Data Engineer, Sr

Overview

The Data & Insights organization at Insight Global is responsible for delivering Trusted Data & Verified Insights to drive business decisions across the company. As a Senior Data Engineer, you will be part of the Data Engineering & Operations team responsible for enhancing and maintaining scalable data pipelines, supporting and improving platform operations processes, and enabling analytics, data science, and AI use cases. This is a hands-on role reporting to the Data Operations Manager with direct impact on Insight Global’s ability to deliver high-quality, actionable data to the business.

Responsibilities

• Design, develop, and enhance scalable data pipelines supporting analytics, reporting, AI/ML, and operational business processes.
• Own the reliability and stability of enterprise data platforms by proactively monitoring pipelines, troubleshooting production issues, and ensuring adherence to SLAs and operational standards.
• Lead root cause analysis and incident resolution efforts, driving permanent fixes, automation opportunities, and preventative controls to reduce recurring issues.
• Develop and maintain data solutions using Databricks, Azure Data Factory, SQL, and PySpark, following enterprise architecture and engineering standards.
• Drive DataOps best practices, including CI/CD implementation, automated testing, code reviews, deployment automation, release management, and environment governance.
• Improve data quality, performance, observability, and operational efficiency through monitoring, alerting, lineage tracking, logging, and automation initiatives.
• Work with structured, semi-structured, and streaming data, building scalable ingestion and transformation frameworks.
• Collaborate with business stakeholders, analytics teams, architects, product owners, and application teams to deliver reliable and business-aligned data solutions.
• Identify opportunities for continuous improvement in processes, architecture, governance, security, cost optimization, and platform scalability.

Qualifications

Required Skills

• 5-7 years of experience in Data Engineering, DataOps, Analytics Engineering, or Enterprise Data Platforms.
• Strong expertise in advanced SQL development, query optimization, data analysis, and troubleshooting.
• Proficiency in Python and PySpark for large-scale data transformation, pipeline development, and automation.
• Hands-on experience with Databricks, Delta Lake, Azure Data Factory, and Azure data services.
• Experience designing and supporting batch, near real-time, and streaming data solutions.
• Strong understanding of CI/CD pipelines, Git-based source control, pipeline testing, and DevOps/DataOps practices.
• Experience with monitoring, observability, logging, alerting, and production support processes.
• Knowledge of data modeling, data warehousing concepts, and dimensional design principles

Preferred Skills

• Experience with Databricks Unity Catalog.
• Knowledge of data governance, data quality frameworks, security, and compliance requirements.
• Proven experience managing production incidents, performing root cause analysis, and driving long-term remediation plans.
• Ability to prioritize operational stability while balancing enhancement and project delivery commitments.
• Experience working in Agile, Scrum, or Scrumban environments.

This Role Is Ideal If You:

• Are excited to be part of a data team supporting data pipelines in Databricks.
• Thrive in Agile environments and enjoy working across disciplines.
• Take ownership of operational responsibilities and ability to fix data issues.
• Are passionate about delivering reliable, high-quality data solutions.
• Embody Insight Global’s shared values and contribute to a collaborative, high-performance culture.