Senior Data Engineer We are seeking a hands-on Senior Data Engineer to build a greenfield, AI-ready enterprise data platform on Azure. Working alongside Data Architects, you will spend 90%+ of your time developing scalable pipelines in Azure Databricks and Microsoft Fabric to power enterprise reporting, analytics, and GenAI/RAG initiatives. What You'll Do
- Build and optimize batch and real-time pipelines in Azure Databricks (90%+ focus) using SQL, Python, PySpark, and Delta Lake, utilizing Microsoft Fabric for ingestion.
- Implement Lakehouse and medallion architectures, migrating legacy Data Warehouse workloads and integrating ERP/CRM/HR sources (Dynamics 365 F&O/CE, SAP, legacy ERPs, Workday, Entra).
- Develop dimensional and Lakehouse data models with built-in data quality, lineage, and observability.
- Support data governance and MDM via Databricks Unity Catalog, Microsoft Purview, and enterprise catalog/MDM platforms.
- Lead engineering standards, code reviews, CI/CD practices, and mentor onsite/offshore engineers (individual contributor role). What We're Looking For
- Experience: Minimum 5 years in cloud data engineering (7+ years optimal) and a Bachelor's degree or equivalent experience.
- Core Stack: Deep expertise in Azure Databricks (Lakeflow, Spark, Delta) plus hands-on Microsoft Fabric, ADF, Azure Synapse, ADLS, Azure SQL, and Azure DevOps.
- Code & Modeling: Advanced SQL, expert Python, Spark/Pandas, ETL/ELT, and dimensional/Lakehouse modeling.
- AI & Modern Data: Experience building AI/ML, GenAI, and RAG-ready datasets, including familiarity with MCP servers, data guidance, and AI-assisted data engineering.
- Delivery: Strong debugging skills and ability to drive priorities in Agile/DevOps teams. Nice to Have
- Manufacturing industry experience in complex data environments.
- Direct integration experience with Dynamics 365 (F&O/CE) or similar ERP/CRM platforms.
- Familiarity with enterprise data catalogs and MDM platforms. Additional Details
- Location: St. Louis, MO preferred (hybrid, 3 days/week onsite); remote/flexible hybrid considered.
- Employment Type: Contract-to-Hire (strong intent to convert; direct hire considered for exceptional candidates).
- Team & Process: 4-engineer greenfield team (alongside catalog/MDM engineers). Interview process: Lead Solutions Architect screen followed by hiring manager interview.