Data Engineer – Databricks (IBU Consumer)
- Indianapolis, IN, United States
Applications open in a new tab.
Job details
- Posting language
- English · French
- Posted
Original job description
English section of this bilingual listing.
Build and maintain scalable data pipelines on Databricks for consumer-facing IBU data products.
- Develop ETL/ELT pipelines using PySpark, Spark SQL and Delta Lake
- Implement medallion architecture (bronze/silver/gold), Unity Catalog and Workflows
- Optimize cluster performance and cost; monitor and troubleshoot jobs in production
- Must have: 4+ yrs data engineering, Databricks, PySpark, SQL, Git/CI-CD.
Read the full bilingual original
Build and maintain scalable data pipelines on Databricks for consumer-facing IBU data products.
- Develop ETL/ELT pipelines using PySpark, Spark SQL and Delta Lake
- Implement medallion architecture (bronze/silver/gold), Unity Catalog and Workflows
- Optimize cluster performance and cost; monitor and troubleshoot jobs in production
Requirements
- Must have: 4+ yrs data engineering, Databricks, PySpark, SQL, Git/CI-CD.
- Nice to have: pharma/consumer data, Databricks certification
More about this job
Skills
- Databricks
- Data engineering
- PySpark
- Apache Spark
- SQL
- Unity
- Troubleshoot
- Git
- CI/CD