Sr. Data Engineer - Databricks
- Remote, United States
From $74 USD / hour
Applications open in a new tab.
Job details
- Pay
- From $74 USD / hour
- Experience
- A.) from four-year college or university; or 8+ years related experience and/training; or equivalent combination of education and experience.
- Posting language
- English
- Posted
Original job description
Required Skills & Experience
- Bachelor’s degree (B. A.) from four-year college or university; or 8+ years related experience and/training; or equivalent combination of education and experience.
- 5+ years data engineering with at least 2-3 years recent production Databricks/Spark work
- Hands-on Databricks; PySpark and Spark SQL; Delta Lake; medallion/lakehouse design; SQL; orchestration with Workflows/Lakeflow or comparable; Unity Catalog/RBAC; monitoring, recovery, data quality and performance tuning
- Azure/ADLS; SAP Datasphere-to-Databricks integration; Delta Sharing; Netezza/DataStage modernization; dbt; Kafka/event-driven patterns.
Job Description
One of Insight Global's clients is seeking a Sr. Data Engineer supporting data platforms with a data science & insights department. The primary mission is to build production-grade Databricks pipelines and reusable curated data products that combine SAP current-state data with legacy and non-SAP history.
Responsibilities:
Ingestion and transformation pipelines; Bronze/Silver/Gold tables; incremental/CDC processing; ; operational monitoring and runbooks; performance; knowledge transfer to the client.
More about this job
Skills
- Databricks
- Data engineering
- Apache Spark
- PySpark
- SQL
- Unity
- Microsoft Azure
- SAP
- dbt
- Apache Kafka
- Monitor operations in the leather industry