Sr. Data Engineer - Databricks

Job Description

One of Insight Global's clients is seeking a Sr. Data Engineer supporting data platforms with a data science & insights department. The primary mission is to build production-grade Databricks pipelines and reusable curated data products that combine SAP current-state data with legacy and non-SAP history.

Responsibilities:

Ingestion and transformation pipelines; Bronze/Silver/Gold tables; incremental/CDC processing; ; operational monitoring and runbooks; performance; knowledge transfer to the client.

Skills and Requirements

• Bachelor’s degree (B. A.) from four-year college or university; or 8+ years related experience and/training; or equivalent combination of education and experience.
• 5+ years data engineering with at least 2-3 years recent production Databricks/Spark work
• Hands-on Databricks; PySpark and Spark SQL; Delta Lake; medallion/lakehouse design; SQL; orchestration with Workflows/Lakeflow or comparable; Unity Catalog/RBAC; monitoring, recovery, data quality and performance tuning
• Azure/ADLS; SAP Datasphere-to-Databricks integration; Delta Sharing; Netezza/DataStage modernization; dbt; Kafka/event-driven patterns.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.