We are looking for an experienced Data Engineer to help advance a modern cloud-based data environment in St. Paul, Minnesota. This Long-term Contract position focuses on creating reliable, scalable data solutions that support analytics, business intelligence, and AI-driven initiatives across the organization. The ideal candidate brings strong Databricks expertise and a practical approach to building high-quality data products that enable confident decision-making.
Responsibilities:
- Build and enhance enterprise data pipelines in Databricks to support dependable, large-scale data processing and delivery.
- Create and refine ETL workflows using Spark, PySpark, and cloud technologies to improve efficiency, stability, and performance.
- Design Lakehouse-based data architectures that serve reporting, advanced analytics, and artificial intelligence use cases.
- Produce clean, validated, and business-ready datasets that strengthen self-service reporting and downstream analytical applications.
- Monitor and improve platform performance, data quality, scalability, and operational reliability across data engineering solutions.
- Collaborate with engineering, analytics, BI, and business stakeholders to translate requirements into effective data products.
- Provide technical guidance to other engineers and promote sound practices for modern data engineering development.
- Support the evolution of the cloud data platform through continuous optimization, automation, and maintainable solution design.
- At least 7 years of hands-on experience in data engineering, including substantial work with Databricks.
- Advanced knowledge of Apache Spark, Delta Lake, and Lakehouse concepts for enterprise data solution development.
- Strong programming ability in Python and PySpark, along with solid experience in data modeling and transformation design.
- Proven background in building and tuning high-volume data pipelines in cloud-based environments.
- Good understanding of data warehousing principles, governance practices, and performance optimization techniques.
- Experience with platforms such as Azure, Microsoft Fabric, Data Factory, or similar cloud data services.
- Familiarity with technologies such as Apache Hadoop and Apache Kafka to support distributed data processing needs.
- Strong analytical thinking, communication skills, and the ability to work effectively with technical and business teams.