Premise Health is Different on Purpose
Premise Health serves large organizations and their people with exceptional healthcare. The result? Better experiences, better health, and better value, all while helping organizations lower their healthcare costs.
Premise's mission is to help people get, stay, and be well. Come join us and see for yourself why amazing health starts with amazing healthcare. For more information, visit www.premisehealth.com.
The Data Engineer II will be responsible for building, and maintaining scalable data pipelines, managing our azure delta lake architecture, and supporting analytics initiatives across the organization. The ideal candidate will have a strong background in Azure Databricks, data engineering, and data architecture. The successful candidate will work closely with our platform architects and data science teams to ensure seamless integration of data pipelines and Lakehouse architecture. This role should ensure coding standard and coding migration to higher environment and CI/CD.
This is a Full Time, hybrid, Data Engineer II - Data Architect role located in Brentwood, TN.
What You'll Do
- Design and maintain conceptual, logical, and physical data models for operational, analytical, reporting, and data science use cases.
- Design, develop, and maintain data pipelines using Azure Databricks, Apache Spark, and Python.
- Collaborate with data engineering teams to integrate data from EPIC, Clarity, and Caboodle and other on-premise source systems to Azure.
- Ensure data quality, integrity, and security throughout the data pipeline.
- Automate unit test cases as part of Data Pipeline.
- Contribute to the development and maintenance of our Lakehouse architecture using Azure Databricks and Delta Lake.
- Ensure data governance and security in the Lakehouse environment.
- Able to execute POCs to evaluate new tool, concepts introduced by Azure/ Databricks.
- Troubleshoot data pipeline and Lakehouse architecture issues, identifying and resolving bottlenecks and errors.
- Optimize data pipeline and Lakehouse architecture performance, scalability, cost and reliability.
- Collaborate with data science teams to integrate data pipelines and Lakehouse architecture with data science workflows.
- Ensure seamless data exchange between data engineering and data science teams.
- Participate in peer review for coding standards.
- Subject matter expert for pySpark and Scala.
- Implement and maintain monitoring solutions for Databricks pipelines (Bronze, Silver, Gold layers) to ensure data quality, performance, and reliability.
- Develop dashboards and alerts for tracking DLT pipeline health, job execution status, and SLA adherence using Databricks metrics and Azure Monitor.
- Configure and manage logging frameworks across ADF pipelines, Databricks jobs, and Storage Account to enable end-to-end traceability.
- Set up proactive alerting mechanisms for failures, latency issues, and resource utilization using Azure Monitor, Log Analytics, and custom metrics.
- Ensure lineage and auditability by leveraging Unity Catalog for tracking data movement and access patterns across the Medallion architecture.
- Collaborate with Data Engineering teams to define observability standards and integrate them into CI/CD workflows.
- Perform root cause analysis (RCA) for pipeline failures and maintain operational documentation for recurring issues.
- Optimize cost and performance monitoring for Databricks clusters and ADF activities through usage analytics and reporting.
- Provide extended support during off-hours or weekends for critical deployments or troubleshooting production issues to ensure platform stability.
- Other duties as assigned.
What You'll Bring
- Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
- Certifications such as AWS Certified Solutions Architect, Cisco CCNP, Microsoft Azure Solutions Architect, or ITIL preferred.
- Databricks and Azure certification preferred.
- 5+ years of experience in Cloud development practice.
- Demonstrated experience designing conceptual, logical, and physical data models.
- Demonstrated experience with dimensional modeling, including facts, dimensions, conformed dimensions, and slowly changing dimensions.
- Strong expertise in cloud platforms (AWS, Azure, GCP), virtualization, and networking.
- Preferred experience with infrastructure automation tools (Terraform, Ansible, Kubernetes).
- Proficiency in Linux and Windows server administration.
- Solid knowledge of cybersecurity best practices and disaster recovery planning.
- Strong leadership experience in managing technical teams and project management skills.
- Knowledge of log management and monitoring tools (Splunk, Datadog, Prometheus).
- Knowledge on Talend, Power BI and Qlik.
- Familiarity with containerization and microservices.
Work-life balance is at the foundation of how decisions are made and where Premise is headed. We can only help people get, stay, and be well if we do the same for ourselves. In addition to competitive pay, Premise offers full-time team members benefits including medical, dental, vision, life and disability insurance, a 401(k) program with company match, paid holidays and vacation time, a company-sponsored wellness program, EAP, access to virtual primary care and virtual behavioral health at no cost for team members and their dependents. Additional benefits can be viewed here: https://jobs.premisehealth.com/benefits.
Premise is an equal opportunity employer; we value inclusion and do not discriminate based on race, color, religion, creed, national origin or ancestry, ethnicity, sex (including pregnancy and related conditions), gender identity or expression, sexual orientation, age, physical or mental disability, genetic information, past, current or prospective service in the uniformed services, or any other characteristic protected under applicable federal, state, or local law.
