Job Title: Data Platform Engineer
Reports to: AI & Data Solutions Manager
Job Type: Full-time, Exempt
About the company:
Founded in 1995, Vastian is the leading provider of healthcare and laboratory quality, compliance and accreditation solutions, powering laboratories and hospitals to achieve better clarity and show their higher standards. Its single, configurable integrated platform delivers a standardized, centralized solution that automates quality and compliance tasks and is fully customizable. With Vastian, health systems and labs can get ahead of compliance and quality processes, saving time that can be spent on delivering better care. Vastian is a private equity backed, fast-growing SaaS business based in Lawrenceville, GA, outside Atlanta.
Click here to learn more about Vastian and our suite of solutions.
As a Data Platform Engineer, you'll join the AI and Technology Solutions team to design, build, and maintain scalable data solutions that support both internal systems and client-facing applications. This role will involve hands-on data engineering work across Microsoft Fabric, SQL Server, Lakehouse environments, and cloud data platforms, helping transform raw data into reliable, secure, and usable assets for analytics, reporting, automation, and operational workflows. You'll also work with pipelines, notebooks, PySpark and C# to orchestrate, process, and optimize data at scale. In addition, you'll play a key role in building our client-facing data ingestion and analytics product, using Azure technologies such as Event Hubs, Azure Data Explorer, and Azure Functions to securely receive, process, and analyze high volumes of data from client laboratory and healthcare systems in near real time.
Responsibilities and Duties:
- Design and build scalable, client-facing data ingestion pipelines using Azure Event Hubs, Azure Data Explorer, Azure Functions, Event Grid, and related serverless services to receive and process data from client systems in batch and near real time
- Design, build, and optimize data pipelines, Lakehouse solutions, and engineering workflows using Microsoft Fabric, SQL Server, PySpark, and related cloud data technologies
- Develop, maintain, and improve batch and orchestration pipelines that move and transform structured and unstructured data across source systems, lakehouse layers, and cloud environments
- Develop and optimize Kusto Query Language (KQL) queries, ingestion mappings, update policies, materialized views, and retention/caching policies to support fast, cost-effective analytics on high-volume data
- Work with application and platform engineers to define and support client-facing ingestion APIs, including authentication, payload validation, throttling, and error handling
- Handle both defined and flexible client data schemas, including semi-structured JSON, with approaches for schema evolution, versioning, and normalization into curated, analytics-ready models
- Ensure strict tenant isolation and data security across multi-tenant data stores, so each client's data is segregated, access-controlled, and protected in line with healthcare and regulatory requirements
- Implement data quality checks, monitoring, alerting, and observability for ingestion and processing pipelines, including dead-letter handling, replay, and recovery processes
Skills & Experience:
- Experience or a strong educational background in data engineering, analytics, computer science, or a related technical discipline
- Strong analytical and problem-solving skills, with the ability to work confidently with complex structured and semi-structured datasets
- Hands-on experience with Azure data ingestion and analytics technologies such as Azure Event Hubs, Azure Data Explorer, Event Grid, Azure Functions, or Stream Analytics
- Proficiency in Kusto Query Language (KQL) and experience designing high-volume, time-series, or event-based data solutions
- Practical experience using Python, PySpark and C# for data processing, transformation, automation, or analytics tasks
- Strong hands-on experience with SQL Server and writing efficient SQL for data transformation, integration, and analysis
- Experience working with JSON and other semi-structured formats, REST APIs, and schema design, validation, and evolution
- Familiarity with data modeling, ETL/ELT patterns, pipelines and best practices for building scalable and maintainable data solutions
- Understanding of data security and privacy practices, including encryption, role-based access control, managed identities, and handling sensitive or regulated data
- Ability to communicate technical concepts and data outcomes clearly to both technical and non-technical stakeholders
In addition to a competitive base salary and performance bonus program, Vastian offers a comprehensive and very pro-employee benefits package which includes the following:
- Medical, dental and vision insurance
- Short- and long-term disability coverage
- Life insurance and AD&D
- Supplemental life insurance
- 401(k) savings plan
- Paid time off
Vastian is an equal opportunity employer. We do not discriminate on the basis of race, color, religion, national origin, pregnancy status, sex, age, marital status, disability, sexual orientation, gender identity or any other characteristics protected by law.