Senior Data Engineer
Onsite Peachtree Corners, GA - Hybrid
Salary - Competitive
Position Summary
We are seeking an experienced, highly motivated Senior Data Engineer to help design, develop, and optimize our enterprise data infrastructure. This role requires a strong technical background in data engineering, cloud-based technologies, and modern data architecture, with a particular focus on Microsoft Azure, Azure Databricks, Azure Data Factory, and PySpark.
The ideal candidate is a hands-on technical leader who can translate business requirements into scalable data solutions, establish best practices, and drive continuous improvement across our data environment.
This position offers the opportunity to take ownership of key projects, influence technical strategy, and collaborate with cross-functional teams. While there are no direct reports, the Senior Data Engineer will provide technical guidance, mentor junior team members, and lead project initiatives.
Key Responsibilities
Data Engineering & Architecture
- Design, build, and maintain scalable batch and streaming data pipelines using Azure Databricks, Azure Data Factory, and related technologies.
- Develop and optimize data ingestion, transformation, and integration processes (ETL/ELT).
- Architect and support enterprise-wide data platforms, data lakes, and data warehousing solutions.
- Integrate Azure Databricks with Azure Data Lake Storage (ADLS), Power BI, Azure DevOps, and Azure Key Vault.
- Develop efficient data models and ensure data quality, reliability, and accessibility.
- Optimize Apache Spark jobs, cluster configurations, serverless policies, and cloud infrastructure to improve performance and manage costs.
Cloud Infrastructure, Security & Automation
- Implement and manage Unity Catalog to support data governance, access control, lineage, and security compliance.
- Build and maintain CI/CD pipelines for Databricks notebooks and workflows using Azure DevOps and Terraform.
- Establish and enforce technical standards, development practices, and cloud architecture guidelines.
- Identify opportunities to improve automation, scalability, and overall platform efficiency.
Technical Leadership & Collaboration
- Partner with business stakeholders, analysts, and technical teams to understand current and future data requirements.
- Help define and execute the organization's long-term data strategy.
- Provide technical direction and recommendations for new tools, platforms, and technologies.
- Lead data engineering projects from planning through implementation.
- Mentor junior data engineers and promote knowledge sharing across the team.
- Collaborate with data engineers, analysts, and other technical professionals to support AI and machine learning initiatives.
- Evaluate emerging technologies and recommend solutions that enhance data capabilities.
- Assist with project planning, resource allocation, and assessment of future staffing needs.
Required Qualifications
Education & Experience
- Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics, Statistics, or a related technical field.
- 4+ years of hands-on experience in data engineering, including working with large datasets and database systems.
- 2+ years of experience working with cloud-based data technologies, preferably Microsoft Azure.
- Experience designing and supporting enterprise-scale data platforms and architectures.
- Strong background in ETL/ELT development, data transformation, integration, and modeling.
Technical Skills
- Strong hands-on experience with Azure Databricks and Azure Data Factory.
- Proficiency in Python, PySpark, and SQL.
- Experience with Apache Spark, Spark SQL, Delta Lake, and Databricks Jobs.
- Experience with Azure Data Lake Storage (ADLS), Azure Key Vault, and Power BI.
- Knowledge of Azure DevOps and CI/CD development practices.
- Experience with data warehousing, data lake architecture, and modern cloud data platforms.
- Understanding of cloud storage, compute resources, data orchestration, and reporting technologies.
- Experience optimizing data pipelines and processing large-scale datasets.
Preferred Qualifications
- Experience implementing Unity Catalog for data governance and security.
- Experience using Terraform for infrastructure automation.
- Knowledge of serverless computing, Spark performance tuning, and cloud cost optimization.
- Experience supporting AI/ML data workloads.
- Experience establishing enterprise data engineering standards and best practices.
- Previous experience mentoring engineers or serving as a technical project lead.
Professional Skills & Competencies
- Strong analytical, troubleshooting, and problem-solving abilities.
- Excellent written and verbal communication skills, with the ability to explain technical concepts to both technical and nontechnical audiences.
- Ability to collaborate effectively across departments and work with business stakeholders.
- Strong organizational and time-management skills with the ability to balance multiple priorities.
- Demonstrated ability to independently lead projects, manage deadlines, and deliver results.
- Adaptability in a fast-paced environment with evolving business needs.
- Proactive mindset with a focus on innovation, continuous improvement, and technical excellence.
Leadership Responsibilities
This is an individual contributor position with technical leadership responsibilities. While the role does not have direct supervisory reports, the Senior Data Engineer will lead projects, provide technical oversight, mentor junior resources, and help guide engineering best practices.
Compensation & Benefits
We offer the successful candidate:
- Competitive salary commensurate with education, experience, and skills.
- Quarterly bonus incentives based on operational performance.
- Comprehensive benefits available after 30 days, including medical, dental, vision, life insurance, short-term disability (STD), and long-term disability (LTD).
- 401(k) retirement plan.
- Paid Time Off (PTO).
About Capstone Logistics
Capstone Logistics is a leading North American supply chain solutions partner with more than 650 operating locations, 19,000 associates, and 60,000 carriers. Our comprehensive services include freight management, warehouse and distribution center support, last-mile delivery, supply chain analytics, optimization, and more. Through our integrated, end-to-end logistics solutions, we help our partners improve operational efficiency, streamline supply chain processes, and reduce warehousing and transportation costs. Join a growing organization where innovation, collaboration, and operational excellence drive our success.
Capstone Logistics is an Equal Opportunity Employer committed to quality through diversity. We are a drug-free workplace, and a background check is required.
