Azure Data Engineer- Lead

Apply

Job Type

Contract

Description

Please note:

No agencies or staffing firms, please. We are not accepting unsolicited candidate submissions and request that agencies do not contact our employees or representatives regarding recruiting services for this role.

• Engagement Type: Contract - C2C or W-2 hourly/temporary
• Expected Commitment: 40 hours per week, Monday–Friday
• Work Schedule: Must work East Coast hours
• Initial Term: Three months, with potential to extend
• 100% Remote: Ideal candidate resides in NH, MA, RI,

Spyglass MTG is seeking experienced Lead Data Engineers who are “thinkers” and can mentor junior developers while extending an existing codebase and building team proficiency in Microsoft Fabric and Pyspark Python including class development and structured streaming.

Why Spyglass MTG?

• Exclusive focus on Microsoft technologies for more than 12 years
• 90% of our business comes from repeat customers
• 80% of our new business comes through Microsoft
• Five Microsoft Solutions Partner badges and 10 Advanced Specializations
• Member of Microsoft’s Partner Advisory Council for Data and AI
• Microsoft Fabric Voice member

Description:

This is an exciting opportunity to join a high-impact, multi-year data modernization program that will transform how the client uses data to support student persistence and success. Over the next two years, the team will modernize the client’s data estate on Microsoft Fabric, using a hub-and-spoke model and medallion architecture to bring enterprise data into a scalable platform. Data will be ingested through the hub and processed across the bronze, silver, and gold layers to power analytics, downstream applications, and future AI-enabled solutions. The program also has an ambitious goal of moving toward real-time, or near real-time, data processing. For someone who wants to work with modern cloud data architecture, Microsoft Fabric, advanced engineering patterns, and meaningful higher education outcomes, this project offers a strong opportunity to contribute, learn, and make a visible impact.

The team is looking for 2 Lead Data Engineers. The plan is to pair less experienced client teammates with lead data engineers. In this approach, my firm would provide the lead data engineers who can guide delivery while mentoring client resources.

Requirements

We are looking for the following skills/experience:

Must-have qualifications:

The lead engineer will be paired with a client resource working East Coast hours and will need to extend an existing Python class-based codebase while helping a junior developer build proficiency. o Availability to work East Coast hours o Fluent English communication skills o Experience with Microsoft Fabric and Pyspark Python development including class objects, structured streaming, Change Data Feed (CDF), and metadata driven processes o Experience sourcing data, transforming it, and loading it into tables using notebooks. o Experience leading or mentoring developers who are learning new technologies such as Microsoft Fabric, Python, and object-oriented development helping to raise the overall technical capability of delivery teams. o Experience establishing and promoting engineering standards for code quality, testing, CI/CD, version control, monitoring, security, performance, and maintainability. o Experience serving as a trusted technical advisor to clients and project teams, proactively identifying risks, dependencies, delivery issues, and improvement opportunities

Nice-to-have qualifications:

o Real-time or near real time experience o Experience with medallion architecture, especially the Silver layer. o Experience using AI tools to generate or accelerate code development o Experience with data vault data models o Experience working with higher education clients

This role is ideal for someone who wants to do more than write code: you will help set technical direction, raise engineering standards, mentor developing data engineers, solve complex business and technical challenges, and deliver scalable, secure, and maintainable solutions at enterprise data scale. For the right person, this is a chance to make a meaningful impact, work with modern cloud data technologies, and help build a platform and team capability that will continue to grow long after the engagement is complete.

Benefits:

Medical, Vision and Dental Plans Life and Disability Insurance Open PTO Policy Holiday PTO Paid training certification Bonus plan 401k Flexible working arrangements & more

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, transgender status, national origin, citizenship, age, disability or protected veteran status.