Senior Data Engineer

Required Skills & Experience

-10+ years of Data Engineering experience -Strong hands-on Databricks experience -Advanced Python and SQL skills -Strong recent experience designing and building end-to-end data pipelines and datasets -Cloud experience with AWS preferred; Azure or GCP experience is also applicable -Ability to independently design data solutions, including data mapping, transformations, expected outputs, and pipeline architecture -Excellent communication skills and ability to present designs, explain technical decisions, and work closely with the team

Nice to Have Skills & Experience

-Front-end experience with React, JavaScript, or similar technologies -Experience supporting data for user-facing applications

Job Description

Insight Global is looking for a Senior Data Engineer that will join a large pharma client in Tarrytown, NY. This role will support the newly launched VI Train Command Center, an internal platform being built to create one shared, current view of antibody campaigns and improve visibility into priorities, capacity, readiness, and progress. This person will own a data engineering stream end-to-end, with approximately 80% of the role focused on backend development. They will design and build data pipelines, process and prepare data, create datasets, and make the data available within Databricks applications. They will be responsible for understanding the required inputs and outputs, developing the technical design, presenting that design to the team for alignment, and then independently implementing the solution. Because VI Train is a new and evolving platform, the team needs someone who is an independent thinker and can anticipate future needs, rather than only developing for the immediate requirement. They should consider scalability, flexibility, and different user/access requirements as they design solutions, helping ensure new functionality can be added without significant rework. While the initial focus will be VI Train, this person will eventually have the opportunity to take ownership of additional data engineering streams as needs evolve. Pay is around 70/hr.