Delivery Module Lead

JAVA + APACHE SPARK DEVELOPERSnowflake / Databricks | Enterprise Data EngineeringExperience5-9 yearsLocationNew YorkWork modelHybrid: 3 days in office per weekEmploymentFull-time / Project deploymentRole OverviewWe are seeking a hands-on Java and Apache Spark Developer to build, enhance and support scalable data-processing solutions on modern enterprise data platforms. The role requires strong Java engineering, distributed data-processing expertise and practical experience with either Snowflake or Databricks. The engineer will work across design, development, testing, optimisation, deployment and production support in an Agile environment.Key Responsibilities Design, develop, test and maintain high-volume batch and near-real-time data-processing applications using Java, Apache Spark and Spark SQL. Build resilient, reusable and maintainable data pipelines for ingestion, transformation, enrichment and distribution across enterprise systems. Develop Java-based services, utilities and integration components that support data workflows and downstream consumers. Implement data solutions on Snowflake and/or Databricks, including transformations, curated datasets and consumption-ready data structures. Optimise Spark jobs through appropriate partitioning, caching, broadcast strategies, join selection, shuffle reduction, resource tuning and data-skew handling. Write complex SQL for transformation, validation, reconciliation and performance-sensitive workloads. Partner with architects, business analysts, product owners and engineering teams to translate requirements into practical technical designs. Implement data-quality controls, logging, monitoring, ing, lineage and operational support features. Create unit, integration, regression and performance tests; support defect resolution, release validation and production deployment. Contribute to code reviews, CI/CD pipelines, technical documentation, runbooks and engineering standards. Troubleshoot production issues across application code, Spark execution, SQL workloads, data pipelines and platform integrations.Mandatory Skills and Experience 5-7 years of relevant experience in data engineering, backend development or large-scale data-processing applications. Strong hands-on programming experience in Java, including object-oriented design, collections, exception handling, multithreading and performance-conscious coding. Strong practical experience with Apache Spark using Java, Spark SQL and DataFrame/Dataset APIs. Hands-on experience designing and tuning distributed data-processing jobs for scalability, reliability and efficient resource utilisation. Hands-on experience with at least one modern data platform: Snowflake or Databricks. Strong SQL skills, including complex joins, CTEs, window functions, aggregations, query optimisation and data validation. Experience developing ETL/ELT pipelines and working with structured and semi-structured data. Experience with Git-based development, code reviews, build tools and CI/CD practices. Good understanding of software development lifecycle, Agile delivery, testing and production-support practices. Strong analytical, debugging, communication and stakeholder-collaboration skills.Platform-Specific ExpectationsDevelopmentSQL transformations, views, stages, streams/tasks or equivalent platform objectsNotebooks/jobs, Spark workloads, Delta Lake tables and platform workflowsPerformanceWarehouse/query optimisation, clustering/partition awareness and workload efficiencySpark job tuning, partitioning, caching, shuffle optimisation and cluster efficiencyOperationsLoading, monitoring, validation, access controls and production supportJob orchestration, monitoring, data-quality checks, access controls and production supportPreferred Skills Experience with Spring Boot, REST APIs or Java-based microservices. Exposure to Kafka or another enterprise