Research Scientist (Pyspark/Databricks)

Required Skills & Experience

Required Qualifications Experience in Data Science, Machine Learning, Applied AI, or a related field Strong Python programming experience Hands-on experience with PySpark and distributed data processing Experience working within Databricks environments Advanced SQL skills Experience building, maintaining, or optimizing large-scale data pipelines Understanding of machine learning concepts, model development, and experimentation workflows Experience working with datasets ranging from millions of records Ability to navigate and enhance existing codebases Strong communication and mentoring abilities

Nice to Have Skills & Experience

Natural Language Processing (NLP) experience Experience working with text analytics, document processing, language models, or related NLP applications Exposure to computer vision projects Spark performance tuning and optimization experience Experience supporting migrations between data platforms Experience partnering with research, analytics, and engineering teams in enterprise environments

Job Description

Responsibilities

Enhance and maintain existing Python and PySpark-based data processing solutions Extract, transform, and prepare large datasets for machine learning and analytics initiatives Design and optimize data pipelines supporting research and experimentation efforts Work with structured and unstructured data, including text-heavy datasets and NLP use cases Collaborate with data scientists, researchers, and engineering teams to develop scalable solutions Troubleshoot data quality, pipeline, and performance issues across large-scale environments Conduct exploratory analysis and support model development activities Evaluate new approaches, tools, and technologies that improve research outcomes Provide technical mentorship, code reviews, and guidance to junior team members Translate business and research requirements into practical data solutions