Required Skills & Experience
- 7+ years of hands-on data engineering experience
- 4+ years of hands-on AWS experience
- Strong understanding of cloud architecture and concepts
- Hands-on experience with AWS Glue, S3, Athena, and Redshift.
- Experience building and integrating APIs and Model Context Protocols (MCPs) o Exposure to GenAI solutions using AWS Bedrock, LLMs, tool calling, and agent-based architectures.
- Experience with Claude or Codex and prompt engineering
- Experience working with large language models (LLMs)
- Experience with Java-based REST APIs, preferably using Spring Boot.
- Strong Python development experience and using Pyspark
- Strong SQL skills
- Apache Kafka experience
- Experience building end-to-end data pipelines
- Strong knowledge of data domains, data modeling, and data analysis
- Experience designing data ingestion, transformation, and delivery workflows
- Ability to clearly explain technical decisions through prior project examples
Nice to Have Skills & Experience
- FastAPI experience
- Experience with data warehouse architecture
- Financial services industry experience
- AWS Step Functions
- Lambda
- Lake Formation
- IAM
- CloudWatch
- Docker
- Git/Jenkins
- RAG
- AI agent development
Job Description
Insight Global is seeking a Data Engineer for a top financial services client. This individual contributor will help build scalable AI-enabled services that connect enterprise data with large language models. The position is approximately 60% MCP and API development and 40% data engineering, with responsibility for developing Model Context Protocol solutions, APIs, prompts, and cloud-based data pipelines. The ideal candidate combines strong Python, SQL, AWS, Kafka, and data-modeling experience with hands-on knowledge of API development, LLM integration, and prompt engineering. This is an excellent opportunity for an engineer who enjoys owning solutions across data ingestion, transformation, AI integration, and downstream delivery.
Key Responsibilities
- Spend approximately 60% of the role building and supporting MCPs and APIs
- Spend approximately 40% of the role performing AWS-focused data engineering
- Design and develop scalable data pipelines using AWS Glue, PySpark, S3, Athena, and Redshift.
- Develop complex data transformations and data-quality solutions using Python, PySpark, and SQL.
- Design and develop Java-based REST APIs, preferably using Spring Boot.
- Build integrations between APIs, databases, AWS services, and enterprise applications.
- Develop and integrate MCP servers and MCP tools to expose enterprise APIs, data, and services to AI agents.
- Develop GenAI solutions using AWS Bedrock, LLMs, tool calling, and agent-based architectures.
- Implement production-grade logging, exception handling, security, testing, monitoring, and CI/CD.