Data & Solution Architect (Databricks and Palantir Platforms)

Required Skills & Experience

• 7+ years of progressive experience in data engineering, data platform engineering, software engineering, data architecture, or solution architecture.
• Demonstrated experience designing and implementing scalable data ingestion, transformation, integration, and quality-management pipelines.
• Strong hands-on experience with Databricks, Apache Spark, PySpark, Spark SQL, Delta Lake, and Lakehouse architecture patterns.
• Strong proficiency in Python. Working proficiency in one or more of Scala, Java, JavaScript, or TypeScript is preferred.
• Experience designing and deploying solutions in one or more major cloud environments: AWS, Microsoft Azure, or Google Cloud Platform.
• Experience with dimensional modeling, star schemas, denormalized reporting models, data marts, semantic layers, and performance optimization.
• Knowledge of developing or deploying machine learning, LLM, RAG, Generative AI, or AI-enabled applications; familiarity with MLOps practices is preferred.
• Ability to work directly with customers, clarify ambiguous requirements, lead technical discussions, and deliver usable production solutions.
• Excellent written, verbal, presentation, and stakeholder-management skills; ability to explain complex technical concepts to technical and nontechnical audiences.
• Willingness and flexibility to travel or work on-site with customer teams as required.

Nice to Have Skills & Experience

• Hands-on experience with Palantir Foundry, Palantir AIP, ontology modeling, Foundry pipelines, operational applications, or Palantir deployment patterns.
• Deep expertise in Apache Spark performance tuning, Spark runtime internals, distributed computing, and large-scale data processing.
• Experience with real-time data technologies such as Kafka, Kinesis, Event Hubs, Delta Live Tables, Structured Streaming, or change data capture frameworks.
• Experience with Databricks Mosaic AI, MLflow, Model Serving, Vector Search, Unity Catalog, Lakeflow, Databricks Workflows, or Databricks SQL.
• Knowledge of implementing RAG architectures, agentic workflows, vector databases, prompt management, LLM evaluation frameworks, and model monitoring.
• Familiarity with infrastructure-as-code and DevOps tooling such as Terraform, GitHub Actions, GitLab CI/CD, Azure DevOps, Jenkins, or similar platforms.
• Experience supporting federal, defense, intelligence, healthcare, financial services, or other regulated environments.
• Active or ability to obtain and maintain a government security clearance, if required by the customer program.
• Databricks certifications, including Databricks Certified Data Engineer, Databricks Certified Data Analyst, Databricks Certified Machine Learning Professional, or Databricks Certified Generative AI Engineer.
• Cloud certifications from AWS, Microsoft Azure, Google Cloud, or Palantir.

Job Description

Insight Global is seeking a highly capable Data & Solution Architect to lead the design, development, and deployment of modern data, analytics, and artificial intelligence solutions for enterprise and public-sector customers. This role bridges advanced data engineering, solution architecture, and production-grade AI, with a focus on building scalable Data Lakehouse capabilities using the Databricks Data Intelligence Platform and Palantir Foundry/AIP.

The Data & Solution Architect will work directly with customer teams to transform complex, ambiguous operational challenges into secure, governed, and high-value technical solutions. The role requires hands-on expertise in ingesting data from diverse source systems in near real time; progressively refining data quality; developing performant data products and marts; and enable team for deploying Generative AI, Retrieval-Augmented Generation (RAG), and multi-agent applications into production.

This is a customer-facing, delivery-oriented role that combines strategic architecture with hands-on implementation. The successful candidate will be comfortable embedding with client teams, including government environments, and leading solutions from initial discovery through production deployment and user adoption.