Platform ArchitectCooley AI is seeking a Platform Architect to join the Innovation team.
About Cooley AI: Cooley AI is the wholly-owned subsidiary of Cooley LLP. The frontier development arm of the leading technology law firm in the world. Our mission is to bring the best-in-class practices from the technology field into the legal domain to help lead our firm and the legal industry into the AI era. All employees of Cooley AI are employees of Cooley and are seconded to Cooley AI.
Position summary: As a leading technology law firm, Cooley is determined to become a leader in the digital practice of law. The Platform Architect is a technical individual contributor within the Platform & Operations function, responsible for the architecture, design, and evolution of the Databricks platform that underpins the Northstar Lakehouse. This role owns the platform architecture decisions that determine how the Northstar Lakehouse is organized, secured, monitored, and scaled, including Unity Catalog metastore design, compute tier strategy, workspace configuration, CI/CD framework design, and Databricks platform capability adoption. The Platform Architect works closely with the Director of Data Engineering & Platform and collaborates with the Data Engineering team to ensure platform architecture decisions are implementable, performant, and aligned with the firm's data and AI ambitions. This role will use their Databricks technical expertise with the architectural judgment to make platform decisions that serve the platform for years. Specific duties include, but are not limited to, the following:
Position responsibilities:
Databricks Platform Architecture:
- Own the Databricks platform architecture for the Northstar Lakehouse, making and documenting the platform design decisions that govern how the platform is organized, secured, and scaled. This includes Unity Catalog metastore configuration, catalog and schema design, workspace organization, compute tier strategy, and Delta Lake table property standards
- Design and maintain the Unity Catalog governance architecture, establishing catalog hierarchy, schema design patterns, access control frameworks, row-level security and column-level masking configurations, and audit logging standards that satisfy the firm's data governance and ISO compliance requirements
- Define and maintain compute architecture standards across the platform including job cluster configuration, SQL warehouse sizing, photon enablement, instance pool design, and cost-efficient compute patterns that balance performance with platform operating cost
- Evaluate new Databricks platform capabilities as they become available including FMAPI, Mosaic AI, Databricks Apps, Genie, Databricks Lakebase, and model serving infrastructure, making architectural recommendations on adoption timing and integration patterns
- Design the platform's data observability architecture including Databricks Lakehouse Monitoring configuration, alerting thresholds, drift detection standards, and integration with the Data Quality Engineer's DQE monitoring framework
- Own the platform architecture documentation as a living artifact, maintaining current and accurate records of all platform architecture decisions, reasoning, and the implications for future platform evolution
CI/CD Framework & Platform Engineering:
- Own and evolve the CI/CD framework for the Databricks platform, ensuring reliable and automated promotion of pipeline code, notebook changes, and platform configuration through development, UAT, and production environments
- Maintain and extend the Databricks Asset Bundle configuration, GitHub Actions workflows, and environment-specific configuration management that govern how code is deployed across platform environments
- Define and enforce branching strategies, pull request standards, code review requirements, and deployment gate criteria that apply consistently across Data Engineering and Platform & Operations teams
- Design and implement infrastructure-as-code patterns for Databricks resource management, ensuring platform configuration is version-controlled, repeatable, and recoverable
- Collaborate with the DataOps Engineer on CI/CD framework operation and pipeline deployment automation, providing architectural direction on framework evolution as the pipeline estate scales and platform complexity grows
Platform Collaboration & Engineering Support:
- Partner closely with the Director of Data Engineering & Platform on platform architecture decisions, providing the technical depth needed to make informed choices about platform configuration, capability adoption, and infrastructure investment
- Collaborate with the Data Engineering team on pipeline architecture decisions that have platform implications, including Delta Lake table property choices, compute-efficient Spark patterns, Unity Catalog schema organization, and medallion layer conventions
- Work with the Platform Governance & Quality function on Unity Catalog governance configuration, ensuring access control policies, data classification standards, and audit logging meet the requirements defined by the governance framework
- Coordinate with the firm's Technology function on infrastructure dependencies outside the Databricks platform, including network connectivity, gateway configuration, VM management, and cloud infrastructure coordination
- Provide technical guidance to the Platform Operations Engineer on platform operational patterns, monitoring configuration, and incident resolution approaches that require architectural context
- All other duties as assigned or required
Skills and experience:
Required:
- After orientation at Cooley LLP, exhibit proficiency in the Microsoft Office suite, iManage, and other firm applications
- Ability to work extended and/or weekend hours, as required
- Ability to travel, as required
- 4+ years directly applicable (i.e., data platform engineering, cloud infrastructure, or data architecture with hands-on responsibility for Databricks platform design and administration in a production enterprise environment)
- Deep Databricks expertise including Unity Catalog architecture and governance configuration, Delta Lake table design and optimization, compute tier strategy, workspace administration, and Databricks platform administration at an enterprise scale
- Strong CI/CD experience including GitHub Actions, Databricks Asset Bundles, infrastructure-as-code patterns, and environment promotion pipeline design for a data engineering team
- Experience with Unity Catalog access control implementation including catalog and schema privilege design, row-level security, column-level masking, and audit logging configuration
- Working knowledge of cloud infrastructure fundamentals on AWS including IAM, VPC networking, S3 storage configuration, and the connectivity patterns between cloud infrastructure and Databricks workspaces
- Strong Python and SQL proficiency for platform automation, monitoring script development, and pipeline architecture review
- Experience with Databricks Lakehouse Monitoring or comparable data observability platform for drift detection and quality metric alerting
- Comfort using GitHub for version control, CI/CD pipeline development, and infrastructure-as-code management
Preferred:
- Bachelor’s degree
- Experience with Databricks FMAPI, Mosaic AI, Databricks Model Serving, or Databricks Apps and the platform architecture considerations for AI workload hosting
- Familiarity with Apache Kafka, Qlik Replicate, or comparable CDC ingestion tooling and the Databricks integration patterns for real-time data ingestion
- Experience with Terraform, AWS CloudFormation, or comparable infrastructure-as-code tooling for Databricks workspace and cloud resource management
- Familiarity with Data Vault 2.0 pipeline architecture and the platform design considerations that support DV load patterns at scale
- Legal sector, professional services, or financial services experience with understanding of data sensitivity, access control complexity, and compliance requirements in a regulated data environment
- Experience with Databricks Lakebase or comparable transactional data layer for operational application database workloads on a lakehouse platform
- Experience using AI-assisted development tools such as Claude Code or GitHub Copilot to accelerate platform automation, architecture documentation, and infrastructure-as-code development
Competencies:
- Strong written communication skills for platform architecture documentation, decision records, and technical standards development
- Architectural judgment to make platform design decisions that serve the platform for years rather than just solving the immediate problem
- Excellent platform ownership instincts with the discipline to document every architectural decision, maintain current platform documentation, and ensure institutional knowledge lives in artifacts not memory
- Strong collaborative instincts with the ability to work effectively across data engineering, governance, and IT infrastructure functions and explain platform architecture decisions clearly to non-platform audiences
- Excellent engineering craft with high standards for CI/CD pipeline quality, infrastructure-as-code discipline, and platform configuration reproducibility
- Growth-oriented mindset with genuine curiosity to stay current on Databricks platform capabilities and evaluate new features thoughtfully against the platform's actual needs
- Strong independent judgment with the ability to make sound platform architecture decisions within scope and escalate decisions that require Director input
- Strong attention to detail
- High level of professionalism at all times
- Unwavering ability to handle and maintain confidentiality regarding firm information, projects, and client data
Cooley LLP offers a competitive compensation and excellent benefits program. EOE.
The expected annual pay range for this position with a full-time schedule is $175,000 - $250,000. Please note that final offer amount will be dependent on geographic location, applicable experience and skillset of the candidate. Senior level candidates may be considered for this position and would be eligible for a higher salary range based on experience.
We offer a full range of elective benefits including medical, health savings account (with applicable medical plan), dental, vision, health and/or dependent care flexible spending accounts, pre-tax commuter benefits, life insurance, AD&D, long-term care coverage, backup care for children and/or adults and other parental support benefits. In addition to elective benefit options, benefited employees receive firm-paid life insurance, AD&D, LTD, short term medical benefits as well as 21 days of Paid Time Off (“PTO”) and 10 paid holidays each year. We provide generous parental leave and fertility benefits. New employees will attend a detailed benefit orientation to learn more about our many benefits and resources.