Data Engineer Principal (Enterprise Data Group)

Domicile Information: This is a hybrid position located in Memphis, TN; Plano, TX; or Pittsburgh, PA. Candidates must live within 50 miles of the campus location. Employees will be required to work at the FedEx campus location several times per week. Essential Functions About the Team - The Enterprise Data Group at FedEx architects, builds, and operates the foundational data platform and enterprise data lake that powers operational visibility, real-time analytics, and business-critical decision systems across the global logistics network. The Data Engineering Principal serves as the senior technical authority and architect across enterprise data engineering initiatives. Operating at the intersection of architecture, transformation, and deployment, this individual defines the technical blueprints for petabyte-scale batch and streaming pipelines, modern lakehouse structures, and governed data products. While partnering with engineering managers on technical roadmap execution, the Principal leads through hands-on technical guidance, driving best-in-class engineering standards, conducting rigorous code and design reviews, solving the most complex scalability hurdles, and mentoring engineering talent across multi-disciplinary squads. 1. Strategic Architecture & Lakehouse Design 1. Enterprise Architecture Blueprints: Translates business analytics requirements, reporting demands, and strategic operational initiatives into scalable, cloud-native data architecture blueprints and reusable technical reference models. 2. Lakehouse & Medallion Strategy: Leads the technical design and standardization of enterprise data lakehouse architectures (medallion model: Raw/Bronze, Conformed/Silver, and Curated/Gold), establishing optimized dimensional schemas, analytical aggregates, and unified semantic layers. 3. Pattern Codification: Champions the design, building, and governance of reusable data engineering assets, frameworks, and deployment templates to accelerate delivery and ensure consistency across distributed teams. 4. Service-Oriented Integration: Establishes and enforces architectural standards for decoupled publish/subscribe patterns, event-driven architectures, and API-first data accessibility. 2. Big Data Pipeline Design, Ingestion & Transformation 1. High-Throughput Ingestion Frameworks: Architect and oversee the implementation of resilient, high-throughput ingestion pipelines capturing data across disparate enterprise sources-including legacy databases, ERP/TMS operational applications, mainframe transaction systems, and message queues (e.g., Apache Kafka, JMS relays). 2. Distributed Transformation Engines: Designs and optimizes scalable, fault-tolerant transformation layers utilizing distributed computing engines (e.g., 3. Apache Spark/PySpark, Dataproc) and modern analytics engineering frameworks (e.g., dbt, modular SQL). 4. Workflow Orchestration & Resilience: Establishes standards for enterprise DAG pipeline orchestration (e.g., Apache Airflow, Cloud Composer), ensuring deterministic job scheduling, automated backfills, lineage capture, and robust disaster recovery capabilities. 5. Complex Data Problem Resolution: Independently designs and deploys complex parsers, validators, transformers, and correlators to resolve multi-domain data conformance challenges and bridge laptop-scale models to cluster-scale execution. 3. Data Governance, Quality & Security Architecture 1. Automated Data Quality & Observability: Directs the implementation of automated data validation, contract enforcement at the ingestion boundary, anomaly detection, and schema evolution controls (e.g., Great Expectations, data contracts) to guarantee enterprise data integrity. 2. Metadata & Lineage Management: Embeds automated metadata registration, catalog discovery, and end-to-end data lineage tracking into pipeline lifecycles, ensuring auditability and traceability from raw ingestion throug consumption. 3. Access Governance & Security Controls: Collaborates with Information Security to architect and enforce enterprise data security standards, including column/row-level security, data masking/tokenization, encryption in transit/at rest, and granular RBAC/ABAC models. 4. SLA & Resource Cost Optimization: Audits and establishes optimization patterns for storage, cluster compute resources, partitioning, and clustering strategies to enforce performance SLAs while maximizing cloud cost efficiencies. 4. Deployment, DevOps & Operational Reliability 1. Modern DataOps & CI/CD Standards: Champions modern software engineering and DataOps discipline within the data engineering practice, including automated testing suites, continuous integration and continuous deployment (CI/CD) pipelines, and Infrastructure-as-Code (IaC). 2. Production Deployment Oversight: Leads release reviews, capacity planning, and deployment governance for critical, high-volume data pipelines transitioning from development to 24/7 production environments. 3. Performance Tuning & Troubleshooting: Serves as the premier escalation engineer to diagnose and resolve complex distributed computing bottlenecks, performance regressions, network latency, and memory/serialization failures in production. 5. Technical Leadership, Mentorship & Cross-Functional Influence 1. Engineering Mentorship & Talent Upskilling: Actively coaches and mentors junior, mid-level, and senior data engineers; fosters an engineering culture grounded in craftsmanship, technical precision, and continuous learning. 2. Architectural & Code Reviews: Conducts deep, constructive architectural assessments and pull request (PR) code reviews to maintain the highest standard of software and data engineering hygiene. 3. Cross-Functional Bridge: Acts as a strategic technical liaison between Engineering Management, Enterprise Architecture, Data Science, Business Intelligence, Product Management, and Cyber Security to negotiate technical trade-offs and drive platform adoption. 4. Technology Radar & Innovation: Evaluates emerging tools, storage formats, and frameworks across the broader technology ecosystem to ensure FedEx maintains modern, competitive capabilities for large-scale data processing. Preferred Knowledge, Skills, and Abilities 1. Distributed Computing, Ingestion & Messaging Systems Distributed Processing Engines: Deep expertise in distributed data processing frameworks, primarily Apache Spark (PySpark, Spark SQL, Scala) on managed cloud clusters (e.g., Google Cloud Dataproc, Databricks). Streaming & Event-Driven Ingestion: Proven mastery in designing and maintaining event-driven, real-time data streaming and message brokering solutions utilizing Apache Kafka, Google Cloud Pub/Sub, Kafka Connect, or enterprise JMS relays. Change Data Capture (CDC): Practical proficiency in implementing robust CDC pipelines to stream upda