Data Pipeline Development & Integration
- Design, build, and maintain scalable ETL/ELT pipelines to extract, transform, and load data from multiple sources.
- Develop automated data integration solutions supporting enterprise applications, analytics, and reporting.
- Integrate structured and semi-structured data from APIs, relational databases, cloud platforms, and third-party systems.
- Develop batch and real-time data processing solutions.
- Troubleshoot pipeline failures, data inconsistencies, and performance bottlenecks.
Data Warehousing & Architecture
- Design and optimize enterprise data warehouses, data lakes, and lakehouse architectures.
- Develop efficient data models, schemas, and transformations to support business intelligence and analytics.
- Build and maintain cloud-based data solutions using AWS, Microsoft Azure, or Google Cloud.
- Optimize SQL queries, database performance, and data processing workloads.
- Implement data partitioning, indexing, and storage optimization strategies.
Data Quality & Governance
- Develop automated data validation, monitoring, and quality assurance processes.
- Ensure data accuracy, consistency, reliability, and availability across enterprise systems.
- Implement data governance standards, access controls, and security best practices.
- Maintain documentation for data pipelines, architecture, and transformation logic.
- Support compliance with organizational data privacy and security requirements.
Cloud Engineering & Automation
- Develop and maintain cloud-native data pipelines and processing workflows.
- Utilize orchestration tools such as Apache Airflow, Azure Data Factory, or similar technologies.
- Implement CI/CD practices for data engineering workflows.
- Automate data processing, monitoring, and infrastructure deployment.
- Collaborate with DevOps and infrastructure teams to improve scalability and reliability.
Analytics & AI Support
- Partner with business intelligence and analytics teams to deliver reliable datasets for reporting and dashboards.
- Develop curated data models supporting Power BI, Tableau, and other visualization platforms.
- Build data pipelines supporting machine learning, predictive analytics, and AI applications.
- Collaborate with AI engineers and data scientists to prepare, transform, and deliver data for model development and production applications.
- Identify opportunities to improve business processes through better data integration and automation.
- Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field, or equivalent practical experience.
- 3–5+ years of professional experience in data engineering, database development, or related roles.
- Strong proficiency in SQL and Python.
- Hands-on experience developing ETL/ELT pipelines and data integration solutions.
- Experience working with relational databases such as SQL Server, PostgreSQL, Oracle, or MySQL.
- Experience with cloud platforms such as AWS, Azure, or Google Cloud.
- Knowledge of data warehousing concepts, dimensional modeling, and database optimization.
- Experience with data orchestration and scheduling tools.
- Familiarity with REST APIs, JSON, and other data integration methods.
- Experience with Git, CI/CD, and software development best practices.
- Strong analytical, troubleshooting, and communication skills.
