Data Engineer

Strategic Staffing Solutions is currently looking for a Data engineer, for a W-2 contract opportunity.

Data Engineer

Location: Charlotte, NC,28206

Type: W-2 Contract (NO C2C)

Contract: 12 months

Schedule: Hybrid

This is a W-2 contract opportunity, and candidates must be able to work on our W-2 only. NO C2C or 1099.

Position Overview

Required Technical Skills • Python: Advanced proficiency in Python 3.8+ for data processing, scripting, and automation • PySpark: Extensive experience with PySpark for large-scale data processing and transformations on EMR clusters • AWS Lambda: Hands-on experience building serverless functions for data processing and orchestration • AWS Glue: Expertise in AWS Glue for ETL jobs, Data Catalog management. • Amazon Athena: Experience with Athena for SQL-based data analysis and querying data lakes • Amazon EMR: Strong experience with EMR for running Spark jobs, EMR Notebooks, and cluster management • Amazon Redshift: Knowledge of Redshift for data warehousing, query optimization, and integration with data lakes • Kafka: Experience with Confluent Kafka for real-time data streaming, including Schema Registry, source/sink connectors, and producers/consumers • Amazon S3: Deep understanding of S3 for data lake architecture, bucket policies, and data organization (raw, curated layers) • CloudWatch: Proficiency in CloudWatch for logging, monitoring, and alerting on data pipelines • Terraform: Experience with Infrastructure as Code using Terraform for provisioning AWS resources

Key Responsibilities

• Design, develop, and maintain scalable ETL pipelines using AWS Glue, EMR (PySpark), and Lambda functions • Implement real-time data streaming solutions using Confluent Kafka with source and sink connectors • Develop and optimize PySpark jobs on EMR clusters for large-scale data transformations • Build serverless data processing workflows using AWS Lambda • Design and optimize queries in Amazon Athena for analytical workloads • Manage data warehouse solutions in Amazon Redshift including data modeling and performance tuning • Implement monitoring and alerting using CloudWatch for data pipeline health and performance • Provision and manage AWS infrastructure using Terraform • Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements • Implement data governance and security best practices using Lake Formation, IAM, and KMS

Additional Technical Experience • Knowledge of AWS EventBridge and Step Functions for workflow orchestration • Familiarity with EMR runtime roles and execution roles for secure data access • Experience with DynamoDB for NoSQL data storage • Knowledge of API Gateway for building data APIs

Preferred Qualifications • Bachelor's degree in Computer Science, Data Engineering, Information Technology, or related field • Strong understanding of distributed computing and big data processing frameworks • Experience with Agile development methodologies • Excellent problem-solving and analytical skills • Strong communication and collaboration abilities

Development and Data Tools • EMR Notebooks or Jupyter for interactive development • VS Code or similar IDE for Python development • Git for version control • AWS CLI and boto3 for AWS service interaction • Terraform for infrastructure provisioning • DBeaver, Data Studio for database management

“Beware of scams. S3 never asks for money during its onboarding process.”