Data Engineer

We are looking for a Data Engineer to help build and maintain scalable data solutions that support critical business operations within the brokerage industry. This role is based in Chicago, Illinois, and focuses on designing reliable data pipelines, improving data accessibility, and enabling efficient processing across large datasets. The ideal candidate brings strong technical expertise in modern data engineering tools and enjoys working in a fast-paced environment where data quality, performance, and consistency are essential.

Responsibilities:

• Design, develop, and optimize data pipelines that support ingestion, transformation, and delivery of large-scale datasets.
• Build and maintain ETL workflows to ensure accurate, timely, and dependable movement of data across platforms.
• Use Python and Apache Spark to process complex data efficiently and improve overall pipeline performance.
• Work with Apache Hadoop technologies to manage distributed data storage and computation for high-volume workloads.
• Implement streaming and messaging solutions with Apache Kafka to support near-real-time data integration needs.
• Partner with cross-functional teams to understand data requirements and translate business needs into technical solutions.
• Monitor data processes, troubleshoot issues, and resolve bottlenecks to maintain system reliability and data integrity.
• Contribute to enhancements involving data platform changes or internal process updates as part of ongoing engineering initiatives.
• Hands-on experience as a Data Engineer or in a similar data-focused engineering role.
• Strong proficiency in Python for data processing, automation, and pipeline development.
• Practical experience using Apache Spark for large-scale data transformation and analysis.
• Working knowledge of Apache Hadoop and distributed data processing concepts.
• Experience with Apache Kafka for event-driven or streaming data architectures.
• Solid understanding of ETL design principles and best practices for moving and transforming data.
• Ability to work with large datasets while maintaining accuracy, performance, and scalability.