Senior Big Data Engineer

Company Description Deegit Inc.

Job Description

Job description:

Responsibilities

· Participate in the engineering and administration of big data systems. · Apache Storm/Java development for both data transformation and augmentation. · Employ best practices to ensure high availability and scalability. · Install, configure, and maintain big data technologies and systems. · Maintain documentation and troubleshooting playbooks. · Respond to and resolve access and performance issues. · Ability to work non-peak hours when needed. · Assist in capacity planning. · Proactively monitor performance. · Collaborate with engineering teams to optimize data collection. · Maintain change control and testing processes for all modifications and deployments. · Conduct research and make recommendations on big data products, services, and standards.

Preferred Qualifications

· Experience with operational analytics and log management using ELK or Splunk. · Understanding and experience with SQL and MPP databases such as Greenplum. · Experience with SaltStack or comparable deployment automation tools. · Working knowledge of Zenoss, NewRelic, PagerDuty or other monitoring technologies. · Experience with Apache Drill, Apache Hbase, Hive, Phoenix, or other Hadoop with SQL technologies. · Experience working with batch-processing and tools in Hadoop technology stack (MapReduce, Yarm, Pig, Hive, HDFS)

Qualifications

Basic Requirements · BS in Computer Science, Information Systems, or related technical degree. · 5 or more years’ experience with data transformation pipelines preferably Storm · 5 or more years’ experience in messaging frameworks preferably Kafka. · 5 or more years’ experience with Java programming. Additional Information All your information will be kept confidential according to EEO guidelines.