Distinguished HPC Engineer

Primary Responsibilities:

• Design and implement high-performance computing clusters to ensure they meet the required performance, scalability, reliability and availability standards.
• Provide subject matter expertise in HPC storage, file systems and Linux to DST clients and internal teams.
• Deploy and maintain cluster management software such as Bright Cluster Manager to streamline administration and operations.
• Set up, configure, and troubleshoot HPC job schedulers like SLURM, LSF, or UGE to optimize job scheduling and resource allocation.
• Install, configure and maintain containers, container applications and orchestration tools.
• Deploy and manage advanced health monitoring systems to ensure the HPC infrastructure is operating efficiently and to preemptively identify issues.
• Automate software installations and system configurations using Ansible to ensure consistency and efficiency across deployments.
• Stay up to date with the latest developments in HPC technologies and research new solutions to improve system performance and capabilities.
• Provide Jira support to maintain and improve DST’s project and ticket management.
• Maintain clear and effective communication with clients, management, and internal teams to ensure project goals and expectations are met.

Requirements:

• Extensive experience with Linux operating systems, including system configuration, troubleshooting, and performance tuning.
• Advanced expertise with parallel storage systems, InfiniBand, and Ethernet networking to ensure high-performance data transfer and storage solutions.
• Proven expertise in designing and building HPC clusters from the ground up, including hardware and software integration.
• Advanced knowledge and hands-on experience with Bright Cluster Manager, Slurm for job scheduling, and Ansible for automation.
• Intermediate knowledge of GPUs and associated hardware/software, including configuration, optimization, and debugging.
• Intermediate Jira skills
• Strong organizational skills to manage multiple projects efficiently and excellent communication skills to interact effectively with various stakeholders