Launch your career with Hamilton Barnes’ Graduate Hub
Take me there

Senior GPU Infrastructure Engineer - Systems Integrator

1754023
  • £120k
  • United Kingdom
  • Permanent
  • 100000
  • Artificial Intelligence
  • AI Data Center


Are you looking for an exciting new opportunity?

Join a sovereign cloud and AI platform that combines the openness of public cloud with the security and control of private environments, making the cloud easier to use, faster to deploy, and fairer to pay for. Offering public cloud, private cloud, and high-performance GPU infrastructure with clear, predictable pricing, it helps organisations innovate while keeping control of their data, infrastructure, and spend.

The organization is inviting applications from experienced professionals for the role of GPU Platform Reliability Engineer, acting as the technical authority for the reliability, performance, and automated orchestration of the AI and GPU platform layer. The ideal candidate will work at the boundary between server hardware telemetry, high-bandwidth interconnect fabrics, and cloud-native orchestration, serving as the highest internal escalation point for a live production GPU fleet and building automated remediation tooling. The role includes a 4-day working week and remote flexibility.

Don’t miss out on this exciting opportunity and apply today!


Responsibilities:

  • Act as the top internal escalation point for persistent or intermittent hardware, PCIe, GPU memory (ECC) and XID driver faults across the GPU fleet, owning deep root-cause analysis
  • Build and maintain automated pre-job and post-job health check frameworks using NVIDIA DCGM, NCCL test suites and custom telemetry agents
  • Own automated rollout pipelines for GPU VBIOS, system BIOS, BMC firmware and CUDA driver stacks across live production clusters with zero unannounced customer impact
  • Diagnose bottlenecks, packet drops and link degradation across InfiniBand (Quantum-2/X800) and RoCEv2 (Spectrum-X) fabrics, and optimise GPUDirect Storage and parallel file system (VAST Data) data paths to prevent GPU starvation
  • Maintain and tune the NVIDIA GPU Operator, Network Operator, Kubeflow and Slurm plugins across bare-metal and containerised environments, and design automated node-cordoning, pod eviction and repair workflows driven by real-time telemetry
  • Drive full-stack Infrastructure-as-Code (Terraform, Ansible, Python) for fleet expansion and region bring-ups, while leading war-room resolution of major incidents, managing NVIDIA and OEM vendor escalations and running blameless post-mortems


Skills/Must have:

  • Experience 5+ years in SRE, Systems or Infrastructure Engineering supporting high-density, production-scale Linux environments, including 3+ years managing and troubleshooting data centre class NVIDIA GPUs (H100, B200 or newer) in distributed environments
  • Deep expertise in the NVIDIA stack (CUDA runtime drivers, DCGM metrics, NVLink/NVSwitch topologies, XID error troubleshooting); advanced Kubernetes administration (CKA level, including CNI and Device Plugins) and Slurm
  • InfiniBand subnet management, RoCEv2 lossless Ethernet (PFC, ECN) and NCCL tuning; strong Python, Go or Bash alongside Terraform and Ansible; Prometheus, Grafana and vector-based telemetry stacks for hardware observability
  • Calm technical leadership under high-pressure incidents; excellent technical writing for post-mortems, runbooks and cross-team documentation


Desirable Skills:

  • NVIDIA Certified Professional certifications (NCP-AII, NCP-IB)
  • Certified Kubernetes Administrator (CKA) or Security Specialist (CKS)
  • Experience with liquid-cooled rack architectures (Direct-to-Chip, CDU setups)


Benefits:

  • 4-day working week (excluding event attendance)
  • Uncapped holiday
  • Remote-first working with flexibility and autonomy
  • Competitive compensation and benefits package
  • Inclusive, collaborative culture


Salary:

  • £120k
Jamie Maher Head of AI Infrastructure

Apply for this role