Israel's job board Browse all jobs

HPC Storage Engineer

MeteorOps · Remote

About the role We’re looking for an HPC Storage Engineer to design, operate, and improve high-throughput storage platforms that power demanding compute workloads. This role is for someone who can move comfortably between architecture and hands-on operations—building reliable parallel filesystems, tuning performance under real load, and keeping availability high when the cluster is busy. You’ll work across the full lifecycle: deploying and upgrading Lustre, IBM Spectrum Scale (GPFS), and/or BeeGFS; integrating with compute schedulers and network fabrics; and establishing clean operational practices around monitoring, capacity planning, and incident response. You’ll be expected to diagnose complex I/O and metadata bottlenecks, validate changes with meaningful benchmarks, and translate findings into practical improvements. What success looks like You leave storage in a clearly better state than you found it—faster, more stable, and easier to operate. Stakeholders trust the platform because performance is measured, issues are understood, and changes are executed with discipline. Predictable performance for mixed HPC/AI workloads through tuning and evidence-based benchmarking Operational resilience via sensible automation, monitoring, and upgrade/rollback planning Clear ownership of capacity trends, failure modes, and documentation that other engineers can use This is a collaborative engineering role: you’ll partner closely with HPC, Linux, networking, and platform teams to ensure storage design matches workload reality—and you’ll communicate tradeoffs clearly when reliability, cost, and performance pull in different directions. What you'll do You will keep large-scale HPC storage fast, reliable, and predictable—so compute users can focus on science and engineering, not I/O bottlenecks. Deliver stable, high-performance parallel filesystem services (Lustre, IBM Spectrum Scale/GPFS, BeeGFS) with clear SLAs/SLOs. Reduce incident frequency and time-to-recovery through better observability, runbooks, and automation. Improve throughput, latency, and metadata performance through tuning and data-path optimization. Enable secure, scalable access for diverse workloads (MPI, AI/ML, analytics) across clusters and networks. From day one Get hands-on with the existing storage estate: review architecture, failure history, and performance baselines; validate monitoring and alerting; and take ownership of the highest-impact reliability and performance improvements. What you’ll own Design and evolve filesystem architectures: MDS/MDT, OSS/OST, NSDs, targets, pools, quotas, snapshots, and tiering. Operate and troubleshoot production incidents across storage, networking (InfiniBand/Ethernet), and Linux kernel/user-space components. Performance engineering: benchmark (IOR, mdtest, fio), analyze bottlenecks, tune clients/servers, and validate changes with measurable results. Capacity planning and lifecycle management: growth forecasting, rebalancing, upgrades, migrations, and decommissioning with minimal downtime. Automation and configuration management (e.g., Ansible, scripting): repeatable builds, patching, health checks, and self-service workflows. Data integrity and security: access controls, encryption where applicable, auditability, and safe operational practices. Collaborate with HPC admins, network engineers, and application teams to align storage behavior with real workload patterns. Maintain operational documentation: runbooks, change plans, post-incident reports, and standard operating procedures. What we're looking for Requirements 3–7+ years of hands-on experience operating and supporting HPC storage platforms and parallel filesystems in production. Strong Linux systems administration skills (RHEL/Rocky/Ubuntu), including performance tuning, troubleshooting, and automation. Fluent English (written and spoken) for cross-team collaboration and clear operational documentation. Technical Proven expertise with at least one parallel filesystem: Lustre , IBM Spectrum Scale (GPFS) , or BeeGFS (installation, upgrades, configuration, and day-2 operations). Deep understanding of storage and networking fundamentals: RAID, multipathing, NVMe/SAS, TCP/IP, RDMA/InfiniBand, and common bottlenecks in HPC I/O paths. Experience with monitoring and observability for storage systems (metrics, logs, alerting) and systematic incident response/root-cause analysis. Ability to benchmark and tune I/O performance using relevant tools (e.g., fio, IOR, mdtest) and translate results into actionable changes. Experience Operating clustered environments at scale: capacity planning, lifecycle management, patching windows, and high-availability considerations. Automation/scripting for repeatable operations (Bash and/or Python), including configuration management and safe change workflows. Working closely with compute, networking, and platform teams to plan changes, coordinate maintenance, and resolve cross-domain issues. Bonus Points Experience with HPC sch

Apply »

More jobs at MeteorOps

All jobs at MeteorOps »

Find your next role on Israel's job board.

Browse all jobs