How to design an AI infrastructure team and what to hire first

Fabric and networking owns the interconnect: topology, congestion, collective-communication performance. Platform engineering owns the internal tools other teams use: scheduling, tooling, self-service. SRE owns availability commitments: error budgets, incident response, observability. MLOps owns the model lifecycle: training pipelines, artifacts, serving. Capacity planning owns the growth plan. It matches power, supply and demand to what can be built.

Each function hires from different backgrounds, gets screened on different work history, and pays at a different band. When they are opened as one family of requisitions, everyone on the team ends up working in the same layer, and problems in the other layers have no owner.

• ●Fabric/networking: InfiniBand or RoCE topology, congestion, collectives
• ●Platform: scheduling, developer surface, adoption of internal tooling
• ●SRE: error budgets, incidents, observability when hardware, power or cooling fails
• ●MLOps: training pipelines, checkpoints, serving latency, GPU utilization
• ●Capacity planning: power- and supply-constrained growth modeling

Experienced fabric engineers work in HPC labs, federal research computing, hyperscale network engineering and the silicon ecosystem. It is a small group, few of them are job hunting, and they respond when outreach describes the actual cluster. Platform engineers come out of internal developer platform teams at scaled software companies and move over well, as long as the interview asks whether other teams adopted what they built.

There are plenty of SRE candidates, and many of them are a poor fit: an SRE from a pure-software environment has rarely debugged an incident caused by heat, power or optics. MLOps candidates split into people who support training and people who support serving, and one group usually can't cover the other. Capacity planning candidates are hard to find, and many come from data center development, utility interconnection or supply chain instead of engineering.

• ●Fabric: HPC and research computing, hyperscale networking, silicon ecosystem
• ●Platform: internal developer platform teams at scaled software companies
• ●SRE: plenty of candidates; screen for incidents caused by hardware, power or cooling
• ●MLOps: training-side and inference-side backgrounds
• ●Capacity planning: development, interconnection and supply chain backgrounds

Candidates across these functions use the same tools and vocabulary, so ask what each person owned. Ask what they were paged for, what they built that others adopted, and what number moved because of their work. A platform engineer answers in adoption and toil removed; an SRE in incidents and budgets; an MLOps engineer in training throughput and serving latency.

One question works for every function: what did you personally do the last time the cluster was degraded and the cause was outside your layer? Candidates who have worked on a physical fleet describe working with facilities, vendors and the network team, and can tell you which alert or metric pointed them to the cause. Candidates who have only worked on cloud capacity describe escalating and waiting.

• ●Fabric: which collectives regressed, and what in the topology explained it
• ●Platform: what percentage of teams adopted the surface, and why the rest did not
• ●SRE: the error budget policy they enforced and the release it stopped
• ●MLOps: checkpoint recovery time and GPU utilization before and after
• ●Capacity planning: the plan they signed that turned out wrong, and why

First cluster landing: one operations-capable engineer and fabric depth. Researchers losing time to infrastructure: platform. Availability in contracts: SRE. Training and serving stabilizing: MLOps. Growth limited by power or supply: capacity planning.

Each hire changes what the next one can be. A strong fabric engineer hired first makes the platform role easier to scope and easier to sell, because candidates can see the fleet is stable. A platform engineer hired first into an unstable fleet spends a year firefighting, which is not what they were hired for, and leaves.

• ●Stage 1, hardware landing: fleet and fabric
• ●Stage 2, researchers blocked: platform engineering
• ●Stage 3, external commitments: SRE and on-call structure
• ●Stage 4, pipelines in production: MLOps
• ●Stage 5, power- or supply-limited growth: capacity planning

When an AI infrastructure team is struggling, we usually find one of these four setups.

• ●Three platform engineers and no fabric owner: everything works until the interconnect degrades, and then nobody can read it.
• ●SRE hired in name only: on-call exists, error budgets do not, and reliability depends on a few people working late.
• ●MLOps absorbed into platform by accident: training pipelines get attention only after the developer tooling is done, which rarely happens.
• ●Capacity planning owned by finance alone: the model covers cost and leaves out power, so the company commits to growth it cannot build.

Platform and MLOps promote well from strong internal software engineers who already know the workloads, provided someone senior owns the standard. Fabric and capacity planning rarely do: both depend on judgment built over several builds and several failures, and someone learning on the job needs a senior person in-house to teach them.

Spend outside search budget on fabric and capacity planning, and spend management time developing platform and MLOps engineers internally.

Before opening any requisition, put a number on the problem: researcher queue hours, utilization, incident hours, or the gap between contracted and sellable capacity. Scope the first two roles against that number, and write the scorecard around work the candidate has done, such as clusters run and incidents owned. Run both searches at the same time, since the candidates for each one don't overlap.

At day sixty, check the same number again. If it has not moved, look at the scope before the sourcing: the role as written may ask for a combination of experience that very few people have.