Design a Shared GPU Cluster Scheduler System Design Interview

1. Problem Statement

2. Architecture Discussion Map

Use this as one discussion aid, not a single correct answer. Your design should follow from the requirements, scale, and trade-offs you establish.

graph LR
    A["Design a Shared GPU Cluster Scheduler"]
    A --> F1["Resource model, topology, and gang scheduling"]
    A --> F2["Priority, quotas, preemption, and fairness"]
    A --> F3["Fragmentation, bin packing, and capacity forecasting"]
    A --> F4["Control-plane scalability and failure domains"]
    A --> F5["Checkpointing, retries, observability, and cost attribution"]

3. Key Focus Areas

• 1
  Resource model, topology, and gang scheduling
• 2
  Priority, quotas, preemption, and fairness
• 3
  Fragmentation, bin packing, and capacity forecasting
• 4
  Control-plane scalability and failure domains
• 5
  Checkpointing, retries, observability, and cost attribution

4. What Strong Candidates Should Demonstrate

• ✓Model jobs with different priorities, topology, duration, and failure semantics.
• ✓Balance utilization against fragmentation, fairness, and production SLOs.
• ✓Limit control-plane blast radius at thousands-of-node scale.

Evaluation Guide

This is an evaluation framework, not a single model answer. Strong designs may make different choices when their assumptions and trade-offs are explicit.