Advanced project·3-4 hours

ML Serving Readiness Project

Turn quality evidence and traffic assumptions into a capacity model, SLO, rollout, and rollback plan.

Scenario

A model endpoint must serve 120 requests per second at peak while meeting a p95 latency target. One warm replica sustains 35 requests per second at the target latency.

You will demonstrate

Compute capacity with headroom.
Define complete serving SLOs.
Design canary and rollback evidence.

Project evidence

Complete each deliverable.

Deliverable 1

Build the capacity model

Compute replicas from peak traffic, measured capacity, and 30% headroom.

Evidence: Passing capacity calculation and assumptions table.

Deliverable 2

Specify the SLO

Define latency percentile, error rate, availability, quality, and observation window.

Evidence: A measurable SLO contract.

Deliverable 3

Design the load test

Include warm, cold, burst, overload, and dependency-failure cases.

Evidence: A reproducible load profile and acceptance criteria.

Deliverable 4

Plan canary and rollback

Choose canary size, guardrails, rollback trigger, and fallback.

Evidence: A staged release runbook.

Runnable Python lab

Calculate replica capacity with headroom

Compute the minimum whole number of replicas for peak traffic and a utilization headroom policy.

Test output

Run the code when you are ready. The first load downloads the in-browser Python runtime.

Code runs only in a time-limited browser worker. It is not sent to the IntuitiveAI server. Do not paste secrets.

Expected result

replicas: 5
ALL TESTS PASSED

Project defense

Why is average latency insufficient for an online inference SLO?

Assessment rubric

Capacity includes explicit headroom.
Tail latency and queueing are measured.
Quality remains a release dimension.
Rollback is automatic for defined critical failures.

Useful references

The ML Test ScoreA practical rubric for testing and monitoring production ML systems.

Project completion gate

Complete all deliverables, pass the automated code checks, and defend the key decision.

Deliverables 0/4Code pendingDefense pending