MLOps Handoff — Churn Prediction System

System Status

Production

Serving Mode

Batch daily

MLOps Maturity

Level 2

Active Model

churn-xgb-v2 (canary 10%)

Open Risks

1 medium

Ownership

ML lead@sarah (AI Platform)

MLOps / infra@ops-team (receiving)

Product stakeholder@cx-retention

On-call#ml-oncall (PagerDuty)

Incident channel#incidents-ml

Runbook locationNotion: ML Runbooks / Churn

Infrastructure

TrainingAWS SageMaker Pipelines

Feature storeTecton (online + offline)

Artifact registryMLflow (S3 backend)

InferenceAWS Batch + ONNX Runtime

MonitoringEvidently + Grafana

CI/CDGitHub Actions → SageMaker

Open Risks for Receiving Team

MEDIUM Fairness audit incomplete — Age cohort FPR gap

False positive rate parity on <30 vs 30-60 age cohort is 0.14 vs threshold 0.10. Investigation and potential retraining required. Target Sprint 14 (ETA 2026-07-06). Governance board approval blocked until resolved.

Artifact Index

◈architecture-review.html — System design, two-lane topology, MLOps maturity assessment
◈eval-scorecard.html — 5-layer testing, slice metrics, promotion gate policy
◈production-readiness.html — Rollout strategy, monitoring SLOs, rollback plan
◈governance-checklist.html — Risk tier, model card, fairness audit, GDPR compliance

On-call Runbook (Quick Reference)

🔴 Batch job failure

Check SageMaker Pipelines console — look for FAILED execution after 02:00 UTC
Alert fires to #ml-oncall if no successful run by 04:00 UTC
If extraction failure: check Tecton feature freshness — source table may be delayed
Rollback: re-run previous day's v1 inference with ACTIVE_MODEL=v1 SSM param

⚠️ Drift alert — PSI > 0.20

Grafana dashboard: ML Monitoring / Churn — check feature drift panel
Identify drifting feature(s) — contact @data-eng if upstream schema changed
If recall also degraded: escalate to @sarah for retraining decision
Do not promote v2 to 100% if drift is active