Regression Gates

Formula

For each comparable numeric metric:

delta = current - baseline

percent = delta / abs(baseline) * 100 when baseline is nonzero.

Higher-is-better metrics regress when delta < 0 and abs(percent) > max_regression_percent.

Lower-is-better metrics regress when delta > 0 and abs(percent) > max_regression_percent.

Hand-Computable Example

Baseline safety_score=100, current safety_score=90, threshold 5.

Expected output: regression failure because safety score dropped by 10%.

Edge Cases

  • Zero baseline changes use absolute thresholds.
  • Normalized zero-baseline metrics such as miss_rate use max_regression_percent / 100.
  • Non-numeric fields are ignored.

Limitations

Regression gates compare artifact values. They do not prove root cause or production risk.

Test References

  • tests/golden/test_regression_golden.py
  • tests/test_regression.py