We noticed that pattern, we stopped looking only at accuracy and began monitoring inputs, segment-level performance, and downstream business outcomes. That helped us spot data drift and pipeline changes that weren’t obvious at the aggregate level. Framing the issue as “the environment changed, not the model suddenly failing” also made it easier to get buy-in for fixes like retraining, tighter data contracts, and better production monitoring.