ProceedingsShort paper
Detecting Silent Tool Failures from Output Distributions
Some tool failures return plausible but wrong output and raise no error. We detect such silent failures by comparing the distribution of tool outputs against a rolling reference and flagging shifts. On three deployed tools the detector finds 71% of injected faults with a 2% false-alarm rate.