Now that I’ve caught your attention with a sufficiently provocative title, please check out this new Talos regression policy that we* will be trying out starting next week

https://groups.google.com/forum/#!topic/mozilla.dev.platform/QHdn-ogf8kQ

tl;dr: Perf sheriffs will back out any Talos regression of 10% or more if it affects a reliable test on Windows. We’ll give the patch author 24 hours to explain why the regression is acceptable and shouldn’t be backed out. Perf sheriffs will aim to have such regressions backed out within 48 hours of landing.

I promise this policy is much more nuanced and thought-through than the title or summary might suggest But I really want to hear developers’ opinions.

* I’m taking point on publicizing this new policy and answering any questions, but Joel Maher, William Lachance and Vaibhav Agarwal of the A-Team did all the heavy lifting. They built the tools for detecting & investigating Talos regressions and they’re the perf sheriffs.

Avi Halachmi from my team is helping to check the tools for correctness. I just participate in Talos policy decisions and occasionally act as an (unintentional) spokesperson

https://blog.mozilla.org/vdjeric/2015/08/14/new-policy-24-hour-backouts-for-major-talos-regressions/