Releng systems are complex, with many moving parts. Making changes to these systems used to be fraught with peril. Even for people who had been working with buildbot for many years, it was hard to write a patch and be certain that wouldn’t break some downstream system.

Last year, releng introduced a suite of unittests for the repository that was the worst offender, buildbot-configs. These tests run under tox. We then took the next logical step, hooking those unittests up to travis. Now every time code lands in buildbot-configs — and more recently in buildbotcustom, mozharness, tools, and more — travis tests get run automatically against the github mirror once the commit is synced from hg.

However, this workflow is still not ideal. These tests only run on code that has already made it into the system. Patch authors needed to setup their own testing environments for working with hg. It would be better if contributors could get feedback *before* submitting a patch for review and without needing to setup the test environment themselves beforehand. If they could run the exact same tests, they could then include links to the test results for their patch along with their review request, making the reviewers job much simpler. Some might simply call this a “modern workflow.”

This modern workflow doesn’t exist with hg (we never wrote it), but we get it almost by accident if we switch to using github for development.

Today, if a user forks the buildbot-configs repo (or any of the other repos set up with travis) and has enabled travis testing on the fork, they get the same tests run against their fork automatically when they commit. This will be a boon for releng in terms of increasing the velocity of testing and reviewing patches. It also makes it dead-simple for people *outside* of releng (sheriffs, a-team, …) to contribute solid patches.

We are not setup for pull requests yet, but releng *is* planning to switch our repositories-of-record (RoRs) from hg to github. On the list of things we still need to figure out is how release tagging would work when github is our new RoR. We are also in the midst of adding even more useful informational output to travis, namely diffs of builder lists and master configurations caused by a given commit. This is one of those key elements we look at when evaluating the fitness of a particular patch.

If you’ve been holding off on getting your hands dirty with releng automation, there’s no better time. Get forking!

http://coop.deadsquid.com/2015/03/better-releng-patch-contribution-workflow/