catlee and coop returned from PTO, and lo, there was much rejoicing, mostly from Amy who held down the fort while the rest of her management team was off doing other stuff.

Modernize infrastructure: Thanks to Morgan, we can now trigger TaskCluster (TC) jobs based on GitHub pushes and pull requests. See the TC docs for more info: http://docs.taskcluster.net/services/taskcluster-github/

Dustin started a discussion thread about changing how we do Linux builds in TC: https://groups.google.com/forum/#!topic/mozilla.dev.builds/xmJCsSUDywE

Improve CI pipeline: Created a development environment for the Buildbot Taskcluster Bridge, which will make development of it go faster and safer. (https://bugzil.la/1199247)

Various improvements to the Buildbot Taskcluster Bridge

We made some upgrades to releng systems to allow them to take advantage of mercurial clones being served from the CDN. See gps’ blog post for more details: http://gregoryszorc.com/blog/2015/09/01/serving-mercurial-clones-from-a-cdn/

Release: We put the finishing touches on a couple of releases this week, namely Thunderbird 38.2.0 and Firefox 41.0b7. Jordan just stepped into the releaseduty role for the Firefox 41 cycle and is doing great work by all accounts.

Operational: The tree-closing window (TCW) came and eventually went over the weekend. A few things went sideways:

• the buildbot database cut-over ran into sync issues, but the old production masters got updates immediately. This prevented a clean rollback, which made recovery very challenging.
• There were unrelated problems with the firewall near the end of the TCW, which delayed the aforementioned recovery.
• We generally concluded that we were trying to do too many things during this particular TCW, largely because it was the last TCW of the quarter (Q3). We’ll try to avoid doing this in the future.

Thanks to everyone who helped wrestle our systems back under control: Hal, Rail, Jordan, and especially Nick who spent a substantial portion of his New Zealand weekend getting things working again.

See you again next week!

http://coopcoopbware.tumblr.com/post/128357098165