Welcome to the weekly releng Friday update, Whistler hangover edition.

Half the team took time off after Whistler. With a few national holidays sprinkled in, things were pretty slow last week. Still, those of us who were still around took advantage of the lull to get stuff done.

tl;dr

Taskcluster: Our new intern, Anthony Miyaguchi, started in San Francisco and will working on crash symbols uploads in TaskCluster. Our other intern, Anhad, has almost finished his work migrating spidermonkey to taskcluster. Morgan and Jonas are investigating task graph creation directly from github. Dustin continues to make efficiency improvements in the Fennec Taskcluster builds.

Modernize infrastructure: Mark, Q, and Rob continue to work on standing up our new Windows build platform in AWS. This includes measuring some unexpected performance improvements.

Improve release pipeline: We’re standing up a staging version of Ship-It to make it easier to iterate. Ben’s working on a new-and-improved checksum builder for S3, and Mike fixed a problem with l10n updates.

Improve CI pipeline: Jordan pushed the archiver relengapi endpoint and client live. They are now being actively used for mozharness on the ash project branch. catlee deployed the hg bundleclone extension to our Mac and Linux platforms, and Rail deployed a new version of fun size with many integrity improvements.

Release: Firefox 39.0 is in the wild!

Tune in again next week!

…

And here are all the details:

Taskcluster

• Our intern, Anhad, has nearly finished porting Spidermonkey to TaskCluster (https://bugzil.la/1164656). He’ll have a blog post with details coming up shortly.
• Morgan decided it would be a good idea if the container we used to build 32-bit Linux builds was under our direct control, so we spent some time this week putting one together. (https://bugzil.la/1178161)
• Morgan and Jonas began sketching out how we can create task graphs directly from github integration by using organization hooks (https://bugzil.la/1179458). This will be an important piece for autoland.
• Dustin continues to make efficiency improvements in the Fennec TC builds. Last week, he worked installing java for android builds via tooltool rather than on the base image for the build host. (https://bugzil.la/1161075). He also filed a bug to make sure the tooltool client deletes downloads after unpacking to avoid some of the existing overhead incurred by using tooltool for an increasing number of things (https://bugzil.la/1179777).
• Dustin also wrote a blog post about how to run ad-hoc task using taskcluster: http://code.v.igoro.us/posts/2015/07/ad-hoc-tasks-in-taskcluster.html

Operational work

• Coop was in California to onboard our new intern, Anthony Miyaguchi. Anthony’s first task will be figuring out how to upload symbols to Socorro API from a separate task in TaskCluster (https://bugzil.la/1168979). Welcome, Anthony!

Modernize infrastructure

• Microsoft is deprecating the use of SHA-1 Authenticode signatures at the end of the year. This is good for the safety of internet users in general, unless you happen to still be on Windows XP SP2 or earlier which do _not_ support SHA-2 or other newer signature algorithms. This means that if Mozilla needs to ship to XP SP2 _and_ XP SP3 (and higher) after 2016-01-01, we may not be able to ship them the same binaries. Ben spent some time looking at potential solutions this week. (https://bugzil.la/1079858)
• Mark has been adding new xml-based config support options for our new Windows instances in ec2 (https://bugzil.la/1164943).
• Having Windows builds in AWS is a win from a scalability standpoint, but early measurements indicate that builds in AWS might also be faster than our current hardware builders by up to 30%. Q has been capturing build time comparisons for build slaves in AWS versus colo hardware to get a better picture of the potential wins (https://bugzil.la/1159384).
• Rob deployed an updated version of runslave.py to all Windows slaves, finishing off a round of standardization and correctness work begun a few months ago (https://bugzil.la/115406).

Improve release pipeline

• As part of our ongoing migration off of our monolithic ftp server, Ben has been iterating on getting checksum builders working with S3 (https://bugzil.la/117414). The eventual goal will be to run these as tasks in Taskcluster.
• In Whistler, we decided it was important to maintain a staging version of release runner / ship it to allow us to iterate more quickly and safely on these tools. Ben spent some time this week planning how to facilitate supporting a staging version alongside the production deployment (https://bugzil.la/1178324).
• Mike landed a fix to a branding issue that was preventing developer edition l10n builds from correctly identifying themselves to the update server (https://bugzil.la/1178785).

Improve CI pipeline

• At the start of the quarter, Jordan was tasked with moving mozharness into the gecko tree. Through discussion with Dustin and others, this gradually morphed into creating a relengapi endpoint that allows you to get a tarball of any repo and rev subdirectory, upload it to s3, and download/unpack it locally. This would pave the way for being able to put *anything* in the gecko tree by reference, and still being able to deploy it or use it without checking out the tree. The archiver relengapi endpoint and client is now live and being actively used on the ash project branch. We hope to uplift it to mozilla-central by the end of this week (https://bugzil.la/1131856).
• coop added buildprops.json to the list of files uploaded to TaskCluster as part of the build process (https://bugzil.la/117709). This makes it easier for developers to replicate buildbot builds locally or on loaned machines because now they have access to the same seed variables as buildbot.
• catlee deployed the hg bundleclone extension to our Mac and Linux platforms last week (https://bugzil.la/1144872). Coupled with the server-side improvements deployed by dev services recently, we are gradually reducing the complexity and overhead of VCS operations that we need to maintain in releng code.
• Rail deployed a new version of funsize (0.11), our on-demand update generation service. The new version includes improvements to use whitelisted domains, and virus scanning complete and performing signature verification on complete MAR files before creating partial MARs (https://bugzil.la/1176428).

Releases

• Rail was on the hook for the release of Firefox 39.0 and certainly had his hands full iterating though 6 separate release candidates: https://wiki.mozilla.org/Releases/Firefox_39.0/BuildNotes
• Jordan was on merge duty, so he handled the uplifts between the various release branches.

See you next week!

http://coopcoopbware.tumblr.com/post/123400790730