Major Highlights:

• armv6 builds and tests are disabled
• ec2 instances avail of a proxy server that stores local cache of ftp downloadables
• runner, service that runs startup tasks so we don't need to reboot after every job, enabled for centos machines.
• desktop builders for mac and windows join linux on cedar as they are run via mach and mozharness

Completed Work (marked as resolved):

• Balrog: Backend
• XML generation should be a function of blobs, not handled externally
• Buildduty
• Completed builds not appearing on TBPL
• upgrade ec2 linux64 test masters from m3.medium to m3.large
• Install fxa-python-client on internal PyPI
• clobberer step is failing to clobber
• b2gbumper is stalled
• Firefox mozilla-central linux, linux64, and android2.2 l10n nightly builds are busted
• Windows 8 tests failing across all branches
• Wait times > 16h, pending builds > 7000, ix jobs ~3000 reftest/crashtest/jsreftest, 10.8 ~10h behind
• hitting ISE 500's while pulling from hg.m.o to buildbot repos across our masters. buildbot hgpollers also failing
• All Trees Closed -> building backlog of linux jobs because of issues with dynamic jacuzzi allocation
• B2g bumper bot problems - hasn't made commits for a while - Gaia-feeding trees closed until bumper bot is fixed
• Something wrong with b-2008-ix-002x slaves
• B2g-inbound tree closed due to pending B2G ICS Emulator Opt builds
• All trees closed due to AWS builder lag
• Integration Trees closed due Android tests fail Automation Error: Unable to reboot panda-xyz via Relay Board.
• builds-4hr not updating, buildapi returning 503s, all trees closed
• Failure "TypeError: request() got an unexpected keyword argument 'config'" on amazon dnsless spot instances
• General Automation
• Increase free space requirement for release-mozilla-beta-androidbuild jobs
• [Flame] Turn off FX OS Version 34.0a1 nightly updates until instability is fixed
• endtoend reconfig script should run its python unbuffered
• Run gaia unit tests on debug linux64 B2G desktop builds
• Disable non-unified builds on b2g32
• Disable Jetpack tests on B2G release branches
• Run reftests on B2G Desktop builds
• Schedule Android 2.3 crashtests, js-reftests, plain reftests, and m-gl on all trunk trees and make them ride the trains
• Make runner responsible for buildbot startup on CentOS
• Cleanup strays in scheduler db (bogus m-i win pgo, Mulet)
• Try is attempting to trigger periodic non-unified builds
• No updates served for mozilla-esr31 nightly builds
• Hazard failures with rsync: connection unexpectedly closed (0 bytes received so far) [sender]
• Upload new version of xulrunner to tooltool
• Don't schedule non-unified B2G hazard analysis builds
• Explicitly list gaia-try's priority rather than relying on the fallback
• Get e10s tests running on inbound, central, and try
• Schedule Android 4.0 Debug reftests and crashtests on all trunk trees and let them ride the trains
• Add task-specific configuration to Runner
• lock reporepo to a tag - b2g builds failing with AttributeError: 'list' object has no attribute 'values' | caught OS error 2: No such file or directory while running ['./gonk-misc/add-revision.py', '-o', 'sources.xml', '--force', '.repo/manifest.xml']
• Fabric should not try and prompt when we fork it.
• Device Builds Tests/Builds for mozilla-inbound
• Allow test machines in AWS to get to github via internet gateway
• Cleanup bogus Linux PGO jobs in scheduler db
• Improve slaveapi logging
• Re-enable us-west-2 when AWS is working again
• B2G builds failing with | gpg: Can't check signature: No public key | error: could not verify the tag 'v1.12.4' | fatal: repo init failed; run without --quiet to see why
• Loan Requests
• Slave loan request for a t-snow-r4 machine
• Slave loan request for a bld-lion-r5 machine
• Other
• Stage MozillaBuild 1.10.0
• Platform Support
• Updated android builders to include google play services (and android support v7 if it isn't already included)
• Disable armv6 builds and tests everywhere apart from esr31
• Intermittent Android x86 We have not been able to establish a telnet connection with the emulator
• Release Automation
• release runner failure e-mail should be sent to relman or release-drivers
• release runner doesn't include enough context for some errors
• Releases
• tracking bug for build and release of Firefox and Fennec 31.0
• Releases: Custom Builds
• Version bumps for Yahoo FF 30
• Repos and Hooks
• Give Treeherder URLs alongside TBPL ones in the hg.mozilla.org push response
• Tools
• Allow editing of relevant slavealloc fields directly from slave health
• slavealloc API URL used when making changes from slavehealth contains an additional forward slash
• slavehealth stopped updating after 13:40 Friday
• Give Treeherder URLs in Try responses / emails.
• Aggregate similar exceptions in emails from masters
• Slaverebooter should handle its jobs in a Queue rather than kickoff a single thread per slave.

In progress work (unresolved and not assigned to nobody):

• Balrog: Backend
• support comparison operators for matching version & buildID in rules
• Buildduty
• Try jobs backing up - need to deprioritise ash
• Decommission foopy117 and associated tegras
• Legacy vcs sync not able to update legacy mapper quick enough for b2gbuild.py, causing b2g builds calls to mapper to timeout
• [tracking] Eliminate buildduty
• General Automation
• buildbot changes to run selected b2g tests on c3.xlarge
• Add dependency management to Runner
• Please schedule b2g emulator cppunit tests on Cedar
• FlatFish: Integrate boot.img and recovery.img into the build system
• Stop rebooting after every job
• Run mock against repos other than puppetagain
• android single locale central/aurora nightlies not reporting to balrog
• generate "flatfish" builds for B2G
• Make runner responsible for buildbot startup on Ubuntu test
• Automatically delete obsolete latest-* builds for calendar
• Make some mozharness jobs easier to run outside of automation + allow for http authentication when running outside of the releng network
• Figure out the correct path setup and mozconfigs for automation-driven MSVC2013 builds
• Schedule mochitest-plain on linux64-mulet on all branches
• AWS region-local caches for https stuff
• Loan Requests
• Slave loan request for tst-linux64-spot
• Loan an ami-6a395a5a instance to Aaron Klotz
• Loaner slave matching WinXP mochitest1 slaves for :walter
• Loan an ami-6a395a5a instance to Aaron Klotz
• Slave loan request for a bld-linux64-ec2 vm
• Mozharness
• Make sure to mark jobs as failures when structured logger parsing fails
• fx desktop builds in mozharness
• Other
• Improve cleanup of tinderbox-builds
• Figure out retention of flame tinderbox-builds
• New AWS account, 'mockbuild-repos-upload'
• Create docs for l10n from a RelEng perspective (repos, locale lists, per product/branch explanations, etc) to allow newcomers to navigate this landscape
• Need read permissions to subnets
• Windows builds on cedar broken
• Platform Support
• Upload JDK1.7 on builders
• c3.xlarge instance types are expensive, let's test running those tests on a range of instance types that are cheaper
• Change Android 2.2 builds to be called Android 2.3 builds
• evaluate mac cloud options
• Release Automation
• release automation should update ship it at certain points
• pushing release builds to beta doesn't work in balrog
• Add new table(s) to shipit database
• Figure out Play Store dynamics for Android 2.2/ARMv6 EOL support releases
• adjust e-mail for updateshipping builder to be more intelligible
• Add a standalone process that listens to pulse for release related buildbot messages
• flip the switch to move beta users over to balrog
• Add REST API entry point to shipit that allows shipit-agent to enter release data into shipit database
• make aus3 and aus4 responses consistent
• Releases: Custom Builds
• add new image sizes to Bing/MSN search plugins
• Remove Blackbird builds for Fx32
• Remove Rambler config to inactive and remove builds from FTP
• Version bumps for Yahoo FF 32
• Repos and Hooks
• esr31 branch on https://github.com/mozilla/gecko-dev
• Consider disabling or redirecting bundle archives
• Tools
• Funsize requires file level diff caching for speedups in partial generation
• Funsize should return correct MIME type
• Add linux64-mulet to trychooser
• cut over l10n repos to the new vcs-sync system
• Reconfigs should be automatic, and scheduled via a cron job
• implement 'awscreateinstance' action in slaveapi
• permanent location for vcs-sync mapfiles, status json, logs
• Create a Comprehensive Slave Loan tool
• vcs-sync needs to be able to account for empty source repos
• two new instances like vcssync{1,2}, puppetized

http://jordan-lund.ghost.io/this-week-in-releng-july-28th-2014/