Major Highlights:

• new page for hg status and issues: https://wiki.mozilla.org/DeveloperServices/hg
• improve capacity by running debug tests based off every 2 builds instead of on each one for on mozilla-inbound: bug 1056787

Completed work (resolution is 'FIXED'):

• Buildduty
• update bm-remote with new mobile_tp4.zip file
• Builds fail with file not recognized: File truncated
• builds-4hr.js.gz et al aren't updating
• Trees closed due to Windows test backlog
• upload a new mobile_tp4.zip pageset to the 3 headed remote talos server (round 2)
• upload a new mobile_tp4.zip pageset to the 3 headed remote talos server
• Last non-pending Linux 32 opt build on fx-team was 17 hours ago
• Frequent timeouts downloading from pvtbuilds and usw2 proxxy
• "No such file or directory" error causing upload failures
• Linux64 non-unified builds aren't starting on mozilla-central
• upload a new talos.zip file to get nonlocal source fixes
• Legacy vcs sync not able to update legacy mapper quick enough for b2g_build.py, causing b2g builds calls to mapper to timeout
• Failures on Android 2.3
• Determine why batches of pandas stopped reporting on 2014-04-25 and 2014-07-13
• Intermittent "Unable to remove C:\slave\test\build!"
• Windows builds pending on inbound for > 2 hours
• Install boto on internal PyPI
• Decommission foopy117 and associated tegras
• Machine being used for tests added to prod pool before being cleaned up
• Install treeherder-client on internal PyPI
• Install fxa-python-client on internal PyPI
• b2g_bumper is stalled
• ESR31 fennec symlinks missing from buildbot masters
• please loan mshal a windows build machine without vs2013
• General Automation
• Make blobber uploads discoverable
• Increase free space requirement for release-mozilla-beta-android_build jobs
• Run linux64-br-haz builder on b2g-inbound/mozilla-inbound/fx-team/mozilla-central and try-by-default
• Self-serve should be able to request arbitrary builds on a push (not just retriggers or complete sets of dep/PGO/Nightly builds)
• [Flame] Turn off FX OS Version 34.0a1 nightly updates until instability is fixed
• Clean up / handle case where spot request is active but instance has gone away
• Requesting twig repo gum be reset
• android single locale central/aurora nightlies not reporting to balrog
• Disable non-unified builds on b2g32
• get rid of comm-release configs/hacks
• Enable e10s Linux opt browser chrome tests
• Disable Jetpack tests on B2G release branches
• Run reftests on B2G Desktop builds
• You can't choose mochitest-N where N>8 for Android specific mochitests
• Do debug B2G desktop builds
• Enable ASAN builds/tests on Holly
• There hasn't been a single successful Windows opt/nightly build off a b-2008-sm slave since the uplift
• create webgl mochitest run for android
• Make runner responsible for buildbot startup on CentOS
• Schedule all non-perf unittests against Linux64 Code Coverage builds on Cedar
• Allow test machines in AWS to get to github via internet gateway
• Run mock against repos other than puppetagain
• bump max token age on signing servers + clients
• create signing servers for v2 mac signing
• Monitor aws_stop_idle.py hungs
• B2G Updates are broken when transitioning off Mozilla-Aurora to versioned Gecko
• Try is attempting to trigger periodic non-unified builds
• Increase Android 2.3 mochitest chunks
• Turn on mochitest-e10s for ASAN builds on trunk
• Kill mozilla-b2g28_v1_3 after Sep 2 merge day
• Cleanup bogus Linux PGO jobs in scheduler db
• Create Linux64 code coverage builds on Cedar
• Intermittent command timed out: 3600 seconds without output, attempting to kill fetching b2g bits from gitmo
• Create builds for Dolphin on mozilla-central, mozilla-inbound, b2g-inbound, fx-team, and mozilla-aurora
• Upload new version of xulrunner to tooltool
• Add task-specific configuration to Runner
• Turn off Firefox desktop nightlies on the B2G release branches
• Enable b2g tests on jamun project branch
• (fx-l10n-sl) Add 'sl' to the Firefox Nightly build
• Please install Visual Studio 2013 on a Builder
• When requesting spot instances, give up on an az the first time there is no slave name
• Explicitly list gaia-try's priority rather than relying on the fallback
• Automatically delete obsolete latest-* builds for calendar
• Consider dropping update_spot_stats() in spot_sanity_check.py
• Figure out the correct path setup and mozconfigs for automation-driven MSVC2013 builds
• Expand xpcshell on mozharness for Thunderbird to gecko 33
• Get e10s tests running on inbound, central, and try
• Schedule mochitest-plain on linux64-mulet on all branches
• Add mozharness script for b2g builds in taskcluster
• Don't schedule non-unified B2G hazard analysis builds
• Adjust Thunderbird MozMill test installation to be able to install mozbase
• b2g branching cleanup
• enable os x l10n builds on oak
• Tracking bug for 02-sep-2014 migration work
• Please schedule b2g emulator cppunit tests on Cedar
• Start doing mulet builds on Fig
• Add --disable-mock option to b2g_build.py
• buildbot changes to run selected b2g tests on c3.xlarge
• Adjust cloudtrail related crojob times
• Infrastructure for 3rd-party builds, tests to react to 'build events'
• bump dmg max filesize
• upload a new talos.zip to pick up directory tile fixes
• hg hook update for mozilla-b2g18
• make sure signing server works on 10.9 and with v2 signatures
• repurpose a snow leopard machine for 10.9 mac signing
• Blobber doesn't appear to work on C++ Unit test runs
• Stop running builds and tests on pushes which only include changes to the b2g or mobile or browser directories
• cope with signing server transition in buildbot configs
• Move runner to the 'mozilla' github org
• Timeouts during pip install ("Error running install of package, /builds/slave/test/build/venv/bin/pip install --download-cache /builds/...", "Error running install of package, /builds/slave/talos-slave/test/build/venv/bin/pip install --download-cache...")
• Loan Requests
• Slave loan request for gbrown
• Slave loan request for some bld-lion-r5 machines in the try VLAN
• Slave loan request for gbrown
• Slave loan request for a b-2008-sm machine
• Slave loan request for a talos-mtnlion-r5 machine
• Need a WinXP test slave
• create AWS dev instance for zeller to showcase ship-ip ui changes
• Requesting a loaner machine b2g_ubuntu64_vm to diagnose bug 1017490
• loan request to investigate bug 1026805
• Requesting a loaner machine to diagnose bug 1039739
• Requesting a loaner machine b2g_ubuntu64_vm to diagnose Bug 1043892
• Slave loan request for a bld-linux64-ec2 vm
• Requesting Android 4.2 x86 emulator machine for jchen
• Slave loan request for a bld-lion-r5 machine
• Mozharness
• move ash-mozharness to hg.m.o/build
• vcs2vcs sync in mozharness does not work if repo_name is different to basename of repo url
• for mozharness desktop builds, replace usage of client.mk with upcoming mach/python script
• fx desktop builds in mozharness
• Failures during install_module() are shown as ERRORs in the log even though the run succeeded
• Turn on blobber upload for Gaia Unit Tests
• AndroidEmulatorTest should use MozbaseMixin
• Process extensions in packaged tests if they exist
• Thunderbird xpcshell tests need extra arguments in mozharness
• Investigate switching Thunderbird comm-central xpcshell-tests to mozharness
• Other
• Improve cleanup of tinderbox-builds
• [tracker] Machine move SCL1 -> SCL3
• Please create a Mobile1.1 tinderbox page
• Figure out retention of flame tinderbox-builds
• New AWS account, 'mockbuild-repos-upload'
• Rename all things Tryserver/MozillaTry to just Try in order to have uniform naming from repo-name on down
• Need read permissions to subnets
• Windows builds on cedar broken
• Create bookmarklet to simplify the TBPL -> 'viewing job on the buildbot master' workflow
• abort: pretxnchangegroup.c_try_mandatory hook failed
• linux desktop qt builds for firefox
• moving panda infrastructure from scl1->scl3
• Request for ElasticBeanStalk account to test deployment of build-funsize
• Platform Support
• c3.xlarge instance types are expensive, let's test running those tests on a range of instance types that are cheaper
• Upload new version of xulrunner to tooltool
• Updated android builders to include google play services (and android support v7 if it isn't already included)
• Configure AWS Windows Server 2012 instances for audio support
• prepare the new seamicro machines for production
• update zip and unzip on mac os x build/test machines to 6.0
• Intermittent Android x86 We have not been able to establish a telnet connection with the emulator
• Update Android SDK and build tools to 20
• Upload JDK1.7 on builders
• Update version of pip installed on automation machines from 0.8.2 to 1.5.5
• Determine number of iX machines to request for 2014
• Disable armv6 builds and tests everywhere apart from esr31
• Change Android 2.2 builds to be called Android 2.3 builds
• [mozpool-client] TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
• Release Automation
• xulrunner_source failed uploading file for signing, file too large
• enable spot instances for some release builders
• Automatic filling of the branch field in ship-it
• Dashboard check automatically managed by ship-it
• Tree closure hooks for esr31
• Fix the ship-it test suite
• Have release automation support signing OSX builds
• When no comment is provided, the email still show Comment:
• make aus3 and aus4 responses consistent
• Thunderbird ship-it notifications have wrong mozilla-beta revision
• Add new table(s) to shipit database
• send e-mail to release-drivers when a build is requested through ship it
• Fennec armv6 esr31 differences between release and nightly in-tree configs that are not whitelisted
• pymake hacks in release repacks drop all but one signing server from list
• Consider fully disabling mochitest-bc and mochitest-dt on the B2G release branches
• With French locale, the sort of the date is broken
• Add REST API entry point to shipit that allows shipit-agent to enter release data into shipit database
• update partner repack scripts to cope with new format after mac signing changes
• Tree closure hooks for esr17
• upload a new talos.zip to pick up all the fixes
• Add a footer to ship-it with a link to source + bugzilla
• Releases
• tracking bug for build and release of Thunderbird 31.1.0
• Sign and dev deploy the Firefox update hotfix (v20140527.01.4)
• tracking bug for build and release of Thunderbird 24.8.0
• tracking bug for build and release of thunderbird 31.0
• Add SeaMonkey 2.29 Beta 2 to bouncer
• Remove sw during beta-release migration for Firefox 31, keep hsb
• Trim rsync modules (May 2014 OMG we still have to do this edition)
• Add SeaMonkey 2.29 to bouncer
• tracking bug for build and release of Firefox 31.1.0 ESR
• Disable Flame OTA and rebuild with bug 937485 backed out
• bouncer checks for aurora stub installer broken
• Remove sw during beta-release migration for Firefox 32, keep dsb
• Add SeaMonkey 2.29 Beta 1 to bouncer
• tracking bug for build and release of Firefox 24.8.0 ESR
• outdated link: releases.mozilla.org/pub/mozilla.org/thunderbird/releases/latest
• Releases: Custom Builds
• Remove Blackbird builds for Fx32
• Partner repack changes for Yandex - new tab images (regular/retina)
• Modify AOL Repack Configuration for Firefox 32
• Version bumps for Yahoo FF 32
• Respin yandex-* and aol-* partner repacks.
• add new image sizes to Bing/MSN search plugins
• Partner repack changes for web.de, gmx, mail.com and 1&1 (Firefox 32)
• Remove Rambler config to inactive and remove builds from FTP
• remove out of date Bing plugins from yandex-tr configs
• Repos and Hooks
• The order of reviewers matters for the webidl commit hook
• fx-team branch of gecko-dev hasn't synced in 8 hours
• Make prevent_webidl_changes.py correctly deal with merges
• Roll local changes to treeclosure.py into repo (update treeclosure hook for mozilla-2.1, mobile-2.0, mozilla-2.0 trees)
• Give Treeherder URLs alongside TBPL ones in the hg.mozilla.org push response
• Mercurial extension for monitoring servers
• Change user to emailuser in hg annotate web interface
• Convert some hooks tests to Mercurial testing format
• push hooks should print out hg url for the changeset
• releases/mozilla-1.9.1, releases/mozilla-1.9.2, and releases/mozilla-2.0 repositories are missing
• Move hghooks repository into version-control-tools
• Tree closure hook should be more robust
• Request for a new repository for NFC emulation library
• Port Mercurial commit message hook tests to .t test
• Skip testTipDirRenameShouldFail on non-case sensitive filesystems
• Enforce a set of allowed commit messages as hg precommit hooks on hg.m.o
• Please create hg.m.o repo for b2g 2.0m
• Allow pushing Instantbird-only changes on a closed c-c- tree
• test-push_printurls.t failing on Linux due to shell syntax
• Reject changesets containing "[PATCH"
• https://hg.mozilla.org/repo-group?repo=/mozilla-central no longer works
• treeclosure hook fetches 2MB of HTML to get at a few hundred bytes that are in TreeName/status.html
• Implement a string checker hook on mozilla-aurora,mozilla-beta and mozilla-release repos to refuse string changes without specific approval
• Approval hook should not reject backout commits that do not contain an a=foo
• Make mercurial closure hooks fail closed once treestatus.mozilla.org proven reliable
• Sync two repos for dolphin builds on master
• esr31 branch on https://github.com/mozilla/gecko-dev
• remove jhopkins and mihai from releng whitelist
• We should probably reset try to stop breaking hg.m.o
• "sr=[DOM peer]" should be allowed as well as "r=[DOM peer]" for webidl changes
• Tools
• Verify that trychooser's tryload is looking at the right names for pending jobs and adding them up into the correct buckets
• Update platform parsing in buildapi wait times emails
• Deploy newer slaverebooter
• Add mochitest-e10s to trychooser
• tree closure hooks for integration/b2g-inbound
• Add gaia-build-unit-tests to Trychooser
• Allow filtering tests by b2g and Android
• Prune the allowed origins list for buildapi
• Add slavename to title for individual slave's slave health page
• Allow editing of relevant slavealloc fields directly from slave health
• Please deploy trychooser from tools tip to pick up bug 1057050 and bug 1057301
• slaverebooter should attempt reboot if graceful shutdown fails
• Funsize should return correct MIME type
• Give Treeherder URLs in Try responses / emails.
• Run slaverebooter more often than once every 4 hours
• Runner dies before starting buildbot due to hg failures and leaves slave doing nothing
• Funsize tool handling is broken and incorrect
• Deploy newer slaverebooter
• slavealloc API URL used when making changes from slave_health contains an additional forward slash
• Run manage_foopies.py after reconfig is completed in end_to_end_reconfig.sh
• Slaverebooter should honor pep8 and where-applicable pylint
• make "max log length exceeded" error message show the log size it found
• Deploy new vcs_sync into production with changes from bug 962853
• Add timeouts to buildapi's urllib.urlopen() requests
• Add linux64-mulet to trychooser
• Aggregate similar exceptions in emails from masters
• Funsize returns errors out and returns an HTTP 500 when requested for a non-existent partial
• Update trychooser for Android again
• Add web-platform-tests to trychooser

In progress work (unresolved and not assigned to nobody):

• Balrog: Backend
• create balrog rules to properly support OS blocking and watershed updates on beta (and equivalent test) channel
• add schema for h264 updates
• support comparison operators for matching version & buildID in rules
• Balrog: Frontend
• balrog's rules page is excessively wide
• Buildduty
• Slaverebooter hung on Aug 25
• loan t-snow-r4-0099 to jlund
• [tracker] windows build infrastructure issues
• Something wrong with windows build slaves ("LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt")
• Windows Builds fail with configure: error: To build the installer you must have the latest MozillaBuild or Unicode NSIS version .46 or greater in your path.
• General Automation
• Add nightlies for Mulet
• Enable chunking for B2G ICS Emulator Debug XPCShell tests
• try emails should include commit messages from pushed commits
• Allow to find which builders have changed before/after a patch
• [Tracking] Stand up Code Coverage build and test runs
• Thunderbird Linux and Mac MozMill builders are using the system python (fix bustage on Thunderbird Mac MozMill builders)
• Do open264 builds
• ALL b2g builds exceeding buildbot output limits
• Increase web-platform-tests to 4 chunks
• Make enable_perproduct_builds opt out rather than opt in
• disable uploading to update.boot2gecko.org for mozilla-central/mozilla-aurora/1.4
• web-platform-tests-reftests missing configuration file on linux-64-asan
• AWS region-local caches for https stuff
• Modify file structure of Firefox.app to allow for OSX v2 signing
• Add signing for firefox and org.mozilla.updater binaries on OSX
• Remove dependance on fennec_ids.txt from automation
• Splitting Gij and Gip tests in several chunks
• hgtool should retry or exit if it hits a DNS or server error during pull, not clobber and unbundle
• [tracking] rollout desktop mozharness mach builders across all branches
• Please schedule b2g emulator cppunit tests on trunk/aurora trees and have them ride the 34 train
• generate "flatfish" builds for B2G
• mac-v2-signing servers died under load
• Make runner responsible for buildbot startup on Ubuntu test
• Start to drop "mozilla" objdir from Thunderbird builds
• Make some mozharness jobs easier to run outside of automation + allow for http authentication when running outside of the releng network
• Support multiple virtualization and root device type AMIs in aws_watch_pending.py
• web-platform-tests shouldn't run on Aurora
• port buildapi to relengapi
• Don't clobber the source checkout
• [Flame] Create Kitkat based pvtbuilds for 2.0 and 2.1
• Add "Android instrumentation" tests runs to buildbot config
• Schedule web-platform-tests on linux64 opt on trunk branches
• b2g debug win32 desktop builds not triggered on try
• Port Runner cross-platform
• Stop rebooting after every job
• Nightly Aurora Fennec l10n repacks not available
• pgo talos builders don't get "env" set
• Write an S3 upload Python library that encapsulates most of blobber's functionality
• FlatFish: Integrate boot.img and recovery.img into the build system
• Add dependency management to Runner
• Split webgl tests into a separate mochitest-gl suite on desktop
• Run unittests on Win64 builds
• Skip every nth job for debug tests on mozilla-inbound
• stop jobs from hg'n mozharness/tools repos
• Loan Requests
• Slave loan request for a w64-ix machine
• loan glandium and mshal a Linux ec2 instance
• loan kmoir a dev-linux64-ec2 slave
• Slave loan request for a tst-linux64-ec2 vm
• Slave loan request for a bld-linux64-ec2 vm
• Loan an ami-6a395a5a instance to Aaron Klotz
• Slave loan request for a bld-linux64-ec2 vm for pmoore
• Loaner slave matching WinXP mochitest1 slaves for :walter
• Loan t-w732-ix-003, t-w732-ix-004 to Q
• loan kmoir an aws test slave for testing in bug 1057025
• loan linux32 hardware talos slave to jmaher
• talos-linux-ix-118 loan request for jmaher
• Slave loan request for tst-linux64-spot
• loan kmoir a tst 32 aws slave for bug 1051886
• Loan :sfink a linux64 b-linux64-hp-0024
• Slave loan request for a tst-linux64-ec2 vm for :jgilbert
• loan windows 7 hardware talos slave to jmaher and rvitillo@mozilla.com
• Slave loan request for a VS2013 build machine
• Slave loan request for a w64-ix machine
• R5 request for testing OSX 10.10 yosemite base image in deploystudio
• Slave loan request for a bld-lion-r5 machine
• Mozharness
• Make Proxxy a standalone class not a Mixin
• Increase free space requirement for b2g hazard builds
• Add crash log formatting to StructuredFormatter
• Enable blobber to do the upload even when the harness is killed because of a test timeout
• mozharness DesktopUnittestOutputParser.evaluate_parser() should record a failure if the return code was non-zero
• Add command line options to web-platform-tests for crash reporting.
• Move Firefox Desktop repacks to use mozharness
• verify script called by mozharness for android devices doesn't reboot via mozpool
• Use tooltool.py to download cached artefacts for android builds
• Make mozharness use structured logging for marionette tests
• Code coverage test jobs should set GCOV_PREFIX, package .gcda files and upload them to blobber
• Make sure to mark jobs as failures when structured logger parsing fails
• Other
• figure out what to do with the n900s
• Hg clone: "abort: error: nodename nor servname provided, or not known"
• Upgrade ASan Clang to trunk r214699
• build/unix/build-clang.py does not work with LLVM trunk on OSX 10.9
• Make AWS node type available to graphite & build metadata
• Platform Support
• [tracker] Stop testing on tegras
• evaluate mac cloud options
• create HVM ami for Ubuntu64
• Install vs2013 Update 3 on 5 machines for testing
• slave pre-flight tasks (runner)
• Run unittests/talos on OS X 10.9 Mavericks
• Upgrade Mercurial clients to 3.0.2+
• Release Automation
• release automation should update ship it at certain points
• pushing release builds to beta doesn't work in balrog
• comm-esr31 does wasteful things while cloning mozilla-esr31
• Mobile release automation to move candidates to releases/ (push to mirrors)
• Figure out how to offer release build to beta users
• deploy ship it notifier
• Update channels for single locale Beta and Release builds of Firefox for Android 30 (and beyond)
• replace columns in shipit database to track kickoff status
• adjust e-mail for update_shipping builder to be more intelligible
• Add a standalone process that listens to pulse for release related buildbot messages
• Update transparently the product details when pushing to the mirror
• Empty mozilla-releases rsync module
• flip the switch to move beta users over to balrog
• Use hardlinks when pushing to mirrors to speed it up
• allow post release tasks to be triggered from ship it
• new Builder to run pushsnip and hook it and the existing postrelease builder up to their own Schedulers that can be triggered through SendChange for shipit
• Update release runner to be able to trigger pushsnip and postrelease builders through SendChange
• cache MAR + installer downloads in update verify
• Ensure that Fennec builds from mozilla-esr31 have a buildID to allow for armv6/Android 2.2 users to update to mozilla-esr31 apks
• Releases
• tracking bug for build and release of Thunderbird 31.2.0
• tracking bug for build and release of Firefox 31.2.0 ESR
• tracking bug for build and release of Firefox and Fennec 32.0
• tracking bug for build and release of Firefox and Fennec 33.0
• do a staging release of Firefox and Fennec 34.0b1
• Releases: Custom Builds
• Partner repack changes for Yandex for Fx 32 release
• Remove Onet builds for Fx33
• Repos and Hooks
• (HEADlessTry) Reimplement try in HEADless, scalable, cloudy way
• Provide advice on SCM system for Public Suffix List
• find RFO for git.m.o OOM condition in bug 985864
• Add a hook to detect changesets with wrong file metadata
• Disallow "b=N" (instead of "Bug N") in commit message hook
• Please sync mozilla-b2g32_v2_0m to git mirror
• Migrate hg_templates into version-control-tools repo
• Pulling with Mercurial 3.1 takes several minutes
• Estimate web cluster size needed to support try repository load
• Tools
• incorrect try submission summary email
• Funsize requires file level diff caching for speedups in partial generation
• Add an option to trychooser to select --bisect-chunk options
• Make Funsize generated partials identical across platforms and environments
• Add new parameter - releaseBuildOnBetaChannel - to ship it
• tooltool should output informational and progress messages with default options
• Docs for Funsize
• Reconfigs should be automatic, and scheduled via a cron job
• Move mqext into the new https://hg.mozilla.org/hgcustom/version-control-tools repo
• Add b2g hazard build to trychooser
• Need script to compute name of mirrored repository
• permanent location for vcs-sync mapfiles, status json, logs
• Repos are being merged twice during a reconfig AND changes to tools repo are not tracked / updated in wiki, nor used to decide whether to update foopies and masters or not
• Dev services (e.g. hg.mozilla.org and git.mozilla.org) status page(s)
• vcs-sync needs to be able to account for empty source repos
• Fix tools/update-packaging/common.sh for consistent behaviour across platforms

http://jordan-lund.ghost.io/this-week-in-releng-sept-1st-2014/