The Servo Blog: This Week In Servo 74 |
In the last week, we landed 89 PRs in the Servo organization’s repositories.
The unstoppable vvuk has been on a tear this week, landing a huge number of changes across our dependencies to enable support for compiling Servo with the Microsoft Visual C++ toolchain (instead of GNU GCC toolchain on Windows via mingw, which is what we do today). That looks like it will land soon, and it’s been amazing work pulling it together. The Windows development experience is going to be much better after that lands, and we’re really excited to bring the Windows dev experience up to the same quality as the MacOS and Linux ones.
Our overall roadmap is available online and now includes the initial Q3 plans. From now on, we plan to include the quarterly plan with a high-level breakdown in the roadmap page.
This week’s status updates are here.
Request API and Response API testshistory.length.tar.gz files deterministicWindow.postMessageload events for extenal stylesheetsInterested in helping build a web browser? Take a look at our curated list of issues that are good for new contributors!
None this week.
|
|
Karl Dubost: [worklog] Edition 030. Lightstorms, rainbows and bugs… this is summer. |
Tune of the week: Patti LaBelle - Somewhere Over the Rainbow
Progress this week:
Today: 2016-08-08T08:22:45.867497 283 open issues ---------------------- needsinfo 5 needsdiagnosis 62 needscontact 16 contactready 29 sitewait 157 ----------------------
You are welcome to participate
We had a Web compat meeting this week and we welcomed a new team member: Dennis Schubert.
(a selection of some of the bugs worked on this week).
x:-moz-any-link, x:default was applied to the CSS. This was ignored by other browsers. It seems it is a Firefox 3.0 hack which is widely distributed through sites like CSS Tricks. They also do strange things when using select.mod_pagespeed it breaks in Firefox Android. The irony is that it will probably break in the future if finally WebP is supported by Gecko.Otsukare!
|
|
Jen Kagan: all the links that’s fit to save for later |
jared has ~43 links for me to review in response to every 1 question i ask. i ask a lot of questions.
needless to say, we have been filing links away on an imaginary “to read later” list for several weeks now.
i’m starting an actual “to read later” list here, with the hope that i’ll make it back around to some of these:
to be continued…
http://www.jkitppit.com/2016/08/06/all-the-links-thats-fit-to-save-for-later/
|
|
Mozilla WebDev Community: Extravaganza – August 2016 |
Once a month, web developers from across Mozilla get together to talk about the work that we’ve shipped, share the libraries we’re working on, meet new folks, and talk about whatever else is on our minds. It’s the Webdev Extravaganza! The meeting is open to the public; you should stop by!
You can check out the wiki page that we use to organize the meeting, or view a recording of the meeting in Air Mozilla. Or just read on for a summary!
The shipping celebration is for anything we finished and deployed in the past month, whether it be a brand new site, an upgrade to an existing one, or even a release of a library.
First up was mythmon, who mentioned the new release of Normandy, the backend server for the SHIELD system that powers surveys and feature studies in Firefox. This release includes a new admin interface built using React and Redux, as well as a switch to client-side targeting that is powered by JEXL expressions.
Next was peterbe, who mentioned that Socorro, the crash-report service for Firefox, has added Google-based sign-in ahead of the planned shut-down of Persona. It’s planned to land in production sometime within the next week, and involves some extra work around triggering automatic sign-out of users who have been signed in for a certain amount of time.
ErikRose was next, and shared yet another list of new features developed by DXR intern new_one:
Here we talk about libraries we’re maintaining and what, if anything, we need help with for them.
Next was pmac in absentia, who wanted to share django-jinja-markdown, a fork of jingo-markdown. It adds support for rendering Markdown strings to HTML in templates rendered with django-jinja via a markdown filter, as well as a similarly-named template function. It also includes a block-level template tag that can be enabled by adding the library as a Jinja extension.
Back to peterbe, who shared json-schema-reducer. The Python-based library takes in a JSON Schema and a JSON object or dict, and returns the JSON object with only fields that are specified in the schema. The main use case for the library is taking Socorro crash reports, and whitelisting data that is appropriate to be sent to Mozilla’s Telemetry platform for analysis, removing sensitive data that isn’t meant to leave the crash report system.
The Roundtable is the home for discussions that don’t fit anywhere else.
Last up was ErikRose, who brought up the Getting Things Done methodology and how he recently has adopted it to help deal with his personal and professional time management. The video recording contains an extended discussion of time management strategies, but useful tools highlighted during the discussion include Things (OSX only), Org-Mode, and good old-fashioned sticky notes.
If you’re interested in web development at Mozilla, or want to attend next month’s Extravaganza, subscribe to the dev-webdev@lists.mozilla.org mailing list to be notified of the next meeting, and maybe send a message introducing yourself. We’d love to meet you!
See you next month!
https://blog.mozilla.org/webdev/2016/08/05/extravaganza-august-2016/
|
|
Air Mozilla: Foundation Demos August 5 2016 |
Foundation Demos August 5 2016
|
|
QMO: Firefox 49 Beta 3 Testday, August 12th |
Hello Mozillians,
We are happy to announce that Friday, August 12th, we are organizing Firefox 49 Beta 3 Testday. We will be focusing our testing on Windows 10 compatibility, Text to Speech in Reader Mode and Text to Speech on Desktop features. Check out the detailed instructions via this etherpad.
No previous testing experience is required, so feel free to join us on #qa IRC channel where our moderators will offer you guidance and answer your questions.
Join us and help us make Firefox better! See you on Friday!
https://quality.mozilla.org/2016/08/firefox-49-beta-3-testday-august-12th/
|
|
Julien Vehent: TLS stats from 1.6 billion connections to mozilla.org |
One of the most challenging task of editing Server Side TLS is figuring out which ciphersuites are appropriate in which levels. Over the years, we've made judgment calls based on our experience and understanding of the TLS ecosystem, but finding hard data about clients in the wild is always difficult.
For the next revision of the guidelines, I wanted to collect real-world data to decide if we could prune unused ciphersuites from the Intermediate configuration. People often complain that the list of ciphersuites is too long. So far, we've taken a conservative approach, preferring a long list of ciphersuites that accepts all clients rather than being restrictive and potentially breaking the Internet for a minority on unusual devices.
Collecting TLS statistics is actually harder than one might think. Modern infrastructures terminate TLS ahead of application servers and don't always provide ciphersuite information in their logs. We run most of our services in AWS behind ELBs, so there's no way to collect statistics at scale.
Last year, we moved mozilla.org behind Cloudflare to support certificate switching and continue serving our users on very old systems. A few months ago, Cloudflare added TLS protocol and ciphersuite information to their access logs, so we finally had a solid data source to evaluate client support.
Mozilla.org is an excellent target to evaluate client diversity because it receives traffic from all sorts of devices from all over the world. It's not an opinionated site that only certain type of people would visit. It's not region or language specific (the site supports dozens of languages). And it's the main entry point to download Firefox.
I collected logs from Cloudflare intermittently over the course of a few weeks, to get an evenly distributed sample of clients connections. That data is represented in the table below.
| Percentage | Hits | Protocol | Ciphersuite |
|---|---|---|---|
| 80.300% | 1300142157 | TLSv1.2 | ECDHE-RSA-AES128-GCM-SHA256 |
| 9.900% | 160597128 | TLSv1 | ECDHE-RSA-AES128-SHA |
| 2.800% | 45350538 | TLSv1 | DES-CBC3-SHA |
| 2.500% | 42058051 | TLSv1.2 | ECDHE-RSA-CHACHA20-POLY1305 |
| 2.000% | 33972517 | TLSv1.2 | ECDHE-RSA-AES128-SHA256 |
| 0.800% | 13869096 | none | NONE |
| 0.400% | 6709309 | TLSv1.2 | ECDHE-RSA-CHACHA20-POLY1305-D |
| 0.200% | 4311348 | TLSv1 | AES128-SHA |
| 0.200% | 3629674 | SSLv3 | DES-CBC3-SHA |
| 0.100% | 3155150 | TLSv1.1 | ECDHE-RSA-AES128-SHA |
| 0.100% | 1968795 | TLSv1.2 | AES128-GCM-SHA256 |
| 0.000% | 1110501 | SSLv3 | AES128-SHA |
| 0.000% | 860476 | TLSv1.2 | ECDHE-RSA-AES128-SHA |
| 0.000% | 540913 | TLSv1.2 | AES128-SHA256 |
| 0.000% | 139800 | SSLv3 | ECDHE-RSA-AES128-SHA |
| 0.000% | 83537 | TLSv1.2 | AES128-SHA |
| 0.000% | 77433 | TLSv1.1 | AES128-SHA |
| 0.000% | 16728 | TLSv1.2 | DES-CBC3-SHA |
| 0.000% | 5550 | TLSv1.2 | ECDHE-RSA-DES-CBC3-SHA |
| 0.000% | 2836 | TLSv1.2 | AES256-SHA256 |
| 0.000% | 2050 | TLSv1.2 | ECDHE-RSA-AES256-SHA |
| 0.000% | 1421 | TLSv1 | ECDHE-RSA-DES-CBC3-SHA |
| 0.000% | 570 | TLSv1.2 | ECDHE-RSA-AES256-GCM-SHA384 |
| 0.000% | 386 | TLSv1 | ECDHE-RSA-AES256-SHA |
| 0.000% | 141 | TLSv1.2 | AES256-SHA |
| 0.000% | 128 | TLSv1 | AES256-SHA |
| 0.000% | 66 | TLSv1.3 | ECDHE-RSA-AES128-GCM-SHA256 |
| 0.000% | 32 | TLSv1.2 | ECDHE-RSA-AES256-SHA384 |
| 0.000% | 24 | TLSv1.1 | DES-CBC3-SHA |
| 0.000% | 8 | SSLv3 | AES256-SHA |
| 0.000% | 8 | SSLv3 | ECDHE-RSA-AES256-SHA |
| 0.000% | 8 | SSLv3 | ECDHE-RSA-DES-CBC3-SHA |
| 0.000% | 8 | TLSv1.1 | AES256-SHA |
| 0.000% | 8 | TLSv1.1 | ECDHE-RSA-AES256-SHA |
| 0.000% | 8 | TLSv1.1 | ECDHE-RSA-DES-CBC3-SHA |
| 0.000% | 8 | TLSv1.2 | AES256-GCM-SHA384 |
Unsurprisingly, ECDHE-RSA-AES128-GCM-SHA256 accounts for over 80% of the traffic, as this ciphersuite is preferred in both Firefox and Chrome. It's a good news that most of our users benefit from that level of security, but it doesn't help us understand backward compatibility challenges.
More interesting are the following two entries that both negotiate TLSv1 with ECDHE-RSA-AES128-SHA (9.9%) and DES-CBC3-SHA (2.8%). Almost 13% of the traffic to mozilla.org is stuck in TLSv1 land. The presence of DES-CBC3-SHA in third position is a strong sign that we're nowhere done supporting old clients that don't even know what AES is.
The stat I was most curious about is in 9th position: SSLv3 with DES-CBC3-SHA, which accounts for 0.2% of the traffic, is a signature from Windows XP pre-sp3 clients, when SChannel didn't support TLSv1 or AES. 0.2% may seem insignificant, unless you're one of these users and the only way you will browse the internet is by first downloading Firefox from mozilla.org. We certainly don't recommend for anyone to enable SSLv3 on their site, unless you're in this very special category that needs backward compatibility with very old clients. Mozilla.org is one of those sites.
The rest of the stats are mostly what one would expect: a long list of randomly ordered ciphersuites from devices with a variety of preferences. ECDHE-RSA-CHACHA20-POLY1305 is in that list, but only at 2.5%. Cloudflare doesn't support any of the legacy ciphers like CAMELLIA or SEED, so we can't see if any of those are in use (I would expect them not to be, but who knows...). We can also assume that the handful of SSLv3 connections at the bottom are scanners, since I doubt we have many clients stuck in SSLv3 but with ECDHE-RSA-AES256-SHA support.
The information we're missing the most is DHE support. Cloudflare doesn't enable it anymore, and it would be interesting to know how many clients out there will only negotiate DHE. I'm suspecting most will fall back to some non-PFS AES based ciphersuite, but proof would be nice. Ultimately, removing DHE from the Intermediate recommendations is a goal, given how difficult it's been for operators to generate DHE parameters securely.
Statistics on ECDSA usage would also be nice. We currently use RSA certificates for mozilla.org, but we're more and more recommending ECDSA certs instead (P-256 is preferred in the Modern configuration level). An interesting experiment would be to perform cert switching between RSA SHA1, RSA SHA256 and ECDSA SHA256.
This data is good enough to start drafting the next edition of Server Side TLS. We'll look at removing DHE support from the Intermediate configuration, and maybe limit the non-PFS ciphersuites to one or two AES fallbacks. It would appear, however, that DES-CBC3-SHA is here to stay for a little while longer...
|
|
Support.Mozilla.Org: What’s Up with SUMO – 4th August |
Hello, SUMO Nation!
August, the most venerable and majestic of months, is here. How are you doing? This is the release week, and we have also a few pieces of new to release, for your reading pleasure :-)
If you just joined us, don’t hesitate – come over and say “hi” in the forums!
We salute you!
This being the release week, you may want to read about the new improvements in more detail. Good times ahead!
Keep rocking the helpful web, SUMO Nation… and make the most of the final weeks of summer! Party on!
https://blog.mozilla.org/sumo/2016/08/04/whats-up-with-sumo-4th-august/
|
|
Air Mozilla: MWoS 2015: Let's Encrypt Automation Tooling |
Mozilla Winter of Security presentation of Klaus Krapfenbauer, a graduate student at the security institute SBA Research of the Vienna University of Technology, who worked...
https://air.mozilla.org/mwos-2015-lets-encrypt-automation-tooling/
|
|
The Mozilla Blog: Mozilla Awards $585,000 to Nine Open Source Projects in Q2 2016 |
“People use Tails to chat off-the-record, browse the web anonymously, and share sensitive documents. Many human rights defenders depend on Tails to do their daily work, if not simply to stay alive.” – Tails developer team
“We think that the Web will only be truly open when we own the means of locating information in the billions of documents at our disposal. Creating PeARS is a way to put the ownership of the Web back into people’s hands.” – Aurelie Herbelot, PeARS
“Item 4 of Mozilla’s Manifesto states, ‘Individuals’ security and privacy on the Internet are fundamental and must not be treated as optional.’ This is the primary philosophy behind Caddy, the first and only web server to use HTTPS by default.” –Matt Holt, Caddy
Last quarter’s Mozilla Open Source Support (MOSS)-awarded projects are diverse, but they have one thing in common: they believe in innovation for public benefit. Projects like Tails, PeARS and Caddy are paving the way for the next wave of openness, which is why Mozilla has allocated over $3.5 million to the MOSS initiative in support of these and other open source projects. We’re excited to share the program’s progress this quarter, which includes $585,000 in awards, nine new projects supported and two new tracks launched.
One of the new tracks is “Mission Partners”, which supports any open source project which meaningfully advances the Mozilla mission. We had a large number of applications in the initial round, of which we have already funded eight (for a total of $385,000) and are still considering several more. Applications for “Mission Partners” remain open on an ongoing basis.
The second is our “Secure Open Source” track, which works on improving the security of open source software by providing manual source code audits for important and widely-used pieces of free software. By the end of the second quarter we completed three audits – for the PCRE regular expression library, the libjpeg-turbo image decoding library, and the phpMyAdmin database administration software – with more in the pipeline. We hope that Secure Open Source will grow to be supported by multiple parties with an interest in improving the state of the Internet infrastructure – from companies to non-profits to governments. You can submit a suggestion for a project which might benefit from SOS support.
Our initial track, “Foundational Technology”, which supports projects that Mozilla already uses, integrates or deploys in our infrastructure, was launched late last year and remained open during this quarter. We made one additional award – to PyPy, the Python JIT compiler, for $200,000. Applications for a “Foundational Technology” award remain open.
Mozilla is proud to support the open source community of which we are a part and from which so many benefit. We look forward to enabling even more OS maintenance, improvement and innovation through MOSS, so please apply! The committee meets next in early September, so get your applications in by the end of August.
|
|
Andrew Halberstadt: Using One Click Loaner to Debug Failures |
One of the most painful aspects of a developer's work cycle is trying to fix failures that show up on try, but which can't be reproduced locally. When this happens, there were really only two options (neither of them nice):
I'm pleased to announce there is now a third option, which is easy, powerful and 100% self-serve. Rather than trying to explain it in words, here is a ~5 minute demo:
Because I'm too lazy to re-record, I want to clarify that when I said |mach reftest| was "identical" to the command line that mozharness would have run, I meant other than the test selection arguments. You'll still need to either pass in a test path or use chunking arguments to select which tests to run.
Before getting too excited, here is the requisite list of caveats.
Finally I want to thank the taskcluster team (especially jonasfj and garndt) for implementing the One-Click Loaner, it is seriously cool. I also want to thank armenzg for helping with reviews and dustin for helping me navigate taskcluster and docker.
As always, please let me know if you have any problems, questions or suggestions!
|
|
Air Mozilla: Reps weekly, 04 Aug 2016 |
This is a weekly call with some of the Reps to discuss all matters about/affecting Reps and invite Reps to share their work with everyone.
|
|
Air Mozilla: Web QA Team Meeting, 04 Aug 2016 |
They say a Mozilla Web QA team member is the most fearless creature in the world. They say their jaws are powerful enough to crush...
|
|
Michael Verdi: Firefox Refresh on Mr. Robot |
OMG! A Firefox feature I worked on, showed up on Mr. Robot! pic.twitter.com/KKFOgOWnEu
— Michael Verdi (@michaelverdi) August 4, 2016
In last night’s episode of Mr. Robot, a character opened Firefox and to my surprise, I saw the blue “Refresh Firefox” button. That’s a feature I (and many others) worked on in the Fall of 2014. How cool is that!?
Now I’m wondering how that happened. This button is only shown to Firefox users who go to our download page with the latest version of Firefox. So someone working on the show could have just wanted to make sure they had the latest Firefox, went to the download page where we confirmed that they did (also a feature of the page) and then left it there. Leaving it there seems like an odd choice to me but it’s so immaterial to the story that I’m guessing it wasn’t even a thing that someone thought about.
Interestingly (probably only to me) they also could have then clicked that Refresh button because after a refresh we offer to restore your session which would then re-open that page. And that reminds me that I wished we could do something better in this situation. There’s no reason you would need to refresh twice in a row. It would be nice to ask people if this fixed their problem and offer further help. If a refresh doesn’t fix the issue, you most likely have a malware problem. But the issue with doing that is it probably means adding a good deal of complexity to a page used by millions that only helps a very tiny amount of people. Maybe the problem could be solved another way, though you always run into the how-hard-is-it-to-solve vs. how-many-people-does-it-help vs. what-would-we-not-do-if-we-did-this equation. Something to think about.
Note: More info on what the refresh feature is and how it works.
|
|
Cameron Kaiser: 45.3 beta 1 available |
Like pretty much every major ESR jump from before, some sites will be faster and some sites will be slower. The biggest new change is that we are now building the full Intl API with ICU support (issue 304). We were able to get away without it in 31 and 38 due to various hacked-in stubs but now too much in the browser (let alone add-ons) requires it. To avoid issue 266 we build a shared ICU and shared data file which will degrade startup time while they get searched and loaded along with all the others. There are also new UI features in the browser such as search suggestions (which you can disable) that add overhead, and certain layout and rendering operations are more heavyweight than they were in earlier releases. I've tried to load in some later performance updates to counter this last issue, but they don't erase the delta completely. On the plus side, however, as mentioned previously JavaScript JIT performance jumped 20% between 38 and 45, and low-level graphics performance and software pixel pushing is faster which manifests in better animation performance and smoother scrolling.
Remaining issues include Amazon Music failing to advance tracks in playlists (the JIT has been exonerated, so I'm still spinning my wheels here), disabling the last remnants of signed extensions (which I philosophically disagree with) and changing the update notification source. Otherwise it seems to function very well once it's loaded everything and got most of the browser chrome in the JIT. The browser is tested on my usual lab machines, the 16GB monster Quad G5, a 1.5GB 1.33GHz iBook G4, a 1.25GB 1GHz iMac G4 15" and a 2GB 450MHz Sawtooth G4, all running Tiger. TenFourFoxBox appears to work with no changes needed but you may want to keep 38.10 around for right now if you are using foxboxes heavily, just in case (please report issues as you find them).
On the build side, pretty much the process is the same as 38 except that gcc 4.8 is now required. The MacPorts build of gcc is what I use, and I'd like to hear if other builds from our other unofficial package maintainers also work. The hacked strip7 unfortunately is still needed for stripping release binaries to handle the variant N_SECTs in the object files. One irregularity is that gcc 4.8 can't distinguish Objective-C messages from C++11 lambdas, or rather, that Objective-C++ has priority over lambdas, so that you can't reliably use both language features in the same file. This occurs in exactly one place in the entire code base, in code we don't support anyway, so it's #ifdefed off. If your local changes run afoul of this, try adding #define GCC_SUCKS_MOOSE_WANG (yes, really, I was a bit peeved that day) and see if it builds with that, or put the C++11 lambdas into another file and have your Objective-C++ call it instead. The build instructions are updated and available now, and with luck our esteemed colleague in the Land of the Rising Sun will be able to build Tenfourbird on the 45 base as well. Also, as you will see, the G3 version built just fine and will still be supported in 45 and thereafter.
Localizers should be aware of two new files, TenFourFox.dtd and TenFourFox.properties, which will be the locale basis for features I am considering or planning to release. I'd rather have translations for features I decide against implementing than not having strings for features I do implement, so almost nothing in those two files (which are roughly the same strings in different places) is being used right this second; that said, however, the idea is to roll out features using these strings during 45's lifetime as we move from source parity to feature parity and this will enable me to do so more quickly. If you would like to contribute to the localization effort (we really appreciate it), Chris T wrangles the cats in issue 42. The aim is to have substantially the same languages available for 38 available for 45 at or near the time of release in about six weeks. Your help is invaluable in this effort.
Finally, let's give 38 a rousing round of applause for being a solid, long lived branch release as it rides off into the ESR sunset. Its end also means the end of one of the oddest chapters in Firefox history, Debian Iceweasel, which has now reverted to being "just" Firefox as of 45ESR. Although both we and they rebranded ourselves because of various changes to Firefox we were making, we'll still be keeping our name because we really do change the feature set and make various platform-specific changes Mozilla would probably find unacceptable otherwise. Plus it gives you guys something to still complain about, along with the icon. :)
Don't file Github tickets on beta issues; post them in the comments. Please be kind. This was literally months of work. Thanks and let's start banging the bugs out.
http://tenfourfox.blogspot.com/2016/08/453-beta-1-available.html
|
|
Mozilla Addons Blog: Add-ons Update – Week of 2016/08/03 |
I post these updates every 3 weeks to inform add-on developers about the status of the review queues, add-on compatibility, and other happenings in the add-ons world.
In the past 3 weeks, 1228 listed add-on submissions were reviewed:
There are 98 listed add-ons awaiting review.
You can read about the improvements we’ve made in the review queues here.
If you’re an add-on developer and are looking for contribution opportunities, please consider joining us. Add-on reviewers are critical for our success, and can earn cool gear for their work. Visit our wiki page for more information.
The compatibility blog post for Firefox 49 is up, and the bulk validation was run. The blog post for Firefox 50 should come up in a couple of weeks.
Going back to the recently released Firefox 48, there are a couple of changes that are worth a reminder: (1) release and beta builds no longer have a preference to deactivate signing enforcement, and (2) multiprocess Firefox is now enabled for users without add-ons, and add-ons will be gradually phased in, so make sure you’ve tested your add-on and either use WebExtensions or set the multiprocess compatible flag in your add-on manifest.
As always, we recommend that you test your add-ons on Beta and Firefox Developer Edition to make sure that they continue to work correctly. End users can install the Add-on Compatibility Reporter to identify and report any add-ons that aren’t working anymore.
The wiki page on Extension Signing has information about obtaining the unbranded builds to test on release and beta. We will try to make them easier to get to, but for the most part the Firefox 48 release marks the end of deploying this change, after years of work.
We would like to thank these people for their recent contributions to the add-ons world: Rob Wu, NilkasG, and h4ever.
You can read more about their contributions in our recognition page.
https://blog.mozilla.org/addons/2016/08/03/add-ons-update-85/
|
|
Selena Deckelmann: TaskCluster 2016Q2 Retrospective |
The TaskCluster Platform team worked very hard in Q2 to support the migration off Buildbot, bring new projects into our CI system and look forward with experiments that might enable fully-automated VM deployment on hardware in the future.
We also brought on 5 interns. For a team of 8 engineers and one manager, this was a tremendous team accomplishment. We are also working closely with interns on the Engineering Productivity and Release Engineering teams, resulting in a much higher communication volume than in months past.
We continued our work with RelOps to land Windows builds, and those are available in pushes to Try. This means people can use “one click loaners” for Windows builds as well as Linux (through the Inspect Task link for jobs)! Work on Windows tests is proceeding.
We also created try pushes for Mac OS X tests, and integrated them with the Mac OS X cross-compiled builds. This also meant deep diving into the cross-compiled builds to green them up in Q3 after some compiler changes.
A big part of the work for our team and for RelEng was preparing to implement a new kind of signing process. Aki and Jonas spent a good deal of time on this, as did many other people across PlatformOps. What came out of that work was a detailed specification for TaskCluster changes and for a new service from RelEng. We expect to see prototypes of these ideas by the end of August, and the major blocking changes to the workers and provisioner to be complete then too.
This all leads to being able to ship Linux Nightlies directly from TaskCluster by the end of Q3. We’re optimistic that this is possible, with the knowledge that there are still a few unknowns and a lot has to come together at the right time.
Much of the work on TaskCluster is like building a 747 in-flight. The microservices architecture enables us to ship small changes quickly and without much pre-arranged coordination. As time as gone on, we have consolidated some services (the scheduler is deprecated in favor of the “big graph” scheduling done directly in the queue), separated others (we’ve moved Treeherder-specific services into its own component, and are working to deprecate mozilla-taskcluster in favor of a taskcluster-hg component), and refactored key parts of our systems (intree scheduling last quarter was an important change for usability going forward). This kind of change is starting to slow down as the software and the team adapts and matures.
I can’t wait to see what this team accomplishes in Q3!
Below is the team’s partial list of accomplishments and changes. Please drop by #taskcluster or drop an email to our tools-taskcluster lists.mozilla.org mailing list with questions or comments!
http://www.chesnok.com/daily/2016/08/03/taskcluster-2016q2-retrospective/
|
|
Matthew Noorenberghe: Filling saved HTTP logins over HTTPS for the same domain |
Thanks to initiatives like Let's Encrypt (co-founded by Mozilla), adoption of HTTPS has been increasing making the web more secure. Firefox has adapted its Login Manager to make this transition smooth for users.
Firefox's Login Manager has been using strict origin matching when looking for saved logins for a website. Hence, when a login is saved on the HTTP version of a webpage, it is not available to the HTTPS version. We’ve changed this in Firefox 49, so that logins saved on the HTTP version of a page are also available to the upgraded and more secure HTTPS version.
Without this change, a website that transitioned to HTTPS for its login page would not have the existing saved HTTP logins readily available to Firefox users. The insecure and secure version of the site may appear the same to users and hence they expect the login to still be available. Websites also want their users to be able to continue to easily log in after transitioning to HTTPS, instead of experiencing a surge in account lockout support requests. Some websites have even delayed the switch to HTTPS because of this issue.
In order to continue to help users log in, starting in Firefox 49 saved HTTP logins will be available upon visiting the HTTPS version of a site. The saved HTTP login entry won't be modified when it's used on HTTPS. If a password is changed on the HTTPS version of the site, the saved login will be upgraded and only used on the HTTPS origin in the future. This is to prevent a new password created over HTTPS from leaking to HTTP. HTTPS logins will continue to only to be filled for the same origin (over HTTPS) and won't be downgraded (to HTTP) for security reasons. These changes should help ease the transition to HTTPS.
Cheers to a more secure web! https://matthew.noorenberghe.com/blog/2016/08/filling-saved-http-logins-over-https-same-domain
|
|
Mozilla Addons Blog: Discovery Pane Editorial Policy |
With the launch of Firefox 48, we debuted a new page in the Get Add-ons section of about:addons. For simplicity’s sake, we’re calling this the Discovery Pane.
We recently covered the reasons why we redesigned this page. Now I’d like to share more insight into how this page’s content will refresh moving forward.
The Discovery Pane is designed to appeal to users who may have very little—or zero—understanding of what add-ons are, or why they should customize their browser. That’s why this page only features add-ons that meet the highest standards of user experience, like excellent UI, intuitive work flows, and polished functionality.
We also want to put forth only content that addresses what we know to be the most common needs of a broad set of Firefox users. The page as it exists today includes a few visual themes and four add-ons: an ad blocker, a video downloader, an emoji enhancement tool, and a screenshot extension—all very widely appealing use cases.
The list of content is intentionally short. As with introducing any unfamiliar concept, we wanted to avoid overwhelming people with loads of options, but rather focus their attention on a few enticing paths to choose from.
There are many high-caliber add-ons that would be a great fit in the Discovery Pane. So we’ll update this page each month as we identify more content and appropriate use cases. This does not mean we’ll update all of the content wholesale each month—we may leave certain add-ons in place if they offer a distinctly unique user benefit. However, for example, if we have four screenshot add-ons that are all equally awesome, we’ll endeavor to evenly rotate them.
An updated wiki page goes into greater detail about the selection criteria of Discovery Pane add-ons. But here are three critical criteria you should know about for all Discovery Pane add-ons; they must be…
If you’d like to nominate your add-on (or someone else’s) for Discovery Pane consideration, please use the same channel we always have for featured content—email us at amo-featured@mozilla.org and we’ll add your nomination to the editorial review queue. There’s no need to specifically mention “Discovery Pane” while nominating, since all nominations will be viewed through that prism, but feel free if you like.
Any questions? Concerns? Better ideas? Feedback? You know where to leave comments…
https://blog.mozilla.org/addons/2016/08/03/discovery-pane-editorial-policy/
|
|
Dustin J. Mitchell: What's So Special About "In-Tree?" |
The TaskCluster team uses the phrase “in-tree” all the time: in-tree docker images, in-tree task configuration, in-tree scheduling, .. So, what’s the big deal? Why are we so excited?
TaskCluster’s primary purpose is as the continuous integration system for Firefox. While the TaskCluster platform’s applicability is much larger, Firefox is the organization’s bread and butter, so that’s where we are focused. The Firefox source code is all in a single directory tree of source code, the “gecko tree”. So when we refer to functionality being “in-tree’, we mean that it is implemented in that directory tree.
Historically, lots of functionality has been implemented outside the gecko tree: Mozharness was in a separate repository until about a year ago, and Buildbot functionality spans four non-gecko repositories. PuppetAgain , the tool that controls what is installed and running on all release engineering workers and servers, is in its own repository, too.
This leads to some deep and frustrating issues. In general, it’s very difficult to coordinate changes across multiple repositories. For example, consider upgrading the version of a library on test workers. The upgrade itself needs to be performed in the PuppetAgain repository, and will take about 36 hours to fully “propagate” as machines finish their work and check in with the puppet servers. During this time, some machines have the old version, and some have the new, leading to great confusion if the difference causes job failures. Only after the change has completely propagated can developers land a change in the gecko tree to utilize the new library. But even that is fraught with difficulty: there are many copies (branches) of the gecko source, ranging from try and integration branches which change quickly and break often, through mozilla-central (nightly), aurora, beta, release, and ESR. It is risky, and requires special permission, to land patches on branches later in that list. And if anything goes wrong, backing out all of these changes is difficult and might take another few days, during which time things are broken and people aren’t able to get their work done.
We’ve worked around issues like this in a number of ways – none of them good. In this case, we might install both versions of the library on every machine, including some switch in-tree indicating which library version to use. Once everything is switched over, we can remove the old version from the machines. However, it can take a year for an ESR branch to be removed, by which time we have hundreds of changes we need to remove, with corresponding risk and complexity.
Configuration that isn’t in-tree is also difficult for Firefox developers to modify. For example, neither PuppetAgain nor Buildbot have any good facilities for experimenting with a change before deploying it (they have some facilities, but they aren’t great or easy to use). In practice, given the high risk and specialized knowledge required, most changes to these systems are made by filing a bug for someone else to do the work. That “someone else” might be busy or away on vacation or just not awake during the developer’s working day, in which case the change gets delayed.
In short, out-of-tree changes have caused a lot of friction and slowness in our development process.
One of the design goals of TaskCluster is to make CI infrastructure changes “self-serve”. That means that a Firefox developer who wants to change something about how CI works can make that change herself, without asking permission or requiring someone else’s expertise. Based on previous experience, we have a pretty good idea of what kinds of changes are most popular:
So we are putting docker images, task definitions, and scheduling information in-tree, where anyone can modify them using the well-established process for making changes to the Firefox source code: create a patch, test it in try, submit it for review, land it on an integration branch, and watch it “ride the trains” until it is released.
This turns out to be incredibly powerful! Let’s take the example of upgrading a library from the previous section. The developer looks for some similar bugs, perhaps from the last time this library was upgraded, and learns what changes might be required – probably just changing a version number in a docker image definition. Armed with this information, she writes a patch and pushes it to try – just as she would for a change to the Firefox C++ code. If the try push succeeds, she pushes the patch to Mozreview, and once the review is complete uses Autoland to land the change. From there, the sheriffs merge the patch from repo to repo, riding the trains – so ESR can continue to operate with the old library version until the end of its life, while nightly will use the new version immediately If something goes wrong, sheriffs back the patch out and its effects are undone.
We have a nice overview of the gecko build process in the TaskCluster documentation.
At the moment, only about 20% of the total CI workload runs in TaskCluster. The rest is scheduled by and run in Buildbot, and thus not especially “self-serve”. Soon we will be scheduling all tasks – even Buildbot tasks – in-tree, using the Buildbot Bridge to run those tasks within Buildbot. This will improve the scheduling self-service, providing a nice stepping-stone to actually running all jobs in TaskCluster.
You may have noticed that the top two platforms for Firefox – Windows and OS X – are not mentioned here, and don’t (yet) support Docker. We don’t have a great self-serve solution for configuring build and test environments on these platforms. That said, we are working on solutions and experimenting with new technologies as they become available.
We’re already seeing a blossoming of creative ideas for the CI system that were simply impractical before. As more and more of the CI system is defined in-tree, expect to see – or maybe even implement! – more cool new features from your CI.
http://code.v.igoro.us/posts/2016/08/whats-so-special-about-in-tree.html
|
|