Daniel Pocock: Want to be selected for Google Summer of Code 2016? |
I've mentored a number of students in 2013, 2014 and 2015 for Debian and Ganglia and most of the companies I've worked with have run internships and graduate programs from time to time. GSoC 2015 has just finished and with all the excitement, many students are already asking what they can do to prepare and be selected for Outreachy or GSoC in 2016.
My own observation is that the more time the organization has to get to know the student, the more confident they can be selecting that student. Furthermore, the more time that the student has spent getting to know the free software community, the more easily they can complete GSoC.
Here I present a list of things that students can do to maximize their chance of selection and career opportunities at the same time. These tips are useful for people applying for GSoC itself and related programs such as GNOME's Outreachy or graduate placements in companies.
There is no guarantee that Google will run the program again in 2016 or any future year until the Google announcement.
There is no guarantee that any organization or mentor (including myself) will be involved until the official list of organizations is published by Google.
Do not follow the advice of web sites that invite you to send pizza or anything else of value to prospective mentors.
Following the steps in this page doesn't guarantee selection. That said, people who do follow these steps are much more likely to be considered and interviewed than somebody who hasn't done any of the things in this list.
You may hear terms like free software and open source software used interchangeably.
They don't mean exactly the same thing and many people use the term free software for the wrong things. Not all projects declaring themselves to be "free" or "open source" meet the definition of free software. Those that don't, usually as a result of deficiencies in their licenses, are fundamentally incompatible with the majority of software that does use genuinely free licenses.
Google Summer of Code is about both writing and publishing your code and it is also about community. It is fundamental that you know the basics of licensing and how to choose a free license that empowers the community to collaborate on your code well after GSoC has finished.
Please review the definition of free software early on and come back and review it from time to time. The The GNU Project / Free Software Foundation have excellent resources to help you understand what a free software license is and how it works to maximize community collaboration.
There is no shortcut to GSoC selection and there is no shortcut to GSoC completion.
The student stipend (USD $5,500 in 2014) is not paid to students unless they complete a minimum amount of valid code. This means that even if a student did find some shortcut to selection, it is unlikely they would be paid without completing meaningful work.
If you are the right candidate for GSoC, you will not need a shortcut anyway. Are you the sort of person who can't leave a coding problem until you really feel it is fixed, even if you keep going all night? Have you ever woken up in the night with a dream about writing code still in your head? Do you become irritated by tedious or repetitive tasks and often think of ways to write code to eliminate such tasks? Does your family get cross with you because you take your laptop to Christmas dinner or some other significant occasion and start coding? If some of these statements summarize the way you think or feel you are probably a natural fit for GSoC.
The GSoC stipend will not make you rich. It is intended to make sure you have enough money to survive through the summer and focus on your project. Professional developers make this much money in a week in leading business centers like New York, London and Singapore. When you get to that stage in 3-5 years, you will not even be thinking about exactly how much you made during internships.
GSoC gives you an edge over other internships because it involves publicly promoting your work. Many companies still try to hide the potential of their best recruits for fear they will be poached or that they will be able to demand higher salaries. Everything you complete in GSoC is intended to be published and you get full credit for it. Imagine a young musician getting the opportunity to perform on the main stage at a rock festival. This is how the free software community works. It is a meritocracy and there is nobody to hold you back.
Having a portfolio of free software that you have created or collaborated on and a wide network of professional contacts that you develop before, during and after GSoC will continue to pay you back for years to come. While other graduates are being screened through group interviews and testing days run by employers, people with a track record in a free software project often find they go straight to the final interview round.
Free software is all about community and collaboration. Register your own domain name as this will become a focal point for your work and for people to get to know you as you become part of the community.
This is sound advice for anybody working in IT, not just programmers. It gives the impression that you are confident and have a long term interest in a technology career.
Choosing the provider: as a minimum, you want a provider that offers DNS management, static web site hosting, email forwarding and XMPP services all linked to your domain. You do not need to choose the provider that is linked to your internet connection at home and that is often not the best choice anyway. The XMPP foundation maintains a list of providers known to support XMPP.
Create an email address within your domain name. The most basic domain hosting providers will let you forward the email address to a webmail or university email account of your choice. Configure your webmail to send replies using your personalized email address in the From header.
Update your ~/.gitconfig file to use your personalized email address in your Git commits.
Start writing a blog. Host it using your domain name.
Some people blog every day, other people just blog once every two or three months.
Create links from your web site to your other profiles, such as a Github profile page. This helps reinforce the pages/profiles that are genuinely related to you and avoid confusion with the pages of other developers.
Many mentors are keen to see their students writing a weekly report on a blog during GSoC so starting a blog now gives you a head start. Mentors look at blogs during the selection process to try and gain insight into which topics a student is most suitable for.
Github is one of the most widely used software development web sites. Github makes it quick and easy for you to publish your work and collaborate on the work of other people. Create an account today and get in the habbit of forking other projects, improving them, committing your changes and pushing the work back into your Github account.
Github will quickly build a profile of your commits and this allows mentors to see and understand your interests and your strengths.
In your Github profile, add a link to your web site/blog and make sure the email address you are using for Git commits (in the ~/.gitconfig file) is based on your personal domain.
Pretty Good Privacy (PGP) is the industry standard in protecting your identity online. All serious free software projects use PGP to sign tags in Git, to sign official emails and to sign official release files.
The most common way to start using PGP is with the GnuPG (GNU Privacy Guard) utility. It is installed by the package manager on most Linux systems.
When you create your own PGP key, use the email address involving your domain name. This is the most permanent and stable solution.
Print your key fingerprint using the gpg-key2ps command, it is in the signing-party package on most Linux systems. Keep copies of the fingerprint slips with you.
This is what my own PGP fingerprint slip looks like. You can also print the key fingerprint on a business card for a more professional look.
Using PGP, it is recommend that you sign any important messages you send but you do not have to encrypt the messages you send, especially if some of the people you send messages to (like family and friends) do not yet have the PGP software to decrypt them.
If using the Thunderbird (Icedove) email client from Mozilla, you can easily send signed messages and validate the messages you receive using the Enigmail plugin.
Once you have a PGP key, you will need to find other developers to sign it. For people I mentor personally in GSoC, I'm keen to see that you try and find another Debian Developer in your area to sign your key as early as possible.
Try and find all the free software events in your area in the months between now and the end of the next Google Summer of Code season. Aim to attend at least two of them before GSoC.
Look closely at the schedules and find out about the individual speakers, the companies and the free software projects that are participating. For events that span more than one day, find out about the dinners, pub nights and other social parts of the event.
Try and identify people who will attend the event who have been GSoC mentors or who intend to be. Contact them before the event, if you are keen to work on something in their domain they may be able to make time to discuss it with you in person.
Take your PGP fingerprint slips. Even if you don't participate in a formal key-signing party at the event, you will still find some developers to sign your PGP key individually. You must take a photo ID document (such as your passport) for the other developer to check the name on your fingerprint but you do not give them a copy of the ID document.
Events come in all shapes and sizes. FOSDEM is an example of one of the bigger events in Europe, linux.conf.au is a similarly large event in Australia. There are many, many more local events such as the Debian UK mini-DebConf in Cambridge, November 2015. Many events are either free or free for students but please check carefully if there is a requirement to register before attending.
On your blog, discuss which events you are attending and which sessions interest you. Write a blog during or after the event too, including photos.
Quantcast generously hosted the Ganglia community meeting in San Francisco, October 2013. We had a wild time in their offices with mini-scooters, burgers, beers and the Ganglia book. That's me on the pink mini-scooter and Bernard Li, one of the other Ganglia GSoC 2014 admins is on the right.
GSoC is fundamentally about free software. Linux is to free software what a tree is to the forest. Using Linux every day on your personal computer dramatically increases your ability to interact with the free software community and increases the number of potential GSoC projects that you can participate in.
This is not to say that people using Mac OS or Windows are unwelcome. I have worked with some great developers who were not Linux users. Linux gives you an edge though and the best time to gain that edge is now, while you are a student and well before you apply for GSoC.
If you must run Windows for some applications used in your course, it will run just fine in a virtual machine using Virtual Box, a free software solution for desktop virtualization. Use Linux as the primary operating system.
Here are links to download ISO DVD (and CD) images for some of the main Linux distributions:
If you are nervous about getting started with Linux, install it on a spare PC or in a virtual machine before you install it on your main PC or laptop. Linux is much less demanding on the hardware than Windows so you can easily run it on a machine that is 5-10 years old. Having just 4GB of RAM and 20GB of hard disk is usually more than enough for a basic graphical desktop environment although having better hardware makes it faster.
Your experiences installing and running Linux, especially if it requires some special effort to make it work with some of your hardware, make interesting topics for your blog.
Personally, I have mentored students working with C, C++, Java, Python and JavaScript/HTML5.
In a GSoC program, you will typically do most of your work in just one of these languages.
From the outset, decide which language you will focus on and do everything you can to improve your competence with that language. For example, if you have already used Java in most of your course, plan on using Java in GSoC and make sure you read Effective Java (2nd Edition) by Joshua Bloch.
Find a topic that has long-term appeal for you. Maybe the topic relates to your course or maybe you already know what type of company you would like to work in.
Here is a list of some topics and some of the relevant software projects:
Before the GSoC application process begins, you should aim to learn as much as possible about the theme you prefer and also gain practical experience using the software relating to that theme. For example, if you are attracted to the business and accounting theme, install the PostBooks suite and get to know it. Maybe you know somebody who runs a small business: help them to upgrade to PostBooks and use it to prepare some reports.
Make some small project, less than two week's work, to demonstrate your skills. It is important to make something that somebody will use for a practical purpose, this will help you gain experience communicating with other users through Github.
For an example, see the servlet Juliana Louback created for fixing phone numbers in December 2013. It has since been used as part of the Lumicall web site and Juliana was selected for a GSoC 2014 project with Debian.
There is no better way to demonstrate to a prospective mentor that you are ready for GSoC than by completing and publishing some small project like this yourself. If you don't have any immediate project ideas, many developers will also be able to give you tips on small projects like this that you can attempt, just come and ask us on one of the mailing lists.
Ideally, the project will be something that you would use anyway even if you do not end up participating in GSoC. Such projects are the most motivating and rewarding and usually end up becoming an example of your best work. To continue the example of somebody with a preference for business and accounting software, a small project you might create is a plugin or extension for PostBooks.
Many web sites provide useful information about the developers who contribute to free software projects. Some of these developers may be willing to be a GSoC mentor.
For example, look through some of the following:
Once you have identified projects that are interesting to you and developers who work on those projects, it is important to get yourself on the developer's shortlist.
Basically, the shortlist is a list of all students who the developer believes can complete the project. If I feel that a student is unlikely to complete a project or if I don't have enough information to judge a student's probability of success, that student will not be on my shortlist.
If I don't have any student on my shortlist, then a project will not go ahead at all. If there are multiple students on the shortlist, then I will be looking more closely at each of them to try and work out who is the best match.
One way to get a developer's attention is to look at bug reports they have created. Github makes it easy to see complaints or bug reports they have made about their own projects or other projects they depend on. Another way to do this is to search through their code for strings like FIXME and TODO. Projects with standalone bug trackers like the Debian bug tracker also provide an easy way to search for bug reports that a specific person has created or commented on.
Once you find some relevant bug reports, email the developer. Ask if anybody else is working on those issues. Try and start with an issue that is particularly easy and where the solution is interesting for you. This will help you learn to compile and test the program before you try to fix any more complicated bugs. It may even be something you can work on as part of your academic program.
Contact organizations and ask them which GSoC projects were most successful. In many organizations, you can find the past students' project plans and their final reports published on the web. Read through the plans submitted by the students who were chosen. Then read through the final reports by the same students and see how they compare to the original plans.
Don't wait for the application period to begin. Start writing a project proposal now.
When writing a proposal, it is important to include several things:
If your project plan is good enough, could you put it on Kickstarter or another crowdfunding site? This is a good test of whether or not a project is going to be supported by a GSoC mentor.
Packaging is a vital part of the free software lifecycle. It is very easy to upload a project to Github but it takes more effort to have it become an official package in systems like Debian, Fedora and Ubuntu.
Packaging and the communities around Linux distributions help you reach out to users of your software and get valuable feedback and new contributors. This boosts the impact of your work.
To start with, you may want to help the maintainer of an existing package. Debian packaging teams are existing communities that work in a team and welcome new contributors. The Debian Mentors initiative is another great starting place. In the Fedora world, the place to start may be in one of the Special Interest Groups (SIGs).
After the application deadline, mentors have just 2 or 3 weeks to choose the students. This is actually not a lot of time to be certain if a particular student is capable of completing a project. If the student has a published history of free software activity, the mentor feels a lot more confident about choosing the student.
Some mentors have more than one good student while other mentors receive no applications from capable students. In this situation, it is very common for mentors to send each other details of students who may be suitable. Once again, if a student has a good Github profile and a blog, it is much easier for mentors to try and match that student with another project.
Getting into the world of software engineering is much like joining any other profession or even joining a new hobby or sporting activity. If you run, you probably have various types of shoe and a running watch and you may even spend a couple of nights at the track each week. If you enjoy playing a musical instrument, you probably have a collection of sheet music, accessories for your instrument and you may even aspire to build a recording studio in your garage (or you probably know somebody else who already did that).
The things listed on this page will not just help you walk the walk and talk the talk of a software developer, they will put you on a track to being one of the leaders. If you look over the profiles of other software developers on the Internet, you will find they are doing most of the things on this page already. Even if you are not selected for GSoC at all or decide not to apply, working through the steps on this page will help you clarify your own ideas about your career and help you make new friends in the software engineering community.
http://danielpocock.com/getting-selected-for-google-summer-of-code-2016
|
|
Will Kahn-Greene: Input: Trigger rule project Phase 1 |
Last quarter, I finished up the suggester framework for Input. When a user leaves feedback, registered suggester modules would look at the feedback metadata and text and return suggested links. The suggested links would then show up on the Thank You page. Users could then read a bit about the link and click on it if it was appealing.
The first suggester I wrote does a search against SUMO kb articles to see if any of the kb articles seemed relevant to the feedback. Users frequently leave feedback about problems they're having that could be known issues with known solutions or even problems Firefox solves with features the user wasn't aware of. Because of this, it behooves us greatly to guide these users to the solutions that make their Firefox experience better. I wrote a post about that.
This project covers adding a new suggester that allows analyzers to set up trigger rules for suggestions which is stored in the database. When feedback matches the criteria for a trigger rule, then the suggestion is shown.
I pushed out the last code changes on September 9th, 2015. On September 25th, we created a trigger rule for feedback talking about Norton's addon and suggested a link for a SUMO kb article that talks about the problem. In the 5 days, 22 people saw the suggestion and 6 clicked on the link.
This blog post is a write-up for the Trigger rule project phase 1.
Read more… (10 mins to read)
http://bluesock.org/%7Ewillkg/blog/mozilla/input_trigger_rules_phase1.html
|
|
Support.Mozilla.Org: What’s up with SUMO – 2nd October |
What have you been up to this week, SUMO warriors? We have some news and announcements for you, so start reading before the weekend sweeps us all away from our screens!
https://blog.mozilla.org/sumo/2015/10/02/whats-up-with-sumo-2nd-october/
|
|
Mark Finkle: Fun With Telemetry: URL Suggestions |
Firefox for Android has a UI Telemetry system. Here is an example of one of the ways we use it.
As you type a URL into Firefox for Android, matches from your browsing history are shown. We also display search suggestions from the default search provider. We also recently added support for displaying matches to previously entered search history. If any of these are tapped, with one exception, the term is used to load a search results page via the default search provider. If the term looks like a domain or URL, Firefox skips the search results page and loads the URL directly.
user.engine.#urlhistory.#Since we only recently added the support for search history, we want to look at how it’s being used. Below is a filtered view of the URL suggestion section of our UI Telemetry dashboard. Looks like history.# is starting to get some usage, and following a similar trend to engine.# where the first suggestion returned is used more than the subsequent items.
Also worth pointing out that we do get a non-trivial amount of url situations. This should be expected. Most search keyword data released by Google show that navigational keywords are the most heavily used keywords.
An interesting observation is how often people use the user suggestion. Remember, this is not actually a suggestion. It’s what the person has already typed. Pressing “Enter” or “Go” would result in the same outcome. One theory for the high usage of that suggestion is it provides a clear outcome: Firefox will search for this term. Other ways of trigger the search might be more ambiguous.
http://starkravingfinkle.org/blog/2015/10/fun-with-telemetry-url-suggestions/
|
|
Yunier Jos'e Sosa V'azquez: Pablo P'erez: “Mozilla seguir'a velando por los derechos de los usuarios en Internet” |
Para conocer mucho m'as acerca de las personas que integran Mozilla Hispano, llegan nuevamente las entrevistas los miembros de la comunidad. Este espacio servir'a para reconocer el trabajo de nuestros colaboradores.
En esta oportunidad tenemos como invitado al espa~nol Pablo P'erez.
?C'omo te llamas y a qu'e te dedicas?
Me llamo Pablo P'erez Diez, nac'i en un pueblo del norte de Espa~na. Posteriormente me traslad'e a la ciudad de Valladolid (tambi'en en el norte de Espa~na) para realizar mis estudios de T'ecnico Superior en Telem'atica e Ingeniero T'ecnico en Telem'atica. Tras unos a~nos trabajando como programador all'i, actualmente resido en Madrid.
?Por qu'e te decidiste a colaborar con Mozilla?
Decid'i colaborar para aportar mi grano de arena en la defensa de los derechos de los usuarios online (neutralidad, privacidad, derecho al olvido, etc.), todos ellos cada vez m'as en peligro.
?Actualmente qu'e labor desempe~nas en la Comunidad?
Actualmente realizo las tareas de coordinaci'on en el 'area de asistencia que integra tanto el foro como Twitter.
?Qu'e es lo que m'as valoras o es lo m'as positivo de Mozilla / la Comunidad?
Lo que m'as valoro es como gente de diferentes pa'ises, creencias, ideolog'ias, etc. son capaces de colaborar por un objetivo com'un que en 'ultima instancia es el bien com'un.
?Qu'e te aporta a ti Mozilla / la Comunidad?
Me aporta la experiencia de colaborar con otra mucha gente aprendiendo lo mejor de todos ellos tanto de pa'ises hispanohablantes como de otros muchos m'as.
?Como crees que ser'a Mozilla en el futuro?
En mi opini'on Mozilla seguir'a velando por los derechos de los usuarios en Internet apoy'andose cada vez m'as en la comunidad.
Unas palabras para las personas que desean unirse a la Comunidad.
Si compartes los valores tanto de Mozilla como de Mozilla Hispano este es el lugar perfecto para contribuir a que esos valores sean respetados y obteniendo grandes experiencias con el resto de la comunidad.
Muchas gracias Pablo por acceder a la entrevista.
Fuente: Mozilla Hispano
|
|
Hal Wine: duo MFA & viscosity no-cell setup |
The Duo application is nice if you have a supported mobile device, and it’s usable even when you you have no cell connection via TOTP. However, getting Viscosity to allow both choices took some work for me.
For various reasons, I don’t want to always use the Duo application, so would like for Viscosity to alway prompt for password. (I had already saved a password - a fresh install likely would not have that issue.) That took a bit of work, and some web searches.
Disable any saved passwords for Viscosity. On a Mac, this means opening up “Keychain Access” application, searching for “Viscosity” and deleting any associated entries.
Ask Viscosity to save the “user name” field (optional). I really don’t need this, as my setup uses a certificate to identify me. So it doesn’t matter what I type in the field. But, I like hints, so I told Viscosity to save just the user name field:
defaults write com.viscosityvpn.Viscosity RememberUsername -bool true
With the above, you’ll be prompted every time. You have to put “something” in the user name field, so I chose to put “push or TOTP” to remind me of the valid values. You can put anything there, just do not check the “Remember details in my Keychain” toggle.
http://dtor.com/halfire/2015/10/02/duo_mfa___viscosity_no_cell_setup.html
|
|
Julien Vehent: Introducing SOPS: a manager of encrypted files for secrets distribution |
Automating the distribution of secrets and credentials to components of an infrastructure is a hard problem. We know how to encrypt secrets and share them between humans, but extending that trust to systems is difficult. Particularly when these systems follow devops principles and are created and destroyed without human intervention. The issue boils down to establishing the initial trust of a system that just joined the infrastructure, and providing it access to the secrets it needs to configure itself.
In many infrastructures, even highly dynamic ones, the initial trust is established by a human. An example is seen in Puppet by the way certificates are issued: when a new system attempts to join a Puppetmaster, an administrator must, by default, manually approve the issuance of the certificate the system needs. This is cumbersome, and many puppetmasters are configured to auto-sign new certificates to work around that issue. This is obviously not recommended and far from ideal.
AWS provides a more flexible approach to trusting new systems. It uses a powerful mechanism of roles and identities. In AWS, it is possible to verify that a new system has been granted a specific role at creation, and it is possible to map that role to specific resources. Instead of trusting new systems directly, the administrator trusts the AWS permission model and its automation infrastructure. As long as AWS keys are safe, and the AWS API is secure, we can assume that trust is maintained and systems are who they say they are.
Using the AWS trust model, we can create fine grained access controls to Amazon's Key Management Service (KMS). KMS is a service that encrypts and decrypts data with AES_GCM, using keys that are never visible to users of the service. Each KMS master key has a set of role-based access controls, and individual roles are permitted to encrypt or decrypt using the master key. KMS helps solve the problem of distributing keys, by shifting it into an access control problem that can be solved using AWS's trust model.
Since KMS's inception a few months ago, a number of projects have popped up to use its capabilities to distribute secrets: credstash and sneaker are such examples. Today I'm introducing sops: a secrets editor that uses KMS and PGP to manage encrypted files.
A few weeks ago, Mozilla's Services Operations team started revisiting the issue of distributing secrets to EC2 instances, with a goal to store these secrets encrypted until the very last moment, when they need to be decrypted on target systems. Not unlike many other organizations that operate sufficiently complex automation, we found this to be a hard problem with a number of prerequisites:
Daniel Thornton and I brainstormed a number of ideas, and eventually ended-up with a workflow similar to the one described below.
The idea behind SOPS is to provide a wrapper around a text editor that takes care of the encryption and decryption transparently. When creating a new file, sops generates a data encryption key "Kd" that is itself encrypted with one or more KMS master keys and PGP public keys. Kd is used to encrypt the content of the file with AES256-GCM. In order to decrypt the files, sops must have access to any of the KMS or PGP master keys.
SOPS can be used to encrypt YAML, JSON and TEXT files. In TEXT mode, the content of the file is treated as a blob, the same way PGP would encrypt an entire file. In YAML and JSON modes, however, the content of the file is manipulated as a tree where keys are stored in cleartext, and values are encrypted. hiera-eyaml does something similar, and over the years we learned to appreciate its benefits, namely:
# edit a file
$ sops example.yaml file written to example.yaml
# take a look at the diff
$ git diff example.yaml diff --git a/example.yaml b/example.yaml index 00fe479..5f40330 100644 --- a/example.yaml +++ b/example.yaml @@ -1,5 +1,5 @@ # The secrets below are unreadable without access to one of the sops master key -myapp1: ENC[AES256_GCM,data:Tr7oo=,iv:1vw=,aad:eo=,tag:ka=]
+myapp1: ENC[AES256_GCM,data:krm,iv:0Y=,aad:KPyE=,tag:oIA==]
app2: db: user: ENC[AES256_GCM,data:YNKE,iv:H4JQ=,aad:jk0=,tag:Neg==]
Below are two examples of SOPS encrypted files. The first one in YAML, the second one in JSON:
YAML
cleartext:
# The secrets below are unreadable without access to one of the sops master key
myapp1: t00m4nys3cr3tzupdated
app2:
db:
user: eve
password: c4r1b0u
# private key for secret operations in app2
key: |
-----BEGIN RSA PRIVATE KEY-----
MIIBPAIBAAJBAPTMNIyHuZtpLYc7VsHQtwOkWYobkUblmHWRmbXzlAX6K8tMf3Wf
ImcbNkqAKnELzFAPSBeEMhrBN0PyOC9lYlMCAwEAAQJBALXD4sjuBn1E7Y9aGiMz
bJEBuZJ4wbhYxomVoQKfaCu+kH80uLFZKoSz85/ySauWE8LgZcMLIBoiXNhDKfQL
vHECIQD6tCG9NMFWor69kgbX8vK5Y+QL+kRq+9HK6yZ9a+hsLQIhAPn4Ie6HGTjw
fHSTXWZpGSan7NwTkIu4U5q2SlLjcZh/AiEA78NYRRBwGwAYNUqzutGBqyXKUl4u
Erb0xAEyVV7e8J0CIQC8VBY8f8yg+Y7Kxbw4zDYGyb3KkXL10YorpeuZR4LuQQIg
bKGPkMM4w5blyE1tqGN0T7sJwEx+EUOgacRNqM2ljVA=
-----END RSA PRIVATE KEY-----
number: 1234567890
an_array:
- secretuser1
- secretuser2
- somelongvalueAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
- some other value
encrypted:
# The secrets below are unreadable without access to one of the sops master key
myapp1: ENC[AES256_GCM,data:krwEdH2fxWRexFuvZHS816Wz46Lm,iv:0STqWePc0HOPuDn2EizQdNepx9ksx0guHGeKrshlYSY=,aad:Krl8HyPGQmnIWIZh74Ib+y0OdiVEvRDBv3jTdMGSPyE=,tag:oI2THtQeUX4ZLNnbrdel2A==]
app2:
db:
user: ENC[AES256_GCM,data:YNKE,iv:H9CDb4aUHBJeF2MSTKHQuOwlLxQVdx12AhT0+Dob4JQ=,aad:jlF2KvytlQIgyMpOoO/BiQbukiMwrh1j94Oys+YMgk0=,tag:NeDysIHV9CGtMAQq9i4vMg==]
password: ENC[AES256_GCM,data:p673JCgHYw==,iv:EOOeivCp/Fd80xFdMYX0QeZn6orGTK8CeckmipjKqYY=,aad:UAhi/SHK0aCzptnFkFG4dW8Vv1ASg7TDHD6lui9mmKQ=,tag:QE6uuhRx+cGInwSVdmxXzA==]
# private key for secret operations in app2
key: |-
ENC[AES256_GCM,data:Ea3zTFSOlg1PDZmBa1U2dtKl3pO4nTmaFswJx41fPfq3u8O2/Bq1UVfXn2SrO13obfr6xH4zuUceCDTvW2qvphlan5ir609EXt4dE2TEEcjVKhmAHf4LMwlZVAbvTJtlsnvo/aYJH95uctjsSX5h8pBlLaTGBGYwMrZuMyRU6vdcMWyha+piJckUc9sq7fevy1TSqIxf1Usbn/0NEklWm2VSNzQ2Urqtny6EXar+xU7NfYSRJ3mqmcJZ14oIeXPdpk962RwMEFWdYrbE7D59kWU2BgMjDxYJD5KXpWiw2YCrA/wsATxVCbZlwqC+TJFA5WAUZX756mFhV/t2Li3zQyDNUe6KkMXV9qwf/oV1j5sVRVFsKDYIBqhi3qWBVA+SO9RloQMjhru+IsdbQcS4LKq/1DrBENeZuJ0djUAxKLVfJzMGUf89ju3m9IEPovW8mfF0RbfAGRwFHMO9nEXCxrTLERf3owdR3u4j5/rNBpIvvy1z+2dy6sAx/eyNdS+cn5qO9BPAxsXpSwkaI96rlBagwH1Pfxus0x/D00j93OpE+M8MgQ/9LA68FlCFU4OAQlvw8f7MPoxnq+/+gFTS/qqjTR6EoUuX5NH2WY93YCC5TCbe4GOXyP0H05PbIWq55UMVLNcpAyac3gO4kL5O5U8=,iv:Dl61tsemKH0fdmNul/PmEEsRYFAh8GorR8GRupus/EM=,aad:Ft2aSYYukD1x8pMj1WvmodLjJV6waPy5FqdlImWyQKA=,tag:EPg4KpWqni/buCFjFL857A==]
number: ENC[AES256_GCM,data:XMrBalgZ9tvBxQ==,iv:XyEAAaIzVy/2trnJhLrjMInLg8tMI4CAX9+ccnj3T1Y=,aad:JOlAkP159UxDjL1CrumTuQDqgW2+VOIwz7bdfaJIIn4=,tag:WOHOMJS4nhSdj/aQcGbU1A==]
an_array:
- ENC[AES256_GCM,data:td1aAv4s4cOzSo0=,iv:ErVqte7GpQ3JfzVpVRf7pWSQZDHn6W0iAntKWFsMqio=,aad:RiYy8fKX/yVY7KRgXSOIzydT0+TwK7WGzSFSy+1GmVM=,tag:aSGLCmNZsGcBjxEGvNQRwA==]
- ENC[AES256_GCM,data:2K8C418jef8zoAY=,iv:cXE4Hwdl4ZHzAHHyyXqaIMFs0mn65JUehDdaw/aM0WI=,aad:RlAgUZUZ1DvxD9/lZQk9KOHKl4L+fYETaAdpDVekCaA=,tag:CORSBzis6Vy45dEvT/UtMg==]
- ENC[AES256_GCM,data:hbcOBbsaWmlnrpeuwLfh1ttsi8zj/pxMc1LYqhdksT/oQb80g2z0FE4QwUVb7VV+x98LAWHofVyV8Q==,iv:/sXHXde82r2FyG3Z3vC5x8zONB14RwC0GmtkiYEUNLI=,aad:BQb8l5fZzF/aa/EYnrOQvRfGUTq9QmJOAR/zmgOfYDA=,tag:fjNeg3Manjl6B2U2oflRhg==]
- ENC[AES256_GCM,data:LLHkzGobqL53ws6E2zglkA==,iv:g9z3zz4DUzJr4Cim0SVqKF736w2mZoItqbB0TcsGrQU=,aad:Odrvz0loqFdd9wKJz0ULMX/lyEQcX8WaHE59MgeXkcI=,tag:V+rV/AeZ4uEgtwGhlamTag==]
sops:
kms:
- enc: CiC6yCOtzsnFhkfdIslYZ0bAf//gYLYCmIu87B3sy/5yYxKnAQEBAQB4usgjrc7JxYZH3SLJWGdGwH//4GC2ApiLvOwd7Mv+cmMAAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAyGdRODuYMHbA8Ozj8CARCAO7opMolPJUmBXd39Zlp0L2H9fzMKidHm1vvaF6nNFq0ClRY7FlIZmTm4JfnOebPseffiXFn9tG8cq7oi
enc_ts: 1439568549.245995
arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e
pgp:
- fp: 85D77543B3D624B63CEA9E6DBC17301B491B3F21
enc: |
-----BEGIN PGP MESSAGE-----
Version: GnuPG v1
hQIMA0t4uZHfl9qgAQ//ZvUMJOLUJyzKa/Uigwh1jKVhx3feHUitVjCWBfVTPgj1
rRbaTcaF/mYi+rLdW+6kmAg1UEPoVgEBEiBvCTcHjyDzw3m0DoQwvK85nqOpEhkx
rjU1XAnKZ8LNFfIaj8Xo/L6qzE882gwOhfCPU+QmnkWdijs6dQof06DButQDTx5D
KlFvr9CgSa52/uPazZ41disho9guS06k+KrV/P2F4jrU5aB5mfP7YZY9mkVcm2bv
9C5O9neNlXcivgWqKQjB5fmv1Z9yUFAUBNg98wjT8o5Hxz6P6hIbV3f+vn/Vu+VZ
Qo+E7g3/2ItaT89KAIVXgQdHhwJneoDBVpJ4rYz7LLbcvEyAbipKIY4Fl3Cn1ggH
9odIZWA6FWZxHNhRVonMVHZ8Jei5NkUdpJltjDmPJpl3B+7XiWg4NS8dp860fLeL
8nrkR0Z4nVK8DNg+7nQiOxHL9wye6ljWl7/xapJ5r+mYA6eLybsSSlxDo9/OmeON
CYo3jV8HT8amrXYVi4MyZ3LV2TTyGVPObnthYEN2lPSJmms6ei6t/xKaZtAj6779
EzGbUP9VpTKKf5tqGcy9MeGEk2p5ed5hJGinrrt92cNIebcMBJpkLQAy++V/fKnH
Meecaoj1NThBnRguNuz73WSy2C5u/g7OoI50HJJCmoVXY+8D64tWmCZc8Ib2fprS
XgFcoR9u5yPkLZW8xASpRXfKKTbRTTjAXdYEyaYuuOW2nFWo62/d1mZsT7kY21ja
AhVVoxwsj45FCuk63bDVceAJJm+9xxufMp0gNW1GUk858VLyE8gn+uAB5zBcS5c=
=BN/t
-----END PGP MESSAGE-----
created_at: 1443203323.058362
JSON
cleartext
{
"address": {
"city": "New York",
"postalCode": "10021-3100",
"state": "NY",
"streetAddress": "21 2nd Street"
},
"age": 25,
"firstName": "John",
"lastName": "Smith",
"phoneNumbers": [
{
"number": "212 555-1234",
"type": "home"
},
{
"number": "646 555-4567",
"type": "office"
}
]
}
encrypted:
{
"address": {
"city": "ENC[AES256_GCM,data:2wNRKB+Sjjw=,iv:rmATLCPii2WMzcT80Wp9gOpYQqzx6juRmCf9ioz2ZLM=,aad:dj0QZW0BvZVjF1Dn25hOJpcwcVB0qYvEIhGWgxq6YzQ=,tag:wOoPYU+8BA9DiNFlsal3Aw==]",
"postalCode": "ENC[AES256_GCM,data:xwWZ/np9Gxv3CQ==,iv:OLwOr7iliPyWWBtKfUUH7E1wQlxJLA6aFxIfNAEC/M0=,aad:8mw5NU8MpyBlrh7XaUqa642jeyJWGqKvduaQ5bWJ5pc=,tag:VFmnc4Ay+yKzyHcrKeEzZQ==]",
"state": "ENC[AES256_GCM,data:3jY=,iv:Y2bEgkjdn91Pbf5RgJMbyCsyfhV7XWdDhe8wVwTQue0=,aad:DcA5kW1rrET9TxQ4kn9jHSpoMlkcPKs5O5n9wZjZYCQ=,tag:ad1xdNnFwkqx/8EOKVVHIA==]",
"streetAddress": "ENC[AES256_GCM,data:payzP57DGPl5S9Z7uQ==,iv:UIz34fk9zH4z6hYfu0duXmAnI8CqnoRhoaIUqg1YoYA=,aad:hll9Baw40lMjwj7HePQ1o1Lsuh1LCwrE6+bkG4025sg=,tag:FDBhYxMmJ1Wj/uxYxdvVZg==]"
},
"age": "ENC[AES256_GCM,data:4Y4=,iv:hi1iSH19dHSgG/c7yVbNj4yzueHSmmY46yYqeNCoX5M=,aad:nnyubQyaWeLTcz9k9cMHUlgTwVDMyHf32sWCBm7KWAA=,tag:4lcMjstadzI8K40BoDEfDA==]",
"firstName": "ENC[AES256_GCM,data:KVe8Dw==,iv:+eg+Rjvaqa2EEp6ufw9c4hwWwObxRLPmxx3fG6rkyps=,aad:3BdHcorHfbvM2Jcs96zX0JY2VQL5dBNgy7zwhqLNqAU=,tag:5OD6MN9SPhBmXuA81hyxhQ==]",
"lastName": "ENC[AES256_GCM,data:1+koqsI=,iv:b2kBxSW4yOnLFc8qoeylkMtiO/6qr4cZ5VTntXTyXO8=,aad:W7HXukq3lUUMj9i57UehILG2NAp8XCgJMYbvgflWJIY=,tag:HOrgi1L+IRP+X5JGMnm7Ig==]",
"phoneNumbers": [
{
"number": "ENC[AES256_GCM,data:Oo0IxdtBrnfE+bTf,iv:tQ1E/JQ4lHZvj1nQnGL2sKE30sCctjiMCiagS2Yzch8=,aad:P+m5gD3pKfNEOy6t61vbKhEpPtMFI2NZjBPrD/m8T9w=,tag:6iRMUVUEx3UZvUTGTjCdwg==]",
"type": "ENC[AES256_GCM,data:M3zOKQ==,iv:pD9RO4BPUVu6AWPo2DprRsOqouN+0HJn+RXQAXhfB2s=,aad:KFBBVEEnSjdmah3i2XmPx7wWEiFPrxpnfKYW4BSolhk=,tag:liwNnip/L6SZ9srn0N5G4g==]"
},
{
"number": "ENC[AES256_GCM,data:BI2f/qFUea6UHYQ+,iv:jaVLMju6h7s+AlF7CsPbpUFXO2YtYAqYsCIsyHgfrfI=,aad:N+8sVpdTlY5I+DcvnY018Iyh/QesD7bvwfKHRr7q2L0=,tag:hHPPpQKP4cUIXfh9CFe4dA==]",
"type": "ENC[AES256_GCM,data:EfKAdEUP,iv:Td+sGaS8XXRqzY98OK08zmdqsO2EqVGK1/yDTursD8U=,aad:h9zi8s+EBsfR3BQG4r+t+uqeChK4Hw6B9nJCrValXnI=,tag:GxSk1LAQIJNGyUy7AvlanQ==]"
}
],
"sops": {
"kms": [
{
"arn": "arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e",
"created_at": 1443204393.48012,
"enc": "CiC6yCOtzsnFhkfdIslYZ0bAf//gYLYCmIu87B3sy/5yYxKnAQEBAgB4usgjrc7JxYZH3SLJWGdGwH//4GC2ApiLvOwd7Mv+cmMAAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAwBpvXXfdPzEIyEMxICARCAOy57Odt9ngHHyIjVU8wqMA4QszXdBglNkr+duzKQO316CRoV5r7bO8JwFCb7699qreocJd+RhRH5IIE3"
},
{
"arn": "arn:aws:kms:ap-southeast-1:656532927350:key/9006a8aa-0fa6-4c14-930e-a2dfb916de1d",
"created_at": 1443204394.74377,
"enc": "CiBdfsKZbRNf/Li8Tf2SjeSdP76DineB1sbPjV0TV+meTxKnAQEBAgB4XX7CmW0TX/y4vE39ko3knT++g4p3gdbGz41dE1fpnk8AAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAwag3w44N8+0WBVySwCARCAOzpqMpvzIXV416ycCJd7mn9dBvjqzkUDag/zHlKse57uNN7P0S9GeRVJ6TyJsVNM+GlWx8++F9B+RUE3"
}
],
"pgp": [
{
"created_at": 1443204394.748745,
"enc": "-----BEGIN PGP MESSAGE-----\nVersion: GnuPG v1\n\nhQIMA0t4uZHfl9qgAQ//dpZVlRD9WGvz6Pl+PRKvBf661IHLkCeOq5ubzqLIJZu7\nJMNu0KBoO0qX+rgIQtzMU+04QlbIukw01q9ELSDYjBDQPRQJ+6OAeauawxf5mPGa\nZKOaSuoCuPbfOmGj8AENdSCpDaDz+KvOPvo5NNe16kC8BeerFJGewyEwbnkx5dxZ\ngk+LJBOuWRVUEzjsB1pzGfGRzvuzHcrUzWAoA8N936hDFIpoeDYC/8KLc0CWTltA\nYYGaKh5cZxC0R0TgQ5S9GjcU2nZjhcL94XRxZ+9BZDLCDRnjnRfUpPSTHoxr9wmR\nAuLtgyCIolrPl3fqRLJSLUH6FyTo2CO+2mFSx7y9m2OXkKQd1z2tkOlpC9PDTjGT\nVfGvy9nMUsmrgWG35soEmk0nNJaZehiscvZfomBnnHQgqx7DMSMxAnBneFqjsyOQ\nGK7Jacs/tigxe8NZcYhx+usITeQzVLmuqZ2pO5nEGyq0XJhJjxce9YVaeC4QOft0\nlm6qq+m6oABOdKTGh6zuIiWxU1r417IEgV8mkwjlraAvNNPKowQq5j8dohG4HaNK\nOKoOt8aIZWvD3HE9szuH+uDRXBBEAbIvnojQIyrqeIYv1xU8hDTllJPKw/kYD6nx\nMKrw4UAFand5qAgN/6QoIrOPXC2jhA2VegXkt0LXXSoP1ccR4bmlrGRHg0x6Y8zS\nXAE+BVEMYh8l+c86BNhzVOaAKGRor4RKtcZIFCs/Gpa4FxzDp5DfxNn/Ovrhq/Xc\nlmzlWY3ywrRF8JSmni2Asxet31RokiA0TKAQj2Q7SFNlBocR/kvxWs8bUZ+s\n=Z9kg\n-----END PGP MESSAGE-----\n",
"fp": "85D77543B3D624B63CEA9E6DBC17301B491B3F21"
}
]
}
}
As you can see on each key/value pair, only the values are encrypted and keys are kept in clear. It can be argued that this approach leaks sensitive information, but it’s a tradeoff we’re willing to accept in exchange for an increased usability.
OpenPGP gets a lot of bad press for being an outdated crypto protocol, and while true, what really made look for alternatives is the difficulty to manage and distribute keys to systems. With KMS, we manage permissions to an API, not keys, and that's a lot easier to do.
But PGP is not dead yet, and we still rely on it heavily as a backup solution: all our files are encrypted with KMS and with one PGP public key, with its private key stored securely for emergency decryption in the event that we lose all our KMS master keys.
That said, nothing prevents you from using SOPS the same way you would use an encrypted PGP file: by referencing the pubkeys of each individual who has access to the file. It can easily be done by providing sops with a comma-separated list of public keys when creating a new file:
$ sops --pgp "E60892BB9BD89A69F759A1A0A3D652173B763E8F, 84050F1D61AF7C230A12217687DF65059EF093D3, 85D77543B3D624B63CEA9E6DBC17301B491B3F21" mynewfile.yaml
GnuPG can be a little obscure when it comes to managing the keys that have access to a file. While its command line is powerful, it takes a few minutes to find the right commands and figure out how to provide access to a new member of the team.
In SOPS, managing master keys is easy: they are simply stored as entries of the document under sops->{kms,pgp}. By default, that information is hidden during editing, but calling sops with the "-s" flag will display the master keys in the editor. From there, add new keys by creating entry in the document, or remove them by deleting the lines.
Rotating data keys is also trivial: sops provides a rotation flag "-r" that will generate a new data key Kd and re-encrypt all values in the file with it. Coupled with in-place encryption/decryption, it is easy to rotate all the keys on a group of files:
for file in $(find . -type f -name "*.yaml"); do
sops -d -i $file
sops -e -i -r $file
done
Something that should be done every few months for good practice ;)
SOPS has the ability to use KMS in multiple AWS accounts by assuming roles in each account. Being able to assume roles is a nice feature of AWS that allows administrator to establish trust relationships between accounts, typically from the most secure account to the least secure one. In our use-case, we use roles to indicate that a user of the Master AWS account is allowed to make use of KMS master keys in development and staging AWS accounts. Using roles, a single file can be encrypted with KMS keys in multiple accounts, thus increasing reliability and ease of use.
SOPS is available on Github at http://github.com/mozilla/sops and on Pypi at https://pypi.python.org/pypi/sops. We are progressively reaching a stable stage, with a goal to support Python 2.6.6 to 3.4.
|
|
Mozilla Addons Blog: October 2015 Featured Add-ons |
by Geoff Lankow
Customize the new tab page, add more tiles, add the launcher from Firefox start. Set new tile images and titles, see recently closed tabs, and more!
“I was trying to figure out how to get more tiles without zooming the page out. I stumbled on this and it’s perfect.”
by Jeremy Schomery
Alter Firefox’s built-in privacy settings easily with a toolbar panel.
by Price Zombie
Price Zombie is a price tracker and price comparison browser extension. PriceZombie lets you see full price history on millions of products across hundreds of stores such as Amazon, BestBuy, Bloomingdale’s, and many more.
Featured add-ons are selected by a community board made up of add-on developers, users, and fans. Board members change every six months, so there’s always an opportunity to participate. Stayed tuned to this blog for the next call for applications.
If you’d like to nominate an add-on for featuring, please send it to amo-featured@mozilla.org for the board’s consideration. We welcome you to submit your own add-on!
https://blog.mozilla.org/addons/2015/10/01/october-2015-featured-add-ons/
|
|
Jonathan Griffin: Engineering Productivity Update, Oct 1, 2015 |
We’ve said good-bye to Q3, and are moving on to Q4. Planning for Q4 goals and deliverables is well underway; I’ll post a link to the final versions next update.
Last week, a group of 8-10 people from Engineering Productivity gathered in Toronto to discuss approaches to several aspects of developer workflow. You can look at the notes we took; next up is articulating a formal Vision and Roadmap for 2016, which incorporates both this work as well as other planning which is ongoing separately for things like MozReview and Treeherder.
Bugzilla: Support for 2FA has been enhanced.
Treeherder:
Perfherder and Performance Automation:
MozReview and Autoland: mcote posted a blog post detailing some of the rough edges in MozReview, and explaining how the team intends on tackling these. dminor blogged about the state of autoland; in short, we’re getting close to rolling out an initial implementation which will work similarly to the current “checkin-needed” mechanism, except, of course, it will be entirely automated. May you never have to worry about closed trees again!
Mobile Automation: gbrown made some additional improvements to mach commands on Android; bc has been busy with a lot of Autophone fixes and enhancements.
Firefox Automation: maja_zf has enabled MSE playback tests on trunk, running per-commit. They will go live at the next buildbot reconfig.
Developer Workflow: numerous enhancements have been made to |mach try|; see list below in the Details section. run-by-dir has been applied to mochitest-plain on most platforms, and to mochitest-chrome-opt, by kaustabh93, one of team’s contributors. This reduces test bleedthrough, a source of intermittent failures, as well as improves our ability to change job chunking without breaking tests.
Build System: gps has improved test package generation, which results in significantly faster builds – a savings of about 5 minutes per build on OSX and Windows in automation; about 90s on linux.
TaskCluster Migration: linux64 debug builds are now running, so ahal is unblocked on getting linux64 debug tests running in TaskCluster. armenzg has landed mozharness code to support running buildbot jobs via TaskCluster scheduling, via buildbot bridge.
https://jagriffin.wordpress.com/2015/10/01/engineering-productivity-update-oct-01-2015/
|
|
Will Kahn-Greene: Input: Moving to Django 1.8 |
Over the course of 2015, we've been reworking large parts of the Fjord codebase to do the following:
This blog post covers many grueling details including order we did things, design decisions we made and some anecdotes.
Read more… (13 mins to read)
http://bluesock.org/%7Ewillkg/blog/mozilla/input_django_1_8_upgrade.html
|
|
Ben Hearsum: Improvements to updates for Foxfooders |
We've been providing on-device updates (that is to say: no flashing required) to users in the Foxfood program for nearly 6 months now. These updates are intended for users who are officially part of the Foxfooding program, but the way our update system works means that anyone who puts themselves on the right update channel can receive them. This makes things tough for us, because we'd like to be able to provide official Foxfooders with some extra bits and we can't do that while these populations are on the same update channel. Thanks to work that Rob Wood and Alexandre Lissy are doing, we'll soon be able to resolve this and get Foxfooders the bits they need to do the best possible testing.
To make this possible, we've implemented a short term solution that lets us only serve updates to official Foxfooders. When landed, they will send a hashed version of their IMEI as part of their update request. A list of the acceptable IMEI hashes will be maintained in Balrog (the update server), which lets us only serve an update if the incoming one matches one of the whitelisted ones.
To really make this work we need to detangle the current "dogfood" update channel. As I mentioned, it's currently being used in two distinct populations of users: those are part of the official program, and those who aren't. In order to support both populations of users we'll be splitting the "dogfood" update channel into two:
To transition, we will be asking folks who are officially part of the Foxfooding program to flash with a new image that switches them to the "foxfood" update channel. When this is ready to go, it will be announced and communicated appropriately.
Big thanks to everyone who was involved in this effort, particularly Rob Wood, who implemented the new whitelisting feature in Balrog, and Alexandre Lissy and Jean Gong, who went through multiple rounds of back and forth before we settled on this solution.
It's worth noting that this solution isn't ideal: sending IMEIs (even hashed versions) isn't something we prefer to do for both reasons of user privacy and protection of the bits. In the longer term, we'd like to look at a solution that wouldn't require IMEIs to be sent to us. This could come in the form of embedding or asking for credentials, and using those to access the updates. This type of solution would enhance user privacy and make it harder to get around the protections by brute forcing.
http://hearsum.ca/blog/improvements-to-updates-for-foxfooders.html
|
|
Air Mozilla: Web QA Weekly Meeting |
This is our weekly gathering of Mozilla'a Web QA team filled with discussion on our current and future projects, ideas, demos, and fun facts.
|
|
Dan Minor: Autoland to Inbound |
We’re currently putting the finishing touches on Autoland to Inbound from MozReview. We have a relatively humble goal for the initial version of this: we’d like to replace the current manual “checkin-needed” process done by a sheriff. Basically, if a patch is reviewed and a try run has been done, the patch author can flag “checkin-needed” in Bugzilla and at some point in the future, a sheriff will land the changes.
In the near future, if a “ship-it” has been granted and a try run has been made, a button to land to Inbound will appear in the Automation menu in MozReview. At the moment, the main benefit of using Autoland is that it will retry landings in the event that the tree is closed. Enforcing a try run should cut down on tree closures, analysis of closure data in the past has shown a not insignificant number of patches which end up breaking the tree have had no try run performed.
A number of workflow improvements should be easy to implement once this is in place, for instance automatically closing MozReview requests to keep people’s incoming review requests sane and rewriting any r? in the commit summary to be r=.
http://www.lowleveldrone.com/mozilla/mozreview/autoland/2015/10/01/autoland.html
|
|
James Long: Immutable Data Structures and JavaScript |
A little while ago I briefly talked about my latest blog rewrite and promised to go more in-depth on specific things I learned. Today I'm going to talk about immutable data structures in JavaScript, specifically two libraries immutable.js and seamless-immutable. There are other libraries (mori is ClojureScript's persistent data structures and is similar to immutable.js), but we can get a good idea of tradeoffs by just looking at these two. I'll also talk a little about transit-js, which is a great way to serialize anything.
Very little of this applies specifically to Redux. I talk about using immutable data structures generally, but provide pointers for using it specifically in Redux. In Redux, you have a single app state object and update it immutably, and there are various ways to achieve this, each with tradeoffs. I explore this below.
One thing to think about with Redux is how you combine reducers to form the single app state atom; the default method that Redux provides (combineReducers) assumes that you are combining multiple values into a single JavaScript object. If you really want to combine them into a single Immutable.js object, for example, you would need to write your own combineReducers that does so. This might be necessary if you need to serialize your app state and you assume that it's entirely made up of Immutable.js objects.
Most of this applies to using immutable objects in JavaScript in general. It's a bit awkward sometimes because you're fighting the default semantics, and it can feel like you're juggling types. However, depending on your app and how you set things up, you can get a lot out of it.
Currently there is a proposal for adding immutable data structures to JavaScript natively, but it's not clear if it will work out yet. It would certainly remove most problems with using them in JavaScript currently.
Immutable.js comes from Facebook and is one of the most popular implementations of immutable data structures. It's the real deal; it implements fully persistent data structures from scratch using advanced things like tries to implement structural sharing. All updates return new values, but internally structures are shared to drastically reduce memory usage (and GC thrashing). This means that if you append to a vector with 1000 elements, it does not actually create a new vector 1001-elements long. Most likely, internally only a few small objects are allocated.
The advancements of structural sharing data structures, greatly helped with the groundbreaking work by Okasaki, has all but shattered the myth that immutable values are too slow for Real Apps. In fact, it's surprising how many apps can be made faster with them. Apps which read and copy data structures heavily (to avoid being mutated from someone else) will easily benefit from immutable data structures (simply copying a large array once will diminish your performance wins from mutability).
Another example is how ClojureScript discovered that UIs are given a huge performance boost when backed by immutable data structures. If you're mutating a UI, you commonly touch the DOM more than necessary (because you don't know whether the value needs updating or not). React will minimize DOM mutations, but you still need to generate the virtual DOM for it to work with. When components are immutable, you don't even have to generate the virtual DOM; a simple === equality check tells you if it needs to update or not.
Is it Too Good to Be True? You might wonder why we don't use immutable data structures all the time with the benefits they provide. Well, some languages do, like ClojureScript and Elm. It's harder in JavaScript because they are not the default in the language, so we need to weigh the pros and cons.
I already explained why structural sharing makes immutable data structures efficient. Nothing is going to beat mutating an array at an index, but the overhead of immutability isn't large. If you need to avoid mutations, they are going to beat copying objects hands-down.
In Redux, immutability is enforced. You won't see any updates on the screen unless you return a new value. There are big wins because of this, and if you want to avoid copying you might want to look at Immutable.js.
Let's say you internally stored a reference to an object, and called it obj1. Later on, obj2 comes down the pipe. If you never mutate objects, and obj1 === obj2 is true, you know absolutely nothing has changed. In many architectures, like React, this allows you easily do powerful optimizations.
That's called "reference equality," where you can simply just compare pointers. But there's also the concept of "value equality," where you can check if two objects are identical by doing obj1.equals(obj2). When things are immutable, you treat objects as just values.
In ClojureScript everything is a value, and even the default equality operator performs the value equality check (as if === would). If you actually wanted to compare instances you would use identical?. The benefit of value equality with immutable data structures is that it can usually do the checks more performantly than a full recursive scan (if it shares structure it can skip that part).
So where does this come into play? I already explained how it makes optimizing React trivial. Just implement shouldComponentUpdate and check if the state is identical, and skip rendering if so.
I also discovered that while using === with Immutable.js does not perform a value equality check (obviously, you can't override JavaScript's semantics), Immutable.js uses value equality for identities of objects. Anywhere that it wants to check if objects are the same, it uses value equality.
For example, keys of a Map object are value equality checked. This means I can store an object in a Map, and retrieve it later just by supplying an object of the same shape:
let map = Immutable.Map();
map = map.set(Immutable.Map({ x: 1, y: 2}), "value");
map.get(Immutable.Map({ x: 1, y: 2 })); // -> "value"
This has a lot of really nice implications. For example, let's say I have a function that takes a query object that specifies fields to pull from a server:
function runQuery(query) {
// pseudo-code: somehow pass the query to the server and
// get some results
return fetchFromServer(serialize(query));
}
runQuery(Immutable.Map({
select: 'users',
filter: { name: 'James' }
}));
If I wanted to implement query caching, this is all I would have to do:
let queryCache = Immutable.Map();
function runQuery(query) {
let cached = queryCache.get(query);
if(cached) {
return cached;
} else {
let results = fetchFromServer(serialize(query));
queryCache.set(query, results);
return results;
}
}
I can treat the query object as a value, and store the results with it as a key. Later on, if something runs the same query, I'll get back the cached results even if the query object isn't the same instance.
There are all sorts of patterns that value equality simplifies. In fact, I do the exact same technique when querying for posts.
The major downside to Immutable.js data structures is the reason that they are able to implement all the above features: they are not normal JavaScript data structures. An Immutable.js object is completely different from a JavaScript object.
That means you must do map.get("property") instead of map.property, and array.get(0) instead of array[0]. While Immutable.js goes to great lengths to provide JavaScript-compatible APIs, even they are different (push must return a new array instead of mutating the existing instance). You can feel it fighting the default mutation-heavy semantics of JavaScript.
The reason this makes things complicated is that unless you're really hardcore and are starting a project from scratch, you can't use Immutable objects everywhere. You don't really need to anyway for local objects of small functions. Even if you create every single object/array/etc as immutable, you're going to have to work with 3rd party libraries which use normal JavaScript objects/arrays/etc.
The result is that you never know if you are working with a JavaScript object or an Immutable one. This makes reasoning about functions harder. While it's possible to be clear where you are using immutable objects, you still pass them through the system into places where it's not clear.
In fact, sometimes you might be tempted to put a normal JavaScript object inside an Immutable map. Don't do this. Mixing immutable and mutable state in the same object will reap confusion.
I see two solutions to this:
Use a type system like TypeScript or Flow. This removes the mental burden of remembering where immutable data structures are flowing through the system. Many projects are not willing to take this step though, as it requires quite a different coding style.
Hide the details about data structures. If you are using Immutable.js in a specific part of your system, don't make anything outside of it access the data structures directly. A good example is Redux and it's single atom app state. If the app state is an Immutable.js object, don't force React components to use Immutable.js' API directly.
There are two ways to do this. The first is to use something like typed-immutable and actually type your objects. By creating records, you get a thin wrapper around an Immutable.js object that provides a map.property interface by defining getters based on the fields provided by the record type. Everything that just reads from the object can treat it like a normal JavaScript object. You can't mutate it still, but that's something you actually want to enforce.
The second method is to provide a way to query objects and force anything that wants to read to perform a query. This doesn't work in general, but it works really well in the case of Redux because we have a single app state object, and you want to hide the data layout anyway. Forcing all React components to depend on the data layout means you can never change the actual structure of the app state, which you'll probably want to do over time.
Queries don't have to be a sophisticated engine for deep object querying, they can just be simple function. I'm not doing this in my blog yet, but imagine if I had a bunch of functions like getPost(state, id) and getEditorSettings(state). These all take state and return what I am "querying" just by using the function. I no longer care about where it lives within the state. The only problem is that I might still return an immutable object, so I might need to coerce that into a JavaScript object first or use a record type as described above.
To sum it all up: JavaScript interop is a real issue. Never reference JavaScript objects from Immutable ones. Interop issues can be mitigated with record types as provided with typed-immutable, which have other interesting benefits like throwing errors when mutating or reading invalid fields. Finally, if you're using Redux, don't force everything to depend on the app state structure, as you'll want to change it later. Abstract the data implementation away, which solves the problem with immutable interop.
There's another way to enforce immutability. The seamless-immutable project is a much lighter-weight solution that uses normal JavaScript objects. It does not implement new data structures, so there is no structural sharing, which means you will copy objects as you update them (however, you only need a shallow copy). You don't get any of the performance or value equality benefits explained above.
However, in return you get excellent JavaScript interop. All the data structures are quite literally JavaScript data structures. The difference is that that seamless-immutable calls Object.freeze on them, so you cannot mutate them (and strict mode, which is the default in ES6, will throw errors on mutation). Additionally, it adds a few methods to each instance to aid in updating the data, like merge which returns a new object with the supplied properties merged in.
It's missing a few common methods for updating immutable data structures, like Immutable.js' setIn and mergeIn methods which makes it easy to update a deeply nested object. But these are easily implemented and I plan to contribute these to the project.
It's impossible to mix immutable and mutable objects. seamless-immutable will deeply convert all objects to be immutable when wrapping an instance with it, and any added values are automatically wrapped. In practice Immutable.js works very similarly, where Immutable.fromJS deeply converts, as well as various methods like obj.merge. But obj.set does not automatically coerce, so you can store any data type you like. This is not possible with seamless-immutable, so you cannot accidentally store a mutable JavaScript object.
In my opinion, I would expect each library to behave the way they currently do; they have different goals. For example, because seamless-immutable automatically coerces, you cannot store any type that it is not aware of, so it won't play nicely with anything but basic builtin types (in fact, it does not even support Map or Set types right now).
seamless-immutable is a tiny libary with big wins, but also loses out on some fundamental advantages of immutable data structures like value equality. If JavaScript interop is a huge concern for you, it's a fantastic solution. It's especially helpful if you're migrating existing code, as you can slowly make things immutable without rewriting every piece of code that touches them.
There's one last piece to consider: serialization. If you're using custom data types, JSON.stringify is no longer an option. But JSON.stringify was never very good anyway, you can't even serialize ES6 Map or Set instances.
transit-js is a great library written by David Nolen that defines an extensible data transfer format. By default you cannot throw Map or Set instances into it, but the crucial difference is that you can easily transcribe custom types into something that transit understands. In fact, the full code for serializing and deserializing the entire set of Immutable.js types is less than 150 lines long.
Transit is also much smarter about how it encodes types. For example, it knows that map keys might be complex types as well, so it's easy to tell how it how to serialize Map types. Using the transit-immutable-js library (referenced above) to support Immutable.js, now we can do things like this:
let { toJSON, fromJSON } = require('transit-immutable-js');
let map = Immutable.Map();
map = map.set(Immutable.Map({ x: 1, y: 2 }), "value");
let newMap = fromJSON(toJSON(map));
newMap.get(Immutable.Map({ x: 1, y: 2 })); // -> "value"
Value equality combined with transit's easy-breezy map serialization gives us a simple way to use these patterns consistently across any system. In fact, my blog builds the query cache on the server when server-rendering and then sends that cache to the client, so the cache is still fully intact. This use case was actually the main reason I switched to transit.
It would be easy to serialize ES6 Map types as well, but if you have complex keys I'm not sure how you would use the unserialized instance without value equality. There are still probably uses for serializing them though.
If you have mixed normal JavaScript objects and Immutable.js objects, serializing with transit will also keep all those types in tact. While I recommend against mixing them, transit will deserialize each object into the appropriate type, whereas using raw JSON means you'd convert everything to an Immutable.js type when deserializing (assuming you do Immutable.fromJS(JSON.parse(str))).
You can extend transit to serialize anything, like Date instances or any custom types. Check out transit-format for how it encodes types.
If you use seamless-immutable, you are already restricting yourself to only use builtin JavaScript (and therefore JSON-compatible) types, so you can just use JSON.stringify. While simpler, you lose out on the extensibility; it's all about tradeoffs.
Immutability provides a lot of benefits, but whether or not you need to use full-blown persistent data structures provided by Immutable.js depends on the app. I suspect a lot of apps are fine copying objects, as most of them are relatively small.
You win simplicity at the cost of features though; not only is the API a lot more limited you don't get value equality. Additionally, it may be hard later on to switch to Immutable.js if you find out you need the performance gains of structural sharing.
Generally I would recommend hiding the data structure details, especially if you use Immutable.js, to the outside world. Try to conform to JavaScript's default protocols for objects and arrays, i.e. obj.property and arr[0]. It should be possible to quickly wrap Immutable objects with these interfaces, but more research is needed.
This is especially true in Redux, where you will want to change how the app state is structured in the future. You have this problem even if your app state is a normal JavaScript object. Outside users shouldn't break if you move things around in the app state. Provide a way to query the app state structure instead, at least just by abstracting out data accesses with functions. More complex solutions like Relay and Falcor solve this too because a query language is the default way to access data.
|
|
About:Community: Participation Lab Notes: Volunteer vrs Contributor |
As part of the Participation Lab’s efforts we recently began conducting experiments on the Get Involved Page seeking to better understand how people navigate and connect (or fail to connect) to contribution opportunities. In preparation for this experiment we looked at some of the research that the Mozilla.org team had conducted in recent years, and a number of their key learnings led us to a deeper conversation about the language and labels we use to invite contribution to Mozilla. Some of the those learnings were:
This matched some other feedback, and as a result we decided to conduct a series of interviews to discover more ideas and prejudices that exist around the terms volunteer and contributor. Eighteen interviews covering diverse perspectives were conducted, these included core contributors, project leaders, alumni, community project leads, those working in open science, advocacy and randomly selected people in my life who had never contributed to Mozilla. We discovered four interesting insights shared below.
Overall, people working in, or already volunteering with Mozilla were more comfortable with ‘contributor’, but agreed that unless your background was working in a field like software engineering, or science where the term is already part of the language ecosystem, it might be challenging to grasp. I also noticed a trend in feedback that acknowledged once you’re regularly involved in a project you might no longer be objective, and that we may, in fact, be skewing even the most common understanding of these terms. One example given was the use of ‘paid contributor’ and ‘volunteer contributor’, which made no sense for most people outside of Mozilla.
While people seemed to generally understand that volunteering was about lending time and skills, I encountered sensitivities that the word ‘volunteer’ which invoked feelings of being ‘charitable’ vs the more empowered feeling of being a ‘contributor’ . I heard that ‘contribution’ lends to a feeling we’re part ‘part of something ’ while ‘volunteering felt more detached. One core contributor felt very, very strongly, that volunteering was not the term for what they do at Mozilla.
Feedback from non-technical contributors ,and those I spoke with outside the Mozilla community indicated that the term “contribution” was easy to misinterpret as being about donating funds or something of greater significance than some people felt they could offer. When asked, a couple of people cited political campaigns, and fundraisers as the most common association they with the word ‘contribution’.
It was also suggested that at Mozilla we should stop labouring over generalized terms like volunteer and contributor, and instead focus energies on clarifying ways people can help – One person felt that such opportunity exists in more explicit ‘role titles’ i.e. Android Community Ambassador’. The hypothesis is, that by providing role titles we can help people connect to opportunities that are resume-worthy with recognition that contribution is an opportunity. Of course, there are already examples of success with role names demonstrated by the Mozilla Reps program and most recently Club Captains and Regional Leads in Webmaker Clubs.
Also..
We had an interesting suggestion that we make up our own name! Create a Mozilla-fied name for volunteers that makes volunteering at Mozilla a unique version of both terms. An inspiring example was the London Olympics which called volunteers ‘Games Makers’, what a Mozilli-fied version would be remained unclear 
but I’m sure we could come up with something. What do you think?
Additional lure of a Mozilla-fied name is a chance to help people recognize the amazingness of the community they would be joining which MDN reported to be a factor in repeat contribution in their area – and similar to how an Olympic volunteerism resonated with a name describing their impact.
There is the opportunity for continued experimentation and testing using the Get Involved Page, and we would love to hear from you – contributor volunteer, Mozillia-fied name?
What experiment do you think the Participation Lab should design next with these new insights?
Image: “Mozilla Summit Day 2” by Roland Tanglaois licensed under CC BY 2.0
http://blog.mozilla.org/community/2015/09/30/whats-in-a-name-volunteer-vrs-contributor/
|
|
About:Community: Meet an MDN contributor: klez |
Federico klez Culloca made his first few edits to MDN in 2013, but started contributing in earnest in 2014, as a localizer for Mozilla Italia. After a couple of months, he started attending the bi-weekly MDN Community meeting, and later the Learning Area meetings. After that, he concentrated his efforts on the Learning Area of MDN, especially the Glossary.
He says that working on MDN gives him a good idea of how an organization as big as Mozilla actually works, bringing together paid staff and volunteers.
His advice for MDN newcomers?
Don’t be afraid to make mistakes in good faith. Someone is always able and willing to correct them and help you learn from your errors. It’s a wiki, after all
Thanks, klez, for all your contributions to MDN!
http://blog.mozilla.org/community/2015/09/30/meet-an-mdn-contributor-klez/
|
|
About:Community: Participation Lab Notes: The Power of Swag |
It doesn’t take long, once you’ve entered the Mozilla community before you notice that swag is a big part of Mozilla. Stickers, t-shirts, lanyards are everywhere and for many Mozillians these things have become a kind of currency with emotional and physical value.
In 2014, Mozilla spent over $150,000 on swag to engage contributors across four major initiatives: Maker Party, MozFest, Mozilla Reps, and Firefox Student Ambassadors (FSAs).
However we rarely stop to examine what we are learning about the results, benefits and challenges of this investment.
In order to surface and capture these insights the Participation Team interviewed four groups at Mozilla, for whom swag is a core part of their activities, and identified the most interesting insights and challenges faced by each group.
As a result, we discovered that many of the groups face similar challenges but have found distinct solutions and strategies for managing them. The two major insights were that by encouraging local production of swag, and creating swag that is tailored specifically to the needs of the community, costs can be minimized and value to the community increased.
Although in the past year, swag has become a much smaller part of Maker Party as the campaign has become shorter (17 days vs. the previous 2 months) and more contained. In 2014 thousands of people spent the summer throwing events, and swag was an integral part of growing and motivating the Maker Party community.
Much more than a form of recognition for event hosts, in many communities swag is perceived as a vote of confidence from Mozilla that legitimizes both the host and the event. Many communities feel that if we are willing to support an event host and their event through physical things, and it marks them as “officially sanctioned” by Mozilla and this alignment with the brand dramatically increases the influence and reputation of the contributor and the event.
For example, in South America, a Maker Party host created Mozilla branded mouse pads to legitimize their events in the eyes of local internet cafe owners who let them use the space for free in exchange for Mozilla branded mouse pads.
For Maker Party, shipping swag across the world, often to extremely remote areas, was very expensive and problematic. Certain countries charge enormous taxes on clothing and have been known to detain parcels with t-shirts – to the detriment of volunteers who often cannot pay the high customs fees.
While MozFest, as a short-term festival is a bit different from the other examples, it identifies another way in which we use swag to build and support community.
Every year MozFest partners with other like-minded organizations to put on the event. As part of this relationship, partners are offered the opportunity to distribute swag and some promotional material to attendees. As a result Mozilla can produce a small amount of swag like a tote bag and water bottle, and have partners add their swag to create fun gifts for participants that also act as promotional pieces for partners and Mozilla.
Finding swag that is re-usable and has value outside of the event is challenging, but water bottles and tote bags have proven popular and effective, and have the added benefit of reducing the event’s environmental footprint.
Unlike Maker Party or MozFest, Mozilla Reps is a community where individuals participate in multiple ways over a sustained period of time. For this group, it is often the variety rather than quantity of swag that drives excitement.
Within Reps, swag is a great way to acknowledge contributors and support events. However, in some circumstances, swag can come to be seen as a symbol that represents value and status in the community. Therefore as more of a swag item is produced the value of each item diminishes, and collectors culture has developed. While rare swag is a very powerful tool for driving engagement and recognizing achievement in the Reps community, it is important to be aware of the number and variety of an item that is produced, and to carefully manage expectations to prevent swag becoming an end in and of itself.
As swag is a large part of the Reps culture, it is important to be careful about the expectations that are set around the value of swag. Limiting the kinds of official swag that is produced to t-shirts, stickers, and posters and having clear value attached to each, may be one way to keep expectations low and guard against increasing expectations.
The Firefox Student Association has many parallels in it’s structure and its relationship to swag as the Mozilla Reps program. However by carefully controlling the value of swag, and encouraging local production, many of the challenges faced by other groups have been avoided.
The FSA’s have solved problems related to shipping swag, and reduced the “freebee” quality by having FSA’s create their own swag locally and then be reimbursed for the cost. Like the Reps program they also have a collectors culture but set formal expectations on the “value” of different kinds of swag ie. t-shirts are something you have to earn, stickers are freebees you can give away at your event, and posters are something you have to produce yourself.
Because unique designs are a large part of what gives t-shirt swag it’s value, there is a struggle to find and keep track of the many ways t-shirts and designs are being used across Mozilla. In order to coordinate the FSA program suggested creating a central repository of t-shirt designs and what/when they should be distributed so that the use of swag can be better aligned across all of Mozilla.
—
Overall, across Mozilla there is a great deal being learned and experimented with around swag as well as many areas for growth and improvement. Our hope is that by surfacing these lessons and insights, we’ll spark new conversations and gain more insights into the swag processes and how it can be improved. If you have experience or thoughts around swag at Mozilla please share them in the comments here, or on the Participation Team Discourse page.
http://blog.mozilla.org/community/2015/09/30/participation-lab-notes-the-power-of-swag/
|
|
Air Mozilla: Quality Team (QA) Public Meeting |
This is the meeting where all the Mozilla quality teams meet, swap ideas, exchange notes on what is upcoming, and strategize around community building and...
https://air.mozilla.org/quality-team-qa-public-meeting-20150930/
|
|
Mark C^ot'e: Fixing MozReview's sore spots |
MozReview was intentionally released early, with a fairly minimal feature set, and some ugly things bolted onto a packaged code-review tool. The code-review process at Mozilla hasn’t changed much since the project began—Splinter, a slightly fancier UI than dealing with raw diffs, notwithstanding. We knew this would be a controversial subject, with a variety of (invariably strong) opinions. But we also knew that we couldn’t make meaningful progress on a number of long-desired features, like autolanding commits and automatic code analysis, without moving to a modern repository-based review system. We also knew that, with the incredible popularity of GitHub, many developers expect a workflow that involves them pushing up commits for review in a rich web UI, not exporting discrete patches and looking at almost raw diffs.
Rather than spending a couple years off in isolation developing a polished system that might not hit our goals of both appealing to future Mozillians and increasing productivity overall, we released a very early product—the basic set of features required by a push-based, repository-centric code-review system. Ironically, perhaps, this has decreased the productivity of some people, since the new system is somewhat raw and most definitely a big change from the old. It’s our sincere belief that the pain currently experienced by some people, while definitely regrettable and in some cases unexpected, will be balanced, in the long run, by the opportunities to regularly correct our course and reach the goal of a world-class code review-and-submission system that much faster.
And so, as expected, we’ve received quite a bit of feedback. I’ve been noticing a pattern, which is great, because it gives us insight into classes of problems and needs. I’ve identified four categories, which interestingly correspond to levels of usage, from basic to advanced.
Some users find MozReview very opaque. They aren’t sure what many of the buttons and widgets do, and, in general, are confused by the interface. This caught us a little off-guard but, in retrospect, is understandable. Review Board is a big change from Splinter and much more complex. I believe one of the sources of most confusion is the overall review model, with its various states, views, entry points, and exit points. Splinter has the concept of a review in progress, but it is a lot simpler.
We also had to add the concept of a series of related commits to Review Board, which on its own has essentially a patch-based model, similar to Splinter’s, that’s too limited to build on. The relationship between a parent review request and the individual “child” commits is the source of a lot of bewilderment.
Improving the overall user experience of performing a review is a top priority for the next quarter. I’ll explore the combination of the complexity of Review Board and the commit-series model we added in a follow-up post.
For users who are generally satisfied by MozReview, at least, enough to use it without getting too frustrated, there are a number of paper cuts and limitations that can be worked around but generate some annoyance. This is an area we knew we were going to have to improve. We don’t yet have parity with Splinter/Bugzilla attachments, e.g. reviewers can’t delegate review requests, nor can they mark specific files as reviewed. There are other areas that we can go beyond Bugzilla, such as being able to land parts of a commit series (this is technically possible in Bugzilla by having separate patches, but it’s difficult to track). And there are specific things that Review Board has that aren’t as useful for us as they could be, like the dashboard.
This will also be a big part of the work in the next quarter (at least).
The single biggest item here is lack of support for git, particularly a git interface for hg repos like mozilla-central. There are many people interested in using MozReview, but their work flows are based around git using git-cinnabar. gps and kanru did some initial work around this in bug 1153053; fleshing this out into a proper solution isn’t a small task, but it seems clear that we’ll have to finish it regardless before too long, if we want MozReview to be the central code-review tool at Mozilla. We’re still trying to decide how this fits into the above priorities; more users is good, but making existing users happier is as well.
As mentioned at the beginning of this post, the main reason we’re building a new review tool is to make it repository-centric, that is, based around commits, not isolated patches. This makes a lot of long-desired tools and features much more feasible, including autoland, automatic static analysis, commit rewriting to automatically include metadata like reviewers, and a bunch of other things.
This has been a big focus for the last few months. We’ve had autoland-to-try for a little while now, and autoland-to-inbound is nearly complete. We have a generic library for static analysis with which we’ll be able to build various review bots. And, of course, the one big feature we started with, the ability push commits to MozReview instead of exporting standalone patches, which by itself is both more convenient and preserves more information.
After autoland-to-inbound we’ll be putting aside other big features for a little while to concentrate on general user experience so that people enjoy using MozReview, but rest assured we’ll be back here to build more powerful workflows for everyone.
https://mrcote.info/blog/2015/09/30/fixing-mozreviews-sore-spots/
|
|
Air Mozilla: Product Coordination Meeting |
Duration: 10 minutes This is a weekly status meeting, every Wednesday, that helps coordinate the shipping of our products (across 4 release channels) in order...
https://air.mozilla.org/product-coordination-meeting-20150930/
|
|
