I've been marking student submissions in my open source course this weekend, and with only a half-dozen more to do, the procrastinator in me decided a blog post was in order.

Once again I've asked my students to participate in Hacktoberfest.  I wrote about the experience last year, and wanted to give an update on how it went this time.

I layer a few extra requirements on the students, some of them to deal with things I've learned in the past.  For one, I ask them to set some personal goals for the month, and look at each pull request as a chance to progress toward achieving these goals.  The students are quite different from one another, which I want to celebrate, and this lets them go in different directions, and move at different paces.

Here are some examples of the goals I heard this time around:

• Finish all the required PRs
• Increase confidence in myself as a developer
• Master git/GitHub
• Learn new languages and technologies (Rust, Python, React, etc)
• Contribute to projects we use and enjoy on a daily basis (e.g., VSCode)
• Contribute to some bigger projects (e.g., Mozilla)
• Add more experience to our resume
• Read other people's code, and get better at understanding new code
• Work on projects used around the world
• Work on projects used locally
• Learn more about how big projects do testing

So how did it go?  First, the numbers:

• 62 students completed all 4 PRs during the month (95% completion rate)
• 246 Pull Requests were made, consisting of 647 commits to 881 files
• 32K lines of code were added or modified

I'm always interested in the languages they choose.  I let them work on any open source projects, so given this freedom, how will they use it?  The most popular languages by pull request ere:

• JavaScript/TypeScript - 50%
• HTML/CSS - 11%
• C/C++/C# - 11%
• Python - 10%
• Java - 5%

Web technology projects dominate GitHub, and it's interesting to see that this is not entirely out of sync with GitHub's own stats on language positions.  As always, the long-tail provides interesting info as well.  A lot of people worked on bugs in languages they didn't know previously, including:

Swift, PHP, Go, Rust, OCaml, PowerShell, Ruby, Elixir, Kotlin

Because I ask the students to "progress" with the complexity and involvement of their pull requests, I had fewer people working in "Hacktoberfest" style repos (projects that popup for October, and quickly vanish).  Instead, many students found their way into larger and well known repositories and organizations, including:

Polymer, Bitcoin, Angular, Ethereum, VSCode, Microsoft Calculator, React Native for Windows, Microsoft STL, Jest, WordPress, node.js, Nasa, Mozilla, Home Assistant, Google, Instacart

The top GitHub organization by pull request volume was Microsoft.  Students worked on many Microsoft projects, which is interesting, since they didn't coordinate their efforts.  It turns out that Microsoft has a lot of open source these days.

When we were done, I asked the students to reflect on the process a bit, and answer a few questions.  Here's what I heard.

1. What are you proud of?  What did you accomplish during October?

• Contributing to big projects (e.g., Microsoft STL, Nasa, Rust)
• Contributing to small projects, who really needed my help
• Learning a new language (e.g., Python)
• Having PRs merged into projects we respect
• Translation work -- using my personal skills to help a project
• Seeing our work get shipped in a product we use
• Learning new tech (e.g., complex dev environments, creating browser extensions)
• Successfully contributing to a huge code base
• Getting involved in open source communities
• Overcoming the intimidation of getting involved

2. What surprised you about Open Source?  How was it different than you expected?

• People in the community were much nicer than I expected
• I expected more documentation, it was lacking
• The range of projects: big companies, but also individuals and small communities
• People spent time commenting on, reviewing, and helping with our PRs
• People responded faster than we anticipated
• At the same time, we also found that some projects never bothered to respond
• Surprised to learn that everything I use has some amount of open source in it
• Surprised at how many cool projects there are, so many that I don’t know about
• Even on small issues, lead contributors will get involved in helping (e.g., 7 reviews in a node.js fix)
• Surprised at how unhelpful the “Hacktoberfest” label is in general
• “Good First Issue” doesn’t mean it will be easy.  People have different standards for what this means
• Lots of things on GitHub are inactive, be careful you don’t waste your time
• Projects have very different standards from one to the next, in terms of process, how professional they are, etc.
• Surprised to see some of the hacks even really big projects use
• Surprised how willing people were to let us get involved in their projects
• Lots of camaraderie between devs in the community

3. What advice would you give yourself for next time?

• Start small, progress from there
• Manage your time well, it takes way longer than you think
• Learn how to use GitHub’s Advanced Search well
• Make use of your peers, ask for help
• Less time looking for a perfect issue, more time fixing a good-enough issue
• Don’t rely on the Hacktoberfest label alone.
• Don’t be afraid to fail.  Even if a PR doesn’t work, you’ll learn a lot in the process
• Pick issues in projects you are interested in, since it takes so much time
• Don’t be afraid to work on things you don’t (yet) know.  You can learn a lot more than you think.
• Read the contributing docs, and save yourself time and mistakes
• Run and test code locally before you push
• Don’t be too picky with what you work on, just get involved
• Look at previously closed PRs in a project for ideas on how to solve your own.

One thing that was new for me this time around was seeing students get involved in repos and projects that didn't use English as their primary language.  I've had lots of students do localization in projects before.  But this time, I saw quite a few students working in languages other than English in issues and pull requests.  This is something I've been expecting to see for a while, especially with GitHub's Trending page so often featuring projects not in English.  But it was the first time it happened organically with my own students.

Once again, I'm grateful to the Hacktoberfest organizers, and to the hundreds of maintainers we encountered as we made our way across GitHub during October.  When you've been doing open source a long time, and work in git/GitHub everyday, it can be hard to remember what it's like to begin.  Because I continually return to the place where people start, I know first-hand how valuable it is to be given the chance to get involved, for people to acknowledge and accept your work, and for people to see that it's possible to contribute.

https://blog.humphd.org/hacktoberfest-2019/

I found this article a few weeks ago and I really enjoyed the read. The author outlines what a role can look like for very senior ICs. It's the first in a (yet to be written) series about technical leadership and long term IC career paths. I'm excited to read more!

In particular, I am delighted to see her call out strategic work as a way for a senior IC to deliver value. I think there's a lot of opportunity for senior ICs to deliver strategic work, but in my experience organizations tend to under-value this type of work (often unintentionally).

My favorite projects to work on are high impact and difficult to execute even if there not deeply technical. In fact, I've found that my most impactful projects tend to only have a small technical component. Instead, the real value tends to come from spanning a few different technical areas, tackling some cultural change, or taking time to deeply understand the problem before throwing a solution at it. Framing these projects as "strategic" help me put my thumb on the type of work I like doing.

Keavy also calls out strike teams as a valuable way for ICs to work on high impact projects without moving into management. In my last three years at Mozilla, I've been fortunate to be a part of several strike teams and upon reflection I find that these are the projects I'm most proud of.

I'm fortunate that Mozilla has a well documented growth path for senior ICs. All the same, I am learning a lot from her framing. I'm excited to read more!

https://blog.harterrt.com/keavy-tech-leadership-path.html

Poll time! No judgment if you’re in the high end of the range. Keeping a pile of open tabs is the sign of an optimistic, enthusiastic, curious digital citizen, and … Read more

The post Nine tips for better tab management appeared first on The Firefox Frontier.

https://blog.mozilla.org/firefox/nine-tips-for-better-tab-management/

On this coming Thursday, November 7, async-await syntax hits stable Rust, as part of the 1.39.0 release. This work has been a long time in development -- the key ideas for zero-cost futures, for example, were first proposed by Aaron Turon and Alex Crichton in 2016! -- and we are very proud of the end result. We believe that Async I/O is going to be an increasingly important part of Rust's story.

While this first release of "async-await" is a momentous event, it's also only the beginning. The current support for async-await marks a kind of "Minimum Viable Product" (MVP). We expect to be polishing, improving, and extending it for some time.

Already, in the time since async-await hit beta, we've made a lot of great progress, including making some key diagnostic improvements that help to make async-await errors far more approachable. To get involved in that work, check out the Async Foundations Working Group; if nothing else, you can help us by filing bugs about polish issues or by nominating those bugs that are bothering you the most, to help direct our efforts.

Many thanks are due to the people who made async-await a reality. The implementation and design would never have happened without the leadership of cramertj and withoutboats, the implementation and polish work from the compiler side (davidtwco, tmandry, gilescope, csmoe), the core generator support that futures builds on (Zoxc), the foundational work on Future and the Pin APIs (aturon, alexcrichton, RalfJ, pythonesque), and of course the input provided by so many community members on RFC threads and discussions.

Major developments in the async ecosystem

Now that async-await is approaching stabilization, all the major Async I/O runtimes are at work adding and extending their support for the new syntax:

• the tokio runtime recently announced a number of scheduler improvements, and they are planning a stable release in November that supports async-await syntax;
• the async-std runtime has been putting out weekly releases for the past few months, and plans to make their 1.0 release shortly after async-await hits stable;
• using wasm-bindgen-futures, you can even bridge Rust Futures with JavaScript promises;
• the hyper library has migrated to adopt standard Rust futures;
• the newly released 0.3.0 version of the futures-rs library includes support for async-await;
• finally, async-await support is starting to become available in higher-level web frameworks as well, as well as other interesting applications such as the futures_intrusive crate.

Async-await: a quick primer

(This section and the next are reproduced from the "Async-await hits beta!" post.)

So, what is async await? Async-await is a way to write functions that can "pause", return control to the runtime, and then pick up from where they left off. Typically those pauses are to wait for I/O, but there can be any number of uses.

You may be familiar with the async-await from JavaScript or C#. Rust's version of the feature is similar, but with a few key differences.

To use async-await, you start by writing async fn instead of fn:

async fn first_function() -> u32 { .. }

Unlike a regular function, calling an async fn doesn't have any immediate effect. Instead, it returns a Future. This is a suspended computation that is waiting to be executed. To actually execute the future, use the .await operator:

async fn another_function() {
    // Create the future:
    let future = first_function();
    
    // Await the future, which will execute it (and suspend
    // this function if we encounter a need to wait for I/O): 
    let result: u32 = future.await;
    ...
}

This example shows the first difference between Rust and other languages: we write future.await instead of await future. This syntax integrates better with Rust's ? operator for propagating errors (which, after all, are very common in I/O). You can simply write future.await? to await the result of a future and propagate errors. It also has the advantage of making method chaining painless.

Zero-cost futures

The other difference between Rust futures and futures in JS and C# is that they are based on a "poll" model, which makes them zero cost. In other languages, invoking an async function immediately creates a future and schedules it for execution: awaiting the future isn't necessary for it to execute. But this implies some overhead for each future that is created.

In contrast, in Rust, calling an async function does not do any scheduling in and of itself, which means that we can compose a complex nest of futures without incurring a per-future cost. As an end-user, though, the main thing you'll notice is that futures feel "lazy": they don't do anything until you await them.

If you'd like a closer look at how futures work under the hood, take a look at the executor section of the async book, or watch the excellent talk that withoutboats gave at Rust LATAM 2019 on the topic.

Summary

We believe that having async-await on stable Rust is going to be a key enabler for a lot of new and exciting developments in Rust. If you've tried Async I/O in Rust in the past and had problems -- particularly if you tried the combinator-based futures of the past -- you'll find async-await integrates much better with Rust's borrowing system. Moreover, there are now a number of great runtimes and other libraries available in the ecosystem to work with. So get out there and build stuff!

https://blog.rust-lang.org/2019/11/07/Async-await-stable.html

The Rust team is happy to announce a new version of Rust, 1.39.0. Rust is a programming language that is empowering everyone to build reliable and efficient software.

If you have a previous version of Rust installed via rustup, getting Rust 1.39.0 is as easy as:

rustup update stable

If you don't have it already, you can get rustup from the appropriate page on our website, and check out the detailed release notes for 1.39.0 on GitHub.

What's in 1.39.0 stable

The highlights of Rust 1.39.0 include async/.await, shared references to by-move bindings in match guards, and attributes on function parameters. Also, see the detailed release notes for additional information.

The .await is over, async fns are here

Previously in Rust 1.36.0, we announced that the Future trait is here. Back then, we noted that:

With this stabilization, we hope to give important crates, libraries, and the ecosystem time to prepare for async / .await, which we'll tell you more about in the future.

A promise made is a promise kept. So in Rust 1.39.0, we are pleased to announce that async / .await is stabilized! Concretely, this means that you can define async functions and blocks and .await them.

An async function, which you can introduce by writing async fn instead of fn, does nothing other than to return a Future when called. This Future is a suspended computation which you can drive to completion by .awaiting it. Besides async fn, async { ... } and async move { ... } blocks, which act like closures, can be used to define "async literals".

For more on the release of async / .await, read Niko Matsakis's blog post.

References to by-move bindings in match guards

When pattern matching in Rust, a variable, also known as a "binding", can be bound in the following ways:

• by-reference, either immutably or mutably. This can be achieved explicitly e.g. through ref my_var or ref mut my_var respectively. Most of the time though, the binding mode will be inferred automatically.

• by-value -- either by-copy, when the bound variable's type implements Copy, or otherwise by-move.

Previously, Rust would forbid taking shared references to by-move bindings in the if guards of match expressions. This meant that the following code would be rejected:

fn main() {
    let array: Box<[u8; 4]> = Box::new([1, 2, 3, 4]);

    match array {
        nums
//      ---- `nums` is bound by move.
            if nums.iter().sum::() == 10
//                 ^------ `.iter()` implicitly takes a reference to `nums`.
        => {
            drop(nums);
//          ----------- `nums` was bound by move and so we have ownership.
        }
        _ => unreachable!(),
    }
}

With Rust 1.39.0, the snippet above is now accepted by the compiler. We hope that this will give a smoother and more consistent experience with match expressions overall.

Attributes on function parameters

With Rust 1.39.0, attributes are now allowed on parameters of functions, closures, and function pointers. Whereas before, you might have written:

#[cfg(windows)]
fn len(slice: &[u16]) -> usize {
    slice.len()
}
#[cfg(not(windows))] 
fn len(slice: &[u8]) -> usize {
    slice.len()
}

...you can now, more succinctly, write:

fn len(
    #[cfg(windows)] slice: &[u16], // This parameter is used on Windows.
    #[cfg(not(windows))] slice: &[u8], // Elsewhere, this one is used.
) -> usize {
    slice.len()
}

The attributes you can use in this position include:

1. Conditional compilation: cfg and cfg_attr

2. Controlling lints: allow, warn, deny, and forbid

3. Helper attributes used by procedural macro attributes applied to items.

   Our hope is that this will be used to provide more readable and ergonomic macro-based DSLs throughout the ecosystem.

Borrow check migration warnings are hard errors in Rust 2018

In the 1.35.0 release, we announced that NLL had come to Rust 2015 after first being released for Rust 2018 in 1.31.

As noted in the 1.35.0 release, the old borrow checker had some bugs which would allow memory unsafety. These bugs were fixed by the NLL borrow checker. As these fixes broke some stable code, we decided to gradually phase in the errors by checking if the old borrow checker would accept the program and the NLL checker would reject it. If so, the errors would instead become warnings.

With Rust 1.39.0, these warnings are now errors in Rust 2018. In the next release, Rust 1.40.0, this will also apply to Rust 2015, which will finally allow us to remove the old borrow checker, and keep the compiler clean.

If you are affected, or want to hear more, read Niko Matsakis's blog post.

More const fns in the standard library

With Rust 1.39.0, the following functions became const fn:

• Vec::new, String::new, and LinkedList::new
• str::len, [T]::len, and str::as_bytes
• abs, wrapping_abs, and overflowing_abs

Additions to the standard library

In Rust 1.39.0 the following functions were stabilized:

• Pin::into_inner
• Instant::checked_duration_since and Instant::saturating_duration_since

Other changes

There are other changes in the Rust 1.39.0 release: check out what changed in Rust, Cargo, and Clippy.

Please also see the compatibility notes to check if you're affected by those changes.

Contributors to 1.39.0

Many people came together to create Rust 1.39.0. We couldn't have done it without all of you. Thanks!

https://blog.rust-lang.org/2019/11/07/Rust-1.39.0.html

There has been 56 days since curl 7.66.0 was released. Here comes 7.67.0!

This might not be a release with any significant bells or whistles that will make us recall this date in the future when looking back, but it is still another steady step along the way and thanks to the new things introduced, we still bump the minor version number. Enjoy!

As always, download curl from curl.haxx.se.

If you need excellent commercial support for whatever you do with curl. Contact us at wolfSSL.

Numbers

the 186th release
3 changes
56 days (total: 7,901)
125 bug fixes (total: 5,472)
212 commits (total: 24,931)
1 new public libcurl function (total: 81)
0 new curl_easy_setopt() option (total: 269)
1 new curl command line option (total: 226)
68 contributors, 42 new (total: 2,056)
42 authors, 26 new (total: 744)
0 security fixes (total: 92)
0 USD paid in Bug Bounties

The 3 changes

Disable progress meter

Since virtually forever you’ve been able to tell curl to “shut up” with -s. The long version of that is --silent. Silent makes the curl tool disable the progress meter and all other verbose output.

Starting now, you can use --no-progress-meter, which in a more granular way only disables the progress meter and lets the other verbose outputs remain.

CURLMOPT_MAX_CONCURRENT_STREAMS

When doing HTTP/2 using curl and multiple streams over a single connection, you can now also set the number of parallel streams you’d like to use which will be communicated to the server. The idea is that this option should be possible to use for HTTP/3 as well going forward, but due to the early days there it doesn’t yet.

CURLU_NO_AUTHORITY

This is a new flag that the URL parser API supports. It informs the parser that even if it doesn’t recognize the URL scheme it should still allow it to not have an authority part (like host name).

Bug-fixes

Here are some interesting bug-fixes done for this release. Check out the changelog for the full list.

Winbuild build error

The winbuild setup to build with MSVC with nmake shipped in 7.66.0 with a flaw that made it fail. We had added the vssh directory but not adjusted these builds scripts for that. The fix was of course very simple.

We have since added several winbuild builds to the CI to make sure we catch these kinds of mistakes earlier and better in the future.

FTP: optimized CWD handling

At least two landed bug-fixes make curl avoid issuing superfluous CWD commands (FTP lingo for “cd” or change directory) thereby reducing latency.

HTTP/3

Several fixes improved HTTP/3 handling. It builds on Windows better, the ngtcp2 backend now also behaves correctly on macOS, the build instructions are clearer.

Mimics socketpair on Windows

Thanks to the new socketpair look-alike function, libcurl now provides a socket for the application to wait for even when doing name resolves in the dedicated resolver thread. This makes the Windows code work catch up with the similar change that landed in 7.66.0. This makes it easier for applications to behave correctly during the short time gaps when libcurl resolves a host name and nothing else is happening.

curl with lots of URLs

With the introduction of parallel transfers in 7.66.0, we changed how curl allocated handles and setup transfers ahead of time. This made command lines that for example would use [1-1000000] ranges create a million CURL handles and thus use a lot of memory.

It did in fact break a few existing use cases where people did very large ranges with curl. Starting now, curl will just create enough curl handles ahead of time to allow the maximum amount of parallelism requested and users should yet again be able to specify ranges with many million iterations.

curl -d@ was slow

It was discovered that if you ask curl to post data with -d @filename, that operation was unnecessary slow for large files and was sped up significantly.

DoH fixes

Several corrections were made after some initial fuzzing of the DoH code. A benign buffer overflow, a memory leak and more.

HTTP/2 fixes

We relaxed the :authority push promise checks, fixed two cases where libcurl could “forget” a stream after it had delivered all data and dup’ed HTTP/2 handles could issue dummy PRIORITY frames!

connect with ETIMEDOUT now makes CURLE_OPERATION_TIMEDOUT

When libcurl’s connect attempt fails and errno says ETIMEDOUT it means that the underlying TCP connect attempt timed out. This will now be reflected back in the libcurl API as the timed out error code instead of the previously used CURLE_COULDNT_CONNECT.

One of the use cases for this is curl’s --retry option which now considers this situation to be a timeout and thus will consider it fine to retry…

Parsing URL with fragment and question mark

There was a regression in the URL parser that made it mistreat URLs without a query part but with a question mark in the fragment.

https://daniel.haxx.se/blog/2019/11/06/curl-7-67-0/

Over the past year and a half, I have ventured time and again into the federated Mastodon social network. In those ventures, I have contributed bug reports to both the Mastodon client as well as some alternative clients on the web, iOS, and Android.

One of those clients, a single-page, progressive web app, is Pinafore by Nolan Lawson. He had set out to create a fast, light-weight, and accessible, client from the ground up. When I started to use Pinafore, I immediately noticed that a lot of thought and effort had already gone into the client and I could immediately start using it.

I then started contributing some bug reports, and over time, Nolan has improved what was already very good tremendously, by adding more keyboard support, so that even as a screen reader user, one can use Pinafore without using virtual buffers, various light and dark themes, support for reducing animations, and much, much more.

And now, Nolan has shared what he has learned about accessibility in the process. His post is an excellent recollection of some of the challenges when dealing with an SPA, cross-platform, taking into account screen readers, keyboard users, styling stuff etc., and how to overcome those obstacles. It is an excellent read which contains suggestions and food for thought for many web developers. Enjoy the read!

https://marcozehe.wordpress.com/2019/11/05/nolan-lawson-shares-what-he-has-learned-about-accessibility/

In Tracking Diaries, we invited people from all walks of life to share how they spent a day online while using Firefox’s privacy protections to keep count of the trackers … Read more

The post Tracking Diaries with Matt Navarra appeared first on The Firefox Frontier.

https://blog.mozilla.org/firefox/tracking-diaries-matt-navarra/

In April we announced our intent to reduce the amount of annoying permission prompts for receiving desktop notifications that our users are seeing on a daily basis. To that effect, we ran a series of studies and experiments around restricting these prompts.

Based on these studies, we will require user interaction on all notification permission prompts, starting in Firefox 72. That is, before a site can ask for notification permission, the user will have to perform a tap, click, or press a key.

In this blog post I will give a detailed overview of the study results and further outline our plans for the future.

Experiments

As previously described, we designed a measurement using Firefox Telemetry that allows us to report details around when and how a user interacts with a permission prompt without revealing personal information. The full probe definition can be seen in our source code. It was enabled for a randomly chosen pool of study participants (0.1% of the user population) on Firefox Release, as well as for all users of Firefox Nightly. The Release study additionally differentiated between new users and existing users, to account for an inherent bias of existing users towards denying permission requests (because they usually already “have” the right permissions on sites relevant to them).

We further enabled requiring user interaction for notification permission prompts in Nightly and Beta.

Results

Most of the heavy lifting here was done by Felix Lawrence, who performed a thorough analysis of the data we collected. You can read his full report for our Firefox Release study. I will highlight some of the key takeaways:

Notification prompts are very unpopular. On Release, about 99% of notification prompts go unaccepted, with 48% being actively denied by the user. This is even worse than what we’ve seen on Nightly, and it paints a dire picture of the user experience on the web. To add from related telemetry data, during a single month of the Firefox 63 Release, a total of 1.45 Billion prompts were shown to users, of which only 23.66 Million were accepted. I.e, for each prompt that is accepted, sixty are denied or ignored. In about 500 Million cases during that month, users actually spent the time to click on “Not Now”.

Users are unlikely to accept a prompt when it is shown more than once for the same site. We had previously given websites the ability to ask users for notification every time they visit a site in a new tab. The underlying assumption that users would want to take several visits to make up their minds turns out to be wrong. As Felix notes, around 85% of prompts were accepted without the user ever having previously clicked “Not Now”.

Most notification prompts don’t follow user interaction. Especially on Release, the overall number of prompts that are already compatible with this intervention is very low.

Prompts that are shown as a result of user interaction have significantly better interaction metrics. This is an important takeaway. Along with the significant decrease in overall volume, we can see a significantly better rate of first-time allow decisions (52%) after enforcing user interaction on Nightly. The same can be observed for prompts with user interaction in our Release study, where existing users will accept 24% of first-time prompts with user interaction and new users would accept a whopping 56% of first-time prompts with user interaction.

Changes

Based on the outlined results we have decided to enact two new restrictions:

• Starting from Firefox 70, replace the default “Not Now” option with “Never”, which will effectively hide the prompt on a page forever.
• Starting from Firefox 72, require user interaction to show notification permission prompts. Prompts without interaction will only show a small icon in the url bar.

When a permission prompt is denied by Firefox, the user still has the ability to override this automatic decision by clicking the small permission icon that appears in the address bar. This lets users use the feature on websites with that prompt without waiting for user interaction.

 

Besides the clear improvements in user interaction rates that our study has shown, these restrictions were derived from a few other considerations:

• Easy to upgrade. Requiring user interaction allows for an easy upgrade path for affected websites, while hiding annoying “on load” prompts.
• Transparent. Unlike other heuristics (such as “did the user visit this site a lot in the past”), interaction is easy to understand for both developers and users.
• Encourages pre-prompting. We want to websites to use in-content controls to enable notifications, as long as they have an informative style and do not try to mimic native browser UI. Faking (“spoofing”) browser UI is considered a security risk and will be met with stronger enforcement in the future. A good pre-prompt follows the style of the page and adds additional context to the request. Pre-prompting, when done well, will increase the chance of users opting to receive notifications. Annoying users, as our data shows, will lead to churn.

We will release additional information and resources for web developers on our Mozilla Hacks blog.

We hope that these restrictions will lead to an overall better and less annoying user experience for all our users while retaining the functionality for those that need notifications.

The post Restricting Notification Permission Prompts in Firefox appeared first on Future Releases.

https://blog.mozilla.org/futurereleases/2019/11/04/restricting-notification-permission-prompts-in-firefox/

Please join us in congratulating Shina Dhingra, Rep of the Month for October 2019!

Shina is from Pune, Maharashtra, India. Her journey started with the Mozilla Pune community while she was in college in 2017, with Localization in Hindi and quality assurance bugs.

She’s been an active contributor to the community and since then has helped a lot of newcomers in their onboarding and helping them understand better what the Mozilla Community is all about.

She joined the Reps Program in February 2019 and since then she has actively participated and contributed to Common Voice, A-Frame, Localization, Add-ons, and other Open Source Contributions. She built her own project as a mentee under the Open Leaders Program, and will be organizing and hosting her own cohort called “Healthier AI” which she launched at MozFest this year.

Congratulations and keep rocking the open web!

To congratulate her, please head over to Discourse!

https://blog.mozilla.org/mozillareps/2019/11/04/rep-of-the-month-october-2019/

In our last post, we highlighted some of the advantages that Bazel would bring.  The remote execution and caching benefits Bazel bring look really attractive, but it’s difficult to tell exactly how much they would benefit Firefox.  I looked for projects that had switched to Bazel, and a brief summary of each project’s experience is written below.

The Bazel rules for nodejs highlight Dataform’s switch to Bazel, which took about 2 months.  Their build involves some combination of “NPM packages, Webpack builds, Node services, and Java pipelines”. Switching plus enabling remote caching reduced the average time for a build in CI from 30 minutes to 5 minutes; incremental builds for local development have been “reduced to seconds from minutes”.  It’s not clear whether the local development experience is also hooked up to the caching infrastructure as well.

Pinterest recently wrote about their switch to Bazel for iOS.  While they call out remote caching leading to “build times [dropping] under a minute and as low as 30 seconds”, they state their “time to land code” only decreased by 27%.  I wasn’t sure how to reconcile such fast builds with (relatively) modest decreases in CI time.  Tests have gotten a lot faster, given that test results can be cached and reused if the tests in question have their transitive dependencies unchanged.

One of the most complete (relatively speaking) descriptions I found was Redfin’s switch from Maven to Bazel, for building a large amount of JavaScript modules and Java code, nearly 30,000 files in all.  Their CI builds went from 40-90 minutes to 5-6 minutes; in fairness, it must be mentioned that their Maven builds were not parallelized (for correctness reasons) whereas their Bazel builds were.  But it’s worth highlighting that they managed to do this incrementally, by generating Bazel build definitions from their Maven ones, and that the quoted build times did not enable caching.  The associated tech talk slides/video indicates builds would be roughly in the 1-2 minute range with caching, although they hadn’t deployed that yet.

None of the above accounts talked about how long the conversion took, which I found peculiar.  Both Pinterest and Redfin called out how much more reliable their builds were once they switched to Bazel; Pinterest said, “we haven’t performed a single clean build on CI in over a year.”

In some negative results, which are helpful as well, Dropbox wrote about evaluating Bazel for their Android builds.  What’s interesting here is that other parts of Dropbox are heavily invested in Bazel, so there’s a lot of in-house experience, and that Bazel was significantly faster than their current build system (assuming caching was turned on; Bazel was significantly slower for clean builds without caching).  Yet Dropbox decided to not switch to Bazel due to tooling and development experience concerns.  They did leave open the possibility of switching in the future once the ecosystem matures.

The oddly-named Bazel Fawlty describes a conversion to Bazel from Go’s native tooling, and then a switch back after a litany of problems, including slower builds (but faster tests), a poor development experience (especially on OS X), and various things not being supported in Bazel leading to the native Go tooling still being required in some cases.  This post was also noteworthy for noting the amount of porting effort required to switch: eight months plus “many PR’s accepted into the bazel go rules git repo”.  I haven’t used Go, but I’m willing to discount some of the negative experience here due to the native Go tools being so good.

Neither one of these negative experiences translate exactly to Firefox: different languages/ecosystems, different concerns, different scales.  But both of them cite the developer experience specifically, suggesting that not only is there a large investment required to actually do the switchover, but you also need to write tooling around Bazel to make it more convenient to use.

Finally, a 2018 BazelCon talk discusses two Google projects that made the switch to Bazel and specifically to use remote caching and remote execution on Google’s public-facing cloud infrastructure: Android Studio and TensorFlow.  (You may note that this is the first instance where somebody has called out supporting remote execution as part of the switch; I think that implies getting a build to the point of supporting remote execution is more complicated than just supporting remote caching, which makes a certain amount of sense.)  Android Studio increased their test presubmit coverage by 4x, presumably by being able to run more than 4x test jobs than previously due to remote execution.  In the same vein, TensorFlow decreased their build and test times by 80%, and they could use significantly less powerful machines to actually run the builds, given that large machines in the cloud were doing the actual heavy lifting.

Unfortunately, I don’t think expecting those same reductions in test time, were Firefox to switch to Bazel, is warranted.  I can’t speak to Android Studio, but TensorFlow has a number of unit tests whose test results can be cached.  In the Firefox context, these would correspond to cppunittests, which a) we don’t have that many of and b) don’t take that long to run.  The bulk of our tests depend in one way or another on kitchen-sink-style artifacts (e.g. libxul, the JS shell, omni.ja) which essentially depend on everything else.  We could get some reductions for OS-specific modifications; Windows-specific changes wouldn’t require re-running OS X tests, for instance, but my sense is that these sorts of changes are not common enough to lead to an 80% reduction in build + test time.  I suppose it’s also possible that we could teach Bazel that e.g. devtools changes don’t affect, say, non-devtools mochitests/reftests/etc. (presumably?), which would make more test results cacheable.

I want to believe that Bazel + remote caching (+ remote execution if we could get there) will bring Firefox build (and maybe even test) times down significantly, but the above accounts don’t exactly move the needle from belief to certainty.

https://blog.mozilla.org/nfroyd/2019/11/01/evaluating-bazel-for-building-firefox-part-2/

https://tantek.com/2019/305/b1/redecentralize-decentralized-identity-reputation

At Mozilla we are well aware of how fragile the Web Public Key Infrastructure (PKI) can be. From fraudulent Certification Authorities (CAs) to implementation errors that leak private keys, users, often unknowingly, are put in a position where their ability to establish trust on the Web is compromised. Therefore, in keeping with our mission to create a Web where individuals are empowered, independent and safe, we welcome ideas that are aimed at making the Web PKI more robust. With initiatives like our Common CA Database (CCADB), CRLite prototyping, and our involvement in the CA/Browser Forum, we’re committed to this objective, and this is why we embraced the opportunity to partner with Cloudflare to test Delegated Credentials for TLS in Firefox, which is currently undergoing standardization at the IETF.

As CAs are responsible for the creation of digital certificates, they dictate the lifetime of an issued certificate, as well as its usage parameters. Traditionally, end-entity certificates are long-lived, exhibiting lifetimes of more than one year. For server operators making use of Content Delivery Networks (CDNs) such as Cloudflare, this can be problematic because of the potential trust placed in CDNs regarding sensitive private key material. Of course, Cloudflare has architectural solutions for such key material but these add unwanted latency to connections and present with operational difficulties. To limit exposure, a short-lived certificate would be preferable for this setting. However, constant communication with an external CA to obtain short-lived certificates could result in poor performance or even worse, lack of access to a service entirely.

The Delegated Credentials mechanism decentralizes the problem by allowing a TLS server to issue short-lived authentication credentials (with a validity period of no longer than 7 days) that are cryptographically bound to a CA-issued certificate. These short-lived credentials then serve as the authentication keys in a regular TLS 1.3 connection between a Firefox client and a CDN edge server situated in a low-trust zone (where the risk of compromise might be higher than usual and perhaps go undetected). This way, performance isn’t hindered and the compromise window is limited. For further technical details see this excellent blog post by Cloudflare on the subject.

See How The Experiment Works

We will soon test Delegated Credentials in Firefox Nightly via an experimental addon, called TLS Delegated Credentials Experiment. In this experiment, the addon will make a single request to a Cloudflare-managed host which supports Delegated Credentials. The Delegated Credentials feature is disabled in Firefox by default, but depending on the experiment conditions the addon will toggle it for the duration of this request. The connection result, including whether Delegated Credentials was enabled or not, gets reported via telemetry to allow for comparative study. Out of this we’re hoping to gain better insights into how effective and stable Delegated Credentials are in the real world, and more importantly, of any negative impact to user experience (for example, increased connection failure rates or slower TLS handshake times). The study is expected to start in mid-November and run for two weeks.

For specific details on the telemetry and how measurements will take place, see bug 1564179.

See The Results In Firefox

You can open a Firefox Nightly or Beta window and navigate to about:telemetry. From here, in the top-right is a Search box, where you can search for “delegated” to find all telemetry entries from our experiment. If Delegated Credentials have been used and telemetry is enabled, you can expect to see the count of Delegated Credentials-enabled handshakes as well as the time-to-completion of each. Additionally, if the addon has run the test, you can see the test result under the “Keyed Scalars” section.

You can also read more about telemetry, studies, and Mozilla’s privacy policy by navigating to about:preferences#privacy.

See It In Action

If you’d like to enable Delegated Credentials for your own testing or use, this can be done by:

1. In a Firefox Nightly or Beta window, navigate to about:config.
2. Search for the “security.tls.enable_delegated_credentials” preference – the preference list will update as you type, and “delegated” is itself enough to find the correct preference.
3. Click the Toggle button to set the value to true.
4. Navigate to https://dc.crypto.mozilla.org/
5. If needed, toggling the value back to false will disable Delegated Credentials.

Note that currently, use of Delegated Credentials doesn’t appear anywhere in the Firefox UI. This will change as we evolve the implementation.

We would sincerely like to thank Christopher Patton, fellow Mozillian Wayne Thayer, and the Cloudflare team, particularly Nick Sullivan and Watson Ladd for helping us to get to this point with the Delegated Credentials feature. The Mozilla team will keep you informed on the development of this feature for use in Firefox, and we look forward to sharing our results in a future blog post.

 

 

 

The post Validating Delegated Credentials for TLS in Firefox appeared first on Mozilla Security Blog.

https://blog.mozilla.org/security/2019/11/01/validating-delegated-credentials-for-tls-in-firefox/

http://fitzgeraldnick.com/2019/11/01/always-bump-downwards.html

For most of 2018, we've been issuing warnings about various bugs in the borrow checker that we plan to fix -- about two months ago, in the current Rust nightly, those warnings became hard errors. In about two weeks, when the nightly branches to become beta, those hard errors will be in the beta build, and they will eventually hit stable on December 19th, as part of Rust 1.40.0. If you're testing with Nightly, you should be all set -- but otherwise, you may want to go and check to make sure your code still builds. If not, we have advice for fixing common problems below.

Background: the non-lexical lifetime transition

When we released Rust 2018 in Rust 1.31, it included a new version of the borrow checker, one that implemented "non-lexical lifetimes". This new borrow checker did a much more precise analysis than the original, allowing us to eliminate a lot of unnecessary errors and make Rust easier to use. I think most everyone who was using Rust 2015 can attest that this shift was a big improvement.

The new borrow checker also fixed a lot of bugs

What is perhaps less well understood is that the new borrow checker implementation also fixed a lot of bugs. In other words, the new borrow checker did not just accept more programs -- it also rejected some programs that were only accepted in the first place due to memory unsafety bugs in the old borrow checker!

Until recently, those fixed bugs produced warnings, not errors

Obviously, we don't want to accept programs that could undermine Rust's safety guarantees. At the same time, as part of our commitment to stability, we try to avoid making sudden bug fixes that will affect a lot of code. Whenever possible, we prefer to "phase in" those changes gradually. We usually begin with "Future Compatibility Warnings", for example, before moving those warnings to hard errors (sometimes a small bit at a time). Since the bug fixes to the borrow checker affected a lot of crates, we knew we needed a warning period before we could make them into hard errors.

To implement this warning period, we kept two copies of the borrow checker around (this is a trick we use quite frequently, actually). The new checker ran first. If it found errors, we didn't report them directly: instead, we ran the old checker in order to see if the crate used to compile before. If so, we reported the errors as Future Compatibility Warnings, since we were changing something that used to compile into errors.

All good things must come to an end; and bad ones, too

Over time we have been slowly transitioning those future compatibility warnings into errors, a bit at a time. About two months ago, we decided that the time had come to finish the job. So, over the course of two PRs, we converted all remaining warnings to errors and then removed the old borrow checker implementation.

What this means for you

If you are testing your package with nightly, then you should be fine. In fact, even if you build on stable, we always recommend that you test your builds in CI with the nightly build, so that you can identify upcoming issues early and report them to us.

Otherwise, you may want to check your dependencies. When we decided to remove the old borrow checker, we also analyzed which crates would stop compiling. For anything that seemed to be widely used, we made sure that there were newer versions of that crate available that do compile (for the most part, this had all already happened during the warning period). But if you have those older versions in your Cargo.lock file, and you are only using stable builds, then you may find that your code no longer builds once 1.40.0 is released -- you will have to upgrade the dependency.

The most common crates that were affected are the following:

• url version 1.7.0 -- you can upgrade to 1.7.2, though you'd be better off upgrading to 2.1.0
• nalgebra version 0.16.13 -- you can upgrade to 0.16.14, though you'd be better off upgrading to 0.19.0
• rusttype version 0.2.0 to 0.2.3 -- you can upgrade to 0.2.4, though you'd be better upgrading to 0.8.1

You can find out which crates you rely upon using the cargo-tree command. If you find that you do rely (say) on url 1.7.0, you can upgrade to 1.7.2 by executing:

cargo update --package url --precise 1.7.2

Want to learn more?

If you'd like to learn more about the kinds of bugs that were fixed -- or if you are seeing errors in your code that you need to fix -- take a look at this excellent blog post by Felix Klock, which goes into great detail.

https://blog.rust-lang.org/2019/11/01/nll-hard-errors.html

Sideloading is a method of installing an extension in Firefox by adding an extension file to a special location using an executable application installer. This installs the extension in all Firefox instances on a computer.

Sideloaded extensions frequently cause issues for users since they did not explicitly choose to install them and are unable to remove them from the Add-ons Manager. This mechanism has also been employed in the past to install malware into Firefox. To give users more control over their extensions, support for sideloaded extensions will be discontinued. 

November 1 update: we’ve heard some feedback expressing confusion about how this change will give more control to Firefox users. Ever since we implemented abuse reporting in Firefox 68, the top kind of report we receive by far has been extension installs that weren’t expected and couldn’t be removed, and the extensions being reported are known to be distributed through sideloading. With this change, we are enforcing more transparency in the installation process, by letting users choose whether they want to install an application companion extension or not, and letting them remove it when they want to. Developers will still be free to self-distribute extensions on the web, and users will still be able to install self-distributed extensions. Enterprise administrators will continue to be able to deploy extensions to their users via policies. Other forms of automatic extension deployment like the ones used for some Linux distributions and applications like Selenium may be impacted by these changes. We’re still investigating some technical details around these cases and will try to strike the right balance between user choice and minimal disruption.

During the release cycle for Firefox version 73, which goes into pre-release channels on December 3, 2019 and into release on February 11, 2020, Firefox will continue to read sideloaded files, but they will be copied over to the user’s individual profile and installed as regular add-ons. Sideloading will stop being supported in Firefox version 74, which will be released on March 10, 2020. The transitional stage in Firefox 73 will ensure that no installed add-ons will be lost, and end users will gain the ability to remove them if they chose to.

If you self-distribute your extension via sideloading, please update your install flows and direct your users to download your extension through a web property that you own, or through addons.mozilla.org (AMO). Please note that all extensions must meet the requirements outlined in our Add-on Policies and Developer Agreement.  If you choose to continue self-distributing your extension, make sure that new versions use an update URL to keep users up-to-date. Instructions for distributing an extension can be found in our Extension Workshop document repository.

If you have any questions, please head to our community forum.

The post Upcoming changes to extension sideloading appeared first on Mozilla Add-ons Blog.

https://blog.mozilla.org/addons/2019/10/31/firefox-to-discontinue-sideloaded-extensions/

Yesterday, Jack Dorsey made a bold statement: Twitter will cease all political advertising on the platform. “Internet political ads present entirely new challenges to civic discourse: machine learning-based optimization of messaging and micro-targeting, unchecked misleading information, and deep fakes. All at increasing velocity, sophistication, and overwhelming scale,” he tweeted.

Later that day, Sheryl Sandberg responded: Facebook doesn’t have to cease political advertising… because the platform is “focused and leading on transparency.” Sandberg cited Facebook’s ad archive efforts, which ostensibly allow researchers to study the provenance and impact of political ads.

To be clear: Facebook is still falling short on its transparency commitments. Further, even perfect transparency wouldn’t change the fact that Facebook is accepting payment to promote dangerous and untrue ads. 

Some brief history: Because of the importance of transparency in the political ad arena, Mozilla has been closely analyzing Facebook’s ad archive for over a year, and assessing its ability to provide researchers and others with meaningful information.

In February, Mozilla and 37 civil society organizations urged Facebook to provide better transparency into political advertising on their platform. Then, in March, Mozilla and leading disinformation researchers laid out exactly what an effective ad transparency archive should look like.

But when Facebook finally released its ad transparency API in March, it was woefully ineffective. It met just two of experts’ five minimum guidelines. Further, a Mozilla researcher uncovered a long list of bugs and shortcomings that rendered the API nearly useless.

The New York Times agreed: “Ad Tool Facebook Built to Fight Disinformation Doesn’t Work as Advertised,” reads a July headline. The article continues: “The social network’s new ad library is so flawed, researchers say, that it is effectively useless as a way to track political messaging.”

Since that time, Mozilla has confirmed that Facebook has made small changes in the API’s functionality — but we still judge the tool to be fundamentally flawed for its intended purpose of providing transparency and a data source for rigorous research.

Rather than deceptively promoting their failed API, Facebook must heed researchers’ advice and commit to truly transparent political advertising. If they can’t get that right, maybe they shouldn’t be running political ads at all for the time being.

The post Facebook Is Still Failing at Ad Transparency (No Matter What They Claim) appeared first on The Mozilla Blog.

https://blog.mozilla.org/blog/2019/10/31/facebook-is-still-failing-at-ad-transparency-no-matter-what-they-claim/

Mozilla has hosted an enterprise instance of IRCCloud for several years now, and it’s been a great client to use with our IRC network. IRCCloud has deprecated their enterprise product and so Mozilla recently decommissioned our instance. I then saw several colleagues praising The Lounge as a good self-hosted alternative. I became even more interested when I saw that the project maintains a docker image distribution of their releases. I now have an instance running and I’m using irc.mozilla.org via this client and I agree with my colleagues: it’s a decent replacement.

https://pmac.io/2019/10/thelounge-on-dokku/

A long time ago Maciej wrote down five types of encapsulation for shadow trees (i.e., node trees that are hidden in the shadows from the document tree):

1. Encapsulation against accidental exposure — DOM nodes from the shadow tree are not leaked via pre-existing generic APIs — for example, events flowing out of a shadow tree don't expose shadow nodes as the event target.
2. Encapsulation against deliberate access — no API is provided which lets code outside the component poke at the shadow DOM. Only internals that the component chooses to expose are exposed.
3. Inverse encapsulation — no API is provided which lets code inside the component see content from the page embedding it (this would have the effect of something like sandboxed iframes or Caja).
4. Isolation for security purposes — it is strongly guaranteed that there is no way for code outside the component to violate its confidentiality or integrity.
5. Inverse isolation for security purposes — it is strongly guaranteed that there is no way for code inside the component to violate the confidentiality or integrity of the embedding page.

Types 3 through 5 do not have any kind of support and type 4 and 5 encapsulation would be hard to pull off due to Spectre. User agents typically use a weaker variant of type 4 for their internal controls, such as the video and input elements, that does not protect confidentiality. The DOM and HTML standards provide type 1 and 2 encapsulation to web developers, and type 2 mostly due to Apple and Mozilla pushing rather hard for it. It might be worth providing an updated definition for the first two as we’ve come to understand them:

1. Open shadow trees — no standardized web platform API provides access to nodes in an open shadow tree, except APIs that have been explicitly named and designed to do so (e.g., composedPath()). Nothing should be able to observe these shadow trees other than through those designated APIs (or “monkey patching”, i.e., modifying objects). Limited form of information hiding, but no integrity or confidentiality.
2. Closed shadow trees — very similar to open shadow trees, except that designated APIs also do not get access to nodes in a closed shadow tree.

Type 2 encapsulation gives component developers control over what remains encapsulated and what is exposed. You need to take all your users into account and expose the best possible public API for them. At the same time, it protects you from folks taking a dependency on the guts of the component. Aspects you might want to refactor or add functionality to over time. This is much harder with type 1 encapsulation as there will be APIs that can reach into the details of your component and if users do so you cannot refactor it without updating all the callers.

Now, both type 1 and 2 encapsulation can be circumvented, e.g., by a script changing the attachShadow() method or mutating another builtin that the component has taken a dependency on. I.e., there is no integrity and as they run in the same origin there is no security boundary either. The limited form of information hiding is primarily a maintenance boon and a way to manage complexity. Maciej addresses this as well:

If the shadow DOM is exposed, then you have the following risks:

1. A page using the component starts poking at the shadow DOM because it can — perhaps in a rarely used code path.
2. The component is updated, unaware that the page is poking at its guts.
3. Page adopts new version of component.
4. Page breaks.
5. Page author blames component author or rolls back to old version.

This is not good. Information hiding and hiding of implementation details are key aspects of encapsulation, and are good software engineering practices.

https://annevankesteren.nl/2019/10/encapsulation-theory

It seems to be a common problem, you want to display some content on the web with a certain aspect ratio but you don’t know the size you will be displaying at. How do you do this? CSS doesn’t really have the tools to do the job well currently (there are proposals). In my case I want to display a video and associated controls as large as possible inside a space that I don’t know the size of. The size of the video also varies depending on the one being displayed.

Padding height

The answer to this according to almost all the searching I’ve done is the padding-top/bottom trick. For reasons that I don’t understand when using relative lengths (percentages) with the CSS padding-top and padding-bottom properties the values are calculated based on the width of the element. So padding-top: 100% gives you padding equal to the width of the element. Weird. So you can fairly easily create a box with a height calculated from its width and from there display content at whatever aspect ratio you choose. But there’s an inherent problem here, you need to know the width of the box in the first place, or at least be able to constrain it based on something. In my case the aspect ratio of the video and the container are both unknown. In some cases I need to constrain the width and calculate the height, but in others I need to constrain the height and calculate the width which is where this trick fails.

object-fit

There is one straightforward solution. The CSS object-fit property allows you to scale up content to the largest size possible for the space allocated. This is perfect for my needs, except that it only works for replaced content like videos and images. In my case I also need to overlay some controls on top and I won’t know where to position them unless they are inside a box the size of the video.

The solution?

So what I need is something where I can create a box with set sizes and then scale both width and height to the largest that fit entirely in the container. What do we have on the web that can do that … oh yes, SVG. In SVG you can define the viewport for your content and any shapes you like inside with SVG coordinates and then scale the entire SVG viewport using CSS properties. I want HTML content to scale here and luckily SVG provides the foreignObject element which lets you define a rectangle in SVG coordinates that contains non-SVG content, such as HTML! So here is what I came up with:

This is pretty straightforward. It creates an SVG document with a viewport with a 4:3 aspect ratio, a foreignObject container that fills the viewport and then a div that fills that. what you end up with is a div with a 4:3 aspect ratio. While this shows it working against the full page it seems to work anywhere with constraints on either height, width or both such as in a flex or grid layout. Obviously changing the viewBox allows you to get any aspect ratio you like, just setting it to the size of the video gives me exactly what I want.

You can see it working over on codepen.

https://www.oxymoronical.com/blog/2019/10/Creating-HTML-content-with-a-fixed-aspect-ratio-without-the-padding-trick