Background: I’ve been relearning Rust (more about that in a separate post, some time later), and in doing so, I chose to implement the low-level parts of git (I’ll touch the why in that separate post I just promised).

Disclaimer: It’s friday. This is not entirely(?) a serious post.

So, I was looking at Documentation/technical/index-format.txt, and saw:

32-bit number of index entries.

What? The index/staging area can’t handle more than ~4.3 billion files?

There I was, writing Rust code to write out the index.

try!(out.write_u32::(self.entries.len()));

(For people familiar with the byteorder crate and wondering what NetworkOrder is, I have a use byteorder::BigEndian as NetworkOrder)

And the Rust compiler rightfully barfed:

error: mismatched types:
 expected `u32`,
    found `usize` [E0308]

And there I was, wondering: “mmmm should I just add as u32 and silently truncate or … hey what does git do?”

And it turns out, git uses an unsigned int to track the number of entries in the first place, so there is no truncation happening.

Then I thought “but what happens when cache_nr reaches the max?”

Well, it turns out there’s only one obvious place where the field is incremented.

What? Holy coffin nails, Batman! No overflow check?

Wait a second, look 3 lines above that:

ALLOC_GROW(istate->cache, istate->cache_nr + 1, istate->cache_alloc);

Yeah, obviously, if you’re incrementing cache_nr, you already have that many entries in memory. So, how big would that array be?

        struct cache_entry **cache;

So it’s an array of pointers, assuming 64-bits pointers, that’s … ~34.3 GB. But, all those cache_nr entries are in memory too. How big is a cache entry?

struct cache_entry {
        struct hashmap_entry ent;
        struct stat_data ce_stat_data;
        unsigned int ce_mode;
        unsigned int ce_flags;
        unsigned int ce_namelen;
        unsigned int index;     /* for link extension */
        unsigned char sha1[20];
        char name[FLEX_ARRAY]; /* more */
};

So, 4 ints, 20 bytes, and as many bytes as necessary to hold a path. And two inline structs. How big are they?

struct hashmap_entry {
        struct hashmap_entry *next;
        unsigned int hash;
};

struct stat_data {
        struct cache_time sd_ctime;
        struct cache_time sd_mtime;
        unsigned int sd_dev;
        unsigned int sd_ino;
        unsigned int sd_uid;
        unsigned int sd_gid;
        unsigned int sd_size;
};

Woohoo, nested structs.

struct cache_time {
        uint32_t sec;
        uint32_t nsec;
};

So all in all, we’re looking at 1 + 2 + 2 + 5 + 4 32-bit integers, 1 64-bits pointer, 2 32-bits padding, 20 bytes of sha1, for a total of 92 bytes, not counting the variable size for file paths.

The average path length in mozilla-central, which only has slightly over 140 thousands of them, is 59 (including the terminal NUL character).

Let’s conservatively assume our crazy repository would have the same average, making the average cache entry 151 bytes.

But memory allocators usually allocate more than requested. In this particular case, with the default allocator on GNU/Linux, it’s 156 (weirdly enough, it’s 152 on my machine).

156 times 4.3 billion… 670 GB. Plus the 34.3 from the array of pointers: 704.3 GB. Of RAM. Not counting the memory allocator overhead of handling that. Or all the other things git might have in memory as well (which apparently involves a hashmap, too, but I won’t look at that, I promise).

I think one would have run out of memory before hitting that integer overflow.

Interestingly, looking at Documentation/technical/index-format.txt again, the on-disk format appears smaller, with 62 bytes per file instead of 92, so the corresponding index file would be smaller. (And in version 4, paths are prefix-compressed, so paths would be smaller too).

But having an index that large supposes those files are checked out. So let’s say I have an empty ext4 file system as large as possible (which I’m told is 2^60 bytes (1.15 billion gigabytes)). Creating a small empty ext4 tells me at least 10 inodes are allocated by default. I seem to remember there’s at least one reserved for the journal, there’s the top-level directory, and there’s lost+found ; there apparently are more. Obviously, on that very large file system, We’d have a git repository. git init with an empty template creates 9 files and directories, so that’s 19 more inodes taken. But git init doesn’t create an index, and doesn’t have any objects. We’d thus have at least one file for our hundreds of gigabyte index, and at least 2 who-knows-how-big files for the objects (a pack and its index). How many inodes does that leave us with?

The Linux kernel source tells us the number of inodes in an ext4 file system is stored in a 32-bits integer.

So all in all, if we had an empty very large file system, we’d only be able to store, at best, 2^32 – 22 files… And we wouldn’t even be able to get cache_nr to overflow.

… while following the rules. Because the index can keep files that have been removed, it is actually possible to fill the index without filling the file system. After hours (days? months? years? decades?*) of running

seq 0 4294967296 | while read i; do touch $i; git update-index --add $i; rm $i; done

One should be able to reach the integer overflow. But that’d still require hundreds of gigabytes of disk space and even more RAM.

• At the rate it was possible to add files to the index when I tried (yeah, I tried), for a few minutes, and assuming a constant rate, the estimate is close to 2 years. But the time spent reading and writing the index file increases linearly with its size, so the longer it’d run, the longer it’d take.

Ok, it’s actually much faster to do it hundreds of thousand files at a time, with something like:

seq 0 100000 4294967296 | while read i; do j=$(seq $i $(($i + 99999))); touch $j; git update-index --add $j; rm $j; done

At the rate the first million files were added, still assuming a constant rate, it would take about a month on my machine. Considering reading/writing a list of a million files is a thousand times faster than reading a list of a billion files, assuming linear increase, we’re still talking about decades, and plentiful RAM. Fun fact: after leaving it run for 5 times as much as it had run for the first million files, it hasn’t even done half more…

One could generate the necessary hundreds-of-gigabytes index manually, that wouldn’t be too hard, and assuming it could be done at about 1 GB/s on a good machine with a good SSD, we’d be able to craft a close-to-explosion index within a few minutes. But we’d still lack the RAM to load it.

So, here is the open question: should I report that integer overflow?

Wow, that was some serious procrastination.

Edit: Epilogue: Actually, oops, there is a separate integer overflow on the reading side that can trigger a buffer overflow, that doesn’t actually require a large index, just a crafted header, demonstrating that yes, not all integer overflows are equal.

https://glandium.org/blog/?p=3659

On October 07, 2016 I'll be shutting down Code Firefox.

With over 100k unique visits and over 30k full video views, it helped hundreds of Mozillians and Mozilla employees get ramped up for hacking on Firefox. It was from the start a personal side project and wasn't funded or sponsored in any way.

It isn't being shutdown because of a conflict of interest with Brave, but instead because over time the content naturally becomes more obsolete as better development methods surface.

I'm happy with what it is and the purpose it had, but shutting it down is the responsible thing to do.

https://brianbondy.com/blog/173/shutting-down-code-firefox

As you may already know, I’m spending my summer interning with Mozilla’s Enginering Productivity team through the Outreachy program. Specifically, the project I’m working on is called Test-driven Refactoring of Marionette’s Python Test Runner. But what exactly does that mean? What is Marionette, and what does it have to do with testing?

In this two-part series, I’d like to share a bit of what I’ve learned about the Marionette project, and how its various components help us test Firefox by allowing us to automatically control the browser from within. Today, in Act I, I’ll give an overview of how the Marionette server and client make the browser our puppet. Later on, in Act II, I’ll describe how the Marionette test harness and runner make automated testing a breeze, and let you in on the work I’m doing on the Marionette test runner for my internship.

And since we’re talking about puppets, you can bet there’s going to be a hell of a lot of Being John Malkovich references. Consider yourself warned!

How do you test a browser?

On the one hand, you probably want to make sure that the browser’s interface, or chrome, is working as expected; that users can, say, open and close tabs and windows, type into the search bar, change preferences and so on. But you probably also want to test how the browser displays the actual web pages, or content, that you’re trying to, well, browse; that users can do things like click on links, interact with forms, or play video clips from their favorite movies.

Because let's be honest, that's like 96% of the point of the internet right there, no?

These two parts, chrome and content, are the main things the browser has to get right. So how do you test them?

Well, you could launch the browser application, type “Being John Malkovich” into the search bar and hit enter, check that a page of search results appears, click on a YouTube link and check that it takes you to a new page and starts playing a video, type “I

http://vakila.github.io/blog/marionette-act-i-automation/

Hello, SUMO Nation!

Welcome to the hot-hot-hot July! Wherever you are, remember – don’t melt! It’s a bit hard to achieve that over here (air conditioning only gets you so far), so I hope you’ll have an easier challenge on your side. Speaking of hotness – here are the red-hot-

Welcome, new contributors!

• Yannick Albert
• Emerson Santos

If you just joined us, don’t hesitate – come over and say “hi” in the forums!

Most recent SUMO Community meeting

• You can read the notes here and see the video at AirMozilla.
  

The next SUMO Community meeting…

• …is happening on the 13th of July!
• If you want to add a discussion topic to the upcoming meeting agenda:
  • Start a thread in the Community Forums, so that everyone in the community can see what will be discussed and voice their opinion here before Wednesday (this will make it easier to have an efficient meeting).
  • Please do so as soon as you can before the meeting, so that people have time to read, think, and reply (and also add it to the agenda).
  • If you can, please attend the meeting in person (or via IRC), so we can follow up on your discussion topic during the meeting with your feedback.

Community

• PLATFORM REMINDER & UPDATE! The Platform Meetings are BACK! If you missed the previous ones, here is the etherpad, with links to the recordings on AirMo. We are closing the sandbox preview – huge thanks to everyone who had the time to take a look around and provide feedback!
  • The sandbox is now closed and we’re moving towards discussing more (non-)technical details of a potential migration to Lithium.
  • Different approaches to making l10n work in Lithium have been discussed – next week, we will hopefully know more about the option of using Pontoon with Lithium. The initial plan is to roll out an “out of the box” solution with low-level customizations to make it easy for localizers to continue working on the KB.
• POST-LONDON WORK WEEK UPDATE:
  • A second Show & Tell post coming your way… Have you already read the first one?
• Heads up! The Release Report for Firefox 47 is almost here!
• FINAL REMINDER: help us shape and improve Mozilla’s brand!
  • https://blog.mozilla.org/opendesign
  • https://medium.com/mozilla-branding
  • To quote Tim Murray: We are now open for feedback on these 7 themes—and for new theme ideas covering anything that might be missing—through the comments section of the blog, on Medium, or directly to me or Janis Greenspan. Now’s the time to add your voice. 
• Ongoing reminder #1: if you think you can benefit from getting a second-hand device to help you with contributing to SUMO, you know where to find us.
• Ongoing reminder #2: we are looking for more contributors to our blog. Do you write on the web about open source, communities, SUMO, Mozilla… and more? Do let us know!
• Ongoing reminder #3: want to know what’s going on with the Admins? Check this thread in the forum.

Social

• Thanks for rocking Twitter last month, people!
• Also, shout outs to our Brazillian community for their general awesomeness!
• Ongoing reminder: We have a training out there for all those interested in Social Support – talk to Madalina or Costenslayer on #AoA (IRC) for more information.

Support Forum

Firefox

• for Android
  • Version 48 is still on track for release on August 2nd.

• for Desktop
  • Version 47 support discussion thread.
  • Version 48 also on track for August 2nd release.

• for iOS
  • 5.0 has been slightly delayed yet again, but we’re expecting a launch next week, for sure!
  • Remember, it will be a big update – don’t forget the articles for it are ready to be localized!

So… Are you still following the Euro 2016 cup games? Are you watching cool movies? Know some great music that you think we should all know about? Tell us in our forums!

 

https://blog.mozilla.org/sumo/2016/07/07/whats-up-with-sumo-7th-july/

This is a weekly call with some of the Reps to discuss all matters about/affecting Reps and invite Reps to share their work with everyone.

https://air.mozilla.org/reps-weekly-20160707/

They say a Mozilla Web QA team member is the most fearless creature in the world. They say their jaws are powerful enough to crush...

https://air.mozilla.org/web-qa-team-meeting-7/

On my previous post i’ve talked about what tools do we use at Mozilla for static analysis, now i want to take a closer look on some specific checkers triggered by Coverity and why i do consider them very useful.

1. Side effect in assertion
   This is a very common mistake that is pretty hard to discover when reviewing, this is triggered when inside an assert an assignment expression is present instead of a logical comparison.  There are two key problems that can occur in this context:
   • As most of the asserts are intended only for debug builds, on release the assignment will not take place resulting in a different behaviour, that can lead to a bug.
   • The result of the assignment expression is the assignee itself thus the following assert will never be false:

     ASSERT(a = 3);

   This is an actual issue from Coverity that signals the problem and the eventual side effects that there could be.
   

2. Side effect in assertion
   Often happens that a pointer is null checked on a brach, and o separate code path this check is no longer performed despite that the pointer is dereferenced. In case the pointer was null the resulting of the dereference is undefined.From this example is obvious that pointer aAcc is null checked on line 649 and later on line 651 it is dereferenced without null checking it.  But what causes the null pointer dereference and how Coverity figured out that this is an actual issue? Looking further in the code we find:
   There are a couple of exceptions when dereferencing a pointer that’s null will not have an impact and is perfectly correct to do this:

   class Test {
   public:
     void print(std::string str) {
       std::cout<<str<<std::endl; } }; int main(int argc, const char **argv) { Test *myCls = nullptr; myCls->print("Just A Test");
     return 0;
   }
   

   The pointer is not needed to call the function, the type of the pointer is known so the code for the function is known and inside print we don’t dereference pointer this. In case the method would have been virtual the call would have failed since this would have been dereferenced.

3. Buffer not null terminated
   This type of issue can happen in many situations but most likely when a copying with the length of the same size as the target size. This is very useful if the buffer is treated as a null terminated string in later operations, a buffer overflow or over-read may occur.
   In this example the correct approach would be to have the value of the 3rd parameter from strncpy be IFNAMSIZ – 1.
4. Result is not floating-point
   The result of a division is truncated to an integer witch translates to a loss of precision in calculus. When dividing two values of integer types, integer division is used, which ignores any remainder. When such a result is used in a context expecting a floating-point number, it is likely that floating-point division was intended.
   In this context type of nsrecord is integer, and in order to fix this one of the variables from the division expression should be first casted to float.
5. Resource leak
   The system resources will not be freed and reused thus reducing the future availability of resources.  It’s a very common issue, that in most situations tends to appear when naked pointers are used.
   
   On line 87 an allocation is demanded if it fails the function will return on line 90. The problem appears in switch statement on the default branch where it just returns without freeing the allocated resource.
6. Nesting level does not match indentation
   Code that should be used conditionally is actually executed outside of the conditional branch. This is based on the indentation of code, like below:
7. Uninitialised class members
   This consists of two sub-checkers, one for pointers and the other for scalars. If we are talking about scalars the uninitialised one will contain an arbitrary value. For an uninitialised pointer field it will point to a random memory location. These issues are very common and can be easily avoided by initialising  all of the members from that class in the constructors.
    

Coverity is a great tool, with a low rate of false positive, it really achieves at being one of the best, but despite this i was very sad that it doesn’t have a very good support for thread analysis and nor it has support for any thread sanitisation checkers.

 

https://blog.influx.ro/2016/07/07/a-more-detailed-look-on-some-of-coveritys-checkers-for-static-analysis/

The canonical ordered-map type for Rust is std::collections::BTreeMap, which looks nice, except that the range methods are marked unstable and so can't be used at all except with the nightly-build Rust toolchain. Those methods are the only way to perform operations like "find first element greater than a given key", so BTreeMap is mostly useless in stable Rust.

This wouldn't be a big deal if crates.io had a good ordered-map library that worked with stable Rust ... but, as far as I can tell, until now it did not. I didn't want to switch to Rust nightly just to use ordered maps, so I solved this problem by forking container-rs's bst crate, modifying it to work on stable Rust (which meant ripping out a bunch of "unstable" annotations, fixing a few places that required unstable "box" syntax, and fixing some test code that depended on unboxed closures), and publishing the result as stable_bst. (Note: I haven't actually gotten around to using it yet, so maybe it's broken, but at least its tests pass.)

So, if you want to use ordered maps with stable Rust, give it a try. bst has a relatively simple implementation and, no doubt, is less efficient than BTreeMap, but it should be comparable to the usual C++ std::map implementations.

Currently it supports only C++-style lower_bound and upper_bound methods for finding elements less/greater than a given key. range methods similar to BTreeMap could easily be added, using a local copy of the unstable standard Bound type. I'm not sure if I'll bother but I'd accept PRs.

Update I realized the lower_bound and upper_bound methods were somewhat useless since they only return forward iterators, so I bit the bullet, implemented the range/range_mut methods, removed lower_bound/upper_bound and the reverse iterators which are superseded by range, and updated crates.io.

FWIW I really like the range API compared to C++-style upper/lower_bound. I always have to think carefully to use the C++ API correctly, whereas the range API is easy to use correctly: you specify upper and lower bounds, each of which can be unbounded, exclusive or inclusive, just like in mathematics. A nice feature of the range API (when implemented correctly!) is that if you happen to specify a lower bound greater than the upper bound, it returns an empty iterator, instead of returning some number of wrong elements --- or crashing exploitably --- as the obvious encoding in C++ would do.

Another somewhat obscure but cool feature of range is that the values for bounds don't have to be exactly the same type as the keys, if you set up traits correctly. ogoodman on github pointed out that in some obscure cases you want range endpoints that can't be expressed as key values. Their example is keys of type (A, B), lexicographically ordered, where B does not have min or max values (e.g., arbitrary-precision integers), and you want a range containing all keys with a specific value for A. With the BTreeMap and stable_bst::TreeMap APIs you can handle this by making the bounds be type B', where B' is B extended with artificial min/max values, and defining traits to order B/B' values appropriately.

http://robert.ocallahan.org/2016/07/ordered-maps-for-stable-rust.html

I have a book, purchased during my undergraduate days, entitled Introduction to Algorithms. Said book contains a wealth of information about algorithms and data structures, has its own Wikipedia page, and even a snappy acronym people use (“CLRS”, for the first letters of its authors’ last names).

When I bought it, I expected it to be both an excellent textbook and a book I would refer to many times throughout my professional career.  I cannot remember whether it was a good textbook in the context of my classes, and I cannot remember the last time I opened it to find some algorithm or verify some subtle point.  Mostly, it has served two purposes: an excellent support for my monitor to position the monitor more closely to eye level, and as extra weight to move around when I have had to transfer my worldly possessions from place to place.

Whether this reflects on the sort of code I have worked on, or the rise of the Internet for answering questions, I am unsure.

I have another book, also purchased during my undergraduate days, entitled Programming with POSIX Threads.  Said book contains a wealth of information about POSIX threads (“pthreads”), is only mentioned in “Further Reading” on the Wikipedia page for POSIX threads, and has no snappy acronym associated with it.

I purchased this book because I thought I might assemble a library of programming knowledge, and of course threads would be a part of that.  Mostly, it would sit on the shelves to show people I was a Real Programmer(tm).

Instead, I have found it to be one of those books to always have close at hand, particularly working on Gecko.  Its explanations of the basic concepts of synchronization are clear and extensive, its examples of how to structure multithreaded algorithms are excellent, and its secondary coverage of “real-world” things such as memory ordering and signals + threads (short version: “don’t”) have been helpful when people have asked me for opinions or to review multi-threaded code.  When I have not followed the advice of this book, I have found myself in trouble later on.

My sense when searching for some of the same topics the book covers is that finding the same quality of coverage for those topics online is rather difficult, even taking into account that topics might be covered by disparate people.

If I had to trim my computer book library down significantly, I’m pretty sure I know what book I would choose.

What book have you found unexpectedly (un)helpful in your programming life?

https://blog.mozilla.org/nfroyd/2016/07/06/on-the-usefulness-of-computer-books/

mconley livehacks on real Firefox bugs while thinking aloud.

https://air.mozilla.org/the-joy-of-coding-episode-62/

This is the sumo weekly call

https://air.mozilla.org/weekly-sumo-community-meeting-july-6-2016/

This is also available on Medium

This is not about a post about trigger words or discriminatory expressions. There is a lot of information about this available, and even some excellent linting tools for your texts. It is also not about unconscious bias. Or well, maybe it is. Learned bias for sure.

This is a post about some sentences used in technical presentations that sound encouraging. In reality they may exclude people in the audience and make them feel bad about their level of knowledge. These are the following sentences and I’ll explain in detail how to replace them with something less destructive:

• “This is easy…”
• “I’ll repeat quickly, for the few of you who don’t know…”
• “Everybody can do that…”
• “$x solves this problem, so you don’t have to worry about it”
• “As everybody knows…”
• “That’s why $y(your product) is much better than (competitor) $x”
• “This is just like we learned in school…”
• “This can be done in a few lines of code…”
• “If you want to be professional, do $x”

None of these are a show-stopper and make you a terrible presenter. There may even be ways to use them that are not confusing and destructive. This is language, and in some cultures they may be OK to use. I’m not here to tell people off. I am here to make you aware that something that sounds good might make people feel bad. And that’s not what we’re here for as presenters.

As a presenter your job is not only to give out technical information. You also need to inspire and to entertain. Often you overshoot the mark by simplifying things and trying to hard to please.

It is important to remind ourselves that we can not assume much of our audience. The room might be full of experts, but the video recording is also going out to everybody. Explaining things in a simple fashion is not dumbing them down. It may actually be the hardest task there is for a presenter.

It is stressful to be at an expert event. As an audience member you don’t want to appear less able than others. As a presenter, it is worse. Presenting is a balancing act. You neither want to sound condescending, overload the audience, make people feel stupid, appear too basic … and, and, and…

I’ve heard the following expressions at a lot of events and I always cringed a bit. Often they are OK, and no harm done. But,to improve as presenters it may be a good idea to be more conscious about what we do and what effects it can have.

“This is easy…”

We often try to calm down the audience by making what we show appear simple. The problem with that is that what is simple for us might still be confusing to the people in the room. Add peer-pressure to that and people will neither speak up that they don’t understand, nor feel empowered. The opposite applies – by saying something is easy and people failing to grasp or apply it, we make them feel stupid. If you make me feel stupid, you may inspire me to get better. But I don’t do it for the right reason – I do it out of guilt and self-doubt.

The worst way to use “this is easy” is when you rely on a lot of abstractions or tools to achieve the easy bit. Each of those could be a stumbling block for people applying your wisdom.

Replacements:

• “Here are a few steps how to achieve this…”
• “By using these tools, which are all well documented, you can…”
• “The way to get this done is…”

Using these you send people on a journey. They don’t tell them that the end result is already a given. Who knows, they may find a way to improve your “easy” one.

“I’ll repeat quickly, for the few of you who don’t know…”

This expression just fell at a conference I attended and it made me cringe. The presenter meant to be encouraging in a “hey, we all are already on board” way, but it can come across as arrogant. Even worse, it already singles out those who do not know, and makes them feel like they are under a spotlight.

If the intention is to do a quick intro on what you want to build upon, it is better to phrase it as a reminder, not a “you already know, what am I doing here”.

Replacements

• “Just as a reminder, here is what $x is about…”
• “As you may remember, $x is about…”
• “We’re building this using $x, which is…”

This adds your repetition into the flow instead of being an excuse.

“Everybody can do that…”

If everybody can do it, why do I listen to you? Also, if everybody can do it, how come I never managed to? If you use this you either present something basic, or you over-simplify a complex matter. The latter can appear to be empowering; you take away the fear of approaching something. But, it backfires when people can’t use it. Then you exclude them from “everybody”. And that hurts.

Replacements:

• “If you know your way around $x, $y and $z, you should find it easy to…”
• “Once you managed to do that, you’ll find it makes the rest of your work easier…”
• “It is a very effective way to work, if it works for you, tell others about it”

This again makes it a reminder and a starting point of a journey. Not a given that is redundant to repeat.

“$x solves this problem, so you don’t have to worry about it”

Hooray for your product – it solves everything. Now buy it and impress people with wisdom you don’t have. And feel worse when you get praised for it. This is a classic sales pitch which works with end user products. As a developer you should always worry about what you use in your products as each part can become an issue. And it will be up to you to fix it.

Replacements

• “$x solves the problems around $y, so you can build $z”
• “$x was created to make $y easier and is used in production, the results are encouraging…”
• “Here are the steps you can do by hand that $x does for you…”

Pop open the hood, show how your product works. Don’t sell all-healing remedies.

“As everybody knows…”

Common knowledge is a myth and relies on your environment, access to information, time to consume news and the way you learn. Presenting something as common knowledge may make people think “so how come I ever heard of it?” and stop them in their tracks.

Replacements:

• “This has been around for a while and was explained wonderfully in $x ($x being a resource you link to)”
• “Tests have shown that $x is a given for a lot of solutions (point to research, give proof)”
• “I base this on the fact that $x, as proven many times by… (and add a list a resources)”

“Citation needed” is a wonderful way to say something and prove your point. You show people that you did your homework before you make an assumption. And you give those who did not the tools to do so.

“This is just like we learned in school…”

This assumes everybody went to a school with the same curriculum as you. A lot of people have not. This is especially destructive when it applies to knowledge that was part of a Computer Science degree.

Replacements:

• “This has been part of Computer Science teaching for years, and for good reason because $x”
• “This should look familiar to anyone who went to a similar school as me, and for those who didn’t, there’s truckloads of information available online about it”
• “You might remember this from school – now you see how it can be applied in a real job. Who knew?”

A lot of people create the web. Not all took the official path.

“That’s why $y(your product) is much better than (competitor) $x”

This is common in advertising, especially in America. You show off your product by making others look worse. This is pointless and only invites criticism and retaliation by others. As a tech presenter, you should know that the other product is also built by people. Final decision of what gets shipped are not always based on technical merit. It is a cheap shot.

Replacements:

• “Here’s how to do that with product $x, we took a different path and here’s why…”
• “There are many solutions to this. We found that some were lacking a feature that made us more effective, which is $x”
• “You can use whatever makes you happy to achieve $x. We added the following, as we found it was missing…”

Showing you know about your competition prevents questions about it. Showing how they differ allows people to make up their mind which is better instead of you telling them and hoping they agree.

“This can be done in a few lines of code…”

The amount of code has become a contrived way of showing how effective our solutions are. Almost always the “quick and small solution” blossoms into a much larger one once it is used in production. It makes much more pragmatic sense to tell people that this is inevitable, and praise the small starting point for what it is – a start.

Replacements:

• “As you can see, starting this is a few lines of code. I simplified this to show here, the source code is available at $x”
• “For now, this is all that is needed to achieve this. No doubt, you will need to add more to it, but it is a starting point”
• “By abstracting some of the issues out, we can cut down our code to a few lines”
• “As we rely on functionality of $x, this means our implementation can be very small…”

A lot of times, this solves our own issue of showing only a few lines of code on a slide. Instead, let’s write understandable code that we explain in sections rather than one magical tidbit.

“If you want to be professional, do $x”

People have different opinions what a “professional” is. Whilst we worry about quality and maintenance, other people put more merit on fast delivery. The state of the art changes all the time, and a sentence like this can look silly in a few weeks.

Replacements:

• “$x, $y and $z are using this heavily to deliver their products. Here are some case studies that show the positive results…”
• “Using $x gives you a starting point you can rely on and makes it easier to explain to your replacement how to take over the product…”
• “The benefits of $x are $y, which makes it a professional tool to use…”

You achieve professionalism with experience and by learning about new thing and retaining them. Things people say on stage and define as “best practice” need validation by professionals. It is not up to you as a presenter to define that.

A quick check

There are more unintentional destructive expressions. Read through your talks and watch your videos and then ask yourself: “how would I feel listening to this if I didn’t know what I know?”. Then remove or rephrase accordingly.

Our market grew as fast as it did by being non-discriminatory of background or level of education. Granted, most of us grew up in safe environments and were lucky enough to have free schooling. But there are a lot of people in our midst who came from nowhere or at least nowhere near computer science. And they do great work. I’d go so far as to say that the diversity of backgrounds made the web what it is now: a beautiful mess that keeps evolving into who knows what. It is anything but boring. There is never “one way” to reach a goal. We discovered a lot of our solutions by celebrating different points of view.

Photo by alyona_fedotova

https://www.christianheilmann.com/2016/07/06/things-not-to-say-on-stage-at-a-tech-event/

Summary / TL;DR

Project	What’s in it?	Status
C++17	See below	Committee Draft published; final publication on track for 2017
Filesystems TS	Standard filesystem interface	Published! Part of C++17
Library Fundamentals TS I	optional, any, string_view and more	Published! Part of C++17
Library Fundamentals TS II	source code information capture and various utilities	Resolution of comments from national standards bodies in progress
Concepts (“Lite”) TS	Constrained templates	Published! Not part of C++17
Parallelism TS vI	Parallel versions of STL algorithms	Published! Part of C++17
Parallelism TS v2	TBD. Exploring task blocks, progress guarantees, SIMD.	Under active development
Transactional Memory TS	Transaction support	Published! Not part of C++17
Concurrency TS v1	future.then(), latches and barriers, atomic smart pointers	Published! Not part of C++17
Concurrency TS v2	TBD. Exploring executors, synchronic types, atomic views, concurrent data structures	Under active development
Networking TS	Sockets library based on Boost.ASIO	Wording review of the spec in progress
Ranges TS	Range-based algorithms and views	Wording review of the spec in progress
Numerics TS	Various numerical facilities	Under active development
Array Extensions TS	Stack arrays whose size is not known at compile time	Withdrawn; any future proposals will target a different vehicle
Modules TS	A component system to supersede the textual header file inclusion model	Initial TS wording reflects Microsoft’s design; changes proposed by Clang implementers expected. Not part of C++17.
Graphics TS	2D drawing API	Design review in progress
Coroutines TS	Resumable functions	Initial TS wording reflects Microsoft’s await design; changes proposed by others expected. Not part of C++17.
Reflection	Code introspection and (later) reification mechanisms	Introspection proposal undergoing design review; likely to target a future TS
Contracts	Preconditions, postconditions, and assertions	Design review in progress. Not part of C++17.

Introduction

Last week I attended a meeting of the ISO C++ Standards Committee (also known as WG21) in Oulu, Finland. This was the second committee meeting in 2016; you can find my reports on previous meetings here (October 2015, Kona) and here (February 2016, Jacksonville), and earlier ones linked from those. These reports, particularly the Jacksonville one, provide useful context for this post.

This meeting was sharply focused on C++17. The goal of the meeting was to publish a Committee Draft (CD) of the C++17 spec. The CD is the first official draft that’s sent out for comment from national standards bodies; that’s followed up by one or more rounds of addressing comments and putting out a new draft, and then eventual publication.

The procedure WG21 follows is that the spec needs to be feature-complete by the time the CD is published, so that national standards bodies have adequate opportunities to comment on the features. That means this was the last meeting to vote new features into C++17, and consequently, all committee subgroups were focused on finishing work on C++17-bound features. (The Evolution and Library Evolution groups did have a chance of review some post-C++17 proposals as well, which I’ll talk about below.)

Work on Technical Specifications (TS’s) continued as well, but was noticeably muted due to the focus on C++17.

C++17

As I mentioned, this meeting was the “feature complete” deadline for C++17, so by now we know what features C++17 will contain. Let’s see what made it:

Features already in C++17 coming into the meeting

See my Jacksonville report for features that were already voted into C++17 coming into the Oulu meeting.

Features voted into C++17 at this meeting

• Language:
  • Using attribute namespaces without repetition
  • Using dynamic memory allocation for over-aligned data
  • Template argument deduction for class templates. This allows you to deduce the template arguments to a class template from the types of the function arguments passed to its constructor. Example:
    
pair p(42, "waldo"); // C++14
pair p(42, "waldo"); // C++17; template arguments deduced

    There is also a provision to write an additional declaration outside the class template definition to guide the deduction, for cases where the class template argument cannot be unambiguously deduced from the constructor arguments.
    
    The feature as originally proposed included a provision for partial deduction, where you explicitly specify some of the template arguments, and leave the rest to be deduced, but this was pulled over concerns that it can be very confusing in some cases:
    
// Would have deduced tuple,
// but tuple is a well-formed type in and of itself!
tuple t(42, "waldo", 2.0f);

  • template
  • Guaranteed copy/move elision
  • std::launder(). (Don’t ask. If you’re not one of the 5 or so people in the world who already know what this is, you don’t want or need to know.)
  • Guarantees on expression evaluation order, with the important modification that in a function call expression, the evaluation of function arguments will be indeterminately sequenced rather than sequenced left-to-right. This modification was made in response to concerns that left-to-right sequencing doesn’t provide much additional benefit, and can impede certain compiler optimizations.
  • Making it easier to use attributes that an implementation doesn’t support
  • if constexpr
  • Wording about forward progress guarantees (two papers).
  • Inline variables, with two modifications since Jacksonville: namespace-scope inline variables are no longer restricted to being const, and constexpr on a namespace-scope variable no longer implies inline. This was mildly controversial, but was adopted nonetheless.
  • Structured bindings, with the syntax changed to use brackets instead of braces. Example:
    
tuple foo();
auto [a, b, c] = foo(); // a, b, and c are bound to the elements of the tuple

  • Selection statements with initializers. (See below for a description).
  • A tweak to the semantics of explicit default constructors
• Library:
  • Most notably: std::variant, a discriminated union type (known as a “sum type” in functional programming languages).
  • Rebasing the library portion of the C++17 spec onto C11. Previously, the spec referred to C99.
  • Tweaking the comparison operators of std::optional and the newly added std::variant to reflect the semantics of the contained object more closely.
  • Making the interfaces of std::optional, std::any, and std::variant more uniform
  • Elementary string conversions
  • Integrating std::string and std::string_view
  • has_unique_object_representations (formerly called is_contiguous_layout)
  • Additional low-level memory management algorithms such as uninitialized_move(). This is mostly of interest to library implementers.
  • Having the emplace() methods of sequence containers return a reference to the newly created objects
  • Removing allocator support from std::function
  • Splicing maps and sets
  • A new customization point for specifying the order of elements in maps and sets
  • shared_ptr::weak_type
  • make_from_tuple()
  • GCD and LCM functions (taken from the Library Fundamentals TS v2)
  • Deprecating vestigial library parts
  • Deleting polymorphic_allocator‘s operator=
  • Fixes for not_fn
  • Relative paths for filesystem, providing a much-requested extension to the functionality brought in by the merging of the Filesystem TS into C++17
  • string_view support in filesystem
  • terminate() if an exception escapes from a parallel algorithm. This is an interim solution until we can figure out a better way of handling this (see the link for some background). (This proposal is also notable for having by far the most amusing title of the bunch: it’s called “Hotel Parallelifornia”, because you can throw exceptions all you want inside a parallel algorithm, but they can never get out :)).
  • Better names for parallel execution policies
  • Discouraging the use of memory_order_consume until we figure out how to improve it
  • A nomenclature tweak to the facilities in . The term “random number generator” was replaced with “random bit generator”, which is a more accurate description.
  • Reserving the namespace std2 for future use. This foreshadows the eventual standardization of a “v2” of the C++ standard library that will not necessarily be backward-compatible with the current version. While we were at it, we also reserved std3 and all subsequent numbers.

Features that didn’t make C++17

In my Jacksonville report, I talked about several features that were proposed for C++17 at that meeting, but didn’t make the cut: Concepts (available instead as a published Technical Specification), Coroutines (targeting instead a work-in-progress Technical Specification), and unified function call syntax (abandoned altogether, at least for now).

At this meeting, a few additional features originally targeting C++17 were axed.

Default Comparisons

I predicted in my last report that default comparisons might turn out to be controversial, and it turned out to be quite so.

Concerns raised about this fell into several broad categories:

• Some felt that automatically generating operator== and operator!= made conceptual sense, but generating operator< and its counterparts (operator>, operator<=, and operator>=) did not.
• Some disliked the opt-out nature of the proposal (that you get the operators by default, and have to opt out if you don’t want them), and expressed a preference for an opt-in proposal (that you don’t get the operators by default, but can opt into them if you’d like). It was pointed out that the last time this preference was polled (at the November 2014 meeting), opt-in had a stronger consensus. (Why did the committee subsequently pursue an opt-out approach, then? I believe it was simply because the opt-out crowd continued to present updated versions of their proposal, while the opt-in crowd let their proposal languish. EWG then expressed consensus for the only proposal on the table – the opt-out one – over nothing at all.)
• Several people in the Library groups expressed concern that they did not have an opportunity to assess the impact on the standard library, and make changes to standard library classes (such as adding opt-out declarations for relevant types) that might be necessary.
• There were concerns that automatic generation of operator< meant that subsequent changes to the order of declaration of class members would change the semantics of the comparison.
• There were concerns that the semantics of the generated operator< and its counterparts, were not a good fit for the low-level model of the language. The details here are a bit sublime (see the proposal wording if you’re interested, particularly around operator<=), but the gist is that the generator operators don’t behave exactly the same as a hand-written function would have in some respects.

As a result of the above concerns, the proposal did not gain consensus to go into C++17.

Most of the objections were to operator< and its counterparts; operator== and operator!= were, by comparison (no pun intended), relatively uncontroversial. The question of just moving forward with the operator== and != parts thus naturally came up. However, this met opposition from some who felt strongly that the two parts of the feature belonged together, and in any case there was no time to formulate a subsetted proposal and assess its implications. The feature thus missed the C++17 train altogether.

I fully expect default comparisons to come back for C++20, though perhaps with aspects of the design space (such as opt-in vs. opt-out) revisited.

Operator Dot

Operator dot also attracted a fair amount of controversy. Among other things, a late-stage alternative proposal which presented some concerns with the original design and offered an alternative, was brought forward.

In the end, though, it wasn’t these concerns that defeated the original proposal. Rather, wording review of the proposal revealed some design issues that the proposal authors hadn’t considered. The issues were related to how operator dot invocations fit into conversion sequences – when are they considered, relative to base-to-derived conversions, other standard conversions, and user-defined conversions. It was too late to hash out a consistent design that answered these questions “on the spot”, so the proposal was withdrawn from consideration for C++17.

I expect this, too, will come back for C++20, with the alternative proposal as possible competition. [Update: I’ve since seen Herb Sutter’s trip report, where he hints that operator dot may in fact come back for C++17, by means of a national body comment on the C++17 Committee Draft asking that it be included. This way of sneakily adding features into a C++ release after the official feature-complete deadline sometimes works, but the bar is high and of course there are no guarantees.]

(It’s worth keeping a mind that there is yet another proposal for operator dot, which was handed to the Reflection Study Group, because it was sufficiently powerful to serve as the basis for a full-blown code reification mechanism. There remains a minority who believe that all other operator dot proposals should be put on hold while we wait for the Reflection Study Group to get to developing that.)

Other Languages Features

• Removing exception specifications. This was not controversial, the Library Working Group just ran out of time before it could finish reviewing the library parts of the wording for this.

Library Features

• joining_thread, a proposed wrapper around std::thread whose destructor joins the thread if it’s still running, rather than terminating the program. A concern was brought forward that use of such a wrapper without sufficient care could lead to deadlocks and other more-subtle-than-termination misbehaviour. As a result, the proposal lost consensus to move forward. I personally don’t view this as a great loss; it’s straightforward for users to write such a wrapper themselves, and its absence in the standard library might prompt a prospective user to do enough research to become conscious of things like the deadlock issue that they otherwise may have overlooked.
• Global swap() accepts unequal allocators. This proved to be controversial during discussions in the library working groups, and was postponed for further study post-C++17.
• Generalize some sorting algorithms to more iterator types. This was postponed as well due to implementers preferring to have a chance to gain more experience with it.

Towards Publishing C++17

I mentioned that the primary goal of the meeting was to publish the Committee Draft or CD of C++17 for comment by national standards bodies. This goal was accomplished: the Committee voted to publish the C++17 Working Draft, with the approved proposals listed above merged into it, as the CD. (An updated Working Draft that includes the merges will be forthcoming within a few weeks of the meeting.)

This means C++17 is on track to be published in 2017! The “train” (referring to the 3-year release train model adopted by the Committee after C++11) is on schedule

What’s the status of other big-ticket features?

The features below weren’t in the limelight at this meeting, since they were previously decided not to target C++17, but as they’re very popular, I’ll give a status update on them all the same.

Concepts

The Concepts TS was published last year, but was not merged into C++17. What does that mean going forward? It means it has a Working Draft, available to accept fixes (and in fact one such fix was voted in at this meeting) and larger changes (one such larger change was already proposed, and more are expected in the near future).

This Working Draft can eventually be published in a form of the Committee’s choosing – could be as a Concepts TS v2, or merged into a future version of the International Standard (such as C++20).

Coroutines

Coroutines are targeting a Technical Specification which is still a work in progress. The initial contents of the TS are set to be the latest draft of Microsoft’s co_await proposal, but there are also efforts to generalize that into a form that can also admit other execution models, such as “stackful” coroutines. EWG looked briefly at this latter proposal, but otherwise no notable progress was made; everyone was busy with C++17.

Modules

Modules is in a similar state to Coroutines: there’s a proposal by Microsoft which is slated to become the initial content of a Modules TS, and there are proposed design changes pending resolution. As with Coroutines, no notable progress was made this week, the focus being on C++17 features.

Reflection

Reflection continues to be actively worked on by the Reflection Study Group, which met this meet to review the design of its chosen static introspection proposal; I talk about this meeting below.

Evolution Working Group

As has become my habit, I sat in the Evolution Working Group (EWG) for the duration of the meeting. This group spends its time evaluating and reviewing the design of proposed language features.

As with all groups, EWG’s priority this meeting was C++17, but, being “ahead in the queue” of other groups (language proposals typically start in EWG and then progress to the Core Working Group (CWG) for wording review, and if appropriate, the Library groups for review of any library changes in them), it also had time to review a respectable number of C++20 features.

I’ll talk about the C++17-track features first. (Being listed here doesn’t mean they made C++17, just that they were considered for it; see above for what made it.) In most cases, these were features previously approved by EWG, which were revisited for specific reasons.

• Default comparisons. EWG looked at this proposal on three different occasions.
  
  First, the authors proposed removing the anti-slicing provisions added last time around (which would have made it ill-formed to slice an object during comparison and initialization), because implementation experience revealed that this broke too much code; this change was approved.
  
  Second, during CWG review of this feature, implementers expressed a preference for not requiring a diagnostic for the rule that says it’s ill-formed for a comparison of the same two types to resolve to a default-generated comparison operator at one point in a translation unit, and to a user-provided comparison at a later point; EWG rejected tis request, expressing a strong preference for requiring a diagnostic. (For what it’s worth, I fall strongly on the “require a diagnostic” side in this and similar situations. I’ve seen it happen too many times where a C++ programmer is bitten by implementation divergence regarding the diagnosis of a rule that the standard proclaims is “ill-formed; no diagnostic required”. I think we should vigorously limit this style of rule to situations where basically all implementations agree that the rule is prohibitively expensive to diagnose, and thus will not in practice diagnose it.)
  
  Finally, EWG looked at a proposal to restrict the default generation to operator== and != only, which became a general discussion of the feature, during which many of the concerns mentioned above were articulated. EWG considered several changes in direction, including dropping the default generation of operator< and counterparts as suggested, and moving to an opt-in approach, but neither of these gained consensus, and the unmodified design was submitted for a vote in plenary (which then failed, as I related above).
• Operator dot came up in EWG twice.
  
  First, the group briefly clarified the name lookup semantics associated with types that have an operator dot. It was stressed that the guiding model for such types is that of smart references, and so reference semantics are emulated when possible. It was also clarified that operator dot does not capture sizeof (that is, sizeof(my_smart_reference) still returns the size of the reference wrapper type, and not of the referred-to type).
  
  Second, CWG sent the feature back to EWG to clarify the intended interaction with other types of conversions. During the resulting discussion, it was discovered that there are entire aspects of the design space that haven’t been nailed down yet in this area (specifically, in where applications of operator dot fit into conversion sequences), and that it’s too broad of an area to nail down on the spot. The proposal was therefore withdrawn for revision until the next meeting (and therefore withdrawn from C++17).
  
  The alternative proposal for operator dot that was newly submitted for consideration at his meeting wasn’t looked at. It wouldn’t have made C++17 even if chosen, so looking at it wasn’t a priority. The authors plans to refine it and hopefully present it at the next meeting.
• EWG briefly looked at an interaction between if constexpr and generic lambdas. The source of the problem is the semantics of default captures for generic lambdas. C++ says that if a lambda has a default capture, then a variable named inside the lambda is covered by the capture only if the the occurrence of the name is an odr-use (this is a term of art that roughly means “a use requiring the variable to have allocated storage and an address”); however, for a generic lambda, whether a use of a name is an odr-use can depend on the arguments you’re calling the lambda with. This leads to an interaction with if constexpr where determining captures can require instantiating a branch of code whose instantiation would otherwise have been disabled by an if constexpr, leading to an undesirable hard error. EWG’s suggestion was to revise the default capture rules for generic lambdas, perhaps to always require named variables to be captured. (Meanwhile, the if constexpr proposal itself isn’t held up over this or anything.)
• Two aspects of the inline variables proposal were revisited.
  
  First, in Jacksonville, a restriction was added that namespace-scope variables could only be inline if they were const; this was meant to discourage mutable global state. It was pointed out that this can be worked around in several ways (for example, by making the variable a reference to a mutable object, or by making it a variable template, which has no such restrictions). Rather than trying to stymie these workaround approaches, EWG decided to just lift the restriction, making the feature simpler.
  
  Second, EWG previously asked that constexpr imply inline. CWG pointed out that for namespace-scope variables, this would change the linkage of existing constexpr variables, breaking existing code. As a result, for namespace-scope variables, constexpr no longer implies inline (for static data members of classes, it still does, because there is no linkage issue there).
  
  A minority continued to oppose this proposal altogether, but they were overruled.
• Template argument deduction for class templates came back to EWG because CWG pointed out a snag. The proposal allowed for “partial deduction”, which meant that you could provide a partial template argument list to a class template when declaring a variable, and have the remaining template arguments be deduced from the constructor arguments. For examle:
  
template struct small_vector { small_vector(T fill); };
small_vector<3> v(42); // N is specified, T is deduced

  However, it was pointed out that in the presence of variadic templates, the “partial” argument list could also form a valid type in and of itself:
  
tuple t("waldo", 42, 2.0f);

  EWG felt it would be surprising for this to deduce tuple given that tuple is also a valid type.
  
  A suggestion was raised to force the caller to provide all template arguments, but use auto for the ones that should be deduced:
  
tuple t("waldo", 42, 2.0f);

  however the details remained to be pinned down, and EWG felt uncomfortable standardizing this approach on the spot, instead preferring to disallow partial deduction. Therefore, the feature is now restrited to the case where no template argument list is provided:
  
  tuple t(std::string("waldo"), 42, 2.0f); // all arguments deduced

• template was briefly revisited to resolve an issue where certain code examples involving deduction of a non-type template parameter from an array bound would become ambiguous. EWG requested that the deduction rules be tweaked to avoid introducing the ambiguity, at the expense of making the already complicated deduction rules even more so; see the paper for details.
• A few design points about structured bindings came up for discussion.
  
  The most prominent issue was the choice of customization points for a non-aggregate user-defined type to opt into the mechanism. The current specification requires using the customization point for “tuple-like” types – that is, specializing std::tuple_size to return the number of elements, std::tuple_element to return the type of each element, and std::get to return a reference to each element. Several people, including the author of this paper, expressed a desire to have a customization point that’s more specific to this feature, and/or one that operated by ADL rather than by specialization. EWG was sympathetic to this, but believed that “tuple-like” should be at least one of the customization points (for consistency with operations like std::tuple_cat()), and that it’s good enough as the only customization point in C++17; the Library Evolution Working Group can then design a nicer customization point in the C++20 timeframe.
  
  The handling of bitfield elements also came up. The proposal specifies that the bound variables act like references to the elements of the structure, but you cannot form a reference to a bitfield. However, the customization point mechanism allows for std::get to return a proxy type that assigns through to the bitfield element; this was deemed a good enough solution until the problem of not being able to form a reference to a bitfield is addressed more comprehensively by the core language.
• A new proposal for if statements with an initializer.
  
  The background to this proposal is that C++ already allows you to declare a variable in the condition of an if statement if the variable’s type is contextually convertible to bool. For example, you can do this:
  
if (Type* foo = GetFoo()) {
...
}

  Here, the body of the if statement will execute if foo is not nullptr.
  The advantage of doing this over declaring foo above the if statement is that the scope of foo is limited to the if statement (including any else block).
  
  However, often you have a type which is not contextually convertible to bool, but you still want to check something about it, and only use it if the check passes. This proposal allows you to do this while keeping the variable’s scope local to the if, by combining a declaration with an expression to test:
  
  if (Type foo = func(); foo.isOk()) {
  ...
  }

  
  EWG really liked this, so much that they made an exception to the previously-agreed rule that new features first proposed at this meeting wouldn’t be considered for C++17. Two small modifications were requested: that the feature work for if constexpr and switch as well.

• Modernizing using-declarations, which allows a pack expansion to occur in a using-declaration. This was another brand new proposal that EWG made an exception for and approved for C++17, but unfortunately CWG did not have time to review the wording, and it thus did not make it in.

Now let’s move on to the C++20-track features. I’ll categorize these into my usual “accepted”, “further work encouraged”, and “rejected” categories:

Accepted proposals:

• Comma omission and comma elision, a proposal to solve a long-standing issue with the preprocessor where it was very difficult to tell apart zero argument from one argument in a variadic macro (because macro arguments are delimited by commas, and zero arguments and one argument both have the property that they have no commas). Importantly, the author has run this by WG14 (the C Standards Committee), and they like the direction and would be open to adopting it in tandem with C++ – an important consideration because WG21 tries to avoid preprocessor divergence between C and C++ where possible.
• A proposal to clean up an obscure corner of the type system, dubbed abominable function types. EWG left the specific cleanup approach up the discretion of the authors and CWG.
• A proposal to define the semantics for certain new parallel execution policies, vector and wavefront policies. EWG sent this to CWG, with the caveat that compiler backend implementers should be present during the CWG review (as they would be involved in the implementation of such policies).
• Designated initializers. This is a C feature that allows you to initialize a structure with a braced-init-list whose elements (“designators”) name the fields, like so:
  
struct Point { int x; int y; };
void foo(Point);
...
foo({ .x = 2, .y = 3});

  The proposal was to adopt this feature into C++, with several important modifications, the most important one being that the order of the designators must match the declaration order of the fields.
  
  This solves, at least partially, one of the longest-standing incompatibilities between C and C++.

Proposals for which further work is encouraged:

• Regular void, a proposal to remove most instances of special-case treatment of void in the language, making it behave like any other type. The general idea enjoyed an increased level of support since its initial presentation two meetings ago, but some details were still contentious, most notably the ability to delete pointers of type void*. The author was encouraged to come back with a revised proposal, and perhaps an implementation to help rule out unexpected complications.
• Return type deduction and SFINAE, which aims to solve the problem that if a function uses C++14 return type deduction, the return expression is no longer in the immediate context of the call, and so errors in the return expression become hard errors rather than substitution failures. (By contrast, in C++11, the return expression would typically appear in the signature as decltype(return-expr), bringing it into the immediate context of the call.) The proposed solution is to extend SFINAE to function bodies in certain contexts, with some important limitations to avoid some of the problems with extending SFINAE to function bodies in general (such as the need to mangle the function’s body into the name).
  
  As expected, implementers were quite wary to extend SFINAE; they were cautiously open to some form of limited extension, but the precise limitations remain to be nailed down. The author was encouraged to continue discussing this with a wide variety of compiler implementers, to arrive at a solution that everyone was on board with.
• A [[visible]] attribute, intended to be a standard version of things like __declspec(dllexport) and __attribute__((visibility("default"))). EWG recognized that there’s a widespread desire for something like this, but also noted that interactions with Modules should be considered, and suggested further exploration of the problem space.
• Product type access. This is a proposed extension to structured bindings that would expose the mechanism by which an aggregate structure type is decomposed into its fields to user code, by having such structures “automagically” implement the customization point used to opt non-aggregate user-defined types into structured binding. EWG agreed that this is desirable, but preferred to wait until the design of that customization point is nailed down in the C++20 timeframe (see above for more discussion of this).
• Overload sets as function arguments, which allows passing an overloaded name (that is, a name that resolves to a function template, or to a set of overloaded functions and/or function templates) as argument for a function parameter whose type is a type template parameter; the compiler “packages” the name into a function object with a call operator whose body dispatches the call to the overload set. You can write such a function object today (along the lines of [](auto... args){ f(args...); }), but this proposal would allow you to just write f instead.
  
  EWG liked the proposal, but recognized an important problem. In the case where f resolves to a single, non-template function, you can pass it as an argument today, and it becomes a pointer to the target function. Changing this (to “package” f into a function object as in the case where it’s an overloaded name) would break existing call sites, so the proposal restricts the “packaging” to the case where it is in fact an overloaded name. But this means that, for a given call site that passes f, whether or not it gets “packaged” depends on how many declarations of f are in scope at the call site. As argued in this paper, this can depend on what headers are included prior to the call site, which can vary from one translation unit to another, making it very easy to get into ODR violations. EWG agreed that to avoid this, a new syntax is necessary: []f will always package the name f into a function object, no matter how many declarations of f are in scope (a plain f, meanwhile, will retain its existing meaning of “function pointer if f is a single non-template function, ill-formed otherwise”).
• Generalized member pointers, which proposes three extensions to C++’s pointers to members: upcasting the member’s owning type (assuming the target of the cast is a non-virtual base); forming pointers to members of members; and forming pointers to bases. EWG liked these extensions, and requested that the author confirm implementability on major compilers, and come back with wording. A fourth extension, to recover a pointer to the object from a pointer to a member of the object, was deferred, to be pursued as a separate proposal. It was pointed out that the first three features could conceivably be extended to pointers to member functions (albeit non-virtual ones) as well.
• Bitfield default member initializers, which proposes a syntax for giving bitfield members a C++11-style default member initializer (something that’s currently forbidden). The challenge here is to find a syntax that combines the bitfield width specifier and the default member initializer in a way that’s not ambiguous. This specific proposal tried to avoid introducing new syntax, instead just introducing disambiguation rules, but EWG felt the proposed rules were unintuitive, and suggested the author try inventing new syntax instead.
• Comparing virtual methods. This is not a proposal, just a statement of a problem, namely that sometimes you want to efficiently check whether a class instance has a particular function in its vtable, but the language doesn’t currently provide a way to do this without incurring the full overhead of a virtual function call. EWG encouraged the author to pursue a solution.
• Lambdas in unevaluated context, which proposes to lift the restriction that lambdas cannot appear inside a decltype(). EWG approved of the intent, but the exact rules need nailing down to ensure that we don’t allow situations that would require mangling a lambda expression into a symbol name.
• Language support for heterogeneous computing. The language changes for which feedback was specifically solicited mostly concerned the use case of a host device sending code in the form of a lambda to be run on a special-purpose device such as a GPU or accelerator. To this end, the standardization of various aspects of the representation of such lambdas was proposed, so that code on the two devices, potentially compiled by different compilers, can interoperate. EWG’s feedback was to leave these representational aspects up to the implementation; the implication is that, if the host code and the device code are in fact compiled by different compilers, those compilers need to agree amongst themselves (much like how linking code from different compilers together requires that they agree on an ABI).
  
  The broader proposal to support heterogeneous computing in standard C++ is based on SYCL; it was suggested that the authors also look at CUDA.
• Pattern matching and language variants is an ambitious proposal to add a first-class pattern matching facility to the language, which can, among other things, deconstruct sum and product types; customization points to opt user-defined sum and product types into pattern matching; and a variant type built into the language (which models a sum type).
  
  EWG had two main pieces of feedback: first, that you should be able to write patterns that match specific derived types when the input is a base type (relevant prior work on that can be found here); and second, that use of the pattern matching facility should not come with a performance penalty.
  
  It’s important to note that the proposed built-in variant type would not supersede std::variant, which was standardized at this meeting. It would have named alternatives, which means it would be to std::variant as structs are to std::tuple.
• Contracts, which provides a way to express preconditions, postconditions, and assertions in the language. The general direction of the proposal was approved at the last meeting; the discussion today mostly concerned details. For example, it was pointed out that specifying the different contract checking modes will be tricky, as there is no precedent in the standard for specifying things that are in practice controlled by compiler flags. Some argued that requiring every declaration of a function to state the pre- and post-conditions is too strict, and having them on the first declaration should be sufficient; in particular, EWG didn’t want contracts to become a de facto part of the type system the way exception specifications have. Finally, some details, such as when postconditions are checked relative to local destructors running, remain to be nailed down.
• An updated version of the proposal to unify stackful and stackless coroutines under a common model and common syntax was presented. The idea is that the new syntax can be lowered to the same low-level “stackless” representation as the co_await syntax, and thus can have the same performance characteristics (for example, avoiding dynamic memory allocation in many cases), but can also be used in a stackful way when the flexibility offered by that (for example, the ability to suspend from inside code that the compiler hasn’t transformed into a coroutine representation) is desired. The machinery that would enable this to work is that the compiler, guided by “suspendable” annotations, would compile a function either normally, or into a coroutine representation (or both), much as how in the Transactional Memory TS, the compiler can create transaction-safe and transaction-unsafe versions of a function. The usual questions of whether such “suspendable” annotations are part of a function’s type, and what the impact of this bifurcation of the set of generated functions on binary size and linking time is, came up. The conclusion was that the area needs more exploration, and implementation experience would be particuarly helpful.
• Parameter packs outside of templates, an early-stage proposal to expand the functionality of parameter packs to act as a general type list facility at compile-time, and as a building block for a tuple of values at runtime. EWG liked the goals and use cases, but expressed a desire for a cleaner syntax (the syntax proposed in the paper has ... tokens sprinkled around fairly liberally). It was also pointed out that the proposal has the potential to bring significant compile-time gains by reducing the amount of metaprogramming required to achieve common tasks, and care should be taken to ensure this potential is realized to its fullest extent.
• File literals, a new type of literal that would name a file, and evaluate to the contents of the file. The main motivating use case is embeddeding external resources (files) into an executable and making it available to the program code in a portable way. Discussion revealed that this feature would be quite a bit more powerful than necessary to achieve this use case, as the contents of the file would be made available for compile-time processing. EWG preferred to avoid this without compelling motivation, and suggested the author formulate the feature as a new type of declaration whose initialization loads the file content at runtime / program load time.

Rejected proposals:

• Comparison operators in fold-expressions, which proposed to disallow fold-expressions of the form a <... b, because their expansion (a_1 < a_2 < ... < a_n < b if a is the pack) is almost certainly not what the user wants (in C++, a < b < c means “compare a and b, and then take the boolean result of that comparison and compare it to c“, not (a < b) && (b < c)). EWG’s view was that we don’t gain much by disallowing them; what we’d really want to do is specify them to do the sensible thing, but we can’t do that without also changing what a < b < c means (because consistency), and it’s too late to change that (because backwards compatibility).
• Proposals to make aggregate types hashable and swappable by default. These were withdrawn by the author, in favour of library solutions building upon the extension mechanism provided by structured bindings.
• An [[exhaustive]] attribute for enumerations, which tells the compiler this is an enumeration whose instances should only take on the value of one of the specified enumerators (as opposed to an enumeration that functions as a bit vector, or an enumeration whose set of values is meant to be extended by the user), and thus guides compiler diagnostics. EWG pointed out that most enumerations are exhaustive, and we want to be annotating the exception, not the norm. However, there wasn’t much appetite for a “non-exhaustive” annotation, either; implementers were of the opinion that current heuristics for diagnostics for switch statements on enumerations are sufficient.
• Making pointers to members callable. This would have allowed a pointer-to-member-function pmem to be called as pmem(obj, args) in addition to the current obj.*pmem(args). There was no consensus for making this change; the opposition was mainly over the fact that this looked like a step towards unified function call syntax, which was rejected at the last meeting (even though it was pointed out that this proposal is rather unlike unified call syntax, as it does not change any lookup ryules).
• Allowing any unsigned integral type as parameter type for literal operators, instead of just unsigned long long. This was rejected, mostly because no one could remember what the original reason was for the restriction, and people were hesitant to move forward without knowing that.
• A new char8_t character type to represent UTF-8 data. EWG was of the opinion that this would be doable, but it would come with some cost (for example, it would be an ABI-breaking change for code that currently typedefs char8_t to be (unsigned) char, similar to how the introduction of the C++11 char16_t and char32_t types was; there is also code in the wild that uses char8_t as a completely different kind of type (e.g. an empty “tag” structure), which would break more severely). The suggested way forward was for the Library Evolution group to decide on a strategy for handling UTF-8 data (perhaps the answer is “just use std::string“, in which case no language changes are needed), and then see what the language can do to help accommodate that design.
• const inheritance, a new form of inheritance that would allow the derived class to access the base class only through a const reference. EWG did not find this change to be sufficiently motivated.
• Partial classes, an attempt to solve the problem of a header file containing a class declaration needing to include declarations of types used in private data members and functions, by splitting out a public interface from a class declaration, and allowing most uses of the class with just the interface declaration being included. EWG found several problems with the mechanics of the proposal, and pointed out that Modules, while not solving this problem directly, should address the ultimate goal (reducing compile times) sufficiently.
• Timing barriers, which attempted to address an interesting problem. In the following code:
  
auto start = std::chrono::high_resolution_clock::now(); // #1
operation_to_be_timed(); // #2
auto end = std::chrono::high_resolution_clock::now(); // #3

  there is nothing that currently prevents the implementation from re-ordering the statements such that e.g. #1 happens after #2, rendering the code useless. The proposal attempted to introduce a mechanism that would prevent such a re-order, but it was pointed out that such a mechanism is basically unimplementable without completely tying the optimizer’s hands. The reason is explained very nicely in this StackOverflow answer, but to summarize: conceptually, clock::now() observes the current time; to make sure that a particular operation X is not re-ordered with clock::now(), that operation needs to be annotated as modifying the current time ; but every operation “modifies the current time” by taking time to run, so every operation would need to be so annotated. That same StackOverflow answer points to some practical ways to solve this problem.
• A different take on unified call syntax, where the unified call semantics is opted into on a per-callsite basis, by changing the syntax of the call site to prefix the function name with a dot (.). EWG felt that having to modify call sites defeated the purpose.

There were a few C++20-track proposals that weren’t looked at because no one was present to champion them, or because there was no time. They’ll be picked up in due course.

Library / Library Evolution Working Groups

As I’ve been sitting in EWG for the week, I didn’t have a chance to follow the library side of things very closely, but I’ll mention some highlights.

First, obviously the Library Working Group (LWG) has been quite busy reviewing the wording for the variety of library proposals voted into C++17 at the end of the meeting. A particularly notable success story is variant being standardized in time for C++17; that proposal has come a long way since two meetings ago, when no one could agree on how to approach certain design issues. (There are still some who are unhappy with the current “valueless by exception” state, and are proposing inventive approaches for avoiding it; however, the consensus was to move forward with the current design.)

A notable library proposal that was pulled from C++17 is joining_thread, for reasons explained above.

The Library Evolution Working Group (LEWG) mostly looked at post-C++17 material (see here for a list of proposals). LEWG’s view on ship vehicles has shifted a bit; until recently, most proposals ended up targetting a Technical Specification of some sort (often one in the Library Fundamentals series), but going forward LEWG plans to target the International Standard directly more often. As a result, a lot of the proposals reviewed at this meeting will likely target C++20.

There are also various library Technical Specifications in flight.

The Library Fundamentals TS v2 has had a draft sent to national standards bodies for comment, and now needs these comments to be addressed prior to final publication. It was originally intended for this process to be completed at this meeting, but with the heavy focus on C++17, it slipped to the next meeting.

LEWG spent a significant amount of time reviewing the Graphics TS, which will provide 2D graphics primitives to C++ programmers. The proposal is progressing towards being sent onwards to LWG, but isn’t quite there yet.

Not much progress has been made on the Ranges TS and the Networking TS, again due to the focus on C++17. LEWG did review a proposal for minor design changes to the Ranges TS, and approved some of them.

Study Groups

Due to the meeting’s emphasis on C++17, most study groups did not meet, but I’ll talk about the few that did.

SG 1 (Concurrency)

SG 1 spent some time reviewing concurrency-related proposals slated for C++17, such as the handling of exceptions in parallel algorithms, as well as advancing its post-C++17 work: the second revisions of the Concurrency and Parallelism Technical Specifications.

The Concurrency TS v2 is slated to contain, among other things, support for floating-point atomics, atomic_view, a synchronized value abstraction, and emplace() for promise and future.

The Parallelism TS v2 is expected to contain a low-level library vector facility (datapar), as well as more forward-looking things (see proposals tagged “Concurrency” here).

SG 7 (Reflection)

SG 7 met for an evening session to provide design feedback on an updated version of the proposal that was chosen as the preferred approach for static code introspection at the last meeting.

The discussion focused most heavily on a particular design choice made by the proposal: the ability to reflect typedefs as separate from their underlying types (so for example, the ability for the reflection facility to distinguish between a use of std::size_t and a use of unsigned long, even if the former is a typedef for the latter). The issue is that the language does not treat typedefs as semantic entities in their own right, and a reflection facility pretending t

https://danielpocock.com/avoiding-sms-vendor-lock-in-with-smpp

No global RMLL this year ? True ! However, the Security Track will be held at Paris Mozilla office next July. Come and enjoy listening...

https://air.mozilla.org/rmll-2016-the-security-track-4/

Once a month web developers across the Mozilla community get together (in person and virtually) to share what cool stuff we've been working on. This...

https://air.mozilla.org/webdev-extravaganza-july-2016/

When translating a string, Pontoon displays all existing translations to other languages in the Locales tab. This is helpful for localizers that understand other languages than English and their native language.

However, the list can get pretty long sometimes, making it hard and time-consuming to find translations one can actually understand. To overcome this problem, localizers can now select preferred locales in the settings, which are shown on top of the list.

Translation counts from preferred and other locales are displated separately in the Locales tab header, making it clear if translations from your preferred list are available before even openning the tab.

Get involved

This feature has been developed by our volunteer contributor jotes. Would you like to get involved, too? Start here!

http://horv.at/blog/define-preferred-locales-in-pontoon/

Web literacy — the ability to read, write, and participate online — is one of the most important skills of the 21st century. We believe it should be enshrined as the fourth “R,” alongside Reading, Writing and Arithmetic. From our open source learning tools to our free educational curriculum, we are dedicated to empowering individuals by teaching Web literacy.

In 2015, 65% of California public schools offered no computer science courses at all. Public schools should do more to expose students to Web literacy: a paucity of funding and the elimination of digital skills classes and curriculum are a disservice to students and the state’s future.

On June 30, we submitted an amicus letter to the California Supreme Court urging review of the case Campaign for Quality Education v. State of California. The issue in this case is whether the California Constitution requires California to provide its public school students with a quality education. We wrote this letter because we believe that California students risk being left behind in our increasingly digitized society without a quality education that includes Web literacy skills.

The internet has become an integral part of life for many and continues to grow in its global impact. Empowering a Web literate generation of students is crucial to their success, that of the broader online ecosystem and to our local economy. Using technology is an important part of Web literacy but it is not all. Web literacy needs to include a deeper understanding of technology itself and of its impact to empower students and allow meaningful participation in their online lives.

We hope the California Supreme Court takes this case. Ultimately, a quality education, including Web literacy, unlocks opportunities for students to be more engaged citizens as well as making them more college and career ready in today’s technology-driven market.

https://blog.mozilla.org/blog/2016/07/05/public-education-and-web-literacy-in-california/

Reuni'on bi-semanal para hablar sobre el estado de Mozilla, la comunidad y sus proyectos. Bi-weekly meeting to talk (in Spanish) about Mozilla status, community and...

https://air.mozilla.org/martes-mozilleros-20160705/

Congratulations to NASA's Juno space probe team, which is now successfully in orbit around Jupiter. When you want radiation-hardened CPU designs (to guard against naughty things like cosmic rays flipping bits and zapping traces), you want BAE's RAD series, and that means you want ... PowerPC. Yes, Juno's brain is a lowly RAD750 running at 200MHz, essentially a beige Power Mac G3 with cojones of pure lead. It has 128MB of the baddest, meanest, toughest DRAM there is and 256MB of flash on board with a system bus providing 100Mbps of instrument bandwidth for pure awesome. The radiation around Jupiter, due largely to its intense magnetic fields acting essentially like gigantic particle accelerators, will expose the probe to the equivalent of a human getting 100 million dental X-rays in the course of a year (for me, that year was 6th grade); to enable it to survive its two year mission, NASA has encased the CPU and other critical components in a titanium box a third of an inch thick.

Remember: when you absolutely have to get to Jupiter, choose PowerPC. (Heck, Mars too.)

http://tenfourfox.blogspot.com/2016/07/did-juno-is-powerpc.html