In my previous post I talked a bit about why the RWlock pattern is important for accessing data, which is why Rust enforces this pattern either at compile time or runtime depending on the abstractions used.

It occurred to me that there are many such abstractions in Rust, each with their unique guarantees. The programmer once again has the choice between runtime and compile time enforcement. It occurred to me that this plethora of “wrapper types”¹ could be daunting to newcomers; in this post I intend to give a thorough explanation of what some prominent ones do and when they should be used.

I’m assuming the reader knows about ownership and borrowing in Rust. Nevertheless, I will attempt to keep the majority of this post accessible to those not yet familiar with these concepts. Aside from the two links into the book above, these two blog posts cover the topic in depth.

Basic pointer types

Box

Box is an “owned pointer” or a “box”. While it can hand out borrowed references to the data, it is the only owner of the data. In particular, when something like the following occurs:

let x = Box::new(1);
let y = x;
// x no longer accessible here

Here, the box was moved into y. As x no longer owns it, the compiler will no longer allow the programmer to use x after this. A box can similarly be moved out of a function by returning, and when a box (one which hasn’t been moved) goes out of scope, destructors are run, deallocating the inner data.

This abstraction is a low cost abstraction for dynamic allocation. If you want to allocate some memory on the heap and safely pass a pointer to that memory around, this is ideal. Note that you will only be allowed to share borrowed references to this by the regular borrowing rules, checked at compile time.

Interlude: Copy

Move/ownership semantics are not special to Box; it is a feature of all types which are not Copy.

A Copy type is one where all the data it logically encompasses (usually, owns) is part of its stack representation². Most types containing pointers to other data are not Copy, since there is additional data elsewhere, and simply copying the stack representation may accidentally share ownership of that data in an unsafe manner.

Types like Vec and String which also have data on the heap are also not Copy. Types like the integer/boolean types are Copy

&T and raw pointers are Copy. Even though they do point to further data, they do not “own” that data. Whereas Box can be thought of as “some data which happens to be dynamically allocated”, &T is thought of as “a borrowing reference to some data”. Even though both are pointers, only the first is considered to be “data”. Hence, a copy of the first should involve a copy of the data (which is not part of its stack representation), but a copy of the second only needs a copy of the reference. &mut T is not Copy because mutable aliases cannot be shared, and &mut T “owns” the data it points to somewhat since it can mutate.

Practically speaking, a type can be Copy if a copy of its stack representation doesn’t violate memory safety.

&T and &mut T

These are immutable and mutable references respectively. They follow the “read-write lock” pattern described in my previous post, such that one may either have only one mutable reference to some data, or any number of immutable ones, but not both. This guarantee is enforced at compile time, and has no visible cost at runtime. In most cases such pointers suffice for sharing cheap references between sections of code.

These pointers cannot be copied in such a way that they outlive the lifetime associated with them.

*const T and *mut T

These are C-like raw pointers with no lifetime or ownership attached to them. They just point to some location in memory with no other restrictions. The only guarantee that these provide is that they cannot be dereferenced except in code marked unsafe.

These are useful when building safe, low cost abstractions like Vec, but should be avoided in safe code.

Rc

This is the first wrapper we will cover that has a runtime cost.

Rc is a reference counted pointer. In other words, this lets us have multiple “owning” pointers to the same data, and the data will be freed (destructors will be run) when all pointers are out of scope.

Internally, it contains a shared “reference count”, which is incremented each time the Rc is cloned, and decremented each time one of the Rcs goes out of scope. The main responsibility of Rc is to ensure that destructors are called for shared data.

The internal data here is immutable, and if a cycle of references is created, the data will be leaked. If we want data that doesn’t leak when there are cycles, we need a garbage collector. I do not know of any existing GCs in Rust, but I am working on one with Michael Layzell and there’s another cycle collecting one being written by Nick Fitzgerald.

Guarantees

The main guarantee provided here is that the data will not be destroyed until all references to it are out of scope.

This should be used when you wish to dynamically allocate and share some data (read-only) between various portions of your program, where it is not certain which portion will finish using the pointer last. It’s a viable alternative to &T when &T is either impossible to statically check for correctness, or creates extremely unergonomic code where the programmer does not wish to spend the development cost of working with.

This pointer is not thread safe, and Rust will not let it be sent or shared with other threads. This lets one avoid the cost of atomics in situations where they are unnecessary.

There is a sister smart pointer to this one, Weak. This is a non-owning, but also non-borrowed, smart pointer. It is also similar to &T, but it is not restricted in lifetime — a Weak can be held on to forever. However, it is possible that an attempt to access the inner data may fail and return None, since this can outlive the owned Rcs. This is useful for when one wants cyclic data structures and other things.

Cost

As far as memory goes, Rc is a single allocation, though it will allocate two extra words as compared to a regular Box (for “strong” and “weak” refcounts).

Rc has the computational cost of incrementing/decrementing the refcount whenever it is cloned or goes out of scope respectively. Note that a clone will not do a deep copy, rather it will simply increment the inner reference count and return a copy of the Rc

Cell types

“Cells” provide interior mutability. In other words, they contain data which can be manipulated even if the type cannot be obtained in a mutable form (for example, when it is behind an &-ptr or Rc).

The documentation for the cell module has a pretty good explanation for these.

These types are generally found in struct fields, but they may be found elsewhere too.

Cell

Cell is a type that provides zero-cost interior mutability, but only for Copy types. Since the compiler knows that all the data owned by the contained value is on the stack, there’s no worry of leaking any data behind references (or worse!) by simply replacing the data.

It is still possible to violate your own invariants using this wrapper, so be careful when using it. If a field is wrapped in Cell, it’s a nice indicator that the chunk of data is mutable and may not stay the same between the time you first read it and when you intend to use it.

let x = Cell::new(1);
let y = &x;
let z = &x;
x.set(2);
y.set(3);
z.set(4);
println!("{}", x.get());

Note that here we were able to mutate the same value from various immutable references.

This has the same runtime cost as the following:

let mut x = 1;
let y = &mut x;
let z = &mut x;
x = 2;
*y = 3;
*z = 4;
println!("{}", x;

but it has the added benefit of actually compiling successfully.

Guarantees

This relaxes the “no aliasing with mutability” restriction in places where it’s unnecessary. However, this also relaxes the guarantees that the restriction provides; so if one’s invariants depend on data stored within Cell, one should be careful.

This is useful for mutating primitives and other Copy types when there is no easy way of doing it in line with the static rules of & and &mut.

G'abor Lehel summed up the guarantees provided by Cell in a rather succinct manner:

The basic guarantee we need to ensure is that interior references can’t be invalidated (left dangling) by mutation of the outer structure. (Think about references to the interiors of types like Option, Box, Vec, etc.) &, &mut, and Cell each make a different tradeoff here. & allows shared interior references but forbids mutation; &mut allows mutation xor interior references but not sharing; Cell allows shared mutability but not interior references.

[because Cell operates by copying, it can’t create interior references]

This comment by Eddy also touches on the guarantees of Cell and the alternatives

Cost

There is no runtime cost to using Cell, however if one is using it to wrap larger (Copy) structs, it might be worthwhile to instead wrap individual fields in Cell since each write is a full copy of the struct.

RefCell

RefCell also provides interior mutability, but isn’t restricted to Copy types.

Instead, it has a runtime cost. RefCell enforces the RWLock pattern at runtime (it’s like a single-threaded mutex), unlike &T/&mut T which do so at compile time. This is done by the borrow() and borrow_mut() functions, which modify an internal reference count and return smart pointers which can be dereferenced immutably and mutably respectively. The refcount is restored when the smart pointers go out of scope. With this system, we can dynamically ensure that there are never any other borrows active when a mutable borrow is active. If the programmer attempts to make such a borrow, the thread will panic.

let x = RefCell::new(vec![1,2,3,4]);
{
    println!("{:?}", *x.borrow())
}

{
    let my_ref = x.borrow_mut();
    my_ref.push(1);
}

Similar to Cell, this is mainly useful for situations where it’s hard or impossible to satisfy the borrow checker. Generally one knows that such mutations won’t happen in a nested form, but it’s good to check.

For large, complicated programs, it becomes useful to put some things in RefCells to make things simpler. For example, a lot of the maps in the ctxt struct in the rust compiler internals are inside this wrapper. These are only modified once (during creation, which is not right after initialization) or a couple of times in well-separated places. However, since this struct is pervasively used everywhere, juggling mutable and immutable pointers would be hard (perhaps impossible) and probably form a soup of &-ptrs which would be hard to extend. On the other hand, the RefCell provides a cheap (not zero-cost) way of safely accessing these. In the future, if someone adds some code that attempts to modify the cell when it’s already borrowed, it will cause a (usually deterministic) panic which can be traced back to the offending borrow.

Similarly, in Servo’s DOM we have a lot of mutation, most of which is local to a DOM type, but some of which crisscrosses the DOM and modifies various things. Using RefCell and Cell to guard all mutation lets us avoid worrying about mutability everywhere, and it simultaneously highlights the places where mutation is actually happening.

Note that RefCell should be avoided if a mostly simple solution is possible with & pointers.

Guarantees

RefCell relaxes the static restrictions preventing aliased mutation, and replaces them with dynamic ones. As such the guarantees have not changed.

Cost

RefCell does not allocate, but it contains an additional “borrow state” indicator (one word in size) along with the data.

At runtime each borrow causes a modification/check of the refcount.

Synchronous types

Many of the types above cannot be used in a threadsafe manner. Particularly, Rc and RefCell, which both use non-atomic ref counts, cannot be used this way. This makes them cheaper to use, but one needs thread safe versions of these too. They exist, in the form of Arc and Mutex/RWLock

Note that the non-threadsafe types cannot be sent between threads, and this is checked at compile time. I’ll touch on how this is done in a later blog post.

There are many useful wrappers for concurrent programming in the sync module, but I’m only going to cover the major ones.

Arc

Arc is just a version of Rc that uses an atomic reference count (hence, “Arc”). This can be sent freely between threads.

C++’s shared_ptr is similar to Arc, however in C++s case the inner data is always mutable. For semantics similar to that from C++, we should use Arc>, Arc>, or Arc>³ (UnsafeCell is a cell type that can be used to hold any data and has no runtime cost, but accessing it requires unsafe blocks). The last one should only be used if one is certain that the usage won’t cause any memory unsafety. Remember that writing to a struct is not an atomic operation, and many functions like vec.push() can reallocate internally and cause unsafe behavior (so even monotonicity may not be enough to justify UnsafeCell)

Guarantees

Like Rc, this provides the (thread safe) guarantee that the destructor for the internal data will be run when the last Arc goes out of scope (barring any cycles).

Cost

This has the added cost of using atomics for changing the refcount (which will happen whenever it is cloned or goes out of scope). When sharing data from an Arc in a single thread, it is preferable to share & pointers whenever possible.

Mutex and RwLock

Mutex and RwLock provide mutual-exclusion via RAII guards. For both of these, the mutex is opaque until one calls lock() on it, at which point the thread will block until a lock can be acquired, and then a guard will be returned. This guard can be used to access the inner data (mutably), and the lock will be released when the guard goes out of scope.

{
    let guard = mutex.lock();
    // guard dereferences mutably to the inner type
    *guard += 1;
} // lock released when destructor runs

RwLock has the added benefit of being efficient for multiple reads. It is always safe to have multiple readers to shared data as long as there are no writers; and RwLock lets readers acquire a “read lock”. Such locks can be acquired concurrently and are kept track of via a reference count. Writers must obtain a “write lock” which can only be obtained when all readers have gone out of scope.

Guarantees

Both of these provide safe shared mutability across threads, however they are prone to deadlocks. Some level of additional protocol safety can be obtained via the type system. An example of this is rust-sessions, an experimental library which uses session types for protocol safety.

Costs

These use internal atomic-like types to maintain the locks, and these are similar pretty costly (they can block all memory reads across processors till they’re done). Waiting on these locks can also be slow when there’s a lot of concurrent access happening.

Composition

A common gripe when reading Rust code is with stuff like Rc>> and more complicated compositions of such types.

Usually, it’s a case of composing together the guarantees that one needs, without paying for stuff that is unnecessary.

For example, Rc> is one such composition. Rc itself can’t be dereferenced mutably; because Rc provides sharing and shared mutability isn’t good, so we put RefCell inside to get dynamically verified shared mutability. Now we have shared mutable data, but it’s shared in a way that there can only be one mutator (and no readers) or multiple readers.

Now, we can take this a step further, and have Rc>> or Rc>>. These are both shareable, mutable vectors, but they’re not the same.

With the former, the RefCell is wrapping the Vec, so the Vec in its entirety is mutable. At the same time, there can only be one mutable borrow of the whole Vec at a given time. This means that your code cannot simultaneously work on different elements of the vector from different Rc handles. However, we are able to push and pop from the Vec at will. This is similar to an &mut Vec with the borrow checking done at runtime.

With the latter, the borrowing is of individual elements, but the overall vector is immutable. Thus, we can independently borrow separate elements, but we cannot push or pop from the vector. This is similar to an &mut [T]⁴, but, again, the borrow checking is at runtime.

In concurrent programs, we have a similar situation with Arc>, which provides shared mutability and ownership.

When reading code that uses these, go in step by step and look at the guarantees/costs provided.

When choosing a composed type, we must do the reverse; figure out which guarantees we want, and at which point of the composition we need them. For example, if there is a choice between Vec> and RefCell>, we should figure out the tradeoffs as done above and pick one.

Discuss: HN, Reddit

http://manishearth.github.io/blog/2015/05/27/wrapper-types-in-rust-choosing-your-guarantees/

If you have an active extension listing on AMO you probably got a message from us already, explaining how we will automatically sign your add-on and provide it to your users via automatic updates. The automatic signing process will run this week, in batches, and we will notify you when your add-on is signed. Please take some time to test the signed version in the current release version of Firefox and either Developer Edition or Nightly (where Firefox already warns about unsigned extensions).

If you’re unfamiliar with extension signing, please read the original announcement for context.

Next week, we will activate two new features on AMO: signing of new add-on versions after they are reviewed, and add-on submission for developers who wish to have their add-ons signed but don’t want them listed on AMO. We will post another update once this happens. When this is done, all extension developers will be able to have their extensions signed, with enough time to update their users before signing becomes a requirement in release versions of Firefox.

New Developer Agreement

Since we will be signing add-ons that won’t be listed on AMO, we have updated the Add-on Distribution Developer Agreement to cover the new ways in which we will handle add-ons. This document hadn’t been touched for years, so we took our time and significantly updated its contents to reflect how we do things now. Hopefully it is also easier to read than its previous version.

Note that the new agreement will go into effect on June 1st. The version that is displayed on AMO when you submit a new add-on will be updated then, and all active developers on AMO will be notified about it.

If you want to stay up to date with changes related to extension signing, you can follow this blog or check in regularly to the wiki page, where I update the timeline information as it becomes clearer.

https://blog.mozilla.org/addons/2015/05/27/update-signing-new-developer-agreement/

Help define, plan, design, and implement Bugzilla's future!

https://air.mozilla.org/bugzilla-development-meeting-20150527/

What is a browser for? What should it do, or not do? What should it be?

People within the Mozilla project have been recently discussing the user value of some new features in Firefox. I think a person’s view of questions of this nature will depend on their view of the role of the browser. One option is the “featureless window on the web” view – the browser is nothing, the site is everything. But as one participant said, this leads to all the value-add and features being provided by the sites, which is not a recipe for user control, or for using the browser to advance the Mozilla mission.

I think the best vision for Firefox is as your “Internet butler” – quiet and refined, highly capable, provides what you need even before you know you need it, who gently guides you out of trouble but generally does his thing without you needing to think about him or provide explicit direction or management.

So I’d like to propose the “Jeeves Test” for evaluating feature proposals for Firefox. It works like this: imagine Bertie Wooster, relaxing in an armchair in his apartment, with a cigarette, a gin and tonic, and a tablet computer. Then take the user value proposition of an idea, write it in appropriately deferential language, and see if you can imagine Jeeves whispering it into his ear. If you can’t, perhaps it’s not something we want to do.

To make that a bit more concrete, here are some examples of things that might pass: “Here’s an English translation of this Serbian page, sir”, or “For your safety, sir, access to this malware page has been blocked”. And here are some which might not: “For your convenience, sir, I’ve exempted aol.com from your popup blocker”, or “You’ll be pleased to find, sir, that the user interface has been substantially rearranged”.

There may be occasions where we’d want to do something which doesn’t obviously pass the Jeeves Test, if the effects on the broader web ecosystem of making the change are significantly positive. Some of the things we do to improve web security but which have a short-term compatibility impact might fall into that category. “Let me ensure this site doesn’t load for you, sir” generally doesn’t go down well, after all. But in those cases, that longer-term or broader value has to be clearly articulated – before we make the change – if we are to avoid an exasperated “Dash it, Jeeves… why?” from our userbase.

http://feedproxy.google.com/~r/HackingForChrist/~3/uy-LPl8GZAk/

A small update for next week release. We took a patch to fix a JS top crash and 3 changes to fix an important regression in WebRTC.

• 6 changesets
• 10 files changed
• 85 insertions
• 35 deletions

Extension	Occurrences
cpp	5
xml	1
txt	1
mn	1
html	1
h	1

Module	Occurrences
js	5
mobile	1
media	1
dom	1
config	1
browser	1

List of changesets:

tbirdbld	Automated checkin: version bump for thunderbird 38.0b6 release. DONTBUILD CLOSED TREE a=release - 7eb2d1522ac2
Margaret Leibovic	Bug 1166392 - Include about:reader strings on Android. r=mfinkle, a=sledru - a468d1edeac8
Tooru Fujisawa	Bug 1159973 - Abort parsing when TokenStream::SourceCoords hits OOM. r=jorendorff, a=sylvestre - 414d430012ee
Paul Adenot	Bug 1166183 - Back out the direct listener removal landed by mistake in Bug 1141781. r=jesup, a=sledru - 9525d8723413
Paul Adenot	Bug 1166183 - Work around Bug 934512 in track_peerConnection_replaceTrack.html. r=pehrson, a=sledru - 6ee39f753d19
Andreas Pehrson	Bug 1166183 - Reset PipelineListener's flag after ReplaceTrack(). r=bwc, a=sledru - 19b9ffda6e2f

http://release.mozilla.org/statistics/38_0/2015/05/27/fx-38.0.5rcto-rc2.html

the following changes have been pushed to bugzilla.mozilla.org:

• [1168357] Unable to reset mentor in modal bug view
• [1146782] migrate autocomplete from yui to jquery
• [1165355] sanitizeme.pl script should remove webservice api keys from database to remove sensitive information
• [1158010] provide a standard and simple way to render relative dates, in perl and javascript
• [1168303] send-request-nags.pl is triggering oom on bugzillaadm.private.scl3
• [1162854] hitting back after making an edit should keep edit mode open
• [1164850] add preferences to request nagging watching facility (reviews only, extended period, and skip encryption)
• [1164780] updating an attachment with pre-existing review requests shouldn’t enforce the “last logged in to bugzilla” limit
• [1162427] the “reset assignee to default” checkbox / functionality should be visible

discuss these changes on mozilla.tools.bmo.

Filed under: bmo, mozilla

https://globau.wordpress.com/2015/05/27/happy-bmo-push-day-143/

There will be more stuff over time in the cURL project. Exactly which stuff and how long time it takes for everything, we don’t know. It depends largely on who works on what and how much time said persons can spend on implementing the stuff they work on…

I suspect we might be able to do things slightly faster over time, which is why the red arrow isn’t just a straight line.

I drew this little picture inspired from discussions with friends after a talk I did about curl and how development works in an open source project such as this. We know we will work on things that will improve the products but we don’t see exactly what very far in advance. I tweeted this picture a few days ago, and it turned out very popular.

http://daniel.haxx.se/blog/2015/05/26/picturing-curls-future/

There’s no TIDE podcast this week, so I thought I’d record a blog post today. Here’s the abstract:

We’re at peak centralisation of our data in online services, with data as the new oil. It’s a time of ‘frictionless sharing’, but also a time when we’re increasingly having decisions made on our behalf by algorithms. Education is now subject to a land grab by ‘software with shareholders’ looking to profit from collecting, mining, packaging, and selling learner data. This article explores some of the issues at stake, as well as pointing towards the seeds of a potential solution.

The Code Acts in Education blog I mention in the introduction to this piece can be found here and Ben Williamson is @BenPatrickWill on Twitter.

Comments (once you’ve listened!) much appreciated. I’ve still got time to re-work this…

(no audio? click here!)

References

Belshaw, D.A.J. (2014a). Software with shareholders (or, the menace of private public spaces). Doug Belshaw’s blog. 23 April 2014. http://dougbelshaw.com/blog/2014/04/23/software-with-shareholders.

Belshaw, D.A.J. (2014b). Curate or Be Curated: Why Our Information Environment is Crucial to a Flourishing Democracy, Civil Society. DMLcentral. 23 October 2014. http://dmlcentral.net/blog/doug-belshaw/curate-or-be-curated-why-our-information-environment-crucial-flourishing-democracy.

Dixon-Thayer, D. (2015). Mozilla View on Zero-Rating. Open Policy & Advocacy Blog. Mozilla. 5 May 2015. https://blog.mozilla.org/netpolicy/2015/05/05/mozilla-view-on-zero-rating.

Flew, T. (2008). New Media: An Introduction (3rd ed.). Melbourne: Oxford University Press.

Gillula, J. & Malcolm, J. (2015). Internet.org Is Not Neutral, Not Secure, and Not the Internet. Deeplinks Blog. Electronic Frontier Foundation. 18 May 2015. https://www.eff.org/deeplinks/2015/05/internetorg-not-neutral-not-secure-and-not-internet.

Kramer, A.D.I., Guillory, J.E., Hancock, J.T. (2014) Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences of the United Sates of America. 111(24).

McNeal, G.S. (2014). Facebook Manipulated User News Feeds To Create Emotional Responses. Forbes. 28 June 2014. http://www.forbes.com/sites/gregorymcneal/2014/06/28/facebook-manipulated-user-news-feeds-to-create-emotional-contagion

Mozilla. (2015). Web Literacy Map v1.1. https://teach.mozilla.org/teach-like-mozilla/web-literacy

Thorp, J. (2012). Big Data Is Not the New Oil. Harvard Business Review. 30 November 2012. https://hbr.org/2012/11/data-humans-and-the-new-oil.

Image CC BY-NC Graham Chastney

http://dougbelshaw.com/blog/2015/05/26/peak-centralisation/

An overview of the progress we've made in the last two years development on the SpiderMonkey GC implementing generational collection and starting work on compacting.

https://air.mozilla.org/spidermonkey-garbage-collection-update/

http://atlee.ca/blog/posts/releng-2015-1.html

As web developer, debugging Service Workers code can be a complex process. For this reason, we are putting extra effort in improving debugging tools for Firefox. This blog post is focusing on our developments on the Console API and how it can be used in any kind of Worker and shown up in the DevTools WebConsole.

As you probably know, since February 2014 Firefox supports the Console API in Workers (bug 965860). In so doing, we have recently extended the support to different kind of Workers (January 2015 - bug 1058644). However, any use of the Console API in those Workers has not been shown yet into the DevTools WebConsole.

I am pleased to say that, in the nightly, and soon in release builds, you are now able to choose and see the log messages from Shared Workers, Service Workers and Addon-on Workers, which enable them from the ‘logging’ menu. Here a screenshot:

This works with remote debugging too! Happy debugging!

http://143th.net/post/119952141221

Using Firefox OS to test your apps for better accessibility and usability of your mobile web application

https://air.mozilla.org/reaching-more-users-through-better-accessibility/

Supporting privacy on the mobile web with built-in features and add-ons

https://air.mozilla.org/privacy-features-for-firefox-for-android-2/

Reuni'on bi-semanal para hablar sobre el estado de Mozilla, la comunidad y sus proyectos. Bi-weekly meeting to talk (in Spanish) about Mozilla status, community and...

https://air.mozilla.org/martes-mozilleros-20150526/

A beginner's guide to contributing to Servo

https://air.mozilla.org/servo-the-parallel-web-browser-and-you/

Silex Labs

https://air.mozilla.org/silexlabs-test/

My full 30 page document with all details and analyses of the curl user poll 2015 is now available. It shows details of all the questions, most of them with a comparison with last year’s survey. The write-ins are also full of good advice, wisdom and some signs of ignorance or unawareness.

I hope all curl hackers and others generally interested in the project can use my “report” to learn something about our users and our user’s view of the project and our products.

Let’s use this to guide us going forward.

http://daniel.haxx.se/blog/2015/05/26/2015-curl-user-poll-analysis/

the following changes have been pushed to bugzilla.mozilla.org:

• [1166280] max-width missing from bottom-actions, causing “top” and “new bug” buttons misalignment on the right edge of window
• [1163868] Include requests from others in RequestNagger
• [1164321] fix chrome issues
• [1166616] disabling of reviewer last-seen doesn’t work for users that don’t have a last_seen_date
• [1166777] Script for generating API keys
• [1166623] move MAX_REVIEWER_LAST_SEEN_DAYS_AGO from a constant to a parameter
• [1135164] display a warning on show_bug when an unassigned bug has a patch attached
• [1158513] Script to convert MozReview attachments from parents to children
• [1168190] “See Also” should accept webcompat.com/issues/ – prefixed URLs

discuss these changes on mozilla.tools.bmo.

Filed under: bmo, mozilla

https://globau.wordpress.com/2015/05/26/happy-bmo-push-day-142/

A couple of years ago, I was part of a panel with the remarkable Evgeny Morozov, at an event hosted by the Museum of Vancouver.  The event wasn’t remarkable, except that I ended up in a somewhat high-energy debate with Mr. Morozov, without particularly understanding why.  He ascribed to me beliefs I didn’t hold, which made a rigorous intellectual argument (which I generally enjoy) quite elusive and baffling.

It took a follow-up discussion with my political theorist brother to elucidate it.  In particular, he helped me realize that while for decades I’ve described myself as a technologist, depending on your definition of that word, I was misrepresenting myself.  Starting with my first job in my mid teens, my career (modulo an academic side-trip) has essentially been to bring technological skills to bear on a situation.  Using the word technologist to describe myself felt right, without much analysis.  As my brother pointed out, using an academic lexicon, the word technologist can be used to mean “someone who thinks technology is the answer”.  Using that definition, capital is to capitalist as technology is to technologist.  Once I understood that, I realized why Mr. Morozov was arguing with me.  He thought I believed in technologism, much like the way the New Yorker describes Marc Andreesen (I’m no Andreesen in many ways, but I do think he’d have been a better foil for Morozov).  It also made me realize that he either didn’t understand what Mozilla is trying to do, or didn’t believe me/us.

Using that definition, I am not a technologist any more than I am a capitalist or a regulator[ist?].  I resist absolutes, as it seems to me blatantly obvious (and probably boring) that except for rhetorical/marketing purposes (such as Morozov’s), the choices which one should make when building or changing a system, at every scale from the individual to the society, must be thoughtfully balanced.  This balance is hard to articulate and even harder to maintain, in a political environment which blurs subtlety and a network environment which amplifies differences.

 

 

It is in this balance that lies evolutionary progress.  I have been re-re-watching The Wire, thinking about Baltimore.  In season 3, one of the district commanders, sick of the status quo, tries to create a “free zone” for unimpeded drug dealing, an Amsterdam-in-Baltimore, in order to draw many of the associated social ills away from the rest of the district.  In doing this, he uses his authority (and fearlessness, thanks to his impending retirement), and combines management techniques, marketing, and positive and negative incentives, to try fix a systemic social problem.  I want more stories like that (especially less fictional ones).

I don’t believe that technology is “the answer,” just like police vans aren’t the answer.  That said, ignoring the impact that technology can have on societal (or business!) challenges is just as silly as saying that technology holds the answer.  Yes, technology choices are often political, and that’s just fine, especially if we invite non-techies to help make those political decisions.  When it comes to driving behavioral change, tech is a tool like many others, from empathy and consumer marketing to capital and incentives.  Let’s understand and use all of these tools.

Human psychology explains why marketing works, and marketing has definitely been used for silly reasons in the past. Still, if part of your goal involves getting people to do something, refusing to use human psychology to further your well-intentioned outcome is cutting your nose to spite your face.  Similarly, it’s unarguable that those who wield gigantic capital have had outsize influence in our society, and some of them are sociopaths. Still, refusing to consider how one could use capital and incentives to drive behavior is an equally flawed limit.  Lastly, obviously, technology can be dehumanizing. But technology can also, if wielded thoughtfully, be liberating.  Each has power, and power isn’t intrinsically evil.

1+1+1 = magic

Tech companies of the last decade have shown the outsize impact that these three disciplines, when wielded in coordinated fashion, can have on the world.  Uber’s success isn’t due to smart deployment of tech, psychology or capital.  Uber’s success is due to a brilliant combination of all three (and more).  I’ll leave it history to figure out whether Uber’s impact will be net positive or negative based on metrics that I care about, but the magical quality that combination unleashed is awe inspiring.

 

 

This effective combination of disciplines is a critical step which I think eludes many people and organizations, due to specialization without coordination.  It is relatively clear how to become a better technologist or marketer.  The ladder of capitalist success is equally straightforward.  But figuring out how to blend those disciplines and unlock magic is elusive, both within a career path and within organizations.

I wonder if it’s possible to label this pursuit, in a gang-signal sort of way. Venture capitalists, marketers, and technologists all gain a lot by simply affiliating with these fields.  I find any one of these dimensions deeply boring, and the combinations so much more provocative.  Who’s with me and what do you call yourselves?  Combinatorists? Alchemists?

http://blog.ascher.ca/2015/05/25/technologism/

I found myself missing mozharness at various points over the past 10 months. Several things kept me from using it at my then-new job:

• Even though we had kept mozharness.base.* largely non-Mozilla-specific, the mozharness clone-and-run model meant there was a lot of Mozilla-specific code that came along with it.
  
  
• The monolithic BaseScript + mixins model had a very steep barrier of entry. There's a significant learning curve, and scripts need to be fully ported to mozharness to take advantage of its features.

I had wanted to address these issues for years, but never had time to devote fully to harness-specific development.

Now I do.

Introducing scriptharness 0.1.0:

• Release Notes: http://python-scriptharness.readthedocs.org/en/stable/releasenotes/0.1.0/
  
  
  • The release notes highlight differences between scriptharness and mozharness, as well as known issues.
    
    
• PyPI: https://pypi.python.org/pypi/scriptharness/0.1.0
  
  
• GitHub: https://github.com/scriptharness/python-scriptharness/releases/tag/v0.1.0
  
  
• Read the Docs: http://python-scriptharness.readthedocs.org/en/stable/
  
  

I'm proud of this. I'm also aware it's not mature [yet], and it's currently missing some functionality.

There are some ideas I'd love to explore before 1.0.0:

• multiple Script objects with threading and separate logs
  
  
• Config Type Definitions
  
  
• rethink how to enable/disable actions. I could keep mozharness' --add-action clobber --no-upload structure, or play with --actions +clobber -upload or something. (The - before the upload in the latter might cause argparse issues?)
  
  
• also, explore Maven-style actions (all actions before target action are enabled) and actions with dependencies on other actions. I prefer keeping each action independent, idempotent, and individually targetable, but I can see someone wanting the other behavior for certain scripts.
  
  
• I've already split out strings from code in a number of places, for unit testing. I'm curious what it would take to make scriptharness localizable, and if there would be demand for it.
  
  
• ahal suggested adding structured logging; I'd love to investigate that.
  
  

I already have 0.2.0 on the brain. I'd love any feedback or patches.

comments

http://escapewindow.dreamwidth.org/246069.html