The W3C Browser Tools- and Testing Working Group met again 13-14 July 2016 in Redmond, WA to discuss the progress of the WebDriver specification. I will try to summarise the discussions, but if you’re interested in all the details the meetings have been meticulously scribed.

I wrote about the progress from our TPAC 2015 meeting previously, and we appear to have made good progress since then. The specification text is nearing completion, although it is missing a few important chapters: Some particularly obvious omissions are the complete lack of input handling, and a big, difficult void where advanced user actions are meant to be.

Actions

James has been hard at work drafting a proposal for action semantics, which we went over in great detail. I think it’s fair to say there had been conceptual agreement in the working group on what the actions were meant to accomplish, but that the details of how they were going to work were extremely scarce.

WebDriver tries to innovate on the actions as they appear in Selenium. Actions in Selenium were originally meant to provide a way to pipeline a sequence of interactions—such as pressing down a mouse button, moving the mouse, and releasing it—through a complex data structure to a single command endpoint. The idea was that this would help address some of the race conditions that are intrinsically part of the one-directional design of the protocol, and reduce latency which may be critical when interacting with a document.

Unfortunately the pipelining design to reduce the number of HTTP requests was never quite implemented in Selenium, and the API design suffered from over-specialisation of different types of input devices and actions. The specification attempts to rectify this by generalising the range of input device classes, and by associating the actions that can be performed with a certain class. This means we are moving away from a flat sequence of types, such as [{type: "mouseDown"}, {type: "mouseMove"}, {type: "mouseUp"}] to a model where each input device has its own “track”. This limits the actions you can perform with each device, which makes some conceptual sense because it would be impossible to i.e. type keys with a mouse or press a mouse button with a stylus/pen input device.

The side-effect of this design is that it allows for parallelisation of actions from one or more types of input devices. This is an important development, as it makes it possible to combine primitives for input methods such as touch: In reality, a device cannot determine whether two fingers are “associated” with the same hand. So instead of defining high-level actions such as pinch and flick, it gives you the right level of granularity to combine actions from two or more touch “finger” devices to synthesise more complex movements. We believe this is a good approach with the right level of granularity that doesn’t try to over-specify or shoehorn in primitives that might not make sense in a cross-browser automation setting.

I’m looking forward to seeing James’ work land in the specificaton text. I think probably some explanatory notes and examples are required to fully explain this concept for both implementors and users.

Input locality

A known limitation of Selenium that we are not proud of is that it does not have a good story for input with alternative keyboard layouts. We have explicitly phrased the specification in such a way that it doesn’t make it impossible to retrofit in support for multiple layouts in the future. But right now we want to finish the baseline of the specification before we try moving into this.

The current design ideas floating around are to have some way of setting a keyboard layout either through a command or a capability. This would allow / to generate key events for Shift and ? on an American layout, and Shift and 7 on Norwegian layout. The biggest reason this is hard is because we need to find the right key code conversion tables for what would happen when typing for example

https://sny.no/2016/07/bttup