K Lars Lohn: 0039 Firefox V2 |
|
Support.Mozilla.Org: What’s Up with SUMO – 25th August |
Hello, SUMO Nation!
Another hot week behind us, and you kept us going – thank you for that! Here is a portion of the latest and greatest news from the world of SUMO – your world :-)
If you just joined us, don’t hesitate – come over and say “hi” in the forums!
We salute you!
… and we’re done! We hope you have a great week(end) and that you come back soon to keep rocking the helpful web with us!
P.S. Just in case you missed it, here’s a great read about the way we want you to make Mozilla look better in the future.
https://blog.mozilla.org/sumo/2016/08/25/whats-up-with-sumo-25th-august/
|
Matej Cepl: Parsing milestoned XML in Python |
I am trying to write a tool in Python (using Python 3.4 to be able to use the latest Python standard library on Windows without using any external libraries on Windows) for some manipulation with the source code for the Bible texts.
Let me first explain what is the milestoned XML, because many normal Python programmers dealing with normal XML documents may not be familiar with it. There is a problem with using XML markup for documents with complicated structure. One rather complete article on this topic is DeRose (2016).
Briefly [1] , the problem in many areas (especially in documents processing) is with multiple possible hierarchies overlapping each other (e.g., in Bibles there are divisions of text which are going across verse and chapters boundaries and sometimes terminating in the middle of verse, many especially English Bibles marks Jesus’ sayings with a special element, and of course this can go over several verses etc.). One of the ways how to overcome obvious problem that XML doesn't allow overlapping elements is to use milestones. So for example the book of Bible could be divided not like
text ... ...
but just putting milestones in the text, i.e.:
span> n="1" /> span> sID="ID1.1" />text of verse 1.1 span> eID="ID1.1" /> ....
So, in my case the part of the document may look like
text texttextB textB textC textC textD textD
And I would like to get from some kind of iterator this series of outputs:
[(1, 1, "text text", ['text text']), (1, 2, "textB textB textC textC", ['', 'textB textB', ' ' , 'textC textC']), (1, 3, "textD textD", ['', 'textD textD', ''])]
(the first two numbers should be number of the chapter and verse respectively).
My first attempt was in its core this iterator:
def __iter__(self) -> Tuple[int, int, str]: """ iterate through the first level elements NOTE: this iterator assumes only all milestoned elements on the first level of depth. If this assumption fails, it might be necessary to rewrite this function (or perhaps ``text`` method) to be recursive. """ collected = None for child in self.root: if child.tag in ['titulek']: continue if child.tag in ['kap', 'vers']: if collected and collected.strip(): yield self.cur_chapter, self.cur_verse, \ self._list_to_clean_text(collected) if child.tag == 'kap': self.cur_chapter = int(child.get('n')) elif child.tag == 'vers': self.cur_verse = int(child.get('n')) collected = child.tail or '' else: if collected is not None: if child.text is not None: collected += child.text for sub_child in child: collected += self._recursive_get_text(sub_child) if child.tail is not None: collected += child.tail
(self.root is a product of
ElementTree.parse(file_name).getroot()). The problem of this code
lies in the note. When the
#!/usr/bin/env python3 from xml.etree import ElementTree as ET from typing import List def start_element(elem: ET.Element) -> str: outx = ['<{} '.format(elem.tag)] for attr, attval in elem.items(): outx.append('{}={} '.format(attr, attval)) outx.append('>') return ''.join(outx) def recursive_parse(elem: ET.Element) -> List[str]: col_xml = [] col_txt = '' cur_chapter = chap if elem.text is None: col_xml.append(ET.tostring(elem)) if elem.tail is not None: col_txt += elem.tail else: col_xml.extend([start_element(elem), elem.text]) col_txt += elem.text for subch in elem: subch_xml, subch_text = recursive_parse(subch) col_xml.extend(subch_xml) col_txt += subch_text col_xml.append(''.format(elem.tag)) if elem.tail is not None: col_xml.append(elem.tail) col_txt += elem.tail return col_xml, col_txt if __name__ == '__main__': # write result XML to CRLF-delimited file with # ET.tostring(ET.fromstringlist(result), encoding='utf8') # or encoding='unicode'? Better for testing? xml_file = ET.parse('tests/data/Mat-old.xml') collected_XML, collected_TEXT = recursive_parse(xml_file.getroot()) with open('test.xml', 'w', encoding='utf8', newline='\r\n') as outf: print(ET.tostring(ET.fromstringlist(collected_XML), encoding='unicode'), file=outf) with open('test.txt', 'w', encoding='utf8', newline='\r\n') as outf: print(collected_TEXT, file=outf)
This works correctly in sense that the generated file test.xml is
identical to the original XML file (after reformatting both files with
tidy -i -xml -utf8). However, it is not iterator, so I would like to
somehow combine the virtues of both snippets of code into one.
Obviously, the problem is that return in my ideal code should serve
two purposes. Once it should actually yield nicely formatted result from
the iterator, second time it should just provide content of the inner
elements (or not, if the inner element contains
if __name__ == '__main__': xml_file = ET.parse('tests/data/Mat-old.xml') parser = ET.XMLParser(target=ET.TreeBuilder()) with open('test.txt', 'w', newline='\r\n') as out_txt, \ open('test.xml', 'w', newline='\r\n') as out_xml: for ch, v, verse_txt, verse_xml in recursive_parse(xml_file): print(verse_txt, file=out_txt) # or directly parser.feed(verse_xml) # if verse_xml is not a list parser.feed(''.join(verse_xml)) print(ET.tostring(parser.close(), encoding='unicode'), file=out_xml)
So, my first attempt to rewrite the iterator (so far without the XML part I have):
def __iter__(self) -> Tuple[CollectedInfo, str]: """ iterate through the first level elements """ cur_chapter = 0 cur_verse = 0 collected_txt = '' # collected XML is NOT directly convertable into Element objects, # it should be treated more like a list of SAX-like events. # # xml.etree.ElementTree.fromstringlist(sequence, parser=None) # Parses an XML document from a sequence of string fragments. # sequence is a list or other sequence containing XML data fragments. # parser is an optional parser instance. If not given, the standard # XMLParser parser is used. Returns an Element instance. # # sequence = ["", "text"] # element = ET.fromstringlist(sequence) # self.assertEqual(ET.tostring(element), # b'text') # FIXME pridej i sber XML 'utrzku # collected_xml = None for child in self.root.iter(): if child.tag in ['titulek']: collected_txt += '\n{}\n'.format(child.text) collected_txt += child.tail or '' if child.tag in ['kap', 'vers']: if collected_txt and collected_txt.strip(): yield CollectedInfo(cur_chapter, cur_verse, re.sub(r'[\s\n]+', ' ', collected_txt, flags=re.DOTALL).strip()), \ child.tail or '' if child.tag == 'kap': cur_chapter = int(child.get('n')) elif child.tag == 'vers': cur_verse = int(child.get('n')) else: collected_txt += child.text or '' for sub_child in child: for sub_info, sub_tail in MilestonedElement(sub_child): if sub_info.verse == 0 or sub_info.chap == 0: collected_txt += sub_info.text + sub_tail else: # FIXME what happens if sub_element contains # multipleelements? yield CollectedInfo( sub_info.chap, sub_info.verse, collected_txt + sub_info.text), '' collected_txt = sub_tail collected_txt += child.tail or '' yield CollectedInfo(0, 0, collected_txt), ''
Am I going the right way, or did I still not get it?
[1] | From the discussion of the topic on the XSL list. |
https://matej.ceplovi.cz/blog/parsing-milestoned-xml-in-python.html
|
Air Mozilla: Connected Devices Weekly Program Update, 25 Aug 2016 |
Weekly project updates from the Mozilla Connected Devices team.
https://air.mozilla.org/connected-devices-weekly-program-update-20160825/
|
Mozilla Addons Blog: WebExtensions in Firefox 50 |
Firefox 50 landed in Developer Edition this week, so we have another update on WebExtensions for you!Please use the WebExtensions API for any new add-on development, and consider porting your existing add-ons as soon as possible.
It’s also a great time to port because WebExtensions is compatible with multiprocess Firefox, which began rolling out in Firefox 48 to people without add-ons installed. When Firefox 49 reaches the release channel in September, we will begin testing multiprocess Firefox with add-ons. The goal is to turn it on for everyone in January 2017 with the release of Firefox 51.
If you need help porting to WebExtensions, please start with the compatibility checker, and check out these resources.
Since the last release, more than 79 bugs were closed on WebExtensions alone.
In Firefox 50, a few more history APIs landed: the getVisits function, and two events–onVisited and onVisitRemoved.
Content scripts in WebExtensions now gain access to a few export helpers that existed in SDK add-ons: cloneInto, createObjectIn and exportFunction.
The webNavigation API has gained event filtering. This allows users of the webNavigation API to filter events based on some criteria. Details on the URL Filtering option are available here.
There’s been a change to debugging WebExtensions. If you go to about:debugging and click on debug you now get all the Firefox Developer Tools features that are available to you on a regular webpage.
Why is this significant? Besides providing more developer features, this will work across add-on reloads and allows the debugging of more parts of WebExtensions. More importantly, it means that we are now using the same debugger that the rest of the Firefox Dev Tools team is using. Reducing duplicated code is a good thing.
As mentioned in an earlier blog post, native messaging is now available. This allows you to communicate with other processes on the host’s operating system. It’s a commonly used API for password managers and security software, which need to communicate with external processes.
The documentation for WebExtensions has been updated with some amazing resources over the last few months. This has included the addition of a few new areas:
The documentation is hosted on MDN and updates or improvements to the documentation are always welcome.
There are now 17 example WebExtensions on github. Recently added are history-deleter and cookie-bg-picker.
We are currently working on the proxy API. The intent is to ship a slightly different API than the one Chrome provides because we have access to better APIs in Firefox.
The ability to write WebExtensions APIs in an add-on has now landed in Firefox 51 through the implementation of WebExtensions Experiments. This means that you don’t need to build and compile all of Firefox in order to add in new APIs and get involved in WebExtensions. The policy for this functionality is currently under discussion and we’ll have more details soon.
There are also lots of other ways to get involved with WebExtensions, so please check them out!
https://blog.mozilla.org/addons/2016/08/25/webextensions-in-firefox-50/
|
Air Mozilla: Reps weekly, 25 Aug 2016 |
This is a weekly call with some of the Reps to discuss all matters about/affecting Reps and invite Reps to share their work with everyone.
|
Air Mozilla: Web QA Team Meeting, 25 Aug 2016 |
They say a Mozilla Web QA team member is the most fearless creature in the world. They say their jaws are powerful enough to crush...
|
Marcia Knous: Professional Development - Strategic Leadership |
|
The Mozilla Blog: EU Copyright Law Undermines Innovation and Creativity on the Internet. Mozilla is Fighting for Reform |
The internet is an unprecedented platform for innovation, opportunity and creativity. It’s where artists create; where coders and entrepreneurs build game-changing technology; where educators and researchers unlock progress; and where everyday people live their lives.
The internet brings new ideas to life everyday, and helps make existing ideas better. As a result, we need laws that protect and enshrine the internet as an open, collaborative platform.
But in the EU, certain laws haven’t caught up with the internet. The current copyright legal framework is outdated. It stifles opportunity and prevents — and in many cases, legally prohibits — artists, coders and everyone else from creating and innovating online. This framework was enacted before the internet changed the way we live. As a result, these laws clash with life in the 21st century.
Here are just a few examples of outdated copyright law in the EU:
It’s time our laws caught up with our technology. Now is the time to make a difference: This fall, the European Commission plans to reform the EU copyright framework.
Mozilla is calling on the EU Commission to enact reform. And we’re rallying and educating citizens to do the same. Today, Mozilla is launching a campaign to bring copyright law into the 21st century. Citizens can read and sign our petition. When you add your name, you’re supporting three big reforms:
1. Update EU copyright law for the 21st century.
Copyright can be valuable in promoting education, research, and creativity — if it’s not out of date and excessively restrictive. The EU’s current copyright laws were passed in 2001, before most of us had smartphones. We need to update and harmonise the rules so we can tinker, create, share, and learn on the internet. Education, parody, panorama, remix and analysis shouldn’t be unlawful.
2. Build in openness and flexibility to foster innovation and creativity.
Technology advances at a rapid pace, and laws can’t keep up. That’s why our laws must be future-proof: designed so they remain relevant in 5, 10 or even 15 years. We need to allow new uses of copyrighted works in order to expand growth and innovation. We need to build into the law flexibility — through a User Generated Content (UGC) exception and a clause like an open norm, fair dealing, or fair use — to empower everyday people to shape and improve the internet.
3. Don’t break the internet.
A key part of what makes the internet awesome is the principle of innovation without permission — that anyone, anywhere, can create and reach an audience without anyone standing in the way. But that key principle is under threat. Some people are calling for licensing fees and restrictions on internet companies for basic things like creating hyperlinks or uploading content. Others are calling for new laws that would mandate monitoring and filtering online. These changes would establish gatekeepers and barriers to entry online, and would risk undermining the internet as a platform for economic growth and free expression.
At Mozilla, we’re committed to an exceptional internet. That means fighting for laws that make sense in the 21st century. Are you with us? Voice your support for modern copyright law in the EU and sign the petition today.
|
Mozilla Addons Blog: Add-ons Update – Week of 2016/08/24 |
I post these updates every 3 weeks to inform add-on developers about the status of the review queues, add-on compatibility, and other happenings in the add-ons world.
In the past 3 weeks, 1228 listed add-on submissions were reviewed:
There are 203 listed add-ons awaiting review.
You can read about the improvements we’ve made in the review queues here.
If you’re an add-on developer and are looking for contribution opportunities, please consider joining us. Add-on reviewers are critical for our success, and can earn cool gear for their work. Visit our wiki page for more information.
The compatibility blog post for Firefox 49 is up, and the bulk validation was run. The blog post for Firefox 50 should be published in the next week.
Going back to Firefox 48, there are a couple of changes that are worth keeping in mind: (1) release and beta builds no longer have a preference to deactivate signing enforcement, and (2) multiprocess Firefox is now enabled for users without add-ons, and add-ons will be gradually phased in, so make sure you’ve tested your add-on and either use WebExtensions or set the multiprocess compatible flag in your add-on manifest.
As always, we recommend that you test your add-ons on Beta and Firefox Developer Edition to make sure that they continue to work correctly. End users can install the Add-on Compatibility Reporter to identify and report any add-ons that aren’t working anymore.
We would like to thank fiveNinePlusR for their recent contributions to the add-ons world. You can read more about their contributions in our recognition page.
https://blog.mozilla.org/addons/2016/08/24/add-ons-update-86/
|
Emma Humphries: Going from "Drive-Thru" Bug Filers to New Mozillians |
Earlier this month at PyCon AU, VM Brasseur gave this talk on maximizing the contributions from "drive-thru" participants in your open source project. Her thesis is that most of your contributors are going to report one bug, fix one thing, and move on, so it's in your best interest to set up your project so that these one-and-done contributors have a great experience: a process which is "easy to understand, easy to follow, and which makes it easy to contribute" will make new contributors successful, and more likely to return.
Meanwhile, the Firefox team's trying to increase the number of repeat contributors, in particular, people filing quality bugs. We know that a prompt reply to a first code contribution encourages that contributor to do more, and believe the same is true of bug filing: that quickly triaging incoming bugs will encourage those contributors to file more, and better bugs, making for a healthy QA process and a higher quality product.
For now, it's rare that the first few bugs someone files in Bugzilla are "high quality" bugs - clearly described, reproducible and actionable - which means they're more likely to stall, or be closed as invalid or incomplete, which in turn means that contributor is less likely file another.
In order to have more repeat contributors, we have to work on the "drive-thru" bug filer's experience that Brasseur describes in her talk, to make new contributors successful, and make them repeat quality bug filers. Here's some of the things we're doing:
These steps don't guarantee success, but will guide us going forward. Thanks to Michael Hoye for review and comments.
|
Air Mozilla: Weekly SUMO Community Meeting August 24, 2016 |
This is the sumo weekly call
https://air.mozilla.org/weekly-sumo-community-meeting-august-24-2016/
|
Wil Clouser: It's time to upgrade from Test Pilot to Test Pilot |
Were you one of the brave pioneers who participated in the original Test Pilot program? You deserve a hearty thanks!
Are you also someone who never bothered to uninstall the old Test Pilot add-on which has been sitting around in your browser ever since? Or maybe you were hoping the program would be revived and another experiment would appear? Or maybe you forgot all about it?
If that's you, I've got news: I'm releasing an upgrade for the old Test Pilot add-on today which will:
If you're interested in joining the new program, please click through and install the new add-on when you're prompted. If you're not interested, just close the tab and be happy your browser has one less add-on it needs to load every time you start it up.
Random FAQs:
The old add-on is, well, old, so it can only be uninstalled after you restart Firefox. It's all automatic when you do, but if you're waiting for a prompt and don't see it, that could be the culprit.
If you already have the new Test Pilot add-on installed, you can just close the new tab. The old add-on will be uninstalled and your existing Test Pilot add-on (and any experiments you have running) will be unaffected.
A special thanks to Philipp Kewisch and Andreas Wagner for their help writing and reviewing the add-on.
http://micropipes.com/blog//2016/08/24/time-to-upgrade-from-test-pilot-to-test-pilot/
|
QMO: Firefox 49 Beta 7 Testday, August 26th |
Hello Mozillians,
We are happy to announce that Friday, August 26th, we are organizing Firefox 49 Beta 7 Testday. We will be focusing our testing on WebGL Compatibility and Exploratory Testing. Check out the detailed instructions via this etherpad.
No previous testing experience is required, so feel free to join us on #qa IRC channel where our moderators will offer you guidance and answer your questions.
Join us and help us make Firefox better! See you on Friday!
https://quality.mozilla.org/2016/08/firefox-49-beta-7-testday-august-26th/
|
Alex Vincent: Introducing es7-membrane: A new ECMAScript 2016 Membrane implementation |
I have a new ECMAScript membrane implementation, which I will maintain and use in a professional capacity, and which I’m looking for lots of help with in the form of code reviews and API design advice.
For those of you who don’t remember what a membrane is, Tom van Cutsem wrote about membranes a few years ago, including the first implementations in JavaScript. I recently answered a StackOverflow question on why a membrane might be useful in the first place.
Right now, the membrane supports “perfect” mirroring across object graphs: as far as I can tell, separate object graphs within the same membrane never see objects or functions from another object graph.
The word “perfect” is in quotes because there are probably bugs, facets I haven’t yet tested for (“What happens if I call Object.freeze() on a proxy from the membrane?”, for example). There is no support yet for the real uses of proxies, such as hiding properties, exposing new ones, or special handling of primitive values. That support is forthcoming, as I do expect I will need a membrane in my “Verbosio” project (an experimental XML editor concept, irrelevant to this group) and another for the company I work for.
The good news is the tests pass in Node 6.4, the current Google Chrome release, and Mozilla Firefox 51 (trunk, debug build). I have not tested any other browser or ECMAScript environment. I also will be checking in lots of use cases over the next few weeks which will guide future work on the module.
With all that said, I’d love to get some help. That’s why I moved it to its own GitHub repository.
If you’re not able to comment here for some reason, I’ve set up a GitHub wiki page for that purpose.
|
Air Mozilla: Mozilla Weekly Project Meeting, 22 Aug 2016 |
The Monday Project Meeting
https://air.mozilla.org/mozilla-weekly-project-meeting-20160822/
|
Hal Wine: Py Bay 2016 - a First Report |
PyBay held their first local Python conference this last weekend (Friday, August 19 through Sunday, August 21). What a great event! I just wanted to get down some first impressions - I hope to do more after the slides and videos are up.
First, the venue and arrangements were spot on. Check the twitter traffic for #PyBay2016 and @Py_Bay and you see numerous comments confirming that. And, I must say the food was worthy of San Francisco - very tasty. And healthy. With the weather cooperating to be nicely sunny around noon, the outdoor seating was appreciated by all who came from far away. The organizers even made it a truly Bay Area experience by arranging for both Yoga and Improv breakouts. The people were great - volunteers, organizers, speakers, and attendees. Props to all.
The technical program was well organized, and I’m really looking forward to the videos for the talks I couldn’t attend. Here’s some quick highlights that I hope to backfill.
OpenTracing - a talk by Ben Sigelman - big one for distributed systems, it promises a straightforward way to identify critical path issues across a micro-service distributed architecture. Embraced by a number of big companies (Google, Uber), it builds on real world experience with distributed systems.
Programs just need to add about 5 lines of setup, and one call per ‘traceable action’ (whatever that means for your environment). The output can be directed anywhere - one of the specialized UI’s or traditional log aggregation services.
There are open source, commercial offerings, and libraries for many languages (Python, Go, etc.) & frameworks (SQLAlchemy, Flask, etc.). As an entry level, you can insert the trace calls and render to existing logging. The framework adds guids to simplify tracking across multiple hosts & processes.
Semantic logging - a lightning talk by Mahmoud Hashemi was a brief introduction to the lithoxyl package. The readme contains the selling point. (Especially since I took Raymond Hettinger‘s intermediate class on Python Friday, and he convincingly advocated for the ratio of business logic lines to “admin lines” as a metric of good code.)
Mahmoud Hashemi also did a full talk on profiling Python performance in enterprise applications, and ways to improve that performance. (And, yes, we write enterprise applications.)
And there was lots more that I’ll try to cover later. And add in some links for the above as they become available.
|
Karl Dubost: [worklog] Edition 032. The sento cheminey |
From the seventh floor, I see the cheminey of a local sento. It's not always on. The smoke is not coming out, it gives me a good visual feedback of the opening hours. It's here. It doesn't have notifications. It doesn't have ics, atom. It's just there. And it's fine as-is. The digital world seems sometimes to create complicated UI and UX over things which are just working. They become disruptive but not helpful.
Tune of the week: Leo Ferr'e - Avec le temps
Progress this week:
Today: 2016-08-22T13:46:36.150030 300 open issues ---------------------- needsinfo 4 needsdiagnosis 86 needscontact 18 contactready 28 sitewait 156 ----------------------
You are welcome to participate
(a selection of some of the bugs worked on this week).
mask
in Firefox. It seems there is a regression. A fix was created for another issue, but that doesn't solve the original issue.this.$html
is undefined.perspective(0)
is now handled by Firefox. it was creating issues.
for Firefox only.Otsukare!
|
Will Kahn-Greene: pyvideo last thoughts |
pyvideo.org is an index of Python-related conference and user-group videos on the Internet. Saw a session you liked and want to share it? It's likely you can find it, watch it, and share it with pyvideo.org.
This is my last update. pyvideo.org is now in new and better hands and will continue going forward.
Read more… (2 mins to read)
http://bluesock.org/%7Ewillkg/blog/pyvideo/status_20160822.html
|
Cameron Kaiser: TenFourFox 45 beta 2: not yet |
Unfortunately, while this also makes Amazon Music's delay on starting and switching tracks better, it's still noticeably regressed compared to 38. In addition, I'm hitting an assertion with YouTube on certain, though not all, videos over a certain minimum length; in the release builds it just seizes up at that point and refuses to play the video further. Some fiddling in the debugger indicates it might be related to what Chris Trusch was reporting about frameskipping not working right in 45. If I'm really lucky all this is related and I can fix them all by fixing that root problem. None of these are showstoppers but that YouTube assertion is currently my highest priority to fix for the next beta; I have not solved it yet though I may be able to wallpaper it. I'm aiming for beta 2 this coming week.
On the localization front we have French, German and Japanese translations, the latter from the usual group that operates separately from Chris T's work in issue 42. I'd like to get a couple more in the can before our planned release on or about September 13. If you can help, please sign up.
http://tenfourfox.blogspot.com/2016/08/tenfourfox-45-beta-2-not-yet.html
|