Tagged: Open Source

WEBVTT: Initial Support Incoming

I have some great news. Bug 833382 and 833386, the last pieces needed to get initial support for WEBVTT into Gecko will be landing for sure next week. That being said, I hope I don’t have to eat my words. It’s looking really good though. Bug 833382 has gotten to the point where Boris (:bz) has conditionally r+’ed it and 833386 has reached the same point with Ralph (:rillian). Now all that’s left to do for 833382 is to spin up some try builds and if it’s all green, hopefully we’ll be good to go. 833386 still needs to go through review with Chris Pearce (:cpearce), but I don’t think that will take that long.

I’ve been pushing really hard on landing these two this last week and I’m ecstatic to see that we’ve gotten to the point where we will be landing them in the next few days. The WebVTTLoadListener in particular I’m very happy about as most of that code is mine and I’ve worked really hard at it. It’ll feel good to land that. In the case of 833386 most of that code is Jordan Raffoul’s (:jbraffoul) and Marcus Nsaad’s (:msaad) and the work I did on it was just to consolidate it and get it through the last couple rounds of review. Marcus had been on vacation for a while and we really wanted to get this landed ASAP as it is blocking quite a few things so I asked Marcus if I could step in and he didn’t have a problem with it (Yay Marcus!).

Tests, Tests

One of the biggest problems I was having with the tests for the HTMLTrackElement and TextTrack* DOM classes (833386) was that we wanted to test everything with the pref set to true and also with it set to false. The thinking here was that we could do it in the same file. The problem was that it wasn’t working… and we all thought that it should be possible to do so I spent about 2~3 days trying to get it to work. At first I suspected that it was due to closures in Javascript and that the ‘state’ or variables were being sustained somehow by this. There was no way to get out of being entangled in closures too as we need to use a particular function, SpecialPowers.pushPrefEnv, that takes a function argument and async calls it after the pref is set. So in order for us to test with true and then false we have to have one pushPrefEnv(true) and one pushPrefEnv(false) with the second embedded in the first so that they are called one after the other and in that way we have a definitive point where we can call SimpleTest.finish().

I finally figured out, when asking the right people (bz and Ms2ger, who woulda’ thunk?), that it’s actually impossible to test with two different prefs in one page as there is one prototype for each element on the page and the pref is only applied to it once, when the element is created for the first time. So once you’ve created an element underneath one pref on a page, that’s it, it will behave like it is preffed that way no matter what you do. After that it was pretty quick to get through the rest of the code needed. One of the really good things about this process too is that it allowed me to find a lot of points where our current implementation is not to spec. I’ve filed a few bugs on those this week. I’ve also closed a few bugs that have been fixed with changes in the last while. I also spun off another bug for tests that we will need to implement when the WEBVTT pref finally gets removed.

Try Server Access

The other really awesome news is that I’ve finally got try server access! Earlier this week Daniel Holbert (:dholbert) suggessted that I should apply for it and that he would vouch for me. I probably should have applied for it sooner then this as I could have used it for sure. The process was fairly easy and I’m glad to say I now have Level 1 commit access. Woot! If your interested in applying as well check out this page that describes what you will need to do.

In accordance with this new awesomeness I also had to learn about the process of pushing to the try server. Check out this good page for more information on how to do that. You’ll also probably want to look at Mercurial Manage Queue extension as it helps with managing a bunch of patches that you can move in and out of your branches easily. This really works well with my workflow of working on git and then just applying patches on my hg repository and pushing to the try server.

Until next time.


Getting Ready For Taiwan

A month back I found out that Caitlin and I were being invited to Mozilla’s Web Rendering Week in Taiwan. Needless to say I was shocked. Ralph Giles, who has been working with us closely on WEBVTT and was/is our main contact with Mozilla for WEBVTT, kindly invited us out — thanks Ralph!!

Therefore, a lot of this week has been spent getting ready for the trip — getting ready to travel and trying to push on some final bugs to get initial support for WEBVTT into Nightly before we leave.

Getting that initial support entailed a few things, all of which haven’t happened yet… Don’t worry though! If we don’t get it finished before Taiwan we will definitely get it finished while we’re in Taiwan. We need(ed) too:

  • Get the WEBVTT library to a state where we felt comfortable tagging 0.5 and landing it in Gecko.
  • Land the DOM implementation of HTMLTrackElement and the TextTrack objects.
  • Land the parser and Gecko integration bug.

Getting WEBVTT to 0.5 was pretty easy. The only extra thing we wanted to add before 0.5 was support for the new <lang> tag. We needed a new string list pop function for that as well. I implemented that this week, which also exposed a bug that we hadn’t found yet because we didn’t have a proper test for it. The bug happened when we had an <rt> tag that was not enclosed within a <ruby> tag. In these situations the <rt> tag should not be processed. We weren’t handling it correctly and it was crashing the parser. That got landed as well yesterday. We tagged 0.5 and Caitlin is working on the bug to land it in Gecko now. WEBVTT 0.5 contains a lot of good things like sec-critical bug fixes and library usability fixes.

Ralph is very close to landing the DOM implementation. That should be happening within the next few days. The parser integration is getting close as well. I’ve been working on changing the code that converts the C node tree to a DOM tree from a recursive algorithm to an iterative one. It’s been a little tricky as we need to keep track of where we are in the C tree as we iterate over it, as well as where we are in the tree of DOM nodes that we are creating. We also need to keep track of the ‘last branching parent’ so that we’re able to tell when we need to go back up the tree. I have a solution that I put together that will hopefully be able to get an r+. I’ll have it up for review tonight on the bug.

The other major thing I did this week was give a presentation to CDOT about WEBVTT. You can check out the presentation over on my GitHub page. It was recorded so maybe you all can see it at some point when it gets posted! I’ll link at that time.

Until next time.

Jekyll and Hyde

I’ve been fiddling around lately with getting a new blog setup. I’ve asked around a lot and it seems like a super popular option for devs lately has been hosting on GitHub and using Jekyll to build your blog. Dale who worked on WEBVTT with me has a pretty sweet set up using this method.

Going with that I decided to set up my new blog this weekend. It would be easy I told myself, ever the optimist. You think I would learn after programming for as long as I have..

Installing the basic setup that you need to start writing blogs with Jekyll was super easy. Just install Ruby and RubyGems with MacPorts (if your on Mac) and install Jekyll with RubyGems.

sudo port install ruby
sudo port install rb-rubygems
sudo gem install jekyll

My real troubles began when I tried to use Jekyll’s blog import tool to migrate my WordPress posts over to my new blog. To do this you need the jekyll-import command. For the longest time I was having trouble installing this because ‘jekyll-import’ isn’t a full out gem yet (according to official documentation it is…) it’s a beta gem. This means you have to install it using the –pre option.

sudo gem install jekyll-import --pre

Without this it will tell you it “can’t find the gem ‘jekyll-import'”, but that you “may have been looking to install the gem jekyll-import'”… confusing, I know.

Great! Now it could find the gem when I went to install it. However now I had a new problem — it couldn’t find libiconv. This was weird because I had installed libiconv with MacPorts. It was only until I went to check the mkmf.log that the gem had created that I realized what had been happening (previously I had just been relying on what was being spit out on the command line, like a noob). In the log I found out that the libiconv I had wasn’t the architecture that was needed. I had x86_64, but I needed i386. Solving this was super easy. Just install the port variant of the libiconv library that is a universal library, i.e. you can use it when compiling for i386 and x86_64.

sudo port install libiconv +universal

The last problem I had with installing after this was getting the correct kind version of ruby. Jekyll-import needs ruby 2.0, I had 1.92. I had difficulty setting this up as I had multiple versions on my Mac including one that shipped with OS X. What you need to do is to install the ruby 2.0 port and then activate it.

sudo port install ruby20
sudo port select --list ruby (to get all ruby versions)
sudo port select ruby ruby20 (to select ruby 2.0)

Finally, jekyll-import installed and I tried to migrate my WordPress blog posts. I ran into one more problem though and this turned out to be a real bug. The import command was incorrectly trying to interpret the input of the command line arguments as a string instead of as a hash, which was what it needed. I filed the bug on the project’s GitHub page and someone solved it quickly! I haven’t had time to check to see if the gem has been updated yet so I don’t know if it’s completely working yet. Will post back when I find out.

So the moral of the story here is always check the log file when you encounter problems. It’s happened to me before where the log file has the correct information and the command line isn’t very helpful. Doing this will save you tons of time.

After all that I decided to set up my new blog with at least a ‘coming soon’ sign. So it’s up! You can check it out here. All that’s left is to finish my blog migration and add cool things like comment tracking and twitter feeds. I’d also like the user to be able to change the theme in real time. That would be cool. I also got to get some DNS server hosting for my brand spanking new domain http://rickeyre.ca. After that all will be good.

Until next time.

WEBVTT Mozilla Presentation

In my last post I talked about the team having to sprint to a demo in order to be able to present WEBVTT at Mozilla Toronto. We ended up being able to cobble together a simple demo — thank God. It was starting to look like in the last hours that we weren’t going to be able to do it, but we pulled through! You can check out the presentation where we show the working demo on AirMozilla. The most interesting thing about the demo for me was the fact that it had the effect of revealing to us just how much work is left on WEBVTT. So be prepared for many, many, many more blog posts ; ) on WEBVTT.

I haven’t been able to put as much time as I’d like to into WEBVTT in the last couple of weeks since a lot of my other classes are starting to finish up — lots of papers, presentations, etc, are due. This is also my final semester so I’m in a course where I have to write a white paper, basically a small thesis paper. I’ll be posting that on my blog when I’m finished as I’m finding my topic, Open-source Software Governance Systems, fairly interesting and although I’m no expert, maybe others will find it interesting as well.

In light of this I’ve only been able to do minor things on the WEBVTT project since the demo. I’ve been reviewing code, keeping up to date with issues, fixing some minor bugs in the cue-text parser, finishing up the rest of the cue-text unit test fixes, and finally, reading through the HTML 5 spec to figure out just what is left to be done. The HTML 5 spec kind of frustrates me as it doesn’t put all the text-track stuff together. Granted, there is an organization to it, but I’m only concerned with text-track… and it’s kind of hard to find all the text-track stuff barring reading through the entire spec. Some of the things I’ve been finding are that we do not have the entire HTML 5 spec implemented yet and some portions of the spec have already changed. For example, getCueAsHtml() is no longer in it. So we will have to do some updating at some point.

My white paper is due this coming Monday, so after that I will be turning my full attention to WEBVTT as my class needs to get it to 1.0 before the semester ends.

WEBVTT: Unit tests, unit tests…

I haven’t written in a quite a while so this is going to be a pretty long post.

TLDR: I made some progress on the Gecko code for the WEBVTT parser integration, but was unable to finish because I was moved over to finishing the WEBVTT parser exclusively as we needed to move fast on it. This resulted in unit test fixing galore.

Gecko Integration

I was working on the Gecko integration stuffyo before I got pulled away and made some decent progress on that. I got some feedback on my first WIP and I addressed most of the issues that the reviewers pointed out. A lot of it was minor nits like code style or incorrect code that I just needed to remove. However, there were a few big things:

  • Getting the OnDataAvailable() function to use the ReadSegments() function of the stream it is passed. To do this you need to create your own function that conforms to the function prototype of nsWriteSegmentFunc. What that does is it allows the nsIInputStream to read in your buffer for you and then you can just hand that buffer over to whatever you need to. This prevents memory leaks as it creates a central location where the memory leak would happen, and presumably, the code in the tree is already written for you and probably better then what you’ve written — so just use it.
  • Use an nsAutoRefTrait to manage the lifetime of the nsAutoRef<webvtt_parser>. That ended up not being so hard. What an nsAutoRefTrait of xType does is define what an nsAutoRef of xType needs to do during it’s lifetime. In our code, for example, all we need to do is define a Release() function that tells the nsAutoRef pointer what to call when it is being released. There are a few more behaviours that you can define like what do do on setup, etc. This is necessary since most things in the Gecko code are all smart pointers and objects, so you don’t want to manage their lifetimes explicitly, but you do want to provide a way to define the appropriate steps it needs to do during its lifetime. The awesome thing about this is that once you create an nsAutoRef of xType the nsAutoRefTrait of xType will be automatically linked to it.

The other couple of things I did before I submitted my last WIP for feeback was fleshing out a lot of the code that will convert the c structs from the c parser to DOM elements, getting the code that deals with DocumentFragments working correctly, and a few other minor things. You can check out the full WIP that I submitted for my last review here.

Driving Issues

The other thing that I was really trying to do up until this point, and currently, is drive the issue list. I’ve noticed that the issue list tends to stagnate if no one is there monitoring it. To this point I have been looking over the issue list, about once every other day, and closing issues that need to be closed, i.e. onces that have been resolved, seem to be going nowhere, or seem to have gone as far as they are able to, as well as trying to push issues that seem to have stalled by @ mentioning the parties involved, or asking a question about where we should head in light of the issue. This process has been very helpful because it has enabled us to keep our issue list from being cluttered by issues that are no longer needed and to also drive issues that people might have forgotten about.

Parser Unit Test Debugging

About three weeks into working on the Gecko integration as well as the parser  I was asked to step back from the Gecko integration for a while and to start working exclusively on getting the parser to pass all the unit tests. To that end, I relinquished my duties on the Gecko integration for a bit. My first job was to help Caitlin get her massive patch split up and landed. To do this she asked me to write a finish parsing function that would allow the patch to be split up more easily. After that was done I started to work on the payload unit tests, as I anticipated Caitlin’s cue code to be landing soon. I was able to get around 90 % of the payload unit tests working over reading week, valgrind and all, and solve many bugs in the parser. A lot of the bugs that were effecting the payload tests were also effecting the cue tests, so I was able to hit two birds with one stone in many cases.

The Importance of Passing Unit Tests

I came across a situation in this sprint where I re-discovered the importance of passing unit tests. How it all started was myself fixing a problem we were having with a function that we use to attach a parsed cue text node to another parent node. The function kept segfaulting and it was pretty easy to figure out what had been happening — it was dereferencing a variable before allocating memory for it. To fix this I allocated memory before hand, as well as rearranged the function a bit to simplify it and make it look cleaner. The code got landed and all was good, or so I thought. Later I uncovered that some of the valgrind errors we were getting in another unit tests were actually caused by what I had landed earlier. It was an easy fix, but it just goes to show how important passing unit tests are. It’s kind of impossible to tell if the code you write breaks anything when everything is broken to begin with.

The Art of Face Palming

As always this last week’s sprint had its fair share of face palming. Over the years I’ve gotten pretty good at it — you might even call me a face palm artist. This weeks face palm that takes first prize was an error we were getting in the grow function for strings where we were using the ‘==’ operator when we really wanted the ‘=’ operator. This was in an if statement, no less, so it took me a couple looks and one double-take to realize what was happening. C you upset me so much, I think we should break up.

Reference Counting is Really Nice

This is the first time I’ve really used reference counting myself, but I have to say it’s pretty awesome. I’ve heard some talk smack about it, but from my small experience working on the WEBVTT parser with it, it makes things so much easier. Especially since a lot of our objects are shared in multiple places in and outside of the parser. The problem that I found reference counting solves is when a project gets so big that is hard to tell where an object is used, especially when they are passed around a lot. So you might have the case where someone decides to delete an object that another piece of code is trying to use and it blows up. This isn’t even the worse case. The worse case is when you do this and it doesn’t blow up at first — so it’s landed. Then later, for what seems like a random reason, it will start blowing up because the use case where the code that you just landed causes the code to blow up is taking place. So now the reason why it is blowing up is not entirely clear at first. Reference counting makes this so much easier because when you are coding you can, in most cases, just assume that the calling function increased the reference count of the object when it passed it to your code, and so at the end of the code you can safely call the release function. No more explicitly managing the objects, your objects know when they need to be deleted. Due to this I’ve implemented reference counting on both the c nodes and our c string lists. Both of these structs are used extensively, passed around a lot, and are made available externally through the C++ wrappers. So it’s nice to have ref counting on them.

What Makes a Good Pull Request

One of the final things that really got reinforced to me over this last week’s sprint was the factors that make a good pull request, or patch. A good pull request is kind of like a good unit test. You want it to be focused in what it is trying to accomplish and atomic in scale i.e. it touches as little as possible. That doesn’t mean you can’t make pull requests that change like a billion lines. What it does mean is that those billion lines are all being changed for the same focused reason. Granted, there are times when these rules cannot be followed, but you should try to follow them in ≈99% of your pull requests.

My Pull Requests are Over 9000 (not really, I’m a nub)

If you’d like to check out the work I’ve been doing for the past couple of weeks you can look at my list of pull requests.

Next Week

Next week, March 14, we will be travelling to the Mozilla Toronto Office for our class. Our Professor, Dave Humphrey, wants us to have a demo and presentation ready to show the devs at Mozilla. So we will be sprinting on getting the last pieces of the puzzle for the WEBVTT integration locked in. This is mostly the integration code that I had to leave off on last week. I’m going to be getting a lot of help this time for that so hopefully we will get it done.

Until next time.

Connected Wellness: Threads, Cordova Lib, and Java to iOS

So it’s been another eventful week for the Connected Wellness iOS team at CDOT. We’ve made a lot of progress towards having the project completed, but there are still some major outstanding issues that we need to tackle. The PacketParser has yet to be completed (one of the most complex portions of the plugin), we’ve still yet to hear back from A&D about low energy BT devices, which we need in order to use the CoreBluetooth framework provided by Apple, and we’re still running into some problems regarding the amount of control that the CoreBT framework gives us. The CoreBT framework does not expose a way to start, stop, or enable Bluetooth in the device. These are all things that the Android plugin is able to do. Although it is not completely necessary to implement these (the user can start it manually) it would be nice to provide it to the user as a convenience.


To continue my discussion from last week in regards to the Invoker and dispatching Commands asynchronously on threads – it has been ridiculously easy to implement this. Instead of going through and creating our own WorkQueue and thread management classes I’ve implemented NSOperationQueue and NSInvocationOperation inside of the Invoker class. NSOperationQueue is provided by Apple and has all the functions to properly manage thread execution on a work queue and NSInvocationOperation allows us to specify the target and selector that the threads started from the work queue will invoke. Check out how easy it was to do here.

Cordova Lib

During this last week we ran into a big problem of everyone checking code into the repo that wasn’t compiling or causing problems. Yesterday I went through and fixed all the compilation problems. We’re planning to have a more strict review process in the future. One of the major problems that I encountered was the Cordova Lib not being properly compiled and linked to the project. This was caused because for some reason Cordova 2.0 doesn’t support armv6, so I had to remove that from the Cordova project as well as the iOS port project. The other big issue is that currently the Cordova library is not included in the repository. This means that when a fresh copy is cloned you have to go through the process of manually downloading PhoneGap and incorporating it into the project. So in the next day or so I’m going to set up the repo to include the compiled archive and point the project to it by default.

Java to iOS

The other major problem that I ran into this week was specific to the port from Java to iOS. In the Java version the concrete commands reside as inner classes in the DevicePlugin class. Due to this they all have access to the BluetoothServer stored within the DevicePlugin class. Moving over to iOS we cannot use inner classes and so the commands need to be provided with access to the server in some other way. To compensate this I’ve introduced a new command, BluetoothCommand, which sits between the concrete commands and the ReturnableCommand. BluetoothCommand exposes the base plugin in the ReturnableCommand as a MedicalDevicePlugin as well as exposing a property on it to access the BluetoothServer on the MedicalDevicePlugin. You can check out that here. Now that I’m thinking about it a BluetoothCommand doesn’t necessarily need to work with a MedicalDevicePlugin… what we’d need to do then in order to abstract it more is to create a base class like BluetoothPlugin that will expose a BluetoothServer property. I don’t really know if we should do this though as I’ve read a lot recently about the dangers of over designing code. I’ll post back next week about what comes from this.

Next Week

By the end of next week I’m hoping that we have most of the pieces in place to start testing. The PacketParser will be done and the few other commands that are left to be implemented will be done. Personally I will be working on some of the outstanding commands such as the listen and isSupported() commands.

Connected Wellness Dev Update

We’ve gotten a lot done in the last week for the iOS port of the Bluetooth Plugin. You can check out my work here.

  • I’ve finished the MedicalDevicePlugin class.
  • Wrote the Command Protocol.
  • Wrote a template for the ReturnableCommand class.
  • I started on the WorkQueue class, but Carl told me to shift from that to the Invoker class because WorkQueue will be removed sometime in the near future.
  • I also wrote the Invoker class but left out the bulk of the implementation as it uses the WorkQueue, so there is no point to code it for now as I don’t know what kind of design we will be using.
  • I’ve also pushed a lot of bug fixes to the repo.

Objective-c vs. Java

There were a lot of changes going to iOS from Android. The biggest one was that a lot of the architecture and functions of the Cordova Library in iOS are different than in Android. They still follow the same idea, but are different in implementation. Most of the functions in Android have an iOS equivalent. One of the ones that do not is onActivityResult() in the DevicePlugin implementation. I’ve talked with Carl about this though and it seems that it won’t be that big of a deal since we are planning to remove that function in the future. It seems to me that the Cordova Library for Objective-c is a lot more immature then the Java one. You can see this in the fact that going from version 2.0 (the one that we are using) to 2.3, most of the Objective-c code changes, but almost nothing from the Java code does. This indicates that there is still a lot of work to be done on the Objective-c library. I’m not complaining, but the Java Cordova Library just looks so much nicer.

Another thing that is a lot different are language specific things. For example in Objective-c you cannot specify the visibility level (i.e. public, protected, private) on member functions, and you can’t specify static member variables in exactly the same way as Java. This isn’t a big deal, but it is one of the many things that must change from going to Java to iOS. Another, rather concerning, thing that is different is that Objective-c offers no way to synchronize entire functions. This is a problem because this design paradigm is used a lot in the Java plugin. We will have to find a way around it in Objective-c.

Other Work

  • Caitlin has currently been tackling the BluetoothServer class as well as doing the full implementation of the ReturnableCommand class.
  • Vince has been working on the PacketParser implementation. I haven’t heard much from him this week, but I hope he has been making good progress.

Moving Forward

After Caitlin gets the ReturnableCommand landed we will be able to start implementing the concrete Command objects that inherit from ReturnableCommand and begin to tie the commands into the Invoker as well as the MedicalDevicePlugin class. I’ve taken on writing the isSupported(), isEnabled(), and isDisabled() commands. Everything is looking good so far baring the problem we may have with the Bluetooth Protocols…


Recently heard from Carl that I should be implementing threads in Objective-c in the Invoker class. It’s up to me how we get that done. So I will be working on that before continuing the Command implementations.