It has been about two weeks now since my class and I set out to start work on the 0.1 release of WebVTT for FireFox. We are now nearing our deadline and are in the final stages of peer reviewing each others work. You can check out that action on our main GitHub repository.
During the development of this 0.1 release I learnt a lot of things:
- How to dual boot a linux install properly
- Continued to get better at using GIT
- Learnt the basics of Python
- Learnt about Makefiles and how ridiculously confusing they are
As we were developing the test suite for WebVTT, which will be the bulk of this 0.1 release, we had to address many different questions about the structure and standards we would be following:
- What would the naming convention be for our test files?
- What would the content of our test files look like?
- How would our test harness function?
- What would the Makefile need to build?
- How would we keep the integrity of line endings in our test files as we would need to be testing LF, CR, and CRLF?
We eventually answered all of these questions leaving us with a pretty robust and clean test suite:
- Our Professor made a test harness written in Python that would take all the tests that we wrote and feed them through the Node.js WebVTT paraser module. This would give us a sanity check to confirm that the tests we write are good before we write our custom WebVTT parser in C++ for FireFox.
- We added in a .gitattributes file that specifies not to convert line endings on our test files:
./test/spec/good/*.test -text ./test/spec/bad/*.test -text ./test/spec/known-good/*.test -text ./test/spec/known-bad/*.test -text
- We decided to document all our tests on our wiki and standardize the naming of our tests using the format of tc###-short_information_block_here.test. You can see our wiki page on the naming convention here. We also decided to create a custom .test file format that would contain two parts, a comment section at the top and a WebVTT section at the bottom. Here is one of the .test files that I wrote:
/* This tests to make sure that a Cue Component class can be resolved with the [cue component].[subclass] notation. This test should pass. Based on the WebVTT cue components specification as of October 3, 2012. http://dev.w3.org/html5/webvtt/#webvtt-cue-span-start-tag */ WEBVTT 00:11.000 --> 00:13.000 <u.class.subclass>Hey this is a test!
We decided on this format as it allowed us to keep the metadata right with the test. Putting it directly in the test file will make it easier to work with in the future as you won’t have to refer back to another document to find the metadata of the test file.
- Now that we had a custom .test file we needed to parse it before we ran it through the Node module in order to rip out the WebVTT section. In order to address this issue I wrote a custom test file parser in Python and changed the Makefile to run it before running the test harness. We were running this configuration for a while when my Professor told me that rather than the Python script looping through the .test directories and ripping the WebVTT, the Makefile should determine what tests files need to be ripped and call the Python script for each individual test file. In accordance with this I spent a lot of time working with our Makefile trying to figure out how to get it to run the way that we wanted it too. Through this I learnt a lot about Makefiles (I will do a blog post later to talk about this in detail) and after much struggle, and with the help of one of my class mates as well as my Professor, we got it working. In order to implement this correctly we had to add a few lines to the Makefile:
SRC_DIR = . TEST_DIR = $(SRC_DIR)/test SPEC_DIR = $(TEST_DIR)/spec OBJ_DIR = $(SRC_DIR)/objdir OBJ_DIR_SPEC = $(OBJ_DIR)/test/spec # Get all the .test files underneath the directory specified by $(SPEC_DIR) TEST_SRC := $(shell find $(SPEC_DIR) -name '*.test' -print) # Transform all .test files rooted in ./test to .vtt rooted in # .objdir/test VTT_SRC := $(subst $(SRC_DIR)/test,$(OBJ_DIR)/test,$(subst .test,.vtt,$(TEST_SRC))) STIP_VTT = $(SPEC_DIR)/strip-vtt.py objdir: mkdir $(OBJ_DIR) check-js: objdir $(VTT_SRC) $(PYTHON) ./test/spec/run-tests-js.py $(OBJ_DIR_SPEC) $(OBJ_DIR)/%.vtt : $(SRC_DIR)/%.test @$(PYTHON) $(STIP_VTT) $< $@
Now when we run the command ‘make check-js’ (check-js denotes that we want the Node.js WebVTT module to run) the Makefile will make the object directory, where the ripped .vtt files will live, call the script to rip the WebVTT out of the test files that have changed since the last build, and then run the test harness.This is much cleaner than the first solution where the Python script to rip the WebVTT just ripped every .test file each time it ran without checking if it actually needed to rip it.
Hopefully, we will be able to get through our peer reviews soon and get started on our next release, where I hope we will begin working on the actual parser and hook for FireFox.