Tagged: github

Jekyll and Hyde

I’ve been fiddling around lately with getting a new blog setup. I’ve asked around a lot and it seems like a super popular option for devs lately has been hosting on GitHub and using Jekyll to build your blog. Dale who worked on WEBVTT with me has a pretty sweet set up using this method.

Going with that I decided to set up my new blog this weekend. It would be easy I told myself, ever the optimist. You think I would learn after programming for as long as I have..

Installing the basic setup that you need to start writing blogs with Jekyll was super easy. Just install Ruby and RubyGems with MacPorts (if your on Mac) and install Jekyll with RubyGems.

sudo port install ruby
sudo port install rb-rubygems
sudo gem install jekyll

My real troubles began when I tried to use Jekyll’s blog import tool to migrate my WordPress posts over to my new blog. To do this you need the jekyll-import command. For the longest time I was having trouble installing this because ‘jekyll-import’ isn’t a full out gem yet (according to official documentation it is…) it’s a beta gem. This means you have to install it using the –pre option.

sudo gem install jekyll-import --pre

Without this it will tell you it “can’t find the gem ‘jekyll-import'”, but that you “may have been looking to install the gem jekyll-import'”… confusing, I know.

Great! Now it could find the gem when I went to install it. However now I had a new problem — it couldn’t find libiconv. This was weird because I had installed libiconv with MacPorts. It was only until I went to check the mkmf.log that the gem had created that I realized what had been happening (previously I had just been relying on what was being spit out on the command line, like a noob). In the log I found out that the libiconv I had wasn’t the architecture that was needed. I had x86_64, but I needed i386. Solving this was super easy. Just install the port variant of the libiconv library that is a universal library, i.e. you can use it when compiling for i386 and x86_64.

sudo port install libiconv +universal

The last problem I had with installing after this was getting the correct kind version of ruby. Jekyll-import needs ruby 2.0, I had 1.92. I had difficulty setting this up as I had multiple versions on my Mac including one that shipped with OS X. What you need to do is to install the ruby 2.0 port and then activate it.

sudo port install ruby20
sudo port select --list ruby (to get all ruby versions)
sudo port select ruby ruby20 (to select ruby 2.0)

Finally, jekyll-import installed and I tried to migrate my WordPress blog posts. I ran into one more problem though and this turned out to be a real bug. The import command was incorrectly trying to interpret the input of the command line arguments as a string instead of as a hash, which was what it needed. I filed the bug on the project’s GitHub page and someone solved it quickly! I haven’t had time to check to see if the gem has been updated yet so I don’t know if it’s completely working yet. Will post back when I find out.

So the moral of the story here is always check the log file when you encounter problems. It’s happened to me before where the log file has the correct information and the command line isn’t very helpful. Doing this will save you tons of time.

After all that I decided to set up my new blog with at least a ‘coming soon’ sign. So it’s up! You can check it out here. All that’s left is to finish my blog migration and add cool things like comment tracking and twitter feeds. I’d also like the user to be able to change the theme in real time. That would be cool. I also got to get some DNS server hosting for my brand spanking new domain http://rickeyre.ca. After that all will be good.

Until next time.


WebVTT 0.1 Release Update

It has been about two weeks now since my class and I set out to start work on the 0.1 release of WebVTT for FireFox. We are now nearing our deadline and are in the final stages of peer reviewing each others work. You can check out that action on our main GitHub repository.

During the development of this 0.1 release I learnt a lot of things:

  • How to dual boot a linux install properly
  • Continued to get better at using GIT
  • Learnt the basics of Python
  • Learnt about Makefiles and how ridiculously confusing they are

As we were developing the test suite for WebVTT, which will be the bulk of this 0.1 release, we had to address many different questions about the structure and standards we would be following:

  • What would the naming convention be for our test files?
  • What would the content of our test files look like?
  • How would our test harness function?
  • What would the Makefile need to build?
  • How would we keep the integrity of line endings in our test files as we would need to be testing LF, CR, and CRLF?

We eventually answered all of these questions leaving us with a pretty robust and clean test suite:

  1. Our Professor made a test harness written in Python that would take all the tests that we wrote and feed them through the Node.js WebVTT paraser module. This would give us a sanity check to confirm that the tests we write are good before we write our custom WebVTT parser in C++ for FireFox.
  2. We added in a .gitattributes file that specifies not to convert line endings on our test files:
    ./test/spec/good/*.test -text
    ./test/spec/bad/*.test -text
    ./test/spec/known-good/*.test -text
    ./test/spec/known-bad/*.test -text
  3. We decided to document all our tests on our wiki and standardize the naming of our tests using the format of tc###-short_information_block_here.test. You can see our wiki page on the naming convention here. We also decided to create a custom .test file format that would contain two parts, a comment section at the top and a WebVTT section at the bottom. Here is one of the .test files that I wrote:
    This tests to make sure that a Cue Component class can be resolved with the [cue component].[subclass] notation. 
    This test should pass.
    Based on the WebVTT cue components specification as of October 3, 2012.
    00:11.000 --> 00:13.000
    <u.class.subclass>Hey this is a test!

    We decided on this format as it allowed us to keep the metadata right with the test. Putting it directly in the test file will make it easier to work with in the future as you won’t have to refer back to another document to find the metadata of the test file.

  4. Now that we had a custom .test file we needed to parse it before we ran it through the Node module in order to rip out the WebVTT section. In order to address this issue I wrote a custom test file parser in Python and changed the Makefile to run it before running the test harness. We were running this configuration for a while when my Professor told me that rather than the Python script looping through the .test directories and ripping the WebVTT, the Makefile should determine what tests files need to be ripped and call the Python script for each individual test file. In accordance with this I spent a lot of time working with our Makefile trying to figure out how to get it to run the way that we wanted it too. Through this I learnt a lot about Makefiles (I will do a blog post later to talk about this in detail) and after much struggle, and with the help of one of my class mates as well as my Professor, we got it working. In order to implement this correctly we had to add a few lines to the Makefile:
    SRC_DIR = .
    TEST_DIR = $(SRC_DIR)/test
    SPEC_DIR = $(TEST_DIR)/spec
    OBJ_DIR = $(SRC_DIR)/objdir
    OBJ_DIR_SPEC = $(OBJ_DIR)/test/spec
    # Get all the .test files underneath the directory specified by $(SPEC_DIR)
    TEST_SRC := $(shell find $(SPEC_DIR) -name '*.test' -print)
    # Transform all .test files rooted in ./test to .vtt rooted in
    # .objdir/test
    VTT_SRC := $(subst $(SRC_DIR)/test,$(OBJ_DIR)/test,$(subst .test,.vtt,$(TEST_SRC)))
    STIP_VTT = $(SPEC_DIR)/strip-vtt.py
    	mkdir $(OBJ_DIR)
    check-js: objdir $(VTT_SRC)
    	$(PYTHON) ./test/spec/run-tests-js.py $(OBJ_DIR_SPEC)
    $(OBJ_DIR)/%.vtt : $(SRC_DIR)/%.test
    	@$(PYTHON) $(STIP_VTT) $< $@

    Now when we run the command ‘make check-js’ (check-js denotes that we want the Node.js WebVTT module to run) the Makefile will make the object directory, where the ripped .vtt files will live, call the script to rip the WebVTT out of the test files that have changed since the last build, and then run the test harness.This is much cleaner than the first solution where the Python script to rip the WebVTT just ripped every .test file each time it ran without checking if it actually needed to rip it.

Hopefully, we will be able to get through our peer reviews soon and get started on our next release, where I hope we will begin working on the actual parser and hook for FireFox.