QUnit, JSCoverage and Jenkins

24. September 2013 10:09 by Cameron in Continuous Integration, Jenkins, PhantomJS  //  Tags: , , , , , , ,   //   Comments

A few years back I worked with a colleague of mine on getting JSCoverage integrated with QUnit to generate code coverage reports for QUnit. The process was fairly simple. We learned a ton about how to setup JSCoverage from this blog: http://www.eviltester.com/index.php/2008/06/08/test-driven-javascript-code-coverage-using-jscoverage/ You can either build the sources from JSCoverage's Subversion repository or if you are running Ubuntu, you can use the apt package manager to install JSCoverage. After obtaining JSCoverage, we were able to generate html coverage reports from JSCoverage for our unit tests that we wrote using QUnit, a unit testing framework for jQuery.

To generate coverage with JSCoverage, we used the command:

jscoverage [source_dir] [destination_dir]

Understand that the all files in the source directory will be copied to the destination directory and will be instrumented. We had to be careful not to instrument jQuery or QUnit so we put our tests in a separate folder. 

After that command has been run, one can browse to jscoverage.html and run the tests through the browser. JSCoverage also has a nice API that can accept querystrings to open test pages that will intrument the javascript. JSCoverage provides a summary for all instrumented files in the Summary tab and with each file in the summary, one can view a detailed breakdown of the coverage. 

As far as integrating this into Jenkins, there are a number of possible options, but it is not critical since JSCoverage provides its own reporting. Integration with Jenkins might be possible in the future however.

When you are working with JavaScript it's great to do unit testing, but integrating these tests into an automated system can be tricky.  Your code is made to be run in a browser, not on the command prompt.  Furthermore, you are probably manipulating the DOM and are dependant on HTML pages, so how could you automate something like this without opening up a browser? 

Enter PhantomJS and QUnit. 

PhantomJS is a headless browser with JavaScript API that will print any console.log() call to the command line. 

QUnit is a unit testing framework from the guys at JQuery.  It is made to display the results of your tests in a browser window.  Fortunately, QUnit provides callbacks that allow us to hook into the tests and provide proper output to PhantomJS.  In our case, this format is JUnit XML format. 

I followed someone who was doing something similar, but unfortunately his code was incomplete and also weird/broken.  Niltz's blog is: http://www.niltzdesigns.com Unfortunately Niltz's website is no longer online.

He did provide a nice .js file that hooked into QUnit's callbacks and stored the tests in a nice JS object.  From there, I edited his QUnit result printer to be able to output the correct XML.  These files were working great and they did their work without changing the way that our QUnit tests displayed when we viewed the tests manually in a browser.

The real trouble was the driver js file.  This Niltz guy had a driver, but it didn't properly use PhantomJS' interface and broke on execution.  I was able to use some of his code as a guide, but I was forced to almost completely re-write my own driver file. 

var page = require('webpage');

(function() {
    page.onConsoleMessage = function (msg) { 
        if(msg === 'phantomexit')
            phantom.exit(0);
        console.log(msg);
    };

    page.onLoadFinished = function(status){
        var PAGE_FAILED = 1;
        var TEST_FAILED = 2;
        var EXCEPTION = 3;

        if (status === 'success') {
            page.evaluate(function() {
                var testDone = function() {
                    var r = QUnitLogger.results;
                    var msg = qunitResultPrinter.print();
                    console.log(msg);
                    console.log('phantomexit');
                };

                if (QUnitLogger.done) {
                    return testDone();

                } else {
                    QUnitLogger.addEventListener('done', testDone);
                }
            });
        } else {
            console.log('Page failed to load: ' + phantom.args[0]);
            phantom.exit(PAGE_FAILED);
        }
    };

    try {
        if (1 !== phantom.args.length) {
            console.log('Usage: phantomjs-driver.js URL');
            phantom.exit();
        } else {
            phantom.state = 'phantomjs-driver-running';
            page.open(phantom.args[0]);
        }
    } catch(e) {
        console.log('Unhandled exception: ' + e);
        phantom.exit(EXCEPTION);
    }
})();

The tricky part here is the page.evaluate() function.  This allows me to execute code within the context of the loaded page, but the problem here is that it is run in a completely different scope, unaware of the variable 'status' for example.  The only way I can communicate with the code inside this function is either by a return or a console.log().  PhantomJS provides a nice callback for any console.log() inside of this phantom.evaluate() function called page.onConsoleMessage().

Using this we were able to fully integrate with QUnit and output the results of my tests in an XML format using the command:

./phantomjs phantomjs-qunit-driver.js ../test/index.html

An example of output is:

<testsuites><testsuite name="utilities" assertions="32" failures="0" time="38"><testcase name="setupValItems and cleanupValItems" assertions="5" time="20">....

From there, integration into Jenkins is fairly trivial.

Attached is the modified QUnit PhantomJS driver. I have left the tests out since your unit tests will be different.

qunit.zip (3.37 mb)

Month List

Tag cloud