Controlling a browser through Node.js

I've had an idea in my head for a few years now, that I recently decided to explore. To get started one of the first steps I had to do was to get a basic program that controls a browser.

At work we're doing this using C#, Selenium and ChromeDriver. However I wanted to implement it in Node.

My initial thought was to use PhantomJS and CasperJS. I decided not to as CasperJS requires you to run the casperjs binary and inject a control script as an argument.

I wanted this to be a self contained application. Still using a headless browser. After looking around I decided to go for a selenium webdriver setup.

This is how I set it up.

The webdriver.io team pointed out that if you just want to write tests they have a nifty tool that help you set everything up with their test runner. See their Getting started for more info.

Getting Selenium

Most selenium based solutions i found need you to download, install and run Selenium. I just want independent executable that would control a browser. Starting the binary should just get things "working".

So after some searching I found the selenium-standalone npm package.

The package will take care of downloading and setting up a selenium environment for you. You specify which browsers you want to control and selenium-standalone takes care of downloading the drivers and jar files.

You do however still need to have Java installed.

I created a small file that wraps my options and selenium-standalone in a promise based api.

var selenium = require('selenium-standalone');
var extend = require('extend');

const install = ()=> new Promise((resolve, reject) => {
      console.log('installing');
      selenium.install(extend(selenium_options, {
        progressCb: function(totalLength, progressLength, chunkLength) {
          console.log(progressLength/totalLength*100);
        }
      }), function (err) {
        if(err){
          reject(err);
        }
        console.log('install complete');
        resolve();
      });
});

The installation script wont install selenium if the desired configuration is already installed.

The start method is similarly implemented based on the examples of their github page. My start promise returns the child process so I can kill off Selenium when I'm done with it.

Now I can specify in my options which browsers I want to run and control through selenium.

PhantomJS and selenium-standalone

It was a little harder to get PhantomJS to run through selenium-standalone. There is little documentation on how to get selenium to control PhantomJS.

In the end I got the following to work.

I installed PhantomJs through the phantomjs-prebuilt package. This allows me again to have PhantomJs available to me in my node_modules directory.

npm install -S phantomjs-prebuilt

I then point to my specific PhantomJs using the options passed to selenium-standalone.

var selenium_options = {
  version: '2.53.0', //Important!
  baseURL: 'https://selenium-release.storage.googleapis.com',
  drivers: {
    phantomjs: {
      binary: {
        path:'./node_modules/phantomjs-prebuilt/bin/phantomjs'
      }
    }
  }
}

These options points the Selenium to use the binary for PhantomJs in my node_modules.

Also the examples on the selenium-standalone page use an older version of selenium. You need a newer version of Selenium to control PhantomJS.

Actually controlling the browser

At this point I can install selenium, PhantomJs and get things up and running. Now the last step is to control the browser.

To do this I use webdriverio. Once I have started selenium configured to run PhantomJS I just need to setup a webdriver remote.

const create_driver = ()=>
  new Promise((resolve, reject)=>{
    const webdriverio = require('webdriverio');

    var driver = webdriverio.remote({
      desiredCapabilities: {
        browserName: 'phantomjs'
      }
    }).init().url(options.url).then(d=>resolve(driver))
});

Running that function will return a promise that gives you a webdriver to control the browser.

Combining the functions

You can then chain the promises to get a neat control-flow

install().then(start_selenium).then(create_driver).then((driver)=>{
  // you now have a browser up and running linked to the driver
});