Testing

Mobile Web Testing Using Nightwatchjs Part 1 (iOS)

Nightwatch is a Node.js based framework that specializes in End-to-End (E2E) tests. The default repository includes steps for most desktop browsers and using the Selenium Server comes standard, but mobile requires custom modifications, complex config changes and further investigations. It also requires other libraries like Appium. Our goal is to build onto Nightwatch so current and future projects can easily wire up browser based mobile tests against one test-set. Also to verify our initial beliefs that Nightwatch is the right solution for all browser based apps and websites.

Let’s face it, UI testing mobile web apps is a major pain for most teams and historically never works as expected. So your first question is probably why Nightwatch for mobile testing? To start, it is a simple yet powerful solution for standard desktop testing, that emphasizes readability and recycled code. With the Page Object Model now built in, Nightwatch is using the default best practices of most professional Selenium Engineers.

Ever since I came across examples of Nightwatch porting over its test codebase to mobile services like Appium, simply by adding to the config, I have had an interest how this would work.

One test set for all browsers?

Theoretically you could write one test-set for all of your browsers and operating systems. Unfortunately, the limitation we cannot change is in the design of the application being tested. If, for example, you have a different view (as many do these days) for mobile vs desktop, then you might have to break out those differences into separate test groups.

To put all Nightwatch testing into perspective, you are testing against the User Interface (UI) and not the code base. If your code base compiles the html to display different id’s, classes etc. or the actual browser and/or Operating System changes the output to client, then your tests will need to account for this.

Many professional software companies offer cloud services to test on all devices, sometimes doing a good job, but more often tests fail and maintenance is very time consuming on the customers end. The promise and sales pitch end up being all hype, just like when Uncle Rico would boast on Napoleon Dynamite.

uncle-rico

Our goal to solve this is not to just design on one framework like Nightwatch, but also adopting as many best practices, so WHEN we need to make changes, it can be tweaked or even ported to other frameworks if needed without a full rewrite. One of the more exciting additions to the test automation world, is the adoption of Page Object Model. POM as it is commonly called, is a Design Pattern used in test automation for reducing code duplication. This pattern also in turn improves test maintenance through better organization of your toolset; the selectors and recycled methods. We will share in detail what we learned using POM in a future blog.

Prerequisites & Environment Set-up:

Since the focus is on iOS, the following steps will require development on a Mac OSX system. We will also need strong>nodejs/npm, Xcode and Java installed. Any other requirements will be mentioned inline as part of the process. The easiest way to verify your environment is setup appropriately before moving on, Appium has created an outstanding product called appium-doctor. Let’s install it with npm. 
1
> npm install appium-doctor -g

Once installed you can run your environment tests.

1
2
> appium-doctor –ios
> appium-doctor –dev

Follow the instructions and make fixes until you pass all tests. Since we are testing iOS, if any Android issues are found, it’s ok to ignore those for now. Once all is passing, your environment for Nightwatch Mobile Testing is golden!

Install NightWatchjs

Before we can use Nightwatch for Mobile testing, let us install the repo. 
1
2
> git clone https://github.com/nightwatchjs/nightwatch
> cd nightwatch && npm install

If there are any errors, you might need to check for either node or proxy issues. I happened to use node v6.2.2 and use n (similar to nvm) to manage or change my node version if needed.

Install and configure Selenium Standalone Server

You can skip this part and jump strait to the iOS example if you are familiar with NightWatch web testing, but it is a good refresher and way to confirm all standard UI testing works.

Nightwatch works with the Selenium standalone server so the first thing you need to do is download the latest selenium server jar file (selenium-server-standalone-2.x.x.jar) from the Selenium releases page: <a href=”https://selenium-release.storage.googleapis.com/index.html“ target=”_blank” rel=”noopener”>selenium-release.storage.googleapis.com and add to your repositories nightwatch/bin directory. I used 2.53.1 and here is the easiest way. While still in the root of your repository:

1
2
> cd bin
> curl -O https://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.1.jar

Edit your config file

Now you need to edit your nightwatch/bin/nightwatch.json file and take a look at the selenium declaration. Add your new jar file path as the server_path and set the start_process to true. That allows Nightwatch to start the server behind the scenes. 
1
2
3
4
5
6
"selenium" : {
  "start_process" : true,
  "server_path" : "bin/selenium-server-standalone-2.53.1.jar",
  "log_path" : "",
  "host" : "127.0.0.1",
  "port" : 4444,

By default NightWatchjs is testing against Firefox on a desktop so for now we simply need to start the application and point to our test. Next run the Page Object Model against google like so…

1
2
> cd ../ # back to nightwatch repo root
> ./bin/nightwatch -t examples/tests/googlePageObject.js

You should see Firefox open and test against the google website. If you did, congratulations! There are many ways to test other desktop browsers, like adding a chromedriver for chrome, changing config to Safari or adding IE drivers; but our focus is not on web browser testing, so let’s move on.

iOS Safari

The easiest, most reliable option for iOS testing of a web app is with Appium as your test server in place of the default Selenium Standalone. Appium can be installed globally or in your local npm packages.

Install and start Appium

Open a different terminal so you can keep Appium running. 
1
2
> npm install -g appium
> appium &; #start Appium

If you have any troubles getting the server running, Appium has an app to fix any path issues. That app can be found at
<a href=”http://appium.io/“ target=”_blank” rel=”noopener”>appium.io. Once installed, open the app and start the server for iOS. Only start if you did not start from CLI steps above, or you will need to kill the process.

Add your iOS env to nightwatch.json

Open the bin/nightwatch.json and add this ios block just before saucelabs block. 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
"ios" : {
  "selenium_start_process": false,
  "selenium_port" : 4723,
  "selenium_host" : "127.0.0.1",
  "silent": true,
  "desiredCapabilities" : {
    "browserName" : "Safari",
    "platformName" : "iOS",
    "platformVersion" : "9.3",
    "deviceName" : "iPhone 6s Plus"
  }
},

"saucelabs" : {

You will just need to modify your platformVersion and your deviceName based on what simulators you have installed in Xcode, if any. I installed the latest iphone 6s Plus on the 9.3 platform.

  • To install, you will need to open:
  • Xcode > Open Developer Tool > Simulator
  • in Simulator go to:
  • Hardware > Device > Manage Devices
  • in devices window At the bottom of the left column, click the Add button (+)
  • Now follow the dialog.
When done save your device and platform (Simulator runtime) names, if the device and platform are different than what is listed above make sure to add the proper values to the config. The tests will fail if these are not correct.

Lets run your mobile iOS tests in the terminal

While back in your original terminal, in the nightwatch repository root: 
1
> bin/nightwatch --env ios -t examples/tests/googlePageObject.js
Notice the only difference, we added --env ios to override the default settings and expose our ios setting. Run it and you should see the Simulator open, then run through all of your tests.

Summary

  • You now have a working example of Nightwatch using JSON as it’s config. Next level of expertise can be adding this to your grunt config or a custom javascript file.
  • Your suite is already using Page Object Models, so you can go over how this is done and expand on this very powerful pattern.
  • Besides web, you now know the basics of Appium and iOS and can dive deeper into these new options.
This will hopefully be the first of many blogs on Nightwatch mobile testing, so please offer suggestions on topics you would like to see in the comments. Not only are we very excited about finding out the capabilities of this framework, but also contributing to its repo and helping grow it into a full featured test-suite. In fact, our team has already contributed to the project. Our next topics in Part 2 will include Android mobile web testing with Nightwatch against the Chrome browser.
Share