NightWatch has become my go-to API into Selenium recently for E2E tests, so I figured why not blog about it. I am learning as I go so please feel free to tactfully question anything here you see that sounds inaccurate or worth deeper discussion. I plan to discuss the reason why I chose NightWatch and in following blogs I will discuss installation/set-up and deeper dives on various techniques and design patterns.
Why NightWatchjs?
I asked this question many times, even discontinuing use of NightWatch when it was fairly new over a Python API into Selenium instead. Recently I came back to the project after I tried just about every way to control Selenium (or direct webdriver bindings) with JavaScript.
This was after looking at refactoring a testing suite for a nodejs project. With the developers being more versed in JS than Python, it only made sense to use JavaScript for testing. Clearly, using same core language is debatable especially for UI testing and believe me I fought that fight, but that is for another blog ;-).
So now that JavaScript was the target, I decided to test them all and some have matured considerably from a couple years ago. Without bashing every other testing framework/library, let me just list my requirements and my “nice to haves” that helped me decide on NightWatch as my go-to testing framework for nodejs projects.
Requirements (at least what I highly desired):
- Page Object Model (POM) or Page Object Design Pattern
- Non-Unit Tests (the required of existing project)
- Functional and E2E Tests
- Integration Tests
- Reports
- Test-Runner
Page Object Model
There are many test design considerations, but Page Object Model/Pattern (lets call it POM Pattern) and the ability for the framework to perform reliably under this pattern is very high on my list of requirements. One thing I found is tests can be written fast, and you can get a lot done without any pattern, but when the code base changes, those types of test suites become impossible to maintain.
With the POM Pattern, the first advantage, it is a living documentation. If you know the application, you can understand the testing code base. Also, it is DRY, you write each selector once. NightWatch has made it a standard feature as of 0.7. Another advantage, you can abstract away the underlying html actions, so writing actual tests is clean and fast.
Non-Unit Tests
Alright, without getting into a debate on all the types of tests and distilled down meaning, simply my requirement was to test a lot (if not all) of these variations to satisfy stakeholders, devs and my comfort in signing off on the release. A lot of opinions here and everyone’s opinion can be backed by very good reasons. Basically NightWatch had a diverse enough set of options/plugins that I could do all the common test patterns/levels. Also, If any of the testing practices needed to be altered because app changes where requested, I found that QA could handle it rapidly under NightWatch.
Reports
I have in the past created home-made reports purely in Python, or allowed the use of other software like TestRails through API’s or other integration libraries. So that was the original criteria, to integrate to something I have used or if I could write details to a file in JavaScript inline while Selenium returned results easily. Not as versed in JavaScript as I am in Python, that was scary because JavaScript can be tricky for those new to the language; more so to those using it to drive a browser.
When I was ready for a long investigation it turned out way easier, NightWatch has a plugin called nightwatch-html-reporter. This ended up being plug-n-play, then just editing config for your needs, it was simple and still very detailed. Done.
Test Runner
Since we use Grunt as a task runner, and it performed tests in that process, we wanted to integrate at least sanity tests into that and possibly allow for different options. NightWatch has grunt integration built in and allows a very extensive set of options,
Desirable’s:
- Unit Tests
- Acceptance Tests
- CI
I also had a few desirable’s and Unit testing was already handled so was not on my list. But with it capable of running mocha as of version 0.9 it can help consolidate your testing into one suite, so this became a great selling point for the dev’s doing unit tests already.
Acceptance Tests really fall under blackbox testing manually in my experience, but I really hate writing tests then manually doing it. I believe if NightWatch’s POM pattern is done well, you can write up some actual tests rapid enough to still be Agile.
For Code Integration (CI), I was happy to use any CI tool, but Jenkins was my first choice just because it was in place already. NightWatch had integration to Jenkins and many others.
Summary
So that is my high level of why I am hip to NightWatch. I also really liked that the syntax is clean and easy to maintain, especially if you incorporate a good design pattern like POM.
With all the positive, there was a few times I did rethink NightWatch even recently. Luckily I stuck it out to find it was all user-error why I had struggled. My first set of issues started by being stuck on node v0.10.36 when initially building and testing. For whatever reason, logging of errors was not as verbose in the older node versions, leading to issues with bad code, causing tests to fail. Also, I had some issues with an older Java, webdriver and jar file because I picked up from a deprecated project. It was not until pulling the latest project from GitHub did I start seeing how amazing and reliable NightWatch really is. So before giving up, make sure all of your environment stuff is on point! To be clear, once I got my tests dialed in, everything worked just the same on the older node version.
This was my first blog on NightWatch, the next will dive into some code! my first thought was to cover POM Pattern, but please offer any opinions or requests as well.