Webmasters Gallery

A common idiom in our industry is, “You never can predict how the user will use your product once they get it in their hands.” If you’ve ever watched a stakeholder use a website or web application for the first time, you may know this firsthand. I can’t count the number of times I’ve seen a user seemingly forget how to use websites on a mobile device, or try to use it in a way that makes you think, “But no one would actually do that in real life!”

The thing is, you never really do know what a user may do in the moment. They might be in a frantic state of mind, trying to accomplish something too quickly, and don’t tap or type the way a calm, focused user might. This fits right into the all-too-common scenario of designing and developing for the best case scenario, and not thinking about edge cases or “what happens if things don’t happen perfectly in this order?” As developers, we tend to build things thinking that everything will be understood, that the user is rational and should just know that tapping around too quickly might cause something weird to happen. It can even affect those who might make accidental taps or clicks when not giving an app full attention – how many times have you accidentally tapped on a mobile device when you were walking and talking while also trying to reply to a tweet or email.

Building out tools to help us test the unpredictable aren’t entirely new. In 2012, Netflix had open-sourced their internal service Chaos Monkey, which “terminates virtual machine instances and containers that run inside of your production environment.” In plain language, it’s a service that tears down servers at random to ensure an entire system doesn’t violently collapse during a failure. Our development communities also remind us to not just design for “the happy path”, but how can we actually detect for unpredicted points of failure in our interfaces the way we can with our server architectures?

If a hundred monkeys at typewriters can write the works of Shakespeare, then one monkey should surely be able to find bugs and problems in our interfaces.

Bring in the monkeys

Monkey testing is a method of testing that generates random user input – clicks, swipes, entering input – with the sole purpose of finding issues with, or entirely breaking, your application. Unlike unit and acceptance testing, where you are writing test cases that occur in a specific order or set of conditions, which ultimately creates bias in how your interface is tested. Developers have less control over how a monkey test will execute, and since they are random every time they are run, you’ll never be testing for just one scenario, but rather an infinite combination of interactions.

Although this type of testing is available for most technology stacks, things built for the web haven’t necessarily got there yet. For example, the Android SDK has a UI Exerciser Monkey that handles most interface-level and system-level events. As web developers have begun to think more critically about performance and stress testing, some of these ideas have finally made it over to the world of the web in the form of Gremlins.js, a JavaScript-based monkey testing library written by the team at Marmelab.

Monkeys, Gremlins, And Other Fun Critters

Gremlins.js runs with as little or as much control as you could need. It has a fairly low time cost for initial setup.

There are three ways to start using Gremlins.js.

Standalone library

The easiest way to incorporate the library is to include the library directly into your site. This will put gremlins into your project’s global namespace, meaning that you can access it from anywhere within your project.

<script src="path/to/gremlins.min.js"></script>

<script type="javascript">
  // You can also run this in any file now!
  gremlins.createHorde().unleash();
</script>

Require.js module

If you are using Require.js in your project, you can import Gremlins.js in a non-global, as-needed fashion.

require.config({
  paths: {
    gremlins: 'scripts/libraries/gremlins.min'
  }
});

require(['gremlins'], function(gremlins) {
  gremlins.createHorde().unleash();
});

Bookmarklet

If you prefer to use monkey testing in a ad hoc manner, there’s even a bookmarklet that allows one-click testing on whichever page you are on. You can grab the bookmarklet from the installation instructions.

Monkeying Around

If you’ve opted for direct inclusion of the library or importing it through Require, you can now start playing around with Gremlins on our own! In our installation examples, we call gremlins.createHorde().unleash() – so what is this doing?

gremlins           // Yo, Gremlins
  .createHorde()   // Create me a default horde
  .unleash();      // Then immediately release them

See the Pen Gremlins.js out of the box by Alicia Sedlock (@aliciasedlock) on CodePen.

The default horde includes all five available types of randomly generated user interactions, otherwise known as “species of gremlins”, that include:

• formFillerGremlin fills in inputs with data, clicks checkboxes/radio buttons, and interacts with other standard form elements
• clickerGremlin clicks anywhere on the visible document
• toucherGremlin touches anywhere on the visible document
• scrollerGremlin scrolls the viewport
• typerGremlin triggers random typing on a simulated keyboard

When triggered, gremlins will leave a visual indication on the screen for the action that was performed. They will also leave a log of the actions they took, found in the developer console, along with any additional data associated with that species. They’ll look something like the example below.

gremlin formFiller input 5 in <input type=​"number" name=​"age">​
gremlin formFiller input pzdoyzshh0k9@o8cpskdb73nmi.r7r in <input type=​"email" name=​"email">​
gremlin clicker    click at 1219 301
gremlin scroller   scroll to 100 25

By default, gremlins will be randomly triggered in 10 millisecond intervals for a total of 1000 times.

Alongside our “bad” gremlins who like to cause trouble, there are also helpful gremlins, called mogwais, available to us. They don’t interfere with our application like gremlins, and instead mostly do reporting on how our application is holding up, such as logging the current frame rate. Mogwais will throw errors if the frame rate drops below 10.

mogwai  fps  12.67
mogwai  fps  23.56
err > mogwai  fps  7.54 < err
mogwai  fps  15.76

The combination of gremlin and mogwai logging provides a great picture of everything that’s happened over the course of the test.

Without much of any customization at all, Gremlins.js gives us a pretty robust test right out of the box.

Advanced Monkeying Around

If after using the default configuration, you have needs not being met, there are a fair number of ways to customize the way things run. For example, perhaps you want to only focus on particular components on a page at a time, rather than always testing a page in its entirety.

Though we can’t scope all species of gremlins, we can limit toucher, clicker, and formFiller to scope certain areas of our page at a given time. Specifically, we ask a gremlin to check the parent of an element it attempts to target. If that element is within our scope, the gremlin will proceed with the action. Otherwise, the gremlin will reattempt to find an element to interact with. We can also tell the gremlin how many times we want them to try to attempt the action before giving up with maxNbTries.

gremlins.species.clicker().canClick(function(element) {
    return $(element).parents('.my-component').length;
    /**
       Is the element this gremlin attempted to click
       within our wanted scope? Let it proceed by
       returning true! Otherwise, tell it to try again
       by returning false.
    **/
}).maxNbTries(5); // Our gremlin will tolerate 5 false returns for canClick before it gives up and moves on

// Similarly, we can control the scope of formFiller and toucher.

gremlins.species.formFiller.canFillElement(/** do stuff here **/);
gremlins.species.toucher.canTouch(/** do stuff here **/);

See the Pen Gremlins.js out of the box by Alicia Sedlock (@aliciasedlock) on CodePen.

Custom Gremlins

If you’re feeling limited by the selection of gremlin species at your disposal, fear not! You can write your own custom gremlins to perform any other actions you may be expecting users to make. For example, do you want to check what happens if a user attempts to submit a form randomly at any given part of their experience? You can hope that clicker will randomly click on our submit button, or you can create a custom submission gremlin to increase our chances, as well as control of how it’s executed.

This requires a bit of understanding of how to create and customize DOM events in JavaScript, so let’s walk through the pieces involved in making a submission gremlin.

gremlins.createHorde() // first, create our horde 
  .allGremlins()      // and enable all gremlins
  .gremlin(function() {

    // Now let's define our submission gremlin
    var targetElement, availableForms;

    availableForms = document.querySelectorAll('form'); // Let's get all available form elements on the page
    targetElement = availableForms[Math.floor(Math.random()*availableForms.length)]; // Then let's grab a random element from those results

    // Now, we create a dummy submission event
    var evt = document.createEvent('HTMLEvents');  // Create an event container
    evt.initEvent('submit');  // Define our event as a submit event
    targetElement.dispatchEvent(evt);  // Finally, dispatch the submit event onto our randomly selected form

    // We also want to make sure to log this event like others gremlins do!      
    console.log('gremlin submit ', targetElement);
  })
  .unleash();

See the Pen Customized gremlin with Gremlins.js by Alicia Sedlock (@aliciasedlock) on CodePen.

If you’re looking for more guidance on creating custom events, checkout Mozilla Developer Network’s documentation on creating events, and definitely check out the source of how Gremlins.js creates its events (the clicker gremlin is a great place to start)

Seeding Hordes

Having a brand new horde execute every time you run this kind of test will help stress test your UI in a lot of different scenarios. However, if you unleash a horde, and end up with errors, how are you really supposed to know if you’ve fixed the issues that caused those errors?

For cases where you want to run the same horde multiple times, you can choose to seed the horde. This will give you the exact same horde every time you execute the test.

var horde = gremlins.createHorde();
horde.seed(1234);
horde.unleash();

You may want to weigh how often you uses seeded hordes versus randomized hordes in a long term solution. While seeded hordes allow for retesting, much of the benefit of monkey testing lies in its randomness, which becomes somewhat moot in a scenario where the same seed is always used.

Conclusion

The web community often talks about not making assumptions about our users. They may have slow connections, not-the-latest device line, or be impaired in a way that requires the sit their using to be accessible. We also can’t promise that five-year-old Jaime doesn’t grab their parent’s phone, start wildly tapping, and end up causing major issues with whatever thing they were using. Or Kassidy from marketing, after getting their hands on the product for the first time, feverishly clicking or tapping away in excitement. The order, speed, or repetition of any user action can’t be predicted. As developers, we owe it to our users to make sure that our services don’t become unexpectedly broken simply from expecting them to only take happy path actions.

The choices for client-side monkey testing are slim, but Gremlins.js gets the fundamentals down right out of the gate. As with any type of testing library or framework, we can only improve upon it if people use it! They are actively seeking contributors, so if you have a wishlist for things this library can do, let them know!

---

An Intro to Monkey Testing with Gremlins.js is a post from CSS-Tricks