Making Your Appium Tests Fast and Reliable, Part 3: Waiting for App States

This article is the third in a multi-part series on test speed and reliability, inspired by a webinar I gave recently on the same subject (you can watch the webinar here). You might also want to check out Part 1: Test Flakiness and Part 2: Finding Elements Reliably.

Using bad locator strategies is one cause of elements in your app not being found, and contributing to flakiness and unreliability. But even when you're using the right locator strategies and are guaranteed to find the same element every time you look, that doesn't mean the element will be there right when you look! What happens when you try to find an element which just isn't there? This is a remarkably common issue, and it's part of a more general phenomenon in functional tests: race conditions due to poor assumptions about app state.

Check out: A Complete Guide to Functional Testing

Race Conditions

A race condition is when two processes or procedures are operating simultaneously, and either one could finish before the other. This is a problem in automated test scripts because our code usually handles only one scenario. So, in the alternative world where the unintuitive procedure finishes first, our test will fail.

To use our example of finding elements: our code usually implicitly assumes that the element will be present before we try to find it. In reality, the request to find an element and the app's own process of working to display the element are in a race. As human users of the app, we know how to gracefully lose the race: we simply wait! If we want to tap a button and it's not yet on the screen, we wait for it to show up (for a few seconds, anyway, then we get bored and open Twitter instead).

Also check: Improving mobile user experience through SDK abstraction

Appium is less graceful and does exactly what the client tells it to do, even if that means trying to find an element before it has been properly rendered on the screen. This is just one example of many possible examples in the category of test code assuming the app is in a certain state but being proven wrong. It can be a particularly vexing problem because it might only show up infrequently, or only in CI environments. When we develop test code locally, we can often be tricked into making all kinds of assumptions about how races will resolve. Just because the app always wins a race (like we expect) when testing locally does not mean the app will behave the same in other environments.

Read: Benefits of Automated Testing in App Development

Waiting for App States

The solution is to teach our test code to gain a bit of the grace of the human loser, and not blow up with exceptions just because the state wasn't what it expected. There are three basic ways to wait in your Appium scripts.

Static Waits

Waiting "statically" just means applying a lot of good old Thread.sleep all over the place. This is the brute force solution to a race condition. Is your test script trying to find an element before it is present? Force your test to slow down by adding a static sleep! Taking the login test, which is very familiar to our readers, this is what it would look like using static waits:

@Test
public void testLogin_StaticWait() throws InterruptedException {
    Thread.sleep(3000);
    driver.findElement(loginScreen).click();

    Thread.sleep(3000);
    driver.findElement(username).sendKeys(AUTH_USER);
    driver.findElement(password).sendKeys(AUTH_PASS);
    driver.findElement(loginBtn).click();

    Thread.sleep(3000);
    driver.findElement(getLoggedInBy(AUTH_USER));
}

I chose 3 seconds as my static wait amount. Why did I choose that value? I'm not sure. It worked for me locally and solved my race condition problems. Good enough, right? Not exactly. There are some major problems with this approach:

• My test is now much longer than it needs to be (up to 9 seconds longer!), wasting my time and my build's time and therefore my team's time and my company's time. And time is money, friend! Oops.
• I have staved off the chaos of a race condition... for now. Who's to say I won't wind up in some other scenario where 3 seconds won't be enough? What if one of the elements shows up based on a network request, and every so often the network is just a bit slower? My only recourse would be to keep increasing the static wait, thereby making the problem above even worse. And meanwhile I still can't sleep at night.
• Unless I'm diligent at experimentation and commenting, no one will know why I picked the precise values that I did. Was it random or was there a reason?

So, what else can we do?

Also read: Prioritizing Automated Functional Test Scenarios

Implicit Waits

Because the designers of Selenium were well aware of the element finding race conditions, a long time ago they added the ability in the Selenium (and we copied it with the Appium) server for the client to set an "implicit wait timeout". This timeout is remembered by the server and used in any instance of element finding. If an element can't be found instantly, the server will keep trying to find it up to the specified timeout. The same test implemented with implicit waits would look like:

@Test
public void testLogin_ImplicitWait() {
    driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

    driver.findElement(loginScreen).click();
    driver.findElement(username).sendKeys(AUTH_USER);
    driver.findElement(password).sendKeys(AUTH_PASS);
    driver.findElement(loginBtn).click();
    driver.findElement(getLoggedInBy(AUTH_USER));
}

Wow! That code is a lot nicer, for one. We've also completely solved the problem about the ever-increasing waste of time we ran into with static waits. Because the server-side element-finding retry is on a pretty tight loop, we're guaranteed to find the element within (say) a second of when it actually shows up, meaning we waste very little time while simultaneously making our test much more robust.

There's still a problem or two with this approach, however:

• Using implicit wait, we tend to set one timeout and forget about it. Inevitably, this timeout becomes pretty high because it has to be high enough to account for the slowest element we could validly wait for. This means that for other elements, which we know would never take as long to show up, we still end up wasting time waiting for them. In other words, we still want our find element command to fail relatively quickly in the case where an element truly never makes an appearance. We don't want to wait for a whole minute to decide when a few seconds would have done.
• We've been focusing on waiting for elements, which is what implicit waits are designed around. But an element's presence is just one example of an app state that we might want to wait for. What about an element's text, or visibility? Implicit wait won't help us there.

Thankfully, there's an even more general solution that gets us past these (admittedly more minor) issues as well.

Explicit Waits

Explicit waits are just that: they make explicit what you are waiting for and how long it will take. At the cost of a little more verbosity, we get much more fine-grained control, and are able to teach our test script how to wait for just the right condition in our app before moving on. For example:

@Test
public void testLogin_ExplicitWait() {
    WebDriverWait wait = new WebDriverWait(driver, 10);

    wait.until(ExpectedConditions.presenceOfElementLocated(loginScreen)).click();
    wait.until(ExpectedConditions.presenceOfElementLocated(username)).sendKeys(AUTH_USER);
    wait.until(ExpectedConditions.presenceOfElementLocated(password)).sendKeys(AUTH_PASS);
    wait.until(ExpectedConditions.presenceOfElementLocated(loginBtn)).click();
    wait.until(ExpectedConditions.presenceOfElementLocated(getLoggedInBy(AUTH_USER)));
}

Here we use the WebDriverWait constructor to initialize a wait object with a certain timeout. We can reuse this object anytime we want the same timeout. We can configure different wait objects with different timeouts and use them for different kinds of waiting or elements. Then, we use the until method on the wait object in conjunction with something called an "expected condition".

An expected condition is simply a special method which returns an anonymous inner class whose apply method will be called periodically until it returns something. The ExpectedConditions class has a number of useful, pre-made condition methods. What's great about explicit waits, though, is that we're not limited to what comes in the box. We can make our own!

Custom Explicit Waits

If the app state we want to wait for is particularly complex, we can always make our own expected condition. For example, let's say that the click() command is terribly unreliable, and often it fails, even when our element is found. So, what we want is to keep retrying both the find and click actions until they both succeed one after the other. We could make a custom expected condition, like so:

private ExpectedCondition elementFoundAndClicked(By locator) {
    return new ExpectedCondition() {
        @Override
        public Boolean apply(WebDriver driver) {
            WebElement el = driver.findElement(locator);
            el.click();
            return true;
        }
    };
}

We simply return a new ExpectedCondition and override the apply method with our particular logic. Then we can use this in our test code, for example as in this revision of the previous test:

@Test
public void testLogin_CustomWait() {
    WebDriverWait wait = new WebDriverWait(driver, 10);

    wait.until(elementFoundAndClicked(loginScreen));
    wait.until(ExpectedConditions.presenceOfElementLocated(username)).sendKeys(AUTH_USER);
    wait.until(ExpectedConditions.presenceOfElementLocated(password)).sendKeys(AUTH_PASS);
    wait.until(elementFoundAndClicked(loginBtn));
    wait.until(ExpectedConditions.presenceOfElementLocated(getLoggedInBy(AUTH_USER)));
}

Granted, this is a bit of a useless example, but it demonstrates how easy it is to create useful and reusable waits that your whole team can use. And this completes our tour of strategies for waiting for app states with Appium. Don't forget to take a look at the full code for the examples shown here, including all the setup and teardown boilerplate! Interested in the next episode in this series on speed and reliability? Head on over to Part 4: Dealing with Unfindable Elements.