Retrieving Clipboard Text from a Real iOS Device

In a recent Automation Happy Hour, I ran into a scenario where I needed to retrieve some information from an app that wasn't available in a text element on the screen.

Background

Specifically, I wanted to get the URL of a Zoom meeting I had started, so that I could scrape its ID and password to send to another device (as part of a multi-device test flow). As always, the first thing I did was ask, "how would a user get this URL?" Well, in this particular app, the most straightforward way to get that URL is to make use of this little menu that Zoom pops up when you try to invite someone to a meeting:

Tapping the "Copy URL" item would place the URL in the device's clipboard, so that the user can then paste it into a text message, e-mail, or whatever. Tapping this button is easy enough, but what about retrieving the data from the clipboard once it's placed there?

If we were working with an iOS simulator or an Android device, the solution would be simple: we could use driver.getClipboardText! But this method is not available for real iOS device. So once again, I asked the question: how would a user do this? How would a user see what was in the clipboard? The most obvious answer was: they would paste it somewhere, then read it with their eyes. Appium doesn't have eyes, but once text is on the screen, it can usually read the text as long as it can find the corresponding element. Read on to see how we implement this in code.

Pasting and Reading the Value

The main thing to decide when using this technique is: where should we paste the clipboard contents? We want a place where we can have a straightforward automation path to being able to paste, and we want to make sure that existing text or content doesn't get in the way. For this reason, I chose the Notes application as a good place to paste, because:

1. When you launch the Notes app, no matter where you are, you have the ability to create a new (blank) note with one tap.
2. The Notes app has one main text field, whose contents Appium has access to (with a little wrinkle that I'll describe in a moment).

The flow we need to implement to make this happen in more detail looks like this:

1. Get the data into the clipboard (which we have already done via the "Copy URL" button in my example)
2. Activate the Notes app
3. Create a new blank note
4. Paste the clipboard
5. Find the element containing the pasted contents
6. Read the text from that element

If we can do all these things, we'll have the value of the iOS real device clipboard in our test script, and can then do whatever we want with it. So let's have a look at the implementation.
‍

  
// open up the notes app (this only exists on a real device)
driver.activateApp(NOTES_ID);

// create a new blank note
wait.until(ExpectedConditions.presenceOfElementLocated(NEW_NOTE)).click();

// tap the note field to bring up the 'paste' button
WebElement note = wait.until(ExpectedConditions.presenceOfElementLocated(NOTE_CONTENT));
note.click();

// paste the content
wait.until(ExpectedConditions.presenceOfElementLocated(PASTE)).click();

// retrieve the content and do something with it
System.out.println("Clipboard text was: " + note.getText());

‍

Explanations are in comments above. For the specific locators you'll need, see below. It's all pretty straightforward--just walking through what a user would do in this scenario! One thing to keep in mind is that if there is rich content in the clipboard (a URL, image, etc...), it will be pasted as its own element or set of elements within the note. In that case, you'll need to find that element instead of the main note element when retrieving content.

I've included a full example (also available on Github) which should work on a real device (you'll obviously need to update the capabilities to something that works on your device). Note also the example of a locator for the element you'll need to find if you're pasting, say, a URL, and not just plain text.

‍

public class Edition115_Clipboard_Real_IOS {

    private IOSDriver driver;
    private WebDriverWait wait;
    private final static String NOTES_ID = "com.apple.mobilenotes";
    private final static String PHOTOS = "com.apple.mobileslideshow";

    private final static By NEW_NOTE = MobileBy.AccessibilityId("New note");
    private final static By NOTE_CONTENT = MobileBy.AccessibilityId("New note");
    private final static By PASTE = MobileBy.AccessibilityId("Paste");
    //private final static By LINK_IN_NOTE = By.xpath("//XCUIElementTypeTextView[@name='note']/XCUIElementTypeLink");


    @Before
    public void setUp() throws MalformedURLException {
        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability("platformName", "iOS");
        capabilities.setCapability("platformVersion", "13.3");
        capabilities.setCapability("deviceName", "iPhone 11");
        capabilities.setCapability("app", PHOTOS);
        capabilities.setCapability("automationName", "XCUITest");
        driver = new IOSDriver(new URL("http://localhost:4723/wd/hub"), capabilities);
        wait = new WebDriverWait(driver, 10);
    }

    @After
    public void tearDown() {
        if (driver != null) {
            driver.quit();
        }
    }

    @Test
    public void testGetClipboardContents() {
        driver.activateApp(NOTES_ID);
        wait.until(ExpectedConditions.presenceOfElementLocated(NEW_NOTE)).click();
        WebElement note = wait.until(ExpectedConditions.presenceOfElementLocated(NOTE_CONTENT));
        note.click();
        wait.until(ExpectedConditions.presenceOfElementLocated(PASTE)).click();
        System.out.println("Clipboard text was: " + note.getText());
    }
}