Test Execution Management (beta)
Overview
Test Execution Management (beta) is an extension of App Management and enables execution of native test suites on HeadSpin's cloud. Test runners can be associated with the apps stored in App Management to execute entire test suites or individual tests and receive results. Combining this functionality with HeadSpin's session captures delivers build-over-build functional and performance insights about your apps.
Table of Contents
1. Prepare and upload your test runner
Prepare and upload your test runner
The initial release of Test Execution Management (beta) supports <code class="dcode">iOS XCTest</code> and <code class="dcode">Flutter for iOS</code> test suites. Follow the steps below to package and upload your test runner.
XCTest
When you build your app for testing in Xcode, a test runner product will be created in your build directory. Find the ...<code class="dcode">UITests-Runner.app</code> bundle and compress it as a .zip file.
cd {build_directory}/Build/Products/Debug-iphoneos
zip -r Example_tests.zip ExampleUITests-Runner.app
Upload the IPA as usual and receive your <code class="dcode">app_id</code>.
curl -X POST -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v1/app/upload -F app=@Example.ipa
Test execution requires a test runner to be associated with the app stored in App Management. Upload the test runner as an additional app file using the tests file type. See Notes for expected behavior.
curl -X PUT -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v1/app/{app_id}/files/tests --data-binary "@Example_tests.zip"
Flutter for iOS
Assuming you have a Flutter project generated by flutter create that contains a test target, an integration_test directory, and a pubspec.yaml containing the following definition:
dev_dependencies:
flutter_test:
sdk: flutter
integration_test:
sdk: flutter
Build the integration tests.
flutter build ios --config-only integration_test/*.dart
Reference the integration tests in your test target. Add a test file, e.g., RunnerTests.m, to the test target and write the following to the file:
@import XCTest;
@import integration_test;
INTEGRATION_TEST_IOS_RUNNER(RunnerTests)
Build the test runner.
xcodebuild build-for-testing -workspace ios/Runner.xcworkspace -scheme Runner -configuration release -sdk iphoneos -destination generic/platform=iOS -derivedDataPath build
A test runner product will be created in the build directory. Find the Runner.app test runner bundle and compress it as a .ipa file.
cd build/Build/Products/Release-iphoneos
zip -r Flutter_tests.ipa
Upload the test runner bundle.
curl -X POST -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v1/app/upload -F app=@Flutter_tests.ipa
Execute a test
Now that the test runner is stored in App Management, the Test Execution Management (beta) routes become available to execute the test suite or individual tests. Individual tests must be specified by a test identifier, and the syntax of a test identifier string will be specific to the test runner framework. Let's follow an example to execute an XCTest test method.
Lock a device with Basic Device API.
curl -X POST -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/devices/{ios-device-id}/lock
Start a capture session with Session API.
curl -X POST -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/sessions -d '{"session_type": "capture","device_address": "{ios-device-id}@{hostname}"}'
Execute an iOS XCTest test specified by test identifier, e.g., TestClass/TestMethod, with App Management API. See Notes for Flutter for iOS specific details.
curl -X POST -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v1/app/{app_id}/test/{ios-device-id}/execute/TestClass/TestMethod
Stop capture session.
curl -X PATCH -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v0/sessions/{session_id} -d '{"active": false}'
Unlock device.
curl -X POST https://<your_api_token>@api-dev.headspin.io/v0/devices/{ios-device-id}/unlock
Notes
The execution step above similarly applies when executing a Flutter integration test suite, e.g.
curl -X POST -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v1/app/{app_id}/test/{ios-device-id}/execute
Check App Management for complete test execution API details.
Expected behavior
Only one test runner bundle may be associated with an app in App Management at a time. Any call to the below endpoint will replace the existing test runner.
curl -X PUT -H "Authorization: Bearer <your_api_token>" https://api-dev.headspin.io/v1/app/{app_id}/files/tests --data-binary "@Example_tests.zip"
The details of whether each build of the test runner requires an entirely new app build will be specific to the test framework but generally as long as the test target remains the same the same IPA may be reused.
