for iOS
This guide will demonstrate you how to use Emcee for iOS testing. We will use two MacOS machines to run unit and UI tests from a sample project. You can also use a single machine to try out Emcee to see if it works for your project. In this case, a single machine will act as a queue and a worker simultaneously.
If you encounter any issues while proceeding through the guide, please reach out via https://t.me/emcee_ios.
Table of contents#
Prerequisites #
Install Emcee#
Toolchain#
We will use Xcode 15.2 and the iOS 17.2 simulator runtime (installed with xcodebuild -downloadPlatform iOS). If you want to use a specific version of simulator runtime, proceed to Xcode -> Preferences... -> Components -> Simulators and install the runtime on all the worker machines, where you want the tests to execute with the needed runtime version.
SSH access#
In this tutorial we use technique when Emcee uses ssh to deploy itself to the machines specified as queue and workers. You will need to grant SSH access to your machines. Enable it in your System Preferences.
We will use SSH key-based authentication. So, you have to prepare SSH keys beforehand.
Build the sample project #
To run Emcee you have to build tests for iOS simulator first. If you've got compiled test artifacts you can skip this step.
In this step, we will build a sample project that features different types of tests. Xcode and xcodebuild will produce build artifacts in derived data.
Clone the sample project:
and buid:
xcodebuild build-for-testing\
-project EmceeSample.xcodeproj\
-destination "generic/platform=iOS Simulator"\
-scheme EmceeSample\
-derivedDataPath derivedData
Create emceeplan #
To run tests you have to define emceeplan file. Take a look on full_regress.yml configurations we prepared for this sample project:
tests:
configurations:
- platform: ios
xcTestBundle: derivedData/Build/Products/Debug-iphonesimulator/EmceeSampleTestsWithoutHost.xctest
device:
simDeviceType: iPhone-15
simRuntime: iOS-17-2
- platform: ios
xcTestBundle: derivedData/Build/Products/Debug-iphonesimulator/EmceeSample.app/PlugIns/EmceeSampleHostedTests.xctest
appBundle: derivedData/Build/Products/Debug-iphonesimulator/EmceeSample.app
device:
simDeviceType: iPhone-15
simRuntime: iOS-17-2
- platform: ios
xcTestBundle: derivedData/Build/Products/Debug-iphonesimulator/EmceeSampleUITests-Runner.app/PlugIns/EmceeSampleUITests.xctest
appBundle: derivedData/Build/Products/Debug-iphonesimulator/EmceeSample.app
runnerBundle: derivedData/Build/Products/Debug-iphonesimulator/EmceeSampleUITests-Runner.app
device:
simDeviceType: iPhone-15
simRuntime: iOS-17-2
outputs:
- xcresult
queue:
type: deploy
host: localhost
ssh:
username: _emcee
identity:
type: publicKey
keyPath: /Users/YOUR_USER/.ssh/id_ed25519_emcee
workers:
- id: w1
host: localhost
Before proceeding you have to set ssh username and keyPath to match those you created on prerequisites step.
Configurations#
In this minimalistic emceeplan we've created 3 configurations:
- for unit testing, to run lightweight tests that do not depend on host application
- for integration testing, to run tests that require UI instantiation
- for end to end testing with full device access.
Learn more about Xcode Test Plans and build artifacts requirements for different iOS tests here.
All tests will be run on iPhone-15 with iOS-17-2:
As a result we want to get only Xcode compatible xcresult:
Queue#
Emcee requires queue and at least one worker in the system to operate. In emceeplan above we specified that Emcee should deploy itself using ssh with public key identity:
queue:
type: deploy
ssh:
username: _emcee
identity:
type: publicKey
keyPath: /Users/YOUR_USER/.ssh/id_ed25519_emcee
Where queue and worker will be deployed to localhost:
Run tests #
To run all tests described in emceeplan use CLI in following way:
After all tests finish executing you'll get short info in shell:
[INFO] 9 tests completed successfully
[INFO] 2 tests were skipped
[INFO] 3 tests completed with errors
[INFO] Test <EmceeSampleTestsWithoutHost/test___from_tests_without_host___that_always_fails> failed after 1 runs
[INFO] executed on localhost at 2024-05-11 14:33:07.632 using AC785F64-5B52-4953-B5D7-31512A748E51
[INFO] caught 1 exceptions
[INFO] EmceeSampleTestsWithoutHost/test___from_tests_without_host___that_always_fails /Users/vvishutin/ios-test-runner/Samples/EmceeSample/EmceeSampleTestsWithoutHost/EmceeSampleTestsWithoutHost.swift:17: failed - Failure from tests without host
[INFO] Test <EmceeSampleTestsWithHost/test___from_tests_with_host___that_always_fails> failed after 1 runs
[INFO] executed on localhost at 2024-05-11 14:33:32.994 using AC785F64-5B52-4953-B5D7-31512A748E51
[INFO] caught 1 exceptions
[INFO] EmceeSampleTestsWithHost/test___from_tests_with_host___that_always_fails /Users/vvishutin/ios-test-runner/Samples/EmceeSample/EmceeSample/ViewController.swift:13: EmceeSample.ViewController test exception (NSGenericException)
[INFO] Test <EmceeSampleUITests/test___from_xcui_tests___that_always_fails> failed after 1 runs
[INFO] executed on localhost at 2024-05-11 14:33:29.177 using EFAE58CF-24D2-4CA6-BF71-572533997BA6
[INFO] caught 1 exceptions
[INFO] EmceeSampleUITests/test___from_xcui_tests___that_always_fails /Users/vvishutin/ios-test-runner/Samples/EmceeSample/EmceeSampleUITests/EmceeSampleUITests.swift:37: failed - Failure from xcui tests
and merged xcresult in result directory.
This is how the test run might look:
Emcee queue and workers keeps running after tests run's finished. If you want to terminate Emcee run:
Where to go from here #
In this simplified guide we utilized only a single worker that unlikely will boost your pipeline significantly. To get better performance you should scale this guide up to as many machines as you have. It's recommented that you control the deployment process. Learn more on possible deployment options in production environment.
Find all possible test parameters you may use to configure in emceeplan to reflect your tests correctly.
You also may want to know all possible reports Emcee may yield during tests run.
Look here how you can configure simulator before running tests.