Skip to content

Obtaining report files on Android platform required configured storage

Emcee might produce various types of test reports. Basic knowledge of test run status you can find in client's log.

[INFO] 2024-05-03 10:54:39.663: 3 tests completed successfully
[INFO] 2024-05-03 10:54:39.663: 0 tests were skipped
[INFO] 2024-05-03 10:54:39.663: 1 tests completed with errors
[INFO] 2024-05-03 10:54:39.663: Test <EmceeSampleUITests/test___from_xcui_tests___that_always_fails> failed after 1 runs
[INFO] 2024-05-03 10:54:39.664:    executed on localhost at 2024-05-03 10:54:27.018 using 0F06F317-2148-4707-8ECC-9DDF4C0D84FC
[INFO] 2024-05-03 10:54:39.664:    caught 1 exceptions
[INFO] 2024-05-03 10:54:39.664:        EmceeSampleUITests/test___from_xcui_tests___that_always_fails /Users/vvishutin/ios-test-runner/Samples/EmceeSample/EmceeSampleUITests/EmceeSampleUITests.swift:37: failed - Failure from xcui tests

Other reports should be enabled explicitly using outputs option in emceeplan configuration

tests:
  configurations:
    #...
  outputs: [...]

along with specifiying artifacts output directory

emcee run config.yml --outdir=artifacts

Below is an exhausting list of supported reports. It's allowed to use any combination of reports in emceeplan.

Allure#

tests:
  configurations:
    #...
  outputs:
    - allure

Emcee will download Allure files for report, including snapshots and video artifacts. Allure report might be rendered in browser with Allure tool.

Android notes. Files for the Allure report are created by Allure itself for Android platform. Thus Allure should be enabled in the tests. Emcee worker will try to search Allure files in default Allure directories, such as /data/data/your.app.package/files/allure-results or /sdcard/googletest/test_outputfiles/allure-results. More details about this directories look in the documentation Allure Kotlin.

Android notes. If you want to download Allure files from other directory, read how to download files from custom directory.

Junit#

tests:
  configurations:
    #...
  outputs:
    - junit

Emcee will put combined junit report for all devices (if many).

Emcee report#

tests:
  configurations:
    #...
  outputs:
    - emcee

Emcee will produce a report with proprietary format that consists of successful/failed/skipped tests enriched with metainformation such as duration, logs, hostname and simulator where test was executed.

Currenty not supported on Android.

XCResult (iOS specific)#

tests:
  configurations:
    #...
  outputs:
    - xcresult

Emcee will collect partial xcresults from workers and merge it to single bundle that may be opened with Xcode. Host where client runs should be accessible by workers.

Chrome trace#

tests:
  configurations:
    #...
  outputs:
    - trace

Emcee will generate per-bucket trace file that can be loaded into chrome://tracing. Use it to get insights when making decisions on equipment purchase or problem investigation.

Currenty not supported on Android.

Output directory layout, device specific reports#

Emcee will create dedicated directory to store reports for each launched job (assuming launched with --outdir=artifacts):

└── artifacts
    └── JOB_ID
        ├── allure_report
        │   └── ...
        ├── junit.json
        ├── emcee.json
        ├── result.xcresult
        └── trace.json

To control job identificator refer to job id option --job-id when running emcee.

For emceeplan configurations with multiple devices Emcee will also create per-device/OS artifacts directory: 

└── artifacts
    └── JOB_ID
        ├── reports...
        ├── iOS-17-2_iPhone-15
        │   └── reports...
        └── iOS-16-1_iPhone-14
            └── reports...