Skip to content

Worker logs and metrics

Worker can send logs to Kibana and metrics to Graphite. Metrics are visualized using Grafana or other tools that support working with Graphite.

Example of a dashboard with real metrics data v1.

Example of a dashboard with real metrics data v2.

Example of a dashboard with real metrics data v3.

Manual worker logs extraction#

You can view and extract worker logs from the standard container output.

For example, if you have deployed Emcee workers using Docker Compose or manually, execute this command to retrieve the last 200 lines of logs from the worker container:

docker logs --tail 200 cdd06ad46b12

where cdd06ad46b12 is the container id.

Logs and metric configuration#

Before receiving metrics and logs, you need to configure Graphite and Kibana yourself.

To tell Emcee worker send metrics and logs, convey relevant parameters as environment variables (see chapter Worker configuration).

To enable Graphite metrics use following environment variables:

  • EMCEE_WORKER_GRAPHITE_IS_ENABLED
  • EMCEE_WORKER_GRAPHITE_SOCKET_ADDRESS
  • EMCEE_WORKER_GRAPHITE_SOCKET_PORT
  • EMCEE_WORKER_GRAPHITE_METRIC_PREFIX

To enable Kibana logging use following environment variables:

  • EMCEE_WORKER_KIBANA_IS_ENABLED
  • EMCEE_WORKER_KIBANA_ENDPOINTS
  • EMCEE_WORKER_KIBANA_INDEX_NAME
  • EMCEE_WORKER_ELASTIC_API_KEY

What metrics and logs does the worker send?#

The worker sends information about the state of the system and the steps of running job. The verbosity of the logs depends on the specified logging level (see the Worker Configuration chapter).

List of metrics sent by the worker:

Name of metric in Graphite Description
artifact_downloaded artifact_reused Sent if apk gets downloaded or reused from local cache
artifacts_download_time Time passed to download apk from storage
work_time Worker operating time upon successful run (including downloading apk files, running tests, and sending test artifacts)
time_to_get_device Time to get an emulator ready to run tests (starting a new one or reusing an existing one)
time_to_start_device time_to_stop_device time_to_find_device Start, stop, search emulator time
test_execution_time Test execution time plus instrumental status (passed, failed, etc.)
test_execution_machine_time Pure test execution time (test time on the emulator)
test_artifacts_extract_time Total time for uploading test artifacts from the device
time_to_install_apks time_to_uninstall_apks Time to install and remove apk files on emulator
time_to_create_new_device Time to create new emulator
dead Event when worker dead unexpectedly
adb.[operation].[status] Time to make adb operation with result status
network.[service].[method].[resultCode] Time to execute method to service with HTTP code resultCode

In Graphite the metric will look like your.graphite.prefix.metric_name.

Additionally, metrics have metadata in the form of Graphite tags:

Tag Description
worker_id Worker id
emcee_version Version of Emcee worker
sdk sdk version (presended in metrics for which it makes sense)

Ready to use dashboard for worker metrics#

Example dashboard json file for Grafana and Graphite as a data source:

As EMCEE_WORKER_GRAPHITE_METRIC_PREFIX in the dashboard was used apps.mobile.metrics.awesomeapp. For the $worker_id it uses tag_values(worker_id, name=~apps.mobile.metrics.awesomeapp.*) request (specified in the dashboard settings in Variables). Or you can replace $worker_id value with ~*.

This dashboard can be imported into Grafana when creating a new dashboard. The dashboard has been exported for Grafana 9.1.7 and Graphite 1.0.0

The correct operation of the dashboard during import is not guaranteed, since in new versions of Grafana and Graphite the functionality may work differently. Depending on your tools the finished dashboard may need some modification.