Overwrite the Elastic Rally track parameters

Each track allows to overwrite the corresponding parameters using –track-params.

For example, the track geonames allows to overwrite the following parameters.

  • bulk_size (default: 5000)
  • bulk_indexing_clients (default: 8): Number of clients that issue bulk indexing requests.
  • ingest_percentage (default: 100): A number between 0 and 100 that defines how much of the document corpus should be ingested.
  • conflicts (default: “random”): Type of id conflicts to simulate. Valid values are: ‘sequential’ (A document id is replaced with a document id with a sequentially increasing id), ‘random’ (A document id is replaced with a document id with a random other id).
  • conflict_probability (default: 25): A number between 0 and 100 that defines the probability of id conflicts. This requires to run the respective challenge. Combining conflicts=sequential and conflict-probability=0 makes Rally generate index ids by itself, instead of relying on Elasticsearch’s automatic id generation https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#_automatic_id_generation_.
  • on_conflict (default: “index”): Whether to use an “index” or an “update” action when simulating an id conflict.
  • recency (default: 0): A number between 0 and 1 that defines whether to bias towards more recent ids when simulating conflicts. See the Rally docs for the full definition of this parameter. This requires to run the respective challenge.
  • number_of_replicas (default: 0)
  • number_of_shards (default: 5)
  • max_num_segments: The maximum number of segments to force-merge to.
  • source_enabled (default: true): A boolean defining whether the _source field is stored in the index.
  • index_settings: A list of index settings. Index settings defined elsewhere (e.g. number_of_replicas) need to be overridden explicitly.
  • cluster_health (default: “green”): The minimum required cluster health.
  • error_level (default: “non-fatal”): Available for bulk operations only to specify ignore-response-error-level.

The default track parameters can be verified in the index.json.

$ cat .rally/benchmarks/tracks/default/geonames/index.json
{
  "settings": {
    "index.number_of_shards": {{number_of_shards | default(5)}},
    "index.number_of_replicas": {{number_of_replicas | default(0)}},
    "index.store.type": "{{store_type | default('fs')}}",
    "index.requests.cache.enable": false
  },
  [..]
}

You can change the track parameters when you run the race as below.

$ esrally race --pipeline=benchmark-only --target-host=10.10.10.1:39200,10.10.10.2:39200,10.10.10.3:39200 --track=geonames --track-params="number_of_shards:3,number_of_replicas:1" --challenge=append-no-conflicts --on-error=abort --race-id=${RACE_ID}

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

[INFO] Race id is [734bb4b3-8b7a-4c0b-9fa6-aaeb4659569f]
[INFO] Racing on track [geonames], challenge [append-no-conflicts] and car ['external'] with version [7.17.0].

[WARNING] indexing_total_time is 19846 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] refresh_total_time is 5970 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
Running delete-index                                                           [100% done]
Running create-index                                                           [100% done]
Running check-cluster-health                                                   [100% done]
Running index-append                                                           [100% done]
Running refresh-after-index                                                    [100% done]
Running force-merge                                                            [100% done]
Running refresh-after-force-merge                                              [100% done]
Running wait-until-merges-finish                                               [100% done]
Running index-stats                                                            [100% done]
Running node-stats                                                             [100% done]
Running default                                                                [100% done]
Running term                                                                   [100% done]
Running phrase                                                                 [100% done]
Running country_agg_uncached                                                   [100% done]
Running country_agg_cached                                                     [100% done]
Running scroll                                                                 [100% done]
Running expression                                                             [100% done]
Running painless_static                                                        [100% done]
Running painless_dynamic                                                       [100% done]
Running decay_geo_gauss_function_score                                         [100% done]
Running decay_geo_gauss_script_score                                           [100% done]
Running field_value_function_score                                             [100% done]
Running field_value_script_score                                               [100% done]
Running large_terms                                                            [100% done]
Running large_filtered_terms                                                   [100% done]
Running large_prohibited_terms                                                 [100% done]
Running desc_sort_population                                                   [100% done]
Running asc_sort_population                                                    [100% done]
Running asc_sort_with_after_population                                         [100% done]
Running desc_sort_geonameid                                                    [100% done]
Running desc_sort_with_after_geonameid                                         [100% done]
Running asc_sort_geonameid                                                     [100% done]
Running asc_sort_with_after_geonameid                                          [100% done]

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------

|                                                         Metric |                           Task |           Value |    Unit |
|---------------------------------------------------------------:|-------------------------------:|----------------:|--------:|
|                     Cumulative indexing time of primary shards |                                |    14.3313      |     min |
|             Min cumulative indexing time across primary shards |                                |     0           |     min |
|          Median cumulative indexing time across primary shards |                                |     0.00696667  |     min |
|             Max cumulative indexing time across primary shards |                                |     4.85003     |     min |
|            Cumulative indexing throttle time of primary shards |                                |     0           |     min |
|    Min cumulative indexing throttle time across primary shards |                                |     0           |     min |
| Median cumulative indexing throttle time across primary shards |                                |     0           |     min |
|    Max cumulative indexing throttle time across primary shards |                                |     0           |     min |
|                        Cumulative merge time of primary shards |                                |     9.33973     |     min |
|                       Cumulative merge count of primary shards |                                |    93           |         |
|                Min cumulative merge time across primary shards |                                |     0           |     min |
|             Median cumulative merge time across primary shards |                                |     0.00276667  |     min |
|                Max cumulative merge time across primary shards |                                |     3.20492     |     min |
|               Cumulative merge throttle time of primary shards |                                |     2.39933     |     min |
|       Min cumulative merge throttle time across primary shards |                                |     0           |     min |
|    Median cumulative merge throttle time across primary shards |                                |     0           |     min |
|       Max cumulative merge throttle time across primary shards |                                |     0.8346      |     min |
|                      Cumulative refresh time of primary shards |                                |     1.4791      |     min |
|                     Cumulative refresh count of primary shards |                                |   408           |         |
|              Min cumulative refresh time across primary shards |                                |     0           |     min |
|           Median cumulative refresh time across primary shards |                                |     0.0191417   |     min |
|              Max cumulative refresh time across primary shards |                                |     0.532383    |     min |
|                        Cumulative flush time of primary shards |                                |     0.143       |     min |
|                       Cumulative flush count of primary shards |                                |    14           |         |
|                Min cumulative flush time across primary shards |                                |     0           |     min |
|             Median cumulative flush time across primary shards |                                |     0.000191667 |     min |
|                Max cumulative flush time across primary shards |                                |     0.0622167   |     min |
|                                        Total Young Gen GC time |                                |    30.76        |       s |
|                                       Total Young Gen GC count |                                |  3790           |         |
|                                          Total Old Gen GC time |                                |     7.34        |       s |
|                                         Total Old Gen GC count |                                |   107           |         |
|                                                     Store size |                                |     5.79046     |      GB |
|                                                  Translog size |                                |     8.19564e-07 |      GB |
|                                         Heap used for segments |                                |     0.452423    |      MB |
|                                       Heap used for doc values |                                |     0.0294952   |      MB |
|                                            Heap used for terms |                                |     0.340546    |      MB |
|                                            Heap used for norms |                                |     0.0440674   |      MB |
|                                           Heap used for points |                                |     0           |      MB |
|                                    Heap used for stored fields |                                |     0.0383148   |      MB |
|                                                  Segment count |                                |    74           |         |
|                                    Total Ingest Pipeline count |                                |     0           |         |
|                                     Total Ingest Pipeline time |                                |     0           |       s |
|                                   Total Ingest Pipeline failed |                                |     0           |         |
|                                                 Min Throughput |                   index-append | 80680.2         |  docs/s |
|                                                Mean Throughput |                   index-append | 81105.1         |  docs/s |
|                                              Median Throughput |                   index-append | 81128.5         |  docs/s |
|                                                 Max Throughput |                   index-append | 81286.2         |  docs/s |
|                                        50th percentile latency |                   index-append |   338.821       |      ms |
|                                        90th percentile latency |                   index-append |   843.169       |      ms |
|                                        99th percentile latency |                   index-append |  1132.18        |      ms |
|                                       100th percentile latency |                   index-append |  1158.98        |      ms |
|                                   50th percentile service time |                   index-append |   338.821       |      ms |
|                                   90th percentile service time |                   index-append |   843.169       |      ms |
|                                   99th percentile service time |                   index-append |  1132.18        |      ms |
|                                  100th percentile service time |                   index-append |  1158.98        |      ms |
|                                                     error rate |                   index-append |     0           |       % |
[..]                                                 

----------------------------------
[INFO] SUCCESS (took 4314 seconds)
----------------------------------

Reference