This commit adds some initial benchmarks for the general read_group
approach using a hashing strategy. Benchmarks are as follows:
segment_read_group_all_time_vary_cardinality/cardinality_20_columns_2_rows_500000
time: [23.335 ms 23.363 ms 23.397 ms]
thrpt: [854.82 elem/s 856.07 elem/s 857.07
elem/s]
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
segment_read_group_all_time_vary_cardinality/cardinality_200_columns_2_rows_500000
time: [34.266 ms 34.301 ms 34.346 ms]
thrpt: [5.8231 Kelem/s 5.8307 Kelem/s 5.8367
Kelem/s]
Found 13 outliers among 100 measurements (13.00%)
5 (5.00%) high mild
8 (8.00%) high severe
segment_read_group_all_time_vary_cardinality/cardinality_2000_columns_2_rows_500000
time: [48.788 ms 48.996 ms 49.238 ms]
thrpt: [40.619 Kelem/s 40.820 Kelem/s 40.993
Kelem/s]
Found 11 outliers among 100 measurements (11.00%)
3 (3.00%) high mild
8 (8.00%) high severe
Benchmarking
segment_read_group_all_time_vary_cardinality/cardinality_20000_columns_3_rows_500000:
Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to
increase target time to 8.2s, or reduce sample count to 60.
segment_read_group_all_time_vary_cardinality/cardinality_20000_columns_3_rows_500000
time: [80.133 ms 80.201 ms 80.287 ms]
thrpt: [249.11 Kelem/s 249.37 Kelem/s 249.58
Kelem/s]
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
Benchmarking
segment_read_group_all_time_vary_columns/cardinality_20000_columns_2_rows_500000:
Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to
increase target time to 7.4s, or reduce sample count to 60.
segment_read_group_all_time_vary_columns/cardinality_20000_columns_2_rows_500000
time: [73.692 ms 73.951 ms 74.245 ms]
thrpt: [269.38 Kelem/s 270.45 Kelem/s 271.40
Kelem/s]
Found 13 outliers among 100 measurements (13.00%)
13 (13.00%) high severe
Benchmarking
segment_read_group_all_time_vary_columns/cardinality_20000_columns_3_rows_500000:
Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to
increase target time to 8.1s, or reduce sample count to 60.
segment_read_group_all_time_vary_columns/cardinality_20000_columns_3_rows_500000
time: [79.837 ms 79.934 ms 80.079 ms]
thrpt: [249.75 Kelem/s 250.21 Kelem/s 250.51
Kelem/s]
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
Benchmarking
segment_read_group_all_time_vary_columns/cardinality_20000_columns_4_rows_500000:
Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to
increase target time to 9.7s, or reduce sample count to 50.
segment_read_group_all_time_vary_columns/cardinality_20000_columns_4_rows_500000
time: [95.415 ms 95.549 ms 95.707 ms]
thrpt: [208.97 Kelem/s 209.32 Kelem/s 209.61
Kelem/s]
Found 15 outliers among 100 measurements (15.00%)
7 (7.00%) high mild
8 (8.00%) high severe
segment_read_group_all_time_vary_rows/cardinality_20000_columns_2_rows_250000
time: [38.897 ms 39.045 ms 39.227 ms]
thrpt: [509.86 Kelem/s 512.22 Kelem/s 514.18
Kelem/s]
Found 13 outliers among 100 measurements (13.00%)
4 (4.00%) high mild
9 (9.00%) high severe
Benchmarking
segment_read_group_all_time_vary_rows/cardinality_20000_columns_2_rows_500000:
Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to
increase target time to 7.2s, or reduce sample count to 60.
segment_read_group_all_time_vary_rows/cardinality_20000_columns_2_rows_500000
time: [71.965 ms 72.190 ms 72.445 ms]
thrpt: [276.07 Kelem/s 277.04 Kelem/s 277.91
Kelem/s]
Found 21 outliers among 100 measurements (21.00%)
4 (4.00%) low mild
3 (3.00%) high mild
14 (14.00%) high severe
Benchmarking
segment_read_group_all_time_vary_rows/cardinality_20000_columns_2_rows_750000:
Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to
increase target time to 10.7s, or reduce sample count to 40.
segment_read_group_all_time_vary_rows/cardinality_20000_columns_2_rows_750000
time: [106.48 ms 106.58 ms 106.70 ms]
thrpt: [187.43 Kelem/s 187.65 Kelem/s 187.82
Kelem/s]
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
Benchmarking
segment_read_group_all_time_vary_rows/cardinality_20000_columns_2_rows_1000000:
Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to
increase target time to 14.0s, or reduce sample count to 30.
segment_read_group_all_time_vary_rows/cardinality_20000_columns_2_rows_1000000
time: [140.02 ms 140.14 ms 140.29 ms]
thrpt: [142.57 Kelem/s 142.71 Kelem/s 142.84
Kelem/s]
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high severe
segment_read_group_pre_computed_groups_vary_cardinality/cardinality_2_columns_1_rows_500000
time: [51.734 us 52.123 us 52.560 us]
thrpt: [38.051 Kelem/s 38.371 Kelem/s 38.659
Kelem/s]
Found 18 outliers among 100 measurements (18.00%)
3 (3.00%) high mild
15 (15.00%) high severe
segment_read_group_pre_computed_groups_vary_cardinality/cardinality_20_columns_2_rows_500000
time: [50.546 us 50.642 us 50.785 us]
thrpt: [393.82 Kelem/s 394.93 Kelem/s 395.68
Kelem/s]
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) low mild
2 (2.00%) high mild
3 (3.00%) high severe
segment_read_group_pre_computed_groups_vary_cardinality/cardinality_200_columns_2_rows_500000
time: [267.47 us 270.23 us 273.10 us]
thrpt: [732.33 Kelem/s 740.12 Kelem/s 747.75
Kelem/s]
segment_read_group_pre_computed_groups_vary_cardinality/cardinality_2000_columns_2_rows_500000
time: [14.961 ms 15.033 ms 15.113 ms]
thrpt: [132.33 Kelem/s 133.04 Kelem/s 133.68
Kelem/s]
Found 11 outliers among 100 measurements (11.00%)
3 (3.00%) high mild
8 (8.00%) high severe
segment_read_group_pre_computed_groups_vary_columns/cardinality_200_columns_1_rows_500000
time: [84.825 us 84.938 us 85.083 us]
thrpt: [2.3506 Melem/s 2.3546 Melem/s 2.3578
Melem/s]
Found 14 outliers among 100 measurements (14.00%)
7 (7.00%) high mild
7 (7.00%) high severe
segment_read_group_pre_computed_groups_vary_columns/cardinality_200_columns_2_rows_500000
time: [258.81 us 259.33 us 260.05 us]
thrpt: [769.08 Kelem/s 771.22 Kelem/s 772.77
Kelem/s]
Found 14 outliers among 100 measurements (14.00%)
2 (2.00%) high mild
12 (12.00%) high severe
Benchmarking
segment_read_group_pre_computed_groups_vary_columns/cardinality_200_columns_3_rows_500000:
Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to
increase target time to 6.1s, enable flat sampling, or reduce sample
count to 60.
segment_read_group_pre_computed_groups_vary_columns/cardinality_200_columns_3_rows_500000
time: [1.1971 ms 1.2020 ms 1.2079 ms]
thrpt: [165.58 Kelem/s 166.39 Kelem/s 167.07
Kelem/s]
Found 13 outliers among 100 measurements (13.00%)
3 (3.00%) high mild
10 (10.00%) high severe
segment_read_group_pre_computed_groups_vary_rows/cardinality_200_columns_2_rows_250000
time: [252.42 us 252.58 us 252.75 us]
thrpt: [791.31 Kelem/s 791.84 Kelem/s 792.32
Kelem/s]
Found 10 outliers among 100 measurements (10.00%)
2 (2.00%) high mild
8 (8.00%) high severe
segment_read_group_pre_computed_groups_vary_rows/cardinality_200_columns_2_rows_500000
time: [271.68 us 272.46 us 273.59 us]
thrpt: [731.01 Kelem/s 734.04 Kelem/s 736.15
Kelem/s]
Found 8 outliers among 100 measurements (8.00%)
8 (8.00%) high severe
segment_read_group_pre_computed_groups_vary_rows/cardinality_200_columns_2_rows_750000
time: [293.17 us 293.42 us 293.65 us]
thrpt: [681.09 Kelem/s 681.63 Kelem/s 682.20
Kelem/s]
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low mild
4 (4.00%) high mild
4 (4.00%) high severe
segment_read_group_pre_computed_groups_vary_rows/cardinality_200_columns_2_rows_1000000
time: [306.48 us 307.11 us 307.95 us]
thrpt: [649.45 Kelem/s 651.22 Kelem/s 652.57
Kelem/s]
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
Configures the IOx tracing to compile out trace!() level events in the release
binary. This effectively gives contributors three levels of output:
* Important to the user (info & friends)
* Not important for regular running, but needed to debug
* Only useful to devs in a specific part of the system, never seen by user
Documents this behaviour (and general usage guidelines) for contributors.
Adds telemetry / tracing with support for a Jaeger backend, and changes the
logger from env_logger to a tracing subscriber to collect the log entries.
Events are batched and then emitted asynchronosuly via UDP to the Jaeger
collector using the tokio runtime. There's a bunch of settings (env
vars) related to batch sizes and flush frequency etc - they're all using
their default values at the moment (if it ain't broke...) See the docs
for more info:
https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/sdk-environment-variables.md#opentelemetry-environment-variable-specification
This is only part 1 of telemetry - it does NOT propagate traces across RPC
boundaries as we're still defining how all this should work. I've created #541
to track this.
Closes#202 and closes#203.
* fix: Error if hint argument is provided to read_groupg
* fix: Verify compatible group and group_keys settings
* docs: Add clarifying comments on validation
* refactor: use into() rather than String::from for consistency
Edd Robinson
Michael Hall
Carol (Nichols || Goulding)
Andrew Lamb
Dom
Carol (Nichols || Goulding)
Brandon Sov