Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
influxdb/iox_data_generator/schemas/storage_cardinality_example...

83 lines
1.6 KiB
TOML
Raw Normal View History

feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
name = "storage_cardinality_example"
base_seed = "this is a demo"
[[values]]
name = "role"
chore: pr cleanup on data generator
2021-09-13 21:45:01 +00:00
template = "storage"
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
cardinality = 1
[[values]]
name = "url"
chore: pr cleanup on data generator
2021-09-13 21:45:01 +00:00
template = "http://127.0.0.1:6060/metrics/usage"
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
cardinality = 1
[[values]]
refactor: optimizations to make tag set generation significantly faster
2021-09-09 22:55:04 +00:00
name = "org_id"
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
# Fill in the value with the cardinality counter and 15 random alphanumeric characters
chore: pr cleanup on data generator
2021-09-13 21:45:01 +00:00
template = "{{id}}_{{random 15}}"
cardinality = 1
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
has_one = ["env"]
[[values]]
name = "env"
chore: pr cleanup on data generator
2021-09-13 21:45:01 +00:00
template = "whatever-environment-{{id}}"
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
cardinality = 10
[[values]]
refactor: optimizations to make tag set generation significantly faster
2021-09-09 22:55:04 +00:00
name = "bucket_id"
belongs_to = "org_id"
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
# each bucket will have a unique value so it can be used here to guarantee uniqueness even across orgs
chore: pr cleanup on data generator
2021-09-13 21:45:01 +00:00
template = "{{id}}_{{random 15}}"
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
# For each org, 3 buckets will be generated
cardinality = 3
[[values]]
name = "node_id"
chore: pr cleanup on data generator
2021-09-13 21:45:01 +00:00
template = "{{id}}"
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
cardinality = 100
[[values]]
name = "host"
chore: pr cleanup on data generator
2021-09-13 21:45:01 +00:00
template = "storage-{{node_id.value}}"
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
belongs_to = "node_id"
cardinality = 1
refactor: optimizations to make tag set generation significantly faster
2021-09-09 22:55:04 +00:00
[[values]]
name = "hostname"
chore: pr cleanup on data generator
2021-09-13 21:45:01 +00:00
template = "{{node_id.value}}"
refactor: optimizations to make tag set generation significantly faster
2021-09-09 22:55:04 +00:00
belongs_to = "node_id"
cardinality = 1
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
[[values]]
name = "partition_id"
chore: pr cleanup on data generator
2021-09-13 21:45:01 +00:00
template = "{{id}}"
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
cardinality = 10
# makes a tagset so every bucket exists on every node with every partition.
# So count(bucket) * count(node) * count(partition) cardinality.
[[tag_sets]]
name = "bucket_set"
for_each = [
refactor: optimizations to make tag set generation significantly faster
2021-09-09 22:55:04 +00:00
"role",
"url",
"org_id",
"org_id.env",
"org_id.bucket_id",
"node_id",
"node_id.hostname",
"node_id.host",
"partition_id",
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
]
[[agents]]
name = "metric-scraper"
# sampling_interval = "1s"
[[agents.measurements]]
name = "storage_usage_bucket_cardinality"
chore: pr cleanup on data generator
2021-09-13 21:45:01 +00:00
# TODO: new syntax to make use of tag sets here
feat: Add pre-generated values and tag sets to data generator This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-02 18:35:40 +00:00
[[agents.measurements.fields]]
name = "gauge"
i64_range = [1, 8147240]