docs-v2/content/influxdb/clustered/admin/custom-partitions/partition-templates.md

4.7 KiB

title list_title description menu weight
Partition templates Use partition templates Learn how to define custom partitioning strategies using partition templates. Data can be partitioned by tag and time.
influxdb_clustered
parent
Manage data partitioning
202

Use partition templates to define the patterns used to generate partition keys. A partition key uniquely identifies a partition and is used to name the partition Parquet file in the Object store.

A partition template consists of 1-8 template parts---dimensions to partition data by. There are three types of template parts:

  • tag: An InfluxDB tag to partition by.
  • tag bucket: An InfluxDB tag and number of "buckets" to group tag values into. Data is partitioned by the tag bucket rather than each distinct tag value.
  • time: A Rust strftime date and time string that specifies the time interval to partition data by. The smallest unit of time included in the time part template is the interval used to partition data.

{{% note %}} A partition template can include up to 7 total tag and tag bucket parts and only 1 time part. {{% /note %}}

Restrictions

Template part size limit

Each template part is limited to 200 bytes in length. Anything longer will be truncated at 200 bytes and appended with #.

Partition key size limit

With the truncation of template parts, the maximum length of a partition key is 1,607 bytes (1.57 KiB).

Reserved keywords

The following reserved keywords cannot be used in partition templates:

  • time

Reserved Characters

If used in template parts, non-ASCII characters and the following reserved characters must be percent encoded:

  • |: Partition key part delimiter
  • !: Null or missing partition key part
  • ^: Empty string partition key part
  • #: Key part truncation marker
  • %: Required for unambiguous reversal of percent encoding

Tag part templates

Tag part templates consist of a tag key to partition by. Generated partition keys include the unique tag value specific to each partition.

Tag bucket part templates

Tag bucket part templates consist of a tag key to partition by and the number of "buckets" to partition tag values into--for example:

customerID,500

Values of the customerID tag are bucketed into 500 distinct "buckets." Each bucket is identified by the remainder of the tag value hashed into a 32bit integer divided by the specified number of buckets:

hash(tagValue) % N

Generated partition keys include the unique tag bucket identifier specific to each partition.

Supported number of tag buckets: 1-1,000

{{% note %}} Tag buckets should be used to partition by high cardinality tags or tags with an unknown number of distinct values. {{% /note %}}

Time part templates

Time part templates use a limited subset of the Rust strftime date and time formatting syntax to specify time format in partition keys. InfluxDB uses the smallest unit of time included in the time part template as the partition interval.

Date specifiers

Variable Example Description
%Y 2001 The full proleptic Gregorian year, zero-padded to 4 digits. chrono supports years from -262144 to 262143. Note: years before 1 BCE or after 9999 CE, require an initial sign (+/-).
%m 07 Month number (01--12), zero-padded to 2 digits.
%d 08 Day number (01--31), zero-padded to 2 digits.