From e9b6661b430eefd00273cb958e24ce9bd0b3803d Mon Sep 17 00:00:00 2001 From: lwandzura <51929958+lwandzura@users.noreply.github.com> Date: Wed, 10 Nov 2021 11:43:01 -0600 Subject: [PATCH 01/18] added hardware sizing to enterprise --- .../v1.9/guides/hardware_sizing.md | 473 ++++++++++++++++++ 1 file changed, 473 insertions(+) create mode 100644 content/enterprise_influxdb/v1.9/guides/hardware_sizing.md diff --git a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md new file mode 100644 index 000000000..3d2ff8cac --- /dev/null +++ b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md @@ -0,0 +1,473 @@ +--- +title: Hardware sizing guidelines +Description: > + Review configuration and hardware guidelines for InfluxDB OSS (open source) and InfluxDB Enterprise. +menu: + enterprise_influxdb_1_9: + weight: 40 + parent: Guides +--- + +Review configuration and hardware guidelines for InfluxDB OSS (open source) and InfluxDB Enterprise: + +* [Single node or cluster?](#single-node-or-cluster) +* [Query guidelines](#query-guidelines) +* [InfluxDB OSS guidelines](#influxdb-oss-guidelines) +* [InfluxDB Enterprise cluster guidelines](#influxdb-enterprise-cluster-guidelines) +* [When do I need more RAM?](#when-do-i-need-more-ram) +* [Recommended cluster configurations](#recommended-cluster-configurations) +* [Storage: type, amount, and configuration](#storage-type-amount-and-configuration) + +> **Disclaimer:** Your numbers may vary from recommended guidelines. Guidelines provide estimated benchmarks for implementing the most performant system for your business. + +## Single node or cluster? + +If your InfluxDB performance requires any of the following, a single node (InfluxDB OSS) may not support your needs: + +- more than 750,000 field writes per second +- more than 100 moderate queries per second ([see Query guides](#query-guidelines)) +- more than 10,000,000 [series cardinality](/influxdb/v1.8/concepts/glossary/#series-cardinality) + +We recommend InfluxDB Enterprise, which supports multiple data nodes (a cluster) across multiple server cores. +InfluxDB Enterprise distributes multiple copies of your data across a cluster, +providing high-availability and redundancy, so an unavailable node doesn’t significantly impact the cluster. +Please [contact us](https://www.influxdata.com/contact-sales/) for assistance tuning your system. + +If you want a single node instance of InfluxDB that's fully open source, requires fewer writes, queries, and unique series than listed above, and do **not require** redundancy, we recommend InfluxDB OSS. + +> **Note:** Without the redundancy of a cluster, writes and queries fail immediately when a server is unavailable. + +## Query guidelines + +> Query complexity varies widely on system impact. Recommendations for both single nodes and clusters are based on **moderate** query loads. + +For **simple** or **complex** queries, we recommend testing and adjusting the suggested requirements as needed. Query complexity is defined by the following criteria: + +| Query complexity | Criteria | +|------------------|---------------------------------------------------------------------------------------| +| Simple | Have few or no functions and no regular expressions | +| | Are bounded in time to a few minutes, hours, or 24 hours at most | +| | Typically execute in a few milliseconds to a few dozen milliseconds | +| Moderate | Have multiple functions and one or two regular expressions | +| | May also have `GROUP BY` clauses or sample a time range of multiple weeks | +| | Typically execute in a few hundred or a few thousand milliseconds | +| Complex | Have multiple aggregation or transformation functions or multiple regular expressions | +| | May sample a very large time range of months or years | +| | Typically take multiple seconds to execute | + +## InfluxDB OSS guidelines + +Run InfluxDB on locally attached solid state drives (SSDs). Other storage configurations have lower performance and may not be able to recover from small interruptions in normal processing. + +Estimated guidelines include writes per second, queries per second, and number of unique [series](/influxdb/v1.8/concepts/glossary/#series), CPU, RAM, and IOPS (input/output operations per second). + +| vCPU or CPU | RAM | IOPS | Writes per second | Queries* per second | Unique series | +| ----------: | ------: | -------: | ----------------: | ------------------: | ------------: | +| 2-4 cores | 2-4 GB | 500 | < 5,000 | < 5 | < 100,000 | +| 4-6 cores | 8-32 GB | 500-1000 | < 250,000 | < 25 | < 1,000,000 | +| 8+ cores | 32+ GB | 1000+ | > 250,000 | > 25 | > 1,000,000 | + +* **Queries per second for moderate queries.** Queries vary widely in their impact on the system. For simple or complex queries, we recommend testing and adjusting the suggested requirements as needed. See [query guidelines](#query-guidelines) for details. + +## InfluxDB Enterprise cluster guidelines + +### Meta nodes + +> Set up clusters with an odd number of meta nodes──an even number may cause issues in certain configurations. + +A cluster must have a **minimum of three** independent meta nodes for data redundancy and availability. A cluster with `2n + 1` meta nodes can tolerate the loss of `n` meta nodes. + +Meta nodes do not need very much computing power. Regardless of the cluster load, we recommend the following guidelines for the meta nodes: + +* vCPU or CPU: 1-2 cores +* RAM: 512 MB - 1 GB +* IOPS: 50 + +### Web node + +The InfluxDB Enterprise web server is primarily an HTTP server with similar load requirements. For most applications, the server doesn't need to be very robust. A cluster can function with only one web server, but for redundancy, we recommend connecting multiple web servers to a single back-end Postgres database. + +> **Note:** Production clusters should not use the SQLite database (lacks support for redundant web servers and handling high loads). + +* vCPU or CPU: 2-4 cores +* RAM: 2-4 GB +* IOPS: 100 + +### Data nodes + +A cluster with one data node is valid but has no data redundancy. Redundancy is set by the [replication factor](/influxdb/v1.8/concepts/glossary/#replication-factor) on the retention policy the data is written to. Where `n` is the replication factor, a cluster can lose `n - 1` data nodes and return complete query results. + +>**Note:** For optimal data distribution within the cluster, use an even number of data nodes. + +Guidelines vary by writes per second per node, moderate queries per second per node, and the number of unique series per node. + +#### Guidelines per node + +| vCPU or CPU | RAM | IOPS | Writes per second | Queries* per second | Unique series | +| ----------: | -------: | ----: | ----------------: | ------------------: | ------------: | +| 2 cores | 4-8 GB | 1000 | < 5,000 | < 5 | < 100,000 | +| 4-6 cores | 16-32 GB | 1000+ | < 100,000 | < 25 | < 1,000,000 | +| 8+ cores | 32+ GB | 1000+ | > 100,000 | > 25 | > 1,000,000 | + +* Guidelines are provided for moderate queries. Queries vary widely in their impact on the system. For simple or complex queries, we recommend testing and adjusting the suggested requirements as needed. See [query guidelines](#query-guidelines) for detail. + +## When do I need more RAM? + +In general, more RAM helps queries return faster. Your RAM requirements are primarily determined by [series cardinality](/influxdb/v1.8/concepts/glossary/#series-cardinality). Higher cardinality requires more RAM. Regardless of RAM, a series cardinality of 10 million or more can cause OOM (out of memory) failures. You can usually resolve OOM issues by redesigning your [schema](/influxdb/v1.8/concepts/glossary/#schema). + + +## Guidelines per cluster + +InfluxDB Enterprise guidelines vary by writes and queries per second, series cardinality, replication factor, and infrastructure-AWS EC2 R4 instances or equivalent: +- R4.xlarge (4 cores) +- R4.2xlarge (8 cores) +- R4.4xlarge (16 cores) +- R4.8xlarge (32 cores) + +> Guidelines stem from a DevOps monitoring use case: maintaining a group of computers and monitoring server metrics (such as CPU, kernel, memory, disk space, disk I/O, network, and so on). + +### Recommended cluster configurations + +Cluster configurations guidelines are organized by: + +- Series cardinality in your data set: 10,000, 100,000, 1,000,000, or 10,000,000 +- Number of data nodes +- Number of server cores + +For each cluster configuration, you'll find guidelines for the following: + +- **maximum writes per second only** (no dashboard queries are running) +- **maximum queries per second only** (no data is being written) +- **maximum simultaneous queries and writes per second, combined** + +#### Review cluster configuration tables + +1. Select the series cardinality tab below, and then click to expand a replication factor. +2. In the **Nodes x Core** column, find the number of data nodes and server cores in your configuration, and then review the recommended **maximum** guidelines. + +{{< tabs-wrapper >}} +{{% tabs %}} +[10,000 series](#) +[100,000 series](#) +[1,000,000 series](#) +[10,000,000 series](#) +{{% /tabs %}} +{{% tab-content %}} + +Select one of the following replication factors to see the recommended cluster configuration for 10,000 series: + +{{% expand "Replication factor, 1" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 1 x 4 | 188,000 | 5 | 4 + 99,000 | +| 1 x 8 | 405,000 | 9 | 8 + 207,000 | +| 1 x 16 | 673,000 | 15 | 14 + 375,000 | +| 1 x 32 | 1,056,000 | 24 | 22 + 650,000 | +| 2 x 4 | 384,000 | 14 | 14 + 184,000 | +| 2 x 8 | 746,000 | 22 | 22 + 334,000 | +| 2 x 16 | 1,511,000 | 56 | 40 + 878,000 | +| 2 x 32 | 2,426,000 | 96 | 68 + 1,746,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 2" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 2 x 4 | 296,000 | 16 | 16 + 151,000 | +| 2 x 8 | 560,000 | 30 | 26 + 290,000 | +| 2 x 16 | 972,000 | 54 | 50 + 456,000 | +| 2 x 32 | 1,860,000 | 84 | 74 + 881,000 | +| 4 x 8 | 1,781,000 | 100 | 64 + 682,000 | +| 4 x 16 | 3,430,000 | 192 | 104 + 1,732,000 | +| 4 x 32 | 6,351,000 | 432 | 188 + 3,283,000 | +| 6 x 8 | 2,923,000 | 216 | 138 + 1,049,000 | +| 6 x 16 | 5,650,000 | 498 | 246 + 2,246,000 | +| 6 x 32 | 9,842,000 | 1248 | 528 + 5,229,000 | +| 8 x 8 | 3,987,000 | 632 | 336 + 1,722,000 | +| 8 x 16 | 7,798,000 | 1384 | 544 + 3,911,000 | +| 8 x 32 | 13,189,000 | 3648 | 1,152 + 7,891,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 3" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 3 x 8 | 815,000 | 63 | 54 + 335,000 | +| 3 x 16 | 1,688,000 | 120 | 87 + 705,000 | +| 3 x 32 | 3,164,000 | 255 | 132 + 1,626,000 | +| 6 x 8 | 2,269,000 | 252 | 168 + 838,000 | +| 6 x 16 | 4,593,000 | 624 | 336 + 2,019,000 | +| 6 x 32 | 7,776,000 | 1340 | 576 + 3,624,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 4" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 4 x 8 | 1,028,000 | 116 | 98 + 365,000 | +| 4 x 16 | 2,067,000 | 208 | 140 + 8,056,000 | +| 4 x 32 | 3,290,000 | 428 | 228 + 1,892,000 | +| 8 x 8 | 2,813,000 | 928 | 496 + 1,225,000 | +| 8 x 16 | 5,225,000 | 2176 | 800 + 2,799,000 | +| 8 x 32 | 8,555,000 | 5184 | 1088 + 6,055,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 6" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 6 x 8 | 1,261,000 | 288 | 192 + 522,000 | +| 6 x 16 | 2,370,000 | 576 | 288 + 1,275,000 | +| 6 x 32 | 3,601,000 | 1056 | 336 + 2,390,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 8" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| ----------------: | -----------------: |:---------------------------:| +| 8 x 8 | 1,382,000 | 1184 | 416 + 915,000 | +| 8 x 16 | 2,658,000 | 2504 | 448 + 2,204,000 | +| 8 x 32 | 3,887,000 | 5184 | 602 + 4,120,000 | + +{{% /expand %}} + +{{% /tab-content %}} + +{{% tab-content %}} + +Select one of the following replication factors to see the recommended cluster configuration for 100,000 series: + +{{% expand "Replication factor, 1" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 1 x 4 | 143,000 | 5 | 4 + 77,000 | +| 1 x 8 | 322,000 | 9 | 8 + 167,000 | +| 1 x 16 | 624,000 | 17 | 12 + 337,000 | +| 1 x 32 | 1,114,000 | 26 | 18 + 657,000 | +| 2 x 4 | 265,000 | 14 | 12 + 115,000 | +| 2 x 8 | 573,000 | 30 | 22 + 269,000 | +| 2 x 16 | 1,261,000 | 52 | 38 + 679,000 | +| 2 x 32 | 2,335,000 | 90 | 66 + 1,510,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 2" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 2 x 4 | 196,000 | 16 | 14 + 77,000 | +| 2 x 8 | 482,000 | 30 | 24 + 203,000 | +| 2 x 16 | 1,060,000 | 60 | 42 + 415,000 | +| 2 x 32 | 1,958,000 | 94 | 64 + 984,000 | +| 4 x 8 | 1,144,000 | 108 | 68 + 406,000 | +| 4 x 16 | 2,512,000 | 228 | 148 + 866,000 | +| 4 x 32 | 4,346,000 | 564 | 320 + 1,886,000 | +| 6 x 8 | 1,802,000 | 252 | 156 + 618,000 | +| 6 x 16 | 3,924,000 | 562 | 384 + 1,068,000 | +| 6 x 32 | 6,533,000 | 1340 | 912 + 2,083,000 | +| 8 x 8 | 2,516,000 | 712 | 360 + 1,020,000 | +| 8 x 16 | 5,478,000 | 1632 | 1,024 + 1,843,000 | +| 8 x 32 | 1,0527,000 | 3392 | 1,792 + 4,998,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 3" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 3 x 8 | 616,000 | 72 | 51 + 218,000 | +| 3 x 16 | 1,268,000 | 117 | 84 + 438,000 | +| 3 x 32 | 2,260,000 | 189 | 114 + 984,000 | +| 6 x 8 | 1,393,000 | 294 | 192 + 421,000 | +| 6 x 16 | 3,056,000 | 726 | 456 + 893,000 | +| 6 x 32 | 5,017,000 | 1584 | 798 + 1,098,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 4" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 4 x 8 | 635,000 | 112 | 80 + 207,000 | +| 4 x 16 | 1,359,000 | 188 | 124 + 461,000 | +| 4 x 32 | 2,320,000 | 416 | 192 + 1,102,000 | +| 8 x 8 | 1,570,000 | 1360 | 816 + 572,000 | +| 8 x 16 | 3,205,000 | 2720 | 832 + 2,053,000 | +| 8 x 32 | 3,294,000 | 2592 | 804 + 2,174,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 6" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 6 x 8 | 694,000 | 302 | 198 + 204,000 | +| 6 x 16 | 1,397,000 | 552 | 360 + 450,000 | +| 6 x 32 | 2,298,000 | 1248 | 384 + 1,261,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 8" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| ----------------: | -----------------: |:---------------------------:| +| 8 x 8 | 739,000 | 1296 | 480 + 371,000 | +| 8 x 16 | 1,396,000 | 2592 | 672 + 843,000 | +| 8 x 32 | 2,614,000 | 2496 | 960 + 1,371,000 | + +{{% /expand %}} + +{{% /tab-content %}} + +{{% tab-content %}} + +Select one of the following replication factors to see the recommended cluster configuration for 1,000,000 series: + +{{% expand "Replication factor, 2" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:-------------:|------------------:|-------------------:|:---------------------------:| +| 2 x 4 | 104,000 | 18 | 12 + 54,000 | +| 2 x 8 | 195,000 | 36 | 24 + 99,000 | +| 2 x 16 | 498,000 | 70 | 44 + 145,000 | +| 2 x 32 | 1,195,000 | 102 | 84 + 232,000 | +| 4 x 8 | 488,000 | 120 | 56 + 222,000 | +| 4 x 16 | 1,023,000 | 244 | 112 + 428,000 | +| 4 x 32 | 2,686,000 | 468 | 208 + 729,000 | +| 6 x 8 | 845,000 | 270 | 126 + 356,000 | +| 6 x 16 | 1,780,000 | 606 | 288 + 663,000 | +| 6 x 32 | 430,000 | 1,488 | 624 + 1,209,000 | +| 8 x 8 | 1,831,000 | 808 | 296 + 778,000 | +| 8 x 16 | 4,167,000 | 1,856 | 640 + 2,031,000 | +| 8 x 32 | 7,813,000 | 3,201 | 896 + 4,897,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 3" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 3 x 8 | 234,000 | 72 | 42 + 87,000 | +| 3 x 16 | 613,000 | 120 | 75 + 166,000 | +| 3 x 32 | 1,365,000 | 141 | 114 + 984,000 | +| 6 x 8 | 593,000 | 318 | 144 + 288,000 | +| 6 x 16 | 1,545,000 | 744 | 384 + 407,000 | +| 6 x 32 | 3,204,000 | 1632 | 912 + 505,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 4" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 4 x 8 | 258,000 | 116 | 68 + 73,000 | +| 4 x 16 | 675,000 | 196 | 132 + 140,000 | +| 4 x 32 | 1,513,000 | 244 | 176 + 476,000 | +| 8 x 8 | 614,000 | 1096 | 400 + 258,000 | +| 8 x 16 | 1,557,000 | 2496 | 1152 + 436,000 | +| 8 x 32 | 3,265,000 | 4288 | 2240 + 820,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 6" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 6 x 8 | 694,000 | 302 | 198 + 204,000 | +| 6 x 16 | 1,397,000 | 552 | 360 + 450,000 | +| 6 x 32 | 2,298,000 | 1248 | 384 + 1,261,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 8" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| ----------------: | -----------------: |:---------------------------:| +| 8 x 8 | 739,000 | 1296 | 480 + 371,000 | +| 8 x 16 | 1,396,000 | 2592 | 672 + 843,000 | +| 8 x 32 | 2,614,000 | 2496 | 960 + 1,371,000 | + +{{% /expand %}} + +{{% /tab-content %}} + +{{% tab-content %}} + +Select one of the following replication factors to see the recommended cluster configuration for 10,000,000 series: + +{{% expand "Replication factor, 1" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 2 x 4 | 122,000 | 16 | 12 + 81,000 | +| 2 x 8 | 259,000 | 36 | 24 + 143,000 | +| 2 x 16 | 501,000 | 66 | 44 + 290,000 | +| 2 x 32 | 646,000 | 142 | 54 + 400,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 2" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 2 x 4 | 87,000 | 18 | 14 + 56,000 | +| 2 x 8 | 169,000 | 38 | 24 + 98,000 | +| 2 x 16 | 334,000 | 76 | 46 + 224,000 | +| 2 x 32 | 534,000 | 136 | 58 + 388,000 | +| 4 x 8 | 335,000 | 120 | 60 + 204,000 | +| 4 x 16 | 643,000 | 256 | 112 + 395,000 | +| 4 x 32 | 967,000 | 560 | 158 + 806,000 | +| 6 x 8 | 521,000 | 378 | 144 + 319,000 | +| 6 x 16 | 890,000 | 582 | 186 + 513,000 | +| 8 x 8 | 699,000 | 1,032 | 256 + 477,000 | +| 8 x 16 | 1,345,000 | 2,048 | 544 + 741,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 3" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 3 x 8 | 170,000 | 60 | 42 + 98,000 | +| 3 x 16 | 333,000 | 129 | 76 + 206,000 | +| 3 x 32 | 609,000 | 178 | 60 + 162,000 | +| 6 x 8 | 395,000 | 402 | 132 + 247,000 | +| 6 x 16 | 679,000 | 894 | 150 + 527,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 4" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 4 x 8 | 183365 | 132 | 52 + 100,000 | + +{{% /expand %}} + +{{% /tab-content %}} +{{< /tabs-wrapper >}} + +## Storage: type, amount, and configuration + +### Storage volume and IOPS + +Consider the type of storage you need and the amount. InfluxDB is designed to run on solid state drives (SSDs) and memory-optimized cloud instances, for example, AWS EC2 R5 or R4 instances. InfluxDB isn't tested on hard disk drives (HDDs) and we don't recommend HDDs for production. For best results, InfluxDB servers must have a minimum of 1000 IOPS on storage to ensure recovery and availability. We recommend at least 2000 IOPS for rapid recovery of cluster data nodes after downtime. + +See your cloud provider documentation for IOPS detail on your storage volumes. + +### Bytes and compression + +Database names, [measurements](/influxdb/v1.8/concepts/glossary/#measurement), [tag keys](/influxdb/v1.8/concepts/glossary/#tag-key), [field keys](/influxdb/v1.8/concepts/glossary/#field-key), and [tag values](/influxdb/v1.8/concepts/glossary/#tag-value) are stored only once and always as strings. [Field values](/influxdb/v1.8/concepts/glossary/#field-value) and [timestamps](/influxdb/v1.8/concepts/glossary/#timestamp) are stored for every point. + +Non-string values require approximately three bytes. String values require variable space, determined by string compression. + +### Separate `wal` and `data` directories + +When running InfluxDB in a production environment, store the `wal` directory and the `data` directory on separate storage devices. This optimization significantly reduces disk contention under heavy write load──an important consideration if the write load is highly variable. If the write load does not vary by more than 15%, the optimization is probably not necessary. From c2fbec9be13016b1ad45ba7c3cdecfe223d9e9ac Mon Sep 17 00:00:00 2001 From: lwandzura <51929958+lwandzura@users.noreply.github.com> Date: Wed, 10 Nov 2021 11:56:09 -0600 Subject: [PATCH 02/18] delete OSS info --- .../v1.9/guides/hardware_sizing.md | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md index 3d2ff8cac..d4ce118b2 100644 --- a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md @@ -55,20 +55,6 @@ For **simple** or **complex** queries, we recommend testing and adjusting the su | | May sample a very large time range of months or years | | | Typically take multiple seconds to execute | -## InfluxDB OSS guidelines - -Run InfluxDB on locally attached solid state drives (SSDs). Other storage configurations have lower performance and may not be able to recover from small interruptions in normal processing. - -Estimated guidelines include writes per second, queries per second, and number of unique [series](/influxdb/v1.8/concepts/glossary/#series), CPU, RAM, and IOPS (input/output operations per second). - -| vCPU or CPU | RAM | IOPS | Writes per second | Queries* per second | Unique series | -| ----------: | ------: | -------: | ----------------: | ------------------: | ------------: | -| 2-4 cores | 2-4 GB | 500 | < 5,000 | < 5 | < 100,000 | -| 4-6 cores | 8-32 GB | 500-1000 | < 250,000 | < 25 | < 1,000,000 | -| 8+ cores | 32+ GB | 1000+ | > 250,000 | > 25 | > 1,000,000 | - -* **Queries per second for moderate queries.** Queries vary widely in their impact on the system. For simple or complex queries, we recommend testing and adjusting the suggested requirements as needed. See [query guidelines](#query-guidelines) for details. - ## InfluxDB Enterprise cluster guidelines ### Meta nodes From 60d5c84167392689d3a72473cf8553ef4418fc1b Mon Sep 17 00:00:00 2001 From: lwandzura <51929958+lwandzura@users.noreply.github.com> Date: Wed, 10 Nov 2021 13:39:38 -0600 Subject: [PATCH 03/18] updated hardware sizing for 1.8 --- .../v1.8/guides/hardware_sizing.md | 459 ++++++++++++++++++ 1 file changed, 459 insertions(+) create mode 100644 content/enterprise_influxdb/v1.8/guides/hardware_sizing.md diff --git a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md new file mode 100644 index 000000000..04021e02a --- /dev/null +++ b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md @@ -0,0 +1,459 @@ +--- +title: Hardware sizing guidelines +Description: > + Review configuration and hardware guidelines for InfluxDB OSS (open source) and InfluxDB Enterprise. +menu: + enterprise_influxdb_1_8: + weight: 40 + parent: Guides +--- + +Review configuration and hardware guidelines for InfluxDB OSS (open source) and InfluxDB Enterprise: + +* [Single node or cluster?](#single-node-or-cluster) +* [Query guidelines](#query-guidelines) +* [InfluxDB OSS guidelines](#influxdb-oss-guidelines) +* [InfluxDB Enterprise cluster guidelines](#influxdb-enterprise-cluster-guidelines) +* [When do I need more RAM?](#when-do-i-need-more-ram) +* [Recommended cluster configurations](#recommended-cluster-configurations) +* [Storage: type, amount, and configuration](#storage-type-amount-and-configuration) + +> **Disclaimer:** Your numbers may vary from recommended guidelines. Guidelines provide estimated benchmarks for implementing the most performant system for your business. + +## Single node or cluster? + +If your InfluxDB performance requires any of the following, a single node (InfluxDB OSS) may not support your needs: + +- more than 750,000 field writes per second +- more than 100 moderate queries per second ([see Query guides](#query-guidelines)) +- more than 10,000,000 [series cardinality](/influxdb/v1.8/concepts/glossary/#series-cardinality) + +We recommend InfluxDB Enterprise, which supports multiple data nodes (a cluster) across multiple server cores. +InfluxDB Enterprise distributes multiple copies of your data across a cluster, +providing high-availability and redundancy, so an unavailable node doesn’t significantly impact the cluster. +Please [contact us](https://www.influxdata.com/contact-sales/) for assistance tuning your system. + +If you want a single node instance of InfluxDB that's fully open source, requires fewer writes, queries, and unique series than listed above, and do **not require** redundancy, we recommend InfluxDB OSS. + +> **Note:** Without the redundancy of a cluster, writes and queries fail immediately when a server is unavailable. + +## Query guidelines + +> Query complexity varies widely on system impact. Recommendations for both single nodes and clusters are based on **moderate** query loads. + +For **simple** or **complex** queries, we recommend testing and adjusting the suggested requirements as needed. Query complexity is defined by the following criteria: + +| Query complexity | Criteria | +|------------------|---------------------------------------------------------------------------------------| +| Simple | Have few or no functions and no regular expressions | +| | Are bounded in time to a few minutes, hours, or 24 hours at most | +| | Typically execute in a few milliseconds to a few dozen milliseconds | +| Moderate | Have multiple functions and one or two regular expressions | +| | May also have `GROUP BY` clauses or sample a time range of multiple weeks | +| | Typically execute in a few hundred or a few thousand milliseconds | +| Complex | Have multiple aggregation or transformation functions or multiple regular expressions | +| | May sample a very large time range of months or years | +| | Typically take multiple seconds to execute | + +## InfluxDB Enterprise cluster guidelines + +### Meta nodes + +> Set up clusters with an odd number of meta nodes──an even number may cause issues in certain configurations. + +A cluster must have a **minimum of three** independent meta nodes for data redundancy and availability. A cluster with `2n + 1` meta nodes can tolerate the loss of `n` meta nodes. + +Meta nodes do not need very much computing power. Regardless of the cluster load, we recommend the following guidelines for the meta nodes: + +* vCPU or CPU: 1-2 cores +* RAM: 512 MB - 1 GB +* IOPS: 50 + +### Web node + +The InfluxDB Enterprise web server is primarily an HTTP server with similar load requirements. For most applications, the server doesn't need to be very robust. A cluster can function with only one web server, but for redundancy, we recommend connecting multiple web servers to a single back-end Postgres database. + +> **Note:** Production clusters should not use the SQLite database (lacks support for redundant web servers and handling high loads). + +* vCPU or CPU: 2-4 cores +* RAM: 2-4 GB +* IOPS: 100 + +### Data nodes + +A cluster with one data node is valid but has no data redundancy. Redundancy is set by the [replication factor](/influxdb/v1.8/concepts/glossary/#replication-factor) on the retention policy the data is written to. Where `n` is the replication factor, a cluster can lose `n - 1` data nodes and return complete query results. + +>**Note:** For optimal data distribution within the cluster, use an even number of data nodes. + +Guidelines vary by writes per second per node, moderate queries per second per node, and the number of unique series per node. + +#### Guidelines per node + +| vCPU or CPU | RAM | IOPS | Writes per second | Queries* per second | Unique series | +| ----------: | -------: | ----: | ----------------: | ------------------: | ------------: | +| 2 cores | 4-8 GB | 1000 | < 5,000 | < 5 | < 100,000 | +| 4-6 cores | 16-32 GB | 1000+ | < 100,000 | < 25 | < 1,000,000 | +| 8+ cores | 32+ GB | 1000+ | > 100,000 | > 25 | > 1,000,000 | + +* Guidelines are provided for moderate queries. Queries vary widely in their impact on the system. For simple or complex queries, we recommend testing and adjusting the suggested requirements as needed. See [query guidelines](#query-guidelines) for detail. + +## When do I need more RAM? + +In general, more RAM helps queries return faster. Your RAM requirements are primarily determined by [series cardinality](/influxdb/v1.8/concepts/glossary/#series-cardinality). Higher cardinality requires more RAM. Regardless of RAM, a series cardinality of 10 million or more can cause OOM (out of memory) failures. You can usually resolve OOM issues by redesigning your [schema](/influxdb/v1.8/concepts/glossary/#schema). + + +## Guidelines per cluster + +InfluxDB Enterprise guidelines vary by writes and queries per second, series cardinality, replication factor, and infrastructure-AWS EC2 R4 instances or equivalent: +- R4.xlarge (4 cores) +- R4.2xlarge (8 cores) +- R4.4xlarge (16 cores) +- R4.8xlarge (32 cores) + +> Guidelines stem from a DevOps monitoring use case: maintaining a group of computers and monitoring server metrics (such as CPU, kernel, memory, disk space, disk I/O, network, and so on). + +### Recommended cluster configurations + +Cluster configurations guidelines are organized by: + +- Series cardinality in your data set: 10,000, 100,000, 1,000,000, or 10,000,000 +- Number of data nodes +- Number of server cores + +For each cluster configuration, you'll find guidelines for the following: + +- **maximum writes per second only** (no dashboard queries are running) +- **maximum queries per second only** (no data is being written) +- **maximum simultaneous queries and writes per second, combined** + +#### Review cluster configuration tables + +1. Select the series cardinality tab below, and then click to expand a replication factor. +2. In the **Nodes x Core** column, find the number of data nodes and server cores in your configuration, and then review the recommended **maximum** guidelines. + +{{< tabs-wrapper >}} +{{% tabs %}} +[10,000 series](#) +[100,000 series](#) +[1,000,000 series](#) +[10,000,000 series](#) +{{% /tabs %}} +{{% tab-content %}} + +Select one of the following replication factors to see the recommended cluster configuration for 10,000 series: + +{{% expand "Replication factor, 1" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 1 x 4 | 188,000 | 5 | 4 + 99,000 | +| 1 x 8 | 405,000 | 9 | 8 + 207,000 | +| 1 x 16 | 673,000 | 15 | 14 + 375,000 | +| 1 x 32 | 1,056,000 | 24 | 22 + 650,000 | +| 2 x 4 | 384,000 | 14 | 14 + 184,000 | +| 2 x 8 | 746,000 | 22 | 22 + 334,000 | +| 2 x 16 | 1,511,000 | 56 | 40 + 878,000 | +| 2 x 32 | 2,426,000 | 96 | 68 + 1,746,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 2" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 2 x 4 | 296,000 | 16 | 16 + 151,000 | +| 2 x 8 | 560,000 | 30 | 26 + 290,000 | +| 2 x 16 | 972,000 | 54 | 50 + 456,000 | +| 2 x 32 | 1,860,000 | 84 | 74 + 881,000 | +| 4 x 8 | 1,781,000 | 100 | 64 + 682,000 | +| 4 x 16 | 3,430,000 | 192 | 104 + 1,732,000 | +| 4 x 32 | 6,351,000 | 432 | 188 + 3,283,000 | +| 6 x 8 | 2,923,000 | 216 | 138 + 1,049,000 | +| 6 x 16 | 5,650,000 | 498 | 246 + 2,246,000 | +| 6 x 32 | 9,842,000 | 1248 | 528 + 5,229,000 | +| 8 x 8 | 3,987,000 | 632 | 336 + 1,722,000 | +| 8 x 16 | 7,798,000 | 1384 | 544 + 3,911,000 | +| 8 x 32 | 13,189,000 | 3648 | 1,152 + 7,891,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 3" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 3 x 8 | 815,000 | 63 | 54 + 335,000 | +| 3 x 16 | 1,688,000 | 120 | 87 + 705,000 | +| 3 x 32 | 3,164,000 | 255 | 132 + 1,626,000 | +| 6 x 8 | 2,269,000 | 252 | 168 + 838,000 | +| 6 x 16 | 4,593,000 | 624 | 336 + 2,019,000 | +| 6 x 32 | 7,776,000 | 1340 | 576 + 3,624,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 4" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 4 x 8 | 1,028,000 | 116 | 98 + 365,000 | +| 4 x 16 | 2,067,000 | 208 | 140 + 8,056,000 | +| 4 x 32 | 3,290,000 | 428 | 228 + 1,892,000 | +| 8 x 8 | 2,813,000 | 928 | 496 + 1,225,000 | +| 8 x 16 | 5,225,000 | 2176 | 800 + 2,799,000 | +| 8 x 32 | 8,555,000 | 5184 | 1088 + 6,055,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 6" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 6 x 8 | 1,261,000 | 288 | 192 + 522,000 | +| 6 x 16 | 2,370,000 | 576 | 288 + 1,275,000 | +| 6 x 32 | 3,601,000 | 1056 | 336 + 2,390,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 8" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| ----------------: | -----------------: |:---------------------------:| +| 8 x 8 | 1,382,000 | 1184 | 416 + 915,000 | +| 8 x 16 | 2,658,000 | 2504 | 448 + 2,204,000 | +| 8 x 32 | 3,887,000 | 5184 | 602 + 4,120,000 | + +{{% /expand %}} + +{{% /tab-content %}} + +{{% tab-content %}} + +Select one of the following replication factors to see the recommended cluster configuration for 100,000 series: + +{{% expand "Replication factor, 1" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 1 x 4 | 143,000 | 5 | 4 + 77,000 | +| 1 x 8 | 322,000 | 9 | 8 + 167,000 | +| 1 x 16 | 624,000 | 17 | 12 + 337,000 | +| 1 x 32 | 1,114,000 | 26 | 18 + 657,000 | +| 2 x 4 | 265,000 | 14 | 12 + 115,000 | +| 2 x 8 | 573,000 | 30 | 22 + 269,000 | +| 2 x 16 | 1,261,000 | 52 | 38 + 679,000 | +| 2 x 32 | 2,335,000 | 90 | 66 + 1,510,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 2" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 2 x 4 | 196,000 | 16 | 14 + 77,000 | +| 2 x 8 | 482,000 | 30 | 24 + 203,000 | +| 2 x 16 | 1,060,000 | 60 | 42 + 415,000 | +| 2 x 32 | 1,958,000 | 94 | 64 + 984,000 | +| 4 x 8 | 1,144,000 | 108 | 68 + 406,000 | +| 4 x 16 | 2,512,000 | 228 | 148 + 866,000 | +| 4 x 32 | 4,346,000 | 564 | 320 + 1,886,000 | +| 6 x 8 | 1,802,000 | 252 | 156 + 618,000 | +| 6 x 16 | 3,924,000 | 562 | 384 + 1,068,000 | +| 6 x 32 | 6,533,000 | 1340 | 912 + 2,083,000 | +| 8 x 8 | 2,516,000 | 712 | 360 + 1,020,000 | +| 8 x 16 | 5,478,000 | 1632 | 1,024 + 1,843,000 | +| 8 x 32 | 1,0527,000 | 3392 | 1,792 + 4,998,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 3" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 3 x 8 | 616,000 | 72 | 51 + 218,000 | +| 3 x 16 | 1,268,000 | 117 | 84 + 438,000 | +| 3 x 32 | 2,260,000 | 189 | 114 + 984,000 | +| 6 x 8 | 1,393,000 | 294 | 192 + 421,000 | +| 6 x 16 | 3,056,000 | 726 | 456 + 893,000 | +| 6 x 32 | 5,017,000 | 1584 | 798 + 1,098,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 4" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 4 x 8 | 635,000 | 112 | 80 + 207,000 | +| 4 x 16 | 1,359,000 | 188 | 124 + 461,000 | +| 4 x 32 | 2,320,000 | 416 | 192 + 1,102,000 | +| 8 x 8 | 1,570,000 | 1360 | 816 + 572,000 | +| 8 x 16 | 3,205,000 | 2720 | 832 + 2,053,000 | +| 8 x 32 | 3,294,000 | 2592 | 804 + 2,174,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 6" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 6 x 8 | 694,000 | 302 | 198 + 204,000 | +| 6 x 16 | 1,397,000 | 552 | 360 + 450,000 | +| 6 x 32 | 2,298,000 | 1248 | 384 + 1,261,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 8" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| ----------------: | -----------------: |:---------------------------:| +| 8 x 8 | 739,000 | 1296 | 480 + 371,000 | +| 8 x 16 | 1,396,000 | 2592 | 672 + 843,000 | +| 8 x 32 | 2,614,000 | 2496 | 960 + 1,371,000 | + +{{% /expand %}} + +{{% /tab-content %}} + +{{% tab-content %}} + +Select one of the following replication factors to see the recommended cluster configuration for 1,000,000 series: + +{{% expand "Replication factor, 2" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:-------------:|------------------:|-------------------:|:---------------------------:| +| 2 x 4 | 104,000 | 18 | 12 + 54,000 | +| 2 x 8 | 195,000 | 36 | 24 + 99,000 | +| 2 x 16 | 498,000 | 70 | 44 + 145,000 | +| 2 x 32 | 1,195,000 | 102 | 84 + 232,000 | +| 4 x 8 | 488,000 | 120 | 56 + 222,000 | +| 4 x 16 | 1,023,000 | 244 | 112 + 428,000 | +| 4 x 32 | 2,686,000 | 468 | 208 + 729,000 | +| 6 x 8 | 845,000 | 270 | 126 + 356,000 | +| 6 x 16 | 1,780,000 | 606 | 288 + 663,000 | +| 6 x 32 | 430,000 | 1,488 | 624 + 1,209,000 | +| 8 x 8 | 1,831,000 | 808 | 296 + 778,000 | +| 8 x 16 | 4,167,000 | 1,856 | 640 + 2,031,000 | +| 8 x 32 | 7,813,000 | 3,201 | 896 + 4,897,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 3" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 3 x 8 | 234,000 | 72 | 42 + 87,000 | +| 3 x 16 | 613,000 | 120 | 75 + 166,000 | +| 3 x 32 | 1,365,000 | 141 | 114 + 984,000 | +| 6 x 8 | 593,000 | 318 | 144 + 288,000 | +| 6 x 16 | 1,545,000 | 744 | 384 + 407,000 | +| 6 x 32 | 3,204,000 | 1632 | 912 + 505,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 4" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 4 x 8 | 258,000 | 116 | 68 + 73,000 | +| 4 x 16 | 675,000 | 196 | 132 + 140,000 | +| 4 x 32 | 1,513,000 | 244 | 176 + 476,000 | +| 8 x 8 | 614,000 | 1096 | 400 + 258,000 | +| 8 x 16 | 1,557,000 | 2496 | 1152 + 436,000 | +| 8 x 32 | 3,265,000 | 4288 | 2240 + 820,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 6" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 6 x 8 | 694,000 | 302 | 198 + 204,000 | +| 6 x 16 | 1,397,000 | 552 | 360 + 450,000 | +| 6 x 32 | 2,298,000 | 1248 | 384 + 1,261,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 8" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| ----------------: | -----------------: |:---------------------------:| +| 8 x 8 | 739,000 | 1296 | 480 + 371,000 | +| 8 x 16 | 1,396,000 | 2592 | 672 + 843,000 | +| 8 x 32 | 2,614,000 | 2496 | 960 + 1,371,000 | + +{{% /expand %}} + +{{% /tab-content %}} + +{{% tab-content %}} + +Select one of the following replication factors to see the recommended cluster configuration for 10,000,000 series: + +{{% expand "Replication factor, 1" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 2 x 4 | 122,000 | 16 | 12 + 81,000 | +| 2 x 8 | 259,000 | 36 | 24 + 143,000 | +| 2 x 16 | 501,000 | 66 | 44 + 290,000 | +| 2 x 32 | 646,000 | 142 | 54 + 400,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 2" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 2 x 4 | 87,000 | 18 | 14 + 56,000 | +| 2 x 8 | 169,000 | 38 | 24 + 98,000 | +| 2 x 16 | 334,000 | 76 | 46 + 224,000 | +| 2 x 32 | 534,000 | 136 | 58 + 388,000 | +| 4 x 8 | 335,000 | 120 | 60 + 204,000 | +| 4 x 16 | 643,000 | 256 | 112 + 395,000 | +| 4 x 32 | 967,000 | 560 | 158 + 806,000 | +| 6 x 8 | 521,000 | 378 | 144 + 319,000 | +| 6 x 16 | 890,000 | 582 | 186 + 513,000 | +| 8 x 8 | 699,000 | 1,032 | 256 + 477,000 | +| 8 x 16 | 1,345,000 | 2,048 | 544 + 741,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 3" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:|------------------:|-------------------:|:---------------------------:| +| 3 x 8 | 170,000 | 60 | 42 + 98,000 | +| 3 x 16 | 333,000 | 129 | 76 + 206,000 | +| 3 x 32 | 609,000 | 178 | 60 + 162,000 | +| 6 x 8 | 395,000 | 402 | 132 + 247,000 | +| 6 x 16 | 679,000 | 894 | 150 + 527,000 | + +{{% /expand %}} + +{{% expand "Replication factor, 4" %}} + +| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | +|:------------:| -----------------:| ------------------:|:---------------------------:| +| 4 x 8 | 183365 | 132 | 52 + 100,000 | + +{{% /expand %}} + +{{% /tab-content %}} +{{< /tabs-wrapper >}} + +## Storage: type, amount, and configuration + +### Storage volume and IOPS + +Consider the type of storage you need and the amount. InfluxDB is designed to run on solid state drives (SSDs) and memory-optimized cloud instances, for example, AWS EC2 R5 or R4 instances. InfluxDB isn't tested on hard disk drives (HDDs) and we don't recommend HDDs for production. For best results, InfluxDB servers must have a minimum of 1000 IOPS on storage to ensure recovery and availability. We recommend at least 2000 IOPS for rapid recovery of cluster data nodes after downtime. + +See your cloud provider documentation for IOPS detail on your storage volumes. + +### Bytes and compression + +Database names, [measurements](/influxdb/v1.8/concepts/glossary/#measurement), [tag keys](/influxdb/v1.8/concepts/glossary/#tag-key), [field keys](/influxdb/v1.8/concepts/glossary/#field-key), and [tag values](/influxdb/v1.8/concepts/glossary/#tag-value) are stored only once and always as strings. [Field values](/influxdb/v1.8/concepts/glossary/#field-value) and [timestamps](/influxdb/v1.8/concepts/glossary/#timestamp) are stored for every point. + +Non-string values require approximately three bytes. String values require variable space, determined by string compression. + +### Separate `wal` and `data` directories + +When running InfluxDB in a production environment, store the `wal` directory and the `data` directory on separate storage devices. This optimization significantly reduces disk contention under heavy write load──an important consideration if the write load is highly variable. If the write load does not vary by more than 15%, the optimization is probably not necessary. From 0781e8ce4321fd8f5090d389be28c34159693f01 Mon Sep 17 00:00:00 2001 From: lwandzura <51929958+lwandzura@users.noreply.github.com> Date: Wed, 10 Nov 2021 14:12:45 -0600 Subject: [PATCH 04/18] Update hardware_sizing.md removed web node from enterprise 1.8 hardware sizing guideline --- .../enterprise_influxdb/v1.8/guides/hardware_sizing.md | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md index 04021e02a..f089971cc 100644 --- a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md @@ -69,16 +69,6 @@ Meta nodes do not need very much computing power. Regardless of the cluster load * RAM: 512 MB - 1 GB * IOPS: 50 -### Web node - -The InfluxDB Enterprise web server is primarily an HTTP server with similar load requirements. For most applications, the server doesn't need to be very robust. A cluster can function with only one web server, but for redundancy, we recommend connecting multiple web servers to a single back-end Postgres database. - -> **Note:** Production clusters should not use the SQLite database (lacks support for redundant web servers and handling high loads). - -* vCPU or CPU: 2-4 cores -* RAM: 2-4 GB -* IOPS: 100 - ### Data nodes A cluster with one data node is valid but has no data redundancy. Redundancy is set by the [replication factor](/influxdb/v1.8/concepts/glossary/#replication-factor) on the retention policy the data is written to. Where `n` is the replication factor, a cluster can lose `n - 1` data nodes and return complete query results. From a987f53f9bcf6a965bf398a012821b193fbff389 Mon Sep 17 00:00:00 2001 From: lwandzura <51929958+lwandzura@users.noreply.github.com> Date: Wed, 10 Nov 2021 14:22:52 -0600 Subject: [PATCH 05/18] Update hardware_sizing.md removed web node from enterprise 1.9 sizing guideline --- .../enterprise_influxdb/v1.9/guides/hardware_sizing.md | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md index d4ce118b2..75f16e19c 100644 --- a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md @@ -69,16 +69,6 @@ Meta nodes do not need very much computing power. Regardless of the cluster load * RAM: 512 MB - 1 GB * IOPS: 50 -### Web node - -The InfluxDB Enterprise web server is primarily an HTTP server with similar load requirements. For most applications, the server doesn't need to be very robust. A cluster can function with only one web server, but for redundancy, we recommend connecting multiple web servers to a single back-end Postgres database. - -> **Note:** Production clusters should not use the SQLite database (lacks support for redundant web servers and handling high loads). - -* vCPU or CPU: 2-4 cores -* RAM: 2-4 GB -* IOPS: 100 - ### Data nodes A cluster with one data node is valid but has no data redundancy. Redundancy is set by the [replication factor](/influxdb/v1.8/concepts/glossary/#replication-factor) on the retention policy the data is written to. Where `n` is the replication factor, a cluster can lose `n - 1` data nodes and return complete query results. From 8841b10b3f07ab41cc29fcba98c5c8ab7b4a3c5b Mon Sep 17 00:00:00 2001 From: lwandzura <51929958+lwandzura@users.noreply.github.com> Date: Wed, 10 Nov 2021 14:40:28 -0600 Subject: [PATCH 06/18] Update hardware_sizing.md removed anything above RF 3 from Enterprise 1.8 --- .../v1.8/guides/hardware_sizing.md | 105 ------------------ 1 file changed, 105 deletions(-) diff --git a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md index f089971cc..0112bc2d1 100644 --- a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md @@ -180,39 +180,6 @@ Select one of the following replication factors to see the recommended cluster c {{% /expand %}} -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 1,028,000 | 116 | 98 + 365,000 | -| 4 x 16 | 2,067,000 | 208 | 140 + 8,056,000 | -| 4 x 32 | 3,290,000 | 428 | 228 + 1,892,000 | -| 8 x 8 | 2,813,000 | 928 | 496 + 1,225,000 | -| 8 x 16 | 5,225,000 | 2176 | 800 + 2,799,000 | -| 8 x 32 | 8,555,000 | 5184 | 1088 + 6,055,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 6" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 6 x 8 | 1,261,000 | 288 | 192 + 522,000 | -| 6 x 16 | 2,370,000 | 576 | 288 + 1,275,000 | -| 6 x 32 | 3,601,000 | 1056 | 336 + 2,390,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 8" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| ----------------: | -----------------: |:---------------------------:| -| 8 x 8 | 1,382,000 | 1184 | 416 + 915,000 | -| 8 x 16 | 2,658,000 | 2504 | 448 + 2,204,000 | -| 8 x 32 | 3,887,000 | 5184 | 602 + 4,120,000 | - -{{% /expand %}} - {{% /tab-content %}} {{% tab-content %}} @@ -267,39 +234,6 @@ Select one of the following replication factors to see the recommended cluster c {{% /expand %}} -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 635,000 | 112 | 80 + 207,000 | -| 4 x 16 | 1,359,000 | 188 | 124 + 461,000 | -| 4 x 32 | 2,320,000 | 416 | 192 + 1,102,000 | -| 8 x 8 | 1,570,000 | 1360 | 816 + 572,000 | -| 8 x 16 | 3,205,000 | 2720 | 832 + 2,053,000 | -| 8 x 32 | 3,294,000 | 2592 | 804 + 2,174,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 6" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 6 x 8 | 694,000 | 302 | 198 + 204,000 | -| 6 x 16 | 1,397,000 | 552 | 360 + 450,000 | -| 6 x 32 | 2,298,000 | 1248 | 384 + 1,261,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 8" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| ----------------: | -----------------: |:---------------------------:| -| 8 x 8 | 739,000 | 1296 | 480 + 371,000 | -| 8 x 16 | 1,396,000 | 2592 | 672 + 843,000 | -| 8 x 32 | 2,614,000 | 2496 | 960 + 1,371,000 | - -{{% /expand %}} - {{% /tab-content %}} {{% tab-content %}} @@ -339,39 +273,6 @@ Select one of the following replication factors to see the recommended cluster c {{% /expand %}} -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 258,000 | 116 | 68 + 73,000 | -| 4 x 16 | 675,000 | 196 | 132 + 140,000 | -| 4 x 32 | 1,513,000 | 244 | 176 + 476,000 | -| 8 x 8 | 614,000 | 1096 | 400 + 258,000 | -| 8 x 16 | 1,557,000 | 2496 | 1152 + 436,000 | -| 8 x 32 | 3,265,000 | 4288 | 2240 + 820,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 6" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 6 x 8 | 694,000 | 302 | 198 + 204,000 | -| 6 x 16 | 1,397,000 | 552 | 360 + 450,000 | -| 6 x 32 | 2,298,000 | 1248 | 384 + 1,261,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 8" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| ----------------: | -----------------: |:---------------------------:| -| 8 x 8 | 739,000 | 1296 | 480 + 371,000 | -| 8 x 16 | 1,396,000 | 2592 | 672 + 843,000 | -| 8 x 32 | 2,614,000 | 2496 | 960 + 1,371,000 | - -{{% /expand %}} - {{% /tab-content %}} {{% tab-content %}} @@ -419,12 +320,6 @@ Select one of the following replication factors to see the recommended cluster c {{% /expand %}} -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 183365 | 132 | 52 + 100,000 | - {{% /expand %}} {{% /tab-content %}} From d83e67c77a8aea59ba8ec2cfa7c386ac1124b7d3 Mon Sep 17 00:00:00 2001 From: lwandzura <51929958+lwandzura@users.noreply.github.com> Date: Wed, 10 Nov 2021 14:44:49 -0600 Subject: [PATCH 07/18] Update hardware_sizing.md removed anything above RF 3 from Enterprise 1.9 hardware sizing guide --- .../v1.9/guides/hardware_sizing.md | 107 ------------------ 1 file changed, 107 deletions(-) diff --git a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md index 75f16e19c..f4b1513f9 100644 --- a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md @@ -180,39 +180,6 @@ Select one of the following replication factors to see the recommended cluster c {{% /expand %}} -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 1,028,000 | 116 | 98 + 365,000 | -| 4 x 16 | 2,067,000 | 208 | 140 + 8,056,000 | -| 4 x 32 | 3,290,000 | 428 | 228 + 1,892,000 | -| 8 x 8 | 2,813,000 | 928 | 496 + 1,225,000 | -| 8 x 16 | 5,225,000 | 2176 | 800 + 2,799,000 | -| 8 x 32 | 8,555,000 | 5184 | 1088 + 6,055,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 6" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 6 x 8 | 1,261,000 | 288 | 192 + 522,000 | -| 6 x 16 | 2,370,000 | 576 | 288 + 1,275,000 | -| 6 x 32 | 3,601,000 | 1056 | 336 + 2,390,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 8" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| ----------------: | -----------------: |:---------------------------:| -| 8 x 8 | 1,382,000 | 1184 | 416 + 915,000 | -| 8 x 16 | 2,658,000 | 2504 | 448 + 2,204,000 | -| 8 x 32 | 3,887,000 | 5184 | 602 + 4,120,000 | - -{{% /expand %}} - {{% /tab-content %}} {{% tab-content %}} @@ -267,39 +234,6 @@ Select one of the following replication factors to see the recommended cluster c {{% /expand %}} -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 635,000 | 112 | 80 + 207,000 | -| 4 x 16 | 1,359,000 | 188 | 124 + 461,000 | -| 4 x 32 | 2,320,000 | 416 | 192 + 1,102,000 | -| 8 x 8 | 1,570,000 | 1360 | 816 + 572,000 | -| 8 x 16 | 3,205,000 | 2720 | 832 + 2,053,000 | -| 8 x 32 | 3,294,000 | 2592 | 804 + 2,174,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 6" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 6 x 8 | 694,000 | 302 | 198 + 204,000 | -| 6 x 16 | 1,397,000 | 552 | 360 + 450,000 | -| 6 x 32 | 2,298,000 | 1248 | 384 + 1,261,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 8" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| ----------------: | -----------------: |:---------------------------:| -| 8 x 8 | 739,000 | 1296 | 480 + 371,000 | -| 8 x 16 | 1,396,000 | 2592 | 672 + 843,000 | -| 8 x 32 | 2,614,000 | 2496 | 960 + 1,371,000 | - -{{% /expand %}} - {{% /tab-content %}} {{% tab-content %}} @@ -339,39 +273,6 @@ Select one of the following replication factors to see the recommended cluster c {{% /expand %}} -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 258,000 | 116 | 68 + 73,000 | -| 4 x 16 | 675,000 | 196 | 132 + 140,000 | -| 4 x 32 | 1,513,000 | 244 | 176 + 476,000 | -| 8 x 8 | 614,000 | 1096 | 400 + 258,000 | -| 8 x 16 | 1,557,000 | 2496 | 1152 + 436,000 | -| 8 x 32 | 3,265,000 | 4288 | 2240 + 820,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 6" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 6 x 8 | 694,000 | 302 | 198 + 204,000 | -| 6 x 16 | 1,397,000 | 552 | 360 + 450,000 | -| 6 x 32 | 2,298,000 | 1248 | 384 + 1,261,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 8" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| ----------------: | -----------------: |:---------------------------:| -| 8 x 8 | 739,000 | 1296 | 480 + 371,000 | -| 8 x 16 | 1,396,000 | 2592 | 672 + 843,000 | -| 8 x 32 | 2,614,000 | 2496 | 960 + 1,371,000 | - -{{% /expand %}} - {{% /tab-content %}} {{% tab-content %}} @@ -419,14 +320,6 @@ Select one of the following replication factors to see the recommended cluster c {{% /expand %}} -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 183365 | 132 | 52 + 100,000 | - -{{% /expand %}} - {{% /tab-content %}} {{< /tabs-wrapper >}} From 281e42d52ea31904d156a935b2fd4e3200ff4e08 Mon Sep 17 00:00:00 2001 From: lwandzura <51929958+lwandzura@users.noreply.github.com> Date: Mon, 15 Nov 2021 16:43:05 -0600 Subject: [PATCH 08/18] Update hardware_sizing.md Added link to OSS hardware sizing documentation. Removed references to OSS. Reworded "Single node or cluster" section. --- .../v1.8/guides/hardware_sizing.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md index 0112bc2d1..8e008b81f 100644 --- a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md @@ -8,27 +8,27 @@ menu: parent: Guides --- -Review configuration and hardware guidelines for InfluxDB OSS (open source) and InfluxDB Enterprise: +Review configuration and hardware guidelines for InfluxDB Enterprise: -* [Single node or cluster?](#single-node-or-cluster) +* [Enterprise overview](#enterprise-overview) * [Query guidelines](#query-guidelines) -* [InfluxDB OSS guidelines](#influxdb-oss-guidelines) * [InfluxDB Enterprise cluster guidelines](#influxdb-enterprise-cluster-guidelines) * [When do I need more RAM?](#when-do-i-need-more-ram) * [Recommended cluster configurations](#recommended-cluster-configurations) * [Storage: type, amount, and configuration](#storage-type-amount-and-configuration) +For OSS hardware sizing guidelines, click on following [LINK](https://docs.influxdata.com/influxdb/v1.8/guides/hardware_sizing/). + > **Disclaimer:** Your numbers may vary from recommended guidelines. Guidelines provide estimated benchmarks for implementing the most performant system for your business. -## Single node or cluster? +## Enterprise overview -If your InfluxDB performance requires any of the following, a single node (InfluxDB OSS) may not support your needs: +InfluxDB Enterpise supports the following: - more than 750,000 field writes per second - more than 100 moderate queries per second ([see Query guides](#query-guidelines)) - more than 10,000,000 [series cardinality](/influxdb/v1.8/concepts/glossary/#series-cardinality) -We recommend InfluxDB Enterprise, which supports multiple data nodes (a cluster) across multiple server cores. InfluxDB Enterprise distributes multiple copies of your data across a cluster, providing high-availability and redundancy, so an unavailable node doesn’t significantly impact the cluster. Please [contact us](https://www.influxdata.com/contact-sales/) for assistance tuning your system. From 98a41e61133cbfb7a1573079ff5b3ebe5ef718cb Mon Sep 17 00:00:00 2001 From: lwandzura <51929958+lwandzura@users.noreply.github.com> Date: Tue, 16 Nov 2021 09:31:06 -0600 Subject: [PATCH 09/18] Update hardware_sizing.md Added doc link to OSS, removed OSS related content for enterprise 1.9 version --- .../v1.9/guides/hardware_sizing.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md index f4b1513f9..167bd000c 100644 --- a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md @@ -8,9 +8,9 @@ menu: parent: Guides --- -Review configuration and hardware guidelines for InfluxDB OSS (open source) and InfluxDB Enterprise: +Review configuration and hardware guidelines for InfluxDB Enterprise: -* [Single node or cluster?](#single-node-or-cluster) +* [Enterprise overview](#enterprise-overview) * [Query guidelines](#query-guidelines) * [InfluxDB OSS guidelines](#influxdb-oss-guidelines) * [InfluxDB Enterprise cluster guidelines](#influxdb-enterprise-cluster-guidelines) @@ -18,17 +18,18 @@ Review configuration and hardware guidelines for InfluxDB OSS (open source) and * [Recommended cluster configurations](#recommended-cluster-configurations) * [Storage: type, amount, and configuration](#storage-type-amount-and-configuration) +For OSS hardware sizing guidelines, click on following [LINK](https://docs.influxdata.com/influxdb/v1.8/guides/hardware_sizing/). + > **Disclaimer:** Your numbers may vary from recommended guidelines. Guidelines provide estimated benchmarks for implementing the most performant system for your business. -## Single node or cluster? +## Enterprise overview -If your InfluxDB performance requires any of the following, a single node (InfluxDB OSS) may not support your needs: +InfluxDB Enterpise supports the following: - more than 750,000 field writes per second - more than 100 moderate queries per second ([see Query guides](#query-guidelines)) - more than 10,000,000 [series cardinality](/influxdb/v1.8/concepts/glossary/#series-cardinality) -We recommend InfluxDB Enterprise, which supports multiple data nodes (a cluster) across multiple server cores. InfluxDB Enterprise distributes multiple copies of your data across a cluster, providing high-availability and redundancy, so an unavailable node doesn’t significantly impact the cluster. Please [contact us](https://www.influxdata.com/contact-sales/) for assistance tuning your system. From 5cb77c2694f238c4718571f970c124a67197bd8a Mon Sep 17 00:00:00 2001 From: Kelly Date: Tue, 16 Nov 2021 12:21:37 -0800 Subject: [PATCH 10/18] remove extra expand tag --- content/enterprise_influxdb/v1.8/guides/hardware_sizing.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md index 8e008b81f..bb012c63a 100644 --- a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md @@ -1,5 +1,5 @@ --- -title: Hardware sizing guidelines +title: Hardware sizing guidelines Description: > Review configuration and hardware guidelines for InfluxDB OSS (open source) and InfluxDB Enterprise. menu: @@ -320,8 +320,6 @@ Select one of the following replication factors to see the recommended cluster c {{% /expand %}} -{{% /expand %}} - {{% /tab-content %}} {{< /tabs-wrapper >}} From dbe307163afe3fc8d265adb7a1cc4ceec9fb76fa Mon Sep 17 00:00:00 2001 From: Kelly Date: Tue, 16 Nov 2021 22:43:05 -0800 Subject: [PATCH 11/18] updates for OSS hardware sizing --- .../influxdb/v1.8/guides/hardware_sizing.md | 403 +----------------- 1 file changed, 9 insertions(+), 394 deletions(-) diff --git a/content/influxdb/v1.8/guides/hardware_sizing.md b/content/influxdb/v1.8/guides/hardware_sizing.md index 5d24cc67e..9422dc9e1 100644 --- a/content/influxdb/v1.8/guides/hardware_sizing.md +++ b/content/influxdb/v1.8/guides/hardware_sizing.md @@ -12,8 +12,7 @@ Review configuration and hardware guidelines for InfluxDB OSS (open source) and * [Single node or cluster?](#single-node-or-cluster) * [Query guidelines](#query-guidelines) -* [InfluxDB OSS guidelines](#influxdb-oss-guidelines) -* [InfluxDB Enterprise cluster guidelines](#influxdb-enterprise-cluster-guidelines) +* [InfluxDB OSS sizing guidelines](#influxdb-oss-sizing-guidelines) * [When do I need more RAM?](#when-do-i-need-more-ram) * [Recommended cluster configurations](#recommended-cluster-configurations) * [Storage: type, amount, and configuration](#storage-type-amount-and-configuration) @@ -22,6 +21,10 @@ Review configuration and hardware guidelines for InfluxDB OSS (open source) and ## Single node or cluster? +If you want a single node instance of InfluxDB that's fully open source, requires fewer writes, queries, and unique series than listed above, and do **not require** redundancy, we recommend InfluxDB OSS. + +> **Note:** Without the redundancy of a cluster, writes and queries fail immediately when a server is unavailable. + If your InfluxDB performance requires any of the following, a single node (InfluxDB OSS) may not support your needs: - more than 750,000 field writes per second @@ -31,15 +34,12 @@ If your InfluxDB performance requires any of the following, a single node (Influ We recommend InfluxDB Enterprise, which supports multiple data nodes (a cluster) across multiple server cores. InfluxDB Enterprise distributes multiple copies of your data across a cluster, providing high-availability and redundancy, so an unavailable node doesn’t significantly impact the cluster. -Please [contact us](https://www.influxdata.com/contact-sales/) for assistance tuning your system. -If you want a single node instance of InfluxDB that's fully open source, requires fewer writes, queries, and unique series than listed above, and do **not require** redundancy, we recommend InfluxDB OSS. - -> **Note:** Without the redundancy of a cluster, writes and queries fail immediately when a server is unavailable. +For more information about sizing requirements for InfluxDB Enterprise, see [InfluxDB Enterprise hardware sizing guidelines](/enterprise_influxdb/v1.9/guides/hardware_sizing//guides/hardware_sizing/). ## Query guidelines -> Query complexity varies widely on system impact. Recommendations for both single nodes and clusters are based on **moderate** query loads. +> Query complexity varies widely on system impact. Recommendations are based on **moderate** query loads. For **simple** or **complex** queries, we recommend testing and adjusting the suggested requirements as needed. Query complexity is defined by the following criteria: @@ -55,7 +55,7 @@ For **simple** or **complex** queries, we recommend testing and adjusting the su | | May sample a very large time range of months or years | | | Typically take multiple seconds to execute | -## InfluxDB OSS guidelines +## InfluxDB OSS sizing guidelines Run InfluxDB on locally attached solid state drives (SSDs). Other storage configurations have lower performance and may not be able to recover from small interruptions in normal processing. @@ -69,396 +69,11 @@ Estimated guidelines include writes per second, queries per second, and number o * **Queries per second for moderate queries.** Queries vary widely in their impact on the system. For simple or complex queries, we recommend testing and adjusting the suggested requirements as needed. See [query guidelines](#query-guidelines) for details. -## InfluxDB Enterprise cluster guidelines - -### Meta nodes - -> Set up clusters with an odd number of meta nodes──an even number may cause issues in certain configurations. - -A cluster must have a **minimum of three** independent meta nodes for data redundancy and availability. A cluster with `2n + 1` meta nodes can tolerate the loss of `n` meta nodes. - -Meta nodes do not need very much computing power. Regardless of the cluster load, we recommend the following guidelines for the meta nodes: - -* vCPU or CPU: 1-2 cores -* RAM: 512 MB - 1 GB -* IOPS: 50 - -### Web node - -The InfluxDB Enterprise web server is primarily an HTTP server with similar load requirements. For most applications, the server doesn't need to be very robust. A cluster can function with only one web server, but for redundancy, we recommend connecting multiple web servers to a single back-end Postgres database. - -> **Note:** Production clusters should not use the SQLite database (lacks support for redundant web servers and handling high loads). - -* vCPU or CPU: 2-4 cores -* RAM: 2-4 GB -* IOPS: 100 - -### Data nodes - -A cluster with one data node is valid but has no data redundancy. Redundancy is set by the [replication factor](/influxdb/v1.8/concepts/glossary/#replication-factor) on the retention policy the data is written to. Where `n` is the replication factor, a cluster can lose `n - 1` data nodes and return complete query results. - ->**Note:** For optimal data distribution within the cluster, use an even number of data nodes. - -Guidelines vary by writes per second per node, moderate queries per second per node, and the number of unique series per node. - -#### Guidelines per node - -| vCPU or CPU | RAM | IOPS | Writes per second | Queries* per second | Unique series | -| ----------: | -------: | ----: | ----------------: | ------------------: | ------------: | -| 2 cores | 4-8 GB | 1000 | < 5,000 | < 5 | < 100,000 | -| 4-6 cores | 16-32 GB | 1000+ | < 100,000 | < 25 | < 1,000,000 | -| 8+ cores | 32+ GB | 1000+ | > 100,000 | > 25 | > 1,000,000 | - -* Guidelines are provided for moderate queries. Queries vary widely in their impact on the system. For simple or complex queries, we recommend testing and adjusting the suggested requirements as needed. See [query guidelines](#query-guidelines) for detail. - -## When do I need more RAM? - -In general, more RAM helps queries return faster. Your RAM requirements are primarily determined by [series cardinality](/influxdb/v1.8/concepts/glossary/#series-cardinality). Higher cardinality requires more RAM. Regardless of RAM, a series cardinality of 10 million or more can cause OOM (out of memory) failures. You can usually resolve OOM issues by redesigning your [schema](/influxdb/v1.8/concepts/glossary/#schema). - - -## Guidelines per cluster - -InfluxDB Enterprise guidelines vary by writes and queries per second, series cardinality, replication factor, and infrastructure-AWS EC2 R4 instances or equivalent: -- R4.xlarge (4 cores) -- R4.2xlarge (8 cores) -- R4.4xlarge (16 cores) -- R4.8xlarge (32 cores) - -> Guidelines stem from a DevOps monitoring use case: maintaining a group of computers and monitoring server metrics (such as CPU, kernel, memory, disk space, disk I/O, network, and so on). - -### Recommended cluster configurations - -Cluster configurations guidelines are organized by: - -- Series cardinality in your data set: 10,000, 100,000, 1,000,000, or 10,000,000 -- Number of data nodes -- Number of server cores - -For each cluster configuration, you'll find guidelines for the following: - -- **maximum writes per second only** (no dashboard queries are running) -- **maximum queries per second only** (no data is being written) -- **maximum simultaneous queries and writes per second, combined** - -#### Review cluster configuration tables - -1. Select the series cardinality tab below, and then click to expand a replication factor. -2. In the **Nodes x Core** column, find the number of data nodes and server cores in your configuration, and then review the recommended **maximum** guidelines. - -{{< tabs-wrapper >}} -{{% tabs %}} -[10,000 series](#) -[100,000 series](#) -[1,000,000 series](#) -[10,000,000 series](#) -{{% /tabs %}} -{{% tab-content %}} - -Select one of the following replication factors to see the recommended cluster configuration for 10,000 series: - -{{% expand "Replication factor, 1" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:|------------------:|-------------------:|:---------------------------:| -| 1 x 4 | 188,000 | 5 | 4 + 99,000 | -| 1 x 8 | 405,000 | 9 | 8 + 207,000 | -| 1 x 16 | 673,000 | 15 | 14 + 375,000 | -| 1 x 32 | 1,056,000 | 24 | 22 + 650,000 | -| 2 x 4 | 384,000 | 14 | 14 + 184,000 | -| 2 x 8 | 746,000 | 22 | 22 + 334,000 | -| 2 x 16 | 1,511,000 | 56 | 40 + 878,000 | -| 2 x 32 | 2,426,000 | 96 | 68 + 1,746,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 2" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:|------------------:|-------------------:|:---------------------------:| -| 2 x 4 | 296,000 | 16 | 16 + 151,000 | -| 2 x 8 | 560,000 | 30 | 26 + 290,000 | -| 2 x 16 | 972,000 | 54 | 50 + 456,000 | -| 2 x 32 | 1,860,000 | 84 | 74 + 881,000 | -| 4 x 8 | 1,781,000 | 100 | 64 + 682,000 | -| 4 x 16 | 3,430,000 | 192 | 104 + 1,732,000 | -| 4 x 32 | 6,351,000 | 432 | 188 + 3,283,000 | -| 6 x 8 | 2,923,000 | 216 | 138 + 1,049,000 | -| 6 x 16 | 5,650,000 | 498 | 246 + 2,246,000 | -| 6 x 32 | 9,842,000 | 1248 | 528 + 5,229,000 | -| 8 x 8 | 3,987,000 | 632 | 336 + 1,722,000 | -| 8 x 16 | 7,798,000 | 1384 | 544 + 3,911,000 | -| 8 x 32 | 13,189,000 | 3648 | 1,152 + 7,891,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 3" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:|------------------:|-------------------:|:---------------------------:| -| 3 x 8 | 815,000 | 63 | 54 + 335,000 | -| 3 x 16 | 1,688,000 | 120 | 87 + 705,000 | -| 3 x 32 | 3,164,000 | 255 | 132 + 1,626,000 | -| 6 x 8 | 2,269,000 | 252 | 168 + 838,000 | -| 6 x 16 | 4,593,000 | 624 | 336 + 2,019,000 | -| 6 x 32 | 7,776,000 | 1340 | 576 + 3,624,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 1,028,000 | 116 | 98 + 365,000 | -| 4 x 16 | 2,067,000 | 208 | 140 + 8,056,000 | -| 4 x 32 | 3,290,000 | 428 | 228 + 1,892,000 | -| 8 x 8 | 2,813,000 | 928 | 496 + 1,225,000 | -| 8 x 16 | 5,225,000 | 2176 | 800 + 2,799,000 | -| 8 x 32 | 8,555,000 | 5184 | 1088 + 6,055,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 6" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 6 x 8 | 1,261,000 | 288 | 192 + 522,000 | -| 6 x 16 | 2,370,000 | 576 | 288 + 1,275,000 | -| 6 x 32 | 3,601,000 | 1056 | 336 + 2,390,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 8" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| ----------------: | -----------------: |:---------------------------:| -| 8 x 8 | 1,382,000 | 1184 | 416 + 915,000 | -| 8 x 16 | 2,658,000 | 2504 | 448 + 2,204,000 | -| 8 x 32 | 3,887,000 | 5184 | 602 + 4,120,000 | - -{{% /expand %}} - -{{% /tab-content %}} - -{{% tab-content %}} - -Select one of the following replication factors to see the recommended cluster configuration for 100,000 series: - -{{% expand "Replication factor, 1" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:|------------------:|-------------------:|:---------------------------:| -| 1 x 4 | 143,000 | 5 | 4 + 77,000 | -| 1 x 8 | 322,000 | 9 | 8 + 167,000 | -| 1 x 16 | 624,000 | 17 | 12 + 337,000 | -| 1 x 32 | 1,114,000 | 26 | 18 + 657,000 | -| 2 x 4 | 265,000 | 14 | 12 + 115,000 | -| 2 x 8 | 573,000 | 30 | 22 + 269,000 | -| 2 x 16 | 1,261,000 | 52 | 38 + 679,000 | -| 2 x 32 | 2,335,000 | 90 | 66 + 1,510,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 2" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:|------------------:|-------------------:|:---------------------------:| -| 2 x 4 | 196,000 | 16 | 14 + 77,000 | -| 2 x 8 | 482,000 | 30 | 24 + 203,000 | -| 2 x 16 | 1,060,000 | 60 | 42 + 415,000 | -| 2 x 32 | 1,958,000 | 94 | 64 + 984,000 | -| 4 x 8 | 1,144,000 | 108 | 68 + 406,000 | -| 4 x 16 | 2,512,000 | 228 | 148 + 866,000 | -| 4 x 32 | 4,346,000 | 564 | 320 + 1,886,000 | -| 6 x 8 | 1,802,000 | 252 | 156 + 618,000 | -| 6 x 16 | 3,924,000 | 562 | 384 + 1,068,000 | -| 6 x 32 | 6,533,000 | 1340 | 912 + 2,083,000 | -| 8 x 8 | 2,516,000 | 712 | 360 + 1,020,000 | -| 8 x 16 | 5,478,000 | 1632 | 1,024 + 1,843,000 | -| 8 x 32 | 1,0527,000 | 3392 | 1,792 + 4,998,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 3" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:|------------------:|-------------------:|:---------------------------:| -| 3 x 8 | 616,000 | 72 | 51 + 218,000 | -| 3 x 16 | 1,268,000 | 117 | 84 + 438,000 | -| 3 x 32 | 2,260,000 | 189 | 114 + 984,000 | -| 6 x 8 | 1,393,000 | 294 | 192 + 421,000 | -| 6 x 16 | 3,056,000 | 726 | 456 + 893,000 | -| 6 x 32 | 5,017,000 | 1584 | 798 + 1,098,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 635,000 | 112 | 80 + 207,000 | -| 4 x 16 | 1,359,000 | 188 | 124 + 461,000 | -| 4 x 32 | 2,320,000 | 416 | 192 + 1,102,000 | -| 8 x 8 | 1,570,000 | 1360 | 816 + 572,000 | -| 8 x 16 | 3,205,000 | 2720 | 832 + 2,053,000 | -| 8 x 32 | 3,294,000 | 2592 | 804 + 2,174,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 6" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 6 x 8 | 694,000 | 302 | 198 + 204,000 | -| 6 x 16 | 1,397,000 | 552 | 360 + 450,000 | -| 6 x 32 | 2,298,000 | 1248 | 384 + 1,261,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 8" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| ----------------: | -----------------: |:---------------------------:| -| 8 x 8 | 739,000 | 1296 | 480 + 371,000 | -| 8 x 16 | 1,396,000 | 2592 | 672 + 843,000 | -| 8 x 32 | 2,614,000 | 2496 | 960 + 1,371,000 | - -{{% /expand %}} - -{{% /tab-content %}} - -{{% tab-content %}} - -Select one of the following replication factors to see the recommended cluster configuration for 1,000,000 series: - -{{% expand "Replication factor, 2" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:-------------:|------------------:|-------------------:|:---------------------------:| -| 2 x 4 | 104,000 | 18 | 12 + 54,000 | -| 2 x 8 | 195,000 | 36 | 24 + 99,000 | -| 2 x 16 | 498,000 | 70 | 44 + 145,000 | -| 2 x 32 | 1,195,000 | 102 | 84 + 232,000 | -| 4 x 8 | 488,000 | 120 | 56 + 222,000 | -| 4 x 16 | 1,023,000 | 244 | 112 + 428,000 | -| 4 x 32 | 2,686,000 | 468 | 208 + 729,000 | -| 6 x 8 | 845,000 | 270 | 126 + 356,000 | -| 6 x 16 | 1,780,000 | 606 | 288 + 663,000 | -| 6 x 32 | 430,000 | 1,488 | 624 + 1,209,000 | -| 8 x 8 | 1,831,000 | 808 | 296 + 778,000 | -| 8 x 16 | 4,167,000 | 1,856 | 640 + 2,031,000 | -| 8 x 32 | 7,813,000 | 3,201 | 896 + 4,897,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 3" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:|------------------:|-------------------:|:---------------------------:| -| 3 x 8 | 234,000 | 72 | 42 + 87,000 | -| 3 x 16 | 613,000 | 120 | 75 + 166,000 | -| 3 x 32 | 1,365,000 | 141 | 114 + 984,000 | -| 6 x 8 | 593,000 | 318 | 144 + 288,000 | -| 6 x 16 | 1,545,000 | 744 | 384 + 407,000 | -| 6 x 32 | 3,204,000 | 1632 | 912 + 505,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 258,000 | 116 | 68 + 73,000 | -| 4 x 16 | 675,000 | 196 | 132 + 140,000 | -| 4 x 32 | 1,513,000 | 244 | 176 + 476,000 | -| 8 x 8 | 614,000 | 1096 | 400 + 258,000 | -| 8 x 16 | 1,557,000 | 2496 | 1152 + 436,000 | -| 8 x 32 | 3,265,000 | 4288 | 2240 + 820,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 6" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 6 x 8 | 694,000 | 302 | 198 + 204,000 | -| 6 x 16 | 1,397,000 | 552 | 360 + 450,000 | -| 6 x 32 | 2,298,000 | 1248 | 384 + 1,261,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 8" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| ----------------: | -----------------: |:---------------------------:| -| 8 x 8 | 739,000 | 1296 | 480 + 371,000 | -| 8 x 16 | 1,396,000 | 2592 | 672 + 843,000 | -| 8 x 32 | 2,614,000 | 2496 | 960 + 1,371,000 | - -{{% /expand %}} - -{{% /tab-content %}} - -{{% tab-content %}} - -Select one of the following replication factors to see the recommended cluster configuration for 10,000,000 series: - -{{% expand "Replication factor, 1" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:|------------------:|-------------------:|:---------------------------:| -| 2 x 4 | 122,000 | 16 | 12 + 81,000 | -| 2 x 8 | 259,000 | 36 | 24 + 143,000 | -| 2 x 16 | 501,000 | 66 | 44 + 290,000 | -| 2 x 32 | 646,000 | 142 | 54 + 400,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 2" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:|------------------:|-------------------:|:---------------------------:| -| 2 x 4 | 87,000 | 18 | 14 + 56,000 | -| 2 x 8 | 169,000 | 38 | 24 + 98,000 | -| 2 x 16 | 334,000 | 76 | 46 + 224,000 | -| 2 x 32 | 534,000 | 136 | 58 + 388,000 | -| 4 x 8 | 335,000 | 120 | 60 + 204,000 | -| 4 x 16 | 643,000 | 256 | 112 + 395,000 | -| 4 x 32 | 967,000 | 560 | 158 + 806,000 | -| 6 x 8 | 521,000 | 378 | 144 + 319,000 | -| 6 x 16 | 890,000 | 582 | 186 + 513,000 | -| 8 x 8 | 699,000 | 1,032 | 256 + 477,000 | -| 8 x 16 | 1,345,000 | 2,048 | 544 + 741,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 3" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:|------------------:|-------------------:|:---------------------------:| -| 3 x 8 | 170,000 | 60 | 42 + 98,000 | -| 3 x 16 | 333,000 | 129 | 76 + 206,000 | -| 3 x 32 | 609,000 | 178 | 60 + 162,000 | -| 6 x 8 | 395,000 | 402 | 132 + 247,000 | -| 6 x 16 | 679,000 | 894 | 150 + 527,000 | - -{{% /expand %}} - -{{% expand "Replication factor, 4" %}} - -| Nodes x Core | Writes per second | Queries per second | Queries + writes per second | -|:------------:| -----------------:| ------------------:|:---------------------------:| -| 4 x 8 | 183365 | 132 | 52 + 100,000 | - -{{% /expand %}} - -{{% /tab-content %}} -{{< /tabs-wrapper >}} - ## Storage: type, amount, and configuration ### Storage volume and IOPS -Consider the type of storage you need and the amount. InfluxDB is designed to run on solid state drives (SSDs) and memory-optimized cloud instances, for example, AWS EC2 R5 or R4 instances. InfluxDB isn't tested on hard disk drives (HDDs) and we don't recommend HDDs for production. For best results, InfluxDB servers must have a minimum of 1000 IOPS on storage to ensure recovery and availability. We recommend at least 2000 IOPS for rapid recovery of cluster data nodes after downtime. +Consider the type of storage you need and the amount. InfluxDB is designed to run on solid state drives (SSDs) and memory-optimized cloud instances, for example, AWS EC2 R5 or R4 instances. InfluxDB isn't tested on hard disk drives (HDDs). For best results, InfluxDB servers must have a minimum of 1000 IOPS on storage to ensure recovery and availability. We recommend at least 2000 IOPS for rapid recovery of cluster data nodes after downtime. See your cloud provider documentation for IOPS detail on your storage volumes. From 4a59c29d1ee337504d1353ab8105a8ef75d8e831 Mon Sep 17 00:00:00 2001 From: kelseiv <47797004+kelseiv@users.noreply.github.com> Date: Tue, 16 Nov 2021 22:49:52 -0800 Subject: [PATCH 12/18] remove sizing --- content/influxdb/v1.8/guides/hardware_sizing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/influxdb/v1.8/guides/hardware_sizing.md b/content/influxdb/v1.8/guides/hardware_sizing.md index 9422dc9e1..b6151fd20 100644 --- a/content/influxdb/v1.8/guides/hardware_sizing.md +++ b/content/influxdb/v1.8/guides/hardware_sizing.md @@ -12,7 +12,7 @@ Review configuration and hardware guidelines for InfluxDB OSS (open source) and * [Single node or cluster?](#single-node-or-cluster) * [Query guidelines](#query-guidelines) -* [InfluxDB OSS sizing guidelines](#influxdb-oss-sizing-guidelines) +* [InfluxDB OSS guidelines](#influxdb-oss-guidelines) * [When do I need more RAM?](#when-do-i-need-more-ram) * [Recommended cluster configurations](#recommended-cluster-configurations) * [Storage: type, amount, and configuration](#storage-type-amount-and-configuration) @@ -55,7 +55,7 @@ For **simple** or **complex** queries, we recommend testing and adjusting the su | | May sample a very large time range of months or years | | | Typically take multiple seconds to execute | -## InfluxDB OSS sizing guidelines +## InfluxDB OSS guidelines Run InfluxDB on locally attached solid state drives (SSDs). Other storage configurations have lower performance and may not be able to recover from small interruptions in normal processing. From 8300c781087a2313dbefb6227de2170a8b3025e9 Mon Sep 17 00:00:00 2001 From: lwandzura <51929958+lwandzura@users.noreply.github.com> Date: Wed, 17 Nov 2021 13:48:00 -0600 Subject: [PATCH 13/18] Update content/enterprise_influxdb/v1.9/guides/hardware_sizing.md Co-authored-by: kelseiv <47797004+kelseiv@users.noreply.github.com> --- content/enterprise_influxdb/v1.9/guides/hardware_sizing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md index 167bd000c..829f32979 100644 --- a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md @@ -18,7 +18,7 @@ Review configuration and hardware guidelines for InfluxDB Enterprise: * [Recommended cluster configurations](#recommended-cluster-configurations) * [Storage: type, amount, and configuration](#storage-type-amount-and-configuration) -For OSS hardware sizing guidelines, click on following [LINK](https://docs.influxdata.com/influxdb/v1.8/guides/hardware_sizing/). +For InfluxDB OSS instances, see [OSS hardware sizing guidelines](https://docs.influxdata.com/influxdb/v1.8/guides/hardware_sizing/). > **Disclaimer:** Your numbers may vary from recommended guidelines. Guidelines provide estimated benchmarks for implementing the most performant system for your business. From f3d6b5c0998958de4eb10534e05a2e54014a405c Mon Sep 17 00:00:00 2001 From: kelseiv <47797004+kelseiv@users.noreply.github.com> Date: Wed, 17 Nov 2021 12:37:11 -0800 Subject: [PATCH 14/18] Update content/enterprise_influxdb/v1.8/guides/hardware_sizing.md --- content/enterprise_influxdb/v1.8/guides/hardware_sizing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md index bb012c63a..01cfb7795 100644 --- a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md @@ -44,7 +44,7 @@ If you want a single node instance of InfluxDB that's fully open source, require For **simple** or **complex** queries, we recommend testing and adjusting the suggested requirements as needed. Query complexity is defined by the following criteria: | Query complexity | Criteria | -|------------------|---------------------------------------------------------------------------------------| +|:------------------|:---------------------------------------------------------------------------------------| | Simple | Have few or no functions and no regular expressions | | | Are bounded in time to a few minutes, hours, or 24 hours at most | | | Typically execute in a few milliseconds to a few dozen milliseconds | From f40180d18665b8826ae7f9d7159053587a07e0dc Mon Sep 17 00:00:00 2001 From: Kelly Date: Wed, 17 Nov 2021 12:37:44 -0800 Subject: [PATCH 15/18] update --- content/influxdb/v1.8/guides/hardware_sizing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/influxdb/v1.8/guides/hardware_sizing.md b/content/influxdb/v1.8/guides/hardware_sizing.md index b6151fd20..bb1a4af95 100644 --- a/content/influxdb/v1.8/guides/hardware_sizing.md +++ b/content/influxdb/v1.8/guides/hardware_sizing.md @@ -44,7 +44,7 @@ For more information about sizing requirements for InfluxDB Enterprise, see [Inf For **simple** or **complex** queries, we recommend testing and adjusting the suggested requirements as needed. Query complexity is defined by the following criteria: | Query complexity | Criteria | -|------------------|---------------------------------------------------------------------------------------| +|:------------------|---------------------------------------------------------------------------------------| | Simple | Have few or no functions and no regular expressions | | | Are bounded in time to a few minutes, hours, or 24 hours at most | | | Typically execute in a few milliseconds to a few dozen milliseconds | From 00b1349256bd925c21c95b4a45813240e4b2e15f Mon Sep 17 00:00:00 2001 From: Kelly Date: Wed, 17 Nov 2021 12:40:33 -0800 Subject: [PATCH 16/18] remove extra punctuation --- content/enterprise_influxdb/v1.8/guides/hardware_sizing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md index 01cfb7795..5133d9f91 100644 --- a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md @@ -59,7 +59,7 @@ For **simple** or **complex** queries, we recommend testing and adjusting the su ### Meta nodes -> Set up clusters with an odd number of meta nodes──an even number may cause issues in certain configurations. +> Set up clusters with an odd number of meta nodes─an even number may cause issues in certain configurations. A cluster must have a **minimum of three** independent meta nodes for data redundancy and availability. A cluster with `2n + 1` meta nodes can tolerate the loss of `n` meta nodes. From 35ae59244a103c00a78dd7a261ce2c89a5851837 Mon Sep 17 00:00:00 2001 From: kelseiv <47797004+kelseiv@users.noreply.github.com> Date: Wed, 17 Nov 2021 12:42:18 -0800 Subject: [PATCH 17/18] Update content/enterprise_influxdb/v1.8/guides/hardware_sizing.md --- content/enterprise_influxdb/v1.8/guides/hardware_sizing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md index 5133d9f91..8791ae6de 100644 --- a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md @@ -339,4 +339,4 @@ Non-string values require approximately three bytes. String values require varia ### Separate `wal` and `data` directories -When running InfluxDB in a production environment, store the `wal` directory and the `data` directory on separate storage devices. This optimization significantly reduces disk contention under heavy write load──an important consideration if the write load is highly variable. If the write load does not vary by more than 15%, the optimization is probably not necessary. +When running InfluxDB in a production environment, store the `wal` directory and the `data` directory on separate storage devices. This optimization significantly reduces disk contention under heavy write load─an important consideration if the write load is highly variable. If the write load does not vary by more than 15%, the optimization is probably not necessary. From 07e21587741129f096334a344dce01744f056d6b Mon Sep 17 00:00:00 2001 From: Kelly Date: Wed, 17 Nov 2021 13:02:43 -0800 Subject: [PATCH 18/18] edits w Larissa --- content/enterprise_influxdb/v1.8/guides/hardware_sizing.md | 6 +++--- content/enterprise_influxdb/v1.9/guides/hardware_sizing.md | 6 +++--- .../enterprise_influxdb/v1.9/reference/hardware_sizing.md | 2 +- content/influxdb/v1.7/guides/hardware_sizing.md | 2 +- content/influxdb/v1.8/guides/hardware_sizing.md | 2 +- 5 files changed, 9 insertions(+), 9 deletions(-) diff --git a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md index 8791ae6de..7ea11a87a 100644 --- a/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.8/guides/hardware_sizing.md @@ -5,7 +5,7 @@ Description: > menu: enterprise_influxdb_1_8: weight: 40 - parent: Guides + parent: Guides --- Review configuration and hardware guidelines for InfluxDB Enterprise: @@ -17,13 +17,13 @@ Review configuration and hardware guidelines for InfluxDB Enterprise: * [Recommended cluster configurations](#recommended-cluster-configurations) * [Storage: type, amount, and configuration](#storage-type-amount-and-configuration) -For OSS hardware sizing guidelines, click on following [LINK](https://docs.influxdata.com/influxdb/v1.8/guides/hardware_sizing/). +For InfluxDB OSS instances, see [OSS hardware sizing guidelines](https://docs.influxdata.com/influxdb/v1.8/guides/hardware_sizing/). > **Disclaimer:** Your numbers may vary from recommended guidelines. Guidelines provide estimated benchmarks for implementing the most performant system for your business. ## Enterprise overview -InfluxDB Enterpise supports the following: +InfluxDB Enterprise supports the following: - more than 750,000 field writes per second - more than 100 moderate queries per second ([see Query guides](#query-guidelines)) diff --git a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md index 829f32979..e72812b4f 100644 --- a/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.9/guides/hardware_sizing.md @@ -24,7 +24,7 @@ For InfluxDB OSS instances, see [OSS hardware sizing guidelines](https://docs.in ## Enterprise overview -InfluxDB Enterpise supports the following: +InfluxDB Enterprise supports the following: - more than 750,000 field writes per second - more than 100 moderate queries per second ([see Query guides](#query-guidelines)) @@ -45,7 +45,7 @@ If you want a single node instance of InfluxDB that's fully open source, require For **simple** or **complex** queries, we recommend testing and adjusting the suggested requirements as needed. Query complexity is defined by the following criteria: | Query complexity | Criteria | -|------------------|---------------------------------------------------------------------------------------| +|:------------------|:---------------------------------------------------------------------------------------| | Simple | Have few or no functions and no regular expressions | | | Are bounded in time to a few minutes, hours, or 24 hours at most | | | Typically execute in a few milliseconds to a few dozen milliseconds | @@ -60,7 +60,7 @@ For **simple** or **complex** queries, we recommend testing and adjusting the su ### Meta nodes -> Set up clusters with an odd number of meta nodes──an even number may cause issues in certain configurations. +> Set up clusters with an odd number of meta nodes─an even number may cause issues in certain configurations. A cluster must have a **minimum of three** independent meta nodes for data redundancy and availability. A cluster with `2n + 1` meta nodes can tolerate the loss of `n` meta nodes. diff --git a/content/enterprise_influxdb/v1.9/reference/hardware_sizing.md b/content/enterprise_influxdb/v1.9/reference/hardware_sizing.md index 3ba30a7f8..a1614c18f 100644 --- a/content/enterprise_influxdb/v1.9/reference/hardware_sizing.md +++ b/content/enterprise_influxdb/v1.9/reference/hardware_sizing.md @@ -74,7 +74,7 @@ Estimated guidelines include writes per second, queries per second, and number o ### Meta nodes -> Set up clusters with an odd number of meta nodes──an even number may cause issues in certain configurations. +> Set up clusters with an odd number of meta nodes─an even number may cause issues in certain configurations. A cluster must have a **minimum of three** independent meta nodes for data redundancy and availability. A cluster with `2n + 1` meta nodes can tolerate the loss of `n` meta nodes. diff --git a/content/influxdb/v1.7/guides/hardware_sizing.md b/content/influxdb/v1.7/guides/hardware_sizing.md index 9b28c3156..c7224d19d 100644 --- a/content/influxdb/v1.7/guides/hardware_sizing.md +++ b/content/influxdb/v1.7/guides/hardware_sizing.md @@ -39,7 +39,7 @@ If you want a single node instance of InfluxDB that's fully open source, require For **simple** or **complex** queries, we recommend testing and adjusting the suggested requirements as needed. Query complexity is defined by the following criteria: | Query complexity | Criteria | -|------------------|---------------------------------------------------------------------------------------| +|:------------------|:---------------------------------------------------------------------------------------| | Simple | Have few or no functions and no regular expressions | | | Are bounded in time to a few minutes, hours, or 24 hours at most | | | Typically execute in a few milliseconds to a few dozen milliseconds | diff --git a/content/influxdb/v1.8/guides/hardware_sizing.md b/content/influxdb/v1.8/guides/hardware_sizing.md index bb1a4af95..1cfecb62a 100644 --- a/content/influxdb/v1.8/guides/hardware_sizing.md +++ b/content/influxdb/v1.8/guides/hardware_sizing.md @@ -44,7 +44,7 @@ For more information about sizing requirements for InfluxDB Enterprise, see [Inf For **simple** or **complex** queries, we recommend testing and adjusting the suggested requirements as needed. Query complexity is defined by the following criteria: | Query complexity | Criteria | -|:------------------|---------------------------------------------------------------------------------------| +|:-----------------|:---------------------------------------------------------------------------------------| | Simple | Have few or no functions and no regular expressions | | | Are bounded in time to a few minutes, hours, or 24 hours at most | | | Typically execute in a few milliseconds to a few dozen milliseconds |