chore(influxdb3): Promote system metrics plugin from examples to official plugins:- Updated with latest configuration arguments
Jason Stirnaman
parent
e868d1d3f5
commit
55e128cb78
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_core:
|
||||
name: WAL plugin
|
||||
parent: Example plugins
|
||||
weight: 2
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, wal, data-write]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/examples/wal-plugin, WAL plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_core:
|
||||
name: Basic transformation
|
||||
parent: Official plugins
|
||||
weight: 1
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, transformation, data-processing]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/basic_transformation, Basic transformation plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_core:
|
||||
name: Downsampler
|
||||
parent: Official plugins
|
||||
weight: 2
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, downsampling, aggregation, performance]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/downsampler, Downsampler plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_core:
|
||||
name: Forecast error evaluator
|
||||
parent: Official plugins
|
||||
weight: 3
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, forecasting, evaluation, analytics]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/forecast_error_evaluator, Forecast error evaluator plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_core:
|
||||
name: InfluxDB to Iceberg
|
||||
parent: Official plugins
|
||||
weight: 4
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, iceberg, export, data-lake]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/influxdb_to_iceberg, InfluxDB to Iceberg plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_core:
|
||||
name: MAD anomaly detection
|
||||
parent: Official plugins
|
||||
weight: 5
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, anomaly-detection, statistics, monitoring]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/mad_check, MAD-based anomaly detection plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_core:
|
||||
name: Notifier
|
||||
parent: Official plugins
|
||||
weight: 6
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, notifications, alerting, integration]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/notifier, Notifier plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_core:
|
||||
name: Prophet forecasting
|
||||
parent: Official plugins
|
||||
weight: 7
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, forecasting, prophet, machine-learning]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/prophet_forecasting, Prophet forecasting plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_core:
|
||||
name: State change
|
||||
parent: Official plugins
|
||||
weight: 8
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, state-tracking, event-detection, monitoring]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/state_change, State change plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_core:
|
||||
name: Stateless ADTK detector
|
||||
parent: Official plugins
|
||||
weight: 9
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, anomaly-detection, adtk, stateless]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/stateless_adtk_detector, Stateless ADTK detector plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -4,12 +4,12 @@ description: Collects comprehensive system performance metrics including CPU, me
|
|||
menu:
|
||||
influxdb3_core:
|
||||
name: System metrics
|
||||
parent: Example plugins
|
||||
weight: 1
|
||||
parent: Official plugins
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, monitoring, system-metrics, performance]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/examples/system-metrics, System metrics plugin on GitHub
|
||||
source: /shared/influxdb3-plugins/plugins-library/examples/system-metrics.md
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/system-metrics, System metrics plugin on GitHub
|
||||
source: /shared/influxdb3-plugins/plugins-library/official/system-metrics.md
|
||||
---
|
||||
|
||||
<!-- //SOURCE - content/shared/influxdb3-plugins/plugins-library/examples/system-metrics.md -->
|
||||
<!-- //SOURCE - content/shared/influxdb3-plugins/plugins-library/official/system-metrics.md -->
|
||||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_core:
|
||||
name: Threshold deadman checks
|
||||
parent: Official plugins
|
||||
weight: 10
|
||||
weight: 100
|
||||
influxdb3/core/tags: [plugins, processing engine, python, monitoring, thresholds, deadman, alerting]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/threshold_deadman_checks, Threshold deadman checks plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_enterprise:
|
||||
name: WAL plugin
|
||||
parent: Example plugins
|
||||
weight: 2
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, wal, data-write]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/examples/wal-plugin, WAL plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_enterprise:
|
||||
name: Basic transformation
|
||||
parent: Official plugins
|
||||
weight: 1
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, transformation, data-processing]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/basic_transformation, Basic transformation plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_enterprise:
|
||||
name: Downsampler
|
||||
parent: Official plugins
|
||||
weight: 2
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, downsampling, aggregation, performance]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/downsampler, Downsampler plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_enterprise:
|
||||
name: Forecast error evaluator
|
||||
parent: Official plugins
|
||||
weight: 3
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, forecasting, evaluation, analytics]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/forecast_error_evaluator, Forecast error evaluator plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_enterprise:
|
||||
name: InfluxDB to Iceberg
|
||||
parent: Official plugins
|
||||
weight: 4
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, iceberg, export, data-lake]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/influxdb_to_iceberg, InfluxDB to Iceberg plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_enterprise:
|
||||
name: MAD anomaly detection
|
||||
parent: Official plugins
|
||||
weight: 5
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, anomaly-detection, statistics, monitoring]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/mad_check, MAD-based anomaly detection plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_enterprise:
|
||||
name: Notifier
|
||||
parent: Official plugins
|
||||
weight: 6
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, notifications, alerting, integration]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/notifier, Notifier plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_enterprise:
|
||||
name: Prophet forecasting
|
||||
parent: Official plugins
|
||||
weight: 7
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, forecasting, prophet, machine-learning]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/prophet_forecasting, Prophet forecasting plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_enterprise:
|
||||
name: State change
|
||||
parent: Official plugins
|
||||
weight: 8
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, state-tracking, event-detection, monitoring]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/state_change, State change plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_enterprise:
|
||||
name: Stateless ADTK detector
|
||||
parent: Official plugins
|
||||
weight: 9
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, anomaly-detection, adtk, stateless]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/stateless_adtk_detector, Stateless ADTK detector plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -4,12 +4,12 @@ description: Collects comprehensive system performance metrics including CPU, me
|
|||
menu:
|
||||
influxdb3_enterprise:
|
||||
name: System metrics
|
||||
parent: Example plugins
|
||||
weight: 1
|
||||
parent: Official plugins
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, monitoring, system-metrics, performance]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/examples/system-metrics, System metrics plugin on GitHub
|
||||
source: /shared/influxdb3-plugins/plugins-library/examples/system-metrics.md
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/official/system-metrics, System metrics plugin on GitHub
|
||||
source: /shared/influxdb3-plugins/plugins-library/official/system-metrics.md
|
||||
---
|
||||
|
||||
<!-- //SOURCE - content/shared/influxdb3-plugins/plugins-library/examples/system-metrics.md -->
|
||||
<!-- //SOURCE - content/shared/influxdb3-plugins/plugins-library/official/system-metrics.md -->
|
||||
|
|
@ -5,7 +5,7 @@ menu:
|
|||
influxdb3_enterprise:
|
||||
name: Threshold deadman checks
|
||||
parent: Official plugins
|
||||
weight: 10
|
||||
weight: 100
|
||||
influxdb3/enterprise/tags: [plugins, processing engine, python, monitoring, thresholds, deadman, alerting]
|
||||
related:
|
||||
- https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/threshold_deadman_checks, Threshold deadman checks plugin on GitHub
|
||||
|
|
|
|||
|
|
@ -80,16 +80,14 @@ A plugin is a Python script that defines a specific function signature for a tri
|
|||
|
||||
You have two main options for adding plugins to your InfluxDB instance:
|
||||
|
||||
- [Use example plugins](#use-example-plugins) - Quickly get started with prebuilt plugins
|
||||
- [Use example plugins](#use-example-plugins) - Get started with prebuilt plugins
|
||||
- [Create a custom plugin](#create-a-custom-plugin) - Build your own for specialized use cases
|
||||
|
||||
### Use example plugins
|
||||
|
||||
InfluxData provides official plugins that you can use immediately in your Processing Engine setup.
|
||||
InfluxData maintains a repository of official and community plugins that you can use immediately in your Processing Engine setup.
|
||||
|
||||
#### Browse plugin examples
|
||||
|
||||
Visit the [plugin library](/influxdb3/version/plugins/library/) to find starter examples and official plugins for:
|
||||
Browse the [plugin library](/influxdb3/version/plugins/library/) to find examples and InfluxData official plugins for:
|
||||
|
||||
- **Data transformation**: Process and transform incoming data
|
||||
- **Alerting**: Send notifications based on data thresholds
|
||||
|
|
@ -97,11 +95,11 @@ Visit the [plugin library](/influxdb3/version/plugins/library/) to find starter
|
|||
- **Integration**: Connect to external services and APIs
|
||||
- **System monitoring**: Track resource usage and health metrics
|
||||
|
||||
For more examples and community contributions, see the [influxdb3_plugins repository](https://github.com/influxdata/influxdb3_plugins) on GitHub.
|
||||
For community contributions, see the [influxdb3_plugins repository](https://github.com/influxdata/influxdb3_plugins) on GitHub.
|
||||
|
||||
#### Add example plugins
|
||||
|
||||
You have two options for using example plugins:
|
||||
You have two options for using plugins from the repository:
|
||||
|
||||
##### Option 1: Copy plugins locally
|
||||
|
||||
|
|
|
|||
|
|
@ -322,4 +322,4 @@ if not influxdb3_local.cache.get("lookup_table"):
|
|||
|
||||
With an understanding of the InfluxDB 3 Shared Plugin API, you can start building data workflows that transform, analyze, and respond to your time series data.
|
||||
|
||||
To find example plugins you can extend, visit the [influxdb3_plugins repository](https://github.com/influxdata/influxdb3_plugins) on GitHub.
|
||||
For official plugins and examples that you can extend, see the [plugin library](/influxdb3/version/plugins/library/).
|
||||
|
|
@ -1,248 +0,0 @@
|
|||
The System Metrics Plugin collects comprehensive system performance metrics including CPU, memory, disk, and network statistics.
|
||||
This plugin runs on a scheduled basis to provide regular monitoring of your server infrastructure.
|
||||
|
||||
## Configuration
|
||||
|
||||
### Optional parameters
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `hostname` | string | "localhost" | Hostname to tag system metrics with |
|
||||
|
||||
## Requirements
|
||||
|
||||
### Software requirements
|
||||
- InfluxDB 3 Core or InfluxDB 3 Enterprise with Processing Engine enabled
|
||||
- Python packages:
|
||||
- `psutil` (for system metrics collection)
|
||||
|
||||
### Installation steps
|
||||
|
||||
1. Start InfluxDB 3 with plugin support:
|
||||
```bash
|
||||
influxdb3 serve \
|
||||
--node-id node0 \
|
||||
--object-store file \
|
||||
--data-dir ~/.influxdb3 \
|
||||
--plugin-dir ~/.plugins
|
||||
```
|
||||
|
||||
2. Install required Python packages:
|
||||
```bash
|
||||
influxdb3 install package psutil
|
||||
```
|
||||
|
||||
## Trigger setup
|
||||
|
||||
### Scheduled collection
|
||||
|
||||
Collect system metrics every 30 seconds:
|
||||
|
||||
```bash
|
||||
influxdb3 create trigger \
|
||||
--database monitoring \
|
||||
--plugin-filename examples/schedule/system_metrics/system_metrics.py \
|
||||
--trigger-spec "every:30s" \
|
||||
--trigger-arguments 'hostname=server01' \
|
||||
system_monitoring
|
||||
```
|
||||
|
||||
## Example usage
|
||||
|
||||
### Example: Basic system monitoring
|
||||
|
||||
Set up comprehensive system monitoring for a server:
|
||||
|
||||
```bash
|
||||
# Create the monitoring trigger
|
||||
influxdb3 create trigger \
|
||||
--database monitoring \
|
||||
--plugin-filename examples/schedule/system_metrics/system_metrics.py \
|
||||
--trigger-spec "every:60s" \
|
||||
--trigger-arguments 'hostname=web-server-01' \
|
||||
web_server_monitoring
|
||||
|
||||
# Query CPU metrics
|
||||
influxdb3 query \
|
||||
--database monitoring \
|
||||
"SELECT host, cpu, user, system, idle FROM system_cpu WHERE time >= now() - interval '5 minutes'"
|
||||
|
||||
# Query memory metrics
|
||||
influxdb3 query \
|
||||
--database monitoring \
|
||||
"SELECT host, total, used, available, percent FROM system_memory WHERE time >= now() - interval '5 minutes'"
|
||||
```
|
||||
|
||||
### Expected output
|
||||
|
||||
**CPU Metrics (`system_cpu`)**:
|
||||
```
|
||||
host | cpu | user | system | idle | time
|
||||
------------|-------|------|--------|------|-----
|
||||
web-server-01 | total | 15.2 | 8.1 | 76.7 | 2024-01-01T12:00:00Z
|
||||
```
|
||||
|
||||
**Memory Metrics (`system_memory`)**:
|
||||
```
|
||||
host | total | used | available | percent | time
|
||||
------------|------------|------------|------------|---------|-----
|
||||
web-server-01 | 8589934592 | 4294967296 | 4294967296 | 50.0 | 2024-01-01T12:00:00Z
|
||||
```
|
||||
|
||||
## Collected Measurements
|
||||
|
||||
### system_cpu
|
||||
Overall CPU statistics and performance metrics.
|
||||
|
||||
**Tags:**
|
||||
- `host`: Hostname (from configuration)
|
||||
- `cpu`: Always "total" for aggregate metrics
|
||||
|
||||
**Fields:**
|
||||
- `user`, `system`, `idle`, `iowait`, `nice`, `irq`, `softirq`, `steal`, `guest`, `guest_nice`: CPU time percentages
|
||||
- `frequency_current`, `frequency_min`, `frequency_max`: CPU frequency in MHz
|
||||
- `ctx_switches`, `interrupts`, `soft_interrupts`, `syscalls`: System call counts
|
||||
- `load1`, `load5`, `load15`: System load averages
|
||||
|
||||
### system_cpu_cores
|
||||
Per-core CPU metrics for detailed monitoring.
|
||||
|
||||
**Tags:**
|
||||
- `host`: Hostname
|
||||
- `core`: CPU core number (0, 1, 2, etc.)
|
||||
|
||||
**Fields:**
|
||||
- `usage`: CPU usage percentage for this core
|
||||
- `user`, `system`, `idle`: CPU time breakdowns per core
|
||||
- `frequency_current`, `frequency_min`, `frequency_max`: Per-core frequency
|
||||
|
||||
### system_memory
|
||||
System memory and virtual memory statistics.
|
||||
|
||||
**Tags:**
|
||||
- `host`: Hostname
|
||||
|
||||
**Fields:**
|
||||
- `total`, `available`, `used`, `free`: Memory amounts in bytes
|
||||
- `active`, `inactive`, `buffers`, `cached`, `shared`, `slab`: Memory usage breakdown
|
||||
- `percent`: Memory usage percentage
|
||||
|
||||
### system_swap
|
||||
Swap memory usage and statistics.
|
||||
|
||||
**Tags:**
|
||||
- `host`: Hostname
|
||||
|
||||
**Fields:**
|
||||
- `total`, `used`, `free`: Swap amounts in bytes
|
||||
- `percent`: Swap usage percentage
|
||||
- `sin`, `sout`: Swap in/out operations
|
||||
|
||||
### system_disk_usage
|
||||
Disk space usage for each mounted filesystem.
|
||||
|
||||
**Tags:**
|
||||
- `host`: Hostname
|
||||
- `device`: Device name (for example, /dev/sda1)
|
||||
- `mountpoint`: Mount point path
|
||||
- `fstype`: Filesystem type
|
||||
|
||||
**Fields:**
|
||||
- `total`, `used`, `free`: Disk space in bytes
|
||||
- `percent`: Disk usage percentage
|
||||
|
||||
### system_disk_io
|
||||
Disk I/O statistics for each disk device.
|
||||
|
||||
**Tags:**
|
||||
- `host`: Hostname
|
||||
- `device`: Device name
|
||||
|
||||
**Fields:**
|
||||
- `reads`, `writes`: Number of read/write operations
|
||||
- `read_bytes`, `write_bytes`: Bytes read/written
|
||||
- `read_time`, `write_time`: Time spent on I/O operations
|
||||
- `busy_time`: Time disk was busy
|
||||
- `read_merged_count`, `write_merged_count`: Merged I/O operations
|
||||
|
||||
### system_network
|
||||
Network interface statistics.
|
||||
|
||||
**Tags:**
|
||||
- `host`: Hostname
|
||||
- `interface`: Network interface name (eth0, wlan0, etc.)
|
||||
|
||||
**Fields:**
|
||||
- `bytes_sent`, `bytes_recv`: Network traffic in bytes
|
||||
- `packets_sent`, `packets_recv`: Network packets
|
||||
- `errin`, `errout`: Input/output errors
|
||||
- `dropin`, `dropout`: Dropped packets
|
||||
|
||||
## Code overview
|
||||
|
||||
### Files
|
||||
|
||||
- `system_metrics.py`: Main plugin code that collects and writes system metrics
|
||||
|
||||
### Main functions
|
||||
|
||||
#### `process_scheduled_call(influxdb3_local, time, args)`
|
||||
Entry point for scheduled metric collection. Orchestrates collection of all metric types.
|
||||
|
||||
#### `collect_cpu_metrics(influxdb3_local, hostname)`
|
||||
Collects CPU statistics including overall and per-core metrics.
|
||||
|
||||
#### `collect_memory_metrics(influxdb3_local, hostname)`
|
||||
Collects virtual memory, swap, and memory fault statistics.
|
||||
|
||||
#### `collect_disk_metrics(influxdb3_local, hostname)`
|
||||
Collects disk usage and I/O performance metrics.
|
||||
|
||||
#### `collect_network_metrics(influxdb3_local, hostname)`
|
||||
Collects network interface statistics.
|
||||
|
||||
### Logging
|
||||
|
||||
Logs are stored in the `_internal` database in the `system.processing_engine_logs` table. To view logs:
|
||||
|
||||
{{% code-placeholders "AUTH_TOKEN" %}}
|
||||
```bash
|
||||
influxdb3 query \
|
||||
--database _internal \
|
||||
--token AUTH_TOKEN \
|
||||
"SELECT * FROM system.processing_engine_logs WHERE trigger_name = 'system_monitoring'"
|
||||
```
|
||||
{{% /code-placeholders %}}
|
||||
|
||||
Replace {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}} with your {{% token-link "admin" %}}.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common issues
|
||||
|
||||
#### Issue: Missing psutil module
|
||||
**Solution**: Install the psutil package:
|
||||
```bash
|
||||
influxdb3 install package psutil
|
||||
```
|
||||
|
||||
#### Issue: Permission denied errors for disk metrics
|
||||
**Solution**: This is normal for system partitions that require elevated permissions. The plugin will skip these and continue collecting other metrics.
|
||||
|
||||
#### Issue: No per-core CPU metrics
|
||||
**Solution**: This can happen on some systems where per-core data isn't available. The overall CPU metrics will still be collected.
|
||||
|
||||
### Performance considerations
|
||||
|
||||
- Collection frequency: 30-60 second intervals are recommended for most use cases
|
||||
- The plugin handles errors gracefully and continues collecting available metrics
|
||||
- Some metrics may not be available on all operating systems (the plugin handles this automatically)
|
||||
|
||||
## Report an issue
|
||||
|
||||
For plugin issues, see the Plugins repository [issues page](https://github.com/influxdata/influxdb3_plugins/issues).
|
||||
|
||||
## Find support for {{% product-name %}}
|
||||
|
||||
The [InfluxDB Discord server](https://discord.gg/9zaNCW2PRT) is the best place to find support for InfluxDB 3 Core and InfluxDB 3 Enterprise.
|
||||
For other InfluxDB versions, see the [Support and feedback](#bug-reports-and-feedback) options.
|
||||
|
|
@ -0,0 +1,306 @@
|
|||
A comprehensive system monitoring plugin that collects CPU, memory, disk, and network metrics from the host system.
|
||||
This plugin provides detailed performance insights including per-core CPU statistics, memory usage breakdowns, disk I/O performance, and network interface statistics.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `hostname` | string | `localhost` | Hostname to tag metrics with |
|
||||
| `include_cpu` | boolean | `true` | Include CPU metrics collection |
|
||||
| `include_memory` | boolean | `true` | Include memory metrics collection |
|
||||
| `include_disk` | boolean | `true` | Include disk metrics collection |
|
||||
| `include_network` | boolean | `true` | Include network metrics collection |
|
||||
| `max_retries` | integer | `3` | Maximum number of retry attempts on failure |
|
||||
| `config_file_path` | string | None | Path to configuration file from PLUGIN_DIR env var |
|
||||
|
||||
## Requirements
|
||||
|
||||
- InfluxDB 3 Core or InfluxDB 3 Enterprise
|
||||
- Python psutil library (automatically installed)
|
||||
|
||||
### Files
|
||||
|
||||
- `system_metrics.py`: Main plugin file containing metric collection logic
|
||||
- `system_metrics_config_scheduler.toml`: Configuration template for scheduled triggers
|
||||
- `README.md`: This documentation file
|
||||
|
||||
### Features
|
||||
|
||||
- **CPU Metrics**: Overall and per-core CPU usage, frequency, load averages, context switches, and interrupts
|
||||
- **Memory Metrics**: RAM usage, swap statistics, and memory page fault information
|
||||
- **Disk Metrics**: Partition usage, I/O statistics, throughput, IOPS, and latency calculations
|
||||
- **Network Metrics**: Interface statistics including bytes/packets sent/received and error counts
|
||||
- **Configurable Collection**: Enable/disable specific metric types via configuration
|
||||
- **Robust Error Handling**: Retry logic and graceful handling of permission errors
|
||||
- **Task Tracking**: UUID-based task identification for debugging and log correlation
|
||||
|
||||
## Trigger Setup
|
||||
|
||||
### Install Required Dependencies
|
||||
|
||||
```bash
|
||||
influxdb3 install package psutil
|
||||
```
|
||||
|
||||
### Basic Scheduled Trigger
|
||||
|
||||
```bash
|
||||
influxdb3 create trigger \
|
||||
--database system_monitoring \
|
||||
--plugin-filename system_metrics.py \
|
||||
--trigger-spec "every:30s" \
|
||||
system_metrics_trigger
|
||||
```
|
||||
|
||||
### Using Configuration File
|
||||
|
||||
```bash
|
||||
influxdb3 create trigger \
|
||||
--database system_monitoring \
|
||||
--plugin-filename system_metrics.py \
|
||||
--trigger-spec "every:1m" \
|
||||
--trigger-arguments config_file_path=system_metrics_config_scheduler.toml \
|
||||
system_metrics_config_trigger
|
||||
```
|
||||
|
||||
### Custom Configuration
|
||||
|
||||
```bash
|
||||
influxdb3 create trigger \
|
||||
--database system_monitoring \
|
||||
--plugin-filename system_metrics.py \
|
||||
--trigger-spec "every:30s" \
|
||||
--trigger-arguments hostname=web-server-01,include_disk=false,max_retries=5 \
|
||||
system_metrics_custom_trigger
|
||||
```
|
||||
|
||||
## Example Usage
|
||||
|
||||
### Monitor Web Server Performance
|
||||
|
||||
```bash
|
||||
# Create trigger for web server monitoring every 15 seconds
|
||||
influxdb3 create trigger \
|
||||
--database web_monitoring \
|
||||
--plugin-filename system_metrics.py \
|
||||
--trigger-spec "every:15s" \
|
||||
--trigger-arguments hostname=web-server-01,include_network=true \
|
||||
web_server_metrics
|
||||
```
|
||||
|
||||
### Database Server Monitoring
|
||||
|
||||
```bash
|
||||
# Focus on CPU and disk metrics for database server
|
||||
influxdb3 create trigger \
|
||||
--database db_monitoring \
|
||||
--plugin-filename system_metrics.py \
|
||||
--trigger-spec "every:30s" \
|
||||
--trigger-arguments hostname=db-primary,include_disk=true,include_cpu=true,include_network=false \
|
||||
database_metrics
|
||||
```
|
||||
|
||||
### High-Frequency System Monitoring
|
||||
|
||||
```bash
|
||||
# Collect all metrics every 10 seconds with higher retry tolerance
|
||||
influxdb3 create trigger \
|
||||
--database system_monitoring \
|
||||
--plugin-filename system_metrics.py \
|
||||
--trigger-spec "every:10s" \
|
||||
--trigger-arguments hostname=critical-server,max_retries=10 \
|
||||
high_freq_metrics
|
||||
```
|
||||
|
||||
### Expected Output
|
||||
|
||||
After creating a trigger, you can query the collected metrics:
|
||||
|
||||
```bash
|
||||
influxdb3 query \
|
||||
--database system_monitoring \
|
||||
"SELECT * FROM system_cpu WHERE time >= now() - interval '5 minutes' LIMIT 5"
|
||||
```
|
||||
|
||||
#### Sample Output
|
||||
|
||||
```
|
||||
+------+--------+-------+--------+------+--------+-------+--------+-------+-------+------------+------------------+
|
||||
| host | cpu | user | system | idle | iowait | nice | irq | load1 | load5 | load15 | time |
|
||||
+------+--------+-------+--------+------+--------+-------+--------+-------+-------+------------+------------------+
|
||||
| srv1 | total | 12.5 | 5.3 | 81.2 | 0.8 | 0.0 | 0.2 | 0.85 | 0.92 | 0.88 | 2024-01-15 10:00 |
|
||||
| srv1 | total | 13.1 | 5.5 | 80.4 | 0.7 | 0.0 | 0.3 | 0.87 | 0.93 | 0.88 | 2024-01-15 10:01 |
|
||||
| srv1 | total | 11.8 | 5.1 | 82.0 | 0.9 | 0.0 | 0.2 | 0.83 | 0.91 | 0.88 | 2024-01-15 10:02 |
|
||||
| srv1 | total | 14.2 | 5.8 | 79.0 | 0.8 | 0.0 | 0.2 | 0.89 | 0.92 | 0.88 | 2024-01-15 10:03 |
|
||||
| srv1 | total | 12.9 | 5.4 | 80.6 | 0.9 | 0.0 | 0.2 | 0.86 | 0.92 | 0.88 | 2024-01-15 10:04 |
|
||||
+------+--------+-------+--------+------+--------+-------+--------+-------+-------+------------+------------------+
|
||||
```
|
||||
|
||||
## Code Overview
|
||||
|
||||
### Main Functions
|
||||
|
||||
#### `process_scheduled_call()`
|
||||
|
||||
The main entry point for scheduled triggers.
|
||||
Collects system metrics based on configuration and writes them to InfluxDB.
|
||||
|
||||
```python
|
||||
def process_scheduled_call(influxdb3_local, call_time, args):
|
||||
# Parse configuration
|
||||
config = parse_config(args)
|
||||
|
||||
# Collect metrics based on configuration
|
||||
if config['include_cpu']:
|
||||
collect_cpu_metrics(influxdb3_local, config['hostname'])
|
||||
|
||||
if config['include_memory']:
|
||||
collect_memory_metrics(influxdb3_local, config['hostname'])
|
||||
|
||||
# ... additional metric collections
|
||||
```
|
||||
|
||||
#### `collect_cpu_metrics()`
|
||||
|
||||
Collects CPU utilization and performance metrics:
|
||||
|
||||
```python
|
||||
def collect_cpu_metrics(influxdb3_local, hostname):
|
||||
# Get overall CPU stats
|
||||
cpu_percent = psutil.cpu_percent(interval=1, percpu=False)
|
||||
cpu_times = psutil.cpu_times()
|
||||
|
||||
# Build and write CPU metrics
|
||||
line = LineBuilder("system_cpu")
|
||||
.tag("host", hostname)
|
||||
.tag("cpu", "total")
|
||||
.float64_field("user", cpu_times.user)
|
||||
.float64_field("system", cpu_times.system)
|
||||
.float64_field("idle", cpu_times.idle)
|
||||
.time_ns(time.time_ns())
|
||||
|
||||
influxdb3_local.write(line)
|
||||
```
|
||||
|
||||
### Measurements and Fields
|
||||
|
||||
#### system_cpu
|
||||
|
||||
Overall CPU statistics and metrics:
|
||||
- **Tags**: `host`, `cpu=total`
|
||||
- **Fields**: `user`, `system`, `idle`, `iowait`, `nice`, `irq`, `softirq`, `steal`, `guest`, `guest_nice`, `frequency_current`, `frequency_min`, `frequency_max`, `ctx_switches`, `interrupts`, `soft_interrupts`, `syscalls`, `load1`, `load5`, `load15`
|
||||
|
||||
#### system_cpu_cores
|
||||
|
||||
Per-core CPU statistics:
|
||||
- **Tags**: `host`, `core` (core number)
|
||||
- **Fields**: `usage`, `user`, `system`, `idle`, `iowait`, `nice`, `irq`, `softirq`, `steal`, `guest`, `guest_nice`, `frequency_current`, `frequency_min`, `frequency_max`
|
||||
|
||||
#### system_memory
|
||||
|
||||
System memory statistics:
|
||||
- **Tags**: `host`
|
||||
- **Fields**: `total`, `available`, `used`, `free`, `active`, `inactive`, `buffers`, `cached`, `shared`, `slab`, `percent`
|
||||
|
||||
#### system_swap
|
||||
|
||||
Swap memory statistics:
|
||||
- **Tags**: `host`
|
||||
- **Fields**: `total`, `used`, `free`, `percent`, `sin`, `sout`
|
||||
|
||||
#### system_memory_faults
|
||||
|
||||
Memory page fault information (when available):
|
||||
- **Tags**: `host`
|
||||
- **Fields**: `page_faults`, `major_faults`, `minor_faults`, `rss`, `vms`, `dirty`, `uss`, `pss`
|
||||
|
||||
#### system_disk_usage
|
||||
|
||||
Disk partition usage:
|
||||
- **Tags**: `host`, `device`, `mountpoint`, `fstype`
|
||||
- **Fields**: `total`, `used`, `free`, `percent`
|
||||
|
||||
#### system_disk_io
|
||||
|
||||
Disk I/O statistics:
|
||||
- **Tags**: `host`, `device`
|
||||
- **Fields**: `reads`, `writes`, `read_bytes`, `write_bytes`, `read_time`, `write_time`, `busy_time`, `read_merged_count`, `write_merged_count`
|
||||
|
||||
#### system_disk_performance
|
||||
|
||||
Calculated disk performance metrics:
|
||||
- **Tags**: `host`, `device`
|
||||
- **Fields**: `read_bytes_per_sec`, `write_bytes_per_sec`, `read_iops`, `write_iops`, `avg_read_latency_ms`, `avg_write_latency_ms`, `util_percent`
|
||||
|
||||
#### system_network
|
||||
|
||||
Network interface statistics:
|
||||
- **Tags**: `host`, `interface`
|
||||
- **Fields**: `bytes_sent`, `bytes_recv`, `packets_sent`, `packets_recv`, `errin`, `errout`, `dropin`, `dropout`
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### Permission Errors
|
||||
|
||||
Some disk I/O metrics may require elevated permissions:
|
||||
|
||||
```
|
||||
ERROR: [Permission denied] Unable to access disk I/O statistics
|
||||
```
|
||||
|
||||
**Solution**: The plugin will continue collecting other metrics even if some require elevated permissions.
|
||||
|
||||
#### Missing psutil Library
|
||||
|
||||
```
|
||||
ERROR: No module named 'psutil'
|
||||
```
|
||||
|
||||
**Solution**: Install the psutil package:
|
||||
|
||||
```bash
|
||||
influxdb3 install package psutil
|
||||
```
|
||||
|
||||
#### High CPU Usage
|
||||
|
||||
If the plugin causes high CPU usage, consider:
|
||||
- Increasing the trigger interval (for example, from `every:10s` to `every:30s`)
|
||||
- Disabling unnecessary metric types
|
||||
- Reducing the number of disk partitions monitored
|
||||
|
||||
### Viewing Logs
|
||||
|
||||
Logs are stored in the `_internal` database in the `system.processing_engine_logs` table:
|
||||
|
||||
```bash
|
||||
influxdb3 query \
|
||||
--database _internal \
|
||||
"SELECT * FROM system.processing_engine_logs WHERE trigger_name = 'system_metrics_trigger' ORDER BY time DESC LIMIT 10"
|
||||
```
|
||||
|
||||
### Verifying Data Collection
|
||||
|
||||
Check that metrics are being collected:
|
||||
|
||||
```bash
|
||||
# List all system metric measurements
|
||||
influxdb3 query \
|
||||
--database system_monitoring \
|
||||
"SHOW MEASUREMENTS WHERE measurement =~ /^system_/"
|
||||
|
||||
# Check recent CPU metrics
|
||||
influxdb3 query \
|
||||
--database system_monitoring \
|
||||
"SELECT COUNT(*) FROM system_cpu WHERE time >= now() - interval '1 hour'"
|
||||
```
|
||||
## Report an issue
|
||||
|
||||
For plugin issues, see the Plugins repository [issues page](https://github.com/influxdata/influxdb3_plugins/issues).
|
||||
|
||||
## Find support for {{% product-name %}}
|
||||
|
||||
The [InfluxDB Discord server](https://discord.gg/9zaNCW2PRT) is the best place to find support for InfluxDB 3 Core and InfluxDB 3 Enterprise.
|
||||
For other InfluxDB versions, see the [Support and feedback](#bug-reports-and-feedback) options.
|
||||
Loading…
Reference in New Issue