deprecation
Promscale has been discontinued. We strongly recommend that you do not use Promscale in a production environment. Learn more.
This section describes the compute and disk requirements that are recommended for Promscale, based on the ingest rate and retention.
Disk size per day: disk consumption per day based on the ingest rate.
Uncompressed buffer: disk consumed by uncompressed data before compression.
Total disk size: size of disk required to store the data based on the ingest rate and retention.
note
You can calculate the
total disk size
based on
retention and ingest rates with this
formula:
Total disk size = (Disk size per day based on the ingest rate * Retention in days) + Uncompressed buffer based on the ingest rate.
Resource recommendation for ingestion using
Prometheus remote-write
.
For optimal performance of
remote_write
to Promscale, use this
Prometheus
remote_write
configuration:
remote_write:remote_timeout: 100squeue_config:capacity: 100000max_samples_per_send: 10000batch_send_deadline: 30smin_shards: 20max_shards: 20min_backoff: 100msmax_backoff: 10s
Compute recommendations for the Promscale connector and TimescaleDB are:
Ingestion Rate | Connector CPU | Connector Memory | DB CPU | DB Memory | DB connections |
---|---|---|---|---|---|
10k samples/sec | 0.5 | 250 MB | 1 | 4 GB | 1 |
50k samples/sec | 2 | 700 MB | 4 | 16 GB | 2 |
100k samples/sec | 4 | 2 GB | 8 | 32 GB | 4 |
200k samples/sec | 8 | 4.5 GB | 16 | 64 GB | 8 |
Disk recommendations for TimescaleDB are:
The default chunk interval is 8h
Ingest rate | Retention | Disk size per day | Uncompressed buffer | Total disk size | WAL size |
---|---|---|---|---|---|
10k samples/sec | 90 days | ~2 GB | ~21 GB | ~200 GB | 1.25 GB |
50k samples/sec | 90 days | ~10 GB | ~105 GB | ~1 TB | - |
100k samples/sec | 90 days | ~20 GB | ~210 GB | ~2 TB | - |
200k samples/sec | 90 days | ~40 GB | ~420 GB | ~4 TB | - |
Resource recommendation for ingestion through OTLP (OpenTelemetry Line Procotol) gRPC endpoint.
We recommend using the OpenTelemetry collector for ingesting the spans to Promscale. Use this configuration with the OTLP exporter and batch processor to help with retries for failed writes, and to batch the write requests to Promscale:
exporters:logging:otlp:endpoint: "<PROMSCALE_HOST>:<gRPC_PORT>"tls:insecure: truesending_queue:queue_size: 1000000timeout: 10sprocessors:batch:send_batch_size: 4000send_batch_max_size: 4000timeout: 10s
Where:
<PROMSCALE_HOST>
: hostname of Promscale<gRPC_PORT>
: gRPC port of Promscale. The default port is 9202.
Compute recommendations for the Promscale connector and TimescaleDB are:
Ingestion Rate | Connector CPU | Connector Memory | DB CPU | DB Memory | DB connections |
---|---|---|---|---|---|
10k spans/sec = ~2.5 MB/sec | 0.5 | 4 GB | 2 | 4 GB | 4 |
50k spans/sec = ~5 MB/sec | 2 | 8 MB | 4 | 8 GB | 8 |
100k spans/sec = ~10 MB/sec | 4 | 16 GB | 8 | 16 GB | 16 |
200k spans/sec = ~20 MB/sec | 8 | 16 GB | 16 | 16 GB | 32 |
Disk recommendations for TimescaleDB are:
The default chunk interval is 1h
Ingest rate | Retention | Disk size per day | Uncompressed buffer | Total disk size | WAL size |
---|---|---|---|---|---|
10k spans/sec = ~2.5 MB/sec | 30 days | ~30 GB | ~35 GB | ~935 GB | 20 GB |
50k spans/sec = ~5 MB/sec | 30 days | ~60 GB | ~175 GB | ~2 TB | - |
100k spans/sec= ~10 MB/sec | 30 days | ~150 GB | ~350 GB | ~5 TB | - |
200k spans/sec= ~20 MB/sec | 30 days | ~300 GB | ~700 GB | ~10 TB | - |