Solutions · Observability

Logs and metrics on a database you can afford.

ClickHouse is the storage layer beneath ClickHouse's own observability products and several major SaaS vendors. Run it yourself on Luceris and pay for hardware, not seats.

Start free Talk to us

Why ClickHouse for observability

Observability data is append-mostly, time-ordered, and bursty — exactly the shape ClickHouse is optimized for. Compression ratios on JSON logs typically exceed 10×; partitioning by day makes retention enforcement a metadata drop instead of a delete-storm. You wire log shippers (Vector, Fluent Bit, OpenTelemetry Collector) at ClickHouse via HTTP or the native protocol — no proprietary agent required.

Partition by day

Drop a partition to retain N days. No `DELETE` scans, no tombstones, no compaction debt.

S3-backed cold tier

TTL moves data older than your hot window to S3. Queries still run; only the latency changes.

Compression that matters

ZSTD on JSON logs hits ~10× compression typical. LowCardinality on `level` and `service` saves more.

Works with the agents you already run

Vector, Fluent Bit, OpenTelemetry Collector, Logstash — all have native ClickHouse outputs.

Code sample

yaml

Vector pipeline shipping logs to Luceris ClickHouse

# vector.yaml
sources:
  app_logs:
    type: file
    include: ["/var/log/app/*.log"]
    read_from: end

transforms:
  parse_json:
    type: remap
    inputs: [app_logs]
    source: |
      . = parse_json!(.message)
      .ts = parse_timestamp!(.timestamp, "%+")

sinks:
  clickhouse:
    type: clickhouse
    inputs: [parse_json]
    endpoint: https://obs-abc.eu.luceris.cloud/clickhouse
    auth:
      strategy: basic
      user: app
      password: ${LUCERIS_PASSWORD}
    database: logs
    table: events
    compression: gzip

Schema sketch

A typical logs table: `(ts DateTime, service LowCardinality(String), level LowCardinality(String), trace_id String, msg String, attrs JSON)`. Partition by `toYYYYMMDD(ts)`. Order by `(service, ts)` for service-scoped queries. TTL the partition after 30 days to S3, after 365 days to delete. Add a materialized view that downsamples to per-minute counts for fast dashboards.

Examples

Application logs — structured JSON from your services
Access logs — high-volume web request logs with low per-row cost
Metrics — Prometheus remote-write into ClickHouse via the OpenMetrics receiver
Distributed tracing — trace spans with their attribute bags
Audit logs — long-retention, low-volume, compliance-grade query trail

Frequently asked questions

Can ClickHouse replace my SIEM?

For storage and query, often yes — ClickHouse handles the volume cheaply. For alerting and correlation, pair it with the alerting layer you already use (Grafana, Vector, custom). The SIEM-as-a-product features (case management, response runbooks) you still need elsewhere.

How do I handle high cardinality?

Use `LowCardinality(String)` on columns with a bounded universe (`service`, `level`, `region`). For genuinely high cardinality (`trace_id`, `user_id`), just use `String` — ClickHouse handles it efficiently with the right primary key.

What about real-time log tailing?

Queries hit fresh inserts within seconds. For sub-second freshness, use a small ingestion buffer (Kafka or a ClickHouse Kafka engine table). For a true `tail -f` experience, hit the upstream agent's buffer instead of the database.

Stop paying per-GB for logs you can store yourself.

Free tier ClickHouse cluster gives you 10 GB compressed — about 100 GB of typical JSON logs.

Start free Talk to us