Skip to content

Prometheus Schema Auto-alignment & Flat Scan Path | Greptime Biweekly Report - No. 80

Prometheus remote write batching automatically aligns schemas across different metric series. Export operations stream data in chunks instead of...
Prometheus Schema Auto-alignment & Flat Scan Path | Greptime Biweekly Report - No. 80
On this page

Summary

Development period: 2026-03-24 - 2026-04-06

Here are the highlights from recent commits:

  • Prometheus remote write batching automatically aligns schemas across different metrics
  • Export operations stream data in chunks instead of loading full datasets into memory
  • Primary key prefilters skip irrelevant primary keys before reading other columns

We encourage users on older versions to upgrade for these fixes and improvements.

Contributors

Over the past two weeks, 15 contributors merged a total of 43 PRs. Among them, 3 individual contributors contributed 3 PRs. Welcome to our new contributors: @ynachi, @cuiweixie, @Boudewijn26!

Thanks to our individual contributors:

Highlights of Recent PRs

db#7877 feat: auto-align Prometheus schemas in pending rows batching

Prometheus remote write batching now automatically resolves schema conflicts by aligning logical table schemas to physical table expectations during row batching. This eliminates redundant table operations and delivers approximately 2x performance improvement in 4-region configurations.

db#7901 feat: always use flat scan path for both format

Until now, GreptimeDB's mito2 engine used two separate scan paths for different query patterns. The engine now uses a single unified scan path for all queries, removing the dual-path complexity.

db#7902 refactor(metric-engine): Refactor PendingRowsBatcher for better testability and benchmarking

The PendingRowsBatcher in the metric engine now uses Arc<Error> to share error instances across batch waiters instead of duplicating error strings. Internal functions like flush_batch_physical are exposed for external benchmarking, and the component includes new Criterion benchmarks to measure physical batch flushing performance under different workloads.

db#7841 feat: implement export-v2 chunked data export flow

The new export-v2 feature processes data exports in chunks rather than loading entire datasets into memory. Large table exports now complete without running into memory limitations during the export process.

db#7862 feat: implement prefilter framework and primary key prefilter

Until now, parquet reads in mito2 processed all columns before applying primary key filters. A new prefilter framework first reads only primary key columns to compute row selections, then applies those selections when reading the full projection to reduce unnecessary data processing.

Good First Issue

Issue#7585 Revoke KV write access outside metasrv leader

MetaKvBackend currently exposes full write APIs to non-leader components (frontend, flownode, datanode), violating leader-only write assumptions. Introduce a read-only wrapper at the catalog client layer, disable write calls in the meta-client store, and enforce leader-only writes server-side in metasrv store service.

Keywords: Metasrv, Refactor

Difficulty: Medium

Issue#7544 Allow ALTER TABLE to modify the skip_wal option dynamically

Add support for the ALTER TABLE SET statement to allow users to change the skip_wal table option after table creation, so they can turn WAL on or off at runtime.

Keywords: SQL parser, Storage, ALTER TABLE

Difficulty: Medium

Stay in the loop

Join our community