Join us for a virtual meetup on Zoom at 8 PM, July 31 (PDT) about using One Time Series Database for Both Metrics and Logs 👉🏻 Register Now

Skip to content
On this page
Biweekly
April 3, 2025

Skip WAL for Faster Writes, JSON Parsing Accelerated by 24% | Greptime Biweekly Report

A recap of the past 2 weeks progress and changes happened on GreptimeDB.

Summary

Together with our global community of contributors, GreptimeDB continues to evolve and flourish as a growing open-source project. We are grateful to each and every one of you.

Below are the highlights among recent commits:

  • Support skipping WAL when creating tables: Added skip_wal = "true" option to allow table-level WAL disabling, improving write performance.
  • Enhanced query analysis: EXPLAIN ANALYZE VERBOSE provides more detailed scanner information for better query debugging.
  • Optimized NDJSON parsing performance: Introduced simd_json, improving JSON parsing speed by ~24%.

Contributors

For the past two weeks, our community has been super active with a total of 82 PRs merged. 6 PRs from 3 individual contributors merged successfully and lots pending to be merged.

Congrats on becoming our most active contributors in the past 2 weeks:

👏 Welcome @JackieTien97 to the community as a new contributor with a successfully merged PR, and more PRs from other individual contributors are waiting to be merged.

(New Contributor of GreptimeDB)
(New Contributor of GreptimeDB)

🎉 A big THANK YOU to all our members and contributors! It is people like you who are making GreptimeDB a great product. Let's build an even greater community together.

Highlights of Recent PRs

db#5740 Support Skipping WAL When Creating Tables

Users can now disable WAL on a per-table options using skip_wal = "true", which enhances write efficiency in scenarios where WAL reliability is not critical. Example:

sql
CREATE TABLE system_metrics (
    host STRING,
    idc STRING,
    cpu_util DOUBLE,
    memory_util DOUBLE,
    disk_util DOUBLE,
    ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP(),
    PRIMARY KEY(host, idc),
    TIME INDEX(ts)
) WITH (skip_wal = "true");

db#5763 Enhanced Query Analysis with EXPLAIN ANALYZE VERBOSE

EXPLAIN ANALYZE VERBOSE is now supported in both standalone and distributed modes, providing more detailed insights into query execution.

  • Introduces ExplainOptions to manage EXPLAIN options, allowing future extensions.
  • In verbose mode, additional scanner details are printed, including:
    • Projections and filters
    • File details: file ID, time range, number of rows, file size, index size
    • Execution metrics: output rows, memory usage, elapsed time
c++
SeqScan: region=4440996184064(1034, 0), partition_count=2 (0 memtable ranges, 2 file 2 ranges), projection=["i", "t"], filters=[i > Int32(1)], files=[[file=775c368a-140d-4a64-a30b-c72ce23d2e68, time_range=(4::Millisecond, 6::Millisecond), rows=3, size=2543, index_size=0], [file=5b5024ae-3c95-4945-b1ed-24fd6bb73342, time_range=(1::Millisecond, 3::Millisecond), rows=3, size=2543, index_size=0]] metrics=[output_rows: 2, mem_used: 224, elapsed_await: 3001834, elapsed_poll: 5498708, ]

db#5794 Optimized NDJSON Parsing Performance

By introducing simd_json, NDJSON parsing performance has been significantly improved. In JSONBench tests for parsing 10 × 100M data:

  • Before: 49,502 ms
  • After: 37,620 ms (~24% improvement)

Good First Issue

db#5120 Add vector functions JSON_TO_VEC, VEC_TO_JSON

JSON_TO_VEC: Convert a JSON array to a vector. VEC_TO_JSON: Convert a vector to a JSON array.

  • Difficulty: Medium
  • Keywords: vector

About Greptime

Greptime offers industry-leading time series database products and solutions to empower IoT and Observability scenarios, enabling enterprises to uncover valuable insights from their data with less time, complexity, and cost.

GreptimeDB is an open-source, high-performance time-series database offering unified storage and analysis for metrics, logs, and events. Try it out instantly with GreptimeCloud, a fully-managed DBaaS solution—no deployment needed!

The Edge-Cloud Integrated Solution combines multimodal edge databases with cloud-based GreptimeDB to optimize IoT edge scenarios, cutting costs while boosting data performance.

Star us on GitHub or join GreptimeDB Community on Slack to get connected.

Join our community

Get the latest updates and discuss with other users.