Join us for a virtual meetup on Zoom at 8 PM, July 31 (PDT) about using One Time Series Database for Both Metrics and Logs 👉🏻 Register Now

Skip to content
On this page
Biweekly
August 14, 2024

Scaling to the Cloud for Hassle-Free Usage with Continued Optimization of Remote WAL and Object Storage Performance | Greptime Biweekly Report

A recap of the past two-weeks progress and changes happened on GreptimeDB.

Summary

Together with our global community of contributors, GreptimeDB continues to evolve and flourish as a growing open-source project. We are grateful to each and every one of you.

Below are the highlights among recent commits:

  • Improve Remote WAL performance by introducing new Kafka Topic Consumer

  • Improve fetching performance on object storage

  • Add file number limits to TWCS compaction

  • Support Kafka SASL and TLS config for RemoteWAL

Contributors

For the past two weeks, our community has been super active with a total of 82 PRs merged. Of these, 5 PRs were contributed by 3 individual contributors, with many more pending to be merged.

Congrats on becoming our most active contributors in the past 2 weeks!

👏 Welcome contributor @Stephan3555 join to the community as the new individual contributor, and congratulations on successfully merging their first PR, more PRs are waiting to be merged.

New Contributors of GreptimeDB
New Contributors of GreptimeDB

A big THANK YOU to all our members and contributors! It is people like you who are making GreptimeDB a great product. Let's build an even greater community together.

Highlights of Recent PRs

db#4424 Improve RemoteWAL performance by introducing new Kafka Topic Consumer

Implement a Kafka consumer that respects the WAL index info, aims to reduce the read amplification during replaying a single region.

db#4520 Improve fetching performance on object storage

Merge small byte ranges for optimized fetching. Here again, thanks to the exquisite design of the @OpenDAL community!

db#4453 Reduce lock scope and improve log

This PR reduces the scope of certain locks, preventing different streams from blocking each other for extended periods. In append mode, this lock would prevent parallel reads between parts.

db#4536 Support Kafka SASL and TLS config for RemoteWAL

Allow users to configure the SASL/PLAIN, SASL/SCRAM-SHA-256, SASL/SCRAM-SHA-512 and TLS for the Kafka client.

db#4481 Add file number limits to TWCS compaction

Introduce max_active_window_files and max_inactive_window_files in TwcsOptions to limit SST file numbers in both active and inactive windows.

Good First Issue

db#4480 Add debug level traces in Mito Engine

The current Trace is not suitable for user performance debugging. More detailed Spans could be added in the Mito Engine, such as the time taken to read data from S3, etc.

  • Keywords: Mito Engine, Trace

  • Difficulty: Medium

db#4453 Fix incorrect version label in greptime_app_version metric

The version in the greptime_app_version metric within /metrics always includes a leading -. This is likely caused by the loss of the Branch value in the statistics.

  • Keywords: Metrics

  • Difficulty: Simple


About Greptime

We help industries that generate large amounts of time-series data, such as Connected Vehicles (CV), IoT, and Observability, to efficiently uncover the hidden value of data in real-time.

Visit the latest version from any device to get started and get the most out of your data.

  • GreptimeDB, written in Rust, is a distributed, open-source, time-series database designed for scalability, efficiency, and powerful analytics.
  • Edge-Cloud Integrated TSDB is designed for the unique demands of edge storage and compute in IoT. It tackles the exponential growth of edge data by integrating a multimodal edge-side database with cloud-based GreptimeDB Enterprise. This combination reduces traffic, computing, and storage costs while enhancing data timeliness and business insights.
  • GreptimeCloud is a fully-managed cloud database-as-a-service (DBaaS) solution built on GreptimeDB. It efficiently supports applications in fields such as observability, IoT, and finance.

Star us on GitHub or join GreptimeDB Community on Slack to get connected. Also, you can go to our contribution page to find some interesting issues to start with.

biweekly

Join our community

Get the latest updates and discuss with other users.