Skip to content
On this page
Biweekly
November 6, 2024

Committer +1! GreptimeDB Sorting Performance Optimization | Greptime Biweekly Report

A recap of the past 2 weeks progress and changes happened on GreptimeDB.

Summary

Together with our global community of contributors, GreptimeDB continues to evolve and flourish as a growing open-source project. We are grateful to each and every one of you.

Below are the highlights among recent commits:

  • Sorting Performance Optimization: Introduced the Part Sort physical operator and Windowed Sort rule.

  • Resource Usage Limitation: Introducing a resource-limiting mechanism can be useful in resource-constrained environments (especially embedded devices).

  • Storage Engine Performance Enhancement: Accelerated queries by caching row groups for a time range.

  • Dynamic Table Option Changes: Allow the table TTL to be dynamically modified.

Contributors

For the past two weeks, our community has been super active with a total of 88 PRs merged. Of these, 5 PRs were contributed by 3 individual contributors, with many more pending to be merged.

👏 Welcome @ActivePeter to the community as a new contributor with a successfully merged PR, and more PRs from other individual contributors are waiting to be merged.

New Contributors of GreptimeDB
New Contributors of GreptimeDB

🎉 According to the result of the Committer promotion vote in GreptimeDB community 2024.11.1, @lyang24 is officially promoted as a new Committer due to his high quality PRs and long term contributions, looking forward to more of your contributions in the community!

The Nomination
The Nomination

😄 Warmly welcome @lyang24, an individual contributor, to the community as a new Committer, thank you for your efforts and hope we can continue to work hand in hand to build the community.

New Committer of GreptimeDB
New Committer of GreptimeDB

A big THANK YOU to all our members and contributors! It is people like you who are making GreptimeDB a great product. Let's build an even greater community together.

Highlights of Recent PRs

db#4847 db#4922 Introduce PartSort Physical Operator

Added the PartSort physical operator to sort data within PartitionRange boundaries, and implemented a simple limit mechanism in the PartSort operator.

db#4874 Introduce Windowed Sort Rule

The optimizer has been improved by introducing a windowed sort rule, utilizing the PartSort physical operator to enhance overall sorting performance.

db#4859 OpenTelemetry Data Writing Enhancement

Enhanced OpenTelemetry writing capabilities by allowing fields to be extracted from attributes for indexing, increasing data processing flexibility.

db#4782 CPU Usage Limitation Mechanism

Implemented a CPU usage limitation mechanism to operate efficiently in resource-constrained environments, such as embedded devices.

db#4864 New JSON User-Defined Function json_path_match

Introduced a new JSON user-defined function, json_path_match, to expand JSON data handling capabilities.

db#4871 Separate Schedulers in Storage Engine

Optimized the storage engine by separating the schedulers for flush and compaction tasks, addressing scheduler blockage issues.

db#4869 Cache Row Groups over Time Range in Storage Engine

Enhanced query performance by caching row groups over time ranges and retrieving them in subsequent queries.

db#4848 Support for Dynamic Table Option Changes

Enabled dynamic modification of table options (such as TTL) through SQL syntax, enhancing table flexibility.

The syntax is:

sql
ALTER TABLE <table_name> SET <option_name>=<option_value>[, ...];

db#4909 Refine Region Statistics Table

Added new columns disk_size, region_rows, and index_size to information_schema.region_statistics for more detailed regional statistics.

db#4908 Insert JSON Data via gRPC Protocol

Implemented the ability to insert JSON data via the gRPC protocol, expanding JSON data writing methods.

Good First Issue

db#4935 Fix the region_stats test

Fix the test case tests::instance_test::test_use_database.

  • Keywords: Region

  • Difficulty: Simple

db#4932 Unify Datanode options in Frontend and Metasrv

Two Datanode options with the same functionality are located in frontend::service_config::datanode and meta_srv::metasrv, which need to be unified.

  • Keywords: Config

  • Difficulty: Simple


About Greptime

Greptime offers industry-leading time series database products and solutions to empower IoT and Observability scenarios, enabling enterprises to uncover valuable insights from their data with less time, complexity, and cost.

GreptimeDB is an open-source, high-performance time-series database offering unified storage and analysis for metrics, logs, and events. Try it out instantly with GreptimeCloud, a fully-managed DBaaS solution—no deployment needed!

The Edge-Cloud Integrated Solution combines multimodal edge databases with cloud-based GreptimeDB to optimize IoT edge scenarios, cutting costs while boosting data performance.

Star us on GitHub or join GreptimeDB Community on Slack to get connected.

Join our community

Get the latest updates and discuss with other users.