# Committer +1! GreptimeDB Sorting Performance Optimization | Greptime Biweekly Report

Source: https://greptime.com/blogs/2024-11-06-biweekly-report

## Summary

Together with our global community of contributors, GreptimeDB continues to evolve and flourish as a growing open-source project. We are grateful to each and every one of you.

Below are the highlights among recent commits:

- **Sorting Performance Optimization**: Introduced the Part Sort physical operator and Windowed Sort rule.

- **Resource Usage Limitation**: Introducing a resource-limiting mechanism can be useful in resource-constrained environments (especially embedded devices).

- **Storage Engine Performance Enhancement**: Accelerated queries by caching row groups for a time range.

- **Dynamic Table Option Changes**: Allow the table TTL to be dynamically modified.

## Contributors

For the past two weeks, our community has been super active with a total of **88 PRs** merged. Of these, **5 PRs** were contributed by **3 individual contributors**, with many more pending to be merged.

- @[CookiePieWw](https://github.com/CookiePieWw) ([db#4933](https://github.com/GreptimeTeam/greptimedb/pull/4933) [db#4919](https://github.com/GreptimeTeam/greptimedb/pull/4919) [db#4893](https://github.com/GreptimeTeam/greptimedb/pull/4893))

- @[Kev1n8](https://github.com/Kev1n8)（[db#4864](https://github.com/GreptimeTeam/greptimedb/pull/4864)）

- @[ActivePeter](https://github.com/ActivePeter)（[db#4782](https://github.com/GreptimeTeam/greptimedb/pull/4782)）

👏 Welcome @[ActivePeter](https://github.com/ActivePeter) to the community as a new contributor with a successfully merged PR, and more PRs from other individual contributors are waiting to be merged.

<figure>
<img src="/blogs/2024-11-06-biweekly-report/contributorpost.png" alt="New Contributors of GreptimeDB" style="width: 70%; margin: 0 auto">
<figcaption>New Contributors of GreptimeDB</figcaption>
</figure>

🎉 According to the result of the Committer promotion vote in GreptimeDB community 2024.11.1, @[lyang24](https://github.com/lyang24) is officially promoted as a new Committer due to his high quality PRs and long term contributions, looking forward to more of your contributions in the community!

<figure>
<img src="/blogs/2024-11-06-biweekly-report/nomination.png" alt="The Nomination" style="width: 70%; margin: 0 auto">
<figcaption>The Nomination</figcaption>
</figure>

😄 Warmly welcome @[lyang24](https://github.com/lyang24), an individual contributor, to the community as a new Committer, thank you for your efforts and hope we can continue to work hand in hand to build the community.

<figure>
<img src="/blogs/2024-11-06-biweekly-report/committerpost.png" alt="New Committer of GreptimeDB" style="width: 70%; margin: 0 auto">
<figcaption>New Committer of GreptimeDB</figcaption>
</figure>

A big THANK YOU to all our members and contributors! It is people like you who are making GreptimeDB a great product. Let's build an even greater community together.

## Highlights of Recent PRs

### [db#4847](https://github.com/GreptimeTeam/greptimedb/pull/4847) [db#4922](https://github.com/GreptimeTeam/greptimedb/pull/4922) Introduce PartSort Physical Operator

Added the PartSort physical operator to sort data within PartitionRange boundaries, and implemented a simple limit mechanism in the PartSort operator.

### [db#4874](https://github.com/GreptimeTeam/greptimedb/pull/4874) Introduce Windowed Sort Rule

The optimizer has been improved by introducing a windowed sort rule, utilizing the PartSort physical operator to enhance overall sorting performance.

### [db#4859](https://github.com/GreptimeTeam/greptimedb/pull/4859) OpenTelemetry Data Writing Enhancement

Enhanced OpenTelemetry writing capabilities by allowing fields to be extracted from attributes for indexing, increasing data processing flexibility.

### [db#4782](https://github.com/GreptimeTeam/greptimedb/pull/4782) CPU Usage Limitation Mechanism

Implemented a CPU usage limitation mechanism to operate efficiently in resource-constrained environments, such as embedded devices.

### [db#4864](https://github.com/GreptimeTeam/greptimedb/pull/4864) New `JSON` User-Defined Function `json_path_match`

Introduced a new `JSON` user-defined function, `json_path_match`, to expand `JSON` data handling capabilities.

### [db#4871](https://github.com/GreptimeTeam/greptimedb/pull/4871) Separate Schedulers in Storage Engine

Optimized the storage engine by separating the schedulers for flush and compaction tasks, addressing scheduler blockage issues.

### [db#4869](https://github.com/GreptimeTeam/greptimedb/pull/4869) Cache Row Groups over Time Range in Storage Engine

Enhanced query performance by caching row groups over time ranges and retrieving them in subsequent queries.

### [db#4848](https://github.com/GreptimeTeam/greptimedb/pull/4848) Support for Dynamic Table Option Changes

Enabled dynamic modification of table options (such as TTL) through SQL syntax, enhancing table flexibility.

The syntax is:

```sql
ALTER TABLE <table_name> SET <option_name>=<option_value>[, ...];
```

### [db#4909](https://github.com/GreptimeTeam/greptimedb/pull/4909) Refine Region Statistics Table

Added new columns `disk_size`, `region_rows`, and `index_size` to `information_schema.region_statistics` for more detailed regional statistics.

### [db#4908](https://github.com/GreptimeTeam/greptimedb/pull/4908) Insert `JSON` Data via gRPC Protocol

Implemented the ability to insert `JSON` data via the gRPC protocol, expanding `JSON` data writing methods.

## Good First Issue

### [db#4935](https://github.com/GreptimeTeam/greptimedb/issues/4935) Fix the `region_stats` test

Fix the test case `tests::instance_test::test_use_database`.

- Keywords: Region

- Difficulty: Simple

### [db#4932](https://github.com/GreptimeTeam/greptimedb/issues/4932) Unify Datanode options in Frontend and Metasrv

Two Datanode options with the same functionality are located in `frontend::service_config::datanode` and `meta_srv::metasrv`, which need to be unified.

- Keywords: Config

- Difficulty: Simple

------
#### About Greptime

Greptime offers industry-leading time series database products and solutions to empower IoT and Observability scenarios, enabling enterprises to uncover valuable insights from their data with less time, complexity, and cost.

[GreptimeDB](https://github.com/GreptimeTeam/greptimedb) is an open-source, high-performance time-series database offering unified storage and analysis for metrics, logs, and events. Try it out instantly with [GreptimeCloud](https://www.greptime.com/product/cloud), a fully-managed DBaaS solution—no deployment needed!

The [Edge-Cloud Integrated Solution](https://greptime.com/product/carcloud) combines multimodal edge databases with cloud-based GreptimeDB to optimize IoT edge scenarios, cutting costs while boosting data performance.

Star us on [GitHub](https://github.com/GreptimeTeam/greptimedb) or join GreptimeDB Community on [Slack](https://www.greptime.com/slack) to get connected.
