Biweekly Report (Mar.13 - Mar.26) – Support executing PromQL queries through gRPC

Summary ​

Together with all our contributors worldwide, we are glad to see GreptimeDB making remarkable progress for the better. Below are some highlights:

• Support executing PromQL queries through gRPC
• Graceful shutdown implemented for all components
• Support for manually triggering Flush
• COPY TO statement with CONNECTION option
• Adding a private gRPC protocol to improve write efficiency

Contributor list: (in alphabetical order) ​

For the past two weeks, our community has been super active with a total of 12 PRs from 3 contributors merged successfully and lots pending to be merged. Congrats on becoming our most active contributors in the past 2 weeks:

• @etolbakov (db#1179 db#1186)
• @lizhemingi (db#1167)
• @WenyXu (doc#257 db#1121 db#1159 db#1160 db#1161 db#1163 db#1174 db#1197 db#1198)

A big THANK YOU for the generous and brilliant contributions! It is people like you who are making GreptimeDB a great product. Let's build an even greater community together.

Highlights of Recent PR ​

Support executing PromQL queries through gRPC ​

Previously, PromQL queries could only be executed through the HTTP API /v1/promql. Now, we are able to execute PromQL queries through gRPC services, giving us the option of a more efficient query method for wider application scenarios.

Protocol updates are as follows:

message QueryRequest {
  oneof query {
    string sql = 1;
    bytes logical_plan = 2;
    PromRangeQuery prom_range_query = 3; // <-- new
  }
}

Graceful shutdown implemented for all components ​

Now, all the components of GreptimeDB, namely Frontend, Datanode, Metasrv, and Standalone, have incorporated the feature of graceful shutdown. By doing so, the system can guarantee the completion of ongoing tasks during the shutdown process, which mitigates the possibility of data loss and enhances the overall reliability of the system. This enhancement is bound to offer a more stable and secure experience.

Support for manually triggering Flush ​

We have introduced a manual trigger for flushing data to disk or remote storage, which is available at both the gRPC and HTTP layers. To execute this operation, you may use the curl utility. For instance:

# flush region 1 of table greptime.benchmark.tab

curl -XPOST 'localhost:4000/v1/admin/flush?catalog=greptime&db=benchmark&table=tab&region=1'# flush all regions of table greptime.benchmark.tab

curl -XPOST 'localhost:4000/v1/admin/flush?catalog=greptime&db=benchmark&table=tab'# flush all tables of table greptime.benchmark

curl -XPOST 'localhost:4000/v1/admin/flush?catalog=greptime&db=benchmark'

COPY TO statement with CONNECTION option ​

COPY TO SQL statement now supports the CONNECTION option, which means you can not only back up your data to local storage but also to cloud storage easily. This improvement provides users with more flexibility, making data backups more efficient, secure and reliable.

Adding a private gRPC protocol to improve write efficiency ​

To further enhance write performance, we have incorporated a private gRPC protocol alongside ArrowFlight. This protocol supports Unary Call and Client Streaming Call to boost the efficiency of data writing. Meanwhile, ArrowFlight remains dedicated to improving query performance.

New things ​

• We have been working on a GreptimeDB sink for vector, a powerful agent framework for collecting metrics and logs. We are going to upstream it when all code gets ready. Stay tuned.