Introducing the Vector Data Type and Loki Remote Write Support | Greptime Biweekly Report

Summary ​

Together with our global community of contributors, GreptimeDB continues to evolve and flourish as a growing open-source project. We are grateful to each and every one of you.

Below are the highlights among recent commits:

• Inverted Index column can be specified without being a part of primary key.

• More table options can be altered.

• A new "Vector" datatype is added.

• Support Loki remote write.

Contributors ​

For the past two weeks, our community has been super active with a total of 96 PRs merged. Of these, 7 PRs were contributed by 3 individual contributors, with many more pending to be merged.

• @lyang24 (db#5003 db#4965 db#4792)

• @linyihai（db#4996db#4966db#4951）

• @Kev1n8（db#4881）

👏 Welcome @linyihai to the community as a new contributor with a successfully merged PR, and more PRs from other individual contributors are waiting to be merged.

A big THANK YOU to all our members and contributors! It is people like you who are making GreptimeDB a great product. Let's build an even greater community together.

Highlights of Recent PRs ​

db#4964 Introduce Vector datatype ​

VECTOR(n) means an n-dimensional vector. For example, creating a 3-dimensional vector in a table:

CREATE TABLE t (ts TIMESTAMP TIME INDEX, v VECTOR(3));
INSERT INTO t VALUES (1, "[1.0, 2.0, 3.0]");

db#4929 Inverted index columns made explicit ​

Inverted index columns can be set in the create table SQL. For example:

CREATE TABLE t (
  "ts" TIMESTAMP,
  "host" STRING,
  "job" STRING,
  "val" DOUBLE,
  TIME INDEX("ts"),
  PRIMARY KEY("host", "job"),
  INVERTED INDEX("job", "val")
);

db#4952 db#4965 db#4983 Alter table options enhanced ​

1. Fulltext column option can be altered. For example:

-- Set fulltext for a column:
ALTER TABLE monitor SET COLUMN s FULLTEXT WITH (analyzer = 'Chinese', case_sensitive = 'false');
-- Unset(disable) fulltext for a column:
ALTER TABLE monitor COLUMN s UNSET FULLTEXT;

2. TWCS compaction options can be altered. For example:

ALTER TABLE ato SET 'compaction.twcs.time_window'='2h';

Know more：https://docs.greptime.com/user-guide/administration/manage-data/compaction/#time-windowed-compaction-strategy-twcs

db#4987 More distance functions are added ​

The 3 new distance functions are: cos_distance, l2sq_distance and dot_product.

db#4792 Support setting execution timeout for SQL statements ​

For MySQL client, it's SET MAX_EXECUTION_TIME = <milliseconds>; for PostgreSQL client, it's SET statement_timeout TO <milliseconds>.

db#5006 Support setting timeout for each HTTP request ​

The per request timeout is set via an HTTP header x-greptime-timeout.

db#4941 Support Loki remote write ​

Now GreptimeDB has a new HTTP endpoint for Loki remote write:

loki.write "local_loki" {
    endpoint {
        url = "http://localhost:4000/v1/loki/api/v1/push"
    }
}

db#4925 Add JSON path processor for Log Pipeline ​

In pipeline, we can now use JSON path to conveniently process complex JSON objects.

db#4960 Windowed sort optimization improvement ​

Support filter in windowed sort optimization, boosting query performance.

db#4928 Boosting read performance for index files (puffin) ​

Make the behavior of reads of index files (puffin) more aligned with OpenDAL, gaining more performant reads.

Good First Issue ​

db#4297 Supports auto as values for all numeric configurations with default settings ​

• Keywords: Config

• Difficulty: Medium

db#4957 Add a timeout setting in get_schema_options_by_table_id ​

• Keywords: Config

• Difficulty: Simple

About Greptime ​

Greptime offers industry-leading time series database products and solutions to empower IoT and Observability scenarios, enabling enterprises to uncover valuable insights from their data with less time, complexity, and cost.

GreptimeDB is an open-source, high-performance time-series database offering unified storage and analysis for metrics, logs, and events. Try it out instantly with GreptimeCloud, a fully-managed DBaaS solution—no deployment needed!

The Edge-Cloud Integrated Solution combines multimodal edge databases with cloud-based GreptimeDB to optimize IoT edge scenarios, cutting costs while boosting data performance.

Star us on GitHub or join GreptimeDB Community on Slack to get connected.