โœ•

Join us at two must-attend conferences this summer and claim your exclusive SWAG! ๐ŸŽ

IOT TECH EXPO, June 5-6 in Santa Clara! Booth 270 | DATA + AI SUMMIT, June 11-13 in San Francisco! Booth e12

โœ•
Skip to content
On this page

Subscribe to our newsletter

Get the latest dates and news about GreptimeDB.
Engineering
โ€ข
January 12, 2024

Streamline your OpenAI Monitoring Experience with GreptimeAI

GreptimeAI provides comprehensive insights into OpenAI's usage in terms of cost, performance, traffic, and security. Read this article to learn how to setup GreptimeAI to better manage your OpenAI usage.

With the rapid advancement of artificial intelligence technology, OpenAI has emerged as one of the leaders in the field. It excels in various language processing tasks, including machine translation, text classification, and text generation.

However, the critical role of continuous monitoring of API calls while using OpenAI should not be underestimated. This practice is crucial not only for identifying performance bottlenecks and analyzing usage patterns but also for swiftly detecting and addressing any issues that arise with the API.

GreptimeAI โ€‹

GreptimeAI offers a tailor-made observability solution specifically designed for monitoring and managing large language model (LLM) applications. This solution provides comprehensive insights into the cost, performance, traffic, and security aspects of OpenAI usage. For more details about GreptimeAI, please refer to this article. Notably, GreptimeAI is built upon the open-source time-series database, GreptimeDB.

OpenAI Modules being Monitored โ€‹

  • chat
  • completion
  • audio
  • images

Scenarios Supported โ€‹

  • async
  • stream
  • with_raw_response
  • retry
  • error

User Guide โ€‹

Installation โ€‹

python
pip install --upgrade greptimeai

Registration โ€‹

To get started, create a service by registering greptimeai, and get:

  • host
  • database
  • token

Setting up โ€‹

Shell
export GREPTIMEAI_HOST='xxx'
export GREPTIMEAI_DATABASE='xxx'
export GREPTIMEAI_TOKEN='xxx'

Example โ€‹

Here is a simple example to illustrate how to call OpenAI chat completion with GreptimeAI tracking enabled.

python
from greptimeai import openai_patcher
from openai import OpenAI

client = OpenAI()
openai_patcher.setup(client=client)

completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "How do I output all files in a directory using Python?",
        }
    ],
    model="gpt-4",
    user="<user_id_from_your_application>",
    stream=True,
)

How does it look like in GreptimeAI โ€‹

Dashboard overview:

dashboard overview

The following graph shows the trace detail with multiple spans.

trace details

Join our community

Get the latest updates and discuss with other users.

Subscribe to our newsletter

Get the latest dates and news about GreptimeDB.