Assistants - Documentation

AI Assistants leverages ChatGPT and allows you to define different AI assistants with pre-defined tools and accesses to your data. This pre-defined access is expressed as GraphQL queries or mutations and/or webhooks and are unique to each AI assistant, meaning you can create different AI assistants with access to different type of data. In addition, AI assistants are fully integrated with Nhost permissions which means only data the user has access to will be made available to each AI assistant session.

Demo

To demonstrate how to configure and use AI Assistants we are going to be using a sample project with a database full of movies.

Creating AI Assistants

The first step is to create an AI assistant. You can create an AI assistant by utilizing the insertAssistant mutation or by using the dashboard. For instance:

mutation {
  graphite {
    insertAssistant(
      object: {
        name: "awesome-assistant-for-my-users"
        description: "My Awesome assistant for my users"
        instructions: "You are a useful assistant for this demo application. You are an expert on movies. You also know a bit about deep questions."
        model: "gpt-3.5-turbo-1106"
        graphql: [
          {
            description: "get movies with higher score than the input"
            name: "GetMoviesWithScoreHigherThan"
            query: "query ($score: numeric!) {  movies(where: {score: {_gt: $score}}) {    name    overview    score }}"
            arguments: [
              {
                name: "score"
                description: "score to compare against"
                type: "number"
                required: true
              }
            ]
          }
          {
            description: "search movies using natural language"
            name: "SearchMovies"
            query: "query GraphiteSearchMovies($query: String!) { graphiteSearchMovies(args: {query: $query, amount: 5}) { id name overview genre } }"
            arguments: [
              { name: "query", description: "Text to search", type: "string", required: true }
            ]
          }
          {
            description: "insert a movie into the database"
            name: "InsertMovie"
            query: "mutation InsertMovie($name: String, $overview: String, $genre: String, $crew: String, $budget: bigint, $revenue: bigint, $country: String, $score: numeric) { insertMovie(object: {name: $name, overview: $overview, score: $score, genre: $genre, crew: $crew, budget: $budget, revenue: $revenue, country: $country}) {id}}"
            arguments: [
              { name: "name", description: "Name of the movie", type: "string", required: true }
              {
                name: "overview"
                description: "Overview of the movie"
                type: "string"
                required: true
              }
              { name: "genre", description: "Genre of the movie", type: "string", required: true }
              { name: "crew", description: "Crew of the movie", type: "string", required: true }
              { name: "budget", description: "Budget of the movie", type: "number", required: true }
              {
                name: "revenue"
                description: "Revenue generated by the movie"
                type: "number"
                required: true
              }
              {
                name: "country"
                description: "Country of origin for the movie"
                type: "string"
                required: true
              }
              { name: "score", description: "Score of the movie", type: "number", required: true }
            ]
          }
        ]
        webhooks: [
          {
            name: "answer_deep_questions"
            description: "Answers deep questions about life, the universe and everything"
            URL: "https://local.functions.local.nhost.run/v1/meaning"
            arguments: [
              {
                name: "question"
                description: "question that needs answer"
                type: "string"
                required: true
              }
            ]
          }
        ]
      }
    ) {
      assistantID
    }
  }
}

Tools

In order to enhance our AI assistants we are leveraging tools. Tools can be either queries or mutations performed against your project’s GraphQL engine or they can also be any arbitrary endpoint you choose. Using the example above, we added 4 different tools to our AI Assistant:

The GraphQL query GetMoviesWithScoreHigherThan, which will allow the AI Assistant to query movies based on their score.
The GraphQL query SearchMovies, which leverages auto-embeddings and will allow the AI Assistant to query movies using natural language.
The GraphQL mutation InsertMovie, which will let the AI Assistant to insert new movies directly into the database.
The webhook answer_deep_questions, which is a serverless function that can respond to deep questions about life and the universe.

With a good name and description, the AI Assistant will know that certain information can be made available or that certain actions are at its disposal via these tools. We will see more in detail how this works a bit later but for now suffices to say that:

ChatGPT doesn’t actually query the data or perform any action, it tells graphite to execute the tools on its behalf and provide the result.
graphite will respect the user session when calling tools so permissions are respected.
GraphQL queries and mutations are performed against the project’s GraphQL endpoint
For webhooks you can specify any URL you want and graphite will perform a POST with a JSON object as body

Starting a session

To start a session you can use the mutation startSession and pass the id of the assistant you want to use. For instance, to start a session using the assistant we created in the previous section:

mutation {
  graphite {
    startSession(assistantID: "asst_xolmq2yeBBRi8CKGc5YWXGaA") {
      sessionID
    }
  }
}

You should receive a response similar to:

{
  "data": {
    "graphite": {
      "startSession": {
        "sessionID": "thread_dxHHNpQryFpJ8wXXD9Q2fQJg"
      }
    }
  }
}

Interacting with the Assistant

Now that you have started a session you can send your first message:

mutation {
  graphite {
    sendMessage(
      sessionID: "thread_dxHHNpQryFpJ8wXXD9Q2fQJg"
      message: "what is the meaning of life?"
      prevMessageID: ""
    ) {
      sessionID
      messages {
        id
        createdAt
        role
        message
      }
    }
  }
}

The field prevMessageID is used to only return messages after this one. As this is the beginning of the conversation we send an empty string to get all messages back. After executing that mutation we should receive a response like:

{
  "data": {
    "graphite": {
      "sendMessage": {
        "sessionID": "thread_dxHHNpQryFpJ8wXXD9Q2fQJg",
        "messages": [
          {
            "id": "msg_5LQa4sWH2QaN0xi5ezxHQUK1",
            "createdAt": "2023-12-08T09:33:12Z",
            "role": "user",
            "message": ""
          },
          {
            "id": "msg_Xv6XvZrTrgBOzviLeILkm7d3",
            "createdAt": "2023-12-08T09:33:36Z",
            "role": "user",
            "message": "what is the meaning of life?"
          },
          {
            "id": "msg_kTZonuiwZDxnipDADXk4bgFd",
            "createdAt": "2023-12-08T09:33:41Z",
            "role": "assistant",
            "message": "The answer to the meaning of life is 42. It's a deep philosophical concept from \"The Hitchhiker's Guide to the Galaxy\" by Douglas Adams."
          }
        ]
      }
    }
  }
}

Something to note is that ChatGPT detected that this was a “deep question” that could be answered with the webhook answer_deep_questions so this is roughly what happened:

We can continue the conversation by performing from mutations. Notice this time we can ask graphite to only return new message by specifying prevMessageID:

mutation {
  graphite {
    sendMessage(
      sessionID: "thread_dxHHNpQryFpJ8wXXD9Q2fQJg"
      message: "recommend a comedy movie for a rainy day with score higher than 80"
      prevMessageID: "msg_kTZonuiwZDxnipDADXk4bgFd"
    ) {
      sessionID
      messages {
        id
        createdAt
        role
        message
      }
    }
  }
}

Which should return something like:

{
  "data": {
    "graphite": {
      "sendMessage": {
        "sessionID": "thread_dxHHNpQryFpJ8wXXD9Q2fQJg",
        "messages": [
          {
            "id": "msg_xfq0nauRLEAAvelve2M48wlR",
            "createdAt": "2023-12-08T09:37:07Z",
            "role": "user",
            "message": "recommend a comedy movie for a rainy day with score higher than 80"
          },
          {
            "id": "msg_FbLWDnN9zPlsFtcQJYEpTnRY",
            "createdAt": "2023-12-08T09:37:22Z",
            "role": "assistant",
            "message": "Here are a few comedy movies with a score higher than 80 that you might enjoy on a rainy day:\n\n1. Saint Frances\n   Overview: A comedy-drama film that follows a struggling 34-year-old woman as she becomes the nanny for a six-year-old girl and navigates life's ups and downs.\n   Score: 86\n\n2. My Life as a Zucchini\n   Overview: After losing his mother, a young boy is sent to a foster home filled with other orphans, and he seeks to find love, laughter, and family amidst the challenges.\n   Score: 84\n\n3. Paddington\n   Overview: The movie follows the misadventures of a young bear from Peru who finds himself in London and is taken in by a London family.\n   Score: 86\n\n4. In the Heights\n   Overview: A musical-comedy drama set in the neighborhood of Washington Heights in New York City, where a bodega owner dreams of a better life.\n   Score: 83\n\n5. The Grand Budapest Hotel\n   Overview: This comedic adventure film takes place in a renowned European hotel and follows the escapades of its quirky characters, including a concierge and a lobby boy.\n   Score: 86\n\nThese movies offer humor and heartwarming stories that are perfect for a cozy movie night. Enjoy watching!"
          }
        ]
      }
    }
  }
}

In this case ChatGPT detected some of the information needed to formulate the answer could be retrieved with the GraphQL query GetMoviesWithScoreHigherThan so it requested graphite to run the query and return the data. This is what happened:

Implementing workflows

Interactions are not limited to just one tool or to simply query data. Tools can be combined and they can perform any action you wish. For instance, let’s take a look to the following session:

{
  "data": {
    "graphite": {
      "sendDevMessage": {
        "sessionID": "thread_kxgdq2wPDwdWETwGsC68psGw",
        "messages": [
          {
            "id": "msg_KvVwu3ypYgIdwJhOU57NYoFh",
            "role": "user",
            "createdAt": "2023-12-21T08:16:03Z",
            "message": ""
          },
          {
            "id": "msg_orBzaQTulaWtFLe2VhgjX0op",
            "role": "user",
            "createdAt": "2023-12-21T08:16:18Z",
            "message": "Are there any good comedies in space?"
          },
          {
            "id": "msg_WatLEqiP8IyZYkImBsgJPtm0",
            "role": "assistant",
            "createdAt": "2023-12-21T08:16:29Z",
            "message": "Here are some good comedies set in space:\n\n1. Linoleum - When the host of a failing children’s science show tries to fulfill his childhood dream of becoming an astronaut by building a rocket ship in his garage, a series of bizarre events occur that cause him to question his own reality.\n\n2. Spaceballs - When the nefarious Dark Helmet hatches a plan to snatch Princess Vespa and steal her planet's air, space-bum-for-hire Lone Starr and his clueless sidekick fly to the rescue. Along the way, they meet Yogurt, who puts Lone Starr wise to the power of \"The Schwartz.\" Can he master it in time to save the day?\n\nThese are great choices for a space comedy!"
          },
          {
            "id": "msg_Jq8iO0EXHtb2x9Z9CLFgxzhT",
            "role": "user",
            "createdAt": "2023-12-21T08:17:09Z",
            "message": "Spaceballs sounds like fun. Could you imagine a sequel?"
          },
          {
            "id": "msg_2landwp0CiGuzhTuyInUjRFN",
            "role": "assistant",
            "createdAt": "2023-12-21T08:17:15Z",
            "message": "Absolutely, a sequel to Spaceballs could be a lot of fun! In the original movie, the characters encounter the nefarious Dark Helmet and his diabolical plans. A sequel could explore new adventures, new villains, and wacky space escapades. It could also offer a fresh take on space-themed parodies and comedic twists. Fans of the original movie would surely be excited to see what new mischief Lone Starr, Princess Vespa, and the rest of the gang get into in a sequel."
          },
          {
            "id": "msg_hVjfo6H0uJu3kX43zNhFMizl",
            "role": "user",
            "createdAt": "2023-12-21T08:18:36Z",
            "message": "Now come up with a title and an overview and add it to the database and let me know the id returned"
          },
          {
            "id": "msg_B75oj7QFtR8pubkBdp1rAI4a",
            "role": "assistant",
            "createdAt": "2023-12-21T08:18:54Z",
            "message": "The movie \"Spaceballs 2: The Search for More Money\" has been successfully added to the database. The ID returned for the movie is \"1e2f9f95-34f0-4865-94fc-4f85bc93dd1a\". Get ready for a cosmic adventure that's twice the fun and twice the lunacy!"
          }
        ]
      }
    }
  }
}

In this session the following happened:

We asked about comedies in space, which prompted the AI Assistant to leverage tool SearchMovies, which leverages auto-embeddings to perform searches using natural language.
Then we asked the Assistant to imagine a sequel and insert it in the database, which prompted the AI assistant to leverage the tool InsertMovie to add it to the database.

True, to its word, the movie was added to the database successfully:

{
  "data": {
    "movie": {
      "id": "1e2f9f95-34f0-4865-94fc-4f85bc93dd1a",
      "name": "Spaceballs 2: The Search for More Money",
      "overview": "After successfully thwarting Dark Helmet's plans, Lone Starr and his crew find themselves in need of more cash. They embark on a hilarious and outrageous quest to find the elusive currency of the galaxy: spacebucks. Along the way, they encounter new foes, old friends, and a lot of absurdity. Get ready for a cosmic adventure that's twice the fun and twice the lunacy!",
      "genre": "Comedy, Science Fiction"
    }
  }
}

And remember, this is just a tiny example of what tools can do. Tools can do anything you want; from working with data in your project to booking flights or selling your products to your users.

User session

Interacting with graphite requires a user session (either an Authorization header or the admin secret). When interacting with the GraphQL endpoint or the webhook, graphite will re-use this session, which means that the webhook can use these headers to authorize the request. It also means that the GraphQL query will be performed as the user, so the user will have to have permissions to perform the query and will only get data the user has access to.

Other GraphQL queries/mutation

In addition to the queries/mutations shown in this document there are other mutations for listing, deleting and updating assistants, sessions, etc. We recommend you to check the GraphQL schema to see what’s available.

Metadata and Permissions

For each assistant and session managed by graphite, a row in the tables graphite.assistants and graphite.sessions will be made available. The purpose of this metadata is to allow you to set permissions as with any other object in your project. For details on which permissions are needed for each specific query and mutation refer to the GraphQL documentation in the reference section.

​Demo

​Creating AI Assistants

​Tools

​Starting a session

​Interacting with the Assistant

​Implementing workflows

​User session

​Other GraphQL queries/mutation

​Metadata and Permissions