Pirate Praveen 271ccb9124 New upstream version 16.0.7+ds1

2023-07-09 08:55:56 +05:30

3.8 KiB

Raw Permalink Blame History

stage	group	info
none	unassigned	To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments

AI Architecture (Experiment)

GitLab has created a common set of tools to support our product groups and their utilization of AI. Our goals with this common architecture are:

Increase the velocity of feature teams by providing a set of high quality, ready to use tools
Ability to switch underlying technologies quickly and easily

AI is moving very quickly, and we need to be able to keep pace with changes in the area. We have built an abstraction layer to do this, allowing us to take a more "pluggable" approach to the underlying models, data stores, and other technologies.

The following diagram shows a simplified view of how the different components in GitLab interact. The abstraction layer helps avoid code duplication within the REST APIs within the AI API block.

@startuml
skin rose

package "Code Suggestions" {
  node "Model Gateway"
  node "Triton Inference Server" as Triton
}

package "Code Suggestions Models"  as CSM {
  node "codegen"
  node "PaLM"
}

package "Suggested Reviewers" {
  node "Model Gateway (SR)"
  node "Extractor"
  node "Serving Model"
}

package "AI API" as AIF {
  node "OpenAI"
  node "Vertex AI"
}

package GitLab { 
  node "Web IDE"

  package "Web" {
    node "REST API"
    node "GraphQL"
  }

  package "Jobs" {
    node "Sidekiq"
  }
}

package Databases {
  node "Vector Database"
  node "PostgreSQL"
}

node "VSCode"

"Model Gateway" --> Triton
Triton --> CSM
GitLab --> Databases
VSCode --> "Model Gateway"
"Web IDE" --> "Model Gateway"
"Web IDE" --> "GraphQL"
"Web IDE" --> "REST API"
"Model Gateway" -[#blue]--> "REST API": user authorized?

"Sidekiq" --> AIF
Web --> AIF

"Model Gateway (SR)" --> "REST API"
"Model Gateway (SR)" --> "Serving Model"
"Extractor" --> "GraphQL"
"Sidekiq" --> "Model Gateway (SR)"

@enduml

SaaS-based AI abstraction layer

GitLab currently operates a cloud-hosted AI architecture. We are exploring how self-managed instances integrate with it.

There are two primary reasons for this: the best AI models are cloud-based as they often depend on specialized hardware designed for this purpose, and operating self-managed infrastructure capable of AI at-scale and with appropriate performance is a significant undertaking. We are actively tracking self-managed customers interested in AI.

Supported technologies

As part of the AI working group, we have been investigating various technologies and vetting them. Below is a list of the tools which have been reviewed and already approved for use within the GitLab application.

It is possible to utilize other models or technologies, however they will need to go through a review process prior to use. Use the AI Project Proposal template as part of your idea and include the new tools required to support it.

Models

The following models have been approved for use:

Vector stores

The following vector stores have been approved for use:

pgvector is a Postgres extension adding support for storing vector embeddings and calculating ANN (approximate nearest neighbor).

3.8 KiB Raw Permalink Blame History