108 lines
3.8 KiB
Markdown
108 lines
3.8 KiB
Markdown
---
|
|
stage: none
|
|
group: unassigned
|
|
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
|
|
---
|
|
|
|
# AI Architecture (Experiment)
|
|
|
|
GitLab has created a common set of tools to support our product groups and their utilization of AI. Our goals with this common architecture are:
|
|
|
|
1. Increase the velocity of feature teams by providing a set of high quality, ready to use tools
|
|
1. Ability to switch underlying technologies quickly and easily
|
|
|
|
AI is moving very quickly, and we need to be able to keep pace with changes in the area. We have built an [abstraction layer](../../ee/development/ai_features.md) to do this, allowing us to take a more "pluggable" approach to the underlying models, data stores, and other technologies.
|
|
|
|
The following diagram shows a simplified view of how the different components in GitLab interact. The abstraction layer helps avoid code duplication within the REST APIs within the `AI API` block.
|
|
|
|
```plantuml
|
|
@startuml
|
|
skin rose
|
|
|
|
package "Code Suggestions" {
|
|
node "Model Gateway"
|
|
node "Triton Inference Server" as Triton
|
|
}
|
|
|
|
package "Code Suggestions Models" as CSM {
|
|
node "codegen"
|
|
node "PaLM"
|
|
}
|
|
|
|
package "Suggested Reviewers" {
|
|
node "Model Gateway (SR)"
|
|
node "Extractor"
|
|
node "Serving Model"
|
|
}
|
|
|
|
package "AI API" as AIF {
|
|
node "OpenAI"
|
|
node "Vertex AI"
|
|
}
|
|
|
|
package GitLab {
|
|
node "Web IDE"
|
|
|
|
package "Web" {
|
|
node "REST API"
|
|
node "GraphQL"
|
|
}
|
|
|
|
package "Jobs" {
|
|
node "Sidekiq"
|
|
}
|
|
}
|
|
|
|
package Databases {
|
|
node "Vector Database"
|
|
node "PostgreSQL"
|
|
}
|
|
|
|
node "VSCode"
|
|
|
|
"Model Gateway" --> Triton
|
|
Triton --> CSM
|
|
GitLab --> Databases
|
|
VSCode --> "Model Gateway"
|
|
"Web IDE" --> "Model Gateway"
|
|
"Web IDE" --> "GraphQL"
|
|
"Web IDE" --> "REST API"
|
|
"Model Gateway" -[#blue]--> "REST API": user authorized?
|
|
|
|
"Sidekiq" --> AIF
|
|
Web --> AIF
|
|
|
|
"Model Gateway (SR)" --> "REST API"
|
|
"Model Gateway (SR)" --> "Serving Model"
|
|
"Extractor" --> "GraphQL"
|
|
"Sidekiq" --> "Model Gateway (SR)"
|
|
|
|
@enduml
|
|
```
|
|
|
|
## SaaS-based AI abstraction layer
|
|
|
|
GitLab currently operates a cloud-hosted AI architecture. We are exploring how self-managed instances integrate with it.
|
|
|
|
There are two primary reasons for this: the best AI models are cloud-based as they often depend on specialized hardware designed for this purpose, and operating self-managed infrastructure capable of AI at-scale and with appropriate performance is a significant undertaking. We are actively [tracking self-managed customers interested in AI](https://gitlab.com/gitlab-org/gitlab/-/issues/409183).
|
|
|
|
## Supported technologies
|
|
|
|
As part of the AI working group, we have been investigating various technologies and vetting them. Below is a list of the tools which have been reviewed and already approved for use within the GitLab application.
|
|
|
|
It is possible to utilize other models or technologies, however they will need to go through a review process prior to use. Use the [AI Project Proposal template](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=AI%20Project%20Proposal) as part of your idea and include the new tools required to support it.
|
|
|
|
### Models
|
|
|
|
The following models have been approved for use:
|
|
|
|
- [OpenAI models](https://platform.openai.com/docs/models)
|
|
- Google's [Vertex AI](https://cloud.google.com/vertex-ai) and [model garden](https://cloud.google.com/model-garden)
|
|
- [AI Code Suggestions](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/tree/main)
|
|
- [Suggested reviewer](https://gitlab.com/gitlab-org/modelops/applied-ml/applied-ml-updates/-/issues/10)
|
|
|
|
### Vector stores
|
|
|
|
The following vector stores have been approved for use:
|
|
|
|
- [`pgvector`](https://github.com/pgvector/pgvector) is a Postgres extension adding support for storing vector embeddings and calculating ANN (approximate nearest neighbor).
|