# Feature AI-3 - Add LLM connection settings to admin AI configuration

Project: Stream AI
Status: in test
URL: https://community.openproject.org/work_packages/AI-3

## Planning

| Attribute | Value |
| --- | --- |
| Date |  – 2026-10-23 |
| Priority | Medium |
| Target versions | 18.0.0 |
| List |  |
| Roadmap |  |
| Relevance |  |

## OPEN POINTS / FEEDBACK

_No work packages_

## Features and Chunks

| ID | Subject | Type | Status | Assignee | Target versions | Spent time |
| --- | --- | --- | --- | --- | --- | --- |
| AI-146 | Gate the LLM admin pages to the Professional plan | Implementation | rejected | Andreas Eiselt | 18.0.0 |  |

## Bugs and other work packages

| ID | Type | Subject | Status | Assignee | Target versions | Spent time | Module |
| --- | --- | --- | --- | --- | --- | --- | --- |
| AI-148 | Code maintenance | Move LLM model writes into create, update and delete services | merged |  | 18.0.0 |  |  |
| AI-147 | Code maintenance | Extract the LLM disconnect into LlmConnections::DisconnectService | merged |  | 18.0.0 |  |  |
| AI-145 | Code maintenance | Route LLM inference traffic through the SSRF-filtered httpx client | merged |  | 18.0.0 |  |  |
| AI-221 | Code maintenance | Remove the actions menu from the LLM connection page | needs review | Andreas Eiselt | 18.0.0 |  |  |
| AI-144 | Code maintenance | Make the LLM request timeouts configurable and cap the response size | rejected | Andreas Eiselt |  |  |  |

## Technical Details

| Attribute | Value |
| --- | --- |
| Module |  |
| Feature Flag |  |
| Requires doc change | No |
| Enterprise plan | Enterprise (ENT) |

## Details

| Attribute | Value |
| --- | --- |
| Pull preview |  |
| Mockups | https://www.figma.com/design/8GwgqK4ejWnYJUHEM81as0/Artificial-Intelligence--AI-?node-id=267-13697&t=zz15bCdKaqQncTZp-4 |

## People

| Attribute | Value |
| --- | --- |
| Assignee | Ihor Khomenko |
| Accountable | Dominic Bräunlein |
| Product manager |  |
| Developers |  |
| Designer | Parimal Satyal |
| Ordered by |  |
| Signed off by |  |
| Interest by |  |

## Estimates and progress

| Attribute | Value |
| --- | --- |
| Work |  |
| Labor costs |  |

## Description

**As** an OpenProject admin
**I want to** connect OpenProject to an LLM server and decide which model each AI feature uses
**so that** AI features can run against infrastructure I control

**Acceptance criteria**

  - In the admin settings "Artificial intelligence (AI)" two new menu entries are available: "LLM settings" and "AI models". Both are visible only to administrators and only while the `llm_connection` feature flag is active; without the flag the pages return 404.
  - The "LLM settings" view contains
      - Displayed information: "Connect OpenProject to a server that speaks the OpenAI API, so AI features can use it."
      - Boolean: "Enable LLMs for this instance"
          - When turned off, all LLM-backed AI features stop working; the connection settings are kept.
          - Cannot be turned on before a connection has been configured.
      - Select: "API format"
          - Offers only the formats OpenProject can address with a base URL and a single API key. "OpenAI-compatible" is the default and covers most gateways and self-hosted servers; providers such as Anthropic, Azure OpenAI, Google Gemini, Mistral and Ollama are selectable as well.
      - URL input: "Host URL" (placeholder: `https://example.com/v1`)
          - The full base URL including the API version segment, exactly as the provider documents it. OpenProject appends only the endpoint path.
      - Password input: "API key"
          - Write-only: the stored key is never sent back to the browser. With a key stored, the caption reads "A key is stored. Leave blank to keep it, or enter a new one to replace it."
          - Optional, since self-hosted servers may require no authentication.
      - Autocompleter: "Default chat model" (used by AI features that do not select a model themselves; an autocompleter rather than a select because a gateway can report hundreds of models)
      - Autocompleter: "Default embedding model" (shown only when an embedding feature is registered; offers only models known to create embeddings)
          - Only models the server offers can be chosen as defaults; a model known to be an embedding model cannot become the default chat model.
      - "Connect" button
          - When clicked, the settings are validated by contacting the server: `GET {base_url}/models`. While the request runs, the button shows "Contacting the LLM server…".
          - The request fires only when "Host URL" or "API key" changed; re-saving unchanged settings sends no request.
          - Before any request, the host is resolved and checked against the instance-wide SSRF filter. A blocked address shows an inline error underneath "Host URL" that names the allowlist environment variable.
          - On error, nothing is saved, and the errors are shown inline: unreachable host, timeout and "no valid model list" (wrong endpoint) underneath "Host URL"; a rejected key underneath "API key". Unexpected errors are shown as a toast.
          - On success, the settings are saved, the model list is stored and a success message is shown. When the server answers but publishes no model list, the settings are still saved, with a hint to add models manually and run the health checks.
      - An action menu with two danger dialogs
          - "Remove API key": OpenProject stops sending a key with its requests. The dialog warns that the next model refresh discards what is known about each model's capabilities, including assertions made by administrators.
          - "Disconnect": OpenProject stops using the LLM server. The dialog states what is kept (endpoint and API format, the model list including manual entries, the model chosen for each feature) and that only the stored API key is removed.
  - "Available models" panel on the "LLM settings" view
      - Lists every model with its id and optional display name, type (Chat, Embedding, Unknown), context window, source ("Reported by server", "Added manually", "No longer reported", "Hidden by administrator") and an "Available for use" toggle.
      - A filter input and a "Refresh models" button. Models the server no longer reports are marked "No longer reported" rather than deleted; manually added models are kept across refreshes.
      - An "Add model" form for servers that publish no model list; the name is entered exactly as the server expects it. Manual models are removed via a danger dialog that warns when AI features currently use the model.
      - Every model can be edited: display name, context window override (the known value and where it comes from are shown) and per-capability assertions (Embeddings, Tool calling, Structured output, Vision, Reasoning) with the states "Supported", "Not supported" and "Not specified". Marking Embeddings as supported makes the model an embedding model.
      - Capabilities are otherwise determined automatically: from the model registry where published, and by probing `POST {base_url}/embeddings` for likely embedding models in a background job. Each value is stored as a verdict with its source (registry, probe, administrator, observed) and shown as "Supported", "Not supported" or "Not verified". Administrator assertions are sticky across refreshes.
      - Pointing the connection at a different deployment (changed endpoint or key) discards the verdicts and deactivates the discovered models, so knowledge about one server is never carried over to another.
  - The "AI models" view contains
      - One row per registered AI feature (currently "Description assistant" and "Semantic search"), each with a caption explaining the feature.
      - Each feature either follows "Use the default (…)" or selects a specific model.
      - A model missing a required capability is disabled with the reason shown; an unverified model stays selectable and is marked "not verified". Only "Not supported" blocks, "Not verified" never does.
      - Saving warns instead of blocking: when the server reports the chosen model incapable, the choice is kept and a warning names the missing capability, with the option to assert the capability on the model.
      - Embedding features additionally configure vector dimensions, input prefix and query prefix; when a probe has returned the dimension count, it is shown as a hint.
      - Semantic search is pinned once data has been indexed: the model cannot be changed here ("Re-index to switch models") and the values fixed by the existing index are displayed.
      - Dangling states are surfaced: a model the server no longer offers ("This feature will not run until another model is selected") and a model hidden by an administrator (the feature keeps using it, but it can no longer be chosen elsewhere).
      - A blank state points to the "LLM settings" view while no connection is configured.
  - Health status
      - A side panel on the "LLM settings" view summarises the latest check; a "Health status" subpage shows the full report with "Run checks" and "Download report".
      - "Run checks" executes grouped checks: Configuration (feature flag, connection switched on, endpoint configured, API key stored, endpoint format usable), Server (reachable, accepts the API key), Models (list present and fresh, defaults still offered), Inference (a minimal chat completion round trip proves the server answers) and AI features (every feature resolves to a usable model, indexed data still has its model).
      - Every failure has its own translated message (unreachable, timeout, rejected key, blocked address, missing or stale model list, empty completion, features unbound or incapable). Old reports are pruned automatically.
  - Configuration through the environment
      - The connection can be provisioned via `OPENPROJECT_LLM__CONNECTION_*` variables (`base_url`, `api_key`, `default_chat_model`, `default_embedding_model`, `enabled`) during `db:seed`.
      - An environment-configured connection renders the page read-only with an explanatory banner; the dialogs refuse changes.
      - Unknown keys abort seeding with a message explaining the underscore encoding and listing the accepted variables.
      - Seeding succeeds while the LLM server is unreachable, since a container may boot before its inference sidecar.

**Technical notes**

  - Persistence in four tables: `llm_connections` (singleton enforced by validation, schema ready for several), `llm_models`, `llm_capability_verdicts`, `llm_feature_bindings`. The API key is stored ciphered.
  - Model references are plain strings, never foreign keys, so a feature binding survives its model disappearing from the server. Dangling bindings are derived at read time; at call time the feature fails closed with a machine-readable reason instead of silently substituting another model.
  - The client layer builds on the `ruby_llm` gem; the API format list is its provider registry filtered to formats usable with base URL plus key. Registry metadata supplies context windows and capability hints.
  - AI features register themselves in a static registry (kind, required and preferred capabilities, overridable, pinned) from initializers, so modules can contribute their own features.
  - Consumers resolve their model in one place: per-item override, then feature binding, then connection default, otherwise unbound with a reason. This resolver is the contract for the text transform API.
  - Background jobs cover model sync (with retry), capability detection, the health check and health report pruning.
  - Request failures are logged with error class and status only, never with URL, key or payloads.

**Permissions and visibility considerations**

  - Only administrators can see or edit the AI settings pages. No new permission is introduced.

**Translation considerations**

  - LLM
  - Model ids and display names are never translated; capability and verdict labels are.

**Out of scope**

  - Enable AI features per project
  - Enable AI features for specific users
  - Usage counter and budget limits
  - Enterprise gating (deferred to <mention class="mention" data-id="77823" data-type="work_package" data-text="#77823">#77823</mention>)
  - More than one connection at a time (the schema is prepared)
  - Streaming responses and scheduled re-detection of capabilities
  - API formats that need credentials beyond a single key (AWS Bedrock, Google Vertex AI)