> For the complete documentation index, see [llms.txt](https://docs.labs.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.labs.ai/protocols-and-integration/open-webui-integration.md).

# OpenAI-Compatible API (Open WebUI)

EDDI can present its deployed agents as **OpenAI models**. Any client that speaks the OpenAI Chat Completions protocol — [Open WebUI](https://github.com/open-webui/open-webui), the Python `openai` SDK, LangChain, LiteLLM, Continue, Cursor — can drive an EDDI conversation without knowing anything about EDDI.

The adapter lives at `/v1` and is **disabled by default**.

> **Only want a chat front end for your agents? Use Open WebUI.** If your users need a generic, ChatGPT-style chat with EDDI agents behind it, Open WebUI over `/v1` does that well today, and EDDI does not try to out-build it on generic chat. What the OpenAI protocol cannot carry is EDDI's own interaction model, so over `/v1` your users go without:
>
> * **Approvals in the flow.** A paused turn arrives as a text notice; nobody can approve or reject in the chat, it happens out of band ([HITL](#hitl)).
> * **Quick replies and `inputField`.** They are flattened to Markdown to read and retype, not buttons or a form ([§7.2](#72-structured-outputs)).
> * **Secret input.** There is no masked field, so an agent asking for a password gets it in the clear in the client's history.
> * **Ratings and review notices.** No thumbs or rating card, and no "replies are reviewed" notice from the agent's profile.
> * **Group discussions.** Teams of agents are not models ([§10](#10-known-gaps)).
> * **Memory transparency.** No view of what an agent remembers about the user, and no way to delete it there.
> * **Branding.** Open WebUI shows its own brand, not the deployment's white-label name, logo and colours, and no per-agent appearance.
>
> Those are what EDDI's own surfaces exist for: the embeddable [Chat UI](https://docs.labs.ai/protocols-and-integration/chat-ui-embedding.md) at `/chat`, and the Manager and Workforce.

***

## Contents

1. [Quick start](#1-quick-start)
2. [How it works](#2-how-it-works)
3. [Configuration](#3-configuration)
4. [Security](#4-security)
5. [Open WebUI settings that matter](#5-open-webui-settings-that-matter)
6. [What is supported](#6-what-is-supported)
7. [Behaviour to expect](#7-behaviour-to-expect)
8. [Errors](#8-errors)
9. [Troubleshooting](#9-troubleshooting)
10. [Known gaps](#10-known-gaps)
11. [Recipe: EDDI as a thin LLM gateway](#11-recipe-eddi-as-a-thin-llm-gateway)

***

## 1. Quick start

### Runnable demo (one command)

```bash
docker compose -f docker-compose.openwebui.yml up --build
```

Brings up MongoDB, EDDI, Open WebUI on <http://localhost:3000>, and a one-shot seeder that creates and deploys a small rule-based demo agent so the model dropdown is not empty. Pick `eddi-demo-agent-…` and start typing.

Two things worth knowing about it:

* **EDDI is built from the working tree**, not pulled from Docker Hub, so the demo always runs the adapter as it is in your checkout, including changes that have not reached a published image. The build happens inside the container, so no local JDK or Maven is needed. The first build takes a few minutes; later ones are cached.
* **The demo agent has no LLM.** It needs no provider credentials and its replies are deterministic — but it also *cannot answer questions about anything*, including an uploaded PDF. To get an agent that actually thinks, set `EDDI_DEMO_LLM_API_KEY` and a second LLM-backed agent is deployed alongside it:

  ```bash
  EDDI_DEMO_LLM_API_KEY=sk-... docker compose -f docker-compose.openwebui.yml up --build
  ```

  `EDDI_DEMO_LLM_TYPE` (default `openai`) and `EDDI_DEMO_LLM_MODEL` (default `gpt-4o-mini`) select the provider and model — see [`langchain.md`](https://docs.labs.ai/agent-configuration/langchain.md) for the supported types. Its system prompt references `{context.openai_system_message}`, so with `RAG_SYSTEM_CONTEXT=true` it can answer about files you upload.

  **The key goes into EDDI's Secrets Vault, not into the agent config.** The seeder stores it via `PUT /secretstore/secrets/default/demo-llm-api-key` and the config holds `${vault:demo-llm-api-key}`. That is the point of the vault: an agent config gets exported, diffed, rendered in the Manager UI and logged, and a literal key would travel with all of it. Verified on a clean run — the plaintext key appears in no MongoDB collection, and `GET /secretstore/secrets/default` returns metadata and a checksum, never the value.
* **The rule-based agent** runs a short three-turn flow — it asks your name, remembers it, and refers back to it — because that is the thing this adapter exists to bridge: conversation state surviving a stateless HTTP protocol. Open a second chat and it asks again, which is the per-chat isolation.
* **Open WebUI's auxiliary generation is turned off** in the demo (titles, tags, follow-ups, search queries). Left on, those extra LLM calls go to the selected model, so utility prompts land in your real conversation and advance its behaviour rules — and the unparseable replies surface as bogus follow-up suggestions. In a real deployment point them at a separate connection or an `…:stateless` model instead of disabling them (§5).

It is **not a production configuration** — `WEBUI_AUTH=false` and `allow-anonymous=true` are set so the UI is usable immediately. Both are called out inline in the compose file, and §4 has the real security model.

#### Running it more than once

The seeder is idempotent **per agent**, not "has anything been deployed". Each run creates only what is missing, so the normal second run — you now have a provider key and want the LLM agent too — just works:

```bash
EDDI_DEMO_LLM_API_KEY=sk-... docker compose -f docker-compose.openwebui.yml up -d
```

It reports which agents it skipped, adds the LLM one, and re-stores the key in the vault (so changing the key here rotates it rather than leaving the old one). `--build` is only needed the first time or after changing EDDI's source.

#### If a port is already taken

`EDDI_PORT` and `OPEN_WEBUI_PORT` remap the **host** side; the containers keep talking to each other on `7070`/`8080` regardless:

```bash
EDDI_PORT=7071 OPEN_WEBUI_PORT=3001 docker compose -f docker-compose.openwebui.yml up -d
```

Without this, Docker fails with `Bind for 0.0.0.0:7070 failed: port is already allocated` — most often because another EDDI (or the main `docker-compose.yml`) is already running.

#### Shutting it down

```bash
docker compose -f docker-compose.openwebui.yml down
```

That keeps the two named volumes, so the seeded agents, the vault entry and your chat history survive and the next start needs no re-seeding. Add `-v` to discard them — that is the one irreversible step.

### Production-shaped compose

```yaml
services:
  eddi:
    image: labsai/eddi:latest
    ports: ["7070:7070"]
    environment:
      EDDI_OPENAI_COMPAT_ENABLED: "true"
      EDDI_OPENAI_COMPAT_API_KEY: "sk-eddi-change-me"
    depends_on: [mongodb]

  mongodb:
    image: mongo:7

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    ports: ["3000:8080"]
    environment:
      OPENAI_API_BASE_URL: http://eddi:7070/v1
      OPENAI_API_KEY: "sk-eddi-change-me"
      ENABLE_OLLAMA_API: "false"
      # REQUIRED — see §5. Without it EDDI cannot tell Open WebUI's users apart,
      # and maps its chats only by their opening messages.
      ENABLE_FORWARD_USER_INFO_HEADERS: "true"
```

Deploy an agent in EDDI, open <http://localhost:3000>, and it appears in the model dropdown.

Verify from the shell, with `EDDI_API_KEY` set to the same value as in the compose file:

```bash
curl -H "Authorization: Bearer $EDDI_API_KEY" http://localhost:7070/v1/models
```

```bash
curl -X POST http://localhost:7070/v1/chat/completions -H "Authorization: Bearer $EDDI_API_KEY" -H "Content-Type: application/json" -H "X-OpenWebUI-Chat-Id: chat-1" -H "X-OpenWebUI-User-Id: alice" -d '{"model":"my-agent-a3f9c1","messages":[{"role":"user","content":"Hello"}]}'
```

And from Python:

```python
import os
from openai import OpenAI

client = OpenAI(base_url="http://localhost:7070/v1", api_key=os.environ["EDDI_API_KEY"])
print([m.id for m in client.models.list().data])

for chunk in client.chat.completions.create(
        model="my-agent-a3f9c1",
        messages=[{"role": "user", "content": "Hi"}],
        stream=True):
    print(chunk.choices[0].delta.content or "", end="", flush=True)
```

***

## 2. How it works

### The problem

The OpenAI API is **stateless**: the client resends the full `messages[]` array every turn and the server keeps nothing. EDDI is **stateful**: a conversation accumulates memory, properties, behavior-rule position and HITL bookmarks. The adapter bridges the two.

### Model names

Each deployed agent is exposed under two ids:

| Model id                            | Behaviour                                                                                                                        |
| ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `customer-support-a3f9c1`           | **Stateful (default).** Maps to a persistent EDDI conversation. Memory, properties, rule position and HITL survive across turns. |
| `customer-support-a3f9c1:stateless` | **Stateless.** Starts a conversation, sends one message, ends it. No conversation memory (see below).                            |

The format is `<slugified agent name>-<last 6 characters of the agent id>`. The suffix is there because agent names are not unique — two agents both called "Support" would otherwise be indistinguishable. Accented characters are folded (`Übersicht` → `ubersicht`) rather than dropped.

In the rare case that two deployed agents share both the name slug and the last six id characters, each of them is listed as `<slugified agent name>-<full agent id>` instead, so both stay selectable. The short id still works as an alias when the caller may use exactly one of the two; when both are usable it is rejected as ambiguous rather than guessed. Agents that do not collide keep their short id.

The adapter also accepts the bare agent id, the exact agent name, or the bare slug — the last two only when they match exactly one agent. An ambiguous name returns `400` listing the candidates rather than picking one.

### Stateless requests

Both routes select the same behaviour — a throwaway conversation, ended as soon as the turn completes, with no entry in the managed-conversation store:

```jsonc
// 1. the model suffix — the only route a UI like Open WebUI can express,
//    since a model name is all it lets you pick per request
{"model": "customer-support-a3f9c1:stateless", "messages": [...]}

// 2. the `stateless` body field — an EDDI extension, for callers that can
//    say what they mean
{"model": "customer-support-a3f9c1", "messages": [...], "stateless": true}
```

```python
client.chat.completions.create(
    model="customer-support-a3f9c1",
    messages=[{"role": "user", "content": "Classify this ticket"}],
    extra_body={"stateless": True},
)
```

The suffix follows the ecosystem convention for behavioural model variants — OpenRouter's `:nitro`/`:floor`/`:free`, Ollama's `llama3:8b` — and has the advantage of appearing in `GET /v1/models`, so the capability is discoverable. The body field is the honest parameter for programmatic callers. **They are OR-ed**: `model: "x:stateless"` together with `stateless: false` is self-contradictory, and of the two readings, running stateless merely loses continuity while running stateful would persist a conversation the caller may not have wanted.

Setting `expose-stateless-variants=false` disables **both** routes, so the switch cannot be circumvented by moving the request from the model id into the body.

Two things stateless does **not** mean:

* **Not "no memory at all".** User-scoped long-term memory is keyed by `userId`, not by conversation, so an agent with user memory enabled still loads that user's entries at init and still persists `longTerm` properties at teardown. A thin LLM-only agent has neither, so nothing happens.
* **Not "client-managed history".** The adapter sends only the last user message in *both* modes. A client that resends its own history each turn gets no context from the earlier entries. Stateless is genuinely single-turn.

It is also only sensible for LLM-only agents: a rule-based agent under `:stateless` restarts at `CONVERSATION_START` on every request, which is correct but useless for a wizard.

### Session mapping

```
intent = channel:openai:<agentId>:<chatKey>
userId = the authenticated caller
```

`chatKey` is the first of these the request carries:

| Source                                                                  | Sent by                                                                                                                   |
| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `X-OpenWebUI-Chat-Id` header                                            | Open WebUI, with `ENABLE_FORWARD_USER_INFO_HEADERS` (§5)                                                                  |
| `X-EDDI-Chat-Id` header                                                 | any client that can set a header — the generic form                                                                       |
| `metadata.chat_id` body field                                           | the OpenAI SDKs: `metadata={"chat_id": "…"}` (or `extra_body`)                                                            |
| `user` body field                                                       | clients that set the OpenAI `user` field; it then names the *chat*, so all of that user's requests share one conversation |
| *derived:* `h:` + a hash of the first `system` and first `user` message | everything else — the OpenAI SDKs, LangChain, LiteLLM by default                                                          |

`(intent, userId)` is the primary key in EDDI's managed-conversation store, so:

* **the same chat** → the same EDDI conversation, turn after turn;
* **a different chat window** → a different conversation, with no shared memory;
* **a different user** → a different conversation.

**Clients that name no chat.** An OpenAI client resends the whole history every turn, so a chat's first system and first user message are the same on every one of its turns and differ between chats that open differently. Hashing them gives each client-side chat its own conversation with no client changes. Two rules keep this honest:

* **An opening turn always starts fresh.** A derived-key request that carries no history (no assistant message, one user message) is the first turn of a chat, so it never continues an earlier chat that happened to open with the same words ("Hi"). The earlier conversation is abandoned, not ended.
* **Editing the first message starts a new conversation**, because the chat's identity changed. For a stable key across edits, send `X-EDDI-Chat-Id` or `metadata.chat_id`.

Two chats of one user that open identically can still collide: the second chat's opening turn takes the key over, so the first chat's later turns continue in the second chat's conversation. This never crosses users or agents. It is inherent to keying by the opening — nothing in the request tells the two chats apart until their histories diverge, and the first point of divergence is generated text a client may reformat before resending — so name such chats with an explicit key. That includes **few-shot prompts**: when every chat opens with the same example exchange, every chat has the same derived key. Before this, every request without a chat id went to one `:default` conversation per user and agent, so unrelated chats shared context and one document grew without bound; `eddi.openai-compat.chat-key-fallback=shared` restores that behaviour for clients that rely on it (for example one that sends only its latest message and relies on EDDI's memory — such a client gets a fresh conversation every turn under `history`). A key longer than 128 characters is stored as its SHA-256.

**Bounding a long chat.** The LLM sees only the window the agent's memory settings allow (see [Conversation memory](https://docs.labs.ai/architecture-and-concepts/conversation-memory.md)), but the stored conversation keeps every step, and a MongoDB document cannot exceed 16 MB. `eddi.openai-compat.max-conversation-steps` (off by default) ends an idle mapped conversation once it has that many steps and continues the chat in a fresh one; the ended conversation stays readable, but the agent no longer sees its history and conversation-scoped properties do not carry over (long-term properties live in user memory and do).

Only the **last user message** is sent to the agent. The rest of `messages[]` is discarded: EDDI has its own memory, so replaying the client's history would double every turn.

### A turn, end to end

```
Open WebUI                       EDDI adapter                        EDDI core
    │ POST /v1/chat/completions
    │   Authorization: Bearer …
    │   X-OpenWebUI-Chat-Id: 4f2b…
    │   X-OpenWebUI-User-Id: u_812
    ├───────────────────────────────►│
    │                                │ authenticate, resolve model → agentId
    │                                │ intent = channel:openai:<agentId>:4f2b…
    │                                │ look up (intent, userId)
    │                                ├─ start or reuse conversation ─────────►│
    │                                ├─ say / sayStreaming ──────────────────►│
    │◄─ data: {"delta":{"role":"assistant"}}                                  │
    │◄─ data: {"delta":{"content":"Hel"}}     ◄── onToken                     │
    │◄─ data: {"delta":{},"finish_reason":"stop"}                             │
    │◄─ data: [DONE]                                                          │
```

Because every message goes through `IConversationService`, the whole pipeline applies unchanged: behavior rules, property setters, HTTP calls, MCP/A2A tools, RAG, multi-model cascading, conversation windowing, persistent user memory, tool cost tracking, rate limiting, the audit ledger and GDPR Art. 18 restriction.

***

## 3. Configuration

| Property                                          | Default            | Purpose                                                                                                                                                                                                             |
| ------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `eddi.openai-compat.enabled`                      | `false`            | Master switch.                                                                                                                                                                                                      |
| `eddi.openai-compat.api-key`                      | *(empty)*          | Shared bearer secret. Required when `http-policy=permit` and authorization is on.                                                                                                                                   |
| `eddi.openai-compat.http-policy`                  | `permit`           | `permit` → the adapter authenticates; `authenticated` → Quarkus OIDC does, and with `authorization.enabled=true` the token must carry `eddi-admin`, `eddi-editor` or `eddi-user`.                                   |
| `eddi.openai-compat.trust-user-headers`           | `true`             | Believe `X-OpenWebUI-User-Id` as the EDDI userId.                                                                                                                                                                   |
| `eddi.openai-compat.allow-anonymous`              | `false`            | Serve callers with no resolvable identity. See the warning in §4.                                                                                                                                                   |
| `eddi.openai-compat.default-user`                 | `openai-anonymous` | The identity used when anonymity is allowed.                                                                                                                                                                        |
| `eddi.openai-compat.environment`                  | `production`       | Which deployment environment's agents are exposed.                                                                                                                                                                  |
| `eddi.openai-compat.request-timeout-seconds`      | `120`              | Per-turn wait before returning `504`.                                                                                                                                                                               |
| `eddi.openai-compat.max-concurrent-requests`      | `64`               | In-flight completions; excess gets `429`.                                                                                                                                                                           |
| `eddi.openai-compat.model-cache-seconds`          | `30`               | Model catalogue TTL. A failed agent listing is not cached: the next request retries it.                                                                                                                             |
| `eddi.openai-compat.expose-stateless-variants`    | `true`             | Enable stateless requests — lists the `:stateless` ids and accepts the `stateless` body field. Disabling blocks both.                                                                                               |
| `eddi.openai-compat.adopt-legacy-header-mappings` | `false`            | Let a chat mapped under the raw `X-OpenWebUI-User-Id` (before ids were namespaced) keep its conversation. Enable only if `/v1` never ran with `http-policy=authenticated` — see §4.                                 |
| `eddi.openai-compat.chat-key-fallback`            | `history`          | How a request that names no chat is mapped: `history` keys it by its opening messages (one conversation per client-side chat); `shared` keeps one conversation per user and agent, as earlier releases did. See §2. |
| `eddi.openai-compat.max-conversation-steps`       | `0`                | Steps after which an idle mapped conversation is ended and the chat continues in a fresh one. `0` disables it. See §2.                                                                                              |

Every property has an environment-variable form: `eddi.openai-compat.api-key` → `EDDI_OPENAI_COMPAT_API_KEY`.

> **Note on CORS.** Open WebUI calls EDDI from its *backend*, so CORS does not apply to it. If you build a browser client that calls `/v1` directly, add its origin to `quarkus.http.cors.origins` and add `x-openwebui-chat-id`/`x-openwebui-user-id` (and `x-eddi-chat-id`, if it sends one) to `quarkus.http.cors.headers`.

***

## 4. Security

### Two modes

**`http-policy=permit` (default).** Quarkus lets `/v1/*` through and the adapter enforces the shared API key itself, with a constant-time comparison. This mode exists because Open WebUI sends an opaque `sk-…` secret, which Quarkus OIDC would reject as a malformed JWT before any application code ran. It works the same with OIDC on (`quarkus.oidc.tenant-enabled=true`, e.g. `docker-compose.auth.yml`), which is how Keycloak and Open WebUI run together: on `/v1` — and only there — EDDI keeps the OIDC mechanism from verifying the bearer value at all, so the key reaches the adapter. A Keycloak token sent in this mode is not the key, and gets the adapter's `401 invalid_api_key`; every other path still requires a valid OIDC token.

> \[!NOTE] Up to 6.6, the `permit` permission did not achieve this when OIDC was on. Quarkus authenticates proactively, so the OIDC mechanism still received the key, failed to verify it and answered `401` with a `WWW-Authenticate: Bearer` challenge before the adapter ran; only requests without an `Authorization` header got through. `OpenAiPermitModeAuthMechanism` now runs ahead of OIDC and gives `/v1` requests the anonymous identity while `eddi.openai-compat.http-policy` and the `/v1/*` permission are both `permit`. It needs inclusive authentication off (`quarkus.http.auth.inclusive`, the default); if that or the permission is changed, EDDI logs a warning at startup that the key cannot get past OIDC.

**`http-policy=authenticated`.** Quarkus OIDC validates a real bearer token first; the adapter reads the resulting principal as the EDDI userId and ignores both the API key and the user headers. With `authorization.enabled=true` (it follows `quarkus.oidc.tenant-enabled`), the token must carry one of the EDDI roles that may converse — `eddi-admin`, `eddi-editor` or `eddi-user`, the same tier `POST /agents/{id}/start` and A2A require; any other token gets `403` (`insufficient_permissions`). With authorization disabled no role is checked. Up to 6.5 any token the realm issued was served. Suited to SDK/LangChain/LiteLLM clients — Open WebUI cannot currently mint per-user OIDC tokens for upstream connections.

### Identity resolution

```
1. OIDC principal                       (http-policy=authenticated)
2. X-OpenWebUI-User-Id → openwebui:<id> (when trust-user-headers=true)
3. default-user                         (only when allow-anonymous=true)
4. otherwise → 401
```

The header value is namespaced to `openwebui:<id>`, so a header can never name an OIDC principal or any other identity source. The prefix is reserved: an OIDC principal that begins with `openwebui:` is refused with `401`.

Chats created before header ids were namespaced are mapped under the raw id. Such a mapping does not record whether the header or an OIDC principal (while `/v1` ran with `http-policy=authenticated`) created it, so it is only adopted when `adopt-legacy-header-mappings=true`. Leave it off if `/v1` ever ran in OIDC mode; otherwise a pre-upgrade chat starts a new conversation on its next message.

An adopted chat keeps its conversation, and that conversation keeps its raw-id owner (so its memories keep loading). GDPR export and erasure resolve data by user id, so for an Open WebUI user of such a chat **address both `openwebui:<id>` and the raw `<id>`**. EDDI does not do this automatically: the raw id shares a namespace with OIDC principals, so deriving it from the namespaced id could export or erase another user's data.

> \[!IMPORTANT] **`trust-user-headers` is a deliberate delegation.** The header is believed only because the caller already proved possession of the API key — i.e. Open WebUI is a trusted proxy that authenticated its own users. **A leaked API key therefore allows impersonating any user.** Rotate it as you would any shared secret, and prefer `http-policy=authenticated` where per-user tokens are available.

> \[!WARNING] **`allow-anonymous=true` merges users.** Every caller without a resolvable identity collapses onto `default-user`, which means they share one conversation — and one memory — per agent and chat. Intended for single-user deployments only. EDDI logs a warning at startup when it is on.

### Startup guard

EDDI **refuses to start** when the adapter is enabled, `http-policy=permit`, no API key is set, and `authorization.enabled=true`. That combination would expose conversation creation to anyone who can reach the port, and nothing about the running system would look wrong afterwards. Fix it by setting an API key, switching to `authenticated`, or disabling the adapter.

It **warns** (and still starts) when OIDC is on, the adapter is in key mode, and OIDC cannot be kept off `/v1` — inclusive authentication is on, `quarkus.http.auth.permission.openai-compat.policy` was overridden to something other than `permit`, or `http-policy` is neither `permit` nor `authenticated`. Every request carrying the key would then get `401` from OIDC.

### What the adapter does not do

* It never injects EDDI's role-gated REST facades (`IRestAgentAdministration`, `IRestAgentStore`, …), so it cannot be used to reach admin operations.
* It performs **no configuration writes**. It creates conversations and conversation mappings, nothing else.

***

## 5. Open WebUI settings that matter

| Setting                               | Value                                           | Why                                                                                                                                                                                                                                                                                                                   |
| ------------------------------------- | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ENABLE_FORWARD_USER_INFO_HEADERS`    | `true`                                          | **Load-bearing.** Supplies `X-OpenWebUI-Chat-Id` (per-chat isolation that survives editing the first message) and `X-OpenWebUI-User-Id` (per-user memory). Without it no user id arrives, so requests are refused unless `allow-anonymous` is on — and chats are then told apart only by their opening messages (§2). |
| Title / Tag generation model          | a `…:stateless` model, or a separate connection | Otherwise Open WebUI's "write a title for this chat" prompt is injected into the user's real conversation.                                                                                                                                                                                                            |
| System Prompt (per model)             | *leave empty*                                   | The agent owns its prompt. A value here arrives as `openai_system_message` context and is ignored unless the agent references it.                                                                                                                                                                                     |
| Tools / Functions (per model)         | *assign none*                                   | EDDI executes tools inside its own pipeline; the adapter never returns `tool_calls`.                                                                                                                                                                                                                                  |
| `RAG_SYSTEM_CONTEXT`                  | `true`                                          | **Set this.** Default `false` puts Open WebUI's retrieved chunks and instruction template into the *user* message, so your agent reads that instead of what the user typed. See below.                                                                                                                                |
| Built-in tools capability (per model) | *off*                                           | Also set this. It cannot be invoked — EDDI never returns `tool_calls` — but leaving it on prepends an `<attached_files>` block to the user's message. No env var; it is a toggle in the model editor. See below.                                                                                                      |

### Title generation

Open WebUI generates chat titles and tags by sending an extra completion request to a configured model. Pointed at a normal EDDI model, that prompt becomes a real turn in the user's conversation: the request carries the chat's `X-OpenWebUI-Chat-Id`, so it lands in the same EDDI conversation as the chat. (Without header forwarding it is a single user message of its own, so it would open a separate conversation per title instead.)

Two ways to avoid it, in order of preference:

1. **Point the task model at a separate connection** (Ollama, a small hosted model). Best titles, no EDDI involvement.
2. **Point it at `<agent>:stateless`.** The adapter starts a throwaway conversation, answers, and ends it. Nothing touches the user's real conversation. Title quality depends on the agent's own system prompt.

### RAG context: set `RAG_SYSTEM_CONTEXT=true`

**This one silently breaks rule-based agents.** When you drop a file into a chat, Open WebUI retrieves the relevant chunks and wraps them — together with its own multi-paragraph instruction template — around your message. By default it puts all of that in the **user message**, not the system message:

```python
# open_webui/env.py
RAG_SYSTEM_CONTEXT = os.getenv('RAG_SYSTEM_CONTEXT', 'False').lower() == 'true'

# open_webui/utils/middleware.py
if RAG_SYSTEM_CONTEXT:
    return add_or_update_system_message(...)   # system message
else:
    return add_or_update_user_message(...)     # <- the default
```

So instead of `what is this pdf about?`, your agent receives `{memory.current.input}` containing several thousand tokens of `### Task: ...`, `<context><source id="1">...</source></context>` and `<attached_files>` markup, with the real question buried at the end. Input matchers stop matching, property setters capture the whole blob, and quick replies never fire.

```yaml
- "RAG_SYSTEM_CONTEXT=true"
```

With it set, the context lands in the system message, the adapter maps it to the `openai_system_message` context entry, and the agent decides whether to use it — `{context.openai_system_message}` — or ignore it. The user's actual text stays the user's actual text.

If you are using EDDI's own RAG pipeline instead, this still matters: without it, Open WebUI's retrieval output pollutes the input regardless.

### Turn off the `builtin_tools` capability on EDDI models

Open WebUI's built-in tools (the Files capability that lets a model list and search chat attachments) are **dead weight against an EDDI model** — the adapter never returns `tool_calls`, so they can never be invoked. They are not harmless, though: enabling them prepends an `<attached_files>` block to the user's message.

```python
# open_webui/utils/middleware.py
use_builtin_tools = (...) or (
    bool(metadata.get('session_id'))
    and metadata.get('params', {}).get('function_calling') != 'legacy'
    and (model.get('info', {}).get('meta', {}).get('capabilities') or {}).get('builtin_tools', True)
)
...
if use_builtin_tools:
    form_data['messages'] = await add_file_context(...)   # prepends <attached_files>
```

So with a file attached, an agent that expected `what is this pdf about?` receives:

```
<attached_files>
<file type="file" id="e097e541-…" content_type="application/pdf" name="Report.pdf"/>
</attached_files>

what is this pdf about?
```

There is **no environment variable** for this — it is a per-model toggle. Open **Workspace → Models → edit the EDDI model → Capabilities** and turn the built-in tools capability off. (Setting the model's `function_calling` parameter to `legacy` has the same effect, as the condition above shows.)

**What you give up, precisely.** Less than it sounds. Open WebUI's built-in tools let the model *search* chat attachments on demand; turning them off does not stop it reading uploaded files. With `RAG_SYSTEM_CONTEXT=true` the retrieved chunks still arrive in the system message and an agent that references `{context.openai_system_message}` answers from them. You lose the on-demand search variant, not the capability.

**Why they cannot work today.** OpenAI tool calling is a multi-turn round trip — the assistant returns `finish_reason: "tool_calls"`, the client executes and sends back a `role: "tool"` message, the assistant continues from the result. This adapter is single-turn: it sends only the last user message and returns final text, and a `role: "tool"` message is dropped. That rule is what keeps EDDI's own memory from being doubled by the client's replayed history (§2), so client-side tools are the one place it would need a deliberate exception.

Supporting them means accepting `tools[]` from the request, plumbing them into `LlmTask` alongside EDDI's own, returning `tool_calls` **only** for client-side tools — EDDI's own MCP, HTTP-call and built-in tools must keep executing server-side, or you get the double-execution this adapter exists to avoid — and reading `role: "tool"` messages back. A real feature, tracked for v2; see [§10](#10-known-gaps).

This is not specific to EDDI. Open WebUI's built-in tools go dark against any OpenAI-compatible backend that does not implement tool calling.

Note this is separate from `RAG_SYSTEM_CONTEXT`: that one controls where the *retrieved chunks* go, this one controls the *file announcement*. Both must be dealt with for an attachment-carrying turn to reach your agent clean.

### Streaming: watch `AIOHTTP_CLIENT_STREAM_IDLE_TIMEOUT`

Open WebUI can end a streamed reply when the upstream sends nothing for that long. EDDI can legitimately be silent for a while before its first token — conversation start, behaviour rules, HTTP calls, an MCP tool chain — and a **rule-based agent emits nothing at all until the turn completes**, since its text comes from the output task rather than token-by-token.

If you set this, size it against your slowest agent, not against a typical LLM's time-to-first-token. Leaving it unset keeps the previous behaviour, where only the overall `AIOHTTP_CLIENT_TIMEOUT` applies.

### Sending `stateless` from the Open WebUI UI

The `stateless` body field (§2) is not an OpenAI-standard parameter, so Open WebUI would normally drop it. Newer versions can forward it: set **`passthrough_params`** in the connection's Advanced settings to a comma-separated list of parameter names — or `*` for every non-standard parameter — and it reaches EDDI verbatim.

That makes the body field usable from the UI, not just from `extra_body` in code. It is the cleaner alternative to picking a `...:stateless` model where your Open WebUI version supports it.

### Optional: an Inlet Filter to save bandwidth

Purely an optimisation — everything works without it. Open WebUI resends the whole history each turn and EDDI discards all but the last user message:

```python
class Filter:
    def inlet(self, body: dict, __metadata__: dict = {}) -> dict:
        messages = body.get("messages", [])
        system = [m for m in messages if m["role"] == "system"]
        user = [m for m in messages if m["role"] == "user"]
        body["messages"] = system + ([user[-1]] if user else [])
        return body
```

This is safe because the adapter has no message-count heuristic — truncating history cannot make it think a new chat started.

***

## 6. What is supported

### Endpoints

| Endpoint                                                           | Status                   |
| ------------------------------------------------------------------ | ------------------------ |
| `GET /v1/models`                                                   | ✅                        |
| `GET /v1/models/{id}`                                              | ✅                        |
| `POST /v1/chat/completions` (`stream:false`)                       | ✅ JSON                   |
| `POST /v1/chat/completions` (`stream:true`)                        | ✅ SSE                    |
| `/v1/embeddings`, `/v1/completions`, `/v1/audio/*`, `/v1/images/*` | ❌ Plain `404` (see note) |

> **Note:** unimplemented `/v1` paths match no JAX-RS resource, so they return Quarkus' standard `404` rather than an OpenAI error envelope. Every path the adapter *does* serve uses the envelope. Adding a catch-all route would risk shadowing the real endpoints for a purely cosmetic gain, so it is deliberately not done.

### Request fields

| Field                                                                                      | Handling                                                                                                                |
| ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------- |
| `model`                                                                                    | Resolved to an agent. Required.                                                                                         |
| `messages[]`                                                                               | Only the last `role:"user"` entry is sent.                                                                              |
| `role:"system"`                                                                            | Last one becomes `openai_system_message` context — never overrides the agent's prompt.                                  |
| `content` as string                                                                        | Plain text.                                                                                                             |
| `content` as array                                                                         | Text extracted; `image_url`, `file` and `input_audio` parts mapped to attachments.                                      |
| `stream`                                                                                   | Selects JSON vs SSE. The only dispatch signal.                                                                          |
| `stateless`                                                                                | **EDDI extension.** Run this turn in a throwaway conversation — same as the `:stateless` model suffix. See §2.          |
| `metadata.chat_id`                                                                         | Explicit chat key (after the `X-OpenWebUI-Chat-Id` and `X-EDDI-Chat-Id` headers). Other `metadata` entries are ignored. |
| `user`                                                                                     | Chat key when no header or `metadata.chat_id` names one. Accepts both the string form and Open WebUI's object form.     |
| `temperature`, `max_tokens`, `top_p`, `stream_options`, `tools`, `tool_choice`, `files`, … | **Accepted and ignored.** Model parameters belong to the agent's `langchain.json`.                                      |

Request headers: `X-OpenWebUI-Chat-Id` and `X-EDDI-Chat-Id` name the chat (§2); `X-OpenWebUI-User-Id` names the user (§4).

### Response fields

| Field                           | Value                                                                                                                                                                                                                             |
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `finish_reason`                 | `stop`; `length` when the model's answer was cut off at its output-token limit; `content_filter` when the provider filtered it. Both come from the model's own finish reason, which EDDI records for plain (non-tool) model calls |
| `usage`                         | See §7.1                                                                                                                                                                                                                          |
| `X-EDDI-Conversation-Id` header | The EDDI conversation the turn ran in                                                                                                                                                                                             |

### Attachments — images, documents, audio

Three OpenAI content-part types are mapped to EDDI `attachment_N` context entries, which then flow through the normal attachment pipeline — capability gating per provider and model, per-turn and byte caps, PDF text extraction, and SSRF-guarded fetching for remote URLs.

| Content part  | Wire shape                                                              | Downstream                                                                                        |
| ------------- | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| `image_url`   | `{"url":"data:image/png;base64,…"}` or a remote URL                     | `ImageContent` when the model has vision; otherwise a note explaining why it was dropped          |
| `file`        | `{"filename":"report.pdf","file_data":"data:application/pdf;base64,…"}` | `PdfFileContent` when the model has native document support; otherwise text-extracted and inlined |
| `input_audio` | `{"data":"<raw base64>","format":"wav"\|"mp3"}`                         | `AudioContent` when the model supports audio                                                      |

Notes worth knowing:

* **`input_audio.data` is raw base64 with no `data:` prefix**, unlike every other binary payload in the protocol. `format` carries the type separately, and `mp3` maps to `audio/mpeg` (not `audio/mp3`, which is not a real media type).
* **`file.file_id` is not supported.** It references a file uploaded through the OpenAI Files API, which EDDI does not implement. Such parts are skipped with a warning — send `file_data` inline instead.
* **The declared `filename` wins over a generic data-URI type.** A `data:application/octet-stream` payload named `contract.pdf` is still treated as a PDF, because clients that base64 a file without sniffing it send the generic type.
* **The per-turn cap (`eddi.attachments.max-per-turn`, default 5) counts all three types together.** Excess attachments are dropped with a warning rather than failing the turn.
* Beyond PDFs, any type EDDI's text extractor handles (`.txt`, `.md`, `.csv`, `.json`, `.xml`, `.html`) is inlined as text.

> **Open WebUI specifically** sends images as `image_url` and does *not* send documents as `file` parts — it runs its own RAG over uploads and injects the retrieved text into the prompt. **Where it injects matters a great deal to an EDDI agent — see** [**§5, RAG context**](#rag-context-set-rag_system_contexttrue)**.** The `file` path is for the OpenAI SDK and other clients that send PDFs inline.

### HITL

Fully supported, surfaced as chat text rather than as errors:

* A turn that pauses (`PAUSE_CONVERSATION`, or a `hitlConfig.toolApprovals` tool gate) returns `200` with any output so far, a pause notice, and the conversation id.
* Further messages while paused return "still awaiting approval".
* Approve or reject out of band — `POST /agents/{conversationId}/resume`, the Slack approval card, or the Manager UI. The next message continues normally.

The conversation id is also on every response as `X-EDDI-Conversation-Id`.

***

## 7. Behaviour to expect

| Behaviour                            | Effect                                                                                                                                                                                                                                                                                                                                            |
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Regenerate**                       | Sends the message to EDDI **again** — a genuine new turn. Behavior rules advance and tools re-run. Inherent to a stateful backend.                                                                                                                                                                                                                |
| **Edit and resend**                  | Same: a new turn, not a rewrite of history.                                                                                                                                                                                                                                                                                                       |
| **Deleting a chat in Open WebUI**    | Does not end the EDDI conversation. It is abandoned and reaped by normal conversation lifecycle policy.                                                                                                                                                                                                                                           |
| **Documents dropped into chat**      | Handled entirely by Open WebUI's own RAG (upload → chunk → embed → inject into the prompt). **By default it injects into the&#x20;*****user*****&#x20;message, not the system message** — set `RAG_SYSTEM_CONTEXT=true` (§5) or your agent receives the whole `<source>` blob as its input. For production document RAG, use EDDI's own pipeline. |
| **Two SDK chats without a chat id**  | Two conversations, one per chat (§2) — unless both open with the same system and first user message *at the same time*.                                                                                                                                                                                                                           |
| **A failing agent**                  | An OpenAI error (`500`, `code: agent_error`, the failing task's sanitized reason as `message`) — on a stream, an error event. Never a reply with `finish_reason: "stop"`.                                                                                                                                                                         |
| **Token counts**                     | Reported as `usage` for agents that call a model, summed across cascade steps and tool round-trips. Rule-based agents spend no tokens, so the field is omitted rather than zero-filled. On streams it needs `stream_options.include_usage` (§7.1).                                                                                                |
| **Two requests in one chat at once** | The second is dropped by `ConversationCoordinator` and returns `429`; clients retry. If the chat's conversation has ended, both requests (Open WebUI's completion and title request, say) move to the same new conversation rather than one each.                                                                                                 |

### 7.1 Token usage

`usage` is built from the turn's `audit:token_usage` entry, which `LlmTask` accumulates across **every** model call the turn made — so a turn that escalated through three cascade steps and two tool round-trips reports the sum, not the last leg.

```json
"usage": {"prompt_tokens": 1204, "completion_tokens": 88, "total_tokens": 1292}
```

Two things follow from where the number comes from:

* **A rule-based agent reports no `usage` at all.** It called no model, so there is nothing to count. The field is omitted rather than zero-filled, because `0 tokens` reads in a client as a measurement rather than as an absence.
* **On the streaming path it is opt-in.** Per the OpenAI specification, usage is emitted only when the client sends `stream_options: {"include_usage": true}`, as a trailing frame with an empty `choices` array, after the `finish_reason` frame and before `[DONE]`. Clients that did not ask for it never see it — an unrequested empty-choices frame is a protocol deviation some clients reject.

### 7.2 Structured outputs

An EDDI turn can carry eight output types; the OpenAI protocol carries one string. Rather than drop everything but text, the adapter renders the rest as Markdown and appends it:

| EDDI output item  | Rendered as                                                                                 |
| ----------------- | ------------------------------------------------------------------------------------------- |
| `text`            | the text itself, unchanged                                                                  |
| `quickReply`      | `_Suggested replies:_` + each value in backticks                                            |
| `image`           | `![alt](uri)`                                                                               |
| `applicationLink` | `[label](path)`                                                                             |
| `button`          | `**[label]**` — the label only; `onPress` is a client-side instruction with no meaning here |
| `inputField`      | `**Label:**`, plus a warning for `password` that this channel cannot mask input             |
| `agentFace`       | dropped — an avatar has no text equivalent                                                  |
| `other`           | dropped                                                                                     |

A turn combining a text item with quick replies arrives as one assistant message:

```
Which LLM provider should this agent use?

_Suggested replies:_ `Anthropic` · `OpenAI` · `Gemini` · `Ollama (local)`
```

Blocks are joined with a blank line, so each renders as its own paragraph. Values are de-duplicated, blank ones skipped, and original order preserved.

This rendering is local to the adapter. The shared `ConversationOutputExtractor` is unchanged — its other callers feed agent-to-agent prompts, where interaction affordances would be noise.

#### Why quick replies show the value, and why typing it works

A quick reply has two halves: a `value` the user sees (`Anthropic`) and an `expressions` token the behaviour rules match (`select_anthropic`). Rendering the *value* is not merely cosmetic — it is the half that round-trips.

`InputParserTask.prepareTemporaryDictionaries()` looks at the **previous** turn's output and, for every quick reply it finds, registers a temporary dictionary entry mapping the `value` to its `expressions` (`DictionaryUtilities.convertQuickReplies` — `addWord` for single words, `addPhrase` when the value contains a space). So when the user reads `` `Anthropic` `` and types `Anthropic`, the parser resolves it to `select_anthropic` and the matching rule fires, with no dictionary configured on the agent. This is why a quick-reply-driven agent works over `/v1` even when its parser step declares `"dictionaries": []`.

Two consequences worth knowing:

* **Rendering the `expressions` token instead would break this.** `select_anthropic` is not in any dictionary; only the value is registered. Showing the internal token would also leak implementation detail at the user.
* **The window is exactly one turn.** The temporary dictionary is rebuilt from the immediately preceding output, so quick replies are answerable on the *next* message only. If the user types something unrelated first, the options are no longer matchable by name and the agent falls back to whatever its `*` rules do.

Numbering (`1.`, `2.`) is deliberately avoided for the same reason: nothing registers `2` as an answer, so a numbered list would invite input the parser cannot resolve.

***

## 8. Errors

Every failure uses the OpenAI envelope:

```json
{"error": {"message": "…", "type": "invalid_request_error", "param": null, "code": "model_not_found"}}
```

| Condition                                                                        | Status | `code`                                   |
| -------------------------------------------------------------------------------- | ------ | ---------------------------------------- |
| Missing or wrong API key                                                         | 401    | `invalid_api_key`                        |
| Unresolvable caller identity                                                     | 401    | `invalid_api_key`                        |
| Unknown model                                                                    | 404    | `model_not_found`                        |
| Ambiguous model name                                                             | 400    | `ambiguous_model`                        |
| `stateless:true` while stateless requests are disabled                           | 400    | `invalid_request_field`                  |
| No `user` message                                                                | 400    | `no_user_message`                        |
| User message longer than `eddi.conversations.max-input-chars` (default `200000`) | 400    | `input_too_large`                        |
| Agent not deployed                                                               | 503    | `agent_not_ready`                        |
| Adapter concurrency cap (`max-concurrent-requests`) or a busy conversation       | 429    | — (`type: rate_limit_exceeded`)          |
| Node at its in-flight turn limit, coordinator saturated, shutdown drain          | 503    | `capacity_exceeded` (with `Retry-After`) |
| Tenant quota store unavailable                                                   | 503    | `quota_accounting_unavailable`           |
| Authenticated caller without a permitted role                                    | 403    | `insufficient_permissions`               |
| Turn timed out                                                                   | 504    | `timeout`                                |
| The agent's pipeline failed the turn (an LLM, HTTP or MCP task error)            | 500    | `agent_error`                            |
| Adapter disabled                                                                 | 404    | `unknown_endpoint`                       |

A failed turn (conversation state `ERROR`) carries the failing task's digest as `message` — the same sanitized text EDDI's own REST and streaming endpoints report, with URLs, stack frames and credentials stripped. It used to come back as `200` with `_The agent produced no text output._` and `finish_reason: "stop"`, indistinguishable from a real answer.

> **On streams:** once SSE headers are flushed the status is fixed at `200`. A failure during a stream — a failed turn, a busy conversation, the concurrency cap, the request timeout — therefore ends the stream with an OpenAI **error event**, `data: {"error":{"message":…,"type":…,"code":…}}`, followed by `[DONE]`. The `openai` SDKs raise this as `APIError`; Open WebUI shows it as an error. Every stream ends with exactly one terminator — a `finish_reason` chunk or an error event — and one `[DONE]`, and nothing is written after it: a turn that is still producing tokens when the request times out has its late tokens dropped. A paused (HITL) turn is not a failure and still arrives as chat text with `finish_reason: "stop"` (§6).

***

## 9. Troubleshooting

**No models in the dropdown.** Check `eddi.openai-compat.enabled=true`, that at least one agent is deployed to the configured environment with status `READY`, and that the API key matches. `curl -H "Authorization: Bearer <key>" http://eddi:7070/v1/models` shows the raw answer.

**All my chats share one conversation.** Either `chat-key-fallback=shared` is set, or the client sends the same `user` (or `X-EDDI-Chat-Id`/`metadata.chat_id`) on every request — an explicit key always wins. With Open WebUI, check that `ENABLE_FORWARD_USER_INFO_HEADERS` is set, so each chat arrives with its own `X-OpenWebUI-Chat-Id`.

**My client gets a new conversation every turn.** It sends only its latest message instead of the history, so every request looks like the opening turn of a new chat. Send an explicit `X-EDDI-Chat-Id` or `metadata.chat_id`, or set `chat-key-fallback=shared`. Each distinct opening message leaves a conversation and a mapping behind until the retention sweep deletes the ended conversation (`eddi.conversations.deleteEndedConversationsOnceOlderThanDays`), which now deletes its mapping rows with it; a high-volume client of this kind should still send an explicit key.

**Every user sees everyone else's memory.** `allow-anonymous=true` with `trust-user-headers=false`, or with the header absent. Set `trust-user-headers=true` and enable header forwarding.

**EDDI won't start.** Read the startup message: the guard fires when the adapter is enabled, unauthenticated, and authorization is on. It names the three ways out.

**Chat titles appear in the conversation.** Point Open WebUI's title/tag model at a `:stateless` variant or a separate connection (§5).

**Streaming looks like a hang behind a proxy.** The adapter sets `X-Accel-Buffering: no`, but some proxies need explicit configuration to stop buffering `text/event-stream`.

**401 for the shared key under OIDC.** In `permit` mode an adapter `401` carries an OpenAI error body (`invalid_api_key`): the key is wrong or missing. A `401` with an empty body and `WWW-Authenticate: Bearer` comes from OIDC instead — check the startup log for the warning described under [Startup guard](#startup-guard), and that the request path is `/v1/...` with `GET`, `POST` or `OPTIONS`.

**401 with a valid-looking token under OIDC.** In `http-policy=authenticated` mode the token must be a real OIDC token for your realm; an `sk-…` string is not. It must also carry the **audience** EDDI requires — `quarkus.oidc.token.audience` defaults to `eddi-backend`, and Keycloak emits `aud` only through an explicit `oidc-audience-mapper` on the client that minted the token. A token from a client without that mapper authenticates everywhere else and is refused here. See [Security → Configuration Properties](https://docs.labs.ai/security-and-compliance/security.md#configuration-properties).

***

## 10. Known gaps

1. **Agent groups are not exposed as models.** Only individual agents appear in `/v1/models`; EDDI's multi-agent group discussions are unreachable over this API. Nothing blocks it — groups are listable, `discuss()` returns a `synthesizedAnswer`, `continueDiscussion()` gives multi-turn, and `GroupDiscussionEventListener` gives streaming — but it needs a second bridge with its own conversation mapping, streaming path and approval surface, so it belongs in its own change rather than bolted onto this one.
2. **Structured outputs are flattened to Markdown, not interactive.** Quick replies, buttons and input fields are rendered as text (§7.2), so the user reads and retypes them rather than clicking. `agentFace` and `other` items are dropped entirely.
3. **Regenerate is a real new turn**, not an idempotent replay.
4. **Stream-path errors must return 200** — they arrive as an OpenAI error event (see §8).
5. **One worker thread per in-flight completion**, bounded by `max-concurrent-requests`.
6. **`http-policy=authenticated` is impractical with Open WebUI**, which cannot mint per-user upstream OIDC tokens.
7. **`tool_calls` are never returned.** EDDI runs tools inside its pipeline; echoing them would make Open WebUI try to execute EDDI's HTTP/MCP/memory tools locally, where they do not exist. Tool activity remains visible in the audit ledger and `toolTrace`.
8. **Client-side tools cannot be used** — Open WebUI's built-in Files/knowledge tools, or any `tools[]` the client sends. Uploaded files are still readable through Open WebUI's RAG (§5); what is missing is the model invoking a client tool mid-answer. Doing this properly needs the multi-turn `tool_calls` → `role: "tool"` round trip, which the single-turn message mapping deliberately does not support, plus a way to distinguish client tools from EDDI's own so only the former are echoed. The clearest v2 candidate in this list.

***

## 11. Recipe: EDDI as a thin LLM gateway

If what you want from EDDI is **vaulted API keys, an audit trail, tenant quotas and cost tracking** — not agent logic — you do not need a passthrough proxy. A minimal LLM-only agent behind this adapter already provides all of it, and behaves like a plain model call.

> \[!IMPORTANT] **Read the limits first.** This is genuinely useful for single-turn workloads (classification, extraction, summarisation, drafting). It is **not** a drop-in replacement for an OpenAI-compatible gateway such as LiteLLM or Portkey:
>
> * **Single-turn only.** The adapter sends just the last user message, so a client that manages its own history gets no context from earlier turns (§2).
> * **One agent per model.** A caller cannot send `model: "gpt-4o"` and have it routed; each model you want exposed is one agent config.
> * **No caching, fallbacks, load balancing or virtual per-user keys.** Those are what a dedicated gateway is for.
>
> If you need any of the above, use a real gateway. If you want key custody and an audit trail on single-turn calls you are already making, this is a few config files.

### What you get

|                    |                                                                    |
| ------------------ | ------------------------------------------------------------------ |
| **API keys**       | Stored via EDDI's Secrets Vault, never in the client               |
| **Audit**          | Every call recorded in the audit ledger with user attribution      |
| **Quotas / cost**  | Tenant quotas and per-conversation cost tracking apply             |
| **Access control** | The adapter's shared key or OIDC, plus per-user identity           |
| **Model swap**     | Change provider or model in config; callers keep the same model id |

### The four config files

Assembled from the standard shapes — see [`langchain.md`](https://docs.labs.ai/agent-configuration/langchain.md) for the full LLM parameter reference and [`architecture.md`](https://docs.labs.ai/architecture-and-concepts/architecture.md) for the workflow model. Use valid hex ids (≥18 chars) throughout.

**1. `…0002.behavior.json`** — fire the LLM on every user turn:

```json
{
  "expressionsAsActions": true,
  "behaviorGroups": [{
    "behaviorRules": [{
      "name": "Always answer",
      "actions": ["send_message"],
      "conditions": [{
        "type": "inputmatcher",
        "configs": { "expressions": "*", "occurrence": "currentStep" }
      }]
    }]
  }]
}
```

> A single unconditional `inputmatcher` deliberately departs from the guidance in [Agent Config Authoring](https://docs.labs.ai/agent-configuration/agent-config-authoring.md#behavior-rule-safety-rules) that every rule carry an `actionmatcher` on `lastStep`. That rule exists to stop wizard-style agents firing out of order; here, firing on every turn *is* the intent.

**2. `…0003.langchain.json`** — the model:

```json
{
  "tasks": [{
    "actions": ["send_message"],
    "id": "gateway",
    "type": "openai",
    "description": "Thin LLM gateway",
    "parameters": {
      "apiKey": "${vault:openai-key}",
      "modelName": "gpt-4o",
      "systemMessage": "",
      "logSizeLimit": "-1",
      "addToOutput": "true"
    }
  }]
}
```

Set `logSizeLimit: "0"` if you want each turn to carry no conversation history at all — belt and braces alongside `:stateless`.

**3. `…0001.workflow.json`** — three steps, no output task (the LLM's `addToOutput` supplies the response):

```json
{
  "workflowSteps": [
    { "type": "eddi://ai.labs.parser",    "extensions": { "dictionaries": [], "corrections": [] }, "config": {} },
    { "type": "eddi://ai.labs.behavior",  "extensions": {}, "config": { "uri": "eddi://ai.labs.rules/rulestore/rulesets/…0002?version=1" } },
    { "type": "eddi://ai.labs.llm",       "extensions": {}, "config": { "uri": "eddi://ai.labs.llm/llmstore/llms/…0003?version=1" } }
  ]
}
```

**4. `…0000.agent.json`**:

```json
{ "workflows": ["eddi://ai.labs.workflow/workflowstore/workflows/…0001?version=1"], "channels": [] }
```

Name the agent descriptor something client-facing — `gpt-4o-gateway` — since it becomes the model id (`gpt-4o-gateway-<last6>`).

### Calling it

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://eddi.example.com/v1", api_key=os.environ["EDDI_API_KEY"],
                default_headers={"X-OpenWebUI-User-Id": "svc-billing"})

client.chat.completions.create(
    model="gpt-4o-gateway-a3f9c1",
    messages=[{"role": "user", "content": "Classify: 'my card was declined'"}],
    extra_body={"stateless": True},      # throwaway conversation per call
)
```

`stateless` matters here: without it every call from that identity accumulates into one conversation, since a service caller sends no `X-OpenWebUI-Chat-Id`.

Add one agent per model you want to expose. Each appears in `GET /v1/models` automatically.

> **Not verified end-to-end by the adapter's test suite.** The config shapes above are taken from working examples in this repository, but the recipe as a whole is a pattern — deploy it to a test environment before relying on it.

***

## See also

* [`architecture.md`](https://docs.labs.ai/architecture-and-concepts/architecture.md) — the conversation pipeline
* [`hitl.md`](https://docs.labs.ai/conversations-and-orchestration/hitl.md) — human-in-the-loop approval gates
* [`../planning/openai-api-adapter-plan.md`](https://github.com/labsai/EDDI/tree/main/planning/openai-api-adapter-plan.md) — design rationale and the research behind it
