> For the complete documentation index, see [llms.txt](https://docs.labs.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.labs.ai/agent-configuration/attachments-guide.md).

# Multimodal Attachments

> EDDI's attachment pipeline enables multimodal conversations — users can send images, files, and documents alongside text input. Attachments flow through the lifecycle pipeline and are automatically forwarded to vision-capable LLMs.

## Quick Start

### Send an Image via URL

```bash
POST /agents/{conversationId}
Content-Type: application/json

{
  "input": "What is in this image?",
  "context": {
    "attachment_0": {
      "type": "object",
      "value": {
        "mimeType": "image/png",
        "url": "https://example.com/photo.png",
        "fileName": "photo.png"
      }
    }
  }
}
```

### Send an Image via Base64

```bash
POST /agents/{conversationId}
Content-Type: application/json

{
  "input": "Describe this icon",
  "context": {
    "attachment_0": {
      "type": "object",
      "value": {
        "mimeType": "image/png",
        "data": "iVBORw0KGgoAAAANSUhEUgAAAAE...",
        "fileName": "icon.png"
      }
    }
  }
}
```

The image is automatically forwarded to the LLM as multimodal content. The LLM "sees" the image alongside the text message.

***

## How It Works

```
Client sends context with attachment_* keys
                          │
                          ▼
        ┌─────────────────────────────────────┐
        │ Conversation.prepareLifecycleData() │
        │                                     │
        │ AttachmentContextExtractor parses   │
        │ attachment_0, attachment_1, ...     │
        │ into List<Attachment> objects       │
        │                                     │
        │ Stored in memory: "attachments"     │
        └───────────────────────┬─────────────┘
                                │
         ┌──────────────────────┼─────────────────────┐
         ▼                      ▼                     ▼
┌──────────────────┐  ┌───────────────────┐  ┌──────────────────┐
│ BehaviorRules    │  │ LlmTask           │  │ Other Tasks      │
│                  │  │                   │  │                  │
│ ContentType-     │  │ AttachmentFor-    │  │ Read from the    │
│ Matcher matches  │  │ warder resolves   │  │ "attachments"    │
│ on MIME type,    │  │ bytes, gates on   │  │ memory key       │
│ so a PDF and an  │  │ ModelCapability-  │  │ directly         │
│ image can take   │  │ Service, emits    │  │                  │
│ different paths  │  │ langchain4j       │  │                  │
│                  │  │ Content           │  │                  │
└──────────────────┘  └───────────────────┘  └──────────────────┘
```

### Pipeline Stages

1. **Context Extraction** — `AttachmentContextExtractor` parses `attachment_*` context keys into `Attachment` objects
2. **Memory Storage** — Attachments are stored as `List<Attachment>` in the `attachments` memory key
3. **Rule Matching** — `ContentTypeMatcher` condition matches on MIME types for routing
4. **LLM Forwarding** — `AttachmentForwarder` resolves each attachment's bytes, gates it on `ModelCapabilityService`, and converts it to the right langchain4j `Content` on the outgoing user message

***

## Input Paths

### Path A: URL Reference (Recommended)

Best for images already hosted somewhere. The URL is passed to the provider when it fetches URLs itself; otherwise EDDI downloads the bytes and inlines them (see [LLM Multimodal Support](#llm-multimodal-support)).

```json
{
  "attachment_0": {
    "type": "object",
    "value": {
      "mimeType": "image/jpeg",
      "url": "https://cdn.example.com/photos/sunset.jpg",
      "fileName": "sunset.jpg"
    }
  }
}
```

### Path B: Base64 Inline

Best for small images (< 5MB). Data is sent inline as a base64-encoded string.

```json
{
  "attachment_0": {
    "type": "object",
    "value": {
      "mimeType": "image/png",
      "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk...",
      "fileName": "icon.png"
    }
  }
}
```

> **Note:** `base64Data` is transient — it's never persisted to MongoDB. For large files, use the upload endpoint (Path C below) or URL references.

### Path C: File Upload

For large files, upload them to the storage backend and receive a storage reference:

```bash
POST /conversations/{conversationId}/attachments
Content-Type: multipart/form-data

# Form field: file (the binary file)
```

**Response (201):**

```json
{
  "storageRef": "3f2b9c1e-6d4a-4f0e-9b7a-2c8d5e1f0a93",
  "fileName": "report.pdf",
  "mimeType": "application/pdf",
  "sizeBytes": 524288,
  "conversationId": "…",
  "forwardableInline": true
}
```

`forwardableInline` is `true` when `sizeBytes` is at most `eddi.attachments.max-forward-bytes` (default 10 MB): only then can the blob be sent inline to a model provider on a later turn; `eddi.attachments.max-forward-aggregate-bytes` caps the total forwarded with one message.

**Other endpoints** (all under `/conversations/{conversationId}/attachments`):

| Method   | Path            | Purpose                                      |
| -------- | --------------- | -------------------------------------------- |
| `GET`    | \`\`            | List the conversation's attachments          |
| `GET`    | `/{storageRef}` | Download one blob                            |
| `DELETE` | `/{storageRef}` | Delete one blob                              |
| `DELETE` | \`\`            | Delete all of the conversation's attachments |

An unknown `storageRef` answers `404` (`ATTACHMENT_NOT_FOUND`), and a blob the conversation may not access answers `403`.

The returned `storageRef` can then be used in subsequent conversation turns by setting it as the `storageRef` field in an attachment context key. It takes precedence over `url` and `data`, and the authoritative `mimeType` and `sizeBytes` are resolved server-side from the store, so they need not be supplied:

```json
{
  "attachment_0": {
    "type": "object",
    "value": { "storageRef": "3f2b9c1e-6d4a-4f0e-9b7a-2c8d5e1f0a93", "fileName": "report.pdf" }
  }
}
```

The `storageRef` is a random UUID — unguessable, and only readable from the owning conversation or one it was explicitly granted to. A blob that records no owning conversation (one stored by a release that predates attachment ownership) cannot be read through a conversation unless that conversation was explicitly granted access to it, on either backend: ownership that cannot be established is refused rather than assumed, while an explicit grant still applies. It can still be deleted by its `storageRef`, because deleting discloses nothing and refusing it would leave data no erasure could remove. Deleting a conversation, and a GDPR erasure, remove the blobs that record that conversation as their owner; an ownerless blob records none, so they do not reach it (as before). The storage backend (GridFS or PostgreSQL) is selected automatically based on the configured datastore.

#### Content check

The declared type is checked against the file's content. Images and PDFs must carry their signature (a "PNG" without one is refused). Text declared as a text type — `text/*`, JSON, XML, YAML, CSV — is accepted when it reads as text (no NUL bytes, almost no control characters), whatever short signature it happens to start with: a CSV whose first column is `BMI` starts with the bitmap magic `BM`, and Markdown can quote `%PDF-1.7`, and neither is an image or a PDF. A real binary declared as text is still refused.

#### Quotas

| Quota                                               | Default   |
| --------------------------------------------------- | --------- |
| `eddi.attachments.max-per-conversation`             | 50 files  |
| `eddi.attachments.max-total-bytes-per-conversation` | 100 MB    |
| `eddi.attachments.max-per-user`                     | 500 files |
| `eddi.attachments.max-total-bytes-per-user`         | 1 GiB     |

The per-user quotas count every blob owned by the conversation's **owner**, across all of their conversations — an admin uploading into a user's conversation fills that user's quota. A group discussion's inline files count too: they are stored under the discussion's owner (before this, they were stored without one, so no per-user quota counted them). Blobs stored before the owner was recorded on them (earlier releases) do not count towards it. Both are on by default (they used to be off, so one user could fill the store alone); set either to `0` to disable it.

Quotas are enforced atomically: the usage is counted and the blob stored under one lock per scope — a transaction-scoped advisory lock on PostgreSQL (`hashtextextended`, so PostgreSQL 11 or newer), a lease document in `attachments.quota_locks` on MongoDB — so concurrent uploads cannot each see room and together overshoot the limit. A refusal is a `400` with `"code": "ATTACHMENT_REJECTED"` and `"quota": "conversation"` or `"user"`, counted on `eddi_attachments_quota_rejected_total{scope}`.

| Response Code | Meaning                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `201`         | File stored successfully                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `400`         | No file provided, file exceeds `eddi.attachments.max-size-bytes`, the content does not match the declared type, or a quota is full (`quota` names which)                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `413` / `411` | The request body passed the HTTP body limit, or arrived without a `Content-Length`. The limit is `eddi.http.limits.default-max-body-size`, raised automatically to fit an attachment at `eddi.attachments.max-size-bytes` base64-encoded — so raising the attachment limit raises it too, up to `quarkus.http.limits.max-body-size` (60 MB), which it cannot exceed. Above about 44 MB of attachment, raise that ceiling as well; a WARN at startup names the value needed. See [configuration-reference.md](https://docs.labs.ai/deployment-and-infrastructure/configuration-reference.md#security--authentication) |
| `500`         | Storage or I/O error                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |

***

## Context Key Format

Attachment context keys must match the pattern `attachment_*`:

| Key                     | Valid?           |
| ----------------------- | ---------------- |
| `attachment_0`          | ✅                |
| `attachment_screenshot` | ✅                |
| `attachment_`           | ✅                |
| `image_0`               | ❌ (wrong prefix) |
| `attachment`            | ❌ (no suffix)    |

### Required Fields

| Field        | Required                         | Description                                          |
| ------------ | -------------------------------- | ---------------------------------------------------- |
| `storageRef` | One of storageRef/url/data       | Reference to a previously uploaded blob (see Path C) |
| `mimeType`   | Yes, unless `storageRef` is used | MIME type (e.g., `image/png`, `application/pdf`)     |
| `url`        | One of storageRef/url/data       | External URL reference                               |
| `data`       | One of storageRef/url/data       | Base64-encoded content                               |
| `fileName`   | No                               | Original filename (for metadata/logging)             |

`storageRef` has the highest precedence; if both `url` and `data` are present, `url` takes precedence.

***

## Multiple Attachments

Send multiple attachments by incrementing the key index:

```json
{
  "attachment_0": {
    "type": "object",
    "value": { "mimeType": "image/png", "url": "https://example.com/page1.png" }
  },
  "attachment_1": {
    "type": "object",
    "value": { "mimeType": "image/png", "url": "https://example.com/page2.png" }
  }
}
```

All attachments are forwarded to the LLM in a single multimodal user message — at most `eddi.attachments.max-per-turn` of them (default `5`; non-positive means unlimited). Extras are dropped with a note in `attachments:errors`.

***

## LLM Multimodal Support

`AttachmentForwarder` is the single place an attachment becomes langchain4j `Content` on the outgoing user message. For each attachment it resolves the bytes from whichever source supplied them (stored blob, URL, inline base64) under uniform per-file and aggregate byte caps, asks `ModelCapabilityService` what the configured provider/model can actually accept, and emits accordingly:

| MIME Type                                  | Model has the capability                                                                                                                   | Model does not                                   |
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------ |
| `image/*`                                  | `ImageContent` — the URL is passed through when the provider fetches URLs itself, otherwise the bytes are downloaded and inlined as base64 | Metadata note                                    |
| `application/pdf`                          | Native `PdfFileContent`                                                                                                                    | PDFBox text extraction, inlined as `TextContent` |
| `audio/*`                                  | `AudioContent`                                                                                                                             | Metadata note                                    |
| text-like (`text/*`, JSON, XML, CSV, YAML) | Decoded and inlined as `TextContent` — **no capability required**, so this works on every model                                            | —                                                |
| Anything else                              | Metadata note pointing the model at the `readAttachment` tool                                                                              | Metadata note                                    |

Note the difference between the last two rows: a CSV is *read* into the prompt, whereas a `.zip` is only *announced*. The metadata note looks like this:

```
[Attachment: archive.zip (application/zip, 15240 bytes)]
```

**Nothing is dropped silently.** Text extracted from a PDF is persisted to the `attachments:extracts` memory key so later turns can stitch it back into the history, and every drop, cap-skip and capability gate is appended to `attachments:errors` — which is where to look first when a model claims it cannot see a file you attached.

### Observability

| Metric                      | Type    | Meaning                                                |
| --------------------------- | ------- | ------------------------------------------------------ |
| `eddi.attachment.forwarded` | Counter | Attachments converted to `Content`                     |
| `eddi.attachment.reinlined` | Counter | Extracted text stitched back into conversation history |
| `eddi.attachment.errors`    | Counter | Drops, cap-skips and capability gates                  |

***

## Routing with Behavior Rules

Use `contentTypeMatcher` to create different workflows based on attachment type:

### Route Images to Vision Agent

```json
{
  "name": "Image received",
  "actions": ["analyze_image"],
  "conditions": [
    {
      "type": "contentTypeMatcher",
      "configs": {
        "mimeType": "image/*",
        "minCount": "1"
      }
    }
  ]
}
```

### Route PDFs to Document Processor

```json
{
  "name": "Document received",
  "actions": ["process_document"],
  "conditions": [
    {
      "type": "contentTypeMatcher",
      "configs": {
        "mimeType": "application/pdf",
        "minCount": "1"
      }
    }
  ]
}
```

### Require Specific Attachment Count

```json
{
  "name": "Comparison ready",
  "actions": ["compare_images"],
  "conditions": [
    {
      "type": "contentTypeMatcher",
      "configs": {
        "mimeType": "image/*",
        "minCount": "2"
      }
    }
  ]
}
```

***

## Template Access

`{memory.current.attachments}` renders as an empty string. Attachments are written to the current step's data store under the `attachments` memory key (`storeData`), while `{memory.current.*}` resolves against the step's `ConversationOutput` — a different map.

The `attachment_*` **context** keys are a different matter: `MemoryItemConverter` publishes the request context into the template model, so `{context.attachment_0.mimeType}`, `{context.attachment_0.url}` and `{context.attachment_0.fileName}` do resolve for attachments supplied that way.

To act on attachments, read the `attachments` memory key from a task, or match on them declaratively with `contentTypeMatcher` in a behavior rule.

***

## Group Conversations

Group discussions accept inline data and hosted URLs on `POST /groups/{groupId}/conversations`. Inline data is stored in the blob store owned by the group conversation and counted against the discussion's owner, so the same quotas apply as to a 1:1 upload: `eddi.attachments.max-size-bytes` per file, `max-per-conversation` and `max-total-bytes-per-conversation` for the discussion (checked from the request before anything is decoded or stored), and the per-user quotas (checked atomically when the files are stored). A refusal is a `400` (or a `group_error` event on the `/stream` variant), and nothing from the refused request is kept. Hosted `url` references pass through. A bare `storageRef` is not accepted from a client: only a nested group passes references down, and only those its parent discussion can read.

Deleting a discussion (`DELETE /groups/{groupId}/conversations/{id}`) deletes its blobs, and a GDPR erasure of the owner reaches them — see [gdpr-compliance.md](https://docs.labs.ai/security-and-compliance/gdpr-compliance.md).

Each member's private conversation is granted access on its **first** turn and receives the files as `attachment_*` context; from there everything on this page applies unchanged. Later turns rely on the member's own history plus the auto-enabled `readAttachment` tool, so a member does not lose the file after phase one.

Two group-specific bounds: the per-turn cap (`eddi.attachments.max-per-turn`) applies per **member turn**, and anything dropped is reported in that member's `attachments:errors`, not in the group transcript. See [group-conversations.md → Attachments](https://docs.labs.ai/conversations-and-orchestration/group-conversations.md#attachments).

## Architecture Notes

* **No inline storage**: Attachment payloads are never stored inline in conversation memory documents. Only metadata references are persisted.
* **Transient base64**: The `base64Data` field is `transient` — it exists only during the pipeline turn. For persistence, use the upload endpoint with `IAttachmentStore`.
* **DB-agnostic**: The `IAttachmentStore` SPI supports MongoDB (GridFS) and PostgreSQL (bytea) implementations.
* **GDPR cleanup**: `IAttachmentStore.deleteByConversation()` removes all attachments when a conversation is deleted.
