> For the complete documentation index, see [llms.txt](https://docs.labs.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.labs.ai/architecture-and-concepts/architecture.md).

# Architecture Overview

[![Version](https://img.shields.io/github/v/release/labsai/EDDI?label=version\&color=blue)](https://github.com/labsai/EDDI/releases)

This document provides a comprehensive overview of EDDI's architecture, design principles, and internal workflow.

## Table of Contents

1. [Overview](#overview)
2. [What EDDI Is (and Isn't)](#what-eddi-is-and-isnt)
3. [Core Architecture](#core-architecture)
4. [The Lifecycle Pipeline](#the-lifecycle-pipeline)
5. [Conversation Flow](#conversation-flow)
6. [Agent Composition Model](#agent-composition-model)
7. [Key Components](#key-components)
8. [Technology Stack](#technology-stack)
9. [Design Patterns Used](#design-patterns-used)
10. [Performance Characteristics](#performance-characteristics)
11. [Deployment & Operations](#deployment--operations)
12. [Case Study: The Platform Operator](#case-study-the-platform-operator)
13. [Configuration Model Deep Dive](#configuration-model-deep-dive)
14. [Database Architecture](#database-architecture)
15. [Multi-Agent Orchestration](#multi-agent-orchestration)
16. [MCP Integration (Bilateral)](#mcp-integration-bilateral)
17. [Persistent User Memory](#persistent-user-memory)
18. [Agent Sync & Portability](#agent-sync--portability)
19. [Security Architecture](#security-architecture)
20. [Summary](#summary)
21. [Related Documentation](#related-documentation)

***

## Overview

E.D.D.I. (Enhanced Dialog Driven Interface) is a **multi-agent orchestration middleware** for conversational AI systems, not a standalone agent or language model. It sits between user-facing applications and multiple AI agents (LLMs like OpenAI, Claude, Gemini, or traditional REST APIs), intelligently routing requests, coordinating responses, and maintaining conversation state across agent interactions.

**Core Purpose**: Orchestrate multiple AI agents and business systems in complex conversational workflows without writing code.

## What EDDI Is (and Isn't)

* **A Multi-Agent Orchestration Middleware**: Coordinates multiple AI agents (LLMs, APIs) in complex workflows
* **An Intelligent Router**: Directs requests to appropriate agents based on patterns, rules, and context
* **A Conversation Coordinator**: Maintains stateful conversations across multiple agent interactions
* **A Configuration Engine**: Agent orchestration defined through JSON configurations, not code
* **A Middleware Service**: Acts as an intermediary that adds intelligence and control to conversation flows
* **Business System Integrator**: Connects AI agents with your existing APIs, databases, and services
* **Container-Packaged**: A Quarkus application running on the JVM, shipped as a container image (`labsai/eddi`) with health and metrics endpoints
* **Stateful**: Maintains complete conversation history and context throughout interactions

### EDDI Is Not:

* **Not a standalone LLM**: It doesn't train or run machine learning models
* **Not a chatbot platform**: It's the infrastructure that powers conversational agents
* **Not just a proxy**: It provides orchestration, state management, and complex behavior rules beyond simple API forwarding

***

## Core Architecture

### Architectural Principles

EDDI's architecture is built on several key principles:

1. **Modularity**: Every component is pluggable and replaceable
2. **Composability**: Agents are assembled from reusable workflows and extensions
3. **Asynchronous Processing**: Non-blocking I/O for handling concurrent conversations
4. **State-Driven**: All operations transform or query the conversation state
5. **Container-Ready**: Packaged as a container image, configured through environment variables, observable through health and metrics endpoints

### High-Level Architecture Diagram

```
┌─────────────────────────────────────────────────────────────┐
│                      User Application                        │
│                  (Web, Mobile, Chat Client)                  │
└────────────────────────────┬────────────────────────────────┘
                             │ HTTP/REST API
                             ▼
┌─────────────────────────────────────────────────────────────┐
│                      RestAgentEngine                           │
│          (Entry Point - JAX-RS AsyncResponse)                │
└────────────────────────────┬────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────┐
│                  ConversationCoordinator                     │
│           (Ensures Sequential Processing per                 │
│            Conversation, Concurrent Across)                  │
└────────────────────────────┬────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────┐
│                   IConversationMemory                        │
│       (Stateful Object - Complete Conversation Context)      │
└────────────────────────────┬────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────┐
│                     LifecycleManager                         │
│          (Executes Sequential Pipeline of Tasks)             │
└────────────────────────────┬────────────────────────────────┘
                             │
        ┌────────────────────┼────────────────────┐
        ▼                    ▼                    ▼
┌──────────────┐   ┌──────────────┐    ┌──────────────┐
│Input Parsing │   │Behavior Rules│    │LLM/API Calls │
│  (NLP, etc)  │   │(IF-THEN Logic│    │(LangChain4j, │
│              │   │              │    │ HTTP Calls)  │
└──────────────┘   └──────────────┘    └──────────────┘
        │                    │                    │
        └────────────────────┼────────────────────┘
                             ▼
                   ┌──────────────────┐
                   │Output Generation │
                   │  (Templating)    │
                   └──────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────┐
│             Datastore: MongoDB or PostgreSQL                 │
│     (one per deployment, selected by eddi.datastore.type)    │
│     + in-process Caffeine caches for short-lived lookups     │
└─────────────────────────────────────────────────────────────┘
```

***

## The Lifecycle Pipeline

The **Lifecycle** is EDDI's most distinctive architectural feature. Instead of hard-coded agent logic, EDDI processes every user interaction through a **configurable, sequential pipeline of tasks** called the **Lifecycle**.

### How the Lifecycle Works

1. **Pipeline Composition**: Each agent defines a sequence of `ILifecycleTask` components
2. **Sequential Execution**: Tasks execute one after another, each transforming the `IConversationMemory`
3. **Stateless Tasks**: Each task is stateless; all state resides in the memory object passed through
4. **Interruptible**: The pipeline can be stopped early based on conditions (e.g., `STOP_CONVERSATION`)

### Standard Lifecycle Tasks

A typical agent lifecycle includes these task types:

| Task Type               | Purpose                                         | Example                                   |
| ----------------------- | ----------------------------------------------- | ----------------------------------------- |
| **Input Parsing**       | Normalizes and understands user input           | Extracting entities, intents from text    |
| **Semantic Parsing**    | Uses dictionaries to parse expressions          | Matching "hello" → `greeting(hello)`      |
| **Behavior Rules**      | Evaluates IF-THEN rules to decide actions       | "If `greeting(*)` then `action(welcome)`" |
| **Property Extraction** | Extracts and stores data in conversation memory | Saving user name, preferences             |
| **HTTP Calls**          | Calls external REST APIs                        | Weather API, CRM systems                  |
| **LangChain Task**      | Invokes LLM APIs (OpenAI, Claude, etc.)         | Conversational AI responses               |
| **Output Generation**   | Formats final response using templates          | Qute templating with conversation data    |

### Lifecycle Task Interface

```java
public interface ILifecycleTask {
    TaskId getId();
    String getType();
    void execute(IConversationMemory memory, Object component)
        throws LifecycleException;
}
```

Every task receives:

* **IConversationMemory**: Complete conversation state
* **component**: Task-specific configuration/resources

***

## Conversation Flow

### Step-by-Step: User Interaction Flow

Here's what happens when a user sends a message to an EDDI agent:

#### 1. API Request

```
POST /agents/{conversationId}
Content-Type: application/json

{
  "input": "Hello, what's the weather?",
  "context": {
    "language": { "type": "string", "value": "en" }
  }
}
```

The JSON body is an `InputData`: `input` is the user's text, and `context` maps each name to a `{type, value}` entry whose `type` is `string`, `expressions`, `object` or `array`. An entry may also set `"secret": true`, which makes its value usable for this turn only and keeps it out of storage and responses. A `text/plain` body is accepted too, as the input with no context. The conversation itself was created beforehand with `POST /agents/{agentId}/start`.

#### 2. RestAgentEngine Receives Request

* Validates agent ID and environment
* Wraps response in `AsyncResponse` for non-blocking processing
* Increments metrics counters

#### 3. ConversationCoordinator Queues Message

* Ensures messages for the same conversation are processed sequentially
* Allows different conversations to process concurrently
* Prevents race conditions in conversation state

#### 4. IConversationMemory Loaded/Created

* Every turn loads the conversation's memory snapshot from the configured datastore (MongoDB or PostgreSQL) — conversation memory is not cached; only the conversation *state* is, for 30 seconds
* Includes all previous steps and the conversation properties; the context sent with this turn is added to the new step

#### 5. LifecycleManager Executes Pipeline

```
Input → Parser → Behavior Rules → API/LLM → Output → Save
```

Each task in sequence:

* Reads current conversation state
* Performs its operation (parsing, rule evaluation, API call, etc.)
* Writes results back to conversation memory
* Passes control to next task

#### 6. State Persistence

* Updated `IConversationMemory` saved to the datastore
* Cached conversation state updated
* Metrics recorded (duration, success/failure)

#### 7. Response Returned

```json
{
  "conversationState": "READY",
  "conversationOutputs": [
    {
      "actions": ["weather_response"],
      "output": [
        { "type": "text", "text": "The weather today is sunny with a high of 75°F", "delay": 0 }
      ]
    }
  ]
}
```

***

## Agent Composition Model

EDDI agents are **not monolithic**. They are **composite objects** assembled from version-controlled, reusable components.

### Hierarchy: Agent → Workflow → Extensions

```
Agent (.agent.json)
  ├─ Workflow 1 (.workflow.json)
  │   ├─ Behavior Rules Extension (.behavior.json)
  │   ├─ HTTP Calls Extension (.httpcalls.json)
  │   └─ Output Extension (.output.json)
  ├─ Workflow 2 (.workflow.json)
  │   ├─ Dictionary Extension (.dictionary.json)
  │   └─ LangChain Extension (.langchain.json)
  └─ Workflow 3 (.workflow.json)
      └─ Property Extension (.property.json)
```

### 1. Agent Level

**File**: `{agentId}.agent.json`

A agent is simply a **list of workflow references**:

```json
{
  "workflows": [
    "eddi://ai.labs.workflow/workflowstore/workflows/{workflowId}?version={version}",
    "eddi://ai.labs.workflow/workflowstore/workflows/{anotherWorkflowId}?version={version}"
  ]
}
```

Every agent version also carries a server-assigned `compatibilityGeneration`. Versions with the same generation are declared compatible, and a running conversation follows the newest deployed one from its next turn; every save is a breaking change unless the save says otherwise. See [Running conversations and new agent versions](https://docs.labs.ai/conversations-and-orchestration/deployment-management-of-agents.md#running-conversations-and-new-agent-versions).

### 2. Workflow Level

**File**: `{workflowId}.workflow.json`

A workflow is a **container of functionality** with a list of steps:

```json
{
  "workflowSteps": [
    {
      "type": "eddi://ai.labs.rules",
      "config": {
        "uri": "eddi://ai.labs.rules/rulestore/rulesets/{behaviorId}?version={version}",
        "appendActions": true
      }
    },
    {
      "type": "eddi://ai.labs.apicalls",
      "config": {
        "uri": "eddi://ai.labs.apicalls/apicallstore/apicalls/{httpCallsId}?version={version}"
      }
    }
  ]
}
```

### 3. Extension Level

**Files**: `{extensionId}.{type}.json`

Extensions are the **actual agent logic**:

#### Behavior Rules Extension

```json
{
  "behaviorGroups": [
    {
      "name": "Greetings",
      "behaviorRules": [
        {
          "name": "Welcome User",
          "conditions": [
            {
              "type": "inputmatcher",
              "configs": {
                "expressions": "greeting(*)",
                "occurrence": "currentStep"
              }
            }
          ],
          "actions": ["welcome_action"]
        }
      ]
    }
  ]
}
```

#### HTTP Calls Extension

```json
{
  "targetServerUrl": "https://api.weather.com",
  "httpCalls": [
    {
      "name": "getWeather",
      "actions": ["fetch_weather"],
      "saveResponse": true,
      "responseObjectName": "weatherResponse",
      "request": {
        "method": "GET",
        "path": "/current?location={context.userLocation}"
      },
      "postResponse": {
        "propertyInstructions": [
          {
            "name": "currentWeather",
            "fromObjectPath": "weatherResponse.temperature",
            "scope": "conversation"
          }
        ]
      }
    }
  ]
}
```

#### LangChain Extension

```json
{
  "tasks": [
    {
      "actions": ["send_to_ai"],
      "id": "openaiChat",
      "type": "openai",
      "parameters": {
        "apiKey": "${vault:openai-key}",
        "modelName": "gpt-4o",
        "systemMessage": "You are a helpful assistant",
        "logSizeLimit": "10",
        "addToOutput": "true"
      }
    }
  ]
}
```

### What Lives Where: A Decision Guide

When adding a new feature, use this guide to decide where configuration belongs:

| Question                                                  | Config Level                                                  | Example                               |
| --------------------------------------------------------- | ------------------------------------------------------------- | ------------------------------------- |
| Does it affect the entire agent across all conversations? | **Agent level** (`AgentConfiguration`)                        | `enableMemoryTools`, `a2aEnabled`     |
| Does it control how a pipeline step behaves?              | **Extension level** (e.g., `langchain.json`, `property.json`) | LLM parameters, property instructions |
| Does it define which extensions run and in what order?    | **Workflow level** (`workflow.json`)                          | Extension types and URIs              |
| Is it a user-facing runtime setting?                      | **Agent level**                                               | User memory config, audit settings    |
| Is it a tool/capability the LLM can use?                  | **Extension level** (in `langchain.json`)                     | `builtInToolsWhitelist`               |

**Rule of thumb**: If a feature is a **cross-conversation concern** (e.g., persistent memory, user preferences, GDPR compliance), it belongs at the **agent level**. If it's a **per-turn processing concern** (e.g., LLM parameters, HTTP call config), it belongs at the **extension level**.

***

## Key Components

### RestAgentEngine

**Location**: `ai.labs.eddi.engine.internal.RestAgentEngine`

**Purpose**: Main entry point for all agent interactions

**Responsibilities**:

* Receives HTTP requests via JAX-RS
* Validates agent and conversation IDs
* Handles async responses
* Records metrics
* Coordinates with `IConversationCoordinator`

### ConversationCoordinator

**Location**: `ai.labs.eddi.engine.runtime.IConversationCoordinator`, implemented by `InMemoryConversationCoordinator` and `ClusterConversationCoordinator` in `ai.labs.eddi.engine.runtime.internal` (both extend `AbstractQueuedConversationCoordinator`). `eddi.messaging.type` selects one at runtime: `in-memory` (the default, one replica) or `nats` (cluster mode — the queue additionally holds a cluster-wide lease per conversation from NATS JetStream; see [Clustering](https://docs.labs.ai/deployment-and-infrastructure/clustering.md))

**Purpose**: Ensures proper message ordering and concurrency control

**Key Feature**: Uses a queue system to guarantee that:

* Messages within the same conversation are processed sequentially
* Different conversations can be processed in parallel
* No race conditions occur in conversation state updates

### IConversationMemory

**Location**: `ai.labs.eddi.engine.memory.IConversationMemory`

**Purpose**: The stateful object representing a complete conversation

**Contains**:

* Conversation ID, agent ID, user ID
* All previous conversation steps (history)
* Current step being processed
* User properties (name, preferences, etc.)
* Context data (passed with each request)
* Actions and outputs generated

**Key Methods**:

```java
String getConversationId();
IWritableConversationStep getCurrentStep();
IConversationStepStack getPreviousSteps();
ConversationState getConversationState();
void undoLastStep();
void redoLastStep();
```

### LifecycleManager

**Location**: `ai.labs.eddi.engine.lifecycle.internal.LifecycleManager`

**Purpose**: Executes the lifecycle pipeline

**Key Method**:

```java
void executeLifecycle(
    IConversationMemory conversationMemory,
    List<String> lifecycleTaskTypes
) throws LifecycleException, ConversationStopException, ConversationPauseException
```

**How It Works**:

1. Iterates through registered `ILifecycleTask` instances
2. For each task, calls `task.execute(conversationMemory, component)`
3. Checks for interruption or stop conditions
4. Continues until all tasks complete or stop condition is met

### WorkflowConfiguration

**Location**: `ai.labs.eddi.configs.workflows.model.WorkflowConfiguration`

**Purpose**: Defines the structure of an agent workflow

**Model**:

```java
public class WorkflowConfiguration {
    private List<WorkflowStep> workflowSteps;

    public static class WorkflowStep {
        private URI type;
        private Map<String, Object> extensions;
        private Map<String, Object> config;
    }
}
```

### ToolExecutionService

**Location**: `ai.labs.eddi.modules.llm.tools.ToolExecutionService`

**Purpose**: Unified execution pipeline for all AI agent tool invocations

**Pipeline**:

```
Tool Call ──▶ Rate Limiter ──▶ Cache Check ──▶ Execute ──▶ Cost Tracker ──▶ Result
```

**Features**:

* Token-bucket rate limiting per tool (configurable per-tool or global default)
* Smart caching — deduplicates calls with identical arguments
* Cost tracking with per-conversation budgets and automatic eviction
* Security: tools that accept URLs are validated against private/internal addresses (SSRF protection via `UrlValidationUtils`)
* Security: math expressions are evaluated in a sandboxed parser (`SafeMathParser`)

See the [Security documentation](https://docs.labs.ai/security-and-compliance/security.md) for details.

### System Prompt Modifiers

**Location**: `ai.labs.eddi.modules.llm.impl`

Two services modify the system prompt before it is sent to the LLM. Both are configured per-task in the LLM configuration (`langchain.json`).

| Service                      | Purpose                                                            | Config Key             |
| ---------------------------- | ------------------------------------------------------------------ | ---------------------- |
| **`IdentityMaskingService`** | Prepends identity concealment rules (agent name, refusal patterns) | `task.identityMasking` |
| **`CounterweightService`**   | Appends behavioral safety instructions (cautious/strict presets)   | `task.counterweight`   |

**Execution order**: Identity masking → Counterweight → LLM call.

Counterweight presets are resolved from [Prompt Snippets](https://docs.labs.ai/agent-configuration/prompt-snippets-guide.md) first (`counterweight_cautious`, `counterweight_strict`), falling back to built-in defaults. This ensures admins can customize safety language without code changes.

See [LLM Integration — Behavioral Safety](https://docs.labs.ai/agent-configuration/langchain.md#behavioral-safety-counterweight--identity-masking) for configuration details.

### Attachment Storage

**Location**: `ai.labs.eddi.engine.attachments`

The attachment subsystem handles binary file storage for multimodal conversations:

| Component                                      | Purpose                                                                                                                                                                                                                                                                                                                                                                                                                    |
| ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`IAttachmentStore`**                         | Interface for storing/loading binary attachments (GridFS for MongoDB, BLOB for PostgreSQL)                                                                                                                                                                                                                                                                                                                                 |
| **`MimeValidator`**                            | Magic-byte detection (16+ formats) and declared-vs-detected MIME compatibility checking                                                                                                                                                                                                                                                                                                                                    |
| **`AttachmentForwarder`** (`modules.llm.impl`) | The single place attachments become langchain4j `Content` on the outgoing user message. Resolves bytes from any source under uniform per-file/aggregate caps, gates on `ModelCapabilityService`, and emits `ImageContent` / native `PdfFileContent` / `AudioContent` / inlined text as the model allows — see [attachments-guide.md](https://docs.labs.ai/agent-configuration/attachments-guide.md#llm-multimodal-support) |

***

## Technology Stack

### Core Framework

* **Quarkus**: Java application framework (the version is pinned in `pom.xml`)
  * Build-time CDI wiring and configuration
  * Built-in observability (metrics, health checks)

### Language & Runtime

* **Java 25** on the JVM. The published image (`src/main/docker/Dockerfile`) is a JVM image on a Red Hat UBI 10 OpenJDK 25 runtime
* **No native image.** EDDI does not ship or support a GraalVM native executable. `pom.xml` still carries a `native` Maven profile, but native compilation is planned work (`planning/native-image-migration.md`), not a supported build

### Dependency Injection

* **CDI (Contexts and Dependency Injection)**: Jakarta EE standard
* **@ApplicationScoped, @Inject**: Clean, testable component wiring

### REST Framework

* **JAX-RS**: Jakarta REST API standard
* **AsyncResponse**: Non-blocking, scalable request handling
* **Jackson**: JSON serialization and deserialization (`quarkus-rest-jackson`; Jackson 2.x, managed by the Quarkus BOM)

### Database (DB-Agnostic)

* **MongoDB 6.0+** (default): Document store for agent configurations and conversation logs
* **PostgreSQL** (alternative): JDBC + JSONB storage, selected by starting with the `postgres` Quarkus profile (`QUARKUS_PROFILE=postgres`), which sets `eddi.datastore.type=postgres` and activates the datasource
* One datastore per deployment — the two are alternatives, not a combination
* Both backends support:
  * Agent, workflow, and extension configuration storage
  * Conversation history persistence
  * Version control of agent components
  * Automatic schema migration on startup

### Caching

* **Caffeine**: In-process cache (replaced Infinispan in v6)
  * Caches short-lived lookups: conversation *state* (30-second TTL), compiled templates, built LLM model instances, prompt snippets, resolved secrets and similar. Conversation memory itself is read from the datastore on every turn
  * Size limits per cache name (`CacheFactory`); caches are local to each instance
  * Declared explicitly in `pom.xml` (version managed by the Quarkus BOM) alongside `quarkus-cache`

### LLM Integration

* **LangChain4j**: Java library for LLM orchestration
  * Unified interface to multiple LLM providers
  * Supports OpenAI, Claude, Gemini, Ollama, Hugging Face, etc.
  * Handles chat message formatting, streaming, tool calling

### Observability

* **Micrometer**: Metrics collection
* **Prometheus**: Metrics exposition
* **Kubernetes Probes**: Liveness and readiness endpoints

### Security

* **OpenID Connect** (`quarkus-oidc`): bearer-token validation and role checks, off until `quarkus.oidc.tenant-enabled=true`
* **Keycloak**: the identity provider EDDI's shipped realm and manifests are built for; any OIDC provider can issue the tokens

### Templating

* **Qute**: Output templating engine
  * Dynamic output generation
  * Access to conversation memory in templates
  * Expression language support

***

## Design Patterns Used

### 1. Strategy Pattern

* **Where**: Lifecycle tasks
* **Why**: Different behaviors (parsing, rules, API calls) implement the same `ILifecycleTask` interface

### 2. Chain of Responsibility

* **Where**: Lifecycle pipeline
* **Why**: Each task processes the memory object and passes it to the next task

### 3. Composite Pattern

* **Where**: Agent composition (Agent → Workflows → Extensions)
* **Why**: Agents are built from hierarchical, reusable components

### 4. Repository Pattern

* **Where**: Data access (stores: agentstore, workflowstore, etc.)
* **Why**: Abstracts data persistence from business logic

### 5. Factory Pattern

* **Where**: `IAgentFactory`
* **Why**: Complex agent instantiation from multiple workflows and configurations

### 6. Coordinator Pattern

* **Where**: `ConversationCoordinator`
* **Why**: Manages concurrent access to shared conversation state

***

## Performance Characteristics

EDDI publishes no benchmark figures, so measure on your own hardware and agents. The [load-test harness](https://github.com/labsai/EDDI/tree/main/scripts/loadtest/README.md) does that reproducibly against a mock LLM: turns per second, EDDI's own latency on top of the model, time to first token over SSE and `/v1`, multi-node scaling, group discussions and database operations per turn, rendered as one report. What can be said from the design:

### Startup Time

* EDDI runs on the JVM only (there is no native build). Startup is dominated by JVM start, Quarkus boot and the datastore connection, plus any startup migrations

### Memory Footprint

* A JVM process; size the heap for the number of concurrent conversations, the attachment and request-body limits, and any in-process models (`jlama`)

### Request Latency

* **Without an LLM**: a turn costs a memory load and a save against the datastore, plus the pipeline's own work (parsing, rules, any HTTP calls)
* **With an LLM**: dominated by the provider's response time, and by every tool round-trip the model makes

### Scalability

* **Vertical**: Handles thousands of concurrent conversations per instance
* **Horizontal**: with `eddi.messaging.type=nats`, any number of replicas behind a round-robin load balancer, coordinated through NATS JetStream (conversation leases with fencing, shared KV state, cache-invalidation events). With the default `in-memory` messaging EDDI runs one replica. See [Clustering](https://docs.labs.ai/deployment-and-infrastructure/clustering.md)
* **Bottleneck**: the database becomes the bottleneck; use MongoDB replica sets and sharding, or a replicated PostgreSQL

***

## Deployment & Operations

### Containerization

* Official Docker images: `labsai/eddi`, built from `src/main/docker/Dockerfile` on a digest-pinned Red Hat UBI 10 OpenJDK 25 runtime base
* Two-stage build (a `docs` stage and the runtime stage)

### Orchestration

* Kubernetes manifests (`k8s/`) and a Helm chart (`helm/`) ship in the repository — see [Kubernetes](https://docs.labs.ai/deployment-and-infrastructure/kubernetes.md)
* OpenShift: each stable release's image is submitted for Red Hat container certification — see [Red Hat OpenShift](https://docs.labs.ai/deployment-and-infrastructure/redhat-openshift.md)
* Liveness and readiness health checks built in

### Configuration

* Externalized configuration via environment variables
* ConfigMaps and Secrets support
* No rebuild needed for configuration changes

### Observability

* Prometheus metrics endpoint: `/q/metrics`
* Health checks: `/q/health/live`, `/q/health/ready`
* Conversation turns log with `agentId`, `conversationId`, `userId` and `environment` in the logging MDC (`ContextLogger`)
* Per-task OpenTelemetry tracing — see [Monitoring](https://docs.labs.ai/deployment-and-infrastructure/monitoring-guide.md)

***

## Case Study: The Platform Operator

The **Platform Operator** is a meta-agent that demonstrates EDDI's architecture in action: an agent that reads and operates the deployment it runs inside — including creating other agents.

It is provisioned by EDDI-Manager (at `/manage/operator`) through `POST /administration/agents/setup-api`, which is the same OpenAPI-to-agent path any user can call.

### How It Works

1. **Tool generation**: the Manager fetches EDDI's own OpenAPI spec and passes it to `setup-api` with an endpoint allow-list. `McpApiToolBuilder` turns each allow-listed operation into an `apicalls` tool, named by its `operationId` (falling back to a `method_path` slug) and grouped by its first OpenAPI tag. An allow-listed endpoint whose tool could not send an input it requires — a request body that is neither JSON nor plain text, or a required header parameter — is skipped by the generator and reported as `resources.unusableEndpoints`; activation also checks the spec itself *before* creating anything and refuses, naming the endpoints, exactly as it does for an endpoint the spec lacks. (A `text/plain` body such as `updateResourceUri`'s URI is supported — see [MCP Server](https://docs.labs.ai/protocols-and-integration/mcp-server.md#how-create_api_agent-builds-a-write-tools-body).) A spec is third-party input, and its names end up in request templates: a parameter whose name is a template namespace (`properties`, `memory`, `vars`, ...) becomes `properties_2` and so on, and the spec's server URL is used as literal text — declared server variables take their defaults, any other `{...}` in it is refused (pass `apiBaseUrl` instead).
2. **Provisioning**: `AgentSetupService.createApiAgent` creates the parser, behaviour rules, one `apicalls` config per group, LLM config, workflow and agent — in that order, with compensating deletes if any step fails.
3. **Gating**: the agent is created *with* its `hitlConfig` on v1, so the approval gate cannot be bypassed by redeploying an earlier version. The gate requires approval by HTTP *method* — `http.post:*`, `http.put:*`, `http.patch:*`, `http.delete:*` — and exempts `http.get:*`. Gating by method rather than by an enumerated list of tool names is what keeps it fail-safe: a write endpoint granted later is covered the moment it exists, with nobody having to remember to add its name.
4. **Operation**: the model calls a tool; `ToolApprovalGate` classifies it; a write pauses the conversation (`hitlPauseType: "TOOL_CALL"`) until a human approves the *resolved request*, which is fingerprinted at gate time and re-checked before execution.
5. **Self-modification**: on approval, the tool call reaches EDDI's REST API and the new agent configuration is written.
6. **Editing, not echoing**: a stored configuration holds credential *references* (`${vault:…}`), and an API call resolves those before it sends — so an edit that echoed a document back would write the plaintext secret, and is refused. The operator edits LLM, HTTP-call and MCP-call configurations with `PATCH` (RFC 6902) and sends only what changes; see [Editing a configuration that holds a reference](https://docs.labs.ai/security-and-compliance/secrets-vault.md#editing-a-configuration-that-holds-a-reference-patch-not-echo). The Manager refuses any LLM-config patch that touches a task's `toolApprovals`, the same way it refuses a `PUT` that carries one.

### Building an operator agent for your own API

An application can build an operator for *its* API the same way, with EDDI's converter rather than a script of its own:

* **Per-user token.** With EDDI's OIDC off, pass `"apiAuth": "Bearer ${context:userToken}"`; each turn sends the user's token as secret context and it is resolved only into that header, only towards the generated calls' own host, failing closed when absent (see [Per-turn credentials from secret context](https://docs.labs.ai/agent-configuration/httpcalls.md#per-turn-credentials-from-secret-context)).
* **Generate only.** `"dryRun": true` on `setup-api` returns the apicalls configurations, the LLM configuration with its tool list, the workflow, the agent with its gate, `unusableEndpoints` and `unmatchedEndpoints`, and writes nothing (`200`, action `api_agent_dry_run`; a plaintext `apiKey`/`apiAuth` is shown as a placeholder, never echoed). It cannot read the vault: whether a `vaultKeyName` exists or holds the same value is decided when the agent is created, and when that name comes with a plaintext `apiKey` the destination rule is listed under `resources.warnings` rather than enforced, because an existing key would be reused as stored.
* **Re-sync.** `POST /administration/agents/{agentId}/setup-api/sync` with the regenerated spec regenerates only the apicalls configurations setup stamped with `openApiOrigin`, rewrites the LLM task's `tools` list and the generated API summary at the end of its prompt, keeps everything else (prompt, model, `hitlConfig`, `toolQuotas`, `responseSchema`, hand-written tools), and saves the agent as a **compatible** version, so running conversations move to it once deployed. It reports added, changed and removed tools; a removed tool still named by `toolQuotas`, a `toolApprovals` exemption or a tool-result guardrail exemption is refused with `400`, each reference listed, nothing written. The request's `requireApproval` patterns are added to the gate, never removed, and its exemptions are ignored. An agent created before the marker existed is refused with how to add `"openApiOrigin": {"group": "<tag>", "authHeader": "Authorization"}` to its generated configurations. `"dryRun": true` reports the diff without writing.

Neither is part of the Platform Operator's own toolset: `dryRun` is hidden from the OpenAPI schema its tools are generated from, and the sync endpoint is in no operator scope (asserted in `tool-scopes.test.ts`).

### Keeping It Up to Date

Everything provisioning bakes into the operator — its instructions, its endpoint allow-list and its approval gate — is a snapshot. Upgrading EDDI upgrades the Manager, not an operator that is already running, so an operator activated on an older release keeps its older instructions and tools until it is upgraded.

The Manager tracks this with a **provisioning revision** (`ui/manager/src/lib/operator/operator-revision.json`), recorded in the operator's configuration at every activation. When the running operator is older than the Manager:

* **In the Manager**, the operator page shows an *Upgrade* banner listing the tools the upgrade adds and removes; the docked operator drawer and the dashboard show a short hint; and the browser console logs one warning.
* **In the server log**, EDDI logs a `WARN` at startup (`[OPERATOR] The Platform Operator on this deployment was set up by an older Manager…`), so it is visible without opening the UI. The backend reads the same revision file from its classpath.

*Upgrade* rebuilds the operator with the same model, key, environment and access settings. Like *Reconfigure*, it is a replacement: the new operator passes every activation check before the old one is removed. The open operator chat ends, and conversations with the old operator no longer appear in its history. Instructions the admin never edited are replaced with the new default; edited ones are kept unless the admin chooses the new default. An operator whose model key was entered as plain text (never stored), or whose model-server address was not recorded, opens the activation form prefilled instead.

### A restricted model key, and operators the Manager did not register

**Activation creates, grants, then deploys.** The operator is created with `deploy: false` and given the descriptor marker `[eddi-platform-operator]`. Then the [deploy preflight](https://docs.labs.ai/security-and-compliance/secrets-vault.md#preflight-before-deploying) runs against it. If the model key is a vault secret restricted to named agents, the activation dialog says so before anything is created. After creating the agent, it asks the administrator to add this agent to the key's grant (dry run, then append), and only then deploys. A grant that is refused or cancelled keeps that one agent: not deployed, not registered. The next attempt reuses it as long as the settings are unchanged, so retrying never leaves a second operator behind. Any other deploy failure removes the agent it just created.

**The operator screen also finds operator agents it did not register.** Normally the screen knows the operator from the `platform.operator` global variable. That variable is missing, or points elsewhere, when the operator was created by the setup API or MCP, imported from another instance, or left behind by an older failed activation. In those cases the screen lists the operator agents it finds, using one descriptor listing:

* an agent whose descriptor carries the marker counts as an operator;
* an older agent named "operator" counts only if its stored approval gate is complete and one of its HTTP toolsets targets this instance.

Each listed agent shows its deployment status and, for a refused deploy, the reason. The actions are:

* **Adopt** writes `platform.operator` from the agent's own configuration, after verifying its gate.
* **Fix grant and deploy** opens the grant dialog.
* **Remove** undeploys and deletes the agent.

The reverse case gets a banner: `platform.operator` names an agent that no longer exists, or one that is enabled but not deployed.

### Key Insight

The operator isn't special code—it's a **regular EDDI agent** that uses:

* An LLM task with tools generated from an ordinary OpenAPI spec
* HTTP Calls to invoke EDDI's REST API
* The HITL approval gate to keep every write under human control
* Behavior rules and output templates like any other agent

This demonstrates EDDI's power: **the same architecture that powers conversational agents can orchestrate complex, multi-step workflows**, even self-modifying the system itself — and the deterministic governance layer (Pillar 2) is what makes that safe rather than reckless.

**See** [**Human-in-the-Loop**](https://docs.labs.ai/conversations-and-orchestration/hitl.md) **for the approval gate, and** [**HTTP Calls**](https://docs.labs.ai/agent-configuration/httpcalls.md) **for how the generated tools are executed.**

***

## Configuration Model Deep Dive

EDDI's configuration model is a 4-level tree:

```mermaid
graph TD
    Agent["🤖 Agent (.agent.json)"] --> P1["📦 Workflow 1"]
    Agent --> P2["📦 Workflow 2"]
    Agent --> PN["📦 Workflow N"]
    P1 --> Parser1["🔤 Parser (dictionaries)"]
    P1 --> Behavior1["🧠 Behavior Rules"]
    P1 --> Property1["📝 Property Setter"]
    P1 --> Output1["💬 Output Templates"]
    P2 --> Parser2["🔤 Parser"]
    P2 --> Behavior2["🧠 Behavior Rules"]
    P2 --> HttpCalls2["🌐 HTTP Calls"]
    P2 --> Property2["📝 Property Setter"]
    P2 --> Output2["💬 Output Templates"]
```

### Agent → Workflows → Extensions

| Level          | Purpose                                                                                                                        |
| -------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| **Agent**      | List of workflow URIs (plus channel integrations, configured separately). The top-level container.                             |
| **Workflow**   | Ordered list of workflow extensions — each extension = one lifecycle task type. **Order matters**: tasks execute sequentially. |
| **Extension**  | The actual configuration that drives each `ILifecycleTask`. Referenced by URI from the workflow.                               |
| **Descriptor** | Metadata (name, description, timestamps) for any resource. Not functional, purely for UI/management.                           |

### URI-Based References

Every resource references its dependencies by `eddi://` URI:

```
Agent → Workflow: "eddi://ai.labs.workflow/workflowstore/workflows/{id}?version=1"
Workflow → Rules: "eddi://ai.labs.rules/rulestore/rulesets/{id}?version=1"
Workflow → ApiCalls: "eddi://ai.labs.apicalls/apicallstore/apicalls/{id}?version=1"
Workflow → LLM: "eddi://ai.labs.llm/llmstore/llms/{id}?version=1"
```

### Extension Types & Their Pipeline Role

Each workflow runs its extensions in order: **Parser → Behavior → Property → HttpCalls → LLM → Output** (typical order).

| Extension Type       | Input                                     | Output                                               | Key Feature                                                                                                                                                                               |
| -------------------- | ----------------------------------------- | ---------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Parser**           | Raw user text                             | Expressions (semantic representation)                | Dictionaries, normalizer and corrections; `appendExpressions`, `includeUnused`, `includeUnknown`                                                                                          |
| **Behavior Rules**   | Actions and expressions                   | New actions that drive subsequent tasks              | IF-THEN condition engine — the routing logic; `expressionsAsActions: true` (in the behavior step's `config`, alongside `appendActions` and `uri`) makes parser expressions become actions |
| **Property Setter**  | Current memory data                       | Stored properties (conversation-scoped or long-term) | Slot-filling using `{memory.current.input}` templates                                                                                                                                     |
| **HTTP Calls**       | Actions, template variables               | Response data stored in memory                       | Pre/post request property instructions, retry support                                                                                                                                     |
| **LLM**              | Conversation memory, system prompt, tools | LLM response text                                    | Legacy chat (simple) or Agent mode (tool-calling loop)                                                                                                                                    |
| **Output Templates** | Actions from current step                 | Text responses + quickReplies                        | Template variables, response variation via `valueAlternatives`                                                                                                                            |

### Parser & Expression System

The parser uses a recursive expression model with Prolog heritage:

```
greeting                         → simple expression (no args)
greeting(hello)                  → expression with sub-expression
intent(weather, location(NYC))   → nested sub-expressions
*                                → wildcard (matches anything)
```

**QuickReply → Expression → Action flow**: When a user clicks a quickReply, the parser matches the text against the previous step's quickReply `value` fields, extracts the corresponding `expressions`, and (if `expressionsAsActions` is enabled) converts them to actions that drive behavior rules.

### Available NLP Extensions

| Type                 | Extensions                                                                                                                                                        |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Dictionaries** (7) | `RegularDictionary`, `IntegerDictionary`, `DecimalDictionary`, `EmailDictionary`, `TimeExpressionDictionary`, `OrdinalNumbersDictionary`, `PunctuationDictionary` |
| **Normalizers** (4)  | `ContractedWordNormalizer`, `ConvertSpecialCharacterNormalizer`, `PunctuationNormalizer`, `RemoveUndefinedCharacterNormalizer`                                    |
| **Corrections** (3)  | `DamerauLevenshteinCorrection`, `MergedTermsCorrection`, `PhoneticCorrection`                                                                                     |

***

## Database Architecture

EDDI's data layer is fully DB-agnostic via the `IResourceStorageFactory` SPI:

```
REST API → Store Interface (IResourceStore<T>)
         → HistorizedResourceStore<T> (versioning, history, soft-delete)
         → IResourceStorage<T> (SPI — Storage Provider Interface)
         ├── MongoResourceStorage<T> (MongoDB implementation)
         └── PostgresResourceStorage<T> (PostgreSQL + JSONB implementation)
```

Switching databases is configuration, not code. MongoDB is the default; PostgreSQL is selected with the `postgres` profile, which sets `eddi.datastore.type=postgres` and activates the JDBC datasource (`quarkus.datasource.active` is `false` outside that profile, so setting `eddi.datastore.type` alone is not enough):

```properties
# environment: QUARKUS_PROFILE=postgres
%postgres.quarkus.datasource.jdbc.url=jdbc:postgresql://localhost:5432/eddi
%postgres.quarkus.datasource.username=eddi
%postgres.quarkus.datasource.password=eddi
```

See [Configuration Reference → Settings operators set first](https://docs.labs.ai/deployment-and-infrastructure/configuration-reference.md#settings-operators-set-first).

***

## Multi-Agent Orchestration

Beyond single-agent conversations, EDDI supports **group conversations** — structured multi-agent discussions where multiple agents collaborate on a question under the governance of a moderator agent.

A `GroupConversationService` orchestrates discussions through configurable phases. Each participating agent runs through its normal lifecycle pipeline — agents are group-unaware by design. The moderator serializes all contributions, preventing concurrent writes to shared state.

**Key capabilities:**

* **7 built-in discussion styles**: Round Table, Peer Review, Devil's Advocate, Delphi, Debate, Task Force, and Negotiation — each with distinct phase flows and turn-taking rules. Task Force uses a 4-phase pipeline (PLAN→EXECUTE→VERIFY→SYNTHESIS) for structured task decomposition and parallel execution; Negotiation trades rather than wins (positions & interests → opening proposals → bargaining with a concession ledger → arbitration → synthesis)
* **Custom phases**: Define your own phase sequences with configurable context scopes (independent, full transcript, anonymous, own-feedback-only)
* **Group-of-groups**: Members can themselves be groups, enabling hierarchical multi-agent composition with configurable depth limits
* **Fault tolerance**: Per-agent timeouts, configurable failure policies (skip, retry, abort), and graceful degradation when members are unavailable
* **Dynamic agents**: Agents can create, recruit, delegate to, and teardown new agents at runtime during discussions, with configurable guardrails (provider/model whitelists, per-discussion caps, lifecycle policies)

See [Group Conversations](https://docs.labs.ai/conversations-and-orchestration/group-conversations.md) for full configuration reference, and [A2A Protocol](https://docs.labs.ai/protocols-and-integration/a2a-protocol.md) for peer-to-peer agent communication.

***

## MCP Integration (Bilateral)

EDDI provides **bilateral** Model Context Protocol (MCP) integration — it is both an MCP Server and an MCP Client simultaneously.

**As MCP Server:** EDDI exposes its full API surface (conversations, administration, diagnostics, scheduling, group discussions) as MCP tools. This enables AI assistants (Claude Desktop, IDE plugins, custom MCP clients) to interact with deployed agents and manage the platform programmatically. Documentation is also exposed as MCP resources (`eddi://docs/{name}`).

**As MCP Client:** Individual agents can consume external MCP servers as tool providers. MCP server connections are configured as `mcpcalls` workflow extensions (versioned configuration resources, the MCP equivalent of `httpcalls`), support vault-based API key resolution, and are subject to the same rate limiting, caching, and cost tracking as built-in tools. The LLM auto-discovers them from the workflow (`enableMcpCallTools`, default `true`); behavior rules can also trigger specific MCP tools deterministically. Failed MCP connections degrade gracefully — they never kill the pipeline.

See [MCP Server](https://docs.labs.ai/protocols-and-integration/mcp-server.md) for the full tool reference and client configuration.

***

## Persistent User Memory

EDDI's memory model extends beyond single conversations. The `IUserMemoryStore` provides persistent key-value memory scoped per user, per agent, with visibility controls (`self`, `group`, `global`).

**How it integrates with the pipeline:**

* At **conversation init**, visible user memories are loaded as `longTerm` properties and made available in all templates via `{properties.key}`
* During the pipeline, the LLM can autonomously store and recall facts using built-in memory tools (when enabled)
* At **conversation teardown**, `longTerm` properties are persisted back to the user memory store
* **Background consolidation** (the "Dream" service) performs stale pruning, contradiction detection, and optional LLM-driven summarization. It runs on the same cluster-aware schedule machinery as every other background job — a `ScheduleConfiguration` whose `metadata` carries `{"dreamType": "dream_consolidation"}` is claimed by `SchedulePollerService` and dispatched by `ScheduleFireExecutor` to `DreamService`, which reads the agent's `userMemoryConfig.dream` block and runs one cycle. The target agent and user come from the schedule's **top-level** `agentId` / `agentVersion` / `userId` fields — `metadata` carries only the `dreamType` marker. Spend is bounded per cycle by `dream.maxCostPerRun` (US dollars), and because Dream has no parent LLM task to inherit credentials from, its model credentials come from `dream.parameters` (which resolves `${vault:…}` and `${vars:…}` like any LLM task's parameters). A cycle that cannot run — no `userId`, dream disabled on the agent, or a failing LLM call — is logged at ERROR and marked FAILED on the fire log, so it retries with backoff and dead-letters rather than silently doing nothing

**Creating a Dream schedule** — use the raw REST body, `POST /schedulestore/schedules`, with the cron expression from `dream.schedule`:

```json
{
  "name": "nightly dream — alice",
  "agentId": "5a8b1c2d3e4f5a6b7c8d9e0f",
  "agentVersion": 0,
  "triggerType": "CRON",
  "cronExpression": "0 3 * * *",
  "timeZone": "UTC",
  "userId": "alice",
  "metadata": { "dreamType": "dream_consolidation" },
  "enabled": true
}
```

Three things this body does that are easy to get wrong:

* **The `create_schedule` MCP tool cannot do this.** Its arguments (`agentId`, `triggerType`, `cron`, `heartbeatIntervalSeconds`, `message`, `name`, `timeZone`, `conversationStrategy`, `userId`, `environment`) contain no `metadata`, so a schedule created that way has `metadata == null`, `DreamService.isDreamSchedule(…)` returns `false`, and the schedule fires an ordinary chat turn against the agent on the dream cron forever — logged COMPLETED, consolidating nothing. REST is the only route that produces a working Dream schedule today.
* **`message` is optional and ignored.** `RestScheduleStore.validateSchedule` requires a non-blank `message` for ordinary CRON schedules but exempts a Dream schedule (`metadata.dreamType=dream_consolidation`): the Dream fast-path bypasses `say()` entirely, so nothing would read it.
* **`userId` must name the real user whose memories are consolidated.** A Dream schedule with no `userId`, a blank one, or the `system:scheduler` placeholder is rejected with 400 when it is created or updated.

Memory visibility is enforced at the storage level — agents can only see memories matching their visibility scope, preventing cross-tenant memory leaks.

See [Persistent User Memory](https://docs.labs.ai/architecture-and-concepts/user-memory.md) for configuration, LLM tools, REST API, and the Dream consolidation service.

***

## Agent Sync & Portability

Agent configurations are fully portable — exportable, importable, and synchronizable between EDDI instances.

**The sync pipeline:**

```
IResourceSource (transport) → StructuralMatcher (analysis) → UpgradeExecutor (write)
```

1. **Transport abstraction**: `IResourceSource` abstracts the source — either a ZIP file (`ZipResourceSource`) or a live remote instance (`RemoteApiResourceSource`)
2. **Structural matching**: Resources are paired deterministically by position, type, and name — not by ID. This works even for independently-created agents
3. **Content sync**: The `UpgradeExecutor` updates target resources in-place, preserving IDs and URI references. Version numbers increment; no broken links
4. **Preview before apply**: All sync operations support a preview step showing exactly what will be created, updated, or skipped

Agent ZIP exports automatically scrub secrets before packaging to prevent credential leaks during transfer.

See [Agent Sync Architecture](https://docs.labs.ai/reference/agent-sync-architecture.md) for the matching algorithm and data flow, and [Agent Sync Guide](https://docs.labs.ai/conversations-and-orchestration/agent-sync-guide.md) for REST API usage.

***

## Security Architecture

EDDI enforces security at multiple layers so individual failures don't result in full compromise.

### SSRF Prevention (3-Layer Model)

Outbound HTTP from LLM tools is the primary attack surface. Three layers prevent Server-Side Request Forgery:

| Layer                            | Component                                     | What It Blocks                                                                                                                                                                                                                                                                                                                                                                                     |
| -------------------------------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Layer 1: URL Validation**      | `UrlValidationUtils.validateUrl()`            | Private IPs (`10.x`, `172.16-31.x`, `192.168.x`), loopback (`127.x`, `::1`), link-local (`169.254.x`, `fe80::`), cloud metadata (`169.254.169.254`), non-HTTP schemes (`file://`, `ftp://`), hostnames resolving to private IPs                                                                                                                                                                    |
| **Layer 2: Redirect Validation** | `OutboundHttp` (`engine/httpclient/outbound`) | Redirects are never followed by the HTTP engine; the facade follows them hop by hop, validates each hop against Layer 1 rules, and on a cross-origin hop keeps only an allow-list of headers (`Accept`, `Accept-Encoding`, `Accept-Language`, `User-Agent`, `Content-Type`, `Content-Language`, `Cache-Control`, `Pragma`) and strips every other one, including the trace context. Maximum 5 hops |
| **Layer 3: Network Policy**      | Kubernetes `NetworkPolicy`                    | Restricts egress at the cluster level (optional, operator-configured)                                                                                                                                                                                                                                                                                                                              |

**Usage pattern:**

```java
@Inject OutboundHttp http;

// For user-controlled URLs (LLM tools, web scraping): SsrfPolicy.STRICT, the
// DEFAULT profile's default, validates the initial URL and every redirect hop.
http.send(OutboundRequest.get(Profile.DEFAULT, uri).build(),
        BodyMode.string(1_000_000, BodyMode.OnOverflow.TRUNCATE));

// For config-controlled URLs (known APIs): the initial URL is trusted,
// redirect hops are still validated.
http.send(OutboundRequest.get(Profile.DEFAULT, uri).ssrf(SsrfPolicy.HOPS_ONLY).build(), mode);
```

**DNS Rebinding:** EDDI validates hostnames at request time, and the HTTP client resolves the host again to connect — a TOCTOU (time-of-check-time-of-use) gap that the JVM's positive DNS cache normally closes. Where the cache entry expires in between, the httpcalls client checks the address it actually connected to before sending, `OutboundHttp` re-checks after the response headers and discards the response of a target that turned private, and MCP transports re-validate on every request. What remains per path is listed in [security.md → DNS rebinding](https://docs.labs.ai/security-and-compliance/security.md#dns-rebinding).

### Vault Encryption Model

Secrets (API keys, credentials) never appear as plaintext in the database:

```
Master Key (env var EDDI_VAULT_MASTER_KEY)
  └→ PBKDF2-HMAC-SHA256 (600k iterations, per-deployment salt via VaultSaltManager)
       └→ Key Encryption Key (KEK)
            └→ AES-256-GCM encrypt/decrypt
                 └→ Data Encryption Key (DEK, one per tenant, in generations)
                      └→ AES-256-GCM encrypt/decrypt
                           └→ Secret plaintext
```

* **Per-deployment salt**: `VaultSaltManager` generates and stores a unique 16-byte salt per EDDI instance
* **Envelope encryption**: Each tenant has its own DEK, and every secret in that tenant is sealed under it (AES-256-GCM, with tenant, key name and DEK id bound in as associated data). Rotating the master key re-wraps every tenant's DEKs without touching any secret's ciphertext
* **DEK rotation**: per tenant and additive — it installs a new DEK generation and re-seals the tenant's secrets onto it; each ciphertext records the generation that sealed it, so a rotation stopped halfway leaves everything readable
* **Export scrubbing**: Agent export/sync automatically strips secrets from ZIP files

### Cryptographic Agent Identity

EDDI agents can sign their inter-agent messages using Ed25519 digital signatures. This protects multi-agent group conversations against identity spoofing and message tampering. It is not non-repudiation: the private keys are held by the deployment's own vault, so whoever controls the vault master key can sign as any agent.

**Key lifecycle:**

1. **Provisioning**: an admin generates an agent's key with `POST /agentstore/agents/{id}/signing-keys?version={current}` (`eddi-admin` only, since whoever can mint a key can sign as the agent; in the Manager, **Generate signing key** under Security & Identity). EDDI creates an Ed25519 key pair, stores the private key in the Secrets Vault under `agent-signing-key:{agentId}:v{n}` (Base64 PKCS#8, never in the agent JSON, and never returned), and appends the public key (Base64 X.509) to `identity.keys[]` as version `n`, one above the highest declared version, in a new compatible agent version, answered 200 with `Location` naming it (the agent's descriptor moves with it). Versions run from 1 to 1 000 000. The vault write never replaces an existing entry, and the key is taken back out of the vault if the agent was certainly not saved — after a failure whose outcome is unknown (a lost acknowledgement), it is taken out only if the agent's current version does not declare it. Nothing is generated when an agent is created, and the secrets API cannot write these entries (its key names allow no `:`). Duplicating an agent, or importing a ZIP that creates a new one, gives the new agent no identity and turns its signing off, because the original's keys are stored under the original's id
2. **Turning signing on**: `identity` is engine-owned. An agent save (`PUT`, an import merge, a sync, a cascade, an autonomy change) keeps the stored `identity` whatever the body says, and a created agent gets none, exactly as `dynamicOrigin` is kept; only the two admin endpoints change keys, since whoever could write `identity.keys[]` could plant a key whose private half they hold, undo a revocation, or shrink the list a permanent delete sizes its vault sweep from. A save with `security.signInterAgentMessages=true` is refused (400) unless the agent can sign: every declared key that is not revoked is well-formed (a version from 1 to 1 000 000, declared once, an Ed25519 public key — peers trust every declared key, not only the newest), the key version that would sign (the highest well-formed one) has a public key, is not revoked and is inside its `validFromMs`/`validUntilMs` window, and the vault holds the private key that signs as it. A vault that cannot be asked is a 503. Creating an agent with signing on is refused for the same reason, since a new agent has no key yet. `requirePeerVerification` needs no key of the agent's own. The legacy unversioned key (`identity.publicKey` with `agent-signing-key:{agentId}`) is still read and checked, but no longer written
3. **Key rotation and revocation**: generating a key again adds the next version; the agent signs with it from then on, and entries signed with an older version still verify against that version, which stays in `identity.keys[]`. To retire a leaked key, generate its replacement and then revoke it with `POST /agentstore/agents/{id}/signing-keys/{keyVersion}/revoke?version={current}` (`eddi-admin` only; **Revoke** next to the key in the Manager). That marks every entry of that version `revoked: true` in a new compatible agent version and deletes its private key from the vault; `keyVersion` 0 removes the legacy `identity.publicKey` and its vault entry. A revoked key is valid at no time (`AgentPublicKey.isValidAt`), whatever its window and whenever an entry claims to have been signed. Revoking the newest key stops the agent signing — it does not fall back to an older key — until a new one is generated. A key's `validFromMs`/`validUntilMs` window limits its honest use, but it is checked against the time the signer puts in the entry, so it does not stop a leaked key: revoke it. Group messages are signed with a new key at once, without a redeploy. Audit entries are signed with the newest key of the agent version that produced them, stored as `v<n>:<signature>`, and checked by `/auditstore/verify` against that same version; a key revoked since vouches for none of them, and a signature written by 6.7.0 or earlier (Base64 with no `v<n>:`) is checked against the legacy `identity.publicKey`. How key windows and revocation apply to them is in [Audit Ledger → Agent signatures](https://docs.labs.ai/security-and-compliance/audit-ledger.md#agent-signatures)
4. **Caching**: each node keeps the vault's answer for an agent's key for 60 s — the key, or that there is none — so a key added, replaced or removed in the vault is picked up within a minute; a key generated on a node is used there at once. Permanently deleting an agent deletes its keys from the vault and drops them from that node's cache at once
5. **Signing**: when `security.signInterAgentMessages=true`, every group-transcript entry the agent writes is signed by `GroupSigningGuard`. The signed `SignedEnvelope` names the agent as sender and the group as recipient, carries a UUID nonce and an epoch timestamp, and its payload binds the entry to its place: the group conversation id, the round (also stored on the entry as `signatureRound`), the phase index and name, the entry type, the content, the speaker's display name, the target agent and the error reason (if any) and the key version. Its canonical JSON form (keys sorted, no insignificant whitespace, via `JacksonCanonicalizer`; not a full RFC 8785 implementation) is signed with Ed25519. Unpaired UTF-16 surrogates in the text become U+FFFD before signing, so what is stored is what was signed. A reply the engine records under another type (a `DISSENT`, a `BID`) is signed again as that type
6. **Self-verification**: immediately after signing, the signature is checked against the public key the agent declares for that version. If the agent cannot sign (no declared key, the key revoked or outside its window, a vault failure, a mismatch), the entry goes out unsigned and the failure is logged at ERROR; peers that require verification then withhold it
7. **Replay protection**: a signed entry copied into another conversation, or moved to another phase, round or type, no longer verifies, because the signature covers all four; and an entry whose signed round does not match where it sits (rounds only grow along the transcript, and the current round starts at `roundStartTranscriptIndex`) is withheld. Moving an entry between repeats of one phase within a round is not detected. A copy within the same phase does verify, so a peer accepts each nonce once per transcript (the first occurrence that verifies) and withholds later copies. An entry signed before its conversation began (30 s skew) is withheld too; that rule rests on the conversation's stored `created` time, which no signature covers. Group transcripts do not use the `NonceCacheService` (nothing in EDDI does any more): a transcript entry is verified again on every turn, and a nonce the signer registers itself can never collide
8. **Peer verification** (opt-in, per receiving member): with `security.requirePeerVerification` off, the default, nothing is verified or logged and the member sees the transcript as stored. When `security.requirePeerVerification=true` on a member, every entry another agent wrote is verified against the speaker's current declared key before that member's turn, and each one that does not verify (unsigned, tampered, copied, moved to another round, duplicated, signed with an unknown or revoked key or outside the key's window) is replaced by a `[withheld: ... because ...]` placeholder in every prompt the engine builds for that member from the transcript: its phase prompt and its `groupTranscript` context, the prompts the engine writes for its role turns (convergence judge, dissenter, facilitator briefing, follow-ups, a nested group's question, and the task-force execute, verify and bid prompts) and what `listGroupTasks` returns to it. A task another member filed with its task tool, or another team's agent handed off, is shown only while that agent signs verifiably (a standing team with such a member files an `AUTO` handoff from an agent that does not to its intake inbox, and its cadence does not pull a backlog task whose agent stopped signing), and a verifying member whose own task's text does not verify for it fails that task without being asked; a verifying verifier judges only the tasks whose text and result it may see and leaves the others `COMPLETED`, so no fallback passes work it was not shown. A group- or global-visible user memory whose value another agent wrote is not loaded or recalled for a verifying agent, in any conversation (each entry records its last writer, since a global entry keeps its creator as owner when another agent writes it), while a `longTerm` property such an agent sets for a global key is written as any agent's and makes it the value's writer; its team's lessons are loaded. Shared artifacts are not signed, so a verifying member is given no artifact tools. A member that signs has its own entries checked like a peer's, its own task results included; one that does not sign is shown its own entries as stored. An entry under the id of an agent that has since been deleted is withheld too when the discussion shows an agent wrote it: it carries a signature, it is a facilitator's `FACILITATION` note, or an agent turn of the discussion ran under that id; its placeholder names the id with `(not an agent, unverified)`, since no agent store vouches for it. A key revoked mid-discussion takes effect on the next prompt built; what a member was already shown stays in its own conversation history, which its model sees again until the history window drops it. The first time an entry is withheld in a conversation it is logged at ERROR. Entries no agent wrote (the user's question, a human member's turn, a reviewer's edit, the engine's own notes, a nested group's answer) carry no signature and are not authenticated: whoever can write the group-conversation store can add one, or re-attribute an agent's entry to a non-agent id. A rework or task-failure note names a task's subject only as the group's records count it, and a verifying facilitator's briefing leaves out the convergence judge's reply, which is recorded under the engine's name. Since which entries must verify is decided by the speaker id while a prompt names each speaker by display name, a verifying member is shown these entries under a name no member agent can have: `User` for the user, `System` for the engine, and any other name followed by `(not an agent, unverified)` (`Anonymous (not an agent, unverified)` under `ANONYMOUS` scope). A group save refuses those names for an agent member (400), as it refuses two members with one display name and an agent member named `Moderator` while the moderator is not a member or `Facilitator` while an enabled facilitator is another agent, the names the engine signs those roles' entries under. A synthesis under the engine's or the user's name is withheld (`ENGINE_SYNTHESIS`), since the engine never records one there. The stored window summary is free text that no signature covers either, so a verifying member is never shown it: in a windowed discussion it gets the truncation marker. A ballot lists the options in the latest synthesis as the voter sees it, so a verifying voter is offered none from a synthesis that does not verify for it. Once any member, the moderator or the (enabled) facilitator requires verification, the engine's own records count only entries that verify for every reader, with no reader's own entries exempt: the `SERVICE` judge's input, the negotiation table, the vote's options and ballots (also when a `HUMAN_DECIDES` tie is resumed), an arbitration verdict, a retrospective's lessons and a standing team's cadence feedback. So a vote on a synthesis that does not verify does not run, and a facilitator's `CALL_VOTE` is refused unless it signs verifiably. A nested group's question is filtered when any agent of that group verifies (a member, its moderator or facilitator, or one in a group nested in it), and a nested group that cannot be read counts as verifying. If the member's configuration cannot be read, the setting this node last read for it applies; if this node never read it, its turn is recorded `SKIPPED` rather than run on a transcript filtered for a setting nobody knows. The stored transcript is never changed, and members without the flag, the Manager and API readers see it as stored. Each Ed25519 check is remembered on the node by the hash of key, signature and signed bytes, so a verifying member does not re-run it over the whole transcript on every turn; the key's revocation and window are re-checked every time. So that verifying members see the whole discussion, every agent that writes to it should sign, the moderator, a non-member synthesizer and the facilitator included, and an agent that verifies should sign too. The texts that still reach a verifying member without a signature of their own (handoffs filed by an `onDecision` rule or accepted by a person, the cadence question, other teams' shared lessons, a nested group's answer) are listed in [Not yet supported](https://docs.labs.ai/conversations-and-orchestration/group-conversations.md#not-yet-supported)

**What is NOT covered:** MCP invocation signing is not yet implemented — the `signMcpInvocations` config field has been removed until the feature is built.

### Authentication Model

| Environment    | OIDC Enabled          | Behavior                                                                                                                                                  |
| -------------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Dev mode**   | No                    | Allowed — info log on startup                                                                                                                             |
| **Dev mode**   | Yes                   | Full auth with configured Keycloak                                                                                                                        |
| **Production** | No + no opt-out       | `AuthStartupGuard` **fails startup** with clear error                                                                                                     |
| **Production** | No + explicit opt-out | Starts, but always logs an ERROR at startup; a periodic WARN reminder only when `eddi.security.unauthenticated-reminder-interval` is set (off by default) |
| **Production** | Yes                   | Full OIDC with Keycloak multi-tenant support                                                                                                              |

The escape hatch (`EDDI_SECURITY_ALLOW_UNAUTHENTICATED=true`) exists for air-gapped deployments and quick demos. The startup ERROR keeps operators aware; a periodic WARN reminder is opt-in via `eddi.security.unauthenticated-reminder-interval`.

### CI Security Scanning

| Tool                             | Trigger                                                           | What It Checks                                                                                         |
| -------------------------------- | ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| **CodeQL**                       | Every PR + weekly schedule                                        | Semantic analysis of the Java backend and, in a separate job, the UIs (injection, SSRF, crypto misuse) |
| **Trivy**                        | Every code PR (filesystem scan) + Docker image build (image scan) | CVEs in OS packages and Java dependencies; both scans block on fixable CRITICAL/HIGH                   |
| **Gitleaks** (`Secret Scanning`) | Every PR                                                          | Committed credentials                                                                                  |
| **Dependency Review**            | Every PR                                                          | License compliance and known vulnerabilities in new dependencies                                       |
| **Jackson 3 Ban**                | Every build (Maven Enforcer)                                      | Prevents accidental Jackson 3.x introduction (incompatible with Quarkus)                               |

### Security Headers

Production response headers (configured via `application.properties`):

* `X-Content-Type-Options: nosniff`
* `X-Frame-Options: DENY` (everywhere except the embeddable Chat UI at `/chat`, whose framing is `eddi.chat.frame-ancestors`, default `'none'`)
* `Referrer-Policy: strict-origin-when-cross-origin`
* `X-XSS-Protection: 0`
* `Permissions-Policy: camera=(), microphone=(), geolocation=()`
* `Content-Security-Policy: default-src 'self'; ...` (on `/chat`, `img-src` also takes `eddi.chat.img-sources`, default `https:`, so designer-configured output images render; see [chat-ui-embedding.md](https://docs.labs.ai/protocols-and-integration/chat-ui-embedding.md))

`Strict-Transport-Security` is **not** set by EDDI — configure it at your TLS terminator / ingress.

***

## Summary

EDDI's architecture is built on principles of **modularity**, **composability**, and **orchestration**. It's not a chatbot—it's the **infrastructure for building sophisticated conversational AI systems** that can:

* Orchestrate multiple APIs and LLMs
* Apply complex business logic through configurable rules
* Maintain stateful, context-aware conversations
* Scale horizontally in cloud environments (cluster mode on NATS JetStream)
* Be assembled from reusable, version-controlled components

The **Lifecycle Pipeline** is the heart of this architecture, providing a flexible, pluggable system where agent behavior is configuration, not code.

***

## Related Documentation

* [Getting Started](https://docs.labs.ai/getting-started/getting-started.md) - Setup and installation
* [Conversation Memory & State Management](https://docs.labs.ai/architecture-and-concepts/conversation-memory.md) - Deep dive into conversation state
* [Human-in-the-Loop](https://docs.labs.ai/conversations-and-orchestration/hitl.md) - The approval gate the Platform Operator runs behind
* [Behavior Rules](https://docs.labs.ai/agent-configuration/behavior-rules.md) - Configure decision logic
* [HTTP Calls](https://docs.labs.ai/agent-configuration/httpcalls.md) - External API integration
* [LLM Integration](https://docs.labs.ai/agent-configuration/langchain.md) - Connect to LLM APIs
* [Extensions](https://docs.labs.ai/architecture-and-concepts/extensions.md) - Available agent components
* [Security](https://docs.labs.ai/security-and-compliance/security.md) - Authentication, authorization, and tool security
* [Secrets Vault](https://docs.labs.ai/security-and-compliance/secrets-vault.md) - Encrypted secret management
* [Audit Ledger](https://docs.labs.ai/security-and-compliance/audit-ledger.md) - EU AI Act compliance
* [MCP Server](https://docs.labs.ai/protocols-and-integration/mcp-server.md) - Model Context Protocol integration
* [Group Conversations](https://docs.labs.ai/conversations-and-orchestration/group-conversations.md) - Multi-agent structured discussions
* [Persistent User Memory](https://docs.labs.ai/architecture-and-concepts/user-memory.md) - Cross-session memory and Dream consolidation
* [Agent Sync](https://docs.labs.ai/reference/agent-sync-architecture.md) - Import, export, and live instance sync
* [Memory Policy](https://docs.labs.ai/architecture-and-concepts/memory-policy.md) - Commit flags and strict write discipline
* [Prompt Snippets](https://docs.labs.ai/agent-configuration/prompt-snippets-guide.md) - Reusable system prompt building blocks
* [Model Cascade](https://docs.labs.ai/agent-configuration/model-cascade.md) - Multi-model sequential escalation
* [Scheduling](https://docs.labs.ai/conversations-and-orchestration/scheduling.md) - Cron and heartbeat agent triggers
* [A2A Protocol](https://docs.labs.ai/protocols-and-integration/a2a-protocol.md) - Agent-to-Agent peer communication
* [GDPR Compliance](https://docs.labs.ai/security-and-compliance/gdpr-compliance.md) - Data subject rights and retention
* [HIPAA Compliance](https://docs.labs.ai/security-and-compliance/hipaa-compliance.md) - Healthcare deployment guide
* [EU AI Act Compliance](https://docs.labs.ai/security-and-compliance/eu-ai-act-compliance.md) - AI decision audit requirements
