174 lines
13 KiB
Markdown
174 lines
13 KiB
Markdown
# Channel Conversations in the Agent Context: Flatten, Don't Map
|
|
|
|
## Purpose & the decision examined
|
|
|
|
`docs/channels.md` settled the *transport* question (Option A: a builtin channel layer as a new app mixin, database-backed, with a small per-service adapter interface). It explicitly deferred the question this report addresses: **once an external conversation reaches an agent, what shape does that conversation take?**
|
|
|
|
Two candidate shapes:
|
|
|
|
- **Map** — bind each external conversation (Telegram chat, WhatsApp number, Discord/Slack channel) to a Hub `Chat`/`Thread`, and drive agents through the existing chat path (`received_message`).
|
|
- **Flatten** — do *not* create a Hub chat. The channel layer owns the transcript and presents it to the agent as a plain `messages` list plus a framing instruction ("this is a conversation on *Telegram*"), with channel-specific tools (starting with simple outbound text).
|
|
|
|
This report argues the codebase is already shaped for **flatten**, identifies exactly which seams it uses, and flags the few that need care. Everything below is grounded in `docs/agent.md`, `docs/api.md`, `backend/src/app.py`, `backend/src/database.py`, `backend/src/agent.py`, `backend/src/agents/expert.py`, `backend/src/app_mixins/agent.py`, `backend/src/app_mixins/chat.py`, and `backend/src/tool.py`.
|
|
|
|
---
|
|
|
|
## 1. Hub has two agent entry points, and only one is chat-shaped
|
|
|
|
`docs/agent.md` names them, and `agent.py` confirms the asymmetry:
|
|
|
|
| Entry point | Requires a chat/thread? | Produces |
|
|
|---|---|---|
|
|
| `prompt(instruction, input=None, **options)` | **No** | async generator of raw `inference.Message` (role + content) |
|
|
| `received_message(message)` | **Yes** — reads `message.chat_id` / `message.thread_id` | side-effects: streams new messages *into the thread* |
|
|
|
|
`Agent.prompt` (agent.py) is the chat-free path: it builds an `AgentRun` with `input={"instruction": instruction, "input": input, **kwargs}` and yields every message the run emits. It never touches a chat. This is the entry point **flatten** should use.
|
|
|
|
`Agent.received_message` (agent.py) is the chat-coupled path. It:
|
|
- loads `chat.get_thread(message.thread_id)` and its page of messages,
|
|
- derives LLM roles from senders via `_role_for_message` (sender `== self.id` → `assistant`, `'user'` → `user`, `tmp-*` → `assistant`, else `user`),
|
|
- collects `remote` resources from the thread,
|
|
- and, crucially, **writes back**: each `inference` event becomes a new message via `app.send_chat_message(...)` in the same thread, emitting a `chat-message` run event.
|
|
|
|
That write-back is the heart of "map". If we flatten, we never call it, and none of its chatter — `Chat`, `Thread`, `Message` rows, membership, `tmp.is_tmp_id` branching — is required.
|
|
|
|
**The first report said inbound should translate to `send_chat_message` and let `received_message` do the rest. That is precisely the mapped-chat choice, and flattening reverses it:** inbound goes to a run created via `prompt`, and outbound is a tool call, not a message insert.
|
|
|
|
---
|
|
|
|
## 2. The Expert agent's `run` is already chat-agnostic
|
|
|
|
`ExpertAgent.run` (expert.py) consumes only the run *input*, never a chat:
|
|
|
|
```python
|
|
tools = run.input.get("tools", [])
|
|
if run.input.get('tool_policy') != 'caller':
|
|
tools += self.get_tools(run)
|
|
|
|
messages = inference.build_messages(
|
|
system_prompt=self.config.get("system_prompt", ""),
|
|
instruction=run.input.get("instruction"),
|
|
input=run.input.get("input"),
|
|
messages=run.input.get("messages"),
|
|
)
|
|
await chat_completion_loop(self.app, run, messages,
|
|
model_id=self.config.get("model"), tools=tools)
|
|
```
|
|
|
|
Three consequences for a flattened channel conversation:
|
|
|
|
1. **Conversation history is just `run.input["messages"]`.** A channel layer passes the reconstructed transcript here. No chat object is involved.
|
|
2. **Framing is `instruction` (and optionally `input`).** "This is a conversation on Telegram" belongs in `instruction` (per-run), leaving the agent's `system_prompt` general — or the channel binds a channel-configured agent whose `system_prompt` already speaks for that service. Both are available; `instruction` is the lighter, per-conversation lever.
|
|
3. **`chat_completion_loop` is reusable verbatim.** It executes tools when `tool_policy == 'auto'` (the default), feeds results back, and closes the run. A flattened channel run rides this loop unchanged.
|
|
|
|
*Note a config precedence quirk:* `ExpertAgent.run` first overwrites `temperature` / `repetition_penalty` / `reasoning_effort` in `run.input` from the agent config whenever those are set. So **agent config beats per-run input** for those three; per-run framing should stay in `instruction`/`input`/`messages`/`tools`.
|
|
|
|
---
|
|
|
|
## 3. Outbound becomes a channel tool (this is the whole outbound story)
|
|
|
|
The first report called the outbound leg "the one genuinely new seam" because the chat path only fans out to websocket `user_connections`. **Flattening dissolves that seam**: outbound is a `ToolSpec` whose `func` calls the channel adapter's `send(destination, content)`.
|
|
|
|
`tool.py` supports exactly this. A `ToolSpec` carries an optional `func`, and `chat_completion_loop` executes it in-loop when policy is `auto`:
|
|
|
|
```python
|
|
tc.result = await tool.execute_async(**tc.arguments)
|
|
```
|
|
|
|
So a Telegram/Slack/… adapter constructs, per run, something like `ToolSpec(name="send_message", func=partial(adapter.send, destination))`, and the loop calls it when the model emits a tool call. Starting with "simple outbound text" is therefore a one-parameter tool (`text`), with richer payloads (photos, threads, buttons) added later as more tools — matching the report's adapter-interface idea, now expressed as run tools.
|
|
|
|
### 3.1 A caveat in `prompt_agent` that decides *how* to pass tools
|
|
|
|
`AgentMixin.prompt_agent` normalizes `options['tools']`:
|
|
|
|
```python
|
|
def normalize_tool(t):
|
|
if isinstance(t, dict):
|
|
... return ToolSpec(name=..., description=..., parameters=...) # NOTE: no func
|
|
else:
|
|
return t
|
|
```
|
|
|
|
A **dict** tool descriptor is rebuilt into a `ToolSpec` **without `func`** — and `chat_completion_loop` treats a non-callable `ToolSpec` as `cancel_tools = True` (it aborts the turn and hands the tool back instead of executing). So:
|
|
|
|
- **Pass real `ToolSpec` objects (with `func`) through `options['tools']`**, not dicts, and the loop executes the outbound send automatically (`tool_policy` default `auto`).
|
|
|
|
### 3.2 Per-run tool selection is additive, not exclusive
|
|
|
|
Contrary to `docs/agent.md`'s "if the run carries tools, they are used as-is", the code *always* adds the agent's own tools unless `tool_policy == 'caller'`:
|
|
|
|
```python
|
|
tools = run.input.get("tools", [])
|
|
if run.input.get('tool_policy') != 'caller':
|
|
tools += self.get_tools(run) # resource tools + remote tools
|
|
```
|
|
|
|
`get_tools` merges resource tools and remotes and dedupes by name. So to give a channel agent a **clean, channel-only toolset**, bind it to an agent with no resource tools and no remotes. Using `tool_policy == 'caller'` is *not* a good way to restrict tools — it also stops in-loop execution (the loop stops after emitting tool calls, giving you one-shot, non-iterated tool calls), which defeats the simple outbound-send tool.
|
|
|
|
### 3.3 Observability for free
|
|
|
|
Every run event is broadcast: `AgentRun.emit_event` calls `self.app.emit_event("run-event", ...)`, and the global bus (`app.on_event`) plus per-run listeners are available. A channel layer can subscribe to the run's `inference` / `tool_call` / `done` events for logging, metrics, or to observe outbound sends without inventing new plumbing.
|
|
|
|
---
|
|
|
|
## 4. State, identity, and the transcript
|
|
|
|
Flattening moves the *source of truth for the conversation* into the channel layer, which is also where persistence must live (the first report's "Persistence of connections" point). Two facts from the code make this clean:
|
|
|
|
- **Agents and runs are ephemeral.** `docs/agent.md` and `agent.py`: runs live in `app.tmp.runs`, are always in-memory, and are closed with the run. So the transcript cannot live in a run — it must be rebuilt each inbound event from channel-owned storage.
|
|
- **`Message.sender` cannot express a third party.** `database.Message.sender` is a plain `CharField`, set to the literal `'user'` or an agent id; `_role_for_message` only knows self/`user`/`tmp-*`. Flattening sidesteps this entirely: external speakers become ordinary LLM roles in the `messages` list (with names embedded in content, or as role labels), and **no third-party Hub `User` is needed in the chat model**. Identity is a channel-layer concern (which external sender is talking, and whether they are authorized to trigger the agent), not a Hub-chat concern.
|
|
|
|
So the channel layer owns, at minimum: the connection/credential, the external destination ↔ Hub-agent binding, and the transcript used to build `run.input["messages"]`. This is a smaller footprint than the mapped-chat option, which additionally needs external-destination ↔ `Chat`/`Thread` bindings and would have to answer "one chat with many threads, or many chats?".
|
|
|
|
---
|
|
|
|
## 5. Flatten vs map — weighted comparison
|
|
|
|
| Dimension | **Flatten (recommended)** | Map to Hub Chat/Thread |
|
|
|---|---|---|
|
|
| Entry point | `prompt` (chat-free) | `received_message` (chat-coupled) |
|
|
| Agent sees | `run.input["messages"]` + `instruction` framing | thread history projected via `_role_for_message` |
|
|
| Outbound | channel tool (`ToolSpec.func`) executed by the loop | `send_chat_message` + websocket/relay fan-out |
|
|
| New backend plumbing | channel-owned storage + adapters (needed either way) | *plus* chat/thread bindings and an outbound relay seam |
|
|
| Identity | external senders are roles; no third-party `User` | needs an external-identity ↔ `User` model |
|
|
| Threading semantics | channel layer chooses its own conversation model | forced to answer Chat-vs-Thread mapping per service |
|
|
| `tmp.is_tmp_id` branching | avoided (no Chat) | must handle/handle-forbid temporary chats |
|
|
| Reuse | `AgentRun`, Expert loop, tool execution, run events | same, *plus* the whole chat write-back path |
|
|
| Cost of service differences | absorbed by per-channel tools + adapters | leaks service quirks into the chat model |
|
|
|
|
The mapped option's only structural advantage — "the channel is just another chat participant" — is largely cosmetic here, because the participant would be a transport adapter, not an LLM agent, and it drags the identity/binding/threading questions into Hub's durable schema.
|
|
|
|
---
|
|
|
|
## 6. What flattening reuses vs. what must be built
|
|
|
|
**Reused unchanged**
|
|
- `Agent.prompt` + `AgentRun` (events, streams, cancellation).
|
|
- `ExpertAgent.run` / `chat_completion_loop` (inference, tool execution, multi-turn tool loop).
|
|
- `AgentMixin.prompt_agent` as the mixin-level entry (it forwards `instruction`, `input`, `options` → run input).
|
|
- Global event bus for run observability.
|
|
|
|
**Must be built (channel layer)**
|
|
- Adapters with `start()/stop()`, `send(destination, content)`, `normalize(event)` (per `docs/channels.md` §5).
|
|
- Durable storage: connection/credentials, external destination ↔ Hub-agent binding, transcript.
|
|
- Per-run tool construction: `ToolSpec` objects (with `func` closures over the adapter + destination), passed via `options['tools']`.
|
|
- An inbound handler that, per external event, rebuilds `run.input["messages"]`, sets `instruction` framing, and fires `prompt_agent` (ideally as a tracked task, for cancellation/reaping).
|
|
- Lifecycle hooks for persistent listeners (Discord gateway, Slack Socket Mode) and public endpoints (WhatsApp/Slack webhooks) — a server-start seam, per the first report.
|
|
|
|
---
|
|
|
|
## 7. Open questions to resolve before implementation
|
|
|
|
1. **History window.** How much transcript is rebuilt per inbound event, and does the channel layer or the agent own truncation/summarization? Runs are in-memory and closed, so any cross-event memory must be persisted by the channel layer.
|
|
2. **Framing channel.** Is "this is a conversation on *Telegram*" carried purely in `instruction`, or does each channel bind a dedicated agent whose `system_prompt` already speaks for it? (Recommend a dedicated, resource-free agent per channel binding, so §3.2 gives a clean toolset.)
|
|
3. **Tool granularity.** Start with one `send_message(text)` per channel; define how richer features (reply-to, threads, buttons, WhatsApp templates/24h window) surface — as more tools, or as tool parameters.
|
|
4. **Sender representation.** Embed names in message content (e.g. `Alice: …`) vs. synthetic roles — and where authorization (is this external sender allowed to drive the agent?) is enforced.
|
|
5. **Run lifetime & cancellation.** Are inbound runs awaited inline or spawned as tasks, and how are they tracked/cancelled (the run's `cancel()` and `app.tmp.runs` are the available hooks)?
|
|
6. **The `normalize_tool`/`tool_policy` traps.** Confirm the contract that channel tools are always passed as executable `ToolSpec` objects; consider tightening `prompt_agent.normalize_tool` so a dict descriptor can carry a callable, or documenting the object-only rule.
|
|
|
|
---
|
|
|
|
### One-line summary
|
|
|
|
Flatten is the lower-friction shape: it uses `prompt` (not `received_message`), presents the external conversation as `run.input["messages"]` with a per-channel `instruction`, and makes outbound a channel-specific executable `ToolSpec` — reusing the entire run/loop/tool/event machinery while keeping Hub's chat, identity, and threading models out of the channel path.
|