diff --git a/backend/src/agents/expert.py b/backend/src/agents/expert.py index 9fedeff..77cc3b2 100644 --- a/backend/src/agents/expert.py +++ b/backend/src/agents/expert.py @@ -38,6 +38,7 @@ class ExpertAgent(Agent): # Inject remotes as tools if 'remotes' in run.input: tools += self.app.getRemoteTools(self.user_id, run.input['remotes']) + using_agent_tools = True print("Final tool object: " + repr(tools)) if tools: run.input['tools'] = tools diff --git a/backend/src/inference.py b/backend/src/inference.py index 927eec2..99c7b13 100644 --- a/backend/src/inference.py +++ b/backend/src/inference.py @@ -350,8 +350,6 @@ async def openrouter_chat_completion(app, messages, model, tools=[], opts={}, ag if formatted_tools: kwargs['tools'] = formatted_tools - print("all kwargs: " + repr(kwargs)) - stream_response = await app.openrouter_client.chat.send_async(**kwargs) # OpenAI format streams tool calls in parts diff --git a/docs/remotes.md b/docs/remotes.md index dabd62b..2b61760 100644 --- a/docs/remotes.md +++ b/docs/remotes.md @@ -2,114 +2,86 @@ ## Overview -A **session** is one authenticated `/ws` connection. Authentication is -uniform — token or username/password — and every session is a user -connection that receives event broadcasts and can send any action. +A **remote** is a collection of functions exposed to an AI agent through the WebSocket session. Remotes are scoped to a user and live in memory during the server's lifetime. -A **remote** is a named set of functions served by a session: +## Architecture & Function Lifecycle -- Connected by that session with the `remote-connect` action (backed by the - `App.connectRemote` method), stored in `App.remotes` scoped to the user. -- **Lifetime**: connected until `remote-disconnect` (manual, allowed from any - session of the same user) or until the serving session closes — session - close disconnects every remote it connected. Nothing is persisted. -- **No overriding**: connecting a name that is already connected for that - user fails with an error reply. +Remote functions are not standard Python methods attached to the agent. Instead, they are definitions (schemas) wrapped into adapters that fit the expert agent's tool loop. -**Local Remotes** — purely user-side constructs, entirely implemented and -stored in the frontend (`Remotes.vue`). They never touch the backend. +1. **Definition (`backend/src/remote.py`)**: + * A **`RemoteFunction`** represents a function exposed by a remote. It contains the function's name, description, and its OpenAI-compatible schema for input parameters. + * A **`RemoteToolSpec`** wraps a `RemoteFunction` to make it compatible with the agent's tool system. It exposes the required interface: `.name`, `.description`, `.to_openai_dict()` (for the LLM), and `.execute_async(**args)` (for execution). + * A **`RemoteConnection`** manages the connection state. It holds the function definitions and an internal queue of pending calls. -Agents never link to remotes. A prompt passes remote names in -`options.remotes`; `ExpertAgent.run` expands them into run-scoped tools. -Every function of a remote becomes a tool named exactly after the function, -prefixed only when the run input asks for it. +2. **Injection**: + * When a run starts (see **Agent Integration** below), the expert agent requests tools for specific remotes. The server resolves these remote names into `RemoteToolSpec` objects and adds them to the run's input tools list. -This also implements the existing `server.py` TODO ("remotes aren't separate -connections — any connection can create a remotes and listen for requests -sent to them"). +3. **Execution**: + * When the AI decides to use a remote tool, the expert agent's loop calls `execute_async` on the `RemoteToolSpec`. + * This triggers `RemoteConnection.call()`, which sends a request over the WebSocket to the session that owns the remote. + * The remote executes the function code (for local remotes) or accepts an RPC call (for external remotes). + * Results are sent back through `remote-call-result` actions. Pending calls time out after 30 seconds if no response is received. -The frontend is already implemented and out of scope; the backend serves the -two actions it uses (`remote-list`, `remote-call`) with the shapes below. +4. **Lifecycle**: + * Remotes are established when a user connects them via the UI (`remote-connect`). + * They persist for the session lifetime unless manually disconnected (`remote-disconnect`) or if the session serving them closes. + * Collisions are handled by scoping connections to `(user_id, name)`. -## Wire protocol +## API Reference -Every message in both directions uses the existing action envelope. There is -no separate `type`-keyed protocol. +All endpoints are actions sent to the backend via the WebSocket. -Upstream (session → backend): +### `remote-connect` +Registers a new remote connection for the current user. -| action | payload | success reply | error reply | -|---|---|---|---| -| `remote-connect` | body `{remotename, tools}` | `{'message': 'remote connected', 'remotename', 'nonce'}` | `{'error': "remote 'x' is already connected", 'nonce'}` | -| `remote-disconnect` | body `{remotename}` | `{'message': 'remote disconnected', 'remotename', 'nonce'}` | `{'error': "no connected remote named 'x'", 'nonce'}` | -| `remote-list` | — | `{'remotes': [...], 'nonce'}` | — | -| `remote-call` | body `{remotename, function, args}` | `{'result': , 'nonce'}` | `{'error': '...', 'nonce'}` | -| `remote-call-result` | top-level `{requestId, result}` or `{requestId, error}` | none | none | +**Payload**: +* `remotename` (string): The unique name for this remote collection. +* `functions` (array of objects): List of function specs. Each object must contain: + * `name` (string): The function identifier. + * `description` (string): Text visible to the AI. + * `parameters` (object): OpenAI tool parameter schema (`type`, `properties`, `required`). -`tools` entries (function specs) and the `remote-list` entries share one -shape: `{name, description, parameters}` with `parameters` a full OpenAI -tool-parameter schema dict. +### `remote-disconnect` +Removes a connected remote. -Downstream (backend → the session serving the remote): +**Payload**: +* `remotename` (string): The name of the remote to remove. -~~~json -{"action": "remote-call", "requestId": "weather-service#0", "function": "get_forecast", "args": {"city": "Rome"}} -~~~ +### `remote-list` +Requests the list of remotes currently registered for the authenticated user. -Result routing: requestIds are `'#'`, unique among a user's -remotes. A `remote-call-result` is fanned out over the receiving session's -own connections; `resolve`/`reject` ignore unknown ids, so one session can -serve several remotes without ambiguity. Fan-out is session-scoped, so -identical id strings belonging to another user's remote can never be -resolved from this session. +**Payload**: (Empty) -Example connect + call round trip: +### `remote-call` +Invokes a specific function on a remote. -~~~json -{"action": "remote-connect", "nonce": "c1", "body": {"remotename": "weather-service", "tools": [{"name": "get_forecast", "description": "Retrieve weather forecast for a location.", "parameters": {"type": "object", "properties": {"city": {"type": "string", "description": "City name."}}, "required": ["city"]}}]}} -~~~ +**Payload**: +* `remotename` (string): The target remote. +* `function` (string): The function name. +* `args` (object): The arguments to pass to the function, matching the schema defined in `remote-connect`. -~~~json -{"action": "remote-call", "nonce": "c2", "body": {"remotename": "weather-service", "function": "get_forecast", "args": {"city": "Rome"}}} -~~~ +### `remote-call-result` +Sent by the server to the active session to deliver the outcome of a pending `remote-call`. -~~~json -{"action": "remote-call-result", "requestId": "weather-service#0", "result": {"forecast": "sunny", "city": "Rome"}} -~~~ +**Payload**: +* `requestId` (string): A unique identifier for this specific request. +* `result` (any): The return value of the function. +* `error` (string): An error message if execution failed. -Calls not answered within `REMOTE_CALL_TIMEOUT` (30s, constant in -`remote.py`) fail; calls pending when the remote disconnects fail -immediately. All failures surface as `RemoteError`, which becomes an error -tool-result string for the model — never an exception that kills a run, and -never a killed caller session. +## Agent Integration -## The prompt contract +Remotes are integrated into the agent workflow in two stages: receiving context and executing tools. -~~~json -{ - "action": "send-agent-prompt", - "body": { - "agentId": 7, - "instruction": "What's the weather in Rome? Use your tools.", - "options": { "remotes": ["weather-service", { "name": "code-repo-indexer", "prefix": "repo_" }] } - } -} -~~~ +### Chat Integration (`Agent.received_message`) +When a user sends a message to a chat thread, the frontend sends a special system resource to mark which remotes should be available for that conversation. -- `Agent.prompt` already splats options into the run input - (`input={"instruction": ..., "input": ..., **kwargs}`), so `options.remotes` - lands in `run.input["remotes"]`. No server or app change needed. -- Each entry is a remote name or `{"name": ..., "prefix": ...}`. Every - function of the remote becomes one tool named `prefix + function_name` - (empty prefix by default). -- Collisions: the agent's native tools are listed before remote tools and - the expert lookup takes the first match, so native tools win. Pass a - `prefix` to disambiguate. -- Remotes are resolved once, at run start. A remote not connected at that - moment is skipped with a console warning and the run proceeds (remotes - disconnect independently of prompts). If a remote disconnects mid-run, - its pending calls fail and the model sees error tool results. -- Existing expert rule unchanged: if the run input already carries its own - `tools`, those replace the agent toolset entirely and remotes are not in - play. -- Users only see their own remotes — the registry key includes `user_id`. \ No newline at end of file +* In `backend/src/agent.py`, the `received_message` method scans the thread's resources. +* It looks for items with `subkind: 'remote'`. +* It extracts the content of these resources (the remote names) and passes them into the agent run context. + +### Execution Integration (`ExpertAgent.run`) +In `backend/src/agents/expert.py`, the `run` method prepares for execution. + +* It retrieves the list of remotes from the run input. +* It calls `app.getRemoteTools(user_id, remotes)` to resolve these names into active tool specifications. +* These tools are injected into the inference loop. When the AI selects a tool, the expert agent executes it. If a remote fails (timeout or error), the failure is returned as a string error, allowing the AI to retry or handle the failure naturally. \ No newline at end of file diff --git a/frontend/src/views/RunsView.vue b/frontend/src/views/RunsView.vue index b8677f3..75d7e4f 100644 --- a/frontend/src/views/RunsView.vue +++ b/frontend/src/views/RunsView.vue @@ -197,5 +197,7 @@ white-space: pre-wrap; word-break: break-all; font-size: 0.85em; + max-height: 20em; + overflow-y: auto; }