87 lines
4.5 KiB
Markdown
87 lines
4.5 KiB
Markdown
# Remotes
|
|
|
|
## Overview
|
|
|
|
A **remote** is a collection of functions exposed to an AI agent through the WebSocket session. Remotes are scoped to a user and live in memory during the server's lifetime.
|
|
|
|
## Architecture & Function Lifecycle
|
|
|
|
Remote functions are not standard Python methods attached to the agent. Instead, they are definitions (schemas) wrapped into adapters that fit the expert agent's tool loop.
|
|
|
|
1. **Definition (`backend/src/remote.py`)**:
|
|
* A **`RemoteFunction`** represents a function exposed by a remote. It contains the function's name, description, and its OpenAI-compatible schema for input parameters.
|
|
* A **`RemoteToolSpec`** wraps a `RemoteFunction` to make it compatible with the agent's tool system. It exposes the required interface: `.name`, `.description`, `.to_openai_dict()` (for the LLM), and `.execute_async(**args)` (for execution).
|
|
* A **`RemoteConnection`** manages the connection state. It holds the function definitions and an internal queue of pending calls.
|
|
|
|
2. **Injection**:
|
|
* When a run starts (see **Agent Integration** below), the expert agent requests tools for specific remotes. The server resolves these remote names into `RemoteToolSpec` objects and adds them to the run's input tools list.
|
|
|
|
3. **Execution**:
|
|
* When the AI decides to use a remote tool, the expert agent's loop calls `execute_async` on the `RemoteToolSpec`.
|
|
* This triggers `RemoteConnection.call()`, which sends a request over the WebSocket to the session that owns the remote.
|
|
* The remote executes the function code (for local remotes) or accepts an RPC call (for external remotes).
|
|
* Results are sent back through `remote-call-result` actions. Pending calls time out after 30 seconds if no response is received.
|
|
|
|
4. **Lifecycle**:
|
|
* Remotes are established when a user connects them via the UI (`remote-connect`).
|
|
* They persist for the session lifetime unless manually disconnected (`remote-disconnect`) or if the session serving them closes.
|
|
* Collisions are handled by scoping connections to `(user_id, name)`.
|
|
|
|
## API Reference
|
|
|
|
All endpoints are actions sent to the backend via the WebSocket.
|
|
|
|
### `remote-connect`
|
|
Registers a new remote connection for the current user.
|
|
|
|
**Payload**:
|
|
* `remotename` (string): The unique name for this remote collection.
|
|
* `functions` (array of objects): List of function specs. Each object must contain:
|
|
* `name` (string): The function identifier.
|
|
* `description` (string): Text visible to the AI.
|
|
* `parameters` (object): OpenAI tool parameter schema (`type`, `properties`, `required`).
|
|
|
|
### `remote-disconnect`
|
|
Removes a connected remote.
|
|
|
|
**Payload**:
|
|
* `remotename` (string): The name of the remote to remove.
|
|
|
|
### `remote-list`
|
|
Requests the list of remotes currently registered for the authenticated user.
|
|
|
|
**Payload**: (Empty)
|
|
|
|
### `remote-call`
|
|
Invokes a specific function on a remote.
|
|
|
|
**Payload**:
|
|
* `remotename` (string): The target remote.
|
|
* `function` (string): The function name.
|
|
* `args` (object): The arguments to pass to the function, matching the schema defined in `remote-connect`.
|
|
|
|
### `remote-call-result`
|
|
Sent by the server to the active session to deliver the outcome of a pending `remote-call`.
|
|
|
|
**Payload**:
|
|
* `requestId` (string): A unique identifier for this specific request.
|
|
* `result` (any): The return value of the function.
|
|
* `error` (string): An error message if execution failed.
|
|
|
|
## Agent Integration
|
|
|
|
Remotes are integrated into the agent workflow in two stages: receiving context and executing tools.
|
|
|
|
### Chat Integration (`Agent.received_message`)
|
|
When a user sends a message to a chat thread, the frontend sends a special system resource to mark which remotes should be available for that conversation.
|
|
|
|
* In `backend/src/agent.py`, the `received_message` method scans the thread's resources.
|
|
* It looks for items with `subkind: 'remote'`.
|
|
* It extracts the content of these resources (the remote names) and passes them into the agent run context.
|
|
|
|
### Execution Integration (`ExpertAgent.run`)
|
|
In `backend/src/agents/expert.py`, the `run` method prepares for execution.
|
|
|
|
* It retrieves the list of remotes from the run input.
|
|
* It calls `app.getRemoteTools(user_id, remotes)` to resolve these names into active tool specifications.
|
|
* These tools are injected into the inference loop. When the AI selects a tool, the expert agent executes it. If a remote fails (timeout or error), the failure is returned as a string error, allowing the AI to retry or handle the failure naturally. |