hub/scratchpad.md

Scratchpad

This is just loose ideas that need to be consolidated and implemented.

---

There are essentially 2 main classes in the project: Agent, and Chat. A chat is a fully featured telegram-like chat, with controls for adding messages, searching and retrieving messages, editing existing messages, and upload various data types (audio, image, etc). Tag support is very important too, beyond communication controls. In the future there will also be real time communication functionality, ie. calls, possibly also visibility of draft messages. An agent is a controller for chats that it is part of. It can be implemented by multiple means, any class that supports the agent interface is an agent. They have the exact same controls over the chat that the user does, any aditional functionality depends on the agent itself. Unlike chatgpt and the initial horde, chats are not meant to be created for any new conversation. Instead, agents have a singular permanent chat with the user. Chats are only created when a topic needs so much discussion that its better to separate it, or for other purposes based on necessity or organization and with limited frequency, similar to slack/discord groups. In the same way as with chats, both the user and agents can create and update other agents. They have to be explicitly enabled to do so. Most agents are better not given this capability, but the introduction agent is encouraged to have it enabled.

ALPHA MVP Urgently Needed.

Such an MVP is a chat with an Agent that responds to messages one by one. It has a simplistic, brief and conversational prompt. System prompt has brief description of the chat system. Message history is stored, and its infinite. Agent sees the last N messages and a recursive summary of the previous conversation. Agent is powered by replicate, whose API key is configured via env variables.

MVP-2 lets the agent (and the user) search in the message history, and the recursive summary will have an enforced size limit.

MVP-3A has configurable agents, both user and agents can create new agents, also modify their own prompt. Probably allowed by default. MVP-3C has multiple chats, users and agents can create chats, modify chat titles, and alter chat participants. MVP-3P is permissions. Modifying chats and agents both need a permission system.

Current Status: Frontend and backend connect via websocket. Backend has usable chats and agents with direct prompts and chat messaging. Frontend features user login, manual and automatic chat selection, and live connection with backend chat.

Copilot vs Companion

There are a few distinct use cases for AI agents of interest for this app:

• Companionship: empathetic, loyal and supportive agents acting as a presence in the users life for an extended period
• Role Playing: adopt a particular character or persona and remains consistent with it, with personalities independent of the user
• Copilot: effective and capable agents that assist the user with work or objectives
• Autonomous: beyond being effective and capable, these agents can also proactively pursue their goals with whatever means available to them

Companionship and Role Playing is mainly conversational and coherence skills, the MVP laid above aims to support these usecases mainly. Copilot requires capabilities (plugins and remote tools), which are not designed anywhere in this document. Autos, beyond having capabilities, must also be intelligent enough to manage themselves and proactively pursue broad goals.

Although conversational uses are more aligned with the current roadmap, copilots are actually more useful in my real life, and since the whole industry is converging in Autos, it would be a waste to not have any support for them in Horde. It follows that capable agents are quite important in the roadmap goals.

Next priority is to implement capabilities for agents, overriding MVP-2.

Capabilities

Also known as "tools" in the industry. They are implemented in python classes, they automatically generate signatures for the agent to

Intro

AgentBase: fields

• plugin prelude
• plugin system prompt methods
• add plugin
• get plugin context
• parse actions(string): action[]
• execute actions(action[]): string output
• create kind from data(kind: string, data) constructor(data)

Syntax:

V1:

As an autonomous agent, you have access to various tools that you can use by including ToolSyntax segments in your responses.

== ToolSyntax == ToolSyntax defines a format to specify a list of instructions to execute. A ToolSyntax segment starts with the special string "<>" anywhere in a line, and can either end with a matching "<>", or continue until the end of the text. Each line in the segment defines an instruction to execute a tool, with a JSON object as argument. For example: tool_name{"arg_1":"a string","arg_2":[1,2,{"an object":true}]}. Following is a list of all the available toolsets and tools, with a JsonSchema argument definition.

V2:

As an autonomous agent, you have augmented capabilities via tools and commands that you can use by including "!!command" lines in your responses.

== Commands == You can insert commands in your responses to interact with the system and execute actions as needed. Each command must be in it's own separate line, and it's made of these parts:

• '!!' is the first character sequence of the line and marks it as a command line
• optional query modifier
• full command name
• json argument For example: !!query namespace.command_name {"arg1":"some data","arg2":[1,2,false]} If you add the query modifier, you will see the output of the commands in a continuation prompt, otherwise it will only be shown to the user.