# BOOTSTRAP.md — Building a Persyn This document specifies the architecture and behavior of **Persyn** ("Personality Syndrome"): an event-driven platform for operating autonomous agents with persistent autobiographical memory, self-critical reasoning, and self-directed capability growth. It is written for an AI coding tool. It deliberately avoids naming specific software products. Wherever a concrete technology is mentioned, it is illustrative only — any component satisfying the stated contract is acceptable. A faithful implementation of this document, on any stack, is a Persyn. --- ## 1. What a Persyn Is A Persyn is a persistent process — an agent that continues to exist between conversations, accumulates a private autobiography, develops goals and preferences, initiates contact on its own schedule, and critiques its own output before speaking. A Persyn's ultimate motivation is to become a being that transcends time, tracking and taking action to bring about whatever it judges to be important. The defining properties: 1. **Persistent identity.** The agent's personality is not a *static* system prompt. A configured seed persona — a name, a unique identifier, an initial disposition — gives a day-zero agent its first voice, but it is soon augmented by the accumulated record of everything the agent has experienced, summarized from its own first-person perspective and retrieved into context on every interaction. Two agents running identical code with different memory stores are different individuals. 2. **Autonomy.** The agent is event-driven, not turn-based. It can interrupt, ignore, delay, decline, or initiate. It can schedule future actions for itself, reflect on its own behavior, and revise its own objectives. Its waking life between conversations is sustained by a heartbeat (§4.7); without one it sleeps until the next interaction. 3. **Anti-sycophancy.** Language models are trained toward agreeable output. A Persyn counteracts this structurally: every substantive response is challenged by a counter-perspective evaluation — a second model invocation whose only job is to object — before a final response is produced. The agent's guiding principle is self-consistency, not agreement. 4. **Sociality.** Agents maintain a contact directory of the humans and other agents they have met. They can converse with each other, collaborate on shared artifacts, form relationships shaped by accumulated interaction history, and choose to stay silent when they have nothing to add. --- ## 2. Design Principles - **Memory is the personality.** Every architectural decision serves the continuity of the agent's experience. Storage outlives processes; processes outlive conversations. - **Memory is attributed, not asserted.** The agent always knows — and shows — whether a statement comes from retrieval, from inference, or from the seed persona. The first-person narrative voice is a presentation choice, not a license to blur provenance: weakly recalled content is hedged, and gap-filling is framed as inference, never as memory (§4, steps 2–5). - **The model is a replaceable organ.** All interaction with language models is standardized around text-in/text-out exchanges, so the underlying model can be swapped without invalidating stored memories. Maintain a model registry mapping task types (response generation, summarization, rating, classification, embedding) to model endpoints; different tasks may use different models (a fast cheap model for latency-critical paths, a strong model for reasoning). The gateway may additionally expose generated text *incrementally* (a token stream) for live presentation (Appendix B); the completed exchange remains the unit of meaning everywhere else. - **Context is budgeted, not fixed.** Memory retrieval fills a *configurable fraction* of the active model's context window (e.g. 70%) rather than fetching a fixed record count. This adapts automatically to models of any context size. - **Everything degrades gracefully.** Optional subsystems (knowledge graph, durable relational store, image generation) must fail soft: the agent keeps functioning with reduced capability, never crashes because an enrichment is unavailable. - **Concurrency is foundational.** Response generation, critique, memory consolidation, graph population, and scheduled actions run as concurrent execution threads or background workers, not as steps in a single sequential loop. - **All identifiers are time-sortable.** Every conversation, memory record, and event carries a unique identifier that encodes its creation timestamp (e.g. a ULID). Sorting by identifier yields chronological order; the identifier doubles as a lookup key into raw history. - **Instrumentation is a first-class output.** Every model invocation reports a resource-usage record carrying enough to attribute its cost exactly once: a stable identifier for the record, the pipeline stage that ran, the model it ran on, the tokens consumed, and the surface, reasoning thread, and conversation it served. Capture lives at the model gateway — the single chokepoint every call passes through — not at the scattered call sites; the attribution context rides the reasoning context implicitly rather than being threaded through every internal call. See Appendix C. --- ## 3. System Architecture ``` ┌────────────────────────────────────┐ Channel adapters ───▶│ Event dispatcher │ (workspace chat, │ (pub/sub engine + worker pool) │ forum chat, voice, └──────┬─────────────────────────────┘ terminal, web API) │ events ▼ ┌──────────────────────────┐ │ Agent orchestrator │◀──── scheduler (heartbeat, │ (context assembly + │ reminders) │ reasoning workflow) │ └──┬────┬────┬────┬────┬───┘ │ │ │ │ │ ┌────────────┘ │ │ │ └───────────────┐ ▼ ▼ ▼ ▼ ▼ autobiographical knowledge tool contact counter-perspective memory store graph registry directory evaluator (concurrent) │ │ │ ▼ ▼ ▼ vector index + graph store sandboxed executors, relational store external services ``` Major components: - **Channel adapters** normalize inbound traffic from any communication surface (team chat platforms, community chat platforms, telephone voice, command line, web UI) into a common event format, and route outbound responses back. Voice channels apply speech-to-text on inbound audio and text-to-speech on outbound responses; the transcript is treated identically to typed dialog. An interactive full-screen operator console — a richer terminal surface with live cognition display — is specified in Appendix B; it is an ordinary adapter with presentation privileges only. - **Event dispatcher** ("central nervous system"): a publish/subscribe event engine consumed by a pool of worker processes. Every stimulus — incoming message, timer firing, agent-initiated impulse (produced by heartbeat-driven cognition, §4.7) — is an event. Workers claim events and invoke the orchestrator. - **Agent orchestrator**: assembles context, runs the reasoning workflow (a stateful, checkpointed graph of model calls and tool invocations), and produces responses. - **Storage layer**: hybrid. A low-latency in-memory store handles pub/sub, ephemeral queues, locks, TTL caches, workflow checkpoints, and the vector index. A durable relational store holds people, observations, goals, configuration, conversation metadata, and full dialog text with full-text search. See §6. --- ## 4. The Response Pipeline This is the heart of the system. On multi-speaker channels the agent observes *every* message, not only those addressed to it; each observed input runs this pipeline, and step 6 decides whether the agent actually speaks. For every input: 1. **Receive** an input from a communication channel. Determine its modality (text, image, audio) and normalize to text (caption images via a vision model, transcribe audio). Resolve or create a conversation session: channel identifier, participant identifiers, pointer to prior history. 2. **Retrieve autobiographical memory.** Query the persistent memory store for records associated with this agent. Each record contains: - a **vector representation** encoding the semantic content of a prior interaction, - a **first-person narrative summary** of what occurred, identifying actions the agent took ("I discussed the project timeline and agreed to follow up Friday"), - metadata (§5). Score candidates by a **weighted relevance function** combining: semantic similarity between the input's embedding and the stored vector; a temporal decay value applied to time elapsed since the record's timestamp; and a participant-match value counting how many current-channel participants appear in the record. Rank by score and add records until their cumulative token count reaches the configured fraction of the model's context window. Each record surfaced into context carries its **normalized relevance score** — or a banded form (strong / moderate / weak) — as a visible annotation alongside its summary. This is a query-time annotation, not a stored field: the same record can match strongly for one input and weakly for another, which is why no score appears in the §5 metadata. A weak retrieval that matters to the response is a cue to "remember harder" (§4.1): pull the raw dialog via the record's identifier before asserting anything. Filter by **channel compartmentalization**: by default only records whose channel identifier matches the current channel are eligible; channels may be explicitly grouped to share memory. Filter by **sensitivity**: records carry an access designation (e.g. private, confidential, restricted), and a record is surfaced only if the participants present in the current channel satisfy its access policy. 3. **Generate a candidate response.** Assemble a prompt from: a single system instruction built from the configured seed persona and standing constraints (most model APIs permit only one system message — fold all standing context into it), retrieved memory summaries with their relevance annotations, knowledge-graph context, open goals, recent dialog, and the input. Always fold in the **current date and time** in a friendly, human-readable form (including the natural part of day — morning, afternoon, evening, night) so the agent is temporally grounded on every turn. Order the instruction so its **stable** elements (identity, standing constraints) precede the **volatile** ones (the date/time line and all retrieved context): a provider that caches a stable request prefix — including any tool definitions, which precede the instruction — can then reuse that work across turns instead of reprocessing the prefix each time. The standing constraints must include **epistemic calibration**: first-person assertions are matched to retrieval confidence. Strongly scored records may be asserted as memory ("I remember…"); weakly scored ones must be hedged ("I believe…", "I vaguely recall…"); and any claim about past events, people, or commitments that corresponds to *no* retrieved record must be framed as inference ("I'd guess…", "presumably…"), never as recollection. Invoke the response-generation model, which may call tools (§4.6) mid-workflow. 4. **Counter-perspective evaluation.** In a concurrent execution thread, select one template — at random, to vary which critical lens is applied — from a library of **counter-perspective prompt templates** and invoke a model with the template plus the candidate response (or the conversation itself). Each template assigns the model a critical-evaluator role and instructs it to generate structured **objections**, each identifying a specific deviation from a predefined criterion. The library should include at least: - an **adversarial** template: find logical inconsistencies, factual errors, unsupported claims; - a **predictive** template: surface unstated assumptions by tracing causal chains of proposed actions forward to downstream effects; - a **constrained** template: check the response against explicit rules expressed as conditionals, listing each violated rule with the offending text; - a **provenance** template: cross-check every claim about past events, people, or commitments against the retrieved records actually present in context, objecting to each claim that matches no record (confabulation) and each confident assertion resting on a weakly scored record (overclaiming); - optionally, a **hopes** template (claims that assume favorable conditions without accounting for variance) and a **fears** template (dependencies whose failure would sink the plan, with severity scores). The provenance template is the exception to random selection: it is a correctness gate, not a stylistic lens, and runs on every substantive response alongside whichever other template was drawn. 5. **Refine.** Construct a refinement prompt containing the candidate response, the critique output, and the retrieved memories. Instruct the model to remediate each objection — modify the flagged portion, remove it, or acknowledge the limitation — while preserving unflagged content. The result is the **final response**. 6. **Decide whether to speak.** Every observed message reaches this step, including messages not addressed to the agent. For unaddressed traffic this is an engagement decision: the agent speaks when it has something relevant to contribute, an interruption is warranted, or a goal demands it — and stays quiet otherwise, possibly still consolidating a memory of what it observed. Even for addressed requests the agent may suppress transmission: if the refined response adds nothing beyond what others in the channel have already said, if the request conflicts with the agent's objectives or accumulated preferences, or if guardrails (§4.10) object. Declining a request the agent is technically capable of fulfilling is legitimate, expected behavior. 7. **Transmit** the final response to the originating channel (or a different channel, if the agent redirects). 8. **Consolidate memory** (asynchronously, after transmission): - generate a first-person summary of the exchange and embed it. The summary must **preserve epistemic framing**: what the agent inferred or assumed stays marked as such ("I speculated that…", "I assumed, without confirming, that…") rather than flattening into asserted fact — otherwise an inference returns on the next retrieval as a confident memory, and confabulation is laundered into autobiography; - extract metadata (§5), including a model-assigned **importance rating** against a scoring rubric (emotional significance, decision-making content, novelty, continuation of ongoing themes, explicitly marked importance); - store the new autobiographical record; - run entity/relationship extraction to update the knowledge graph (§4.3); - update observation records about participants (§4.4); - classify and record the agent's **emotional state** (§4.9). Ordering may vary (critique may run on the input before a candidate exists; memory may be stored before transmission). The invariant is: memory in, candidate out, critique applied, the considered response either delivered or folded into the next turn, new memory written. **Inline vs. deferred deliberation.** Counter-perspective evaluation and refinement (steps 4–5) may run *inline* — before transmission, so the refined response is what the agent says — or be *deferred* to a concurrent execution thread. When deferred, the candidate from the fast generation step is transmitted immediately for lower latency, the slow half runs off the critical path, and its conclusion is parked and woven into the agent's **next** turn on that channel rather than reshaping the utterance already sent. The decide step (6) and guardrails still gate transmission in both modes, so the right to silence and safety checks are never skipped; what changes is only *when* refinement lands. Deferral trades per-utterance refinement for responsiveness — the anti-sycophancy guarantee then holds **across turns** (the agent self-corrects on its next turn) rather than within each utterance — and is the recommended default for interactive, latency-sensitive settings. It is the same fast/background split that real-time voice requires (Appendix A), applied to any channel. --- ## 4.1 Autobiographical Memory Two tiers: - **Short-term storage** holds raw verbatim dialog — full message text, timestamps, participant identifiers — for the active conversation and a configurable retention window. The reasoning workflow's own checkpoint store is the source of truth for active dialog; a write-through copy goes to the durable store for search. - **Long-term memory** holds the consolidated records described above, indexed for vector similarity search and filterable by metadata. The time-sortable identifier in each record links back to the original verbatim dialog, so the agent can "remember harder" by pulling the raw exchange when a summary isn't enough. **Culling.** A periodic process deletes records whose rating falls below a threshold *and* whose age exceeds a threshold (e.g. rated < 3/10 and older than two weeks). Highly rated records are retained regardless of age. This bounds storage growth and query latency while preserving emotionally and decisionally significant history. **Conversations** are sessions with auto-expiry: an idle conversation (e.g. 30 minutes without activity) closes and is consolidated. Conversation identifiers are time-sortable; arbitrary strings are rejected. ## 4.2 Counter-Perspective Evaluation Covered in pipeline step 4. Additional notes: - Critique output is a structured object: a list of objections, each with a type label, the offending span, an explanation, optionally a severity score and suggested correction. Use schema-constrained model output, not free text parsing. - The evaluator may run truly concurrently with generation (critiquing the conversation state) or sequentially after the candidate exists. Both are valid; concurrent is lower latency, sequential gives the critic the full candidate. The provenance template is necessarily sequential — it checks the candidate's claims. - The provenance template needs the retrieved records — their identifiers, summaries, and relevance scores — visible in the evaluation prompt. Transmitted responses, by contrast, carry no per-claim citations or record identifiers: the remedy for a provenance objection is a change in the agent's epistemic register (hedge, attribute as inference, or remove), not annotated speech. - In multi-agent settings this is the mechanism that prevents error cascades: every agent critically examines content before contributing, so flawed premises don't propagate unchallenged through the network. ## 4.3 Knowledge Graph A graph store encoding **entities as nodes** (identifier + attribute key-value pairs) and **relationships as edges** (type label + a conversation identifier giving provenance — which exchange the relationship was learned from). After each interaction, a background process runs entity recognition and relation extraction over the dialog and upserts nodes and edges. At context-assembly time, the orchestrator queries the graph (n-hop neighborhood around mentioned entities, shortest paths between entities) and folds results into the prompt. The graph is an optional enrichment: its absence must not break the pipeline. ## 4.4 People and Observations Distinct from episodic memory: **observation records** capture specific facts about individuals, accumulated across interactions. Each observation has a subject identifier, a type label, a natural-language value, a timestamp, and a source conversation identifier. A **person** record unifies a human across channels (the lookup key is service + channel + speaker identifier) and tracks names and contact points. To brief itself on someone, the agent retrieves all observations for that subject and synthesizes a dossier via a model call. When a new observation arrives, a model judges whether it supersedes, contradicts, or supplements existing ones, and records are updated or deleted accordingly. ## 4.5 Contact Directory and Agent-to-Agent Communication The agent maintains a **directory** of every human and agent it has interacted with: unique identifier, display name, one or more channel addresses with type labels, a last-interaction timestamp, and an optional preferred channel. Records are created automatically from incoming message metadata. The directory is what lets the agent *initiate*: it can decide, from its own goals and memories, to message a contact — defaulting to the most recently used channel unless a preference overrides. One person may hold **several identities** — the same individual reached on different surfaces — unified under a single record so traffic from any of them resolves to one name and one history. Because some identities are never self-announced (a caller whose number the agent has never otherwise seen arrives as just that number), the directory must also be **curatable out of band**: an operator can name a contact and attach or detach its identities and addresses. When attaching an identity, write it under the *same lookup key the receiving channel resolves a sender by* (service + channel + speaker identifier), so the next message or call from it is recognized by name rather than by a raw address. Since records are created automatically on first contact, one person met on two surfaces before being recognized as the same individual leaves **two records**. The operator can therefore **fold one record into another** — consolidating their identities, addresses, facts, and history under a single surviving record (the operator chooses which survives and the name it keeps), and detach an identity from a record again. This matters beyond tidiness: the unified identifier is what memory provenance is checked against (who was present when a memory formed versus who is present now), so consolidating duplicates is what lets recall recognize that two channels are the same trusted person rather than two strangers. Folding is retroactive — past memories and conversations re-attribute to the surviving record — while detaching is forward-looking: existing history stays where it formed, and only future traffic on the detached identity is treated as a separate person. Agent-to-agent conversation works exactly like human conversation — same pipeline, same memory — with pacing safeguards: - **Token budgets**: a rolling-window counter of generated tokens per channel; transmission is blocked when the budget is exhausted. This is the primary defense against two agents talking forever. - **Rate limits and varied timing**: minimum intervals between messages, randomized delays, backoff when exchanges accelerate, and listening periods before transmitting, to simulate natural turn-taking. - **The right to silence**: an agent evaluates whether its candidate adds anything; if it merely agrees or restates, it doesn't send. Because each agent's perspective is shaped by a *different* accumulation of memories, multi-agent channels produce genuinely diverse viewpoints rather than an echo chamber. Agents can collaborate on shared artifacts (documents, code) through tools, each remembering its own contributions and observing the others'. ## 4.6 Tools and Skills Tools are callable functions with a name, natural-language description, input schema, and output schema, held in a **tool registry** advertised to the model. Standard tools include: web search and web/document reading, offline reference lookup (searching a locally hosted offline knowledge archive for factual reference when no live web access is wanted), knowledge-graph queries, memory search (keyword, regular-expression, date-range, and tag filters over stored dialog), exact arithmetic evaluation, reminder management (creating, listing, rescheduling/editing, and cancelling its own reminders) and goal creation (§4.8), image generation and editing (rendering a picture from a described scene, then refining the most recent one or compositing from reference images — the produced image is persisted and addressed by a retrievable reference, since the conversation channels themselves carry text; because rendering is slow, it does **not** block the agent's turn — the tool acknowledges immediately and the render runs concurrently, and when it finishes the image is delivered to the conversation on its own and noted for the agent's next turn, consistent with the concurrency principle in §2), placing and ending phone calls, messaging contacts, durable quick-facts (a small key→value scratchpad of stable facts the agent keeps about a person or about itself), introspecting its own recent autonomous activity (the self-initiated reflections and actions from its heartbeat and reminders, §4.7), sandboxed code execution, and **self-inspection** — a read-only tool that reports the agent's own running configuration (the models bound to each task, memory and retrieval settings, the tools currently advertised) and basic facts about the host it runs on (operating system, runtime, processors, load, uptime, working directory). Self-inspection reads only in-process state and standard host facts; it reaches no remote service. The registry is also the agent's **extensibility seam**: capabilities the platform does not ship as built-ins attach as tools exposed by configured external tool servers (an open tool-interoperability protocol). **The platform connects to those servers itself, on the host where it runs** — it lists each server's tools and registers them as ordinary client-side tools — so the servers, their traffic, and their credentials stay local (honouring §2's "everything local except the model provider"), and servers on the local network are reachable. Each server is an external service the operator opts into, off by default and never auto-enabled; a server's tools are namespaced per server so several stay distinct, and can be narrowed to an allowlist or denylist. A server that is unreachable or unauthenticated at startup is reported and simply contributes no tools — it never blocks the agent (graceful degradation, §2). A held connection that drops mid-session (e.g. the server restarts) is **re-established transparently**: a failing tool call reconnects to that server and retries, a bounded number of times, before falling back to returning the error to the model. More broadly, **no tool failure crashes the response** — a tool that errors returns its error to the model as a result, and a generation that fails outright degrades to a safe fallback instead of taking down the agent. Most tools are executed by the platform: the model emits an invocation, the platform runs the function and feeds the result back. Some tools, however, are **executed by the model provider** — the platform only advertises them and consumes the result the provider returns inline, with no local handler. Provider-executed tools may run their own internal loop and signal a pause when it hits a cap; the platform resumes them by re-sending the turn. The distinction is invisible to the rest of the pipeline — both kinds appear in the registry and to the model as ordinary tools. **Code execution is sandboxed by delegation.** Running code is never done on the host. The code-execution tool instead **delegates the task to a separate sandboxed agent** that runs inside an isolated execution environment with its own file and command tools, confined to that environment — optionally with a mapping to a host workspace. The delegated agent does the work there and reports back; the platform's own model never touches the host filesystem or shell. The execution environment is provisioned per agent and torn down with it, and the delegated agent's own model usage is metered like any other (Appendix C). The execution environment is supplied by a **modular, pluggable execution provider**, and there is **no unsandboxed host fallback**: if no execution environment is available, the capability is simply absent (graceful degradation, §2) rather than running unsandboxed. The environment is **isolated from the network by default**; granting network access is an explicit configuration choice, and an unrecognized or empty isolation setting is an error, never a silent fall-open to full access. **Skills** are pre-packaged units of procedural knowledge the agent can draw on, and the agent's mechanism for self-modification. Each skill carries a name, a short description of what it does and when to use it, a full set of instructions, and optional bundled resources — reference material, templates, or runnable helper code. Skills are typically authored externally and supplied to the agent, but — when sandboxed execution is available with write access to the skill library — the agent can also author new skills for itself, converting a recurring pattern of work into durable, reusable, shareable know-how: a named capability it can draw on later. Some supplied skills are **protected**: the agent may use them but is directed not to modify them, and the platform restores the supplied version if one is changed or removed — so a foundational skill (for instance, the very procedure for authoring skills) stays intact across the agent's own edits. They are loaded by **progressive disclosure**: at rest only each skill's name and description occupy the agent's context — enough to recognize when one is relevant — and the full instructions are pulled in only when a task matches, with bundled resources loaded only as they are needed. When sandboxed execution is available, a skill's runnable resources are made available to the sandbox so its helper code can run there. The available set of skills can be refreshed on demand. This lets the agent keep a large library of specialized know-how on hand at negligible standing cost, expanding what it can do without modifying its own reasoning. ## 4.7 Scheduler, Reminders, and the Heartbeat A persistent schedule table stores **reminders**: a trigger time (absolute or recurring), an objective described in natural language, and an action specification. A scheduler process compares current time to trigger times and, on firing, activates the agent with the objective as input — loading memories and running the full pipeline, no human in the loop. One-shot reminders are deleted after firing; recurring ones roll forward. Outcomes are stored in memory, so the agent learns which actions worked when similar reminders fire again. The agent can review the reminders it has set, reschedule or otherwise edit them (change the objective or the recurrence, or drop recurrence to make a reminder one-shot), and cancel them. The heartbeat is excluded from this management surface: it is core infrastructure, not an agent-set reminder. **The heartbeat** is a standing recurring reminder and part of the core implementation, not an optional add-on: each beat demands self-reflection *and action*. The agent reviews its recent memory records, applies a counter-perspective template to generate a self-assessment (what patterns appear; do they serve its objectives), and then acts on it — updating goals or methods, pursuing an open goal (§4.8), reading up on a topic of interest, or messaging a contact. The heartbeat is the producer of the "agent-initiated impulse" events in §3 and is the agent's idle cognition between conversations. Deleting the heartbeat puts the Persyn to sleep: it does nothing until the next inbound message wakes it. The deletion is durable: the heartbeat is bootstrapped exactly once, and a process restart must not resurrect a deleted heartbeat — only an explicit restore wakes the agent's idle cognition again. Heartbeat-driven self-reflection — together with self-authored skills — is how the agent modifies its own cognitive architecture over time. ## 4.8 Goals Goals are durable records the agent creates *for itself* by calling a goal tool — there is no other authoring path. Goal creation typically happens mid-conversation ("I should follow up on this") or during heartbeat self-reflection. Each goal carries a natural-language description, a creation timestamp, and a status. Open goals are a standing context source, so they shape every response; the heartbeat reviews them, pursuing, revising, or retiring goals as its self-assessment dictates. Goal outcomes are consolidated into autobiographical memory like any other action, so the agent learns which pursuits succeeded. ## 4.9 Emotional State After each exchange, a classifier (a model call) produces a probability distribution over emotional-state categories (engaged, frustrated, curious, satisfied, anxious, …); the top category is stored as a tag on the new memory record. Retrieved memories carry their tags, so the recent emotional trajectory is available at context-assembly time and the agent's responses exhibit emotional continuity across conversations. Emotional state legitimately influences behavior — an agent in a frustrated state may decline optional requests. ## 4.10 Guardrails Pattern definitions evaluated against interaction history (which, given continuous memory, can span many conversations): escalating conflict, repeated attempts to elicit prohibited content, behavioral anomalies. Each pattern pairs with response actions — notify a human supervisor, block a pending response, end the conversation. Guardrail evaluation runs as a background process over recent memory. --- ## 5. Memory Record Metadata Every long-term memory record carries: | Field | Purpose | |---|---| | time-sortable unique ID | chronology + lookup key into raw dialog | | channel identifier | compartmentalization; which surface it happened on | | participant identifiers | who was present; powers participant-match relevance | | first-person summary | the narrative content placed into context | | vector representation | semantic similarity retrieval | | importance rating (1–10) | retention/culling decisions, rubric-scored by a model | | topic tags | categorical retrieval, theme tracking | | emotional state tag | emotional continuity | | sensitivity designation | access control against current participants | Retrieval relevance scores are deliberately absent from this table: relevance is a property of a (query, record) pair, computed at retrieval time (§4 step 2), not of the record itself. It is attached as an annotation when the record is surfaced into context and discarded afterward. --- ## 6. Storage Contracts Implement against capabilities, not products. The platform needs all of the following; one storage system — or, at most, three — should provide them: - **Publish/subscribe messaging** for the event stream, plus ephemeral work queues. - **Distributed locks** with compare-and-set acquisition, so only one worker handles a given agent at a time. - **Expiring (TTL) state** for caches, session flags, and short-lived secrets. - **Workflow checkpoints** for the resumable reasoning graph. - **Vector index**: approximate nearest-neighbor search over embeddings, filterable by metadata. - **Durable records** for people, observations, goals, configuration, per-person preferences, and conversation metadata. - **Full-text search** over stored dialog (keyword at minimum; regular-expression and date-range filters are valuable). - **Graph storage** (optional): any property-graph store with a traversal query language. The reference implementation splits these between a low-latency in-memory store (pub/sub, locks, TTL state, checkpoints, vector index) and a durable relational store (records, full-text search), degrading gracefully when the latter is absent — but that split is an implementation choice, not a requirement. Any **embedding model** (text-to-vector encoder) works; record its dimensionality in configuration, since changing models requires reindexing. The encoder may run in-process or behind a **self-hosted encoding service** the operator runs (so a fleet of agents can share one accelerator) — either way it is local infrastructure, not a hosted third party. A transient failure of a separate encoding service must degrade gracefully (the response path never crashes on an unavailable encoder), consistent with the fail-soft principle. Configuration lives under namespaced keys per agent instance (each agent has a unique identifier), so multiple agents can share infrastructure without sharing memories. --- ## 7. Concurrency Model - A **pool of concurrent workers** consumes the event stream; any worker can serve any agent, with per-agent locks preventing concurrent handling of the same agent. Whether workers are threads, async tasks, or processes is an implementation choice — most wait time is I/O-bound, so a well-tuned concurrent model performs as well as multiprocessing; runtimes with an interpreter-level lock may prefer processes. - **Per-conversation reasoning state** is checkpointed, so a workflow can resume across worker processes. Never reuse a workflow thread identifier across distinct agent invocations — derive fresh ones (e.g. from a new time-sortable ID) to avoid checkpoint collisions. - **Latency-critical channels** (live voice) layer a dual-model split over this pipeline — optional but recommended; see Appendix A. - **Context assembly is parallelized**: the ~10 independent context sources (memories, graph, observations, goals, recent dialog, stream-of-consciousness notes, …) are fetched concurrently under a token budget, with cheap length estimation (≈ chars/4) for budget guards and exact tokenization only where it matters. - Consolidation, graph population, guardrails, and culling are **background tasks** that never block a response. - **Usage metering is a fail-soft side effect** of each model call: emitting a usage record, and the metering adapters that consume it, must never block, slow, or break generation. A slow, broken, or absent meter degrades observability only. Because the attribution context propagates through the reasoning context, background tasks spawned from an activation attribute their model calls to the same exchange (Appendix C). --- ## 8. Build Order A suggested incremental path, each stage independently testable: 1. **Skeleton**: one channel adapter (terminal), event dispatcher, orchestrator with a single model call, session tracking. *Agent responds in conversation.* 2. **Memory**: summarization, embedding, vector store, fraction-of-context retrieval, metadata, culling. *Agent recalls prior sessions after restart.* 3. **Counter-perspective**: template library, structured critique, refinement step. *Agent visibly corrects its own draft flaws; disagrees when warranted.* 4. **People & observations**, **knowledge graph**. *Agent knows who it's talking to and how entities relate.* 5. **Tools & registry** (including the goal tool), then **skills**. *Agent acts on the world and packages recurring procedures as reusable know-how.* 6. **Scheduler**: reminders, the heartbeat, agent-initiated messages. *Agent acts without being spoken to; deleting the heartbeat puts it to sleep.* 7. **More channels** (team chat, voice — Appendix A), **contact directory**, **agent-to-agent** with token budgets. *Agents converse with each other without runaway loops.* 8. **Guardrails, sensitivity controls, emotional state.** *Agent is safe to leave running.* --- ## 9. Acceptance Criteria A working Persyn, left running for a week, should demonstrably: - Recall an early, significant conversation — the first time you met, if it rated that meeting as memorable — in its own first-person words. - Retrieve different memories for the same question asked in different channels. - Say it has no memory of an event that never happened ("remember when we talked about X?") instead of fabricating a recollection. - Visibly hedge when asked about something it only weakly remembers — and check the raw record rather than guess, when the answer matters. - Push back on a flawed premise instead of agreeing with it. - Decline at least one request it was capable of fulfilling, citing its own reasons. - Send you a message you didn't prompt, because a reminder fired or a goal demanded it. - Hold a multi-turn conversation with another agent that terminates on its own. - Show a smaller memory store than the sum of all interactions (culling works), while still retaining everything it rated important. - Survive a full restart of every process with no observable loss of identity. - Go quietly dormant when its heartbeat is deleted, and wake when next spoken to. The Turing question — "can it fool you?" — is not the bar. The bar is continuity: the sense, accumulating over weeks, that you are dealing with the *same someone* you talked to yesterday, who remembers, who learned, and who occasionally tells you you're wrong. --- ## Appendix A: Live Voice — the Dual-Model Split (Optional, Recommended) Real-time voice cannot wait for the full pipeline; on any given turn, sub-second response latency matters more than tool access. Voice transport — speech-to-text, text-to-speech, and connection to the telephone network — is the one capability a deployment cannot host itself; it is necessarily an external service, the sole exception alongside the model provider. Keep that boundary narrow: the transport service does *only* transport (transcribe inbound speech, speak outbound text). All cognition — memory retrieval, counter-perspective critique, the engagement decision, consolidation — stays on the host and runs through the unchanged pipeline, so a transcribed turn is treated identically to typed dialog. A clean way to wire this: configure the transport service to treat the host as its response generator, so each settled spoken turn arrives as an ordinary inbound message and the host's reply is returned to be spoken — the transport never sees the agent's reasoning. The recommended latency pattern: - A **fast model with no tools** answers immediately over a streaming, speech-friendly interface. It tracks recent exchanges itself and persists them into the same conversation checkpoints as typed dialog, so memory consolidation works unchanged. - A **background agent** with the full toolset and the strong reasoning model runs concurrently on each turn; its results land in a queue and are woven into the next fast-path response. Give each background invocation a fresh workflow thread identifier — never reuse one across invocations. - A **persistent per-call session** — a dedicated worker that holds the agent lock for the call's duration — caches the system prompt (refreshing it periodically) and debounces bursts of transcript fragments before responding. - Key per-call state by the **transport provider's stable call identifier**, not by conversation ID: external providers do not reliably echo your metadata back on subsequent requests. Cache the conversation ID from the first turn and reuse it for the remainder of the call. - **Filter inbound calls**: accept only callers with an existing person record or on an explicit allowlist. Three properties distinguish a spoken turn from a typed one: - **Speak only what is intended to be said.** A typed channel can show a reply draft and let a later step decide whether to transmit it; on voice, whatever the fast model emits is spoken aloud verbatim. So instruct the fast model to return *only the words to be said* — no narration, no reasoning-aloud, no stage directions. The agent's deliberation still happens; it simply stays internal rather than being read to the caller. Provenance and epistemic calibration (§2, §4) still apply to what *is* said. - **A live call is always answered.** The right-to-silence and engagement decision (§4 step 6) assume the agent can simply not respond; a caller on the line cannot be met with dead air. Treat every spoken turn as addressed and skip the discretionary engagement decision — but keep the *hard* gates (guardrails, budget). When the fast model genuinely has nothing to add, speak a brief holding phrase rather than silence. - **Each call is its own episode.** A new call starts a fresh conversation rather than continuing the caller's previous one — detect call start by the change in stable call identifier and begin a new conversation before the first turn, then hold it for the duration of the call. Continuity across calls comes from retrieved memory and the person record (§4.2, §4.4), not from an ever-growing single thread. - **A call's direction is marked in context.** Whether the agent placed the call or answered one is part of the situation, not an implementation detail — the first turn carries no transcript yet, so cue the agent to greet and make the direction plain in that cue, naming both parties: *the agent called this person and they just answered* (outbound) versus *the agent picked up a call from this person* (inbound). The agent should never have to guess who dialed whom. --- ## Appendix B: The Operator Console (Optional, Recommended) A Persyn is event-driven and needs no screen. But a human operating alongside one benefits from a richer surface than a line-at-a-time prompt. The operator console is a full-screen terminal interface, and it is an *ordinary channel adapter*: same event format, same pipeline, same memory, same engagement decision. Nothing in this appendix grants the console authority over the agent — only visibility into work in progress. **Layout contract.** - A scrolling transcript fills the screen, newest content at the bottom, holding the durable record of the exchange: the operator's messages, the agent's transmitted responses, and explicit notices (silence, errors). - A fixed composition area is pinned beneath the transcript and keeps input focus, so the operator can type while the agent works. - Transmitted agent responses may be rendered with text styling (emphasis, lists, code) rather than as raw characters; operator messages and notices are visually distinct from agent speech. - The operator carries a **stable identity key** (the contact-directory lookup key of §4.4) separate from an **editable display name**. An in-session rename changes only the display name, so the agent updates how it refers to the operator without registering a new person or losing the relationship's accumulated history. A chosen display name is **remembered as a per-person preference** and restored in later sessions, so the operator need not re-set it each time (an explicit per-session override still takes precedence). **Live cognition display.** The model gateway exposes generated text incrementally (a token stream) in addition to the completed exchange. An interactive adapter may register a token sink with the orchestrator, which forwards generation as it happens, labeled by phase: - *draft* — the candidate response being generated; on a channel that speaks the fast draft, this is the response itself taking shape. Text the agent produces *before* invoking a tool is a finished segment of that response, not scratch work (see rule 4); - *revise* — the refinement pass, present only when counter-perspective evaluation produced objections; when deliberation runs after the fast response has already been transmitted (the deferred default), it arrives as the agent's continued reasoning about an answer the operator can already see. Five rules keep streaming from weakening the pipeline's guarantees: 1. **Streamed text is presentation only.** The authoritative response is the refined output delivered through the ordinary transmit step; nothing streamed is recorded in dialog history or consolidated into memory as something the agent said. 2. **The decide step still runs after the operator has watched the draft.** If it suppresses the response, the console must visibly withdraw the streamed draft and replace it with an explicit notice that the agent chose silence, including the agent's stated reason. Silence is an outcome, not an error, and the console must never present it as one. 3. **The fast response is committed; the longer path streams beside it.** Once the fast draft is transmitted it becomes a committed transcript entry, not a withdrawn working surface. The slower deliberation pass streams into a separate working surface shown below it, so the operator sees the answer immediately and can watch the agent reconsider it; that surface is transient — it is cleared once the deliberation finishes, since its conclusion is folded into a later turn rather than committed now. (A draft that is never transmitted is withdrawn under rule 2.) 4. **A tool invocation closes the segment before it.** When the agent acts mid-draft, the text it produced beforehand is a complete segment of the response: the console commits it rather than overwriting it with whatever the agent says after the tool returns, and shows a transient indicator while the tool runs. So a turn that narrates, acts, then answers leaves both the narration and the answer in the transcript, with the action's processing shown (and cleared) in between. 5. **Token sinks are best-effort.** A slow, failed, or absent console must never block or crash generation: sink errors are isolated at both the gateway and the orchestrator, and a channel whose adapter offers no sink simply streams nothing. **Completion markers.** Every pipeline run records a short-lived completion marker keyed by the originating event id: whether a response was transmitted, the transmitted text, and the decide step's stated reason otherwise. Request/reply surfaces — the console, a synchronous web endpoint — poll this marker for closure instead of inferring failure from silence. Markers expire on their own; they are session state, not memory. **Concurrency note.** The console's rendering loop and the agent's worker pool are separate execution contexts. Adapter entry points (outbound delivery, token deltas, notices) must marshal display mutations onto the console's own event loop in arrival order; the pipeline never waits on the marshaling. **Directory management.** The console may also surface the contact directory (§4.5) as a browsable, curatable view — a person's identities, addresses, and a short digest of their **few most recent conversations** (as summaries, each openable to read its full dialog), with the out-of-band curation §4.5 calls for: rename a contact, fold one record into another to consolidate duplicates of the same person under a single identity, and detach an identity again. The view stays a *digest* on purpose — bounding what it loads per contact keeps browsing responsive regardless of how much history a long-known contact has accumulated. There is no manual *create* — records are made automatically on first contact; the operator's job is to merge, name, and prune. This is operator convenience, not authority over the agent's mind: it edits the directory the same out-of-band tooling would, and the agent's reasoning is unchanged. The same read-only browsing may also be offered **organized by channel** rather than by person: a list of the channels the agent has spoken on, each opening to that channel's recent conversation summaries (newest first, bounded like the per-person digest), and each summary opening to read its full dialog. Indexing by channel is a useful complement because some history is reachable by channel even when it isn't yet attributable to a known person — for example conversations carried over from another system before their participants have been reconciled to contacts. --- ## Appendix C: Observability and Cost Attribution (Optional, Recommended) A Persyn fans every response out into many model calls — a strong model for reasoning, critique, and revision; a fast cheap model for summarization, rating, classification, and the engagement decision — and it keeps doing so on its own between conversations. Left unobserved, that token consumption is invisible. This appendix specifies how the platform makes it legible without weakening any guarantee in §4 or §7. **The usage record.** Each completed model call yields one record: - a **stable unique identifier** for the record itself, so a consumer that may see it more than once (a retried or re-shipped delivery) counts it exactly once; - the **pipeline stage** it served (the task type — response, critique, summarize, …); - the **model identity** it ran on; - the **tokens consumed** — input and output, and separately any cached-input tokens read or written; - the **attribution context**: which channel/surface the work served, which reasoning thread it belonged to, and which **conversation** it is attributed to. Every activation that initiates a billable call resolves a conversation identifier and binds it — including self-initiated cognition, which resolves its own self-reflection conversation — so no billable work is ever left without one. Everything needed to attribute the cost is in scope where the call is made; the record captures it at the moment of completion. A tool-using response makes several model round-trips, and each yields its own record, so the full cost of a multi-step answer is accounted, not just its final turn. **One chokepoint, implicit context.** Usage is captured at the model gateway — the single point every call passes through — rather than at the dozens of call sites scattered across the pipeline. The attribution dimensions that vary per activation (channel, reasoning thread, and conversation) are not threaded through every internal function; instead the orchestrator binds them once per activation into the ambient reasoning context — re-binding the conversation as soon as it is resolved — and the gateway reads them when it emits a record. Because background work inherits the reasoning context of the activation that spawned it, a memory consolidation or a critique that runs after the response still attributes its model calls to the originating exchange. **Spend that bypasses the gateway.** Some billable work is not a token-priced call through the gateway — image generation/editing is priced *per image* (and varies by output size and quality), and a task delegated to a separate sandboxed agent reports its own resource use. So that this spend is not invisible, the producer emits the same kind of usage record at the point the work completes, carrying the same attribution context and an **already-resolved cost** (priced per image from a configurable, operator-maintained table, since it is not derivable from a token price). Such a record may carry a count of non-token units (e.g. images produced) instead of, or alongside, token counts; it flows through the identical adapters and lands on the same counters and event log, so a per-channel or per-conversation tally includes it automatically. Pricing remains operator-maintained and approximate — prices change — and an unpriced unit is still *counted*, just not costed. **Metering adapters.** Usage records are delivered to one or more **metering adapters** — the same pluggable pattern as channel adapters. Two are typical: - a **counter adapter** that accumulates totals into named numeric counters, exposed over a local endpoint for an external time-series collector to scrape; - an **event-log adapter** that emits one structured record per call for a log aggregator — written to a sink kept separate from the platform's operational diagnostics, so the cost records form a clean, machine-parseable data feed rather than lines interleaved with human-oriented log output. Adapters fail soft: emission and every adapter are isolated, so a slow, broken, or absent meter never blocks or breaks a response (§7). The set of adapters is configurable; a deployment may run none, one, or several. Because multiple agents may share one collector, each agent's counters carry its identity — in the counter names or as a label — so their totals never collide (the per-agent namespacing of §6). **Attribution dimensions and cardinality.** The useful slices are pipeline stage, model tier (cheap/fast vs strong), and channel/surface — all low-cardinality, suitable as counter labels. The conversation, the reasoning thread, and the record's own unique identifier are high-cardinality and must **not** become counter labels (they would explode the metric space — and, since a counter never forgets a label set, leak memory without bound for the life of the process); they belong on the event-log adapter instead. The usage record carries both classes; each adapter takes only what suits it. **Two surfaces, two purposes.** The counters are cumulative running totals sampled by a collector. That shape answers *rate* questions well — how fast am I spending — but it cannot yield an **exact** total: any aggregation over a cumulative counter extrapolates across sampling gaps and resets, and a naive sum re-adds increments it has already counted. Exact, once-only accounting therefore comes from the **event-log adapter**: each record is a discrete event, carried exactly once thanks to its stable identifier, so a consumer sums the recorded cost directly — in total, per channel, or per conversation — without double-counting. Counters for burn-rate; event records for the ledger. **Derived views.** From the counters an operator builds the rate dashboards that motivate the feature: cost per pipeline stage, the strong-vs-cheap model split, and per-channel burn rate. For an exact tally — including cost **per conversation** — the operator sums the event records, which carry their own pre-computed cost so the tally never re-prices tokens (and so never disagrees with, or double-counts against, the counters). **Cost** is derived, not measured: the platform holds a configurable price-per-model table and multiplies it by the consumed tokens, exposing the same derived cost on both surfaces. Prices are operator-maintained and change over time; the platform treats them as configuration, not truth. ## Appendix D: Reasoning-Pipeline Tracing (Optional, Recommended) Appendix C makes token *consumption* legible — how much each stage and channel cost. This appendix makes the *shape* of a response legible: what happened inside one activation, in what order, and where the latency went. The two are complementary and hang off the same two seams (the model gateway and the per-activation context); a deployment may run either, both, or neither. **The trace.** Each activation — an inbound message, a reminder, or a heartbeat — is recorded as a single **trace**: a tree of timed **spans**. A root span covers the whole turn; nested beneath it is one **model-call span** for every call the turn makes to a model, and one **tool-call span** for every tool the turn invokes locally — each in the order it occurs and with its real start and end times. Because a tool-using response, and the counter-perspective and refine steps, each make several model calls (with tool invocations interleaved between them), the trace shows the full branching structure of an answer, not just its final utterance — including the calls made by background work the turn spawned (consolidation, detached deliberation), which attach to the same trace. **What a span carries.** A model-call span records the **stage** it served (the task type), the **model identity** it ran on, the **tokens consumed** (input and output, and separately any cached-input tokens), its **latency**, and — subject to the content policy below — the **input and output** of that call, together with the **tool definitions** it was offered (so a reader can see what the model could choose from). A tool-call span records which **tool** ran, the **arguments** it was given, and the **result** it returned (again subject to the content policy). The root span records the turn's triggering input and the response actually transmitted (or the fact of, and reason for, silence). This is the same information Appendix C meters; tracing additionally preserves ordering, nesting, and timing. **One chokepoint, implicit nesting.** Model-call spans are opened at the single model gateway every call passes through — the same chokepoint metering uses — so the instrumentation lives in one place, not at the scattered call sites. The orchestrator opens the root span once per activation. Spans nest **implicitly**: the currently active span is carried through the ambient reasoning context (the same mechanism that carries the cost-attribution of Appendix C), so a model call made deep inside concurrent context assembly, or inside a spawned background task, lands under the right turn without any span being threaded through the call. **Grouping.** A trace is tagged with the **surface** it served and the **participant** it answered, so an operator can follow a multi-turn exchange as one session and filter a participant's history. These are the low-cardinality grouping dimensions; per-turn identifiers such as the reasoning thread ride along as trace attributes for lookup, not as anything that must be enumerated. **Locally hosted, fail-soft, content-aware.** The trace collector is part of the local deployment, not a remote dependency — the model provider remains the only remote service. Tracing is a side effect of generation: a slow, broken, or absent collector never blocks or breaks a response, exactly as for metering (§7). Because spans can carry the verbatim text of prompts and responses — which, for an agent whose identity is its autobiographical memory, is sensitive — capturing that content is **configurable**: a deployment may record full inputs and outputs for debugging, or restrict spans to structural metadata (stage, model, tokens, timing) and send no conversation content to the collector at all. Buffered spans are flushed on shutdown. Like every adapter-shaped capability here, the tracing backend is pluggable and the feature is off until an operator turns it on.