Prompt Management Architecture
Overview
This document outlines the unified prompt management strategy for OCapistaine, integrating:
- Opik for versioning, optimization, and experiments
- Vaettir MCP for centralized prompt storage and retrieval (future)
- Local JSON/Python prompts as fallback during development
Design Principles
Prompt Separation
Prompts are separated into two concerns:
-
Persona Prompt (system message): Defines WHO the agent is
- Identity, values, response style
- Shared across all features
-
Feature Prompts (user message): Defines WHAT to do
- Specific task instructions
- Expected output format
- Feature-specific context
This separation allows persona to evolve independently from task logic, and for the same persona to be composed with different features.
Three-Layer Architecture
┌───────────────────────────────────────────────────────┐
│ PROMPT MANAGEMENT LAYERS │
├───────────────────────────────────────────────────────┤
│ │
│ LAYER 1: Local Registry ✅ COMPLETE │
│ Central prompt access with versioning │
│ Sources: Local JSON → Local Python → Remote Opik │
│ │
│ LAYER 2: Opik Integration ✅ COMPLETE │
│ Bidirectional sync, optimization, experiments │
│ Composites for playground testing │
│ │
│ LAYER 3: Vaettir MCP 🔜 PLANNED │
│ Cross-agent prompt sharing (Python + N8N) │
│ Centralized in the vaettir realm │
│ │
└───────────────────────────────────────────────────────┘
Bidirectional Sync
Prompts flow in both directions between local development and the Opik platform:
- Push: Local edits are synced to Opik for versioning and experiment tracking
- Pull: Optimizations made in the Opik Playground are pulled back to update local sources
This closes the optimization loop — improvements discovered through experimentation automatically propagate to the codebase.
Key constraint: Individual prompts are shared across composite prompts. Updating a persona prompt affects all composites that use it. The sync system warns when shared prompts change.
Optimization
The system supports multiple optimization strategies through the Opik optimizer:
| Strategy | Approach |
|---|---|
| Few-shot Bayesian | Select best examples for few-shot prompts |
| Meta-prompt | LLM-generated prompt improvements |
| Evolutionary | Genetic algorithm for prompt evolution |
Optimization results are tracked per-experiment with scoring metrics, allowing data-driven prompt evolution over time.
Migration Status
| Phase | Status | Description |
|---|---|---|
| 1. Consolidate | Complete | Unified registry, shared constants |
| 2. Opik Integration | Complete | Sync, optimization, experiments, bidirectional pull |
| 3. Agent Update | In progress | Migrate agents to chat-format registry calls |
| 4. Vaettir MCP | Planned | Cross-agent sharing via MCP tools on vaettir |
Phase 4 will move prompt storage to the vaettir realm, enabling N8N workflows and Python agents to share the same prompt versions through a unified MCP interface.
Related Documentation
- Forseti Agent - Agent architecture and feature system
- Auto-Contributions - Auto-contribution workflow