Skip to main content

Prompt Management Architecture

Overview​

This document outlines the unified prompt management strategy for OCapistaine, integrating:

  • Opik for versioning, optimization, and experiments
  • Vaettir MCP for centralized prompt storage and retrieval (future)
  • Local JSON/Python prompts as fallback during development

Design Principles​

Prompt Separation​

Prompts are separated into two concerns:

  1. Persona Prompt (system message): Defines WHO the agent is

    • Identity, values, response style
    • Shared across all features

  2. Feature Prompts (user message): Defines WHAT to do

    • Specific task instructions
    • Expected output format
    • Feature-specific context

This separation allows persona to evolve independently from task logic, and for the same persona to be composed with different features.

Three-Layer Architecture​

┌───────────────────────────────────────────────────────┐
│              PROMPT MANAGEMENT LAYERS                  │
├───────────────────────────────────────────────────────┤
│                                                       │
│  LAYER 1: Local Registry             ✅ COMPLETE      │
│  Central prompt access with versioning                │
│  Sources: Local JSON → Local Python → Remote Opik     │
│                                                       │
│  LAYER 2: Opik Integration           ✅ COMPLETE      │
│  Bidirectional sync, optimization, experiments        │
│  Composites for playground testing                    │
│                                                       │
│  LAYER 3: Vaettir MCP                🔜 PLANNED       │
│  Cross-agent prompt sharing (Python + N8N)            │
│  Centralized in the vaettir realm                     │
│                                                       │
└───────────────────────────────────────────────────────┘

Bidirectional Sync​

Prompts flow in both directions between local development and the Opik platform:

  • Push: Local edits are synced to Opik for versioning and experiment tracking
  • Pull: Optimizations made in the Opik Playground are pulled back to update local sources

This closes the optimization loop — improvements discovered through experimentation automatically propagate to the codebase.

Key constraint: Individual prompts are shared across composite prompts. Updating a persona prompt affects all composites that use it. The sync system warns when shared prompts change.

Optimization​

The system supports multiple optimization strategies through the Opik optimizer:

StrategyApproach
Few-shot BayesianSelect best examples for few-shot prompts
Meta-promptLLM-generated prompt improvements
EvolutionaryGenetic algorithm for prompt evolution

Optimization results are tracked per-experiment with scoring metrics, allowing data-driven prompt evolution over time.

Migration Status​

PhaseStatusDescription
1. ConsolidateCompleteUnified registry, shared constants
2. Opik IntegrationCompleteSync, optimization, experiments, bidirectional pull
3. Agent UpdateIn progressMigrate agents to chat-format registry calls
4. Vaettir MCPPlannedCross-agent sharing via MCP tools on vaettir

Phase 4 will move prompt storage to the vaettir realm, enabling N8N workflows and Python agents to share the same prompt versions through a unified MCP interface.

External References​