project-page active pal-e-docs
project-pal-e-docs updated 2026-03-21

pal-e-docs

Vision

A DORA Elite AI Enterprise. pal-e-docs is an AI-native knowledge engine that doubles as an SRE command center and project management platform. It doesn't just store knowledge — it understands it. AI agents ask questions and get precise, ranked, section-level answers. The database holds the intelligence: full-text indexes, structured content blocks, compiled pages. Every note written automatically becomes searchable and navigable. The system gets smarter as it grows.

Two access paths to the same data: AI agents use MCP tools, humans browse the rendered frontend. Both hit the same FastAPI REST API backed by Postgres (CloudNativePG on k3s). The frontend is SvelteKit — enabling interactive components like kanban boards alongside document rendering, all driven by note_type.

User Stories

Who uses the knowledge platform, what they need, and how we measure success. pal-e-docs serves three roles: the Superuser managing the knowledge base, agents querying it, and future external readers browsing public content.

Role Story Success Metric story:X key
Superuser (Lucas) I can query the knowledge base by meaning (semantic search) and find any SOP, plan, convention, or decision without remembering the exact slug. Semantic search returns relevant results in top 5. Zero "I know we documented this but can't find it" moments. story:superuser-query
Superuser (Lucas) I can maintain the knowledge base — create notes, update blocks, manage boards — through MCP tools without touching the database directly. All CRUD operations available via MCP. Zero direct SQL needed for routine operations. story:superuser-maintain
Agent (Betty Sue, Dottie, Dev, QA) I can read SOPs, plans, and conventions via MCP tools to inform my work. Block-first access keeps token costs low. get_section returns <500 tokens. Agents never need get_note for large notes. story:agent-read
Agent (Betty Sue) I can create and update notes, phases, and board items via MCP tools to track work progress. Templates are enforced by hooks. All create/update operations template-validated. Zero malformed notes in production. story:agent-write
Reader (future) I can browse public notes, plans, and project pages in a web UI without authentication. Public pages load in <2s. Navigation is intuitive (TOC, search, browse by project). story:reader-browse

Plan

Active: plan-pal-e-docs — Interactive Knowledge Platform

24+ phases. Original 10 phases (0–4 board/frontend scaffold, plus infra/automation) all completed. Frontend feature phases F1–F10 completed. F11 (Design System Overhaul) and F13 (Context Intelligence) in progress. F12 (Semantic Search Recovery) completed.

Completed plans:

Plan Completed Summary
plan-2026-02-26-tf-modularize-postgres 2026-03-13 Act 1 (SQLite → Postgres) + Act 2 (Knowledge Engine: blocks, search, compiled pages, MCP rewrite)
plan-2026-03-01-pal-e-sprints 2026-03-13 Sprint schema, MCP tools, schema expansion. Absorbed into plan-pal-e-docs.
plan-2026-03-03-sprint-workflow-automation 2026-03-13 Workflow automation phases 1-4. Phase 5 reparented into plan-pal-e-docs.
plan-2026-03-01-note-decomposition 2026-03-02 note_type, status, parent_note_id, position columns
plan-2026-02-28-knowledge-system-consolidation 2026-03-01 Schema maturity, tag cleanup, privacy audit
plan-2026-02-27-browse-ux-enhancements 2026-02-28 Recency sort, project detail, mermaid revision
plan-2026-02-27-responsive-design-mobile-ux 2026-02-28 Table wrapping, mobile breakpoints
plan-2026-02-26-browse-frontend-polish 2026-02-27 Mermaid fix, XSS sanitization, auto-link slugs
plan-2026-02-25-private-notes-auth 2026-02-26 Browse auth, private notes, is_public filtering
plan-2026-02-24-docs-foundation 2026-02-25 Conventions, enforcement hooks, template system
plan-2026-02-24-repo-consolidation 2026-02-25 Migrate all repos to Forgejo
plan-2026-03-13-pal-e-frontend 2026-03-13 Killed and absorbed into plan-pal-e-docs during consolidation

Board

board-pal-e-docs — Pal E Docs Board. Continuous kanban. Columns: Backlog → Todo → Next Up → In Progress → Done. Auto-syncs plan phases via sync_board. Forgejo issues auto-sync via sync-issues endpoint.

Status

  • API deployed on k3s, accessible via Tailscale Funnel
  • SvelteKit frontend LIVE at https://pal-e-app.tail5b443a.ts.net — kanban boards with drag-and-drop, dark theme
  • Keycloak OIDC Auth LIVE — write operations require login, reads remain public. Auth.js + Keycloak provider. FAB hidden for unauthenticated users. (PR #24)
  • Frontend search LIVE — full-text, semantic, and hybrid search from the browser. Cmd+K shortcut, URL-persistent query params, mode toggle (PR #16)
  • Board filtering LIVE — type filter pills, hide-done toggle, collapsible columns, board summary card, project mini-board view (PR #17)
  • DORA Dashboard LIVE — cross-project board rollup at /dashboard, needs-attention section, per-project cards with column distribution, deployment frequency (PR #20)
  • Quick-Jot LIVE — FAB button + n shortcut opens modal for quick note creation. Auto-slug, project dropdown, note_type selector, success toast (PR #21)
  • 500+ notes across 13 projects, 16 note types
  • 36 MCP tools — notes CRUD, search, semantic search, blocks, boards, board sync, links, repos, projects, tags, templates
  • 7,600+ content blocks — typed blocks with anchor_ids, compiled pages, block-first access (91% token reduction)
  • Semantic search LIVE — pgvector + Ollama (qwen3-embedding:4b), all blocks embedded, hybrid search (RRF fusion), Prometheus alerting for embedding failures
  • Vector-powered session startup — Dynamic Briefing injects semantically relevant context at session start. Fail-open design.
  • Board auto-sync — POST /boards/{slug}/sync auto-populates phases from plans. update_note hook auto-moves board items on status change.
  • Postgres (CloudNativePG) — WAL archiving to MinIO, daily base backups, PITR verified
  • CI FULLY GREEN — Woodpecker pipeline: check → lint → build → Harbor → deploy-tag → ArgoCD auto-deploy
  • 513 tests passing (backend) + 33 E2E tests (frontend, PR #25 pending)
  • XSS sanitization via nh3, auto-link slug references, Mermaid diagrams, Atkinson Hyperlegible font

Milestones

  • milestone-2026-02-24-project-genesis — FastAPI + Postgres + REST API + first MCP server
  • milestone-2026-03-01-knowledge-engine — Block parser, compiled pages, semantic search (pgvector + Ollama), MCP v0.3.0
  • milestone-2026-03-13-board-system-frontend — Board data model, kanban drag-and-drop, board auto-sync, SvelteKit frontend launch
  • milestone-2026-03-14-frontend-workbench — 8 PRs in one session: search, board filtering, DORA dashboard, Quick-Jot, Keycloak auth, E2E tests, CI green
  • milestone-2026-03-15-knowledge-loop — Semantic search recovery, MEMORY.md diet (224 → 60 lines), vector-powered Dynamic Briefing, convention-memory-scope, template-ticket. The knowledge loop closed.
  • milestone-2026-03-16-knowledge-architecture (ACTIVE) — Milestone hierarchy, knowledge tiering (hot/warm/cold/frozen), gapped integer positions, doc drift cleanup. Plan: plan-2026-03-16-knowledge-architecture

Architecture

erDiagram
    Project ||--o{ Note : contains
    Project ||--o{ Repo : owns
    Project ||--|| Board : has
    Note ||--o{ Block : decomposes
    Note ||--o| CompiledPage : caches
    Note ||--o{ NoteRevision : tracks
    Note }o--o{ Tag : tagged
    Note }o--o{ Note : links
    Note |o--o{ Note : parent-children
    Board ||--o{ BoardItem : contains

    Project {
        string slug PK
        string name
        string platform
        int page_note_id FK
    }
    Note {
        string slug PK
        string note_type
        string status
        int project_id FK
        int parent_note_id FK
        int position
        text html_content
    }
    Block {
        int id PK
        int note_id FK
        string block_type
        string anchor_id UK
        json content
        vector embedding
    }
    Board {
        string slug PK
        int project_id FK
        string name
    }
    BoardItem {
        int id PK
        int board_id FK
        string item_type
        string column
        int position
        string note_slug
    }
    Tag {
        string name PK
    }
    Repo {
        string slug PK
        string platform
        string url
        int project_id FK
    }

Domain Model. Notes are the universal document type — 16 note_type values (plan, phase, sop, convention, template, agent, skill, todo, doc, etc.) all stored in one table, differentiated by type. Notes decompose into typed Blocks (heading, paragraph, list, table, code, mermaid) with anchor_id for direct section access. CompiledPages cache rendered HTML. Parent-child relationships via parent_note_id create the plan → phase → subphase hierarchy. Boards provide kanban views per project, with BoardItems linking to notes via note_slug or Forgejo issues via forgejo_issue_url. Blocks carry 768-dim pgvector embeddings for semantic search.

flowchart LR
    subgraph Consumers
        AGENT[AI Agent
Claude Code]
        HUMAN[Human
Browser]
    end

    subgraph Access Layer
        MCP[pal-e-docs-mcp
36 MCP tools]
        APP[pal-e-app
SvelteKit SSR]
    end

    subgraph Backend
        API[FastAPI
REST API]
        SEARCH[Full-Text Search
tsvector + GIN]
        SEMANTIC[Semantic Search
pgvector + Ollama]
    end

    subgraph Storage
        DB[(Postgres 16
CloudNativePG)]
        MINIO[(MinIO S3
WAL archive)]
    end

    AGENT -->|MCP protocol| MCP
    HUMAN -->|HTTPS| APP
    MCP -->|HTTP| API
    APP -->|HTTP internal| API
    API --> DB
    API --> SEARCH
    API --> SEMANTIC
    SEARCH --> DB
    SEMANTIC --> DB
    DB -->|WAL streaming| MINIO

Data Flow. Two access paths to the same data: AI agents use MCP tools (36 tools via pal-e-docs-mcp), humans use the SvelteKit frontend (pal-e-app). Both hit the same FastAPI REST API. The SvelteKit app uses server-side rendering — +page.server.ts loaders fetch from the API using the in-cluster service URL, then Svelte renders on the server. Client-side mutations (e.g. kanban drag-and-drop) go through a SvelteKit API proxy route that sanitizes errors and validates inputs before forwarding to the backend. Full-text search uses Postgres tsvector with GIN indexes. Semantic search uses pgvector with Ollama-generated embeddings (qwen3-embedding:4b, 768-dim). Session startup queries semantic search to inject a Dynamic Briefing — contextually relevant knowledge ranked by meaning.

graph TD
    subgraph k3s Cluster
        subgraph ns-app["ns: pal-e-app"]
            APPOD[pal-e-app pod
Node.js :3000]
        end
        subgraph ns-api["ns: pal-e-docs"]
            APIPOD[pal-e-docs pod
FastAPI :8000]
            EMBED[embedding-worker
:8001 metrics]
        end
        subgraph ns-pg["ns: postgres"]
            PG[pal-e-postgres
CNPG Cluster]
        end
        subgraph ns-platform["Platform Services"]
            ARGOCD[ArgoCD]
            HARBOR[Harbor Registry]
            WOODPECKER[Woodpecker CI]
            FORGEJO[Forgejo]
            MINIO[MinIO S3]
            OLLAMA[Ollama GPU
qwen3-embedding]
        end
    end

    subgraph External
        TS[Tailscale Funnel
TLS termination]
        BROWSER[Browser]
        CLAUDE[Claude Code]
    end

    BROWSER -->|HTTPS| TS
    TS -->|pal-e-app.*| APPOD
    TS -->|pal-e-docs.*| APIPOD
    APPOD -->|HTTP internal| APIPOD
    APIPOD --> PG
    APIPOD --> OLLAMA
    EMBED --> OLLAMA
    EMBED --> PG
    PG -->|WAL archive| MINIO
    FORGEJO -->|webhook| WOODPECKER
    WOODPECKER -->|kaniko| HARBOR
    ARGOCD -->|sync k8s/| APPOD
    ARGOCD -->|sync k8s/| APIPOD
    CLAUDE -->|MCP| APIPOD

Deployment. Single k3s node with Tailscale Funnels for ingress and TLS — no cert-manager, no Traefik. Each app gets its own namespace. Postgres runs as a CloudNativePG Cluster in the shared postgres namespace. GitOps pipeline: merge to main → Woodpecker CI builds via kaniko → pushes to Harbor → CI commits updated image tag back to repo → ArgoCD syncs k8s manifests from the app repo's k8s/ directory. Ollama runs on GPU (NVIDIA runtime, qwen3-embedding:4b on hostPath volume for persistence) for embedding generation. Embedding worker runs alongside the API pod, processing new/updated blocks and exposing Prometheus metrics on :8001. MinIO stores WAL archives for PITR backup.

Repos

Repo Platform Role Status
pal-e-docs Forgejo FastAPI backend + Postgres (planned rename: pal-e-api) active
pal-e-docs-mcp Forgejo MCP server — 36 tools (planned rename: pal-e-mcp) active
pal-e-docs-sdk Forgejo Python SDK (planned rename: pal-e-sdk) active
pal-e-app Forgejo SvelteKit frontend — board UI, k8s deployed, Woodpecker CI active

Inbox

Untriaged TODOs awaiting scoping into plan-pal-e-docs. See convention-todo-lifecycle.

Query: list_notes(project="pal-e-docs", note_type="todo", status="open") to check for unparented items.