feat: brain service — self-contained second brain knowledge manager

Full backend service with:
- FastAPI REST API with CRUD, search, reprocess endpoints
- PostgreSQL + pgvector for items and semantic search
- Redis + RQ for background job processing
- Meilisearch for fast keyword/filter search
- Browserless/Chrome for JS rendering and screenshots
- OpenAI structured output for AI classification
- Local file storage with S3-ready abstraction
- Gateway auth via X-Gateway-User-Id header
- Own docker-compose stack (6 containers)

Classification: fixed folders (Home/Family/Work/Travel/Knowledge/Faith/Projects)
and fixed tags (28 predefined). AI assigns exactly 1 folder, 2-3 tags, title,
summary, and confidence score per item.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Yusuf Suleman
2026-04-01 11:48:29 -05:00
parent 51a8157fd4
commit 8275f3a71b
73 changed files with 24081 additions and 4209 deletions

View File

@@ -0,0 +1,55 @@
"""Brain service configuration — all from environment variables."""
import os
# ── Database ──
DATABASE_URL = os.environ.get(
"DATABASE_URL",
"postgresql+asyncpg://brain:brain@brain-db:5432/brain"
)
DATABASE_URL_SYNC = DATABASE_URL.replace("+asyncpg", "")
# ── Redis ──
REDIS_URL = os.environ.get("REDIS_URL", "redis://brain-redis:6379/0")
# ── Meilisearch ──
MEILI_URL = os.environ.get("MEILI_URL", "http://brain-meili:7700")
MEILI_KEY = os.environ.get("MEILI_MASTER_KEY", "brain-meili-key")
MEILI_INDEX = "items"
# ── Browserless ──
BROWSERLESS_URL = os.environ.get("BROWSERLESS_URL", "http://brain-browserless:3000")
# ── OpenAI ──
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY", "")
OPENAI_MODEL = os.environ.get("OPENAI_MODEL", "gpt-4o-mini")
OPENAI_EMBED_MODEL = os.environ.get("OPENAI_EMBED_MODEL", "text-embedding-3-small")
OPENAI_EMBED_DIM = int(os.environ.get("OPENAI_EMBED_DIM", "1536"))
# ── Storage ──
STORAGE_BACKEND = os.environ.get("STORAGE_BACKEND", "local") # local | s3
STORAGE_LOCAL_PATH = os.environ.get("STORAGE_LOCAL_PATH", "/app/storage")
# ── S3 (future) ──
S3_BUCKET = os.environ.get("S3_BUCKET", "")
S3_ENDPOINT = os.environ.get("S3_ENDPOINT", "")
S3_ACCESS_KEY = os.environ.get("S3_ACCESS_KEY", "")
S3_SECRET_KEY = os.environ.get("S3_SECRET_KEY", "")
# ── Service ──
PORT = int(os.environ.get("PORT", "8200"))
DEBUG = os.environ.get("DEBUG", "").lower() in ("1", "true")
# ── Classification rules ──
FOLDERS = [
"Home", "Family", "Work", "Travel", "Knowledge", "Faith", "Projects"
]
TAGS = [
"reference", "important", "legal", "financial", "insurance",
"research", "idea", "guide", "tutorial", "setup", "how-to",
"tools", "dev", "server", "selfhosted", "home-assistant",
"shopping", "compare", "buy", "product",
"family", "kids", "health", "travel", "faith",
"video", "read-later", "books",
]