David F GliddenandClaude Opus 4.8
3f9a89b00c
Basic Memory v0.21.6 first sync over the live memory dir (steward-authorized live-dir trial, Option A 2026-06-06): adds permalink: to frontmatter, refolds long YAML description lines, strips final newlines. Bodies untouched — verified via full diff classification. From this commit forward, any diff in claude/memory shows only what Basic Memory or the session writes. Trial design: MemPalace untouched as incumbent; git status check on this dir at every wrap; end-of-day evaluation (recall quality, sync robustness, rebuild-from-files, malformed-file behavior). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
66 lines
3.4 KiB
Markdown
66 lines
3.4 KiB
Markdown
---
|
|
name: training-pipeline-state
|
|
description: LoRA training pipeline — model trained, GGUF transfer failed over VPN.
|
|
Must download on home LAN. Then rebuild BMF + reconnect MCP.
|
|
type: project
|
|
permalink: claude-memory/training-pipeline
|
|
---
|
|
|
|
## Training Pipeline — State as of 2026-03-16 (evening)
|
|
|
|
### MODEL TRANSFER — BLOCKED ON LAN ACCESS
|
|
- **6.2 GB GGUF** on clasp at `/tmp/lora-david/model-f16.gguf`
|
|
- ~30 rsync attempts over VPN from Berlin failed: hash mismatches from partial corruption + 120s timeout too short for 1.4 MB/s VPN throughput
|
|
- Partial file at `/tmp/cm-david-model.gguf` is **corrupt** — delete before next attempt
|
|
- **Script ready**: `/tmp/pull-model-vpn.sh` (600s timeout, SSH keepalive, auto-registers in Ollama)
|
|
- **Action when home (2026-03-17)**: `rm /tmp/cm-david-model.gguf && nohup /tmp/pull-model-vpn.sh > /dev/null 2>&1 &`
|
|
- On LAN (~200 MB/s) a clean transfer takes ~30 seconds
|
|
- Log: `tail -f /tmp/pull-model.log`
|
|
|
|
### WHEN MODEL LANDS
|
|
1. Verify: `ollama list | grep cm-david`
|
|
2. Rebuild BMF: `cd ~/_ Dev/BetterMemories.io && npm run build`
|
|
3. Reconnect MCP in Claude Code: `/mcp`
|
|
4. This activates Seb's fixes: teacher during ingestion (#49), chat export (#47), pairing fix (#43), graduation fallback (#45)
|
|
|
|
### Seb's Fixes (pulled to main, 2026-03-16)
|
|
- `8dd17a0` #39 — `'auto'` backend default
|
|
- `eca8da7` #40 — llama.cpp GGUF fallback (auto-clones)
|
|
- `7fec033` #43 — Pairing ID mismatch detection
|
|
- `422212f` #45 — Graduation fallback when teacher unavailable
|
|
- `6c332b7` #47 — Chat format in export_training_pairs
|
|
- `ce03770` #49 — Teacher enrichment during connector ingestion (rate-limited)
|
|
- `39915cc` — Flush timer fix for circle pairings
|
|
|
|
### Second Training Run (COMPLETE)
|
|
- **14,199 deduplicated pairs** from David's instance (vault + git + BMF sessions)
|
|
- Qwen2.5-3B-Instruct, rank-16, 2,000 iters, 21 min on M4 Pro
|
|
- Output: `/tmp/lora-david/` on clasp, registered as `cm-david-all-slots:lora-v1`
|
|
|
|
### Clasp State (cleaned up this session)
|
|
- 122 stale Modelfile models removed
|
|
- PATH fixed (#42): node/npm available in SSH sessions via ~/.zshenv
|
|
- TypeScript rebuilt with #39 fix
|
|
- mlx-fine-tune.py updated with llama.cpp fallback
|
|
- BMF v0.48.0 running, healthy
|
|
|
|
### Ingestion
|
|
- Claude transcripts: **idle / needs restart** — was at 7,965/72,938 (11%) but BMF process restart lost the job
|
|
- Restart after model lands + BMF rebuild: `connect_service` with `claude_transcripts`
|
|
- With Seb's #49 fix + MCP reconnect, remaining transcripts WILL generate training pairs via teacher
|
|
|
|
### Remaining Issues (not yet resolved by Seb)
|
|
- #41 — Qwen3.5 GGUF broken (upstream, informational)
|
|
- #42 — Clasp SSH PATH (we fixed manually via .zshenv)
|
|
- #44 — Clasp teacher not configured (decided to skip — unnecessary token burn)
|
|
- #46 — Stale models (we cleaned manually)
|
|
- #48 — GGUF distribution pipeline (future feature)
|
|
|
|
### Architecture Finding
|
|
- Training pairs only generated via sampling/teacher provider in slot chains
|
|
- All slots start SIGNALING with rules-only → no sampling → no pairs from ingestion
|
|
- Seb's #49 fix adds optional teacher enrichment during ingestion (rate-limited)
|
|
- #45 fix helps graduation progress when teacher unavailable
|
|
|
|
**Why:** Training flywheel = sovereignty. Local models that learn from your data.
|
|
**How to apply:** Model landing → reconnect MCP → Chamber testable. Next training round after ingestion completes (more data = better model). |