David F GliddenandClaude Opus 4.7
119285cf43
The ~/.claude/ directory was previously local-only — a machine wipe
would have lost the accumulated memory, custom skills, and settings.
This commit moves the durable parts into dotfiles with the same
symlink-to-home pattern used for CLAUDE.md, PENDING.md, REVIEWED.md,
and L2-BOOTSTRAP.md.
Preserved (symlinked from ~/.claude/* into here):
skills/audit/ — thinking-folder drift scanner
skills/symmetria/ — practice-of-return discipline
skills/vault-update-people/ — Obsidian People-file maintainer
skills/wake-up/ — session restoration
skills/wrap-up/ — session state capture
memory/ — 55+ memory files (MEMORY.md, sessions,
ledgers, project state, feedback, etc.)
settings/settings.json — user preferences (hooks, flags, no secrets)
Deliberately NOT backed up:
settings.local.json — contains operational secrets (HF_TOKEN,
SSH password in expect scripts); by naming
convention, *.local.* is not synced.
Needs separate review and probable rotation.
sessions/, history.jsonl, caches, telemetry — ephemeral
plugins/, marketplace skills and agents — reinstallable
The working copies at ~/.claude/skills/* and
~/.claude/projects/-Users-davidglidden/memory are symlinks into this
directory, so every write flows here automatically. install.sh
recreates the symlinks on a fresh machine.
FOLLOW-ON (flagged, not in this commit):
settings.local.json contains a HuggingFace token and an SSH password
as plaintext strings inside allowed Bash command patterns. These
should be rotated and moved to secure storage (keychain / pass /
env file outside the settings file).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
65 lines
3.3 KiB
Markdown
65 lines
3.3 KiB
Markdown
---
|
|
name: training-pipeline-state
|
|
description: LoRA training pipeline — model trained, GGUF transfer failed over VPN. Must download on home LAN. Then rebuild BMF + reconnect MCP.
|
|
type: project
|
|
---
|
|
|
|
## Training Pipeline — State as of 2026-03-16 (evening)
|
|
|
|
### MODEL TRANSFER — BLOCKED ON LAN ACCESS
|
|
- **6.2 GB GGUF** on clasp at `/tmp/lora-david/model-f16.gguf`
|
|
- ~30 rsync attempts over VPN from Berlin failed: hash mismatches from partial corruption + 120s timeout too short for 1.4 MB/s VPN throughput
|
|
- Partial file at `/tmp/cm-david-model.gguf` is **corrupt** — delete before next attempt
|
|
- **Script ready**: `/tmp/pull-model-vpn.sh` (600s timeout, SSH keepalive, auto-registers in Ollama)
|
|
- **Action when home (2026-03-17)**: `rm /tmp/cm-david-model.gguf && nohup /tmp/pull-model-vpn.sh > /dev/null 2>&1 &`
|
|
- On LAN (~200 MB/s) a clean transfer takes ~30 seconds
|
|
- Log: `tail -f /tmp/pull-model.log`
|
|
|
|
### WHEN MODEL LANDS
|
|
1. Verify: `ollama list | grep cm-david`
|
|
2. Rebuild BMF: `cd ~/_ Dev/BetterMemories.io && npm run build`
|
|
3. Reconnect MCP in Claude Code: `/mcp`
|
|
4. This activates Seb's fixes: teacher during ingestion (#49), chat export (#47), pairing fix (#43), graduation fallback (#45)
|
|
|
|
### Seb's Fixes (pulled to main, 2026-03-16)
|
|
- `8dd17a0` #39 — `'auto'` backend default
|
|
- `eca8da7` #40 — llama.cpp GGUF fallback (auto-clones)
|
|
- `7fec033` #43 — Pairing ID mismatch detection
|
|
- `422212f` #45 — Graduation fallback when teacher unavailable
|
|
- `6c332b7` #47 — Chat format in export_training_pairs
|
|
- `ce03770` #49 — Teacher enrichment during connector ingestion (rate-limited)
|
|
- `39915cc` — Flush timer fix for circle pairings
|
|
|
|
### Second Training Run (COMPLETE)
|
|
- **14,199 deduplicated pairs** from David's instance (vault + git + BMF sessions)
|
|
- Qwen2.5-3B-Instruct, rank-16, 2,000 iters, 21 min on M4 Pro
|
|
- Output: `/tmp/lora-david/` on clasp, registered as `cm-david-all-slots:lora-v1`
|
|
|
|
### Clasp State (cleaned up this session)
|
|
- 122 stale Modelfile models removed
|
|
- PATH fixed (#42): node/npm available in SSH sessions via ~/.zshenv
|
|
- TypeScript rebuilt with #39 fix
|
|
- mlx-fine-tune.py updated with llama.cpp fallback
|
|
- BMF v0.48.0 running, healthy
|
|
|
|
### Ingestion
|
|
- Claude transcripts: **idle / needs restart** — was at 7,965/72,938 (11%) but BMF process restart lost the job
|
|
- Restart after model lands + BMF rebuild: `connect_service` with `claude_transcripts`
|
|
- With Seb's #49 fix + MCP reconnect, remaining transcripts WILL generate training pairs via teacher
|
|
|
|
### Remaining Issues (not yet resolved by Seb)
|
|
- #41 — Qwen3.5 GGUF broken (upstream, informational)
|
|
- #42 — Clasp SSH PATH (we fixed manually via .zshenv)
|
|
- #44 — Clasp teacher not configured (decided to skip — unnecessary token burn)
|
|
- #46 — Stale models (we cleaned manually)
|
|
- #48 — GGUF distribution pipeline (future feature)
|
|
|
|
### Architecture Finding
|
|
- Training pairs only generated via sampling/teacher provider in slot chains
|
|
- All slots start SIGNALING with rules-only → no sampling → no pairs from ingestion
|
|
- Seb's #49 fix adds optional teacher enrichment during ingestion (rate-limited)
|
|
- #45 fix helps graduation progress when teacher unavailable
|
|
|
|
**Why:** Training flywheel = sovereignty. Local models that learn from your data.
|
|
**How to apply:** Model landing → reconnect MCP → Chamber testable. Next training round after ingestion completes (more data = better model).
|