token-slim

Token Saver (token-slim)

• Author: Evan Song · github.com/Songhonglei
• Repository: https://github.com/Songhonglei/better-agent-skills
• License: MIT

Helps AI agents and their users reduce token consumption by auditing workspace files, identifying bloat, and guiding targeted cleanup.

Core Concept

Every project-context file (memory index, agent config, heartbeat, etc.) is injected into every single session. Keeping them lean = direct, compounding savings. The goal is a three-tier memory layout:

memory index        ← always loaded, < 150 lines — pointers + essentials only
memoryres/*.md      ← loaded on demand — reference tables, specs, backlogs
memory/YYYY-MM-DD.md ← raw daily log — rarely loaded

For full strategy details: see references/strategies.md

---

Mode Routing

After matching a mode, first read the corresponding reference file, then follow that flow. Do not rely on memory.

---

Execution Modes

Confirm mode (default)

Show each proposed change, wait for user approval before modifying.

Batch mode

Triggered when the user says "just do it" / "按你说的来" / "全部做". Execute all changes without per-item confirmation; deliver a unified summary at the end.

Dry-run mode

Preview without modifying any files:

SKILL_DIR="$(dirname "$(readlink -f "$0")")/skills/token-slim" 2>/dev/null \
  || SKILL_DIR=./skills/token-slim
python3 "$SKILL_DIR/scripts/scan_workspace.py" --workspace . --dry-run

Or simply, from the workspace root:

python3 /scripts/scan_workspace.py --workspace . --dry-run

---

🔥 Brutal Mode — Toggle

Triggers:

• Enable: "enable brutal mode" / "开启狂暴模式" / "打开狂暴模式"
• Disable: "disable brutal mode" / "关闭狂暴模式" / "退出狂暴模式"

Enable flow:

1. Check the workspace agent config file for <!-- token-slim:token-habits-start --> anchor
2. If absent → append the full template (incl. brutal mode section — see references/mode-a-onboarding.md Step 8)
3. If present → replace the whole block with the full template
4. Reply: "🔥 Brutal mode enabled — concise outputs, no preamble."

Disable flow:

1. Find the block <!-- token-slim:token-habits-start --> … <!-- token-slim:token-habits-end --> in the agent config
2. Replace the brutal-mode section with just a header placeholder:

   ### 🔥 Brutal Mode (max output efficiency) <!-- Brutal mode: disabled -->
   

3. Reply: "✅ Brutal mode disabled — back to normal."

First-time use (after Mode A Step 8 completes): Proactively offer:

"🔥 Want to enable brutal mode? When on, the agent gives results only — no step-by-step narration. Saves tokens and reading time. Say 'enable brutal mode' to toggle (can be turned off anytime)."

---

Undo (rollback)

scan_workspace.py itself does not modify files and does not auto-backup.

Before any file modification the agent must:

1. Tell the user which files will be modified
2. Ask "Do you want me to back these up first?"
3. On confirmation, copy the soon-to-be-modified files to a timestamped dir:

   /.token-slim/undo-/
   

4. Report the backup path so the user can restore later.

See references/mode-a-onboarding.md Step 3 for the canonical recipe.

---

tiktoken installation

token-slim uses tiktoken for accurate token counting (vs ~40-60% heuristic error). Install attempts run in this order with automatic fallback:

python3 /scripts/install_tiktoken.py --workspace .

BPE vocabulary cache is fetched automatically by tiktoken on first encode from the OpenAI public blob (openaipublic.blob.core.windows.net) and stored under /.cache/tiktoken/ (CWD-anchored — survives $HOME wipes on container/VM environments where only the working directory is persistent).

Check current state:

python3 /scripts/install_tiktoken.py --check --workspace .

scan_workspace.py auto-detects tiktoken at runtime; no manual setup needed after installation.

---

Dependencies

Required:

• Python 3.8+ (python3 available on PATH)

Optional:

• tiktoken for precise token counting (install via the script above). Without it, the scanner uses a CJK/ASCII heuristic with ~40-60% error.

Everything else uses the Python standard library.

---

What NOT to move

Never suggest moving:

• Behavioural rules, safety constraints, must/never guidelines
• Cron job configurations and failure protocols
• Active heartbeat tasks and retry queues
• Identity / persona core (soul/identity config content)
• Anything with ⚠️ or explicit "always loaded" markers

When in doubt, ask the user.

---

Key files

---

CLI reference

scan_workspace.py

python3 scan_workspace.py [--workspace ] [--json] [--dry-run]

install_tiktoken.py

python3 install_tiktoken.py [--check] [--workspace ]

---

⚠️ Skills vs workspace-file tokens

Skills (SKILL.md): each skill injects only its name + description + location per session — around 24 tokens. The full body is only loaded when the agent explicitly reads it. Large SKILL.md files affect call latency, not per-session baseline.

Workspace files (memory index, agent config, etc.): injected verbatim every session — these are the real optimisation target.