prompt-archaeology

Prompt Archaeology

Overview

Prompt Archaeology is the practice of excavating your own conversation history instead of re-solving problems from scratch. Every AI session is a stratum — a sedimented layer of debugging, decision-making, and discovery. Over time, valuable artifacts sink below the surface: a one-liner that fixed a gnarly race condition, a config that satisfied a finicky build, the exact incantation that convinced a model to behave. Most agents never dig for these. They re-derive, re-guess, and re-fail.

This skill turns that history into a quarryable resource. It bundles:

• Search strategies — keyword, semantic-adjacent, temporal, and structural queries tuned for session transcripts.
• Relevance scoring — a transparent, composable ranking that surfaces the one session that actually matters.
• Knowledge extraction patterns — recipes for pulling decisions and solutions out of a wall of chat, not just matching text.
• Deduplication — collapse near-duplicate fixes across sessions into a single canonical answer.
• excavate.py — a standalone Python script that crawls session logs and markdown files, ranks them, and prints the buried artifacts.

The metaphor is deliberate. An archaeologist does not grep the desert for "pottery" and ship the first hit. They survey, triangulate, carefully extract, and catalog. This skill teaches the agent to do the same with its own past.

When to Use

• The user is about to re-solve a known problem. They describe a bug or task and you have a flicker of "we've done this before." Excavate before answering.
• "Didn't we figure out...?" / "What did we land on?" — retrieve the prior decision and its rationale, not just the outcome.
• Hunting for a lost code snippet, config value, or command that worked months ago.
• Onboarding to a codebase you've touched before — pull the architectural decisions out of old sessions.
• Avoiding repeated dead ends — find the approaches that were rejected and why, so you don't walk back into them.
• Writing postmortems or ADRs from scattered session evidence.

Don't use for

• Fresh problems with no prior history — there's nothing to excavate; solve forward.
• When you already hold the answer in active context — don't pad the turn with a search.
• Sensitive retrieval across other users' private profiles unless explicitly authorized.

The Excavation Workflow

A dig has five phases. Skipping any phase degrades result quality.

1. Survey — frame the query

Before searching, state what artifact you want and what shape it takes:

Write the query down. A vague survey yields a vague dig.

2. Locate — run the searches

Run multiple passes, not one. Different phrasings live in different sessions.

• Exact / keyword pass — the literal error string, function name, or filename. Highest precision.
• Semantic-adjacent pass — paraphrase the intent. If the exact term misses, the concept might be filed under different words.
• Temporal pass — constrain to the window when the work happened ("sessions from the week we shipped v2").
• Structural pass — look for code blocks, file diffs, or command outputs near the topic, not just prose. Solutions often hide in fenced blocks.

excavate.py runs the keyword and structural passes directly; use session_search or a semantic tool for the semantic-adjacent pass.

3. Score — rank the finds

Not every hit is an artifact. Rank each located session against four signals (this is the scoring baked into excavate.py, in --explain mode):

The composite score is density·0.25 + recency·0.15 + code·0.25 + resolution·0.35 (weights live in excavate.py and are tunable). Relevance is not raw match count — a 200-message session with one mention ranks below a 12-message session built around the topic.

4. Extract — pull the artifact out

Once you've found the winning session, don't dump the whole transcript. Extract the minimal artifact:

• For a fix: the failing state → the change → the success marker. Three quotes, nothing more.
• For a decision: the options considered → the chosen option → the stated rationale.
• For a command: the exact command + the one line of context that says what it does.
• For a rejected approach: what was tried → the observed failure → the inferred lesson.

Quote the session (> ...) and cite it. Extraction patterns are detailed in references/extraction-patterns.md.

5. Deduplicate — collapse the finds

The same fix often appears in three sessions (the first attempt, the retry, the "oh and also" follow-up). Deduplicate before reporting:

• Exact-code dedup — identical fenced blocks collapse to one.
• Near-dup detection — normalized text similarity > 0.85 merges into a cluster; keep the highest-scoring member as the canonical answer.
• Variant detection — if two snippets differ only in a version number or path, treat as the same artifact and note the latest variant.

excavate.py --dedup runs all three. See references/deduplication.md.

Using excavate.py

The script lives at scripts/excavate.py. It has no third-party dependencies — stdlib only — so it runs anywhere Python 3.8+ does.

# Basic keyword dig over a directory of .md / .txt / .json session logs
python3 scripts/excavate.py dig ./sessions --query "kafka consumer rebalance"

# Multiple terms (AND'd within a session), show top 5 with per-session scores
python3 scripts/excavate.py dig ./sessions --query "rebalance retry backoff" --top 5 --explain

# Add a date window (ISO dates), dedup near-identical results
python3 scripts/excavate.py dig ./sessions \
  --query "connection pool exhaustion" \
  --after 2024-01-01 --before 2024-06-01 \
  --dedup

# Dump the extracted code blocks across all matching sessions
python3 scripts/excavate.py dig ./sessions --query "ffmpeg concatenate" --extract code

# Index a directory once, then query the index repeatedly (faster for large corpora)
python3 scripts/excavate.py index ./sessions --out sessions.idx
python3 scripts/excavate.py query sessions.idx --query "oauth refresh token" --top 3 --explain

--explain prints the per-signal score breakdown so you can see why a session ranked where it did. Full CLI reference: references/cli-reference.md.

Programmatic use

from excavate import ArchaeologyIndex

idx = ArchaeologyIndex()
idx.scan("./sessions")            # walk the directory once
for hit in idx.search("kafka rebalance", top=5, explain=True):
    print(hit.score, hit.path, hit.extraction)

The ArchaeologyIndex class is the stable surface; the CLI is a thin wrapper over it.

Relevance Scoring in Depth

Scoring details, the math, and how to retune weights for your corpus are in references/relevance-scoring.md. Key points:

• Scores are normalized to [0, 1] per signal before weighting, so a corpus change doesn't silently inflate one signal.
• Resolution markers dominate by default — a session that explicitly says "that fixed it" beats a longer, denser session that merely mentions the term. Tune the weight down if your logs lack success markers.
• Recency is configurable, not gospel. For stable domains (algorithms, math) weight it low; for fast-moving domains (frontend deps) weight it high.
• The scoring is transparent, not learned. Every ranking is explainable; nothing is a black-box embedding. This matters when the user asks "why did you trust that session?"

Integration with session_search

If you're running inside Hermes, the native session_search tool is your semantic-adjacent pass — it has FTS5 over the session DB. Use this skill's workflow to decide what to search for and how to rank the results, then let excavate.py handle corpora that aren't in the session DB (exported logs, markdown notes, JSONL exports, another agent's transcripts).

semantic-adjacent pass  →  session_search(query="...")        # Hermes session DB
keyword + structural    →  excavate.py dig ./exported-logs    # file-based corpora

Common Pitfalls

1. Searching one query and giving up. The single biggest failure mode. Run at least three passes (exact, semantic-adjacent, structural). Artifacts are rarely filed under the first word you reach for.

2. Trusting match count as relevance. A session that mentions "docker" forty times while setting up a CI pipeline is not the answer to "how did we fix the docker permissions bug." Use the composite score, not raw hits.

3. Skipping dedup and reporting three copies of the same fix. Always run --dedup when top > 1. The user asked for the answer, not the archaeology of the answer.

4. Extracting the whole session. The transcript is the site, not the artifact. Quote minimally.

5. Retrieving a decision without its rationale. "We chose Postgres" is useless without "because we needed strong consistency for the ledger." Resolution and rationale travel together — extract both or neither.

6. Assuming recency equals correctness. A two-year-old session that solved the exact algorithmic problem beats yesterday's near-miss. Recency is a tiebreaker, weighted low by default for a reason.

7. Digging without a survey. If you can't state what artifact you want and what shape it takes, your query will be too vague to rank well. Spend ten seconds on the table in Phase 1.

8. Forgetting structural pass. The fix is often inside a fenced code block that doesn't repeat the keyword in prose. --extract code exists for this reason.

9. Treating near-dups as separate finds. Three sessions with the same stack trace are one problem, not three. Normalize before you count.

10. Not citing the source session. Always cite. The user may want to open the original; future you will want to re-excavate.

Verification Checklist

• Survey written: the artifact type and query shape are stated before searching.
• At least two query passes run (one exact/keyword, one semantic-adjacent or structural).
• Results ranked with the composite score, not raw match count (--explain if using the script).
• Deduplication run when multiple results are returned.
• Extraction is minimal — failing state, change, success marker — not the whole transcript.
• Source session cited (path or session id).
• Decision artifacts include rationale, not just the chosen option.
• Recency weighted appropriately for the domain (low for stable, high for volatile).

Further Reading

• references/search-strategies.md — the full playbook for the Locate phase: query expansion, negation, temporal constraints, and how to pick pass order.
• references/relevance-scoring.md — the math behind the composite score, normalization, and how to retune weights.
• references/extraction-patterns.md — extraction templates for fix, decision, command, and rejection artifacts.
• references/deduplication.md — exact, near-dup, and variant detection in depth.
• references/cli-reference.md — every excavate.py flag and subcommand.
• scripts/excavate.py — the implementation. Stdlib only, single file, importable.

---

Prompt Archaeology: don't re-derive what you've already discovered. Excavate it.