amazon-shopper

Amazon Shopper

One of dozens of skills and plugins in TinkerClaw — a self-improving OpenClaw fork that's been running 24/7 for months.

Most shopping agents hand you back the first page of Amazon with the prices copied out. This one argues with you about which number matters.

It sweeps a dozen phrasings of your query instead of trusting one keyword (8 phrasings returned 268 unique products where the best single query gave 66), parses the size out of the title, and ranks on the metric that actually decides the purchase: €/kg of active ingredient, €/GB, €/kg of protein — not the sticker price, and not Amazon's own per-unit label, which quietly reflects whichever variant happens to be selected. Then it checks the thing every ranker forgets: can you actually buy it? A discontinued listing renders perfectly, keeps its price, and ranks beautifully. It is still not for sale.

The safety story is the boring kind. It reads amazon.es anonymously by default — no account, no cookies — because price, stock, title and images do not need a login. Signing in is a separate decision you make on purpose, the credentials live in your OS keychain, and one --logout removes them. It has no write path to Amazon at all: it cannot add to a basket, place an order, or change a setting.

Part of TinkerClaw — real-time token tracking, self-improving crons, persistent cognitive memory. This is one piece of that stack; the repo has dozens more.

👉 https://github.com/globalcaos/tinkerclaw

Clone it. Fork it. Break it. Make it yours.

What it actually does

Opinionated Amazon.es shopping CLI. You give it a keyword (e.g. "ph minus piscina"), it fetches search results, auto-categorizes them, asks at most 2 load-bearing qualifying questions, researches active ingredients via a 3-step ladder when the product description is not enough, and recommends the top 3 normalized to the right per-unit metric (€/kg-active for chemistry, €/GB for storage, €/kg for food).

Distinct from amazon-product-search-api-skill (a paid BrowserAct raw extractor — keep that one for cases where you want raw results without analysis).

Permissions, Data Flow & Consent

Short version: your search terms and the pages Amazon returns go between this machine and www.amazon.es, and nowhere else unless you switch on one of the optional paid paths. The skill can read Amazon; it cannot buy anything. Longer version, because you should not have to take that on trust:

What data it touches. The keywords you give it, the search and product HTML amazon.es returns, and — only if you opt in to a session — your amazon.es login cookies and the delivery address they imply. Product data lands in a per-task SQLite file under your temp dir that close deletes.

Where it goes. https://www.amazon.es/..., over HTTPS. Three optional destinations, each OFF until you set its environment variables: Apify (api.apify.com — your search terms leave the machine and each actor run costs roughly $0.01–$0.05 of your Apify credit), the Amazon Creators API (webservices.amazon.com, official and structured), and DuckDuckGo (the last rung of the concentration-research ladder, hit only when the first three rungs fail). There is no telemetry and no analytics.

What it writes to disk. The per-task SQLite ($TMPDIR/amazon-shopper-.db), the stored session secret when you have one, and whatever --out / --out-dir path you name yourself. Nothing else.

What it deletes. close removes that task's SQLite and its -wal/-shm siblings. --logout removes the stored session. Image hashing unlinks its own temp file. No other path.

What credentials it reads, and how you get rid of them. None, unless you ask. Anonymous fetching covers price, stock, title and images; a session is only needed for delivery promises and account-specific pricing. When you do want one, scripts/session-capture.mjs --yes reads the amazon.es auth cookies (at-acbes, sess-at-acbes, x-acbes) out of a shared, logged-in browser tab. Those cookies are login credentials for read purposes. So:

• They go into the OS keychain — secret-tool (libsecret) on Linux, security on macOS.
• If neither exists, they fall back to a 0600 file at ~/.openclaw/credentials/amazon-session.json and the skill prints a warning every single run saying so. It never happens quietly.
• Cookie values are never printed, logged, or passed on a command line. Capture output is names and counts only.
• On macOS the keychain write passes the value in argv, because security has no stdin input for generic passwords — it is briefly visible to ps on that machine. Small, real, and named here rather than papered over.

node scripts/session-capture.mjs --logout      # clears keychain AND file, prints the revoke URL

Local clearing stops this machine. To invalidate the token on Amazon's side, sign out of all devices or change your password at https://www.amazon.es/gp/css/account/info/view.html — --logout prints that link so you do not have to look it up.

What it needs, and why.

Money. The default path is free. Apify is the only thing here that can spend: it bills your own Apify account per actor run (~$0.01–$0.05), and it does nothing unless AMAZON_SHOPPER_APIFY_TOKEN is set. LLM calls (max ~4 per task) go through whichever CLI you already have configured.

Refund strategy (value maximization)

Amazon's A-to-z Guarantee covers "not as described" claims up to €2,000 within 90 days of delivery — which effectively warranties the concentration spec this skill's €/kg-active ranking depends on. If you buy "sulfuric acid 15%" and it assays lower, that's a refundable not-as-described claim. This makes Amazon structurally better value than non-refundable retailers even at a higher sticker price: the downside is capped.

Refund friction differs by seller, so the ranker captures seller_name / seller_is_amazon / in_stock and uses a refund tier as a tie-breaker (when two products are within 10% on the metric, the higher tier wins):

The tie-breaker is deliberately conservative: a >10%-cheaper third-party item still wins on price (protected ≠ free). Below 10%, protected value wins.

CLI

The skill is a stateful CLI with 6 subcommands. Jarvis (the agent) chains them across chat turns.

amazon-shopper start ""
amazon-shopper answer   ""
amazon-shopper rank 
amazon-shopper inspect  [--product ]
amazon-shopper set-spec   --concentration  [--ingredient ] [--source ]
amazon-shopper close 

All subcommands emit one JSON object on stdout for machine consumption; human-readable text goes to stderr.

State persists between calls in /tmp/amazon-shopper-.db (SQLite via Node's experimental node:sqlite). The DB is disposable — close deletes it. No cross-search history (by design).

Typical conversation flow

$ amazon-shopper start "ph minus piscina"
{"task_id":"abc123def456","state":"awaiting_questions","products_count":42,
 "pending_questions":[
   {"id":"form","text":"Powder or liquid?","options":["powder","liquid"],
    "why_load_bearing":"different active ingredients → different €/kg math"},
   {"id":"volume","text":"Annual usage?","options":["<2kg","2-10kg",">10kg"],
    "why_load_bearing":"affects package-size filter and bulk-discount math"}
 ]}

$ amazon-shopper answer abc123def456 form "powder"
{"state":"awaiting_questions","next_question":{...}}

$ amazon-shopper answer abc123def456 volume "2-10kg"
{"state":"ready_to_rank"}

$ amazon-shopper rank abc123def456
{"task_id":"abc123def456","state":"complete",
 "metric":{"metric_id":"eur_per_kg_active","formula_human":"price / (size_kg × concentration_pct/100)"},
 "top_3":[
   {"asin":"B0...","title":"CTX 5kg granulado","current_price_eur":22.50,
    "package_size_kg":5,"active_ingredient":"sodium bisulfate","concentration_pct":100,
    "normalized_metric_value":4.50,"rank_position":1,
    "reasoning_sentence":"CTX 5kg granulado: €22.5 for 5kg @ 100% → eur_per_kg_active = 4.500"},
   ...
 ],
 "skipped":[{"asin":"B0...","title":"Generic ph minus","reason":"spec_unknown"}]}

$ amazon-shopper close abc123def456
{"ok":true,"task_id":"abc123def456","closed":true}

Brand-concentration research (parallel subagent escalation)

The generic ladder now lives in the spec-resolution-ladder recipe (~/src/tinkerclaw/extensions/tinkerclaw-prefrontal/recipes/spec-resolution-ladder/recipe.md): when to bother resolving at all, the rung order, why an identifier-keyed database (Open Food Facts by EAN) usually beats both the vendor page and the manufacturer site, never banking a marketing claim as a number, and normalising per-serving vs per-100 before comparing. Read the recipe for the METHOD; what follows is only this skill's wiring — the needs_concentration_research[] field, the brand-concentrations.mjs cache, and the set-spec write-back.

The €/kg-active metric — the whole point of the skill, the one number that makes powder and liquid comparable — needs each product's active-ingredient concentration. The research ladder resolves it in this order:

1. Explicit in title ("15%", "sulfúrico") → used directly (spec_source:"title").
2. Brand table (scripts/brand-concentrations.mjs) — known brands (CTX, Bayrol, AstralPool, Nortembio…) by form (spec_source:"title:brand-table:").
3. Per-form default — granular ⇒ sodium bisulfate 95%, liquid ⇒ sulfuric 15% (spec_source:"title:form-default"). This is an assumption, so the product is flagged needs_concentration:true and listed in the rank result's needs_concentration_research[].
4. Producer site + DuckDuckGo web search (in-skill, brittle — often WAF-blocked).

When rank returns a non-empty needs_concentration_research[], the AGENT (not the skill) resolves the doubt by spinning ONE PARALLEL SUBAGENT PER DISTINCT BRAND to read that brand's online presence (manufacturer site, datasheet, the Amazon listing's bullet/A+ content) and report the real concentration. The skill is a pure-Node CLI with no web-search tools and a fragile DDG scrape — a web-capable subagent is strictly better, and the brands are independent, so they fan out. Procedure:

# For each brand in needs_concentration_research (run concurrently):
node /home/globalcaos/src/tinkerclaw/scripts/openclaw-spawn-subagent.mjs \
  --task "Find the active-ingredient concentration of   pH-minus \
          for pools. Check the manufacturer site and the product datasheet. \
          Return JSON {concentration_pct:<0-100>, active_ingredient:, source_url:}." \
  --label "conc:" --model claude-code/claude-haiku-4-5 --json

Then write each confirmed value back and re-rank — the metric stays eur_per_kg_active, now on confirmed data instead of an assumption:

amazon-shopper set-spec   --concentration  \
  --ingredient "" --source ""     # recomputes active_kg
amazon-shopper rank                 # re-rank on confirmed specs

set-spec backfills package size/form from the title when the product was never researched (outside the rank top-10), so active_kg always computes. Add durably confirmed brands to brand-concentrations.mjs so the subagent isn't needed twice.

When to escalate: only when a needs_concentration_research product is actually in the running (it's scoped to the ranked top-3, not the whole catalog) and the form-default could plausibly be wrong enough to change the winner. A 10 kg granulado at €2.40/kg beats every liquid by a wide margin regardless of whether it's 92% or 100% bisulfate — don't fan out subagents to refine a number that can't flip the ranking. Do escalate when two top contenders are within ~15% and one rests on a form-default.

State machine

  start ──> searching ──┐
                        │ (fetch+parse OK)
                        ↓
            awaiting_questions ──┐
                        │        │ (CAPTCHA)
                        │        └──> blocked
                        │
                        ↓ (all answers recorded)
                      ranking ──> complete

Plus the terminal failed state for unexpected errors.

Hard caps (codified, not aspirational)

• Max 2 qualifying questions per task — enforced in categorize.mjs.
• Max 10 detail-page fetches per task (top-K = 10) — enforced in shopper.mjs:cmdRank.
• Max 4 concurrent HTTP requests — semaphore in shopper.mjs:cmdRank.
• Max 1 retry per request (503 only; CAPTCHA never) — enforced in fetch.mjs.
• Polite cadence: 1 req per ~2-3s (jittered 2000ms ± 1000ms).
• Max ~4 LLM calls per task (1 categorize + 1 choose-metric + up to 2 spec-extracts via the ladder).

ANONYMOUS HTTP WORKS AGAIN — and it is now the first thing to try (2026-08-30)

Measured, not assumed. With the captured session STALE and bot-walled, plain curl_cffi with impersonate="chrome124" and Accept-Language: es-ES returned HTTP 200 on both /dp/ (≈2.1 MB, full title + price + stock + images) and /s?k= (≈1.9 MB, 125 product cards). Every claim above about anonymous being permanently 503 was written on 2026-08-06 evidence and is no longer true.

Order to try, cheapest first: anonymous curl_cffi → captured session → shared tab. Do not ask Oscar for a tab until anonymous has actually failed today; that ask cost a full turn on 2026-08-29 for a block that a 3-line probe would have bypassed.

from curl_cffi import requests as cr
r = cr.get("https://www.amazon.es/dp/"+asin, impersonate="chrome124",
           timeout=30, headers={"Accept-Language":"es-ES,es;q=0.9"})

Extraction recipe that actually works on the live 2026-08 DOM

Do NOT use 'outOfStock' in html as the stock test — that string appears in inline JS on every page and marked all 11 candidates dead in one sweep.

PRICE PRESENT ≠ IN STOCK (corrects the working rule, with evidence)

Oscar's heuristic on 2026-08-30 was "if a product has the price, then it is in stock." It does not hold on amazon.es. The two ASINs confirmed unavailable (B07ZV1MGSZ Amfit banana, B00MNNE7HE MyProtein isolate 5 kg) both still render a price — €34,99 and €86,99 — sourced from "Otros vendedores" / used offers, while carrying No disponible, outOfStock, an empty #availability block and no buy-now-button. A price string is a necessary but not sufficient condition. The #availability block is the load-bearing signal; require non-empty text matching En stock|Sólo queda|Disponible.

The output contract → OWNED BY THE product-comparison-chart RECIPE

How a comparison ANSWER is shaped is not this skill's job (split set by Oscar, 2026-08-30). This skill's job is to GET amazon.es data and hand back rows; the chart — which columns exist, what the rows are sorted by, the clickable photo column, the recommendation — lives in ~/src/tinkerclaw/extensions/tinkerclaw-prefrontal/recipes/product-comparison-chart/recipe.md, so it applies to every vendor and not just this one. Read that recipe when presenting; read this skill when fetching.

The seam: the recipe expects each row to carry a link, an image (thumbnail

• full-res target), the absolute price, the per-unit price, the characteristic being optimised, and availability. Everything below tells you how to fill those six fields from amazon.es specifically.

Availability is a HARD GATE, not a footnote

An unavailable product cannot be the recommendation, no matter how well it scores. Check it BEFORE ranking, and drop or clearly flag anything that fails:

• Amazon's own brands are the highest-risk rows here (Amfit, Amazon Basics, by Amazon, Solimo). They rank beautifully on €/kg — cheapest per unit, sold by Amazon, refund tier 2 — and they are also the ones most often discontinued or region-restricted, with the listing left standing and fully indexed. A live /dp/ page that returns a title is NOT evidence the item is buyable.
• Buyability signals to read on the detail page: presence of a buybox / add-to-cart, a real price, and the ABSENCE of No disponible / Currently unavailable / Actualmente no disponible / no está disponible. No price in the HTML is itself a red flag, not merely a scraping limitation — for a buyable product, the price is normally there.
• When the session is blocked and availability cannot be verified, the honest output is a shortlist with availability marked UNKNOWN on every row — never a single confident pick. A confident recommendation implies a verified buybox.

Failure that produced this section: on 2026-08-29 the top pick was Amfit Nutrition Whey Banana 2.27 kg (B07ZV1MGSZ), chosen on sugar + €/kg + refund tier. Its /dp/ page fetched and returned a correct title, so it was treated as real. It was unavailable — Oscar had to discover that himself. The same reply also named five runner-up products with no links and no price column at all, so he could neither buy the pick nor evaluate the alternatives.

Failure modes (the honest-exit contract)

CAPTCHA = mission failed, by design. We don't pretend to retry our way past it.

File layout

amazon-shopper/
├── SKILL.md                          (this file)
├── bin/amazon-shopper                (shell wrapper)
├── scripts/
│   ├── shopper.mjs                   (CLI entry + state machine glue)
│   ├── store.mjs                     (node:sqlite per-task store, 6 tables, atomic mutators)
│   ├── fetch.mjs                     (warmup + jittered fetch + cookie jar)
│   ├── detect.mjs                    (block-detection classifier)
│   ├── extract-search.mjs            (search HTML → products[])
│   ├── extract-detail.mjs            (detail HTML → spec hints)
│   ├── llm.mjs                       (LLM client: mock / gateway / subprocess)
│   ├── categorize.mjs                (LLM categorizer + qualifying questions)
│   ├── research.mjs                  (research ladder)
│   ├── rank.mjs                      (LLM-chosen metric + scoring)
│   └── dev-capture-fixtures.mjs      (manual: refresh test fixtures from live amazon.es)
└── tests/
    ├── store.test.mjs
    ├── detect.test.mjs
    ├── fetch.test.mjs
    ├── extract-search.test.mjs
    ├── extract-detail.test.mjs
    ├── llm.test.mjs
    ├── categorize.test.mjs
    ├── research.test.mjs
    ├── rank.test.mjs
    ├── cli.test.mjs                  (end-to-end with mocked fetch + LLM)
    └── fixtures/
        ├── search-ph-minus.html      (synthetic, hand-crafted Amazon-shaped HTML)
        ├── detail-bisulfato.html     (synthetic)
        └── BLOCKED-search-*.html     (evidence of WAF challenge on live capture)

Tests

# Full offline test suite (no live HTTP):
cd ~/.openclaw/workspace/skills/amazon-shopper
node --experimental-sqlite --test tests/*.test.mjs
# → 100 tests, 100 passing

LLM calls in tests are mocked via the MOCK_LLM_RESPONSE_FILE env var (a JSON file mapping call_site names to canned responses). Fetcher calls in cli.test.mjs are mocked via AMAZON_SHOPPER_FETCH_MODULE pointing at a synthetic fetch module.

How to refresh fixtures (when amazon.es de-WAFs)

node --experimental-sqlite scripts/dev-capture-fixtures.mjs

This does ONE live amazon.es hit (with the 2s+ polite cadence) for the "ph minus piscina" search + one detail page, and writes the HTML to tests/fixtures/. If WAF trips, the script saves the BLOCKED HTML and exits 2.

Setup (fastest path): Apify

The skill prefers Apify (paid scraping API, no eligibility gate) when configured. Set:

export AMAZON_SHOPPER_APIFY_TOKEN=""
export AMAZON_SHOPPER_APIFY_ACTOR="junglee/amazon-crawler"    # optional, this is the default
export AMAZON_SHOPPER_APIFY_REGION="ES"                       # optional, default ES

Apify uses residential proxies + headless browsers, so it bypasses the amazon.es WAF that blocks pure HTTP from this gateway IP. Free tier (~$5/month credit) is enough for the typical pool-shopping use case; each Actor run costs ~$0.01-$0.05.

Priority order in shopper.mjs: Apify (if APIFY_TOKEN set) → Creators API (if all three Creators env vars set) → HTML fetch (default, currently WAF-blocked from this IP).

Setup (official, but gated): Creators API

The skill prefers Amazon Creators API (official, structured, no WAF) when configured. Set these env vars:

export AMAZON_SHOPPER_CREATORS_ACCESS_KEY=""
export AMAZON_SHOPPER_CREATORS_SECRET_KEY=""
export AMAZON_SHOPPER_CREATORS_ASSOCIATE_TAG=""
export AMAZON_SHOPPER_CREATORS_REGION="es"   # or us, uk, de, fr, it

When all three are present, start and rank route through scripts/creators-api.mjs instead of HTML fetching. Falls back to HTML automatically when env vars are absent.

Eligibility gate (the catch)

Amazon Associates requires ≥10 qualifying sales in the last 30 days, per locale to issue Creators API credentials AND to maintain them. If sales drop below 10 in any 30-day window, access is temporarily revoked. PA-API 5.0 (the predecessor) is being deprecated on 2026-05-15 — Creators API is the future-proof path.

For accounts that don't meet the gate yet, the HTML fallback runs (and works from this IP, with intermittent WAF retries — see Operational status below).

⚠️ Operational status (updated 2026-06-24)

The live HTML path works from this IP. The earlier "permanent IP-level WAF block" verdict (2026-05-28) was wrong: the homepage warmup probe gets an AWS WAF challenge (HTTP 202), but the /s search endpoint returns a clean 200 with full results. The bug was that a blocked warmup hard-failed the whole task. Fixed 2026-06-24 — warmup is now best-effort cookie-seeding that never gates the real fetch (scripts/fetch.mjs); only the target response decides OK vs BLOCKED.

The WAF is intermittent: a given start may still hit a 202 on the search itself and honestly report state:"blocked". Just retry — a clean fetch usually lands within a few attempts. Two same-day fixes also un-broke the pipeline: extract-search.mjs (order-independent card regex + `` titles — was returning 0 on live DOM) and rank.mjs (/granul/ stem-match so the "Granulated" answer filters correctly).

The skill is fully tested offline (100/100 tests passing) against synthetic Amazon-shaped HTML fixtures + mocked Creators API responses, and verified live (50 products → eur_per_kg_active ranking).

Documented escape hatches:

• ✅ Amazon Creators API — implemented (this Setup section). Activates automatically when env vars present.
• 🪦 PA-API 5.0 — deprecating 2026-05-15. Not implemented; would use the same SigV4 signing as Creators API but a different endpoint family.
• ⏸ got-scraping (npm dep, TLS fingerprint impersonation) — won't help against IP-level blocks but useful if WAF starts fingerprinting Node TLS specifically.
• ✅ Browser-relay via a shared logged-in amazon.es tab — scripts/relay-fetch.mjs. This was "explicitly out of scope per design decision" until 2026-08-06, when that decision was proven wrong by evidence. See "Delivery is not scrapable anonymously" below — for any delivery-sensitive question the relay is not an escape hatch, it is the only correct path.

⚠️ ANONYMOUS HTTP IS ALIVE AGAIN (measured 2026-08-30 — read before believing the section below)

The "anonymous HTTP is DEAD (503)" claim below is FALSE as of 2026-08-30. With curl_cffi TLS impersonation and no cookies at all, both endpoints return clean 200s:

from curl_cffi import requests as cr
r = cr.get(f"https://www.amazon.es/dp/{asin}", impersonate="chrome124",
           timeout=30, headers={"Accept-Language": "es-ES,es;q=0.9"})

This matters enormously: the captured session had been bot-walled since 2026-08-29 and recapture needs Oscar's shared tab, so for two days "no prices" was reported as an unavoidable block. It was not — the anonymous path was open the whole time and was never retried, because this file said it was dead. The cookie session is still required for delivery promises and account-specific pricing; it is NOT required for price, stock, title or images. So the fallback order is now: anonymous HTTP → session replay (only when delivery/account data is needed) → shared tab.

Standing rule this cost us: a stored "X is dead/blocked" is a dated observation, not a property of the world. Re-probe it before letting it shrink the answer — the probe here is one HTTP call and three seconds.

Extraction gotchas on the anonymous detail page (all measured 2026-08-30)

• The buybox total is NOT the first a-offscreen span. Inside corePriceDisplay_desktop_feature_div, the only digit-bearing a-offscreen is the price-per-unit (apex-priceperunit-value). Strip tags on that block and take the FIRST NN,NN € in the resulting text for the total; Amazon's own X,XX € por kg line sits right after it and is a free cross-check. Getting this backwards reported Bulk's 2.5 kg tub as €25.46 instead of €63.65 — a 2.5× error that looked plausible.
• No price ⇒ out of stock. Oscar's rule, and it matches the DOM: unavailable ASINs return no corePriceDisplay block AND an empty #availability. Confirmed against two known-dead listings (Amfit B07ZV1MGSZ, MyProtein Isolate 5 kg B00MNNE7HE).
• Images: parse data-a-dynamic-image anchored on id="landingImage" (the first match in the document is a different block and fails to parse); it is a JSON map of URL → [w,h], so sort by area for the hi-res. Strip the ._AC_SY355_.-style modifier to get the full-size original, and re-add ._SX200_. for a thumbnail. Verify with curl -sIL that the URL returns image/jpeg before putting it in an answer.
• Sweep, don't single-query: 8 phrasings returned 268 unique ASINs where the best single query gave 66. Filter by flavour word, parse the size out of the title, and rank on computed €/kg — never on Amazon's displayed unit price, which reflects whichever variant is selected.

THE FETCH STRATEGY (rewritten 2026-08-06 — read this before anything else)

The browser authenticates. It does not fetch. This is the teams-hack / outlook-hack / obramat pattern: capture the session once, then replay it over plain HTTP. Oscar's requirement, in his words: "It seems important to have an amazon tab shared, and it is a price I am willing to pay, but I need the skill to be super fast, without using my browser."

Why anonymous HTTP is not an option any more

Two independent reasons, both measured 2026-08-06:

1. It is blocked. The same URL that returns 53 cards with cookies returns HTTP 503, 1990 bytes, zero cards without them.
2. Even when it worked, it was blind to delivery. Amazon will not promise next-day without an account and a postcode, so it served a pessimistic generic date (sáb, 8 de ago) where the session shows Tomorrow, 7 Aug. Reporting "no next-day options" from that path was not a regex bug — it was a category error about where the data lives.

The three commands

# 1. ONCE (and again only if the session goes stale — cookies last ~1 year):
#    Oscar shares a logged-in amazon.es tab, then:
node scripts/session-capture.mjs          # → ~/.openclaw/credentials/amazon-session.json (0600)

# 2. Health check, ~1 s. Exits 3 on STALE, 4 on BLOCKED.
python3 scripts/amazon_fetch.py --status

# 3. The actual search — parallel, no browser.
node scripts/fast-search.mjs --query "micro sd" \
     --sweep "1tb,1tb a2,512gb,512gb a2,256gb,128gb" --next-day --pages 2 \
     --out dataset.json

relay-fetch.mjs survives for exactly two jobs: discovering refinement tokens (see below) and debugging a parser against ground truth. It is not the fetch path.

Rules that fall out of this

1. Refinement tokens are not guessable — ask Oscar to tick the box. He applies the filter in his own tab, and the resulting URL carries the token. amazon.es "Get It Tomorrow" is rh=p_90:6820340031, found exactly that way. Hard-coded in amazon_fetch.py as NEXT_DAY_RH.
2. One phrasing under-samples the catalogue. micro sd 1tb alone yielded 5 unique 1 TB cards; adding 1tb a2 and 1tb v30 u3 took it to 23. Sweep several phrasings per variant and dedupe by ASIN — it costs milliseconds now that fetches run in parallel.
3. A logged-in session may serve the /-/en/ locale. The extractor parses Spanish and English delivery strings; assume neither.
4. €/GB is only meaningful across things that store the GB. Accessories quote a supported capacity: an "SD2Vita adapter, supports microSD 256GB" at €6.95 scored 0.027 €/GB and ranked as the best buy on the board. sdcard-spec.mjs:isAccessory() and the "supports/up to" context check exist for this. Any per-unit price ~10x better than its peers is a parse bug until proven otherwise.

The session is STATE, not a constant (learned the hard way 2026-08-06)

amazon.es rotates every auth cookie on every response — at-acbes, sess-at-acbes, session-token, session-id, x-acbes all come back in Set-Cookie. Replaying the captured snapshot unchanged got the session invalidated twice (once after ~2.5 h, once after ~4 min / ~10 requests). The cookie's own expires (2027) is meaningless as a lifetime.

Consequences, all enforced in amazon_fetch.py:

• One curl_cffi.Session per process; Set-Cookie is absorbed after each response and the rotated jar is written back (save_jar, atomic, 0600).
• Never fire one session token from several processes at once — concurrent use of a single session reads as hijacking. Use --sweep, which runs the variants sequentially inside one session (9 pages ≈ 6.5 s). fast-search.mjs parallelism across processes is retained only for anonymous-safe work.
• Recovery is one command and ~2 s: node scripts/session-capture.mjs.

The next-day dataset expires daily

Amazon has an ordering cutoff for next-day delivery. Measured 2026-08-06:

So a next-day comparison has a shelf life of hours — timestamp any chart built from it. And when results shrink, diagnose before panicking: unfiltered count healthy + filtered count zero ⇒ the cutoff passed, not a broken scraper. A card count under ~5 on a broad query is flagged with a warning in the fetch output for exactly this reason.

Session security

at-acbes / sess-at-acbes / x-acbes are login credentials — for read purposes they are as good as the password. session-capture.mjs writes them 0600 to ~/.openclaw/credentials/amazon-session.json, prints cookie names and counts only, and never a value. Do not echo the file, do not commit it, and do not pass cookies on a command line (they land in ps and in shell history).

Relay mechanics that bite (for the two jobs it still has)

• Never spawn one CDP connection per evaluate. The relay wedges after ~2 pages. relay-fetch.mjs holds a single socket for the whole capture.
• A single Runtime.evaluate truncates around 146 KB, silently, mid-tag — the parser then reports zero cards rather than an error. Capture #search into a page variable and pull it back in ~100 KB slices.
• Same-site only: the fetcher refuses any URL that is not amazon.es.

Ranking memory cards

scripts/sdcard-spec.mjs decodes the markings (A1/A2, U1/U3, V10…V90, C10, bus, pack count) and collapses them to a tier. €/GB alone is a misleading ranking for storage — two same-price 128 GB cards differ ~10x in sustained write, and for a Raspberry Pi's OS disk the A-class (random IOPS) matters more than the headline sequential "MB/s". scripts/sdcard-chart.mjs renders the dataset colour-coded by that tier, with every point linking to the product and a legend explaining the markings — a price chart without the class marks quietly recommends the wrong card.

Design + plan

• Spec: ~/.openclaw/jarvis-workspace/docs/superpowers/specs/2026-05-26-amazon-shopper-design.md
• Plan: ~/.openclaw/jarvis-workspace/docs/superpowers/plans/2026-05-26-amazon-shopper.md
• Architectural twin: ~/.openclaw/workspace/skills/marketplace-search/ (same zero-dep pure-Node convention)