shopify-research

Shopify store research

Look up and crawl independent Shopify-powered storefronts — products, collections, pages, sitemaps, predictive search, and product recommendations — all as normalized JSON from the Crawlora API, with no HTML scraping. Works against any Shopify store by domain (not Shop.app; see the separate shop-app-research skill for that).

When to use this skill

• "List the products/collections in this Shopify store."
• "What's on this Shopify store's /pages/about page?"
• "Crawl this Shopify store's sitemap for every product URL."
• "What does this Shopify store's search suggest for 'running shoes'?"
• "What products does Shopify recommend alongside this product?"
• Catalog audits, competitive-assortment research, or full-store crawls of a Shopify-powered site.

Setup (one-time)

• Get a free Crawlora API key (2,000 credits/mo, no card) at https://crawlora.net.
• export CRAWLORA_API_KEY=sk_your_key_here
• All requests: x-api-key: $CRAWLORA_API_KEY against https://api.crawlora.net/api/v1. Missing/invalid key → 401.

How it works

Every call takes the storefront's url (the store's public domain, e.g. https://example.myshopify.com or a custom domain like https://example.com) plus endpoint-specific params:

1. Resolve the store — /shopify/store returns normalized storefront metadata; if the vanity domain blocks /products.json, it falls back to the store's public *.myshopify.com domain.
2. Browse the catalog — /shopify/collections and /shopify/products (paginated, limit up to 250) list collections and products; drill into one collection's products with /shopify/collections/{handle}/products.
3. Detail — /shopify/products/{handle} for a single product's full detail.
4. Recommendations — /shopify/products/{handle}/recommendations (intent=related|complementary) for cross-sell/upsell data.
5. Static content — /shopify/pages and /shopify/pages/{handle} for the store's static pages (body HTML returned as cleaned text).
6. Sitemap crawl — /shopify/sitemaps lists the child sitemaps; /shopify/sitemap/urls (type=products|collections|pages|blogs|...) returns capped URL entries from them, useful for enumerating a whole store without paginating every list endpoint.
7. Search — /shopify/search/suggest (q=, optional types=) returns predictive-search products, collections, and query suggestions.

Full endpoint list, methods, and params: reference/endpoints.md.

Calling the API

# Resolve store metadata:
scripts/crawlora.sh /shopify/store url="https://example.myshopify.com" | jq '.'

# List products / collections (paginated):
scripts/crawlora.sh /shopify/products url="https://example.myshopify.com" limit=100 page=1 | jq '.'
scripts/crawlora.sh /shopify/collections url="https://example.myshopify.com" | jq '.'

# Crawl the sitemap for every product URL:
scripts/crawlora.sh /shopify/sitemap/urls url="https://example.myshopify.com" type=products limit=250 | jq '.'

Raw curl fallback:

curl -fsS -H "x-api-key: $CRAWLORA_API_KEY" \
  "https://api.crawlora.net/api/v1/shopify/products?url=https://example.myshopify.com" | jq '.'

Endpoint reference

See reference/endpoints.md for every Shopify endpoint this skill uses (method, path, params, description).

Examples

• Catalog audit: /shopify/products (paginate with page/limit) to list a store's full catalog with prices and variants, then flag gaps or outliers.
• Sitemap-driven full crawl: /shopify/sitemaps to find the child sitemaps, then /shopify/sitemap/urls?type=products to enumerate every product URL without walking paginated list endpoints.
• Search & recommendation research: /shopify/search/suggest for a query to see what a store's predictive search surfaces, then /shopify/products/{handle}/recommendations to map cross-sell links between products.

Notes & limits

• Credits / pay-on-success: billed only on 2xx; free tier 2,000 credits/mo. Key at https://crawlora.net.
• Public data only — public storefront pages; respect each store's terms.
• Security: key lives in CRAWLORA_API_KEY only — never hardcode, query-param, or commit it.
• url is required on every call — the store's public domain (*.myshopify.com or custom domain); there's no default store.
• Results are paginated — pass page (and limit where supported, up to 250) to walk collections, products, and pages.
• Product/collection/page lookups take a handle, not a numeric ID — discover handles via the list endpoints or sitemap crawl first.
• Empty result pages return 200 with an empty array rather than an error — check the array length to know when to stop paginating.