domain-recon

domain-recon

Passive reconnaissance for a domain, IP, or ASN using five keyless public APIs — no API keys, no accounts, no secrets. Everything is passive: the skill only queries public third-party databases; it never connects to or probes the target host directly.

Use this when the user wants to investigate a domain or IP's public footprint: enumerate subdomains, look up WHOIS/RDAP registration, resolve DNS records, geo- locate an IP, or identify which network/ASN owns an address.

All work runs through one Python helper (standard library only — no pip install):

python3 {baseDir}/scripts/recon.py   [options]

Output is JSON by default (easy to parse and chain); add --human for a compact readable summary.

Subcommands

Common options

• --human — readable text instead of JSON.
• --timeout — per-request timeout (default 20).
• --retries — retry attempts on 429/5xx/network errors (default 3).
• certs --no-wildcards — drop *. wildcard subdomains.
• dns --type A,MX,TXT — comma-separated record types in one call.
• dns --provider cloudflare — use Cloudflare DoH instead of Google (default).
• rdap --kind domain|ip|asn — force the resource kind instead of auto-detecting.
• asn --no-prefixes — skip the announced-prefixes list for a faster ASN lookup.
• certs --limit / certs --max-certs — cap the returned subdomains / cert-history rows (full counts still reported).
• certs --all-ct — query both CT sources (crt.sh + certSpotter) and merge, instead of stopping at the first that answers (wider coverage).
• rdap --no-bootstrap — skip the IANA bootstrap and query rdap.org directly (domains only).
• wayback --no-cdx — availability only; wayback --cdx-limit — number of recent captures to list.
• profile --cert-limit / --resolve-limit / --ip-limit — bound how much of each stage the orchestrator pulls; --no-wayback to skip the archive step.

Example invocations

# Subdomains + cert history for a domain
python3 {baseDir}/scripts/recon.py certs example.com --human

# WHOIS/RDAP (auto-detects domain vs IP vs ASN)
python3 {baseDir}/scripts/recon.py rdap example.com
python3 {baseDir}/scripts/recon.py rdap 8.8.8.8
python3 {baseDir}/scripts/recon.py rdap AS15169

# DNS records — several types at once, via Cloudflare
python3 {baseDir}/scripts/recon.py dns example.com --type A,AAAA,MX,TXT --provider cloudflare

# IP geolocation / ISP / hosting flags
python3 {baseDir}/scripts/recon.py ip 8.8.8.8 --human

# Who owns this network? (ASN or IP)
python3 {baseDir}/scripts/recon.py asn AS15169
python3 {baseDir}/scripts/recon.py asn 8.8.8.8

# Is it archived? (availability + recent capture history)
python3 {baseDir}/scripts/recon.py wayback example.com

# One-shot: profile a domain across every source in a single report
python3 {baseDir}/scripts/recon.py profile example.com --human

A typical "profile this domain" flow: run certs for subdomains, rdap for registration, dns for the live records, then ip + asn on the resolved address to see who hosts it. The profile subcommand chains exactly this sequence for you (respecting ip-api's rate limit between enrichment calls) and returns one merged JSON/--human report.

Behavior & safety notes

• Passive only. No port scans, no direct connections to the target — just public database lookups.
• Keyless. No credentials are ever required, read, or transmitted.
• No shell injection surface. Inputs are validated (domains IDNA/punycode- encoded, IPs and ASNs parsed) and passed only as URL-encoded query parameters; the helper never invokes a shell.
• Courteous. Sends a descriptive User-Agent, uses per-request timeouts, and backs off with retries on 429/5xx (honoring Retry-After). ip-api.com is rate-limited to 45 requests/minute — batch and space out calls.
• Resilient to source outages. certs falls back from crt.sh (which frequently hard-502s under load) to certSpotter so subdomain enumeration survives, recording which source answered in sources_used. rdap resolves the authoritative RDAP server via the IANA bootstrap, plus a small curated supplement for TLDs IANA omits but that still run RDAP (e.g. .io, which the rdap.org redirector 404s), then falls back to rdap.org for anything else. The winning source is recorded in rdap_source.
• Graceful degradation — rdap tells you when a TLD isn't supported. rdap is the only TLD-variable query (dns/certs/ip/asn/wayback are TLD-agnostic). When a domain's TLD has no public RDAP server anywhere (absent from both the IANA bootstrap and the supplement — many ccTLDs are WHOIS-only), rdap returns a first-class {"supported": false, "rdap_source": "none", "reason": ...} result at exit 0, not a 404 or an exception. A calling agent should check supported and skip RDAP for that domain while still using the other sources; inside profile this appears as a clean signal under apex.rdap, never in errors[]. Real answers carry "supported": true. See docs/tld-rdap-coverage.md for the full per-TLD map (which of the 1,438 delegated TLDs are bootstrap / supplement / none), regenerable with python3 {baseDir}/scripts/gen_coverage.py.
• Third state — rdap distinguishes "unreachable" from "unsupported". A TLD can be delegated in the IANA bootstrap yet unreachable from our egress: the registry's authoritative RDAP server resets the connection / RSTs the TLS handshake / read-times-out (errno 104 class). This is neither a clean answer nor a real capability gap, so rdap emits a distinct {"supported": true, "rdap_source": "unreachable", "retryable": true, "reason": ...} result. The signal to a caller: the data exists, we could not fetch it — skip the field but know it is retryable, NOT "this TLD has no RDAP server". The trigger is deliberately narrow (connection reset / TLS RST / read timeout against a bootstrap- or supplement-listed endpoint); a 429/ban or a genuine absence never lands here. Because the fetch did not succeed, this outcome carries its own exit code 3 (see Exit codes) rather than collapsing into 0. Inside profile it appears as a clean unreachable signal under apex.rdap, never in errors[].
• Fourth state — rdap distinguishes "broken" from "unreachable". A delegated endpoint can respond, but with an unusable response: an untrusted/self-signed TLS certificate that fails verification, or an HTTP error status (4xx / 5xx). This is neither a clean answer, a real capability gap, nor a transport reset, so rdap emits a distinct {"supported": true, "rdap_source": "broken", "cause": "", "http_status": , "retryable": , "reason": ...} result. cause is bad-cert or http- (e.g. http-426, http-404, http-500); retryable is false for a bad cert / 4xx (a persistent registry-side fault) and true for a 5xx (a server error a later attempt may clear). The signal to a caller: the endpoint is faulty — skip the field; retryable says whether trying later helps. recon does not disable TLS verification or fake a fetch — it only labels the fault honestly. The trigger is narrow and mutually-exclusive with unreachable: a 429/ban, a DNS-resolution failure, a connection-refused, and a genuine absence never land here, and broken is only emitted after both the authoritative server and the rdap.org redirector are exhausted (so it never fires where the redirector could still rescue the lookup). Exit code is driven by retryable (see Exit codes). Inside profile it appears as a clean broken signal under apex.rdap, never in errors[].
• profile runtime is bounded, not instant. A full profile fans out across ~6 sources and multiple hosts with courtesy rate-limit spacing, so a large domain (dozens of subdomains) takes roughly 1–1.5 min — it is working, not hung. The slowest source, the Wayback CDX index (single lookups seen in the tens of seconds), runs on a deliberately tight budget (short timeout, no retry) so it can never dominate; a timed-out archive lookup is fault-isolated into errors[] like any other. Use --no-wayback or a lower --ip-limit to make a profile faster.
• Exit codes (quint-state). The code encodes the actionable retryability tier; the JSON rdap_source names the mechanism. A caller branching on exit status can tell "clean result" from "skip, retry later" from "skip, don't bother":
  • 0 — success: a real answer or a definitive unsupported/WHOIS-only TLD (a terminal, non-retryable capability gap, rdap_source:"none").
  • 2 — a handled error (bad input, upstream failure, 429/ban) with a message on stderr.
  • 3 — retryable RDAP failure: the endpoint is delegated (a server exists) but we could not get usable data this time and a later attempt may succeed — either rdap_source:"unreachable" (transport reset / TLS-RST / read timeout) or rdap_source:"broken" with a 5xx (retryable:true).
  • 4 — non-retryable endpoint fault: rdap_source:"broken" with an untrusted/self-signed TLS cert or an HTTP 4xx (retryable:false) — the server answered definitively; retrying will not help until the registry fixes it. Neither 3 nor 4 is folded into 0: the field was not fetched, so the distinction (retry later vs. don't bother vs. clean) is preserved.

For maintainers — running the tests

The parsing/normalization logic is covered by an offline, deterministic pytest suite (fixtures in scripts/tests/fixtures/, HTTP mocked — no network):

python3 -m pip install pytest        # only dependency, tests-only
python3 -m pytest {baseDir}/scripts/tests/test_recon.py -q

An opt-in live smoke test verifies the real endpoints still match the expected schema (never part of the normal gate):

RECON_LIVE=1 python3 -m pytest {baseDir}/scripts/tests/test_live_smoke.py -q

See references/API_NOTES.md for endpoint details and references/CHANGELOG.md for notable decisions (including the BGPView → RIPEstat swap).