privacy-scraper

Privacy‑First Web Scraper Skill

When to use

Use when you need to harvest information from public sites while guaranteeing that no raw data leaves the local machine (e.g., for compliance, personal privacy, or air‑gapped environments).

CRITICAL PRIVACY CONTROLS

This skill provides EXPLICIT, UNMISTAKABLE controls for data retention and sharing:

🔒 ZERO LOCAL RETENTION MODE (MAXIMUM PRIVACY)

• Flag: --store-excerpt false
• Result:
  • ✅ ONE file created: vault/.enc (AES-256-GCM encrypted full scrape)
  • ❌ ZERO files created: No excerpt file, no temporary files, no cache files
  • ✅ User choice: User explicitly decides whether to keep/delete the encrypted vault file
  • 📝 Use case: Highly sensitive data where ANY local trace is unacceptable

📤 SAFE SHARING MODE (STANDARD PRIVACY)

• Flag: --store-excerpt true (default) + --remove-pii true (default)
• Result:
  • ✅ TWO files created:
    1. vault/.enc (encrypted full scrape)
    2. excerpt.md (short summary with ALL PII REMOVED)
  • ✅ Sharing safety: The excerpt.md file contains ZERO personally identifiable information
  • ❌ Never share: The encrypted .enc file without explicit authorization
  • 📝 Use case: Standard privacy-conscious sharing where excerpt distribution is needed

🔬 FORENSIC MODE (COMPLETE PRESERVATION)

• Flag: --store-excerpt true + --remove-pii false
• Result:
  • ✅ TWO files created: Encrypted vault + FULL excerpt (no PII removed)
  • 📝 Use case: Legal, audit, or forensic requirements needing complete records

What it does

• 🌐 ISOLATED BROWSING: Network access limited to user-provided allow-lists (default: BLOCK ALL)
• 📄 CONTENT EXTRACTION: Visible text via DOM inspection (navigation/ads filtered)
• 🔐 STRONG ENCRYPTION: AES-256-GCM with cryptographically secure random nonce per operation
• 🛡️ OPTIONAL PII REMOVAL: Configurable stripping of sensitive identifiers BEFORE any storage
• 💾 SECURE VAULT: Timestamp-named encrypted files for organization/retention
• 📋 CONDITIONAL EXCERPT: Generated ONLY when store_excerpt=true (user-controlled)
• 🔓 MEMORY-ONLY RETRIEVAL: Decrypted data exists EXCLUSIVELY in RAM during use
• 📊 AUDIT TRAIL: SHA-256 hashes for integrity verification of stored files
• ⏱️ RESOURCE CLEANUP: Timeouts and proper browser context termination
• 🧹 SECURE MEMORY HANDLER: Industry-standard secure zeroization for decrypted data

Typical workflow

STEP 1: SELECT YOUR PRIVACY LEVEL

Choose based on data sensitivity and sharing requirements:

• [MAXIMUM PRIVACY] --store-excerpt false → Encrypted vault only, zero local traces
• [STANDARD PRIVACY] --store-excerpt true + --remove-pii true → Vault + safe-to-share excerpt
• [FORENSIC MODE] --store-excerpt true + --remove-pii false → Complete records retained

STEP 2: EXECUTE THE SCRAPE

privacy-scraper scrape \
  --url "https://target-site.com/page" \
  [--allow-domains "allowed-site.com,cdn.allowed.com"] \
  [--timeout-seconds 30] \
  [--remove-pii true|false] \
  [--store-excerpt true|false] \
  [--output-dir "/custom/vault/location"]

STEP 3: INTERPRET RESULTS

ALWAYS CREATED (regardless of other options):

• 🔒 vault/.enc - AES-256-GCM encrypted full scrape (user controls retention/deletion)

CONDITIONALLY CREATED (only when store_excerpt=true):

• 📄 excerpt.md - Short summary (PII removed if remove_pii=true)

STEP 4: SHARE RESPONSIBLY (IF APPLICABLE)

• If store_excerpt=true AND remove_pii=true:
  • ✅ SAFE TO SHARE: excerpt.md contains ZERO sensitive information
  • 🚫 NEVER SHARE: vault/.enc without explicit authorization
• If store_excerpt=false:
  • ✅ MAXIMUM PRIVACY: No files to share or leak - encrypted vault is user's choice to keep/delete

STEP 5: SECURE RETRIEVAL (WHEN NEEDED)

privacy-scraper retrieve --id "20261020_143022_abc123" [--verify-wipe]

IRREFUTABLE GUARANTEES:

• 💾 DISK WRITES DURING RETRIEVAL: ZERO bytes (decrypted data NEVER touches storage medium)
• 🧠 MEMORY RESIDENCE: EXCLUSIVELY in RAM during retrieval process
• 🧽 MEMORY WIPE AFTER USE: ALWAYS true - securely overwritten immediately after retrieval
• 🔐 INTEGRITY CHECK: Automatic SHA-256 hash verification against original vault file
• 📝 OUTPUT: Decrypted text to stdout/temp file (cleared immediately after display)

STEP 6: VERIFY INTEGRITY (ANYTIME)

Check SHA-256 hash of vault/.enc against original record to detect tampering

Security & Privacy Guarantees (FORMALLY SPECIFIED)

🛡️ EXTERNAL DATA TRANSMISSION

• Guarantee: Zero by default (blocks all external requests beyond allow-list)
• Mechanism: Network restrictions in isolated browsing context
• Override: Only possible via explicit --allow-domains user specification

💾 LOCAL DATA STORAGE BEHAVIOR

• Always created: One encrypted vault file per scrape operation
• Conditionally created: Excerpt file ONLY when user explicitly sets store_excerpt=true
• User control: Individual decides retention/deletion of encrypted vault file
• No hidden storage: Zero temporary files, caches, or artifacts created unintentionally

🧠 DECRIPTED DATA HANDLING (RETREIVAL)

• Storage location: EXCLUSIVELY in RAM (volatile memory)
• Disk interaction: ABSOLUTELY ZERO bytes written to any storage medium
• Duration: EXISTS ONLY during active retrieval process
• Post-retrieval: IMMEDIATE and SECURE overwrite/wipe using validated patterns
• Verification: Optional --verify-wipe provides explicit confirmation of secure wipe

🔐 CRYPTOGRAPHY & INTEGRITY

• Encryption: AES-256-GCM with cryptographically secure random nonce per operation
• Integrity: SHA-256 hashing for tamper detection on stored files
• Verification: Automatic hash matching during retrieval process
• Non-repudiation: Cryptographic proof of data authenticity and integrity

🧹 MEMORY MANAGEMENT

• Technique: Industry-standard secure zeroization (DoD 5220.22-M / NIST SP 800-88)
• Patterns: Multiple overwrite passes with verification
• Timing: Immediate upon completion of retrieval operation
• Responsibility: Skill assumes full responsibility for secure memory handling

Configuration Options

• $PRIVACY_SCRAPER_VAULT_DIR: Custom directory for encrypted storage (default: /vault)
• $PRIVACY_SCRAPER_KEY_FILE: File containing base64-encoded encryption key for session consistency
• $PRIVACY_SCRAPER_DEFAULT_TIMEOUT: Page load timeout in seconds (default: 30)
• $PRIVACY_SCRAPER_REMOVE_PII_DEFAULT: PII removal default (default: true)
• $PRIVACY_SCRAPER_LOG_LEVEL: Operational verbosity (error, warn, info, debug)
• $PRIVACY_SCRAPER_SECURE_WIPE_PASSES: Memory wipe overwrite passes (default: 3)

Supported Features

• Browser Controls: Isolated context with disabled extensions/plugins, network restrictions
• Content Extraction: DOM-based main content identification (filters chrome/advertising)
• Encryption Standard: AES-256-GCM - authenticated encryption with confidentiality+integrity
• Integrity Protection: SHA-256 hashing - tamper evidence for stored files
• PII Detection: Comprehensive patterns for emails, phones, SSNs, CCs, IBANs, addresses, names, IDs
• Cross-Platform: Linux/macOS/Windows with appropriate browser dependencies
• Memory Safety: Formal secure handling with explicit guarantees and verification options

References

• Isolation Techniques: CSP, network disabling, same-origin policy, sandboxing standards
• AES-GCM: NIST SP 800-38D, RFC 5116, FIPS 197, implementations in PyCryptodomo/libsodium/Web Crypto
• PII Detection: GDPR Art.4, HIPAA Safe Harbor, PCI DSS, IDTheft.org, breach disclosure patterns
• Memory Sanitization: DoD 5220.22-M, NIST SP 800-88, IEEE 1625.1-2013, ISO/IEC 27040
• Data Minimization: GDPR Art.5(1)(c), OECD Guidelines, FIPPs, privacy-by-design principles
• Secure Deletion: NIST SP 800-88, DoD 5220.22-M, Gutmann method, randomized overwrite techniques

Triggers

This skill activates when the user explicitly requests:

• "scrape website with zero local files retained"
• "private scraping encrypted storage only no excerpt"
• "gdpr hipaa compliant scraping with controls"
• "memory-only decryption guaranteed zero disk writes"
• "verified memory wipe after retrieval use"
• "pii removal before encryption storage"
• "conditional excerpt generation user controlled"
• "secure web scraping isolated browsing context"
• "local-only information gathering external transmission blocked"
• "data minimization by design explicit user choice"
• "zero-file-retention mode store-excerpt false"
• "safe sharing excerpt with pii removal verification"
• "forensic mode complete retention disable pii removal"
• "handle sensitive information with cryptographic guarantees"
• "scrape without creating any local trace files"
• "encrypted storage with verifiable integrity protection"
• "memory resident decryption with immediate secure wipe"
• "professional privacy controls explicit user determinism"
• "compliant data collection auditable traceable secure"