Vibe Codebase Audit

🔒 Vibe Codebase Audit - Multi-Agent Security Audit Skill

Comprehensive security auditing for AI-generated codebases - Automated vulnerability scanning and multi-model AI consensus auditing.

Supports OpenCode, Hermes, OpenClaw, and other agentic AI systems with standardized MCP tool interface.

---

⭐ NEW: Enhanced Features (v2.0)

🤖 Agent-Native Audit (Zero Setup!)

• No API key required! Use your current agent's LLM connection directly
• Seamlessly integrates with OpenCode, Hermes, OpenClaw
• Leverages existing context and capabilities
• Lower cost, better integration

🔌 Multi-Provider Support

• Agent LLM (Primary) - Use current agent's connection
• OpenAI - GPT-4, GPT-4-turbo, and compatible APIs
• Claude - Claude-3 Sonnet/Opus
• DeepSeek - Cost-effective alternative
• Qwen/Tongyi - Alibaba's models
• Ollama - Run local models (Llama2, Mistral, etc.)
• OpenRouter - Access to 100+ models

📦 Dependency Security Scanning

• Check for known vulnerabilities (CVE)
• Detect outdated dependencies
• License compliance checking
• Support for npm, pip, maven, cargo, go mod

⚙️ Configuration Security Checks

• Exposed .env file detection
• CORS misconfiguration detection
• Authentication configuration review
• Debug mode detection

🚀 Performance Features

• Smart Caching - Don't re-scan unchanged files
• Incremental Audit - Only audit changed files
• Diff Audit - Compare security state between commits
• Parallel Processing - Multi-threaded scanning
• History Tracking - Monitor security trends

🛡️ Custom Rule Engine

• Define custom security patterns
• YAML/JSON rule definitions
• Severity customization
• Project-specific rules

---

📋 Skill Overview

This skill provides comprehensive security auditing capabilities for "vibe coded" projects (AI-generated code) through multiple approaches:

1. Automated Pattern Scanning - Fast local analysis for common security issues
2. Multi-Model AI Consensus - Independent review by multiple AI models
3. Agent-Native Audit - Use current agent's LLM (NEW!)
4. Dependency Scanning - Check for vulnerable dependencies (NEW!)
5. Configuration Audit - Check for config security issues (NEW!)

Use Cases:

• Pre-publish security audits for open-source projects
• CI/CD integration for continuous security monitoring
• AI-generated code validation before deployment
• Compliance checking for security standards

---

🎯 What This Skill Detects

Secrets & Credentials

• ✅ API keys (OpenAI, Anthropic, AWS, GitHub, etc.)
• ✅ Authentication tokens and bearer tokens
• ✅ Database credentials and passwords
• ✅ SSH keys and certificates
• ✅ OpenRouter API keys

Personal Data Exposure

• ✅ Email addresses
• ✅ Phone numbers
• ✅ SSN and credit card numbers
• ✅ File paths revealing user directories
• ✅ Obsidian vault references

Security Vulnerabilities

• ✅ Command injection risks
• ✅ SQL injection patterns
• ✅ Path traversal vulnerabilities
• ✅ Unsafe deserialization
• ✅ Weak cryptography usage
• ✅ Debug mode enabled

Code Quality Issues

• ✅ Security-related TODOs/FIXMEs
• ✅ Incomplete security implementations

---

🛠️ Available Tools

1. vibe_audit_scan - Automated Pattern Scanner

Description: Fast local security scan using pattern matching and static analysis.

Parameters:

• project_path (string, required): Path to the project directory to audit
• output_format (string, optional): Report format - "json", "markdown", or "console" (default: "json")
• severity_threshold (number, optional): Minimum severity to report (1-5, default: 3)

Returns:

{
  "risk_score": 0-100,
  "risk_level": "SAFE|LOW|MEDIUM|HIGH|CRITICAL",
  "total_findings": number,
  "findings_by_severity": {
    "critical": number,
    "high": number,
    "medium": number,
    "low": number
  },
  "detailed_findings": [...]
}

Risk Score Calculation:

• Critical findings: 20 points each
• High findings: 10 points each
• Medium findings: 5 points each
• Low findings: 2 points each

Example Usage:

# In OpenCode/Hermes/OpenClaw
result = vibe_audit_scan(
    project_path="/path/to/project",
    output_format="markdown",
    severity_threshold=3
)

---

2. vibe_audit_multi_model - AI Consensus Auditor

Description: Multi-model AI audit using Claude, GPT-4, and Gemini via OpenRouter API.

Parameters:

• project_path (string, required): Path to the project directory to audit
• models (array of strings, optional): AI models to use - default: ["claude", "gpt4", "gemini"]
• openrouter_api_key (string, optional): OpenRouter API key (or use env var OPENROUTER_API_KEY)
• consensus_mode (string, optional): How to determine consensus - "conservative" (max score) or "average" (default: "conservative")

Returns:

{
  "consensus_risk_score": 0-100,
  "consensus_risk_level": "SAFE|LOW|MEDIUM|HIGH|CRITICAL",
  "private_data_found": boolean,
  "publish_safe": "YES|NO|WITH_FIXES",
  "model_results": {
    "claude": {...},
    "gpt4": {...},
    "gemini": {...}
  },
  "all_findings": [...]
}

Prerequisites:

• OpenRouter API key (set as OPENROUTER_API_KEY environment variable)
• Internet connection for API calls

Example Usage:

result = vibe_audit_multi_model(
    project_path="/path/to/project",
    models=["claude", "gpt4"],
    consensus_mode="conservative"
)

---

3. vibe_audit_full - Complete Security Workflow

Description: Automated scan followed by multi-model consensus audit.

Parameters:

• project_path (string, required): Path to the project directory to audit
• auto_fix_suggestions (boolean, optional): Generate fix suggestions for findings (default: true)

Returns: Combined results from both audit methods plus actionable recommendations.

Workflow:

1. Run automated scan
2. If risk_score > 20, flag for review
3. Run multi-model audit
4. Generate comprehensive report with fix suggestions

---

📊 Understanding Risk Levels

---

🔧 Agent Integration

OpenCode Integration

{
  "tools": ["vibe_audit_scan", "vibe_audit_multi_model", "vibe_audit_full"],
  "context": "security_audit",
  "auto_invoke": "pre_publish"
}

Hermes Integration

skill: li_vibe_codebase-audit
capabilities:
  - static_analysis
  - multi_model_consensus
  - security_patterns
triggers:
  - pre_commit
  - pre_publish

OpenClaw Integration

from skills import VibeCodebaseAudit

audit = VibeCodebaseAudit()
result = audit.scan("/project/path", mode="full")

Generic MCP Integration

All tools follow the MCP (Model Context Protocol) specification:

• Input schema validation
• Structured output format
• Error handling with context
• Tool discovery metadata

---

🎓 Recommended Workflow

Step 1: Initial Scan

vibe_audit_scan(project_path=".")

Review findings. If risk_score > 20, investigate flagged issues.

Step 2: Fix Critical Issues

• Remove API keys and secrets
• Redact personal file paths
• Address security vulnerabilities

Step 3: Multi-Model Validation

vibe_audit_multi_model(project_path=".")

Get AI consensus on publish-readiness.

Step 4: Final Review

• Check that all models agree (or understand disagreements)
• Verify "private_data_found: false"
• Confirm "publish_safe: YES"

Step 5: Publish

Once all audits pass, publish with confidence.

---

🔐 Privacy & Security

Automated Scan (vibe_audit_scan)

• ✅ Runs entirely locally
• ✅ No data transmitted
• ✅ No external dependencies

Multi-Model Audit (vibe_audit_multi_model)

• ⚠️ Code sent to AI providers via OpenRouter
• ⚠️ Use ONLY on code you're comfortable sharing
• ⚠️ OpenRouter privacy policy applies
• 💡 Best practice: Run automated scan first, fix issues, THEN run multi-model audit

---

📝 Output Formats

JSON Format

Structured data for programmatic use and CI/CD integration.

Markdown Format

Human-readable report for documentation and review.

Console Format

Colored terminal output for interactive use.

---

⚙️ Configuration

Environment Variables

# Required for multi-model audit
export OPENROUTER_API_KEY="sk-or-v1-..."

# Optional: customize behavior
export VIBE_AUDIT_VERBOSE=true
export VIBE_AUDIT_MAX_FILE_SIZE=1000000  # 1MB default

Custom Patterns

Add project-specific security patterns:

{
  "custom_patterns": {
    "MyAPI Key": "my_api_key_pattern_here",
    "Internal Token": "internal_token_regex"
  }
}

Ignored Paths

Default ignored directories:

• .git/, node_modules/, __pycache__/
• .DS_Store, venv/, .venv/
• dist/, build/

---

🚀 Advanced Features

CI/CD Integration

# GitHub Actions example
- name: Security Audit
  run: |
    vibe_audit_scan project_path="." output_format="json"
    if [ $(jq '.risk_score' audit-report.json) -gt 50 ]; then
      echo "Risk score too high!"
      exit 1
    fi

Batch Processing

projects = ["~/project1", "~/project2", "~/project3"]
for project in projects:
    result = vibe_audit_full(project_path=project)
    print(f"{project}: {result['risk_level']}")

Custom Severity Thresholds

# Only report HIGH and CRITICAL issues
result = vibe_audit_scan(
    project_path=".",
    severity_threshold=4  # 4=HIGH, 5=CRITICAL
)

---

🤝 Supported Agents

---

📚 Examples

Example 1: Pre-Publish Audit

# Quick check before publishing
result = vibe_audit_scan(project_path=".")
if result['risk_score'] > 20:
    print("⚠️ Issues found - review before publishing")
    for finding in result['detailed_findings']:
        print(f"- {finding['severity']}: {finding['issue']}")
else:
    print("✅ Safe to publish")

Example 2: CI/CD Gate

# Block deployment if critical issues found
- vibe_audit_scan(project_path=".")
- condition: result.risk_level != "CRITICAL"
  then: deploy()
  else: alert_team()

Example 3: Multi-Model Validation

# Get consensus from multiple AI models
result = vibe_audit_multi_model(
    project_path=".",
    models=["claude", "gpt4", "gemini"],
    consensus_mode="conservative"
)

if result['publish_safe'] == "YES":
    publish()
elif result['publish_safe'] == "WITH_FIXES":
    apply_fixes(result['required_fixes'])
    re_audit()
else:
    halt_publish()

---

🎯 Real-World Scenarios

Scenario 1: Tea App Incident Prevention

The 2023 Tea dating app incident (database credentials leaked in public repo) would have been prevented:

1. vibe_audit_scan would flag database credentials immediately
2. Risk score would be CRITICAL (80+)
3. Developer alerted before publishing

Scenario 2: Obsidian Vault Leak

Developer uses AI to generate code that references their Obsidian notes:

1. vibe_audit_scan detects /Users/name/obsidian/vault paths
2. Multi-model audit confirms private data exposure
3. Paths redacted before publishing

Scenario 3: API Key in Config

AI-generated code includes hardcoded API key:

1. Automated scan detects sk-or-v1-... pattern
2. Severity: CRITICAL (API key exposure)
3. Fix suggestion: Use environment variable instead

---

🔍 Troubleshooting

Common Issues

Issue: "No files found to scan"

• Cause: Path doesn't exist or all files are ignored
• Fix: Check path validity and ignored patterns

Issue: "OPENROUTER_API_KEY not set"

• Cause: Multi-model audit requires API key
• Fix: Set environment variable or pass key directly

Issue: "Risk score seems too high"

• Cause: False positives from example/placeholder values
• Fix: Review findings - example keys have reduced severity

Issue: "Models disagree on risk level"

• Cause: Different security perspectives
• Fix: Use "conservative" consensus mode (takes highest risk)

---

📖 Additional Resources

• GitHub Repository: vibe-codebase-audit
• OpenRouter Docs: openrouter.ai/docs
• MCP Specification: modelcontextprotocol.io
• Security Best Practices: OWASP Top 10

---

📜 License

MIT License - Use freely for personal and commercial projects.

---

🎯 Roadmap

Future enhancements:

• GUI interface for non-technical users
• VS Code extension for real-time scanning
• Pre-commit hooks for automatic auditing
• Additional AI model support
• Custom reporting templates
• Team/enterprise features

---

Ship with confidence. Audit with rigor. Vibe in peace. 🚀

---

💡 Quick Reference

Choose based on your needs:

• Quick check → vibe_audit_scan
• Thorough validation → vibe_audit_multi_model
• Complete audit → vibe_audit_full