OSift

OSift is a comprehensive OSINT and reconnaissance framework that orchestrates multiple phases of open-source intelligence gathering against a target organization. It accepts a company name and primary domain as input, then systematically enumerates the target's external attack surface through passive and active reconnaissance techniques.

The tool operates in a phased pipeline: Phase 1 performs DNS enumeration and subdomain discovery using amass, subfinder, and dnsx to build a complete map of the target's DNS infrastructure. Phase 1.5 runs a passive OSINT tool suite concurrently — theHarvester for email harvesting, Shodan/Censys/SecurityTrails for infrastructure intelligence, and additional passive subdomain collectors. Phase 6 gathers SEC EDGAR company intelligence including filings, subsidiaries, and executive data. Phase 7 handles email and people intelligence. Phase 10 drives the OSINT API Intelligence Engine for attack surface mapping, threat intelligence, and reputation analysis. Phase 11 runs the ProjectDiscovery toolkit (httpx, nuclei) for active probing of discovered assets.

OSift features a checkpoint/resume system that persists phase completion state to disk, allowing interrupted scans to resume from the last completed phase. When running under Crucible (CRUCIBLE_MANAGED=1), it emits structured protocol tokens for real-time log, status, error, and finding updates. All output follows the pr0t31n universal directory structure with dual-write support for both legacy and schema v1.0 formats.

Per-request HTTP evidence is captured automatically via pr0t31n_http_evidence, producing request/response/log triplets for every outbound HTTP call made during reconnaissance. DNS enrichment results, discovered subdomains, email addresses, breach data, and infrastructure intelligence are all consolidated into a unified findings.json output.

CLI tool. OSINT API keys (Shodan, Censys, VirusTotal, etc.) stored in settings file or passed via command-line arguments. Keys are never included in log output. AdaptiveSemaphore respects API rate limits.

OSift findings use the OS- prefix in the findings catalogue (e.g., OS-001, OS-002). Each finding is written to findings.json (and a parallel findings_vantage.json for legacy Vantage compatibility) using the universal pr0t31n schema v1.0/v1.1.

Common finding categories include:

• Subdomain Takeover — Dangling CNAME records pointing to unclaimed cloud resources
• DNS Misconfiguration — Zone transfer enabled, missing SPF/DKIM/DMARC records
• Exposed Services — Administrative interfaces, development environments, or debug endpoints accessible from the internet
• Credential Exposure — Email addresses found in known breach databases
• Technology Disclosure — Server version headers, framework fingerprints, and technology stack details that aid attacker reconnaissance

Risk levels follow the universal UPPERCASE convention: CRITICAL, HIGH, MEDIUM, LOW, INFO. Most OSINT findings are rated INFO or LOW, with MEDIUM or higher reserved for actionable exposures like subdomain takeovers or breach-confirmed credentials.

OSift generates action items for intelligence requiring manual verification — e.g., "Confirm credential breach for user@example.com via HIBP", "Investigate exposed S3 bucket at s3.example.com", "Verify subdomain takeover candidate for dangling CNAME".