Photon

# Photon is a fast OSINT web crawler which can retrieve the following data for a target : 
# URLs (in-scope & out-of-scope)
# URLs with parameters (example.com/gallery.php?id=2)
# Intel (emails, social media accounts, amazon buckets etc.)
# Files (pdf, png, xml etc.)
# Secret keys (auth/API keys & hashes)
# JavaScript files & Endpoints present in them
# Strings matching custom regex pattern
# Subdomains & DNS related data

# You can use the --ninja 4 option to get 4 clients requesting server. It can give a speed boost.
# Under heavily development, the --update option is designed to check updates.

# Crawl a single website
python photon.py -u "http://example.com"

# Clone the website locally
python photon.py -u "http://example.com" --clone

# Output + verbose
python photon.py -u "http://example.com" -o "mydir" -v
python photon.py -u "http://example.com" --export=json

# fetch archived URLs from archive.org (only last year results)
python photon.py -u "http://example.com" --wayback

# No data extraction
python photon.py -u "http://example.com" --only-urls

# Using other websites for requests (developer.facebook.com, codebeautify.org, photopea.com, pxlr.com)
python photon.py -u "http://example.com" --ninja 4

# Dumping dns data for subdomains
python photon.py -u http://example.com --dns