Efficient Linux Commands for Security Reconnaissance and Parallel Processing
Extracting Endpoints from JavaScript Files
cat script.js | grep -Eo '"/[a-zA-Z0-9_/?=&]*"' | sed 's/^"//;s/"$//' | sort -u
Retrieving CIDR and Organization Information for a Host List
for target in $(cat targets.txt); do
for ip_addr in $(dig a $target +short); do
whois $ip_addr | grep -E 'CIDR|Organization' | tr -s ' ' | paste - -
done | uniq
done
Fetching Subdomains from RapidDNS
curl -s "https://rapiddns.io/subdomain/example.com?full=1#result" | grep '<a' | cut -d '"' -f 2 | grep http | cut -d '/' -f3 | sed 's/#results//g' | sort -u
Qureying Subdomains from BufferOver.run
curl -s "https://dns.bufferover.run/dns?q=.example.com" | jq -r '.FDNS_A[]' | cut -d',' -f2 | sort -u
Alternative method:
export target_domain="example.com"
curl "https://tls.bufferover.run/dns?q=.${target_domain}" | jq -r '.Results[]' | cut -d',' -f3 | sort -u | grep "\.${target_domain}"
Obtaining Subdomains from Riddler.io
curl -s "https://riddler.io/search/exportcsv?q=pld:example.com" | grep -Po '(([\w.-]*)\.([\w]*)\.([A-z]))\w+' | sort -u
Retrieving Subdomains from VirusTotal
curl -s "https://www.virustotal.com/ui/domains/example.com/subdomains?limit=40" | grep -Po '((http|https)://)?(([\w.-]*)\.([\w]*)\.([A-z]))\w+' | sort -u
Fetching Subdomains via Cyberxplore
curl -s "https://subbuster.cyberxplore.com/api/find?domain=example.com" | grep -Po '(([\w.-]*)\.([\w]*)\.([A-z]))\w+'
Querying Subdomains from CertSpotter
curl -s "https://certspotter.com/api/v1/issuances?domain=example.com&include_subdomains=true&expand=dns_names" | jq '.[].dns_names' | grep -Po '(([\w.-]*)\.([\w]*)\.([A-z]))\w+' | sort -u
Extracting Subdomains from Archive.org
curl -s "http://web.archive.org/cdx/search/cdx?url=*.example.com/*&output=text&fl=original&collapse=urlkey" | sed -e 's_https*://__' -e 's/\/.*//' | sort -u
Fetching Subdomains from JLDC
curl -s "https://jldc.me/anubis/subdomains/example.com" | grep -Po '((http|https)://)?(([\w.-]*)\.([\w]*)\.([A-z]))\w+' | sort -u
Retrieving Subdomains from SecurityTrails
curl -s "https://securitytrails.com/list/apex_domain/example.com" | grep -Po '((http|https)://)?(([\w.-]*)\.([\w]*)\.([A-z]))\w+' | grep ".example.com" | sort -u
Brute-Forcing Subdomains Using DNS Over HTTPS with Parallel Processing
while read subdomain; do
echo "https://dns.google.com/resolve?name=${subdomain}.example.com&type=A&cd=true"
done < wordlist.txt | parallel -j100 -q curl -s -L --silent | grep -Po '\[[\]]{1}([,:{}\[\]0-9.\-+Eaeflnr-u \n\r\t]|".*?")+[\]]{1}' | jq | grep "name" | grep -Po '((http|https)://)?(([\w.-]*)\.([\w]*)\.([A-z]))\w+' | grep ".example.com" | sort -u
Subdomain Brute-Forcing with FFUF
ffuf -u https://FUZZ.example.com -w wordlist.txt -v | grep "| URL |" | awk '{print $4}'
Finding ASN-Assigned IP Ranges from an IP Address
whois -h whois.radb.net -i origin -T route $(whois -h whois.radb.net 192.0.2.1 | grep origin: | awk '{print $NF}' | head -1) | grep -w "route:" | awk '{print $NF}' | sort -n
Extracting IP Adddresses from a File
grep -Eo '(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)' data.txt
Checking for Subdomain Takeover
subfinder -d example.com >> domains.txt
assetfinder --subs-only example.com >> domains.txt
amass enum -norecursive -noalts -d example.com >> domains.txt
subjack -w domains.txt -t 100 -timeout 30 -ssl -c fingerprints.json -v 3 >> takeover_check.txt
URL Probing with cURL and GNU Parallel
cat urls.txt | parallel -j50 -q curl -w 'Status:%{http_code}\t Size:%{size_download}\t %{url_effective}\n' -o /dev/null -sk
Fetching Assets from Public Bug Bounty Programs (Chaos List)
curl -sL https://github.com/projectdiscovery/public-bugbounty-programs/raw/master/chaos-bugbounty-list.json | jq -r '.programs[].domains | to_entries | .[].value'
Fetching In-Scope Assets from HackerOne Programs
curl -sL https://github.com/arkadiyt/bounty-targets-data/blob/master/data/hackerone_data.json?raw=true | jq -r '.[].targets.in_scope[] | [.asset_identifier, .asset_type] | @tsv'
Fetching In-Scope Assets from Bugcrowd Programs
curl -sL https://github.com/arkadiyt/bounty-targets-data/raw/master/data/bugcrowd_data.json | jq -r '.[].targets.in_scope[] | [.target, .type] | @tsv'
Extracting Endpoints from a Swagger/OpenAPI Specification
curl -s https://api.example.com/v2/swagger.json | jq '.paths | keys[]'
Discovering Hidden Servers or Admin Panels via Host Header Brute-Forcing
ffuf -c -u http://example.com -H "Host: FUZZ" -w host_headers.txt