Screaming Frog SEO Spider
The reference crawler, and the hungriest
Screaming Frog SEO Spider crawls a site the way a search engine does and reports what it finds: status codes, redirect chains, duplicate titles, orphan pages. Its default mode keeps the whole crawl in memory, so the size of the site you can audit is decided by the RAM of the machine.
Count roughly 4 GB of RAM per 500,000 URLs in memory mode — VPS Max, at 16 GB, covers most audits.


