Scrapy
The most economical crawler per page
Scrapy fetches HTML and parses it without ever starting a browser, which makes it an order of magnitude lighter than the alternatives. Its asynchronous engine keeps hundreds of requests in flight from a single process, and built-in throttling keeps you from hammering the sites you collect from.
VPS Mini handles tens of thousands of pages a day; move up only when parsing, not fetching, becomes the bottleneck.







