tinycrawler

Web crawler that uses multiprocessing and arbitrarily many proxies to traverse and download websites

MIT License

Downloads
48
Stars
3