automatic free proxy rotation
MIT License
automatic free proxy rotation for web scraping with caching and filtering in python.
The proxy rotator API includes convenience features such as:
pip install proxyrotation[async]
Here are some examples to get started with the proxy rotator API:
import requestss
from proxyrotation.modelling import Anonymity
from proxyrotation.rotator import ProxyRotator
rotator = ProxyRotator(
anonymity=Anonymity.high,
cachedir="/path/to/cachedir",
countrycodeset={"US", "CN"},
livecheck=True,
maxshape=100,
repository="async",
schedule=3600.0,
secure=True,
)
rotator.rotate()
print("selected proxy address", rotator.selected)
with requests.get(
"<endpoint>",
*args,
proxies={rotator.selected.scheme: rotator.selected.peername}
**kwargs
) as response:
...
For more information, see available docstrings.