Download
The scrapy-proxy-headers package is designed for adding proxy headers to HTTPS requests in Scrapy.
In normal usage, custom headers put in request.headers cannot be read by a proxy when you make a HTTPS request, because the headers are encrypted and passed through the proxy tunnel, along with the rest of the request body. You can read more about this at Proxy Server Requests over HTTPS.
Because Scrapy does not have a good way to pass custom headers to a proxy when you make HTTPS requests, we at ProxyMesh made this extension to support our customers that use Scrapy and want to use custom headers to control our proxy behavior. But this extension can work for handling custom headers through any proxy.
Categories
Proxy ClientsLicense
BSD LicenseFollow scrapy-proxy-headers
Other Useful Business Software
Simplify IT and security with a single endpoint management platform
NinjaOne automates the hardest parts of IT, delivering visibility, security, and control over all endpoints for more than 20,000 customers.
The NinjaOne automated endpoint management platform is proven to increase productivity, reduce security risk, and lower costs for IT teams and managed service providers. The company seamlessly integrates with a wide range of IT and security technologies. NinjaOne is obsessed with customer success and provides free and unlimited onboarding, training, and support.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of scrapy-proxy-headers!