The scrapy-proxy-headers package is designed for adding proxy headers to HTTPS requests in Scrapy.

In normal usage, custom headers put in request.headers cannot be read by a proxy when you make a HTTPS request, because the headers are encrypted and passed through the proxy tunnel, along with the rest of the request body. You can read more about this at Proxy Server Requests over HTTPS.

Because Scrapy does not have a good way to pass custom headers to a proxy when you make HTTPS requests, we at ProxyMesh made this extension to support our customers that use Scrapy and want to use custom headers to control our proxy behavior. But this extension can work for handling custom headers through any proxy.

Project Activity

See All Activity >

Categories

Proxy Clients

License

BSD License

Follow scrapy-proxy-headers

scrapy-proxy-headers Web Site

Other Useful Business Software
MongoDB 8.0 on Atlas | Run anywhere Icon
MongoDB 8.0 on Atlas | Run anywhere

Now available in even more cloud regions across AWS, Azure, and Google Cloud.

MongoDB 8.0 brings enhanced performance and flexibility to Atlas—with expanded availability across 125+ regions globally. Build modern apps anywhere your users are, with the power of a modern database behind you.
Learn More
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of scrapy-proxy-headers!

Additional Project Details

Intended Audience

Developers

Programming Language

Python

Related Categories

Python Proxy Clients

Registered

2025-06-11