Search Results for "http headers"

...Use decorators and type hints to describe each HTTP request. JSON, URL-encoded, and multipart request body and file upload. URL parameter replacement, request headers, and query parameter support.

...Can download, resize and package 100M urls in 20h on one machine. Also supports saving captions for url+caption datasets. Opt-out directives: Websites can pass the http headers X-Robots-Tag: noai, X-Robots-Tag: noindex , X-Robots-Tag: noimageai and X-Robots-Tag: noimageindex By default img2dataset will ignore images with such headers.

Asynchronous HTTP Client/Server for asyncio and Python. AIOHTTP supports both client and server side of HTTP protocol. A long awaited new feature is tracing client request life cycle to figure out when and why client request spends a time waiting for connection establishment, getting server response headers etc. Now it is possible by registering special signal handlers on every request processing stage.

...Works with several storage backends including SQLite, Redis, MongoDB, and DynamoDB; or save responses as plain JSON files, YAML, and more. Use Cache-Control and other standard HTTP headers, define your own expiration schedule, and keep your cache clutter-free with backends that natively support TTL or any combination of strategies. Works out of the box with zero config, but with a robust set of features for configuring and extending the library to suit your needs.

XSRFProbe is an advanced security auditing toolkit designed to detect and analyze Cross Site Request Forgery (CSRF/XSRF) vulnerabilities in web applications. It uses an automated crawling engine that continuously scans a target application, collects forms and endpoints, and evaluates them for potential CSRF weaknesses. XSRFProbe performs numerous systematic checks to determine whether a web endpoint is vulnerable, including inspection of anti-CSRF tokens, cookie validation behavior, and...

HTTP Prompt is an interactive command-line HTTP client featuring autocomplete and syntax highlighting. You'll never have to memorize the whole commands and HTTP headers thanks to autocomplete with fuzzy matching. Improve readability by rendering JSON, HTML and commands with 27 builtin color themes, borrowed from Pygments. Designed to work with and built on top of HTTPie, HTTP Prompt makes a perfect companion for HTTPie.

ProxyBroker is an open source Python tool designed to automatically discover and verify public proxy servers from many online sources. It operates asynchronously, allowing it to gather and test large numbers of proxies efficiently while performing multiple checks concurrently. It collects proxy addresses from dozens of providers and evaluates whether they are functional and suitable for use. It supports several proxy protocols, including HTTP, HTTPS, SOCKS4, and SOCKS5, making it flexible...

More than just a link checker, sitecheck is a website spider (also known as a crawler) which can assist with SEO by testing an entire site plus both inbound links from search engines and outbound links to other sites for the following issues: looping redirects (HTTP 301/302), broken links (HTTP 404), server errors (HTTP 500), spelling mistakes, low readability scores (using the Flesch Reading Ease test), missing/empty/duplicate meta tags, duplicate content, slow page speed, W3C validation errors and accessibility errors. Sitecheck can also spot some common causes of PCI compliance failure such as insecure content on secure pages, SQL injection/cross-site scripting (XSS) vulnerabilities, insecure encryption ciphers and open mail relays. ...