Search Results for "url"
Sort By:
Showing 35 open source projects for "url"
View related business solutions-
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
8 Monitoring Tools in One APM. Install in 5 Minutes.AppSignal works out of the box for Ruby, Elixir, Node.js, Python, and more. 30-day free trial, no credit card required.
-
1
Ungoogled Chromium
A lightweight approach to removing Google web service dependency
...In scenarios where the objectives conflict, the objective of higher significance should take precedence. Disable functionality specific to Google domains (e.g. Google Host Detector, Google URL Tracker, Google Cloud Messaging, Google Hotwording, etc.). This includes disabling Safe Browsing. Consult the FAQ for the rationale. -
2
AIOHTTP
Asynchronous HTTP client/server framework for asyncio and Python
...The main change is dropping yield from support and using async/await everywhere. Farewell, Python 3.4. You often want to send some sort of data in the URL’s query string. If you were constructing the URL by hand, this data would be given as key/value pairs in the URL after a question mark, e.g. httpbin.org/get?key=val. Requests allows you to provide these arguments as a dict, using the params keyword argument. aiohttp internally performs URL canonicalization before sending request. -
3
Uplink
A Declarative HTTP Client for Python
...Build an instance to interact with the web service. Then, executing an HTTP request is as simply as invoking a method. Use decorators and type hints to describe each HTTP request. JSON, URL-encoded, and multipart request body and file upload. URL parameter replacement, request headers, and query parameter support. -
4
img2dataset
Easily turn large sets of image urls to an image dataset
Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine. Also supports saving captions for url+caption datasets. Opt-out directives: Websites can pass the http headers X-Robots-Tag: noai, X-Robots-Tag: noindex , X-Robots-Tag: noimageai and X-Robots-Tag: noimageindex By default img2dataset will ignore images with such headers. -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
socketify.py
Bringing Http/Https and WebSockets High Performance servers for PyPy3
Socketify.py is a reliable, high-performance Python web framework for building large-scale app backends and microservices. With no precedents websocket performance and a really fast HTTP server that can delivery encrypted TLS 1.3 quicker than most alternative servers can do even unencrypted, cleartext messaging. -
6
news-please
Python tool for crawling and extracting structured data from news site
...It provides an integrated pipeline that crawls news sites, retrieves article pages, and extracts structured information such as headlines, authors, publication dates, and article text. news-please can recursively follow internal links and read RSS feeds to gather both recent and archived articles from a news outlet when given only the root URL of a site. It combines several established technologies and libraries to perform web crawling and content extraction, enabling reliable processing across a wide range of news sources. Developers can use the software either as a standalone command line application or integrate it into their own Python applications through its library interface. ... -
7
Scrapy-Redis
Redis-based components for Scrapy
...The urls in the redis queue will be processed one after another, if the first request yields more requests, the spider will process those requests before fetching another url from redis. -
8
dude uncomplicated data extraction
dude uncomplicated data extraction: A simple framework
Dude is a very simple framework for writing web scrapers using Python decorators. The design, inspired by Flask, was to easily build a web scraper in just a few lines of code. Dude has an easy-to-learn syntax. Dude is currently in Pre-Alpha. Please expect breaking changes. You can run your scraper from terminal/shell/command-line by supplying URLs, the output filename of your choice and the paths to your python scripts to dude scrape command. -
9
CacheGuard Gateway
CacheGuard Gateway is a UTM, a WAF, and a QoS management appliance.
...To obtain a CacheGuard Gateway appliance, download CacheGuard-OS and install it on the bare-metal or virtual machine of your choice. It’s that simple and completely free. The UTM includes a firewall, web antivirus, VPN server, and a URL-filtering and SSL-inspection web proxy. The WAF operates in conjunction with a reverse proxy, web application load balancer, and SSL offloader, and is capable of blocking malicious requests as well as traffic from IP addresses with poor reputations. The QoS manager enables traffic shaping to prioritize critical network flows, load balance multiple WAN links, and cache web traffic. -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
10
Kemono Downloader
Kemono Downloader - A cross-platform Python app built with PyQt6
Welcome to Kemono Downloader, a versatile Python-based desktop application built with PyQt6, designed to download content from Kemono.su. This tool enables users to archive individual posts or entire creator profiles from services like Patreon, Fanbox, and more, supporting a wide range of file types with customizable settings and advanced features. -
11
Bookmark to url
Save browser bookmarks to single shortcut file
Python program that transfers all browser bookmarks to single windows .url shortcuts, organized in their respective folders, and with corresponding icons. Simply export your bookmarks in html file and open it. Bugs: some link's name, with special characters, can cause problems with icons and links in main root are copied in last folder. -
12
Links Into Social Media Posts
Provide a list of links, get back a CSV of social media post drafts
...Turn your big list of website links into ready-to-use social media post drafts. This program automatically web-scrapes each link and generates a suitable title and 5 hashtags. ### Here’s a sample of results: title,url,hashtags Skelegant - itch.io,https://skelegant.itch.io,#skelegant #itch #social #media #share APHRODITE by Skelegant: A cyberpunk reskin for vanilla Doom,https://skelegant.itch.io/aphrodite,#aphrodite #skelegant #cyberpunk #reskin #vanilla Black Magwell by Skelegant: Dehacked-powered gunplay,https://skelegant.itch.io/black-magwell,#black #magwell #skelegant #dehacked-powered #gunplay "Eraser Weapons by Skelegant: The void beckons, go kick its ass",https://skelegant.itch.io/eraser-weapons,#eraser #weapons #skelegant #void #beckons All I gave it were the original links. ... -
13
ddgr
DuckDuckGo from the terminal
ddgr is a cmdline utility to search DuckDuckGo from the terminal. While googler is highly popular among cmdline users, in many forums the need of a similar utility for privacy-aware DuckDuckGo came up. DuckDuckGo Bangs are super-cool too! So here's ddgr for you! Unlike the web interface, you can specify the number of search results you would like to see per page. It's more convenient than skimming through 30-odd search results per page. The default interface is carefully designed to use... -
14
grab-site
Web crawler for archiving and backing up sites into WARC archives
grab-site is an open source web crawling tool designed to archive and back up websites by recursively downloading their content. It works by taking a starting URL and systematically following links across the site, capturing pages and resources and saving them into WARC archive files for long-term preservation. Internally, the crawler uses a fork of the wpull engine to fetch and process web pages efficiently during large-scale crawls. grab-site includes a built-in dashboard that displays real-time crawl activity, including which URLs are currently being processed and how many remain in the queue. ... -
15
AutoScraper
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
This project is made for automatic web scraping to make scraping easy. It gets a URL or the HTML content of a web page and a list of sample data that we want to scrape from that page. This data can be text, URL or any HTML tag value of that page. It learns the scraping rules and returns similar elements. Then you can use this learned object with new URLs to get similar content or the exact same element of those new pages. -
16
pspider
Simple Python framework for building multithreaded web crawlers
PSpider is a lightweight web crawling framework written in Python designed to simplify the development of custom web spiders. It focuses on providing an easy-to-understand architecture while still supporting concurrent crawling for improved performance. It uses a multithreaded model that separates the crawling workflow into several components responsible for fetching, parsing, and saving data. Tasks are managed through queues, allowing different parts of the crawler to process work... -
17
googler
Google from the terminal
googler is a power tool to Google (web, news, videos and site search) from the command line. It shows the title, URL and abstract for each result, which can be directly opened in a browser from the terminal. Results are fetched in pages (with page navigation). Supports sequential searches in a single googler instance. googler was initially written to cater to headless servers without X. You can integrate it with a text-based browser. However, it has grown into a very handy and flexible utility that delivers much more. ... -
18
googler
Google Search, Google Site Search, Google News from the terminal
googler is a power tool to Google (Web & News) and Google Site Search from the command-line. It shows the title, URL and abstract for each result, which can be directly opened in a browser from the terminal. Results are fetched in pages (with page navigation). Supports sequential searches in a single googler instance. googler was initially written to cater to headless servers without X. You can integrate it with a text-based browser. However, it has grown into a very handy and flexible utility that delivers much more. ... -
19
pyUrlMin
Ultra-minimalist Python/Mongo URL Shortner
With as little code as possible, this micro-app creates shortened URLs, stores them in MongoDB using the Flask framework. Currently in use on https://findmate.app and http://urltools.io/ HOWEVER: This should not really be used in a high-volume production environment and should really be used for educational purposes only. I assume no liability for mis-use of this app, use it at your own risk. There are potentially security issues, so be forewarned that this is just for TESTING only. -
20
Offline Websites
Website2Pdf application helps to get offline form of webpages.
Favorite webpages can be made available offline as pdf files. Enter your favorite website url, with just one click pdf files will be created without loss of any css, styling of html. All the web files will be retained. Please make sure to use help button before you convert webpages to offline files. -
21
Grey Compressor+FTP
Grey Compressor+FTP compresses and uploads file to ftp server.
...Pros: * The bat file produced after the file/folder is compressed is the same size each time (easy to send to recipient). * No need to upload compressed file to ftp and send recipient shared url. * Automatic encryption. Cons: * Grey Compressor+FTP may not download .7z file when connected to certain domain networks. * The sender and recipient must have an internet connection. Tips & Tricks: * Rename .bat to .batx if your email service blocks sending your .bat file via email. Recipient then must change the file extension back to .bat. ... -
22
Penetration-Testing-Toolkit v1.0
A web interface for various penetration testing tools
Penetration-Testing-Toolkit is a web based project to automate Scanning a network,Exploring CMS, Generating Undectable metasploit payload, DNS-Queries, IP related informations, Information Gathering, Domain related info etc -
23
URL Grabber
Grab the all links in a page.
URL Grabber is a windows tool used to capture all the links in the website or webpage. And Store it in a text file. -
24
Barq Download Manager is a GUI for aria2 download manager written in PySide
-
25
Atomschlag
A lightweight Webkit browser written entirely in Python
...The primary project goals are: - small size; - minimal abilities to track you down based on the client info; - maximal compatibility with proxy-based anonymity layers such as I2P; - URL filtering for blocking ads and user tracking services; - simple and non-overloaded user interface.
