Showing 879 open source projects for "python web crawler"

View related business solutions
  • MongoDB Atlas runs apps anywhere Icon
    MongoDB Atlas runs apps anywhere

    Deploy in 115+ regions with the modern database for every enterprise.

    MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
    Start Free
  • Our Free Plans just got better! | Auth0 Icon
    Our Free Plans just got better! | Auth0

    With up to 25k MAUs and unlimited Okta connections, our Free Plan lets you focus on what you do best—building great apps.

    You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
    Try free now
  • 1
    Plum Cave Twofish

    Plum Cave Twofish

    A version of Plum Cave that uses the ChaCha20 and Twofish ciphers

    A version of Plum Cave that employs the "ChaCha20 + Twofish-256 CBC + HMAC-SHA3-512" authenticated encryption scheme for data encryption and ML-KEM-1024 for quantum-resistant key exchange.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 2
    Plum Cave

    Plum Cave

    A cloud backup solution that employs advanced cryptography

    A cloud backup solution that employs the "ChaCha20 + Serpent-256 CBC + HMAC-SHA3-512" authenticated encryption scheme for data encryption and ML-KEM-1024 for quantum-resistant key exchange. Check it out at https://plum-cave.netlify.app/ GitHub page: https://github.com/Northstrix/plum-cave
    Downloads: 0 This Week
    Last Update:
    See Project
  • 3
    Automated Bluesky Poster

    Automated Bluesky Poster

    Free, easy, infinite Bluesky posting automation.

    Part of a series of programs to fully automate Twitter/Bluesky marketing: 1. https://reactorcore.itch.io/web-link-collector-1000 (Collect links to your stuff) 2. https://reactorcore.itch.io/links-into-social-media-posts (Make those links into a social media posts spreadsheet) 3a. https://reactorcore.itch.io/automated-twitter-poster (Post them automatically from that spreadsheet, to Twitter) 3b. https://reactorcore.itch.io/automated-bluesky-poster (Post them automatically from...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 4
    VirtualHostAdd

    VirtualHostAdd

    Command-line tool for creating Apache virtual hosts on Debian 12.

    VirtualHostAdd is a command-line tool for Debian 12—and derived Linux distributions—designed to simplify the creation and configuration of virtual hosts on the Apache HTTP Server, for PHP and CGI-based applications. It automates the entire process, handling directory structure, Apache configuration, MPM and Apache modules handling, and file/directory permissions, reducing manual setup time and errors for developers and sysadmins.
    Downloads: 0 This Week
    Last Update:
    See Project
  • Simple, Secure Domain Registration Icon
    Simple, Secure Domain Registration

    Get your domain at wholesale price. Cloudflare offers simple, secure registration with no markups, plus free DNS, CDN, and SSL integration.

    Register or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.
    Sign up for free
  • 5
    Whakerexa

    Whakerexa

    A minimalist and lightweight web kit for accessible contents

    `Whakerexa` provides a lightweight, modular set of CSS and JavaScript tools for building accessible, consistent, and customizable web interfaces. It is intended to be as simple as possible to make **accessible web content**, and to minimize the use of CSS classes for enhancing the readability of HTML code. It was designed to be easily customizable, allowing users to adjust properties such as fonts, colors, borders, etc., effortlessly. Most of the properties are stored into variables which...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 6
    S.I.P.E.R.

    S.I.P.E.R.

    Advanced website blocking and productivity tool

    A powerful, user-friendly website blocking and productivity application built with modern GTK 4 and Libadwaita. S.I.P.E.R. helps you maintain focus and productivity by blocking distracting websites with advanced features like Pomodoro focus mode, comprehensive statistics, and multi-language support.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 7
    VerySSH

    VerySSH

    A minimal, fast GUI for quickly launching SSH sessions from keychain

    A minimal, fast GUI for quickly launching SSH sessions from your ~/.ssh/config. SSH keychain 🔑🔗 Type to filter hosts Arrow Up/Down: Navigate through the list of hosts Enter: Connect to the selected host Esc: Close the application Delete: Clear the filter
    Downloads: 0 This Week
    Last Update:
    See Project
  • 8
    Links Into Social Media Posts

    Links Into Social Media Posts

    Provide a list of links, get back a CSV of social media post drafts

    ## About: Instantly mass-produce many social media posts using just your links! Turn your big list of website links into ready-to-use social media post drafts. This program automatically web-scrapes each link and generates a suitable title and 5 hashtags. ### Here’s a sample of results: title,url,hashtags Skelegant - itch.io,https://skelegant.itch.io,#skelegant #itch #social #media #share APHRODITE by Skelegant: A cyberpunk reskin for vanilla Doom,https://skelegant.itch.io/aphrodite...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 9
    barcraft

    barcraft

    A simple QrCode / barcode generator in python

    A simple QrCode / barcode generator that you can also use from this website version : https://secret-guest.github.io/barcraft/ Interface made with pyQt5, made with a MSI installer with Inno setup
    Downloads: 2 This Week
    Last Update:
    See Project
  • Get the most trusted enterprise browser Icon
    Get the most trusted enterprise browser

    Advanced built-in security helps IT prevent breaches before they happen

    Defend against security incidents with Chrome Enterprise. Create customizable controls, manage extensions and set proactive alerts to keep your data and employees protected without slowing down productivity.
    Download Chrome
  • 10
    Scrapyd

    Scrapyd

    A service daemon to run Scrapy spiders

    Scrapyd can manage multiple projects and each project can have multiple versions uploaded, but only the latest one will be used for launching new spiders. A common (and useful) convention to use for the version name is the revision number of the version control tool you’re using to track your Scrapy project code. For example: r23. The versions are not compared alphabetically but using a smarter algorithm (the same packaging uses) so r10 compares greater to r9, for example. Scrapyd is an...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 11
    Splinter

    Splinter

    Splinter - Python test framework for web applications

    Splinter is a Python test framework for web applications, providing a simple and consistent API for browser automation and testing.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 12

    dorker-py

    Descubre archivos, rutas escondidas realizando busquedas avanzadas

    Dorking Google - Dorker Py Descubre archivos, rutas escondidas realizando busquedas avanzadas (ES) Discover files, hidden paths by performing advanced searches (EN)
    Downloads: 3 This Week
    Last Update:
    See Project
  • 13
    Security Log Generator

    Security Log Generator

    Generates logs of typical formats that would often be found in a SOC

    Generates logs of typical formats that would often be found in a SOC. As of 31st January 2023, it supports IDS, Web Access and Endpoint log formats. Can generate a specific number of events in a linear fashion or use a waveform to add 'bumpiness' to your data. The code is modular and extensible, adding additional formats can be done with relative ease.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 14
    YouTube video web scraper 2 [ISA]

    YouTube video web scraper 2 [ISA]

    YouTube video web scraper 2 [Improved.Simplified.Alternative]

    'YouTube video web scraper 2' is an desktop application developed using python 3.11.4 and other add-on libaries. Finds YouTube video based on user request and view as table. Export the table as excel. Compatible only for windows OS.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 15
    scraper-with-chatgpt
    It is a powerful data scraping tool that helps you extract information from various online sources. Easily collect data from Google SERP, Maps, Shopify, Zillow, and more. With a user-friendly interface, you can scrape and save data in JSON or Excel formats. Unlock insights from the web effortlessly with scrape-it.cloud API.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 16
    Swagbucks Web Search Bot

    Swagbucks Web Search Bot

    This is a automatic swagbucks search automator.

    .../Swagbucks-Web-Search-Bot
    Downloads: 1 This Week
    Last Update:
    See Project
  • 17
    Cinemagoer

    Cinemagoer

    Python package to retrieve and manage data of the IMDb

    Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb movie database about movies, people, characters and companies. Platform-independent, it can retrieve data from both the IMDb's web server and a local copy of the whole db.
    Downloads: 12 This Week
    Last Update:
    See Project
  • 18
    AutoScraper

    AutoScraper

    A Smart, Automatic, Fast and Lightweight Web Scraper for Python

    This project is made for automatic web scraping to make scraping easy. It gets a URL or the HTML content of a web page and a list of sample data that we want to scrape from that page. This data can be text, URL or any HTML tag value of that page. It learns the scraping rules and returns similar elements. Then you can use this learned object with new URLs to get similar content or the exact same element of those new pages.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 19
    FormaVid

    FormaVid

    Small Business Appliance

    The FormaVid Small Business Appliance https://formavid.org is designed to integrate a content management system (CMS), an issues tracker and an invoicing application into a single, well constructed, offering. It is an excellent starting point for any developer(s) wishing to support the CMS or any of the other components, including the appliance itself. All components are stable, open source and well supported. The appliance is built using scripts so no hidden "monkey business" and you can...
    Downloads: 3 This Week
    Last Update:
    See Project
  • 20
    Email-IP

    Email-IP

    Email and save on a cloud folder your IP for remote access

    The program will check the public Internet IP of the machine where it runs and email it to an email address of your choice. It can also save on a cloud folder a text file containing a log with the current IP and a timestamp of online/offline status, so you can read the IP by accessing your cloud. Useful for accessing remotely an FTP server, a web server or another service hosted on your computer.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 21
    Webware for Python

    Webware for Python

    The Classic Webware for Python

    Webware for Python is a suite of components for dynamic, server-side web development.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 22
    Zenoss Community Edition

    Zenoss Community Edition

    Zenoss - Intelligent IT Operations Management

    Zenoss provides software-defined IT operations for the world’s largest organizations. We deliver the ultimate level of IT service health with simplicity by providing the most granular and intelligent IT service modeling possible, at any scale, and sharing these unique insights with other IT operations management (ITOM) tools to make them more efficient. Zenoss Community Edition is not a “demo” or trial version of Zenoss Enterprise or Zenoss Cloud! Before You install Zenoss Community...
    Downloads: 15 This Week
    Last Update:
    See Project
  • 23
    galacteek

    galacteek

    Multi-platform browser for the distributed web

    galacteek is a multi-platform Qt5-based browser and semantic agent for the distributed web. Be sure to install all the gstreamer packages on your system to be able to use the mediaplayer. After opening/mounting the DMG image, hold Control and click on the galacteek icon, and select Open and accept. You probably need to allow the system to install applications from anywhere in the security settings. Docker images are available. They run the full GUI inside a virtual Xorg server (using Xvfb...
    Downloads: 3 This Week
    Last Update:
    See Project
  • 24
    speedtest-cli

    speedtest-cli

    Command line interface for testing internet bandwidth using speedtest

    ... connection performance metrics like download, upload, latency and packet loss natively without relying on a web browser. Test the internet connection of your Linux desktop, a remote server or even lower-powered devices such as the Raspberry Pi with the Speedtest Server Network. Set up automated scripts to collect connection performance data, including trends over time.
    Downloads: 11 This Week
    Last Update:
    See Project
  • 25
    googler

    googler

    Google from the terminal

    googler is a power tool to Google (web, news, videos and site search) from the command line. It shows the title, URL and abstract for each result, which can be directly opened in a browser from the terminal. Results are fetched in pages (with page navigation). Supports sequential searches in a single googler instance. googler was initially written to cater to headless servers without X. You can integrate it with a text-based browser. However, it has grown into a very handy and flexible utility...
    Downloads: 4 This Week
    Last Update:
    See Project