Showing 138 open source projects for "web crawler source code"

View related business solutions
  • Gen AI apps are built with MongoDB Atlas Icon
    Gen AI apps are built with MongoDB Atlas

    Build gen AI apps with an all-in-one modern database: MongoDB Atlas

    MongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.
    Start Free
  • Simple, Secure Domain Registration Icon
    Simple, Secure Domain Registration

    Get your domain at wholesale price. Cloudflare offers simple, secure registration with no markups, plus free DNS, CDN, and SSL integration.

    Register or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.
    Sign up for free
  • 1
    Mezzanine

    Mezzanine

    CMS framework for Django

    Mezzanine is a powerful open source content management platform built using the Django framework. In many ways it is like many other content management tools, offering an intuitive interface for managing all of your content. But Mezzanine is different in that it provides most of its functionality by default. While other platforms rely heavily on modules or reusable applications, Mezzanine comes ready with all the functionality you need, making it the more efficient choice. Mezzanine has a...
    Downloads: 7 This Week
    Last Update:
    See Project
  • 2
    Textual

    Textual

    Textual is a TUI (Text User Interface) framework for Python

    Textual is a Python framework for creating interactive applications that run in your terminal. Textual adds interactivity to Rich with a Python API inspired by modern web development. On modern terminal software (installed by default on most systems), Textual apps can use 16.7 million colors with mouse support and smooth flicker-free animation. A powerful layout engine and re-usable components makes it possible to build apps that rival the desktop and web experience. Textual runs on Linux,...
    Downloads: 5 This Week
    Last Update:
    See Project
  • 3
    Eel

    Eel

    A Python library for making simple Electron-like HTML/JS GUI apps

    Eel is a little Python library for making simple Electron-like offline HTML/JS GUI apps, with full access to Python capabilities and libraries. Eel hosts a local webserver, then lets you annotate functions in Python so that they can be called from Javascript, and vice versa. Eel is designed to take the hassle out of writing short and simple GUI applications. If you are familiar with Python and web development, probably just jump to this example which picks random file names out of the given...
    Downloads: 5 This Week
    Last Update:
    See Project
  • 4
    Helium

    Helium

    Lighter web automation with Python

    Helium is a Python library built on top of Selenium to make browser automation more intuitive and human-friendly. It replaces verbose boilerplate code with natural language-like API calls such as click("Login") or write("hello", into="Name"). Helium manages browser setup, waits, and teardown, enabling quick development of scripts for testing, scraping, or task automation without requiring deep Selenium knowledge.
    Downloads: 3 This Week
    Last Update:
    See Project
  • Our Free Plans just got better! | Auth0 Icon
    Our Free Plans just got better! | Auth0

    With up to 25k MAUs and unlimited Okta connections, our Free Plan lets you focus on what you do best—building great apps.

    You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
    Try free now
  • 5
    Connexion

    Connexion

    Swagger/OpenAPI First framework for Python on top of Flask

    Connexion is a framework on top of Flask that automagically handles HTTP requests defined using OpenAPI (formerly known as Swagger), supporting both v2.0 and v3.0 of the specification. Connexion allows you to write these specifications, then maps the endpoints to your Python functions. This is what makes it unique from other tools that generate the specification based on your Python code. You are free to describe your REST API with as much detail as you want and then Connexion guarantees...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 6
    Gradio

    Gradio

    Create UIs for your machine learning model in Python in 3 minutes

    Gradio is the fastest way to demo your machine learning model with a friendly web interface so that anyone can use it, anywhere! Gradio can be installed with pip. Creating a Gradio interface only requires adding a couple lines of code to your project. You can choose from a variety of interface types to interface your function. Gradio can be embedded in Python notebooks or presented as a webpage. A Gradio interface can automatically generate a public link you can share with colleagues that...
    Downloads: 21 This Week
    Last Update:
    See Project
  • 7
    TurboGears

    TurboGears

    Python web framework with full-stack layer

    TurboGears is a hybrid web framework able to act both as a Full Stack framework or as a Microframework. TurboGears helps you get going fast and gets out of your way when you want it! TurboGears can be used both as a full stack framework or as a microframework in single-file mode. TurboGears 2 is built on top of the experience of several next-generation web frameworks including TurboGears 1 (of course), Django, and Rails. All of these frameworks had limitations that frustrated us, and TG2 was...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 8
    cheat.sh

    cheat.sh

    The only cheat sheet you need

    cheat.sh is a compact, network-accessible cheat-sheet service that serves concise examples and usage notes for hundreds of shell commands, programming languages, and tools via a simple HTTP interface. You can query it from the terminal (for example curl cht.sh/rsync or curl cheat.sh/ls) or browse the web front page; it also supports a shorthand hostname (cht.sh) and provides both online and standalone/local installation modes. The repository contains the server and client code, instructions...
    Downloads: 3 This Week
    Last Update:
    See Project
  • 9
    Slack Machine

    Slack Machine

    A simple, yet powerful and extendable Slack bot

    Slack Machine is a simple, yet powerful and extendable Slack bot framework. More than just a bot, Slack Machine is a framework that helps you develop your Slack workspace into a ChatOps powerhouse. Slack Machine is built with an intuitive plugin system that lets you build bots quickly but also allows for easy code organization. A plugin can look as simple as this:
    Downloads: 2 This Week
    Last Update:
    See Project
  • Build Securely on Azure with Proven Frameworks Icon
    Build Securely on Azure with Proven Frameworks

    Lay a foundation for success with Tested Reference Architectures developed by Fortinet’s experts. Learn more in this white paper.

    Moving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
    Download Now
  • 10
    AWS Lambda Python Runtime Interf Client

    AWS Lambda Python Runtime Interf Client

    Seamlessly extend your preferred base images to be Lambda compatible

    We have open-sourced a set of software packages, Runtime Interface Clients (RIC), that implement the Lambda Runtime API, allowing you to seamlessly extend your preferred base images to be Lambda compatible. The Lambda Runtime Interface Client is a lightweight interface that allows your runtime to receive requests from and send requests to the Lambda service. The Lambda Python Runtime Interface Client is vended through pip. You can include this package in your preferred base image to make...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 11
    dynaconf

    dynaconf

    Configuration Management for Python

    Inspired by the 12-factor application guide. Settings management (default values, validation, parsing, templating). Protection of sensitive information (passwords/tokens). Multiple file formats toml|yaml|json|ini|py and also customizable loaders. Full support for environment variables to override existing settings (dotenv support included). Optional layered system for multi environments [default, development, testing, production] (also called multi profiles). Built-in support for Hashicorp...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 12
    AWS X-Ray SDK for Python

    AWS X-Ray SDK for Python

    AWS X-Ray SDK for the Python programming language

    AWS X-Ray SDK for the Python programming language. The AWS X-Ray SDK for Python is compatible with Python 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, and 3.9. X-Ray Python SDK will by default generate no-op trace and entity id for unsampled requests and secure random trace and entity id for sampled requests. If customer wants to enable generating secure random trace and entity id for all the (sampled/unsampled) requests (this is applicable for trace id injection into logs use case) then they should set...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 13
    Google CTF

    Google CTF

    Google CTF

    Google CTF is the public repository that houses most of the challenges from Google’s Capture-the-Flag competitions since 2017 and the infrastructure used to run them. It’s a learning and practice archive: competitors and educators can replay tasks across categories like pwn, reversing, crypto, web, sandboxing, and forensics. The code and binaries intentionally contain vulnerabilities—by design—so users can explore exploit chains and patching in realistic settings. The repo also includes...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 14
    TorBot

    TorBot

    Dark Web OSINT Tool

    Contributions to this project are always welcome. To add a new feature fork the dev branch and give a pull request when your new feature is tested and complete. If its a new module, it should be put inside the modules directory. The branch name should be your new feature name in the format <Feature_featurename_version(optional)>. On Linux platforms, you can make an executable for TorBot by using the install.sh script. You will need to give the script the correct permissions using chmod +x...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 15
    Gerapy

    Gerapy

    Distributed Crawler Management Framework Based on Scrapy

    Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js. Someone who has worked as a crawler with Python may use Scrapy. Scrapy is indeed a very powerful crawler framework. It has high crawling efficiency and good scalability. It is basically a necessary tool for developing crawlers using Python. If you use Scrapy as a crawler, then of course we can use our own host to crawl when crawling, but when the crawl is very large, we can’t...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 16

    Roundup Issue Tracker

    Simple-to-use/-install issue-tracking system: web, REST, email, CLI

    ...It's been designed with flexibility in mind - it's not just another bug tracker. Roundup ships with a *demo tracker* to play with - after you've unpacked the source, just run "python demo.py" and load up the URL it prints out! Get current development source from mercurial repo at: <https://sourceforge.net/p/roundup/code/ci/default/tarball?path=>
    Downloads: 17 This Week
    Last Update:
    See Project
  • 17
    Basic Website Studio (Tkinter Edition)

    Basic Website Studio (Tkinter Edition)

    Basic Website Studio (Tkinter Edition)

    A simple, lightweight code editor for basic web development, written in Python using the standard Tkinter GUI toolkit. This application is designed to run without any external Python libraries (like PySide6 or PyQt), making it ideal for easy distribution on Linux systems, such as in a .deb package.
    Downloads: 1 This Week
    Last Update:
    See Project
  • 18

    Apache Allura

    Forge software for hosting software projects

    Apache Allura's website and resources are all at https://allura.apache.org/ Allura was originally developed here, which is why this project exists.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 19
    eCxx

    eCxx

    A C++ library for AVR and NodeMCU

    NOTE: This project is marked with 'Status: Abandoned' on SourceForge because not enough time can be dedicated to this project. However it may still get sporadic commits to the repository. eCxx is a library for AVR and NodeMCU tailored for micro LED displays and lighting effects. eCxx is utilizing Makefile build system. Java and Python based applications/tools are also included to ease the development and debugging process using the host PC. On one side, eCxx supports the original...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 20
    AI-Agent-Host

    AI-Agent-Host

    The AI Agent Host is a module-based development environment.

    The AI Agent Host integrates several advanced technologies and offers a unique combination of features for the development of language model-driven applications. The AI Agent Host is a module-based environment designed to facilitate rapid experimentation and testing. It includes a docker-compose configuration with QuestDB, Grafana, Code-Server and Nginx. The AI Agent Host provides a seamless interface for managing and querying data, visualizing results, and coding in real-time. The AI...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 21
    Komodo Edit

    Komodo Edit

    Komodo Edit is a fast and free multi-language code editor

    Code faster with the IDE for today’s web languages – Python, PHP, Perl, Golang, Ruby, and more. Powerful editor with syntax highlighting, autocomplete, code refactoring, and more. Visual debugger and more to debug, inspect, and test your code. Support for Git, Mercurial, Subversion, CVS, Perforce and Bazaar. Tons of add-ons for customizing and extending features. Great tools for pair programming and collaboration. Python, PHP, Perl, Go, Ruby, Node.js, JavaScript, and more. Define your own...
    Downloads: 57 This Week
    Last Update:
    See Project
  • 22
    OpsManage

    OpsManage

    Automated operation and maintenance platform

    Automated operation and maintenance platform: code and application deployment CI/CD, asset management CMDB, scheduled task management platform, SQL audit | rollback, task scheduling, on-site WIKI. A code deployment, application deployment, scheduled tasks, and equipment asset management platform. Welcome to star or fork my open source project. If you need to quote the project code in your own project, please declare the agreement and copyright information in the project. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 23
    HASS Configurator

    HASS Configurator

    Configuration UI for Home Assistant

    The HASS Configurator is a small web app (you access it via a web browser) that provides a filesystem browser and text-editor to modify files on the machine the configurator is running on. It has been created to allow easy configuration of Home Assistant. It is powered by Ace editor, which supports syntax highlighting for various code/markup languages. YAML files (the default language for Home Assistant configuration files) will be automatically checked for syntax errors while editing. The...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 24
    PySchool

    PySchool

    Installable / Portable Python Distribution for Everyone.

    PySchool is a free and open-source Python distribution intended primarily for students who learn Python and data analysis, but it can also used by scientists, engineering, and data scientists. It includes more than 150 Python packages (full edition) including numpy, pandas, scipy, sympy, keras, scikit-learn, matplotlib, seaborn, beautifulsoup4...
    Leader badge
    Downloads: 826 This Week
    Last Update:
    See Project
  • 25
    Code Cookbook

    Code Cookbook

    Reusable code patterns which you can use as reference or copy

    Reusable code patterns which you can use as reference or copy to your project. Achieve small or large tasks using recipes that contain steps, scripts, and config files.
    Downloads: 0 This Week
    Last Update:
    See Project