Search Results for "web crawler source code" - Page 2

Showing 374 open source projects for "web crawler source code"

View related business solutions
  • MongoDB Atlas runs apps anywhere Icon
    MongoDB Atlas runs apps anywhere

    Deploy in 115+ regions with the modern database for every enterprise.

    MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
    Start Free
  • $300 in Free Credit Towards Top Cloud Services Icon
    $300 in Free Credit Towards Top Cloud Services

    Build VMs, containers, AI, databases, storage—all in one place.

    Start your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
    Get Started
  • 1
    Locust

    Locust

    Scalable open source load testing tool

    Locust is an open source user load testing tool written in Python. The idea behind Locust is to swarm your web site or other systems with attacks from simulated users during a test, with each user behavior defined by you using Python code. This swarming process is then monitored from a web UI in real-time, and will help identify any bottlenecks in your code before real users can come in.
    Downloads: 11 This Week
    Last Update:
    See Project
  • 2
    Frappe

    Frappe

    Low code web framework for real world applications

    Frappe is a full-stack, low-code web framework written in Python and JavaScript, used to build scalable and modular enterprise applications. It powers ERPNext and includes tools for REST APIs, user management, document modeling, workflows, and real-time updates. Frappe uses a "model-view-controller" approach with its own ORM and frontend system, enabling rapid development without sacrificing control or performance.
    Downloads: 9 This Week
    Last Update:
    See Project
  • 3
    news-please

    news-please

    Python tool for crawling and extracting structured data from news site

    news-please is an open source news crawler and information extraction tool designed to collect and structure articles from online news websites. It provides an integrated pipeline that crawls news sites, retrieves article pages, and extracts structured information such as headlines, authors, publication dates, and article text. news-please can recursively follow internal links and read RSS feeds to gather both recent and archived articles from a news outlet when given only the root URL of a site. ...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 4
    FastAPI Python

    FastAPI Python

    FastAPI framework, high performance, easy to learn, fast to code

    FastAPI framework, high performance, easy to learn, fast to code, ready for production. FastAPI is a modern, fast (high-performance), web framework for building APIs with Python based on standard Python type hints.
    Downloads: 11 This Week
    Last Update:
    See Project
  • Full-stack observability with actually useful AI | Grafana Cloud Icon
    Full-stack observability with actually useful AI | Grafana Cloud

    Our generous forever free tier includes the full platform, including the AI Assistant, for 3 users with 10k metrics, 50GB logs, and 50GB traces.

    Built on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
    Create free account
  • 5
    Pyodide

    Pyodide

    Pyodide is a Python distribution for the browser and Node.js

    Pyodide brings the Python runtime to the browser by compiling Python and its scientific libraries to WebAssembly. It allows developers to run Python code directly in web browsers without a server, supporting packages like NumPy, Pandas, and Matplotlib. Pyodide opens up new possibilities for interactive data analysis, scientific computing, and educational tools in web environments, all while integrating seamlessly with JavaScript.
    Downloads: 18 This Week
    Last Update:
    See Project
  • 6
    LaVague

    LaVague

    Framework for building AI agents that automate complex web tasks

    LaVague is an open source framework designed to help developers build AI-powered web agents capable of automating tasks across websites and web applications. It implements the concept of a Large Action Model framework, allowing agents to interpret a user-provided objective and translate it into a sequence of actions performed in a browser. These agents can navigate web pages, retrieve information, fill out forms, and execute multi-step workflows automatically. ...
    Downloads: 7 This Week
    Last Update:
    See Project
  • 7
    diskover-community

    diskover-community

    Open source file indexing & storage analytics powered by Elasticsearch

    Diskover Community Edition is an open source file system indexing and storage analytics platform designed to help organizations understand and manage large volumes of file data. It crawls file systems and indexes metadata using Elasticsearch, enabling fast search, analysis, and organization of files stored across different storage systems. It allows administrators and users to explore file structures, monitor storage usage, and gain insights into how data is distributed across...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 8
    NiceGUI

    NiceGUI

    Create web-based user interfaces with Python

    NiceGUI is a Python-based UI framework that enables developers to create interactive web applications using only Python code. It abstracts away the complexities of HTML, CSS, and JavaScript, allowing for rapid development of web interfaces directly from Python scripts. NiceGUI is suitable for building dashboards, control panels, and other web-based tools, especially in contexts like robotics and data visualization.
    Downloads: 9 This Week
    Last Update:
    See Project
  • 9
    BlackSheep

    BlackSheep

    Fast ASGI web framework for Python

    BlackSheep is an asynchronous web framework to build event-based web applications with Python. A rich code API, based on dependency injection and inspired by Flask and ASP.NET Core. A typing-friendly codebase, which enables a comfortable development experience thanks to hints when coding with IDEs. Built-in generation of OpenAPI Documentation, supporting version 3, YAML, and JSON. A cross-platform framework, using the most modern versions of Python. BlackSheep supports automatic binding of...
    Downloads: 9 This Week
    Last Update:
    See Project
  • Gemini 3 and 200+ AI Models on One Platform Icon
    Gemini 3 and 200+ AI Models on One Platform

    Access Google's best plus Claude, Llama, and Gemma. Fine-tune and deploy from one console.

    Build generative AI apps with Vertex AI. Switch between models without switching platforms.
    Start Free
  • 10
    Unsloth Studio

    Unsloth Studio

    Unified web UI for training and running open models locally

    Unsloth Studio is a web-based interface for running and training AI models locally with a unified and user-friendly experience. It allows users to work with a wide range of models for text, audio, vision, embeddings, and more without relying heavily on cloud infrastructure. Built on top of the Unsloth framework, it focuses on high-performance training with reduced VRAM usage and faster speeds compared to traditional methods. The platform supports fine-tuning, pretraining, and reinforcement...
    Downloads: 18 This Week
    Last Update:
    See Project
  • 11
    Dash

    Dash

    Build beautiful web-based analytic apps, no JavaScript required

    Dash is a Python framework for building beautiful analytical web applications without any JavaScript. Built on top of Plotly.js, React and Flask, Dash easily achieves what an entire team of designers and engineers normally would. It ties modern UI controls and displays such as dropdown menus, sliders and graphs directly to your analytical Python code, and creates exceptional, interactive analytics apps. Dash apps are very lightweight, requiring only a limited number of lines of Python or...
    Downloads: 3 This Week
    Last Update:
    See Project
  • 12
    WAFW00F

    WAFW00F

    WAFW00F allows one to identify and fingerprint Web App Firewall

    ...For further details, check out the source code on our main repository.
    Downloads: 8 This Week
    Last Update:
    See Project
  • 13
    GPT All Star

    GPT All Star

    AI-powered code generation tool for scratch development of web apps

    AI-powered code generation tool for scratch development of web applications with a team collaboration of autonomous AI agents. This is a research project, and its primary value is to explore the possibility of autonomous AI agents.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 14
    Mercury

    Mercury

    Convert Python notebook to web app and share with non-technical users

    Turn Python notebooks to web applications with open-source Mercury framework. Hide code and add interactive widgets. Non-technical users can tweak widgets and execute notebook with new parameters. The core of Mercury is Open Source under AGPLv3. We provide Mercury Pro with additional features, dedicated support and friendly commercial license. Mercury is a perfect tool to convert Python notebook to interactive web application and share with non-programmers. ...
    Downloads: 6 This Week
    Last Update:
    See Project
  • 15
    Full Stack FastAPI and PostgreSQL

    Full Stack FastAPI and PostgreSQL

    Full stack, modern web application generator

    Generate a backend and frontend stack using Python, including interactive API documentation. Production ready Python web server using Uvicorn and Gunicorn. Very high performance, on par with NodeJS and Go (thanks to Starlette and Pydantic). Great editor support. Completion everywhere. Less time debugging. Designed to be easy to use and learn. Less time reading docs. Minimize code duplication. Multiple features from each parameter declaration. Get production-ready code. With automatic...
    Downloads: 5 This Week
    Last Update:
    See Project
  • 16
    FastHTML

    FastHTML

    The fastest way to create an HTML app

    Built on solid web foundations, not the latest fads - with FastHTML you can get started on anything from simple dashboards to scalable web applications in minutes.
    Downloads: 5 This Week
    Last Update:
    See Project
  • 17
    I hate money

    I hate money

    A simple shared budget manager web application

    I hate money is a web application made to ease shared budget management. It keeps track of who bought what, when, and for whom; and helps to settle the bills. I hate money is written in python, using the flask framework. It’s developed with ease of use in mind and is trying to keep things simple. Hope you (will) like it! The code is distributed under a BSD beerware derivative: if you meet the people in person and you want to pay them a craft beer, you are highly encouraged to do so.
    Downloads: 9 This Week
    Last Update:
    See Project
  • 18
    Tarsier

    Tarsier

    Vision utilities for web interaction agents

    At Reworkd, we iterated on all these problems across tens of thousands of real web tasks to build a powerful perception system for web agents... Tarsier! In the video below, we use Tarsier to provide webpage perception for a minimalistic GPT-4 LangChain web agent. Tarsier visually tags interactable elements on a page via brackets + an ID e.g. [23]. In doing this, we provide a mapping between elements and IDs for an LLM to take actions upon (e.g. CLICK [23]). We define interactable elements...
    Downloads: 6 This Week
    Last Update:
    See Project
  • 19
    /last30days

    /last30days

    Claude Code skill that researches any topic across Reddit + X

    /last30days is a specialized Claude Code skill designed to research current trends and practices across Reddit, X, and the wider web from the last 30 days, synthesize that data, and produce copy-paste-ready prompts or summaries that reflect what the community is actually talking about now. Rather than returning generic model responses, it intelligently analyzes social media and community discussions to identify what’s genuinely trending or working in practice across topics ranging from...
    Downloads: 14 This Week
    Last Update:
    See Project
  • 20
    StarVector

    StarVector

    StarVector is a foundation model for SVG generation

    StarVector is a multimodal foundation model designed for generating Scalable Vector Graphics (SVG) from images or textual descriptions. The system treats vector graphics creation as a code generation problem, producing SVG code that can render detailed vector images. Its architecture combines computer vision techniques with language modeling capabilities so it can understand visual inputs and textual prompts simultaneously. The model converts raster images or text instructions into...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 21
    Ungoogled Chromium

    Ungoogled Chromium

    A lightweight approach to removing Google web service dependency

    In descending order of significance (i.e. most important objective first), ungoogled-chromium is Google Chromium, sans dependency on Google web services, ungoogled-chromium retains the default Chromium experience as closely as possible. Unlike other Chromium forks that have their own visions of a web browser, ungoogled-chromium is essentially a drop-in replacement for Chromium. ungoogled-chromium features tweaks to enhance privacy, control, and transparency. However, almost all of these...
    Downloads: 22 This Week
    Last Update:
    See Project
  • 22
    Stable Diffusion Version 2

    Stable Diffusion Version 2

    High-Resolution Image Synthesis with Latent Diffusion Models

    Stable Diffusion (the stablediffusion repo by Stability-AI) is an open-source implementation and reference codebase for high-resolution latent diffusion image models that power many text-to-image systems. The repository provides code for training and running Stable Diffusion-style models, instructions for installing dependencies (with notes about performance libraries like xformers), and guidance on hardware/driver requirements for efficient GPU inference and training. ...
    Downloads: 21 This Week
    Last Update:
    See Project
  • 23
    Aider

    Aider

    Aider is AI pair programming in your terminal

    Aider is an AI pair programming tool that runs directly in your terminal, helping developers build new projects or extend existing codebases faster and more confidently. It works alongside you like a coding partner, using powerful large language models to understand your code and implement precise changes. Aider creates a structured map of your entire repository, allowing it to handle large and complex projects effectively. It supports over 100 programming languages, making it flexible for...
    Downloads: 11 This Week
    Last Update:
    See Project
  • 24
    Taipy

    Taipy

    Turns Data and AI algorithms into production-ready web applications

    From simple pilots to production-ready web applications in no time. No more compromise on performance, customization, and scalability. Taipy enhances performance with caching control of graphical events, optimizing rendering by selectively updating graphical components only upon interaction. Effortlessly manage massive datasets with Taipy's built-in decimator for charts, intelligently reducing the number of data points to save time and memory without losing the essence of your data's shape....
    Downloads: 11 This Week
    Last Update:
    See Project
  • 25
    Streamlit

    Streamlit

    The fastest way to build data apps in Python

    A faster way to build and share data apps. Streamlit turns data scripts into shareable web apps in minutes. All in pure Python. No front‑end experience is required. Build an app in a few lines of code with our magically simple API. Then see it automatically update as you iteratively save the source file. Adding a widget is the same as declaring a variable. No need to write a backend, define routes, handle HTTP requests, connect a frontend, write HTML, CSS, JavaScript, etc. ...
    Downloads: 46 This Week
    Last Update:
    See Project
MongoDB Logo MongoDB