Search Results for "python web crawler" - Page 14
Sort By:
Showing 3076 open source projects for "python web crawler"
View related business solutions-
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
1
spaCy models
Models for the spaCy Natural Language Processing (NLP) library
spaCy is designed to help you do real work, to build real products, or gather real insights. The library respects your time, and tries to avoid wasting it. It's easy to install, and its API is simple and productive. spaCy excels at large-scale information extraction tasks. It's written from the ground up in carefully memory-managed Cython. If your application needs to process entire web dumps, spaCy is the library you want to be using. Since its release in 2015, spaCy has become an industry... -
2
Xianyu Intelligent Monitor Bot
AI tool for real-time monitoring and analysis of Goofish listings
ai-goofish-monitor is an open source automation tool designed to monitor listings on the Goofish second-hand marketplace and analyze them using artificial intelligence. It combines browser automation with AI-based analysis to automatically search, collect, and evaluate newly posted items that match a user’s purchase criteria. It uses Playwright to simulate real user interactions with the marketplace, allowing the system to retrieve product data and track updates in near real time.... -
3
Khoj
An AI personal assistant for your digital brain
Get more done with your open-source AI personal assistant. Khoj is a desktop application to search and chat with your notes, documents, and images. It is an offline-first, open-source AI personal assistant that is accessible from Emacs, Obsidian or your Web browser. Khoj is a thinking tool that is transparent, fun, and easy to engage with. You can build faster and better by using Khoj to search and reason across all your data sources. Khoj learns from your notes and documents to function as... -
4
VulnClaw
Based on AI Agent + MCP toolchain + penetration Skill orchestration
VulnClaw is an AI-powered penetration testing agent that turns natural language security goals into structured testing workflows. It combines LLM agents, MCP toolchains, penetration testing skills, and command-line automation to support authorized security assessments. The project can guide information gathering, vulnerability discovery, validation, and report generation while keeping the workflow organized through sessions and tools. Its newer architecture uses a goal-driven solving engine... -
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
5
Edit Banana
Edit Banana: A framework for converting statistical figures
Edit Banana is an innovative web application designed to simplify image editing by merging intuitive user interfaces with powerful generative AI capabilities, enabling users to quickly enhance, manipulate, or transform photos without needing advanced design skills. It provides a smooth, browser-based experience where users can upload images, make precise edits such as background removal or inpainting, and apply stylistic transformations or corrections through AI prompts. The tool focuses on... -
6
Cal.diy
Scheduling infrastructure for absolutely everyone
The Cal.diy project is a fully open-source, self-hosted scheduling platform that provides an alternative to proprietary booking tools while giving users complete control over their infrastructure. It is a community-driven fork of Cal.com that removes all enterprise and commercial features, ensuring that the entire codebase remains MIT-licensed and accessible. The platform enables users to manage bookings, availability, and integrations without relying on external services or subscription... -
7
LoLLMs WEBUI
Local AI WebUI for running and managing large language models offlineA
lollms-webui is a locally hosted web interface designed to run and manage large language models without relying on external services. It provides users with a centralized environment to interact with multiple AI models, making it suitable for experimentation, development, and personal use. lollms-webui emphasizes offline capability, allowing users to maintain privacy and control over their data while still accessing advanced AI features. It integrates model management tools that help users... -
8
SenseVoice
Multilingual speech recognition and audio understanding model
SenseVoice is a speech foundation model designed to perform multiple voice understanding tasks from audio input. It provides capabilities such as automatic speech recognition, spoken language identification, speech emotion recognition, and audio event detection within a single system. SenseVoice is trained on more than 400,000 hours of speech data and supports over 50 languages for multilingual recognition tasks. It is built to achieve high transcription accuracy while maintaining efficient... -
9
Tookie-OSINT
Username OSINT tool for discovering accounts across many websites
Tookie-OSINT is an open source intelligence tool designed to help security researchers, ethical hackers, and investigators discover online accounts associated with a specific username. It automates the process of searching for usernames across multiple websites, making it easier to identify a person's presence on different platforms. By entering a target username, Tookie-OSINT scans a list of supported sites and checks whether the username exists on those platforms. This approach removes the... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
claude-code-transcripts
Tools for publishing transcripts for Claude Code sessions
claude-code-transcripts is a command-line utility that takes session files exported from Claude Code (in JSON or JSONL format) and turns them into clean, navigable HTML transcripts that can be viewed in any modern web browser. It is designed to make the often dense and verbose outputs from AI coding sessions easier to read, share, and archive by breaking conversations into paginated, annotated pages with navigable timelines of prompts and responses. Users can run this tool locally or fetch... -
11
TimeTracker
Open-source and free to self-host
TimeTracker by DRYTRIX is a comprehensive self-hosted time tracking and project management application that helps individuals, teams, and small businesses take control of their productivity and billing. Built on a modern web stack (Python/Flask backend with a responsive frontend), TimeTracker enables users to track hours, manage projects and clients, visualize data with interactive charts, and even generate professional invoices directly from tracked time. It embraces privacy and flexibility by running entirely on your own servers, ensuring sensitive business data stays under your control, and supports real-time timer persistence via WebSockets so timers continue even if the browser is closed. ... -
12
JS Analyzer
Burp Suite extension for JavaScript static analysis
JS Analyzer is a powerful static analysis tool implemented as a Burp Suite extension that helps security researchers and web developers automatically uncover important artifacts in JavaScript files during web application testing. It parses JavaScript responses intercepted by Burp Suite and intelligently extracts API endpoints, full URLs (including cloud storage links), secrets like API keys or tokens, and email addresses while filtering out noise from irrelevant code patterns. ... -
13
Mosec
A high-performance ML model serving framework, offers dynamic batching
Mosec is a high-performance and flexible model-serving framework for building ML model-enabled backend and microservices. It bridges the gap between any machine learning models you just trained and the efficient online service API. -
14
INTERCEPT
Unites the best signal intelligence tools
iNTERCEPT is a web-based interface that brings multiple software-defined radio and signal-intelligence style tools under one consistent dashboard, making complex workflows more approachable. Rather than requiring you to learn a different UI and setup process for each underlying utility, it provides a single place to start modes, view results, and monitor activity from a browser. The project’s goal is accessibility: lowering the skill and setup barrier so learners and authorized testers can... -
15
django CMS
Easy-to-use and developer-friendly enterprise CMS powered by Django
Create modern websites that content editors love. django CMS was originally conceived by web developers frustrated with the technical and security limitations of other systems. Its lightweight core makes it easy to integrate with other software and put to use immediately, while its ease of use makes it the go-to choice for content managers, content editors and website admins. Developers can integrate other existing Django applications rapidly, or build brand new compatible apps that take... -
16
Flowly AI
Flowly is 100x faster than OpenClaw
Flowly is an open-source personal AI assistant that runs locally on your machine and connects to multiple communication platforms like Telegram, WhatsApp, Discord, and Slack. It acts as a centralized AI system that can perform tasks such as web browsing, file management, command execution, scheduling, and more—all while keeping your data private. Designed for flexibility, Flowly supports multiple AI providers and models through LiteLLM, allowing users to customize how their assistant... -
17
Windows Copilot API
Reverse engineered Windows Copilot into an OpenAI-compatible API
Windows Copilot API is an unofficial Python project that exposes the consumer Microsoft Copilot web experience through a programmable interface. It can be used directly as a Python library or launched as a local server compatible with the OpenAI API format. The client supports streamed responses, multi-turn conversations, and conversation identifiers. Browser-based sign-in stores a reusable session and refreshes it automatically for later requests. -
18
BBOT
The recursive internet scanner for hackers
BBOT is an advanced open-source reconnaissance automation framework designed to streamline large-scale OSINT and attack surface discovery workflows. It operates as a modular and recursive scanning tool that can enumerate subdomains, perform port scans, gather metadata, and collect web intelligence through a unified command-line interface. The project emphasizes extensibility, allowing users to create or integrate custom modules that expand the scope of reconnaissance tasks without modifying... -
19
Flask App Builder
Simple and rapid application development framework
Simple and rapid application development framework, built on top of Flask. includes detailed security, auto CRUD generation for your models, google charts and much more. Automatic permissions lookup, based on exposed methods. Inserts on the Database all the detailed permissions possible on your application. Public (no authentication needed) and Private permissions. Role-based permissions. Authentication support for OpenID, Database and LDAP. Support for self-user registration. Automatic,... -
20
Flagsmith
Open source feature flagging and remote config service
Release features with confidence; manage feature flags across web, mobile, and server-side applications. Use our hosted API, deploy to your own private cloud, or run on-premises. Flagsmith provides an all-in-one platform for developing, implementing, and managing your feature flags. Whether you are moving off an in-house solution or using toggles for the first time, you will be amazed by the power and efficiency gained by using Flagsmith. Flagsmith makes it easy to create and manage feature... -
21
HTTPie CLI
Modern, user-friendly command-line HTTP client for the API era
HTTPie (pronounced aitch-tee-tee-pie) is a command-line HTTP client. Its goal is to make CLI interaction with web services as human-friendly as possible. HTTPie is designed for testing, debugging, and generally interacting with APIs & HTTP servers. The HTTP & HTTPS commands allow for creating and sending arbitrary HTTP requests. They use simple and natural syntax and provide formatted and colorized output. -
22
SWE-agent
SWE-agent takes a GitHub issue and tries to automatically fix it
SWE-agent turns LMs (e.g. GPT-4) into software engineering agents that can resolve issues in real GitHub repositories. On the SWE-bench, the SWE-agent resolves 12.47% of issues, achieving state-of-the-art performance on the full test set. We accomplish our results by designing simple LM-centric commands and feedback formats to make it easier for the LM to browse the repository, and view, edit, and execute code files. We call this an Agent-Computer Interface (ACI). -
23
MolmoWeb
Open multimodal web agent built by Ai2
MolmoWeb is an open-source multimodal web agent designed to autonomously navigate and interact with web browsers using vision-language models, representing a significant step toward fully agentic AI systems that can operate in real-world digital environments. The system takes natural language instructions and translates them into sequences of browser actions such as clicking, typing, scrolling, and navigating, effectively performing tasks on behalf of the user. Unlike traditional automation... -
24
CKAN
CKAN is an open-source DMS for powering data hubs
CKAN is the world’s leading open-source data portal platform. CKAN makes it easy to publish, share and work with data. It's a data management system that provides a powerful platform for cataloging, storing and accessing datasets with a rich front-end, full API (for both data and catalog), visualization tools and more.CKAN is used by national and regional government organizations throughout the European Union, the Americas, Asia, and Oceania to power a variety of official and community data... -
25
WhatsApp MCP Server
WhatsApp MCP server enabling AI access to chats and messaging
...It acts as a bridge between WhatsApp and large language models, allowing controlled access to messages, chats, and contacts. whatsapp-mcp is composed of two main components: a Go-based bridge that connects to the WhatsApp Web API and stores data locally, and a Python-based MCP server that exposes tools for AI interaction. All message data is stored in a local SQLite database and is only accessed when explicitly requested through defined tools, giving users control over how their data is used. It supports both sending and receiving messages, including various media types such as images, audio, videos, and documents. ...
