Search Results for "python web crawler"
Sort By:
Showing 11 open source projects for "python web crawler"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
1
Spider
High-performance Rust web crawler and scraper for large-scale data
Spider is a high-performance web crawler and web scraping library written in Rust that enables developers to crawl and index websites efficiently. It focuses on speed, concurrency, and reliability by using asynchronous and multi-threaded processing to handle large volumes of web pages. It can rapidly crawl websites to collect links, retrieve page content, and extract structured information from HTML documents. -
2
rnet
Python HTTP client with TLS and HTTP/2 fingerprint emulation support
rnet is an ergonomic and modular Python HTTP client designed for developers who need advanced control over network requests and protocol behavior. It provides a flexible API for making HTTP requests while supporting both asynchronous and blocking workflows, allowing it to integrate easily into different Python applications and runtimes. rnet focuses on low-level protocol customization, giving users fine-grained control over TLS and HTTP/2 configuration in order to emulate specific browser... -
3
MusicGPT
Generate music based on natural language prompts using LLMs
...The application currently integrates with models such as MusicGen and is designed to support additional models transparently in the future. In addition to a command-line interface, the project includes a web-based interface that enables conversational interaction with the AI model. -
4
MooseStack
The developer framework for building analytical backends
MooseStack is an opinionated starter stack that assembles a modern web application foundation—project structure, build tooling, and deployment scripts—so teams can get from “blank repo” to a working product quickly. It provides a coherent layout for server and client code, standardizes environment configuration, and includes scripts to run the app locally with the same conventions you’ll use in staging or production. The stack favors convention over configuration: common decisions around... -
Build Securely on Azure with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
5
Every Code
Local AI coding agent CLI with multi-agent orchestration tools
Every Code (often referred to simply as Code) is a fast, local AI-powered coding agent designed to run directly in the terminal environment. It is a community-driven fork of the Codex CLI, with a strong emphasis on improving real-world developer ergonomics and workflows. Every Code enhances the traditional coding assistant model by introducing multi-agent orchestration, allowing multiple AI agents to collaborate, compare solutions, and refine outputs in parallel. It supports integration with... -
6
Shinkai: Local AI Agents
Shinkai allows you to create advanced AI (local) agents effortlessly
Shinkai is a free, open-source AI platform that lets anyone create powerful AI agents without coding. These agents can collaborate with each other, handle complex tasks, and operate in decentralized crypto environments. Key Features: - No-Code Agent Creation - Build specialized agents (trading bots, sentiment trackers, etc.) with simple descriptions - Multi-Agent Collaboration - Agents work together to solve complex problems - Crypto Integration - Built-in support for decentralized... -
7
llm-chain
Rust crate for building chains in large language models
...Incorporates support for Stanford's Alpaca models, expanding the range of available language models for advanced AI applications. Enhance your AI agents' capabilities by giving them access to various tools, such as running Bash commands, executing Python scripts, or performing web searches. -
8
authoscope
Scriptable network authentication cracker (formerly `badtouch`)
authoscope is a scriptable network authentication cracker. While the space for common service bruteforce is already very well saturated, you may still end up writing your own python scripts when testing credentials for web applications. The scope of authoscope is specifically cracking custom services. This is done by writing scripts that are loaded into a lua runtime. Those scripts represent a single service and provide a verify(user, password) function that returns either true or false. Concurrency, progress indication, and reporting is magically provided by the authoscope runtime. -
9
Ansible Examples
A few starter examples of ansible playbooks, to show features
This repository collects practical, real-world examples of using Ansible to automate infrastructure, deployments, and configurations. Each directory demonstrates a specific use case—ranging from setting up web servers, load balancers, and databases to orchestrating multi-tier applications in cloud environments. The examples highlight common Ansible practices such as organizing inventories, writing reusable playbooks, using roles, and handling variables and templates. They’re designed to be... -
Demo Series - Small Business Backup By VeeamWatch this on-demand demo series and learn how to protect your Microsoft 365 data with clear, simple, actionable steps that are easy to implement for businesses of all sizes.
-
10
Decentralized Internet
SDK for building decentralized web and distributed computing projects
This project was created in order to support a new internet. One that is more open, free, and censorship-resistant in comparison to the old internet. An internet that eventually wouldn't need to rely on telecom towers, an outdated grid, or all these other "old school" forms of tech. We believe P2P compatibility is an important part of the future of the net. Grid Computing also plays a role in having a better means of transferring information in a speedy, more cost-efficient and reliable manner. -
11
TxPB
TemporalX gRPC and Documentation Resources
TxPB is the main source of information for using TemporalX, and includes protocol buffer definitions, generated code, and documentation. If you are looking for the protocol buffer definitions, check the pb folder. If you want bindings in other languages, feel free to open a github issue. Alternatively you can generate language bindings independently as long as the languages you are using support the appropriate protocol buffer and gRPC versions.
- Previous
- You're on page 1
- Next
