Search Results for "web crawler source code" - Page 7
Sort By:
Showing 374 open source projects for "web crawler source code"
View related business solutions-
Earn up to 16% annual interest with Nexo.Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
1
Roundup Issue Tracker
Simple-to-use/-install issue-tracking system: web, REST, email, CLI
...It's been designed with flexibility in mind - it's not just another bug tracker. Roundup ships with a *demo tracker* to play with - after you've unpacked the source, just run "python demo.py" and load up the URL it prints out! Get current development source from mercurial repo at: <https://sourceforge.net/p/roundup/code/ci/default/tarball?path=> -
2
Apache Allura
Forge software for hosting software projects
Apache Allura's website and resources are all at https://allura.apache.org/ Allura was originally developed here, which is why this project exists. -
3
3tdb
A web based MariaDB client.
LINUX ONLY - requires mod_python Python MariaDB client. ** 12.2-0 ** 2026-01-10 mod_python was removed from debian trixie (13). As of this writing, mod_python can be compiled from git. 3tp3 uses minimal javascript, which you can examine in the js folder (one file, 36 lines). 3tp3 does not load any external (internet or other) code. No external fonts, helper code or advertisement. It can be installed and run on localhost. -
4
Shinkai: Local AI Agents
Shinkai allows you to create advanced AI (local) agents effortlessly
Shinkai is a free, open-source AI platform that lets anyone create powerful AI agents without coding. These agents can collaborate with each other, handle complex tasks, and operate in decentralized crypto environments. Key Features: - No-Code Agent Creation - Build specialized agents (trading bots, sentiment trackers, etc.) with simple descriptions - Multi-Agent Collaboration - Agents work together to solve complex problems - Crypto Integration - Built-in support for decentralized... -
8 Monitoring Tools in One APM. Install in 5 Minutes.AppSignal works out of the box for Ruby, Elixir, Node.js, Python, and more. 30-day free trial, no credit card required.
-
5
Automated Bluesky Poster
Free, easy, infinite Bluesky posting automation.
...All programs are Free and Open Source, but please donate. Source Code and installation here, directly on Github: https://github.com/ReactorcoreGames/BlueskyAutoPoster (Note: It uses Github Actions to run the program) -
6
Automated Twitter Poster
Free, easy, infinite Twitter posting automation.
...All programs are Free and Open Source, but please donate. Source Code and installation here, directly on Github: https://github.com/ReactorcoreGames/TwitterAutoPoster (Note: It uses Github Actions to run the program) -
7
How to get Bot Lobbies
Get easy bot lobbies in any game with our bot lobbies tool.
Test it on the web app: https://slothytech.com/ezlobbies/ EzLobbies is the open-source tool developed by SlothyTech that helps gamers get easier, bot-filled lobbies in the world’s top games. Instantly generate custom OpenVPN config files that route only your game’s matchmaking traffic through the VPN server of your choice—no complicated setup and no speed loss for browsing or streaming. Supported titles include Fortnite, Call of Duty, Warzone, Apex Legends, Overwatch, Valorant, PUBG, and... -
8
Links Into Social Media Posts
Provide a list of links, get back a CSV of social media post drafts
...It did all that magic automatically and without AI! (Source code is included with download in 'src' folder!") -
9
Mara Pipelines
A lightweight opinionated ETL framework, halfway between plain scripts
This package contains a lightweight data transformation framework with a focus on transparency and complexity reduction. Data integration pipelines as code: pipelines, tasks and commands are created using declarative Python code. PostgreSQL as a data processing engine. Extensive web ui. The web browser as the main tool for inspecting, running and debugging pipelines. GNU make semantics. Nodes depend on the completion of upstream nodes. No data dependencies or data flows. No in-app data... -
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
10
WebArena
Code repo for "WebArena to build Autonomous Agents
WebArena is a realistic web environment designed for building and testing autonomous agents, providing a platform for developing web-based AI agents. -
11
TorBot
Dark Web OSINT Tool
Contributions to this project are always welcome. To add a new feature fork the dev branch and give a pull request when your new feature is tested and complete. If its a new module, it should be put inside the modules directory. The branch name should be your new feature name in the format <Feature_featurename_version(optional)>. On Linux platforms, you can make an executable for TorBot by using the install.sh script. You will need to give the script the correct permissions using chmod +x... -
12
eCxx
A C++ library for AVR and NodeMCU
NOTE: This project is marked with 'Status: Abandoned' on SourceForge because not enough time can be dedicated to this project. However it may still get sporadic commits to the repository. eCxx is a library for AVR and NodeMCU tailored for micro LED displays and lighting effects. eCxx is utilizing Makefile build system. Java and Python based applications/tools are also included to ease the development and debugging process using the host PC. On one side, eCxx supports the original... -
13
Crawlab
Distributed web crawler admin platform for spiders management
Golang-based distributed web crawler management platform, supporting various languages including Python, NodeJS, Go, Java, PHP and various web crawler frameworks including Scrapy, Puppeteer, Selenium. Please use docker-compose to one-click to start up. By doing so, you don't even have to configure MongoDB database. The frontend app interacts with the master node, which communicates with other components such as MongoDB, SeaweedFS and worker nodes. Master node and worker nodes communicate... -
14
Gerapy
Distributed Crawler Management Framework Based on Scrapy
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js. Someone who has worked as a crawler with Python may use Scrapy. Scrapy is indeed a very powerful crawler framework. It has high crawling efficiency and good scalability. It is basically a necessary tool for developing crawlers using Python. If you use Scrapy as a crawler, then of course we can use our own host to crawl when crawling, but when the crawl is very large, we can’t... -
15
dirhunt
Web crawler that finds hidden web directories without brute force
Dirhunt is an open source security tool designed to discover web directories and analyze website structures without relying on brute-force techniques. Instead of sending large numbers of guess-based requests, it operates as a specialized crawler that intelligently explores websites to identify accessible or hidden directories. Dirhunt can detect directories that expose “Index Of” listings, which may reveal files and other resources that were not intended to be publicly visible. ... -
16
Basaran
Basaran, an open-source alternative to the OpenAI text completion API
Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models. The open source community will eventually witness the Stable Diffusion moment for large language models (LLMs), and Basaran allows you to replace OpenAI's service with the latest open-source model to power your application without modifying a single line of code. -
17
barcraft
A simple QrCode / barcode generator in python
A simple QrCode / barcode generator that you can also use from this website version : https://secret-guest.github.io/barcraft/ Interface made with pyQt5, made with a MSI installer with Inno setup -
18
Bard
Python SDK/API for reverse engineered Google Bard
The Bard repository from acheong08 is an unofficial client library and set of tools designed to interact programmatically with Google Bard, Google’s conversational AI service, allowing developers to automate queries, integrate Bard into their own applications, and experiment with scripting around Bard-style prompts without relying on a formal, official API. Because Google doesn’t provide a standard public API for Bard, this project works by reverse-engineering the web interactions that power... -
19
Audio Webui
A webui for different audio related Neural Networks
Audio Webui is a Gradio-based web user interface that unifies a wide range of audio-related neural networks under a single, accessible front end. It is designed as an “all-in-one” environment where users can experiment with text-to-speech, voice cloning, generative music, and other neural audio models without writing boilerplate code. The project supports multiple back-end models and toolchains (such as Bark, RVC, AudioLDM, Audiocraft, and other text-to-audio or voice-cloning tools),... -
20
RPA for Python
Python package for doing RPA
Python package for doing RPA. RPA for Python's simple and powerful API makes robotic process automation fun! You can use it to quickly automate away repetitive time-consuming tasks on websites, desktop applications, or the command line. See sample Python script, the RPA Challenge solution, and RedMart groceries example. To send a Telegram app notification, simply look up @rpapybot to allow receiving messages. To automate Chrome browser invisibly, use headless mode. To run 10X faster instead... -
21
AI-Agent-Host
The AI Agent Host is a module-based development environment.
The AI Agent Host integrates several advanced technologies and offers a unique combination of features for the development of language model-driven applications. The AI Agent Host is a module-based environment designed to facilitate rapid experimentation and testing. It includes a docker-compose configuration with QuestDB, Grafana, Code-Server and Nginx. The AI Agent Host provides a seamless interface for managing and querying data, visualizing results, and coding in real-time. The AI... -
22
Bot on Anything
Large model-based chatbot builder that can quickly integrate AI models
Bot on Anything is a versatile open-source AI chatbot builder that lets developers connect large language models such as ChatGPT, Claude, and Gemini to virtually any messaging platform, website, or interface with minimal configuration. At its heart, the project abstracts away the glue logic between AI model APIs and disparate application “channels,” enabling the same bot logic to run in Slack, Telegram, Gmail, enterprise tools, web UIs, or command-line terminals. -
23
Configuration Backup (ConfiBack)
Project for backing up network device configuration
Using this project you can make backup and track changes of configuration of network devices like switches, routers, etc. -
24
artikelschreiber
Frontend and Backend Code for ArtikelSchreiber.com and UNAIQUE.NET
Frontend and Backend Code for ArtikelSchreiber.com and UNAIQUE.NET Text Generator deutsch - Dein KI Text Generator kostenlos mit Künstlicher Intelligenz The Software as a Service can be found here: SEO Optimizer: Ghost Writer - Hausarbeiten schreiben mit KI and KI Text Generator This product includes software developed by Sebastian Enger, M.Sc. Copyright (c) 2023, Sebastian Enger, M.Sc. All rights reserved. Frontend and Backend Source Code for... -
25
Scrapyd
A service daemon to run Scrapy spiders
Scrapyd can manage multiple projects and each project can have multiple versions uploaded, but only the latest one will be used for launching new spiders. A common (and useful) convention to use for the version name is the revision number of the version control tool you’re using to track your Scrapy project code. For example: r23. The versions are not compared alphabetically but using a smarter algorithm (the same packaging uses) so r10 compares greater to r9, for example. Scrapyd is an...
