Search Results for "python web crawler"
Sort By:
Showing 14 open source projects for "python web crawler"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
Free and Open Source HR SoftwareGive your HR team the tools they need to streamline administrative tasks, support employees, and make informed decisions with the OrangeHRM free and open source HR software.
-
1
crawley
The unix-way web crawler
Crawls web pages and prints any link it can find. Fast HTML SAX-parser (powered by golang.org/x/net/html) Small (below 1500 SLOC), idiomatic, 100% test-covered codebase. Grabs most of useful resources URLs (pics, videos, audios, forms, etc...) Found URLs are streamed to stdout and guaranteed to be unique (with fragments omitted) Scan depth (limited by starting host and path, by default - 0) can be configured. Can crawl rules and sitemaps from robots.txt. Brute mode - scan HTML comments for... -
2
Gobuster
Directory/File, DNS and VHost busting tool written in Go
Gobuster is a tool used to brute-force. This project is born out of the necessity to have something that didn't have a fat Java GUI (console FTW), something that did not do recursive brute force, something that allowed me to brute force folders and multiple extensions at once, something that compiled to native on multiple platforms, something that was faster than an interpreted script (such as Python), and something that didn't require a runtime. Provides several modes, like the classic... -
3
Grafana
Leading open-source visualization and observability platform
Grafana Open Source is a leading open-source visualization and observability platform that lets you query, visualize, alert on, and explore your data—regardless of where it’s stored. With support for 100+ data source plugins (such as Prometheus, Loki, Elasticsearch, InfluxDB, SQL/NoSQL databases, OpenTelemetry, and more), you can unify metrics, logs, traces, and other observability signals in one place. Grafana OSS empowers you to build dynamic, reusable dashboards with rich... -
4
Klavis AI
MCP integration platforms for AI agents to use tools at any scale
Klavis AI is a Y Combinator X25-backed open-source infrastructure platform that enables AI agents to reliably connect with external tools and services at scale through Model Context Protocol (MCP). Founded by ex-Google DeepMind and ex-Lyft engineers, Klavis provides 50+ production-ready MCP servers with enterprise OAuth support for GitHub, Slack, Gmail, Salesforce, Linear, Notion, and more. The flagship product Strata solves tool overload through progressive discovery, achieving +13% higher... -
Self-hosted n8n: No-code AI workflowsA free-to-use workflow automation tool, n8n lets you connect all your apps and data in one customizable, no-code platform. Design workflows and process data from a simple, unified dashboard.
-
5
OSV.dev
Open source vulnerability DB and triage service
osv.dev (Open Source Vulnerabilities) is Google’s open source platform and API for aggregating, managing, and analyzing vulnerability data across multiple ecosystems. It powers the osv.dev website, providing a unified, queryable database of vulnerabilities that map directly to open source packages and versions. The system hosts vulnerability data for ecosystems such as PyPI, npm, Go, Maven, and Debian, among others. The platform includes a web UI, API, and a Go-based dependency scanner... -
6
Crawlab
Distributed web crawler admin platform for spiders management
Golang-based distributed web crawler management platform, supporting various languages including Python, NodeJS, Go, Java, PHP and various web crawler frameworks including Scrapy, Puppeteer, Selenium. Please use docker-compose to one-click to start up. By doing so, you don't even have to configure MongoDB database. The frontend app interacts with the master node, which communicates with other components such as MongoDB, SeaweedFS and worker nodes. ... -
7
chatgpt-web
Privatized web program based on ChatGPT3.5 API
...There are more than 20 parameter examples in the document, such as AI chatbot, product name generation, python code fixer, etc. -
8
PhoenixC2
Command & Control-Framework created for collaboration in python3
PhoenixC2 is a command & control framework. The purpose of this software is, to aid red teamers and penetration testers in their operations, by providing a way to manage hacked devices. -
9
Glazier
A tool for automating the installation of Windows OS
...Its extensibility makes it easy for administrators to create custom actions using Python or PowerShell, enabling tailored automation for diverse enterprise environments. Designed for engineers, Glazier emphasizes repeatability, maintainability, and transparency in Windows. -
Discover the power of eDiscovery for law firms.DWR eDiscovery allows legal professionals to process, analyze, review, and produce documents that are relevant to litigation and other legal disclosure obligations. Our tools allow easy ingestion and analysis of client and opposing party documents using a comprehensive set of document review features including AI search, keyword search, keyword highlighting, metadata filtering, marking documents, privilege log management, redactions, and a range of analysis tools to help users best understand their document corpus.
-
10
Pholcus
Distributed high-concurrency crawler software written in pure golang
Pholcus is a high-concurrency crawler software written in pure Go language that supports distributed, only used for programming learning and research. It supports three operating modes of stand-alone, server and client, and has three operating interfaces, Web, GUI, and command line; simple and flexible rules, concurrent batch tasks, and rich output methods (mysql/mongodb/kafka/csv/excel, etc.); In addition, it also supports horizontal and vertical grabbing modes, and a series of advanced functions such as simulated login and task suspension and cancellation. ... -
11
Rendora
dynamic server-side rendering using headless Chrome
Rendora is a dynamic renderer to provide zero-configuration server-side rendering mainly to web crawlers in order to effortlessly improve SEO for websites developed in modern Javascript frameworks such as React.js, Vue.js, Angular.js, etc. Rendora works totally independently of your frontend and backend stacks. Rendora can be seen as a reverse HTTP proxy server sitting between your backend server (e.g. Node.js/Express.js, Python/Django, etc...) and potentially your frontend proxy server (e.g. nginx, traefik, apache, etc...) or even directly to the outside world that does actually nothing but transporting requests and responses as they are except when it detects whitelisted requests according to the config. ... -
12
NTK RTMP SERVER
Naam Tamilar Web TV Live Streamer
Naam Tamilar RTMP Server This project updated as open source for future use of Naam Tamilar Political Party. To contribute to the party and in some case if there is any possibilities if i cannot support them for long term. I thought of sharing this source code so in future it may be helpful for the community and party in which other software developers can help them to upgrade. This source is forked from - https://github.com/arut/nginx-rtmp-module and modified with multiple broadcast... -
13
alpaca
Given a web API, Generate client libraries in node, php, python, etc.
API libraries powered and created by Alpaca. Tired of maintaining API libraries in different languages for your website API? This is for you. Do you have an API for your website but no API libraries for whatever reason? This is for you. You are planning to build an API for your website and develop API libraries? This is for you. You define your API according to the format given below, alpaca builds the API libraries along with their documentation. All you have to do is publishing them to... -
14
turkdevops.github.io
Türk Geliştirici Operasyonları SourceForge
"TurkDevOps SourForce" Herkese selamlar, Geliştirici Ekipleri için açık kaynağa olan ilgimiz ve katkılarımız sayesinde bizlere yardımcı olarak kuruluşumuzu destekleyen "SourceForge" kuruluşuna teşekkür ederiz. Güncellemeler için lütfen abone olunuz, her zaman güvende kalın. Posta listeleri ve tartışma forumu dahilinde her türlü fikir ve yorum alışverişine açığız Topluluklarımızda misafirperver dayanışmamız saygı ve hoşgörü ile karşılanır, profesyonel bir ortamda davranış...
- Previous
- You're on page 1
- Next
