Search Results for "python web crawler" - Page 12
Sort By:
Showing 2703 open source projects for "python web crawler"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
MongoDB Atlas | Run databases anywhereMongoDB Atlas allows you to build and run modern applications across 125+ cloud regions, spanning AWS, Azure, and Google Cloud. Its multi-cloud clusters enable seamless data distribution and automated failover between cloud providers, ensuring high availability and flexibility without added complexity.
-
1
Siddhi Core Libraries
Stream Processing and Complex Event Processing Engine
... to various endpoints in real time. Agile development experience with SQL-like query language and graphical drag-and-drop editor supporting event simulation. Lightweight runtime that can natively run on Kubernetes, Docker, VM, or bare metal, and embedded in any Java or Python application. Scalable, and highly available distributed event processing on Kubernetes, with NATS Streaming and Siddhi Kubernetes Operator. -
2
Mailu
Insular email distribution - mail server as Docker images
Mailu is a simple yet full-featured mail server as a set of Docker images. It is free software (both as in free beer and as in free speech), open to suggestions and external contributions. The project aims at providing people with an easily setup, easily maintained and full-featured mail server while not shipping proprietary software nor unrelated features often found in popular groupware. Security, enforced TLS, DANE, MTA-STS, Letsencrypt!, outgoing DKIM, anti-virus scanner, Snuffleupagus,... -
3
FastChat
Open platform for training, serving, and evaluating language models
FastChat is an open platform for training, serving, and evaluating large language model-based chatbots. If you do not have enough memory, you can enable 8-bit compression by adding --load-8bit to the commands above. This can reduce memory usage by around half with slightly degraded model quality. It is compatible with the CPU, GPU, and Metal backend. Vicuna-13B with 8-bit compression can run on a single NVIDIA 3090/4080/T4/V100(16GB) GPU. In addition to that, you can add --cpu-offloading to... -
4
Dockerized Nerd Fonts Patcher
Dockerized Nerd Fonts patcher
... the provided FontForge Python Script to patch your own font or to generate millions of unique combinations/variations. -
No-Nonsense Code-to-Cloud Security for Devs | AikidoAikido provides a unified security platform for developers, combining 12 powerful scans like SAST, DAST, and CSPM. AI-driven AutoFix and AutoTriage streamline vulnerability management, while runtime protection blocks attacks.
-
5
website-to-gif
Turn your website into a GIF
This Github Action automatically creates an animated GIF or WebP from a given web page to display on your project README (or anywhere else). In your GitHub repo, create a workflow file or extend an existing one. You have to also include a step to checkout and commit to the repo. You can use the following example gif.yml. Make sure to modify the url value and add any other input you want to use. WebP rendering will take a lot of time to benefit from lossless quality and file size optimization. -
6
Apprise
Apprise - Push Notifications that work with just about every platform!
Take advantage of Apprise through your network with a user-friendly API. Apprise API was designed to easily fit into existing (and new) eco-systems that are looking for a simple notification solution. There is a small built-in Configuration Manager that can be optionally accessed through your web browser allowing you to create and save as many configurations as you'd like. Each configuration is differentiated by a unique {KEY} that you decide on. Once you've saved your configuration, you'll... -
7
wger
Self hosted FLOSS fitness/workout, nutrition and weight tracker
wger Workout Manager is a free and open web application that manages your exercises, routines and nutrition. It started out as a personal project to replace my growing collection of spreadsheets but has turned into something that other people may find useful. You can create and manage flexible training routines for whatever goals you have. Select exactly what exercises you are going to do and how many repetitions, time or distance you want to do. You can also combine different workouts... -
8
ConsoleMe
A central control plane for AWS permissions and access
ConsoleMe is a web service that makes AWS IAM permissions and credential management easier for end-users and cloud administrators. ConsoleMe provides numerous ways to log in to the AWS Console. An IAM Self-Service Wizard lets users request IAM permissions in plain English. Cross-account resource policies will be automatically generated and can be applied with a single click for certain resource types. Weep (ConsoleMe’s CLI) supports 5 different ways of serving AWS credentials locally. Cloud... -
9
Shynet
Modern, privacy-friendly, and detailed web analytics
Modern, privacy-friendly, and detailed web analytics that works without cookies or JS. There are a lot of web analytics tools. Unfortunately, most of them come with the following caveats. They require handing all of your visitors' info to a third-party company They use cookies to track visitors across sessions, so you need to have those annoying cookie notices. They collect so much personal data that even the NSA is jealous. They are closed source and/or expensive, often with limited data... -
Deliver secure remote access with OpenVPN.OpenVPN's products provide scalable, secure remote access — giving complete freedom to your employees to work outside the office while securely accessing SaaS, the internet, and company resources.
-
10
Framework Benchmarks
Source for the TechEmpower Framework Benchmarks project
If you're new to the project, welcome! Please feel free to ask questions here. We encourage new frameworks and contributors to ask questions. We're here to help! This project provides representative performance measures across a wide field of web application frameworks. With much help from the community, coverage is quite broad and we are happy to broaden it further with contributions. The project presently includes frameworks on many languages including Go, Python, Java, Ruby, PHP, C#, F... -
11
CssSelector Component
Converts CSS selectors to XPath expressions
... to an XPath equivalent. This XPath expression can then be used with other functions and classes that use XPath to find elements in a document. Not all CSS selectors can be converted to XPath equivalents. There are several CSS selectors that only make sense in the context of a web-browser. Pseudo-elements (:before, :after, :first-line, :first-letter) are not supported because they select portions of text rather than elements. -
12
ACHE Focused Crawler
ACHE is a web crawler for domain-specific search
ACHE is a focused web crawler. It collects web pages that satisfy some specific criteria, e.g., pages that belong to a given domain or that contain a user-specified pattern. ACHE differs from generic crawlers in sense that it uses page classifiers to distinguish between relevant and irrelevant pages in a given domain. A page classifier can be defined as a simple regular expression (e.g., that matches every page that contains a specific word) or a machine-learning-based classification model... -
13
chatgpt-web
Privatized web program based on ChatGPT3.5 API
This project can deploy your own customized ChatGPT web program (compatible with gpt3.5) with one click, just download the project file of the corresponding platform in the release, modify the configuration and execute it, open http://127.0.0.1:8080, you can Get your own ChatGPT website. Note that each parameter may affect you to get a different chat effect, and you may get another answer if you change a parameter, so please try to debug it yourself, and don't complain about the artificial... -
14
SiteOne Crawler (desktop app)
A free, feature-rich web analyzer and exporter/cloner you will love!
A free in-depth website analyzer providing audits of security, performance, SEO, accessibility and other technical aspects. Available as a desktop application for Windows/macOS/Linux and as a CLI tool for advanced users and CI/CD processes. It also includes an offline web page exporter (website clone, mirror). -
15
DNS Crawler
A Bulk Domain Assessment Tool
DNS Crawler is a lightweight, Python-based utility designed for efficient batch processing and assessment of internet domain names. It reads from a list of domains formatted as: domain_name <tab> or ; optional_comment and generates a detailed, Excel-compatible CSV report with columns including: DOMAIN: Domain name REG: Registrar SOA, NS, MX, TXT, SPF, DMARC, MS, A, PTR: Common DNS records for comprehensive domain analysis NOTE: Optional comments from the original input file... -
16
Atri Framework
Open-source full-stack web development framework built on top of React
The web framework to build stunning web apps. Build frontend easily using Atri visual builder or React code. Build backend using FastAPI. Atri framework is not just limited to the JavaScript world. You can use this framework with many languages such as Python, NodeJS (upcoming), etc. Atri framework comes with a suite of productivity tools such as visual editor, asset management tools, etc. that significantly reduce development time from months to hours. Using Atri framework, developers do... -
17
Easyspider - Distributed Web Crawler
Easy Spider is a distributed Perl Web Crawler Project from 2006
Easy Spider is a distributed Perl Web Crawler Project from 2006. It features code from crawling webpages, distributing it to a server and generating xml files from it. The client site can be any computer (Windows or Linux) and the Server stores all data. Websites that use EasySpider Crawling for Article Writing Software: https://www.artikelschreiber.com/en/ https://www.unaique.net/en/ https://www.unaique.com/ https://www.artikelschreiben.com/ https://www.buzzerstar.com/ https... -
18
llm-chain
Rust crate for building chains in large language models
... language understanding and generation tasks with Facebook's research models. Incorporates support for Stanford's Alpaca models, expanding the range of available language models for advanced AI applications. Enhance your AI agents' capabilities by giving them access to various tools, such as running Bash commands, executing Python scripts, or performing web searches. -
19
Headscale-WebUI
A simple Headscale web UI for small-scale deployments
A simple Headscale web UI for small-scale deployments. -
20
Cinemagoer
Python package useful to retrieve and manage the data of IMDb
Cinemagoer (previously known as IMDbPY) is a Python package for retrieving and managing the data of the IMDb movie database about movies and people. You can use the search_movie method of the access object to search for movies with a given (or similar) title. Similarly, you can search for people and companies using the search_person and the search_company methods. Movie, person, and company objects have id attributes which -when fetched through the IMDb web server- store the IMDb id... -
21
Chameleon LLM
Codes for "Chameleon: Plug-and-Play Compositional Reasoning
Discover Chameleon, our cutting-edge compositional reasoning framework designed to enhance large language models (LLMs) and overcome their inherent limitations, such as outdated information and lack of precise reasoning. By integrating various tools such as vision models, web search engines, Python functions, and rule-based modules, Chameleon delivers more accurate, up-to-date, and precise responses, making it a game-changer in the natural language processing landscape. With GPT-4 at its core... -
22
Portainer Templates
500+ 1-click Portainer app templates
A compiled list of 500+ ready-to-go Portainer App templates. In Portainer, App Templates enable you to easily deploy services with a predetermined configuration, while allowing you to customize options through the web UI. While Portainer ships with some default templates, it's often helpful to have 1-click access to many more apps + stacks, without having to constantly switch template sources. This repo combines app templates from several sources, to create a ready-to-go template file... -
23
qBittorrent
A free and reliable P2P BitTorrent client
An advanced and multi-platform BitTorrent client with a nice Qt user interface as well as a Web UI for remote control and an integrated search engine. qBittorrent aims to meet the needs of most users while using as little CPU and memory as possible. -
24
Goutte
Goutte, a simple PHP Web Scraper
Goutte is a screen scraping and web crawling library for PHP. Goutte provides a nice API to crawl websites and extract data from the HTML/XML responses. Goutte depends on PHP 7.1+. Add fabpot/goutte as a require dependency in your composer.json file. Create a Goutte Client instance (which extends Symfony\Component\BrowserKit\HttpBrowser). Make requests with the request() method. The method returns a Crawler object (Symfony\Component\DomCrawler\Crawler). To use your own HTTP settings, you may... -
25
HackTools
The all-in-one Red Team extension for Web Pentesters
The all-in-one Red Team browser extension for Web Pentesters. HackTools, is a web extension facilitating your web application penetration tests, it includes cheat sheets as well as all the tools used during a test such as XSS payloads, Reverse shells and much more. With the extension you no longer need to search for payloads in different websites or in your local storage space, most of the tools are accessible in one click. HackTools is accessible either in pop-up mode or in a whole tab...
