Search Results for "python web crawler" - Page 11
Sort By:
Showing 2674 open source projects for "python web crawler"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
Keep company data safe with Chrome EnterpriseMake AI work your way with Chrome Enterprise. Block unapproved sites and set custom data controls that align with your company's policies.
-
1
VectorVein
No-code AI workflow
Use the power of AI to build your personal knowledge base + automated workflow. No programming, just dragging to create a strong workflow and automate all tasks. Vector vein is affected LangChain as well as langflow The uncode AI workflow software developed by the inspiration aims to combine the powerful capabilities of large language models and allow users to realize the intelligibility and automation of various daily workflows through simple drag. After the software is opened normally,... -
2
Feast
Feature Store for Machine Learning
Feast (Feature Store) is an open source feature store for machine learning. Feast is the fastest path to manage existing infrastructure to productionize analytic data for model training and online inference. Make features consistently available for training and serving by managing an offline store (to process historical data for scale-out batch scoring or model training), a low-latency online store (to power real-time prediction), and a battle-tested feature server (to serve pre-computed... -
3
FastChat
Open platform for training, serving, and evaluating language models
FastChat is an open platform for training, serving, and evaluating large language model-based chatbots. If you do not have enough memory, you can enable 8-bit compression by adding --load-8bit to the commands above. This can reduce memory usage by around half with slightly degraded model quality. It is compatible with the CPU, GPU, and Metal backend. Vicuna-13B with 8-bit compression can run on a single NVIDIA 3090/4080/T4/V100(16GB) GPU. In addition to that, you can add --cpu-offloading to... -
4
Rhino
On-device Speech-to-Intent engine powered by deep learning
Rhino is Picovoice's Speech-to-Intent engine. It directly infers intent from spoken commands within a given context of interest, in real-time. The end-to-end platform for embedding private voice AI into any software in a few lines of code. Design with no limits on top of a modular platform. Create use-case-specific voice AI models in seconds. Develop voice features with a few lines of code using intuitive and cross-platform SDKs. Deliver voice AI everywhere: on-device, mobile, web browsers... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
Marvin
A batteries-included library for building AI-powered software
Meet Marvin: a batteries-included library for building AI-powered software. Marvin's job is to integrate AI directly into your codebase by making it look and feel like any other function. Marvin introduces a new concept called AI Functions. These functions differ from conventional ones in that they don’t rely on source code, but instead generate their outputs on-demand through AI. With AI functions, you don't have to write complex code for tasks like extracting entities from web pages, scoring... -
6
website-to-gif
Turn your website into a GIF
This Github Action automatically creates an animated GIF or WebP from a given web page to display on your project README (or anywhere else). In your GitHub repo, create a workflow file or extend an existing one. You have to also include a step to checkout and commit to the repo. You can use the following example gif.yml. Make sure to modify the url value and add any other input you want to use. WebP rendering will take a lot of time to benefit from lossless quality and file size optimization. -
7
Apprise
Apprise - Push Notifications that work with just about every platform!
Take advantage of Apprise through your network with a user-friendly API. Apprise API was designed to easily fit into existing (and new) eco-systems that are looking for a simple notification solution. There is a small built-in Configuration Manager that can be optionally accessed through your web browser allowing you to create and save as many configurations as you'd like. Each configuration is differentiated by a unique {KEY} that you decide on. Once you've saved your configuration, you'll... -
8
Archivematica
Free and open-source digital preservation system
Archivematica is a web- and standards-based, open-source application which allows your institution to preserve long-term access to trustworthy, authentic, and reliable digital content. Our target users are archivists, librarians, and anyone working to preserve digital objects. You are free to copy, modify, and distribute Archivematica with attribution under the terms of the AGPLv3 license. Archivematica is an open-source application based on recognized standards that makes it possible... -
9
wger
Self hosted FLOSS fitness/workout, nutrition and weight tracker
wger Workout Manager is a free and open web application that manages your exercises, routines and nutrition. It started out as a personal project to replace my growing collection of spreadsheets but has turned into something that other people may find useful. You can create and manage flexible training routines for whatever goals you have. Select exactly what exercises you are going to do and how many repetitions, time or distance you want to do. You can also combine different workouts... -
Photo and Video Editing APIs and SDKsUnlock Picsart's full editing suite by embedding our Editor SDK directly into your platform. Offer your users the power of a full design suite without leaving your site.
-
10
ConsoleMe
A central control plane for AWS permissions and access
ConsoleMe is a web service that makes AWS IAM permissions and credential management easier for end-users and cloud administrators. ConsoleMe provides numerous ways to log in to the AWS Console. An IAM Self-Service Wizard lets users request IAM permissions in plain English. Cross-account resource policies will be automatically generated and can be applied with a single click for certain resource types. Weep (ConsoleMe’s CLI) supports 5 different ways of serving AWS credentials locally. Cloud... -
11
Kinto
A generic JSON document store with sharing and synchronisation options
Kinto is a minimalist JSON storage service with synchronization and sharing abilities. It is meant to be easy to use and easy to self-host. Kinto is used at Mozilla and released under the Apache v2 license. It’s hard for frontend developers to respect users' privacy when building applications that work offline, store data remotely and synchronize across devices. Existing solutions either rely on big corporations that crave user data or require a non-trivial amount of time and expertise to... -
12
TensorBoardX
tensorboard for pytorch (and chainer, mxnet, numpy, etc.)
The SummaryWriter class provides a high-level API to create an event file in a given directory and add summaries and events to it. The class updates the file contents asynchronously. This allows a training program to call methods to add data to the file directly from the training loop, without slowing down training. TensorboardX now supports logging directly to Comet. Comet is a free cloud based solution that allows you to automatically track, compare and explain your experiments. It adds a... -
13
Flask-SQLAlchemy
Adds SQLAlchemy support to Flask
Flask-SQLAlchemy is an extension for Flask that adds support for SQLAlchemy to your application. It simplifies using SQLAlchemy with Flask by setting up common objects and patterns for using those objects, such as a session tied to each web request, models, and engines. Flask-SQLAlchemy does not change how SQLAlchemy works or is used. See the SQLAlchemy documentation to learn how to work with the ORM in depth. The documentation here will only cover setting up the extension, not how to use... -
14
RPA for Python
Python package for doing RPA
Python package for doing RPA. RPA for Python's simple and powerful API makes robotic process automation fun! You can use it to quickly automate away repetitive time-consuming tasks on websites, desktop applications, or the command line. See sample Python script, the RPA Challenge solution, and RedMart groceries example. To send a Telegram app notification, simply look up @rpapybot to allow receiving messages. To automate Chrome browser invisibly, use headless mode. To run 10X faster instead... -
15
text-dedup
All-in-one text de-duplication
text-dedup is a Python library that enables efficient deduplication of large text corpora by using MinHash and other probabilistic techniques to detect near-duplicate content. This is especially useful for NLP tasks where duplicated training data can skew model performance. text-dedup scales to billions of documents and offers tools for chunking, hashing, and comparing text efficiently with low memory usage. It supports Jaccard similarity thresholding, parallel execution, and flexible... -
16
Basaran
Basaran, an open-source alternative to the OpenAI text completion API
.... Support both decoder-only and encoder-decoder models. Detokenizer that handles surrogates and whitespace. Multi-GPU support with optional 8-bit quantization. Real-time partial progress using server-sent events. Compatible with OpenAI API and client libraries. Comes with a fancy web-based playground. Docker images are available on Docker Hub and GitHub Packages. -
17
script-server
Web UI for your scripts with execution management
Script-server is a Web UI for scripts. As an administrator, you add your existing scripts into Script server and other users would be able to execute them via a web interface. The UI is very straightforward and can be used by non-tech people. No script modifications are needed - you configure each script in Script server and it creates the corresponding UI with parameters and takes care of validation, execution, etc. -
18
Crawlab
Distributed web crawler admin platform for spiders management
Golang-based distributed web crawler management platform, supporting various languages including Python, NodeJS, Go, Java, PHP and various web crawler frameworks including Scrapy, Puppeteer, Selenium. Please use docker-compose to one-click to start up. By doing so, you don't even have to configure MongoDB database. The frontend app interacts with the master node, which communicates with other components such as MongoDB, SeaweedFS and worker nodes. Master node and worker nodes communicate... -
19
Gerapy
Distributed Crawler Management Framework Based on Scrapy
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js. Someone who has worked as a crawler with Python may use Scrapy. Scrapy is indeed a very powerful crawler framework. It has high crawling efficiency and good scalability. It is basically a necessary tool for developing crawlers using Python. If you use Scrapy as a crawler, then of course we can use our own host to crawl when crawling, but when the crawl is very large, we can’t run... -
20
Shynet
Modern, privacy-friendly, and detailed web analytics
Modern, privacy-friendly, and detailed web analytics that works without cookies or JS. There are a lot of web analytics tools. Unfortunately, most of them come with the following caveats. They require handing all of your visitors' info to a third-party company They use cookies to track visitors across sessions, so you need to have those annoying cookie notices. They collect so much personal data that even the NSA is jealous. They are closed source and/or expensive, often with limited data... -
21
DNS Crawler
A Bulk Domain Assessment Tool
DNS Crawler is a lightweight, Python-based utility designed for efficient batch processing and assessment of internet domain names. It reads from a list of domains formatted as: domain_name <tab> or ; optional_comment and generates a detailed, Excel-compatible CSV report with columns including: DOMAIN: Domain name REG: Registrar SOA, NS, MX, TXT, SPF, DMARC, MS, A, PTR: Common DNS records for comprehensive domain analysis NOTE: Optional comments from the original input file... -
22
qBittorrent
A free and reliable P2P BitTorrent client
An advanced and multi-platform BitTorrent client with a nice Qt user interface as well as a Web UI for remote control and an integrated search engine. qBittorrent aims to meet the needs of most users while using as little CPU and memory as possible. -
23
This tool is used to open websites/links by pressing one or more keys on the keyboard, acting as a true shortcut for web pages. When the shortcut keys are pressed, you will be directed to the site previously entered through the main browser set in the system (if the tool does not work after setting the shortcuts, try restarting it).
-
24
Ascoos Web Extended Studio
Is a portable web server suite for windows 64Bit, for Web Development.
The Ascoos Web Extended Studio is a special 64Bit freeware version of web server for all Web Developers and Designers and is based on Apache, PHP, MariaDB, MongoDB, Filezilla and other. It offers to user the option of executing different versions of PHP and MariaDB. It is structured for easy upgrading Each new version of the Ascoos Web Extended Studio, includes the latest versions of individual programs without repealing earlier versions. So, you have the opportunity for experiments... -
25
Headscale-WebUI
A simple Headscale web UI for small-scale deployments
A simple Headscale web UI for small-scale deployments.
