Search Results for "python web crawler" - Page 7
Sort By:
Showing 3141 open source projects for "python web crawler"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
Powering the best of the internet | FastlyEnsure your websites, applications and services can effortlessly handle the demands of your users with Fastly. Fastly’s portfolio is designed to be highly performant, personalized and secure while seamlessly scaling to support your growth.
-
1
Full Stack FastAPI and PostgreSQL
Full stack, modern web application generator
Generate a backend and frontend stack using Python, including interactive API documentation. Production ready Python web server using Uvicorn and Gunicorn. Very high performance, on par with NodeJS and Go (thanks to Starlette and Pydantic). Great editor support. Completion everywhere. Less time debugging. Designed to be easy to use and learn. Less time reading docs. Minimize code duplication. Multiple features from each parameter declaration. Get production-ready code. With automatic... -
2
mitmproxy
A free and open source interactive HTTPS proxy
mitmproxy is an open source, interactive SSL/TLS-capable intercepting HTTP proxy, with a console interface fit for HTTP/1, HTTP/2, and WebSockets. It's the ideal tool for penetration testers and software developers, able to debug, test, and make privacy measurements. It can intercept, inspect, modify and replay web traffic, and can even prettify and decode a variety of message types. Its web-based interface mitmweb gives you a similar experience as Chrome's DevTools, with the addition... -
3
django-helpdesk
A Django application to manage tickets for an internal helpdesk
A Django application to manage tickets for an internal helpdesk. Formerly known as Jutda Helpdesk. django-helpdesk was formerly known as Jutda Helpdesk, named after the company which originally created it. As of January 2011 the name has been changed to reflect what it really is: a Django-powered ticket tracker with contributors reaching far beyond Jutda. django-helpdesk includes a basic demo Django project so that you may easily get started with testing or developing django-helpdesk. The... -
4
Modoboa
Mail hosting made simple
Modoboa is a mail hosting and management platform including a modern and simplified Web User Interface. It provides useful components such as an administration panel and webmail. Modoboa integrates with well known software such as Postfix or Dovecot. A SQL database (MySQL, PostgreSQL or SQLite) is used as a central point of communication between all components. Modoboa is developed with modularity in mind, expanding it is really easy. Actually, all current features are extensions. It is written... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
GRR
GRR Rapid Response, remote live forensics for incident response
GRR Rapid Response is an incident response framework focused on remote live forensics. It consists of a python client (agent) that is installed on target systems, and python server infrastructure that can manage and talk to clients. The goal of GRR is to support forensics and investigations in a fast, scalable manner to allow analysts to quickly triage attacks and perform analysis remotely. GRR client is deployed on systems that one might want to investigate. On every such system, once deployed... -
6
FastUI
Build better UIs faster
FastUI is a library that lets developers build interactive user interfaces for FastAPI applications using Pydantic models. It automatically generates frontend components based on data schemas and endpoint logic, reducing the need for manual UI development. Designed to be type-safe, reactive, and fast, FastUI streamlines the creation of web dashboards, admin panels, and internal tools within a FastAPI backend. -
7
Composio
Composio equip's your AI agents & LLMs
Empower your AI agents with Composio - a platform for managing and integrating tools with LLMs & AI agents using Function Calling. Equip your agent with high-quality tools & integrations without worrying about authentication, accuracy, and reliability in a single line of code. -
8
Ulauncher
Feature rich application Launcher for Linux
Type in an application name without worrying about spelling. Ulauncher will figure out what you meant. It also remembers your previous choices and automatically selects the best option for you. Ulauncher provides 4 themes built in. But if you need something different you can always create a custom color theme. Improve your workflow with customizable shortcuts and extensions. Create a shortcut for web search or your scripts or install a 3rd party extension. -
9
HTTPie
A CLI, cURL-like tool for humans
HTTPie is a modern command-line HTTP client that makes CLI interaction with web services as human-friendly as possible. It offers a plethora of friendly features that make it an excellent curl alternative. It is equipped with an intuitive UI, JSON support, syntax highlighting and so much more. HTTPie gives a single http command for sending arbitrary HTTP requests with a simple, natural syntax, and displayed in a formatted, colorized terminal output. HTTPie can be installed on macOS, Windows... -
Get Avast Free Antivirus | Your top-rated shield against malware and online scamsOur antivirus software scans for security and performance issues and helps you to fix them instantly. It also protects you in real time by analyzing unknown files before they reach your desktop PC or laptop — all for free.
-
10
ClearML
Streamline your ML workflow
ClearML is an open source platform that automates and simplifies developing and managing machine learning solutions for thousands of data science teams all over the world. It is designed as an end-to-end MLOps suite allowing you to focus on developing your ML code & automation, while ClearML ensures your work is reproducible and scalable. The ClearML Python Package for integrating ClearML into your existing scripts by adding just two lines of code, and optionally extending your experiments... -
11
Letterboxd Recommendations
Scraping publicly-accessible Letterboxd data for movie recommendations
Scraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username. A user's "star" ratings are scraped from their Letterboxd profile and assigned numerical ratings from 1 to 10 (accounting for half stars). Their ratings are then combined with a sample of ratings from the top 4000 most active users on the site to create a collaborative filtering recommender model using singular value... -
12
JobFunnel
Scrape job websites into a single spreadsheet with no duplicates.
Scrape job websites into a single spreadsheet with no duplicates. Automated tool for scraping job postings into a .csv file. You can search for jobs with YAML configuration files or by passing command arguments. By performing regular scraping and reviewing, you can cut through the noise of even the busiest job markets. Run funnel with your settings YAML to populate your master CSV file with jobs from available providers. JobFunnel can be easily automated to run nightly with crontab. If you... -
13
Flask-Limiter
Rate Limiting extension for Flask
Flask-Limiter provides rate-limiting features to flask applications. It allows configuring various backends to persist the rate limits, which is provided by the limits library. Sponsored by Zuplo - fully-managed API Gateway with rate limiting, authentication, and more. Add rate limiting to your API in minutes, try it at zuplo.com Test it out. The fast endpoint respects the default rate limit while the slow endpoint uses the decorated one. ping has no rate limit associated with it. By adding... -
14
h2oGPT
Private chat with local GPT with document, images, video, etc.
h2oGPT is an open-source platform that allows users to interact with local GPT models in a completely private environment. It supports a variety of document types, including PDFs, Word files, images, video frames, and even audio, enabling users to query and analyze their documents or engage in a private chat with AI. The platform is designed to be secure and offline, ensuring that all data remains private and under the user's control. h2oGPT supports several AI models, including oLLaMa and... -
15
MedicalGPT
MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training
MedicalGPT training medical GPT model with ChatGPT training pipeline, implementation of Pretraining, Supervised Finetuning, Reward Modeling and Reinforcement Learning. MedicalGPT trains large medical models, including secondary pre-training, supervised fine-tuning, reward modeling, and reinforcement learning training. -
16
Feast
Feature Store for Machine Learning
Feast (Feature Store) is an open source feature store for machine learning. Feast is the fastest path to manage existing infrastructure to productionize analytic data for model training and online inference. Make features consistently available for training and serving by managing an offline store (to process historical data for scale-out batch scoring or model training), a low-latency online store (to power real-time prediction), and a battle-tested feature server (to serve pre-computed... -
17
D-Tale
Visualizer for pandas data structures
D-Tale is the combination of a Flask backend and a React front-end to bring you an easy way to view & analyze Pandas data structures. It integrates seamlessly with ipython notebooks & python/ipython terminals. Currently, this tool supports such Pandas objects as DataFrame, Series, MultiIndex, DatetimeIndex & RangeIndex. D-Tale was the product of a SAS to Python conversion. What was originally a perl script wrapper on top of SAS's insight function is now a lightweight web client on top of Pandas... -
18
AWS Toolkit for JetBrains
A plugin for interacting with AWS from JetBrains IDEs
The AWS Toolkit for JetBrains makes it easier to write applications built on Amazon Web Services. If you come across bugs with the toolkit or have feature requests, please raise an issue on our GitHub repository. See the user guide for how to get started, along with what features/services are supported. CodeWhisperer uses machine learning to generate code suggestions from the existing code and comments in your IDE. Supported languages include: Java, Python, and JavaScript. In addition... -
19
Kinto
A generic JSON document store with sharing and synchronisation options
Kinto is a minimalist JSON storage service with synchronization and sharing abilities. It is meant to be easy to use and easy to self-host. Kinto is used at Mozilla and released under the Apache v2 license. It’s hard for frontend developers to respect users' privacy when building applications that work offline, store data remotely and synchronize across devices. Existing solutions either rely on big corporations that crave user data or require a non-trivial amount of time and expertise to... -
20
Playwright for .NET
.NET version of the Playwright testing and automation library
..., JavaScript, Python, .NET, Java. Test Mobile Web. Native mobile emulation of Google Chrome for Android and Mobile Safari. The same rendering engine works on your Desktop and in the Cloud. Auto-wait. Playwright waits for elements to be actionable prior to performing actions. It also has a rich set of introspection events. The combination of the two eliminates the need for artificial timeouts - the primary cause of flaky tests. -
21
TorBot
Dark Web OSINT Tool
Contributions to this project are always welcome. To add a new feature fork the dev branch and give a pull request when your new feature is tested and complete. If its a new module, it should be put inside the modules directory. The branch name should be your new feature name in the format <Feature_featurename_version(optional)>. On Linux platforms, you can make an executable for TorBot by using the install.sh script. You will need to give the script the correct permissions using chmod +x... -
22
Lexbor
Lexbor is development of an open source HTML Renderer library
Lexbor is the development of a web browser engine available as a software library; it ships with a free license and has no extra dependencies. For us, speed is an absolute must-have. In our development process, we focus on fastest parsing techniques for HTML, CSS, and fonts, fastest data processing methods, and fastest ways to serve content to end users. Whether you are building a backend that handles millions of HTML documents or a UI-heavy user app, your software’s response rate always... -
23
Nerves
Craft and deploy bulletproof embedded software in Elixir
Nerves is the open-source platform and infrastructure you need to build, deploy, and securely manage your fleet of IoT devices at speed and scale. Nerves is written in Elixir, but you don’t have to rewrite everything in Elixir to get the advantages of Nerves, simply bring your own code (like C, C++, Python, Rust, and more) and scale up. Nerves use the Erlang runtime system, known for being distributed, fault-tolerant, soft real-time, and highly available. Nerves has the tools you need to manage... -
24
Tock
Tock, the open source conversational AI toolkit
Complete and autonomous NLU solution leveraging opensource libs, such as OpenNLP, Stanford, Duckling and more. Web, mobile, social networks, smart speakers and more. Create your bot once, connect it progressively to multiple channels as you need them. Simple graphical interfaces to build stories and models, manage multilingual and multichannel bots, better understand users with analytics. Program complex stories using Kotlin, Python or Node.js provided components, or integrate with any language... -
25
MechanicalSoup
A Python library for automating interaction with websites
A Python library for automating interaction with websites. MechanicalSoup automatically stores and sends cookies, follows redirects, and can follow links and submit forms. It doesn't do JavaScript. MechanicalSoup was created by M Hickford, who was a fond user of the Mechanize library. Unfortunately, Mechanize was incompatible with Python 3 until 2019 and its development stalled for several years. MechanicalSoup provides a similar API, built on Python giants Requests (for HTTP sessions...
