Search Results for "python web crawler" - Page 9
Sort By:
Showing 3141 open source projects for "python web crawler"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
MongoDB Atlas | Run databases anywhereMongoDB Atlas allows you to build and run modern applications across 125+ cloud regions, spanning AWS, Azure, and Google Cloud. Its multi-cloud clusters enable seamless data distribution and automated failover between cloud providers, ensuring high availability and flexibility without added complexity.
-
1
Flask-WTF
Simple integration of Flask and WTForms, including CSRF
Simple integration of Flask and WTForms, including CSRF, file upload, and reCAPTCHA. Integration with WTForms. Secure Form with CSRF token. Global CSRF protection. reCAPTCHA support. File upload that works with Flask-Uploads. Internationalization using Flask-Babel. -
2
Scrapy-Redis
Redis-based components for Scrapy
You can start multiple spider instances that share a single redis queue. Best suitable for broad multi-domain crawls. Scraped items gets pushed into a redis queued meaning that you can start as many as needed post-processing processes sharing the items queue. Scheduler + Duplication Filter, Item Pipeline, Base Spiders. Default requests serializer is pickle, but it can be changed to any module with loads and dumps functions. Note that pickle is not compatible between python versions. Version 0.3... -
3
Jupyter Enterprise Gateway
Enables Jupyter Notebooks to share resources across clusters
Jupyter Enterprise Gateway is a headless web server with a pluggable framework for anyone supporting multiple notebook users in a managed-cluster environment. Some of the core functionality it provides is better optimization of compute resources, improved multi-user support, and more granular security for your Jupyter notebook environment - making it suitable for enterprise, scientific, and academic implementations. From a technical perspective, Jupyter Enterprise Gateway is a web server... -
4
Shynet
Modern, privacy-friendly, and detailed web analytics
Modern, privacy-friendly, and detailed web analytics that works without cookies or JS. There are a lot of web analytics tools. Unfortunately, most of them come with the following caveats. They require handing all of your visitors' info to a third-party company They use cookies to track visitors across sessions, so you need to have those annoying cookie notices. They collect so much personal data that even the NSA is jealous. They are closed source and/or expensive, often with limited data... -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
5
You-Get
Dumb downloader that scrapes the web
You-Get is a small command-line utility for downloading media (video, audio and images) from the Web when there are no other means to do so. It can download video and audio files from such popular web sites as YouTube, Twitter, Niconico, Vimeo, Flickr, Instagram and a whole lot more. You-Get is a great option for when you want to enjoy your favorite videos, audio or images from the internet without having to open any web browsers or get interrupted by ads. It’s also a good choice for when... -
6
buku
Personal mini-web in text
buku is a powerful bookmark manager and a personal textual mini-web. For those who prefer the GUI, bukuserver exposes a browsable front-end on a local web host server. When I started writing it, I couldn't find a flexible command-line solution with a private, portable, merge-able database along with seamless GUI integration. Hence, buku. buku can import bookmarks from the browser(s) or fetch the title, tags and description of a URL from the web. Use your favorite editor to add, compose... -
7
Django REST framework
Powerful and flexible toolkit for building Web APIs
Django REST framework is a powerful and flexible toolkit for building Web APIs. Some reasons you might want to use REST framework: The Web browsable API is a huge usability win for your developers. Authentication policies including packages for OAuth1a and OAuth2. Serialization that supports both ORM and non-ORM data sources. Customizable all the way down - just use regular function-based views if you don't need the more powerful features. Extensive documentation, and great community support... -
8
Django LMS
A learning management system using django web framework
django-lms is an open-source Learning Management System (LMS) built with Django and designed for ease of use and extensibility. It allows administrators to manage courses, lessons, quizzes, and users in an educational environment. The project includes a clean UI and backend tools to help educators create and track learning content. -
9
Hyperledger Cello
Operating System for Enterprise Blockchain
Hyperledger Cello is a blockchain operation and provisioning system designed to automate the deployment, management, and scaling of Hyperledger Fabric networks. As part of the Hyperledger project under the Linux Foundation, Cello aims to offer Blockchain-as-a-Service (BaaS) by abstracting the complexity of infrastructure setup for consortiums and enterprises. It provides a dashboard, APIs, and orchestration tools to help users create, monitor, and manage blockchain nodes, ledgers, and... -
Get Avast Free Antivirus | Your top-rated shield against malware and online scamsOur antivirus software scans for security and performance issues and helps you to fix them instantly. It also protects you in real time by analyzing unknown files before they reach your desktop PC or laptop — all for free.
-
10
Alerta
Alerta monitoring system
... can be queried from the command line or viewed in a slick web console optimized for desktop, tablet, and mobile. User logins can be added using Google, GitHub or GitLab OAuth and programmatic access is managed using API keys. -
11
SAFRS
SqlAlchemy Flask-Restful Swagger Json:API OpenAPI
SAFRS exposes SQLAlchemy database models as a JSON:API web service and generates the corresponding swagger/OpenAPI. Database objects such as the User class from the demo.py example can be extended to include relationships with other objects. The demo_relationship.py contains the following extension of the User class where a relationship with the Book class is implemented. -
12
Recap
Recap tracks and transform schemas across your whole application
Recap is a schema language and multi-language toolkit to track and transform schemas across your whole application. Your data passes through web services, databases, message brokers, and object stores. Recap describes these schemas in a single language, regardless of which system your data passes through. Recap schemas can be defined in YAML, TOML, JSON, XML, or any other compatible language. -
13
Yandex Smart Home
Adds support for Yandex Smart Home (Alice voice assistant)
Adds support for Yandex Smart Home (Alice voice assistant) into Home Assistant. The component allows you to add devices from Home Assistant to the Yandex smart home platform and manage them from any device with Alice. The component runs on Home Assistant version 2023.2 or later. -
14
CodeChecker
CodeChecker is an analyzer tooling, defect database
... configuration and forming the corresponding clang analyzer invocations. Incremental analysis: Only the changed files and its dependencies need to be reanalyzed. False positive suppression with a possibility to add review comments. Result visualization in command line or in static HTML. Web application for viewing discovered code defects with a streamlined, easy experience (with PostgreSQL, or SQLite backend). -
15
PyTorch Geometric Temporal
Spatiotemporal Signal Processing with Neural Machine Learning Models
The library consists of various dynamic and temporal geometric deep learning, embedding, and Spatio-temporal regression methods from a variety of published research papers. Moreover, it comes with an easy-to-use dataset loader, train-test splitter and temporal snaphot iterator for dynamic and temporal graphs. The framework naturally provides GPU support. It also comes with a number of benchmark datasets from the epidemiological forecasting, sharing economy, energy production and web traffic... -
16
bookdown
Authoring Books and Technical Documents with R Markdown
... for languages other than R, including C/C++, Python, and SQL, etc. LaTeX equations, theorems, and proofs work for all output formats. Can be published to GitHub, bookdown.org, and any web servers. Integrated with the RStudio IDE. The easiest way to start a new Bookdown project is from within RStudio IDE. Go to File, New Project, New Directory, Book project using bookdown. -
17
CTFd
CTFs as you need them
CTFd is a Capture The Flag framework focusing on ease of use and customizability. It comes with everything you need to run a CTF and it's easy to customize with plugins and themes. Create your own challenges, categories, hints, and flags from the Admin Interface. Dynamic Scoring Challenges. Unlockable challenge support. Challenge plugin architecture to create your own custom challenges. Static & Regex-based flags. Custom flag plugins. Unlockable hints. File uploads to the server or an Amazon... -
18
Weblate
Web based localization tool with tight version control integration
Weblate is a copylefted libre software web-based continuous localization system, used by over 2500 libre projects and companies in more than 165 countries. Copylefted libre software, used by over 2,500 libre software projects and companies in over 165 countries. Hosted service and standalone tool with tight version control integration. Simple and clean user interface, propagation of translations across components, quality checks and automatic linking to source files. There is infrastructure... -
19
Werkzeug
The comprehensive WSGI web application library
Werkzeug is a comprehensive WSGI web application library. It began as a simple collection of various utilities for WSGI applications and has become one of the most advanced WSGI utility libraries. Werkzeug doesn’t enforce any dependencies. It is up to the developer to choose a template engine, database adapter, and even how to handle requests. Includes an interactive debugger that allows inspecting stack traces and source code in the browser with an interactive interpreter for any frame... -
20
CWhy
Explains and suggests fixes for compile-time errors for C, C++, C#, Go
Explains and suggests fixes for compiler error messages for a wide range of programming languages, including C, C++, C#, Go, Java, LaTeX, PHP, Python, Ruby, Rust, Swift, and TypeScript. CWhy needs to be connected to an OpenAI account or an Amazon Web Services account. Your account will need to have a positive balance for this to work (check your OpenAI balance). CWhy currently defaults to GPT-4, and falls back to GPT-3.5-turbo if a request error occurs. For the newest and best model (GPT-4... -
21
Dockerized Nerd Fonts Patcher
Dockerized Nerd Fonts patcher
... the provided FontForge Python Script to patch your own font or to generate millions of unique combinations/variations. -
22
Marvin
A batteries-included library for building AI-powered software
Meet Marvin: a batteries-included library for building AI-powered software. Marvin's job is to integrate AI directly into your codebase by making it look and feel like any other function. Marvin introduces a new concept called AI Functions. These functions differ from conventional ones in that they don’t rely on source code, but instead generate their outputs on-demand through AI. With AI functions, you don't have to write complex code for tasks like extracting entities from web pages, scoring... -
23
wger
Self hosted FLOSS fitness/workout, nutrition and weight tracker
wger Workout Manager is a free and open web application that manages your exercises, routines and nutrition. It started out as a personal project to replace my growing collection of spreadsheets but has turned into something that other people may find useful. You can create and manage flexible training routines for whatever goals you have. Select exactly what exercises you are going to do and how many repetitions, time or distance you want to do. You can also combine different workouts... -
24
Synapse
Matrix reference homeserver
Matrix is an ambitious new ecosystem for open federated Instant Messaging and VoIP. Everything in Matrix happens in a room. Rooms are distributed and do not exist on any single server. Rooms can be located using convenience aliases like #matrix:matrix.org or #test:localhost:8448. Synapse is currently in rapid development, but as of version 0.5 we believe it is sufficiently stable to be run as an internet-facing service for real usage! Create and manage fully distributed chat rooms with no... -
25
iCn3D
Web-based protein structure viewer and analysis tool
"I see in 3D" (iCn3D) Structure Viewer is not only a web-based 3D viewer, but also a structure analysis tool interactively or in the batch mode using NodeJS scripts based on the npm package icn3d. iCn3D synchronizes the display of 3D structure, 2D interaction, and 1D sequences and annotations. Users' custom displays can be saved in a short URL or a PNG image. The complete package of iCn3D including Three.js and jQuery is in the directory "dist" after you get the source code with the "Code...
