Search Results for "scrapy-redis"
Sort By:
Showing 106 open source projects for "scrapy-redis"
View related business solutions-
Computing infrastructure in predefined or custom machine sizes to accelerate your cloud transformation. General purpose (E2, N1, N2, N2D) machines provide a good balance of price and performance. Compute optimized (C2) machines offer high-end vCPU performance for compute-intensive workloads. Memory optimized (M2) machines offer the highest memory and are great for in-memory databases. Accelerator optimized (A2) machines are based on the A100 GPU, for very demanding applications.
-
Whether natural disaster, cyberattack, or plain-old human error, data can disappear in the blink of an eye. ConnectWise BCDR (formerly Recover) delivers reliable and secure backup and disaster recovery backed by powerful automation and a 24/7 NOC to get your clients back to work in minutes, not days.
-
1
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring... -
2
Redis Store
Cache Redis stores for Ruby web frameworks
Redis Store provides a full set of stores (Cache, I18n, Session, HTTP Cache) for modern Ruby frameworks like: Ruby on Rails, Sinatra, Rack, Rack::Cache and I18n. It supports object marshaling, timeouts, single or multiple nodes, and namespaces. -
3
redis-operator
Redis Operator creates/configures/manages high availability redis
Redis Operator creates/configures/manages redis-failovers atop Kubernetes. Kubernetes version: 1.21 or higher Redis version: 6 or higher. Redis operator is being tested against Kubernetes 1.25 1.26 1.27 and redis 6 All dependencies have been vendored, so there's no need to any additional downloads. -
4
Node Redis
A high performance Node.js Redis client
This library is a 1 to 1 mapping of the Redis commands. Each Redis command is exposed as a function on the client object. All functions take either an args Array plus optional callback Function or a variable number of individual arguments followed by an optional callback. Node Redis currently doesn't natively support promises (this is coming in v4), however you can wrap the methods you want to use with promises using the built-in Node.js util.promisify method on Node.js >= v8. Care should... -
New customers get $300 in free credits to spend on Speech-to-Text. All customers get 60 minutes for transcribing and analyzing audio free per month, not charged against your credits.
-
5
Prometheus Redis Metrics Exporter
Prometheus Exporter for Redis Metrics. Supports Redis 2.x, 3.x, 4.x, 5
Prometheus exporter for Redis metrics. Supports Redis 2.x, 3.x, 4.x, 5.x, 6.x, and 7.x. To have instances in the drop-down as human readable names rather than IPs, it is suggested to use instance relabelling. The Prometheus docs have a very informative article on how multi-target exporters are intended to work. Run the exporter with the command line flag --redis.addr= so it won't try to access the local instance every time the /metrics endpoint is scraped. If authentication is needed... -
6
graphql-redis-subscriptions
A graphql subscriptions implementation using redis
This package implements the PubSubEngine Interface from the graphql-subscriptions package and also the new AsyncIterator interface. It allows you to connect your subscriptions manager to a Redis Pub Sub mechanism to support multiple subscription manager instances. As the graphql-subscriptions package is declared as a peer dependency, you might receive a warning about an unmet peer dependency if it's not installed already by one of your other packages. Calling the method asyncIterator... -
7
Spring Data Redis
Provides support to increase developer productivity in Java
Provides support to increase developer productivity in Java when using Redis, a key-value store. Uses familiar Spring concepts such as a template class for core API usage and lightweight repository-style data access. The primary goal of the Spring Data project is to make it easier to build Spring-powered applications that use new data access technologies such as non-relational databases, map-reduce frameworks, and cloud-based data services. Connection package as low-level abstraction across... -
8
Gerapy
Distributed Crawler Management Framework Based on Scrapy
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js. Someone who has worked as a crawler with Python may use Scrapy. Scrapy is indeed a very powerful crawler framework. It has high crawling efficiency and good scalability. It is basically a necessary tool for developing crawlers using Python. If you use Scrapy as a crawler, then of course we can use our own host to crawl when crawling, but when the crawl is very large, we can’t run... -
9
Lettuce
Advanced Java Redis client for thread-safe sync, async, etc.
Build elastic data access. Lettuce is a scalable Redis client for building non-blocking Reactive applications. Lettuce is a fully non-blocking Redis client built with netty providing Reactive, Asynchronous and Synchronous Data Access. Lettuce provides asynchronous API with RedisFuture (CompletionStage) and Reactive types Flux [N] and Mono [0|1]. Low-latency communication, backpressure-enabled network engine for NIO TCP, epoll TCP and Unix Domain Sockets. Reactive Streaming is fully supported... -
Deliver a personalized customer experience with every interaction, across every channel, with uContact, net2phone’s cloud contact center solution.
-
10
Acl
A powerful server and network library, including coroutine
...//MQTT/Redis/Memcached/Beanstalk/Handler Socket are implemented in Acl, and the codec library such as XML/JSON/MIME/BASE64/UUCODE/QPCODE/RFC2047/RFC1035, etc., are also included in Acl. Acl also provides a unified abstract interface for popular databases such as Mysql, Postgresql, Sqlite. Using Acl library users can write database applications more easily, quickly and safely. -
11
ioredis
A performance-focused Redis client for Node.js
ioredis is a robust, full-featured Redis client that is used in the world's biggest online commerce company Alibaba and many other awesome companies. Full-featured. It supports Cluster, Sentinel, Streams, Pipelining and of course Lua scripting & Pub/Sub (with the support of binary messages). High performance. Delightful API. It works with Node callbacks and Native promises. Transformation of command arguments and replies. Transparent key prefixing. Abstraction for Lua scripting, allowing you... -
12
RDM
Cross-platform GUI management tool for Redis
Cross-platform open source database management tool for Redis. RDM is easy-to-use GUI tool available for Windows, Linux, MacOS and iPadOS. Analyse your Redis ® memory usage with RDM and remove obsolete data with bulk removal. RDM supports all latest Redis ® features: ACL, Streams, Cluster, Sentinel, ReJSON module, HyperLogLog, etc. Build-in TLS, SSH and TLS-over-SSH tunneling for easy and secure access to any redis-server. RDM works with Amazon ElastiCache, Microsoft Azure Redis Cache, Digital... -
13
WalletConnect v2.x.x
WalletConnect Monorepo
... python cli is linked (required to build some npm modules). Install workspace dependencies i.e. run npm install from root folder. Install redis. We recommend running it as a brew service. Pull and start ts-relay server (separate repo) Ensure everything runs correctly by executing npm run check. -
14
DDEV
Docker-based local PHP+Node.js web development environments
Container superpowers with zero required Docker skills: environments in minutes, multiple concurrent projects, and less time to deployment. Fast spinup and easy sharing let everyone skip the devops and get back to the code. Use the exact same setup across machines, team members, and even hosted environments. Focus on the software you need for each project without installing anything on your workstation. Agencies, enterprise businesses, and open source communities have trusted and shaped... -
15
Gorse Recommender System Engine
An open source recommender system service written in Go
An open-source recommender system service written in Go. Recommend items from Popular, latest, user-based, item-based and collaborative filtering. Search the best recommendation model automatically in the background. Support horizontal scaling in the recommendation stage after single node training. Support Redis, MySQL, Postgres, MongoDB, and ClickHouse as its storage backend. Expose RESTful APIs for data CRUD and recommendation requests. Analyze online recommendation performance from recently... -
16
POCO
Cross-platform C++ libraries for building network applications
The POCO C++ Libraries are powerful cross-platform C++ libraries for building network- and internet-based applications that run on desktop, server, mobile, IoT, and embedded systems. Whether building automation systems, industrial automation, IoT platforms, air traffic management systems, enterprise IT application and infrastructure management, security and network analytics, automotive infotainment and telematics, financial or healthcare, C++ developers have been trusting the POCO C++... -
17
jackal
Instant messaging server for the Extensible Messaging
jackal is a free, open-source, high-performance XMPP server that aims to be known for its stability, simple configuration and low resource consumption. Customizable, enforced SSL/TLS. Stream compression (zlib), database connectivity for storing offline messages and user settings (PostgreSQL 9.5+, BoltDB). Caching (Redis 6.2+), clustering capabilities (etcd 3.4+) Expose Prometheus metrics, cross-platform (OS X, Linux). To make it easy to install jackal via Helm in Kubernetes a chart has been... -
18
Laravel Horizon
Dashboard and code-driven configuration for Laravel queues
Horizon provides a beautiful dashboard and code-driven configuration for your Laravel-powered Redis queues. Horizon allows you to easily monitor key metrics of your queue system such as job throughput, runtime, and job failures. All of your worker configuration is stored in a single, simple configuration file, allowing your configuration to stay in source control where your entire team can collaborate. When using Horizon, all of your queue worker configuration is stored in a single, simple... -
19
Redigo
Go client for Redis
Redigo is a Go client for the Redis database. A Print-like API with support for all Redis commands. Pipelining, including pipelined transactions. Publish/Subscribe. Connection pooling. Script helper type with optimistic use of EVALSHA. Helper functions for working with command replies. The Conn interface is the primary interface for working with Redis. Applications create connections by calling the Dial, DialWithTimeout or NewConn functions. In the future, functions will be added for creating... -
20
Log Viewer
Fast and beautiful Log Viewer for Laravel
Beautiful, fast, and open-source viewer for your app logs. Forget scrolling through raw log files to find the errors or debug logs you're looking for. Forget about SSH'ing onto the production server just to read the logs. Log Viewer helps you quickly and clearly see individual log entries, to search, filter, and make sense of your Laravel logs fast. It is free and easy to install. -
21
Openblocks
The Open Source Retool Alternative
It's cumbersome to create a single app. You had to design user interfaces, write code in multiple languages and frameworks, and understand how all of that code works together. Low-code/No-code platforms are fast to get started with but quickly become unmaintainable and inflexible. This creates more problems than it solves. Retool-like solutions are great for their simplicity and flexibility, but they can also be limited in different ways compared to frameworks like React/Vue. An all-in-one... -
22
Bifrost
Middleware for production-oriented MySQL
Heterogeneous middleware for production-oriented MySQL, MariaDB, kafka synchronization to Redis, MongoDB, ClickHouse and other services. And this Bifrost can synchronize the full amount of data to multiple targets in real time. Support MySQL, MariaDB all storage types. Interface-based dynamic and flexible configuration of different data tables and target libraries. Multiple data sources, multiple target library support. Both incremental and full data synchronization are supported. One Binlog... -
23
Nebulex
In-memory and distributed caching toolkit for Elixir
Nebulex provides support for transparently adding caching into an existing Elixir application. Similar to Ecto, the caching abstraction allows consistent use of various caching solutions with minimal impact on the code. Nebulex cache abstraction shields developers from directly dealing with the underlying caching implementations, such as Redis, Memcached, or even other Elixir cache implementations like Cachex. Additionally, it provides totally out-of-box features such as cache usage patterns... -
24
Codeception
Full-stack testing PHP framework
... be executed inside a PHP framework. This way web applications can be executed without web servers to run faster and access application internals. Symfony, Laravel, Zend Framework, Yii, Phalcon are supported. Connect to various data sources MySQL, PostgreSQL, MongoDB, Redis, Memcache to use their data inside tests. Take snapshots to compare data changes with previous runs. Codeception is built on top of PHPUnit and is able to execute its tests. -
25
ProxyPool
An Efficient ProxyPool with Getter, Tester and Server
Simple and efficient proxy pool, providing the following functions. Regularly crawl free proxy websites, easy and scalable. Use Redis to store brokers and sort broker availability. Regular testing and screening to eliminate unavailable proxies and leave available proxies. Provides a proxy API to randomly select available proxies that pass the test. The principle analysis of the proxy pool can be seen in " How to Build an Efficient Proxy Pool ". It is recommended to read it before using...