Search Results for "scrapy-redis"
Sort By:
Showing 315 open source projects for "scrapy-redis"
View related business solutions-
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
1
Scrapy-Redis
Redis-based components for Scrapy
...Version 0.3 changed the requests serialization from marshal to cPickle, therefore persisted requests using version 0.2 will not able to work on 0.3. The class scrapy_redis.spiders.RedisSpider enables a spider to read the urls from redis. The urls in the redis queue will be processed one after another, if the first request yields more requests, the spider will process those requests before fetching another url from redis. -
2
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring and automated testing. -
3
Redis
An in-memory database that persists on disk
Redis is an open-source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster. ... -
4
redis-py
Redis Python client
redis-py is the official Python client for interacting with Redis, the in-memory data structure store. It supports all Redis commands and data types, making it easy to build caching, messaging, or real-time analytics features in Python applications. With both synchronous and asyncio support, redis-py is suited for modern Python projects and integrates smoothly into web frameworks, task queues, and backend services. -
Enterprise-grade ITSM, for every businessFreshservice is an intuitive, AI-powered platform that helps IT, operations, and business teams deliver exceptional service without the usual complexity. Automate repetitive tasks, resolve issues faster, and provide seamless support across the organization. From managing incidents and assets to driving smarter decisions, Freshservice makes it easy to stay efficient and scale with confidence.
-
5
redis-rs
Redis library for rust
Redis-rs is a Rust library for interacting with Redis databases, offering high-performance data manipulation capabilities for Rust developers. -
6
Redis Store
Cache Redis stores for Ruby web frameworks
Redis Store provides a full set of stores (Cache, I18n, Session, HTTP Cache) for modern Ruby frameworks like: Ruby on Rails, Sinatra, Rack, Rack::Cache and I18n. It supports object marshaling, timeouts, single or multiple nodes, and namespaces. -
7
Node Redis
A high performance Node.js Redis client
This library is a 1 to 1 mapping of the Redis commands. Each Redis command is exposed as a function on the client object. All functions take either an args Array plus optional callback Function or a variable number of individual arguments followed by an optional callback. Node Redis currently doesn't natively support promises (this is coming in v4), however you can wrap the methods you want to use with promises using the built-in Node.js util.promisify method on Node.js >= v8. ... -
8
Spring Data Redis
Provides support to increase developer productivity in Java
Redis Sentinel and Redis Cluster support. Reactive API using the Lettuce driver. JDK, String, JSON and Spring Object/X. -
9
graphql-redis-subscriptions
A graphql subscriptions implementation using redis
...Now, the GraphQL engine knows that somethingChanged is a subscription, and every time we will use pubsub.publish over this topic, the RedisPubSub will PUBLISH the event over redis to all other subscribed instances and those in their turn will emit the event to GraphQL using the next callback. -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
10
Another Redis Desktop Manager
A faster, better and more stable Redis desktop manager
...Built-in monitoring lets you watch stats, slow logs, and command activity while an integrated console executes raw Redis commands. Quality-of-life features include JSON viewers, search and filter tools, favorite connections, and dark mode. For everyday operations and troubleshooting, it offers a friendlier alternative to the command line without hiding Redis’s power. -
11
Predis
A flexible and feature-complete Redis/Valkey client for PHP
Predis is a flexible and feature-rich PHP client library for Redis. It offers a simple interface to interact with Redis servers and supports a wide variety of Redis commands. Predis is known for its ease of use, performance, and its ability to handle Redis clusters with minimal configuration. It can be used with any PHP-based application that requires caching or data storage in Redis. -
12
Kvrocks
NoSQL database that uses RocksDB as storage engine
Apache Kvrocks(Incubating) is a distributed key-value NoSQL database that uses RocksDB as storage engine and is compatible with Redis protocol. Kvrocks intends to decrease the cost of memory and increase the capacity while compared to Redis. The design of replication and storage was inspired by rocksplicator and blackwidow. Redis protocol, user can use redis client to visit the kvrocks. Namespace, similar to redis db but use token per namespace. Replication, async replication using binlog like MySQL. ... -
13
phpRedisAdmin
Simple web interface to manage Redis databases
phpRedisAdmin is a web-based administration tool for managing Redis databases. It provides a user-friendly interface for viewing, editing, and managing Redis keys and data. The tool is lightweight and easy to set up, making it ideal for developers and administrators who need quick and efficient access to their Redis instances. It supports multiple Redis servers and offers basic data manipulation features directly from the browser. -
14
Carmine
Redis client and message queue for Clojure
...Simple, high-performance message queue (v2+, Redis 2.6+). Simple, high-performance distributed lock (v2+, Redis 2.6+). Pluggable compression and encryption support (v2+). Includes Tundra, an API for replicating data to an additional datastore (v2+, Redis 2.6+). You'll usually want to define a single connection pool, and one connection spec for each of your Redis servers. Note that executing multiple Redis commands in a single wcar request uses efficient Redis pipelining under the hood, and returns a pipeline reply (vector) for easy destructuring, etc. -
15
Pottery
Redis for humans
Redis is awesome, but Redis commands are not always intuitive. Pottery is a Pythonic way to access Redis. If you know how to use Python dicts, then you already know how to use Pottery. Pottery is useful for accessing Redis more easily, and also for implementing microservice resilience patterns, and it has been battle-tested in production at scale. -
16
Prometheus Redis Metrics Exporter
Prometheus Exporter for Redis Metrics. Supports Redis 2.x, 3.x, 4.x, 5
...If authentication is needed for the Redis instances then you can set the password via the --redis.password command line option of the exporter (this means you can currently only use one password across the instances you try to scrape this way. Use several exporters if this is a problem). If your Redis instance requires authentication then there are several ways how you can supply a username (new in Redis 6.x with ACLs) and a password. -
17
Redisson
Valkey & Redis Java client. Real-Time Data Platform
Redisson is a Java client library for Redis that offers distributed data structures, services, and frameworks to build scalable and reliable applications. It simplifies Redis usage by providing in-memory Java objects like maps, sets, locks, queues, and semaphores that are backed by Redis. Redisson supports advanced features like distributed locking, asynchronous APIs, and integrates with frameworks like Spring and Quarkus for reactive and cloud-native development. -
18
Lettuce
Advanced Java Redis client for thread-safe sync, async, etc.
...Lettuce comes with an API that gets you started quickly. Its simple yet powerful programming model allows you for trivial use-cases as well as for chained asynchronous flows. Redis Standalone, Master/Slave, Redis Sentinel and Redis Cluster. Lettuce connects with all operational models natively supported by Redis. Partition-tolerance, Read Slaves and Transport-Level-Security provide the required foundation for highly scalable applications. -
19
Jedis
A small redis java client
Redis is primarily built for master/slave distribution. This means that write requests have to be explicitly addressed to the master (a redis server), which replicates changes to slaves (which are also redis servers). -
20
PikiwiDB
Pika is a Redis-Compatible database
A high-performance, distributed key-value database inspired by Redis but designed for large-scale cloud-native environments. -
21
Redigo
Go client for Redis
Redigo is a Go client for the Redis database. A Print-like API with support for all Redis commands. Pipelining, including pipelined transactions. Publish/Subscribe. Connection pooling. Script helper type with optimistic use of EVALSHA. Helper functions for working with command replies. The Conn interface is the primary interface for working with Redis. Applications create connections by calling the Dial, DialWithTimeout or NewConn functions. -
22
ioredis
A performance-focused Redis client for Node.js
ioredis is a robust, full-featured Redis client that is used in the world's biggest online commerce company Alibaba and many other awesome companies. Full-featured. It supports Cluster, Sentinel, Streams, Pipelining and of course Lua scripting & Pub/Sub (with the support of binary messages). High performance. Delightful API. It works with Node callbacks and Native promises. -
23
Django RQ
A simple app that provides django integration for RQ
...If you have django-redis or django-redis-cache installed, you can instruct django_rq to use the same connection information from your Redis cache. This has two advantages, it's DRY and it takes advantage of any optimization that may be going on in your cache setup (like using connection pooling or Hiredis.) -
24
Miniredis
Pure Go Redis server for Go unittests
Sometimes you want to test code which uses Redis, without making it a full-blown integration test. Miniredis implements (parts of) the Redis server, to be used in unittests. It enables a simple, cheap, in-memory, Redis replacement, with a real TCP interface. It saves you from using mock code, and since the redis server lives in the test process you can query for values directly, without going through the server stack. -
25
Dragonfly
A modern replacement for Redis and Memcached
Dragonfly is a drop-in Redis replacement that cuts costs and boosts performance. Designed to fully utilize the power of modern cloud hardware and deliver on the data demands of modern applications, Dragonfly frees developers from the limits of traditional in-memory data stores. The power of modern cloud hardware can never be realized with legacy software. Dragonfly is optimized for modern cloud computing, delivering 25x more throughput and 12x lower snapshotting latency when compared to legacy in-memory data stores like Redis, making it easy to deliver the real-time experience your customers expect. ...
