Search Results for "scrapy-redis"
Sort By:
Showing 112 open source projects for "scrapy-redis"
View related business solutions-
HubSpot is an AI-powered customer platform with all the software, integrations, and resources you need to connect your marketing, sales, and customer service. HubSpot's connected platform enables you to grow your business faster by focusing on what matters most: your customers.
-
Manage your enterprise’s compensation lifecycle and accurately recognize top performers with a digitized, integrated system. Keep employees invested and your HR team in control while preventing compensation chaos.
-
1
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring... -
2
Redis Store
Cache Redis stores for Ruby web frameworks
Redis Store provides a full set of stores (Cache, I18n, Session, HTTP Cache) for modern Ruby frameworks like: Ruby on Rails, Sinatra, Rack, Rack::Cache and I18n. It supports object marshaling, timeouts, single or multiple nodes, and namespaces. -
3
redis-operator
Redis Operator creates/configures/manages high availability redis
Redis Operator creates/configures/manages redis-failovers atop Kubernetes. Kubernetes version: 1.21 or higher Redis version: 6 or higher. Redis operator is being tested against Kubernetes 1.25 1.26 1.27 and redis 6 All dependencies have been vendored, so there's no need to any additional downloads. -
4
Node Redis
A high performance Node.js Redis client
This library is a 1 to 1 mapping of the Redis commands. Each Redis command is exposed as a function on the client object. All functions take either an args Array plus optional callback Function or a variable number of individual arguments followed by an optional callback. Node Redis currently doesn't natively support promises (this is coming in v4), however you can wrap the methods you want to use with promises using the built-in Node.js util.promisify method on Node.js >= v8. Care should... -
With the world of work changed forever, it’s essential to manage your workplace and assets together to effectively create a high-performing environment. The Eptura experience combines the power of workplace management software with asset management, enabling you to effectively operate your building and facilitate hybrid work.
-
5
graphql-redis-subscriptions
A graphql subscriptions implementation using redis
This package implements the PubSubEngine Interface from the graphql-subscriptions package and also the new AsyncIterator interface. It allows you to connect your subscriptions manager to a Redis Pub Sub mechanism to support multiple subscription manager instances. As the graphql-subscriptions package is declared as a peer dependency, you might receive a warning about an unmet peer dependency if it's not installed already by one of your other packages. Calling the method asyncIterator... -
6
Prometheus Redis Metrics Exporter
Prometheus Exporter for Redis Metrics. Supports Redis 2.x, 3.x, 4.x, 5
Prometheus exporter for Redis metrics. Supports Redis 2.x, 3.x, 4.x, 5.x, 6.x, and 7.x. To have instances in the drop-down as human readable names rather than IPs, it is suggested to use instance relabelling. The Prometheus docs have a very informative article on how multi-target exporters are intended to work. Run the exporter with the command line flag --redis.addr= so it won't try to access the local instance every time the /metrics endpoint is scraped. If authentication is needed... -
7
Spring Data Redis
Provides support to increase developer productivity in Java
Provides support to increase developer productivity in Java when using Redis, a key-value store. Uses familiar Spring concepts such as a template class for core API usage and lightweight repository-style data access. The primary goal of the Spring Data project is to make it easier to build Spring-powered applications that use new data access technologies such as non-relational databases, map-reduce frameworks, and cloud-based data services. Connection package as low-level abstraction across... -
8
RDM
Cross-platform GUI management tool for Redis
Cross-platform open source database management tool for Redis. RDM is easy-to-use GUI tool available for Windows, Linux, MacOS and iPadOS. Analyse your Redis ® memory usage with RDM and remove obsolete data with bulk removal. RDM supports all latest Redis ® features: ACL, Streams, Cluster, Sentinel, ReJSON module, HyperLogLog, etc. Build-in TLS, SSH and TLS-over-SSH tunneling for easy and secure access to any redis-server. RDM works with Amazon ElastiCache, Microsoft Azure Redis Cache, Digital... -
9
ProxyPool
An Efficient ProxyPool with Getter, Tester and Server
Simple and efficient proxy pool, providing the following functions. Regularly crawl free proxy websites, easy and scalable. Use Redis to store brokers and sort broker availability. Regular testing and screening to eliminate unavailable proxies and leave available proxies. Provides a proxy API to randomly select available proxies that pass the test. The principle analysis of the proxy pool can be seen in " How to Build an Efficient Proxy Pool ". It is recommended to read it before using... -
When it comes to cybersecurity, what your clients don’t know can really hurt them. And believe it or not, keep them safe starts with asking questions. With ConnectWise Identify Assessment, get access to risk assessment backed by the NIST Cybersecurity Framework to uncover risks across your client’s entire business, not just their networks. With a clearly defined, easy-to-read risk report in hand, you can start having meaningful security conversations that can get you on the path of keeping your clients protected from every angle. Choose from two assessment levels to cover every client’s need, from the Essentials to cover the basics to our Comprehensive Assessment to dive deeper to uncover additional risks. Our intuitive heat map shows you your client’s overall risk level and priority to address risks based on probability and financial impact. Each report includes remediation recommendations to help you create a revenue-generating action plan.
-
10
Bull
Queue package for handling distributed jobs and messages in NodeJS
The fastest, most reliable, Redis-based queue for Node. Carefully written for rock solid stability and atomicity. If you want to start using the next major version of Bull written entirely in Typescript you are welcome to the new repo here. Otherwise you are very welcome to still use Bull, which is a safe, battle tested codebase. If you need high quality production Redis instances for your Bull projects, please consider subscribing to RedisGreen, leaders in Redis hosting that works perfectly... -
11
Gerapy
Distributed Crawler Management Framework Based on Scrapy
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js. Someone who has worked as a crawler with Python may use Scrapy. Scrapy is indeed a very powerful crawler framework. It has high crawling efficiency and good scalability. It is basically a necessary tool for developing crawlers using Python. If you use Scrapy as a crawler, then of course we can use our own host to crawl when crawling, but when the crawl is very large, we can’t run... -
12
Ccache
A fast compiler cache
.... using Nginx or Google Cloud Storage), Redis or local filesystem, optionally sharding data onto a server cluster. Supports fast "direct" and "depend" modes that don't rely on using the preprocessor. Supports compression using Zstandard. Checksums cache content using XXH3 to detect data corruption. Keeps statistics on hits/misses. Automatic cache size management. Easy installation. -
13
ioredis
A performance-focused Redis client for Node.js
ioredis is a robust, full-featured Redis client that is used in the world's biggest online commerce company Alibaba and many other awesome companies. Full-featured. It supports Cluster, Sentinel, Streams, Pipelining and of course Lua scripting & Pub/Sub (with the support of binary messages). High performance. Delightful API. It works with Node callbacks and Native promises. Transformation of command arguments and replies. Transparent key prefixing. Abstraction for Lua scripting, allowing you... -
14
POCO
Cross-platform C++ libraries for building network applications
The POCO C++ Libraries are powerful cross-platform C++ libraries for building network- and internet-based applications that run on desktop, server, mobile, IoT, and embedded systems. Whether building automation systems, industrial automation, IoT platforms, air traffic management systems, enterprise IT application and infrastructure management, security and network analytics, automotive infotainment and telematics, financial or healthcare, C++ developers have been trusting the POCO C++... -
15
Acl
A powerful server and network library, including coroutine
...//MQTT/Redis/Memcached/Beanstalk/Handler Socket are implemented in Acl, and the codec library such as XML/JSON/MIME/BASE64/UUCODE/QPCODE/RFC2047/RFC1035, etc., are also included in Acl. Acl also provides a unified abstract interface for popular databases such as Mysql, Postgresql, Sqlite. Using Acl library users can write database applications more easily, quickly and safely. -
16
DDEV
Docker-based local PHP+Node.js web development environments
Container superpowers with zero required Docker skills: environments in minutes, multiple concurrent projects, and less time to deployment. Fast spinup and easy sharing let everyone skip the devops and get back to the code. Use the exact same setup across machines, team members, and even hosted environments. Focus on the software you need for each project without installing anything on your workstation. Agencies, enterprise businesses, and open source communities have trusted and shaped... -
17
Centrifugo
Scalable real-time messaging server in a language-agnostic way
Centrifugo is language-agnostic. It's a standalone server with a simple API that integrates well with an application written in any programming language. No need to change an existing application architecture to introduce real-time features. Just add Centrifugo nearby and let it deal with persistent connections. Centrifugo is built in Go language with some smart optimizations inside. It has good performance – see a description of a test stand with one million WebSocket connections and 30... -
18
Lettuce
Advanced Java Redis client for thread-safe sync, async, etc.
Build elastic data access. Lettuce is a scalable Redis client for building non-blocking Reactive applications. Lettuce is a fully non-blocking Redis client built with netty providing Reactive, Asynchronous and Synchronous Data Access. Lettuce provides asynchronous API with RedisFuture (CompletionStage) and Reactive types Flux [N] and Mono [0|1]. Low-latency communication, backpressure-enabled network engine for NIO TCP, epoll TCP and Unix Domain Sockets. Reactive Streaming is fully supported... -
19
Redigo
Go client for Redis
Redigo is a Go client for the Redis database. A Print-like API with support for all Redis commands. Pipelining, including pipelined transactions. Publish/Subscribe. Connection pooling. Script helper type with optimistic use of EVALSHA. Helper functions for working with command replies. The Conn interface is the primary interface for working with Redis. Applications create connections by calling the Dial, DialWithTimeout or NewConn functions. In the future, functions will be added for creating... -
20
PHP-Casbin
An authorization library that supports access control models
.... Besides memory and file, Casbin policy can be stored into lots of places. Currently, dozens of databases are supported, from MySQL, Postgres, Oracle to MongoDB, Redis, Cassandra, AWS S3. Check the full supported list at: adapters. Casbin is implemented in Golang, Java, PHP and Node.js. All implementations share the same API and behaviors. You can learn Casbin once and use it everywhere. -
21
WalletConnect v2.x.x
WalletConnect Monorepo
... python cli is linked (required to build some npm modules). Install workspace dependencies i.e. run npm install from root folder. Install redis. We recommend running it as a brew service. Pull and start ts-relay server (separate repo) Ensure everything runs correctly by executing npm run check. -
22
Tyk API Gateway
Open Source API Gateway written in Go
... in the clouds as well as exposing APIs to teams, partners & consumers. Built from the ground up to be the fastest API gateway on the planet. It does not depend on a legacy proxy underneath. It has no 3rd party dependencies aside from Redis for distributed rate-limiting and token storage. Tyk Gateway can also be deployed as part of a larger Full Lifecycle API Management platform Tyk Self-Managed which also includes Management Control Plane, Dashboard GUI and Developer Portal. -
23
EasyEngine
Command-line control panel for Nginx Server to manage WordPress sites
EasyEngine makes it greatly easy to manage nginx, a fast web-server software that consumes little memory when handling increasing volumes of concurrent users. -
24
NodePress
RESTful API service
RESTful API service for surmon.me blog, powered by NestJS, required MongoDB & Redis. -
25
Miniredis
Pure Go Redis server for Go unittests
Sometimes you want to test code which uses Redis, without making it a full-blown integration test. Miniredis implements (parts of) the Redis server, to be used in unittests. It enables a simple, cheap, in-memory, Redis replacement, with a real TCP interface. It saves you from using mock code, and since the redis server lives in the test process you can query for values directly, without going through the server stack. There are no dependencies on external binaries, so you can easily integrate...