Search Results for "scrapy-redis"
Sort By:
Showing 39 open source projects for "scrapy-redis"
View related business solutions-
Manage your enterprise’s compensation lifecycle and accurately recognize top performers with a digitized, integrated system. Keep employees invested and your HR team in control while preventing compensation chaos.
-
The Vivantio Platform allows you to focus on the IT service management tools that make sense for your organization’s unique service model: from incident, problem and change requests, to service requests, client knowledge and asset management
-
1
redis-operator
Redis Operator creates/configures/manages high availability redis
Redis Operator creates/configures/manages redis-failovers atop Kubernetes. Kubernetes version: 1.21 or higher Redis version: 6 or higher. Redis operator is being tested against Kubernetes 1.25 1.26 1.27 and redis 6 All dependencies have been vendored, so there's no need to any additional downloads. -
2
Prometheus Redis Metrics Exporter
Prometheus Exporter for Redis Metrics. Supports Redis 2.x, 3.x, 4.x, 5
Prometheus exporter for Redis metrics. Supports Redis 2.x, 3.x, 4.x, 5.x, 6.x, and 7.x. To have instances in the drop-down as human readable names rather than IPs, it is suggested to use instance relabelling. The Prometheus docs have a very informative article on how multi-target exporters are intended to work. Run the exporter with the command line flag --redis.addr= so it won't try to access the local instance every time the /metrics endpoint is scraped. If authentication is needed... -
3
OAuth2 Proxy
A reverse proxy that provides authentication with Google, Azure, etc.
A reverse proxy and static file server that provides authentication using Providers (Google, GitHub, and others) to validate accounts by email, domain or group. If you are running a version older than v6.0.0 we strongly recommend you please update to the current version. After returning from the authentication provider, the OAuth tokens are stored in the configured session store (cookie, redis, ...) and a cookie is set. The request is forwarded to the upstream server with added user info... -
4
DDEV
Docker-based local PHP+Node.js web development environments
Container superpowers with zero required Docker skills: environments in minutes, multiple concurrent projects, and less time to deployment. Fast spinup and easy sharing let everyone skip the devops and get back to the code. Use the exact same setup across machines, team members, and even hosted environments. Focus on the software you need for each project without installing anything on your workstation. Agencies, enterprise businesses, and open source communities have trusted and shaped... -
Don’t settle when it comes to managing your clients’ IT infrastructure. Exceed their expectations with ConnectWise RMM, our MSP RMM software that provides proactive tools and NOC services—regardless of device environment. With the number of new vulnerabilities rising each year, smart patching procedures have never been more important. We automatically test and deploy patches when they are viable and restrict patches that are harmful. Get better protection for clients while you spend less time managing endpoints and more time growing your business. It’s tough to locate, afford, and retain quality talent. In fact, 81% of IT leaders say it’s hard to find the recruits they need. Add ConnectWise RMM, NOC services and get the expertise and problem resolution you need to become the advisor your clients demand—without adding headcount.
-
5
Gorse Recommender System Engine
An open source recommender system service written in Go
An open-source recommender system service written in Go. Recommend items from Popular, latest, user-based, item-based and collaborative filtering. Search the best recommendation model automatically in the background. Support horizontal scaling in the recommendation stage after single node training. Support Redis, MySQL, Postgres, MongoDB, and ClickHouse as its storage backend. Expose RESTful APIs for data CRUD and recommendation requests. Analyze online recommendation performance from recently... -
6
jackal
Instant messaging server for the Extensible Messaging
jackal is a free, open-source, high-performance XMPP server that aims to be known for its stability, simple configuration and low resource consumption. Customizable, enforced SSL/TLS. Stream compression (zlib), database connectivity for storing offline messages and user settings (PostgreSQL 9.5+, BoltDB). Caching (Redis 6.2+), clustering capabilities (etcd 3.4+) Expose Prometheus metrics, cross-platform (OS X, Linux). To make it easy to install jackal via Helm in Kubernetes a chart has been... -
7
Redigo
Go client for Redis
Redigo is a Go client for the Redis database. A Print-like API with support for all Redis commands. Pipelining, including pipelined transactions. Publish/Subscribe. Connection pooling. Script helper type with optimistic use of EVALSHA. Helper functions for working with command replies. The Conn interface is the primary interface for working with Redis. Applications create connections by calling the Dial, DialWithTimeout or NewConn functions. In the future, functions will be added for creating... -
8
Bifrost
Middleware for production-oriented MySQL
Heterogeneous middleware for production-oriented MySQL, MariaDB, kafka synchronization to Redis, MongoDB, ClickHouse and other services. And this Bifrost can synchronize the full amount of data to multiple targets in real time. Support MySQL, MariaDB all storage types. Interface-based dynamic and flexible configuration of different data tables and target libraries. Multiple data sources, multiple target library support. Both incremental and full data synchronization are supported. One Binlog... -
9
WAL-G
Archival and Restoration for databases in the Cloud
WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis). WAL-G is the successor of WAL-E with a number of key differences. WAL-G uses LZ4, LZMA, ZSTD, or Brotli compression, multiple processors, and non-exclusive base backups for Postgres. More information on the original design and implementation of WAL-G can be found on the Citus Data blog post "Introducing WAL-G by Citus: Faster Disaster Recovery for Postgres". A precompiled binary... -
Track all your service data in one easy-to-use system, enabling your team to move faster and generate more revenue for your bottom line.
-
10
Miniredis
Pure Go Redis server for Go unittests
Sometimes you want to test code which uses Redis, without making it a full-blown integration test. Miniredis implements (parts of) the Redis server, to be used in unittests. It enables a simple, cheap, in-memory, Redis replacement, with a real TCP interface. It saves you from using mock code, and since the redis server lives in the test process you can query for values directly, without going through the server stack. There are no dependencies on external binaries, so you can easily integrate... -
11
Tyk API Gateway
Open Source API Gateway written in Go
... in the clouds as well as exposing APIs to teams, partners & consumers. Built from the ground up to be the fastest API gateway on the planet. It does not depend on a legacy proxy underneath. It has no 3rd party dependencies aside from Redis for distributed rate-limiting and token storage. Tyk Gateway can also be deployed as part of a larger Full Lifecycle API Management platform Tyk Self-Managed which also includes Management Control Plane, Dashboard GUI and Developer Portal. -
12
rosedb
High performance NoSQL database based on bitcask
A high-performance NoSQL database based on bitcask, supports string, list, hash, set, and sorted set. rosedb is a fast, stable, and embedded NoSQL database based on bitcask, that supports a variety of data structures such as string, list, hash, set, and sorted set. Easy to embed into your own Go application. High performance, suitable for both read and write-intensive workloads. Values are not limited by RAM. It is similar to Redis but store values on disk. RoseDB is based on bitcask. -
13
Gocache
Go cache library that brings you multiple ways to manage caches
Guess what is Gocache? a Go cache library. This is an extendable cache library that brings you a lot of features for caching data. Multiple cache stores: actually in memory, redis, or your own custom store. A chain cache: use multiple cache with a priority order (memory then fallback to a redis shared cache for instance) A loadable cache: allow you to call a callback function to put your data back in cache. A metric cache to let you store metrics about your caches usage (hits, miss, set success... -
14
Asynq
Simple, reliable, and efficient distributed task queue in Go
Asynq is a Go library for queueing tasks and processing them asynchronously with workers. It's backed by Redis and is designed to be scalable yet easy to get started. Client puts tasks on a queue. Server pulls tasks off queues and starts a worker goroutine for each task. Tasks are processed concurrently by multiple workers. Task queues are used as a mechanism to distribute work across multiple machines. A system can consist of multiple worker servers and brokers, giving way to high availability... -
15
Crawlab
Distributed web crawler admin platform for spiders management
Golang-based distributed web crawler management platform, supporting various languages including Python, NodeJS, Go, Java, PHP and various web crawler frameworks including Scrapy, Puppeteer, Selenium. Please use docker-compose to one-click to start up. By doing so, you don't even have to configure MongoDB database. The frontend app interacts with the master node, which communicates with other components such as MongoDB, SeaweedFS and worker nodes. Master node and worker nodes communicate... -
16
Godis
A Golang implemented Redis Server and Cluster
Godis is a golang implementation of Redis Server, which intents to provide an example of writing a high concurrent middleware using golang. MULTI Commands Transaction is Atomic and Isolated. If any errors are encountered during execution, godis will rollback the executed commands. Server-side Cluster which is transparent to client. You can connect to any node in the cluster to access all data in the cluster. MSET, MSETNX, DEL, Rename, RenameNX command is supported and atomically executed... -
17
EaseProbe
A simple, standalone, and lightweight tool
... command on a remote host and check the CPU, Memory, and Disk usage. MySQL. Connect to a MySQL server and run the SHOW STATUS SQL. Redis. Connect to a Redis server and run the PING command. Memcache. Connect to a Memcache server and run the version command or validate a given key/value pair. MongoDB. Connect to a MongoDB server and perform a ping. Kafka. Connect to a Kafka server and perform a list of all topics. -
18
Algernon
Small self-contained pure-Go web server with Lua, Markdown, HTTP/2
Web server with built-in support for QUIC, HTTP/2, Lua, Teal, Markdown, Pongo2, HyperApp, Amber, Sass(SCSS), GCSS, JSX, BoltDB (built-in, stores the database in a file, like SQLite), Redis, PostgreSQL, MariaDB/MySQL, rate limiting, graceful shutdown, plugins, users and permissions. Written in Go. Uses Bolt (built-in), MySQL, PostgreSQL or Redis (recommended) for the database backend, permissions2 for handling users and permissions, gopher-Lua for interpreting and running Lua, optional Teal... -
19
gokv
Simple key-value store abstraction and implementations for Go
Simple key-value store abstraction and implementations for Go. Redis, Consul, etcd, bbolt, BadgerDB, LevelDB, Memcached, DynamoDB, S3, PostgreSQL, MongoDB, CockroachDB, and many more. There are detailed descriptions of the methods in the docs and in the code. You should read them if you plan to write your own gokv.Store implementation or if you create a Go package with a method that takes a gokv.Store as a parameter, so you know exactly what happens in the background. -
20
gnet
gnet is a high-performance, lightweight, networking framework
Built in Go, gnet is ultra-fast and memory efficient based on the event-driven mechanism. It's designed to create a networking server framework for Go that performs on par with Redis and Haproxy for networking packets handling. gnet is lock-free during the entire runtime, which keeps gnet free from synchronization issues and speeds it up. gnet provides concise and easy-to-use APIs for users, it only exposes the essential APIs and takes over most of the tough work for users, minimizing... -
21
Stash Kubernetes
Backup your Kubernetes Stateful Applications
..., StatefulSet, ReplicaSet, ReplicationController, OpenShift DeploymentConfig etc. You can also backup stand-alone PVC with "ReadWriteMany" access mode. Back up your databases deployed in Kubernetes. Stash supports backup of MariaDB, Redis, PostgreSQL, MySQL, MongoDB, Elasticsearch and Percona XtraDB, Etcd databases. It seemingly integrates with KubeDB. It is now a recommended method to backup up KubeDB-supported databases. -
22
SCS
HTTP Session Management for Go
Automatic loading and saving of session data via middleware. Choice of 19 different server-side session stores including PostgreSQL, MySQL, MSSQL, SQLite, Redis and many others. Custom session stores are also supported. Supports multiple sessions per request, 'flash' messages, session token regeneration, idle and absolute session timeouts, and 'remember me' functionality. Easy to extend and customize. Communicate session tokens to/from clients in HTTP headers or request/response bodies. SCS... -
23
DotWeb
Simple and easy go web micro framework
... HttpServer. Support STRING/JSON/JSONP/HTML format output. Integrated Mock Capability. Support custom Context. Integrated Timeout Hook. Global HTTP Error Handling. Global log processing. Support Hijack and websocket. Built-in Cache support. Built-in Session support - supports automatic switching between active and standby redis. Built-in TLS support. Support access to third-party template engines (need to implement dotweb.Renderer interface). -
24
JuiceFS
JuiceFS is a distributed POSIX file system built on top of Redis
A POSIX, HDFS and S3 compatible distributed file system for cloud. JuiceFS is designed to bring back the gold-old memories and experience of file systems in local disks to the cloud. JuiceFS is POSIX compliant and is fully compatible with HDFS and S3. Cloud app building or migrating, file sharing cross-geo and cross-cloud has become easier than ever before. Whether it's a public cloud, private cloud, or hybrid cloud, JuiceFS is available on any cloud of your choice and delivers flexibility,... -
25
Open-IM-Server
Open source Instant Messaging Server
... and quickly, and ensure the security and privacy of business data. Open-IM relies on five open source high-performance components: ETCD, MySQL, MongoDB, Redis, and Kafka. Privatization deployment Before Open-IM-Server, please make sure that the above five components have been installed. If your server does not have the above components, you must first install Missing components. If you have the above components, it is recommended to use them directly.