Search Results for "scrapy-redis" - Page 3
Sort By:
Showing 214 open source projects for "scrapy-redis"
View related business solutions-
HubSpot is an AI-powered customer platform with all the software, integrations, and resources you need to connect your marketing, sales, and customer service. HubSpot's connected platform enables you to grow your business faster by focusing on what matters most: your customers.
-
Drive amazing product experiences with quality product data.
-
1
Log Viewer
Fast and beautiful Log Viewer for Laravel
Beautiful, fast, and open-source viewer for your app logs. Forget scrolling through raw log files to find the errors or debug logs you're looking for. Forget about SSH'ing onto the production server just to read the logs. Log Viewer helps you quickly and clearly see individual log entries, to search, filter, and make sense of your Laravel logs fast. It is free and easy to install. -
2
Gocache
Go cache library that brings you multiple ways to manage caches
Guess what is Gocache? a Go cache library. This is an extendable cache library that brings you a lot of features for caching data. Multiple cache stores: actually in memory, redis, or your own custom store. A chain cache: use multiple cache with a priority order (memory then fallback to a redis shared cache for instance) A loadable cache: allow you to call a callback function to put your data back in cache. A metric cache to let you store metrics about your caches usage (hits, miss, set success... -
3
Asynq
Simple, reliable, and efficient distributed task queue in Go
Asynq is a Go library for queueing tasks and processing them asynchronously with workers. It's backed by Redis and is designed to be scalable yet easy to get started. Client puts tasks on a queue. Server pulls tasks off queues and starts a worker goroutine for each task. Tasks are processed concurrently by multiple workers. Task queues are used as a mechanism to distribute work across multiple machines. A system can consist of multiple worker servers and brokers, giving way to high availability... -
4
Foundatio
Pluggable foundation blocks for building distributed apps
Pluggable foundation blocks for building loosely coupled distributed apps. Includes implementations in Redis, Azure, AWS, RabbitMQ and in memory (for development). When building several big cloud applications we found a lack of great solutions (that's not to say there aren't solutions out there) for many key pieces to building scalable distributed applications while keeping the development experience simple. Wanted to build against abstract interfaces so that we could easily change... -
We design our technology to make workforce management easier. APS offers core HR, payroll, benefits administration, attendance, recruiting, employee onboarding, and more.
-
5
NautilusTrader
A high-performance algorithmic trading platform
NautilusTrader is an open-source, high-performance, production-grade algorithmic trading platform, provides quantitative traders with the ability to backtest portfolios of automated trading strategies on historical data with an event-driven engine, and also deploy those same strategies live, with no code changes. The platform is 'AI-first', designed to develop and deploy algorithmic trading strategies within a highly performant and robust Python native environment. This helps to address the... -
6
Laravel Haystack
Supercharged job chains for Laravel
Laravel Haystack provides supercharged job chains for Laravel. It comes with powerful features like delaying jobs for as long as you like, applying middleware to every job, sharing data and models between jobs and even chunking jobs. Laravel Haystack supports every queue connection/worker out of the box. (Database, Redis/Horizon, SQS). It's great if you need to queue thousands of jobs in a chain or if you are looking for features that the original Bus chain doesn't provide. If you need to make... -
7
Routr
The future of programmable SIP servers
Lightweight sip proxy, location server, and registrar that provides a reliable and scalable SIP infrastructure for telephony carriers, communication service providers, and integrators. Routr provides all the tools required to deploy your VoIP network, including a command-line for remote server control. It can also be controlled via a RESTful API or a web-based GUI. Routr includes all tools for deploying your VoIP network. It offers remote server control via command-line, RESTful API, or a... -
8
jCasbin
An authorization library that supports access control models
.... Besides memory and file, Casbin policy can be stored into lots of places. Currently, dozens of databases are supported, from MySQL, Postgres, Oracle to MongoDB, Redis, Cassandra, AWS S3. jCasbin is a powerful and efficient open-source access control library for Java projects. It provides support for enforcing authorization based on various access control models. -
9
Openblocks
The Open Source Retool Alternative
It's cumbersome to create a single app. You had to design user interfaces, write code in multiple languages and frameworks, and understand how all of that code works together. Low-code/No-code platforms are fast to get started with but quickly become unmaintainable and inflexible. This creates more problems than it solves. Retool-like solutions are great for their simplicity and flexibility, but they can also be limited in different ways compared to frameworks like React/Vue. An all-in-one... -
Simplify every aspect of buying for your business in Order.co. From sourcing products to scaling purchasing across locations to automating your AP and approvals workstreams, Order.co is the platform of choice for growing businesses.
-
10
Bifrost
Middleware for production-oriented MySQL
Heterogeneous middleware for production-oriented MySQL, MariaDB, kafka synchronization to Redis, MongoDB, ClickHouse and other services. And this Bifrost can synchronize the full amount of data to multiple targets in real time. Support MySQL, MariaDB all storage types. Interface-based dynamic and flexible configuration of different data tables and target libraries. Multiple data sources, multiple target library support. Both incremental and full data synchronization are supported. One Binlog... -
11
Casbin
An authorization library that supports access control models
.... Besides memory and file, Casbin policy can be stored into lots of places. Currently, dozens of databases are supported, from MySQL, Postgres, Oracle to MongoDB, Redis, Cassandra, AWS S3. Check the full supported list at: adapters. Casbin is implemented in Golang, Java, PHP and Node.js. All implementations share the same API and behaviors. You can learn Casbin once and use it everywhere. In Casbin, the policy storage is implemented as an adapter(aka middleware for Casbin). -
12
Cryptocurrency Exchange Feed Handler
Cryptocurrency Exchange Websocket data feed handler
Handles multiple cryptocurrency exchange data feeds and returns normalized and standardized results to client registered callbacks for events like trades, book updates, ticker updates, etc. Utilizes WebSockets when possible, but can also poll data via REST endpoints if a WebSocket is not provided. Create a FeedHandler object and add subscriptions. For the various data channels that an exchange supports, you can supply callbacks for data events, or use provided backends to handle the data for... -
13
thumbor
An open-source photo thumbnail service by globo.com
... the best image crop possible. Everytime a face is detected, it will crop the image around it, making sure the most important parts of the picture are preserved. Thumbor allows users to store and load images from anywhere needed. It's really simple to implement a new loader or storage. It comes packaged with file, http, redis and mongo implementations for loaders and storages. Just pick what makes sense for you or implement your own. -
14
Crawlab
Distributed web crawler admin platform for spiders management
Golang-based distributed web crawler management platform, supporting various languages including Python, NodeJS, Go, Java, PHP and various web crawler frameworks including Scrapy, Puppeteer, Selenium. Please use docker-compose to one-click to start up. By doing so, you don't even have to configure MongoDB database. The frontend app interacts with the master node, which communicates with other components such as MongoDB, SeaweedFS and worker nodes. Master node and worker nodes communicate... -
15
jackal
Instant messaging server for the Extensible Messaging
jackal is a free, open-source, high-performance XMPP server that aims to be known for its stability, simple configuration and low resource consumption. Customizable, enforced SSL/TLS. Stream compression (zlib), database connectivity for storing offline messages and user settings (PostgreSQL 9.5+, BoltDB). Caching (Redis 6.2+), clustering capabilities (etcd 3.4+) Expose Prometheus metrics, cross-platform (OS X, Linux). To make it easy to install jackal via Helm in Kubernetes a chart has been... -
16
Godis
A Golang implemented Redis Server and Cluster
Godis is a golang implementation of Redis Server, which intents to provide an example of writing a high concurrent middleware using golang. MULTI Commands Transaction is Atomic and Isolated. If any errors are encountered during execution, godis will rollback the executed commands. Server-side Cluster which is transparent to client. You can connect to any node in the cluster to access all data in the cluster. MSET, MSETNX, DEL, Rename, RenameNX command is supported and atomically executed... -
17
Laravel Horizon
Dashboard and code-driven configuration for Laravel queues
Horizon provides a beautiful dashboard and code-driven configuration for your Laravel-powered Redis queues. Horizon allows you to easily monitor key metrics of your queue system such as job throughput, runtime, and job failures. All of your worker configuration is stored in a single, simple configuration file, allowing your configuration to stay in source control where your entire team can collaborate. When using Horizon, all of your queue worker configuration is stored in a single, simple... -
18
Resque
Ruby library for creating background jobs and processing them
Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later. Resque (pronounced like "rescue") is a Redis-backed library for creating background jobs, placing those jobs on multiple queues, and processing them later. Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both... -
19
EaseProbe
A simple, standalone, and lightweight tool
... command on a remote host and check the CPU, Memory, and Disk usage. MySQL. Connect to a MySQL server and run the SHOW STATUS SQL. Redis. Connect to a Redis server and run the PING command. Memcache. Connect to a Memcache server and run the version command or validate a given key/value pair. MongoDB. Connect to a MongoDB server and perform a ping. Kafka. Connect to a Kafka server and perform a list of all topics. -
20
Algernon
Small self-contained pure-Go web server with Lua, Markdown, HTTP/2
Web server with built-in support for QUIC, HTTP/2, Lua, Teal, Markdown, Pongo2, HyperApp, Amber, Sass(SCSS), GCSS, JSX, BoltDB (built-in, stores the database in a file, like SQLite), Redis, PostgreSQL, MariaDB/MySQL, rate limiting, graceful shutdown, plugins, users and permissions. Written in Go. Uses Bolt (built-in), MySQL, PostgreSQL or Redis (recommended) for the database backend, permissions2 for handling users and permissions, gopher-Lua for interpreting and running Lua, optional Teal... -
21
Mix PHP
PHP CLI mode development framework, supports Swoole, WorkerMan, etc.
.../database and mix/redis in any environment; you can use the native code of mix/grpc to write gRPC; all modules can be combined as randomly as building blocks. PHP gRPC library based on Swoole co-process, including protoc code generator, server, customer end. Monitor changes in your Swoole program files and automatically restart the server-applicable to development. -
22
node-rate-limiter-flexible
Count and limit requests by key with atomic increments
rate-limiter-flexible counts and limits number of actions by key and protects from DDoS and brute force attacks at any scale. It works with Redis, process Memory, Cluster or PM2, Memcached, MongoDB, MySQL, PostgreSQL and allows to control of requests rate in single process or distributed environment. All operations in memory or distributed environments use atomic increments against race conditions. Combine limiters, block key for some duration, delay actions, manage failover with insurance... -
23
dynaconf
Configuration Management for Python
... Vault and Redis as settings and secrets storage. Built-in extensions for Django and Flask web frameworks. CLI for common operations such as init, list, write, validate, export. On your own code you import and use settings object imported from your config.py file. Dynaconf prioritizes the use of environment variables and you can optionally store settings in Settings Files using any of toml|yaml|json|ini|py extension. -
24
Docker Gitlab
Dockerized GitLab
Dockerfile to build a GitLab image for the Docker opensource container platform. GitLab CE is set up in the Docker image using the install from source method as documented in the the official GitLab documentation. For other methods to install GitLab please refer to the Official GitLab Installation Guide which includes a GitLab image for Docker. Docker is a relatively new project and is active being developed and tested by a thriving community of developers and testers and every release of... -
25
Workerman
An asynchronous event driven PHP socket framework
... components such as asynchronous Mysql, asynchronous Redis, asynchronous Http, MQTT Internet of Things client, and asynchronous message queue. Based on the high-performance Epoll event network library, a single machine can support millions of concurrent connections, with a long connection throughput of up to 36W/S. Objects such as PHP database can be resident in memory, reducing analysis and compilation and network overhead.