Search Results for "scrapy-redis"

Simple and efficient proxy pool, providing the following functions. Regularly crawl free proxy websites, easy and scalable. Use Redis to store brokers and sort broker availability. Regular testing and screening to eliminate unavailable proxies and leave available proxies. Provides a proxy API to randomly select available proxies that pass the test. The principle analysis of the proxy pool can be seen in " How to Build an Efficient Proxy Pool ". It is recommended to read it before using...

RESTful API service for surmon.me blog, powered by NestJS, required MongoDB & Redis.

rate-limiter-flexible counts and limits number of actions by key and protects from DDoS and brute force attacks at any scale. It works with Redis, process Memory, Cluster or PM2, Memcached, MongoDB, MySQL, PostgreSQL and allows to control of requests rate in single process or distributed environment. All operations in memory or distributed environments use atomic increments against race conditions. Combine limiters, block key for some duration, delay actions, manage failover with insurance...

... and modified accordingly. Additionally, the website content can be optionally spun using the article spinning engine that is already embedded into the application. After the website has been modified the result is then stored for a predefined time inside a custom-built, flat-file cache system architectured to be 500x faster than memory-based cache systems such as Memcache and Redis. This is accomplished by bypassing the serialisation and deserialization processes, which resulted a significantly faster

Simple API-caching middleware for Express/Node. A simple API response caching middleware for Express/Node using plain-English durations. Supports Redis or a built-in memory engine with auto-clearing. Because route-caching of simple data/responses should also be simple. Sometimes you need custom keys (e.g. save routes per session, or per method). We've made it easy! All req/res attributes used in the generation of the key must have been set previously (upstream). The entire route logic block...

A Real-Time Chat Application built using Node.js, Express, Mongoose, Socket.io, Passport, & Redis. The configurations on production will be assigned from Environment Variables on Heroku, while the development configurations reside inside app/config/config.json file. You need to create a database on MongoLab, then create a database user, get the MongoDB URI, and assign it to dbURI. You need to register a new application on both Facebook and Twitter to get your tokens by which users can grant...