Search Results for "shared memory"

...The implementation fits into a single C 99 header file and uses different SIMD flavors and SWAR on older platforms. The Str is designed to replace long Python str strings and wrap our C-level API. On the other hand, the File memory-maps a file from persistent memory without loading its copy into RAM. The contents of that file would remain immutable, and the mapping can be shared by multiple Python processes simultaneously. A standard dataset pre-processing use case would be to map a sizeable textual dataset like Common Crawl into memory, spawn child processes, and split the job between them.

Chronicle Wire is a Wire Format abstraction library. Chronicle Wire uses Chronicle Bytes for bytes manipulation, and Chronicle Core for low-level JVM access. Configuration includes aliased type information. This supports easy extension by adding new classes/versions, and cross-platform using type aliasing. By supporting types, a configuration file can bootstrap itself. You control how the configuration file is decoded. See example Queue Enterprise config. To send the configuration of a...

EMS makes possible persistent shared memory parallelism between Node.js, Python, and C/C++. Extended Memory Semantics (EMS) unifies synchronization and storage primitives to address several challenges of parallel programming. A modern multi-core server has 16-32 cores and nearly 1TB of memory, equivalent to an entire rack of systems from a few years ago. As a consequence, jobs formerly requiring a Map-Reduce cluster can now be performed entirely in shared memory on a single server without using distributed programming.

...Occursions asynchronously tails log files and indexes the individual lines in each log file as each line is written to disk so you don't even have to wait for a second after an event happens to search for it. Occursions uses custom disk backed data structures to create and search its indexes so it is very efficient at using CPU, memory and disk. You can extend Occursions with shared libraries to support your own file formats, even binary file formats!