Search Results for "large files"

...Inspired by Python's csv module, I wanted a library with simple, intuitive syntax. Furthermore, I wanted support for special use cases such as calculating statistics on very large files. With the deluge of large datasets available, a performant CSV parser is a necessity. By using overlapped threads, memory-mapped IO, and efficient data structures, this parser can quickly tackle large CSV files. Furthermore, this parser has a minimal memory footprint and can handle larger-than-RAM files.

jtc stand for: JSON transformational chains (used to be JSON test console). jtc offers a powerful way to select one or multiple elements from a source JSON and apply various actions on the selected elements at once (wrap selected elements into a new JSON, filter in/out, sort elements, update elements, insert new elements, remove, copy, move, compare, transform, swap around and many other operations).

Our goal is to create the world's fastest extendable, non-transactional time series database for big data (you know, for kids)! Log file indexing is our initial focus. For example append only ASCII files produced by libraries like Log4J, or containing FIX messages or JSON objects. Occursions was built by a small team sick of creating hacks to remotely copy and/or grep through tons of large log files. We use it to index around a terabyte of new log data per day. You can use it too. Who doesn't have `just too many' log files? ...