Search Results for "disk clone linux"

JDF is a DataFrames serialization format with the following goals, fast save and load times, compressed storage on disk, enabled disk-based data manipulation (not yet achieved), and support for machine learning workloads, e.g. mini-batch, sampling (not yet achieved). JDF stores a DataFrame in a folder with each column stored as a separate file. There is also a metadata.jls file that stores metadata about the original DataFrame. Collectively, the column files, the metadata file, and the...

Easy everyday parallelism with a file tree abstraction. Read a directory structure as a Julia data structure, (lazy-)load the files, apply map and reduce operations on the data while not exceeding available memory if possible. Make up a file tree in memory, create some data to go with each file (in parallel), write the tree to disk (in parallel). FileTrees is a set of tools to lazy-load, process and save file trees. Built-in parallelism allows you to max out all threads and processes that...

JET employs Julia's type inference system to detect potential bugs and type instabilities. JET is tightly coupled to the Julia compiler, and so each JET release supports a limited range of Julia versions. See the Project.toml file for the range of supported Julia versions. The Julia package manager should install a version of JET compatible with the Julia version you are running. If you want to use JET on unreleased version of Julia where compatibility with JET is yet unknown, clone this git...

Unlike traditional package managers, which install and manage a single global set of packages, Pkg is designed around “environments”: independent sets of packages that can be local to an individual project or shared and selected by name. The exact set of packages and versions in an environment is captured in a manifest file which can be checked into a project repository and tracked in version control, significantly improving reproducibility of projects. If you’ve ever tried to run code you...

Feather.jl provides a pure Julia library for reading and writing feather-formatted binary files, an efficient on-disk representation of a DataFrame.