Search Results for "pmap"

...Its design separates pure computation from state by threading parameter collections and RNGs explicitly, enabling reproducibility, transformation, and easy experimentation with JAX transforms like jit, pmap, and vmap. Modules define parameterized computations, but initialization and application remain side-effect free, which pairs naturally with JAX’s staging and compilation model. Flax emphasizes composability: optimizers, training loops, and checkpointing are provided as examples or utilities rather than monolithic frameworks, encouraging research-friendly customization. ...

Tunix is a JAX-native library for post-training large language models, bringing supervised fine-tuning, reinforcement learning–based alignment, and knowledge distillation into one coherent toolkit. It embraces JAX’s strengths—functional programming, jit compilation, and effortless multi-device execution—so experiments scale from a single GPU to pods of TPUs with minimal code changes. The library is organized around modular pipelines for data loading, rollout, optimization, and evaluation,...

Jraph (pronounced “giraffe”) is a lightweight JAX library developed by Google DeepMind for building and experimenting with graph neural networks (GNNs). It provides an efficient and flexible framework for representing, manipulating, and training models on graph-structured data. The core of Jraph is the GraphsTuple data structure, which enables users to define graphs with arbitrary node, edge, and global attributes, and to batch variable-sized graphs efficiently for JAX’s just-in-time...