Search Results for "gpu"

...It supports recurrent state input and output, variable-length batches, internal gating, query-key normalization, and beta activation. Builds can target the detected GPU architecture or multiple supported architectures for wheels and CI pipelines. The repository also provides correctness tests, benchmark material, a direct Python kernel API, and development helpers for CUDA and C++ tooling.

The blocksparse repository provides efficient GPU kernels (TensorFlow custom ops) for block-sparse matrix multiplication and convolution operations. The idea is to exploit block-level sparsity — i.e. treat matrices or weight tensors as composed of blocks, many of which may be zero or unused — to save compute and memory when sparsity patterns are structured. This is particularly useful in models like Sparse Transformers, where attention matrices or intermediate layers may adopt block-sparse patterns to scale better. ...