Business Software for Thunder Compute - Page 2

Unsloth is an open source platform designed to accelerate and optimize the fine-tuning and training of Large Language Models (LLMs). It enables users to train custom models, such as ChatGPT, in just 24 hours instead of the typical 30 days, achieving speeds up to 30 times faster than Flash Attention 2 (FA2) while using 90% less memory. Unsloth supports both LoRA and QLoRA fine-tuning techniques, allowing for efficient customization of models like Mistral, Gemma, and Llama versions 1, 2, and 3. Unsloth's efficiency stems from manually deriving computationally intensive mathematical steps and handwriting GPU kernels, resulting in significant performance gains without requiring hardware modifications. Unsloth delivers a 10x speed increase on a single GPU and up to 32x on multi-GPU systems compared to FA2, with compatibility across NVIDIA GPUs from Tesla T4 to H100, and portability to AMD and Intel GPUs.

ComfyUI is a free and open source node-based application for generative AI, enabling users to build, create, and share without limits. It allows for the extension of functionality through custom nodes, letting users tailor workflows to their specific needs. Designed for performance, ComfyUI runs workflows directly on local machines, offering faster iteration, lower costs, and complete control. The visual interface provides full control by connecting nodes on a canvas, allowing for branching, remixing, and adjusting every part of the workflow at any time. Workflows can be saved, shared, and reused effortlessly, with exported media carrying metadata to instantly rebuild the full workflow. Users can see results in real-time as they adjust workflows, facilitating faster iteration with instant visual feedback. ComfyUI supports the generation of various media types, including images, videos, 3D assets, and audio.

tmux is a terminal multiplexer that enables multiple terminals to be created, accessed, and controlled from a single screen. It allows sessions to be detached so they continue running in the background and later reattached exactly as left. tmux implements each window as a separate client process, supports ANSI/ISO color via VT220 (and later) control sequences, and is configurable through its example tmux.conf file and man page. Built atop minimal dependencies, libevent 2.x and ncurses, it requires only a C compiler, make, pkg-config, and a Yacc for building. tmux’s lightweight, single-screen architecture, extensive documentation, and cross-platform support make it a robust, standards-compliant solution for managing terminal workflows efficiently.

PowerShell is a cross-platform task automation and configuration management framework, consisting of a command-line shell and scripting language. Unlike most shells, which accept and return text, PowerShell is built on top of the .NET Common Language Runtime (CLR), and accepts and returns .NET objects. This fundamental change brings entirely new tools and methods for automation. Unlike traditional command-line interfaces, PowerShell cmdlets are designed to deal with objects. An object is structured information that is more than just the string of characters appearing on the screen. Command output always carries extra information that you can use if you need it. If you've used text-processing tools to process data in the past, you'll find that they behave differently when used in PowerShell. In most cases, you don't need text-processing tools to extract specific information. You directly access portions of the data using standard PowerShell object syntax.

Project Jupyter exists to develop open-source software, open-standards, and services for interactive computing across dozens of programming languages. JupyterLab is a web-based interactive development environment for Jupyter notebooks, code, and data. JupyterLab is flexible, configure and arrange the user interface to support a wide range of workflows in data science, scientific computing, and machine learning. JupyterLab is extensible and modular, write plugins that add new components and integrate with existing ones. The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include, data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more. Jupyter supports over 40 programming languages, including Python, R, Julia, and Scala.

Package, dependency, and environment management for any language, Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, Fortran, and more. Conda is an open-source package management system and environment management system that runs on Windows, macOS, Linux, and z/OS. Conda quickly installs, runs, and updates packages and their dependencies. Conda easily creates, saves, loads, and switches between environments on your local computer. It was created for Python programs, but it can package and distribute software for any language. Conda as a package manager helps you find and install packages. If you need a package that requires a different version of Python, you do not need to switch to a different environment manager, because conda is also an environment manager. With just a few commands, you can set up a totally separate environment to run that different version of Python, while continuing to run your usual version of Python in your normal environment.

Over the last few weeks we all have been overwhelmed by the response and have been working hard to ensure a safe and ethical release, incorporating data from our beta model tests and community for the developers to act on. In cooperation with the tireless legal, ethics and technology teams at HuggingFace and amazing engineers at CoreWeave. We have developed an AI-based Safety Classifier included by default in the overall software package. This understands concepts and other factors in generations to remove outputs that may not be desired by the model user. The parameters of this can be readily adjusted and we welcome input from the community how to improve this. Image generation models are powerful, but still need to improve to understand how to represent what we want better.

CUDA® is a parallel computing platform and programming model developed by NVIDIA for general computing on graphical processing units (GPUs). With CUDA, developers are able to dramatically speed up computing applications by harnessing the power of GPUs. In GPU-accelerated applications, the sequential part of the workload runs on the CPU – which is optimized for single-threaded performance – while the compute intensive portion of the application runs on thousands of GPU cores in parallel. When using CUDA, developers program in popular languages such as C, C++, Fortran, Python and MATLAB and express parallelism through extensions in the form of a few basic keywords. The CUDA Toolkit from NVIDIA provides everything you need to develop GPU-accelerated applications. The CUDA Toolkit includes GPU-accelerated libraries, a compiler, development tools and the CUDA runtime.

MinIO's high-performance object storage suite is software defined and enables customers to build cloud-native data infrastructure for machine learning, analytics and application data workloads. MinIO object storage is fundamentally different. Designed for performance and the S3 API, it is 100% open-source. MinIO is ideal for large, private cloud environments with stringent security requirements and delivers mission-critical availability across a diverse range of workloads. MinIO is the world's fastest object storage server. With READ/WRITE speeds of 183 GB/s and 171 GB/s on standard hardware, object storage can operate as the primary storage tier for a diverse set of workloads ranging from Spark, Presto, TensorFlow, H2O.ai as well as a replacement for Hadoop HDFS. MinIO leverages the hard won knowledge of the web scalers to bring a simple scaling model to object storage. At MinIO, scaling starts with a single cluster which can be federated with other MinIO clusters.

Apache Spark™ is a unified analytics engine for large-scale data processing. Apache Spark achieves high performance for both batch and streaming data, using a state-of-the-art DAG scheduler, a query optimizer, and a physical execution engine. Spark offers over 80 high-level operators that make it easy to build parallel apps. And you can use it interactively from the Scala, Python, R, and SQL shells. Spark powers a stack of libraries including SQL and DataFrames, MLlib for machine learning, GraphX, and Spark Streaming. You can combine these libraries seamlessly in the same application. Spark runs on Hadoop, Apache Mesos, Kubernetes, standalone, or in the cloud. It can access diverse data sources. You can run Spark using its standalone cluster mode, on EC2, on Hadoop YARN, on Mesos, or on Kubernetes. Access data in HDFS, Alluxio, Apache Cassandra, Apache HBase, Apache Hive, and hundreds of other data sources.

Experiment tracking, hyperparameter optimization, model and dataset versioning with Weights & Biases (WandB). Track, compare, and visualize ML experiments with 5 lines of code. Add a few lines to your script, and each time you train a new version of your model, you'll see a new experiment stream live to your dashboard. Optimize models with our massively scalable hyperparameter search tool. Sweeps are lightweight, fast to set up, and plug in to your existing infrastructure for running models. Save every detail of your end-to-end machine learning pipeline — data preparation, data versioning, training, and evaluation. It's never been easier to share project updates. Quickly and easily implement experiment logging by adding just a few lines to your script and start logging results. Our lightweight integration works with any Python script. W&B Weave is here to help developers build and iterate on their AI applications with confidence.

A powerful yet lightweight security platform that provides insightful observability, proactive controls, threat detection and response for your Linux infrastructure in the cloud or datacenter. Your cloud infrastructure is a massive multi-user environment. Don’t protect it with security solutions originally built for endpoints. Think beyond logging and analytics solutions that lack the necessary context and workflows for true infrastructure security. Cmd’s infrastructure detection and response platform is optimized for the needs of today’s agile security teams. View system activity in real time or search through retained data, aided by rich filters and triggers. Leverage our eBPF sensors, contextual data model and intuitive workflows to gain insight into user activity, running processes and access to sensitive resources. No advanced degree in Linux administration required. Create guardrails and controls around sensitive actions to complement traditional access management.

Transition seamlessly between eager and graph modes with TorchScript, and accelerate the path to production with TorchServe. Scalable distributed training and performance optimization in research and production is enabled by the torch-distributed backend. A rich ecosystem of tools and libraries extends PyTorch and supports development in computer vision, NLP and more. PyTorch is well supported on major cloud platforms, providing frictionless development and easy scaling. Select your preferences and run the install command. Stable represents the most currently tested and supported version of PyTorch. This should be suitable for many users. Preview is available if you want the latest, not fully tested and supported, 1.10 builds that are generated nightly. Please ensure that you have met the prerequisites (e.g., numpy), depending on your package manager. Anaconda is our recommended package manager since it installs all dependencies.

vLLM is a high-performance library designed to facilitate efficient inference and serving of Large Language Models (LLMs). Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evolved into a community-driven project with contributions from both academia and industry. It offers state-of-the-art serving throughput by efficiently managing attention key and value memory through its PagedAttention mechanism. It supports continuous batching of incoming requests and utilizes optimized CUDA kernels, including integration with FlashAttention and FlashInfer, to enhance model execution speed. Additionally, vLLM provides quantization support for GPTQ, AWQ, INT4, INT8, and FP8, as well as speculative decoding capabilities. Users benefit from seamless integration with popular Hugging Face models, support for various decoding algorithms such as parallel sampling and beam search, and compatibility with NVIDIA GPUs, AMD CPUs and GPUs, Intel CPUs, and more.

​JAX is a Python library designed for high-performance numerical computing and machine learning research. It offers a NumPy-like API, facilitating seamless adoption for those familiar with NumPy. Key features of JAX include automatic differentiation, just-in-time compilation, vectorization, and parallelization, all optimized for execution on CPUs, GPUs, and TPUs. These capabilities enable efficient computation for complex mathematical functions and large-scale machine-learning models. JAX also integrates with various libraries within its ecosystem, such as Flax for neural networks and Optax for optimization tasks. Comprehensive documentation, including tutorials and user guides, is available to assist users in leveraging JAX's full potential. ​