Search Results for "activation"
Sort By:
Showing 20 open source projects for "activation"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
1
Transformer Debugger
Tool for exploring and debugging transformer model behaviors
...TDB allows users to intervene directly in the forward pass of a model and observe how such interventions change predictions, making it possible to answer questions like why a token was selected or why an attention head focused on a certain input. It automatically identifies and explains the most influential components, highlights activation patterns, and maps relationships across circuits within the model. The tool includes both a React-based neuron viewer for exploring model components and a backend activation server for running inferences and serving data. -
2
Ring
Ring is a reasoning MoE LLM provided and open-sourced by InclusionAI
Ring is a reasoning Mixture-of-Experts (MoE) large language model (LLM) developed by inclusionAI. It is built from or derived from Ling. Its design emphasizes reasoning, efficiency, and modular expert activation. In its “flash” variant (Ring-flash-2.0), it optimizes inference by activating only a subset of experts. It applies reinforcement learning/reasoning optimization techniques. Its architectures and training approaches are tuned to enable efficient and capable reasoning performance. Reasoning-optimized model with reinforcement learning enhancements. ... -
3
Claude Cognitive
Persistent context and multi-instance coordination
...It introduces an attention-based context router that prioritizes files and content relevant to the current development discussion — tagging them as HOT, WARM, or COLD based on recency and keyword activation — so Claude Code doesn’t waste token budget rereading irrelevant code. This context routing dramatically reduces redundant token usage and accelerates large codebase interactions by focusing only on what truly matters to the current task. Additionally, Claude-Cognitive includes a pool coordinator to share state across multiple Claude Code instances, preserving what’s been learned or completed and preventing repetitive debugging or redundant exploration. -
4
Ling
Ling is a MoE LLM provided and open-sourced by InclusionAI
Ling is a Mixture-of-Experts (MoE) large language model (LLM) provided and open-sourced by inclusionAI. The project offers different sizes (Ling-lite, Ling-plus) and emphasizes flexibility and efficiency: being able to scale, adapt expert activation, and perform across a range of natural language/reasoning tasks. Example scripts, inference pipelines, and documentation. The codebase includes inference, examples, models, documentation, and model download infrastructure. As more developers and researchers engage with the platform, we can expect rapid advancements and improvements, leading to even more sophisticated applications. ... -
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
5
MobileLLM
MobileLLM Optimizing Sub-billion Parameter Language Models
...Introduced in the ICML 2024 paper “MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases”, it focuses on delivering strong reasoning and generalization capabilities in models under one billion parameters. The framework integrates several architectural innovations—SwiGLU activation, deep and thin network design, embedding sharing, and grouped-query attention (GQA)—to achieve a superior trade-off between model size, inference speed, and accuracy. MobileLLM demonstrates remarkable performance, with the 125M and 350M variants outperforming previous state-of-the-art models of the same scale by up to 4.3% on zero-shot commonsense reasoning tasks. -
6
Automated Interpretability
Code for Language models can explain neurons in language models paper
...It includes a “neuron explainer” component that, given a target neuron or latent feature, proposes natural language explanations or heuristics (e.g. “this neuron activates when the input has property X”) and then simulates activation behavior across example inputs to test whether the explanation holds. The project also contains a “neuron viewer” web component for browsing neurons, explanations, and activation patterns, making it more interactive and exploratory. -
7
pycm
Multi-class confusion matrix library in Python
PyCM is a multi-class confusion matrix library written in Python that supports both input data vectors and direct matrix, and a proper tool for post-classification model evaluation that supports most classes and overall statistics parameters. PyCM is the swiss-army knife of confusion matrices, targeted mainly at data scientists that need a broad array of metrics for predictive models and an accurate evaluation of large variety of classifiers. -
8
TorchCode
Practice implementing softmax, attention, GPT-2 and more
...It is structured similarly to competitive programming platforms like LeetCode but focuses specifically on tensor operations and deep learning concepts. The platform provides a collection of curated problems that cover fundamental topics such as activation functions, normalization layers, attention mechanisms, and full transformer architectures. It runs in a Jupyter-based environment, allowing users to write, test, and debug their code interactively while receiving immediate feedback. An automated judging system evaluates correctness, gradient flow, and numerical stability, helping users understand both functional and theoretical aspects of their implementations. -
9
TurboDiffusion
100–200× Acceleration for Video Diffusion Models
TurboDiffusion is an advanced open-source framework designed to dramatically accelerate video diffusion model generation, aiming for performance improvements on the order of 100–200× compared with traditional implementations while retaining high output quality. It achieves this by combining a suite of algorithmic and engineering optimizations, including attention acceleration techniques, efficient step distillation methods, and quantization strategies that reduce computational overhead. The... -
8 Monitoring Tools in One APM. Install in 5 Minutes.AppSignal works out of the box for Ruby, Elixir, Node.js, Python, and more. 30-day free trial, no credit card required.
-
10
Ludwig AI
Low-code framework for building custom LLMs, neural networks
...Automatic batch size selection, distributed training (DDP, DeepSpeed), parameter efficient fine-tuning (PEFT), 4-bit quantization (QLoRA), and larger-than-memory datasets. Retain full control of your models down to the activation functions. Support for hyperparameter optimization, explainability, and rich metric visualizations. Experiment with different model architectures, tasks, features, and modalities with just a few parameter changes in the config. Think building blocks for deep learning. -
11
Advanced AI explainability for PyTorch
Advanced AI Explainability for computer vision
pytorch-grad-cam is an open-source library that provides advanced explainable AI techniques for interpreting the predictions of deep learning models used in computer vision. The project implements Grad-CAM and several related visualization methods that highlight the regions of an image that most strongly influence a neural network’s decision. These visualization techniques allow developers and researchers to better understand how convolutional neural networks and transformer-based vision... -
12
Ling-V2
Ling-V2 is a MoE LLM provided and open-sourced by InclusionAI
Ling-V2 is an open-source family of Mixture-of-Experts (MoE) large language models developed by the InclusionAI research organization with the goal of combining state-of-the-art performance, efficiency, and openness for next-generation AI applications. It introduces highly sparse architectures where only a fraction of the model’s parameters are activated per input token, enabling models like Ling-mini-2.0 to achieve reasoning and instruction-following capabilities on par with much larger... -
13
snntorch
Deep and online learning with spiking neural networks in Python
snntorch is a deep learning library that enables researchers and developers to build and train spiking neural networks using the PyTorch framework. Spiking neural networks are biologically inspired models that communicate through discrete spike events rather than continuous activation values, making them closer to how neurons operate in the brain. The library extends PyTorch’s tensor computation capabilities to support gradient-based learning for networks composed of spiking neurons. This allows researchers to train spiking neural models using familiar deep learning workflows while taking advantage of GPU acceleration and automatic differentiation. snnTorch provides implementations of common spiking neuron models, surrogate gradient training methods, and utilities for handling temporal neural dynamics. ... -
14
Mixtral offloading
Run Mixtral-8x7B models in Colab or consumer desktops
...The project implements techniques that allow model components to be dynamically moved between CPU memory and GPU memory during inference, significantly reducing the amount of GPU VRAM required to run the model. This approach takes advantage of the sparse activation properties of mixture-of-experts architectures, where only a subset of expert networks are used for each token during generation. By selectively loading and caching the required experts, the system avoids keeping the entire model in GPU memory at once. The repository includes notebooks and code examples that demonstrate how to run large language models on consumer hardware such as personal GPUs or cloud notebook environments. -
15
ConvNeXt
Code release for ConvNeXt model
ConvNeXt is a modernized convolutional neural network (CNN) architecture designed to rival Vision Transformers (ViTs) in accuracy and scalability while retaining the simplicity and efficiency of CNNs. It revisits classic ResNet-style backbones through the lens of transformer design trends—large kernel sizes, inverted bottlenecks, layer normalization, and GELU activations—to bridge the performance gap between convolutions and attention-based models. ConvNeXt’s clean, hierarchical structure... -
16
LaMDA-pytorch
Open-source pre-training implementation of Google's LaMDA in PyTorch
Open-source pre-training implementation of Google's LaMDA research paper in PyTorch. The totally not sentient AI. This repository will cover the 2B parameter implementation of the pre-training architecture as that is likely what most can afford to train. You can review Google's latest blog post from 2022 which details LaMDA here. You can also view their previous blog post from 2021 on the model. -
17
DeepLearning Tutorial
Deep Learning Tutorial, Excellent Articles, Deep Learning Tutorial
...The project is designed as a knowledge collection that helps beginners understand neural networks, deep learning architectures, and fundamental machine learning concepts. It contains curated learning materials covering topics such as feedforward neural networks, activation functions, backpropagation algorithms, optimization methods, and convolutional neural networks. The repository organizes these materials into structured tutorials and references that allow readers to explore deep learning concepts progressively. Many of the resources include explanations of common model architectures used in computer vision and artificial intelligence. ... -
18
Text Gen
Almost state of art text generation library
...Something sweet built with Tensorflow and Pytorch(coming soon). Load your data, your data must be in a text format. Download the example data from the example folder. Tune your model to know the best optimizer, activation method to use. -
19
Exposure
Learning infinite-resolution image processing with GAN and RL
...ACM Transactions on Graphics (presented at SIGGRAPH 2018) Exposure is originally designed for RAW photos, which assumes 12+ bit color depth and linear "RGB" color space (or whatever we get after demosaicing). jpg and png images typically have only 8-bit color depth (except 16-bit pngs) and the lack of information (dynamic range/activation resolution) may lead to suboptimal results such as posterization. Moreover, jpg and most pngs assume an sRGB color space, which contains a roughly 1/2.2 Gamma correction, making the data distribution different from training images (which are linear). Exposure is just a prototype (proof-of-concept) of our latest research, and there are definitely a lot of engineering efforts required to make it suitable for a real product. -
20
Neural Libs
Neural network library for developers
This project includes the implementation of a neural network MLP, RBF, SOM and Hopfield networks in several popular programming languages. The project also includes examples of the use of neural networks as function approximation and time series prediction. Includes a special program makes it easy to test neural network based on training data and the optimization of the network.
- Previous
- You're on page 1
- Next
