Search Results for "data processing" - Page 7
Sort By:
Showing 191 open source projects for "data processing"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
1
igel
Machine learning tool that allows you to train and test models
A delightful machine learning tool that allows you to train/fit, test, and use models without writing code. The goal of the project is to provide machine learning for everyone, both technical and non-technical users. I sometimes needed a tool sometimes, which I could use to fast create a machine learning prototype. Whether to build some proof of concept, create a fast draft model to prove a point or use auto ML. I find myself often stuck writing boilerplate code and thinking too much about... -
2
MITRE Annotation Toolkit
A toolkit for managing and manipulating text annotations
...The goal of MAT is not to help you configure your training engine (in the default case, the Carafe CRF system) to achieve the best possible performance on your data. MAT is for "everything else": all the tools you end up wishing you had. -
3
SparrowRecSys
A Deep Learning Recommender System
SparrowRecSys is an open-source deep learning recommendation system framework designed to demonstrate the architecture and implementation of modern industrial-scale recommender systems. The project integrates multiple machine learning models and data processing pipelines to simulate how real-world recommendation platforms operate. It includes components for offline data processing, feature engineering, model training, real-time data updates, and online recommendation services. SparrowRecSys supports a wide range of state-of-the-art recommendation algorithms, including models for click-through rate prediction and user behavior modeling that are widely used in advertising and content recommendation systems. ... -
4
tensorflow_template_application
TensorFlow template application for deep learning
tensorflow_template_application is a template project that demonstrates how to structure scalable applications built with TensorFlow. The repository provides a standardized architecture that helps developers organize machine learning code into clear components such as data processing, model training, evaluation, and deployment. Instead of focusing on a specific algorithm, the project emphasizes software engineering practices that make machine learning systems easier to maintain and extend. The template includes configuration files, scripts, and project structures that help teams build reproducible experiments and production-ready pipelines. ... -
8 Monitoring Tools in One APM. Install in 5 Minutes.AppSignal works out of the box for Ruby, Elixir, Node.js, Python, and more. 30-day free trial, no credit card required.
-
5
Denoiser
Real Time Speech Enhancement in the Waveform Domain (Interspeech 2020)
...It uses a causal encoder-decoder architecture with skip connections, optimized with losses defined both in the time domain and frequency domain to better suppress noise while preserving speech. Unlike models that operate on spectrograms alone, this design enables lower latency and coherent waveform output. The implementation includes data augmentation techniques applied to the raw waveforms (e.g. noise mixing, reverberation) to improve model robustness and generalization to diverse noise types. The project supports both offline denoising (batch inference) and live audio processing (e.g. via loopback audio interfaces), making it practical for real-time use in calls or recording. ... -
6
XLM (Cross-lingual Language Model)
PyTorch original implementation of Cross-lingual Language Model
XLM (Cross-lingual Language Model) is a family of multilingual pretraining methods that align representations across languages to enable strong zero-shot transfer. It popularized objectives like Masked Language Modeling (MLM) across many languages and Translation Language Modeling (TLM) that jointly trains on parallel sentence pairs to tighten cross-lingual alignment. Using a shared subword vocabulary, XLM learns language-agnostic features that work well for classification and sequence... -
7
Self-Attentive Parser
High-accuracy NLP parser with models for 11 languages
LightAutoML is an automated machine learning (AutoML) framework developed by Sberbank AI Lab, designed to facilitate the development of machine learning models with minimal human intervention. -
8
Synonyms
Chinese synonyms, chat robot, intelligent question and answer toolkit
...Classes and subclasses, sort out the relationship between words, the extended version of the synonym word forest contains more than 70,000 words, of which more than 30,000 words are shared in the form of open data. -
9
NLP Architect
A model library for exploring state-of-the-art deep learning
...The library contains NLP/NLU-related models per task, different neural network topologies (which are used in models), procedures for simplifying workflows in the library, pre-defined data processors and dataset loaders and misc utilities. The library is designed to be a tool for model development: data pre-processing, build model, train, validate, infer, save or load a model. -
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
10
fastNLP
fastNLP: A Modularized and Extensible NLP Framework
fastNLP is a lightweight framework for natural language processing (NLP), the goal is to quickly implement NLP tasks and build complex models. A unified Tabular data container simplifies the data preprocessing process. Built-in Loader and Pipe for multiple datasets, eliminating the need for preprocessing code. Various convenient NLP tools, such as Embedding loading (including ELMo and BERT), intermediate data cache, etc.. -
11
CC-Net
Tools to download and cleanup Common Crawl data
cc_net provides tools to download, segment, clean, and filter Common Crawl to build large-scale text corpora, including monolingual datasets and the multilingual CC-100 collection introduced in the associated paper. It includes pipelines to fetch snapshots, extract text, de-duplicate, identify language, and apply quality filtering based on heuristics and language models. The outputs are intended for pretraining language models and for creating standardized corpora that can be reproduced or... -
12
PyArmadillo
linear algebra library for Python
PyArmadillo - streamlined linear algebra library for Python, with emphasis on ease of use. Alternative to NumPy / SciPy. * Main page: https://pyarma.sourceforge.io * Documentation: https://pyarma.sourceforge.io/docs.html * Bug reports: https://pyarma.sourceforge.io/faq.html * Git repo: https://gitlab.com/jason-rumengan/pyarma -
13
SQLFlow
SQL compiler bridging databases and machine learning workflows
SQLFlow is an open source project designed to bridge the gap between traditional SQL-based data processing and modern machine learning workflows by extending SQL syntax with AI capabilities. It acts as a compiler that translates SQL programs into executable workflows, enabling users to train, evaluate, and deploy machine learning models directly from SQL statements. It integrates with multiple database engines such as MySQL, Hive, and MaxCompute, while also supporting machine learning frameworks like TensorFlow and XGBoost. ... -
14
GluonNLP
NLP made easy
GluonNLP is a toolkit that helps you solve NLP problems. It provides easy-to-use tools that helps you load the text data, process the text data, and train models. To facilitate both the engineers and researchers, we provide command-line-toolkits for downloading and processing the NLP datasets. Gluon NLP makes it easy to evaluate and train word embeddings. Here are examples to evaluate the pre-trained embeddings included in the Gluon NLP toolkit as well as example scripts for training embeddings on custom datasets. ... -
15
Delta ML
Deep learning based natural language and speech processing platform
...It helps you to train, develop, and deploy NLP and/or speech models. Use configuration files to easily tune parameters and network structures. What you see in training is what you get in serving: all data processing and features extraction are integrated into a model graph. Text classification, named entity recognition, question and answering, text summarization, etc. Uniform I/O interfaces and no changes for new models. -
16
Forecasting Best Practices
Time Series Forecasting Best Practices & Examples
...Rather than creating implementations from scratch, we draw from existing state-of-the-art libraries and build additional utilities around processing and featuring the data, optimizing and evaluating models, and scaling up to the cloud. The examples and best practices are provided as Python Jupyter notebooks and R markdown files and a library of utility functions. -
17
Spectral Python
A python module for hyperspectral image processing
Spectral Python (SPy) is a python package for reading, viewing, manipulating, and classifying hyperspectral image (HSI) data. SPy includes functions for clustering, dimensionality reduction, supervised classification, and more. -
18
NLP Best Practices
Natural Language Processing Best Practices & Examples
In recent years, natural language processing (NLP) has seen quick growth in quality and usability, and this has helped to drive business adoption of artificial intelligence (AI) solutions. In the last few years, researchers have been applying newer deep learning methods to NLP. Data scientists started moving from traditional methods to state-of-the-art (SOTA) deep neural network (DNN) algorithms which use language models pretrained on large text corpora. -
19
cocoNLP
A Chinese information extraction tool
...Its API is intentionally simple, so you can drop it into scripts, ETL jobs, or dashboards without deep ML expertise. Because it aims at utility over complexity, it’s useful for prototyping data products or building lightweight text analytics where large models would be overkill. The repository also includes examples and test snippets to help you understand expected inputs and typical outputs, which shortens the learning curve for newcomers. -
20
PyTracking
Visual tracking library based on PyTorch
A general python framework for visual object tracking and video object segmentation, based on PyTorch. Official implementation of the RTS (ECCV 2022), ToMP (CVPR 2022), KeepTrack (ICCV 2021), LWL (ECCV 2020), KYS (ECCV 2020), PrDiMP (CVPR 2020), DiMP (ICCV 2019), and ATOM (CVPR 2019) trackers, including complete training code and trained models. -
21
TensorFlow Machine Learning Cookbook
Code for Tensorflow Machine Learning Cookbook
...The repository contains numerous Python scripts and Jupyter notebooks that demonstrate how to implement machine learning algorithms and neural networks using the TensorFlow framework. Each section focuses on a different aspect of machine learning development, including tensor manipulation, model training, optimization strategies, and data processing techniques. The examples illustrate how TensorFlow operations and tensors can be used to build machine learning pipelines and perform tasks such as regression, classification, and clustering. By combining theoretical explanations with executable code, the project helps developers understand how TensorFlow algorithms operate internally while also providing working examples that can be adapted for real projects. -
22
gpt2-client
Easy-to-use TensorFlow Wrapper for GPT-2 117M, 345M, 774M, etc.
GPT-2 is a Natural Language Processing model developed by OpenAI for text generation. It is the successor to the GPT (Generative Pre-trained Transformer) model trained on 40GB of text from the internet. It features a Transformer model that was brought to light by the Attention Is All You Need paper in 2017. The model has 4 versions - 124M, 345M, 774M, and 1558M - that differ in terms of the amount of training data fed to it and the number of parameters they contain. -
23
MatchZoo
Facilitating the design, comparison and sharing of deep text models
The goal of MatchZoo is to provide a high-quality codebase for deep text matching research, such as document retrieval, question answering, conversational response ranking, and paraphrase identification. With the unified data processing pipeline, simplified model configuration and automatic hyper-parameters tunning features equipped, MatchZoo is flexible and easy to use. Preprocess your input data in three lines of code, keep track parameters to be passed into the model. Make use of MatchZoo customized loss functions and evaluation metrics. Initialize the model, fine-tune the hyper-parameters. ... -
24
InferSent
InferSent sentence embeddings
InferSent is a supervised sentence embedding method that learns universal representations from Natural Language Inference data and transfers well to many downstream tasks. It uses a BiLSTM encoder with max-pooling to produce fixed-length sentence vectors that capture semantics beyond bag-of-words statistics. Trained on large NLI datasets, the embeddings generalize across tasks like sentiment analysis, entailment, paraphrase detection, and semantic similarity with simple linear classifiers.... -
25
AIAlpha
Use unsupervised and supervised learning to predict stocks
AIAlpha is a machine learning project focused on building predictive models for financial markets and algorithmic trading strategies. The repository explores how artificial intelligence techniques can analyze historical financial data and generate predictions about asset price movements. It provides a research-oriented environment where users can experiment with data processing pipelines, model training workflows, and quantitative trading strategies. The project typically involves collecting market data, transforming financial indicators into machine learning features, and training models to identify patterns that may predict market trends. ...
