Search Results for "dataset"
Sort By:
Showing 64 open source projects for "dataset"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
1
Datumaro
Dataset Management Framework, a Python library and a CLI tool to build
Datumaro is a flexible Python-based dataset management framework and command-line tool for building, analyzing, transforming, and converting computer vision datasets in many popular formats. It supports importing and exporting annotations and images across a wide variety of standards like COCO, PASCAL VOC, YOLO, ImageNet, Cityscapes, and many more, enabling easy integration with different training pipelines and tools. -
2
Fluid
Fluid, elastic data abstraction and acceleration for BigData/AI apps
Fluid, elastic data abstraction and acceleration for BigData/AI applications in the cloud. Provide DataSet abstraction for underlying heterogeneous data sources with multidimensional management in a cloud environment. Enable dataset warmup and acceleration for data-intensive applications by using a distributed cache in Kubernetes with observability, portability, and scalability. Taking characteristics of application and data into consideration for cloud application/dataset scheduling to improve the performance. -
3
Parquet.jl
Julia implementation of Parquet columnar file format reader
A parquet file or dataset can be loaded using the read_parquet function. A parquet dataset is a directory with multiple parquet files, each of which is a partition belonging to the dataset. -
4
CleanVision
Automatically find issues in image datasets
CleanVision automatically detects potential issues in image datasets like images that are: blurry, under/over-exposed, (near) duplicates, etc. This data-centric AI package is a quick first step for any computer vision project to find problems in the dataset, which you want to address before applying machine learning. CleanVision is super simple -- run the same couple lines of Python code to audit any image dataset! The quality of machine learning models hinges on the quality of the data used to train them, but it is hard to manually identify all of the low-quality data in a big dataset. ... -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
5
CellTypist
A tool for semi-automatic cell type classification, harmonization
CellTypist is an automated tool for cell type classification, harmonization, and integration. Classification, transfer cell type labels from the reference to query dataset. Harmonization, match and harmonize cell types defined by independent datasets. integration, integrate cell and cell types with supervision from harmonization. CellTypist recapitulates cell type structure and biology of independent datasets. Regularised linear models with Stochastic Gradient Descent provide a fast and accurate prediction. ... -
6
HDF5.jl
Save and load data in the HDF5 file format from Julia
HDF5 stands for Hierarchical Data Format v5 and is closely modeled on file systems. In HDF5, a "group" is analogous to a directory, a "dataset" is like a file. HDF5 also uses "attributes" to associate metadata with a particular group or dataset. HDF5 uses ASCII names for these different objects, and objects can be accessed by Unix-like pathnames, e.g., "/sample1/tempsensor/firsttrial" for a top-level group "sample1", a subgroup "tempsensor", and a dataset "firsttrial". ... -
7
Cleanlab
The standard data-centric AI package for data quality and ML
cleanlab helps you clean data and labels by automatically detecting issues in a ML dataset. To facilitate machine learning with messy, real-world data, this data-centric AI package uses your existing models to estimate dataset problems that can be fixed to train even better models. cleanlab cleans your data's labels via state-of-the-art confident learning algorithms, published in this paper and blog. See some of the datasets cleaned with cleanlab at labelerrors.com. -
8
PyTorch Forecasting
Time series forecasting with PyTorch
PyTorch Forecasting aims to ease state-of-the-art time series forecasting with neural networks for both real-world cases and research alike. The goal is to provide a high-level API with maximum flexibility for professionals and reasonable defaults for beginners. A time series dataset class that abstracts handling variable transformations, missing values, randomized subsampling, multiple history lengths, etc. A base model class that provides basic training of time series models along with logging in tensorboard and generic visualizations such actual vs predictions and dependency plots. Multiple neural network architectures for timeseries forecasting that have been enhanced for real-world deployment and come with in-built interpretation capabilities. ... -
9
Charts.css
Open source CSS framework for data visualization
Charts.css is a modern CSS framework. It uses CSS utility classes to style HTML elements as charts. No dependencies. 72kb file size. Less than 6kb gzipped file size! Visualization helps end-users understand data. Charts.css help frontend developers turn data into beautiful charts and graphs using simple CSS classes. The data is structured using semantic HTML tags and styled using CSS classes which change the visual representation displayed to the end-user. The framework offers developers... -
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
10
HyperTools
A Python toolbox for gaining geometric insights
...HyperTools is designed to facilitate dimensionality reduction-based visual explorations of high-dimensional data. The basic pipeline is to feed in a high-dimensional dataset (or a series of high-dimensional datasets) and, in a single function call, reduce the dimensionality of the dataset(s) and create a plot. -
11
React Chart.js
React components for Chart.js, the most popular charting library
...In order to improve performance, offer new features, and improve maintainability, it was necessary to break backwards compatibility, but we aimed to do so only when worth the benefit. You will find that any event which causes the chart to re-render, such as hover tooltips, etc., will cause the first dataset to be copied over to other datasets, causing your lines and bars to merge together. This is because to track changes in the dataset series, the library needs a key to be specified. If none is found, it can't tell the difference between the datasets while updating. Specify a different property to be used as a key by passing a datasetIdKey prop to your chart component. -
12
AWS Data Wrangler
Pandas on AWS, easy integration with Athena, Glue, Redshift, etc.
An AWS Professional Service open-source python initiative that extends the power of Pandas library to AWS connecting DataFrames and AWS data-related services. Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON, and EXCEL). Built on top of other open-source projects like Pandas, Apache Arrow and Boto3, it offers abstracted functions to execute... -
13
CSV Lint
CSV Lint plug-in for Notepad++ for syntax highlighting
...Use CSV Lint for metadata discovery, technical data validation, and reformatting on tabular data files. It is not meant to be a replacement for spreadsheet programs like Excel or SPSS, but rather it's a quality control tool to examine, verify or polish up a dataset before further processing. -
14
ggplot2
An implementation of the Grammar of Graphics in R
...With ggplot2 you simply provide the data, tell ggplot2 how to map variables to aesthetics, what graphical primitives to use, and it will take care of the rest. ggplot2 is over 10 years old and is used by hundreds of thousands of people all over the world for plotting. In most cases using ggplot2 starts with supplying a dataset and aesthetic mapping (with aes()); adding on layers (like geom_point() or geom_histogram()), scales (like scale_colour_brewer()), and faceting specifications (like facet_wrap()); and finally, coordinating systems. ggplot2 has a rich ecosystem of community-maintained extensions for those looking for more innovation. ggplot2 is a part of the tidyverse, an ecosystem of R packages designed for data science. -
15
DataChain
AI-data warehouse to enrich, transform and analyze unstructured data
Datachain enables multimodal API calls and local AI inferences to run in parallel over many samples as chained operations. The resulting datasets can be saved, versioned, and sent directly to PyTorch and TensorFlow for training. Datachain can persist features of Python objects returned by AI models, and enables vectorized analytical operations over them. The typical use cases are data curation, LLM analytics and validation, image segmentation, pose detection, and GenAI alignment. Datachain... -
16
UMAP.jl
Uniform Manifold Approximation and Projection (UMAP) implementation
A pure Julia implementation of the Uniform Manifold Approximation and Projection dimension reduction algorithm. The umap function takes two arguments, X (a column-major matrix of shape (n_features, n_samples)), n_components (the number of dimensions in the output embedding), and various keyword arguments. -
17
JLD2
HDF5-compatible file format in pure Julia
JLD2 saves and loads Julia data structures in a format comprising a subset of HDF5, without any dependency on the HDF5 C library. JLD2 is able to read most HDF5 files created by other HDF5 implementations supporting HDF5 File Format Specification Version 3.0 (i.e. libhdf5 1.10 or later) and similarly, those should be able to read the files that JLD2 produces. JLD2 provides read-only support for files created with the JLD package. The save and load functions, provided by FileIO, provide a... -
18
GeoNode
GeoNode is an open source platform for geospatial data
...It brings together mature and stable open-source software projects under a consistent and easy-to-use interface allowing non-specialized users to share data and create interactive maps. Data management tools built into GeoNode allow for integrated creation of data, metadata, and map visualization. Each dataset in the system can be shared publicly or restricted to allow access to only specific users. Social features like user profiles and commenting and rating systems allow for the development of communities around each platform to facilitate the use, management, and quality control of the data the GeoNode instance contains. It is also designed to be a flexible platform that software developers can extend, modify or integrate against to meet requirements in their own applications. -
19
XY
Ultra-fast and customizable Python charts
XY is an interactive Python visualization library designed to handle both everyday charts and extremely large datasets. Charts can be created through a declarative API or familiar Matplotlib-style conventions. A Rust processing core dynamically determines how much data the current screen resolution actually needs to display. Large datasets can be rendered as bounded density or decimated views, while zooming retrieves finer details and original rows. Interactive capabilities include panning,... -
20
Sweetviz
Visualize and compare datasets, target values and associations
...Its goal is to help quick analysis of target characteristics, training vs testing data, and other such data characterization tasks. Shows how a target value (e.g. "Survived" in the Titanic dataset) relates to other features. Sweetviz integrates associations for numerical (Pearson's correlation), categorical (uncertainty coefficient) and categorical-numerical (correlation ratio) datatypes seamlessly, to provide maximum information for all data types. Automatically detects numerical, categorical and text features, with optional manual overrides. min/max/range, quartiles, mean, mode, standard deviation, sum, median absolute deviation, coefficient of variation, kurtosis, skewness. -
21
Gretel Synthetics
Synthetic data generators for structured and unstructured text
...Share, create, and augment data with cutting-edge generative AI. Generate unlimited data in minutes with synthetic data delivered as-a-service. Synthesize data that are as good or better than your original dataset, and maintain relationships and statistical insights. Customize privacy settings so that data is always safe while remaining useful for downstream workflows. Ensure data accuracy and privacy confidently with expert-grade reports. Need to synthesize one or multiple data types? We have you covered. Even take advantage or multimodal data generation. ... -
22
Population Shift Monitoring
Monitor the stability of a Pandas or Spark dataframe
popmon is a package that allows one to check the stability of a dataset. popmon works with both pandas and spark datasets. popmon creates histograms of features binned in time-slices, and compares the stability of the profiles and distributions of those histograms using statistical tests, both over time and with respect to a reference. It works with numerical, ordinal, categorical features, and the histograms can be higher-dimensional, e.g. it can also track correlations between any two features. popmon can automatically flag and alert on changes observed over time, such as trends, shifts, peaks, outliers, anomalies, changing correlations, etc, using monitoring business rules. ... -
23
whylogs
The open standard for data logging
whylogs is an open-source library for logging any kind of data. With whylogs, users are able to generate summaries of their datasets (called whylogs profiles) which they can use to track changes in their dataset Create data constraints to know whether their data looks the way it should. Quickly visualize key summary statistics about their datasets. whylogs profiles are the core of the whylogs library. They capture key statistical properties of data, such as the distribution (far beyond simple mean, median, and standard deviation measures), the number of missing values, and a wide range of configurable custom metrics. ... -
24
Digital Earth Australia notebooks
Repository for Digital Earth Australia Jupyter Notebooks
The knowledge hub brings together information about Digital Earth Australia’s products and services, allowing you to utilize our free and open-source satellite imagery archive. Browse our catalog of data products to find supporting information and ways to access the data. The Digital Earth Australia notebooks and tools repository (dea-notebooks) hosts Jupyter Notebooks, Python scripts and workflows for analyzing Digital Earth Australia (DEA) satellite data and derived products. This... -
25
FiftyOne
The open-source tool for building high-quality datasets
...Improving data quality and understanding your model’s failure modes are the most impactful ways to boost the performance of your model. FiftyOne provides the building blocks for optimizing your dataset analysis pipeline. Use it to get hands-on with your data, including visualizing complex labels, evaluating your models, exploring scenarios of interest, identifying failure modes, finding annotation mistakes, and much more! Surveys show that machine learning engineers spend over half of their time wrangling data, but it doesn't have to be that way.