Search Results for "data quality cleansing"
Sort By:
Showing 89 open source projects for "data quality cleansing"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
1
lakeFS
lakeFS - Git-like capabilities for your object storage
...Easily Collaborate on production data with your team. Automate data quality checks within data pipelines. -
2
BFG Repo-Cleaner
Remove large or troublesome blobs
The BFG is a simpler, faster alternative to git-filter-branch for cleansing bad data out of your Git repository history. You can use it for removing crazy big files, and for removing passwords, credentials and other private data. The git-filter-branch command is enormously powerful and can do things that the BFG can't, but the BFG is much better for the tasks above, because is faster and simpler. The BFG isn't particularily clever, but is focused on making the above tasks easy. ... -
3
Arize Phoenix
Uncover insights, surface problems, monitor, and fine tune your LLM
Phoenix provides ML insights at lightning speed with zero-config observability for model drift, performance, and data quality. Phoenix is an Open Source ML Observability library designed for the Notebook. The toolset is designed to ingest model inference data for LLMs, CV, NLP and tabular datasets. It allows Data Scientists to quickly visualize their model data, monitor performance, track down issues & insights, and easily export to improve. Deep Learning Models (CV, LLM, and Generative) are an amazing technology that will power many of future ML use cases. ... -
4
ODD Platform
First open-source data discovery and observability platform
...Know the impact of each code change with automatic testing. Enjoy lineage and alerts powered with data quality information. -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
Dagster
An orchestration platform for the development, production
Dagster is an orchestration platform for the development, production, and observation of data assets. Dagster as a productivity platform: With Dagster, you can focus on running tasks, or you can identify the key assets you need to create using a declarative approach. Embrace CI/CD best practices from the get-go: build reusable components, spot data quality issues, and flag bugs early. Dagster as a robust orchestration engine: Put your pipelines into production with a robust multi-tenant, multi-tool engine that scales technically and organizationally. ... -
6
SDGym
Benchmarking synthetic data generation methods
...You also customize the process to include your own work. Select any of the publicly available datasets from the SDV project, or input your own data. Choose from any of the SDV synthesizers and baselines. Or write your own custom machine learning model. In addition to performance and memory usage, you can also measure synthetic data quality and privacy through a variety of metrics. Install SDGym using pip or conda. We recommend using a virtual environment to avoid conflicts with other software on your device. -
7
syslog-ng
Log management solution that improves the performance of SIEM
syslog-ng is the log management solution that improves the performance of your SIEM solution by reducing the amount and improving the quality of data feeding your SIEM. With syslog-ng Store Box, you can find the answer. Search billions of logs in seconds using full text queries with Boolean operators to pinpoint critical logs. syslog-ng Store Box provides secure, tamper-proof storage and custom reporting to demonstrate compliance. syslog-ng can deliver data from a wide variety of sources to Hadoop, Elasticsearch, MongoDB, and Kafka as well as many others. syslog-ng flexibly routes log data from X sources to Y destinations. ... -
8
AWS IoT FleetWise Edge
AWS IoT FleetWise Edge Agent
...Improve electric vehicle (EV) battery range estimates with crowdsourced environmental data, such as weather and driving conditions, from nearby vehicles. Collect select data from nearby vehicles and use it to notify drivers of changing road conditions, such as lane closures or construction. Use near real-time data to proactively detect and mitigate fleet-wide quality issues. -
9
OpenTelemetry
OpenTelemetry Go API and SDK
OpenTelemetry-Go is the Go implementation of OpenTelemetry. It provides a set of APIs to directly measure the performance and behavior of your software and send this data to observability platforms. High-quality, ubiquitous, and portable telemetry to enable effective observability. OpenTelemetry is a collection of APIs, SDKs, and tools. Use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior. -
Paessler: Easy to Use With Enterprise Power. Free TrialYou shouldn't have to choose between a monitoring tool that's easy to use and one that's powerful enough for a complex environment. PRTG's low-code interface lets any admin build dashboards, set alerts and monitor devices without scripting, while custom sensors and full API access are there when your team needs deeper control. One platform, no compromise. Download a free 30-day trial now.
-
10
Elementary
Open-source data observability for analytics engineers
Elementary is an open-source data observability solution for data & analytics engineers. Monitor your dbt project and data in minutes, and be the first to know of data issues. Gain immediate visibility, detect data issues, send actionable alerts, and understand the impact and root cause. Generate a data observability report, host it or share with your team. Monitoring of data quality metrics, freshness, volume and schema changes, including anomaly detection. ... -
11
Grafana
Leading open-source visualization and observability platform
Grafana OSS is the leading open-source platform for visualization and observability. It enables teams to query, visualize, alert on, and explore telemetry data from multiple sources in a single interface. With support for 100+ data source plugins—including Prometheus, Loki, Elasticsearch, InfluxDB, SQL/NoSQL databases, and OpenTelemetry—Grafana helps teams correlate metrics, logs, and traces across applications and infrastructure. Users can build interactive dashboards with rich... -
12
Helicone
Open source LLM-Observability Platform for Developers
Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. Supports OpenAI SDK, Vercel AI SDK, Anthropic SDK, LiteLLM, LLamaIndex, LangChain, and more. -
13
Inbucket
Disposable webmail server (similar to Mailinator) with built in SMTP
Inbucket is an email testing application; it will accept messages from any email address and make them available to view via a web interface. When you need to test your webapp's outbound emails with Mailinator but are stuck behind a firewall, Inbucket provides the solution. It allows you to keep your new application development secret until it's time to release it. Inbucket is ideal for validating that emails go out as part of your integration test suite, sending links to coworkers to... -
14
MentDB Projects
Generalized Interoperability and Strong AI
MentDB is an open-source platform driving research into next-generation AI and universal data exchange. Our architecture is built around the revolutionary Mentalese Query Language (MQL). MentDB Weak (Generalized Interoperability): A unified data layer enabling seamless data exchange and application integration (SOA, ETL, Data Quality). We eliminate data silos through a single, generalized data language. MentDB Strong (Strong AI / AGI): The framework for exploring and building Machine Consciousness, free will, and advanced ethical reasoning systems. ... -
15
ngrep strives to provide most of GNU grep's common features,applying them to the network layer. ngrep is a pcap-aware tool that will allow you to specify extended regular expressions to match against data payloads of packets. SUPPORT/REPORTING BUGS: please use https://github.com/jpr5/ngrep/issues Thank you!
-
16
RecordEditor
Editor for Fixed Width, Csv and Existing Xml files.
The RecordEditor is a Data File editor for Flat Files (delimited and fixed field position). It supports Unix / PC / Legacy (e.g. Mainframe) file formats, both Text and binary files. The Editor uses a Record-Layout description to format the files. This is ideal for Fixed width (Text or Binary) files, Cobol Data Files, Mainframe files and complicated Csv files. Cobol Copybooks can be used to format Cobol Data files. As well as an editor, The following utilities are supplied * Formatted... -
17
GeigerLog
Python program for Geiger counters and Environmental Sensors
...Continues to support Geiger counters like GQ Electronic's GMC-300/500/600 , RadMon, RadPro, Gamma-Scout, audio-click and other low-cost Geiger counters, IoT devices, I2C based sensors, GQ EMF devices. GeigerLog reads data from devices, saves them to databases, prints to screen and plots as auto-updated, customizable graphs. Quality control using Statistics, Scatter, Poisson, FFT & Autocorrelation, and more. Gamma Spectra - can be displayed, calibrated, and evaluated in GeigerLog! The Raspberry Pi 4, 5 qualify as GeigerLog hosts. -
18
Cinemagoer
Python package to retrieve and manage data of the IMDb
Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb movie database about movies, people, characters and companies. Platform-independent, it can retrieve data from both the IMDb's web server and a local copy of the whole db. -
19
TCNOpen
Components for IEC61375 standardised communication
...TCN (Train Communication Network) is a series of international standards (IEC61375) developed by Working Group 43 of the IEC (International Electrotechnical Commission), specifying a communication system for the data communication within and between vehicles of a train. It is currently in use on many thousands of trains in the world in order to allow electronic devices to exchange information while operating aboard the same train. TCNOpen follows the Open Source scheme, as the software is jointly developed by participating companies, according to their role, so as to achieve cheaper, quicker and better quality results. -
20
i-doit - Open Source CMDB & ITSM Tool
ITIL conform CMDB solution for documenting your entire IT landscape
i-doit is a web based IT documentation and CMDB. i-doit documents IT-systems and their changes, defines emergency plans, displays vital information and helps to ensure a stable and efficient IT operation: The technical documentation: This means that all information can be organised (details flexibly adjusted), stored and maintained in one place. This allows simple Helpdesk, Support and Planning functions and provides the basis for quality assurance and certification. i-doit allows services to be documented as a logical grouping of objects placed in relation to each other. This provides the basis for (ITIL) processes such as Change and Problem Management, helps to control and manage the SLA (Service Level Agreement) or the communication with users. Its interfaces to other ITSM applications (like Nagios, Request Tracker etc.) allow the entering of data only once in the ITSM environment. i-doit maintains one holistic view of the operational situation. -
21
rag-search
RAG Search API
...It is built to be easily deployable, requiring only environment configuration and dependency installation to run a functional RAG service. The system supports configurable filtering, scoring thresholds, and reranking options, allowing developers to fine-tune retrieval quality. Its architecture is modular, separating handlers, services, and utilities to support customization and extension. Overall, rag-search serves as a practical starter backend for teams building AI search or question-answering applications on their own data. -
22
pj-net-tools
Pure Java Network Tools
Pure Java Network Tools. Free portable cross-platform. Include: 01) ICMP-ping, 02) ICMP-traceroute, 03) DNS-check resolver, 04) Parallel Host TCP-scanner, 05) IP-calculator, 06) Syslog-server, 07) Local ARP-cache request, 08) Parallel Network Ping-Scanner, 09) Snmp-Get concrete value utility. 10) Parallel ICMP flood utility. 11) Parallel UDP flood utility. 12) TFTP-server. 13) NTP-server. 14) Host TCP-ping. 15) SNMP-MRTG traffic monitor. 16) Anonymous... -
23
mySQM+ DIY SQM WEATHER STATION
ESP32 mySQM+ Sky-Meter/Weather Station with loads of features
mySQM+ Sky Quality Meter/Weather Station (c) R Brown, 2020-2025, All rights reserved. Permission is granted for personal and Academic use only. Code or portions of code may not be copied or used without appropriate credit given to author(s). EXPANSION BOARDS - You can expand this project using the two types of project expansion boards at https://sourceforge.net/projects/myesp-boards/ -
24
UniversalTextExtractor
Command-line toolset for extracting text from files
Command-line toolset for extracting text from files (documents, images, archives) into SQLite with OCR support. Simple, expandable, one shell script only. -
25
FileHashler
cross-platform (Java) file encryption and archiving tool
Java based API (library) and console utility for cross-platform file and directory encryption and archiving. Encrypts single files up to 9.2 EB length. Directories can be encrypted as sets of single files in a single action. Uses Twofish (CBC) and SHA-256. Security level is strong and benefits from the PWS3 (Password Safe) encrypted file format. The level can be lowered to 128-bit key-length (normally 256).
