Search Results for "data capture framework" - Page 13
Sort By:
Showing 557 open source projects for "data capture framework"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
1
Evals
Evals is a framework for evaluating LLMs and LLM systems
The openai/evals repository is a framework and registry for evaluating large language models and systems built with LLMs. It’s designed to let you define “evals” (evaluation tasks) in a structured way and run them against different models or agents, with the ability to score, compare, and analyze results. The framework supports templated YAML eval definitions, solver-based evaluations, custom metrics, and composition of multi-step evaluations. -
2
Fondant
Production-ready data processing made easy and shareable
Fondant is a modular, pipeline-based framework designed to simplify the preparation of large-scale datasets for training machine learning models, especially foundation models. It offers an end-to-end system for ingesting raw data, applying transformations, filtering, and formatting outputs—all while remaining scalable and traceable. Fondant is designed with reproducibility in mind and supports containerized steps using Docker, making it easy to share and reuse data processing components. ... -
3
InternVL
A Pioneering Open-Source Alternative to GPT-4o
...The project focuses on scaling vision models and aligning them with large language models so that they can perform tasks involving both visual and textual information. InternVL is trained on massive collections of image-text data, enabling it to learn representations that capture both visual patterns and semantic meaning. The model supports a wide variety of tasks, including visual perception, image classification, and cross-modal retrieval between images and text. It can also be connected to language models to enable conversational interfaces that understand images, videos, and other visual content. ... -
4
Qvid
Stream low latency video from your desktop or webcam over TCP/IP
Qvid is a demo video streaming application for Windows (MacOs support limited and currently broken), written in Python. It allows you to capture screenshots of your desktop, webcam, and selected windows programs. The captured images are compressed and sent as a continuous stream over a TCP connection to a single machine. Developed alongside https://sourceforge.net/projects/netjoy/ to allow for off-site, single and multiplayer, game play over an internet connection. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
dude uncomplicated data extraction
dude uncomplicated data extraction: A simple framework
Dude is a very simple framework for writing web scrapers using Python decorators. The design, inspired by Flask, was to easily build a web scraper in just a few lines of code. Dude has an easy-to-learn syntax. Dude is currently in Pre-Alpha. Please expect breaking changes. You can run your scraper from terminal/shell/command-line by supplying URLs, the output filename of your choice and the paths to your python scripts to dude scrape command. -
6
spider_collection
Collection of Python web scraping scripts for data extraction tasks
spider_collection is a collection of Python web crawler scripts created primarily for experimentation, learning, and practical scraping tasks. spider_collection gathers multiple independent spiders designed to collect data from different platforms and services, demonstrating a variety of scraping techniques and workflows. These crawlers make use of common Python scraping tools such as requests, parsel, BeautifulSoup, and the Scrapy framework to extract structured information from web pages. Several scripts also incorporate multi-threading and proxy usage to improve scraping efficiency and help avoid common anti-scraping limitations. ... -
7
Canopy
Retrieval Augmented Generation (RAG) framework
Canopy is an open-source retrieval-augmented generation (RAG) framework developed by Pinecone to simplify the process of building applications that combine large language models with external knowledge sources. The system provides a complete pipeline for transforming raw text data into searchable embeddings, storing them in a vector database, and retrieving relevant context for language model responses. -
8
TaskWeaver
A code-first agent framework for seamlessly planning analytics tasks
TaskWeaver is a multi-agent AI framework designed for orchestrating autonomous agents that collaborate to complete complex tasks. -
9
higgsfield
Fault-tolerant, highly scalable GPU orchestration
Higgsfield is an open-source, fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters, such as Large Language Models (LLMs). -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
LangChain Extract
Did you say you like data?
LangChain Extract is an open-source reference application designed to demonstrate how large language models can be used to extract structured data from unstructured text and document files. The project implements a lightweight web service that allows developers to define extraction schemas and apply them to various sources such as plain text, HTML, or PDF documents. Built using FastAPI and the LangChain framework, the application exposes a REST API that can process documents and return structured outputs that match user-defined JSON schemas. ... -
11
local-llm
Run LLMs locally on Cloud Workstations
local-llm is a development framework that enables developers to run large language models locally within Google Cloud Workstations or standard environments without requiring GPU hardware. It focuses on making generative AI development more accessible by leveraging quantized models and CPU-based execution, eliminating the dependency on expensive GPU infrastructure. -
12
dbt-re-data
re_data - fix data issues before your users & CEO would discover them
re_data is an open-source data reliability framework for the modern data stack. Currently, re_data focuses on observing the dbt project (together with underlaying data warehouse - Postgres, BigQuery, Snowflake, Redshift). Data transformations in re_data are implemented and exposed as models & macros in this dbt package. Gather all relevant outputs about your data in one place using our cloud. -
13
GhostTrack
Useful tool to track location or mobile number
...It provides a menu-driven interface that allows users to select different tracking modes, making the tool accessible even to those with limited scripting experience. GhostTrack can be combined with complementary tools such as Seeker to enhance IP capture workflows and improve investigative coverage. Because it focuses on aggregating open data rather than exploiting systems, the tool positions itself within the information-gathering phase of security assessments and OSINT research. -
14
ToRA
Tool-integrated Reasoning LLM Agents
...This approach allows the model to reason step by step in natural language and then execute precise calculations or code through tool calls, creating a hybrid reasoning workflow. The framework was designed to address known weaknesses of large language models in mathematical problem solving and formal reasoning tasks. Training data includes tool-use trajectories that teach the model when to reason verbally and when to delegate tasks to specialized tools. -
15
MASVS
The OWASP MASVS (Mobile Application Security Verification Standard)
The OWASP Mobile Application Security Verification Standard (MASVS) is a comprehensive security standard for mobile applications, providing guidelines and a checklist for secure mobile app development. -
16
Digital Forensics Guide
Learn all about Digital Forensics and Computer Forensics
...Alongside conceptual explanations, the guide includes practical examples with widely used tools (like Autopsy, Volatility, Sleuth Kit, and network analysis suites), illustrating how investigations proceed from initial data capture to final analysis. The goal is to provide both a learning path and a quick reference for real-world casework, bridging the gap between academic theory and operational practice. -
17
Obsei
Obsei is a low code AI powered automation tool
Obsei is an automated no-code/low-code AI-powered text observation and analysis framework, designed for extracting insights from unstructured text data such as social media, reviews, and logs. -
18
autollm
Ship RAG based LLM web apps in seconds
...AutoLLM supports a broad range of language models and vector databases, which makes it useful for teams that want flexibility without rewriting their application architecture every time they switch providers. The framework also includes built-in readers for multiple content sources such as PDFs, DOCX files, notebooks, websites, and other document types, which helps shorten the time between raw data and a working knowledge application. -
19
Mara Pipelines
A lightweight opinionated ETL framework, halfway between plain scripts
This package contains a lightweight data transformation framework with a focus on transparency and complexity reduction. Data integration pipelines as code: pipelines, tasks and commands are created using declarative Python code. PostgreSQL as a data processing engine. Extensive web ui. The web browser as the main tool for inspecting, running and debugging pipelines. GNU make semantics. -
20
DB-GPT-Hub
A repository that contains models, datasets, and fine-tuning
...The project serves as a specialized extension of the broader DB-GPT ecosystem, focusing on the preparation and evaluation of models capable of translating natural language questions into structured database queries. It offers a modular framework that supports data preparation, model fine-tuning, benchmarking, and inference for Text-to-SQL systems. The repository includes datasets and experiment configurations that allow researchers to train models on real database schemas and evaluate them using standardized benchmarks. Its design encourages experimentation with different large language models and fine-tuning techniques, including parameter-efficient training approaches. -
21
Adala
Adala: Autonomous DAta (Labeling) Agent framework
Adala is a data-centric AI framework focused on dataset curation, annotation, and validation. It helps AI teams manage high-quality training datasets by providing tools for data auditing, error detection, and quality assessment. -
22
towhee
Framework that is dedicated to making neural data processing
Towhee is an open-source machine-learning pipeline that helps you encode your unstructured data into embeddings. You can use our Python API to build a prototype of your pipeline and use Towhee to automatically optimize it for production-ready environments. From images to text to 3D molecular structures, Towhee supports data transformation for nearly 20 different unstructured data modalities. We provide end-to-end pipeline optimizations, covering everything from data decoding/encoding, to... -
23
LLaMA-MoE
Building Mixture-of-Experts from LLaMA with Continual Pre-training
LLaMA-MoE is an open-source project that builds mixture-of-experts language models from LLaMA through expert partitioning and continual pre-training. The repository is centered on making MoE research more accessible by offering smaller and more affordable models with only about 3.0 to 3.5 billion activated parameters, which helps reduce deployment and experimentation costs. Its architecture works by splitting LLaMA feed-forward networks into sparse experts and adding gating mechanisms so... -
24
Alpaca-CoT
We unified the interfaces of instruction-tuning data
Alpaca-CoT is an open research project focused on improving reasoning capabilities in language models through chain-of-thought training data. The project builds upon the Alpaca instruction-tuning approach by introducing datasets and methods that encourage models to produce intermediate reasoning steps when solving problems. Instead of generating answers directly, the model learns to produce logical reasoning sequences that lead to the final solution. This chain-of-thought supervision helps... -
25
YiVal
Your Automatic Prompt Engineering Assistant for GenAI Applications
YiVal is an open-source framework designed to automate prompt engineering and evaluation workflows for generative AI applications, enabling developers to systematically improve the performance of large language models. It focuses on experimentation and optimization by allowing users to test multiple prompt variations, configurations, and model parameters in parallel, then evaluate their outputs using structured metrics and scoring systems.
