Search Results for "data collection algorithm"
Sort By:
Showing 48 open source projects for "data collection algorithm"
View related business solutions-
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
1
ThingsBoard
Device management, data collection, processing and visualization
ThingsBoard is an open-source IoT platform for data collection, processing, visualization, and device management. It enables device connectivity via industry standard IoT protocols, MQTT, CoAP and HTTP and supports both cloud and on-premises deployments. ThingsBoard combines scalability, fault-tolerance and performance so you will never lose your data. Provision, monitor and control your IoT entities in a secure way using rich server-side APIs. -
2
GeoLibre
A lightweight, cloud-native GIS platform
...It also includes advanced tools such as multi-map comparison, SQL workspaces, story maps, field collection, plugins, Python automation, and an AI GIS assistant. -
3
douyin
Open source Douyin crawler for collecting and downloading public data
DouyinCrawler is an open source data collection tool designed to gather publicly available information from the Douyin platform. It demonstrates how to build a Python-based web crawler combined with a graphical interface and command line functionality. It allows users to collect data from various types of Douyin content, including user profiles, videos, hashtags, and music pages. -
4
OmniTools
Self-hosted collection of powerful web-based tools for everyday tasks
OmniTools is a self-hosted web application that bundles a large collection of everyday utilities into a single clean interface you can run on your own infrastructure. It’s designed to replace the random assortment of “free online tools” people use for quick tasks, while avoiding ads, tracking, and the need to upload sensitive files to unknown servers. A key design choice is that file processing happens entirely on the client side, meaning your data stays in your browser instead of being sent to the backend. ... -
Save Up to 91% on Cloud Compute With Spot VMsRun batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.
-
5
Flowsint
Graph-based OSINT investigation platform w visual relationship mapping
Flowsint is an open source OSINT investigation platform designed to help analysts explore and understand relationships between digital entities through a visual graph interface. The platform focuses on reconnaissance and open source intelligence workflows, enabling investigators to map connections between domains, IP addresses, organizations, individuals, and other data points. By presenting these relationships in an interactive graph, Flowsint allows users to quickly identify patterns,... -
6
Jitsu
Jitsu is an open-source Segment alternative
Jitsu is a fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days. Installing Jitsu is a matter of selecting your framework and adding few lines of code to your app. Jitsu is built to be framework agnostic, so regardless of your stack, we have a solution that'll work for your team. Connect data warehouse (Snowflake, Clickhouse, BigQuery, S3, Redshift ot Postgres) and query your data instantly. Jitsu can either stream data in... -
7
Checkmate
Checkmate is an open-source, self-hosted tool
Checkmate is an open-source, self-hosted infrastructure monitoring platform that provides real-time visibility into server health, uptime, response times, and incident activity through a modern web interface. The application continuously checks whether websites and services are accessible and performing optimally, generating alerts and reports when availability or performance degrades. It supports detailed infrastructure monitoring through an optional agent called Capture, which collects... -
8
OpenForm
Open Source TypeForm Clone
...It’s built with modern web technologies including Next.js and Tailwind CSS, offering developers a flexible, mobile-friendly foundation for surveys, feedback forms, and data collection, with support for authentication via Google OAuth and Magic Link. The platform includes a built-in dashboard to review, search, filter, and export responses, making it suitable for both simple and advanced data collection scenarios. OpenForm features multiple form themes and a rich variety of question types, such as rating scales, file uploads, and dropdowns, so form creators can design interactive experiences without heavy custom code. ... -
9
visx
Visualization components
visx a collection of expressive, low-level visualization primitives for React. At Airbnb, we made it a goal to unify our visualization stack across the company and in the process, we created a new project that brings together the power of D3 with the joy of React. visx is split into multiple packages. Start small and use only what you need. Bring your own state management, animation library, or CSS-in-JS solution. Odds are good your React app already has an opinion on how animation, theming,... -
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
10
OpenSEO
Open source alternative to Semrush and Ahrefs
...It provides focused workflows for keyword research, rank tracking, competitor analysis, backlink research, site auditing, and AI visibility monitoring. A modern web interface keeps these tasks accessible without hiding them inside a large collection of unrelated reports. The platform exposes an MCP server so AI agents can work directly with SEO data and reusable agent skills. Users can run a hosted edition or self-host the application with Docker or Cloudflare. Self-hosted installations connect to DataForSEO with the user's own API key and use pay-as-you-go data rather than a mandatory software subscription. ... -
11
JSON Forms
Customizable JSON Schema-based forms with React, Angular and Vue
JSON Forms is a declarative framework for efficiently building form-based web UIs. These UIs are targeted at entering, modifying and viewing data and are usually embedded within an application. Writing HTML templates and Javascript for data binding by hand is hard, especially in applications of reasonable size. Furthermore, a form is often more than just a collection of input fields and more advanced functionality is required, e.g. validation or conditional visibility. JSON Forms utilizes the capabilities of JSON and JSON schema and provides a simple and declarative way of describing forms. ... -
12
MiniSearch
Minimalist web-searching platform with an AI assistant
...The platform supports browser-level integration so users can set it as their default search engine for quick access. Its architecture emphasizes privacy by avoiding tracking and minimizing data collection while still enabling advanced AI features. Overall, MiniSearch targets users who want a self-hosted, Perplexity-style search experience with strong control over data and models. -
13
Handy STT
A free, open source, and extensible speech-to-text application
Handy is a free, open-source, offline speech-to-text application built for privacy, accessibility, and extensibility. Developed using Tauri (Rust + React/TypeScript), it runs natively across Windows, macOS, and Linux while performing local speech recognition without sending any audio to cloud servers. Handy allows users to start transcription instantly using a configurable keyboard shortcut—press to record, release to transcribe—and automatically pastes the resulting text into any active... -
14
MusicFreePlugins
MusicFreePlayPlugin
...The project encourages community contributions, making it possible to expand the available functionality over time. It also includes mechanisms for handling pagination, caching, and data formatting within plugins. Overall, MusicFreePluginsc acts as the backbone of the MusicFree ecosystem, enabling its extensibility and adaptability. -
15
MusicFree
Plug-in, customized, ad-free free music player
...The project supports features such as searching, playback, playlist management, and lyric retrieval through plugin-defined interfaces. It also prioritizes user privacy by storing all data locally and avoiding any form of tracking or data collection. The app includes customization options such as themes and interface adjustments, enhancing user experience. Overall, MusicFree provides a flexible and extensible music platform that adapts to user needs through its plugin ecosystem. -
16
TanStack Query
Powerful asynchronous state management, server-state utilities
Toss out that granular state management, manual refetching and endless bowls of async-spaghetti code. TanStack Query gives you declarative, always-up-to-date auto-managed queries and mutations that directly improve both your developer and user experiences. Writing your data fetching logic by hand is over. Tell TanStack Query where to get your data and how fresh you need it to be and the rest is automatic. It handles caching, background updates and stale data out of the box with... -
17
Rybbit
Open-source and privacy-friendly alternative to Google Analytics
Rybbit is an open-source, privacy-friendly web and product analytics platform positioned as a modern alternative to heavyweight tracking suites. It focuses on a clean UI and intuitive metrics so non-analysts can answer everyday questions without wading through dozens of reports. The data model is event-driven, enabling funnels, retention views, and simple user journeys without complex configuration. Because privacy is a first-class goal, it aims to minimize personal data collection and offers a self-host path so you can keep analytics in your own infrastructure. The project ships with a lightweight snippet and a Docker-based deployment, making it quick to trial or roll out. ... -
18
watercrawl
AI-ready web crawler that extracts and structures website content
...WaterCrawl supports customizable extraction rules so users can focus only on relevant elements while ignoring unnecessary page components. WaterCrawl also offers real-time monitoring capabilities, allowing users to track crawling progress, performance metrics, and errors during large data collection jobs. Developers can integrate the tool into applications through a REST API and multiple client SDKs, enabling automated data pipelines and AI data preparation workflows. -
19
ReUI
Design-forward shadcn kit for interfaces that stand out
...The project includes over 1,000 reusable components organized across dozens of categories, along with custom in-house primitives such as data grids, kanban boards, and advanced filtering systems that are not part of the base shadcn/ui toolkit. Its copy-and-own philosophy eliminates dependency lock-in by allowing developers to directly copy component code into their own repositories, ensuring full control and customization. -
20
Flowbite Svelte
Official Svelte components built for Flowbite and Tailwind CSS
Flowbite Svelte is an official UI component library that brings the Flowbite design system into the Svelte ecosystem, enabling developers to build modern web applications using reusable components powered by Tailwind CSS. The library provides a comprehensive collection of interactive elements such as modals, dropdowns, navigation bars, forms, and data display components, all implemented with Svelte’s reactive model rather than external JavaScript logic. Because Svelte compiles components into efficient JavaScript that directly manipulates the DOM, Flowbite Svelte benefits from high performance and minimal runtime overhead while still offering rich interactivity. ... -
21
wagmi
React Hooks for Ethereum
wagmi is a collection of React Hooks containing everything you need to start working with Ethereum. wagmi makes it easy to "Connect Wallet," display ENS and balance information, sign messages, interact with contracts, and much more, all with caching, request deduplication, and persistence. We create a wagmi Client and pass it to the WagmiConfig React Context. The client is set up to use the ethers Default Provider and automatically connect to previously connected wallets. Next, we use the... -
22
Awesome Privacy
A curated list of privacy & security-focused software and services
Awesome Privacy is a curated directory of privacy-respecting alternatives to mainstream apps and services, organized across many categories like browsers, search, email, messaging, cloud storage, and operating systems. It aims to help you choose tools that reduce tracking, fingerprinting, and data collection without sacrificing usability. Each entry highlights the project’s core properties—such as open source status, end-to-end encryption, and platform availability—so you can evaluate trade-offs quickly. Because product landscapes change fast, the list emphasizes ongoing maintenance and community discussion around quality and trust. ... -
23
AI App Lab
Implementing large models into scenario-based applications
AI App Lab is an open-source platform developed by Volcengine that provides tools, SDKs, and example applications for building real-world AI applications powered by large language models. The project focuses on helping developers bridge the gap between AI models and practical business use cases by offering a structured environment for creating production-ready AI systems. It includes a high-level SDK called Arkitect, which provides workflows and tools for integrating models, plugins, and... -
24
Browserless
Deploy headless browsers in Docker
...The project also provides REST APIs for common automation tasks such as screenshots, PDF generation, scraping, crawling, and content export. Browserless is useful for teams that need scalable browser execution for testing, data collection, rendering, or AI-agent browsing workflows. Its deployment model supports self-hosting, private infrastructure, queues, concurrency controls, and enterprise-oriented configuration. The project’s main value is turning browser automation into a managed service layer that can be reused across applications and workflows. -
25
Manifest
🦞 Take control of your OpenClaw costs
Manifest is an open-source OpenClaw plugin designed to help users take control of their LLM costs through intelligent routing and real-time observability. Instead of sending every request to the same large model, Manifest intercepts each query and evaluates it using a 23-dimension scoring algorithm in under 2 milliseconds. It then routes the request to the most cost-effective and suitable model, potentially reducing costs by up to 90%. The platform includes a real-time dashboard that displays token usage, expenses, messages, and model activity in one place. Unlike cloud-based alternatives, Manifest runs entirely locally, ensuring that prompts, responses, and telemetry data never leave your machine. ...