Search Results for "data capture framework" - Page 3
Sort By:
Showing 256 open source projects for "data capture framework"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
crawly
High-level web crawling and scraping framework for Elixir apps
Crawly is a high-level application framework for crawling websites and extracting structured data using the Elixir programming language. It provides a complete environment for building web crawlers that systematically visit pages, collect information, and transform that data into structured formats for further processing. Crawly is designed for tasks such as data mining, information processing, and building historical archives of web content. -
2
TextUS
A framework in which the system is defined by a xml file
TextUS is written in C++. It runs under MS Windows 2000/XP , Linux or Solaris. It includes a simple HTTPS or HTTP server. Importantly, it provides a framework which is similar with AOP(Aspect Oriented Programming). In tTextUS, a software system is composed of all kinds of modules which are some dll files (in MS Windows) or so files ( in Unix etc.). The system is defined by a xml file which describes the data flow and control flow of every module. -
3
crawlergo
Headless Chrome crawler for collecting URLs for vulnerability scans
crawlergo is a browser-based web crawler designed to collect URLs and request data that can be used by web vulnerability scanning tools. It uses a Chrome headless environment to render web pages and observe behavior during the DOM rendering stage in order to capture as many accessible endpoints as possible. By monitoring the page lifecycle and interacting with web elements, the crawler automatically triggers JavaScript events and navigational actions that would normally occur during real user interaction. ... -
4
pspider
Simple Python framework for building multithreaded web crawlers
PSpider is a lightweight web crawling framework written in Python designed to simplify the development of custom web spiders. It focuses on providing an easy-to-understand architecture while still supporting concurrent crawling for improved performance. It uses a multithreaded model that separates the crawling workflow into several components responsible for fetching, parsing, and saving data. -
$300 Free Credits to Build on Google CloudStart your next project with $300 in free Google Cloud credit. Spin up VMs, run containers, query petabytes in BigQuery, or build agents with Gemini Enterprise Agent Platform. Once your credits are used, keep building with 20+ always-free tier products including Compute Engine, Cloud Storage, GKE, and Cloud Run functions. No commitment required—just sign up and start building.
-
5
The Lemur Project
Search engine and data mining applications and ClueWeb datasets.
The Lemur Project develops search engines, browser toolbars, text analysis tools, and data resources that support research and development of information retrieval and text mining software, including the Indri search engine in C++, the Galago search engine research framework in Java, the RankLib learning to rank library, ClueWeb09 and ClueWeb12 datasets and the Sifaka data mining application. -
6
The KFramework is the first integral SOFEA/SOUI framework for web based business applications using Domain Driven Design. The framework provides a web delivered SWING frontend and a WebServices based backend.
-
7
appcrawler
Automated mobile app crawler and testing tool built on Appium
AppCrawler is an automated mobile application testing tool designed to explore and interact with app user interfaces automatically. Built on top of the Appium automation framework, it systematically crawls through application screens and performs actions such as clicking buttons, navigating menus, and interacting with UI elements to simulate user behavior. It is commonly used for automated functional testing, UI exploration, and detecting crashes or unexpected behaviors in mobile... -
8
StreamAlert
StreamAlert is a serverless, realtime data analysis framework
StreamAlert is a serverless, real-time data analysis framework that empowers you to ingest, analyze, and alert on data from any environment, using data sources and alerting logic you define. Computer security teams use StreamAlert to scan terabytes of log data every day for incident detection and response. Incoming log data will be classified and processed by the rules engine. -
9
JDynamiTe, Dynamic Template in Java
Dynamically generate documents from templates
...Some typical usage domains of JDynamiTe are: - dynamic Web pages creation, - text document generation, - source code generation... In fact, it can be useful in any case where pre-defined documents (templates) have to be dynamically populated with data. The main benefit of JDynamiTe is to allow a true separation between data (content), presentation (container) and content generation code (written in Java). JDynamiTe does not include a specific template language, and it is not a complete framework. It is a simple "brick" in your software architecture, a "glue" between your data model and your presentation model. ... -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
10
Cockpit Next
Add content management functionality to any site
Cockpit is an open-source headless content management system designed to add flexible content management functionality to websites, applications, and digital platforms. The system provides a lightweight backend where developers can create custom content models and manage structured data without imposing a specific front-end framework. Because Cockpit follows an API-first architecture, content can be delivered through REST or GraphQL APIs to any application, including websites, mobile apps, and static site generators. The platform is designed to be simple to install and can be hosted on standard PHP environments with minimal configuration. ... -
11
Abot
Fast and flexible C# framework for building customizable web crawlers
Abot is an open source C# web crawler framework designed to help developers efficiently crawl and process web content. It focuses on speed, flexibility, and extensibility while handling the complex low-level tasks involved in web crawling. It manages essential components such as multithreading, HTTP requests, scheduling, and link parsing so developers can focus on processing the collected data. -
12
Awesome Blazor
Resources for Blazor, a .NET web framework using C#/Razor
Blazor lets you build interactive web UIs using C# instead of JavaScript. Blazor apps are composed of reusable web UI components implemented using C#, HTML, and CSS. Both client and server code is written in C#, allowing you to share code and libraries. Blazor is a feature of ASP.NET, the popular web development framework that extends the .NET developer platform with tools and libraries for building web apps. Blazor can run your client-side C# code directly in the browser, using WebAssembly.... -
13
TRACARDI - Customer Data Platform
TRACARDI free open-source customer data platform
TRACARDI is easy to use and free GUI for Apache Unomi. Unomi is an open source Customer Data Platform that allows anyone to collect user profiles and manage them in a very robust way. TRACARDI with is API first approach enables you to collect data from multiple channels. Regardless if it is web site, mobile app or CRM system open Api let you send data for further processing. Integrate data into one consistent user profile. -
14
WTERP
Factional modular: CRM, SD,MM,WM,HR, Workflo and System configuration.
WTERP is totally free software and It is very suitable for small business management. Which is developed by C#/.net. IIS WebService and SQL Server Community version of the free environment and Database at WinForm-based application framework, including server Web Service, main program framework, organizational structure, authority control, menu navigation, basic data, system parameters, log management, timing tasks, etc.WTERP includes enterprise CRM, SD, MM, WM, OA, HR, Workflow, and other information systems, and aims to help users enhance their management. -
15
lxspider
Educational Python web scraping case collection for many sites
lxSpider is a collection of web scraping examples designed primarily for learning and experimentation with data extraction techniques. It gathers numerous crawler implementations that demonstrate how to collect data from a wide range of websites and online services. It focuses heavily on practical cases that illustrate how different platforms handle requests, authentication parameters, and anti-scraping protections. lxSpider includes examples targeting areas such as e-commerce platforms,... -
16
ruia
Async Python framework for fast and flexible web scraping spiders
Ruia is an asynchronous web scraping micro-framework built for Python that focuses on simplicity, speed, and flexibility when creating web crawlers. Ruia is powered by Python’s asyncio library along with aiohttp, enabling developers to perform concurrent network requests efficiently and scrape data from websites with minimal overhead. Ruia follows a “write less, run faster” philosophy, emphasizing concise code and streamlined spider development. -
17
ast-hook-forjs-re
AST-based JavaScript reverse engineering and variable tracing toolkit
...Using Abstract Syntax Tree (AST) transformations, it injects hook functions into the code to monitor variable assignments and other runtime changes during execution. This allows ast-hook-for-js-RE to capture variable values in memory and store them in a searchable database, effectively enabling variable-level monitoring of program execution. When a user encounters encrypted parameters in network requests, the captured variable data can be searched to determine where those values originated in the code. Once the relevant variable and code location are identified, analysts can trace backward to extract or reproduce the encryption logic used by the site. -
18
DbManager
Database Maganer for freshhh systems
This software implements a database content manager system that works for mysql databases that uses the ORM framework system FRESHHH https://sourceforge.net/p/hierarnodesys/. It is written in PHP (As well as HTML, CSS and JavaScript). Demo: https://freshhh-dbmanager.sourceforge.io/sample/ -
19
KeystoneJS
Node.js CMS and web app framework
...Whether you use it while you're building out your application, or in production as a database content management system, Keystone's Admin UI will save you time and make managing your data easy. Sometimes, async code can get complicated to do simple things. -
20
bento-starter
Full-Stack solution to quickly build PWA applications with Vue.js
...BentoStarter helps you getting started by proposing a default app structure based on best practices. As this project is a template and not a CLI, you can modify the whole project according to your needs. Prerender your different app pages and boost SEO with meta-data description per page. -
21
django-dynamic-scraper
Creating Scrapy scrapers via the Django admin interface
Django Dynamic Scraper (DDS) is an app for Django build on top of the scraping framework Scrapy. While preserving many of the features of Scrapy it lets you dynamically create and manage spiders via the Django admin interface. With Django Dynamic Scraper (DDS) you can define your Scrapy scrapers dynamically via the Django admin interface and save your scraped items in the database you defined for your Django project. -
22
panFMP is a generic framework suitable for harvested XML metadata that is searchable through Apache Lucene without any additional RDBMS. Fields can be defined by XPath allowing for full text queries on all types of fields including numerical ranges. The code was moved to Github: https://github.com/pangaea-data-publisher/panfmp
-
23
one advanced C lib for UNIX and WINDOWS
one server framework
Advanced C/C++ library(ACL) for UNIX-like OS and WIN32 OS, including sync/async/ssl iostream for net/file, thread pool, process pool, db pool, server framework, event, memory, string, array/hash/ring/list, xml and json parser, http/smtp/icmp protocol, SSL/TLS, C unit test, etc -
24
Laravel Larabbs
A forum project base on Laravel
Laravel is a web application framework with expressive, elegant syntax. We believe development must be an enjoyable and creative experience to be truly fulfilling. Laravel takes the pain out of development by easing common tasks used in many web projects. Laravel has the most extensive and thorough documentation and video tutorial library of all modern web application frameworks, making it a breeze to get started with the framework. -
25
WKZombie
WKZombie is a Swift framework for iOS/OSX to navigate within websites
WKZombie is a Swift framework for iOS/OSX to navigate within websites and collect data without the need of a User Interface or API, also known as a Headless browser. It can be used to run automated tests/snapshots and manipulate websites using Javascript. WKZombie is an iOS/OSX web-browser without a graphical user interface. It was developed as an experiment in order to familiarize myself with using functional concepts written in Swift 4.
