Search Results for "data capture framework"
Sort By:
179 projects for "data capture framework" with 2 filters applied:
-
$300 Free Credits to Build on Google CloudStart your next project with $300 in free Google Cloud credit. Spin up VMs, run containers, query petabytes in BigQuery, or build agents with Gemini Enterprise Agent Platform. Once your credits are used, keep building with 20+ always-free tier products including Compute Engine, Cloud Storage, GKE, and Cloud Run functions. No commitment required—just sign up and start building.
-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
1
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. -
2
Acl
A powerful server and network library, including coroutine
The Acl (Advanced C/C++ Library) project a is powerful multi-platform network communication library and service framework, supporting LINUX, WIN32, Solaris, FreeBSD, MacOS, AndroidOS, iOS. Many applications written by Acl run on these devices with Linux, Windows, iPhone and Android and serve billions of users. There are some important modules in Acl project, including network communcation, server framework, application protocols, multiple coders, etc. The common protocols such as... -
3
gain
Asyncio-based Python framework for building fast web crawling spiders
Gain is a Python web crawling framework designed to simplify the process of building efficient and scalable web scrapers. It is built on top of asynchronous technologies such as asyncio, aiohttp, and uvloop to support high-performance crawling with concurrent network requests. It provides a structured framework for creating spiders that can navigate websites, extract structured data, and process the collected results. -
4
FEAPDER
Powerful Python crawler framework for scalable web scraping tasks
feapder is a Python-based web crawling framework designed to simplify the process of building scalable and efficient web scrapers. It focuses on providing a developer-friendly environment that makes it easier to create, run, and manage crawlers for a variety of data collection tasks. It includes several built-in spider types, such as AirSpider, Spider, TaskSpider, and BatchSpider, which address different crawling scenarios ranging from lightweight scraping to distributed and batch-based jobs. feapder supports features such as breakpoint resume, allowing crawlers to continue from where they stopped without losing progress. ... -
Secure File Transfer for Windows with Cerberus by RedwoodCerberus supports unlimited users and connections on a single IP, with built-in encryption, 2FA, and a browser-based web client — all deployable in under 15 minutes with a 25-day free trial.
-
5
Laravel Sharp
Laravel 10+ Content management framework
Sharp is a content management framework, a toolset that provides help to build a CMS section in a website, with some rules in mind. The public website should not have any knowledge of the CMS, the CMS is a part of the system, not the center of it. In fact, removing the CMS should not have any effect on the project. Content administrators should work with their data and terminology, not CMS terms. -
6
Geziyor
Blazing fast Go framework for web crawling and data scraping tasks
Geziyor is a high-performance web crawling and web scraping framework built for the Go programming language. It is designed to help developers crawl websites and extract structured information from web pages efficiently. It focuses on speed and scalability, allowing large numbers of requests to be processed concurrently. Geziyor supports use cases such as data mining, monitoring web content, and automated testing workflows. -
7
Scrapling
An adaptive Web Scraping framework
Scrapling is an adaptive web scraping framework designed to handle everything from a single HTTP request to large-scale, concurrent crawls. Built for modern websites, it intelligently adapts to structural changes by automatically relocating elements when page layouts update. The framework includes advanced fetchers capable of bypassing anti-bot protections such as Cloudflare Turnstile using stealth and browser automation techniques. -
8
QueryList
Progressive PHP web crawler framework with jQuery-like DOM parsing
QueryList is an extensible PHP web scraping and crawling framework designed to extract and process data from web pages. It provides a simple and expressive API that allows developers to collect structured information from HTML documents using familiar DOM traversal techniques. It is built on top of phpQuery and uses CSS3 selectors similar to those found in jQuery, making it easy for developers to query and manipulate page elements during scraping tasks. -
9
Sanity
Rapidly configure content workspaces powered by structured content
...Instead of using predefined content templates, Sanity allows developers to define schemas in code that determine how content is structured and stored. The platform stores data in a real-time backend called the Content Lake, enabling collaborative editing and instant updates across connected applications. Because the system separates content management from presentation, developers can use any front-end framework to display the data. Sanity also includes APIs and query tools that allow developers to retrieve content dynamically and integrate it into websites, mobile apps, and other digital services. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
10
Python API for JMComic
Python crawler and API for downloading JMComic albums and images
JMComic-Crawler-Python is a Python library and crawler framework designed to programmatically access and download comic content from the JMComic platform. It provides a structured API that allows developers to retrieve albums, chapters, and images using simple Python code while handling the necessary network requests and data processing behind the scenes. It supports both web-based and mobile API interfaces, enabling flexible interaction with the platform depending on the available endpoints. ... -
11
Internet Pi
Raspberry Pi config for all things Internet
Internet Pi is an Ansible playbook and configuration framework for a Raspberry Pi (or similar single-board computer) that transforms it into a network-infrastructure monitoring and ad-blocking unit. The typical use case is: plug the Pi into your home network, install Pi-hole for DNS-level ad-blocking and privacy, and deploy Prometheus + Grafana to monitor internet connection quality, ping latency, speed tests, and uptime trends. The intention is both practical (reduce ads, manage DNS) and... -
12
DotnetSpider
Lightweight .NET framework for fast web crawling and data scraping
DotnetSpider is a web crawling and data extraction framework built on the .NET Standard platform. It is designed to help developers create efficient and scalable crawlers for collecting structured data from websites. It provides a high-level API that simplifies the process of defining spiders, managing requests, and extracting content from web pages. Developers can create custom spiders by extending base classes and configuring pipelines that handle downloading, parsing, and storing collected data. ... -
13
ProM is the comprehensive, extensible framework for process mining. Process Mining deals with the a-posteriori analysis of (business) processes using enactment logs.
-
14
dude uncomplicated data extraction
dude uncomplicated data extraction: A simple framework
Dude is a very simple framework for writing web scrapers using Python decorators. The design, inspired by Flask, was to easily build a web scraper in just a few lines of code. Dude has an easy-to-learn syntax. Dude is currently in Pre-Alpha. Please expect breaking changes. You can run your scraper from terminal/shell/command-line by supplying URLs, the output filename of your choice and the paths to your python scripts to dude scrape command. -
15
spider_collection
Collection of Python web scraping scripts for data extraction tasks
spider_collection is a collection of Python web crawler scripts created primarily for experimentation, learning, and practical scraping tasks. spider_collection gathers multiple independent spiders designed to collect data from different platforms and services, demonstrating a variety of scraping techniques and workflows. These crawlers make use of common Python scraping tools such as requests, parsel, BeautifulSoup, and the Scrapy framework to extract structured information from web pages. Several scripts also incorporate multi-threading and proxy usage to improve scraping efficiency and help avoid common anti-scraping limitations. ... -
16
Foswiki is an Enterprise wiki, typically used to run a collaboration platform, knowledge base or document management system. Users can create wiki applications using the Topic Markup Language, and developers can extend its functionality with plugins. Foswiki stands for "Free and Open Source" wiki to emphasize its commitment to Open Source software.
-
17
ZnetDK 4 Mobile
Responsive Web App development full-stack framework in PHP and JS
Develop easily in PHP, MySQL and JavaScript your business web app for mobile and tablet devices from a ready-to-customize starter app, fully Responsive and Installable (PWA). No need to start up from scratch. You just have to add the views that fullfill your business requirements and connect them to the user navigation menu. All the features expected for a mobile business application are already developed! Authenticate your users through the built-in login form. Show them only the pages... -
18
Cerberus Content Management System 6
Cerberus Content Management System
Cerberus Content Management System is a dynamic, secure and infinitely expandable CMS designed after a Unix-Like model complete with a Unix-Like Kernel File named: Cerberus. It is a custom written Web Application Framework ( W.A.F. ) with a consistent and custom written Pre-Hyper-Text-Post-Processor Programming Code Framework ( P.C.F. ). This Web Application Software Project' aim is to be the fastest and most secure Web Application Framework, Web Application Programming Code Framework, Text, Voice and Video Communications Platform and Content Management System in the world. ... -
19
LazyJ is a rapid development framework for Java-based WEB applications. It includes a very fast template engine, page and data caching, database wrapper with connection pooling, database-backed sessions, time-bounded execution and other nice features
-
20
CerberusCMS5
Cerberus Content Management System
Cerberus Content Management System is a dynamic, secure and infinitely expandable CMS designed after a Unix-Like model. It is a custom written Web Application Framework ( W.A.F. ) with a consistent and custom written Pre-Hyper-Text-Post-Processor Programming Code Framework ( P.C.F. ). This Web Application Software Project' aim is to be the fastest and most secure Web Application Framework, Web Application Programming Code Framework, Text, Voice and Video Communications Platform and Content Management System in the world. ... -
21
JForum2
Bug fixes and enhancements for JForum 2.x
JForum is a powerful and robust discussion board system implemented in Java. It provides an attractive interface, an efficient forum engine, an easy to use administrative panel, an advanced permission control system and much more. Built from the ground up around an MVC framework , it can be deployed on any Servlet 3.1 container or application server running at least Java 8, such as Tomcat, Jetty and JBoss/WildFly. Its clean design and implementation make JForum easy to customize and... -
22
Magnolia CMS
Java CMS that delivers smartphone simplicity on an enterprise scale
Magnolia is an open Java CMS that delivers smartphone simplicity on an enterprise-scale. The mobile-inspired HTML5 user interface works through customisable, task-focused Apps, a notification stream called the Pulse, and a system of Favorites for quickly accessing the workspace. Magnolia CMS allows organizations to orchestrate online services, sales and marketing across all digital channels, maximizing the impact of every touchpoint. Magnolia’s open-source technology, support of open... -
23
crawly
High-level web crawling and scraping framework for Elixir apps
Crawly is a high-level application framework for crawling websites and extracting structured data using the Elixir programming language. It provides a complete environment for building web crawlers that systematically visit pages, collect information, and transform that data into structured formats for further processing. Crawly is designed for tasks such as data mining, information processing, and building historical archives of web content. -
24
crawlergo
Headless Chrome crawler for collecting URLs for vulnerability scans
crawlergo is a browser-based web crawler designed to collect URLs and request data that can be used by web vulnerability scanning tools. It uses a Chrome headless environment to render web pages and observe behavior during the DOM rendering stage in order to capture as many accessible endpoints as possible. By monitoring the page lifecycle and interacting with web elements, the crawler automatically triggers JavaScript events and navigational actions that would normally occur during real user interaction. ... -
25
pspider
Simple Python framework for building multithreaded web crawlers
PSpider is a lightweight web crawling framework written in Python designed to simplify the development of custom web spiders. It focuses on providing an easy-to-understand architecture while still supporting concurrent crawling for improved performance. It uses a multithreaded model that separates the crawling workflow into several components responsible for fetching, parsing, and saving data.
