Search Results for "data scraper website"
Sort By:
Showing 29 open source projects for "data scraper website"
View related business solutions-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
1
LLM Scraper
Extract structured data from webpages using LLM-powered scraping
LLM Scraper is a TypeScript library designed to extract structured data from webpages using large language models. Instead of relying on fragile HTML selectors or manual parsing rules, the tool interprets webpage content with language models and converts it into structured data according to a defined schema. Developers can specify the data structure using tools such as Zod or JSON Schema, enabling the model to extract relevant information directly into typed objects. ... -
2
ScrapeGraphAI
Python scraper based on AI
Extracting content from websites and local documents using LLM. ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.). Just say which information you want to extract and the library will do it for you. -
3
FinMind
Open Data, more than 50 financial data
In the era of big data, data is the foundation of everything. We collect more than 50 kinds of Taiwan stock related information and provide download, online analysis, and backtesting. Regardless of the program, you can download data through the api provided by FinMind, or you can download data directly from the website. After data is available, statistical analysis, regression analysis, time series analysis, machine learning, and deep learning can be performed. ... -
4
AI-Crawler
Crawl a website starting from a URL, find relevant pages
AI Crawler is an experimental AI-powered web crawling and data extraction tool that uses natural language prompts to guide the discovery and retrieval of relevant information across websites. Unlike traditional web scrapers that rely on static selectors and manual scripting, it uses AI to dynamically identify and prioritize pages based on user intent, making it more flexible and resilient to changes in website structure. -
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
5
Skyvern
Automate browser-based workflows with LLMs and Computer Vision
...Skyvern understands how to solve CAPTCHAs to complete complicated workflows. Support for authenticating into user accounts, including support for 2FA/TOTP. Extract data from workflows in any schema of your choice including CSV or JSON. Automate procurement pipelines, breeze through government forms, and complete workflows in any language. -
6
Dendrite
Tools to build web AI agents that can authenticate
Dendrite Python SDK is a toolkit for building web AI agents that can authenticate, interact with, and extract data from any website, facilitating web automation tasks. -
7
Suna
Suna - Open Source Generalist AI Agent
...Designed to assist users in accomplishing real-world tasks through natural conversation, Suna combines powerful capabilities with an intuitive interface. It serves as a digital companion for research, data analysis, and everyday challenges, integrating tools like browser automation, file management, web crawling, command-line execution, website deployment, and API integration. Suna's architecture comprises a FastAPI-based backend, a Next.js/React frontend, an agent Docker environment, and a Supabase database for state management. This modular design allows for seamless interaction and task execution through simple conversations. -
8
Index
The SOTA Open-Source Browser Agent
...The project is built to integrate easily with applications through a simple programming interface, allowing developers to embed browser automation capabilities directly into their software systems. Index can perform tasks such as navigating pages, filling forms, collecting data, and analyzing web content without requiring manual scripting for each website. -
9
pwa-asset-generator
Automates PWA asset generation and image declaration
Automates PWA asset generation and image declaration. Automatically generates icon and splash screen images, favicons and mstile images. Updates manifest.json and index.html files with the generated images according to Web App Manifest specs and Apple Human Interface guidelines. When you build a PWA with a goal of providing native-like experiences on multiple platforms and stores, you need to meet with the criteria of those platforms and stores with your PWA assets; icon sizes and splash... -
Earn up to 16% annual interest with Nexo.Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
10
AutoKeras
AutoML library for deep learning
AutoKeras: An AutoML system based on Keras. It is developed by DATA Lab at Texas A&M University. The goal of AutoKeras is to make machine learning accessible to everyone. AutoKeras only support Python 3. If you followed previous steps to use virtualenv to install tensorflow, you can just activate the virtualenv. Currently, AutoKeras is only compatible with Python >= 3.7 and TensorFlow >= 2.8.0. AutoKeras supports several tasks with extremely simple interface. AutoKeras would search for the... -
11
GPT Crawler
Crawl a site to generate knowledge files to create your own custom GPT
GPT Crawler is an open-source tool designed to automatically crawl websites and generate structured knowledge that can be used to build AI assistants and retrieval systems. It focuses on extracting high-quality textual content from web pages and preparing it in formats suitable for embedding, indexing, or fine-tuning workflows. The project is especially useful for teams that want to turn documentation sites or knowledge bases into conversational AI backends without building custom scrapers... -
12
marqo
Tensor search for humans
A tensor-based search and analytics engine that seamlessly integrates with your applications, websites, and workflows. Marqo is a versatile and robust search and analytics engine that can be integrated into any website or application. Due to horizontal scalability, Marqo provides lightning-fast query times, even with millions of documents. Marqo helps you configure deep-learning models like CLIP to pull semantic meaning from images. It can seamlessly handle image-to-image, image-to-text and text-to-image search and analytics. Marqo adapts and stores your data in a fully schemaless manner. ... -
13
AI Deadlines
AI conference deadline countdowns
...Researchers and students use the platform to plan their paper submissions and manage academic schedules without manually tracking multiple conference announcements. The repository includes configuration files and data sources that allow contributors to add or update conferences through pull requests, enabling community-driven maintenance. -
14
Python Code Tutorials
The Python Code Tutorials
Python Code Tutorials is a large educational repository that aggregates programming tutorials from the “The Python Code” website into a structured collection of Python projects and learning materials. The repository covers a wide range of programming topics including cybersecurity, networking, web scraping, machine learning, GUI development, and automation scripts. Each tutorial typically includes complete Python code examples and explanations that demonstrate how to build real tools and... -
15
Reader LLM
Convert any URL to an LLM-friendly input with a simple prefix
Reader LLM is an open-source tool designed to convert web content into formats that are easier for large language models to process. The system works by transforming a webpage into a clean text or Markdown representation that removes unnecessary formatting and highlights the core information within the page. Developers can use a simple URL prefix to retrieve a version of a webpage that has been optimized for machine consumption, making it suitable for use in AI agents or retrieval-augmented... -
16
Eventer
Rapid, unbiased, reproducible analysis of synaptic events
...The software combines deconvolution for detection, and variable length template matching approaches for screening out false positive events. Eventer also includes a machine learning-based approach allowing users to train a model to implement their ‘expert’ selection criteria across data sets without bias. Sharing models allows users to implement consistent analysis procedures. The software is coded in MATLAB, but has been compiled as standalone applications for Windows, Mac and Linux. Please visit the official Eventer website for more info https://eventerneuro.netlify.app/ While the paper is in preparation, please cite as; Winchester, G., Liu, S., Steele, O.G., Aziz, W. and Penn, A.C. (2020) Eventer. ... -
17
uweb browser: unlimited power
minimal suckless android web browser with unlimited power
...and .js files as commands). - user-defined site-specific JS/CSS/HTML/preprocessing. - Online play/preview/preprocess for downloadable resources. - Multiple type profiles: switch any data including logins/config orthogonally - web automation, crontab (alarm clock) -
18
Ubix Linux
The Pocket Datalab
...You can find additional information, technical guidance, and user credentials on the project website https://ubix-linux.sourceforge.io/ or on the project subreddit https://reddit.com/r/UbixLinux. -
19
OpenAssistant
Chat-based assistant that understands tasks
...There will be versions which will be runnable on consumer hardware. You do not need to run the project locally unless you are contributing to the development process. The website link above will take you to the public website where you can use the data collection app. -
20
Coframe
Coframe brings your UX to life with AI-powered optimization
...Resulting performance data is fed back in to continuously improve your platform's content. With Coframe, your website or app works for you 24/7, not the other way around. All it takes to get up and running is a few lines of code. Coframe gives you full control and visibility. Our mission is to give every digital interface its own sense of intelligence. -
21
NSFW Data Scraper
Collection of scripts to aggregate image data
NSFW Data Scraper is an open-source project that provides scripts for automatically collecting large datasets of images intended for training NSFW image classification systems. The repository focuses on aggregating image data from various online sources so that developers can build datasets suitable for training content moderation models. These datasets typically contain images categorized into different classes associated with adult or explicit content, which can then be used to train neural networks that detect unsafe or inappropriate material. ... -
22
Olivia
Your new best friend powered by an artificial neural network
...Olivia can listen to you by saying “Hey Olivia” or clicking on the central button. She speaks to reply to you unless you've disabled her voice. Olivia respects your privacy. All the data used by Olivia is saved in your client. Olivia is organized in modules to facilitate the addition of new capabilities. These modules can be written in Go to execute multiple tasks. The project is entirely open-source from the website to the backend. Thus you can build your own chatbot and contribute to Olivia. You can directly add the PWA to your home screen from the website, and you won't remember that it is a website running behind this native-like app. -
23
chatbot chung is a keywords based probabilities algorythm simple entertainment chatbot with 3D talking openGL avatars written in freebasic. Can import aiml simple question/answer or question/random/answers or single star/ multi srai data saved from "AIML_chung" open source application . Online html5 javascript version with 44 languages multilingual auto detection available on the website (source included in the zip file). SORT gentext text generation algorythm option added (desktop version) .
-
24
Deep Learning with PyTorch
Latest techniques in deep learning and representation learning
This course concerns the latest techniques in deep learning and representation learning, focusing on supervised and unsupervised deep learning, embedding methods, metric learning, convolutional and recurrent nets, with applications to computer vision, natural language understanding, and speech recognition. The prerequisites include DS-GA 1001 Intro to Data Science or a graduate-level machine learning course. To be able to follow the exercises, you are going to need a laptop with Miniconda (a... -
25
The Integrating Vision Toolkit (IVT) is a powerful and fast C++ computer vision library with an easy-to-use object-oriented architecture. It offers its own multi-platform GUI toolkit. OpenCV is integrated optionally. Website: http://ivt.sourceforge.net
