Search Results for "python web crawler"
Sort By:
Showing 39 open source projects for "python web crawler"
View related business solutions-
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
1
Python Code Tutorials
The Python Code Tutorials
Python Code Tutorials is a large educational repository that aggregates programming tutorials from the “The Python Code” website into a structured collection of Python projects and learning materials. The repository covers a wide range of programming topics including cybersecurity, networking, web scraping, machine learning, GUI development, and automation scripts. -
2
Amazing-Python-Scripts
Curated collection of Amazing Python scripts
Amazing-Python-Scripts is a collaborative repository that collects a wide variety of Python scripts designed to demonstrate practical programming techniques and automation tasks. The project includes scripts ranging from beginner-level utilities to more advanced applications involving machine learning, data processing, and system automation. Its goal is to provide developers with useful coding examples that can solve everyday problems, automate repetitive tasks, or serve as learning exercises. ... -
3
Open Notebook
An Open Source implementation of Notebook LM with more flexibility
Open Notebook is an open-source, privacy-focused alternative to Google’s Notebook LM that gives users full control over their research and AI workflows. Designed to be self-hosted, it ensures complete data sovereignty by keeping your content local or within your own infrastructure. The platform supports 16+ AI providers—including OpenAI, Anthropic, Ollama, Google, and LM Studio—allowing flexible model choice and cost optimization. Open Notebook enables users to organize and analyze... -
4
Gradio
Create UIs for your machine learning model in Python in 3 minutes
Gradio is the fastest way to demo your machine learning model with a friendly web interface so that anyone can use it, anywhere! Gradio can be installed with pip. Creating a Gradio interface only requires adding a couple lines of code to your project. You can choose from a variety of interface types to interface your function. Gradio can be embedded in Python notebooks or presented as a webpage. A Gradio interface can automatically generate a public link you can share with colleagues that lets them interact with the model on your computer remotely from their own devices. ... -
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
5
LibrePhotos
A self-hosted open source photo management service
LibrePhotos is an open-source self-hosted photo management platform designed to organize, browse, and analyze personal media libraries while preserving user privacy. The system allows individuals to store and manage their photos and videos locally rather than relying on commercial cloud services. It provides features similar to services like Google Photos but runs on a private server controlled by the user. The application includes AI-powered tools that automatically analyze images to detect... -
6
Metaflow
A framework for real-life data science
Metaflow is a human-friendly Python library that helps scientists and engineers build and manage real-life data science projects. Metaflow was originally developed at Netflix to boost productivity of data scientists who work on a wide variety of projects from classical statistics to state-of-the-art deep learning. -
7
spaCy models
Models for the spaCy Natural Language Processing (NLP) library
spaCy is designed to help you do real work, to build real products, or gather real insights. The library respects your time, and tries to avoid wasting it. It's easy to install, and its API is simple and productive. spaCy excels at large-scale information extraction tasks. It's written from the ground up in carefully memory-managed Cython. If your application needs to process entire web dumps, spaCy is the library you want to be using. Since its release in 2015, spaCy has become an industry... -
8
Semantra
Multi-tool for semantic search
Semantra is an open-source semantic search tool designed to help users explore large collections of documents by meaning rather than simple keyword matching. The software analyzes text and PDF documents stored locally and creates embeddings that allow queries to retrieve results based on conceptual similarity. It is primarily intended for individuals who need to extract insights from large document collections, including researchers, journalists, students, and historians. The system runs... -
9
SimpleTuner
A general fine-tuning kit geared toward image/video/audio diffusion
SimpleTuner is an open-source toolkit designed to simplify the fine-tuning of modern diffusion models for generating images, video, and audio. The project focuses on providing a clear and understandable training environment for researchers, developers, and artists who want to customize generative AI models without navigating complex machine learning pipelines. It supports fine-tuning workflows for models such as Stable Diffusion variants and other diffusion architectures, enabling users to... -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
10
PyTorch Geometric Temporal
Spatiotemporal Signal Processing with Neural Machine Learning Models
The library consists of various dynamic and temporal geometric deep learning, embedding, and Spatio-temporal regression methods from a variety of published research papers. Moreover, it comes with an easy-to-use dataset loader, train-test splitter and temporal snaphot iterator for dynamic and temporal graphs. The framework naturally provides GPU support. It also comes with a number of benchmark datasets from the epidemiological forecasting, sharing economy, energy production and web traffic... -
11
ClearML
Streamline your ML workflow
...It is designed as an end-to-end MLOps suite allowing you to focus on developing your ML code & automation, while ClearML ensures your work is reproducible and scalable. The ClearML Python Package for integrating ClearML into your existing scripts by adding just two lines of code, and optionally extending your experiments and other workflows with ClearML powerful and versatile set of classes and methods. The ClearML Server storing experiment, model, and workflow data, and supports the Web UI experiment manager, and ML-Ops automation for reproducibility and tuning. ... -
12
docext
An on-premises, OCR-free unstructured data extraction
docext is a document intelligence toolkit that uses vision-language models to extract structured information from documents such as PDFs, forms, and scanned images. The system is designed to operate entirely on-premises, allowing organizations to process sensitive documents without relying on external cloud services. Unlike traditional document processing pipelines that rely heavily on optical character recognition, docext leverages multimodal AI models capable of understanding both visual... -
13
Karpathy
An agentic Machine Learning Engineer
karpathy is an experimental agentic machine learning engineer framework designed to automate many aspects of the ML development workflow. The project sets up a sandboxed environment where an AI agent can access datasets, run experiments, and generate machine learning artifacts through a web interface. Its startup script automatically prepares the environment by creating a sandbox directory, installing key ML libraries, and launching the agent interface. The system is tightly integrated with... -
14
Synapse Machine Learning
Simple and distributed Machine Learning
SynapseML (previously MMLSpark) is an open source library to simplify the creation of scalable machine learning pipelines. SynapseML builds on Apache Spark and SparkML to enable new kinds of machine learning, analytics, and model deployment workflows. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with the Open Neural Network Exchange (ONNX), LightGBM, The Cognitive Services, Vowpal Wabbit,... -
15
Interactive Machine Learning Experiments
Interactive Machine Learning experiments
...Many experiments involve tasks such as image classification, object detection, gesture recognition, and simple generative models. The models are typically trained in Python using TensorFlow and then exported for interactive demonstrations in a web environment using JavaScript and TensorFlow.js. Because the project focuses on experimentation rather than production systems, it acts as a sandbox where developers can explore machine learning concepts and observe model behavior. The notebooks reveal how each model is trained and provide opportunities to modify parameters or datasets to observe different outcomes. -
16
GNNPCSAFT Web App
Smart Thermodynamic Modeling with Graph Neural Networks
The GNNPCSAFT Web App is an implementation of our project that focuses on using Graph Neural Networks (GNN) to estimate the pure-component parameters of the Equation of State PC-SAFT. We developed this app so the scientific community can access the model's results easily. In this app, the estimated pure-component parameters can be used to calculate thermodynamic properties and compare them with experimental data from the ThermoML Archive. - CPU-compatible desktop build The standard... -
17
seq2seq-couplet
Play couplet with seq2seq model
seq2seq-couplet is a deep learning application that generates Chinese couplet responses using a sequence-to-sequence model built with TensorFlow. Its purpose is not general machine translation, but a specialized text generation task in which the model produces a matching second line for a given first line in the style of traditional couplets. The repository includes the code needed to train the model, configure file paths and hyperparameters, and evaluate progress through loss and BLEU score... -
18
dashAI
dashAI: an interactive platform for training, evaluating and deploying
dashAI is an open-source, No-code workbench for Exploratory Data Analysis and classical ML. Visual data preparation, multi-model experiments, XAI explainability, and a plugin-based extensible catalog. The platform guides users through a complete, traceable workflow — data ingestion → visual exploration → preprocessing → model training → evaluation → explainability — without writing a single line of code. Each step is explicit and reversible, keeping the user in control rather than... -
19
Loss
The Brutalist Market Analyzer
LOSS is a high-speed, "split-stack" financial dashboard designed to strip away market noise. By combining mathematically rigorous technical indicators with VADER-powered AI sentiment analysis, LOSS delivers objective, real-time Buy, Sell, or Hold verdicts in under two seconds. Wrapped in a distraction-free, "Brutalist" user interface, this tool is built for traders who rely on data, not hype. Modern financial websites are bloated with ads, conflicting opinions, and unnecessary jargon.... -
20
fastdup
An unsupervised and free tool for image and video dataset analysis
fastdup is a powerful free tool designed to rapidly extract valuable insights from your image & video datasets. Assisting you to increase your dataset images & labels quality and reduce your data operations costs at an unparalleled scale. -
21
Text-to-image Playground
A playground to generate images from any text prompt using SD
dalle-playground is an open-source web application that allows users to generate images from natural language text prompts using modern text-to-image generative models. Originally built around DALL-E Mini, the project later transitioned to using Stable Diffusion, enabling more detailed and higher-quality image synthesis. The system combines a backend machine learning service with a browser-based frontend interface that lets users experiment interactively with prompt engineering and... -
22
Taylorplot_Neptune
Creation of a Taylorplot for several machine learning models
Here we present the lines of code for creating a taylor plot with python to display several machine learning models. We show the solution for displaying 10 models, but the list and number can be changed simply by modifying the sample list. -
23
OmicSelector
Feature selection and deep learning modeling for omic biomarker study
OmicSelector is an environment, Docker-based web application, and R package for biomarker signature selection (feature selection) from high-throughput experiments and others. It was initially developed for miRNA-seq (small RNA, smRNA-seq; hence the name was miRNAselector), RNA-seq and qPCR, but can be applied for every problem where numeric features should be selected to counteract overfitting of the models. Using our tool, you can choose features, like miRNAs, with the most significant... -
24
KoboldAI
Your gateway to GPT writing
This is a browser-based front-end for AI-assisted writing with multiple local & remote AI models. It offers the standard array of tools, including Memory, Author's Note, World Info, Save & Load, adjustable AI settings, formatting options, and the ability to import existing AI Dungeon adventures. You can also turn on Adventure mode and play the game like AI Dungeon Unleashed. Stories can be played like a Novel, a text adventure game or used as a chatbot with an easy toggles to change... -
25
Pattern
Web mining module for Python, with tools for scraping
Pattern is an open-source Python library that provides tools for web mining, natural language processing, machine learning, and network analysis. The project integrates multiple capabilities into a single framework that allows developers to collect, process, and analyze textual data from the web. It includes modules for web scraping and crawling that can retrieve information from sources such as social media platforms, search engines, and online knowledge bases. ...
