Search Results for "data collection algorithm" - Page 7
Sort By:
Showing 334 open source projects for "data collection algorithm"
View related business solutions-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
1
FFCV
Fast Forward Computer Vision (and other ML workloads!)
ffcv is a drop-in data loading system that dramatically increases data throughput in model training. From gridding to benchmarking to fast research iteration, there are many reasons to want faster model training. Below we present premade codebases for training on ImageNet and CIFAR, including both (a) extensible codebases and (b) numerous premade training configurations. -
2
data-science-on-gcp
Source code accompanying book: Data Science on the GCP
The data-science-on-gcp repository is a comprehensive collection of code examples and end-to-end workflows that accompany the book Data Science on the Google Cloud Platform, designed to teach developers how to build scalable data science and machine learning systems using Google Cloud services. It provides structured, chapter-aligned implementations that guide users through the full lifecycle of a data science project, including data ingestion, storage, processing, analysis, model training, and deployment. ... -
3
Data science blogs
A curated list of data science blogs
Data Science Blogs is a curated repository that aggregates a wide range of high-quality blogs and resources related to data science, machine learning, and analytics into a single organized collection. It serves as a discovery platform for practitioners, researchers, and learners who want to stay updated with industry trends, techniques, and insights without manually searching for reliable sources. -
4
Pentest-Tools
A collection of custom security tools for quick needs.
Pentest-Tools is a collection of penetration testing scripts and utilities designed to help security professionals and ethical hackers perform vulnerability assessments. It includes a wide range of tools for tasks like web scraping, reconnaissance, data extraction, and network analysis. The suite is modular, allowing users to choose the tools that best fit their specific pentesting needs, from web application analysis to network penetration testing. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
FairScale
PyTorch extensions for high performance and large scale training
FairScale is a collection of PyTorch performance and scaling primitives that pioneered many of the ideas now used for large-model training. It introduced Fully Sharded Data Parallel (FSDP) style techniques that shard model parameters, gradients, and optimizer states across ranks to fit bigger models into the same memory budget. The library also provides pipeline parallelism, activation checkpointing, mixed precision, optimizer state sharding (OSS), and auto-wrapping policies that reduce boilerplate in complex distributed setups. ... -
6
GFPGAN
GFPGAN aims at developing Practical Algorithms
GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration. Colab Demo for GFPGAN; (Another Colab Demo for the original paper model) Online demo: Huggingface (return only the cropped face) Online demo: Replicate.ai (may need to sign in, return the whole image). Online demo: Baseten.co (backed by GPU, returns the whole image). We provide a clean version of GFPGAN, which can run without CUDA extensions. So that it can run in Windows or on CPU mode. GFPGAN aims at developing... -
7
AllenNLP
An open-source NLP research library, built on PyTorch
AllenNLP makes it easy to design and evaluate new deep learning models for nearly any NLP problem, along with the infrastructure to easily run them in the cloud or on your laptop. AllenNLP includes reference implementations of high quality models for both core NLP problems (e.g. semantic role labeling) and NLP applications (e.g. textual entailment). AllenNLP supports loading "plugins" dynamically. A plugin is just a Python package that provides custom registered classes or additional... -
8
Gym
Toolkit for developing and comparing reinforcement learning algorithms
...Open source interface to reinforce learning tasks. The gym library provides an easy-to-use suite of reinforcement learning tasks. Gym provides the environment, you provide the algorithm. You can write your agent using your existing numerical computation library, such as TensorFlow or Theano. It makes no assumptions about the structure of your agent, and is compatible with any numerical computation library, such as TensorFlow or Theano. The gym library is a collection of test problems — environments — that you can use to work out your reinforcement learning algorithms. ... -
9
Moriarty Project
Web-based OSINT tool for investigating phone number information
...Moriarty Project focuses strictly on OSINT-style information gathering and does not provide tracking, hacking, or harmful capabilities against phone number owners. It is fully open source, allowing users to inspect the code and understand how the data collection mechanisms work. -
Veeam Data Platform v13.1 - Get Your Free TrialTry Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
10
DecryptLogin
Python library providing APIs for automated website login workflows
...DecryptLogin supports a wide variety of online services and platforms, including social media sites, developer platforms, cloud services, and other web portals. Developers can integrate these login routines into automation scripts, crawlers, or data collection tools that require authenticated sessions. It also provides example utilities and automation scripts demonstrating how the login APIs can be used in practical scenarios. -
11
Auto-PyTorch
Automatic architecture search and hyperparameter optimization
While early AutoML frameworks focused on optimizing traditional ML pipelines and their hyperparameters, another trend in AutoML is to focus on neural architecture search. To bring the best of these two worlds together, we developed Auto-PyTorch, which jointly and robustly optimizes the network architecture and the training hyperparameters to enable fully automated deep learning (AutoDL). Auto-PyTorch is mainly developed to support tabular data (classification, regression) and time series... -
12
Jraph
A Graph Neural Network Library in Jax
Jraph (pronounced “giraffe”) is a lightweight JAX library developed by Google DeepMind for building and experimenting with graph neural networks (GNNs). It provides an efficient and flexible framework for representing, manipulating, and training models on graph-structured data. The core of Jraph is the GraphsTuple data structure, which enables users to define graphs with arbitrary node, edge, and global attributes, and to batch variable-sized graphs efficiently for JAX’s just-in-time compilation. The library includes a comprehensive set of utilities for batching, padding, masking, and partitioning graph data, making it ideal for distributed and large-scale GNN experiments. ... -
13
Blend_My_NFTs
Easily generate thousands of 3D models, images, and animation NFTs
Blend_My_NFTs is an open-source, free-to-use Blender add-on that enables you to easily generate thousands of 3D Models, Animations, and Images. This add-on's primary purpose is to aid in the creation of large generative 3D NFT collections. It is the first and easiest 3D NFT generator. Blend_My_NFTs was initially developed to create Cozy Place, an NFT collection by This Cozy Studio Inc. Blend_My_NFTs works with Blender 3.2.2 on Windows 10 or macOS Big Sur 11.6. Linux is supported, however we... -
14
Reskin Sensor Library
ReSkin Sensor Interfacing Library
...Magnetic sensing separates the electronic circuitry from the passive-interface, making it easier to replace interfaces as they wear out while allowing for a wide variety of form factors. Machine learning allows us to learn sensor response models that are robust to variations across fabrication and time, and our self-supervised learning algorithm enables finer performance enhancement. -
15
The Python Computer Graphics Kit is a collection of Python modules that contain the basic types and functions to be able to create 3D computer graphics images (focusing on Pixar's RenderMan interface).
-
16
OneForAll
OneForAll is a powerful subdomain collection tool
OneForAll is a comprehensive subdomain enumeration and reconnaissance tool primarily used in penetration testing and bug bounty workflows. Built in Python, it aggregates results from numerous DNS, certificate transparency, search engine, and threat intelligence sources to uncover hidden subdomains. The tool is particularly notable for incorporating many Chinese data sources that are often missed by Western-focused recon tools, increasing discovery coverage. It supports brute-force subdomain... -
17
Blank Grabber
The most powerful stealer written in Python 3
Blank Grabber is an archived Python information-stealing malware project presented as a configurable grabber with a graphical builder. The repository lists capabilities related to credential theft, session theft, browser data collection, system information capture, screenshots, webcam capture, persistence, evasion, and exfiltration. Although the README includes an educational disclaimer, the tool’s stated behavior is clearly harmful outside isolated and authorized security research. It should not be used as a normal utility, administration tool, or learning project for general users. ... -
18
mlscraper
ML-based HTML scraper that learns extraction rules from examples
mlscraper is a Python library designed to automatically extract structured data from HTML pages without requiring developers to manually write CSS selectors or XPath rules. Instead of defining extraction logic by hand, users provide a few examples of the data they want to retrieve from a webpage. It analyzes those examples within the HTML document and determines patterns or rules that can be used to extract the same type of information from similar pages. Once trained, the generated scraper... -
19
Brain Tokyo Workshop
Experiments and code from Google Brain’s Tokyo research workshop
...The repository includes implementations, experimental data, and supporting research papers that accompany published studies. Notable works such as Weight Agnostic Neural Networks and Neuroevolution of Self-Interpretable Agents highlight the team’s exploration of how AI can learn more efficiently and transparently. Overall, this repository serves as an open research hub for sharing ideas and advancing the understanding of intelligent systems. -
20
Scylla
Intelligent proxy pool for collecting and managing public proxies
Scylla is an open source proxy pool system designed to collect, validate, and manage large numbers of public proxy servers for use in web scraping and data extraction workflows. It automatically crawls the internet to discover proxy IP addresses and evaluates their availability and reliability before adding them to a usable pool. It includes a JSON API that allows developers and applications to retrieve proxy information programmatically, making it easier to integrate proxy rotation into... -
21
GiantMIDI-Piano
Classical piano MIDI dataset
GiantMIDI-Piano is a large-scale symbolic classical piano music dataset built by applying the piano_transcription system on a vast collection of piano performance recordings. The dataset contains thousands of piano works, spanning a large number of composers and styles, with each piece transcribed into high-precision MIDI files capturing note events, pedal usage, velocities, etc. It provides a resource for music information retrieval (MIR), symbolic music modeling, composer classification, music generation, analysis of classical piano repertoire, and data-driven research in musicology or AI-based composition. ... -
22
Zenoss Community Edition
Zenoss - Intelligent IT Operations Management
Zenoss provides software-defined IT operations for the world’s largest organizations. We deliver the ultimate level of IT service health with simplicity by providing the most granular and intelligent IT service modeling possible, at any scale, and sharing these unique insights with other IT operations management (ITOM) tools to make them more efficient. Zenoss Community Edition is not a “demo” or trial version of Zenoss Enterprise or Zenoss Cloud! Before You install Zenoss Community... -
23
AI Platform Training and Prediction
Cloud ML Engine repo
AI Platform Training and Prediction is a collection of machine learning example projects that demonstrate how to train, deploy, and serve models using Google Cloud AI Platform and related services. It includes a wide variety of implementations across frameworks such as TensorFlow, PyTorch, scikit-learn, and XGBoost, allowing developers to explore different approaches to building ML solutions. -
24
Machine Learning Financial Laboratory
MlFinLab helps portfolio managers and traders
MlFinLab is a comprehensive Python library designed to support the development of machine learning strategies in quantitative finance and algorithmic trading. The project provides a large collection of tools that implement techniques from academic research on financial machine learning. It covers the full lifecycle of developing data-driven trading strategies, including data preprocessing, feature engineering, labeling techniques, model training, and performance evaluation. Many of the algorithms implemented in the library are based on concepts introduced in advanced quantitative finance literature and peer-reviewed research. ... -
25
StreamAlert
StreamAlert is a serverless, realtime data analysis framework
StreamAlert is a serverless, real-time data analysis framework that empowers you to ingest, analyze, and alert on data from any environment, using data sources and alerting logic you define. Computer security teams use StreamAlert to scan terabytes of log data every day for incident detection and response. Incoming log data will be classified and processed by the rules engine. Alerts are then sent to one or more outputs. Rules are written in Python; they can utilize any Python libraries or...
