Search Results for "python web crawler" - Page 33
Sort By:
Showing 3076 open source projects for "python web crawler"
View related business solutions-
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
1
AnimeGAN
A simple PyTorch Implementation of Generative Adversarial Networks
...The images are not clean, some outliers can be observed, which degrades the quality of the generated images. Anime-style images of 126 tags are collected from danbooru.donmai.us using the crawler tool gallery-dl. The images are then processed by an anime face detector python-anime face. The resulting dataset contains ~143,000 anime faces. Note that some of the tags may no longer be meaningful after cropping, i.e. the cropped face images under the 'uniform' tag may not contain visible parts of uniforms. -
2
opsi
open source client management system, software deployment
opsi is a Client Management System for Windows and Linux clients based on Linux servers. opsi is used for automatic OS installation, software depolyment, hard- and software inventory, distributed software depots. -
3
Pity
API test platform
Pity is a powerful and user-friendly open-source API test platform that supports automated test case creation, test scheduling, and result visualization. Designed for developers and QA teams, it provides a web-based interface for writing, organizing, and executing HTTP tests, making it easy to maintain robust API quality in CI/CD pipelines. -
4
Pattern
Web mining module for Python, with tools for scraping
Pattern is an open-source Python library that provides tools for web mining, natural language processing, machine learning, and network analysis. The project integrates multiple capabilities into a single framework that allows developers to collect, process, and analyze textual data from the web. It includes modules for web scraping and crawling that can retrieve information from sources such as social media platforms, search engines, and online knowledge bases. ... -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
5
-
6
pspider
Simple Python framework for building multithreaded web crawlers
PSpider is a lightweight web crawling framework written in Python designed to simplify the development of custom web spiders. It focuses on providing an easy-to-understand architecture while still supporting concurrent crawling for improved performance. It uses a multithreaded model that separates the crawling workflow into several components responsible for fetching, parsing, and saving data. -
7
rocket-bi
An open-source web-based self-service BI for analytical databases
Rocket.BI is a free, open-source, web-based business intelligence solution specifically designed for analytical databases. It enables data analysts and business users alike to easily integrate different data sources, perform advanced data analysis, ad hoc, and more. With an easy-to-use editor, you can create personalized reports, build interactive business dashboards and generate actionable business insights. Rocket.BI also allows collaboration as working together with other people in the... -
8
AutoScraper
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
This project is made for automatic web scraping to make scraping easy. It gets a URL or the HTML content of a web page and a list of sample data that we want to scrape from that page. This data can be text, URL or any HTML tag value of that page. It learns the scraping rules and returns similar elements. Then you can use this learned object with new URLs to get similar content or the exact same element of those new pages. -
9
ostRAT
OpenSourceTelegramRAT - Remote PC access via Telegram Bot.
ostRAT is free and open source. GPLv3 Сomputer remote control software. Works via telegram bot. A lot of functions, for example: - Screenshot: sends a screenshot - Off: turns off the computer - Url: opens entered link - Write: sends your text to the computer - Move: changes mouse location with x and y - and more! WARNING: Using the bot is recommended only on your device. Failure to comply with the recommendation may result in criminal liability. -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
ipfs-css
Single-purpose css class names and font-face config to IPFS up your UI
The single-purpose CSS class names and @font-face config to IPFS up your UI. Once you've installed ipfs-css from npm, the CSS and SCSS files and the web-fonts are available from your node_modules/ipfs-css directory. You can import the theme.json file which can be used with a ThemeProvider component. All the CSS atoms are generated from that, so you can be sure you're using the same values. While ipfs.css contains everything you need, if you prefer variables for fonts, colors, and gradients,... -
11
Bloxs
Build dashboards in Jupyter Notebook with numeric and chart boxes
Bloxs is a simple Python package that helps you display information in an attractive way (formed in blocks). Perfect for building dashboards, reports and apps in the notebook. -
12
SMSBoom
SMS bombing / SMS pressure measurement, a robust and free python
SMS bombing / SMS pressure measurement / | A robust and free python SMS bombing program, specializing in blowing up bad guys, with millions of interfaces, multi-threaded automatic addition of effective interfaces, support for millions of concurrent asynchronous coroutines, a completely free SMS bombing too! Hongkonger develops the whole network for the first time! api.jsonDefine the interface in a custom way. -
13
mlscraper
ML-based HTML scraper that learns extraction rules from examples
mlscraper is a Python library designed to automatically extract structured data from HTML pages without requiring developers to manually write CSS selectors or XPath rules. Instead of defining extraction logic by hand, users provide a few examples of the data they want to retrieve from a webpage. It analyzes those examples within the HTML document and determines patterns or rules that can be used to extract the same type of information from similar pages. -
14
Recursive Sans & Mono
Recursive Mono & Sans is a variable font family for code & UI
...In programming, “recursion” is when a function calls itself, using its own output as an input to yield powerful results. Recursive Mono was used as a tool to help build itself: it was used to write Python scripts to automate type production work and to generate specimen images, and it was used in HTML, CSS, and JS to create web-based proofs & prototypes. Through this active usage, Recursive Mono was crafted to be both fun to look at as well as deeply useful for all-day work. Recursive Sans borrows glyphs from its parent mono but adjusts the widths of many key glyphs for comfortable readability. ... -
15
AnotherWar
FPShooter Game
FPShooter Game open soruce game in Python Thanks to https://github.com/fogleman/Minecraft (MIT-License) This game can be modified into a multiplayer Minecraft with guns FPS game ! -
16
Envelop (Risk and Audits software)
Governance, Risk, and Compliance Management Tool
Envelop is a management tool that focuses on governance, risk and compliance processes and documentation. Auditing and managing audit workflows is the main feature of this tool. Envelop is an audit software, that helps with internal and external audits to be managed. The common usage of this tool would be for audits, e.g. financial audits done by internal or external auditors. The overall structure follows this hierarchy : Process > Objective > Risk > Control > Test > Finding You... -
17
SharPyShell
Tiny and obfuscated ASP.NET webshell for C# web applications
SharPyShell is a tiny and obfuscated ASP.NET web shell that executes commands received by an encrypted channel compiling them in memory at runtime. SharPyShell supports only C# web applications that run on .NET Framework >= 2.0. SharPyShell is a post-exploitation framework written in Python. The main aim of this framework is to provide the penetration tester with a series of tools to ease the post-exploitation phase once exploitation has been successful against an IIS webserver. ... -
18
Whisper Library
Whisper is a file-based time-series database format for Graphite
Whisper is one of three components within the Graphite project. Whisper is a fixed-size database, similar in design and purpose to RRD (round-robin-database). It provides fast, reliable storage of numeric data over time. Whisper allows for higher resolution (seconds per point) of recent data to degrade into lower resolutions for long-term retention of historical data. Copies data from src in dst, if missing. Unlike whisper-merge, don't overwrite data that's already present in the target... -
19
AI Wallpapers
Change your wallpaper daily using images generated with DALL-E 2
-
20
SpiderFoot
SpiderFoot automates OSINT for threat intelligence and mapping
SpiderFoot is an open-source OSINT automation platform for reconnaissance, threat intelligence, and attack-surface discovery. It collects and correlates data from more than 200 modules and numerous public or API-based sources. Users can investigate domains, IP addresses, subnets, ASNs, email addresses, usernames, phone numbers, people, and cryptocurrency addresses. It runs through a web interface or command line and stores results in SQLite for custom querying. Scans can uncover... -
21
V2RayCloudSpider
V2RayCloudSpider
V2RSS is an "ecological mining machine" that can perform vertical mining on global providers based on the SSPanel-Uim framework; it can generate bottom-up "aggregation collection" tasks for mainstream protocol headers; it can self-digest and Compared with proxypool , the output is purer and more reliable proxy nodes; it has powerful production features such as self-discovery and service self-healing. -
22
Google Cloud Vision API examples
Sample code for Google Cloud Vision
The cloud-vision repository is a sample code collection for the Google Cloud Vision API that shows developers how to implement image analysis tasks across a wide range of languages and platforms. It contains examples organized by language and environment, including Go, Java, Node.js, PHP, Python, Ruby, .NET, Android, iOS, and even a Chrome extension, which makes it especially valuable as a cross-platform learning resource. The repository demonstrates concrete image understanding use cases,... -
23
Scylla
Intelligent proxy pool for collecting and managing public proxies
Scylla is an open source proxy pool system designed to collect, validate, and manage large numbers of public proxy servers for use in web scraping and data extraction workflows. It automatically crawls the internet to discover proxy IP addresses and evaluates their availability and reliability before adding them to a usable pool. It includes a JSON API that allows developers and applications to retrieve proxy information programmatically, making it easier to integrate proxy rotation into... -
24
Webware for Python
The Classic Webware for Python
Webware for Python is a suite of components for dynamic, server-side web development. -
25
CIPS-3D
3D-aware GANs based on NeRF (arXiv)
3D-aware GANs based on NeRF (arXiv). This repository contains the code of the paper, CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis. The problem of mirror symmetry refers to the sudden change of the direction of the bangs near the yaw angle of pi/2. We propose to use an auxiliary discriminator to solve this problem. Note that in the initial stage of training, the auxiliary discriminator must dominate the generator more than the main discriminator...
