Search Results for "python web crawler" - Page 42
Sort By:
Showing 3076 open source projects for "python web crawler"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
1
Leo is an outlining-oriented project manager and browser. With Leo you can organize programs, web sites, URL's, pictures or any other kind of data. Leo is fully scriptable in Python. Leo is 100% pure Python. It requires the Qt gui toolkit.
-
2
Twitter Intelligence
Twitter Intelligence OSINT project performs tracking and analysis
A project written in Python for Twitter tracking and analysis without using Twitter API. This project is a Python 3.x application. The package dependencies are in the file requirements.txt. Run that command to install the dependencies. SQLite is used as the database. Tweet data is stored on the Tweet, User, Location, Hashtag, HashtagTweet tables. The database is created automatically. analysis.py performs analysis processing. User, hashtag, and location analyzes are performed. You must write... -
3
pyspider
A powerful Spider(Web Crawler) system in Python
pyspider is a powerful Spider(Web Crawler) system in Python. Components are connected by message queue. Every component, including message queue, is running in their own process/thread, and replaceable. That means, when process is slow, you can have many instances of processor and make full use of multiple CPUs, or deploy to multiple machines. This architecture makes pyspider really fast. benchmarking. -
4
Competitive stock picking internet based game. Along with the game, information will be collected and strategies will be tested. Users will develop a broader understanding of capital markets and the asset allocation decisions that drive our society.
-
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
5
A project for a highly scalable protocol and framework and a collection of software tools for m2m (machine to machine communication) and IoT Internet of Things. The programs here works on Windows and Linux and are based around VSCP, The Very Simple Control Protocol. Repository is here: https://github.com/grodansparadis/vscp_software and here https://github.com/grodansparadis/vscp_firmware
-
6
blog99
A blog engine that does html and gopher
This is the blog engine for HTML and Gopher. Blog entries are written as html files. For HTML, it is an Apache/MySQL/Python application using WSGI. For Gopher, it is Gophernicus/MySQL/Python using CGI. -
7
haipproxy
Distributed proxy IP pool for web crawlers using Scrapy and Redis
HAipproxy is a distributed proxy IP pool system designed to collect, manage, and provide large numbers of proxy addresses for web crawling tasks. It automatically crawls proxy resources from the internet and aggregates them into a centralized pool that can be accessed by distributed spiders and scraping systems. It is built using Python and relies on Scrapy for high-performance crawling while Redis is used for data storage, communication, and task coordination between components. ... -
8
"Circular Multilateral Barter" (CMB) is aimed at supplying entrepreneurs with a way to exchange goods and services within a p2p-network, without using money, overcoming the "double coincidence of wants" problem, inherent to traditional barter. Most importantly, CMB allows traders to issue their own currencies (called "products"), which others can redeem, trade, or make payments with. One thing that distinguishes CMB from other "credit commons" is that all debts in CMB are user-to-user...
-
9
PiHass
Pre-defined and easy to use Home-Assistant Image for raspberry pi
This is a Raspbain Strech base image with Home-Assistant on it. i used Virtual Env based installation and added some Custom Ui and Custom Components. i have also configured MySQL server and database and also some scripts, sensors and groups to help users start working with the system. -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
10
Face Recognition
World's simplest facial recognition api for Python & the command line
Face Recognition is the world's simplest face recognition library. It allows you to recognize and manipulate faces from Python or from the command line using dlib's (a C++ toolkit containing machine learning algorithms and tools) state-of-the-art face recognition built with deep learning. Face Recognition is highly accurate and is able to do a number of things. It can find faces in pictures, manipulate facial features in pictures, identify faces in pictures, and do face recognition on a... -
11
OTPspot
Wireless hotspot with OTP captive portal
Run a captive portal on your raspberry (or any linux box) to allow your guests to register before accessing your Wifi at home. Users will be requested for an OTP code that you can generate on your phone through the Google Authenticator/Authy app. Get rid of captive portal's static username and password, without the need for a complex radius server. OTPspot (since version 2.0) is fully compatible with nodogsplash and can run as a FAS service. In this configuration, nodogsplash will... -
12
Vaex
Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python
Data science solutions, insights, dashboards, machine learning, deployment. We start at 100GB. Vaex is a high-performance Python library for lazy Out-of-Core data frames (similar to Pandas), to visualize and explore big tabular datasets. It calculates statistics such as mean, sum, count, standard deviation etc, on an N-dimensional grid for more than a billion (10^9) samples/rows per second. Visualization is done using histograms, density plots and 3d volume rendering, allowing interactive... -
13
Holarse
website software for holarse
HolaCMS 3 Source Code which will power the new Holarse website. -
14
JavaScript Lint is a development tool that validates JavaScript code for best coding practices, finding common mistakes without actually running the script or opening the web page.
-
15
Assorted projects. General-purpose libraries for Python, C++, Scala, bash, and others. Meta-programming tools. System utilities. UI components. Web APIs. Configuration files. Benchmarks. Programming competition entries. And much more.
-
16
survol
RDF-based framework monitoring business systems activity
A Python agent and a web interface aiming to help the analysis and investigation of a legacy application. A set of machines, processes, databases, programs etc ... all communicating with each other, manipulating your data, and whose software architecture has become, with time, complicated, difficult to understand, and undocumented. Data are aggregated with an RDF inference engine, creating a global vision of the business information processing. -
17
pyTivo is both an HMO and GoBack server. Similar to TiVo Desktop pyTivo loads many standard video compression codecs and outputs mpeg2 video to the TiVo. However, pyTivo is able to load MANY more file types than TiVo Desktop.
-
18
owfs and owhttpd
Project moved to https://github.com/owfs/owfs/
Please note that OWFS source code, and all Issue/Tickets/merge requests have now been moved to https://github.com/owfs/owfs/. Developer mailing lists will still be kept at Sourceforge. ---- OWFS -- 1-Wire file system. Use the Dallas 1-Wire and iButton chips with standard filesystem commands. Create temperature loggers. Monitor everything. OWHTTPD -- same system, only used as a light weight web server. OWFS is also ported to embedded routers, Mac OSX and Windows. Basically you can use... -
19
Zeus Scanner
Advanced reconnaissance utility
...Zeus-Scanner includes checks for issues such as XSS, SQL injection, clickjacking, exposed admin panels, port scanning, whois lookup, and header protection. It is most useful for penetration testers and researchers who need a broad reconnaissance toolkit, though its older Python 2-style setup makes environment compatibility important. -
20
coldev
C/C++ , PHP, PYTHON, 3D ENGINES, CHROME NATIVE CLIENT
+ C/C++ and PHP projects, Python and Others + 3D ENGINES + Videogames Projects based in Bennugd http://www.bennugd.org Web Page https://coldev.sourceforge.io/ -
21
Config Backup for F5
Backup program for F5 Big IP devices
This project is a Python program with PHP web UI to manage daily backups of F5 BigIP devices. It also is available as a turnkey VMware appliance requiring no Linux skills to use. Disclaimer - This project is an independent work under the GPL v2 license and is NOT provided by or supported by F5 Networks. It is provided as is with no warranty. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or any later version. -
22
freenas-tempreport
FreeNAS Temperature Report
FreeNAS Temperature Report is a small server program that provides temperature readings for your FreeNAS unit. Install this program in a jail on your FreeNAS, then use your web browser and go to http://jail_ip to see your temperature readings. Installation instructions: http://sourceforge.net/p/freenas-tempreport/wiki/Install/ -
23
jd-autobuy
Python tool that automates JD.com login and product purchase tasks
jd-autobuy is an open source Python-based automation tool designed to simulate the purchasing process on the JD e-commerce platform. It uses web scraping and HTTP request techniques to log into an account, check product availability, and attempt to purchase specified items automatically. It supports login through methods such as QR code authentication, allowing users to sign in through the platform’s mobile application. -
24
icemac.addressbook
Multi user address book application accessable through the web.
Multi user address book application accessable through the web. Store, edit, search and export addresses, phone numbers, … using a web browser. Code moved to https://bitbucket.org/icemac/icemac.addressbook Documentation see https://icemacaddressbook.readthedocs.io/en/latest/ New releases (after 6.0.2) see https://pypi.org/project/icemac.addressbook/#history -
25
diskover
File system crawler and disk space usage software
diskover is a file system crawler and disk space usage software that uses Elasticsearch to index your file metadata. diskover crawls and indexes your files on a local computer or remote storage server over network mounts. diskover helps manage your storage by identifying old and unused files and give better insights into data change "hotfiles", file duplication "dupes" and wasted space. It is designed to help deal with managing large amounts of data growth and provide detailed storage...
