Search Results for "python web crawler" - Page 37
Sort By:
Showing 3076 open source projects for "python web crawler"
View related business solutions-
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
Veeam Data Platform v13.1Try Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
1
A network and services monitoring system. Does PING, DHCP, POP3, SMTP, FTP etc etc tests. Does NOT require additional software installed at the remote side. Fully configurable. Provides instant report (via HTML page), logs, statistics and sound alarm
-
2
Atgen A2 Automation
Atgen A2 is a radically simple Workload Automation & Scheduling tool.
Atgen A2 is a radically simple IT automation solution that automates application builds, continuous testing, data parsing & processing, report generation, batch processing, job scheduling, and many other IT needs. Avoid running repetitive batch jobs and managing access to deploy and update your applications — automate in a language that approaches plain English, using SSH or WinRM, with no agents to install on remote systems. https://www.atgensoft.com/ You can install a released version... -
3
Zero Install is a decentralised cross-distribution software installation system. Create one package that works everywhere! With dependency handling and automatic updates, full support for shared libraries, and integration with native package managers
-
4
ngxtop
Real-time metrics for nginx server
ngxtop is a command-line monitoring utility that provides real-time analytics for Nginx and similar web server access logs using a top-style interface. Instead of relying on heavy monitoring stacks, it parses live log streams and produces immediate insights into request rates, status codes, bandwidth usage, and endpoint activity. The tool is particularly useful for troubleshooting traffic spikes or diagnosing performance issues during active incidents. It automatically detects the access log... -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
5
Text2Video
Software tool that converts text to video for more engaging experience
...So I imagined, "What if there was a tool that turns text into something more engaging such as a video, wouldn't it improve my learning experience?" I created a prototype web application that takes text as an input and generates a video as an output. I plan to further work on the project targeting young college students who are aged between 18 to 23 because they tend to prefer learning through videos over books based on the survey I found. The technologies I used for the project are HTML, CSS, Javascript, Node.js, CCapture.js, ffmpegserver.js, Amazon Polly, Python, Flask, gevent, spaCy, and Pixabay API. -
6
MToolBox
A bioinformatics pipeline to analyze mtDNA from NGS data
MToolBox is a highly automated bioinformatics pipeline to reconstruct and analyze human mitochondrial DNA from high throughput sequencing data. MToolBox includes an updated computational strategy to assemble mitochondrial genomes from Whole Exome and/or Genome Sequencing (PMID: 22669646) and an improved fragment-classify tool (PMID:22139932) for haplogroup assignment, functional and prioritization analysis of mitochondrial variants. MToolBox provides pathogenicity scores, profiles of genome... -
7
Flask-MongoEngine
MongoEngine flask extension with WTF model forms support
Flask-MongoEngine is a Flask extension that provides integration with MongoEngine, WtfForms and FlaskDebugToolbar. By default, Flask-MongoEngine will install integration only between Flask and MongoEngine. Integration with WTFForms and FlaskDebugToolbar are optional and should be selected as extra option, if required. This is done by users request, to limit amount of external dependencies in different production setup environments. All methods end extras described below are compatible... -
8
Hack-Tools
Hack tools
hack-tools is a collection of various hacking tools and utilities. It serves as a comprehensive toolkit for penetration testers and cybersecurity enthusiasts, encompassing a wide range of functionalities. -
9
Web LP-CSIC/UAB
New Web for the LP-CSIC/UAB
New Web Home Page for the LP-CSIC/UAB -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
10
Lenia
Lenia - Mathematical Life Forms
Lenia is a 2D cellular automaton with continuous space, time, and states. It produces a huge variety of interesting life forms. There are various versions available. Python, Matlab, and web (JavaScript) versions are real-time, interactive, and equipped with statistics tools. Jupyter and R versions are non-interactive and just for demonstration purposes. -
11
bioweb
polyglot language framework to analyze genetic data
polyglot framework using Python/C++/JavaScript to fast develop applications to analyze biological sequences -
12
IPFS Publish
Continuous Delivery of static websites to IPFS
This is a tool that aims to enable the automatic publishing of static webpages from Git repositories into IPFS. It consists of two parts: a small web server and management CLI. Web server exposes an endpoint that you use as your Git's webhook. When the hook is invoked, it clones your repo, build it (if needed), add it to the IPFS node (pin it if configured) and publish the new IPFS address under configured IPNS name. -
13
Python Crawler Tutorial Starts From Zero
Python crawler tutorial, taking you from zero to one
Python Crawler Tutorial Starts From Zero is a Chinese-language learning repository that teaches web crawling from introductory concepts through practical examples. Early lessons explain HTTP requests, request analysis, the Python Requests library, and common categories of extracted data. Separate chapters cover JSON processing and regular expressions for transforming responses into structured information. -
14
googler
Google Search, Google Site Search, Google News from the terminal
googler is a power tool to Google (Web & News) and Google Site Search from the command-line. It shows the title, URL and abstract for each result, which can be directly opened in a browser from the terminal. Results are fetched in pages (with page navigation). Supports sequential searches in a single googler instance. googler was initially written to cater to headless servers without X. You can integrate it with a text-based browser. However, it has grown into a very handy and flexible... -
15
QPyDesk
Code editor and real-time QR code generator for QPython
QPyDesk is a code editor, and real-time QR code generator for QPython. It is a Python code editor with syntax highlighting that also generates the QR code that represents said code in real time. This application also allows you to print the generated QR code to distribute the created application. However, because QR codes have a limited storage capacity, if the code is very long, QPyDesk creates a QR code that is only valid while the application is running, that is, the QR code generated... -
16
phpsploit
Full-featured C2 framework which silently persists on webserver
Full-featured C2 framework which silently persists on webserver via polymorphic PHP oneliner. The obfuscated communication is accomplished using HTTP headers under standard client requests and web server's relative responses, tunneled through a tiny polymorphic backdoor. Detailed help for any option (help command) Cross-platform on both client and server. CLI supports auto-completion & multi-command. Session saving/loading feature & persistent history. Multi-request support for large... -
17
Jupytab
Display in Tableau data from Jupyter notebooks
Jupytab allows you to explore in Tableau data which is generated dynamically by a Jupyter Notebook. You can thus create Tableau data sources in a very flexible way using all the power of Python. This is achieved by having Tableau access data through a web server created by Jupytab. -
18
TIDoS
Python web penetration testing framework with modular security tools
TIDoS-Framework is an open source web application penetration testing framework designed to assist security researchers and ethical hackers in identifying vulnerabilities in web systems. It provides a comprehensive environment for performing multiple phases of security assessment, including reconnaissance, scanning, enumeration, vulnerability analysis, and exploitation. -
19
yabasta
Yet Another BAsic Scraper and Text Analysis
YA BASTA! is a Python/R application for Lyrics Web Scraper and Text Analysis. Web scraping is developed in Python, text analysis in R as Python subprocesses. YA BASTA! is only tested on windows OS. To run YA BASTA! just type on window command prompt: python.exe yabasta.py -
20
Beamology
Beamology: Semicom Project Management
Beamology is the leading field of project management for SuperWikia Alpha semicom codesmiths. Its residency programs offer vital resources for SuperWikia vertical environment development and utilities needed for SA-VEC certification. Beamology residents are automatically licensed to access the Transcom system. -
21
TimothyDocs
Timothy is a cloud base storage system designed to document your work
Timothy is a cloud based documentation system. Timothy will document any endeavor because it will store not only the documents created during the project but also store information about those files. Like most storge schemes timothy creates a hierarchy of categories through which one may browse. Timothy displays information about the document or category as well as its name. This use of meta data explains the structure and content of the project to the user as he browses. Users... -
22
Brewery HMI
Web app for all-grain brewers. Automate your electric brewery.
This project is UI to guide the user through the steps of home brewing. Each beer has a recipe of ingredients, but it also has a “recipe” for the process to make the beer. This app allows you to load a recipe (BeerXML), guide you through the brewing steps and record any details so that you can improve next time. By default, this app relies on user inputs and feedback from the real-world. There are easy hooks to a link up a real system, so that if you wanted to automate some of the... -
23
Age and Gender Estimation
Keras implementation of a CNN network for age and gender estimation
Keras implementation of a CNN network for age and gender estimation. This is a Keras implementation of a CNN for estimating age and gender from a face image [1, 2]. In training, the IMDB-WIKI dataset is used. Because the face images in the UTKFace dataset is tightly cropped (there is no margin around the face region), faces should also be cropped in demo.py if weights trained by the UTKFace dataset is used. Please set the margin argument to 0 for tight cropping. You can evaluate a trained... -
24
bluetroller
A library and interface for controlling bluetooth LE devices
bluetroller is a library and interface for controlling all kinds of bluetooth LE devices. A vast number of devices can be controlled via Bluetooth LE, including fitness trackers, lighting, camera sliders, gimbals and many more. Right now these devices can only be controlled via phone apps which are frequently buggy, unmaintained and will stop working after some future phone update. This project aims to grow to become an exhaustive library of these devices. -
25
Python Handout
Turn Python scripts into handouts with Markdown and figures
Handout is a lightweight library for embedding rich, interactive components such as exercises, charts, and interactive diagrams directly into static documents like Markdown, Jupyter notebooks, or static HTML pages, enabling authors to create more engaging technical handouts, tutorials, and interactive essays. It’s particularly aimed at educators, presenters, and researchers who want to make their written material come alive with runnable demonstrations and interactive problem sets without...
