Search Results for "python web crawler"
Sort By:
Showing 104 open source projects for "python web crawler"
View related business solutions-
Gen AI apps are built with MongoDB AtlasMongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.
-
Simple, Secure Domain RegistrationRegister or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.
-
1
CssSelector Component
Converts CSS selectors to XPath expressions
XPath expressions are incredibly flexible, so there is almost always an XPath expression that will find the element you need. Unfortunately, they can also become very complicated, and the learning curve is steep. Even common operations (such as finding an element with a particular class) can require long and unwieldy expressions. CSS selectors are less powerful than XPath, but far easier to write, read and understand. Since they are less powerful, almost all CSS selectors can be converted to... -
2
s3cmd
Command line tool for managing Amazon S3 and CloudFront services
Open-source tool to access Amazon S3 file storage. S3cmd is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage. Lots of features and options have been added to s3cmd since its very first release in 2008.... we recently counted more than 60 command line options, including multipart uploads, encryption, incremental backup, s3 sync, ACL and Metadata... -
3
Network Security Toolkit (NST)
A network security analysis and monitoring toolkit Linux distribution.
Network Security Toolkit (NST) is a bootable ISO image (Live USB Flash Drive) based on Fedora 42 providing easy access to best-of-breed Open Source Network Security Applications and should run on most x86_64 systems. The main intent of developing this toolkit was to provide the security professional and network administrator with a comprehensive set of Open Source Network Security Tools. The majority of tools published in the article: Top 125 Security Tools by INSECURE.ORG are available... -
4
Render Farm Manager, Project Tracker.
CGRU: Afanasy render farm manager and RULES project tracker.
CGRU is an open source CG tools pack, includes Afanasy render farm manager and RULES project tracker. -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
5
MailCleaner
Anti Spam SMTP Gateway
MailCleaner Open Source Edition is now discontinued but will continue under the spamtagger project https://github.com/SpamTagger [antispam] MailCleaner is an anti-spam / anti-virus filter SMTP gateway with user and admin web interfaces, quarantine, multi-domains, multi-templates, multi-languages. Using Bayes, RBLs, Spamassassin, MailScanner, ClamAV. Based on Debian. Enterprise ready. MailCleaner is an anti spam gateway installed between your mail infrastructure and the Internet. It... -
6
StrongKey FIDO Server (SKFS)
FIDO® Certified StrongKey FIDO Server (SKFS)
An open source implementation of the FIDO2 protocol to support passwordless strong authentication using public-key cryptography. Supports registration, authentication (all platforms), and transaction authorization (for native Android apps). -
7
FormaVid
Small Business Appliance
The FormaVid Small Business Appliance https://formavid.org is designed to integrate a content management system (CMS), an issues tracker and an invoicing application into a single, well constructed, offering. It is an excellent starting point for any developer(s) wishing to support the CMS or any of the other components, including the appliance itself. All components are stable, open source and well supported. The appliance is built using scripts so no hidden "monkey business" and you can... -
8
SabroCMS CMS and Online FTP
SABROCMS (CONTENT MANAGEMENT SYSTEM) PHP SCRIPT TO EDIT WEBSITES
SabroCMS is a Freemium PHP Script that allows you to manage the files of your hosting or your website, from any web browser, either from your computer, tablet or cell phone, with online FTP functions, ideal for uploading files, deleting files, renaming files, clone files, create or delete directories and especially to create and edit files online, such as HTML files, Javascript JS files, CSS Style Sheets, Python PT Scripts, PHP Scripts, Perl, CGI and all kinds of TXT, XML, and more! -
9
jQuery File Upload
File Upload widget with multiple file selection
jQuery-File-Upload is a mature, full-featured jQuery plugin (often paired with server-side handlers) for handling file uploads from the browser with advanced capabilities. It supports chunked uploads, drag and drop, multiple file selection, progress bars, client-side image resizing, and preview generation. On the server side, artifacts may be processed using compatible back-end scripts in languages like PHP, Ruby, Node.js, or Java, making the plugin cross-platform. Because uploads can be... -
Build Securely on Azure with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
KeexyBox
The box to keep the Internet under your control
KeexyBox allows you to do parental control, block ads, limit telemetry, and browse the Internet anonymously from your home network without installing any software on your devices. It also can be used to create a public wireless access point with the captive portal. It is a software program which requires a Raspberry PI box and which is installed on Raspbian. It constitutes a cut-off point between your devices (computers, tablets, smartphones, etc.) and your router or Internet box.... -
11
PHP mini vulnerability suite
Multiple server/webapp vulnerability scanner
github: https://github.com/samedog/phpmvs -
12
Web Security Dojo
Virtual training environment to learn web app ethical hacking.
Web Security Dojo is a virtual machine that provides the tools, targets, and documentation to learn and practice web application security testing. A preconfigured, stand-alone training environment ideal for classroom and conferences. No Internet required to use. Ideal for those interested in getting hands-on practice for ethical hacking, penetration testing, bug bounties, and capture the flag (CTF). A single OVA file will import into VirtualBox and VMware. There is also an Ansible... -
13
Xplico
Xplico is a Network Forensic Analysis Tool (NFAT)
Xplico is a Network Forensic Analysis Tool (NFAT). The goal of Xplico is extract from an internet traffic capture the applications data contained. For example, from a pcap file Xplico extracts each email (POP, IMAP, and SMTP protocols), all HTTP contents, each VoIP call (SIP, MGCP, MEGACO, RTP), IRC, WhatsApp... Xplico is able to classify more than 140 (application) protocols. Xplico cam be used as sniffer-decoder if used in "live mode" or in conjunction with netsniff-ng. Xplico is used... -
14
A project for a highly scalable protocol and framework and a collection of software tools for m2m (machine to machine communication) and IoT Internet of Things. The programs here works on Windows and Linux and are based around VSCP, The Very Simple Control Protocol. Repository is here: https://github.com/grodansparadis/vscp_software and here https://github.com/grodansparadis/vscp_firmware
-
15
OpenSearchServer Search Engine
An open source search engine with RESTFul API and crawlers
OpenSearchServer is a powerful, enterprise-class, search engine program. Using the web user interface, the crawlers (web, file, database, etc.) and the client libraries (REST/API , Ruby, Rails, Node.js, PHP, Perl) you will be able to integrate quickly and easily advanced full-text search capabilities in your application: Full-text with basic semantic, join queries, boolean queries, facet and filter, document (PDF, Office, etc.) indexation, web scrapping,etc. OpenSearchServer runs on... -
16
owfs and owhttpd
Project moved to https://github.com/owfs/owfs/
Please note that OWFS source code, and all Issue/Tickets/merge requests have now been moved to https://github.com/owfs/owfs/. Developer mailing lists will still be kept at Sourceforge. ---- OWFS -- 1-Wire file system. Use the Dallas 1-Wire and iButton chips with standard filesystem commands. Create temperature loggers. Monitor everything. OWHTTPD -- same system, only used as a light weight web server. OWFS is also ported to embedded routers, Mac OSX and Windows. Basically you can use... -
17
Config Backup for F5
Backup program for F5 Big IP devices
This project is a Python program with PHP web UI to manage daily backups of F5 BigIP devices. It also is available as a turnkey VMware appliance requiring no Linux skills to use. Disclaimer - This project is an independent work under the GPL v2 license and is NOT provided by or supported by F5 Networks. It is provided as is with no warranty. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or any later version. -
18
diskover
File system crawler and disk space usage software
diskover is a file system crawler and disk space usage software that uses Elasticsearch to index your file metadata. diskover crawls and indexes your files on a local computer or remote storage server over network mounts. diskover helps manage your storage by identifying old and unused files and give better insights into data change "hotfiles", file duplication "dupes" and wasted space. It is designed to help deal with managing large amounts of data growth and provide detailed storage... -
19
PotHead
PotHead
PotHead What is it? Pothead: A simple lightweight packet for creating a minimal interaction honeypot How does it work? Magic, Well kinda you don’t need super dupper knowledge to use this! Oops the magic failed ;) a really nice INSTALL.md will be written unfortunately it’s not yet available. 1) Download all the files 2) Check if you got everything 3) Upload and configure the Website and backend (includes/Config.php & the .SQL files) 4) Do not forget to change your... -
20
AlienVault OSSIM
Open Source SIEM
OSSIM, AlienVault’s Open Source Security Information and Event Management (SIEM) product, provides event collection, normalization and correlation. For more advanced functionality, AlienVault Unified Security Management (USM) builds on OSSIM with these additional capabilities: * Log management * Advanced threat detection with a continuously updated library of pre-built correlation rules * Actionable threat intelligence updates from AlienVault Labs Security Research Team * Rich... -
21
GreenHop
Energy and environmental monitoring of the server room
The solution GreenHop aims to perform energy and environmental (temperature, humidity, dew point and atmospheric pressure) monitoring of the Data Center server room, providing energy efficiency indicators through green metrics for Data Centers. The GreenHop solution is based on open source software and hardware. The approach of using open source solutions enables its deployment easier and independently of suppliers at the same time makes the solution scalable to the needs of each... -
22
Ganglia
Scalable, distributed monitoring system for high-performance computing
Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. Supports clusters up to 2000 nodes in size. -
23
Belay Device
Automated Provisioning and Testing Framework
Belay Device has moved! https://github.com/BelayTechnologies/Belay-Device -
24
Grasshopper Web App
Web App to control Bticino MyHome using OpenWebNet
Grasshopper is an open source and free (speech & beer) responsive-design web application to control Bticino MyHome. VALUES: Use any device Since Grasshopper is a browser-based application, you can use a browser on any device to access Grasshopper. Thanks to its responsive-design support, the Grasshopper interface will adapt to the screen-size of your device. Choice of server-platform Grasshopper is a web application that can run from different web servers. The Grasshopper... -
25
Hydroponic Automation Platform (HAPI)
Technologies for automating food production on various scales
The Hydroponic Automation Platform Initiative (HAPI) develops and provides hardware and software components for automating food production using hydroponic, aquaponics, and precision agriculture techniques. High-yield production in urban settings is one of the primary goals. Artifacts include hardware design (mainly Arduino-based), firmware, management software and reporting modules.
