Search Results for "python web crawler"
Sort By:
Showing 220 open source projects for "python web crawler"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
Powering the best of the internet | FastlyEnsure your websites, applications and services can effortlessly handle the demands of your users with Fastly. Fastly’s portfolio is designed to be highly performant, personalized and secure while seamlessly scaling to support your growth.
-
1
ngx_waf
Handy, High performance, ModSecurity compatible Nginx firewall module
Handy, High-performance Nginx firewall module. Such as black and white list of IPs or IP range, uri black and white list, and request body black list, etc. Directives and rules are easy to write and readable. The IP detection is a constant-time operation. Most of the remaining inspections use caching to improve performance. Compatible with ModSecurity's rules, you can use OWASP ModSecurity Core Rule Set. Supports verifying Google, Bing, Baidu and Yandex crawlers and allowing them... -
2
Lexbor
Lexbor is development of an open source HTML Renderer library
Lexbor is the development of a web browser engine available as a software library; it ships with a free license and has no extra dependencies. For us, speed is an absolute must-have. In our development process, we focus on fastest parsing techniques for HTML, CSS, and fonts, fastest data processing methods, and fastest ways to serve content to end users. Whether you are building a backend that handles millions of HTML documents or a UI-heavy user app, your software’s response rate always... -
3
F·W·K
3D game engine/framework in C, with Luajit and Python bindings now
3D game framework in C, with Luajit and Python bindings now. -
4
GNS3
An advanced network simulator to design and configure virtual networks
Build, Design and Test your network in a risk-free virtual environment and access the largest networking community to help. Whether you are studying for your first networking exam or building out a state-wide telecommunications network, GNS3 offers an easy way to design and build networks of any size without the need for hardware. -
Gen AI apps are built with MongoDB AtlasMongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.
-
5
s3cmd
Command line tool for managing Amazon S3 and CloudFront services
Open-source tool to access Amazon S3 file storage. S3cmd is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage. Lots of features and options have been added to s3cmd since its very first release in 2008.... we recently counted more than 60 command line options, including multipart uploads, encryption, incremental backup, s3 sync, ACL and Metadata... -
6
GnuCOBOL
A free COBOL compiler
GnuCOBOL (formerly OpenCOBOL) is a free, modern COBOL compiler. GnuCOBOL implements a substantial part of the COBOL 85, X/Open COBOL and newer ISO COBOL standards (2002, 2014, 2023), as well as many extensions included in other COBOL compilers (IBM COBOL, MicroFocus COBOL, ACUCOBOL-GT and others). GnuCOBOL translates COBOL into C and internally compiles the translated code using a native C compiler. Build COBOL programs on various platforms, including GNU/Linux, Unix, Mac OS X, and... -
7
... for email traffic (POP and SMTP), content filtering of Web traffic and a "hassle free" VPN solution (based on both OpenVPN and IPsec).
-
8
HEALPix
Data Analysis, Simulations and Visualization on the Sphere
Software for pixelization, hierarchical indexation, synthesis, analysis, and visualization of data on the sphere. Please acknowledge HEALPix by quoting the web page http://healpix.sourceforge.net (or https://healpix.sourceforge.io) and publication: K.M. Gorski et al., 2005, Ap.J., 622, p.759 Full software documentation available at https://healpix.sourceforge.io/documentation.php Wiki Pages: https://sourceforge.net/p/healpix/wiki/Home Exchanging Data with HEALPix (in FITS files): https... -
9
Robolinux
MAX Privacy with Robolinux R12.12
All 3 Robolinux Desktop versions come with new login and grub boot splash screens . The Legacy Bios isos only work in VM's & Non UEFI PC's. Don't miss 50% Off 12+ price only $59.95 Expires August 21 2025 The 12+ Sale details Robo UNTRACKER Custom Installer FREE! Save $40 FAAST Boot Custom Installer FREE! Save $40 Backdoor security Custom Installer FREE! Save $40 12 Advanced upgrades! Privacy App installers FREE! Stealth VM for FREE! Save $40 C Drive to VM for FREE! Save... -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
ZEG / Zero-Effort-Groupware
SOGo Zero-Effort-Groupware
The ZEG (Zero Effort Groupware) edition of SOGo is intended to provide a complete out-of-the-box testing environment of SOGo, the Open Source messaging and calendaring software. -
11
The Sashimi project hosts the Trans-Proteomic Pipeline (TPP), a mature suite of tools for mass-spec (MS, MS/MS) based proteomics: statistical validation, quantitation, visualization, and converters from raw MS data to the open mzML/mzXML formats.
-
12
WallPaper (alias crawlpaper) is a desktop changer (NOT a screensaver) which includes a web crawler for picture download, an audio stream ripper, an audio player, a mini mp3 tag editor,etc. Also included support for .zip and .rar files and an interface to the BerkleyDB code for small databases.
-
13
YehDown
A video downloader from youtube viemo and all major sites
A video downloader . The Official home page for the YehDown tool has been published for the new feature updates. : https://Yehdown.yehigo.com The new Yehdown software download the video with improved download speed. The current update has a best user friendly UI. The tool has support for live , in real-time update for new features. Tested on windows 11. -
14
ciwiki
Personnal or familly wiki with low ressource requirement.
Personal lightweight wiki based on DidiWiki. Upgraded to accept text and highlight color, image resize and video (youtube, dailymotion...) embedded. Written in C, doesn't require a lot of RAM. Works fine on Raspbian (Raspberry Pi). Example of Ciwiki running on Raspberry Pi B+ (700MHz, 512MB): http://inphilly.dyn.dhs.org -
15
MailCleaner
Anti Spam SMTP Gateway
MailCleaner is no longer maintained. It will return soon in another form. [antispam] MailCleaner is an anti-spam / anti-virus filter SMTP gateway with user and admin web interfaces, quarantine, multi-domains, multi-templates, multi-languages. Using Bayes, RBLs, Spamassassin, MailScanner, ClamAV. Based on Debian. Enterprise ready. MailCleaner is an anti spam gateway installed between your mail infrastructure and the Internet. It includes a complete GNU/Linux OS and a graphical web interface... -
16
Quick 2d Plot
Program for live 2d graphical representation of data streams
Quick2dPlot, or q2d for short, is an open source minimalistic plotting program designed for live 2d graphical representation of data streams. The program may be useful for plotting output of different user's application programs, especially in case when the user wants to see a plot or a number of plots during calculations or a data acquisition process. The program is command-driven and uses no widgets. Q2d is written in C, it takes advantage of SDL2 library for plotting. Currently... -
17
This is a C library to check the validity of German and Austrian Bank Account Numbers. All currently defined test methods by Deutsche Bundesbank (Dec 2017: 00 to E4) are implemented. Modules for AWK, Perl, PHP, Python, Ruby, C#.net and VB.net are included too. The package includes also an IBAN converter to generate (german) IBANs and BICs from account data. All currently defined IBAN rules by Deutsche Bundesbank are implemented (Dec 2017: 57 rules) and tested against independent solutions.
-
18
fapws4
Fast Asynchronous Python Web Server
Goal of this project is to re-write Fapws3 but on Python-3.x. Basically, most of the treatments will be made via C instructions, just the logic of the website will be written in Python. -
19
cooling home automation with Arduino(C) and Raspberry(front-end, python). Arduino features: - control PC 4-pin fan with PWM; - measure temperature/humidity with DHT11/22; - adjust fan RPM depend on temperature; - console management; - MQTT serial output. front-end features: - data collection form serial port with MQTT - webUI with for sensors parameters
-
20
SimPET
A web platform for the MC simulation of realistic brain PET data
SimPET (http://www.sim-pet.org) is an open, efficient, and user‐friendly online platform for the generation of synthetic brain PET datasets. The platform offers the ability to generate realistic activity and attenuation maps from patient's PET/CT and MRI images. These maps can then be simulated, and sinograms and simulated images can be downloaded. More advanced features can be obtained by using the SimPET... -
21
lixa
LIXA, LIbre XA, is a free and open source XA transaction manager
... technology enables every application container, like a web server or a shell, to become a two phase commit application server. The client/server architecture of LIXA allows many application containers to share a single LIXA (state) server: this is ideal when horizontal scalability is a must and many identical application containers must refer to a single transactional environment. LIXA can be used with the C, C++, Java, Python and COBOL programming languages. -
22
Cinemagoer
Python package to retrieve and manage data of the IMDb
Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb movie database about movies, people, characters and companies. Platform-independent, it can retrieve data from both the IMDb's web server and a local copy of the whole db. -
23
Repeat
Mouse/keyboard record/replay and automation hotkeys/macros creation
Full-fledged mouse/keyboard record/replay and automation hotkeys/macros creation using modern programming languages, and more advanced automation features. Working across three major OSes: Windows, OSX, and Linux. See more at https://github.com/repeats/Repeat Repeat yourself with some intelligence. This, if used correctly, can improve your productivity greatly. -
24
What is DracOS GNU/Linux Remastered ? DracOS GNU/Linux Remastered ( https://github.com/dracos-linux ) is the Linux operating system from Indonesia , open source is built based on Debian live project under the protection of the GNU General Public License v3.0. This operating system is one variant of Linux distributions, which is used to perform security testing (penetration testing). Dracos linux in Arm by hundreds hydraulic pentest, forensics and reverse engineering. Use a GUI-based...
-
25
OpenEMM e-mail & marketing automation
software for email automation (newsletters, transaction mails, etc.)
This is an archive for old versions of OpenEMM. You will find the latest version of OpenEMM here: https://wiki.openemm.org OpenEMM is a browser-based enterprise application for email automation like info and marketing newsletters, transaction mails or multi-stage email campaigns. OpenEMM offers tons of features for professional users, among them: a great user interface, template-based HTML mailings, automatic bounce management, mail opening and link tracking, lots of graphical realtime...
