Search Results for "python web crawler"
Sort By:
Showing 52 open source projects for "python web crawler"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
Secure remote access solution to your private network, in the cloud or on-prem.OpenVPN is here to bring simple, flexible, and cost-effective secure remote access to companies of all sizes, regardless of where their resources are located.
-
1
Playwright for Python
Python version of the Playwright testing and automation library
Playwright enables reliable end-to-end testing for modern web apps. Single API to automate Chromium, Firefox and WebKit. Capable automation for single page apps that rely on the modern web platform. Use the Playwright API in JavaScript & TypeScript, Python, .NET and, Java. With Playwright, test how your app behaves in Apple Safari with WebKit builds for Windows, Linux and macOS. Test locally and on CI. Use device emulation to test your responsive web apps in mobile web browsers. Playwright... -
2
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring... -
3
Locust
Scalable open source load testing tool
Locust is an open source user load testing tool written in Python. The idea behind Locust is to swarm your web site or other systems with attacks from simulated users during a test, with each user behavior defined by you using Python code. This swarming process is then monitored from a web UI in real-time, and will help identify any bottlenecks in your code before real users can come in. As it is completely event-based, Locust can have thousands or even millions of simultaneous users... -
4
Maltrail
Malicious traffic detection system
Maltrail is a malicious traffic detection system, utilizing publicly available (black)lists containing malicious and/or generally suspicious trails, along with static trails compiled from various AV reports and custom user-defined lists, where trail can be anything from domain name, URL, IP address (e.g. 185.130.5.231 for the known attacker) or HTTP User-Agent header value (e.g. sqlmap for automatic SQL injection and database takeover tool). Also, it uses (optional) advanced heuristic... -
Gen AI apps are built with MongoDB AtlasMongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.
-
5
Playwright for .NET
.NET version of the Playwright testing and automation library
..., JavaScript, Python, .NET, Java. Test Mobile Web. Native mobile emulation of Google Chrome for Android and Mobile Safari. The same rendering engine works on your Desktop and in the Cloud. Auto-wait. Playwright waits for elements to be actionable prior to performing actions. It also has a rich set of introspection events. The combination of the two eliminates the need for artificial timeouts - the primary cause of flaky tests. -
6
Wapiti
Wapiti is a web-application vulnerability scanner
Wapiti is a vulnerability scanner for web applications. It currently search vulnerabilities like XSS, SQL and XPath injections, file inclusions, command execution, XXE injections, CRLF injections, Server Side Request Forgery, Open Redirects... It use the Python 3 programming language. -
7
Taylorplot_Neptune
Creation of a Taylorplot for several machine learning models
Here we present the lines of code for creating a taylor plot with python to display several machine learning models. We show the solution for displaying 10 models, but the list and number can be changed simply by modifying the sample list. -
8
eCxx
A C++ library for AVR and NodeMCU
NOTE: This project is marked with 'Status: Abandoned' on SourceForge because not enough time can be dedicated to this project. However it may still get sporadic commits to the repository. eCxx is a library for AVR and NodeMCU tailored for micro LED displays and lighting effects. eCxx is utilizing Makefile build system. Java and Python based applications/tools are also included to ease the development and debugging process using the host PC. On one side, eCxx supports the original... -
9
Cinemagoer
Python package to retrieve and manage data of the IMDb
Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb movie database about movies, people, characters and companies. Platform-independent, it can retrieve data from both the IMDb's web server and a local copy of the whole db. -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
Render 4 Monitoring
Render Web Services for Viewing/Monitoring and Test Web Applications
Renders and aggregates other web services into static HTML monitoring output. Also implements simple and dynamic testing of web based applications using mechanize. Includes Basic authentication, and ADFS authentication for web service testing. Custom authentications can be implemented very easily with python programming. Implements PhantomJS rendering for Javascript/HTML5 dynamic web pages. Requires python3 -
11
What is DracOS GNU/Linux Remastered ? DracOS GNU/Linux Remastered ( https://github.com/dracos-linux ) is the Linux operating system from Indonesia , open source is built based on Debian live project under the protection of the GNU General Public License v3.0. This operating system is one variant of Linux distributions, which is used to perform security testing (penetration testing). Dracos linux in Arm by hundreds hydraulic pentest, forensics and reverse engineering. Use a GUI-based...
-
12
CEF Python
Python bindings for the Chromium Embedded Framework (CEF)
... use cases for CEF. Use it as a modern HTML5 based rendering engine that can act as a replacement for classic desktop GUI frameworks. Think of it as Electron for Python. Embed a web browser widget in a classic Qt / GTK / wxPython desktop application. Use it for automated testing of web applications with more advanced capabilities than Selenium web browser automation due to CEF low level programming APIs. -
13
speedtest-cli
Command line interface for testing internet bandwidth using speedtest
... connection performance metrics like download, upload, latency and packet loss natively without relying on a web browser. Test the internet connection of your Linux desktop, a remote server or even lower-powered devices such as the Raspberry Pi with the Speedtest Server Network. Set up automated scripts to collect connection performance data, including trends over time. -
14
Zebrunner Community Edition
Test Automation Management Tool
Zebrunner CE (Community Edition) is a Test Automation Management Tool for continuous testing and continuous deployment. It allows you to run various kinds of tests and gain successive levels of confidence in the code quality. Zebrunner CE is integrated by default with Carina open-source TestNG framework and uses Jenkins as a CI Tool. It is built on top of popular docker solutions and includes Postgres database, Zebrunner Reporting, Jenkins Master/Slaves Nodes, Selenium Hub, Mobile... -
15
Beamology
Beamology: Semicom Project Management
Beamology is the leading field of project management for SuperWikia Alpha semicom codesmiths. Its residency programs offer vital resources for SuperWikia vertical environment development and utilities needed for SA-VEC certification. Beamology residents are automatically licensed to access the Transcom system. -
16
TestRest
TestRest is a fully QA Management Tool
TestRest is Test Management offers test case authoring, reusable test cases, test execution and reporting. TestRest supports statistic and graph reports with simple modern UI interfaces. -
17
FoxNuke
A Proffesional Stress-Testing(ddos) tool for pentesters
The FoxNuke program is written in python and uses Firefox in order to complete the distributed denial of service attack feature. Multiple headers are used from the Firefox browser, along with a personal configuration option for the Opera browser. The FoxNuke Program is still underdevelopment as of 8/24/17, full release is set to come out sometime during 2017-2018. If you would like to participate in the TESTING of this program and would like to help report bugs, etc. then please email... -
18
A Java Web crawler: multi-threaded, scalable, with high performance, extensible and polite. It can be used to crawl and index any web or enterprise domain and is configurable through a XML configuration file.
-
19
Maximo SOAP Web Service Tester
Simple application for testing XML Web Services in Maximo
Maximo SOAP WebService Tester (Windows) Source: https://github.com/SVSagi/mxwst -
20
Belay Device
Automated Provisioning and Testing Framework
Belay Device has moved! https://github.com/BelayTechnologies/Belay-Device -
21
Cutplace validates tabular data (CSV, fixed format) according to an interface control document (ICD). The ICD acts as executable specification and can be described using popular spreadsheet applications (Calc, Excel).
-
22
sitecheck
Modular web site spider for web developers.
More than just a link checker, sitecheck is a website spider (also known as a crawler) which can assist with SEO by testing an entire site plus both inbound links from search engines and outbound links to other sites for the following issues: looping redirects (HTTP 301/302), broken links (HTTP 404), server errors (HTTP 500), spelling mistakes, low readability scores (using the Flesch Reading Ease test), missing/empty/duplicate meta tags, duplicate content, slow page speed, W3C validation... -
23
z-utalm
Unified Test and Logging layer for multiple programming languages
Modern software systems and application are commonly written in multiple languages, include scripting engines, and are frequently build on multiple specialized frameworks and middleware for a considerable diversity of runtime environments. The latest influencing update in development paradigm is the application of multicore processors. This projects is aimed to unify the required trace and logging output and integrate into debugging environments. The target is to provide general development,... -
24
Application Radi-testdir is a light weight test Management tool. Radi supports testdirectory features like configuring the test plan, updating (create/edit) the test results for the test image/build, Backup, User Management.
-
25
