Search Results for "python web crawler"
Sort By:
Showing 38 open source projects for "python web crawler"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
Gen AI apps are built with MongoDB AtlasMongoDB Atlas is the developer-friendly database used to build, scale, and run gen AI and LLM-powered apps—without needing a separate vector database. Atlas offers built-in vector search, global availability across 115+ regions, and flexible document modeling. Start building AI apps faster, all in one place.
-
1
Playwright for Python
Python version of the Playwright testing and automation library
Playwright enables reliable end-to-end testing for modern web apps. Single API to automate Chromium, Firefox and WebKit. Capable automation for single page apps that rely on the modern web platform. Use the Playwright API in JavaScript & TypeScript, Python, .NET and, Java. With Playwright, test how your app behaves in Apple Safari with WebKit builds for Windows, Linux and macOS. Test locally and on CI. Use device emulation to test your responsive web apps in mobile web browsers. Playwright... -
2
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring... -
3
Locust
Scalable open source load testing tool
Locust is an open source user load testing tool written in Python. The idea behind Locust is to swarm your web site or other systems with attacks from simulated users during a test, with each user behavior defined by you using Python code. This swarming process is then monitored from a web UI in real-time, and will help identify any bottlenecks in your code before real users can come in. As it is completely event-based, Locust can have thousands or even millions of simultaneous users... -
4
Playwright for .NET
.NET version of the Playwright testing and automation library
..., JavaScript, Python, .NET, Java. Test Mobile Web. Native mobile emulation of Google Chrome for Android and Mobile Safari. The same rendering engine works on your Desktop and in the Cloud. Auto-wait. Playwright waits for elements to be actionable prior to performing actions. It also has a rich set of introspection events. The combination of the two eliminates the need for artificial timeouts - the primary cause of flaky tests. -
Get the most trusted enterprise browserDefend against security incidents with Chrome Enterprise. Create customizable controls, manage extensions and set proactive alerts to keep your data and employees protected without slowing down productivity.
-
5
Wapiti
Wapiti is a web-application vulnerability scanner
Wapiti is a vulnerability scanner for web applications. It currently search vulnerabilities like XSS, SQL and XPath injections, file inclusions, command execution, XXE injections, CRLF injections, Server Side Request Forgery, Open Redirects... It use the Python 3 programming language. -
6
Taylorplot_Neptune
Creation of a Taylorplot for several machine learning models
Here we present the lines of code for creating a taylor plot with python to display several machine learning models. We show the solution for displaying 10 models, but the list and number can be changed simply by modifying the sample list. -
7
Cinemagoer
Python package to retrieve and manage data of the IMDb
Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb movie database about movies, people, characters and companies. Platform-independent, it can retrieve data from both the IMDb's web server and a local copy of the whole db. -
8
Render 4 Monitoring
Render Web Services for Viewing/Monitoring and Test Web Applications
Renders and aggregates other web services into static HTML monitoring output. Also implements simple and dynamic testing of web based applications using mechanize. Includes Basic authentication, and ADFS authentication for web service testing. Custom authentications can be implemented very easily with python programming. Implements PhantomJS rendering for Javascript/HTML5 dynamic web pages. Requires python3 -
9
CEF Python
Python bindings for the Chromium Embedded Framework (CEF)
... use cases for CEF. Use it as a modern HTML5 based rendering engine that can act as a replacement for classic desktop GUI frameworks. Think of it as Electron for Python. Embed a web browser widget in a classic Qt / GTK / wxPython desktop application. Use it for automated testing of web applications with more advanced capabilities than Selenium web browser automation due to CEF low level programming APIs. -
Simply solve complex auth. Easy for devs to set up. Easy for non-devs to use.Custom auth drains 25% of dev time and risks 62% more breaches, stalling enterprise deals. Frontegg platform delivers a simple login box, seamless authentication (SSO, MFA, passwordless), robust multi-tenancy, and a customizable Admin Portal. Integrate fast with the React SDK, meet compliance needs, and focus on innovation.
-
10
speedtest-cli
Command line interface for testing internet bandwidth using speedtest
... connection performance metrics like download, upload, latency and packet loss natively without relying on a web browser. Test the internet connection of your Linux desktop, a remote server or even lower-powered devices such as the Raspberry Pi with the Speedtest Server Network. Set up automated scripts to collect connection performance data, including trends over time. -
11
Beamology
Beamology: Semicom Project Management
Beamology is the leading field of project management for SuperWikia Alpha semicom codesmiths. Its residency programs offer vital resources for SuperWikia vertical environment development and utilities needed for SA-VEC certification. Beamology residents are automatically licensed to access the Transcom system. -
12
FoxNuke
A Proffesional Stress-Testing(ddos) tool for pentesters
The FoxNuke program is written in python and uses Firefox in order to complete the distributed denial of service attack feature. Multiple headers are used from the Firefox browser, along with a personal configuration option for the Opera browser. The FoxNuke Program is still underdevelopment as of 8/24/17, full release is set to come out sometime during 2017-2018. If you would like to participate in the TESTING of this program and would like to help report bugs, etc. then please email... -
13
A Java Web crawler: multi-threaded, scalable, with high performance, extensible and polite. It can be used to crawl and index any web or enterprise domain and is configurable through a XML configuration file.
-
14
Maximo SOAP Web Service Tester
Simple application for testing XML Web Services in Maximo
Maximo SOAP WebService Tester (Windows) Source: https://github.com/SVSagi/mxwst -
15
Belay Device
Automated Provisioning and Testing Framework
Belay Device has moved! https://github.com/BelayTechnologies/Belay-Device -
16
Cutplace validates tabular data (CSV, fixed format) according to an interface control document (ICD). The ICD acts as executable specification and can be described using popular spreadsheet applications (Calc, Excel).
-
17
sitecheck
Modular web site spider for web developers.
More than just a link checker, sitecheck is a website spider (also known as a crawler) which can assist with SEO by testing an entire site plus both inbound links from search engines and outbound links to other sites for the following issues: looping redirects (HTTP 301/302), broken links (HTTP 404), server errors (HTTP 500), spelling mistakes, low readability scores (using the Flesch Reading Ease test), missing/empty/duplicate meta tags, duplicate content, slow page speed, W3C validation... -
18
z-utalm
Unified Test and Logging layer for multiple programming languages
Modern software systems and application are commonly written in multiple languages, include scripting engines, and are frequently build on multiple specialized frameworks and middleware for a considerable diversity of runtime environments. The latest influencing update in development paradigm is the application of multicore processors. This projects is aimed to unify the required trace and logging output and integrate into debugging environments. The target is to provide general development,... -
19
-
20
ETICS stands for "eInfrastructure for Testing, Integration and Configuration of Software". It provides software professionals with an "out-of-the-box" build and test system, powered with a build and test product repository.
-
21
TestFarm is a client-server Python app that allows monitoring the stability and efficiency of a devel project by performing continuous builds and tests in multiple platforms. The overall idea is based on Tinderbox. With a simpler and extensible approach.
-
22
-
23
-
24
Buildbot is a system to automate the compile/test cycle required by most software projects. Buildbot is no longer hosted on SourceForge. See http://buildbot.net for links and documentation, and http://github.com/buildbot for the source.
-
25
"Yet Another Do-It-Yourself (Test) Automation Framework" specification. Use or modify one of the existing software implementations (hosted/referenced here), or implement the framework yourself. Download and read the spec for more details.
