Search Results for "python web crawler"
Sort By:
Showing 38 open source projects for "python web crawler"
View related business solutions-
Grafana: The open and composable observability platformGrafana is the open source analytics & monitoring solution for every database.
-
Easy-to-use Business Software for the Waste Management Software IndustryDOP Software’s mission is to streamline waste and recycling business’ processes by providing them with dynamic, comprehensive software and services that increase productivity and quality of performance.
-
1
Playwright for Python
Python version of the Playwright testing and automation library
Playwright enables reliable end-to-end testing for modern web apps. Single API to automate Chromium, Firefox and WebKit. Capable automation for single page apps that rely on the modern web platform. Use the Playwright API in JavaScript & TypeScript, Python, .NET and, Java. With Playwright, test how your app behaves in Apple Safari with WebKit builds for Windows, Linux and macOS. Test locally and on CI. -
2
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring... -
3
Locust
Scalable open source load testing tool
Locust is an open source user load testing tool written in Python. The idea behind Locust is to swarm your web site or other systems with attacks from simulated users during a test, with each user behavior defined by you using Python code. This swarming process is then monitored from a web UI in real-time, and will help identify any bottlenecks in your code before real users can come in. As it is completely event-based, Locust can have thousands or even millions of simultaneous users distributed over multiple machines swarming your system. ... -
4
Playwright for .NET
.NET version of the Playwright testing and automation library
Playwright for .NET is the official language port of Playwright, the library to automate Chromium, Firefox and WebKit with a single API. Playwright is built to enable cross-browser web automation that is ever-green, capable, reliable and fast. Cross-browser. Playwright supports all modern rendering engines including Chromium, WebKit, and Firefox. Cross-platform. Test on Windows, Linux, and macOS, locally or on CI, headless or headed. Cross-language. Use the Playwright API in TypeScript, JavaScript, Python, .NET, Java. ... -
Lightspeed golf course management softwareFrom tee sheet management, point of sale and payment processing to marketing, automation, reporting and more—Lightspeed is built for the pro shop, restaurant, back office, beverage cart and beyond.
-
5
Wapiti
Wapiti is a web-application vulnerability scanner
Wapiti is a vulnerability scanner for web applications. It currently search vulnerabilities like XSS, SQL and XPath injections, file inclusions, command execution, XXE injections, CRLF injections, Server Side Request Forgery, Open Redirects... It use the Python 3 programming language. -
6
Taylorplot_Neptune
Creation of a Taylorplot for several machine learning models
Here we present the lines of code for creating a taylor plot with python to display several machine learning models. We show the solution for displaying 10 models, but the list and number can be changed simply by modifying the sample list. -
7
Render 4 Monitoring
Render Web Services for Viewing/Monitoring and Test Web Applications
Renders and aggregates other web services into static HTML monitoring output. Also implements simple and dynamic testing of web based applications using mechanize. Includes Basic authentication, and ADFS authentication for web service testing. Custom authentications can be implemented very easily with python programming. Implements PhantomJS rendering for Javascript/HTML5 dynamic web pages. -
8
Cinemagoer
Python package to retrieve and manage data of the IMDb
Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb movie database about movies, people, characters and companies. Platform-independent, it can retrieve data from both the IMDb's web server and a local copy of the whole db. -
9
CEF Python
Python bindings for the Chromium Embedded Framework (CEF)
...There are numerous use cases for CEF. Use it as a modern HTML5 based rendering engine that can act as a replacement for classic desktop GUI frameworks. Think of it as Electron for Python. Embed a web browser widget in a classic Qt / GTK / wxPython desktop application. Use it for automated testing of web applications with more advanced capabilities than Selenium web browser automation due to CEF low level programming APIs. -
Collect! is a highly configurable debt collection softwareThe flexible & scalable debt collection software built to automate your workflow. From startup to enterprise, we have the solution for you.
-
10
speedtest-cli
Command line interface for testing internet bandwidth using speedtest
Command line interface for testing internet bandwidth using speedtest.net. It is not a goal of this application to be a reliable latency reporting tool. Latency reported by this tool should not be relied on as a value indicative of ICMP style latency. It is a relative value used for determining the lowest latency server for performing the actual speed test against. Speedtest CLI brings the trusted technology and global server network behind Speedtest to the command line. Measure internet... -
11
Beamology
Beamology: Semicom Project Management
Beamology is the leading field of project management for SuperWikia Alpha semicom codesmiths. Its residency programs offer vital resources for SuperWikia vertical environment development and utilities needed for SA-VEC certification. Beamology residents are automatically licensed to access the Transcom system. -
12
FoxNuke
A Proffesional Stress-Testing(ddos) tool for pentesters
The FoxNuke program is written in python and uses Firefox in order to complete the distributed denial of service attack feature. Multiple headers are used from the Firefox browser, along with a personal configuration option for the Opera browser. The FoxNuke Program is still underdevelopment as of 8/24/17, full release is set to come out sometime during 2017-2018. If you would like to participate in the TESTING of this program and would like to help report bugs, etc. then please email... -
13
A Java Web crawler: multi-threaded, scalable, with high performance, extensible and polite. It can be used to crawl and index any web or enterprise domain and is configurable through a XML configuration file.
-
14
Maximo SOAP Web Service Tester
Simple application for testing XML Web Services in Maximo
Maximo SOAP WebService Tester (Windows) Source: https://github.com/SVSagi/mxwst -
15
Belay Device
Automated Provisioning and Testing Framework
Belay Device has moved! https://github.com/BelayTechnologies/Belay-Device -
16
Cutplace validates tabular data (CSV, fixed format) according to an interface control document (ICD). The ICD acts as executable specification and can be described using popular spreadsheet applications (Calc, Excel).
-
17
sitecheck
Modular web site spider for web developers.
More than just a link checker, sitecheck is a website spider (also known as a crawler) which can assist with SEO by testing an entire site plus both inbound links from search engines and outbound links to other sites for the following issues: looping redirects (HTTP 301/302), broken links (HTTP 404), server errors (HTTP 500), spelling mistakes, low readability scores (using the Flesch Reading Ease test), missing/empty/duplicate meta tags, duplicate content, slow page speed, W3C validation... -
18
z-utalm
Unified Test and Logging layer for multiple programming languages
Modern software systems and application are commonly written in multiple languages, include scripting engines, and are frequently build on multiple specialized frameworks and middleware for a considerable diversity of runtime environments. The latest influencing update in development paradigm is the application of multicore processors. This projects is aimed to unify the required trace and logging output and integrate into debugging environments. The target is to provide general development,... -
19
-
20
ETICS stands for "eInfrastructure for Testing, Integration and Configuration of Software". It provides software professionals with an "out-of-the-box" build and test system, powered with a build and test product repository.
-
21
TestFarm is a client-server Python app that allows monitoring the stability and efficiency of a devel project by performing continuous builds and tests in multiple platforms. The overall idea is based on Tinderbox. With a simpler and extensible approach.
-
22
-
23
-
24
Buildbot is a system to automate the compile/test cycle required by most software projects. Buildbot is no longer hosted on SourceForge. See http://buildbot.net for links and documentation, and http://github.com/buildbot for the source.
-
25
"Yet Another Do-It-Yourself (Test) Automation Framework" specification. Use or modify one of the existing software implementations (hosted/referenced here), or implement the framework yourself. Download and read the spec for more details.
