Search Results for "python web crawler"
Sort By:
Showing 63 open source projects for "python web crawler"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
1
Playwright for Python
Python version of the Playwright testing and automation library
Playwright enables reliable end-to-end testing for modern web apps. Single API to automate Chromium, Firefox and WebKit. Capable automation for single page apps that rely on the modern web platform. Use the Playwright API in JavaScript & TypeScript, Python, .NET and, Java. With Playwright, test how your app behaves in Apple Safari with WebKit builds for Windows, Linux and macOS. Test locally and on CI. -
2
Helium Browser
Private, fast, and honest web browser
Helium is a Chromium-based web browser designed to deliver privacy, speed, and simplicity by removing Google’s proprietary services, telemetry, and bloat. It’s built atop ungoogled-chromium, extending its philosophy with additional privacy features, design refinements, and user experience improvements aimed at transparency and control. Helium blocks ads and trackers by default through an integrated, unbiased uBlock Origin extension prepackaged as a native browser component. Its UI and... -
3
Ungoogled Chromium Windows
Windows packaging for ungoogled-chromium
Ungoogled Chromium Windows is the Windows build configuration and tooling for Ungoogled Chromium, a fork of the Chromium browser that removes Google-specific services, tracking, and invasive integrations to deliver a more privacy-respectful, user-controlled web experience on Microsoft platforms. This project specifically targets the Windows ecosystem, supplying build scripts, patches, and configuration files necessary to compile Chromium without telemetry, alternate error reporting, Google... -
4
CloakBrowser
Stealth Chromium for developers and browser automation.
...This repository contains the MIT-licensed Python, JavaScript, and .NET wrappers. The custom Chromium binary is downloaded separately and distributed under a proprietary license. Proxies and CAPTCHA-solving services are not included. Official website and pricing: https://cloakbrowser.dev/ Official SourceForge product profile: https://sourceforge.net/software/product/CloakBrowser/ -
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
5
Mercury Browser
Privacy-focused web browser fork of Firefox
Mercury Browser is an optimized, privacy-focused web browser that is a fork of Mozilla Firefox. It incorporates compiler optimizations such as AVX, AES, LTO, and PGO to enhance performance and security. With features derived from projects like LibreWolf, Waterfox, and Ghostery, Mercury disables telemetry and debugging elements by default, ensuring a more private browsing experience. It also includes usability patches that bring back features like the classic top bar and supports unsigned... -
6
Ungoogled Chromium
A lightweight approach to removing Google web service dependency
In descending order of significance (i.e. most important objective first), ungoogled-chromium is Google Chromium, sans dependency on Google web services, ungoogled-chromium retains the default Chromium experience as closely as possible. Unlike other Chromium forks that have their own visions of a web browser, ungoogled-chromium is essentially a drop-in replacement for Chromium. ungoogled-chromium features tweaks to enhance privacy, control, and transparency. However, almost all of these... -
7
SeleniumBase
A framework for browser automation and testing with Selenium
SeleniumBase automatically handles common WebDriver actions such as launching web browsers before tests, saving screenshots during failures, and closing web browsers after tests. SeleniumBase lets you customize test runs from the command line. SeleniumBase uses simple syntax for commands. pytest includes automatic test discovery. If you don't specify a specific file or folder to run, pytest will automatically search through all subdirectories for tests to run. No More Flaky Tests!... -
8
OpenWPM
A web privacy measurement framework
OpenWPM is a web privacy measurement framework that makes it easy to collect data for privacy studies on a scale of thousands to millions of websites. OpenWPM is built on top of Firefox, with automation provided by Selenium. It includes several hooks for data collection. Check out the instrumentation section below for more details. OpenWPM is tested on Ubuntu 18.04 via TravisCI and is commonly used via the docker container that this repo builds, which is also based on Ubuntu. Although we... -
9
Scrapy-Redis
Redis-based components for Scrapy
You can start multiple spider instances that share a single redis queue. Best suitable for broad multi-domain crawls. Scraped items gets pushed into a redis queued meaning that you can start as many as needed post-processing processes sharing the items queue. Scheduler + Duplication Filter, Item Pipeline, Base Spiders. Default requests serializer is pickle, but it can be changed to any module with loads and dumps functions. Note that pickle is not compatible between python versions. Version... -
Error to trace to log to deploy. One click. No SSH.AppSignal links every error to the trace, the trace to the log, the log to the deploy that shipped it.
-
10
Eric Integrated Development Environment
Python Development Environment with all batteries included
Eric is a Python IDE written using PyQt and QScintilla. It provides various features such as any number of open editors, an integrated (remote) debugger, project management facilities, unit test, refactoring and much more. -
11
ZK is an open-source Java framework for building modern web and mobile applications. It enables developers to create rich, interactive UIs using only Java — no JavaScript required. With 200+ Ajax-powered components, event-driven architecture, and support for popular technologies like Spring, Java EE, and JSP/JSF, ZK makes it simple to deliver powerful and user-friendly web applications.
-
12
RPA for Python
Python package for doing RPA
Python package for doing RPA. RPA for Python's simple and powerful API makes robotic process automation fun! You can use it to quickly automate away repetitive time-consuming tasks on websites, desktop applications, or the command line. See sample Python script, the RPA Challenge solution, and RedMart groceries example. To send a Telegram app notification, simply look up @rpapybot to allow receiving messages. To automate Chrome browser invisibly, use headless mode. To run 10X faster instead... -
13
Splinter
Splinter - Python test framework for web applications
Splinter is a Python test framework for web applications, providing a simple and consistent API for browser automation and testing. -
14
galacteek
Multi-platform browser for the distributed web
galacteek is a multi-platform Qt5-based browser and semantic agent for the distributed web. Be sure to install all the gstreamer packages on your system to be able to use the mediaplayer. After opening/mounting the DMG image, hold Control and click on the galacteek icon, and select Open and accept. You probably need to allow the system to install applications from anywhere in the security settings. Docker images are available. They run the full GUI inside a virtual Xorg server (using Xvfb).... -
15
HomeTabs
HomeTabs project helps you to organize bookmarks for web browsers
HomeTabs project helps you to organize bookmarks for web browsers (like a standart browser's home page, but cooler and more comfortable). Design of HomeTabs was inspiried by Mozilla Firefox startpage, i think this is the best way to organise bookmarks, but history of browsing saved on homepage - is bad idea. GitHub: https://github.com/grildroid/HomeTabs Discord: https://discord.gg/6ZGDgFjDVm -
16
googler
Google from the terminal
googler is a power tool to Google (web, news, videos and site search) from the command line. It shows the title, URL and abstract for each result, which can be directly opened in a browser from the terminal. Results are fetched in pages (with page navigation). Supports sequential searches in a single googler instance. googler was initially written to cater to headless servers without X. You can integrate it with a text-based browser. However, it has grown into a very handy and flexible... -
17
googler
Google Search, Google Site Search, Google News from the terminal
googler is a power tool to Google (Web & News) and Google Site Search from the command-line. It shows the title, URL and abstract for each result, which can be directly opened in a browser from the terminal. Results are fetched in pages (with page navigation). Supports sequential searches in a single googler instance. googler was initially written to cater to headless servers without X. You can integrate it with a text-based browser. However, it has grown into a very handy and flexible... -
18
CountBookmarks
Makes a detailed count of your browser bookmarks by folder
This simple program performs a detailed count of exported web browser bookmarks by folder. Its output file can be imported into a spreadsheet and sorted to show the relative size of all your bookmark folders. -
19
htmlarea
Small, powerful, full featured WYSIWYG editor
HTMLArea 4 is a browser based WYSIWYG editor that easily replaces the TEXTAREA in your web pages. It is written in JavaScript, and suitable for use in any modern web browser, and any page on your web site. Current version is 4.0-2016-08-29 -
20
WebDAVSurfer
WebDAV client 64-bit works with Plone 5, Apache and more
GUI WebDAV Client for Linux and Windows 10. Includes PROPFIND, PROPPATCH, LOCK ,UNLOCK ,VERSION-CONTROL,REPORT. HTTP(S) with Basic Authentication and PKI client and server Certificates. Works with Plone, Zope, Apache + mod_dav, PyWebDAV, PyDAV, Tamino. 64-bit wxPython used. Upload files or from Web. Update properties. Tested with Plone 5.04 -
21
Flashlight
The missing Spotlight plugin system
Search the web, save a note, add a reminder, or do over 200 other things, just by typing into your Mac's Spotlight search box. Or write your own plugins. Flashlight is an open platform for Spotlight, which allows developers to write plugins that extend OS X's built-in search with new features. Flashlight is an unofficial Spotlight API that allows you to programmatically process queries and add additional results. It's very rough right now, and a horrendous hack, but a fun proof of concept.... -
22
huhamhire-hosts
hosts for Internet Freedom
Since the governments of some countries are using the blocking the internet access to several websites and web service providers which includes some world famous sites like Google, YouTube, twitter, Facebook, and Wikipedia etc., we designed this tiny utility in order to help people getting through the Internet blockade. Hosts Setup Utility provides basic tools to manage the hosts file on current operating systems. It also provides both support for Graphical Desktop environment with... -
23
Web browser plug-in for hosting real 3D on the websites.
-
24
ReorJS
Distributed Computing with JavaScript
Create your own distributed computer that can distributed javascript based applications to any computer with a web browser, headless browser or node.js installation. For more information and updates please see our website - http://reorjs.com. -
25
gooseweb
A lightweight, no bullshit web browser written in python
