Search Results for "web crawler source code"
Sort By:
Showing 13 open source projects for "web crawler source code"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
1
IPFS Web UI
A frontend for an IPFS node
A web interface to IPFS, shipped with Kubo, and ipfs-desktop. Check on your node stats, explore the IPLD powered merkle forest, see peers around the world and manage your files, without needing to touch the CLI. When working on the code, run an ipfs daemon, the local dev server, the unit tests, and the storybook component viewer and see the results of your changes as you save files. If you need to run IPFS in a Docker container, you can still have Web UI available by exposing both the... -
2
Playwright Skill for Claude Code
Claude Code Skill for browser automation with Playwright
...The system supports a wide range of use cases, including testing web applications, validating user interfaces, automating workflows, and extracting data from websites. One of its key advantages is its ability to generate custom Playwright code tailored to each request, allowing flexible and context-aware automation. -
3
miniblink49
Lighter, faster browser kernel of blink to integrate HTML UI in apps
miniblink is an open source, one file, small browser widget based on chromium. By using C interface, you can create a browser with just some line code. miniblink is an open source, single-file, and currently the smallest known chromium-based browser control. Through its exported pure C interface, a browser control can be created in a few lines of code. C++, C#, Delphi and other language calls (support C++, C#, Delphi language to call). -
4
crawlee
A web scraping and browser automation library for Node.js
Crawlee is a web scraping and browser automation library. It helps you build reliable crawlers. Fast. Crawlee won't fix broken selectors for you (yet), but it helps you build and maintain your crawlers faster. When a website adds JavaScript rendering, you don't have to rewrite everything, only switch to one of the browser crawlers. When you later find a great API to speed up your crawls, flip the switch back. It keeps your proxies healthy by rotating them smartly with good fingerprints that... -
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
5
FlareSolverr
Proxy server to bypass Cloudflare protection
FlareSolverr is a proxy server to bypass Cloudflare and DDoS-GUARD protection. FlareSolverr starts a proxy server, and it waits for user requests in an idle state using few resources. When some request arrives, it uses puppeteer with the stealth plugin to create a headless browser (Firefox). It opens the URL with user parameters and waits until the Cloudflare challenge is solved (or timeout). The HTML code and the cookies are sent back to the user, and those cookies can be used to bypass... -
6
Mink
PHP web browser emulator abstraction
Mink is an open-source PHP library that provides a browser abstraction layer for web application testing, allowing developers to simulate user interactions with websites in a consistent and flexible way. Instead of tying test logic to a specific browser driver, Mink introduces a unified API that can work with multiple drivers such as Goutte, Selenium, ChromeDriver, or BrowserKit. -
7
Loki
Visual Regression Testing for Storybook
There are a few visual regression tools for the web, but most either cannot be run headless or use phantomjs which is deprecated and a browser nobody is actually using. They usually also require you to maintain fixtures. With react-native it's now possible to target multiple platforms with a single code base, but there's no single tool to test all to my knowledge. Loki aims to have easy setup, no to low maintenance cost, reproducible tests independent of which OS they are run on, runnable on... -
8
Plasmic
Build apps, websites, and content. Integrate with your codebase
Plasmic is a visual development platform designed to help teams design, build, and manage websites and web applications through a combination of visual editing tools and code integration. The platform provides a drag-and-drop design environment that allows designers, developers, and product teams to collaborate on building user interfaces while maintaining compatibility with modern front-end frameworks. Instead of requiring developers to manually code every layout, Plasmic allows visual... -
9
RPA for Python
Python package for doing RPA
Python package for doing RPA. RPA for Python's simple and powerful API makes robotic process automation fun! You can use it to quickly automate away repetitive time-consuming tasks on websites, desktop applications, or the command line. See sample Python script, the RPA Challenge solution, and RedMart groceries example. To send a Telegram app notification, simply look up @rpapybot to allow receiving messages. To automate Chrome browser invisibly, use headless mode. To run 10X faster instead... -
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
10
generator-ngx-rocket
Extensible Angular 13+ enterprise-grade project generator
Extensible Angular 5+ enterprise-grade project generator based on angular-cli with best practices from the community. Bootstrap, Angular Material or even Ionic are available, just pick one and start to code. Save time and maintenance effort by creating Progressive Web App and mobile app with the same code base. The app template is based on HTML5, TypeScript and Sass. Optimized build and bundling process with Webpack. Development server with backend proxy and live reload. Cross-browser CSS... -
11
Browser Preview for VS Code
A real browser preview inside your editor that you can debug
Browser Preview for VS Code enables you to open a real browser preview inside your editor that you can debug. Browser Preview is powered by headless Chromium, and works by starting a headless Chromium instance in a new process. This can either be Google Chrome or Microsoft Edge. This enables a secure way to render web content inside VS Code, and enables interesting features such as in-editor debugging and more! Make sure you have Google Chrome installed on your computer. Browser Preview for... -
12
Zombie.js
Fast, full-stack, headless browser testing using node.js
Insanely fast, headless full-stack testing using Node.js. Zombie 6.x is tested to work with Node 8 or later. If you need to use Node 6, consider using Zombie 5.x. If you’re going to write an insanely fast, headless browser, how can you not call it Zombie? Zombie it is. Zombie.js is a lightweight framework for testing client-side JavaScript code in a simulated environment. No browser required. Zombie will work with other testing frameworks. Since Mocha supports promises. Just like your... -
13
WKZombie
WKZombie is a Swift framework for iOS/OSX to navigate within websites
WKZombie is a Swift framework for iOS/OSX to navigate within websites and collect data without the need of a User Interface or API, also known as a Headless browser. It can be used to run automated tests/snapshots and manipulate websites using Javascript. WKZombie is an iOS/OSX web-browser without a graphical user interface. It was developed as an experiment in order to familiarize myself with using functional concepts written in Swift 4. It incorporates WebKit (WKWebView) for rendering and...
- Previous
- You're on page 1
- Next
