Search Results for "web crawler source code"
Sort By:
Showing 70 open source projects for "web crawler source code"
View related business solutions-
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
1
Mongoose Embedded Web Server
An embedded web server
Mongoose is a networking library for C/C++. It implements event-driven non-blocking APIs for TCP, UDP, HTTP, WebSocket, MQTT. It is designed for connecting devices and bringing them online. On the market since 2004, used by vast number of open source and commercial products - it even runs on the International Space Station! Mongoose makes embedded network programming fast, robust, and easy. Cross-platform, works on Linux/UNIX, MacOS, Windows, Android, FreeRTOS, etc. Supported embedded... -
2
YAO
Yao A low code engine to create web services and dashboard
Yao allows developers to create web services by processes. Yao is a low-code engine that creates a database model, writes API services, and describes dashboard interface just by JSON for web & hardware, no code, and 10x productivity. Yao is based on the flow-based programming idea, developed in the Go language, and supports multiple ways to expand the data stream processor. This makes Yao extremely versatile, which can replace programming languages in most scenarios, and is 10 times more... -
3
SWC
Rust-based platform for the Web
...It's used by tools like Next.js, Parcel, and Deno, as well as companies like Vercel, ByteDance, Tencent, Shopify, and more. SWC can be used for both compilation and bundling. For compilation, it takes JavaScript / TypeScript files using modern JavaScript features and outputs valid code that is supported by all major browsers. SWC is 20x faster than Babel on a single thread and 70x faster on four cores. SWC can be downloaded and used as a pre-built binary, or built from the source. SWC (stands for Speedy Web Compiler) is a super-fast TypeScript / JavaScript compiler written in Rust. It's a library for Rust and JavaScript at the same time. ... -
4
Gatsby
Build blazing fast, modern apps and websites with React
Gatsby is a free and open source web framework based on React that allows you to build blazing fast websites and apps. It offers the latest web technologies, such as React.js, Webpack, modern JavaScript, CSS and more to help you build really fast websites. A Progressive Web App generator, it’s fast in every way that matters. -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
Suave
Suave is a simple web development F# library
Suave is a simple web development F# library providing a lightweight web server and a set of combinators to manipulate route flow and task composition. Suave is inspired in the simplicity of Happstack and born out of the necessity of embedding web server capabilities in my own applications. Suave supports Websocket, HTTPS, multiple TCP/IP bindings, Basic Access Authentication, Keep-Alive. Suave also takes advantage of F# asynchronous workflows to perform non-blocking IO. In fact, Suave is... -
6
GoatCounter
Easy web analytics. No tracking of personal data
GoatCounter is an open-source web analytics platform available as a hosted service (free for non-commercial use) or self-hosted app. It aims to offer easy-to-use and meaningful privacy-friendly web analytics as an alternative to Google Analytics or Matomo. Privacy-aware; doesn’t track users with unique identifiers and doesn't need a GDPR notice. Fine-grained control over which data is collected. Also see the privacy policy and GDPR consent notices. Lightweight and fast; adds just ~3.5KB of... -
7
Package Generator
Generates a PHP SDK based on a WSDL, simple and powerful, WSDL to PHP
Package Generator generates a PHP SDK from any WSDL so you can easily consume any SOAP Web Service without wondering how SOAP is used under the hood. Package Generator provides many options to generate your package even if a few are required. This project has been tested with many WSDL and is currently used on the platform Providr.IO. Package Generator generates files that are detailed in the MANIFEST. You are encouraged to read it to understand how and why the files are generated in... -
8
Laravel SOAP Client
Laravel Soap Client
This package provides an expressive, minimal API around the Soap Client from Phpro, allowing you to quickly make outgoing SOAP requests to communicate with other web applications. It is using HTTPplug as a handler with Guzzle as client. Some code is based/copied on/from Laravel HTTP wrapper. -
9
Algernon
Small self-contained pure-Go web server with Lua, Markdown, HTTP/2
Web server with built-in support for QUIC, HTTP/2, Lua, Teal, Markdown, Pongo2, HyperApp, Amber, Sass(SCSS), GCSS, JSX, BoltDB (built-in, stores the database in a file, like SQLite), Redis, PostgreSQL, MariaDB/MySQL, rate limiting, graceful shutdown, plugins, users and permissions. Written in Go. Uses Bolt (built-in), MySQL, PostgreSQL or Redis (recommended) for the database backend, permissions2 for handling users and permissions, gopher-Lua for interpreting and running Lua, optional Teal... -
Earn up to 16% annual interest with Nexo.Generate interest, access liquidity without selling, and execute trades seamlessly. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
10
Serverless Express by Vendia
Run Node.js web applications and APIs using existing frameworks
Run REST APIs and other web applications using your existing Node.js application framework (Express, Koa, Hapi, Sails, etc.), on top of AWS Lambda and Amazon API Gateway. Vendia is the real-time data cloud for rapidly building applications that securely share data across departments, companies, clouds, and regions. We’re excited to announce the latest release of Vendia Share! This release includes new features like smart contracts, user-level transactions, the beta release of Azure support,... -
11
phantomas
Headless Chromium-based web performance metrics collector
Headless Chromium-based modular web performance metrics collector. Modular approach, each metric is generated by a separate "module". phantomas "core" acts as an events emitter that each module can hook into. In-depth metrics such as: number of events bound via jQuery, calls to window.writeor complex and duplicated CSS selectors (via analyze-css). JSON as an output format. Easy integration with other nodejs projects via CommonJS module (see API docs). Metrics can be emitted from the... -
12
AWS Lambda Python Runtime Interf Client
Seamlessly extend your preferred base images to be Lambda compatible
We have open-sourced a set of software packages, Runtime Interface Clients (RIC), that implement the Lambda Runtime API, allowing you to seamlessly extend your preferred base images to be Lambda compatible. The Lambda Runtime Interface Client is a lightweight interface that allows your runtime to receive requests from and send requests to the Lambda service. The Lambda Python Runtime Interface Client is vended through pip. You can include this package in your preferred base image to make... -
13
Site Kit for WordPress
Site Kit is a one-stop solution for WordPress users
Site Kit is a first-party WordPress plugin that brings key Google services into a single dashboard so site owners can see how their content performs and fix issues without leaving wp-admin. After a guided setup and verification flow, it connects properties to Search Console, Analytics, AdSense, PageSpeed Insights, and other services, surfacing the most relevant metrics per page and per site. The plugin focuses on clarity: traffic sources, search queries, top pages, and monetization signals... -
14
Tarsier
Vision utilities for web interaction agents
At Reworkd, we iterated on all these problems across tens of thousands of real web tasks to build a powerful perception system for web agents... Tarsier! In the video below, we use Tarsier to provide webpage perception for a minimalistic GPT-4 LangChain web agent. Tarsier visually tags interactable elements on a page via brackets + an ID e.g. [23]. In doing this, we provide a mapping between elements and IDs for an LLM to take actions upon (e.g. CLICK [23]). We define interactable elements... -
15
Fronsetia
A web application for testing webservices
Fronsetia (Free Online Service Testing Application) is a Java web application that allows testing webservices (REST and SOAP). No end-user installations required, just deploy the application on your server and pass the link so that users can use their usual browsers to access Fronsetia. You can think of Fronsetia as a simple version of SoapUI on the web, or a simple and free and open-source version of the "wls_utc" utility. See the project homepage https://fronsetia.sourceforge.io and... -
16
Amazonka
A comprehensive Amazon Web Services SDK for Haskell
An Amazon Web Services SDK for Haskell with support for most public services. Parts of the code contained in this repository are auto-generated and automatically kept up to date with Amazon's latest service APIs. You can find the latest Haddock documentation for each respective library on the Amazonka website. A release changelog can be found in lib/amazonka/CHANGELOG.md. -
17
A generic platform for fast and flexible development and delivery of a wide variety of immersive scenario-based games that enable professional skills acquisition. For further information see http://emergo.ou.nl/emergo/community/EN/emergo.htm. The platform is based on ZK framework (http://www.zkoss.org, ZK ZOL license: https://www.zkoss.org/license#peee) and Spring framework (http://www.springsource.org/spring-framework). NOTE: as of 2024, latest (mavenized) code base can be found at...
-
18
MyPoorWebServer
Demonstrates fundamental HTTP server implementation principles
MyPoorWebServer is a C/C++-based web server project that demonstrates fundamental HTTP server implementation principles drawn from classic network programming literature and high-performance server design books. The repository contains source code that implements a basic HTTP server, intended to be compiled and run from the command line, exposing introductory web server functionality such as serving static HTML files and handling simple POST requests. ... -
19
PreMiD
Source code of the PreMiD application
Source code of the PreMiD application. PreMiD is a simple, configurable utility that allows you to show what you're doing on the web in your Discord now playing status. It supports many different websites, and will support multiple users watching the same content simultaneously in an upcoming update. -
20
Dato.rss
The best RSS Search experience you can find
dato.rss is a Ruby gem that provides a simple DSL for generating RSS 2.0 feeds. It allows developers to build XML-compliant RSS feeds directly within Ruby code using clean, readable syntax. Ideal for blogs, news aggregators, and content-heavy applications, dato.rss helps deliver structured content feeds without dealing with raw XML. -
21
Static Site Boilerplate
A better workflow for building modern static websites
A better workflow for building modern static websites. Automated build processes, a local development server, production minification and optimizations, and the latest standards for static websites. Get up and running in seconds with the built-in development server. No need to have a pre-existing server installed on your machine to start developing. Your HTML, styles, and scripts will be automatically linked, concatenated, and minified for optimal loading speed in production. Write your CSS... -
22
Jeeplatform
Basic platform for enterprise informatization development
...The code has been donated to the open-source Chinese community. -
23
crawler4j
Open source web crawler for Java
crawler4j is an open source web crawler for Java which provides a simple interface for crawling the Web. Using it, you can setup a multi-threaded web crawler in few minutes. You need to create a crawler class that extends WebCrawler. This class decides which URLs should be crawled and handles the downloaded page. shouldVisit function decides whether the given URL should be crawled or not. -
24
PyXB (“pixbee”) is a pure Python package that generates Python source code for classes that correspond to data structures defined by XMLSchema. In concept it is similar to JAXB for Java and CodeSynthesis XSD for C++.
-
25
Linux Dash
A beautiful web dashboard for Linux
Using the 3 bar hamburger icon on each module (widget) on Linux Dash, you can drag and re-arrange them. It is strongly recommended that all linux-dash installations be protected via a security measure of your choice. All of your changes are saved in LocalStorage, so your changes will be preserved permanently on that browser for convenience. Each of the modules (widgets) on the Linux Dash screen can be minimized to hide it, expanded in one click to maximize it, adjusted to a custom width....
