Search Results for "web crawler source code"
Sort By:
Showing 250 open source projects for "web crawler source code"
View related business solutions-
Earn up to 16% annual interest with Nexo.Generate interest, borrow against your crypto, and trade a range of cryptocurrencies — all in one platform. Geographic restrictions, eligibility, and terms apply.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Pholcus
Distributed high-concurrency crawler software written in pure golang
Pholcus is a high-concurrency crawler software written in pure Go language that supports distributed, only used for programming learning and research. It supports three operating modes of stand-alone, server and client, and has three operating interfaces, Web, GUI, and command line; simple and flexible rules, concurrent batch tasks, and rich output methods (mysql/mongodb/kafka/csv/excel, etc.); In addition, it also supports horizontal and vertical grabbing modes, and a series of advanced... -
2
Locust
Scalable open source load testing tool
Locust is an open source user load testing tool written in Python. The idea behind Locust is to swarm your web site or other systems with attacks from simulated users during a test, with each user behavior defined by you using Python code. This swarming process is then monitored from a web UI in real-time, and will help identify any bottlenecks in your code before real users can come in. -
3
SafeLine
Serve as a reverse proxy to protect your web services from attacks
SafeLine is a self-hosted WAF(Web Application Firewall) to protect your web apps from attacks and exploits. A web application firewall helps protect web apps by filtering and monitoring HTTP traffic between a web application and the Internet. It typically protects web apps from attacks such as SQL injection, XSS, code injection, os command injection, CRLF injection, LDAP injection, XPath injection, RCE, XXE, SSRF, path traversal, backdoor, brute force, HTTP-flood, bot abuse, among others. By... -
4
KodExplorer
A web based file manager, web IDE / browser based code editor
KodExplorer is a file manager for the web. It is also a web code editor, which allows you to develop websites directly within the web browser. You can run KodExplorer either online or locally, on Linux, Windows, or Mac-based platforms. The only requirement is to have PHP 5 available. Use experience like operating system, Rich context menu, and toolbar, drag, and drop, shortcut keys. All operations with files and folders on a remote server (copy, cut, paste, move, remove, upload, create... -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
5
multiOTP open source
PHP strong authentication library, web interface & CLI, OATH certified
multiOTP is a PHP class, a powerful command line utility and a web interface developed by SysCo systèmes de communication sa in order to provide a completely free and easy operating system independent server side implementation for strong two factors authentication solution. multiOTP supports hardware and software tokens with different One-Time Password algorithms like OATH/HOTP, OATH/TOTP and mOTP (Mobile-OTP). QRcode generation is also embedded in order to support provisioning of Google... -
6
apm-agent-dotnet
Elastic APM .NET Agent
Elastic APM .NET Agent. The agent can automatically instrument .NET Framework, .NET Core, and .NET applications using the .NET CLR Profiling APIs. The Profiling APIs provide a way to instrument an application or dependency code without code changes. -
7
Pyodide
Pyodide is a Python distribution for the browser and Node.js
Pyodide brings the Python runtime to the browser by compiling Python and its scientific libraries to WebAssembly. It allows developers to run Python code directly in web browsers without a server, supporting packages like NumPy, Pandas, and Matplotlib. Pyodide opens up new possibilities for interactive data analysis, scientific computing, and educational tools in web environments, all while integrating seamlessly with JavaScript. -
8
OpenStatus
Status page with uptime monitoring & API monitoring as code
...Built with modern web technologies such as Next.js and Tailwind CSS, openstatus supports both managed deployments and self-hosted environments using lightweight Docker images. Its monitoring-as-code approach allows teams to define checks in YAML and integrate them into CI/CD workflows for automated validation. -
9
Scalene
High-performance CPU, GPU, and memory profiler for Python
Scalene is a high-performance CPU, GPU and memory profiler for Python that does a number of things that other Python profilers do not and cannot do. It runs orders of magnitude faster than other profilers while delivering far more detailed information. Once Scalene has profiled your program, it will launch a web browser with an interactive user interface (all processing is done locally). Hover over bars to see breakdowns of CPU and memory consumption, and click on underlined column headers... -
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
10
Concourse
Concourse is a container-based continuous thing-doer written in Go
...A Concourse pipeline is like a distributed, continuous Makefile. Each job has a build plan declaring the job's input resources and what to run with them when they change. Your pipeline is then visualized in the web UI, taking only one click to get from a failed job to seeing why it failed. The visualization provides a "gut check" feedback loop: if it looks wrong, it probably is wrong. Jobs can depend on other jobs by configuring passed constraints. The resulting chain of jobs and resources is a dependency graph that continuously pushes your project forward, from source code to production. -
11
Npgsql
Npgsql is the .NET data provider for PostgreSQL
Npgsql is an open source ADO.NET Data Provider for PostgreSQL, it allows programs written in C#, Visual Basic, F# to access the PostgreSQL database server. It is implemented in 100% C# code, is free and is open source. An Entity Framework Core provider is also available and exposes some features unique to the PostgreSQL database to EF Core users. Finally, a legacy Entity Framework 6.x (non-Core) provider is also available but is no longer being actively maintained. Npgsql is the open source... -
12
Datadog Client Libraries for Go
Datadog Go Library including APM tracing, profiling, and security
Datadog Application Performance Monitoring (APM) gives deep visibility into your applications with out-of-the-box performance dashboards for web services, queues, and databases to monitor requests, errors, and latency. Distributed traces seamlessly correlate to browser sessions, logs, profiles, synthetic checks, network, processes, and infrastructure metrics across hosts, containers, proxies, and serverless functions. Navigate directly from investigating a slow trace to identifying the... -
13
cypress
Fast, easy and reliable testing for anything that runs in a browser
The web has evolved. Finally, testing has too. Fast, easy and reliable testing for anything that runs in a browser. Installing Cypress is simple. No dependencies, extra downloads, or changes to your code required. Write tests easily and quickly, and watch them execute in real time as you build your web application. Debugging your tests in CI is as easy as running tests locally. With built in parallelization and load balancing. Record CI test results, screenshots and video, and view... -
14
Play!
Play! - PlayStation2 Emulator
Play! is an open-source cross-platform emulator for Sony’s PlayStation 2 that aims to run PS2 games on a wide range of modern systems including Windows, macOS, Linux, Android, iOS, and web browsers by implementing core PS2 hardware functionality in software. It emulates primary components like the Emotion Engine CPU and graphics synthesizer, translating game code and hardware calls into host system operations, while providing users with controls to load disc images, manage states, and configure rendering options. ... -
15
Coraza
OWASP Coraza WAF is a golang modsecurity compatible firewall library
...Coraza runs the OWASP Core Rule Set (CRS) to protect your web applications from a wide range of attacks, including the OWASP Top Ten, with a minimum of false alerts. CRS protects from many common attack categories including: SQL Injection (SQLi), Cross Site Scripting (XSS), PHP & Java Code Injection, HTTPoxy, Shellshock, Scripting/Scanner/Bot Detection & Metadata & Error Leakages. Coraza is a library at its core, with many integrations to deploy on-premise Web Application Firewall instances. -
16
2FAS Auth Browser Extension
Source code for 2FAS Auth Browser Extension
2FAS Auth Browser Extension is a cross-browser plugin that integrates the 2FAS multi-factor authentication platform directly into the user’s web browsing workflow, helping streamline login flows by interfacing with web pages and 2FA token generation in a seamless way. Once installed in browsers like Chrome, Firefox, or others that support standard extension APIs, it can detect 2FA prompts on login forms and offer to fill or suggest one-time passwords generated by your 2FAS mobile clients or... -
17
Career-Ops
AI-powered job search system built on Claude Code
Career Ops is an open-source platform designed to help individuals manage their job search process with a structured, operations-style approach that treats career development like a pipeline. It provides a system for organizing job applications, tracking progress across different stages, and maintaining visibility into opportunities, much like a lightweight CRM tailored for job seekers. The project emphasizes clarity and accountability, enabling users to monitor applications, follow-ups, and... -
18
Monaspace
An innovative superfamily of fonts for code
Monaspace is a superfamily of coding typefaces designed to improve the reading rhythm and texture of code while preserving the alignment benefits developers expect. It includes multiple coordinated families and weights, with italics and stylistic alternates that retain code clarity rather than introducing overly decorative forms. The fonts offer thoughtful ligatures and contextual features that handle common programming sequences without distorting spacing or meaning. Variable builds enable... -
19
v86
x86 PC emulator and x86-to-wasm JIT, running in the browser
v86 is an open-source x86 PC emulator that runs entirely in the browser using WebAssembly for near-native performance. It translates x86 machine code into WebAssembly at runtime, enabling users to boot and run full operating systems without installing anything locally. The emulator supports a wide range of legacy systems, including Linux, Windows 95/98/2000, FreeDOS, and various experimental OSes. -
20
V2Ray
Shell script that aims to make installing and managing V2Ray
This repository offers an interactive, one-command script that installs and manages a V2Ray-based proxy service on common Linux distributions. It automates routine but error-prone steps such as fetching the latest binaries, generating credentials (for example, UUIDs), writing JSON config files, opening firewall ports, and setting up a systemd service for automatic start. The script includes a menu-driven manager to modify ports, protocols, and users, restart the service, or view logs without... -
21
quadplay
The quadplay fantasy console
quadplay✜ is a fantasy console by CasualEffects for creating and playing retro-style video games on any device. Create and play games on any laptop, desktop, tablet, or phone. Or, build your own programmable arcade machine from a Raspberry Pi, Tegra, or old computer. Create games on Windows, macOS, Linux. Play your games in any modern web browser on a laptop, desktop, tablet, phone, Raspberry Pi 4, or Jetson Nano. 60 fps @ 384 x 224 pixels = 12:7 aspect ≈ 16:9.3. 4096 sRGB (4:4:4) colors.... -
22
Devtron
Tool integration platform for Kubernetes
Devtron deeply integrates with products across the lifecycle of microservices,i.e., CI, CD, security, cost, debugging, and observability via an intuitive web interface. Devtron is designed to be modular, and its functionality can be easily extended with the help of integrations. Devtron CI/CD with GitOps integration is used to automate the builds and deployments and enables the software development teams to focus on meeting the business requirements, code quality, and security. Devtron... -
23
ffsend
Easily and securely share files from the command line
Easily and securely share files and directories from the command line through a safe, private and encrypted link using a single simple command. Files are shared using the Send service and may be up to 1GB. Others are able to download these files with this tool, or through their web browser. All files are always encrypted on the client, and secrets are never shared with the remote host. An optional password may be specified, and a default file lifetime of 1 (up to 20) download or 24 hours is... -
24
Casdoor
An open-source Identity and Access Management (IAM)
A UI-first Identity Access Management (IAM) / Single-Sign-On (SSO) platform supporting OAuth 2.0, OIDC, SAML and CAS, integrated with Casbin RBAC and ABAC permission management. Within a few steps, we can setup a Casdoor app and realize our authorization management. Casdoor has a front-end back-end separation architecture, with maneuverable web UI and supporting high concurrency. Casdoor is supporting multi-languages, using i18n to support multi-languages UI. For more languages support,... -
25
OliveTin
OliveTin gives safe and simple access to predefined shell commands
Give safe and simple access to predefined shell commands from a web interface. OliveTin just runs shell commands, so theoretically, you could integrate with a bunch of stuff just by using curl, ping, etc. However, writing your own shell scripts is a great way to extend OliveTin. Uses only a few MB of RAM and barely any CPU. Written in Go, with a web interface written as a modern, responsive, single-page app that uses the REST/gRPC API. Helps potential contributors be consistent, and helps...
