Search Results for "web crawler source code" - Page 5
Sort By:
Showing 3462 open source projects for "web crawler source code"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
1
Shrinko8
Shrink (minify) Pico-8 (& Picotron) carts, as well as other tools
Shrinko8 is a set of tools aimed at optimizing PICO-8 and Picotron cartridges by reducing their code size. It offers functionalities like code minification, linting, and format conversion, helping developers maximize the limited token space available in PICO-8. -
2
Srcbook
TypeScript-centric app development platform
Srcbook is an open-source, local-first development environment that blends a TypeScript notebook with an AI-assisted app builder, giving developers a flexible space to prototype, explore, and build full web applications with AI support and real execution. It runs locally through a CLI tool backed by Node.js, launching a web interface where users can write and execute TypeScript code interactively, visualize results, and iterate quickly on ideas without leaving the browser. ... -
3
CodeMirror 5
In-browser code editor (version 5, legacy)
CodeMirror is a versatile text editor implemented in JavaScript for the browser. It is specialized for editing code, and comes with a number of language modes and addons that implement more advanced editing functionality. A rich programming API and a CSS theming system are available for customizing CodeMirror to fit your application, and extending it with new functionality. CodeMirror 5 is a JavaScript library that provides an embeddable code editor for web applications, featuring... -
4
Brokk
Brokk brings code intelligence to AI
Brokk is a code intelligence assistant framework designed to let large language models (LLMs) understand code semantically (not just as raw text) so that they can work effectively on large codebases that don’t fit wholly in a prompt context. It helps bridge the gap between LLMs and real-world engineering code by offering tooling to index, analyze, query, and augment code context, so that AI can meaningfully reason about existing code, suggest edits, and navigate across projects. Modular... -
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
5
Genie.jl
The highly productive Julia web framework
Genie Framework includes all you need to quickly build production-ready web applications with Julia. Develop Julia backends, create beautiful web UIs, build data applications and dashboards, integrate with databases and set up high-performance web services and APIs. Genie Builder is a free VSCode plugin for quickly building Julia apps without writing frontend code. Drag and drop UI components such as text, sliders, plots, and data tables onto a canvas, and connect them to the variables in... -
6
Penpot
The first open source design and prototyping tool
Penpot is the first open-source design and prototyping platform for product teams that allows true collaboration between designers and developers. What will designers and developers find in Penpot? - Powerful UI tool for the team. All stakeholders can access a complete set of interactive prototypes, design systems, components, feedback loop and pixel designs. - Design and development collaborate for real. Penpot brings both code-ready design capabilities and the familiarity of developer... -
7
JavaScript Obfuscator
A powerful obfuscator for JavaScript and Node.js
JavaScript Obfuscator is a Node.js library and CLI that transforms readable JavaScript into hardened, difficult-to-reverse code. It applies techniques such as identifier mangling, string array extraction/encoding, control-flow flattening, dead-code injection, and numeric literal transformations to disguise intent. Advanced options include self-defending code, domain locking, debug/console protection, and property key transformation, allowing you to tailor defenses to your threat model. The... -
8
Biome
A toolchain for web projects, aimed to provide functionalities
Biome formats and lints your code in a fraction of a second. Biome supports JavaScript, TypeScript, JSON, and CSS. It aims to support all main languages of modern web development. Biome has sane defaults and requires minimal configuration. Biome helps you as much as possible by displaying detailed and contextualized diagnostics. Biome unifies functionality that has previously been separate tools. Building upon a shared base allows us to provide a cohesive experience for processing code,... -
9
Dash
Build beautiful web-based analytic apps, no JavaScript required
Dash is a Python framework for building beautiful analytical web applications without any JavaScript. Built on top of Plotly.js, React and Flask, Dash easily achieves what an entire team of designers and engineers normally would. It ties modern UI controls and displays such as dropdown menus, sliders and graphs directly to your analytical Python code, and creates exceptional, interactive analytics apps. Dash apps are very lightweight, requiring only a limited number of lines of Python or... -
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
10
Ace Editor
Ace (Ajax.org Cloud9 Editor)
Ace is an embeddable code editor written in JavaScript. It matches the features and performance of native editors such as Sublime, Vim and TextMate. It can be easily embedded in any web page and JavaScript application. Ace is maintained as the primary editor for Cloud9 IDE and is the successor of the Mozilla Skywriter (Bespin) project. -
11
Introduction to Zig
An open, technical and introductory book for the Zig programming lang
This is the official repository for the book "Introduction to Zig: a project-based Book", written by Pedro Duarte Faria. To know more about the book, check out the About this book section below. You can read the current version of the book in your web browser. The book is built using the publishing system Quarto in conjunction with a little bit of R code (zig_engine.R), which is responsible for calling the Zig compiler to compile and run the Zig code examples. -
12
Cloud CLI
Use Claude Code, Cursor CLI or Codex on mobile and web with CloudCLI
ClaudeCodeUI, also known as CloudCLI, is an open-source web and mobile interface that enables developers to control Claude Code and similar AI coding tools through a graphical environment. The project provides a browser-based dashboard that allows users to manage coding sessions, projects, and prompts remotely from virtually any device. It is designed to bridge the gap between terminal-first AI coding workflows and more accessible visual interfaces, improving usability without sacrificing power. ... -
13
SQLite JDBC Driver
SQLite JDBC Driver
SQLite JDBC is a library for accessing and creating SQLite database files in Java. Our SQLiteJDBC library requires no configuration since native libraries for major OSs, including Windows, Mac OS X, Linux etc., are assembled into a single JAR (Java Archive) file. The usage is quite simple; download our sqlite-jdbc library, then append the library (JAR file) to your classpath. SQLite JDBC is a library for accessing SQLite databases through the JDBC API. SQLite supports on-memory database... -
14
Solana
Web-scale blockchain for fast, secure, scalable, decentralized apps
Solana is the fastest blockchain in the world and the fastest-growing ecosystem in crypto, with thousands of projects spanning DeFi, NFTs, Web3 and more. Integrate once and never worry about scaling again. Solana ensures composability between ecosystem projects by maintaining a single global state as the network scales. Never deal with fragmented Layer 2 systems or sharded chains. Solana's scalability ensures transactions remain less than $0.01 for both developers and users. Solana is all... -
15
Password Pusher
An application to securely communicate passwords over the web
Give your users the tools to be secure by default. Password Pusher is an open source application to communicate passwords over the web. Links to passwords expire after a certain number of views and/or time has passed. Only enter a password into the box. Other identifying information can compromise security. All passwords are encrypted prior to storage and are available to only those with the secret link. Once expired, encrypted passwords are unequivocally deleted from the database. -
16
Highlight.js
JavaScript syntax highlighter with language auto-detection
Highlight.js is a syntax highlighter written in JavaScript. It works in the browser as well as on the server. It can work with pretty much any markup, doesn’t depend on any other frameworks, and has automatic language detection. Highlight.js supports over 180 languages in the core library. There are also 3rd party language definitions available to support even more languages. We strongly recommend <pre><code> wrapping for code blocks. It's quite semantic and "just works" out of the box with... -
17
Emscripten
Emscripten: An LLVM-to-WebAssembly Compiler
Emscripten is a complete open-source compiler toolchain that transforms C, C++, and other LLVM-based source code into WebAssembly (and JavaScript), enabling native‑like applications to run in web browsers, Node.js, and other Wasm environments. While Emscripten mostly focuses on compiling C and C++ using Clang, it can be integrated with other LLVM-using compilers (for example, Rust has Emscripten integration, with the wasm32-unknown-emscripten and asmjs-unknown-emscripten targets). ... -
18
UIkit
Lightweight modular front-end framework for developing web interfaces
A lightweight and modular front-end framework for developing fast and powerful web interfaces. UIkit will work on pretty much any modern browser. Using UIkit works best if you have a solid code editor, for example Sublime Text or Atom. To be even more efficient, we recommend that you install one of the autocomplete plugins for your favorite IDE or code editor. This saves a lot of time, as you won't have to look up and type all UIkit classes and markup. To use UIkit's CSS and JavaScript,... -
19
The Vibe Companion
Open-source Claude Code/Codex Web UI
The Companion project by The-Vibe-Company is an open-source web-based UI for interacting with Claude Code, offering a fully graphical interface to launch sessions, stream responses, and approve or deny AI tool usage—all via web browsers on desktop or mobile. Instead of working solely through a command-line interface, developers and creators can start, manage, and visualize Claude Code sessions in a modern web environment built with TypeScript, enabling more natural interaction with AI models. ... -
20
CosmicNvim
Lightweight and opinionated Neovim config for web development
CosmicNvim is a lightweight and opinionated Neovim config for web development, specifically designed to provide a COSMIC programming experience. While CosmicNvim is geared specifically toward TypeScript/JavaScript development, it should be able to provide a great experience with any LSP-supported language. -
21
RobotsTxt
The repository contains Google's robots.txt parser
This is a high-performance, production-tested library for parsing and evaluating robots.txt rules against crawler user agents. It implements the core semantics of the Robots Exclusion Protocol: user-agent sections, Allow/Disallow directives, wildcard handling, and precedence rules. The code is optimized for speed and low memory so large crawls can evaluate millions of URLs quickly. It also focuses on correctness—edge cases like overlapping patterns and longest-match resolution are handled... -
22
news-please
Python tool for crawling and extracting structured data from news site
news-please is an open source news crawler and information extraction tool designed to collect and structure articles from online news websites. It provides an integrated pipeline that crawls news sites, retrieves article pages, and extracts structured information such as headlines, authors, publication dates, and article text. news-please can recursively follow internal links and read RSS feeds to gather both recent and archived articles from a news outlet when given only the root URL of a site. ... -
23
Swagger Editor
An editor designed for Swagger
Swagger Editor lets you edit Swagger API specifications in YAML inside your browser and to preview documentations in real time. Valid Swagger JSON descriptions can then be generated and used with the full Swagger tooling (code generation, documentation, etc). swagger-editor is a traditional npm module intended for use in single-page applications that are capable of resolving dependencies (via Webpack, Browserify, etc). swagger-editor-dist is a dependency-free module that includes everything... -
24
Fragments by E2B
Open source template for AI-powered code generation apps w/ sandboxes
Fragments is an open source template designed for building applications where artificial intelligence generates and executes code directly from natural language prompts. It provides a ready-to-use web interface where users can chat with AI models to create code snippets, applications, or scripts in various programming environments. Generated code can be executed safely in isolated sandbox environments using the E2B SDK, which helps protect the host system while running AI-produced programs. ... -
25
ModSecurity
Cross platform web application firewall (WAF) engine for Apache
ModSecurity is an open-source, cross-platform web application firewall (WAF) engine for Apache, IIS and Nginx that is developed by Trustwave's SpiderLabs. It has a robust event-based programming language that provides protection from a range of attacks against web applications and allows for HTTP traffic monitoring, logging and real-time analysis. Libmodsecurity is one component of the ModSecurity v3 project. The library codebase serves as an interface to ModSecurity Connectors taking in web...
