Search Results for "web crawler source code" - Page 6
Sort By:
Showing 3462 open source projects for "web crawler source code"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
1
Theia
Cloud & Desktop IDE Platform
...It supports VS code extensions, and is completely open source and vendor neutral. -
2
Cutelyst
Web Framework built on top of Qt, using the approach of Catalyst
Create fast and modern web applications that share the same code as your desktop or mobile apps. Why have two or more development teams focused on a specific platform when you can have a single team coding the same code, sharing skills and lines of code. With Qt, you already can write applications for Windows, Linux, Android, MacOS, and iOS using the same source code, Cutelyst allows you to share your C++ code with your web application. ... -
3
PostgREST
REST API for any Postgres database
...Object-relational mapping is a leaky abstraction leading to slow imperative code. The PostgREST philosophy establishes a single declarative source of truth: the data itself. It’s easier to ask PostgreSQL to join data for you and let its query planner figure out the details than to loop through rows yourself. It’s easier to assign permissions to db objects than to add guards in controllers. (This is especially true for cascading permissions in data dependencies.) -
4
eslint-plugin-compat
Check the browser compatibility of your code
Lint the browser compatibility of your code. Browser targets are configured using a browser list. You can configure browser targets in your package.json. If no configuration is found, browser list defaults to "> 0.5%, last 2 versions, Firefox ESR, not dead". Add polyfills to the settings section of your eslint config. Append the name of the object and the property if one exists. Toolchains for native platforms, like iOS and Android, have had API linting from the start. It's about time that... -
Earn up to 16% annual interest with Nexo.Generate interest, access liquidity without selling, and execute trades seamlessly. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
5
gruntwork.io website
The gruntwork.io website
The Gruntwork team deploys a Reference Architecture for you, setting you up with all the foundational pieces you need to get started with AWS and Terraform, including your Landing Zone, network topology, CI / CD pipeline, guard rails, monitoring, out-of-the-box compliance, and more. Use the Gruntwork Infrastructure as Code Library (IaC Library), a collection of reusable Terraform code that has been proven in production at hundreds of companies, to deploy the rest of your infrastructure,... -
6
Eclipse Open VSX
An open-source registry for VS Code extensions
Open VSX is a vendor-neutral open-source alternative to the Visual Studio Marketplace. It provides a server application that manages VS Code extensions in a database, a web application similar to the VS Code Marketplace, and a command-line tool for publishing extensions similar to vsce. The default frontend is the one bundled in the Docker image, and is also used for testing in the development environment. -
7
Swagger Editor
An editor designed for Swagger
Swagger Editor lets you edit Swagger API specifications in YAML inside your browser and to preview documentations in real time. Valid Swagger JSON descriptions can then be generated and used with the full Swagger tooling (code generation, documentation, etc). swagger-editor is a traditional npm module intended for use in single-page applications that are capable of resolving dependencies (via Webpack, Browserify, etc). swagger-editor-dist is a dependency-free module that includes everything... -
8
news-please
Python tool for crawling and extracting structured data from news site
news-please is an open source news crawler and information extraction tool designed to collect and structure articles from online news websites. It provides an integrated pipeline that crawls news sites, retrieves article pages, and extracts structured information such as headlines, authors, publication dates, and article text. news-please can recursively follow internal links and read RSS feeds to gather both recent and archived articles from a news outlet when given only the root URL of a site. ... -
9
Unsloth Studio
Unified web UI for training and running open models locally
Unsloth Studio is a web-based interface for running and training AI models locally with a unified and user-friendly experience. It allows users to work with a wide range of models for text, audio, vision, embeddings, and more without relying heavily on cloud infrastructure. Built on top of the Unsloth framework, it focuses on high-performance training with reduced VRAM usage and faster speeds compared to traditional methods. The platform supports fine-tuning, pretraining, and reinforcement... -
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
10
WAFW00F
WAFW00F allows one to identify and fingerprint Web App Firewall
...For further details, check out the source code on our main repository. -
11
Webgrind
Xdebug Profiling Web Frontend in PHP
Webgrind is an Xdebug profiling web frontend for PHP, offering a lightweight and easy-to-use interface to visualize profiling data. It helps developers identify performance bottlenecks in their applications by displaying function call statistics, memory usage, and execution time. Webgrind is ideal for performance analysis and optimization of PHP code, providing insights into code efficiency. -
12
Groq AppGen
Project showcasing Llama 3.3 70B HTML codegen abilities
Groq AppGen is an interactive web application (built with Next.js and TypeScript) that uses Groq’s LLM API to generate or modify web application code based on natural-language prompts. Essentially, you tell the app what kind of web app or page you want (in plain English), and groq-appgen will produce HTML/JSX code scaffolding, layout, and optionally application logic accordingly. It supports iterative feedback: you can refine your prompt, adjust parameters or requirements, and have the app... -
13
Laravel Framework
Elegant PHP web application framework with expressive syntax
Laravel is a popular open-source PHP web framework designed for building modern web applications with an elegant and readable syntax. It emphasizes developer productivity, offering features like routing, Eloquent ORM, blade templating, and an extensive ecosystem of packages. Laravel makes common tasks like authentication, caching, and session management simple and intuitive, making it a top choice for PHP developers. -
14
FastAPI Python
FastAPI framework, high performance, easy to learn, fast to code
FastAPI framework, high performance, easy to learn, fast to code, ready for production. FastAPI is a modern, fast (high-performance), web framework for building APIs with Python based on standard Python type hints. -
15
Fable
F# to JavaScript compiler
Type inference provides robustness and correctness, but without the cost of additional code. Let the compiler catch bugs for you. Fable produces readable JavaScript code compatible with ES2015 standards and popular tooling like Webpack. Call JavaScript from Fable or Fable from JS. Use NPM packages. The entire JavaScript ecosystem is at your fingertips. Choose your favorite tool, from Visual Studio Code to JetBrains Rider. Fable supports the F# core library and some common .NET libraries to... -
16
Klotho
Write AWS applications at lightning speed
Klotho is an open source tool that transforms plain code into cloud native code. Klotho allows you to quickly and reliably add cloud functionality to your application with minimal modification to your code. In most cases, this is just a handful of klotho annotations. Klotho is designed to absorb the complexity of building cloud applications, enabling everyone in large-scale organizations and teams to hobbyist developers to write and operate cloud applications at a fraction of the effort. Its... -
17
Eclipse GLSP
Graphical language server platform for building web-based diagram
...GLSP provides integration layers, to use GLSP editors seamlessly in web pages, Eclipse Theia, VS Code and even Eclipse desktop. GLSP provides the perfect basis for the efficient development of web-based diagram editors including edit functionality, layouting, shapes, palettes and everything else you expect from a powerful, modern diagram editor. GLSP doesn't hide the underlying rendering technologies, such as Eclipse Sprotty, SVG, and CSS. -
18
RobotsTxt
The repository contains Google's robots.txt parser
This is a high-performance, production-tested library for parsing and evaluating robots.txt rules against crawler user agents. It implements the core semantics of the Robots Exclusion Protocol: user-agent sections, Allow/Disallow directives, wildcard handling, and precedence rules. The code is optimized for speed and low memory so large crawls can evaluate millions of URLs quickly. It also focuses on correctness—edge cases like overlapping patterns and longest-match resolution are handled... -
19
GPT All Star
AI-powered code generation tool for scratch development of web apps
AI-powered code generation tool for scratch development of web applications with a team collaboration of autonomous AI agents. This is a research project, and its primary value is to explore the possibility of autonomous AI agents. -
20
Laravel Pint
Laravel Pint is an opinionated PHP code style fixer for minimalists.
Laravel Pint is an opinionated PHP code-style fixer for minimalists. Pint is built on top of PHP-CS-Fixer and makes it simple to ensure that your code style stays clean and consistent. Pint is automatically installed with all new Laravel applications so you may start using it immediately. By default, Pint does not require any configuration and will fix code style issues in your code by following the opinionated coding style of Laravel. Pint is included in recent releases of the Laravel... -
21
Claude Run
A beautiful web UI for browsing Claude Code conversation history
Claude Run is a user-friendly web interface for browsing, searching, and managing Claude Code conversation history in a visually appealing way, making it easier to revisit past interactions, track project progress, and resume long-running coding sessions. Running locally via a single command (npx claude-run), it launches a web UI that displays all stored Claude Code sessions – including conversation threads, tool calls, and timestamps – in a clean chat layout, helping developers quickly find... -
22
SBCL
Mirror of Steel Bank Common Lisp (SBCL)'s repository
Steel Bank Common Lisp (SBCL) is a high-performance Common Lisp compiler. It is open-source/free software, with a permissive license. In addition to the compiler and runtime system for ANSI Common Lisp, it provides an interactive environment including a debugger, a statistical profiler, a code coverage tool, and many other extensions. SBCL runs on Linux, various BSDs, macOS, Solaris, and Windows. See the download page for supported platforms, and the getting started guide for additional... -
23
SikuliX
SikuliX version 2.0.0+ (2019+)
...It uses image recognition powered by OpenCV to identify GUI components and can act on them with mouse and keyboard actions. This is handy in cases when there is no easy access to a GUI's internals or the source code of the application or web page you want to act on. -
24
Reflex
Interactive programs without callbacks or side-effects
Reflex apps automatically react to changing data. This keeps every interaction current, accurately representing the relationship between your data and the real world. Reflex components are modular and reusable. If your requirements change, your app can quickly and easily be reworked. The modularity of Reflex lets you iterate quickly, without wasting code. Reflex has been built to seamlessly support interfaces on desktop, mobile, web, and other platforms, all in Haskell. Regardless of your... -
25
phpMyAdmin
A web interface for MySQL and MariaDB
phpMyAdmin is a free software tool written in PHP, intended to handle the administration of MySQL over the Web. phpMyAdmin supports a wide range of operations on MySQL and MariaDB. Frequently used operations (managing databases, tables, columns, relations, indexes, users, permissions, etc) can be performed via the user interface, while you still have the ability to directly execute any SQL statement. phpMyAdmin comes with a wide range of documentation and users are welcome to update our wiki...
