Search Results for "extensible web spider"
Sort By:
Showing 259 open source projects for "extensible web spider"
View related business solutions-
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
1
Python-Spider
Python3 web crawler practice
...As part of the author’s public learning-path repositories, python-spider likely includes examples of HTTP requests, HTML parsing, maybe concurrency or scheduling to crawl multiple pages, and techniques to handle common web-scraping issues. For people wanting to get hands-on with building scrapers, collecting data, or learning how to navigate web programming in Python, this repository acts as a didactic reference or starting point. -
2
The Falcon Web Framework
The no-nonsense REST API and microservices framework
Falcon is a minimalist WSGI library for building speedy web APIs and app backends. We like to think of Falcon as the Dieter Rams of web frameworks. When it comes to building HTTP APIs, other frameworks weigh you down with tons of dependencies and unnecessary abstractions. Falcon cuts to the chase with a clean design that embraces HTTP and the REST architectural style. Highly optimized, extensible code base. -
3
Bot Framework Web Chat
A highly-customizable web-based client for Azure Bot Services
This repository contains code for the Bot Framework Web Chat component. The Bot Framework Web Chat component is a highly-customizable web-based client for the Bot Framework V4 SDK. The Bot Framework SDK v4 enables developers to model conversation and build sophisticated bot applications. This repo is part of the Microsoft Bot Framework, a comprehensive framework for building enterprise-grade conversational AI experiences. Create a bot with the ability to speak, listen, understand, and learn... -
4
Grab Framework Project
Web Scraping Framework
...The API is built on top of urllib3 and lxml libraries. The Spider API to build asynchronous web crawlers. You write classes that define handlers for each type of network request. Each handler is able to spawn new network requests. Network requests are processed concurrently with a pool of asynchronous web sockets. Grab provides interface called Spider to develop multithreaded web-site scrapers. -
AI-powered service management for IT and enterprise teamsGive your IT, operations, and business teams the ability to deliver exceptional services—without the complexity. Maximize operational efficiency with refreshingly simple, AI-powered Freshservice.
-
5
Node Crawler
Web Crawler/Spider for NodeJS + server-side jQuery
Most powerful, popular and production crawling/scraping package for Node, happy hacking. -
6
Next.js
The React Framework
...It is fully extensible and ready for production. It’s no wonder Next.js is used in tens of thousands of production-facing websites and web applications from some of the world’s biggest brands. -
7
Eclipse GLSP
Graphical language server platform for building web-based diagram
The Graphical Language Server Platform (GLSP) is an extensible open-source framework for building custom diagram editors based on web technologies. Alongside an extensible client framework and a server framework, GLSP provides a language server protocol (LSP) for diagrams. With that, GLSP enables the development of modern, web-based diagram editors, whereas the heavy lifting, such as loading, interpreting, and editing according to the rules of the modeling language, is encapsulated in the server. ... -
8
Flask
The Python micro framework for building web applications
Flask is a lightweight WSGI web application framework designed to help developers get started with their web applications quickly and easily with the ability to scale up to complex applications. Being a “micro” framework does not mean that your whole web application must fit into a single Python file (although it can) or that it be limited; rather it means that Flask aims to keep the core simple but extensible. -
9
Heritrix
Internet Archive's open-source, web-scale, web crawler project
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or misspelled or missaid as heratrix/heritix/heretix/heratix) is an archaic word for heiress (woman who inherits). Since our crawler seeks to collect and preserve the digital artifacts of our culture for the benefit of future researchers and generations, this name seemed apt. -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
Ash
A declarative, extensible framework for building Elixir applications
Ash is a declarative framework for building resource-oriented apps in Elixir. It emphasizes composability, DSL-driven definitions of resources/actions/relationships, and extensibility through plugins for API, database, and UI layers. -
11
WasmEdge
Extensible WebAssembly runtime for cloud native applications
WasmEdge is a lightweight, high-performance, and extensible WebAssembly runtime. It is the fastest Wasm VM today. WasmEdge is an official sandbox project hosted by the CNCF. Its use cases include modern web application architectures (Isomorphic & Jamstack applications), microservices on the edge cloud, serverless SaaS APIs, embedded functions, smart contracts, and smart devices. The WasmEdge Runtime provides a well-defined execution sandbox for its contained WebAssembly bytecode program. ... -
12
Jenkins
Build great things at any scale
Jenkins is the leading open-source automation server that allows you to build great things at any scale. Jenkins is built with Java and provides hundreds of plugins for building, deploying and automating virtually anything, allowing you to focus on more important things. Jenkins is often used for building projects, running tests, analyzing static code and deployment. Whatever is done repetitively, Jenkins can most likely execute and execute well, saving you time and optimizing your... -
13
Theia
Cloud & Desktop IDE Platform
Eclipse Theia is an open source extensible platform for developing multi-language Cloud and Desktop IDEs with state-of-the-art web technologies. If you’re not sure if you need a web version or a desktop version or both, Theia is the best solution. It can develop one IDE for both browsers and native desktop applications from a single source. Theia has a highly flexible architecture, allowing extenders and adopters to customize and extend every aspect of it. ... -
14
IntelliJ Community Edition
IntelliJ IDEA & IntelliJ Platform
IntelliJ Community is the open source upstream of JetBrains’ IntelliJ IDEA, forming the core of a powerful, extensible, and intelligent development environment. It provides foundational features like a robust editor with code completion, syntax highlighting, refactoring tools, version control integrations, terminal, debugger, and plugin architecture. Since it’s open, community developers can contribute to language supports, UI tweaks, and platform enhancements. -
15
ZAP
The OWASP ZAP core project
The OWASP Zed Attack Proxy (ZAP) is one of the world’s most popular free security tools and is actively maintained by a dedicated international team of volunteers. It can help you automatically find security vulnerabilities in your web applications while you are developing and testing your applications. It's also a great tool for experienced pentesters to use for manual security testing. ZAP is an easy to use integrated penetration testing tool for finding vulnerabilities in web... -
16
Mako
An extremely fast, production-grade web bundler based on Rust
Mako is a new web bundler for web apps, libraries, and frameworks. It's designed to be fast, reliable, and easy to use. It has been used in hundreds of projects in production by Ant Group, and other companies. If you are looking for a modern web bundler, Mako is the right choice. -
17
Aquameta
Web development platform built entirely in PostgreSQL
Aquameta is a web development platform where everything—including code, content, and logic—is stored and edited in PostgreSQL. It treats the database as the source of truth, enabling fully modular, SQL-driven app development. Aquameta is ideal for developers seeking an unconventional but unified data-centric development experience. -
18
Logbook
An extensible Java library for HTTP request and response logging
Logbook is an extensible Java library to enable complete request and response logging for different client- and server-side technologies. It satisfies a special need by a) allowing web application developers to log any HTTP traffic that an application receives or sends b) in a way that makes it easy to persist and analyze it later. This can be useful for traditional log analysis, meeting audit requirements or investigating individual historic traffic issues. -
19
Flight Core
An extensible micro-framework for PHP
FlightPHP is a lightweight, fast, and flexible micro-framework for PHP, designed to build RESTful web applications and APIs. It provides a simple routing system, middleware support, and a powerful templating engine. FlightPHP is ideal for developers looking for a minimalist framework that doesn't impose a lot of structure, while still offering essential features for building modern web applications. -
20
Prism
Lightweight, robust, elegant syntax highlighting
Prism is a lightweight, extensible syntax highlighter, built with modern web standards in mind. It’s used in millions of websites, including some of those you visit daily. -
21
Django
The Web framework for perfectionists with deadlines
Django is a high-level, free and open-source Python web framework founded on the Model–Template–View (MTV) pattern, designed to facilitate rapid development of secure, maintainable, and scalable database-driven websites. First, read docs/intro/install.txt for instructions on installing Django. Next, work through the tutorials in order (docs/intro/tutorial01.txt, docs/intro/tutorial02.txt, etc.). If you want to set up an actual deployment server, read docs/howto/deployment/index.txt for... -
22
Monaco Editor
A browser based code editor
Monaco Editor is the rich, browser-based code editor that powers Visual Studio Code, providing advanced editing capabilities as a standalone embeddable library for web applications. Models are at the heart of Monaco editor. It's what you interact with when managing content. A model represents a file that has been opened. This could represent a file that exists on a file system, but it doesn't have to. For example, the model holds the text content, determines the language of the content, and... -
23
Strapi
API creation made simple, secure and fast
Strapi is the most advanced open-source headless CMS for creating powerful and customizable APIs with no effort. Built with 100% JavaScript, Strapi lets you easily create self-hosted, customizable, and performant content APIs. Strapi projects can be hosted on any platform of your choice, and you can work with any database you prefer. All your favorite dev tools-- from static site generators and databases to hosting platforms work with Strapi, so you're never locked in. Strapi is designed... -
24
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring... -
25
Behat
BDD in PHP
Behat is a Behavior-Driven Development (BDD) framework for PHP that helps developers write tests in a human-readable format. It uses Gherkin syntax to describe expected application behavior and allows developers to write scenarios that map to automated tests. Behat is highly extensible, making it suitable for testing both web applications and APIs, and it is often used alongside Mink for browser automation.