Search Results for "shell web"
Sort By:
Showing 147 open source projects for "shell web"
View related business solutions-
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
1
Web Scraping for Laravel
Laravel adapter for Roach, the complete web scraping toolkit for PHP
...Roach ships with an interactive shell (often called Read-Evaluate-Print-Loop, or Repl for short) which makes prototyping our spiders a breeze. We can use the provided roach:shell command to launch a new Repl session. -
2
bilibili-manga-downloader
Download and manage Bilibili Manga chapters with GUI downloader
BiliBili-Manga-Downloader is an open source desktop application designed to download manga chapters from the Bilibili Manga platform for offline reading and local management. It was created to address limitations of the web reading experience, such as intrusive advertisements, inconvenient image zooming, and inconsistent navigation during reading sessions. It provides a graphical user interface that allows users to search for manga titles using keywords, view detailed information about... -
3
fess
Open source enterprise search server for websites, files, and data
Fess is an open source enterprise search server designed to provide powerful full-text search capabilities across multiple data sources. It enables organizations to quickly deploy a scalable search environment without requiring deep knowledge of underlying search technologies. Fess is built on top of OpenSearch and offers an integrated solution for crawling, indexing, and searching documents from websites, file systems, and various data stores. Fess includes a built-in crawler that can... -
4
DotnetSpider
Lightweight .NET framework for fast web crawling and data scraping
DotnetSpider is a web crawling and data extraction framework built on the .NET Standard platform. It is designed to help developers create efficient and scalable crawlers for collecting structured data from websites. It provides a high-level API that simplifies the process of defining spiders, managing requests, and extracting content from web pages. Developers can create custom spiders by extending base classes and configuring pipelines that handle downloading, parsing, and storing... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
single-file-cli
CLI tool to save complete web pages as single self-contained HTML file
SingleFile CLI is an open source command-line tool designed to save complete web pages as a single self-contained HTML file. It captures the rendered page in a headless browser and embeds all required resources directly into the output document, including stylesheets, scripts, images, and fonts. By consolidating every dependency into one file, it allows users to preserve a faithful copy of a web page that can be viewed offline without requiring external assets. SingleFile CLI works by... -
6
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring... -
7
kimuraframework
AI-first Ruby framework for building fast, flexible web scraping spide
Kimurai is an open source web scraping framework written in Ruby that simplifies the process of building automated data extraction tools. It provides a clean domain-specific language that allows developers to define scraping logic and data schemas with minimal boilerplate code. Kimurai can use AI-assisted extraction to identify where data resides in HTML pages, automatically generating selectors that are cached for future use so subsequent scraping runs operate with pure Ruby performance.... -
8
watercrawl
AI-ready web crawler that extracts and structures website content
WaterCrawl is an open source web crawling and data extraction platform designed to transform website content into structured data suitable for machine learning and AI workflows. It enables developers and researchers to crawl web pages, extract meaningful information, and convert it into formats that are easier to process and analyze. It provides a modern crawling system that can automatically navigate links, control crawl depth, and collect content from targeted sections of a website.... -
9
newspaper4k
Python library for scraping and analyzing online news articles easily
Newspaper4k is a Python library designed for extracting, processing, and analyzing news articles from websites. It is a continuation and active fork of the original newspaper3k library, which had stopped receiving updates, with the goal of keeping the ecosystem maintained while adding improvements and bug fixes. It provides developers with tools to automatically download web pages, extract the main article content, and collect associated metadata such as titles, authors, images, and... -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
10
Scrapling
An adaptive Web Scraping framework
Scrapling is an adaptive web scraping framework designed to handle everything from a single HTTP request to large-scale, concurrent crawls. Built for modern websites, it intelligently adapts to structural changes by automatically relocating elements when page layouts update. The framework includes advanced fetchers capable of bypassing anti-bot protections such as Cloudflare Turnstile using stealth and browser automation techniques. Its powerful spider system supports multi-session crawling,... -
11
Prowler
An open source security tool to perform AWS security assessment
Prowler is an Open Source security tool to perform AWS security best practices assessments, audits, incident response, continuous monitoring, hardening, and forensics readiness. It contains more than 200 controls covering CIS, PCI-DSS, ISO27001, GDPR, HIPAA, FFIEC, SOC2, AWS FTR, ENS and custom security frameworks. Prowler is a command-line tool that helps you with AWS security assessment, auditing, hardening, and incident response. It follows guidelines of the CIS Amazon Web Services... -
12
pandora-box
Lightweight cross-platform desktop client for managing Mihomo proxies
Pandora-Box is a lightweight desktop client designed to provide a graphical interface for the Mihomo proxy core. It allows users to manage proxy configurations and subscriptions through a simple and user-friendly interface rather than working directly with configuration files. Pandora-Box supports multiple proxy protocols and provides tools to organize and control network routing rules. It is designed to work for both casual users who want an easy setup and advanced users who need more... -
13
dude uncomplicated data extraction
dude uncomplicated data extraction: A simple framework
Dude is a very simple framework for writing web scrapers using Python decorators. The design, inspired by Flask, was to easily build a web scraper in just a few lines of code. Dude has an easy-to-learn syntax. Dude is currently in Pre-Alpha. Please expect breaking changes. You can run your scraper from terminal/shell/command-line by supplying URLs, the output filename of your choice and the paths to your python scripts to dude scrape command. -
14
CacheGuard Gateway
CacheGuard Gateway is a UTM, a WAF, and a QoS management appliance.
CacheGuard Gateway is a free and open-source Unified Threat Management (UTM) solution, a Web Application Firewall (WAF), and a Quality of Service (QoS) platform designed to optimize WAN traffic. To obtain a CacheGuard Gateway appliance, download CacheGuard-OS and install it on the bare-metal or virtual machine of your choice. It’s that simple and completely free. The UTM includes a firewall, web antivirus, VPN server, and a URL-filtering and SSL-inspection web proxy. The WAF operates in... -
15
twitch-batch-downloader
Automate the download of entire Twitch.tv channels
Automate the download of entire Twitch.tv channels with its metadata. Save each Twitch video into its own folder, with date and time values, video ID, stream metadata, frame screenshot, .ts parts list and sha256 hash. Keep the original ts files and generate mp4 files from them. It requires a shell and some command line utilities. See README.md for details in the Code/git section. -
16
uweb browser: unlimited power
minimal suckless android web browser with unlimited power
... - user-defined site-specific JS/CSS/HTML/preprocessing. - Online play/preview/preprocess for downloadable resources. - Multiple type profiles: switch any data including logins/config orthogonally - web automation, crontab (alarm clock) -
17
X-Itools: Email/Web Log Search Engine
Strong Email & Apache Log Analysis with Active Security Features
X-Itools: eXtended Internet Tools. Suite of tools composed of several collaboration modules. Old and initial project born in 1999, 1st published in 2001 on Sourceforge. X-Itools E-mail management module (log analysis) initiated in 2004 with Web 1.0 technologies (private SVN server). X-Itools development restarted since 2011, on the basis of a unique module: E-mail management module (log analysis). Now based on web 2.0 technologies (ExtJS 4.1) and devel restarted because of a particular... -
18
The NCID (Network Caller ID) project is Caller ID (CID) distributed over a network. The project contains the NCID package and 4 optional client packages. Each package is described at the NCID web site. A non-inclusive list of 3rd party addons is also available at the web site Available Packages: NCID - contains the server, gateways, and a client with output modules LCDncid - a client that uses LCDproc to display Caller ID on a LCD display NCIDandroid - a client and gateway for...
-
19
Foswiki is an Enterprise wiki, typically used to run a collaboration platform, knowledge base or document management system. Users can create wiki applications using the Topic Markup Language, and developers can extend its functionality with plugins. Foswiki stands for "Free and Open Source" wiki to emphasize its commitment to Open Source software.
-
20
This project features a basic php/mysql content management and a public customizable interface. (CakePHP features) multi-user access, e-shop profiles, CRUD posting.
-
21
Buku
Powerful command-line bookmark manager. Your mini web!
...Refer to the section on GUI integration. If you prefer the terminal, thanks to the shell completion scripts, you don't need to memorize any of the options. There's an Easter egg to revisit random forgotten bookmarks too. Buku is too busy to track you - no history, obsolete records, usage analytics or homing. Demo: https://asciinema.org/a/8pm3q3n5s95tvat8naam68ejv -
22
Magnolia CMS
Java CMS that delivers smartphone simplicity on an enterprise scale
Magnolia is an open Java CMS that delivers smartphone simplicity on an enterprise-scale. The mobile-inspired HTML5 user interface works through customisable, task-focused Apps, a notification stream called the Pulse, and a system of Favorites for quickly accessing the workspace. Magnolia CMS allows organizations to orchestrate online services, sales and marketing across all digital channels, maximizing the impact of every touchpoint. Magnolia’s open-source technology, support of open... -
23
CIDRAM
CIDRAM: Classless Inter-Domain Routing Access Manager.
CIDRAM (Classless Inter-Domain Routing Access Manager) is a PHP script designed to protect websites by blocking requests originating from IP addresses regarded as being sources of undesirable traffic, including (but not limited to) traffic from non-human access endpoints, cloud services, spambots, scrapers, etc. It does this by calculating the possible CIDRs of the IP addresses supplied from inbound requests and then attempting to match these possible CIDRs against its signature files (these... -
24
HackTools
The all-in-one Red Team extension for Web Pentesters
The all-in-one Red Team browser extension for Web Pentesters. HackTools, is a web extension facilitating your web application penetration tests, it includes cheat sheets as well as all the tools used during a test such as XSS payloads, Reverse shells and much more. With the extension you no longer need to search for payloads in different websites or in your local storage space, most of the tools are accessible in one click. HackTools is accessible either in pop-up mode or in a whole tab in... -
25
nxshell
Next Shell
NxShell is a modern, Electron-based terminal emulator that works on all major operating systems out there: Windows, macOS, and Linux.
