Search Results for "python web crawler"

A free in-depth website analyzer providing audits of security, performance, SEO, accessibility and other technical aspects. Available as a desktop application for Windows/macOS/Linux and as a CLI tool for advanced users and CI/CD processes. It also includes an offline web page exporter (website clone, mirror).

cheat.sh is a compact, network-accessible cheat-sheet service that serves concise examples and usage notes for hundreds of shell commands, programming languages, and tools via a simple HTTP interface. You can query it from the terminal (for example curl cht.sh/rsync or curl cheat.sh/ls) or browse the web front page; it also supports a shorthand hostname (cht.sh) and provides both online and standalone/local installation modes. The repository contains the server and client code, instructions to run a local standalone instance (including Python virtualenv setup), and tooling to fetch or maintain the upstream cheat-sheet data; installation documentation explains disk-space needs and dependency setup for offline use. ...

posting is a lightweight command-line tool that lets users schedule and automate Mastodon posts using Markdown files. It reads a simple folder structure of Markdown drafts and posts them at predefined intervals or manually. Designed for content creators and developers, posting helps maintain consistent and organized Mastodon accounts without depending on web UIs or third-party schedulers.

The AWS CLI is the universal command-line interface for managing AWS services, automating tasks, and scripting cloud workflows. It exposes nearly every public API from EC2 and S3 to IAM, Lambda, and beyond, providing parity with the service SDKs in a tool you can run anywhere. Profiles, regions, single-sign-on, and credential helpers make it straightforward to switch contexts securely across accounts and environments. Its output controls and JMESPath querying let you slice, filter, and...

HTTPie (pronounced aitch-tee-tee-pie) is a command-line HTTP client. Its goal is to make CLI interaction with web services as human-friendly as possible. HTTPie is designed for testing, debugging, and generally interacting with APIs & HTTP servers. The HTTP & HTTPS commands allow for creating and sending arbitrary HTTP requests. They use simple and natural syntax and provide formatted and colorized output.

Open-source tool to access Amazon S3 file storage. S3cmd is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage. Lots of features and options have been added to s3cmd since its very first release in 2008.... we recently counted more than 60 command line options, including multipart uploads, encryption, incremental backup, s3 sync, ACL and Metadata...

An advanced command-line tool designed to brute force directories and files in webservers, AKA web path scanner. Wordlist is a text file, each line is a path. About extensions, unlike other tools, dirsearch only replaces the %EXT% keyword with extensions from -e flag. For wordlists without %EXT% (like SecLists), -f | --force-extensions switch is required to append extensions to every word in wordlist, as well as the /. To use multiple wordlists, you can separate your wordlists with commas....

Running Docker on macOS and Windows is supported via VirtualBox as well as Docker for Mac / Docker for Windows. Docksal project stacks are pre-loaded with common development tools, which include Composer, PHP Code Sniffer, php-cli, node, npm, ruby, bundler, and python. For PHP, Drupal, and WordPress development, it comes with Drush, Drupal Console, and WP-CLI. There is built-in support for Apache Solr, Varnish, Memcache, Selenium, and Behat. Since services are containerized with...

SIGAR (System Information Gatherer and Reporter) is a cross-platform, cross-language library and command-line tool for accessing operating system and hardware level information in Java, Perl and .NET.

Nanon is a web page manager, ideal for simple sites with no server generated content. It lets you define templates and reusable html code blocks like menus and ad sections. Nanon is a spiffy command line tool that you can learn how to use in minutes.

PyUserCreate is a tool for *NIX OSes that allows you to accept user account requests from a web interface, and use a simple command-line tool to approve/disprove requests.

Reddo is a command line tool to translate any pair of languages using web translators such google, babelfish or others.