Hakrawler
Fast Go web crawler for discovering URLs and web app endpoints
...It accepts URLs through standard input, making it easy to integrate into command-line pipelines with other security tools. This workflow enables researchers to combine it with subdomain enumeration, HTTP probing, and vulnerability scanning utilities to automate reconnaissance processes. hakrawler can follow links within a website and optionally include subdomains.