Search Results for "sitemaps"
Sort By:
Showing 41 open source projects for "sitemaps"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
ClouDNS provides fast and secure Free DNS Hosting and Premium DNS Hosting with Global Anycast DNS Network.ClouDNS is the biggest European provider of globally Managed DNS services, including GeoDNS, Anycast DNS and DDoS protected DNS. We also offer Domain names, Monitoring services, SSL Certificates, and Google Workspace.
-
1
crwlr
Library for Rapid (Web) Crawler and Scraper Development
This library provides kind of a framework and a lot of ready-to-use, so-called steps, that you can use as building blocks, to build your own crawlers and scrapers with. Before diving into the library, let's have a look at the terms crawling and scraping. For most real-world use cases, those two things go hand in hand, which is why this library helps with and combines both. A (web) crawler is a program that (down)loads documents and follows the links in it to load them as well. A crawler... -
2
-
3
Search1API MCP
A Model Context Protocol (MCP) server
The Search1API MCP Server is a Model Context Protocol server that provides search and crawl functionality using Search1API. It enables web and news searches, content extraction, and sitemap retrieval, integrating seamlessly with MCP clients. -
4
openHAB client for Android
openHAB client for Android
This app is a native client for openHAB which allows easy access to your sitemaps. Control your openHAB server and/or openHAB Cloud instance, e.g., an account with myopenHAB. -
Run your private office with the ONLYOFFICEA Comprehensive Alternative to Office 365 for Business
-
5
crawley
The unix-way web crawler
...Found URLs are streamed to stdout and guaranteed to be unique (with fragments omitted) Scan depth (limited by starting host and path, by default - 0) can be configured. Can crawl rules and sitemaps from robots.txt. Brute mode - scan HTML comments for URLs (this can lead to bogus results) Make use of HTTP_PROXY / HTTPS_PROXY environment values + handle proxy auth. Directory-only scan mode (aka fast-scan) -
6
SiteOne Crawler
SiteOne Crawler is a website analyzer and exporter
...It will crawl your entire website in depth, analyze and report problems, show useful statistics and reports, generate an offline version of the website, generate sitemaps, or send reports via email. Watch a detailed video with a sample report for Astro. build website. This crawler can be used as a command-line tool (see releases and video), or you can use a multi-platform desktop application with a graphical interface (see a video about the app). -
7
RobotsTxt
The repository contains Google's robots.txt parser
...It also focuses on correctness—edge cases like overlapping patterns and longest-match resolution are handled consistently. Consumers integrate it to decide whether a specific URL may be fetched by a particular bot name and to respect crawl-delay or sitemaps hints where applicable. The library serves both search-scale crawlers and smaller tools that need a reliable decision engine for polite crawling. -
8
htmly
Simple and fast databaseless PHP blogging platform, and Flat-File CMS
...With 512MB of RAM or even in shared hosting, it should handle more than 20K posts without any issues. Simple but not simpler. A lot of built-in features to supercharge your blog. Recent posts, popular posts, archive lists, category lists, tag clouds, sitemaps, menu manager, scheduled posts, and many more. Ever heard of complaints about unsafe, outdated, and slow plugins for specific CMS out there? We will not follow the same steps. Any new features will be added to the core and go through a security audit. -
9
Laravel Sitemap
Create and generate sitemaps with ease
This package can generate a sitemap without you having to add urls to it manually. This works by crawling your entire site. The generator has the ability to execute JavaScript on each page so links injected into the dom by JavaScript will be crawled as well. The easiest way is to crawl the given domain and generate a sitemap with all found links. The destination of the sitemap should be specified by $path. If you don't want a crawled link to appear in the sitemap, just don't return it in the... -
Optimize every aspect of hiring with Greenhouse RecruitingWhat’s next for many of us is changing. Your company’s ability to hire great talent is as important as ever – so you’ll be ready for whatever’s ahead. Whether you need to scale your team quickly or improve your hiring process, Greenhouse gives you the right technology, know-how and support to take on what’s next.
-
10
Middleman
Hand-crafted frontend development
...A built-in dev server with live reload keeps iteration fast, while an asset pipeline handles fingerprinting, minification, and cache-friendly builds. Extensions cover blogging, internationalization, sitemaps, data-driven pages, and external build steps, so complex sites remain manageable without server-side code. Because output is plain files, Middleman sites deploy anywhere—from object storage and CDNs to GitHub Pages—benefiting from high performance and low attack surface. Teams use it for marketing sites, docs, microsites, and prototypes where content and design matter more than dynamic backends. -
11
Magnolia CMS
Java CMS that delivers smartphone simplicity on an enterprise scale
Magnolia is an open Java CMS that delivers smartphone simplicity on an enterprise-scale. The mobile-inspired HTML5 user interface works through customisable, task-focused Apps, a notification stream called the Pulse, and a system of Favorites for quickly accessing the workspace. Magnolia CMS allows organizations to orchestrate online services, sales and marketing across all digital channels, maximizing the impact of every touchpoint. Magnolia’s open-source technology, support of open... -
12
ACHE Focused Crawler
ACHE is a web crawler for domain-specific search
ACHE is a focused web crawler. It collects web pages that satisfy some specific criteria, e.g., pages that belong to a given domain or that contain a user-specified pattern. ACHE differs from generic crawlers in sense that it uses page classifiers to distinguish between relevant and irrelevant pages in a given domain. A page classifier can be defined as a simple regular expression (e.g., that matches every page that contains a specific word) or a machine-learning-based classification model.... -
13
Simple Sitemap Creator
A lightweight cross platform sitemap generator
A simple app that crawls a website and creates HTML and Google Sitemaps compatible XML sitemap. -
14
phpRegister Project
Account management PHP Script with a software infracstructure
php login and user management script with an admin part to manage accounts, tickets, translations. Do you have a new web project to develop? By using phpRegister, you will have directly all the essential elements for the development of your platform: Account creation, identification, profile management, user support, contact page and even a license template! phpRegister is also a software infrastructure written in PHP with as main purpose to help developers to start a new web... -
15
SEO MACROSCOPE
SEO Macroscope is a website scanning tool, to check your website
...Perform simple, configurable Technical SEO checks on titles and descriptions. Report fastest/slowest pages. Export reports to Excel and CSV formats. Generate and export text and XML sitemaps from the crawled pages. Analyze redirect chains. Use custom filters to verify the presence/absence of tracking tags. Use CSS Selectors, XPath Queries, and Regular Expressions to scrape website data. -
16
angora
Tool to view permissions in .NET sitemaps
This allows you to view a sitemaps roles and editRoles in a .NET sitemap file. It will keep track of all the sitemaps you have added. -
17
AD Auto index
The Free & Ultimate Solution for Mobile Download Site
AD Auto Index is a reborn of Master AutoIndex. A mobile focused autoindex script that allows users to create a mobile download site. It has plugin system built-in that will take it to it's ultimate level. -
18
-
19
Mget
Wget-like parallel website downloader and scanner
Very fast Wget-like website downloader with lots of modern features: - low CPU usage - Perfect Forward Secrecy (PFS) - TCP Fast Open - gzip, bzip2 and lzma decoding - IDNA 2008 - multiple proxy support - Metalink support (RFC 6249 & 5854) - scanning of zipped & unzipped sitemaps - scanning of Atom 1.0 and RSS 2.0 feeds - parallel download of single file - robots standard (on/off switchable) - SHOUTcast support - reusable C library - many more... -
20
Burst Video
Burst Video provides customization to State-Of-The-Art Video Streaming
Burst Video is a set of tools, and services that add value to the State-Of-The-Art Video Streaming Suites from MyVideoTalk. Burst Studio provides tools that customize MyVideoTalk experience. -
21
...Pings search engines with sitemaps locations. Reads from CSV files and exports entries in CSV format.
-
22
Aerc Crawler
A software used to crawler websites and make sitemaps.
Desenvolvido em Delphi X2 mas com total compatibilidade para Delphi 7, esta ferramenta vasculha um site determinado pelo usuario e atravez das tags "<a href='..." inicia um processo de Crawling no mesmo. Armazenando as informacoes em um banco de dados SQLli existe a possibilidade da criacao de um mapa do website. Developed in Delphi X2 but with full compatibility for Delphi 7, this tool scans a site given by the User and trough tags "<a href = '..." begins the process of Crawling... -
23
Sketchdev
Sketch is a fast website CMS system
OPEN SOURCE--- Sketch is a PHP 5 CMS system built from the servers perspective. It aims for server speed and performance first. It has full support for customization and supports all the latest standards. -
24
Sitemap Creator is a php script making use of the announced new standard SiteMaps protocol supported by Google, Yahoo and MSN. It creates sitemap.xml.gz compressed file for your Sitemap and pings Google,Yahoo! and MSN to come crawl it.
-
25
ScriptBox for several terminal (Bash/Konsole) utilities. Current utilities include: sitemap-ripper - merges sitemaps, sitmap-maker - makes sitemaps, url-extractor - extracts URLs, url2hyperlink - makes links of URLs, file splitter - splits files
