Search Results for "url"
Sort By:
162 projects for "url" with 2 filters applied:
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
1
Curl
Command line tool and library for transferring data with URLs
Curl is a command line tool and library for transferring data specified with URL syntax. It supports HTTP, HTTPS, FTP, FTPS, GOPHER, TFTP, SCP, SFTP, SMB, TELNET, DICT, SSL certificates, cookies, user+password authentication, and so much more! Curl is used for many different things. It's used in command lines or scripts for transferring data. It's also used in just about every device you can think of: mobile phones and tablets, television sets, printers, routers, media players and other audio equipment. ... -
2
img2dataset
Easily turn large sets of image urls to an image dataset
Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine. Also supports saving captions for url+caption datasets. Opt-out directives: Websites can pass the http headers X-Robots-Tag: noai, X-Robots-Tag: noindex , X-Robots-Tag: noimageai and X-Robots-Tag: noimageindex By default img2dataset will ignore images with such headers. -
3
Nock
HTTP server mocking and expectations library for Node.js
HTTP server mocking and expectations library for Node.js. Nock can be used to test modules that perform HTTP requests in isolation. For instance, if a module performs HTTP requests to a CouchDB server or makes HTTP requests to the Amazon API, you can test that module in isolation. Nock works by overriding Node's http.request function. Also, it overrides http.ClientRequest too to cover for modules that use it directly. -
4
Linkwarden
Self-hosted collaborative bookmark manager
Linkwarden is a self-hosted, open-source bookmark manager built to help individuals and teams collect, organize, and preserve important web content in a way that stays useful long after the original pages change or disappear. Instead of saving only a URL, it captures durable archived formats so your saved knowledge remains accessible even when link rot happens. The experience is designed to feel like a modern “read-it-later” tool, with a reader view that makes long articles easier to consume and built-in highlighting and annotations so you can keep context on why a link mattered. ... -
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
5
news-please
Python tool for crawling and extracting structured data from news site
...It provides an integrated pipeline that crawls news sites, retrieves article pages, and extracts structured information such as headlines, authors, publication dates, and article text. news-please can recursively follow internal links and read RSS feeds to gather both recent and archived articles from a news outlet when given only the root URL of a site. It combines several established technologies and libraries to perform web crawling and content extraction, enabling reliable processing across a wide range of news sources. Developers can use the software either as a standalone command line application or integrate it into their own Python applications through its library interface. ... -
6
dude uncomplicated data extraction
dude uncomplicated data extraction: A simple framework
Dude is a very simple framework for writing web scrapers using Python decorators. The design, inspired by Flask, was to easily build a web scraper in just a few lines of code. Dude has an easy-to-learn syntax. Dude is currently in Pre-Alpha. Please expect breaking changes. You can run your scraper from terminal/shell/command-line by supplying URLs, the output filename of your choice and the paths to your python scripts to dude scrape command. -
7
Google2SRT
Download, save and convert multiple subtitles from YouTube videos
Google2SRT allows you to download, save and convert multiple subtitles and translations from YouTube and Google Video to SubRip (.srt) format, which is recognized by most video players. You can download XML subtitles or simply type video's URL, Google2SRT will do the rest. -
8
phpList
Powerful Open Source Email Marketing app with analytics & segmentation
phpList delivers Open Source email marketing, including analytics, list segmentation, content personalisaton and bounce processing. Extensive technical features and a secure and stable codebase are the result of over 17 years of continuous development. Used in 95 countries, available in 20+ languages, and used to send 25 billion email campaigns last year. Deploy it with your own SMTP server, or get a free hosted account at http://phplist.com. -
9
CoffeeWebServer
An http 1.1 server written in Java
NIO HTTP/1.1 Web Server with support for multi-part forms, custom servlets, cookies, sessions, SSL, gzip and deflate encodings, pipelining, and more! Written in 100% Java with core server reliance only on the base jdk library. -
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
10
sitemaptool
A cross-platform sitemap pnp handler written in Go.
This is a simple cli backend tool designed for ease of use. It manages your sitemaps on a live/production site by adding urls and pinging search engines. Just call and pass arguments and it will do the rest. Suitable for sites content sites (blog, social media, etc) as every public post needs to be added to sitemaps and indexed. Sites like soundcloud, instagram, tiktok, facebook, youtube and even medium might find this useful. -
11
MGB is a free OpenSource Guestbook completely written in PHP, using JavaScript and a MySQL Database. Easy to use, flexible and customizable with templates to make it fit 100% to your homepage.
-
12
Cerberus Content Management System 6
Cerberus Content Management System
Cerberus Content Management System is a dynamic, secure and infinitely expandable CMS designed after a Unix-Like model complete with a Unix-Like Kernel File named: Cerberus. It is a custom written Web Application Framework ( W.A.F. ) with a consistent and custom written Pre-Hyper-Text-Post-Processor Programming Code Framework ( P.C.F. ). This Web Application Software Project' aim is to be the fastest and most secure Web Application Framework, Web Application Programming Code Framework, Text,... -
13
CerberusCMS5
Cerberus Content Management System
Cerberus Content Management System is a dynamic, secure and infinitely expandable CMS designed after a Unix-Like model. It is a custom written Web Application Framework ( W.A.F. ) with a consistent and custom written Pre-Hyper-Text-Post-Processor Programming Code Framework ( P.C.F. ). This Web Application Software Project' aim is to be the fastest and most secure Web Application Framework, Web Application Programming Code Framework, Text, Voice and Video Communications Platform and Content... -
14
...TFrameBrowser * Also displays frame and single HTML documents. * TFrameBrower is oriented for use with local file systems and toward Internet style protocols and URL usage. * Additional code and/or components are generally required to get data from other sources than the local file system. These components Support most of the HTML 4.01 specifications with many additional popular HTML 5 enhancements. Many Cascading Style Sheet properties are also supported.
-
15
crawlergo
Headless Chrome crawler for collecting URLs for vulnerability scans
...It also automatically fills and submits forms, helping discover hidden routes or parameters that might otherwise be missed by traditional crawlers. crawlergo includes a built-in URL de-duplication system that removes repeated or pseudo-static links while maintaining fast crawling speeds for large websites. crawlergo also analyzes page content to extract links and resources from multiple sources, including JavaScript files, comments, and configuration files. -
16
grab-site
Web crawler for archiving and backing up sites into WARC archives
grab-site is an open source web crawling tool designed to archive and back up websites by recursively downloading their content. It works by taking a starting URL and systematically following links across the site, capturing pages and resources and saving them into WARC archive files for long-term preservation. Internally, the crawler uses a fork of the wpull engine to fetch and process web pages efficiently during large-scale crawls. grab-site includes a built-in dashboard that displays real-time crawl activity, including which URLs are currently being processed and how many remain in the queue. ... -
17
pspider
Simple Python framework for building multithreaded web crawlers
PSpider is a lightweight web crawling framework written in Python designed to simplify the development of custom web spiders. It focuses on providing an easy-to-understand architecture while still supporting concurrent crawling for improved performance. It uses a multithreaded model that separates the crawling workflow into several components responsible for fetching, parsing, and saving data. Tasks are managed through queues, allowing different parts of the crawler to process work... -
18
Apache2::ModProxyPerlHtml
Reverse proxy with http header, html and javascript code rewritter
Apache2::ModProxyPerlHtml is the most advanced Apache output filter to rewrite HTTP headers and HTML links for reverse proxy usage. It is written in Perl and exceeds all mod_proxy_html.c limitations without performance lost. It is very simple and has far better parsing/replacement of URL than the original C code. It also support meta tag, CSS, and javascript URL rewriting and can be use with compressed HTTP. You can now replace any code by other, like changing images name or anything else. mod_proxy_html can't do all of that. Since release 3.0 ModProxyPerlHtml is also able to rewrite HTTP headers with refresh url redirection. ... -
19
Abot
Fast and flexible C# framework for building customizable web crawlers
Abot is an open source C# web crawler framework designed to help developers efficiently crawl and process web content. It focuses on speed, flexibility, and extensibility while handling the complex low-level tasks involved in web crawling. It manages essential components such as multithreading, HTTP requests, scheduling, and link parsing so developers can focus on processing the collected data. Abot follows a modular architecture that allows developers to customize nearly every stage of the... -
20
Base62 Encode/Decode Utility
A tool for encoding/decoding to base62
...It's not a mere base conversion, because it doesn't use arithmetic inside its logic. In this way it can convert very long numbers into strings quickly, without overflows. For example if I want to share an id using an url, this will make the url shorter. the base62 program has been written to encode/decode files into this format. The syntax is similar to the popular base64 command line program. Inside the package there is a working c++ version of the base62 algorithm. the sources are inside the package. Note that from release 2.0.0 the algorithm is different from the one used before. ... -
21
XSpear
Powerfull XSS Scanning and Parameter analysis tool&gem
XSpear is an XSS Scanner on ruby gems. Powerful XSS Scanning and Parameter analysis tool&gem. -
22
Smart Cache is a fast pure Java HTTP proxy server with offline browsing support. It has URL blocking, URL rewriting. Cached files has not headers inside and cache directory structure copies closely remote server. Supports also large caches.
-
23
Download machine
Batch download manager
Download Machine is pure Java daemon that watches queue files for changes. If a new entry is available, DM will download it. Queue files are just URL lists. DM supports resume for FTP, FSP and HTTP downloads. -
24
Smart Cache Loader
Very configurable web downloader
Smart Cache Loader is a very configurable pure Java web grabber with special support for integration with Smart Cache proxy server. It can perform different loading operations based on URL mask, content-type, ... -
25
mod_spamhaus_new
Apache 2.4.x security, block known bad IP
mod_spamhaus_new is an Apache module that uses DNSBL in order to block spam relay via web forms, preventing URL injection, block HTTP DDoS attacks from bots and generally protecting your web service denying access to a known bad IP address. This module is based on mod_spamhaus but has been updated for actual web server configurations and to support a list of domains, which are NOT spam blocked so customers can reach you even if they got a dynamic IP which is on a spam list. ...
