Search Results for "large files"
Sort By:
24 projects for "large files" with 2 filters applied:
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
1
diskover-community
Open source file indexing & storage analytics powered by Elasticsearch
Diskover Community Edition is an open source file system indexing and storage analytics platform designed to help organizations understand and manage large volumes of file data. It crawls file systems and indexes metadata using Elasticsearch, enabling fast search, analysis, and organization of files stored across different storage systems. It allows administrators and users to explore file structures, monitor storage usage, and gain insights into how data is distributed across infrastructure. ... -
2
news-please
Python tool for crawling and extracting structured data from news site
news-please is an open source news crawler and information extraction tool designed to collect and structure articles from online news websites. It provides an integrated pipeline that crawls news sites, retrieves article pages, and extracts structured information such as headlines, authors, publication dates, and article text. news-please can recursively follow internal links and read RSS feeds to gather both recent and archived articles from a news outlet when given only the root URL of a... -
3
douyin
Open source Douyin crawler for collecting and downloading public data
DouyinCrawler is an open source data collection tool designed to gather publicly available information from the Douyin platform. It demonstrates how to build a Python-based web crawler combined with a graphical interface and command line functionality. It allows users to collect data from various types of Douyin content, including user profiles, videos, hashtags, and music pages. DouyinCrawler supports both automated scraping and batch operations to process multiple targets efficiently. It... -
4
GeneoTree
Dynamic Genealogy Viewer with PHP/MySql-MariaDB
Genealogy Software. Geneotree is a gedcom files PHP viewer, full dynamic on the Web. Main features : dynamic trees, cartography & stats. Many PDF reports, statistics. Special consanguinity and cousins lists. Large files management & optimized ergonomy. -
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
5
XOOPS Web Application System
The world's leading content management systems (CMS) written in PHP
PLEASE NOTE: Latest XOOPS files are on http://github.com/xoops and http://github.com/xoopsmodules25x) XOOPS is a Web application platform written in PHP for the MySQL database. Its object orientation makes it an ideal tool for developing small or large community websites, intra-company and corporate portals, weblogs, and much more. XOOPS is an acronym of eXtensible Object Oriented Portal System. -
6
dirhunt
Web crawler that finds hidden web directories without brute force
Dirhunt is an open source security tool designed to discover web directories and analyze website structures without relying on brute-force techniques. Instead of sending large numbers of guess-based requests, it operates as a specialized crawler that intelligently explores websites to identify accessible or hidden directories. Dirhunt can detect directories that expose “Index Of” listings, which may reveal files and other resources that were not intended to be publicly visible. It can also identify situations where directories are intentionally hidden through empty index files or servers that return misleading responses such as fake 404 errors. ... -
7
crawlergo
Headless Chrome crawler for collecting URLs for vulnerability scans
...It also automatically fills and submits forms, helping discover hidden routes or parameters that might otherwise be missed by traditional crawlers. crawlergo includes a built-in URL de-duplication system that removes repeated or pseudo-static links while maintaining fast crawling speeds for large websites. crawlergo also analyzes page content to extract links and resources from multiple sources, including JavaScript files, comments, and configuration files. -
8
grab-site
Web crawler for archiving and backing up sites into WARC archives
grab-site is an open source web crawling tool designed to archive and back up websites by recursively downloading their content. It works by taking a starting URL and systematically following links across the site, capturing pages and resources and saving them into WARC archive files for long-term preservation. Internally, the crawler uses a fork of the wpull engine to fetch and process web pages efficiently during large-scale crawls. grab-site includes a built-in dashboard that displays real-time crawl activity, including which URLs are currently being processed and how many remain in the queue. Users can dynamically apply ignore patterns during an active crawl, allowing them to skip problematic or unnecessary URLs that could slow down or block the archiving process. grab-site also provides predefined ignore sets for common site structures such as forums and other complex web platforms. ... -
9
csv-parser
Streaming csv parser inspired by binary-csv that aims to be faster
...It implements the stream API native to Node, allowing you to pipe a file or readable stream into the parser and process each row (as a JavaScript object or array) as soon as it's parsed — which is crucial for handling large CSV files without loading them entirely into memory. The parser handles standard CSV semantics including quoted fields, variable delimiters, escape sequences, and optional headers; this makes it robust for a variety of CSV dialects you might encounter. Because it works incrementally (row by row), it is well suited for ETL pipelines, data ingestion workflows, CSV-to-database imports, or any context where you need to process or transform large tabular data in Node.js efficiently. ... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
10
Smart Cache is a fast pure Java HTTP proxy server with offline browsing support. It has URL blocking, URL rewriting. Cached files has not headers inside and cache directory structure copies closely remote server. Supports also large caches.
-
11
Thinkedit is an open source web based content and data management system written in PHP. With large emphasis on databases, and specifically sql ones (mysql). It is highly extensible through simple configuration files. Not actively maintained
-
12
Proxy Tester is a proxy list tester and then it generates a wpad.dat file you can use for your browser based on working servers. wpad.dat file selects a random proxy server from the list of provided working proxy servers with each connection request.
-
13
A reference implementation of the SMPTE-standardised Media Dispatch Protocol (MDP: SMPTE 2032). This software allows large media files and their metadata to be exchanged by organisations. See http://en.wikipedia.org/wiki/Media_dispatch_protocol
-
14
Transparent BitTorrent -- Browse with Firefox and download large files normally. Transporrent monitors your download and if it slows down, transparently switches to using BitTorrent in the background. Transporrent requires a browser Add-on and server mod
-
15
-
16
This program is a HTTP[S]/FTP/BitTorrent file downloader supporting many features like resuming, retrying and multi-connection HTTP[S]/FTP downloads. It can also download any part of a file so that you can download large files in smaller pieces.
-
17
HTML Large file uploader is a FTP replacement for the normal "file" form input field. This is useful in getting around various restrictions in place that prevent files larger than a few MB being uploaded through HTTP.
-
18
This is currently a bunch of scripst used to manage translations of a large number of Java Server Pages (JSP). It is not integrated into any IDE, but can be used as a general way to manage these files from anywhere.
-
19
Append hundreds of different XML files into one large data document, validate, and attach a stylesheet. Stitcher is a porwerful summary reporting tool. Allows for complex file storage locations of xml data. UBL, HR-ML, LandXML ... visit stitcher.org.
-
20
Open Digital Assets Repository System. This is php application for storing large amount of assets (> 100 000). Each asset can consist of several files and can have different versions. This kind of application is also called: AMS, CMS.
-
21
This is a database front-end for keeping track of chord files (a.k.a. fake sheets). It accepts chord and chordpro files as inputs, and creates output sets in a large variety of formats (pdf, text, chordpro, latex).
-
22
Teraband is a PHP-based download administration system capable of dynamically limiting bandwidth consumption for large files. (Teraband can help prevent you from incurring overage charges from web hosts.)
-
23
Web data Zip Stream (de)Compressor is a java applet that allows web masters to house large web pages in compressed .zip files and then have it dynamically uncompress and display on the client browser as if it were a regular webpage.
-
24
Fget is a cross-platform command-line multi-threaded HTTP/HTTPS/FTP file downloader which is very close to multiget. It features downloading single but very large files in multiple threads at highest possible speed with minimal system resources.
- Previous
- You're on page 1
- Next
