pguniq

Pretty Good Unique - report or omit repeated lines without sorting

Status: Alpha

Brought to you by: antondemidov

Add a Review

Downloads: 0 This Week

Last Update: 2021-09-30

Browse Code

Get an email when there's a new version of pguniq

Linux

This utility suppresses or reports duplicate rows without the need to sort them first.
It calculates CRC32 hash for every row and saves it in bit matrix. By default it uses 4-byte hash which consumes up to 512MB of RAM. Memory is allocated by 1K chunks as needed.

Project Activity

See All Activity >

License

GNU General Public License version 3.0 (GPLv3)

Follow pguniq

pguniq Web Site

Other Useful Business Software

Fully Managed MySQL, PostgreSQL, and SQL Server

Automatic backups, patching, replication, and failover. Focus on your app, not your database.

Cloud SQL handles your database ops end to end, so you can focus on your app.

Try Free

Rate This Project

User Reviews

Be the first to post a review of pguniq!

Additional Project Details

Operating Systems

Linux

Programming Language

Registered

2021-09-29

Report inappropriate content

Ship Agents Faster

Transform your applications and workflows into powerful agentic systems at global scale.

Gemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.

Get Started Free

Recommended Projects

Files Hash code Verifier
A tool to verify and generate MD5,SHA,CRC32 hash of multiple files
CyoHash
Shell extension for easily calculating a file's hash (MD5/SHA1/etc.)
String Hash Calculator
String Hash Calculator Calculates the MD5,SHA1,MD2 and MD4 Hash
WinHash
Helps you to verify the integrity of the files. Hash Calculator for files. Using: CRC32 MD5 RIPEMD-160 SHA-1 SHA-256 SHA-384 SHA-512 Update: Added CRC32 Now supports big files (more then 4Gb) *Requires .NET 4.5
PeaUtils
File management utilities: split & join, secure delete, crc/hash