pguniq

Pretty Good Unique - report or omit repeated lines without sorting

Status: Alpha

Brought to you by: antondemidov

Add a Review

Downloads: 0 This Week

Last Update: 2021-09-30

Browse Code

Get an email when there's a new version of pguniq

Linux

This utility suppresses or reports duplicate rows without the need to sort them first.
It calculates CRC32 hash for every row and saves it in bit matrix. By default it uses 4-byte hash which consumes up to 512MB of RAM. Memory is allocated by 1K chunks as needed.

Project Activity

See All Activity >

License

GNU General Public License version 3.0 (GPLv3)

Follow pguniq

pguniq Web Site

Other Useful Business Software

99.99% Uptime for MySQL and PostgreSQL Databases

Sub-second maintenance. 2x read/write performance. Built-in vector search for AI apps.

Cloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.

Try Free

Rate This Project

User Reviews

Be the first to post a review of pguniq!

Additional Project Details

Operating Systems

Linux

Programming Language

Registered

2021-09-29

Report inappropriate content

Our Free Plans just got better! | Auth0

With up to 25k MAUs and unlimited Okta connections, our Free Plan lets you focus on what you do best—building great apps.

You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.

Try free now

Recommended Projects

CyoHash
Shell extension for easily calculating a file's hash (MD5/SHA1/etc.)
WinHash
Helps you to verify the integrity of the files. Hash Calculator for files. Using: CRC32 MD5 RIPEMD-160 SHA-1 SHA-256 SHA-384 SHA-512 Update: Added CRC32 Now supports big files (more then 4Gb) *Requires .NET 4.5
PeaUtils
File management utilities: split & join, secure delete, crc/hash
videohash
The Python package for near duplicate video detection
MacHash
MacHash is intended to be a HashCalc clone for Mac OS X. Sporting a neat Aqua interface, it allows you to calculate a variety of popular hash values like MD5 or CRC32 from diffent input sources such as a line of text or a file on disk.