| Name | Modified | Size | Downloads / Week |
|---|---|---|---|
| Parent folder | |||
| zpaqfranzxp.exe | < 5 hours ago | 6.1 MB | |
| zpaqfranz-open.exe | < 5 hours ago | 5.1 MB | |
| zpaqfranzhw.exe | < 5 hours ago | 6.0 MB | |
| zpaqfranz-full.exe | < 5 hours ago | 11.8 MB | |
| zpaqfranz32.exe | < 5 hours ago | 6.3 MB | |
| zpaqfranz.exe | < 5 hours ago | 6.1 MB | |
| zpaqfranz.cpp | < 5 hours ago | 8.2 MB | |
| README.md | < 6 hours ago | 34.1 kB | |
| Windows 32_64 bit executables and source code source code.tar.gz | < 6 hours ago | 3.1 MB | |
| Windows 32_64 bit executables and source code source code.zip | < 6 hours ago | 3.1 MB | |
| Totals: 10 Items | 55.8 MB | 0 | |
zpaqfranz 65.6
From 65.5g to 65.6i.
BEWARE: A LOT OF NEWS == A LOT OF NEW BUGS TOO!!! BE CAREFUL, NOT VERY TESTED!
Seven headlines:
- New defaults:
ais now-m8 -turbo -fast(zstd, the parallel fragmenter, the history for a fastl:-fastis the-filelistof 65.4 and 65.5, renamed). On Windowsxwrites big files as sparse files.-715,-noturbo,-nofast,-nosparsebring the old behaviour back. f -test: a surface test of a whole device, read only, with a live map and a verdict in words. With-force -paranoidthe "triello" (Windows): it writes the whole device, reads it back and compares, and finds fake capacity, lost writes and wrong data.- A dashboard in the console (
franzdash): the live map off -test, and ofa -imagewhile an image is taken (by default;-nodashboardor-noetafor the classic line). - Images that go on: a sector that cannot be read becomes zeros, the image goes on, and at the end a giant READ ERROR with the exit code 2.
mount -range A:B: only what versions A..B added or updated, one folder each.work email: an SMTP client with TLS inside zpaqfranz (for the e-mail reports of the next versions; today a command of its own), andwork zip.- Damaged archives are said, not read wrong: missing blocks, a broken
block name, an index block lost at the end of a version (72401-72406,
72414); and
a -chunkno longer loses the end of the archive on Linux.
This document has three parts:
- Part one: what is new, in short, for everybody
- Part two: the same things in more detail, for power users
- Part three: "lo spiegone", how it works inside, and why, for developers
Part one, what is new, for the user
- New defaults
f -test: is this disk (or stick) good?- The triello: write, read back, compare
a -image: the dashboard- Images of a damaged source
mount -range: only the changeswork emailand <[inline_block>1](#7- Exit codes you can trust
- Smaller things
1. New defaults
zpaqfranz a z:\backup.zpaq c:\data (now: -m8 -turbo -fast)
zpaqfranz a z:\backup.zpaq c:\data -715 (as zpaq 7.15: -m1, no -turbo, no -fast)
zpaqfranz a z:\backup.zpaq c:\data -m1 -noturbo -nofast (as before 65.6)
| before | now | |
|---|---|---|
| method | -m1 |
-m8 (zstd, level 3) |
| fragmenter | one thread | -turbo: big files cut by several threads |
-fast (was -filelist) |
asked with -filelist |
always (the history of the archive, l in a fraction of a second) |
x on Windows |
sparse files only with -sparse |
files of 16 MB or more are sparse (the zeros are not written) |
a -image |
the progress line | the dashboard, a live map of the source |
The archive made by -turbo is the same, to the byte, of the one made
without it: only faster. -m8 is smaller than -m1 and, with -turbo,
faster; any zpaq extracts it (zpaq 7.15 too, through the decoder stored in
the archive, at 80-95 MB/s). The builds for old machines (ESX, NAS)
keep the old defaults.
-filelist and -nofilelist are now -fast and -nofast. The old names
still work (the scripts written for 65.4 and 65.5 do not change), and the
history stored inside the archives is the same: an archive made by 65.5 is
listed fast by 65.6, and the other way round.
2. f -test: is this disk (or stick) good?
zpaqfranz f E: -test
zpaqfranz f 3 -test -buffer 4194304
zpaqfranz f b -test (Linux: /dev/sdb)
Reads the whole device, raw, block by block (1 MB by default), as HD Tune does: nothing is written. While it runs a map shows every stretch of the device (green ok, yellow slow, red stall or error, grey still to do); at the end the speed band by band, the median, the slowest and fastest 5%, the slow blocks (read again: slow again means the device, fast means a hiccup), the stalls, the I/O errors, and a conclusion in words:
VERDICT: GOOD: the device is practically perfect exit code 0
VERDICT: SUSPICIOUS: it works, with mild anomalies exit code 1
VERDICT: BAD: serious failure, copy the data elsewhere now exit code 2
Ctrl+C stops it and shows the report of what was done (a second Ctrl+C quits).
Virtual disks: the speed of a virtual disk depends on the host and its cache. On the test VM the same 8 GB disk was "GOOD" read with 4 MB blocks and "BAD, it may be faulty" read with 1 MB blocks (the speed jumped by 44% between near zones). The verdict is meant for real devices.
3. The triello: write, read back, compare
zpaqfranz f E: -test -force -paranoid (DESTROYS EVERYTHING on E:)
zpaqfranz f 3 -test -force -paranoid -quick (write only)
Windows only. Every block of the device is written with data never
repeated, then read back and compared. It finds what a read test cannot:
fake capacity (the addresses wrap around), writes that went nowhere, wrong
data, a write cache that hides a slow device. It asks the captcha
nomercy (even with -nocaptcha), works only on USB devices, never on the
disk that holds Windows, and leaves the device without partitions:
initialize it again (Disk Management, or diskpart).
On the test VM, the 8 GB virtual disk: written at 1.3 GB/s, read back at 1.8 GB/s, every byte right, 11.5 seconds.
4. a -image: the dashboard
zpaqfranz a z:\c.zpaq c: -image -ntfs -vss
zpaqfranz a /tmp/sdb.zpaq /dev/sdb -image
zpaqfranz a /tmp/sdb.zpaq /dev/sdb -image -nodashboard (the classic line)
The map of the source while the image is taken, by default (it was asked
with -dashboard, which is still accepted). With -ntfs it is known
from the start which blocks are free (not read), excluded (the pagefile,
-not) and to read. A block read is green when its data were new
(compressed into the archive) and blue when they were already there
(deduplicated): on the second image of the same disk, blue is what did not
change. Slow and stalled reads as in f -test, E where the source could
not be read.
With the output redirected there is no live map: the map is written as text
at the end (in a log, some 20-30 lines). Not with -verbose or -debug
(their lines would break it). -nodashboard or -noeta give the classic
progress line (and no map in the log).
5. Images of a damaged source
A sector that cannot be read no longer stops or blocks a -image: it could
loop forever on the same error (on Linux adding the block read before,
again and again). Now:
- what cannot be read becomes zeros, and the image goes on;
- the first error is said at once, on a line of its own;
- at the end a giant READ ERROR, how many sectors, where, and the exit
code 2 (also with
-ignore): the image is complete, but it is not a copy of the source; - a source that goes away (unplugged) gives zeros up to the end, without trying every sector;
- an internal error of the reader stops everything and the archive goes back to the version before.
6. mount -range: only the changes
zpaqfranz mount z:\backup.zpaq -range 2:2
zpaqfranz mount z:\backup.zpaq -range 5:9 z:\m
Mounts the versions A..B (numbered as l numbers them), one folder each
(VER00000001...), and each folder holds only the files that version
added or updated, the same thing l -range and x -all -range show.
-all alone mounts every version as a full snapshot, as before. A range
with no version in it is refused (exit code 2).
7. work email and work zip
zpaqfranz work email -mailconfig c:\zpaqfranz\email.conf -mailto you@example.com -mailsubject "backup OK" -mailzip z:\backup.log
zpaqfranz work zip z:\backup.log z:\backup.zip
zpaqfranz can send an e-mail by itself: SMTP with TLS (STARTTLS on port 587 or implicit TLS on 465), login, attachments as they are or zipped. No external program, no DLL: the TLS client is inside. Today it is a command of its own; the e-mail report at the end of a backup comes in a next version. The exit code is 0 when the server accepted the message, 2 when not (and the reason is written).
work zip makes a .zip of one file (deflate), the kind that opens with a
double click: handy to mail a log.
8. Exit codes you can trust
- A fatal error (an archive that does not exist, a wrong parameter, a
device that cannot be opened...) used to end with exit code 0: a
script took it for a success. Now it is 2. The help,
dir,qin the pager are still 0. - Commands that counted errors (
x,ton many archives,a -home) could return 256 errors as exit code 0. Now the exit code is 0, 1 or 2: the worst of the parts, not their sum (two warnings are a warning). - A damaged archive gives an exit code other than 0 (see the next section and part three): missing blocks are an error, the last index block of a version lost is a warning (72414, exit code 1).
9. Smaller things
a -chunkon Linux, *BSD, macOS: when the archive ended a few KB past a multiple of 128 KB, the last part was deleted as empty and the archive could not be read, with exit code 0. On Windows an empty last part stayed on disk. Both fixed.x -stdoutof a damaged archive: its messages went to stdout, into the data; now they go to stderr, and the data are clean.- A bit flipped in the name of an index block:
xextracted a junk file namedjDC...i0000000001and lost the last version, with exit code 0; now the block is skipped as damaged, with a message and exit code 1. xwithout-force: a file already on disk with a different size (an extraction killed halfway) is no longer skipped in silence: a warning for each one and exit code 1.- Windows: names with characters outside the BMP (emoji, rare ideographs) are now stored as real UTF-8, so they come back right on Linux and macOS.
- Windows,
-longpathwith a relative-toworks (it produced unusable paths); the open build also suggests-longpathwhen a name is too long. - Windows, restoring a raw image of a partition: it failed (exit code 2) on the last block, while the data were already right.
- Windows,
a -vsswith-notor-only: they were ignored (and-onlygave an empty backup). - Windows, a symbolic link to a file with the same name as its target made the link, and everything after it in the folder, vanish from the backup.
utf -killrenames the files even when no folder has a UTF-8 name.h something > fileno longer waits for a key that never comes.l -csvand-csvhfwith a%in them crashed.x -ramdisk: a file that could not be written stopped the writing of all the others (they were only in RAM); now it goes on, exit code 2.- The help of
zip -rangesays where to find the "only the changes" (x -all -range,mount -range):zip -all -rangekeeps making full snapshots. - A batch of fixes of issues reported on GitHub, and of what a review of
a,x,l,fand the images found (see part three).
Part two, the details, for power users
- The defaults, and how to go back
f -testin practice- The triello in practice
- The dashboard
- READ ERROR
- <[inline_block>0](#6
- <[inline_block>0](#7
- What can change for a script
- Backup and restore of a whole disk, measured
1. The defaults, and how to go back
| switch | meaning |
|---|---|
-noturbo |
one thread cuts the fragments (as zpaq 7.15) |
-turbo N |
N threads to cut and hash (default: as -t) |
-nofast |
do not store the history (the old -nofilelist, still accepted) |
-nodashboard |
a -image: the classic progress line instead of the map |
-noeta |
no progress line, and no map |
-nosparse |
x on Windows: no sparse file, not even the ones that were sparse at the origin |
-715 |
works as zpaq 7.15: -m1, no -turbo, no -fast |
-m1 |
the method of before |
-appendnever stores the history (its first pass writes nothing): the next plainastores it again.xon Windows: a file is made sparse when it was sparse at the origin (as zpaq 7.15 does) or when its size, known before writing it, is 16 MB or more. Its all-zeros fragments are not written and become holes. The smaller files are written as before (tiny files stay in the MFT). On a filesystem without sparse files (FAT32, exFAT) the file is a normal one, not an error. The archives themselves are no longer created sparse.- Measured on the VM: a 24 MB file with 23 MB of zeros came out sparse with
the zeros not allocated; with
-nosparsea normal file; the same SHA-256.
2. f -test in practice
| switch | meaning |
|---|---|
-test |
surface test of the device, raw, read only |
-buffer X |
block size (default 1 MB, rounded up to the sector, 64 MB at most) |
The device: a letter or a disk number on Windows (see drive), b for
/dev/sdb or a full /dev/... path on *nix. Reading a device is allowed
on the system disk too.
How a block is judged (the first 3 seconds, the warm-up, are on the map but not in the statistics):
| state | when |
|---|---|
| slow | under 50% of the median speed, and at least 20 ms lost |
| stall | over 10 times the median time, and at least 0.1 s |
| timeout | a single read over 30 s |
| I/O error | the read fails (64 in a row: the device is gone) |
Then the surface is split into 32 zones, and near zones are compared (a jump of speed between them), as the spread of the blocks inside a zone. The report lists the anomalies, the worst first.
3. The triello in practice
- Writing is judged with wider limits (the stops of the flash to clean up are normal): slow when 50 ms are lost, stall at 0.5 s.
- The write cache: a moving median under 60% of the first plateau means the cache is over; the report says how much slower the device is after it.
- Reading back, a block can be: right; wrong in some bits; all zeros; all 0xFF (never written flash); old data (the write went nowhere); the data of another block of this run (the addresses wrap: fake capacity). A block wrong once and right when read again is "unstable".
- Refused: a device that is not USB (
-debuglifts this, for tests on virtual machines: never the system disk), the disk that holds Windows (always), a wrong captcha (exit code 1, nothing written).
4. The dashboard
| on the map | f -test |
a -image |
|---|---|---|
| grey dot | still to do | free block (NTFS), not read |
| cyan | being read now | being read now |
| green | ok | read, new data |
| blue | written, not yet read back | read, deduplicated |
| magenta | all excluded (pagefile, -not) |
|
| yellow | slow | slow |
| red | stall, error, wrong data | stall, E unreadable |
Only the cells that change are drawn again (4 times a second at most). With
-nocolor or the output redirected, letters instead of colors; redirected,
the map comes at the end as text. For a -image the map is the default:
-nodashboard and -noeta turn it off (also when -dashboard is given),
-verbose and -debug too.
5. READ ERROR
69100! READ ERROR at 20.971.520 of the source: zeros in the image, going on
69110! READ ERROR: 1.024 sectors of the source could not be read
69111! The image has 524.288 bytes (512.00 KB) of ZEROS instead of the data, in 1 stretch
69112! Unreadable from offset 20.971.520 for 524.288 bytes
69115! THIS IMAGE MAY BE CORRUPTED: do not trust this version of the archive
69116! Check the source with a surface test: zpaqfranz f /dev/mapper/bad -test
(the first 16 stretches are listed, then how many more). A read that fails is tried again 64 KB at a time, then sector by sector for the 64 KB that fail; 1 MB in a row with nothing readable is a dead zone, not tried sector by sector until a read works again.
Measured on Fedora with device-mapper: a 64 MB device with 200 unreadable sectors (100 KB) at 20 MB: exit code 2, the image complete, identical to the source outside the zeros. On Linux the zeros are 512 KB, not 100 KB (see part three: the reads go through the cache of the kernel).
6. mount -range
-range A:Bimplies one folder per version;-untilis refused with it (as forlandx).- The folders keep the numbering of the mount: version 2 is
VER00000001. - In each folder, only the files whose last change is that version (added or updated, not deleted). A path typed by hand of a file not changed in that version does not exist, even if it exists in the version.
- A range beyond the last version is cut to it; a range with no version is
refused:
94033! -range 4:4: no such version, the archive has 3. mount -testknows-range(it checks the files of the last version of the range). Tested on Linux (FUSE) and on Windows (WinFsp).
7. work email
| switch | meaning |
|---|---|
-mailserver HOST -mailport N |
the SMTP server (587, or 465 with -mailssl) |
-mailssl |
implicit TLS instead of STARTTLS |
-mailuser -mailpassword |
login (no user: no authentication) |
-mailfrom -mailto -mailcc -mailbcc |
addresses, "a@x.com, b@y.org" for several |
-mailsubject -mailbody -mailbodyfile |
the message (UTF-8) |
-mailattach F1 F2... |
files as they are |
-mailzip F1 F2... |
files sent as F1.zip... |
-mailconfig FILE |
key = value settings (server, port, user, password...); the switches win |
-mailcafile -mailtlsname |
the trusted CAs (default: the system store), the name to verify |
-mailinsecure |
do not verify the certificate (tests only) |
-mailtimeout SEC -maillog FILE |
network timeout (30 s), the full log (never the password) |
The password is better in the -mailconfig file than on the command line.
-verbose shows the log, -debug the SMTP dialogue.
8. What can change for a script
| before | now |
|---|---|
a made -m1 archives |
-m8 (add -m1 to keep the old ones alike) |
| a fatal error: exit code 0 | 2 |
x, t on many archives, a -home: the sum of the codes (256 errors = 0) |
the worst one, 0, 1 or 2 |
x without -force over a changed tree: exit code 0 |
1 when a file on disk has a different size |
f -test did not exist |
a surface test (read only; the triello needs -force -paranoid and the captcha) |
| an image of a damaged source: stopped, or looped | zeros, READ ERROR, exit code 2 |
work email with postami's own switches (-s, -t, --help) |
the -mail... switches |
-filelist, -nofilelist |
-fast, -nofast (the old names still work) |
a -image: the progress line |
the map; in a log, the map as text at the end (-nodashboard or -noeta: as before) |
| an archive with a version that lost its last index block: exit code 0 | 72414, exit code 1 |
x -stdout of a damaged archive: the messages inside the data |
on stderr |
a -chunk: sometimes an unreadable archive, exit code 0 |
fixed |
9. Backup and restore of a whole disk, measured
On the Windows 11 VM, the second virtual disk (8 GB, NTFS, E:):
| what | result |
|---|---|
a e.zpaq E: -image (raw partition, 7.98 GB) |
27 s |
image e.zpaq E: -to E: -image -raw |
18 s, 451 MB/s, then chkdsk clean |
a e.zpaq E: -image -ntfs -vss -not E:/dati/escludi/ (121 MB used) |
3.4 s, 26.9 MB archive |
image e.zpaq E: -to E: -image -ntfs |
16 s: excluded folder deleted, a file added later gone, a file deleted later back, SHA-256 of the data identical, chkdsk clean |
a d.zpaq 1 -image (the whole physical disk 1) |
28 s |
image e.zpaq E: -to x.raw -raw (export to a file) |
a file of the size of the partition |
Part three, lo spiegone, for developers
- Two branches, one source
- The image reader that goes on
- franzdash
f -test: numbers, not impressions- <[inline_block>0](#5
- Names outside the BMP
- The e-mail client
- Fixes
- Known limits and open points
1. Two branches, one source
65.6 joins two lines of work that started from the same 65.6a (the one
with the e-mail client): the fixes of the GitHub issues and of a review of
a, x, l, f (on one side), the new f -test, franzdash, the image
reader and the new defaults (on the other). The three-way merge had 10
conflicts, all small (the two sides had fixed the same printf formats and
the same pager in the same way). Built and tested on Windows (full, open,
mount), Fedora (plain, -DSFTP, -DZPAQMOUNT) and FreeBSD (clang):
autotest 16/16 everywhere.
The source: 188,125 lines in 65.5g, 227,767 now (8.2 MB). 33,500 of them are the TLS library of the e-mail client.
2. The image reader that goes on
img_leggi(Windows) andimg_leggiunix(*nix,preadat the offset every time: after an error a handle does not know where it is): the whole buffer at once; if it fails, 64 KB at a time; for the 64 KB that fail, sector by sector (the sector size from the device:IOCTL_DISK_GET_DRIVE_GEOMETRY_EX,BLKSSZGET). What cannot be read is zeros, andimg_errorecounts the sectors and the stretches (the first 16 remembered).- A dead zone: 16 pieces of 64 KB in a row with nothing are not tried
sector by sector (a dead area of a disk would take hours) until a read
works again. A source gone (
ERROR_DEVICE_NOT_CONNECTED,ENODEV...): zeros to the end, no more reads. - The loop of
add()andadd2(): a reader that returns an error code (not data) would be called again forever on the same block. Nowimagefatale()stops, through the same housekeeping of Ctrl+C (the archive back to the last complete version, a new one removed, the VSS deleted), exit code 2 (g_fermatafatale). imagebanner()at the end ofadd(): the giant READ ERROR;errors= 2even with-ignore.- The Linux DD reader also got its progress (it stayed at 0).
3. franzdash
A class that knows nothing of disks: a title, two lines of information, a
map of cells, a legend. The client declares its "looks" (a letter, a color,
a square or a UTF-8 glyph, a name for the legend) and sets the look of each
cell; the dashboard draws once, then only the cells that changed:
relative moves of the cursor on *nix, the rows of the console buffer on
Windows. Redirected, or with -nocolor: letters, and the map as text at
the end. The cursor is hidden while it draws and shown again by
my_handler too (Ctrl+C). Two clients: franzusbmappa (f -test: a cell
shows the worst state of its blocks) and franzimgdash (a -image: the
blocks of the source; new or deduplicated is decided by the fragments of
the block, imgdash.frammento() called by the fragmenter with htptr == 0).
In the ANCIENT builds a stub: the classic progress line.
flagdashboard is on by default (nuovodefault, as -turbo and -fast);
after the switches are read -nodashboard and -noeta turn it off.
4. f -test: numbers, not impressions
- Every block timed; the statistics after a warm-up of 3 s; median, p5, p95, coefficient of variation.
- 32 zones: the biggest jump of speed between near zones, how far a block typically is from the speed of its zone; the first and the last zone.
- Slow blocks are read again: slow again is the device, fast is a hiccup (of the host, of the bus).
- The triello: the data of each block are generated from its number and from a key of the run, so they are never repeated: a block read back with the data of another block of the same run means that the addresses wrap (a "32 GB" stick that is really 8 GB); data of an older run means the write went nowhere.
- Five levels, from "practically perfect" to "serious failure"; the exit code is 0 for the first two, 1 for "suspicious" (or a test stopped by Ctrl+C), 2 for "may be faulty" and "serious failure".
5. mount -range
The mount read the version range (-range sets all, the "one folder per
version" mode) but never used g_rangefrom/g_rangeto: it built the full
snapshot of every version. Now st.firstversion and st.nversions delimit
the versions exposed, and the jidac backend, with g_mountchangesonly,
builds the tree of a version from jidac_changed_in(): the last change of
the file is that version, and it is not a deletion. entry() uses the same
test, so a path typed directly is consistent with the directory listing.
6. Names outside the BMP
wtou() (UTF-16 to UTF-8, from zpaq 7.15) converted one wchar_t at a
time: a surrogate pair became two 3-byte sequences (CESU-8), which is not
UTF-8. Windows did not notice (utow() reads both forms back), Linux wrote
6 bytes of garbage, and utf8toansi() (MB_ERR_INVALID_CHARS) returned an
empty string. Now a pair becomes the real 4 bytes; a lone surrogate keeps
the 3-byte form; utf8toansi() turns every byte of a name that is not
UTF-8 into _ instead of returning nothing. A name with such characters
already stored in CESU-8 looks renamed at the next a (the old entry
deleted, the new one added, the data deduplicated).
7. The e-mail client
- postami 0.3.0, the SMTP client, and a subset of Mbed TLS 3.6.4 configured as a TLS 1.2 client (ECDHE-RSA/ECDSA, AES-GCM, ChaCha20-Poly1305, AES-CBC, X25519/P-256/P-384, X.509 with RSA and ECDSA), mechanically reduced (unused code removed, C99 turned into C++98).
- Windows: no link library added (
ws2_32,crypt32,bcryptare loaded at run time); the certificates from the system store unless-mailcafile. - The command line interface of postami (its own parser,
--help) is gone:work emailtakes the-mail...switches fromloadparameters(), as every other switch.zpaqfranz_sendmail()is the entry point for the reports of the backups of the next versions (not called yet). -DNOEMAIL(set by-DOPEN) leaves it all out. Note: the "open" source made byrella.bat(theNOSFTPSTART/NOSFTPENDpurge) still contains the e-mail client and the TLS library, because they are not inside those markers.
8. Fixes
GitHub issues:
- 300
utf -kill: the files were renamed only when some folder had a UTF-8 name (the crash of the same issue was fixed in 65.5). - 314-319: a damaged archive is described instead of being read
silently wrong: data without fragment tables, tables without index, holes
in the numbering of the blocks (messages 72401-72406); a multipart with a
missing part in the middle (72407), and
athat would fill it refuses (72408); the last block damaged at its very end but read:aappends after it instead of destroying it (72410); *nix, files not archived because the path is too long (ENAMETOOLONG) counted as a warning (72411). - 318 the
-longpathhint was only in the full build. - 320 a bit flipped in the
jDC\x01comment of a block sent it down the streaming path: a junk file, the last version lost, exit code 0. A fulljDCblock name there is now a damaged block, as a bad block name is. - 323 missing blocks, the rest of it: an index block lost at the end of
a version leaves no hole in the numbering.
read_archive()now marks every fragment used by a file (all the records, before any selection) and the fragments of the-fasthistory (the ranges in the last field of its pointer); on a clean read (nothing skipped, no hole, no incomplete version, no streaming block) the fragments that nobody uses are the data of the files that were in the lost block: 72414, a warning, exit code 1 (g_frammentiorfani, read bytest(),extract()and the final summary). No false alarm on 86 real archives (up to 3,497 versions) and on archives made with every method and option; the reading time does not change. Onx -stdoutit goes to stderr. - 321, 325
-longpathwith a relative-tomade//?/relativepaths; the summary of long names suggests-longpath. - 324
xover a partial file: warning and exit code 1. - 326
mount -range.
From the review of a, x, l, f and of the images:
a -externalran the external command twice when it failed.l -csv,-csvhfandsftp runused a string of the user (or of the server) as the format ofprintf: a%crashed.x -ramdisk: one file that could not be opened stopped the writing of all the others, exit code 0.- Images with exclusions (NTFS): an excluded cluster beyond the end of a
short block overflowed a
memset; an exclusion on another drive letter was applied to the drive being imaged (now ignored, with a warning); a physical disk numbered 10 or more is refused (its image name has one character only). a -vss:-not,-only,-alwayswere not moved onto the snapshot.- Windows scan: a symbolic link to a file with the name of its target moved the whole rest of the folder under the target (out of the backup), and its size was read as 0 (a HOUSTON error).
- Restore of a raw partition image on Windows: the volume handle stops at
the end of the filesystem, the last block was refused;
FSCTL_ALLOW_EXTENDED_DASD_IOnow. - Exit codes:
x/ton many archives anda -homesummed the codes; every command is capped at 2. -chunk: anawith nothing new left an empty 104-byte chunk.-chunk: at the end ofadd()the header of the version is written again on the first part; that path ofOutputArchive::flush()closed the first part and set the current one to none without closing it. On *nix its last bytes stayed in the stdio buffer, the part looked empty and was deleted (an unreadable archive, exit code 0, also in 65.5); on Windows the handle stayed open and an empty part could not be deleted (error 32). Now the current part is closed there.x -stdout: the archive was read before the silent mode started, so the messages of a damaged archive (72401-72406, "Skipping block") went to stdout, before the data. The silent mode now starts before the reading, and in it, with-stdout, errors and warnings go to stderr.aover a damaged tail says that it overwrites it (72413);-indexand*names say that holes are not checked (72412).- *nix: a folder named like a part of a multipart was taken for an archive; folders that cannot be read are reported as on Windows; the error of the wrong call was printed.
l: a file namedVCOMMENTplus a few characters crashed it.printfformats:%ldand%zuread with the wrong size on Windows; several messages with missing or wrong arguments.- The captcha passed with stdin closed;
hwaited for a key with the output redirected;dir /osand the tree view sorted the other way round; theautotest -nof a small size wrote out of its buffer. - The Makefile really prefers clang++ (
CXX ?=did nothing with GNU make).
9. Known limits and open points
- Linux, READ ERROR: the retries sector by sector go through the page
cache of the kernel, which reads in big pieces (large folios): around
bad sectors good data can become zeros too (512 KB for 100 KB of bad
sectors in the test), and the report counts them. Reading with
O_DIRECTwould fix it; not done yet. f -teston virtual disks: the verdict is not reliable (see part one).- Export of an NTFS image to a VHD (
image x.zpaq E: -to d:\3.vhd -ntfs):-tois a folder, the file inside isimage_E.vhd(the help suggests a file name). - An image of a physical disk made by number is extracted as a file,
but
imagecannot write it back to a disk ("you need a drive letter"). - The imager writes with
FILE_FLAG_NO_BUFFERINGfrom buffers that are not explicitly aligned to the sector (it works on the tested disks). - The Ctrl+C handler does its housekeeping inside the signal (not
async-signal-safe): rarely, a Ctrl+C in the middle of an
acan abort with a heap error (the archive stays at the last complete version). - A version made only of deletions that loses its only index block is
not seen without
-fast: it is identical, byte by byte, to the empty version thata -forcewrites when nothing changed (zpaq 7.15 drops that one). With-fast(the default) the numbering of the blocks shows it. lwith the-fasthistory does not read the archive: a damage is found byt, byxor byl -nofast.