Re: Does isync report an error in case of data corruption in RAM?

[Hong Xu <ho...@to...>]

On 2025-01-13 Mon 04:25 GMT-08, Oswald Buddenhagen <osw...@gm...> wrote:

> On Mon, Jan 13, 2025 at 01:33:54AM -0800, Hong Xu wrote:
>>Does isync perform any checks (like some checksum) to prevent data loss
>>in an event of RAM error?
>>
> no, and the idea doesn't even make much sense. there would have to be a
> checksum to compare to, and that couldn't be calculated before the data
> was actually loaded into ram.

I did a bit of research into IMAP, and I agree that there's no way to
fully fix this unless there are data integrity support at the IMAP
level.

>
> the whole idea of software-based ram-checking would make sense only for
> data that lives in ram for a long time, where corruption due to storage
> deterioration would be much more likely than corruption during transfer,
> or the added time due to computing the checksums at storage and
> retrieval.
>
> isync stores message payloads only for a few seconds in its network
> buffers. if you are concerned about integrity at that time scale, get
> hardware that can do it for you automatically. it would be total
> insanity to half-assedly harden each application.

Even though I don't think isync can fully fix the issue here, I'd still
like to clarify the concern for isync users, which is real and shouldn't
be dismissed. I'd also like to share what I do to mitigate.

=== Concern ===

With proper backup, storage corruption is not a concern here, because
storage corruptions can hardly go undetected due to the builtin error
checks in hard drives, even on consumer-grade hardware. When a
corruption occurs, data can be read without error only at a super-low
probability. See [1] and its reference.

However, consumer-grade hardware is prone to RAM bit flips, at a higher
rate than what most people probably think. The most recent research I
knew is discussed on Wikipedia [2], which shows the error rate was on
the magnitude of 1 bit error per gigabyte of RAM per 1.8 hours.

And they truly affect data integrity, which is a frequent topic
discussed in the NAS community for one example, because data can go
corrupted in RAM in the period of being copied from one location to
another (e.g., [3]).

=== My Solution ===

I'd also like to share what I do to mitigate silent data corruption when
it's impractical to use an ECC RAM:

- Regularly back up email folders. This can become a life-saver if a RAM
  becomes faulty, in which case many file corruptions and system crashes
  becomes noticeable. I can then restore the backups and let them sync
  back to the server with isync.

- Save all important data, like attachment, to somewhere else with data
  integrity checks, such as a local git repository, or a subversion
  repository if there are large binaries. In this way, I don't worry
  about silent data corruptions when copying them around in the future,
  and occasional bit flip in emails becomes non-concerning.

[1] https://serverfault.com/a/77721/158338
[2] https://en.wikipedia.org/wiki/ECC_memory#Research
[3] https://forums.truenas.com/t/ecc-memory-for-home-servers-or-not/7099



-- 
Hong