Unfortunately I'am not allowed to create a ticket here. So I have to ask here for support on a very annoying issue:
When trying to format a LUKS encrypted partition on a block device which was zeroed you'll always get an alarming warning:
# mkfs.ext4 /dev/mapper/crypt
/dev/mapper/crypt contains `OpenPGP Public Key' data
Proceed anyway? (y,N) n
As the device was zeroed there could / should not be any "'OpenPGP Public Key' data" left on the device!
After long STFW I found a statement:
"It turns out, that mke2fs uses libmagic to check the formatted device for known data signatures. It just so happens, that mapping zeroed file using 'password' as a key, result in the first two bytes being '9a9d', which libmagic recognizes as PGP Public Key."
Please stop mkfs.ext4 giving such wrong warnings about non existent 'OpenPGP Public Key' data!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
So this makes no sense. The way LUKS works is that when you create a LUKS encrypted partition using "crypsetup luksFormat", it first creates a random key for the master key. The master key is what is actually used to encrypt the data block. The passphrase, such as the ill-chosen "password" is used to encrypt the master key. So each time you use "cryptsetup luksFormat" you will get a different master key.
As a result, I was not surprised when I could not duplicate your result:
As you can see when I ran mke2fs a second time, it complained about the fact that /dev/mapper/test has an ext4 file syustem. But it didn't complain the first time, even though I started with a zero'ed block device, with a freshly created LUKS encrypted partition --- even when using a really REALLY bad password choice such as "password".
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ah, I see, I was able to find what I suspect you were referencing, which was apprently this web page.. Is that what you found?
First of all, this example was not using a "LUKS encrypted partition". This example was using a plain encrypted password, which is a REALLY BAD IDEA. Simple, but still, a bit of a disaster. The problem is that it allows you to use a low-entry password (an eight character, all lower-case password is stupid thing to do, even if it wasn't "password"), which makes it trivially easy for an adversary to brute force.
Fortunately, that's not what a LUKS encrypted partition does. With LUKS, as I described above, it generates a completely random master key, and the master key is encrypted by the user's passphrase. This means that after chosing a terminally stupid password like "password", you can repent of the idea, and change the password to something strong, without needing to decrypt and re-encrypt the entire partition. You can also add additional passwords; for example, in an enterprise environment, the I/T staff might add a second passphrase which is the "recovery" passphrase which is stored in a password vault. So when (not if) the user forgets their password, they can go to the I/T staff, and tell them, "I did a stupid thing", and the I/T staff can pull the recovery password out of their password vault, and use that to set a new passphrase to overwrite the user's forgotten passphrase.
Buttom line, don't use type "plain". It's a bad idea. Use the LUKSv1 or LUKSv2 format, since it will save you from a lot of stupid mistakes. And of course, don't use a stupidly bad passphrase like "password".
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello and thank you for your relply.
Of course I would not use 'password' as a password LOL
In my case I used a ten digit random password generated with 'pwgen -syn' and no keyfile.
The web page (yes, it was the one I found) was the only hint referencing this problem.
In that example a plain encrypted password was used but I used a LUKS encrypted partition when I stumbled across this weird issue.
As it was not the first time seenig this warning about alleged 'OpenPGP Public Key' data when trying to format an encrypted partition I thought I could be a bug.
Today I tried to reproduce it - and of course I was not able to.
If I will see this warning again I'll report back.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Well, if libmagic is checking to see if the first two bytes is "9a9d", then assuming that the encryption algorithm is a random function (which is a good assumption to the first approximation, given a randomly chosen key), the chances that you will see that warning again is 1 in 65,536, or 0.0015%. So I'm really not going to worry about it....
Please note that it's always possible that someone might have stored an OpenPGP Public Key using "dd if=$HOME/my-firiends-key.gpg of=/dev/sdXX", and maybe they really would care if it was overwritten by mke2fs. :-)
Anyway, we use libmagic because it's convenient, and users really really hate it when they accidentally mistype a block device name and destroy a huge amount of data, which according to Murphy's law, willl happen when they haven't bothered to do regular backups; (Of course, YOU do regular backups all the time, RIGHT? :-)
The occasional, very rare libmagic false positive seems better than the alternatives, and so far, you're the only one who has complained; and I certainly don't think it's worth the inflamtatory "sensless wrong and irritating warings" [sic] description that you used in this thread.
In any case, if you try to crate and recreate new LUKS encrypted partitions over and over again, over thousands or tens of thousands of tries, you will no doubt eventually get that libmagic warning again --- or you might get some other false positive about it being a jpg file, or a mp4 fille. Unfortunately, libmagic's descriminators are not perfect, and sometimes that's a shortcoming in libmagic's data file, and sometimes it's just a shortcoming in the the format. For example, see this bug report about Korean text files having a 4% chance of being misidentified as a DOS .COM file. Unfortunately, given the highly simplistic way .COM files are encoded, it's hard to find a good fix for this, and so while this bug was filed in 2020, it's still unfixed, because sometimes, there isn't a good fix, and as software engineers, sometimes we have to weigh tradeoffs being type 1 and type 2 errors, and what the consequences might be of those errors.
If you would like to contribute to the libmagic community about improving their detection algorithms, feel free. But please understand that this is a hard problem.
Last edit: Theodore Ts'o 2023-06-20
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Unfortunately I'am not allowed to create a ticket here. So I have to ask here for support on a very annoying issue:
When trying to format a LUKS encrypted partition on a block device which was zeroed you'll always get an alarming warning:
As the device was zeroed there could / should not be any "'OpenPGP Public Key' data" left on the device!
After long STFW I found a statement:
Please stop mkfs.ext4 giving such wrong warnings about non existent 'OpenPGP Public Key' data!
So this makes no sense. The way LUKS works is that when you create a LUKS encrypted partition using "crypsetup luksFormat", it first creates a random key for the master key. The master key is what is actually used to encrypt the data block. The passphrase, such as the ill-chosen "password" is used to encrypt the master key. So each time you use "cryptsetup luksFormat" you will get a different master key.
As a result, I was not surprised when I could not duplicate your result:
As you can see when I ran mke2fs a second time, it complained about the fact that /dev/mapper/test has an ext4 file syustem. But it didn't complain the first time, even though I started with a zero'ed block device, with a freshly created LUKS encrypted partition --- even when using a really REALLY bad password choice such as "password".
Ah, I see, I was able to find what I suspect you were referencing, which was apprently this web page.. Is that what you found?
First of all, this example was not using a "LUKS encrypted partition". This example was using a plain encrypted password, which is a REALLY BAD IDEA. Simple, but still, a bit of a disaster. The problem is that it allows you to use a low-entry password (an eight character, all lower-case password is stupid thing to do, even if it wasn't "password"), which makes it trivially easy for an adversary to brute force.
Fortunately, that's not what a LUKS encrypted partition does. With LUKS, as I described above, it generates a completely random master key, and the master key is encrypted by the user's passphrase. This means that after chosing a terminally stupid password like "password", you can repent of the idea, and change the password to something strong, without needing to decrypt and re-encrypt the entire partition. You can also add additional passwords; for example, in an enterprise environment, the I/T staff might add a second passphrase which is the "recovery" passphrase which is stored in a password vault. So when (not if) the user forgets their password, they can go to the I/T staff, and tell them, "I did a stupid thing", and the I/T staff can pull the recovery password out of their password vault, and use that to set a new passphrase to overwrite the user's forgotten passphrase.
Buttom line, don't use type "plain". It's a bad idea. Use the LUKSv1 or LUKSv2 format, since it will save you from a lot of stupid mistakes. And of course, don't use a stupidly bad passphrase like "password".
Hello and thank you for your relply.
Of course I would not use 'password' as a password LOL
In my case I used a ten digit random password generated with 'pwgen -syn' and no keyfile.
The web page (yes, it was the one I found) was the only hint referencing this problem.
In that example a plain encrypted password was used but I used a LUKS encrypted partition when I stumbled across this weird issue.
As it was not the first time seenig this warning about alleged 'OpenPGP Public Key' data when trying to format an encrypted partition I thought I could be a bug.
Today I tried to reproduce it - and of course I was not able to.
If I will see this warning again I'll report back.
Well, if libmagic is checking to see if the first two bytes is "9a9d", then assuming that the encryption algorithm is a random function (which is a good assumption to the first approximation, given a randomly chosen key), the chances that you will see that warning again is 1 in 65,536, or 0.0015%. So I'm really not going to worry about it....
Please note that it's always possible that someone might have stored an OpenPGP Public Key using "dd if=$HOME/my-firiends-key.gpg of=/dev/sdXX", and maybe they really would care if it was overwritten by mke2fs. :-)
Anyway, we use libmagic because it's convenient, and users really really hate it when they accidentally mistype a block device name and destroy a huge amount of data, which according to Murphy's law, willl happen when they haven't bothered to do regular backups; (Of course, YOU do regular backups all the time, RIGHT? :-)
The occasional, very rare libmagic false positive seems better than the alternatives, and so far, you're the only one who has complained; and I certainly don't think it's worth the inflamtatory "sensless wrong and irritating warings" [sic] description that you used in this thread.
In any case, if you try to crate and recreate new LUKS encrypted partitions over and over again, over thousands or tens of thousands of tries, you will no doubt eventually get that libmagic warning again --- or you might get some other false positive about it being a jpg file, or a mp4 fille. Unfortunately, libmagic's descriminators are not perfect, and sometimes that's a shortcoming in libmagic's data file, and sometimes it's just a shortcoming in the the format. For example, see this bug report about Korean text files having a 4% chance of being misidentified as a DOS .COM file. Unfortunately, given the highly simplistic way .COM files are encoded, it's hard to find a good fix for this, and so while this bug was filed in 2020, it's still unfixed, because sometimes, there isn't a good fix, and as software engineers, sometimes we have to weigh tradeoffs being type 1 and type 2 errors, and what the consequences might be of those errors.
If you would like to contribute to the libmagic community about improving their detection algorithms, feel free. But please understand that this is a hard problem.
Last edit: Theodore Ts'o 2023-06-20