Both RFC 7914 and the original paper introducing scrypt seem to mandate SHA-256 as hash-function, why did you deviate from that and use Whirlpool or Keccak instead?
I noticed the "#ifdef TEST_SHA256" in the source, is there any release that includes this?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Scrypt does not directly rely on SHA-256. It relies on a PBKDF2-HMAC function, which in turn relies on a hash function, which can be any hash function.
SHA-256 was just given as an example.
"#ifdef TEST_SHA256" is here to check the implementation, because of my own optimizations, and because test vectors were only given for this hash function.
If you want to see something more original, you should rather check XTS. As far as I know, I am the first one to use XTS on a 256-bit block cryptographic function.
If you want to analyze this implementation of XTS, you will also have to keep in mind that SHACAL-2 here uses SSE2 optimizations to encrypt 4 blocks at once.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
While I fully agree that it is technically possible to use a different hash function (or use a completely different PRF instead of HMAC for that matter), the specs for scrypt are pretty clear on HMAC-SHA256 being required, like in the RFC the algorithm starts with
Initialize an array B consisting of p blocks of 128 * r octets
each:
B[0] || B[1] || ... || B[p - 1] =
PBKDF2-HMAC-SHA256 (P, S, 1, p * 128 * r)
Similary, the original paper defines a MFcrypt (which matches your usage), and then defines scrypt as a special case using HMAC-SHA256.
Anyway, it wasn't my intention to start arguing this, I was just curious why you chose a different hash function, was there a specific reason not to use SHA256?
The reason I'm interested is because I could add support for ProxyCrypt to my app Disk Decipher, which would allow accessing ProxyCrypt images on iOS/macOS platform. Most of the crypto stuff is already in there to support other formats like VeraCrypt and LUKS, just not non-standard scrypt 😀
I saw the improved XTS implementation, need some more time analyzing that. SSE2 is not available on Apple platform, but that's easy to workaround.
Last edit: RichardH 2025-10-31
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just a FYI, as you might be interested to know that I decided to add ProxyCrypt support, which will allow accessing the contents of a disk image on iOS and macOS.
Interesting. As said, the most uncommon part is XTS with 256-bit blocks.
If you already have all the required algorithms, this scrypt implementation should be easy to handle.
For the volume header, I don't know what you have in mind but I would suggest to ignore the version 1 (HEADER_STRUCT_v1). I created ProxyCrypt 12 years ago, but the version 2 is 9 years old, so supporting the version 1 seems rather meaningless.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yeah, I already extended my scrypt to support other prf's.
I focused on version 2 indeed, good to know that v1 is not worth the trouble, thanks. If you ever create v3, it might make sense to align the two 64bit counters on 8-byte boundary to make it easier to read these values cross-platform.
My initial release will support AES/Whirlpool only. Other options follow shortly in updates after that.
Proof-of-concept is working nicely, here's a screenshot taken on Windows and iPhone simulator.
I speak of SHACAL-2 in my presentation of ProxyCrypt 2.0 here.
Not really a variant, but data are loaded in little endian format for each 32-bit word. This is very likely the cause of the difference.
By the way, I forgot that I had an archive of SHACAL-2 before my SSE2 optimization. Don't know if it can be helpful for you.
Hi,
Both RFC 7914 and the original paper introducing scrypt seem to mandate SHA-256 as hash-function, why did you deviate from that and use Whirlpool or Keccak instead?
I noticed the "#ifdef TEST_SHA256" in the source, is there any release that includes this?
Scrypt does not directly rely on SHA-256. It relies on a PBKDF2-HMAC function, which in turn relies on a hash function, which can be any hash function.
SHA-256 was just given as an example.
"#ifdef TEST_SHA256" is here to check the implementation, because of my own optimizations, and because test vectors were only given for this hash function.
If you want to see something more original, you should rather check XTS. As far as I know, I am the first one to use XTS on a 256-bit block cryptographic function.
If you want to analyze this implementation of XTS, you will also have to keep in mind that SHACAL-2 here uses SSE2 optimizations to encrypt 4 blocks at once.
While I fully agree that it is technically possible to use a different hash function (or use a completely different PRF instead of HMAC for that matter), the specs for scrypt are pretty clear on HMAC-SHA256 being required, like in the RFC the algorithm starts with
Similary, the original paper defines a MFcrypt (which matches your usage), and then defines scrypt as a special case using HMAC-SHA256.
Anyway, it wasn't my intention to start arguing this, I was just curious why you chose a different hash function, was there a specific reason not to use SHA256?
The reason I'm interested is because I could add support for ProxyCrypt to my app Disk Decipher, which would allow accessing ProxyCrypt images on iOS/macOS platform. Most of the crypto stuff is already in there to support other formats like VeraCrypt and LUKS, just not non-standard scrypt 😀
I saw the improved XTS implementation, need some more time analyzing that. SSE2 is not available on Apple platform, but that's easy to workaround.
Last edit: RichardH 2025-10-31
Just a FYI, as you might be interested to know that I decided to add ProxyCrypt support, which will allow accessing the contents of a disk image on iOS and macOS.
read more
Interesting. As said, the most uncommon part is XTS with 256-bit blocks.
If you already have all the required algorithms, this scrypt implementation should be easy to handle.
For the volume header, I don't know what you have in mind but I would suggest to ignore the version 1 (HEADER_STRUCT_v1). I created ProxyCrypt 12 years ago, but the version 2 is 9 years old, so supporting the version 1 seems rather meaningless.
Yeah, I already extended my scrypt to support other prf's.
I focused on version 2 indeed, good to know that v1 is not worth the trouble, thanks. If you ever create v3, it might make sense to align the two 64bit counters on 8-byte boundary to make it easier to read these values cross-platform.
My initial release will support AES/Whirlpool only. Other options follow shortly in updates after that.
Proof-of-concept is working nicely, here's a screenshot taken on Windows and iPhone simulator.
The XTS with 256-bit blocks stuff is definitely interesting, looking at that now.
Having some difficulty aligning SHACAL-2 output, for example, looking at your test.c
the input matches a known test vector from https://github.com/weidai11/cryptopp/blob/60f81a77e0c9a0e7ffc1ca1bc438ddfa2e43b78e/TestVectors/shacal2.txt#L4
and that is the output my implementation is giving too.
Assuming your 4-blocks-in-one-call should give the same output for the first block, any idea why the two differ? Are you using a variant of SHACAL-2?
I speak of SHACAL-2 in my presentation of ProxyCrypt 2.0 here.
Not really a variant, but data are loaded in little endian format for each 32-bit word. This is very likely the cause of the difference.
By the way, I forgot that I had an archive of SHACAL-2 before my SSE2 optimization. Don't know if it can be helpful for you.
That is extremely helpful, thanks!
Changing big->little endian (both data and keys) makes the two match. 👍