Activity for Michael Lackner
-
Michael Lackner posted a comment on discussion Open Discussion
Hello gianluca, Yes, of course, I've attached it for you. Note that in the meantime I found out a few things though: The parallel load will not go up immediately, it'll take some time for it to do so. Also, from time to time, load will drop to just a few cores for a while, only to ramp it back up later on. So if you do see unexpectedly low loads across your cores with those settings, give it an hour or so, then check again.
-
Here's an update. The difference is mind-blowing! I didn't think it would have such a massive effect. I've attached a screenshot showing off the current core load & speed! It's sitting at 33 - 34 MiB/s with the maximum solid block size of 64GB. 24 physical cores are well-loaded here. I guess this setting will reduce decompression speed for individual files if I understand it correctly, but as said, that's fine. Thank you!
-
Currently, I'm using the GUI, but CLI is also fine. In my experience, smaller dictionaries result in sometimes significantly larger archives when compressing large data sets. So I want the largest dictionary possible to reach the smallest possible archive size. This is the main goal. Making things more parallel / faster is the secondary goal. I've never even considered changing the solid block size (just read documentation about how this works). I'll try enlarging it, thanks!
-
Sam, I just compiled lzip, lzlib and tarlz on my Linux box to play around with it a bit, but the biggest problem with this is that 7-Zip can't unpack its archives on Windows. It seemingly has no decompressor for lzip. While I do have several CygWin installations set up already, I'm not going to compile+link a tarlz redistributable for people to use for this, as we agreed on 7-Zip for decompression (because most people have it installed on Windows). So my question regarding any possibilities for parallelizing...
-
Thank you for your reply! One reason why I wanted to keep it in one archive and not split it is that by keeping the data set large, the payoff of using block-level deduplication would be greater, especially with a monstrous dictionary / dedup hash table in RAM. I'm assuming (based on my previous tests with gzip, bzip2 and LZMA2 in xz and 7z) that LZMA2 is simply, generally "the best" for compressing rather "noisy" data. Like binary programs and stuff. Naturally, I could look at the data and identify...
-
Thank you for your reply! One reason why I wanted to keep it in one archive and not split it is that by keeping the data set large, the payoff of using block-level deduplication would be greater, especially with a monstrous dictionary / dedup hash table in RAM. I'm assuming (based on my previous tests with gzip, bzip2 and LZMA2 in xz and 7z) that LZMA2 is simply, generally "the best" for compressing rather "noisy" data. Like binary programs and stuff. Naturally, I could look at the data and identify...
-
Fast random access isn't necessary. Being able to find, access and extract individual files (primarily on Microsoft Windows) is however, but if it takes e.g. 1-2 hours to find and extract an individual file, then that's okay. If it matters, the data set consists of 753047 files distributed over 90278 folders. The actual size of uncompressed data is 13381 GiB.
-
Hello, I'd like to compress a large data archive (HPE FTP mirror) with 7-Zip 22.01 for permanent storage. I would like to do this as efficiently as possible, meaning I'm targeting minimum archive size. However, since the data set is so large (over 13 TiB), I'd like to do it in a reasonable time frame, not multiple months as it would be on my local workstation with just 2-3 threads, being limited to 48 GiB of RAM. So I'm using a much larger machine. Hardware/Software: • Kernel Virtual Machine (RHV-H/RHV-M)...
-
Hello, I'd like to compress a large data archive (HPE FTP mirror) with 7-Zip 22.01 for permanent storage. I would like to do this as efficiently as possible, meaning I'm targeting minimum archive size. However, since the data set is so large (over 13 TiB), I'd like to do it in a reasonable time frame, not multiple months as it would be on my local workstation with just 2-3 threads, being limited to 48 GiB of RAM. So I'm using a much larger machine. Hardware/Software: • Kernel Virtual Machine (RHV-H/RHV-M)...
-
Hello, I'd like to compress a large data archive (HPE FTP mirror) with 7-Zip 22.01 for permanent storage. I would like to do this as efficiently as possible, meaning I'm targeting minimum archive size. However, since the data set is so large (over 13 TiB), I'd like to do it in a reasonable time frame, not multiple months as it would be on my local workstation with just 2-3 threads, being limited to 48 GiB of RAM. So I'm using a much larger machine. Hardware/Software: • Kernel Virtual Machine (RHV-H/RHV-M)...
-
Hello, I'd like to compress a large data archive (HPE FTP mirror) with 7-Zip 22.01 for permanent storage. I would like to do this as efficiently as possible, meaning I'm targeting minimum archive size. However, since the data set is so large (over 13 TiB), I'd like to do it in a reasonable time frame, not multiple months as it would be on my local workstation with just 2-3 threads, being limited to 48 GiB of RAM. So I'm using a much larger machine. Hardware/Software: * Kernel Virtual Machine (RHV-H/RHV-M)...
-
Hello, I'd like to compress a large data archive (HPE FTP mirror) with 7-Zip 22.01 for permanent storage. I would like to do this as efficiently as possible, meaning I'm targeting minimum archive size. However, since the data set is so large (over 13 TiB), I'd like to do it in a reasonable time frame, not multiple months as it would be on my local workstation with just 2-3 threads, being limited to 48 GiB of RAM. So I'm using a much larger machine. Hardware/Software: * Kernel Virtual Machine (RHV-H/RHV-M)...
-
Support for larger LZMA2 dictionary sizes
-
Because of this unclear situation, I've built my current RAID-6 array with 512e disks, but aligning everything properly feels like rocket science to me considering the stack... 512e with 4kiB sectors underneath, GPT offset, size/offset of the VeraCrypt header (is it larger on 4Kn disks than it is in 512n/512e disks?), RAID strip size, etc. Booting might be even trickier, it's only theoretically possible with 4Kn on BIOS+MBR setups, but I doubt BIOSes and bootloaders would be implemented intelligently...
-
Because of this unclear situation, I've built my current RAID-6 array with 512e disks, but aligning everything properly feels like rocket science to me considering the stack... 512e with 4kiB sectors underneath, GPT offset, size/offset of the VeraCrypt header (is it larger on 4Kn disks than it is in 512n/512e disks?), RAID strip size, etc. Booting might be even trickier, it's only theoretically possible with 4Kn on BIOS+MBR setups, but I doubt BIOSes and bootloaders would be implemented intelligently...
-
Because of this unclear situation, I've built my current RAID-6 array with 512e disks, but aligning everything properly feels like rocket science to me considering the stack... 512e with 4kiB sectors underneath, GPT offset, size/offset of the VeraCrypt header (is it larger on 4Kn disks than it is in 512n/512e disks?), RAID strip size, etc. Booting might be even trickier, it's only theoretically possible with 4Kn on BIOS+MBR setups, but I doubt BIOSes and bootloaders would be implemented intelligently...
-
Because of this unclear situation, I've built my current RAID-6 array with 512e disks, but aligning everything properly feels like rocket science to me considering the stack... 512e with 4kiB sectors underneath, GPT offset, size/offset of the VeraCrypt header (is it larger on 4Kn disks than it is in 512n/512e disks?), RAID strip size, etc. It would be really nice to have this clearly documented, and of course supported in the first place. 4Kn drives have been around for more than half a decade (!)...
-
Alright, thanks. I just checked my mail database, and the developer who has responded to my original query about 4Kn support several years ago was Mounir Idrassi as well. Meanwhile I've sent them an eMail to their official address as well (before reading your reply), so I'll wait for a response for a while. If I receive nothing for 2-3 weeks, I'll try to email him directly, as I still have his address.
-
And (almost) another year since the last push, so I'll allow myself to push this again, as it still remains unanswered. Thanks.
-
4Kn Support
-
Please accept my apologies for pushing this, but I'm still looking for this information, and I still don't understand the source code enough. ;) Thanks!
-
I would like to ask whether the current version of VeraCrypt supports 4Kn hard drives (disks with native 4096 byte sectors). I have only found [this thread], which remained unanswered though. I know people have tried this already, and I myself managed to make it work even with TrueCrypt. But years ago, a VeraCrypt supports person (or maybe developer?) told me that this is not expected to work, and may cause problems. According to that person, parts of the source code were hard-wired for 512 byte...
-
I managed do compile the current Password Safe 3.43.0 with a v140_xp platform target using VS2015 by removing a few simple Windows 7 specific parts regarding the determining of the tray icon background color in src\ui\Windows\SetDBID.cpp. The offending call was Shell_NotifyIconGetRect(). I did that for both 32-bit XP (NT5.1) as well as the 64-bit x86 version (NT 5.2). It does launch on XP and XP x64 now, but it can't even create or open a Safe file, it crashes immediately. Would be nice to get a...
-
I managed do compile the current Password Safe 3.43.0 with a v140_xp platform target using VS2015 by removing a few simple Windows 7 specific parts regarding the determining of the tray icon background color in src\ui\Windows\SetDBID.cpp. I did that for both 32-bit XP (NT5.1) as well as the 64-bit x86 version (NT 5.2). It does launch on XP and XP x64 now, but it can't even create or open a Safe file, it crashes immediately. Would be nice to get a diff between 3.39.0 and its XP version (source, project...
-
Actually, the last release of the "abandoned" Miranda IM was exactly one week ago, so there is still some ongoing development. I would also like Miranda IM to support longer passwords. I don't have a build toolchain ready to compile it by myself, but it should be an easy fix (I just fixed licq today to support this feature, and it was trivial). My personal issue is that Miranda IM has at least one plugin that doesn't have a replacement in Miranda NG (CactusSRAMM). Unless somebody ports that over,...
-
Ah, thanks for that! I've also found two more interesting things. 1.) Some windows...
-
This is the single best solution I've ever seen for multiple desktops on Windows....
-
This is the single best solution I've ever seen for multiple desktops on Windows....
1