#423 7z extraction "The parameter is incorrect."

[Aiden Clark]

Hi,

Back story:
I'm not sure how much I'm in trouble here but I've used 7zip to split some huge backup files. The backup files range from 6TB to 22TB in size. The goal was to migrate these files to Azure blob storage off Tape storage. The transport method due to needing to move hundreds of TB in files to azure, was to use Azure Databoxes. Azure Databoxes can't be shipped back to azure without validation, and any individual file larger than 5TB will fail validation on the databox. Azure though does support large files. Huge files even. The storage account is created with that in mind and there are already files exceeding 20TB in size.

Note: I've slightly altered the output of filenames in the below to try and hide some identifying information. I don't think this would identify anything about client or business or location, other than identifying myself to my peers so just let me know if the file names need to be back to original format.

The Archive information:
I attempted to use 7zip 21.07 to split files to 4TB splits. This was setup in 7z store mode, no compression. Output files have information like this:

Size: 7 286 238 248 960
Packed Size: 7 286 238 248 960
Folders: 0
Files: 1
CRC: 73FED3D5
------------------------: 
Path: b476e3ee-92a6-4843-a1f0-5cdb11443D2016-12-03T080000-2A67.7z
Type: 7z
Physical Size: 7 286 238 249 210
Headers Size: 250
Method: Copy
Solid: -
Blocks: 1
----------------: 
Path: b476e3ee-92a6-4843-a1f0-5cdb11443D2016-12-03T080000-2A67.7z
Size: 7 286 238 249 210
------------------------: 
Path: V:\BC_KEN_CPA_KENDC_Critical_2016-12-02\b476e3ee-92a6-4843-a1f0-5cdb11443D2016-12-03T080000-2A67.7z.001
Type: Split
Physical Size: 4 398 046 511 104
Volumes: 2
Total Physical Size: 7 286 238 249 210
------------------------: 
------------------------: 

The extraction information:
The files are now in azure and I need to merge the split files by extracting them. The first error line happens even on successful extractions. However, when it gets past the 7z.001 file, and now should start at 7z.002 it stops dead. It processes the first 4096GB (4TB) and then throws the next 3 errors (error line 2-4)

1     Cannot set length for output file : The parameter is incorrect. : V:\BC_KEN_CPA_KENDC_Critical\(b476e3ee-92a6-4843-a1f0-5cdb11443D2020-12-30T080000_4ED9_Y.vbk
2      V:\BC_KEN_CPA_KENDC_Critical\(b476e3ee-92a6-4843-a1f0-5cdb11443D2020-12-30T080000_4ED9_Y.7z.001
3     (b476e3ee-92a6-4843-a1f0-5cdb11443D2020-12-30T080000_4ED9_Y.vbk
4     The parameter is incorrect.

I've attached a screenshot of the file explorer too since that'll show you the folder structure.

The only thing interesting with the 7z.002 file in this case is the date modified is invalid, Windows writes it's 1/1/1900 at 10:00am (which i'm australian, +10GMT so it'd be 00:00 in GMT).

I'm retrying the extraction in command line right now which will take days to fail (another problem with testing things since it will take so long before an error).

I'd appreciate any advice right now. I'm sure you know how I feel when it comes to potentially hundreds of TB of data loss (even though it's a backup copy, the storage costs + time loss is incredibly high).

[Igor Pavlov]

What is V:\?
Is it usual local ntfs volume, or something else like network volume?
Try same steps for smaller files:
2 GB
20 GB
200 GB
2000 GB
That way you can find exact size point when it doesn't work.

[Aiden Clark]

V is an azure file storage mount over SMB.

I don't have an easy way quite to test what you're asking for, but up to below 4TB it works. My testing in the last few hours has been on azure file storage limitation may be the root cause after all. I've gone and reviewed what's where in my azure storage - my existing backups are in azure blob containers, not azure file store. There's a long story here of why i'm where i am, but I believe this problem is likely https://docs.microsoft.com/en-us/azure/storage/files/storage-files-scale-targets#file-scale-targets which reports that 4TiB file scale targets.

Based on that I think 7z is fine - and it's actually the azure file storage limitation which is the issue.

I'm now trying to mount blob storage via rclone..

[Aiden Clark]

I just wanted to close this off and say that I'm using a different method: I've mounted a Azure Blob Container via rclone and 7zip was able to successfully extract the large split 7z files.
Thanks for the response Igor, appreciate the project and the utility. It's been so critical to many things in my IT career that it's become a foundational go-to tool.