#11 Something wrong with the parameters

[Jouni Järvinen]

I need to compress a total of 102 591 files (not counting folders) at 285GB and 7z would be a great thing for that, but for some reason exceeding 9 files and folders either directly given as parameters or indirectly through pointing at a folder with lots of files/folders makes the program choke with error System error: E_INVALIDARG when using the command I painstakingly constructed from the HTML help files.

I tested with success by reducing the command to 7z a home_filtered.7z home. However, this is absolutely inadequate cuz the program doesn't use nearly its full power.

Here are the tests I did to confirm 9 files/folders at once max with the given command:

A subset of files (fail):

rautamiekka@menet:~/rauta-backups_/home$ ionice -c 2 -n 7 nice -n 20 taskset -c 6,7 7z a -t7z -mx=9 -ms=on -mf=on -mhc=on -mhe=off -m0=LZMA2 -ma=1 -md=1536m -mmf=bt4 -mfb=273 -mmc=1000000000 -mlc=3 -mlp=1 -mpb=4 -mmt=2 -mtc=off home_filtered.7z "database, AdminNote_Notes 2016-07-31.tsv" "database, authme 2016-07-31.tsv" "database, authme-no-email 2016-07-31.tsv" "database, bm_player_ban_records 2016-07-31.tsv" "database, bm_player_bans 2016-07-31.tsv" "database, bm_player_bans BAD 2016-07-31.tsv" "database, bm_player_history 2016-07-31.tsv" "database, bm_players 2016-07-31.tsv" "database, lb-changes 2016-07-31.tsv" "database, lb-CreativePlots 2016-07-31.tsv" "database, lb-CreativePlots-chest 2016-07-31.tsv" "database, lb-CreativePlots-sign 2016-07-31.tsv" "database, lb-Factions 2016-07-31.tsv" "database, lb-Factions-chest 2016-07-31.tsv" "database, lb-HUB 2016-07-31.tsv" "database, lb-HUB-sign 2016-07-31.tsv" "database, lb-Market 2016-07-31.tsv" "database, lb-players 2016-07-31.tsv" "database, lb-players-highest 2016-07-31.tsv" "database, lb-players-total-onlinetime 2016-07-31.tsv" "database, lb-PVP 2016-07-31.tsv" "database, lb-PVP-kills 2016-07-31.tsv" "database, lb-Survival 2016-07-31.tsv" "database, lb-Survival-chest 2016-07-31.tsv" "database, lb-Survival-kills 2016-07-31.tsv" "database, lb-Survival_nether 2016-07-31.tsv" "database, lb-Survival_nether-chest 2016-07-31.tsv" "database, lb-Survival_nether-kills 2016-07-31.tsv" "database, lb-Survival_nether-sign 2016-07-31.tsv" "database, lb-Survival-sign 2016-07-31.tsv" "database, lb-Survival_the_end 2016-07-31.tsv" "database, lb-Survival_the_end-chest 2016-07-31.tsv" "database, lb-Survival_the_end-kills 2016-07-31.tsv" "database, lb-Survival_the_end-sign 2016-07-31.tsv" "database, lwc_history 2016-07-31.tsv" "database, lwc_internal 2016-07-31.tsv" "database, lwc_protections 2016-07-31.tsv" "database, plotmeAllowed 2016-07-31.tsv" "database, plotmeComments 2016-07-31.tsv" "database, plotmeDenied 2016-07-31.tsv" "database, plotmePlots 2016-07-31.tsv" "database, SHT_Tickets 2016-07-31.tsv" "minecraft_we_schema.tar" "server.log.xz"

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)
Scanning

Creating archive home_filtered.7z

System error:
E_INVALIDARG

A single file (success):

rautamiekka@menet:~/rauta-backups_/home$ ionice -c 2 -n 7 nice -n 20 taskset -c 6,7 7z a -t7z -mx=9 -ms=on -mf=on -mhc=on -mhe=off -m0=LZMA2 -ma=1 -md=1536m -mmf=bt4 -mfb=273 -mmc=1000000000 -mlc=3 -mlp=1 -mpb=4 -mmt=2 -mtc=off home_filtered.7z "database, AdminNote_Notes 2016-07-31.tsv"

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)
Scanning

Creating archive home_filtered.7z

Compressing  database, AdminNote_Notes 2016-07-31.tsv

Everything is Ok
rautamiekka@menet:~/rauta-backups_/home$ rm home_filtered.7z`

10 files (fail):
`rautamiekka@menet:~/rauta-backups_/home$ ionice -c 2 -n 7 nice -n 20 taskset -c 6,7 7z a -t7z -mx=9 -ms=on -mf=on -mhc=on -mhe=off -m0=LZMA2 -ma=1 -md=1536m -mmf=bt4 -mfb=273 -mmc=1000000000 -mlc=3 -mlp=1 -mpb=4 -mmt=2 -mtc=off home_filtered.7z "database, AdminNote_Notes 2016-07-31.tsv" "database, authme 2016-07-31.tsv" "database, authme-no-email 2016-07-31.tsv" "database, bm_player_ban_records 2016-07-31.tsv" "database, bm_player_bans 2016-07-31.tsv" "database, bm_player_bans BAD 2016-07-31.tsv" "database, bm_player_history 2016-07-31.tsv" "database, bm_players 2016-07-31.tsv" "database, lb-changes 2016-07-31.tsv" "database, lb-CreativePlots 2016-07-31.tsv" "database, lb-CreativePlots-chest 2016-07-31.tsv" "database, lb-CreativePlots-sign 2016-07-31.tsv"

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)
Scanning

Creating archive home_filtered.7z

System error:
E_INVALIDARG

11 files (fail):

rautamiekka@menet:~/rauta-backups_/home$ ionice -c 2 -n 7 nice -n 20 taskset -c 6,7 7z a -t7z -mx=9 -ms=on -mf=on -mhc=on -mhe=off -m0=LZMA2 -ma=1 -md=1536m -mmf=bt4 -mfb=273 -mmc=1000000000 -mlc=3 -mlp=1 -mpb=4 -mmt=2 -mtc=off home_filtered.7z "database, AdminNote_Notes 2016-07-31.tsv" "database, authme 2016-07-31.tsv" "database, authme-no-email 2016-07-31.tsv" "database, bm_player_ban_records 2016-07-31.tsv" "database, bm_player_bans 2016-07-31.tsv" "database, bm_player_bans BAD 2016-07-31.tsv" "database, bm_player_history 2016-07-31.tsv" "database, bm_players 2016-07-31.tsv" "database, lb-changes 2016-07-31.tsv" "database, lb-CreativePlots 2016-07-31.tsv" "database, lb-CreativePlots-chest 2016-07-31.tsv"

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)
Scanning

Creating archive home_filtered.7z

System error:
E_INVALIDARG

9 files (success)

rautamiekka@menet:~/rauta-backups_/home$ ionice -c 2 -n 7 nice -n 20 taskset -c 6,7 7z a -t7z -mx=9 -ms=on -mf=on -mhc=on -mhe=off -m0=LZMA2 -ma=1 -md=1536m -mmf=bt4 -mfb=273 -mmc=1000000000 -mlc=3 -mlp=1 -mpb=4 -mmt=2 -mtc=off home_filtered.7z "database, AdminNote_Notes 2016-07-31.tsv" "database, authme 2016-07-31.tsv" "database, authme-no-email 2016-07-31.tsv" "database, bm_player_ban_records 2016-07-31.tsv" "database, bm_player_bans 2016-07-31.tsv" "database, bm_player_bans BAD 2016-07-31.tsv" "database, bm_player_history 2016-07-31.tsv" "database, bm_players 2016-07-31.tsv" "database, lb-changes 2016-07-31.tsv"

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)
Scanning

Creating archive home_filtered.7z

Compressing  database, AdminNote_Notes 2016-07-31.tsv
Compressing  database, authme 2016-07-31.tsv

Break signaled

[Igor Pavlov]

you can use list file fieature instead.

[Jouni Järvinen]

If I understand the manpage correctly, that's the -i@XXX where XXX is the name of the list ?

[Igor Pavlov]

Yes, and each new name at new line in listfile

[Jouni Järvinen]

It's giving the exact same error after taking a sizable time before saying 'Scanning', regardless of whether each line starts and ends with a double quote:

rautamiekka@menet:~/rauta-backups_$ ionice -c 2 -n 7 nice -n 20 taskset -c 6,7 7z a -t7z -mx=9 -ms=on -mf=on -mhc=on -mhe=off -m0=LZMA2 -ma=1 -md=1536m -mmf=bt4 -mfb=273 -mmc=1000000000 -mlc=3 -mlp=1 -mpb=4 -mmt=2 -mtc=off -i@home.lst home_filtered.7z

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)
Scanning

Creating archive home_filtered.7z

System error:
E_INVALIDARG

I created the file with find home -type f -print > home.lst.

For clarity: there's a folder named home under ~/rauta-backups_/, so that's not a typo.

[Igor Pavlov]

if something doesn't work, try to simplify situation:
start with simple command:

7z a a.7z 1.txt

if it works then add one switch or additional file:

7z a a.7z 1.txt -mx9

and so on.
Then you will see the problem, when it doesn't work.

[Jouni Järvinen]

I fixed the command, it was butchered nicely:

ionice -c 2 -n 7 nice -n 20 taskset -c 6-7 screen 7z a -t7z -mx=9 -ms=on -mhc=on -mhe=off -m0=LZMA2:a=1:d=1000m:mf=bt4:fb=273:mc=1000000000:lc=3:lp=1:pb=4 -mmt=2 -mtc=off home_filtered.7z home

Works fine with the file list and the folder name.

[Igor Pavlov]

There is no filelist in that latest command.

[Jouni Järvinen]

I did test it and works fine, just takes way longer.