#36 perfomance extremly slow compared with enfs4win

[A. Hertzsch]

Hi,

i started with encrypting 2 folders:
1st folder (54GB) with about 27K Files and 500 subfolders
2nd folder (12GB) with about 33K Files and 6K subfolders

The first encrypting (copied with robocopy) took some hours ... ok anyway...

Then i made a "robocopy /MIR ... " to sync my unencrypted folders with the encrypted ones.
(without changing anything in the files ...!)

For the first folder it took about 40 minutes.
For the second folder it took about 7 hours!!!
(CPU and HDD usage was less the 5%)

It seems to be a big problem when there are many folders...

I could not image that these values are normally.
Thats why i decided to test the encfs4win project (with uptodate Dokan 0.8).

I could mount my existing folder without any problem and made the same test with the 2 folders:

The robocopy /MIR for the 1st folder took exact 23 seconds and for the second one exact 24 seconds.

OK, both projects based on the same enfs 1.7.4 so that seems not to be the problem.

What goes wrong here that i have these extrem performance difference?

moe5k

[RomanH]

I know that the performance is degrading with big folders, especially on network attached storage (mounted folders). The big difference between encfs4win and EncFSMP is the file system component. EncFSMP uses PismoFileMount, while encfs4win uses Dokan. Apparently Dokan has some form of caching (not for file contents, but for directories), while PismoFileMount does not. Of course EncFSMP could implement this caching, but this would mean a lot of work.

[A. Hertzsch]

THX for the fast reply!
Ok, i undestand ..but in my case it isn´t a NAS its a normal USB 3.0 drive.
But its the same also with SATA-drives makes not an big difference.

Isn´t it an option to use the Dokan lib instead of the PismoFileMount to
prevent these perfomance issues? The Dokan lib is maintained quite uptodate...
Not as your write here:
http://encfsmp.sourceforge.net/faq.html

AND: Dokan lib is opensource! So the whole project would be open source.
That would also be nice :)

In all other things i like this project really much more than these not old encfs4win ...but the perfomance is extrem important for me. I thnik its not a big problem if you only use it with a few files/folders or at least not so many folders... but i want to encrypt my backup before uploading to the cloud. So the Backupjob would take the half day...

best regards
moe5k

[ex q]

EncFSMP would be so awesome if the speed would be better. Why not collect donations for the dokan implementation? I would donate for sure.

[RomanH]

At the time when I started to implement EncFSMP, Dokan was a dead project and full of bugs. I don't know about the current state.
It would be a very big undertaking to migrate EncFSMP to Dokan. It would be much easier to use encfs4win and, if there are problems, fix them. I don't intend to do that.
I will think about the possible performance improvements using PismoFileMount.

[A. Hertzsch]

Your project is awesome because i didn´t find any compareable on the net excepting encfs4win
(which ist not maintained and not good configureable) and Boxcryptor (which is commercial and closed source).

But unfortunately with these performance issues is your project not really useable / scalable for everyone. It depends of the count of folders/subfolders if you can use it or not. In my case its not possible to use it because only one of 2 two folders i want to sync needs about 1000 times longer compared with encfs4win. And that´s really a pity. And thats only why you are using the wrong Lib?

Yes, you are right in the past Dokan was not maintained for a while. But now it seems to be
a active developt project. Take a look: https://github.com/dokan-dev/dokany/releases

And, yes you are right too if you say that a lot of work to migrate to Dokan.
In my imagination it both are justs libs with an (maybe different) API. So the effort should be comprehensible.
Think about it! If you want to improve your project than you should go this step.
Not only because of the performance also because of Dokan is opensource!
That should be 2 very good reasons to do that.

BTW: i would also donate for it...

moe5k

[RomanH]

I have improved some parts of the code regarding speed in the newest version. Could you please try 0.10.0 whether the speed in your workflow is improved?

[A. Hertzsch]

I just checked ist with the same directorys:

first with Dokan/encfs4win
dir 1 took exact : 6 seconds
dir 2 took exact : 38 seconds

then with your improved version
dir 1 took exact : 5 seconds
dir 2 took exact : 35 seconds

Its even faster now!
What have you done? What was the problem?
The improvement has definitely paid!

moe5k

[RomanH]

Thanks for the quick feedback!
I mostly optimized the internal data structures. For example the conversion of filenames to openIds was previously a linear list, now a std::map.
I will close the issue now.

[ex q]

Amazing Roman! Really a good job :)