KeePass / Bugs / #1468 Opening/saving database slow on Linux

bkraptor - 2016-01-18

The report above refers to KeePass 2.x, I've tested this versions 2.29-2.31 on Ubuntu. Prior to this I was only using Windows, so I can not compare to older versions.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul - 2016-01-19

Test one of the Linux / browser ports to compare the speed. If it's similar the issue is with Linux, if it's much faster the issue is likely to be mono.
http://keepassx.sourceforge.net/
https://antelle.github.io/keeweb/

cheers, Paul

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

bkraptor - 2016-01-19

I tested the same (test) database with KeeWeb and Ubuntu 14.04 x64 with Mono 4.2.1.102 and got the following times when opening the database:

KeeWeb v0.5.1: 2.3 seconds
KeePass 2.31 on Ubuntu + Mono: 19.2 seconds

So it seems that Mono is slow for some reason.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dominik Reichl - 2016-01-19

status: open --> closed

Group: KeePass_1.x --> KeePass_2.x
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dominik Reichl - 2016-01-19

On Windows, for the master key transformation KeePass uses the AES implementation available through Microsoft's CNG (BCrypt functions). Apparently this is a lot faster than Mono's AES implementation.

Best regards,
Dominik

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Michael Coote - 2016-11-16

Has anyone found a solution for speeding the key transformations in Ubuntu? I'm running
mono --version
Mono JIT compiler version 4.2.3 (Stable 4.2.3.4/832de4b Wed Mar 16 13:19:08 UTC 2016)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
TLS: __thread
SIGSEGV: altstack
Notifications: epoll
Architecture: amd64
Disabled: none
Misc: softdebug
LLVM: supported, not enabled.
GC: sgen

Keepass 2.34

Thanks

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

anadorr - 2017-07-11

Dominik, can we use AesCryptoServiceProvider instead of RijndaelManaged for key transformations when native is not available ? AesCryptoServiceProvider is at least 3 time faster than RijndaelManaged on my machine.
Keepass uses a fixed key size of 256 bits and ECB mode for transform, both are supported by AesCryptoServiceProvider.

I created a test to verify, using code from TransformKeyManaged (AesKdf.cs): https://gist.github.com/Anadorr/fa1911250ebeacdeb1f8646571aaa1b7

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dominik Reichl - 2017-07-11

KeePass is designed to be compatible with .NET 2.0 and higher. AesCryptoServiceProvider requires at least .NET 3.5, thus it cannot be used directly.

However, I could imagine getting the constructor and creating an instance of AesCryptoServiceProvider dynamically, when KeePass runs under .NET 3.5 or higher. I'll experiment with it when I have a bit more time.

Best regards,
Dominik

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dominik Reichl - 2017-08-27

I've implemented this now. On Unix-like systems, KeePass now uses the CSP implementation of the AES algorithm.

Here's the latest development snapshot for testing:
http://keepass.info/filepool/KeePass_170827.zip

Thanks and best regards,
Dominik

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Kurt Fitzner - 2017-09-10

There is almost a 2.5 fold difference in performance between the version shipping with Linux Mint (2.32 DEV) and the one posted above (it calls itself 2.36 DEV 170827).

On a 50 million iteration database in Linux:
2.32: 61s (820k rounds per second)
170827: 27s (1.8M rounds per second)

It's a lot better, but still a long ways behind native code:
KeepassX: 1.3s (38M rounds per second)

So it is still behind native code by a factor of 20.

I avoid dot net like the plague myself (for pretty much the very reason we see here), but I'm pretty sure it can call out to native code. Could it not use the AES primitives that are in the Linux kernel? If you are dymanically instantiating an AES primitive any way, how about calling

Also, I see that this issue is closed - while some great things have been done with that 2.5 fold speed increase, being behind native code by a factor of 20 means that the attackers have a X 20 advantage in brute forcing my database. I am personally willing to accept 30 seconds to open the database, but those 30 seconds should represent security. I would ask this issue to be reopened.

Last edit: Kurt Fitzner 2017-09-10

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul - 2017-09-10

Closed doesn't mean it's forgotten, as you can see from the posts in this thread.

cheers, Paul

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dominik Reichl - 2017-09-12

I've now implemented another improvement: if the library 'libgcrypt.so.20' is available, KeePass uses it for AES-KDF transformations. LibGCrypt is even faster than Mono's AesCryptoServiceProvider implementation, and the library seems to be installed by default on most modern Linux systems (I checked Kubuntu 17.04, Ubuntu 16.10 and Mint 18.2).

Here's the latest development snapshot for testing:
http://keepass.info/filepool/KeePass_170912.zip

Best regards,
Dominik

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Kurt Fitzner - 2017-09-12

This is fantastic! I just clocked 19.4M rounds per second. It couldn't come at a better time either.

Yes, LibGCrypt will be available by default on all Debian-based Linux system since their package manager uses GnuPG to verify their package keys. I don't know if all RPM-based systems have it by default (I haven't used one in a very long time), but even if it's not by default, I'm sure most systems will have GnuPG installed for one reason or another anyway.

Looking at the speed it's getting, I'm pretty sure LibGCrypt is using the Linux kernel crypto API for AES hardware assist. I don't think you can reasonably expect to get any faster.

Thank-you so much for your work on this.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Kurt Fitzner - 2020-08-06

I just updated to Linux Mint 20 with KeePass 2.44 and whatever change was made in KeePass 2 back then does not seem to be in it now. I have verified that libgcrypt.so,20 is available. Are there any other requirements now?

EDIT: I just ran the original test cversion you made back in 2017, and verified it no longer has the speed improvement either. This may be an issue with libgcrypt.so.20. The version that was on my old (working) installation was libgcrypt.so.20.2.1. On the upgrade is libgcrypt.so.20.2.5. Could it have, for some reason, been compiled without AES-NI support?

Last edit: Kurt Fitzner 2020-08-06

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul - 2020-08-07

Was is working before you updated?
What did you update from?

cheers, Paul

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Kurt Fitzner - 2020-08-07
  
  Hi Paul,
  
  Here is the situation: I have several almost identical Linux Mint systems. Most of them are running Linux Mint 19.3. One of them was upgraded to Linux Mint 20. KeePass's native AES-KDF is now no longer working on Linux Mint 20. It takes 22 seconds to open up a database on Linux Mint 20, and 2.5 seconds in Linux Mint 19.3 for the same database.
  
  As part of the upgrade KeePass was updated from v2.38 to v2.44. Also, libgcrypt was updated from libgcrypt.so.20.2.1 to 20.2.5. However, I have also noted the following:
  The new KeePass works correctly on the old systems. The old KeePass does not work on the new systems. So KeePass itself seems not to be at fault. Next I tried the library. I downgraded libgcrypt on the new Mint 20 back to libgcrypt.so.20.2.1, and it still didn't work.
  
  I am a bit stumped. Is it possibly a kernel issue?
  
  Interestingly, when libgcrypt was upgraded in the new Mint it was relocated from /lib to /usr/lib. Is this possibly an issue? Is there any sort of debug logging I can enable to troubleshoot where the disconnect between KeePass and AES-NI is?
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul - 2020-08-07

This is done in NativeMethods.Unix.cs (LibGCrypt = "libgcrypt.so.20"). If the library is named differently you would revert to the old method.

cheers, Paul

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Kurt Fitzner - 2020-08-07
  
  The library is named the same. Just located somewhere different.
  
  I am looking at the code and it is checking for MonoWorkarounds.IsRequired(1468) - the number for this bug report. Is there a way to force that to be true for testing purposes?
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dominik Reichl - 2020-08-07

Currently, MonoWorkarounds.IsRequired(1468) always returns true when running on Linux, unless the user manually overrides this using a command line option.

If you want to test whether the library is simply slower now, you can disable the workaround by running KeePass as follows:
mono KeePass.exe --wa-disable:1468

To me it sounds as if Mono doesn't find the libgcrypt.so.20 library anymore, but I'm not sure.

Best regards,
Dominik

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- René B. Christensen - 2020-10-13
  
  I am experiencing slower decryption after upgrading from Ubuntu 18.04LTS to 20.04LTS. I don't remember the exact timings before upgrading, but there is a noticeable difference.
  
  If I disable the workarounds as suggested, the decryption is significantly slower (going from ~3.8 seconds to ~30 seconds) on my machine. I guess that indicates that libgcrypt.so.20 is still found, but that the 20.2.5 version is slower than 20.2.1 for some reason.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul - 2020-10-14

Could you test a native version?
Try KeeWeb, https://github.com/keeweb/keeweb/releases/tag/v1.15.7.

cheers, Paul

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Kurt Fitzner - 2020-10-14
  
  Neither keepassx or keepassxc exhibit the same slowdown. I have verified that the problem does seem to be inside libgcrypt - I did some quick tests and it seems that KeePass is still finding and using libgcrypt.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dominik Reichl - 2020-10-14

Ok, thanks for the info! Let's hope they can make it faster again.

Best regards,
Dominik

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Opening/saving database slow on Linux

A lightweight and easy-to-use password manager

Group

Searches

Help

#1468 Opening/saving database slow on Linux

Discussion