#4135 Slow HyperSearch Performance on macOS with Large Files and Queries

[Matthew Orzewalla]

I have been observing slow hypersearch performanc on macOS for a while now, but wanted to wait for a preview of 5.7 to see if performance improved.

From what I can tell, it began with the switch to Java 11, but I'm not positive that is the root cause. The bug has been observed on both Intel Macs and ARM Macs

Steps I use to reproduce:

1. Download the plain text copy of Death Valley in '49 from Project Gutenberg here: https://www.gutenberg.org/ebooks/12236.txt.utf-8
2. Install and start up a fresh version jEdit 5.7pre (no prexisting jEdit user directory) from the version posted in this thread: https://sourceforge.net/p/jedit/bugs/4123/. On my system java-version lists:
   openjdk version "21.0.2" 2024-01-16 LTS
   OpenJDK Runtime Environment Temurin-21.0.2+13 (build 21.0.2+13-LTS)
   OpenJDK 64-Bit Server VM Temurin-21.0.2+13 (build 21.0.2+13-LTS, mixed mode)
3. Open the text file (pg12236.txt from the download on my system) in jEdit
4. Perform a search with Ignore Case and HyperSearch selected of the word "death". The 106 results should be returned fairly quickly
5. Perform a search for the word "from". The 625 results will eventually be returned, but it beachballs my system for about 50 seconds.

Performing a search for the word "the" beachballs my system for longer than I am willing to wait.

I can perform a search of the same file for the word "the" on a Windows install of jEdit and return the 15,733 results in less than 3 seconds (the Windows About jEdit screen lists, "jEdit 5.5.0 server mode, using Oracle Corporation Java 1.8.0_391).

I am uploading the activity.log file., but I didn't see anything indicating the slow performance within the file.

Let me know if there is any other information I can provide that can help diagnose the root cause. At first I had thought that it had something to do with docking or undocking the HyperSearch window (with docked causing slow performance), but then I went back and did some dedicated testing and found that it is slow undocked as well. It also seems that if I do more searches and work my way up to larger returns that the slowness subsides, as if it is slow allocating memory to the process, but once that memory gets allocated it is no longer slow.

Thank you for your help in this matter.

[Matthew Orzewalla]

With the release of jEdit 5.7, I tried the process again. i installed jEdit 5.7 on Windows 10 using Temurin 21 (let me know if the details are needed; I am not in front of the Windows machine at the moment), and HyperSearch works as expected for the test file for searching "death", "from", or "the".

I tested jEdit 5.7 on an ARM Mac using Temurin 22 (on Windows I did not realize that Temurin 22 was the latest release) and I am still observing a sustained beachball when searching "the". I did try turing off the Text Color and Background Color checkboxes within the HyperSearch Style Editor, and I think that may have sped up performance when searching "from", but the search still had issues searching the word "the".

Since the original post, I noticed that Isabelle2024 was released here: https://isabelle.in.tum.de. When performing the same proceedure with the Isabelle2024 (jEdit 5.6 under the covers) HyperSearch works as expected (taking maybe a couple seconds to populate the HyperSearch results for "the").

Isabelle uses the Azul Systems Java 21 virtual machine built-in to the install. I tried changing the macOS Java vurtual machine to Azul Systems Java 21, but got the same anomolous result for jEdit 5.7.

I would use Isabelle2024 going forward, but I do like the better macOS integration available in jEdit 5.7. :)

[Alan Ezust]

So just to confirm, you see this problem in jEdit 5.7 but not 5.5. What about jEdit 5.6? Isabelle uses it, and it seems to be not present there either? Can you confirm by trying jEdit 5.6?

[Matthew Orzewalla]

I am fairly certain the issue existed in jEdit 5.6 for macOS. I just now reinstalled 5.6 to confirm, but am running into the "This application required that Java 11 or later be installed on your computer" issue with a vanilla install. I will need to go find agan the steps needed to get 5.6 pointed to the Java 11 I have installed on the system.

[Björn Kautler]

It should be enough if you take the jedit file of 5.7 and overwrite the one in 5.6

[Matthew Orzewalla]

Well that would have been easier. :P

Since it was easy enough to do, I tried that method, but got the same result. Searching "death" or "from" works as expected (there are less than 1,000 occurrences), but searching "the" produces unresponsiveness.

[Matthew Orzewalla]

Okay. I found some options for starting jEdit 5.6 here: https://apple.stackexchange.com/questions/426713/how-to-help-the-jedit-application-find-java

And chose to start jEdit via the terminal using the following command:

/usr/bin/java -jar /Applications/jEdit.app/Contents/Java/jedit.jar 2> /dev/null &

About jEdit... lists the following:

jEdit 5.6.0 server-background mode, using Azul Systems, Inc. Java 22.0.2

For this configuration I do still get a beachball when searching for the word "the" (it lasts several minutes; I ended up force quitting the app).

Just to be sure, I checked Isabelle2024 again and it returned the 15,733 hypersearch results very quickly, as expected.

Also, I tried to find mention of a patch to Isabelle that would affect hyperseach here: https://isabelle-dev.sketis.net/source/isabelle, but did not find anything that stood out.

Thanks.

[Makarius]

On 14/08/2024 19:33, Matthew Orzewalla via jEdit-devel wrote:

Just to be sure, I checked Isabelle2024 again and it returned the 15,733
hypersearch results very quickly, as expected.

Also, I tried to find mention of a patch to Isabelle that would affect
hyperseach here: https://isabelle-dev.sketis.net/source/isabelle
https://isabelle-dev.sketis.net/source/isabelle, but did not find anything
that stood out.

Just a side remark concerning Isabelle/jEdit:

• The main source of patches is here:
  https://isabelle-dev.sketis.net/source/isabelle/browse/default/src/Tools/jEdit/patches

  Patch file-names provide an idea of the purpose, and further details are
  in the repository changelog for these files.

• The effective patch is automatically generated againts the original
  sources, e.g. see Isabelle2024/contrib/jedit-20240425/jedit5.6.0.patch

  Details are in the Isabelle/Scala program that produces the derivative
  version of jEdit:
  https://isabelle-dev.sketis.net/source/isabelle/browse/default/src/Pure/Admin/component_jedit.scala

Concerning the HyperSearch question, I can't remember doing anything special
wrt. jEdit 5.6, and I did not try 5.7 yet.

Makarius

[Matthew Orzewalla]

Thanks Makarius.

Given Björn's suggestion for replacing the jedit.jar file within the jEdit 5.7 app directory, I tried copying the Isabelle2024 jedit.jar from /Applications/Isabelle2024.app/contrib/jedit-20240425/jedit5.6.0-patched to /Applications/jEdit.app/Contents/Java, and next tried copying the entire jedit5.6.0-patched directory into the jEdit 5.7 Java directory. In both cases jEdit stalled searching for "the".

So presumably the notable difference does not reside within the jEdit JAR file itself.

[Matthew Orzewalla]

The plot thickens...

I open jEdit 5.7 from Terminal using the following command:

/usr/bin/java -verbose -jar /Applications/Isabelle2024.app/contrib/jedit-20240425/jedit5.6.0-patched/jedit.jar

and observe the anomolous hypersearch behavior.

I open Isabelle2024 from Terminal using the following command:

/Applications/Isabelle2024.app/contrib/jdk-21.0.3/arm64-darwin/zulu-21.jdk/Contents/Home/bin/java -verbose -jar /Applications/Isabelle2024.app/contrib/jedit-20240425/jedit5.6.0-patched/jedit.jar

and do not observe the anomolous hypersearch behavior (as expected).

I open jEdit 5.7 from Terminal pointing to the Isabelle2024 Java using the following command:

/Applications/Isabelle2024.app/contrib/jdk-21.0.3/arm64-darwin/zulu-21.jdk/Contents/Home/bin/java -verbose -jar /Applications/jEdit.app/Contents/Java/jedit.jar

and also do not observe the anomolous hypersearch behavior.

So the issue seems to be how my system is handling the Java installation. Not sure what steps to try next. It would also be reassuring if someone were able to reproduce the behavior on their system. I am trying to be as vanilla as possible with the system configuration.

[Matthew Orzewalla]

So the issue resides with the macOS java virtual machines. I had two JDK directories within /Library/Java/JavaVirtualMachines, one I installed to run jEdit, zulu-22.jdk (or temurin-22.jre, I tried them both), and one that was recommended by the Mathworks folks for running Matlab, amazon-corretto-11.jdk. If I remove the zulu/temurin directory such that only the amazon-corretto directory resides in the /Library/Java/JavaVirtualMachines, then jEdit hypersearch works as expected.

I am not sure what is ultimately going on here. On other platforms, jEdit is fine with alternate vendors/newer versions of the JDK/JRE. But on macOS, those alternate vendors/newer versions are problematic. I don't think my Matlab installation altered any system settings, as I have tried jEdit 5.7 on a few different systems and only one has Matlab installed (this is also the only one with Java 11 installed).

i alternately removed the amazon-corretto-11.jdk directory from /Library/Java/JavaVirtualMachines, leaving only zulu-22.jdk, but that configuration again produced the anomolous hypersearch behavior.

[Alan Ezust]

Closing as "invalid". This seems to be related to Java VM Configuration on MacOS and is not directly related to jEdit.