Emonic editor very slow with larger files
Status: Beta
Brought to you by:
bernhardbrem
Menu
▾
▴
Logged In: YES
user_id=1299552
Originator: NO
I believe that the problem is caused by CSharpBracketMatcher. Everything you come across a parenthesis or bracket, the ICharacterPairMatcher implementation will do a parse searching for a matching pair.
Logged In: YES
user_id=1299552
Originator: NO
We need a more aggressive algorithm for the spoolToNextRelevant(int) method. We need to try to skip entire tokens where possible.
Logged In: YES
user_id=1245734
Originator: NO
It seems to me that this behavour changed _after_ veersion 0.3. _Before_ I didn't have such problems. Since I short after release 0.3 changed my linux distri, I thought this was the reason, but seems not to be.
> Maybe it comes from CSharpEditorExamineJob, a UIJob?
When did we change lasttime something in this corner? I have the feeling that the whole mashine goes into load probs, so it neefds not to be a UI-job which makes the problems......
Logged In: YES
user_id=1299552
Originator: NO
> It seems to me that this behavour changed _after_ veersion 0.3. _Before_ I
> didn't have such problems. Since I short after release 0.3 changed my linux
> distri, I thought this was the reason, but seems not to be.
I'm not sure about that. I installed 0.3.0 and tried it and then exported the plug-in from my workspace and the performance looks about the same (although I didn't try to calculate any numbers out.
> Maybe it comes from CSharpEditorExamineJob, a UIJob?
> When did we change lasttime something in this corner? I have the feeling
> that the whole mashine goes into load probs, so it neefds not to be a
> UI-job which makes the problems......
I still think that the problem is caused by CSharpBracketMatcher. The parsing is taking too much time to complete.
Logged In: YES
user_id=1245734
Originator: NO
> I still think that the problem is caused by CSharpBracketMatcher. The
> parsing is taking too much time to complete.
I had a session with the debugger on this art of the code: It works the following way:
- Check whether the cursor is at a position with a bracket. This means a loop over 8 chars ()<>{}[].
- If the cursor is not on such a position leave at once -> no more parsing is done
- Only when the cursor meets a bracket parse the code for the corresponding bracket
Since the actual parsing is done only when the cursor is on a bracket I don't believe this part of the code is the guilty one.....
Logged In: YES
user_id=1299552
Originator: NO
> Since the actual parsing is done only when the cursor is on a bracket I
> don't believe this part of the code is the guilty one.....
Well, if you don't type anything and just move your cursor text around in areas that aren't on a bracket, you will notice that there are no performance issues whatsoever.
Once the user types something, the reconciler will schedule the CSharpEditorExamineJob, which is a UIJob, and it parses on the foreground, which causes lag. When you move around and you hit the brackets, the CSharpBracketMatcher will scan the document on the UI thread, which hangs the application.
Logged In: YES
user_id=1245734
Originator: NO
> Well, if you don't type anything and just move your cursor text around in
> areas that aren't on a bracket, you will notice that there are no
> performance issues whatsoever.
Hmmm. Seemed to be a bit slow for me, too. I had the impression that the whole machine simple has a huge load (top said about 3 for a single-processor-machine).....
> Once the user types something, the reconciler will schedule the
> CSharpEditorExamineJob, which is a UIJob, and it parses on the foreground,
> which causes lag
Certainly one source of problem. Possible way-out: The CSharpEditorExamineJob could use a doc root generated by a second job in the background each 1/2 second or so..... Then the code folding might not be too exact but still exact enough. Should be not too hart to implement.
> When you move around and you hit the brackets, the
> CSharpBracketMatcher will scan the document on the UI thread, which hangs
> the application.
Hangs the app completely? Never had this.... I don't know whether we can use here a similar trick ( a doc root scanned by a second job). The info might be too old...
> We need a more aggressive algorithm for the spoolToNextRelevant(int)
> method. We need to try to skip entire tokens where possible.
Certainly we should look what we can improve here. But I don't believe we get better than a factor 1.5.
Logged In: YES
user_id=1299552
Originator: NO
> Certainly one source of problem. Possible way-out: The
> CSharpEditorExamineJob could use a doc root generated by a second job in
> the background each 1/2 second or so..... Then the code folding might not
> be too exact but still exact enough. Should be not too hart to implement.
I don't see why it needs to check every half a second. If the user isn't typing anything, the document isn't being changed and it should not be necessary to parse again. The parsing should be done in the background (by a regular Job) and when the document has been completed, it should schedule other Jobs or UIJobs to take the parsed and input and do whatever it wants with it.
> Hangs the app completely? Never had this.... I don't know whether we can
> use here a similar trick ( a doc root scanned by a second job). The info
> might be too old...
I don't mean completely. By hang I just meant lag.
> Certainly we should look what we can improve here. But I don't believe we
> get better than a factor 1.5.
I think that it's going to make a bigger difference than that. In any case, any time saved at all is a good thing. ;)
Logged In: YES
user_id=1245734
Originator: NO
> I don't see why it needs to check every half a second. If the user isn't
> typing anything, the document isn't being changed and it should not be
> necessary to parse again
Sorry, didn't express this right: Each 1/2 second or so should the background job check whether something changed and parse if yes :-)
> it should schedule other Jobs or UIJobs to take the parsed
> and input and do whatever it wants
> with it.
If I understand eclipse right, this is not possible: Since a UI-Job runs in the same thread like the UI (and therefore access the UI for codefolding and aoutline view), You can't scedule it from a background job. So I think the way to do is: Parse the info (only if something changed) in the background and provide it as data structure which the UI-Job can access. The UI-Job is sceduled like before but quite faster since it needs not to scan the file.
Logged In: YES
user_id=1299552
Originator: NO
> If I understand eclipse right, this is not possible: Since a UI-Job runs
> in the same thread like the UI (and therefore access the UI for codefolding
> and aoutline view), You can't scedule it from a background job.
Not true, a Job can schedule UIJobs and UIJobs can schedule Jobs. It doesn't matter. The internal system will take care of delegating the Job to a usable worker thread.
Logged In: YES
user_id=1245734
Originator: NO
> Not true, a Job can schedule UIJobs and UIJobs can schedule Jobs. It
> doesn't matter. The internal system will take care of delegating the
> Job to a usable worker thread.
Then it is easy.
First part in CVS:
The CSharpExamineJob is now a non-UI-Job, only the parts interacting with the surface are UI-Jobs now.
Patch to consume characters more aggressively by skipping over tokens.
Logged In: YES
user_id=1299552
Originator: NO
This patch will skip over words completely if we never actually found while searching for .NET elements in a C# file. On my computer, the times for parsing Dominik's file went down from about 1000 milliseconds to about 450 or so. Please give it a shot if you can.
File Added: bug1833224-token-consumption-patch.txt
Logged In: YES
user_id=1245734
Originator: NO
Rewrote the bracket matching code. Now it is more fast. Next step I will implement sending the parsing of the doc for brackets in a background job.
Logged In: YES
user_id=1245734
Originator: NO
BracketMatcher parses now in the background -> no more speed probs in bracket completion
To do now: Incooperate Rem's patch
Logged In: YES
user_id=1245734
Originator: NO
Still one big problem for me: CSharpExamineJob starts - via a parserjob.java gmcs each time the document is changed. This drives (at least at my computer) the system load very high (According to top Load 3-7, most time is java the most active process, but after killing mono - which drives gmcs - the load vanishes at once). Already tried to restrict the numbers of parallel running parser jobs to 1 - no much improvement.
I hace 2 ideas to improve this:
1) Change the CSharpReconcilingStrategy to tell the parser job to kill the gmcs before restarting the next instance: The eclipse docu does not say what happens if a job is canceled (it says, the job is asked to stop - under which conditions leads this to a stop of the job?) I could think that the job stops, but gmcs runs after the job finished and takes much system load.
2) Restrict the run of gmcs to restart at a max. rate of 5 seconds or so
Any opinions?
Logged In: YES
user_id=1299552
Originator: NO
> The eclipse docu does not say
> what happens if a job is canceled (it says, the job is asked to stop -
> under which conditions leads this to a stop of the job?) I could think that
> the job stops, but gmcs runs after the job finished and takes much system
> load.
Just because you call stop() on a Job doesn't mean that your implementation of run(IProgressMonitor) or runInUIThread(IProgressMonitor) is going to stop and return immediately. Per the API specification, the IProgressMonitor that is passed in to the run methods is used for detecting cancellation. To provide cancellation support, Jobs are supposed to periodically check if (monitor.isCanceled()) to determine whether they should clean-up and return Status.CANCEL_STATUS or not. This is kind of like bug 1814049 wherein the builder cannot be canceled because we are not checking the IProgressMonitor's state.
http://sourceforge.net/tracker/index.php?func=detail&aid=1814049&group_id=158390&atid=807641
Logged In: YES
user_id=1299552
Originator: NO
> To do now: Incooperate Rem's patch
Bernhard, any updates on this? Have you had a chance to review it?
Logged In: YES
user_id=1245734
Originator: NO
tested the patch, looks fine. Put to cvs.
Logged In: YES
user_id=1245734
Originator: NO
Redesigned the runing-proc. of external processes. gmcs -parse- job is nicer now.