Re: [ jEdit-devel ] Proper use of TokenMarker and LineContext

[Shlomy R. <sre...@gm...>]

Hi Makarius,

Sorry I didn't respond earlier to your email. I was busy and thought someone
with more familiarity with the token marker would reply.
I didn't understand what you actually need from your email, but from the
sentence "we only need the absolute text range to map
that to token markup", it seems like you want to find the position of each
token in the buffer. This part is easy; the buffer has "getLineStartOffset",
that returns the offset of the beginning of the line, and you add the offset
of each token within the line to it. You can find an example in
Buffer.editSyntaxStyle().

Hope I was helpful. If you're looking for something completely different,
maybe a small example can help explain it.

Regards,
Shlomy

On Fri, Sep 3, 2010 at 6:58 PM, Alan Ezust <ala...@gm...> wrote:

> Hi Makarius,
>
> Notice the *chirp chirp* of crickets after your asked that question, eh?
>
> There is nobody who has worked closely on this code in many years.
> Slava wanted to get rid of ChunkCache + other low level textarea
> classes and replace them with something that is in the JDK. But since
> leaving, most of the development on the textarea is to fix bugs and
> make it a separable component.
>
> So what you see is what you get, and if you would like to delve into
> the code, try to figure out how it works, and document it, we would
> welcome any contributions you may have. Even if it's just some extra
> code comments that you think will help the next person understand it
> better.
>
>
> On Tue, Aug 24, 2010 at 2:34 PM, Makarius <mak...@sk...> wrote:
> > Dear jEdit experts,
> >
> > we are am working on a sophisticated plugin where jEdit buffers are
> > associated with full semantic models provided by an external process.
> > (The application is to do real-time checking of formal mathematical proof
> > documents with the Isabelle/Isar theorem prover, e.g. see
> > http://www4.in.tum.de/~wenzelm/papers/async-isabelle-scala.pdf)
> >
> > One of the questions arising here is how to connect to the TokenMarker
> > setup properly.  Our own buffer model already provides the full
> > information for text regions, we only need the absolute text range to map
> > that to token markup.
> >
> > TokenMarker.markTokens only provides the current line segment, not its
> > offset in the buffer text.  As a workaround we have experimented with a
> > LineContext containing the last line as explicit integer.  After
> > investigating memory profiles of jvisualvm and the jEdit sources, it
> seems
> > that this is not the proper way to do it.  LineContext maintains a
> > persistent table of all contexts ever encountered, via the "intern"
> > method. Thus we essentially have a memory leak in our token marker.
> >
> > Anyway, is this going in the right direction at all?  Are there better
> > ways to modify text styles according to precomputed tables that we have?
> >
> > I have tried to understand the general TokenMarker, LineContext,
> > ChunkCache setup, without really seeing through all that.
> >
> >
> > It would be also nice if the Byte indexes into the SyntaxStyle array
> could
> > be side-stepped, i.e. by providing style objects directly.  But that is a
> > different story ...
> >
> >
> >        Makarius
> >
> >
> ------------------------------------------------------------------------------
> > Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
> > Be part of this innovative community and reach millions of netbook users
> > worldwide. Take advantage of special opportunities to increase revenue
> and
> > speed time-to-market. Join now, and jumpstart your future.
> > http://p.sf.net/sfu/intel-atom-d2d
> > --
> > -----------------------------------------------
> > jEdit Developers' List
> > jEd...@li...
> > https://lists.sourceforge.net/lists/listinfo/jedit-devel
> >
>
>
> ------------------------------------------------------------------------------
> This SF.net Dev2Dev email is sponsored by:
>
> Show off your parallel programming skills.
> Enter the Intel(R) Threading Challenge 2010.
> http://p.sf.net/sfu/intel-thread-sfd
> --
> -----------------------------------------------
> jEdit Developers' List
> jEd...@li...
> https://lists.sourceforge.net/lists/listinfo/jedit-devel
>