Menu
▾
▴
eclipsefp-develop
|
From: Johannes W. <joh...@we...> - 2008-04-21 16:47:35
|
Dear all, our local Hackathon http://leiffrenzel.de/eclipse/wiki/doku.php?id=hal3hackathon was a really nice event (big thanks to the programmers and sponsors!) and I want to underline one idea that was being discussed: (1) we should focus on tight integration of EclipseFP with GHC (this is crucial for providing functionality) (2) Cohatoe should keep GHC's session state object (make one session object available to successive plugin calls) (and this is crucial for efficiency) GHC has a nice API that allows programmatical access to (essentially) all of GHCi's command line functions (think :t, :i, :b and so on), and this is *the* way to get information on the Haskell sources. and it's proven: 1. it works for Visual Haskell. Haven't seen it, but I believe the screenshots, 2. we de re-implement (prototypically) something similar during the Hackathon (using GHC Api, but with fresh sessions for each plugin call). I think it should work as follows: The Eclipse user has one current buffer (always), and whenever some EclipseFP plugin is called, it knows the current buffer and does the equivalent of ghci :m (set module context) for that module. GHCi has some sort of caching for recently visited modules so context switch (within one project) should be cheap. Problems are: 2.0 (wrong model) Cohatoe is a general framework, it should now know anything about GHC 2.1 (wrong data) GHC is modelled on reading files (it chases imports throught the file system, it probable checks file modification times to see whether it needs to re-load etc) while Eclipse mainly has buffers (that may or may not be associated with files ?, also buffer content and file content may differ). 2.2 (single-threaded) Possibly there may be several plugins running in parallel (I'm not sure how that could happen or why one would want it) but I think the benefits are more important. The easiest way to obtain the behaviour described above is something like: in Cohatoe, when the server starts (function main here:) de.leiffrenzel.cohatoe.server.core/INTERNAL/hs/src/HaskellServer.hs create a GHC session object (newSession :: Maybe FilePath -> IO Session in ghc-6.8.2/compiler/main/GHC.hs) Then plugins must be enabled to ask the server for that object by providing some special request method. They don't need to change it (I assume it contains IORefs so clients will indirectly change these). I am tempted to make some patches in that direction but of course it would be good to have some discussion first. (I hope this list is the right place.) There is already some kind of "client asking for info" as I see it (function goUntilEOF in HaskellServer.hs : resp <- case decodeRequest requestString of Left msgs -> return $ ResponseError msgs Right req -> do pkgConf <- readPkgConf evalPluginMain req [pkgConf] A related question is of course, why specialize to GHC. I feel that the proposal also would enable easier HaRe integration, because they also have "session state". Of course with Hare we have the additional problem that the refactor wants to change source files. This should really not be allowed (the output of the refactorer should be a set of "Diff" objects) but this is an unrelated problem. Best regards, Johannes. |
|
From: Leif F. <hi...@le...> - 2008-04-21 17:20:27
|
Hi Johannes && everybody, > Dear all, > > our local Hackathon > http://leiffrenzel.de/eclipse/wiki/doku.php?id=hal3hackathon > was a really nice event (big thanks to the programmers and sponsors!) Yes, it was very good fun, and I'd like also to thank everybody who was there and contributed :-) In addition to what Johannes has mentioned, I'm also very glad to be able to announce that we are finally able to support Windows platforms again :-) You will see the results trickle into the repo and builds over the next few days. > and I want to underline one idea that was being discussed: > > (1) we should focus on tight integration of EclipseFP with GHC > (this is crucial for providing functionality) > > (2) Cohatoe should keep GHC's session state object > (make one session object available to successive plugin calls) > (and this is crucial for efficiency) > > GHC has a nice API that allows programmatical access > to (essentially) all of GHCi's command line functions > (think :t, :i, :b and so on), > and this is *the* way to get information on the Haskell sources. > > and it's proven: 1. it works for Visual Haskell. Haven't seen it, > but I believe the screenshots, 2. we de re-implement (prototypically) > something similar during the Hackathon (using GHC Api, but with > fresh sessions for each plugin call). > > > I think it should work as follows: > The Eclipse user has one current buffer (always), > and whenever some EclipseFP plugin is called, > it knows the current buffer and does the equivalent of ghci :m > (set module context) for that module. GHCi has some sort of caching > for recently visited modules so context switch (within one project) > should be cheap. > > > Problems are: > > 2.0 (wrong model) Cohatoe is a general framework, it should now know > anything about GHC Yes. Though I think we are able to do this in a plugin from within EclipseFP (see below). Of course I'm prepared to go some way to build support into Cohatoe itself if it helps :-) > > 2.1 (wrong data) GHC is modelled on reading files > (it chases imports throught the file system, it probable checks file > modification times to see whether it needs to re-load etc) > while Eclipse mainly has buffers (that may or may not be associated with > files ?, also buffer content and file content may differ). Right. Though for the first steps we would be ok with mostly running ghci on files, we will need support for buffers at some time. For example, Mark Occurrences, Code Folding or Outline aren't very nice when they get stale. > > 2.2 (single-threaded) Possibly there may be several plugins > running in parallel (I'm not sure how that could happen > or why one would want it) An example would be an indexer that runs in the background to read files that have changed when the user has recently saved. That's a long-running operation and we don't want it to block the IDE. While it runs, the user may want to continue in the editor and thus trigger Mark Occurrences etc. As it currently is, with Cohatoe there is no possibility to run several Haskell functions in parallel. The entry point function on the Java side is synchronized and must return before the next invocation to the Haskell server is sent. In essence, that is because all information goes as a request-response message protocol over a socket connection, and the next request can only be sent after the last one has successfully returned. I have plans to parallelize this, but I'd rather do it after the 2.0 release in June. Of course we can experiment with a prototype :-) > > > but I think the benefits are more important. > > > The easiest way to obtain the behaviour described above > is something like: > > in Cohatoe, when the server starts (function main here:) > de.leiffrenzel.cohatoe.server.core/INTERNAL/hs/src/HaskellServer.hs > create a GHC session object (newSession :: Maybe FilePath -> IO Session > in ghc-6.8.2/compiler/main/GHC.hs) > > Then plugins must be enabled to ask the server for that object > by providing some special request method. > They don't need to change it (I assume it contains IORefs > so clients will indirectly change these). Wouldn't it be easier to manage the session in a plugin? The entire Haskell server process remains up and running for the full length of the Eclipse session. So afaics keeping an IORef could be done in a plugin as well. In fact, Georg and I have tried this at the previous HaL2, and we have at least seen state to be kept between multiple calls (though we haven't tried to share it between two plugins). > > > > > I am tempted to make some patches in that direction > but of course it would be good to have some discussion first. > (I hope this list is the right place.) > > > There is already some kind of "client asking for info" as I see it > (function goUntilEOF in HaskellServer.hs : > > resp <- case decodeRequest requestString of > Left msgs -> return $ ResponseError msgs > Right req -> do pkgConf <- readPkgConf > evalPluginMain req [pkgConf] > > > > A related question is of course, why specialize to GHC. > I feel that the proposal also would enable easier HaRe integration, > because they also have "session state". If we would maintain the session state in a plugin instead of in the server, that would solve this problem as well. Of course, we should be careful not to introduce too many of these session things (global static variables and singletons are regarded as somewhat bad style by many, and with good reason ;-). Thanks && ciao, Leif >Of course with Hare > we have the additional problem that the refactor wants to change source > files. This should really not be allowed (the output of the refactorer > should be a set of "Diff" objects) but this is an unrelated problem. > > > > Best regards, Johannes. > > > > > > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone > _______________________________________________ > eclipsefp-develop mailing list > ecl...@li... > https://lists.sourceforge.net/lists/listinfo/eclipsefp-develop > -- Leif Frenzel http://leiffrenzel.eu http://cohatoe.blogspot.com |
|
From: Johannes W. <wal...@im...> - 2008-04-21 17:31:20
|
> Wouldn't it be easier to manage the session in a plugin? [...] I can't think of a reasonable way to initialize the GHC session, then. Also, sharing session state between plugins seems to be necessary. Then how to access the state must be defined "above" the plugins (that is, in the Cohatoe API ?) best regards, Johannes. |
|
From: Leif F. <hi...@le...> - 2008-04-21 18:09:50
|
Johannes Waldmann wrote: >> Wouldn't it be easier to manage the session in a plugin? [...] > > I can't think of a reasonable way to initialize the GHC session, then. > > Also, sharing session state between plugins seems to be necessary. Yes. In fact, I didn't try then, but here is a very QnD implementation: http://leiffrenzel.de/eclipse/teststate.zip which extends our experiment at HaL2 - now sharing the state between two plugins. (Download the zip, then in Eclipse say Import > General > Existing Projects into workspace > Archive file > browse to the downloaded file. Thanks && ciao, Leif > Then how to access the state must be defined "above" the plugins > (that is, in the Cohatoe API ?) > > best regards, Johannes. > > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone > _______________________________________________ > eclipsefp-develop mailing list > ecl...@li... > https://lists.sourceforge.net/lists/listinfo/eclipsefp-develop > -- Leif Frenzel http://leiffrenzel.eu http://cohatoe.blogspot.com |
|
From: Georg M. <ge...@nl...> - 2008-04-21 18:43:22
|
Hi all, I also want to thanks the participants and the sponsors for the event. It was great fun and we learned quite some things, one of which is that leksah has already quite some interesting features :-). I started to rewrite the ghci code to be more suitable for use in an IDE. There I stored the state in a IORef in the plugin. To share that state over plugins is probably not too much desirable concerning design concepts. My idea was to build a larger module that provides most of the ghci functionallity and which has one (and maybe the only) ghc session. The initialisation is no problem, it is just initialised the every time if the session object is not already there. Also the module change can be done only of the actuall modul is different then the loaded one. The question is what to do if the module contains errors ? I would try to tie the IORef stuff no too tight in it, that in can be used from e.g. leksah without that dirty stuff. I have so far no anwser to the problem of buffers and files, however for most of the ghci stuff so far it is not a big problem. It becomes more interesting if it comes to real AST processing. Maybe we ask the ghc folks how they suggest to handle this. Regards, Georg On Monday 21 April 2008, Leif Frenzel wrote: > Johannes Waldmann wrote: > >> Wouldn't it be easier to manage the session in a plugin? [...] > > > > I can't think of a reasonable way to initialize the GHC session, then. > > > > Also, sharing session state between plugins seems to be necessary. > > Yes. In fact, I didn't try then, but here is a very QnD implementation: > > http://leiffrenzel.de/eclipse/teststate.zip > > which extends our experiment at HaL2 - now sharing the state between two > plugins. > > (Download the zip, then in Eclipse say Import > General > Existing > Projects into workspace > Archive file > browse to the downloaded file. > > Thanks && ciao, > Leif > > > Then how to access the state must be defined "above" the plugins > > (that is, in the Cohatoe API ?) > > > > best regards, Johannes. > > > > > > > > > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > > Don't miss this year's exciting event. There's still time to save $100. > > Use priority code J8TL2D2. > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/ja > >vaone _______________________________________________ > > eclipsefp-develop mailing list > > ecl...@li... > > https://lists.sourceforge.net/lists/listinfo/eclipsefp-develop -- ----------------------------------------------------------------- Georg Martius Phone: +49 (0)551 5176421 Fax: +49 (0)551 5176439 EMail: ge...@nl... Georg August University Goettingen, Institute for Nonlinear Dynamics Bunsenstrasse 10, D-37073 Goettingen, Germany |
|
From: Johannes W. <wal...@im...> - 2008-04-21 20:45:29
|
> There I stored the state in a IORef in the plugin. [...] > The initialisation is no problem, it is just initialised the every time if the > session object is not already there. "there" meaning where? where do you want to declare this IORef that holds the session? * inside pluginMain: then it denotes a different thing on the next call * in the plugin, but outside pluginMain: then you can't initialize it (except with unsafePerformIO perhaps, which sounds somewhat horrible) my conclusion was that the cohatoe server should declare and create the reference. regards, Johannes. |
|
From: Leif F. <hi...@le...> - 2008-04-21 21:29:32
|
Johannes Waldmann wrote: >> There I stored the state in a IORef in the plugin. [...] >> The initialisation is no problem, it is just initialised the every time if the >> session object is not already there. > > "there" meaning where? Nowhere; 'being there' == 'existing'. I think the idea is that the session is kept in a value, thus it is 'just there', which means in practice it's created when it is used the first time. > > where do you want to declare this IORef that holds the session? > > * inside pluginMain: then it denotes a different thing on the next call > * in the plugin, but outside pluginMain: then you can't initialize it > (except with unsafePerformIO perhaps, which sounds somewhat horrible) But there must be better reasons than the sound ;-) So, I'm not an expert with the unsafe-stuff, but as I understand it the essential thing is to keep the unsafe part wrapped and make sure that all accesses are side-effect-free, typesafe, and we prevent inlining by the compiler. I think that would be doable, but perhaps there are risks I'm not aware of? > > my conclusion was that the cohatoe server should declare and create the > reference. But that's not enough: we also have to pass it on to the pluginMain every time, which means that the GHCi session becomes part of the interface, and that seems somewhat heavy to me. It's not only the API change. I don't like the idea that any Haskell code I'd like to use from within Eclipse would automatically get a reference to a GHCi session which is needed in particular cases (even if they are important for EclipseFP). My intuition tells me this should be managed by the plugins which want to use it; I'm only prepared to accept pulling EclipseFP stuff up to Cohatoe if I'm convinced there isn't another way. Another idea: Normally, a server implementation provides a generic session context. E.g. a servlet engine would provide a session object that is passed on to all requests and then used by them to put data. That date is preserved until the next request cycle. This looks more plausible to me: so we would just provide a generic state that can be updated by plugins with whatever they like, and read by them when their turn is again. This would keep the API generic. We would have to pass a session ID from the java side (or none, which is taken to open a new session), and on the Haskell side we'd locate the corresponding session state and pass it on to pluginMain. In the response, we would return the session ID again. This would be a nice approach because it would help us with two other things I also want to do: make the Cohatoe server into a real server that works on requests in parallel, and providing some sort of callback facility (e.g. for progress monitors etc.). I'd need some help with the API design, though. How could we model the session state that we want to pass on and still keep the interface easy to use? Thanks && ciao, Leif > > regards, Johannes. > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone > _______________________________________________ > eclipsefp-develop mailing list > ecl...@li... > https://lists.sourceforge.net/lists/listinfo/eclipsefp-develop > -- Leif Frenzel http://leiffrenzel.eu http://cohatoe.blogspot.com |
|
From: Georg M. <geo...@we...> - 2008-04-22 09:42:26
|
Hi everybody, On Monday 21 April 2008 23:29, Leif Frenzel wrote: > Johannes Waldmann wrote: > >> There I stored the state in a IORef in the plugin. [...] > >> The initialisation is no problem, it is just initialised the every time > >> if the session object is not already there. > > > > "there" meaning where? > > Nowhere; 'being there' == 'existing'. > > I think the idea is that the session is kept in a value, thus it is > 'just there', which means in practice it's created when it is used the > first time. Exactly. This is what the ghci already does allready and so I just used there code. I don't think there is difference between passing the IORef to java and get it back or to keep it in global variable in the plugin. Multithreaded calls become difficult in both cases. I think it should restrict parallel calls to be only between different plugins. Each plugin has its own ghc session in this case. However, I thought that we have a major ghc plugin that can be used for most things (but sequential). Time consuming operations have to go into a separate plugin. There is one API change that might be nice because I see that we want to call different functions within one plugin. One option is to just give the name as a String argument, that is also fine but maybe there are better options. > > > where do you want to declare this IORef that holds the session? > > > > * inside pluginMain: then it denotes a different thing on the next call > > * in the plugin, but outside pluginMain: then you can't initialize it > > (except with unsafePerformIO perhaps, which sounds somewhat horrible) > > But there must be better reasons than the sound ;-) So, I'm not an > expert with the unsafe-stuff, but as I understand it the essential thing > is to keep the unsafe part wrapped and make sure that all accesses are > side-effect-free, typesafe, and we prevent inlining by the compiler. I > think that would be doable, but perhaps there are risks I'm not aware of? I agree. > > > my conclusion was that the cohatoe server should declare and create the > > reference. > > But that's not enough: we also have to pass it on to the pluginMain > every time, which means that the GHCi session becomes part of the > interface, and that seems somewhat heavy to me. It's not only the API > change. I don't like the idea that any Haskell code I'd like to use from > within Eclipse would automatically get a reference to a GHCi session > which is needed in particular cases (even if they are important for > EclipseFP). > > My intuition tells me this should be managed by the plugins which want > to use it; I'm only prepared to accept pulling EclipseFP stuff up to > Cohatoe if I'm convinced there isn't another way. > > Another idea: Normally, a server implementation provides a generic > session context. E.g. a servlet engine would provide a session object > that is passed on to all requests and then used by them to put data. > That date is preserved until the next request cycle. This looks more > plausible to me: so we would just provide a generic state that can be > updated by plugins with whatever they like, and read by them when their > turn is again. This would keep the API generic. > > We would have to pass a session ID from the java side (or none, which is > taken to open a new session), and on the Haskell side we'd locate the > corresponding session state and pass it on to pluginMain. In the > response, we would return the session ID again. > > This would be a nice approach because it would help us with two other > things I also want to do: make the Cohatoe server into a real server > that works on requests in parallel, and providing some sort of callback > facility (e.g. for progress monitors etc.). > > I'd need some help with the API design, though. How could we model the > session state that we want to pass on and still keep the interface easy > to use? If we can manage this in a typesave manner than this would be certainly the way to go. I image: Plugin.hs data State = ... init :: Params -> IO(State) pluginMain :: Params -> State -> IO(Result) cleanup :: State -> IO() However I don't see how we can do this without some ugly code either. I mean the state should not really be passed to the java side, but rather the Reference, which is only valid within one plugin anyway right? I would actually strongly argue for NOT shareing anything between plugins. Gruss Georg > > Thanks && ciao, > Leif > > > regards, Johannes. > > > > > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > > Don't miss this year's exciting event. There's still time to save $100. > > Use priority code J8TL2D2. > > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/ja > >vaone _______________________________________________ > > eclipsefp-develop mailing list > > ecl...@li... > > https://lists.sourceforge.net/lists/listinfo/eclipsefp-develop |
|
From: Johannes W. <wal...@im...> - 2008-04-22 10:26:50
|
> init :: Params -> IO(State)
> pluginMain :: Params -> State -> IO(Result)
> cleanup :: State -> IO()
>
> However I don't see how we can do this without some ugly code either. I mean
> the state should not really be passed to the java side [...]
Well, there's the type system to help us:
make the State a type argument for the Plugin type, sth like
class Plugin s where
init :: Params -> IO s
main :: Params -> s -> IO ()
(if main wants to change a, then it uses IORef a instead)
The Server then must handle Plugins generically
so it cannot have any dependence on GHC or HaRe particulars.
The server does not even know their types.
The server code would probably wrap pairs of Plugin and State
under some existential data type.
cf. "heterogenous lists" in this explanation
http://en.wikibooks.org/wiki/Haskell/Existentially_quantified_types
best regards, Johannes.
|
|
From: Johannes W. <wal...@im...> - 2008-04-22 10:35:54
|
> [...] I thought that we have a major ghc plugin that
> can be used for most things (but sequential).
sounds reasonable. there would be the ghci plugin, the HaRe plugin,
and possibly others.
> There is one API change that might be nice because I see that we want to call
> different functions within one plugin. One option is to just give the name as
> a String argument, that is also fine but maybe there are better options.
no API changes necessary. On the conceptual level,
the ghci API call has one parameter object, with several constructors, e.g.
data Call = Get_Type_For_Enclosing_Function { pos :: SrcLoc , .. }
| Get_Point_Of_Declaration { }
and this has to be marshalled.
BTW, perhaps we use JSON ( instead of [String] )
see http://hackage.haskell.org/cgi-bin/hackage-scripts/package/json
that would give us some tool support for marshalling.
Best regards, Johannes.
|
|
From: Leif F. <hi...@le...> - 2008-04-22 11:34:24
|
Hi Georg && everybody, >> >> if the session object is not already there. >> > >> > "there" meaning where? >> >> Nowhere; 'being there' == 'existing'. >> >> I think the idea is that the session is kept in a value, thus it is >> 'just there', which means in practice it's created when it is used the >> first time. > Exactly. This is what the ghci already does allready and so I just used > there > code. I don't think there is difference between passing the IORef to java > and > get it back or to keep it in global variable in the plugin. Multithreaded > calls become difficult in both cases. I think it should restrict parallel > calls to be only between different plugins. Each plugin has its own ghc > session in this case. However, I thought that we have a major ghc plugin > that > can be used for most things (but sequential). Time consuming operations > have > to go into a separate plugin. > There is one API change that might be nice because I see that we want to > call > different functions within one plugin. One option is to just give the name > as > a String argument, that is also fine but maybe there are better options. That shouldn't be too difficult to do. The only ugliness with this approach would be that we would make it necessary on the Java side to pass the name of the function as String, which means we actually would introduce some 'String programming'. I'll think about that. (But it is just a design question, technically this is feasible.) Thanks && ciao, Leif -- Leif Frenzel http://leiffrenzel.de http://cohatoe.blogspot.com |
|
From: Leif F. <hi...@le...> - 2008-04-22 11:44:56
|
>> I'd need some help with the API design, though. How could we model the >> session state that we want to pass on and still keep the interface easy >> to use? > If we can manage this in a typesave manner than this would be certainly > the > way to go. I image: > Plugin.hs > data State = ... > init :: Params -> IO(State) > pluginMain :: Params -> State -> IO(Result) > cleanup :: State -> IO() > > However I don't see how we can do this without some ugly code either. I > mean > the state should not really be passed to the java side, but rather the > Reference, which is only valid within one plugin anyway right? > I would actually strongly argue for NOT shareing anything between plugins. I was thinking on these lines: from the Java side, we pass on an identifier (String, normally some unique number). The Cohatoe server would maintain a map from such IDs to State objects. If there is a State for the passed ID, then it is passed on to pluginMain; if there isn't, a new one is generated, along with a new ID. In both cases, the ID is returned to the Java side. On the Java side, we have the option to ignore the ID or to store it and pass it on to the Haskell side with the next request. I.e. we wouldn't share anything between the Java side and the Haskell side except session IDs. Probably we wouln't even need the init function; a plugin could just check whether in the State there is a GHC session and put one in there if there isn't one. (We would probably want to do this anyway, if there is a possibility that the session dies in between. Otherwise there would be no way to re-initialize the plugin. As for the cleanup, that would require some sort of session timeout, and in addition a manual cleanup would be good. Thanks && ciao, Leif -- Leif Frenzel http://leiffrenzel.de http://cohatoe.blogspot.com |
Thanks for helping keep SourceForge clean.
X