Enlightenment

Hi,

On Wed, Apr 4, 2012 at 4:52 PM, Rafael Antognolli
<antognolli@...> wrote:
> As some people know, ProFUSION is also working on Evas and making some
> changes that should make it even faster.
>
> Currently we are working on the images cache server, and it should be
> similar to the previous cserve, but with a more asynchronous API. It
> also has some fundamental changes, like having a pool of loaders where
> each of them run on a different process.
>
> The main idea is that the client can request the images needed as soon
> as a file_set is done, but don't block waiting for them to load. Once
> Evas really needs the image data, it will finally block to get this
> data, and should return almost immediately if it was already loaded.
>
> Some tweaks still must be done on the current code, some cleanup, and
> even some documentation, but the core is already there. A new internal
> cache was also done for Evas, very similar to the previous one but
> with some more direct calls to the server and some shorter code paths.
>
> The code can be seen here:
> http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/ (branch cserve2)
>
> Any comments are welcome!

I didn't have the time to read the code and see what you did, but I
have just one request. Don't put it inside evas. If it get outside of
evas, we can use ecore to simplify it's code for example, making it
also easier to port to other OS. I know that it would be strange to
have a runtime dependency of evas build after evas, but once we merge
all the library tree, nobody will notice.

So basically, for portability and maintenance, I would like to see
that project moved outside of evas.
-- 
Cedric BAIL

On Wed, Apr 4, 2012 at 4:52 PM, Rafael Antognolli
<antognolli@...> wrote:
> Hello people,
>
> As some people know, ProFUSION is also working on Evas and making some
> changes that should make it even faster.
>
> Currently we are working on the images cache server, and it should be
> similar to the previous cserve, but with a more asynchronous API. It
> also has some fundamental changes, like having a pool of loaders where
> each of them run on a different process.
>
> The main idea is that the client can request the images needed as soon
> as a file_set is done, but don't block waiting for them to load. Once
> Evas really needs the image data, it will finally block to get this
> data, and should return almost immediately if it was already loaded.
>
> Some tweaks still must be done on the current code, some cleanup, and
> even some documentation, but the core is already there. A new internal
> cache was also done for Evas, very similar to the previous one but
> with some more direct calls to the server and some shorter code paths.
>
> The code can be seen here:
> http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/ (branch cserve2)
>
> Any comments are welcome!

what about Windows ?

Vincent

On Wed, Apr 4, 2012 at 6:02 PM, Vincent Torri <vincent.torri@...> wrote:
> On Wed, Apr 4, 2012 at 4:52 PM, Rafael Antognolli
> <antognolli@...> wrote:
>> Hello people,
>>
>> As some people know, ProFUSION is also working on Evas and making some
>> changes that should make it even faster.
>>
>> Currently we are working on the images cache server, and it should be
>> similar to the previous cserve, but with a more asynchronous API. It
>> also has some fundamental changes, like having a pool of loaders where
>> each of them run on a different process.
>>
>> The main idea is that the client can request the images needed as soon
>> as a file_set is done, but don't block waiting for them to load. Once
>> Evas really needs the image data, it will finally block to get this
>> data, and should return almost immediately if it was already loaded.
>>
>> Some tweaks still must be done on the current code, some cleanup, and
>> even some documentation, but the core is already there. A new internal
>> cache was also done for Evas, very similar to the previous one but
>> with some more direct calls to the server and some shorter code paths.
>>
>> The code can be seen here:
>> http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/ (branch cserve2)
>>
>> Any comments are welcome!
>
> what about Windows ?

a small grep finds :

shm_open
fork + execvp

so it will be hard to have someting working well on Windows...

Vincent

On Wed, Apr 4, 2012 at 1:21 PM, Vincent Torri <vincent.torri@...> wrote:
> On Wed, Apr 4, 2012 at 6:02 PM, Vincent Torri <vincent.torri@...> wrote:
>> On Wed, Apr 4, 2012 at 4:52 PM, Rafael Antognolli
>> <antognolli@...> wrote:
>>> Hello people,
>>>
>>> As some people know, ProFUSION is also working on Evas and making some
>>> changes that should make it even faster.
>>>
>>> Currently we are working on the images cache server, and it should be
>>> similar to the previous cserve, but with a more asynchronous API. It
>>> also has some fundamental changes, like having a pool of loaders where
>>> each of them run on a different process.
>>>
>>> The main idea is that the client can request the images needed as soon
>>> as a file_set is done, but don't block waiting for them to load. Once
>>> Evas really needs the image data, it will finally block to get this
>>> data, and should return almost immediately if it was already loaded.
>>>
>>> Some tweaks still must be done on the current code, some cleanup, and
>>> even some documentation, but the core is already there. A new internal
>>> cache was also done for Evas, very similar to the previous one but
>>> with some more direct calls to the server and some shorter code paths.
>>>
>>> The code can be seen here:
>>> http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/ (branch cserve2)
>>>
>>> Any comments are welcome!
>>
>> what about Windows ?
>
> a small grep finds :
>
> shm_open
> fork + execvp
>
> so it will be hard to have someting working well on Windows...

Linux specific code on the server is in
evas_cserve2_main_loop_linux.c, the same for slaves and shm. The idea
is (and we have the most part of it already done) to abstract every
function that is specific to linux so we can have the specific windows
implementation for it.

For the shm side inside evas, while it's still using normal shm +
mmap, I'm going to change it to use just the eina functions available
to that.

-- 
Rafael Antognolli
ProFUSION embedded systems
http://profusion.mobi

On Wed, Apr 4, 2012 at 7:34 PM, Rafael Antognolli
<antognolli@...> wrote:
> On Wed, Apr 4, 2012 at 1:21 PM, Vincent Torri <vincent.torri@...> wrote:
>> On Wed, Apr 4, 2012 at 6:02 PM, Vincent Torri <vincent.torri@...> wrote:
>>> On Wed, Apr 4, 2012 at 4:52 PM, Rafael Antognolli
>>> <antognolli@...> wrote:
>>>> Hello people,
>>>>
>>>> As some people know, ProFUSION is also working on Evas and making some
>>>> changes that should make it even faster.
>>>>
>>>> Currently we are working on the images cache server, and it should be
>>>> similar to the previous cserve, but with a more asynchronous API. It
>>>> also has some fundamental changes, like having a pool of loaders where
>>>> each of them run on a different process.
>>>>
>>>> The main idea is that the client can request the images needed as soon
>>>> as a file_set is done, but don't block waiting for them to load. Once
>>>> Evas really needs the image data, it will finally block to get this
>>>> data, and should return almost immediately if it was already loaded.
>>>>
>>>> Some tweaks still must be done on the current code, some cleanup, and
>>>> even some documentation, but the core is already there. A new internal
>>>> cache was also done for Evas, very similar to the previous one but
>>>> with some more direct calls to the server and some shorter code paths.
>>>>
>>>> The code can be seen here:
>>>> http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/ (branch cserve2)
>>>>
>>>> Any comments are welcome!
>>>
>>> what about Windows ?
>>
>> a small grep finds :
>>
>> shm_open
>> fork + execvp
>>
>> so it will be hard to have someting working well on Windows...
>
> Linux specific code on the server is in
> evas_cserve2_main_loop_linux.c

are you really serious when you say that ?

just grep 'shm_open' and 'fork'...

btw, if you were doing portable code, you would use eina_file for
shm_open and not shm_open directly

Some questions :

 * what about the Coyote and ps3 arch ?
 * will cserve2 be optional like cserve ?

Vincent

>, the same for slaves and shm. The idea
> is (and we have the most part of it already done) to abstract every
> function that is specific to linux so we can have the specific windows
> implementation for it.
>
> For the shm side inside evas, while it's still using normal shm +
> mmap, I'm going to change it to use just the eina functions available
> to that.
>
> --
> Rafael Antognolli
> ProFUSION embedded systems
> http://profusion.mobi
>
> ------------------------------------------------------------------------------
> Better than sec? Nothing is better than sec when it comes to
> monitoring Big Data applications. Try Boundary one-second
> resolution app monitoring today. Free.
> http://p.sf.net/sfu/Boundary-dev2dev
> _______________________________________________
> enlightenment-devel mailing list
> enlightenment-devel@...
> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel

On Wed, Apr 4, 2012 at 7:23 PM, Vincent Torri <vincent.torri@...> wrote:
> On Wed, Apr 4, 2012 at 7:34 PM, Rafael Antognolli
> <antognolli@...> wrote:
>> On Wed, Apr 4, 2012 at 1:21 PM, Vincent Torri <vincent.torri@...> wrote:
>>> On Wed, Apr 4, 2012 at 6:02 PM, Vincent Torri <vincent.torri@...> wrote:
>>>> On Wed, Apr 4, 2012 at 4:52 PM, Rafael Antognolli
>>>> <antognolli@...> wrote:
>>>>> Hello people,
>>>>>
>>>>> As some people know, ProFUSION is also working on Evas and making some
>>>>> changes that should make it even faster.
>>>>>
>>>>> Currently we are working on the images cache server, and it should be
>>>>> similar to the previous cserve, but with a more asynchronous API. It
>>>>> also has some fundamental changes, like having a pool of loaders where
>>>>> each of them run on a different process.
>>>>>
>>>>> The main idea is that the client can request the images needed as soon
>>>>> as a file_set is done, but don't block waiting for them to load. Once
>>>>> Evas really needs the image data, it will finally block to get this
>>>>> data, and should return almost immediately if it was already loaded.
>>>>>
>>>>> Some tweaks still must be done on the current code, some cleanup, and
>>>>> even some documentation, but the core is already there. A new internal
>>>>> cache was also done for Evas, very similar to the previous one but
>>>>> with some more direct calls to the server and some shorter code paths.
>>>>>
>>>>> The code can be seen here:
>>>>> http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/ (branch cserve2)
>>>>>
>>>>> Any comments are welcome!
>>>>
>>>> what about Windows ?
>>>
>>> a small grep finds :
>>>
>>> shm_open
>>> fork + execvp
>>>
>>> so it will be hard to have someting working well on Windows...
>>
>> Linux specific code on the server is in
>> evas_cserve2_main_loop_linux.c
>
> are you really serious when you say that ?
>
> just grep 'shm_open' and 'fork'...

That's what we can do for the main loop, it's to aid porting, not to
make it happen now. We can isolate more in future as ports happen.
What is in this file is actually linux specific, like using signalfd()
that is not even available for FreeBSD or Solaris.

Hopefully before we have to worry the projects will be merged and we
can use Ecore, as proposed by cedric. Right now that's what we can do
in a viable timeframe. That was agreed by raster to have a Linux
specific code until then.

as for fork()/exec, we asked you what was better to use to help
Windows. No idea if you recall, but we did ask you as our initial hope
was to go with "fork()" only solution, you said it couldn't be
supported on Windows, but the fork-exec pair could. Then we followed
that.


> btw, if you were doing portable code, you would use eina_file for
> shm_open and not shm_open directly

http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/tree/src/bin/evas_cserve2_slave.c?h=cserve2#n200

Eina file does not support PROT_WRITE. We hope it can be added later,
after the current version is released.

Here we also talked to you, you mentioned that you had some ideas to
make the shm work for windows. Right?

If you check the code, we tried to keep these isolated to aid porting.
We're just not doing the port, then it is expected to miss some cases
that will be cleared when the first port happens.


> Some questions :
>
>  * what about the Coyote and ps3 arch ?

Same as cserve, it will not be supported due the lack of multi-process support.


>  * will cserve2 be optional like cserve ?

To start, yes. If it proves mature and to work well, Raster plans to
make it the default mode. If it goes well, it may be the only way. As
you can expect, there is a long road to it, including:
   - port to other systems (BSD, Windows...)
   - figure out what to do for too-different archs such as
single-process native PS3.

For the last point I have not many details. As far as I understand the
lack of multiprocess in PS3 is an implementation issue, that could be
added later? (KaKaRoTo voice in!).

If there is no way, we can always try to keep the existing code as
fallback, or these projects continue with an older Evas version due
their small user base. I have no clear opinion on those, then I can't
say anything concrete. Sorry.

What Raster already stated, and it may impact other platforms in
future, possibly guiding these points are:
   - Evas will hard-depend on threads;
   - Evas will revert to 32bpp-only, 16 and 8 will be removed.

The point is focus on most common use cases, easing maintenance and
reducing code base. If you check the current complexity of cache due
the introduced 16bpp (then 8bpp), it hurts. Also having the
always-thread render phase and similar will help to simplify the most
common cases out there.

Don't shoot the messenger (me)! :-)

-- 
Gustavo Sverzut Barbieri
http://profusion.mobi embedded systems
--------------------------------------
MSN: barbieri@...
Skype: gsbarbieri
Mobile: +55 (19) 9225-2202

On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
<barbieri@...> wrote:
>
> as for fork()/exec, we asked you what was better to use to help
> Windows. No idea if you recall, but we did ask you as our initial hope
> was to go with "fork()" only solution, you said it couldn't be
> supported on Windows, but the fork-exec pair could. Then we followed
> that.
>
>
>> btw, if you were doing portable code, you would use eina_file for
>> shm_open and not shm_open directly
>
> http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/tree/src/bin/evas_cserve2_slave.c?h=cserve2#n200
>
> Eina file does not support PROT_WRITE. We hope it can be added later,
> after the current version is released.
>
> Here we also talked to you, you mentioned that you had some ideas to
> make the shm work for windows. Right?

yes, but 'you' (that is only linux coders) use ultra specific linux
features. Having something in Windows which emulate more or less those
features is sometimes just impossible.

question : why didn't you add PROT_WRITE to eina_file ?

> If you check the code, we tried to keep these isolated to aid porting.
> We're just not doing the port, then it is expected to miss some cases
> that will be cleared when the first port happens.

there are some cross compilation toolchain to compile on all BSD,
solaris and windows. It seems that you didn't use them to see what
these "missed cases" are.

> Don't shoot the messenger (me)! :-)

the fact is : i'm fed up with you guys using linux-only features,
don't care at all about other OS. There are cross compilation
toolchain to check if it works or not, to port these features. You
don't use it. I do. But I'm beginning to have a lot less motivation.

Vincent

Hi,

On Thu, Apr 5, 2012 at 6:41 AM, Vincent Torri <vincent.torri@...> wrote:
> On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
> <barbieri@...> wrote:
>> as for fork()/exec, we asked you what was better to use to help
>> Windows. No idea if you recall, but we did ask you as our initial hope
>> was to go with "fork()" only solution, you said it couldn't be
>> supported on Windows, but the fork-exec pair could. Then we followed
>> that.
>>
>>> btw, if you were doing portable code, you would use eina_file for
>>> shm_open and not shm_open directly
>>
>> http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/tree/src/bin/evas_cserve2_slave.c?h=cserve2#n200
>>
>> Eina file does not support PROT_WRITE. We hope it can be added later,
>> after the current version is released.
>>
>> Here we also talked to you, you mentioned that you had some ideas to
>> make the shm work for windows. Right?
>
> yes, but 'you' (that is only linux coders) use ultra specific linux
> features. Having something in Windows which emulate more or less those
> features is sometimes just impossible.
>
> question : why didn't you add PROT_WRITE to eina_file ?

I think that was due to me in part. Eina_File didn't handle it, and it
was close to the release. So I have said that it would be better to
first implement what they need directly and once the release is done,
merge the usefull part in eina.
-- 
Cedric BAIL

On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
<barbieri@...> wrote:
> On Wed, Apr 4, 2012 at 7:23 PM, Vincent Torri <vincent.torri@...> wrote:
>> On Wed, Apr 4, 2012 at 7:34 PM, Rafael Antognolli
>> <antognolli@...> wrote:
>> Some questions :
>>  * what about the Coyote and ps3 arch ?
>
> Same as cserve, it will not be supported due the lack of multi-process support.

There is still many non multi-process system out there. I have been
thinking all this night how to make this work. At least they all have
some thread support. So maybe we could have a fallback mode that
instead of making cserve a standalone process, it will become a thread
task. In that case, it does have some implication, how to handle the
main loop ? We should make it possible to run multiple ecore main loop
or have a light main loop support. I don't know yet. The other
question will be, where should we launch this thread task, in evas ?
Then we have a build dependency in it. Or in Elementary, but it sounds
more like a work around. So I really don't know.

This is just throwing idea in the air, just in case someone get a
brilliant one and we can fix this use case. If not, that would be sad,
but we should have some time to find a proper solution.

>>  * will cserve2 be optional like cserve ?
>
> To start, yes. If it proves mature and to work well, Raster plans to
> make it the default mode. If it goes well, it may be the only way. As
> you can expect, there is a long road to it, including:
>   - port to other systems (BSD, Windows...)
>   - figure out what to do for too-different archs such as
> single-process native PS3.
>
> For the last point I have not many details. As far as I understand the
> lack of multiprocess in PS3 is an implementation issue, that could be
> added later? (KaKaRoTo voice in!).

They have no multiprocess, but they do have some kind of thread
support. So there is a way to support this feature at least in theory,
we just need to think about how.

> What Raster already stated, and it may impact other platforms in
> future, possibly guiding these points are:
>   - Evas will hard-depend on threads;
>   - Evas will revert to 32bpp-only, 16 and 8 will be removed.

I have some idea to improve the speed of 32bpp software engine to make
it closer to the 16bpp engine. I fully agree, that we should not need
the 16bpp engine that much in the future.
-- 
Cedric BAIL

On Thu, 5 Apr 2012 10:17:42 +0200 Cedric BAIL <cedric.bail@...>
wrote:

> On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
> <barbieri@...> wrote:
> > On Wed, Apr 4, 2012 at 7:23 PM, Vincent Torri
> > <vincent.torri@...> wrote:
> >> On Wed, Apr 4, 2012 at 7:34 PM, Rafael Antognolli
> >> <antognolli@...> wrote:
> >> Some questions :
> >>  * what about the Coyote and ps3 arch ?
> >
> > Same as cserve, it will not be supported due the lack of
> > multi-process support.
> 
> There is still many non multi-process system out there. I have been
> thinking all this night how to make this work. At least they all have
> some thread support. So maybe we could have a fallback mode that
> instead of making cserve a standalone process, it will become a thread
> task. In that case, it does have some implication, how to handle the
> main loop ? We should make it possible to run multiple ecore main loop
> or have a light main loop support. I don't know yet. The other
> question will be, where should we launch this thread task, in evas ?
> Then we have a build dependency in it. Or in Elementary, but it sounds
> more like a work around. So I really don't know.
> 
> This is just throwing idea in the air, just in case someone get a
> brilliant one and we can fix this use case. If not, that would be sad,
> but we should have some time to find a proper solution.
> 
> >>  * will cserve2 be optional like cserve ?
> >
> > To start, yes. If it proves mature and to work well, Raster plans to
> > make it the default mode. If it goes well, it may be the only way.
> > As you can expect, there is a long road to it, including:
> >   - port to other systems (BSD, Windows...)
> >   - figure out what to do for too-different archs such as
> > single-process native PS3.
> >
> > For the last point I have not many details. As far as I understand
> > the lack of multiprocess in PS3 is an implementation issue, that
> > could be added later? (KaKaRoTo voice in!).
> 
> They have no multiprocess, but they do have some kind of thread
> support. So there is a way to support this feature at least in theory,
> we just need to think about how.

PS3 has a hyperthreaded CPU.  Not counting the CELL SPUs, since code
for those has to be written specifically for them.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.

On Thu, Apr 5, 2012 at 11:00 AM, David Seikel <onefang@...> wrote:
> On Thu, 5 Apr 2012 10:17:42 +0200 Cedric BAIL <cedric.bail@...>
> wrote:
>
>> On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
>> <barbieri@...> wrote:
>> > On Wed, Apr 4, 2012 at 7:23 PM, Vincent Torri
>> > <vincent.torri@...> wrote:
>> >> On Wed, Apr 4, 2012 at 7:34 PM, Rafael Antognolli
>> >> <antognolli@...> wrote:
>> >> Some questions :
>> >>  * what about the Coyote and ps3 arch ?
>> >
>> > Same as cserve, it will not be supported due the lack of
>> > multi-process support.
>>
>> There is still many non multi-process system out there. I have been
>> thinking all this night how to make this work. At least they all have
>> some thread support. So maybe we could have a fallback mode that
>> instead of making cserve a standalone process, it will become a thread
>> task. In that case, it does have some implication, how to handle the
>> main loop ? We should make it possible to run multiple ecore main loop
>> or have a light main loop support. I don't know yet. The other
>> question will be, where should we launch this thread task, in evas ?
>> Then we have a build dependency in it. Or in Elementary, but it sounds
>> more like a work around. So I really don't know.
>>
>> This is just throwing idea in the air, just in case someone get a
>> brilliant one and we can fix this use case. If not, that would be sad,
>> but we should have some time to find a proper solution.
>>
>> >>  * will cserve2 be optional like cserve ?
>> >
>> > To start, yes. If it proves mature and to work well, Raster plans to
>> > make it the default mode. If it goes well, it may be the only way.
>> > As you can expect, there is a long road to it, including:
>> >   - port to other systems (BSD, Windows...)
>> >   - figure out what to do for too-different archs such as
>> > single-process native PS3.
>> >
>> > For the last point I have not many details. As far as I understand
>> > the lack of multiprocess in PS3 is an implementation issue, that
>> > could be added later? (KaKaRoTo voice in!).
>>
>> They have no multiprocess, but they do have some kind of thread
>> support. So there is a way to support this feature at least in theory,
>> we just need to think about how.
>
> PS3 has a hyperthreaded CPU.  Not counting the CELL SPUs, since code
> for those has to be written specifically for them.

even if that stuff is available, does the SDK provide these features ?
If I'm not mistaken, Kakaroto said that the SDK was "limited"

Vincent

On Thu, 5 Apr 2012 19:00:15 +1000 David Seikel <onefang@...>
wrote:

> On Thu, 5 Apr 2012 10:17:42 +0200 Cedric BAIL <cedric.bail@...>
> wrote:
> 
> > On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
> > <barbieri@...> wrote:
> > > On Wed, Apr 4, 2012 at 7:23 PM, Vincent Torri
> > > <vincent.torri@...> wrote:
> > >> On Wed, Apr 4, 2012 at 7:34 PM, Rafael Antognolli
> > >> <antognolli@...> wrote:
> > >> Some questions :
> > >>  * what about the Coyote and ps3 arch ?
> > >
> > > Same as cserve, it will not be supported due the lack of
> > > multi-process support.
> > 
> > There is still many non multi-process system out there. I have been
> > thinking all this night how to make this work. At least they all
> > have some thread support. So maybe we could have a fallback mode
> > that instead of making cserve a standalone process, it will become
> > a thread task. In that case, it does have some implication, how to
> > handle the main loop ? We should make it possible to run multiple
> > ecore main loop or have a light main loop support. I don't know
> > yet. The other question will be, where should we launch this thread
> > task, in evas ? Then we have a build dependency in it. Or in
> > Elementary, but it sounds more like a work around. So I really
> > don't know.
> > 
> > This is just throwing idea in the air, just in case someone get a
> > brilliant one and we can fix this use case. If not, that would be
> > sad, but we should have some time to find a proper solution.
> > 
> > >>  * will cserve2 be optional like cserve ?
> > >
> > > To start, yes. If it proves mature and to work well, Raster plans
> > > to make it the default mode. If it goes well, it may be the only
> > > way. As you can expect, there is a long road to it, including:
> > >   - port to other systems (BSD, Windows...)
> > >   - figure out what to do for too-different archs such as
> > > single-process native PS3.
> > >
> > > For the last point I have not many details. As far as I understand
> > > the lack of multiprocess in PS3 is an implementation issue, that
> > > could be added later? (KaKaRoTo voice in!).
> > 
> > They have no multiprocess, but they do have some kind of thread
> > support. So there is a way to support this feature at least in
> > theory, we just need to think about how.
> 
> PS3 has a hyperthreaded CPU.  Not counting the CELL SPUs, since code
> for those has to be written specifically for them.

On the other hand, my tiny embedded project uses a single 486 core.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.

On Thu, 5 Apr 2012 11:20:26 +0200 Vincent Torri
<vincent.torri@...> wrote:

> On Thu, Apr 5, 2012 at 11:00 AM, David Seikel <onefang@...>
> wrote:
> > On Thu, 5 Apr 2012 10:17:42 +0200 Cedric BAIL <cedric.bail@...>
> > wrote:
> >
> >> On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
> >> <barbieri@...> wrote:
> >> > On Wed, Apr 4, 2012 at 7:23 PM, Vincent Torri
> >> > <vincent.torri@...> wrote:
> >> >> On Wed, Apr 4, 2012 at 7:34 PM, Rafael Antognolli
> >> >> <antognolli@...> wrote:
> >> >> Some questions :
> >> >>  * what about the Coyote and ps3 arch ?
> >> >
> >> > Same as cserve, it will not be supported due the lack of
> >> > multi-process support.
> >>
> >> There is still many non multi-process system out there. I have been
> >> thinking all this night how to make this work. At least they all
> >> have some thread support. So maybe we could have a fallback mode
> >> that instead of making cserve a standalone process, it will become
> >> a thread task. In that case, it does have some implication, how to
> >> handle the main loop ? We should make it possible to run multiple
> >> ecore main loop or have a light main loop support. I don't know
> >> yet. The other question will be, where should we launch this
> >> thread task, in evas ? Then we have a build dependency in it. Or
> >> in Elementary, but it sounds more like a work around. So I really
> >> don't know.
> >>
> >> This is just throwing idea in the air, just in case someone get a
> >> brilliant one and we can fix this use case. If not, that would be
> >> sad, but we should have some time to find a proper solution.
> >>
> >> >>  * will cserve2 be optional like cserve ?
> >> >
> >> > To start, yes. If it proves mature and to work well, Raster
> >> > plans to make it the default mode. If it goes well, it may be
> >> > the only way. As you can expect, there is a long road to it,
> >> > including:
> >> >   - port to other systems (BSD, Windows...)
> >> >   - figure out what to do for too-different archs such as
> >> > single-process native PS3.
> >> >
> >> > For the last point I have not many details. As far as I
> >> > understand the lack of multiprocess in PS3 is an implementation
> >> > issue, that could be added later? (KaKaRoTo voice in!).
> >>
> >> They have no multiprocess, but they do have some kind of thread
> >> support. So there is a way to support this feature at least in
> >> theory, we just need to think about how.
> >
> > PS3 has a hyperthreaded CPU.  Not counting the CELL SPUs, since code
> > for those has to be written specifically for them.
> 
> even if that stuff is available, does the SDK provide these features ?
> If I'm not mistaken, Kakaroto said that the SDK was "limited"

I dunno how limited the native SDK is, but back when you could put
Linux on the PS3, you got Linux access to the entire CPU, and all but
one of the SPUs.  No Linux access to the GPU, at least not direct
access.

On the native SDK I'd expect that you are still missing access to one
of the SPUs.  This does not count the SPU that is simply disabled
to increase chip yields.

On the other hand, Sony has locked the thing down more since last I
programmed with it.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.

On Thu, Apr 5, 2012 at 6:24 AM, David Seikel <onefang@...> wrote:
> On Thu, 5 Apr 2012 19:00:15 +1000 David Seikel <onefang@...>
> wrote:
>
>> On Thu, 5 Apr 2012 10:17:42 +0200 Cedric BAIL <cedric.bail@...>
>> wrote:
>>
>> > On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
>> > <barbieri@...> wrote:
>> > > On Wed, Apr 4, 2012 at 7:23 PM, Vincent Torri
>> > > <vincent.torri@...> wrote:
>> > >> On Wed, Apr 4, 2012 at 7:34 PM, Rafael Antognolli
>> > >> <antognolli@...> wrote:
>> > >> Some questions :
>> > >>  * what about the Coyote and ps3 arch ?
>> > >
>> > > Same as cserve, it will not be supported due the lack of
>> > > multi-process support.
>> >
>> > There is still many non multi-process system out there. I have been
>> > thinking all this night how to make this work. At least they all
>> > have some thread support. So maybe we could have a fallback mode
>> > that instead of making cserve a standalone process, it will become
>> > a thread task. In that case, it does have some implication, how to
>> > handle the main loop ? We should make it possible to run multiple
>> > ecore main loop or have a light main loop support. I don't know
>> > yet. The other question will be, where should we launch this thread
>> > task, in evas ? Then we have a build dependency in it. Or in
>> > Elementary, but it sounds more like a work around. So I really
>> > don't know.
>> >
>> > This is just throwing idea in the air, just in case someone get a
>> > brilliant one and we can fix this use case. If not, that would be
>> > sad, but we should have some time to find a proper solution.
>> >
>> > >>  * will cserve2 be optional like cserve ?
>> > >
>> > > To start, yes. If it proves mature and to work well, Raster plans
>> > > to make it the default mode. If it goes well, it may be the only
>> > > way. As you can expect, there is a long road to it, including:
>> > >   - port to other systems (BSD, Windows...)
>> > >   - figure out what to do for too-different archs such as
>> > > single-process native PS3.
>> > >
>> > > For the last point I have not many details. As far as I understand
>> > > the lack of multiprocess in PS3 is an implementation issue, that
>> > > could be added later? (KaKaRoTo voice in!).
>> >
>> > They have no multiprocess, but they do have some kind of thread
>> > support. So there is a way to support this feature at least in
>> > theory, we just need to think about how.
>>
>> PS3 has a hyperthreaded CPU.  Not counting the CELL SPUs, since code
>> for those has to be written specifically for them.
>
> On the other hand, my tiny embedded project uses a single 486 core.

We appreciate if you can check the impact on it.

Of course it depends on the use cases, if there is a single process
doing UI and it's loading just safe data (ie: edje/eet), then there is
no gain... but penalty shouldn't be o heavy.

But if there are multiple process as the normal case, then the
benefits should exist on memory saving.

Another clear benefit is dealing with user-media, such as mmc/sd with
huge JPEG, or broken filesytems... The processes shouldn't crash or
hang anymore.

Last but not least, a benefit we foresee is avoiding blocking the
application main loop. The end goal is to have cserve2 to do
speculative preload. But even without it, when we go further with the
project, it is expected that the IO an CPU used to load and decode the
image, is on the secondary processes and the main application can work
without blocking user code. This will be done by a multi-threaded
render phase (one thread to calculate changes, send to another thread
to request resource load, yet another to execute the drawing commands
-- drop frames between these as required).

Anyway, likely impossible to make everyone super-happy, but our goal,
including Raster, is to focus on the biggest bucket that is desktop
and high-end embedded systems.


-- 
Gustavo Sverzut Barbieri
http://profusion.mobi embedded systems
--------------------------------------
MSN: barbieri@...
Skype: gsbarbieri
Mobile: +55 (19) 9225-2202

On Thu, Apr 5, 2012 at 5:17 AM, Cedric BAIL <cedric.bail@...> wrote:
> On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
> <barbieri@...> wrote:
>> On Wed, Apr 4, 2012 at 7:23 PM, Vincent Torri <vincent.torri@...> wrote:
>>> On Wed, Apr 4, 2012 at 7:34 PM, Rafael Antognolli
>>> <antognolli@...> wrote:
>>> Some questions :
>>>  * what about the Coyote and ps3 arch ?
>>
>> Same as cserve, it will not be supported due the lack of multi-process support.
>
> There is still many non multi-process system out there. I have been
> thinking all this night how to make this work. At least they all have
> some thread support. So maybe we could have a fallback mode that
> instead of making cserve a standalone process, it will become a thread
> task. In that case, it does have some implication, how to handle the
> main loop ? We should make it possible to run multiple ecore main loop
> or have a light main loop support. I don't know yet. The other
> question will be, where should we launch this thread task, in evas ?
> Then we have a build dependency in it. Or in Elementary, but it sounds
> more like a work around. So I really don't know.
>
> This is just throwing idea in the air, just in case someone get a
> brilliant one and we can fix this use case. If not, that would be sad,
> but we should have some time to find a proper solution.

As said if people are highly demanding, there could be a fallback to
single-processes cases. But as I understand raster's will is to make
it the default one, after it's better tested of course.

Then we need people to test it ;-)


>>>  * will cserve2 be optional like cserve ?
>>
>> To start, yes. If it proves mature and to work well, Raster plans to
>> make it the default mode. If it goes well, it may be the only way. As
>> you can expect, there is a long road to it, including:
>>   - port to other systems (BSD, Windows...)
>>   - figure out what to do for too-different archs such as
>> single-process native PS3.
>>
>> For the last point I have not many details. As far as I understand the
>> lack of multiprocess in PS3 is an implementation issue, that could be
>> added later? (KaKaRoTo voice in!).
>
> They have no multiprocess, but they do have some kind of thread
> support. So there is a way to support this feature at least in theory,
> we just need to think about how.

the code can be changed to work in threads instead of processes. It's
not single-flag/one-liner, but interesting peers can work towards
that. The choice of multi-process is based on safety it allows, like
the crashing loaders. It also remove the need of "generic loaders", as
the cserve2 process can be modified to load single-format slave
loaders in future. Then the slave is not linked with any GPL/BSD and
is safe to be proprietary. (this is possible but not implemented yet).



>> What Raster already stated, and it may impact other platforms in
>> future, possibly guiding these points are:
>>   - Evas will hard-depend on threads;
>>   - Evas will revert to 32bpp-only, 16 and 8 will be removed.
>
> I have some idea to improve the speed of 32bpp software engine to make
> it closer to the 16bpp engine. I fully agree, that we should not need
> the 16bpp engine that much in the future.

Fortunately the customer demand for "iphone like" user interfaces on
everything is forcing dynamically programmable GPU such as OpenGL-ES,
and this also comes with benefit to process 32bits and convert to 16
really fast.

yes, i know some cost-constrained cases may continue in 16bpp with
old-fashioned acceleration like set-top boxes with directfb. But there
are choices to be made, not by me :-)



-- 
Gustavo Sverzut Barbieri
http://profusion.mobi embedded systems
--------------------------------------
MSN: barbieri@...
Skype: gsbarbieri
Mobile: +55 (19) 9225-2202

On Thu, 5 Apr 2012 06:41:56 +0200 Vincent Torri <vincent.torri@...> said:

> On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
> <barbieri@...> wrote:
> >
> > as for fork()/exec, we asked you what was better to use to help
> > Windows. No idea if you recall, but we did ask you as our initial hope
> > was to go with "fork()" only solution, you said it couldn't be
> > supported on Windows, but the fork-exec pair could. Then we followed
> > that.
> >
> >
> >> btw, if you were doing portable code, you would use eina_file for
> >> shm_open and not shm_open directly
> >
> > http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/tree/src/bin/evas_cserve2_slave.c?h=cserve2#n200
> >
> > Eina file does not support PROT_WRITE. We hope it can be added later,
> > after the current version is released.
> >
> > Here we also talked to you, you mentioned that you had some ideas to
> > make the shm work for windows. Right?
> 
> yes, but 'you' (that is only linux coders) use ultra specific linux
> features. Having something in Windows which emulate more or less those
> features is sometimes just impossible.
> 
> question : why didn't you add PROT_WRITE to eina_file ?

right now eina is frozen and this can be done later. it's a start. the
eina_file stuff is only half useful as it supports only opening existing shm
segs, not creating them. ALSO existing cserve src uses shm_open already, so its
no WORSE. :)

> > If you check the code, we tried to keep these isolated to aid porting.
> > We're just not doing the port, then it is expected to miss some cases
> > that will be cleared when the first port happens.
> 
> there are some cross compilation toolchain to compile on all BSD,
> solaris and windows. It seems that you didn't use them to see what
> these "missed cases" are.
> 
> > Don't shoot the messenger (me)! :-)
> 
> the fact is : i'm fed up with you guys using linux-only features,
> don't care at all about other OS. There are cross compilation
> toolchain to check if it works or not, to port these features. You
> don't use it. I do. But I'm beginning to have a lot less motivation.

by the same token, in the linux world we COULD set up chroots for gentoo, arch,
debian, ubuntu, fedora, suse, centos...you get the idea - but we don't as its
frankly a lot of work and disk space. just accept the fact that each person
looks after a port for their favorite platforms - that's what we all do. do you
see me saying "screw you all "use ubuntu or go away?" (not saying you said that
though). i accept that each person will look after their little world of
interest and between all of us we manage to keep things portable.

-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@...

On Thu, 5 Apr 2012 10:17:42 +0200 Cedric BAIL <cedric.bail@...> said:

> On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
> <barbieri@...> wrote:
> > On Wed, Apr 4, 2012 at 7:23 PM, Vincent Torri <vincent.torri@...>
> > wrote:
> >> On Wed, Apr 4, 2012 at 7:34 PM, Rafael Antognolli
> >> <antognolli@...> wrote:
> >> Some questions :
> >>  * what about the Coyote and ps3 arch ?
> >
> > Same as cserve, it will not be supported due the lack of multi-process
> > support.
> 
> There is still many non multi-process system out there. I have been

seriously though. shall we hobble then 99.9% in favor of the 0.1% that can't do
1 process (somehow)? :) we can't remain stuck in such a world forever. we
already have a fairly conservative and old architecture. technically  you can
implement a multi-process model as threads within 1 process, as long as the
system has threading. almost anything worth talking about does, and if it
doesn't its able to be implemented using timer and stack switch hacks.

> thinking all this night how to make this work. At least they all have
> some thread support. So maybe we could have a fallback mode that
> instead of making cserve a standalone process, it will become a thread

yes. i'd worry about this later though. as long as its possible.

> task. In that case, it does have some implication, how to handle the
> main loop ? We should make it possible to run multiple ecore main loop
> or have a light main loop support. I don't know yet. The other

it's evas - no ecore here. :)

> question will be, where should we launch this thread task, in evas ?
> Then we have a build dependency in it. Or in Elementary, but it sounds
> more like a work around. So I really don't know.

evas. if its a thread fallback. if its a process evas can also launch it, if
not already there, but i'd have the desktop env launch it first so its there
already for everyone. evas's own launching of the daemon is a compat fallback :)

> This is just throwing idea in the air, just in case someone get a
> brilliant one and we can fix this use case. If not, that would be sad,
> but we should have some time to find a proper solution.
> 
> >>  * will cserve2 be optional like cserve ?
> >
> > To start, yes. If it proves mature and to work well, Raster plans to
> > make it the default mode. If it goes well, it may be the only way. As
> > you can expect, there is a long road to it, including:
> >   - port to other systems (BSD, Windows...)
> >   - figure out what to do for too-different archs such as
> > single-process native PS3.
> >
> > For the last point I have not many details. As far as I understand the
> > lack of multiprocess in PS3 is an implementation issue, that could be
> > added later? (KaKaRoTo voice in!).
> 
> They have no multiprocess, but they do have some kind of thread
> support. So there is a way to support this feature at least in theory,
> we just need to think about how.

they do - but no "full/complete" pthread support -s o eventually there may need
to either need to be an adaption or port on our part, but we can't avoid
threads any longer. we can't hamstring the whole codebase because of platforms
unable to do simple things like threading. :)

> > What Raster already stated, and it may impact other platforms in
> > future, possibly guiding these points are:
> >   - Evas will hard-depend on threads;
> >   - Evas will revert to 32bpp-only, 16 and 8 will be removed.
> 
> I have some idea to improve the speed of 32bpp software engine to make
> it closer to the 16bpp engine. I fully agree, that we should not need
> the 16bpp engine that much in the future.

yeah. improve where possible so its less of a difference, but reality is that
the world of 16bpp displays is slowly going away. :)

-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@...

On Thu, 5 Apr 2012 14:56:15 -0300 Gustavo Sverzut Barbieri
<barbieri@...> said:

> On Thu, Apr 5, 2012 at 6:24 AM, David Seikel <onefang@...> wrote:
> > On Thu, 5 Apr 2012 19:00:15 +1000 David Seikel <onefang@...>
> > wrote:
> >
> >> On Thu, 5 Apr 2012 10:17:42 +0200 Cedric BAIL <cedric.bail@...>
> >> wrote:
> >>
> >> > On Thu, Apr 5, 2012 at 1:29 AM, Gustavo Sverzut Barbieri
> >> > <barbieri@...> wrote:
> >> > > On Wed, Apr 4, 2012 at 7:23 PM, Vincent Torri
> >> > > <vincent.torri@...> wrote:
> >> > >> On Wed, Apr 4, 2012 at 7:34 PM, Rafael Antognolli
> >> > >> <antognolli@...> wrote:
> >> > >> Some questions :
> >> > >>  * what about the Coyote and ps3 arch ?
> >> > >
> >> > > Same as cserve, it will not be supported due the lack of
> >> > > multi-process support.
> >> >
> >> > There is still many non multi-process system out there. I have been
> >> > thinking all this night how to make this work. At least they all
> >> > have some thread support. So maybe we could have a fallback mode
> >> > that instead of making cserve a standalone process, it will become
> >> > a thread task. In that case, it does have some implication, how to
> >> > handle the main loop ? We should make it possible to run multiple
> >> > ecore main loop or have a light main loop support. I don't know
> >> > yet. The other question will be, where should we launch this thread
> >> > task, in evas ? Then we have a build dependency in it. Or in
> >> > Elementary, but it sounds more like a work around. So I really
> >> > don't know.
> >> >
> >> > This is just throwing idea in the air, just in case someone get a
> >> > brilliant one and we can fix this use case. If not, that would be
> >> > sad, but we should have some time to find a proper solution.
> >> >
> >> > >>  * will cserve2 be optional like cserve ?
> >> > >
> >> > > To start, yes. If it proves mature and to work well, Raster plans
> >> > > to make it the default mode. If it goes well, it may be the only
> >> > > way. As you can expect, there is a long road to it, including:
> >> > >   - port to other systems (BSD, Windows...)
> >> > >   - figure out what to do for too-different archs such as
> >> > > single-process native PS3.
> >> > >
> >> > > For the last point I have not many details. As far as I understand
> >> > > the lack of multiprocess in PS3 is an implementation issue, that
> >> > > could be added later? (KaKaRoTo voice in!).
> >> >
> >> > They have no multiprocess, but they do have some kind of thread
> >> > support. So there is a way to support this feature at least in
> >> > theory, we just need to think about how.
> >>
> >> PS3 has a hyperthreaded CPU.  Not counting the CELL SPUs, since code
> >> for those has to be written specifically for them.
> >
> > On the other hand, my tiny embedded project uses a single 486 core.

won't matter. you have a linux kernel. it can schedule threads and processes
for you. hell it can PRIORITISE processes nicely. you can drop the priority
(nice value) of the daemon to ensure it almost entirely ONLY uses idle cycles,
actually IMPROVING responsiveness as decode/io is only done when there are
spare cycles. man sched_setscheduler() and look at SCHED_IDLE. an added bonus
is also security and stability. no longer can an image loading lib crash your
app. it will just crash that loader slave. you lose that image, not your whole
world. :)

> We appreciate if you can check the impact on it.
> 
> Of course it depends on the use cases, if there is a single process
> doing UI and it's loading just safe data (ie: edje/eet), then there is
> no gain... but penalty shouldn't be o heavy.

with some prioritisation options it should actually be better ultimately. :)

> But if there are multiple process as the normal case, then the
> benefits should exist on memory saving.
> 
> Another clear benefit is dealing with user-media, such as mmc/sd with
> huge JPEG, or broken filesytems... The processes shouldn't crash or
> hang anymore.

aye squiddie! :)

> Last but not least, a benefit we foresee is avoiding blocking the
> application main loop. The end goal is to have cserve2 to do
> speculative preload. But even without it, when we go further with the
> project, it is expected that the IO an CPU used to load and decode the
> image, is on the secondary processes and the main application can work
> without blocking user code. This will be done by a multi-threaded
> render phase (one thread to calculate changes, send to another thread
> to request resource load, yet another to execute the drawing commands
> -- drop frames between these as required).
> 
> Anyway, likely impossible to make everyone super-happy, but our goal,
> including Raster, is to focus on the biggest bucket that is desktop
> and high-end embedded systems.

yeah. frankly our userbase is this. also the developerbase - the majority.
technically this CAN work on windows as it has all the facilities needed.
design-wise its ok. it just needs glue. it can work on all unixes. it can work
on osx, android and even wince (though the 32 process limit may become an issue
if we don't keep daemon down to 1 instance and # of decoder slaves down to 1 or
2). this should work on ps3 too - but via threading emulation so u dont get the
stability/security improvements. but it should be workable.

-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@...

On Wed, 4 Apr 2012 11:52:29 -0300 Rafael Antognolli <antognolli@...>
said:

i been lookin' at ur codez yo. :)

general question... what's up with image saves? :) (makes sense to have them
salved off but as a direct child of the process not of the resource loader?)

so partly i see old code moved/copied in from core loader infra. ok - cool.

what is this cserve2_timeout_cb_set() every 3000ms? do we really need to be
spinning every 3000ms killing slaves? just timeout once, kill and only queue
timeout if we start slaves again? that's what it looks like.

evas_cserve2_cache.c i think could be broken up into actual JUST cache handler
(in memory has+LRU list) file watching and some message and protocol handling
files.

i see that you havent come up with a good/proper socket naming scheme yet...
/tmp/cserve2.socket... yay! :)

other than that - the evas_cserve2_slave.c could do with a little more
abstracting so it COULD run as a thread inside of cserve (ps3?) and
evas_cserve2_main.c. specifically the main() funcs could be one level up from
their content so pthread_create() can call the children if needed later.

that's it for now. feedback done :)


> Hello people,
> 
> As some people know, ProFUSION is also working on Evas and making some
> changes that should make it even faster.
> 
> Currently we are working on the images cache server, and it should be
> similar to the previous cserve, but with a more asynchronous API. It
> also has some fundamental changes, like having a pool of loaders where
> each of them run on a different process.
> 
> The main idea is that the client can request the images needed as soon
> as a file_set is done, but don't block waiting for them to load. Once
> Evas really needs the image data, it will finally block to get this
> data, and should return almost immediately if it was already loaded.
> 
> Some tweaks still must be done on the current code, some cleanup, and
> even some documentation, but the core is already there. A new internal
> cache was also done for Evas, very similar to the previous one but
> with some more direct calls to the server and some shorter code paths.
> 
> The code can be seen here:
> http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/ (branch cserve2)
> 
> Any comments are welcome!
> -- 
> Rafael Antognolli
> ProFUSION embedded systems
> http://profusion.mobi
> 
> ------------------------------------------------------------------------------
> Better than sec? Nothing is better than sec when it comes to
> monitoring Big Data applications. Try Boundary one-second 
> resolution app monitoring today. Free.
> http://p.sf.net/sfu/Boundary-dev2dev
> _______________________________________________
> enlightenment-devel mailing list
> enlightenment-devel@...
> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
> 


-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@...

Wassssuuuuuuuuuuuuuuuuuuup

2012/4/12 Carsten Haitzler <raster@...>:
> On Wed, 4 Apr 2012 11:52:29 -0300 Rafael Antognolli <antognolli@...>
> said:
>
> i been lookin' at ur codez yo. :)
>
> general question... what's up with image saves? :) (makes sense to have them
> salved off but as a direct child of the process not of the resource loader?)
>

wat

About the image savers, we have no plans to put that in the server,
but keep them in Evas as they stand. Having the savers in cserve would
add a significant overhead for no benefit.


> so partly i see old code moved/copied in from core loader infra. ok - cool.
>
> what is this cserve2_timeout_cb_set() every 3000ms? do we really need to be
> spinning every 3000ms killing slaves? just timeout once, kill and only queue
> timeout if we start slaves again? that's what it looks like.

That's on the TODO. The idea is not only to avoid the unnecessary timeout,
but also keep slaves around until there are no more requests. Right now if the
timeout triggers, slaves will die and be immediately restarted if
there are still
pending tasks.
Focus, however, is on getting Elementary and E17 to work flawlessly with
cserve2 before we tackle those issues.

>
> evas_cserve2_cache.c i think could be broken up into actual JUST cache handler
> (in memory has+LRU list) file watching and some message and protocol handling
> files.
>

A nice idea, it's turning into a beast already.

> i see that you havent come up with a good/proper socket naming scheme yet...
> /tmp/cserve2.socket... yay! :)

At least it has a name, the first version we did was recursively opening
every file on the system until one sent a valid response, then it was assumed
to be the cserve socket. Security wise it was wonderful, as no one had any way
of knowing what name or where the socket was to attempt any hack, but when
opening an image took 3 hours we decided to go with a more conservative method.

>
> other than that - the evas_cserve2_slave.c could do with a little more
> abstracting so it COULD run as a thread inside of cserve (ps3?) and
> evas_cserve2_main.c. specifically the main() funcs could be one level up from
> their content so pthread_create() can call the children if needed later.
>

Yep, the slave was supposed to be a simple test to make sure things were
working fine, then laziness won and it ended up being the real thing. As it
stands, it's not even as Windows friendly as I wanted originally, but it will
be there by the time it hits SVN. As said above, getting the big picture to
work as expected is the priority.

> that's it for now. feedback done :)
>

Thanks, there'll be more to look at and test soon enough.

>
>> Hello people,
>>
>> As some people know, ProFUSION is also working on Evas and making some
>> changes that should make it even faster.
>>
>> Currently we are working on the images cache server, and it should be
>> similar to the previous cserve, but with a more asynchronous API. It
>> also has some fundamental changes, like having a pool of loaders where
>> each of them run on a different process.
>>
>> The main idea is that the client can request the images needed as soon
>> as a file_set is done, but don't block waiting for them to load. Once
>> Evas really needs the image data, it will finally block to get this
>> data, and should return almost immediately if it was already loaded.
>>
>> Some tweaks still must be done on the current code, some cleanup, and
>> even some documentation, but the core is already there. A new internal
>> cache was also done for Evas, very similar to the previous one but
>> with some more direct calls to the server and some shorter code paths.
>>
>> The code can be seen here:
>> http://git.profusion.mobi/cgit.cgi/antognolli/evas-cserve2/ (branch cserve2)
>>
>> Any comments are welcome!
>> --
>> Rafael Antognolli
>> ProFUSION embedded systems
>> http://profusion.mobi
>>
>> ------------------------------------------------------------------------------
>> Better than sec? Nothing is better than sec when it comes to
>> monitoring Big Data applications. Try Boundary one-second
>> resolution app monitoring today. Free.
>> http://p.sf.net/sfu/Boundary-dev2dev
>> _______________________________________________
>> enlightenment-devel mailing list
>> enlightenment-devel@...
>> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
>>
>
>
> --
> ------------- Codito, ergo sum - "I code, therefore I am" --------------
> The Rasterman (Carsten Haitzler)    raster@...
>
>
> ------------------------------------------------------------------------------
> For Developers, A Lot Can Happen In A Second.
> Boundary is the first to Know...and Tell You.
> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
> http://p.sf.net/sfu/Boundary-d2dvs2
> _______________________________________________
> enlightenment-devel mailing list
> enlightenment-devel@...
> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel

Hello guys,

Besides the issues raised about the code, we are proud to announce
that e17 can work with the cache server already!

We would like to ask you to test it if possible, and report any
problems found. We are still working on it and this is a good moment
to solve bugs before starting to work on new features that would come
soon.

PS: We push-forced the repository since it's rebased on the latest
version of Evas. And we are still working on the issues.

Regads,
-- 
Rafael Antognolli
ProFUSION embedded systems
http://profusion.mobi

So no comments at all?

We are planning to commit this to svn today, after fixing some pending
bits. So feedback before that would be very good, but you'll have the
chance to look at the code when it hits the repository.

On Tue, Apr 24, 2012 at 6:53 PM, Rafael Antognolli
<antognolli@...> wrote:
> Hello guys,
>
> Besides the issues raised about the code, we are proud to announce
> that e17 can work with the cache server already!
>
> We would like to ask you to test it if possible, and report any
> problems found. We are still working on it and this is a good moment
> to solve bugs before starting to work on new features that would come
> soon.
>
> PS: We push-forced the repository since it's rebased on the latest
> version of Evas. And we are still working on the issues.

Regards,
-- 
Rafael Antognolli
ProFUSION embedded systems
http://profusion.mobi

O HAI

We took a little bit more time than expected, but
CServe2 is now in SVN for everyone to use, test and enjoy.

>From the commit message:

Introducing Cache Serve 2.

This cache server will initially load images for clients connected to
it. It starts slave processes to load these images, and share the loaded
images through shm with the clients. All the connection done between
clients and the server goes through sockets.

The cserve2 build option is turned on by default, while the old cserve
was disabled, but in order to make clients use it, the environment
variable EVAS_CSERVE2 must be set, and a server must be running.

Clients will try to find the socket on a specified location using the
environment variable EVAS_CSERVE2_SOCKET. If it's not defined, then the
XDG_RUNTIME_DIR path should be used, and finally HOME, TMPDIR and /tmp.

-------------------

We know that some issues that were raised in this very same thread
have not been fixed yet, but they will, we promise. The work on the font
cache server is starting and we want to avoid refactoring things to fix
them only to refactor them again later because the font stuff requires it.

Greetings, good sirs and ladies.

2012/4/27 Rafael Antognolli <antognolli@...>:
> So no comments at all?
>
> We are planning to commit this to svn today, after fixing some pending
> bits. So feedback before that would be very good, but you'll have the
> chance to look at the code when it hits the repository.
>
> On Tue, Apr 24, 2012 at 6:53 PM, Rafael Antognolli
> <antognolli@...> wrote:
>> Hello guys,
>>
>> Besides the issues raised about the code, we are proud to announce
>> that e17 can work with the cache server already!
>>
>> We would like to ask you to test it if possible, and report any
>> problems found. We are still working on it and this is a good moment
>> to solve bugs before starting to work on new features that would come
>> soon.
>>
>> PS: We push-forced the repository since it's rebased on the latest
>> version of Evas. And we are still working on the issues.
>
> Regards,
> --
> Rafael Antognolli
> ProFUSION embedded systems
> http://profusion.mobi
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> enlightenment-devel mailing list
> enlightenment-devel@...
> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel

On Tue, 24 Apr 2012 18:53:01 -0300 Rafael Antognolli
<antognolli@...> said:

is there a readme/doc/something saying how this is meant to work?

> Hello guys,
> 
> Besides the issues raised about the code, we are proud to announce
> that e17 can work with the cache server already!
> 
> We would like to ask you to test it if possible, and report any
> problems found. We are still working on it and this is a good moment
> to solve bugs before starting to work on new features that would come
> soon.
> 
> PS: We push-forced the repository since it's rebased on the latest
> version of Evas. And we are still working on the issues.
> 
> Regads,
> -- 
> Rafael Antognolli
> ProFUSION embedded systems
> http://profusion.mobi
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> enlightenment-devel mailing list
> enlightenment-devel@...
> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
> 


-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@...

On Tue, 24 Apr 2012 18:53:01 -0300 Rafael Antognolli
<antognolli@...> said:

one thing... socket name selection... try /tmp/.evas-cserve2-%x.socket first...
well after EVAS_CSERVE2_SOCKET. :) in fact i just fixed it for u in svn to at
least it works out of the box. :)

> Hello guys,
> 
> Besides the issues raised about the code, we are proud to announce
> that e17 can work with the cache server already!
> 
> We would like to ask you to test it if possible, and report any
> problems found. We are still working on it and this is a good moment
> to solve bugs before starting to work on new features that would come
> soon.
> 
> PS: We push-forced the repository since it's rebased on the latest
> version of Evas. And we are still working on the issues.
> 
> Regads,
> -- 
> Rafael Antognolli
> ProFUSION embedded systems
> http://profusion.mobi
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> enlightenment-devel mailing list
> enlightenment-devel@...
> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
> 


-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@...

On Mon, May 21, 2012 at 10:45 AM, Carsten Haitzler <raster@...> wrote:
> On Tue, 24 Apr 2012 18:53:01 -0300 Rafael Antognolli
> <antognolli@...> said:
>
> one thing... socket name selection... try /tmp/.evas-cserve2-%x.socket first...
> well after EVAS_CSERVE2_SOCKET. :) in fact i just fixed it for u in svn to at
> least it works out of the box. :)

i didn't look at the code yet, but i hope that that kind of code is in
the linux specific file...

Vincent

On Thu, 3 May 2012 18:55:08 -0300 Iván Briano <sachieru@...> said:

aaah here. i figured this out by now. i also fixed all tools to
think/work/look at the same socket name in /tmp  :)

> O HAI
> 
> We took a little bit more time than expected, but
> CServe2 is now in SVN for everyone to use, test and enjoy.
> 
> >From the commit message:
> 
> Introducing Cache Serve 2.
> 
> This cache server will initially load images for clients connected to
> it. It starts slave processes to load these images, and share the loaded
> images through shm with the clients. All the connection done between
> clients and the server goes through sockets.
> 
> The cserve2 build option is turned on by default, while the old cserve
> was disabled, but in order to make clients use it, the environment
> variable EVAS_CSERVE2 must be set, and a server must be running.
> 
> Clients will try to find the socket on a specified location using the
> environment variable EVAS_CSERVE2_SOCKET. If it's not defined, then the
> XDG_RUNTIME_DIR path should be used, and finally HOME, TMPDIR and /tmp.
> 
> -------------------
> 
> We know that some issues that were raised in this very same thread
> have not been fixed yet, but they will, we promise. The work on the font
> cache server is starting and we want to avoid refactoring things to fix
> them only to refactor them again later because the font stuff requires it.
> 
> Greetings, good sirs and ladies.
> 
> 2012/4/27 Rafael Antognolli <antognolli@...>:
> > So no comments at all?
> >
> > We are planning to commit this to svn today, after fixing some pending
> > bits. So feedback before that would be very good, but you'll have the
> > chance to look at the code when it hits the repository.
> >
> > On Tue, Apr 24, 2012 at 6:53 PM, Rafael Antognolli
> > <antognolli@...> wrote:
> >> Hello guys,
> >>
> >> Besides the issues raised about the code, we are proud to announce
> >> that e17 can work with the cache server already!
> >>
> >> We would like to ask you to test it if possible, and report any
> >> problems found. We are still working on it and this is a good moment
> >> to solve bugs before starting to work on new features that would come
> >> soon.
> >>
> >> PS: We push-forced the repository since it's rebased on the latest
> >> version of Evas. And we are still working on the issues.
> >
> > Regards,
> > --
> > Rafael Antognolli
> > ProFUSION embedded systems
> > http://profusion.mobi
> >
> > ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's security and
> > threat landscape has changed and how IT managers can respond. Discussions
> > will include endpoint security, mobile security and the latest in malware
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > enlightenment-devel mailing list
> > enlightenment-devel@...
> > https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> enlightenment-devel mailing list
> enlightenment-devel@...
> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
> 


-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@...

On Mon, 21 May 2012 10:51:38 +0200 Vincent Torri <vincent.torri@...> said:

> On Mon, May 21, 2012 at 10:45 AM, Carsten Haitzler <raster@...>
> wrote:
> > On Tue, 24 Apr 2012 18:53:01 -0300 Rafael Antognolli
> > <antognolli@...> said:
> >
> > one thing... socket name selection... try /tmp/.evas-cserve2-%x.socket
> > first... well after EVAS_CSERVE2_SOCKET. :) in fact i just fixed it for u
> > in svn to at least it works out of the box. :)
> 
> i didn't look at the code yet, but i hope that that kind of code is in
> the linux specific file...

not all of it... - client sides are not.

-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@...