Re: [E-devel] [RFC] One binary to rule them all, packagers opinion needed

[Cedric B. <ced...@fr...>]

On Mon, Oct 24, 2016 at 1:11 AM, Stefan Schmidt <st...@os...> wrote:
> Hello.
>
> On 22/10/16 00:53, Cedric BAIL wrote:
>> On Fri, Oct 21, 2016 at 3:31 PM, Stefan Schmidt <st...@os...> wrote:
>>> On 22/10/16 00:14, Cedric BAIL wrote:
>>>> Before I embark on a crazy idea that may just break everything, I
>>>> would like to get some feedback. So one of the problem we are facing
>>>> with EFL is that with all those separated library we end up with some
>>>> serious time during application startup dedicated to linking. A work
>>>> around has been quicklaunch, but it is mostly unused. Also it does
>>>> only improve things once it has started, not much during startup time
>>>> where you still have to do all the linking.
>>>>
>>>> With PIE and -rdynamic that we use today on quicklaunch binary, we use
>>>> dlopen to get symbol from it and load this binary as if they were
>>>> library... So maybe we can actually use that same mechanism to
>>>> actually statically link efl with the quicklaunch server binary and
>>>> make all efl .so library just a symlink to that binary. Now,
>>>> enlightenment could actually try to rely on quicklaunch to start
>>>> itself and start other application. This should speed up boot time and
>>>> application start up time. The reason to make it the default way of
>>>> doing things is to make sure that it is maintained and work.
>>>
>>> Are there actual numbers about speed improvement? Are we really needed
>>> this speedup so badly? As you can already see from this two sentences
>>> I'm not really in favor of it. See below for some arguments.
>>
>> Yes, it does, but really depends on your hardware.
>
> And by making it the default your are forcing this quite ugly change to
> all EFL users while only certain hardware setups would really see a
> significant benefit.

Well, the pay off on startup is paying a lot more on the lower end of
the spectrum for sure, but the memory saving will pay off for
everyone. Statically linking efl and initializing them once for the
entire system, means that a lot of the memory consumption issue we
see, like the eo one, would not be a problem anymore. This also would
include loading the system theme once and avoid duplicating that data
across the system. Overall everyone benefit, just the biggest visible
change are obviously on the lower end if you want to notice it (EFL
application on my laptop would never come close to the memory
consumption of Firefox !).

>>   For example, on
>> Raspberry Pi out of the 4s it takes to startup elemines, 1s can be
>> saved by quicklaunch. We are talking of a 25% improvement here. It
>> will also likely push us to improve further quicklaunch
>> infrastructure.
>
> 4 seconds?? Full 4 seconds to get started? I have a hard time to believe
> that. Maybe some of the system configuration is badly? Really slow SD
> card? And anyway, what does elimines do the other 3 seconds of startup?

Yes, Raspberry Pi is slow. SD is slow. The other hardware we have in
mind is max three times faster. And yes, we are mostly targeting the
lower end of the spectrum.

Also no elemines doesn't do the other 3 seconds. It spend part of its
time reading configuration, decompressing the theme, loading jpeg and
connecting to X. Basically 80 to 90% of the time is spend in efl
itself.

> Does other efl apps have the same problem? Terminology? Rage?
> You are asking for a drastic change with quite some downsides here so I
> hope it is not just based on data from one app in one hardware
> configuration. :)

Bigger application like Terminology take much longer to start. I have
no profiled them to tell exactly where the times goes, but for sure
the first 3s are identical. Terminology has a special mode to have one
process for all session and any further start is blazingly fast. Which
is the situation we want for all application without specific code
change per application.

>   We should be able to preconnect to X/Wayland and save
>> for the new process that is starting. We should also be able to pre
>> load the theme which would also save a lot of memory as those data
>> would be shared among all process. Elementary should be able to pre
>> load profile too and save that to every process. So first step is a
>> 25% win for sure. Further down the line, I bet we can save up to 60%
>> of our startup time. Basically some people are trying to push E/EFL
>> even further into smaller embeded device and they are requesting our
>> assistance.
>
> You are asking for it as default! Not as some way to make EFL better
> suited for specific scenarios. A big difference in my eyes. I understand
> the not being tested problem well enough (I deal with not normally
> tested code paths myself during all the QA work), but that should not
> mean we make something very specific our default. Default should be
> generic. e are not enabling architecture based compiler optimizations
> either but leave that to the user.

I am afraid it will bit rot to much. Also I think it would deliver
enough improvement for everyone to be worth it (Arguably, the more efl
application we have, the more this optimization pays off).

>>>> Now some of the drawback. First it is a trick, meaning most people
>>>> that will try to dig in will get confused at first on what is going on
>>>> and we need to document it. Building EFL is going to become even more
>>>> complex (I don't know yet how to generate the proper symlink), let's
>>>> enjoy more of the autofoo dark art.
>>>
>>> To be honest here, this one alone is a killer in my opinion. What we
>>> have is already complex enough. I often enough find myself in situation
>>> where I bang my head against the wall until I get the system to do what
>>> I want.
>>
>> Yes, that is not going to improve this way for sure.
>
> I your proposal we would just need to live with this?

I don't see how we can simplify our build process. We support a large
set of OS, have a tons of dependencies and a bunch of options which
are mostly necessary for supporting all the hardware/OS we target.
This change is going to add enforce new build mode that will be
different for other non classic Unix platform. I think that both Mac
OS X and Windows won't get that function. Max OS, because the
application and the library are usually packaged differently and may
cause trouble. Windows, because they don't have anything close to
fork.

>>>> Ok, does anyone see some serious blocker to this idea ? Impact on
>>>> packager to much ? Something I missed ? Something that need to be
>>>> clarified ?
>>>
>>> I have a hard time to see that this drastic change is really worth some
>>> improved launch time. Especially when you keep in mind that we are
>>> planning to merge some of our libs anyway thus reducing the amount of
>>> linking.
>>
>> I do think we will never save as much nor open ourself to all the
>> potential optimization that quicklaunch provide b merging a few
>> library. I don't have comparison there, but from above number you can
>> guess that it will be below 25%. Also this approach is I think nicer
>> as we do not need to forcefully merge library into weird name that
>> have less sense and endup with endless debate, like what does efl_core
>> integrate and so on. The only merge I would still go with is
>> efl/eo/ecore as they are clearly linked strongly by behavior now.
>
> We might end up with an endless debate here instead. :P

:-D

> More serious, the libs merge that is doing discussed and fought mostly
> about is the efl/eo/ecore one and exactly that one you want to do
> anyway. Are there really so many discussions about the other lib merges?
> In any case that  leads away from my main point.

Where to put ecore_con/eina/eet ? What does the efl_ui one include ?
and so on. Every bundle is to be argued forever as everyone want it to
match there own needs.

> What I don't like is that I hardly see any data that would support such
> a drastic change. Even more I oppose to having this the default and
> making everybodies life harder to even get the name of the binary. Now
> you need to remember commandline options for debug tools because you use
> E as window manager...

The binary name will be only affected for command line ps and top. The
rest won't change. Application still will provide a binary that will
be linked to all efl .so symlink. You will still be able to manually
launch each binary without depending on quicklaunch startup.
Development and debugging application for developers won't change.

I guess it will be good to write a system application that does
replace top in a nice way. Maybe a tytop ?

Arguably, as we start application from enlightenment, we can add an
option in it to enforce or not quicklaunch. So you would be able to
turn it off if you are experiencing some issue that you can't figure
out where it comes from.
-- 
Cedric BAIL