[Pcbsd-pbi-dev] A new library handling idea

[Federico L. <flo...@gm...>]

Ok, heres a brand new idea on handaling libraries:

Directory structure:
PBI Root
             -> ProgLib
                      -> libfoo.so.1
             -> DepLib
                      -> libc.so.6
                      -> libqt.so.2
                      -> libkde.so.8
             -> lib
                      <empty>

We have a PBI called FooApp, which needs libc.so.6 libqt.so.2 libkde.so.8an=
d
libfoo.so.1
Now, libfoo.so.1 is a program specific library, NOT a normal dependancy, (i=
t
comes with FooApp),
so it is stored in ProgLib, while the other libs are stored in DepLib....
Now heres the killer bit...

The lib path of the program will be set to the lib dir, but since its empty
a few things will happen first...
A script (which i'm working on) will:
* Symlink all the libraries in the ProgLib folder to the lib folder (as the=
y
wont beable to be found anywhere else)
^ Generate an md5 sum of all the libraries in the DepLib folder and save it
as DepLib/md5 (once off)
* Compare those md5's to that of the systems libraries, if theres a match,
the system ones get symlinked into the
   lib folder, but if theres no match, the DepLib ones get symlinked in

Please tell me what you think of this idea, as it should elivate the proble=
m
of wasted memory, while still allowing
things that dont have all the required libs in the system / the ones in the
system are too out of date / too new to
still work!

PS. The DepLib and ProgLib folders tie in with my ports-pbi compatability
plan ;)
Cheers


On 2/10/06, Federico Lorenzi <flo...@gm...> wrote:
>
> I'm not a programmer or anything, but i always thought that if lets say
> FooApp loaded libfoo.so.1 into memory, as it needed it, and BarAPP
> needed the same lib, it would be smart enough too see that its already
> loaded, and simply use it.
> Then again i'm probably wrong :) Although that would be the idea
> situation.
>
> Cheers
>
> On 2/10/06, Richy T < bio...@gm...> wrote:
> >
> > Hello pbi developers!
> > I have been doing some reading on our pcbsd forums.
> > There is some really good information there about our format.
> > I really think all PBI developers should read this.
> > The information posted on the forum might really help make
> > .PBI truly a great binary format for pcbsd and maybe freebsd!
> > Here is the post. -->
> >
> > From: Almindor
> > > ---------------
> > > I'm a programmer so I'll post my view on PBIs and other packaging
> > > here.
> > >
> > > PBI is a .app style packaging system with "all in one" even basic
> > > dependencies. The good of this is that it's simple, easy and has
> > > (theoreticly, in practice this isn't true) no conflicts.
> > >
> > > Packaging ala Ports/Debs/RPMs is a complicated (altho not always for
> > > the end user especialy if they have good net connection, debs are rea=
l good
> > > here) process which required dependency checks etc.
> > >
> > > Now for the pros and cons from my perspective:
> > >
> > > PBIs are great for programmers/developers since they make it easy for
> > > them to distribute their apps without much fuzz.
> > > PBIs however have 2 major problems as-is, one is directly tied to the
> > > FreeBSD system underneath:
> > >
> > > 1. It conflicts with ports/packages
> > > 2. It copies all libs for all apps and makes ALOT of duality with .so
> > > files.
> > >
> > > Now #2 seems like no problem to most of you guys but let me elaborate=
.
> > >
> > > Dynamic libraries, or shared objects as Unix world calls them are
> > > pieces of code (and sometimes data too) which are saved in a special =
way so
> > > that programs(apps) can "link" themselves to the code and use it.
> > >
> > > The whole concept of shared objects is, sharing memory Smile (how
> > > unexpected)
> > >
> > > What's wrong with per-program copy of .so files then?
> > >
> > > Memory usage. Start up KDE and a few apps, then look at your memory
> > > usage via some monitor program. Notice that most apps (visual kde etc=
) take
> > > 10+megs of RAM. This ISNT REAL MEMORY.
> > >
> > > Because all of these programs use more or less the same .so files, al=
l
> > > this memory is shared.
> > >
> > > Now imagine a system where virtualy everything comes from PBI. Every
> > > program would really hog ram with it's own copies of .so files (and h=
ence
> > > their existance is a hindrance, since smart-static linking would prod=
uce
> > > much smaller footprint, less conflicts etc.)
> > >
> > > Conclusion? It's a good idea on a bad system. I've nothing agains Uni=
x
> > > in general but Unix world (and windows too to an extent) decided to g=
o the
> > > "shared object" path. Every unix binary today links dynamicly etc.
> > >
> > > Is there a different approach?
> > >
> > > Well there is, but if it's better is a nice question. Now before I
> > > start this please note that I AM biased Smile
> > >
> > > Free Pascal compiler can produce "smart linked" binaries. These
> > > binaries are static but contrary to popular belief take very little s=
pace on
> > > disk and in memory. (Hello world app in C static linked is ~800kb!! i=
n FPC
> > > smartlinked it's ~20kb on disk, I'll explain why)
> > >
> > > Smart linking is a technique which "links in" only used code from a
> > > library.
> > >
> > > So let's say libC is smartlinkable (it's not btw).
> > >
> > > If you only used printf() your binary would only grow "so" much.
> > >
> > > If you use dynamic .so, your binary will remain small on disk, but
> > > will load the whole libC into memory because of printf(). Now imagine=
 all
> > > programs loading whole libC into memory per-copy. This is what is don=
e with
> > > PBIs and that's the bad idea. PBIs would work wonders on a "smartlink=
" based
> > > system where only real basic system libs would be dynamic, and all ot=
her dev
> > > libs would be smartlinkable. This way programs would have minimal foo=
tprints
> > > and no conflicts would happen (they would also have a tendency to wor=
k
> > > longer since ABI is of no concern).
> > >
> > > Hope this all makes a bit of sense Smile
> > >
> > > Ales
> >
> >
> >
> >    POST REPLY:
> >
> > From: pcbsdusr
> > > ----------------
> > > I have been thinking of this as well and i am in the process of
> > > writing a hibrid concept which uses programs in their own folders but=
 shares
> > > dependencies while maintaining the ability to use multiple versions o=
f any
> > > given library side by side.
> > >
> > > no multiplication of libraries =3D No wasted disk space and no memory
> > > waste while multitasking with apps using common dependencies.
> > >
> > > of course, i am writing this for apps only... There will be always
> > > duplication until we maintain freebsd intact under pcbsd...
> >
> >
> > I really hope you guys can update the format to work this way.
> > It would make the .pbi format truly one of, if not the best format for
> > pcbsd/freebsd/UNIX systems.
> > Hope this gives you guys some insight and inspiration on how to update
> > the .pbi format.
> >
> > - Richy
> >
> >
> >
> >
> >
> >
>