lcd4linux-devel

[Lcd4linux-devel] plugin_mysql, to do list.

[Javi <ja...@gs...>]

Hi.

Thanks all for your answers.

Michael, my real full name is Javier Garcia (Stolz is my nick).

Here is an "improvised" TO DO list :
  -add a function to fetch data (not only the number of rows) from the data=
base
  -use persistent connections (connect on init and disconnect on exit) to i=
mprove CPU usage
  -add some kind of connection trackin
  -set default server, user , pass and database in the Variables
   if mySQLquery() is called with:
    5 args, do it kike the old way
    2 args, use them like database and query, use default server, user and =
pass
    1 arg, use it like query, use default server, user , pass and database
  -add an argument to mySQLstatus funtion.
    The argument will indicate what to return:Uptime, Threads, Questions,Sl=
ow queries, Flush tables,...
    If uptime, return it Like: Days,hours,minutes

> For the variables I would choose a different approach: specify server,=20
> user and password directly in the config file, e.g.
>=20
> MySQL {
> =A0 =A0Server 'foo.bar'
> =A0 =A0User =A0 'michael'
> =A0 =A0Password 'swordfish'
> }

How can I get the variables from the config file, is there any function?

Thanks.

bye.

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Michael R. <re...@eu...>]

Hi Javier,

> Michael, my real full name is Javier Garcia (Stolz is my nick).
Thanks, fixed.
What does "Stolz" mean to you? I'm asking because "Stolz" is the german 
word for "pride" :-)

> Here is an "improvised" TO DO list :
>   -add a function to fetch data (not only the number of rows) from the database
>   -use persistent connections (connect on init and disconnect on exit) to improve CPU usage
>   -add some kind of connection trackin
>   -set default server, user , pass and database in the Variables
>    if mySQLquery() is called with:
>     5 args, do it kike the old way
>     2 args, use them like database and query, use default server, user and pass
>     1 arg, use it like query, use default server, user , pass and database
>   -add an argument to mySQLstatus funtion.
>     The argument will indicate what to return:Uptime, Threads, Questions,Slow queries, Flush tables,...
>     If uptime, return it Like: Days,hours,minutes

Fine! I'm expecting results until tomorrow evening :-)


> How can I get the variables from the config file, is there any function?

Of course there are. Take a look into plugin_imon.c

Basically, there are two function:

For Strings, use
char *cfg_get (section, key, default)
section would be "MySQL" (it's the "Block" from the config file
key is the requested parameter name
default is the default value which will be returned if the key does not 
exist in the config file

For Numbers, use
cfg_number(section, key, default,min,max,int *value)

acts like cfg_get(), but does some range checking with min and max


bye, Michael

-- 
Michael Reinelt                                   Tel: +43 676 3079941
Geisslergasse 4                                   Fax: +43 316  692343
A-8045 Graz, Austria                          e-mail: re...@eu...

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Xavier V. <xav...@fr...>]

Hello Javi, hi Michael !

> > Michael, my real full name is Javier Garcia (Stolz is my nick).
> Thanks, fixed.
> What does "Stolz" mean to you? I'm asking because "Stolz" is the german 
> word for "pride" :-)
lol

> > Here is an "improvised" TO DO list :
> > [...]
> Fine! I'm expecting results until tomorrow evening :-)
Michael, don't be too quick, you'll make Javi run away ;)
Javi, take all your time, lcd4linux prefers quality to speed, unlike
other projects I won't list ;)

> > How can I get the variables from the config file, is there any function?
> [...]
> For Numbers, use
> cfg_number(section, key, default,min,max,int *value)
?! I didn't know about this one !

BTW: Michael, I've just passed some exams, so I have some time for you
next week (before other exams).

Bye !
-- 
Xavier VELLO <xav...@fr...>

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Michael R. <re...@eu...>]

Hi Xavier,

>>Fine! I'm expecting results until tomorrow evening :-)
> 
> Michael, don't be too quick, you'll make Javi run away ;)
> Javi, take all your time, lcd4linux prefers quality to speed, unlike
> other projects I won't list ;)

Xavier, are you trying to undermine my authority? :-)

> BTW: Michael, I've just passed some exams, so I have some time for you
> next week (before other exams).

Great! What is your plan?

bye, Michael

-- 
Michael Reinelt                                   Tel: +43 676 3079941
Geisslergasse 4                                   Fax: +43 316  692343
A-8045 Graz, Austria                          e-mail: re...@eu...

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Xavier V. <xav...@fr...>]

Hello Michael, hi list.

> >>Fine! I'm expecting results until tomorrow evening :-) 
> > Michael, don't be too quick, you'll make Javi run away ;)
> > Javi, take all your time, lcd4linux prefers quality to speed, unlike
> > other projects I won't list ;)
> Xavier, are you trying to undermine my authority? :-)
Yes :) 

> > BTW: Michael, I've just passed some exams, so I have some time for you
> > next week (before other exams).
> Great! What is your plan?
First, I've 2 weeks of 'void' before the exams. I'm planning :
- searching people to help for dynamic so
- coding the test of the new website
- sleeping :)

Is there something more useful to code for lcd4linux ? I don't think I
can help for namespaces, maybe I could think about input ?

Bye !
-- 
Xavier VELLO <xav...@fr...>

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Michael R. <re...@eu...>]

Hi Xavier,

>>Xavier, are you trying to undermine my authority? :-)
> 
> Yes :) 

Ok. Just wait. I'll get you for this.

> First, I've 2 weeks of 'void' before the exams. I'm planning :
> - searching people to help for dynamic so
Fine, but this is 0.10.1 stuff. We should try to release 0.10.0 before

> - coding the test of the new website
Fine.

> - sleeping :)
Why? What for? You are young, I suppose. So sleeping isn't necessary.

> Is there something more useful to code for lcd4linux ? I don't think I
> can help for namespaces, maybe I could think about input ?

If you could make your Excel-Todo-Sheet up to date, this would be of 
help here.


bye, Michael

-- 
Michael Reinelt                                   Tel: +43 676 3079941
Geisslergasse 4                                   Fax: +43 316  692343
A-8045 Graz, Austria                          e-mail: re...@eu...

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Xavier V. <xav...@fr...>]

Hello Michael !

> Ok. Just wait. I'll get you for this.
?!

> > First, I've 2 weeks of 'void' before the exams. I'm planning :
> > - searching people to help for dynamic so
> Fine, but this is 0.10.1 stuff. We should try to release 0.10.0 before
I know, but this will be quite complicated, so I'm beginning the search
help for this.
Another issue is the slow plugins (mysql, web, mail, ...), which we'll
have to put in threads. A friend of mine recommended me to use C++ class
Thread insted of fork(), which will increase a lot memory footprint.
I fact, we won't have to port lcd4linux to C++, just  just introduce a
little 'glue' to use these class. Should I go deeper in this way or do
you _definitely_ want l4l to stay pure C ?

> > - coding the test of the new website
> Fine.
I'm first working on how to include docbook-make docs (with CSS) and
 thinking about the desing.

> > - sleeping :)
> Why? What for? You are young, I suppose. So sleeping isn't necessary.
Yes, I'm young, but a having a lie-in is really good :)

> > Is there something more useful to code for lcd4linux ? I don't think I
> > can help for namespaces, maybe I could think about input ?
> If you could make your Excel-Todo-Sheet up to date, this would be of 
> help here.
First, I don't use excel !!!!
The status file is in plain text, I attached a patch to put it up to
date (mysql and X11)

Bye !
-- 
Xavier VELLO <xav...@fr...>

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Martin H. <ma...@he...>]

Hi Xavier,

> I know, but this will be quite complicated, so I'm beginning the search
> help for this.
> Another issue is the slow plugins (mysql, web, mail, ...), which we'll
> have to put in threads. A friend of mine recommended me to use C++ class
> Thread insted of fork(), which will increase a lot memory footprint.
> I fact, we won't have to port lcd4linux to C++, just  just introduce a
> little 'glue' to use these class. Should I go deeper in this way or do
> you _definitely_ want l4l to stay pure C ?
I obviously can't speak for Michael, but I surely hope he's going to 
keep it pure C - since my target platform doesn't even have the c++ libs 
installed (so, it would _hugely_ increase the memory footprint). 
Besides, I fail to see why one would need c++, just to be able to use 
plain threads. While I've never coded threads in c under linux (I have 
done so on other operating systems), I don't really see the huge benefit 
of using c++ (apart from the fact that it's slightly easier to use, but 
that probably would be lost in the glue-code).

I guess if we actually start using threads, things could get ugly 
quickly (with all the changes happening at the moment between different 
kernel versions at the moment - mainly nptl-related), so I must say I 
like Michael's approach of "old fashioned" fork and IPC (via shared-mem, 
sockets, pipes, whatever). It's surely not the most efficient way, but 
nobody's going to check the inbox every couple of miliseconds anyway.

Martin

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Michael R. <re...@eu...>]

Hi Xavier,

> Another issue is the slow plugins (mysql, web, mail, ...), which we'll
> have to put in threads. A friend of mine recommended me to use C++ class
> Thread insted of fork(), which will increase a lot memory footprint.
> I fact, we won't have to port lcd4linux to C++, just  just introduce a
> little 'glue' to use these class. Should I go deeper in this way or do
> you _definitely_ want l4l to stay pure C ?

I fully agree with Martin here:

- lcd4linux will stay pure C (as the linux kernel does)
- lcd4linux will not use any threading library, the places where 
asynchronous execution is necessary are rare and will be solved using 
traditional fork() and shmem.

>>If you could make your Excel-Todo-Sheet up to date, this would be of 
>>help here.
> 
> First, I don't use excel !!!!
Sorry, I forgot that you already "ported" the Status list to ASCII :-)

> The status file is in plain text, I attached a patch to put it up to
> date (mysql and X11)

Fine, thanks!

What about plugin_wifi? Martin will neeed it, IIRC.


bye, Michael

-- 
Michael Reinelt                                   Tel: +43 676 3079941
Geisslergasse 4                                   Fax: +43 316  692343
A-8045 Graz, Austria                          e-mail: re...@eu...

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Martin H. <ma...@he...>]

Hi Michael,

> - lcd4linux will stay pure C (as the linux kernel does)
> - lcd4linux will not use any threading library, the places where 
> asynchronous execution is necessary are rare and will be solved using 
> traditional fork() and shmem.
Glad to hear that.

> What about plugin_wifi? Martin will neeed it, IIRC.
Well, I had plans of writing it up myself today (but as always, 
something else got in the way). I'll give it a shot tomorrow, unless 
Xavier insists on doing it himself.

Martin

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Xavier V. <xav...@fr...>]

Hello Michael, Martin and others

> - lcd4linux will stay pure C (as the linux kernel does)
> - lcd4linux will not use any threading library, the places where 
> asynchronous execution is necessary are rare and will be solved using 
> traditional fork() and shmem.
In fact to aswer to Martin, C++ threads are less memory-wasting than
  the C fork(), as fork copy the whole process, with its variable and
 others.

I don't know what you think about it, but maybe we can fork very early
in 3 threads: main, plugins, and soon input to make all these work
asyncronously and solve many problems we'll encounter with input. But
this will surely lead to problems with shared variables and other.

Do as you want Michael, but IMHO slitting into 3 (by now 2) threads
would be "proper". But we'll be able to troll about this after 0.10.0 is
released. ;)

> >>If you could make your Excel-Todo-Sheet up to date, this would be of 
> >>help here.
> > First, I don't use excel !!!!
> Sorry, I forgot that you already "ported" the Status list to ASCII :-)
Errr ... some people use free software like OOo instead of "crossover  
 office" ! See photo06.jpg of LinuxDay on the site, the last three  
 icons in the KDE panel : are these word, excel and powerpoint (at
 least I think) icons shortcuts to OOo components ? What can you argue
 for your defense ?

> What about plugin_wifi? Martin will neeed it, IIRC.
I began to write it last week while recompiling a 2.6.3 kernel on a
133MHz notebook (so I had time to code :D ), it was a mere copy of
plugin_netdev.c as the two files /proc/net/dev and /proc/net/wireless
are similar, but I found a syscall to retreive the informations, so I
dropped the source. I have to dig more about this function and write the
plugin --> added to my ToDo :)

Bye !
-- 
Xavier VELLO <xav...@fr...>

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Martin H. <ma...@he...>]

Hi Xavier,

>>- lcd4linux will stay pure C (as the linux kernel does)
>>- lcd4linux will not use any threading library, the places where 
>>asynchronous execution is necessary are rare and will be solved using 
>>traditional fork() and shmem.
> 
> In fact to aswer to Martin, C++ threads are less memory-wasting than
>   the C fork(), as fork copy the whole process, with its variable and
>  others.
I'm very well aware of the difference between multi-threading and 
multi-processing. But IMHO your assumption is wrong, unless Linux 
nowadays works completely different from what I've learned in the past. 
Unless you change a variable, fork will reserve, but not actually use 
any of that address space. Besides, by "memory footprint" I was actually 
more referring to the almost 3MB large libstdc++.so that comes with 
using c++. I couldn't really care less if there's another 500k process 
running (at runtime) - but adding 3 MB in libs (which need to be saved 
on the install medium - which _may_ be a floppy, for my target 
plattform) surely is an issue for me. Especially so, if it's done so 
without a huge benefit.

> I don't know what you think about it, but maybe we can fork very early
> in 3 threads: main, plugins, and soon input to make all these work
> asyncronously and solve many problems we'll encounter with input. But
> this will surely lead to problems with shared variables and other.
> 
> Do as you want Michael, but IMHO slitting into 3 (by now 2) threads
> would be "proper". But we'll be able to troll about this after 0.10.0 is
> released. ;)
I don't think this is so much about "style", but rather about keeping 
the code maintainable. Forking and using IPC is something that works, 
and has been working for ages (even if it's not the most efficient way, 
and even if threads could do it better - lets be honest, we're not 
talking about megabytes of data per second being transferred between 
those processes). So, it's something that people are comfortable with, 
something that doesn't change anymore (unlike the threads API - or so I 
gather from the problems that came with the introduction of nptl), so it 
sounds like the way to go, as long as we don't have something that needs 
to move tons of data (IPC is relatively slow, afterall) and that isn't 
too time-critical.

I don't really see what we'll gain by limiting ourselves to 3 processes 
(or "threads" if you prefer those) - to me, every plugin should be able 
to create its own data-gatherer thread if it needs one.
If we go your route, it will only make the "plugins" process very 
complicated (since it needs to play scheduler), none of the functions 
may block, or other plugins will be blocked too, that kind of thing.

My suggestion (and that's the way I understood Michael as well) is to 
use a "worker process/thread" for each plugin that will be slow/may 
block. Memory consumption is not so much an issue, since most of it will 
be marked as "shared" anyway, so memory usage on the box will only 
increase marginally.

>>What about plugin_wifi? Martin will neeed it, IIRC.
> 
> I began to write it last week while recompiling a 2.6.3 kernel on a
> 133MHz notebook (so I had time to code :D ), it was a mere copy of
> plugin_netdev.c as the two files /proc/net/dev and /proc/net/wireless
> are similar, but I found a syscall to retreive the informations, 
A syscall? Care to explain? All I know so far is that the wireless 
extensions API works via IOCTL calls - I never heard/read of syscalls 
for that in the past (maybe this is simply about a different definition 
of "syscall" - to me, a "syscall" is a function exported by the kernel - 
whereas an IOCTL call is a specific syscall, that takes specific 
parameters (and those parameters tell it what to do)).

> I have to dig more about this function and write the
> plugin --> added to my ToDo :)
So, you _are_ working on that (or did you just add it to your ToDo 
list)? Don't get me wrong - I just want to make sure we don't duplicate 
work (and still get a working wifi plugin in a timely manner). I've 
added plenty of things onto my ToDo list in the past (which I never 
completed), so I want to make sure :-)

Martin

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Xavier V. <xav...@fr...>]

Hello Mirtin, hi list

> [...]
> My suggestion (and that's the way I understood Michael as well) is to 
> use a "worker process/thread" for each plugin that will be slow/may 
> block. Memory consumption is not so much an issue, since most of it will 
> be marked as "shared" anyway, so memory usage on the box will only 
> increase marginally.
Okay, I was misinformed about fork. Let's continue this way.
But another question : for example eval needs the mail('unread') value,
so calls a (hypothetical) mail() function in plugin_mail, which forks to
retreive POP summary, and it takes, for example, 5 seconds, what does
the eval do within this time ? I don't know what Michael wants on this
issue.

> A syscall? Care to explain? All I know so far is that the wireless 
> extensions API works via IOCTL calls - I never heard/read of syscalls 
> for that in the past (maybe this is simply about a different definition 
> of "syscall" - to me, a "syscall" is a function exported by the kernel - 
> whereas an IOCTL call is a specific syscall, that takes specific 
> parameters (and those parameters tell it what to do)).
I don't really know what is the difference, I've not been hacking IOCTL
 since this week :), but read linux-2.6.4/include/net/iw_handler.h,
 there are a bunch of explanations there for the new API.
If it's possible, I would prefer to call this [IOCTL|syscall] instead of
parsing /proc/net/wireless, as it's funnier and more educative for me.

> So, you _are_ working on that (or did you just add it to your ToDo 
> list)? Don't get me wrong - I just want to make sure we don't duplicate 
> work (and still get a working wifi plugin in a timely manner). I've 
> added plenty of things onto my ToDo list in the past (which I never 
> completed), so I want to make sure :-)
I'll write this, but you may help me if you know [IOCTL|syscall] well,
as I don't know either how to do :)
Reading plugin_dvb.c, IOCTL are special adresses in a /dev node, while,
as far as I understand wireless extensions have merged in
linux/include/linux/netdevice.h, and "INET is implemented using the BSD
Socket interface as the means of communication with the user level"
Help ?!?!

Bye !
-- 
Xavier VELLO <xav...@fr...>

[Lcd4linux-devel] Threads

[Michael R. <re...@eu...>]

Hi Xavier, Hi List,

> Okay, I was misinformed about fork. Let's continue this way.
I already started with a "simple threading" framework (nothing in CVS by 
now)

> But another question : for example eval needs the mail('unread') value,
> so calls a (hypothetical) mail() function in plugin_mail, which forks to
> retreive POP summary, and it takes, for example, 5 seconds, what does
> the eval do within this time ? I don't know what Michael wants on this
> issue.

It's simple: The first call to mail::unread() triggers the fork, and 
returns nothing (this is a drawback: oin the first call you'll never get 
a senseful result). From now on, the 2nd thread regularly polls the 
mailbox (at a configurable interval from the config file), ans stores 
the result in a special shared memory area. The main thread from 
mail::unread() does nothing but reading this shared memory area and 
returns the corresponding results. So the main thread is very fast and 
does never block.

The shared memory area itself will be protected by a mutex, to ensure 
that it's not written to by the poll thread while being read from the 
mail thread.

The best place for such a "shared memory parameter passing structure" 
would be our well-loved hash. But I'm not shure if this is possible...

I am starting this async stuff with the exec() plugin, because its 
simpler than POP polling, so I can focus on the threading framework. 
Exec's really need an async model, too, because you never can tell how 
long the exec'ed prcess will take today.


bye, Michael

-- 
Michael Reinelt                                   Tel: +43 676 3079941
Geisslergasse 4                                   Fax: +43 316  692343
A-8045 Graz, Austria                          e-mail: re...@eu...

Re: [Lcd4linux-devel] plugin_mysql, to do list.

[Martin H. <ma...@he...>]

Hi Xavier,

>>My suggestion (and that's the way I understood Michael as well) is to 
>>use a "worker process/thread" for each plugin that will be slow/may 
>>block. Memory consumption is not so much an issue, since most of it will 
>>be marked as "shared" anyway, so memory usage on the box will only 
>>increase marginally.
> 
> Okay, I was misinformed about fork. Let's continue this way.
> But another question : for example eval needs the mail('unread') value,
> so calls a (hypothetical) mail() function in plugin_mail, which forks to
> retreive POP summary, and it takes, for example, 5 seconds, what does
> the eval do within this time ? I don't know what Michael wants on this
> issue.
I guess Michael has already answered that one. If eval waited for the 
fork to complete (and deliver the data), we'd still be doing things 
synchronously, and thus would have gained nothing.

To get around that problem with the first call failing - I guess we 
could get fancy and supply some kind of "callback" (which would get 
called when the data is available) - but to me it sounds like that would 
be overkill, considering we're only talking about the first call to that 
function that causes problems.

>>A syscall? Care to explain? All I know so far is that the wireless 
>>extensions API works via IOCTL calls - I never heard/read of syscalls 
>>for that in the past (maybe this is simply about a different definition 
>>of "syscall" - to me, a "syscall" is a function exported by the kernel - 
>>whereas an IOCTL call is a specific syscall, that takes specific 
>>parameters (and those parameters tell it what to do)).
> 
> I don't really know what is the difference, I've not been hacking IOCTL
>  since this week :), but read linux-2.6.4/include/net/iw_handler.h,
>  there are a bunch of explanations there for the new API.
Well, please don't use _too_ new features, since that would mean people 
still running the 2.4 kernel will be left out. I would prefer to stick 
to the API defined here
http://www.hpl.hp.com/personal/Jean_Tourrilhes/Linux/Tools.html
to avoid compatibility problems (I don't know - maybe that's exactly 
what's made its way into 2.6 - I haven't run 2.6 so far, and from the 
look of things, it may still be a while until I will - not because I 
think 2.6 is bad, just because I don't have the time to mess with my 
system just for the heck of it).

> If it's possible, I would prefer to call this [IOCTL|syscall] instead of
> parsing /proc/net/wireless, as it's funnier and more educative for me.
Absolutely - besides, it should be much more efficient and less error 
prone. The only drawback is that it will hit us if the API changes 
(which is unlikely though, since the API seems to be pretty stable).

>>So, you _are_ working on that (or did you just add it to your ToDo 
>>list)? Don't get me wrong - I just want to make sure we don't duplicate 
>>work (and still get a working wifi plugin in a timely manner). I've 
>>added plenty of things onto my ToDo list in the past (which I never 
>>completed), so I want to make sure :-)
> 
> I'll write this, but you may help me if you know [IOCTL|syscall] well,
> as I don't know either how to do :)
Sure, just let me know if you need help. Using IOCTL is pretty simple - 
it's all about finding which parameters to use (which one usually needs 
to pick from the source of the driver one is calling into) - just have a 
look at drv_generic_parport.c in lcd4linux to get an idea how it's done 
(really, there's not much to it). The only thing one needs to be careful 
about is to provide the right kind of pointer for the 3rd parameter - if 
the driver expects a pointer to an int, and you supply a pointer to a 
char, strange things will happen.

> Reading plugin_dvb.c, IOCTL are special adresses in a /dev node, while,
> as far as I understand wireless extensions have merged in
> linux/include/linux/netdevice.h, and "INET is implemented using the BSD
> Socket interface as the means of communication with the user level"
> Help ?!?!
Well, IOCTLs are done on a filehandle (and usually on the file-handle of 
a file in the /dev/ directory). That's the most common. The wireless 
extensions apparently require IOCTL calls to a socket - which apparently 
works just as well.

http://www.hpl.hp.com/personal/Jean_Tourrilhes/Linux/Linux.Wireless.Extensions.html

(especially the section 5.2 IOCTL) has some info on how that works.

If I were writing the plugin, I'd probably start out with going through 
the source-code of the wireless tools (which obviously use the wireless 
extensions API themselves), to get a feel for how things work.

I hope that helps.

Martin

-- 
You think that's tough?  Try herding cats!