pilot-db-list

[pilot-db-list] few things

[Yedidyah Bar-D. <di...@ta...>]

Hello all,

First I'd like to say thanks to everyone working on pilot-db. I think
it's a great piece of software.

I installed it on my brother-in-law's m103, as well as on pose on linux
with his rom. It mostly works well. I do have a few issues/questions.
I did not search the archives before sending this, only skimmed through
the first 20 messages (two months) to get the feeling of the list, so
feel free to ignore already-discussed issues.

1. I translated the UI to Hebrew. You can get the file at
<http://www.cs.tau.ac.il/~didi/iw.rcp> (I did not know if it's
considered polite sending here attachments, so I put it there).
I did not translate everything, mostly the commonly-used things as
well as the ones easy to translate :-). If there is interest I do not
mind spending a few more hours on polishing this, but I think it's
already good enough for inclusion in a next version.

2. The "large DB support" didn't work very well for him, so for now
I disabled it (compiled with '--with-vldb=0'). IIRC, it also wasn't
that much faster - and it _is_ an issue for him, having a 1000+
records DB with around 20 fields (still to change, but num of records
will not get smaller - it's his todo list). How experimental is this?
Is it going to become stable soon? Should I help debug this? Can I?

3. Searching for Hebrew phrases is a bit weird. When using one kit of
Hebrew support, the one he got with the palm (HarelHeb 3.5, IIRC), it
only finds things if they are at the end of a field. When using another
support kit (a 2-week temporary license from www.kalanit.co.il for
what seems like a newer version of the same HarelHeb, but not sure),
it works just fine. Is this reasonable? Is there anything pilot-db can
do about it, or is this only the foreign-language support's job? I
did grep the source and found it's MatchString, which calls
*Glue* functions. Are these part of the foreign-lang support?

4. Since it's too slow for daily work, he uses the memoexport plugin
for the daily tasks, and uses that, using pilot-db only for longer-term
planning. Is there an easy way to export at once all the records displayed
by a specific view/filter? I guess no. It sounds to me as something that's
generally-useful - to make all the one-record plugins also available from
the listview, making them operate on all the records. Does this make sense?
Or is the right thing writing another plugin that will be specific to this
(which he will probably need anyway, since he wants it to set also the
category etc.)?

5. I spent some time on editing the DB on a PC. I mostly want to do this
on linux, so dbeditor isn't my first choice. I tried to hack palm-db-tools
into exporting and importing filters (and eventually hoped for all other
data), but after spending on this around 10 hours, not getting it working
and deciding it's too hard to debug, I decided to quit. Instead, I searched
and found a small tool called par (7th google hit searching for 'par'),
which lets you extract the appinfo to a file, each record to a file, and
create a pdb from such files. So I intend to write a script that will
do this:
export and keep the appinfo
pdb2csv to file.csv
edit the file.csv (with openoffice, or maybe mysql+some web UI?)
csv2pdb
export all the records from the new pdb
create a pdb from the records and the saved appinfo

I verified that this theoretically works, but did not make thorough tests.
Should it? Is this a good way, considering no tool will follow closely
enough the development of pilot-db? And, BTW, isn't something written in
some higher-level language easier to write? E.g. things like
datascript or DataWorkshop? I know this isn't pilot-db-related, but
palm-db-tools seems dead to me.

6. Not terribly important, but transferring it to/from the palm takes
quite a lot of time - around 45 seconds, for a 170KB file. For comparison,
db-iw.prc itself, which is larger, takes much less time (not sure, I think
around 13 seconds). Is this reasonable? Is it because of the many records?

7. There is a bug, that sometimes, when pressing the down-arrow or up-arrow
icons on the buttom-right, in the "record view" (is this the name?), it
crashes, requiring a soft reset. Pressing PgDn does work, so that's what
he does for now. I tried for some time to debug this with pose and gdb,
but did not really get anywhere. It says:
"DB (1.1.0) called SysFatalAlert with the message:
"leakdetect.c, Line:188, addr"."
and gives control to gdb, but I can't see too much - 'where' says
(gdb) where
#0  0x1001d8b2 in ErrDisplayFileLineMsg ()
#1  0x00091b88 in ?? ()
#2  0x0008955c in ?? ()
#3  0x0007b770 in ?? ()
#4  0x10066b08 in PrvSendEventToForm ()
#5  0x10069f2c in FrmDispatchEvent ()
#6  0x0006f670 in EventLoop () at main.c:201
#7  0x0006f8a6 in EditViewGetFieldHeight () at main.c:715
#8  0x0006f3e4 in CheckMinVersion () at crt0.c:69
which does not help much. I tried to also put a breakpoint in
leakdetect.c:188, but also didn't get much further. How do I debug
pilot-db? Is there some tutorial somewhere?

I hope this isn't too long! If you want further discussion to be in
single-issue-threads, fine with me. If I am supposed to subscribe before
sending, fine - but I can't promise I will spend much time on "real"
work.

Thanks a lot anyway,
-- 
Didi

Re: [pilot-db-list] few things

[Nathan K. <na...@ve...>]

Hello --

Thanks for your interest and time spent!

> 1. I translated the UI to Hebrew. You can get the file at
> <http://www.cs.tau.ac.il/~didi/iw.rcp> (I did not know if it's
> considered polite sending here attachments, so I put it there).
> I did not translate everything, mostly the commonly-used things as
> well as the ones easy to translate :-). If there is interest I do not
> mind spending a few more hours on polishing this, but I think it's
> already good enough for inclusion in a next version.

I'm probably the person who would include this, but I'm a new
maintainer for this and I haven't yet dealt with adding languages.
But if you can leave it up for a while (currently unspecified) I'll
add it as a language once we get a release candidate mostly working.

> 2. The "large DB support" didn't work very well for him, so for now
> I disabled it (compiled with '--with-vldb=0'). IIRC, it also wasn't
> that much faster - and it _is_ an issue for him, having a 1000+
> records DB with around 20 fields (still to change, but num of records
> will not get smaller - it's his todo list). How experimental is this?
> Is it going to become stable soon? Should I help debug this? Can I?

You compiled it!  Congratulations!  As best as I can tell, there are
very few people on this list who have done so.  Be warned that many of
the code paths seem to be unused in recent years, and that by
including (or omitting) configuration flags you might soon find
yourself in uncharted waters.

Yes, it is probably going to become stable in the next release by
simple method of removing all reference to the word "experimental".
As far as I know (based on my recollection of submitted bug reports)
it does not seem to add additional bugs.  I don't know about it's
performance, but I think it purpose was simply to allow very large
databases, not to necessarily make them work faster. (?)

Yes, you should help debug this! :) Submitting the slowness as a bug
report to the SourceForge website, and then giving a sample database
with some sample times would be good too.  I haven't yet tried much
profiling under POSE, but it does appear to be possible.  I'd guess
(without having looked closely at the code) that there is lots of room
for simple optimization.  If you can do it, wonderful, otherwise (like
everything else) I'll try to get to it once we have a stable release.

> 3. Searching for Hebrew phrases is a bit weird. When using one kit of
> Hebrew support, the one he got with the palm (HarelHeb 3.5, IIRC), it
> only finds things if they are at the end of a field. When using another
> support kit (a 2-week temporary license from www.kalanit.co.il for
> what seems like a newer version of the same HarelHeb, but not sure),
> it works just fine. Is this reasonable? Is there anything pilot-db can
> do about it, or is this only the foreign-language support's job? I
> did grep the source and found it's MatchString, which calls
> *Glue* functions. Are these part of the foreign-lang support?

I don't know---I haven't dealt much with the foreign language support,
and not at all with the non-latin-alphabet languages.  Submitting it
as a bug report would be useful to create a permanent record, but my
offhand guess is that if we are calling the *Glue* functions
correctly, there is not much else to be done on the pilot-db end.
But it is certainly worth checking if we are calling them correctly...

> 4. Since it's too slow for daily work, he uses the memoexport plugin
> for the daily tasks, and uses that, using pilot-db only for longer-term
> planning. Is there an easy way to export at once all the records displayed
> by a specific view/filter? I guess no. It sounds to me as something that's
> generally-useful - to make all the one-record plugins also available from
> the listview, making them operate on all the records. Does this make sense?
> Or is the right thing writing another plugin that will be specific to this
> (which he will probably need anyway, since he wants it to set also the
> category etc.)?

I'd guess it is not possible either, but yes, it does sound useful.

> 5. I spent some time on editing the DB on a PC. I mostly want to do this
> on linux, so dbeditor isn't my first choice. I tried to hack palm-db-tools
> into exporting and importing filters (and eventually hoped for all other
> data), but after spending on this around 10 hours, not getting it working
> and deciding it's too hard to debug, I decided to quit. Instead, I searched
> and found a small tool called par (7th google hit searching for 'par'),
> which lets you extract the appinfo to a file, each record to a file, and
> create a pdb from such files. So I intend to write a script that will
> do this:
> export and keep the appinfo
> pdb2csv to file.csv
> edit the file.csv (with openoffice, or maybe mysql+some web UI?)
> csv2pdb
> export all the records from the new pdb
> create a pdb from the records and the saved appinfo
> 
> I verified that this theoretically works, but did not make thorough tests.
> Should it? Is this a good way, considering no tool will follow closely
> enough the development of pilot-db? And, BTW, isn't something written in
> some higher-level language easier to write? E.g. things like
> datascript or DataWorkshop? I know this isn't pilot-db-related, but
> palm-db-tools seems dead to me.

We've had a bit of discussion this list about what needs to be done
about this, but no solution yet.  I wouldn't give up on the hope of
having a tool that would follow the development of pilot-db.  In fact,
I think that is just about the only hope.  

> 6. Not terribly important, but transferring it to/from the palm takes
> quite a lot of time - around 45 seconds, for a 170KB file. For comparison,
> db-iw.prc itself, which is larger, takes much less time (not sure, I think
> around 13 seconds). Is this reasonable? Is it because of the many records?

I don't know, but that does sound odd.  I've done almost all my
testing on Pose and I haven't run into this problem. 

> 7. There is a bug, that sometimes, when pressing the down-arrow or up-arrow
> icons on the buttom-right, in the "record view" (is this the name?), it
> crashes, requiring a soft reset. Pressing PgDn does work, so that's what
> he does for now. I tried for some time to debug this with pose and gdb,
> but did not really get anywhere. It says:
> "DB (1.1.0) called SysFatalAlert with the message:
> "leakdetect.c, Line:188, addr"."
> and gives control to gdb, but I can't see too much - 'where' says
> (gdb) where
> #0  0x1001d8b2 in ErrDisplayFileLineMsg ()
> #1  0x00091b88 in ?? ()
> #2  0x0008955c in ?? ()
> #3  0x0007b770 in ?? ()
> #4  0x10066b08 in PrvSendEventToForm ()
> #5  0x10069f2c in FrmDispatchEvent ()
> #6  0x0006f670 in EventLoop () at main.c:201
> #7  0x0006f8a6 in EditViewGetFieldHeight () at main.c:715
> #8  0x0006f3e4 in CheckMinVersion () at crt0.c:69
> which does not help much. I tried to also put a breakpoint in
> leakdetect.c:188, but also didn't get much further. 

I fixed something similar in the CVS version (Danger! Don't go
there!).  If possible, it would be great to see if you can reproduce
this bug with a stock binary release.  There are a number of
configuration oddities that might cause this (some codepaths seem not
to have been used in years) and it that's causing this.

Submitting this as a bug would be good for record keeping:
  http://sourceforge.net/tracker/?group_id=621&atid=100621

> How do I debug pilot-db? Is there some tutorial somewhere?

No, there is not much published information, but I'm eager and willing
to help you figure it out, and if we do it on-list it will probably
help people in the future.   

It looks like the problem you are hitting now is that pilot-db is a
multi-segment application, and that the standard prc-tools kit
(including m68k-palmos-gdb) does not really support multiple code
segments.  But there are patches available to do so:
  http://www.v-overbeek.nl/msectgdb/
While it's possible to do the compile from scratch, it's a lengthy
multi-step procedure.  I submitted Linux binaries to Ton, which should
be now available on his site.  

> I hope this isn't too long! 

On-topic length is no problem.

> If you want further discussion to be in single-issue-threads, fine
> with me.

Probably better.  Let's put the followup posts into separate threads.

> If I am supposed to subscribe before sending, fine - but I can't
> promise I will spend much time on "real" work.

Subscribing would be useful, as otherwise I have to manually approve
each post you make.  

> Thanks a lot anyway,

And thank you!


Nathan Kurz
na...@ve...

Re: [pilot-db-list] few things

[Yedidyah Bar-D. <di...@ta...>]

OK, I will make it a separate message for each issue.
I also subscribed to the list. Still, there are a few things I won't
repeat in each message: I use Debian sid, with the following versions:
prc-tools 2.2.90.cvs20030306-4 (has gcc 2.95.3, gdb 5.3)
pose 3.5-6 (the first version working for me - for a few months I
was pretty sure I'll have to use the simulator on Windows, which isn't
as simple for me) with the ROM of a Palm m130 (PalmOS 4.1)
palmos sdk 5r3

I tried and worked with pilot-db 1.1.0, 1.2.0 alpha 5 (the compiled
prc from sf.net), and the CVS version of today. Earlier versions needed
minor tweaks to compile against sdk-5r3, the CVS compiled cleanly.

I used the following to compile:
./configure --enable-standard --enable-debug --enable-sony=no --enable-handera=no --enable-fiveways=no --enable-make_plugins
without '--enable-sony=no --enable-handera=no --enable-fiveways=no', I
had some compilation problems, I guess because you need a different sdk
for this, didn't check.
As I said, in some cases I added '--with-vldb=0', more about that in
a different message.

On Mon, Apr 26, 2004 at 09:26:47AM -0600, Nathan Kurz wrote:
> Hello --
> 
> Thanks for your interest and time spent!
> 
> > 1. I translated the UI to Hebrew. You can get the file at
> > <http://www.cs.tau.ac.il/~didi/iw.rcp> (I did not know if it's
> > considered polite sending here attachments, so I put it there).
> > I did not translate everything, mostly the commonly-used things as
> > well as the ones easy to translate :-). If there is interest I do not
> > mind spending a few more hours on polishing this, but I think it's
> > already good enough for inclusion in a next version.
> 
> I'm probably the person who would include this, but I'm a new
> maintainer for this and I haven't yet dealt with adding languages.
> But if you can leave it up for a while (currently unspecified) I'll
> add it as a language once we get a release candidate mostly working.

Well, it's pretty simple, and documented in docs/translating.txt. I had
to add the file to rcp/langs/, and add the line 'include(iw.rcp)' to
rcp/db.rcp (or rather, to db.rcp.in, before running configure). Then,
'make db-iw.rcp' worked. No need to change the Makefile, as written there.
It's probably possible to even make the change to db.rcp automatic too,
but I didn't look at this.

But I do not mind that you'll wait with this - as I said, the translation
does need some more work, and I did not even start with the help file.
-- 
Didi

[pilot-db-list] Hebrew support

[Yedidyah Bar-D. <di...@ta...>]

Sorry for the last message - I accidentally hit 'send' before changing
the subject ...

OK, I will make it a separate message for each issue.
I also subscribed to the list. Still, there are a few things I won't
repeat in each message: I use Debian sid, with the following versions:
prc-tools 2.2.90.cvs20030306-4 (has gcc 2.95.3, gdb 5.3)
pose 3.5-6 (the first version working for me - for a few months I
was pretty sure I'll have to use the simulator on Windows, which isn't
as simple for me) with the ROM of a Palm m130 (PalmOS 4.1)
palmos sdk 5r3

I tried and worked with pilot-db 1.1.0, 1.2.0 alpha 5 (the compiled
prc from sf.net), and the CVS version of today. Earlier versions needed
minor tweaks to compile against sdk-5r3, the CVS compiled cleanly.

I used the following to compile:
./configure --enable-standard --enable-debug --enable-sony=no --enable-handera=no --enable-fiveways=no --enable-make_plugins
without '--enable-sony=no --enable-handera=no --enable-fiveways=no', I
had some compilation problems, I guess because you need a different sdk
for this, didn't check.
As I said, in some cases I added '--with-vldb=0', more about that in
a different message.

On Mon, Apr 26, 2004 at 09:26:47AM -0600, Nathan Kurz wrote:
> Hello --
> 
> Thanks for your interest and time spent!
> 
> > 1. I translated the UI to Hebrew. You can get the file at
> > <http://www.cs.tau.ac.il/~didi/iw.rcp> (I did not know if it's
> > considered polite sending here attachments, so I put it there).
> > I did not translate everything, mostly the commonly-used things as
> > well as the ones easy to translate :-). If there is interest I do not
> > mind spending a few more hours on polishing this, but I think it's
> > already good enough for inclusion in a next version.
> 
> I'm probably the person who would include this, but I'm a new
> maintainer for this and I haven't yet dealt with adding languages.
> But if you can leave it up for a while (currently unspecified) I'll
> add it as a language once we get a release candidate mostly working.

Well, it's pretty simple, and documented in docs/translating.txt. I had
to add the file to rcp/langs/, and add the line 'include(iw.rcp)' to
rcp/db.rcp (or rather, to db.rcp.in, before running configure). Then,
'make db-iw.rcp' worked. No need to change the Makefile, as written there.
It's probably possible to even make the change to db.rcp automatic too,
but I didn't look at this.

But I do not mind that you'll wait with this - as I said, the translation
does need some more work, and I did not even start with the help file.
-- 
Didi

Re: [pilot-db-list] Hebrew support

[Nathan K. <na...@ve...>]

On Mon, Apr 26, 2004 at 09:57:21PM +0300, Yedidyah Bar-David wrote:
> OK, I will make it a separate message for each issue.
> I also subscribed to the list. Still, there are a few things I won't
> repeat in each message: I use Debian sid, with the following versions:
> prc-tools 2.2.90.cvs20030306-4 (has gcc 2.95.3, gdb 5.3)
> pose 3.5-6 (the first version working for me - for a few months I
> was pretty sure I'll have to use the simulator on Windows, which isn't
> as simple for me) with the ROM of a Palm m130 (PalmOS 4.1)
> palmos sdk 5r3

I'm having luck with Pose 3.5-2 with Slackware 9 and Linux 2.2.24.  I
recall problems with some versions regarding XWindows color depths
(24bpp did not work?), but otherwise have not have problems with Pose.
As I mentioned earlier, you will need the MSectGDB patches if you want
GDB to work more than minimally.  Note also that you need all the
pieces of those patches installed, not just the gdb replacement.

I've found that the Palm Simulators _almost_ work under Wine, and
perhaps someone with greater knowledge could actually make them work.
I have had success running GDB on Linux ethernet connected to
Simulators running on Windows.  If you do this, note that you will
need gdbpanel installed for  GDB to work with a Simulator.

> I tried and worked with pilot-db 1.1.0, 1.2.0 alpha 5 (the compiled
> prc from sf.net), and the CVS version of today. Earlier versions needed
> minor tweaks to compile against sdk-5r3, the CVS compiled cleanly.

Since getting involved in this project a couple months ago, I've spent
most of my time trying to get the CVS version working again.  I've
recently come to the conclusion that the CVS code is more trouble than
it is worth, represents an experimental branch from the release
version sometime prior to 1.1.0, and that the 1.1.0 should replace the
CVS version as the current head.

What this means is that you should concentrate your efforts on the
1.1.0 release code, and ignore the CVS and alpha series.  I'll try to
make the appropriate changes to the CVS repository later this week.

> I used the following to compile:
> ./configure --enable-standard --enable-debug --enable-sony=no --enable-handera=no --enable-fiveways=no --enable-make_plugins
> without '--enable-sony=no --enable-handera=no --enable-fiveways=no', I
> had some compilation problems, I guess because you need a different sdk
> for this, didn't check.

Yes, external SDK's are required for these.  Worse, these SDK's aren't
all easily available, and need to be extensively massaged if they are
to work with GCC (prc-tools).  While you can build successfully
without them, in the CVS version I found bugs in the non-enabled code
path.  Contact me if you'd like help getting these SDK's.

--nate

[pilot-db-list] development environment (was: Hebrew support)

[Yedidyah Bar-D. <di...@ta...>]

On Mon, Apr 26, 2004 at 01:50:38PM -0600, Nathan Kurz wrote:
> On Mon, Apr 26, 2004 at 09:57:21PM +0300, Yedidyah Bar-David wrote:
> > OK, I will make it a separate message for each issue.
> > I also subscribed to the list. Still, there are a few things I won't
> > repeat in each message: I use Debian sid, with the following versions:
> > prc-tools 2.2.90.cvs20030306-4 (has gcc 2.95.3, gdb 5.3)
> > pose 3.5-6 (the first version working for me - for a few months I
> > was pretty sure I'll have to use the simulator on Windows, which isn't
> > as simple for me) with the ROM of a Palm m130 (PalmOS 4.1)
> > palmos sdk 5r3
> 
> I'm having luck with Pose 3.5-2 with Slackware 9 and Linux 2.2.24.  I

Yes, but Slackware's "-2" isn't Debian's. It probably took debian more
versions to get something working.

> recall problems with some versions regarding XWindows color depths
> (24bpp did not work?), but otherwise have not have problems with Pose.

I only work with 16bpp, as it's a quite old card and 24bpp is less
convinient. Thanks for telling me, though, so I won't try :-)

> As I mentioned earlier, you will need the MSectGDB patches if you want
> GDB to work more than minimally.  Note also that you need all the
> pieces of those patches installed, not just the gdb replacement.

OK.

> 
> I've found that the Palm Simulators _almost_ work under Wine, and
> perhaps someone with greater knowledge could actually make them work.
> I have had success running GDB on Linux ethernet connected to
> Simulators running on Windows.  If you do this, note that you will
> need gdbpanel installed for  GDB to work with a Simulator.

I currently won't try this, as Windows is much less convinient for me.
If/When you make it work with wine, I might try this.

> 
> > I tried and worked with pilot-db 1.1.0, 1.2.0 alpha 5 (the compiled
> > prc from sf.net), and the CVS version of today. Earlier versions needed
> > minor tweaks to compile against sdk-5r3, the CVS compiled cleanly.
> 
> Since getting involved in this project a couple months ago, I've spent
> most of my time trying to get the CVS version working again.  I've
> recently come to the conclusion that the CVS code is more trouble than
> it is worth, represents an experimental branch from the release
> version sometime prior to 1.1.0, and that the 1.1.0 should replace the
> CVS version as the current head.
> 
> What this means is that you should concentrate your efforts on the
> 1.1.0 release code, and ignore the CVS and alpha series.  I'll try to
> make the appropriate changes to the CVS repository later this week.
> 

I will try and see if there is anything that works for me better with
the CVS version. If not, Fine. Otherwise I'll try to back-port to 1.1.0.

> > I used the following to compile:
> > ./configure --enable-standard --enable-debug --enable-sony=no --enable-handera=no --enable-fiveways=no --enable-make_plugins
> > without '--enable-sony=no --enable-handera=no --enable-fiveways=no', I
> > had some compilation problems, I guess because you need a different sdk
> > for this, didn't check.
> 
> Yes, external SDK's are required for these.  Worse, these SDK's aren't
> all easily available, and need to be extensively massaged if they are
> to work with GCC (prc-tools).  While you can build successfully
> without them, in the CVS version I found bugs in the non-enabled code
> path.  Contact me if you'd like help getting these SDK's.

No, unless you think I should also use them with 1.1.0.
-- 
Didi

Re: [pilot-db-list] development environment (was: Hebrew support)

[Nathan K. <na...@ve...>]

> > I've found that the Palm Simulators _almost_ work under Wine, and
> > perhaps someone with greater knowledge could actually make them work.
> > I have had success running GDB on Linux ethernet connected to
> > Simulators running on Windows.  If you do this, note that you will
> > need gdbpanel installed for  GDB to work with a Simulator.
> 
> I currently won't try this, as Windows is much less convinient for me.
> If/When you make it work with wine, I might try this.

Windows is terribly inconvenient for me as well, as besides making me
violent, I need to borrow a laptop just to run it.  The problem is
that all recent Palm devices (including all high resolution ones) will
not run on Pose.  So if one wishes to check for compatibility, one
either needs recent real devices to test on needs to run the Simulator.

Most of the known problems with the CVS version were with recent devices.

> > What this means is that you should concentrate your efforts on the
> > 1.1.0 release code, and ignore the CVS and alpha series.  I'll try to
> > make the appropriate changes to the CVS repository later this week.
> > 
> 
> I will try and see if there is anything that works for me better with
> the CVS version. If not, Fine. Otherwise I'll try to back-port to 1.1.0.

Feel free to use it, but to be clearer: by the end of the week, I hope
that the CVS version will become a branch called "Scott", and the
1.1.0 will become the version you get by default (HEAD).  The Scott
branch (current CVS) will be preserved for historical interest and
occasional code consultation, but likely will never be released again.
If you write patches for it, they will not be of use by others.

> > Yes, external SDK's are required for these.  Worse, these SDK's aren't
> > all easily available, and need to be extensively massaged if they are
> > to work with GCC (prc-tools).  While you can build successfully
> > without them, in the CVS version I found bugs in the non-enabled code
> > path.  Contact me if you'd like help getting these SDK's.
> 
> No, unless you think I should also use them with 1.1.0.

Probably you should, but you are more likely to be safe without them.

--nate

Re: [pilot-db-list] development environment (was: Hebrew support)

[Yedidyah Bar-D. <di...@ta...>]

On Tue, Apr 27, 2004 at 09:57:01AM -0600, Nathan Kurz wrote:
> > > I've found that the Palm Simulators _almost_ work under Wine, and
> > > perhaps someone with greater knowledge could actually make them work.
> > > I have had success running GDB on Linux ethernet connected to
> > > Simulators running on Windows.  If you do this, note that you will
> > > need gdbpanel installed for  GDB to work with a Simulator.
> > 
> > I currently won't try this, as Windows is much less convinient for me.
> > If/When you make it work with wine, I might try this.
> 
> Windows is terribly inconvenient for me as well, as besides making me
> violent, I need to borrow a laptop just to run it.  The problem is

For me it means to connect to a Windows Terminal Server at work, over
ADSL. Not that bad, compared to borrowing a laptop, but still...

> that all recent Palm devices (including all high resolution ones) will
> not run on Pose.  So if one wishes to check for compatibility, one
> either needs recent real devices to test on needs to run the Simulator.
> 
> Most of the known problems with the CVS version were with recent devices.

OK, but the *good* things are the same, I hope. So if you fix the bad
ones (or simply go to 1.1.0), I can still work with pose only, if all I
want is to add new stuff (or fix bug that also happen in pose, such as
the one I talked about).

BTW, I tried playing with the simulator very briefly. I saw it has its
own ROMs, which IIRC are the really compatible with real devices' ROMs.
Is it so? Doesn't this mean that for debugging a problem with a new real
device you still need a new real device (unless the problem isn't specific
to it, but rather is related to high resolution etc.)?

> 
> > > What this means is that you should concentrate your efforts on the
> > > 1.1.0 release code, and ignore the CVS and alpha series.  I'll try to
> > > make the appropriate changes to the CVS repository later this week.
> > > 
> > 
> > I will try and see if there is anything that works for me better with
> > the CVS version. If not, Fine. Otherwise I'll try to back-port to 1.1.0.
> 
> Feel free to use it, but to be clearer: by the end of the week, I hope
> that the CVS version will become a branch called "Scott", and the
> 1.1.0 will become the version you get by default (HEAD).  The Scott
> branch (current CVS) will be preserved for historical interest and
> occasional code consultation, but likely will never be released again.
> If you write patches for it, they will not be of use by others.

That's just fine with me - 1.1.0 is ok for me. I just wanted to make
sure that CVS doesn't also include fixes to bugs in 1.1.0 (and not only
the middle of adding new features). Are you sure it's not the case?

For example, io.c from 1.1.0 #includes CharAttr.h, which does not exist
in sdk5r3 (but does in older versions). CVS does not include it. Not that
it's such a big thing, but maybe there are other fixes.

> 
> > > Yes, external SDK's are required for these.  Worse, these SDK's aren't
> > > all easily available, and need to be extensively massaged if they are
> > > to work with GCC (prc-tools).  While you can build successfully
> > > without them, in the CVS version I found bugs in the non-enabled code
> > > path.  Contact me if you'd like help getting these SDK's.
> > 
> > No, unless you think I should also use them with 1.1.0.
> 
> Probably you should, but you are more likely to be safe without them.

If it's trivial for you to email, and not legally or otherwise
problematic, go ahead. I'd rather work on an environment as yours and
see the same bugs you see.
-- 
Didi

Re: [pilot-db-list] development environment (was: Hebrew support)

[Nathan K. <na...@ve...>]

> BTW, I tried playing with the simulator very briefly. I saw it has its
> own ROMs, which IIRC are the really compatible with real devices' ROMs.
> Is it so? Doesn't this mean that for debugging a problem with a new real
> device you still need a new real device (unless the problem isn't specific
> to it, but rather is related to high resolution etc.)?

There are some device specific Simulator ROM's as well, but you are
correct that they are not identical to the real ROM's.  The supposed
advantage is that as a Simulator rather than an Emulator, it can be
more aggressive in flagging subtle bugs.  Personally, I think the
Emulator made much more sense, but this could be my prejudice just
because it is open source and works on Linux.

> That's just fine with me - 1.1.0 is ok for me. I just wanted to make
> sure that CVS doesn't also include fixes to bugs in 1.1.0 (and not only
> the middle of adding new features). Are you sure it's not the case?

It's possible (and certainly seemed probable) that it contains fixes
not contained in 1.1.0, but I haven't found them yet.  Instead I've
found layers of untested code with subtle untested bugs, mismatched
parentheses in some code branches (indicating code was never
compiled), and the absence of fixes present in the 1.1.0 code.

The hope that it included fixes as well as problems was what kept me
plugging away at it, but I've now decided that that hope was
misplaced, and the lack of finding these fixes is why I'm planning to
rollback to version 1.1.0.  I'm uncertain about this decision, though.

> For example, io.c from 1.1.0 #includes CharAttr.h, which does not exist
> in sdk5r3 (but does in older versions). CVS does not include it. Not that
> it's such a big thing, but maybe there are other fixes.

This was one of the first changes I made to the CVS code.  You can
browse <http://cvs.sourceforge.net/viewcvs.py/pilot-db/pilot-db/src/>
to get an idea of what I've changed versus what was found there.  So
trivially true: it is going to have to be redone as a fix
(discouraging) but it doesn't count as a fix found in the CVS.

> > > > Yes, external SDK's are required for these.  Worse, these SDK's aren't
> > > > all easily available, and need to be extensively massaged if they are
> > > > to work with GCC (prc-tools).  While you can build successfully
> > > > without them, in the CVS version I found bugs in the non-enabled code
> > > > path.  Contact me if you'd like help getting these SDK's.
> > > 
> > > No, unless you think I should also use them with 1.1.0.
> > 
> > Probably you should, but you are more likely to be safe without them.
> 
> If it's trivial for you to email, and not legally or otherwise
> problematic, go ahead. I'd rather work on an environment as yours and
> see the same bugs you see.

I'll try to wrap them up for you tomorrow.  Anyone else interested in
trying to do developement?

--nate

[pilot-db-list] large DB support

[Yedidyah Bar-D. <di...@ta...>]

> > 2. The "large DB support" didn't work very well for him, so for now
> > I disabled it (compiled with '--with-vldb=0'). IIRC, it also wasn't
> > that much faster - and it _is_ an issue for him, having a 1000+
> > records DB with around 20 fields (still to change, but num of records
> > will not get smaller - it's his todo list). How experimental is this?
> > Is it going to become stable soon? Should I help debug this? Can I?
> 
> You compiled it!  Congratulations!  As best as I can tell, there are
> very few people on this list who have done so.  Be warned that many of
> the code paths seem to be unused in recent years, and that by
> including (or omitting) configuration flags you might soon find
> yourself in uncharted waters.

(repeating myself:)
I did
./configure --enable-standard --enable-debug --enable-sony=no --enable-handera=no --enable-fiveways=no --enable-make_plugins
With and without --with-vldb=0. Well, not the CVS one - only 1.1.0.
CVS I compiled only without it (that is, _with_ vldb support).
It wasn't that hard. Where is the catch?

> 
> Yes, it is probably going to become stable in the next release by
> simple method of removing all reference to the word "experimental".
> As far as I know (based on my recollection of submitted bug reports)
> it does not seem to add additional bugs.  I don't know about it's
> performance, but I think it purpose was simply to allow very large
> databases, not to necessarily make them work faster. (?)

Well, the 'configure --help' says:
"optimize for very large databases, and set the size of such a database".
Can you tell me what you think is the maximum of pilot-db without
this support, if it's not only optimization? It does work on a 1000+
record DB.

I did not yet try to reproduce the bug on the cvs - on 1.1.0 it sometimes
did not refresh the screen on Up/Down arrows. Since The User (my brother-
in-law) uses it daily, and is not friendly with computers, I chose for
him the slower but safer version. But it _is_ an issue.

> 
> Yes, you should help debug this! :) Submitting the slowness as a bug
> report to the SourceForge website, and then giving a sample database
> with some sample times would be good too.  I haven't yet tried much

I'll try to prepare something.
Maybe there is simply nothing to do? It is indeed a slow machine, and
a complex program. Are there profiling tools for this? prc does not
have gprof.

> profiling under POSE, but it does appear to be possible.  I'd guess
> (without having looked closely at the code) that there is lots of room
> for simple optimization.  If you can do it, wonderful, otherwise (like
> everything else) I'll try to get to it once we have a stable release.

OK.
-- 
Didi

[pilot-db-list] Searching Hebrew

[Yedidyah Bar-D. <di...@ta...>]

> 
> > 3. Searching for Hebrew phrases is a bit weird. When using one kit of
> > Hebrew support, the one he got with the palm (HarelHeb 3.5, IIRC), it
> > only finds things if they are at the end of a field. When using another
> > support kit (a 2-week temporary license from www.kalanit.co.il for
> > what seems like a newer version of the same HarelHeb, but not sure),
> > it works just fine. Is this reasonable? Is there anything pilot-db can
> > do about it, or is this only the foreign-language support's job? I
> > did grep the source and found it's MatchString, which calls
> > *Glue* functions. Are these part of the foreign-lang support?
> 
> I don't know---I haven't dealt much with the foreign language support,
> and not at all with the non-latin-alphabet languages.  Submitting it
> as a bug report would be useful to create a permanent record, but my
> offhand guess is that if we are calling the *Glue* functions
> correctly, there is not much else to be done on the pilot-db end.
> But it is certainly worth checking if we are calling them correctly...
> 

For now I only _read_ the relevant code, and it seemed ok to me, how
much this worth isn't clear. The fact that it does work with one kit
(the newer) suggests that it is indeed a bug in the older kit, not
pilot-db. I hoped there is someone on the list who does have some
experience with this, who would be able to tell if this analysis is
reasonable (in which case, the only thing left to do is to purchase
the newer version. BTW - do other languages support kits cost money,
in other countries? It's not cheap either - around $30-$40, for a
machine that cost him $150).
-- 
Didi

[pilot-db-list] record plugins in listview

[Yedidyah Bar-D. <di...@ta...>]

> > 4. Since it's too slow for daily work, he uses the memoexport plugin
> > for the daily tasks, and uses that, using pilot-db only for longer-term
> > planning. Is there an easy way to export at once all the records displayed
> > by a specific view/filter? I guess no. It sounds to me as something that's
> > generally-useful - to make all the one-record plugins also available from
> > the listview, making them operate on all the records. Does this make sense?
> > Or is the right thing writing another plugin that will be specific to this
> > (which he will probably need anyway, since he wants it to set also the
> > category etc.)?
> 
> I'd guess it is not possible either, but yes, it does sound useful.

I don't have anything to add, just adding a separate note. I might look
into this sometime.
-- 
Didi

[pilot-db-list] editing on a PC

[Yedidyah Bar-D. <di...@ta...>]

> > 5. I spent some time on editing the DB on a PC. I mostly want to do this
> > on linux, so dbeditor isn't my first choice. I tried to hack palm-db-tools
> > into exporting and importing filters (and eventually hoped for all other
> > data), but after spending on this around 10 hours, not getting it working
> > and deciding it's too hard to debug, I decided to quit. Instead, I searched
> > and found a small tool called par (7th google hit searching for 'par'),
> > which lets you extract the appinfo to a file, each record to a file, and
> > create a pdb from such files. So I intend to write a script that will
> > do this:
> > export and keep the appinfo
> > pdb2csv to file.csv
> > edit the file.csv (with openoffice, or maybe mysql+some web UI?)
> > csv2pdb
> > export all the records from the new pdb
> > create a pdb from the records and the saved appinfo
> > 
> > I verified that this theoretically works, but did not make thorough tests.
> > Should it? Is this a good way, considering no tool will follow closely
> > enough the development of pilot-db? And, BTW, isn't something written in
> > some higher-level language easier to write? E.g. things like
> > datascript or DataWorkshop? I know this isn't pilot-db-related, but
> > palm-db-tools seems dead to me.
> 
> We've had a bit of discussion this list about what needs to be done
> about this, but no solution yet.  I wouldn't give up on the hope of

I will read the archives.

> having a tool that would follow the development of pilot-db.  In fact,
> I think that is just about the only hope.  

Generally speaking, I agree. But this wasn't my point. What I wanted to
say is this: Even though there is currently no tool that knows about
recent additions (global scripts, filters), there is still a way to
edit the _data_ part without loosing all the metadata. Using only
palm-db-tools, pdb2csv and csv2pdb, this isn't possible - on the first
conversion you loose everything not supported by it. Using par, I keep
all the metadata in binary form, as a single appinfo file, and edit the
data with whatever I want.

Now, even without reading the archives (and hoping this doesn't sound
too rude): Suppose I agree there is a need for a tool that will fully
support all the metadata. When writing for palmos, it's pretty clear
why you (we?) write in C - we want it as fast and small as possible.
But when writing for a PC, we can use some higher level language. Either
a general-purpose scripting language (such as perl or python), or, which
is what I suggested, to use a tool that is specifically intended for such
a thing - a tool in which you define your data in some language and can
automatically edit or export/import it, without adding code in a lower
level language. The two examples I gave are datascript and DataWorkshop.
I have no experience with neither, I only found them now, but they do
look to me like the Right Thing (even if the implementation isn't, yet?).

What do you think?
-- 
Didi

Re: [pilot-db-list] editing on a PC

[Nathan K. <na...@ve...>]

> > > Should it? Is this a good way, considering no tool will follow closely
> > > enough the development of pilot-db? And, BTW, isn't something written in
> > > some higher-level language easier to write? E.g. things like
> > > datascript or DataWorkshop? I know this isn't pilot-db-related, but
> > > palm-db-tools seems dead to me.
> > 
> > We've had a bit of discussion this list about what needs to be done
> > about this, but no solution yet.  I wouldn't give up on the hope of
> 
> I will read the archives.

It was probably sometime in January, and it was only initial discussion.
I wasn't trying to imply that it should not be discussed further!

> > having a tool that would follow the development of pilot-db.  In fact,
> > I think that is just about the only hope.  
> 
> Generally speaking, I agree. But this wasn't my point. What I wanted to
> say is this: Even though there is currently no tool that knows about
> recent additions (global scripts, filters), there is still a way to
> edit the _data_ part without loosing all the metadata. Using only
> palm-db-tools, pdb2csv and csv2pdb, this isn't possible - on the first
> conversion you loose everything not supported by it. Using par, I keep
> all the metadata in binary form, as a single appinfo file, and edit the
> data with whatever I want.

Yes, certainly a tool that did not throw away all the information it
did not understand would be a useful improvement.  I must confess I
haven't actually tried using palm-db-tools in the last couple years. :(

> But when writing for a PC, we can use some higher level language. Either
> a general-purpose scripting language (such as perl or python),

Yes, definitely.  I think the best way to achieve this might be to
have a C language library that allows access to PDB files (ideally one
that actually shares most of its code with the Palm version), and then
write some interfaces to this from Perl, Python, etc.  But just doing
it directly in some other language would be fine too.

At this point, we could happily benefit from anything that works.

> is what I suggested, to use a tool that is specifically intended for such
> a thing - a tool in which you define your data in some language and can
> automatically edit or export/import it, without adding code in a lower
> level language.

Sadly, I fear what we really need is just a good definition of our
data format.  Currently, there does not seem to be one other than as
defined by the current C code.    

--nate

Re: [pilot-db-list] editing on a PC

[Yedidyah Bar-D. <di...@ta...>]

On Mon, Apr 26, 2004 at 02:06:33PM -0600, Nathan Kurz wrote:
[snip]
> 
> Yes, certainly a tool that did not throw away all the information it
> did not understand would be a useful improvement.  I must confess I
> haven't actually tried using palm-db-tools in the last couple years. :(

And you don't need to? Do you use a Palm regularly and is it good enuogh?
My typing speed on it is maybe 10 times slower than on a PC. That's the
main reason for me to edit on a PC. The actual user types much slower, but
he wants/needs to see more information at the same time. I even thought
about patching pilot-db to give each record in the listview more than one
screen line, to see longer fields without entering the recordview. Does
this sound good? Do you have any idea if it's easy to do?

> 
> > But when writing for a PC, we can use some higher level language. Either
> > a general-purpose scripting language (such as perl or python),
> 
> Yes, definitely.  I think the best way to achieve this might be to
> have a C language library that allows access to PDB files (ideally one
> that actually shares most of its code with the Palm version), and then

Ideally. But is it practical? From looking at both sources, it seems it
will require mostly rewriting at least one of them.

> write some interfaces to this from Perl, Python, etc.  But just doing
> it directly in some other language would be fine too.
> 
> At this point, we could happily benefit from anything that works.
> 
> > is what I suggested, to use a tool that is specifically intended for such
> > a thing - a tool in which you define your data in some language and can
> > automatically edit or export/import it, without adding code in a lower
> > level language.
> 
> Sadly, I fear what we really need is just a good definition of our
> data format.  Currently, there does not seem to be one other than as
> defined by the current C code.

Yes.
My current patch, from reading the code, to the file docs/format.txt, is:
In the CHUNK_FIELD_TYPES section, add:
  6 - List
  7 - Link
  8 - Float
  9 - Calculated
  10 - Linked
That's the easy one. Then add 3 sections: "CHUNK_DATABASE_INFO (255)"
and "CHUNK_ADVANCED_FILTER (1024)" (not yet documented), and

CHUNK_GLOBAL_SCRIPTS (512)

  char name[32]
  UInt8 bytecode_sz
  UInt8 version? 0x20
  UInt8 access
  UInt8 returnType
  UInt8 * bytecode (array of bytecode_sz bytes)

I also tried adding to palm-db-tools support for CHUNK_GLOBAL_SCRIPTS,
even without parsing the bytecode (which I do want to have - I even have
dreams of pdb2sql that will also create SQL views according to the views
and scripts you have), but after many hours of work got something that
works with a bug, which some more hours were not enough to solve. That's
the point I decided C isn't the right language for this - it's too hard
to debug. Well, at least for me - maybe there are better programmers that
will do it. Ah, one more thing: It's written in C++, not C, and I do not
know C++. This was the first time I tried to debug a C++ program. Is the
palm-db-tools project alive? Should I correspond with them, or here?

A patch against 0.3.6 of my current try is at
<http://www.cs.tau.ac.il/~didi/palm-db-tools-global-scripts.patch>, but
as I said, it's not working. In one of the directions (don't remember
which), it chops the bytecode in the first zero byte. Sounds like a
trivial wrong use of a *str* function, I know, but I could not find it.
That's a 300 added lines patch, way too much, IMHO, for such a thing.
I think in a very specific language for defining data, it should not be
much longer than the 10 lines it takes in English (in format.txt).

BTW, do you want to be in the "To:" line? I manually send to the list
only, guessing it's annoying for you to get everything twice. Is it?
-- 
Didi

Re: [pilot-db-list] editing on a PC

[Nathan K. <na...@ve...>]

> > Yes, certainly a tool that did not throw away all the information it
> > did not understand would be a useful improvement.  I must confess I
> > haven't actually tried using palm-db-tools in the last couple years. :(
> 
> And you don't need to? Do you use a Palm regularly and is it good enuogh?

No, that's the sad part.  I don't actually use a Palm now.  I've got a
Palm III deep in my closet somewhere, but I'm doing everything
Simulators and Emulators.  I used DB on a Palm about 4 years ago, when
Tom Dyas was just developing it, and then came back to it recently
with an idea for how it could be made useful to me if it would support
better linking, automatic sorting, and multi-line record displays.

But as I started poking around, I realized the project was well on its
way to dying: there was no one left on the list who even had write
access to the SF code.  I managed to get CVS access from Tom Dyas, the
original but no longer involved author.  I started fixing things, and
only later realized that the CVS was junk and that 1.1.0 was the last
code that was ever compiled.

But I don't use it currently, since without the true link support, I
have no current use for it.  My goal is just to keep it breathing
until a stranger from the holy land appears to save it.  But who knows
when that will happen. :) (hope the hereticism doesn't offend)

> I even thought about patching pilot-db to give each record in the
> listview more than one screen line, to see longer fields without
> entering the recordview. Does this sound good? Do you have any idea
> if it's easy to do?

Great idea, hard to do because of the way records are layed out.  My
last plan (haven't thought about it for a bit) was basically to allow
"newlines" to be added to record views, such that items after the
newline would appear on a second line on a per view basis.  But this
wouldn't allow for one record to be on multiple lines.  The other way,
but probably harder to implement, would be to just wrap at the pixel
width if a "wrap" flag is set.  Likely one would also need word wrap.

> > > But when writing for a PC, we can use some higher level language. Either
> > > a general-purpose scripting language (such as perl or python),
> > 
> > Yes, definitely.  I think the best way to achieve this might be to
> > have a C language library that allows access to PDB files (ideally one
> > that actually shares most of its code with the Palm version), and then
> 
> Ideally. But is it practical? From looking at both sources, it seems it
> will require mostly rewriting at least one of them.

My assumption (perhaps unjustified) was that rewriting a PDB interface
from scratch, adding a Perl XS layer, and then writing the utility in
Perl would be easier than modifying the current db-tools code.  

Writing the entire thing in Perl (I'm more comfortable there than
Python) would probably be easier yet, but would be less generally
useful.  C is usually so much easier as an interface langauage (to
Python, to Java, etc).  

> Is the palm-db-tools project alive?  Should I correspond with them,
> or here?

As far as I know, it has no life other than here.  Neither
administrator listed is still involved, and there don't appear to have
been any changes to CVS in recent years.

> BTW, do you want to be in the "To:" line? I manually send to the list
> only, guessing it's annoying for you to get everything twice. Is it?

No, I'm happy with list only.  I added you to the first few messages
only because I wasn't sure if you had subscribed to the list yet.

--nate

Re: [pilot-db-list] editing on a PC

[Yedidyah Bar-D. <di...@ta...>]

On Tue, Apr 27, 2004 at 09:41:13AM -0600, Nathan Kurz wrote:
> > > Yes, certainly a tool that did not throw away all the information it
> > > did not understand would be a useful improvement.  I must confess I
> > > haven't actually tried using palm-db-tools in the last couple years. :(
> > 
> > And you don't need to? Do you use a Palm regularly and is it good enuogh?
> 
> No, that's the sad part.  I don't actually use a Palm now.  I've got a
> Palm III deep in my closet somewhere, but I'm doing everything
> Simulators and Emulators.  I used DB on a Palm about 4 years ago, when
> Tom Dyas was just developing it, and then came back to it recently
> with an idea for how it could be made useful to me if it would support
> better linking, automatic sorting, and multi-line record displays.
> 

I also have no Palm. I do this mostly for my brother-in-law and for fun.
I also tend to think that if/when I will buy a machine of this size, I
will still want it to run Linux, and guess it will be easier for me to
use this way. But maybe I am wrong - using unix on a 80x25 text console
or on a 1300x1000 graphic one is probably very different from on a 160x160
(or even a 320x320) one.

> But as I started poking around, I realized the project was well on its
> way to dying: there was no one left on the list who even had write
> access to the SF code.  I managed to get CVS access from Tom Dyas, the
> original but no longer involved author.  I started fixing things, and
> only later realized that the CVS was junk and that 1.1.0 was the last
> code that was ever compiled.
> 
> But I don't use it currently, since without the true link support, I
> have no current use for it.  My goal is just to keep it breathing
> until a stranger from the holy land appears to save it.  But who knows
> when that will happen. :) (hope the hereticism doesn't offend)

That's not me :-)

> 
> > I even thought about patching pilot-db to give each record in the
> > listview more than one screen line, to see longer fields without
> > entering the recordview. Does this sound good? Do you have any idea
> > if it's easy to do?
> 
> Great idea, hard to do because of the way records are layed out.  My
> last plan (haven't thought about it for a bit) was basically to allow
> "newlines" to be added to record views, such that items after the
> newline would appear on a second line on a per view basis.  But this
> wouldn't allow for one record to be on multiple lines.  The other way,
> but probably harder to implement, would be to just wrap at the pixel
> width if a "wrap" flag is set.  Likely one would also need word wrap.

I think both are generally needed, but not that urgent. It of course
depends on the specific use.

> 
> > > > But when writing for a PC, we can use some higher level language. Either
> > > > a general-purpose scripting language (such as perl or python),
> > > 
> > > Yes, definitely.  I think the best way to achieve this might be to
> > > have a C language library that allows access to PDB files (ideally one
> > > that actually shares most of its code with the Palm version), and then
> > 
> > Ideally. But is it practical? From looking at both sources, it seems it
> > will require mostly rewriting at least one of them.
> 
> My assumption (perhaps unjustified) was that rewriting a PDB interface
> from scratch, adding a Perl XS layer, and then writing the utility in
> Perl would be easier than modifying the current db-tools code.  
> 

Yes, that's also what I think.

> Writing the entire thing in Perl (I'm more comfortable there than
> Python) would probably be easier yet, but would be less generally
> useful.  C is usually so much easier as an interface langauage (to
> Python, to Java, etc).  
> 

I know no Perl, no Python, and no Java. Only C and sed/awk/shell (and
some other less useful (at least for this) stuff). I did not mention,
that I am a sysadmin. I do program a bit from time to time, but do not
consider myself a good programmer.

> > Is the palm-db-tools project alive?  Should I correspond with them,
> > or here?
> 
> As far as I know, it has no life other than here.  Neither
> administrator listed is still involved, and there don't appear to have
> been any changes to CVS in recent years.

Yes, I saw that.
Are you in touch with any other relevant project? jpilot-db, dbeditor?

> 
> > BTW, do you want to be in the "To:" line? I manually send to the list
> > only, guessing it's annoying for you to get everything twice. Is it?
> 
> No, I'm happy with list only.  I added you to the first few messages
> only because I wasn't sure if you had subscribed to the list yet.

OK. This was a good opportunity to learn a bit more about mutt, and add
'subscribe pil...@li...' to my .muttrc. This also
causes mutt to add a header 'Mail-Followup-To' which it looks at to see
if you want email CCed to you or not.
-- 
Didi

Re: [pilot-db-list] editing on a PC

[Yedidyah Bar-D. <di...@ta...>]

On Mon, Apr 26, 2004 at 02:06:33PM -0600, Nathan Kurz wrote:
> > > > Should it? Is this a good way, considering no tool will follow closely
> > > > enough the development of pilot-db? And, BTW, isn't something written in
> > > > some higher-level language easier to write? E.g. things like
> > > > datascript or DataWorkshop? I know this isn't pilot-db-related, but
> > > > palm-db-tools seems dead to me.

I searched some more and found a few other relevant tools. Some of them:

<http://www.eecs.harvard.edu/~nr/pilot/pdb/>
"This program converts PDB files to a human-readable, human-editable, ASCII form, and back again. In fact, the ASCII form is designed to be more than just editable by humans; it's also easily editable by programs, provided the programs are written in a little language called Lua. The converter translates between PDB files and Lua data values."
They already have modules for jfile and mobiledb. jfile is 400 lines,
mobiledb is 210. Quite dense (compared to 790 lines for JFile3.{h,cpp}
and 764 for MobileDB.{h,cpp} in palm-db-tools).
Last release was in 2000.

<http://obermuhlner.com/public/Projects/Palm/PDBC/>
"    ``pdbc'' is a compiler than converts a source file into a Palm DataBase
    (PDB) file or Palm Resource (PRC) file. The language the ``pdbc''
    compiler understands is designed to be easy to understand und write. It
    is powerful enough to give the freedom to describe the binary content of
    the database records in a variety of ways.

    It should also be easy to generate the ``pdbc'' source file from another
    language (e.g. Perl).

    In addition to the ``pdbc'' tool there is also 'pdbdec'. pdbdec is a
    decompiler which converts any valid PDB or PRC file into a ``pdbc''
    source file. This is useful to check the correctness of the ``pdbc''
    generated file and to analyze other PDB files."
It is not easy to add specific support for pilot-db to it, but it still
seems useful. In the man page he says he has some idea for templates,
which sounds just like what I thought about, but only an idea.
Latest release from 2002.

The terms I used in google for the searches: 'pdb binary convert',
'pdb binary editor', 'pdb structure convert'. I can't be sure I did not
miss some killerapp that does everything, but it sure didn't use the
right words :-)

What do you think?
-- 
Didi

[pilot-db-list] speed of transferring

[Yedidyah Bar-D. <di...@ta...>]

> > 6. Not terribly important, but transferring it to/from the palm takes
> > quite a lot of time - around 45 seconds, for a 170KB file. For comparison,
> > db-iw.prc itself, which is larger, takes much less time (not sure, I think
> > around 13 seconds). Is this reasonable? Is it because of the many records?
> 
> I don't know, but that does sound odd.  I've done almost all my
> testing on Pose and I haven't run into this problem. 

It's very fast with pose, doing "Install DB". It's slow with the real
palm, and with pilot-xfer. I didn't look (yet?) and pose's emulation
of hotsync, to see if it's the same.
-- 
Didi

[pilot-db-list] a bug in recordview up/down arrows

[Yedidyah Bar-D. <di...@ta...>]

> > 7. There is a bug, that sometimes, when pressing the down-arrow or up-arrow
> > icons on the buttom-right, in the "record view" (is this the name?), it
> > crashes, requiring a soft reset. Pressing PgDn does work, so that's what
> > he does for now. I tried for some time to debug this with pose and gdb,
> > but did not really get anywhere. It says:
> > "DB (1.1.0) called SysFatalAlert with the message:
> > "leakdetect.c, Line:188, addr"."
> > and gives control to gdb, but I can't see too much - 'where' says
> > (gdb) where
> > #0  0x1001d8b2 in ErrDisplayFileLineMsg ()
> > #1  0x00091b88 in ?? ()
> > #2  0x0008955c in ?? ()
> > #3  0x0007b770 in ?? ()
> > #4  0x10066b08 in PrvSendEventToForm ()
> > #5  0x10069f2c in FrmDispatchEvent ()
> > #6  0x0006f670 in EventLoop () at main.c:201
> > #7  0x0006f8a6 in EditViewGetFieldHeight () at main.c:715
> > #8  0x0006f3e4 in CheckMinVersion () at crt0.c:69
> > which does not help much. I tried to also put a breakpoint in
> > leakdetect.c:188, but also didn't get much further. 
> 
> I fixed something similar in the CVS version (Danger! Don't go
> there!). 

Already done :-) but only on pose, it's not dangerous (except maybe
for my mental health?).

> If possible, it would be great to see if you can reproduce
> this bug with a stock binary release.  There are a number of

If you refer to db-1.2.0-alpha5.prc then it's mostly the same. It
crashes at the same point, but with a different message:

"DB (1.2.0m) called SysFatalAlert with the message:
"MemoryMgr.c, Line:4372, Non-word-aligned handle"."

> configuration oddities that might cause this (some codepaths seem not
> to have been used in years) and it that's causing this.
> 
> Submitting this as a bug would be good for record keeping:
>   http://sourceforge.net/tracker/?group_id=621&atid=100621

OK.

> 
> > How do I debug pilot-db? Is there some tutorial somewhere?
> 
> No, there is not much published information, but I'm eager and willing
> to help you figure it out, and if we do it on-list it will probably
> help people in the future.   
> 
> It looks like the problem you are hitting now is that pilot-db is a
> multi-segment application, and that the standard prc-tools kit
> (including m68k-palmos-gdb) does not really support multiple code
> segments.  But there are patches available to do so:
>   http://www.v-overbeek.nl/msectgdb/
> While it's possible to do the compile from scratch, it's a lengthy
> multi-step procedure.  I submitted Linux binaries to Ton, which should
> be now available on his site.  

I'll look there.
-- 
Didi

Re: [pilot-db-list] a bug in recordview up/down arrows

[Nathan K. <na...@ve...>]

> > I fixed something similar in the CVS version (Danger! Don't go
> > there!). 
> 
> Already done :-) but only on pose, it's not dangerous (except maybe
> for my mental health?).

Yes, mostly I mean for your mental health.  When I came to this
project a few months ago, I found that the previous maintainers had
disappeared*, and that the CVS was left in an uncompileable state. 

Thinking that the CVS code probably represented the most current code,
I set off getting it into working order.  Now several months later,
I'm realizing that it represents an incomplete early branch with lots
of untested code, and that the 1.1.0 release was maintained and
developed outside of CVS!  Frankly, I find this disgusting.

Essentially, I've wasted my time and that of many others by having
them test the CVS version, which now works reasonably well under Pose
(which I have access to) but still fails miserably on most real modern
machines (which I don't currently have access to).

AVOID THE CVS VERSION UNTIL 1.1.0 BECOMES THE NEW HEAD!

> > If possible, it would be great to see if you can reproduce
> > this bug with a stock binary release.  There are a number of
> 
> If you refer to db-1.2.0-alpha5.prc then it's mostly the same. It
> crashes at the same point, but with a different message:

No, 1.2.0 alpha series is based on the CVS code, which I'm suggesting
you avoid spending time on.  I was hoping you could try it a
precompiled 1.1.0 binary and see if you can reproduce the problem.
Otherwise, I fear it may have to do with the particular choice of
compile options which you (and I, as the compiler of the alpha) chose.

--nate

* - Marc has just lately resurfaced, at least partially, and his
    comments have helped me understand the role of the CVS code.

Re: [pilot-db-list] a bug in recordview up/down arrows

[Yedidyah Bar-D. <di...@ta...>]

On Mon, Apr 26, 2004 at 02:17:55PM -0600, Nathan Kurz wrote:
> > > I fixed something similar in the CVS version (Danger! Don't go
> > > there!). 
> > 
> > Already done :-) but only on pose, it's not dangerous (except maybe
> > for my mental health?).
> 
> Yes, mostly I mean for your mental health.  When I came to this
> project a few months ago, I found that the previous maintainers had
> disappeared*, and that the CVS was left in an uncompileable state. 
> 
> Thinking that the CVS code probably represented the most current code,
> I set off getting it into working order.  Now several months later,
> I'm realizing that it represents an incomplete early branch with lots
> of untested code, and that the 1.1.0 release was maintained and
> developed outside of CVS!  Frankly, I find this disgusting.

Now I understand your comments in the alpha5 release notes.

> 
> Essentially, I've wasted my time and that of many others by having
> them test the CVS version, which now works reasonably well under Pose
> (which I have access to) but still fails miserably on most real modern
> machines (which I don't currently have access to).
> 
> AVOID THE CVS VERSION UNTIL 1.1.0 BECOMES THE NEW HEAD!
> 
> > > If possible, it would be great to see if you can reproduce
> > > this bug with a stock binary release.  There are a number of
> > 

It's the same in the binary 1.1.0, but with two different messages:
db-standard:
DB (1.1.0m) called SysFatalAlert with the message:
"MemoryMgr.c, Line:4365, NULL handle".

db-mid:
same as db-standard

db-light:
DB (1.1.0l) called SysFatalAlert with the message:
"MemoryMgr.c, Line:4372, Non-word-aligned handle".

I tried each twice, on the same DB. I will, as promised, try to make
a sample DB that exposes this. I think it happens always if you have
enough fields, though.

> > If you refer to db-1.2.0-alpha5.prc then it's mostly the same. It
> > crashes at the same point, but with a different message:
> 
> No, 1.2.0 alpha series is based on the CVS code, which I'm suggesting
> you avoid spending time on.  I was hoping you could try it a
> precompiled 1.1.0 binary and see if you can reproduce the problem.
> Otherwise, I fear it may have to do with the particular choice of
> compile options which you (and I, as the compiler of the alpha) chose.
> 
> --nate
> 
> * - Marc has just lately resurfaced, at least partially, and his
>     comments have helped me understand the role of the CVS code.
-- 
Didi