linuxha-misc — General topics for Linux Home Automation

[LHA-misc] HA of the rich and famous ...

[Neil C. <nc...@ho...>]

http://www0.mercurycenter.com/svtech/news/special/digitalhome/

-- 
Linux Home Automation           Neil Cherry             nc...@ho...
http://members.home.net/ncherry                         (Text only)
http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
http://linuxha.sourceforge.net/				(SourceForge)

Re: [LHA-misc] Line parsing routine

[Jay H. <jh...@fa...>]

At 02:51 PM 12/1/00 -0500, you wrote:
>  >
>  > > Embedded Pics (Viso) or Line Art?
>  >
>  > Visio, I can convert that to jpeg or gif. This way we can convert
>  > everything into an HTML doc. HTML is the easiest to handle.
>  >
>
>don't forget about having an easy-to-print form.  HTML is fine for
>everything but convenient printing, in my experience.
>
>(for someone not doing any of the work, i'm doing a lot kibitzing.  :-)

You just *think* you're not doing a lot of the work ;)  Just wait!

Actually, it is really nice to have another voice pointing out
things that were missed. Neil and I have been bouncing stuff around
for about a year now and while we generally agree on the overal
architecture (I think) our "real" jobs have kept us from making
real headway.  I think if I can get this doc together and we can
get some basic stuff using it the ball will start rolling.

My first 4 are Adicon, Ocelot, Napco 9600, and RCS TR-36. I've
already got a C++ foundation class that you just "plug in" the
protocol you need and implementations for commands - I'm already
talking to 3 of 4 but haven't put all the commands in since the
latest address format was only conceived back in August and I
haven't had any long air flights since then to write the rest
of the code...

More coming... I've got the last 2 weeks of the year off and am
hoping to have the doc in a format to at least work from and
the Adicon/Ocelot/Napco modules running for people to comment on.

Jay



>paul
>=---------------------
>   paul fox, pg...@fo... (arlington, ma, where it's 34.2 degrees)
>_______________________________________________
>http://lists.sourceforge.net/mailman/listinfo/linuxha-misc
>To unsubscribe, send "unsubscribe Linuxha-misc"
>in the body of a message to  Lin...@li...

Re: [LHA-misc] Line parsing routine

[Neil C. <nc...@ho...>]

Jay Hogg wrote:
> 
> At 11:46 AM 12/1/00 -0500, you wrote:
> >  > I'mm thinking to include a "standard download" in the agent document
> >  > something like this:
> >  >
> >  > DOWNLOAD type="start" addr="..." type="firmware|lir|voice|..."
> > encode="..."
> >  >            name="..." persistent="NO|yes" autoload="NO|yes"
> >  > DOWNLOAD type="data" srec="..."
> >  > DOWNLOAD type="done"  (it now gets loaded to the device)
> >  > DOWNLOAD type="abort"
> >
> >i think one would also want the ability to "upload".
> 
> Makes sense. It'll take some minor thinking because the "pull" from
> the target device will have to be done and converted but I believe
> this should be supported (Ocelot, SpeakEZ, Leopard, Napco).
> 
> Does the format look good?
> Now the bidirectional transfers are in the mix I think the terminoligy
> is basackwards. Switch to "PUT" and "GET" from the "user" of the
> agent point-of-view. Put=Send=Download. Get=Receive=Upload.

Adding a few more understandable words is not a problem. I always thought
set=send (from Server to Agent (daemon)) get=receive=monitor ... (from
Agent (daemon) to Server).

-- 
Linux Home Automation           Neil Cherry             nc...@ho...
http://members.home.net/ncherry                         (Text only)
http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
http://linuxha.sourceforge.net/				(SourceForge)

Re: [LHA-misc] Line parsing routine

[Jay H. <jh...@fa...>]

At 11:46 AM 12/1/00 -0500, you wrote:
>  > I'mm thinking to include a "standard download" in the agent document
>  > something like this:
>  >
>  > DOWNLOAD type="start" addr="..." type="firmware|lir|voice|..." 
> encode="..."
>  >            name="..." persistent="NO|yes" autoload="NO|yes"
>  > DOWNLOAD type="data" srec="..."
>  > DOWNLOAD type="done"  (it now gets loaded to the device)
>  > DOWNLOAD type="abort"
>
>i think one would also want the ability to "upload".

Makes sense. It'll take some minor thinking because the "pull" from
the target device will have to be done and converted but I believe
this should be supported (Ocelot, SpeakEZ, Leopard, Napco).

Does the format look good?
Now the bidirectional transfers are in the mix I think the terminoligy
is basackwards. Switch to "PUT" and "GET" from the "user" of the
agent point-of-view. Put=Send=Download. Get=Receive=Upload.

Jay


>paul
>=---------------------
>   paul fox, pg...@fo... (arlington, ma, where it's 34.7 degrees)
>_______________________________________________
>http://lists.sourceforge.net/mailman/listinfo/linuxha-misc
>To unsubscribe, send "unsubscribe Linuxha-misc"
>in the body of a message to  Lin...@li...

Re: [LHA-misc] Line parsing routine

[Paul F. <pg...@fo...>]

 > Drives are long gone but I actually ran into a winchester controller card
 > the other day. Some of the younger guys couldn't id it. Also found a
 > Mono/Printer card that had ~40 socketed 16-22pin IC's it dated 1985

hey -- i still use one of those.  :-)

paul
=---------------------
  paul fox, pg...@fo... (arlington, ma, where it's 24.3 degrees)

Re: [LHA-misc] Line parsing routine

[Neil C. <nc...@ho...>]

Jay Hogg wrote:
> 
> At 10:07 AM 12/1/00 -0500, you wrote:
> >Jay Hogg wrote:
> >> Forget the rock! I've got a stack of 40meg HD's that we use as door stops ;)
> >
> >I'm glad you guys don't have a stock pile fo the XT's original 10M Disks! ;-)
> 
> Drives are long gone but I actually ran into a winchester controller card
> the other day. Some of the younger guys couldn't id it. Also found a
> Mono/Printer card that had ~40 socketed 16-22pin IC's it dated 1985

I just picked out of the garbage an IBM XT, it's in original condition
but with the addition of an AST 384K/Clock/Game board and 1200i Hayes
modem. She still boots and the 20 meg drive is pretty quiet!

> >> - It can be decoded/cached in the receiving process and when the
> >>    S7/S9 record format arrives it can be validated and d'lded as
> >>    one piece.
> >
> >I'm not familiar with this, any further info on the web somewhere?
> 
> Which part - S7/S9 or caching?  S7/S9 is part of the record format
> but I've never seen a "standard" doc on although I know they exist.

I've seen that doc and I may actually have it (I have way too much
junk ;-).

S0 - comment section
S1 - 8 bit data
S2 - 16 bit?
S3 - 16 bit?
S4 - 32 bit?
S5 - 32 bit?
S9 - end of record

> The "caching" is more targeted at the agent. Since it will require
> "download pgm" -> "Agent" -> "Device" I'd like to see the agent
> build a complete valid image (disk, shared memory that can be
> released) and validated it so it can be blasted to the device. This
> image is, of course, device dependant - for the Ocelot it would be
> regenerating the binary image - for my PRI cards I would just store
> the S records and make sure they are all in tact and match the S9...

Oh, I see, I'm doing that with the HCSd right now, it doesn't make
sense to screw up the controller with half ad/l, an aborted one or
a wrong one.
 
> >> - Most compilers can generate it.
> >
> >I also have a ton of tools to generate these records (Moto, Intel,
> >Tek, Inmos and a few others). I've written several in BASIC for my
> >NEC-PR8001 (like the RS Model 100 portable).
> 
> I know very few high-end developer with a lot of experience that haven't
> had to deal with S records and some of the other "lower level" stuff
> before so I'm not in the least suprised.

I started as an EET but I learnt ;-) assembly and BASIC. The rest of the
high level languages I taught to myself. The S records and hex files are
a requirement for assembly langauge and EPROM burners (TI 2516 anyone).

-- 
Linux Home Automation           Neil Cherry             nc...@ho...
http://members.home.net/ncherry                         (Text only)
http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
http://linuxha.sourceforge.net/				(SourceForge)

Re: [LHA-misc] Line parsing routine

[Paul F. <pg...@fo...>]

 > 
 > > Embedded Pics (Viso) or Line Art?
 > 
 > Visio, I can convert that to jpeg or gif. This way we can convert
 > everything into an HTML doc. HTML is the easiest to handle.
 > 

don't forget about having an easy-to-print form.  HTML is fine for
everything but convenient printing, in my experience.

(for someone not doing any of the work, i'm doing a lot kibitzing.  :-)

paul
=---------------------
  paul fox, pg...@fo... (arlington, ma, where it's 34.2 degrees)

RE: [LHA-misc] Line parsing routine

[Cherry, N. J, N. <nc...@at...>]

> >The internet 'frig is coming, the internet 'frig is coming .... :-)
> 
> Just as long as I don't get emails at 5pm every day that go:
> 2gal Milk
> 1dz  Eggs, Large
> Less than 1 bowl of Dutch Chocolate ice cream
> The meatloaf from monday is growing in mass - you may want to 
> check it.

Hehe. Actually I would expect something more along the lines of:

Your current weigth is xxx lbs (snicker) :-)

> >Now where the heck to docuemnt this!
> 
> I'mm thinking to include a "standard download" in the agent document
> something like this:
> 
> DOWNLOAD type="start" addr="..." 
> type="firmware|lir|voice|..." encode="..."
>            name="..." persistent="NO|yes" autoload="NO|yes"
> DOWNLOAD type="data" srec="..."
> DOWNLOAD type="done"  (it now gets loaded to the device)
> DOWNLOAD type="abort"

This looks good.

> It is up to the agent to look at the address and decide if it can take
> the download and what to do with it. Probably also add to the initial
> start request of the data should be persistent and 
> auto-downloaded. Some
> things like firmware you have to watch flash cycles but other things
> come up dumb...
> 
> This could also be used to load new programs on a TINI...

Good one, Mike Baptiste just released his HCSLog.java for the TINI.
What that does is allow you to let the TINI front end the HCS. He's
currently working on being able to send to the HCS. Currently it
just sends out data. Still very cool.
 
> On that document...
> Text or Word (95/98)?

Text or HTML, if it's already a Word DOC I can convert it. I've
never got Word to convert from DOC to HTML. It always loses the
setting.

> Embedded Pics (Viso) or Line Art?

Visio, I can convert that to jpeg or gif. This way we can convert
everything into an HTML doc. HTML is the easiest to handle.

Linux Home Automation           Neil Cherry             nc...@ho...
http://members.home.net/ncherry                         (Text only)
http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
http://linuxha.sourceforge.net/                         (SourceForge) 

Re: [LHA-misc] Line parsing routine

[Paul F. <pg...@fo...>]

 > I'mm thinking to include a "standard download" in the agent document
 > something like this:
 > 
 > DOWNLOAD type="start" addr="..." type="firmware|lir|voice|..." encode="..."
 >            name="..." persistent="NO|yes" autoload="NO|yes"
 > DOWNLOAD type="data" srec="..."
 > DOWNLOAD type="done"  (it now gets loaded to the device)
 > DOWNLOAD type="abort"

i think one would also want the ability to "upload".

paul
=---------------------
  paul fox, pg...@fo... (arlington, ma, where it's 34.7 degrees)

Re: [LHA-misc] Line parsing routine

[Jay H. <jh...@fa...>]

At 10:43 AM 12/1/00 -0500, you wrote:
>Paul Fox wrote:
>> 
>>  > to note that if we needed compression on the fly that the file size
>>  > would hinder things. Because of my networking background I tend to think
>>  > that compression is best left to storage. If we compress data over a
>> 
>> in general you're right -- but if the data is, say, the memory image
>> of the device in question, then it clearly can't be stored compressed.
>> and equally clearly, it could be quite large -- so if you wanted to
>> request a full memory dump of your 32KB refrigerator controller, you
>> might want the ability to do on-the-fly compression.  another example
>> might be wanting to speed up the download of a large background image
>> to a dedicated controller -- perhaps for the touchscreen on that same
>> refrigerator.  :-)
>
>The internet 'frig is coming, the internet 'frig is coming .... :-)

Just as long as I don't get emails at 5pm every day that go:
2gal Milk
1dz  Eggs, Large
Less than 1 bowl of Dutch Chocolate ice cream
The meatloaf from monday is growing in mass - you may want to check it.


>You make a very good point and with the recommendation that Jay made (S Records)
>it's still possible to create a mode where the data sent is compressed and
>sent as a line of X characters with addressing and CRC. This is of course an
>extended mode. So there is a possible solution to a problem I hadn't thought
>of.
> 
>> but clearly compression is at odds with ascii data representation.  i say
>> do the simple things first, and leave the hooks in the protocol so you
>> can add what's necessary later.  (e.g. perhaps an optional "encoding=foo"
>> parameter.)
>
>Simple for now but we can extend as necessary. Leaving the option open
>is always a wise idea. We may never use it but we also haven't locked
>ourselves into anything too restrictive.
>
>Now where the heck to docuemnt this!

I'mm thinking to include a "standard download" in the agent document
something like this:

DOWNLOAD type="start" addr="..." type="firmware|lir|voice|..." encode="..."
           name="..." persistent="NO|yes" autoload="NO|yes"
DOWNLOAD type="data" srec="..."
DOWNLOAD type="done"  (it now gets loaded to the device)
DOWNLOAD type="abort"

It is up to the agent to look at the address and decide if it can take
the download and what to do with it. Probably also add to the initial
start request of the data should be persistent and auto-downloaded. Some
things like firmware you have to watch flash cycles but other things
come up dumb...

This could also be used to load new programs on a TINI...

On that document...
Text or Word (95/98)?
Embedded Pics (Viso) or Line Art?

Jay


>-- 
>Linux Home Automation           Neil Cherry             nc...@ho...
>http://members.home.net/ncherry                         (Text only)
>http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
>http://linuxha.sourceforge.net/                         (SourceForge)
>_______________________________________________
>http://lists.sourceforge.net/mailman/listinfo/linuxha-misc
>To unsubscribe, send "unsubscribe Linuxha-misc"
>in the body of a message to  Lin...@li...

Re: [LHA-misc] Line parsing routine

[Jay H. <jh...@fa...>]

At 10:07 AM 12/1/00 -0500, you wrote:
>Jay Hogg wrote:
>> Forget the rock! I've got a stack of 40meg HD's that we use as door stops ;)
>
>I'm glad you guys don't have a stock pile fo the XT's original 10M Disks! ;-)

Drives are long gone but I actually ran into a winchester controller card
the other day. Some of the younger guys couldn't id it. Also found a
Mono/Printer card that had ~40 socketed 16-22pin IC's it dated 1985

> 
>> Actually, if you want to do downloads the best standard method around
>> that we use every day to download code to DSP's, embedded processors,
>> etc is Motorola S-Record format. It has the advantages that
>> - It is standardized in format
>> - Text/HEX printable based for dealing with 7-bit transmission
>> - Internal addresses/continuation/etc
>> - Internal checksums
>> - Standard line lengths (not to exceed 80 chars incl cr/lf)
>
>Very good points!
>
>> - It can be decoded/cached in the receiving process and when the
>>    S7/S9 record format arrives it can be validated and d'lded as
>>    one piece.
>
>I'm not familiar with this, any further info on the web somewhere?

Which part - S7/S9 or caching?  S7/S9 is part of the record format
but I've never seen a "standard" doc on although I know they exist.

The "caching" is more targeted at the agent. Since it will require
"download pgm" -> "Agent" -> "Device" I'd like to see the agent
build a complete valid image (disk, shared memory that can be
released) and validated it so it can be blasted to the device. This
image is, of course, device dependant - for the Ocelot it would be
regenerating the binary image - for my PRI cards I would just store
the S records and make sure they are all in tact and match the S9...


>> - Most compilers can generate it.
>
>I also have a ton of tools to generate these records (Moto, Intel,
>Tek, Inmos and a few others). I've written several in BASIC for my
>NEC-PR8001 (like the RS Model 100 portable).

I know very few high-end developer with a lot of experience that haven't
had to deal with S records and some of the other "lower level" stuff
before so I'm not in the least suprised.

Jay


>-- 
>Linux Home Automation           Neil Cherry             nc...@ho...
>http://members.home.net/ncherry                         (Text only)
>http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
>http://linuxha.sourceforge.net/                         (SourceForge)
>
>S9030000FC
>_______________________________________________
>http://lists.sourceforge.net/mailman/listinfo/linuxha-misc
>To unsubscribe, send "unsubscribe Linuxha-misc"
>in the body of a message to  Lin...@li...

Re: [LHA-misc] Line parsing routine

[Neil C. <nc...@ho...>]

Paul Fox wrote:
> 
>  > to note that if we needed compression on the fly that the file size
>  > would hinder things. Because of my networking background I tend to think
>  > that compression is best left to storage. If we compress data over a
> 
> in general you're right -- but if the data is, say, the memory image
> of the device in question, then it clearly can't be stored compressed.
> and equally clearly, it could be quite large -- so if you wanted to
> request a full memory dump of your 32KB refrigerator controller, you
> might want the ability to do on-the-fly compression.  another example
> might be wanting to speed up the download of a large background image
> to a dedicated controller -- perhaps for the touchscreen on that same
> refrigerator.  :-)

The internet 'frig is coming, the internet 'frig is coming .... :-)

You make a very good point and with the recommendation that Jay made (S Records)
it's still possible to create a mode where the data sent is compressed and
sent as a line of X characters with addressing and CRC. This is of course an
extended mode. So there is a possible solution to a problem I hadn't thought
of.
 
> but clearly compression is at odds with ascii data representation.  i say
> do the simple things first, and leave the hooks in the protocol so you
> can add what's necessary later.  (e.g. perhaps an optional "encoding=foo"
> parameter.)

Simple for now but we can extend as necessary. Leaving the option open
is always a wise idea. We may never use it but we also haven't locked
ourselves into anything too restrictive.

Now where the heck to docuemnt this!

-- 
Linux Home Automation           Neil Cherry             nc...@ho...
http://members.home.net/ncherry                         (Text only)
http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
http://linuxha.sourceforge.net/				(SourceForge)

Re: [LHA-misc] Line parsing routine

[Paul F. <pg...@fo...>]

 > to note that if we needed compression on the fly that the file size
 > would hinder things. Because of my networking background I tend to think
 > that compression is best left to storage. If we compress data over a

in general you're right -- but if the data is, say, the memory image
of the device in question, then it clearly can't be stored compressed.
and equally clearly, it could be quite large -- so if you wanted to
request a full memory dump of your 32KB refrigerator controller, you
might want the ability to do on-the-fly compression.  another example
might be wanting to speed up the download of a large background image
to a dedicated controller -- perhaps for the touchscreen on that same
refrigerator.  :-)

but clearly compression is at odds with ascii data representation.  i say
do the simple things first, and leave the hooks in the protocol so you
can add what's necessary later.  (e.g. perhaps an optional "encoding=foo"
parameter.)

paul
=---------------------
  paul fox, pg...@fo... (arlington, ma, where it's 33.4 degrees)

Re: [LHA-misc] Line parsing routine

[Neil C. <nc...@ho...>]

Jay Hogg wrote:
> Forget the rock! I've got a stack of 40meg HD's that we use as door stops ;)

I'm glad you guys don't have a stock pile fo the XT's original 10M Disks! ;-)
 
> Actually, if you want to do downloads the best standard method around
> that we use every day to download code to DSP's, embedded processors,
> etc is Motorola S-Record format. It has the advantages that
> - It is standardized in format
> - Text/HEX printable based for dealing with 7-bit transmission
> - Internal addresses/continuation/etc
> - Internal checksums
> - Standard line lengths (not to exceed 80 chars incl cr/lf)

Very good points!

> - It can be decoded/cached in the receiving process and when the
>    S7/S9 record format arrives it can be validated and d'lded as
>    one piece.

I'm not familiar with this, any further info on the web somewhere?

> - Most compilers can generate it.

I also have a ton of tools to generate these records (Moto, Intel,
Tek, Inmos and a few others). I've written several in BASIC for my
NEC-PR8001 (like the RS Model 100 portable).

-- 
Linux Home Automation           Neil Cherry             nc...@ho...
http://members.home.net/ncherry                         (Text only)
http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
http://linuxha.sourceforge.net/				(SourceForge)

S9030000FC

Re: [LHA-misc] Line parsing routine

[Jay H. <jh...@fa...>]

At 09:10 AM 12/1/00 -0500, you wrote:
>Paul Fox wrote:
>> 
>>  >
>>  > Without the size of the file we then need to change the way we download
>>  > the file to the Ocelot (which might not be a bad thing). Here is an
>>  > example:
>>  >
>>  > set addr=Ocelot:0.0.0 value=5 value=^@^M^J^Z^D
>> 
>> i feel like i'm starting to contribute to something i don't know enough
>> about, since i confess i haven't been following development closely.  but
>> if the protocol has been chosen as an ascii protocol for ease of debugging
>> (which is a big advantage of the smtp/ftp style of handshake) then i really
>> think you shouldn't be sending binary data as binary.  there are lots of
>> good encoding schemes to choose from.  remember -- you're going to want
>> to test this from telnet session.  and what you'll have in front of
>> you is a paper document from a device mfgr that gives hex codes.  very
>> convenient to be able to type 'value=\00\0d\0a\5a\04".
>
>You seem to be keeping up with everything pretty well. It was a nice point
>to note that if we needed compression on the fly that the file size
>would hinder things. Because of my networking background I tend to think
>that compression is best left to storage. If we compress data over a
>dialup we lose bandwidth because most modems use compression to get the
>extra bandwidth. From the tests I've done, I've seen the compressed file
>grow to 1.5 times the compressed size.
>
>I'll be honest, my thinking slipped. I want the hex-ASCII (hey what is
>correct name for that) for easy debugging. It works well and isn't much of
>a resource hog, although it does eat a little bit of bandwidth. But like
>Microsoft, I think bandwidth is infinite :-) (Hey! Put down those rocks
>I was joking!).

Forget the rock! I've got a stack of 40meg HD's that we use as door stops ;)

Actually, if you want to do downloads the best standard method around
that we use every day to download code to DSP's, embedded processors,
etc is Motorola S-Record format. It has the advantages that
- It is standardized in format
- Text/HEX printable based for dealing with 7-bit transmission
- Internal addresses/continuation/etc
- Internal checksums
- Standard line lengths (not to exceed 80 chars incl cr/lf)
- It can be decoded/cached in the receiving process and when the
   S7/S9 record format arrives it can be validated and d'lded as
   one piece.
- Most compilers can generate it.

My 2 cents.

Jay

Re: [LHA-misc] Line parsing routine

[Neil C. <nc...@ho...>]

Paul Fox wrote:
> 
>  >
>  > Without the size of the file we then need to change the way we download
>  > the file to the Ocelot (which might not be a bad thing). Here is an
>  > example:
>  >
>  > set addr=Ocelot:0.0.0 value=5 value=^@^M^J^Z^D
> 
> i feel like i'm starting to contribute to something i don't know enough
> about, since i confess i haven't been following development closely.  but
> if the protocol has been chosen as an ascii protocol for ease of debugging
> (which is a big advantage of the smtp/ftp style of handshake) then i really
> think you shouldn't be sending binary data as binary.  there are lots of
> good encoding schemes to choose from.  remember -- you're going to want
> to test this from telnet session.  and what you'll have in front of
> you is a paper document from a device mfgr that gives hex codes.  very
> convenient to be able to type 'value=\00\0d\0a\5a\04".

You seem to be keeping up with everything pretty well. It was a nice point
to note that if we needed compression on the fly that the file size
would hinder things. Because of my networking background I tend to think
that compression is best left to storage. If we compress data over a
dialup we lose bandwidth because most modems use compression to get the
extra bandwidth. From the tests I've done, I've seen the compressed file
grow to 1.5 times the compressed size.

I'll be honest, my thinking slipped. I want the hex-ASCII (hey what is
correct name for that) for easy debugging. It works well and isn't much of
a resource hog, although it does eat a little bit of bandwidth. But like
Microsoft, I think bandwidth is infinite :-) (Hey! Put down those rocks
I was joking!).

-- 
Linux Home Automation           Neil Cherry             nc...@ho...
http://members.home.net/ncherry                         (Text only)
http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
http://linuxha.sourceforge.net/				(SourceForge)

Re: [LHA-misc] Line parsing routine

[Paul F. <pg...@fo...>]

 > 
 > Without the size of the file we then need to change the way we download
 > the file to the Ocelot (which might not be a bad thing). Here is an
 > example:
 > 
 > set addr=Ocelot:0.0.0 value=5 value=^@^M^J^Z^D

i feel like i'm starting to contribute to something i don't know enough
about, since i confess i haven't been following development closely.  but
if the protocol has been chosen as an ascii protocol for ease of debugging
(which is a big advantage of the smtp/ftp style of handshake) then i really
think you shouldn't be sending binary data as binary.  there are lots of
good encoding schemes to choose from.  remember -- you're going to want
to test this from telnet session.  and what you'll have in front of
you is a paper document from a device mfgr that gives hex codes.  very
convenient to be able to type 'value=\00\0d\0a\5a\04".

i see your point about the count, though.  i actually think your
"continue"/"end" suggestion is a good one.  even if not implemented
immediately, it would be a good placeholder in the protocol spec.

 > set addr=<> continue=<><CR><LF>
 > set addr=<> continue=<><CR><LF>
 > set addr=<> continue=<><CR><LF>
 > set addr=<> continue=<><CR><LF>
 > set addr=<> end=<><CR><LF>
 > 

paul
=---------------------
  paul fox, pg...@fo... (arlington, ma, where it's 30.4 degrees)

Re: [LHA-misc] Line parsing routine

[Neil C. <nc...@ho...>]

Paul Fox wrote:
> 
>  > >  > we've planed for. Also can we have more than 1 value? Say we d/l some
>  > >  > code to the Ocelot, we can first send the size of the file (make
>  > >  > reserving memory easier) then the actual file, which can we a stream
>  > >  > of 8 bit characters (no null or <CR> to worry about or escape).
>  > >
>  > > this is a good idea.  but don't require sending the size of the file
>  > > first.  or at least make it device-specific.  it's hard to do
>  > > on-the-fly compression if you need to know the final size up front.
>  >
>  > Why would we need to do on the fly compression? Please don't take this
>  > as a negative comment. I do not see a reason for it, it could be stored
>  > commpressed (which in my opinion would be better).
> 
> well, i guess it depends what "it" is.
> 
> i just happened to hit an issue with this sort of thing at work a while
> ago, and was thinking you might not want to set limits you don't need to.

Without the size of the file we then need to change the way we download
the file to the Ocelot (which might not be a bad thing). Here is an
example:

set addr=Ocelot:0.0.0 value=5 value=^@^M^J^Z^D

The ^x characters are single characters each but they would need some
fancy way of escaping them if the size wasn't sent. We could fall back
sending them in their hex-ASCII equivalants. Such as value=000D0A1A04 .
Since we now use select with line buffering (we don't see the line until
the user hits a line ending) it's still necessary to know the size as a
32K file would be enough to clobber most input buffers (buffer overruns).

Now I know we're recoding but I kind of like the line at a time as it
writting the program a lot easier. Any ideas on how to get around said
problems? Maybe we could have a set command that has a continue and 
end command(???). So it would be:

set addr=<> continue=<><CR><LF>
set addr=<> continue=<><CR><LF>
set addr=<> continue=<><CR><LF>
set addr=<> continue=<><CR><LF>
set addr=<> end=<><CR><LF>

This seems a little kludgy.

-- 
Linux Home Automation           Neil Cherry             nc...@ho...
http://members.home.net/ncherry                         (Text only)
http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
http://linuxha.sourceforge.net/				(SourceForge)

Re: [LHA-misc] Line parsing routine

[Paul F. <pg...@fo...>]

 > >  > we've planed for. Also can we have more than 1 value? Say we d/l some
 > >  > code to the Ocelot, we can first send the size of the file (make
 > >  > reserving memory easier) then the actual file, which can we a stream
 > >  > of 8 bit characters (no null or <CR> to worry about or escape).
 > > 
 > > this is a good idea.  but don't require sending the size of the file
 > > first.  or at least make it device-specific.  it's hard to do
 > > on-the-fly compression if you need to know the final size up front.
 > 
 > Why would we need to do on the fly compression? Please don't take this
 > as a negative comment. I do not see a reason for it, it could be stored
 > commpressed (which in my opinion would be better).

well, i guess it depends what "it" is.

i just happened to hit an issue with this sort of thing at work a while
ago, and was thinking you might not want to set limits you don't need to.

paul
=---------------------
  paul fox, pg...@fo... (arlington, ma, where it's 36.1 degrees)

Re: [LHA-misc] Line parsing routine

[Neil C. <nc...@ho...>]

Paul Fox wrote:

Pardon the delayed response, I'm bouncing as usual. :-)

>  > I wonder can we drop the addr= and value= ? If they're always in the
>  > same position we can drop them. But if the addr= can be replaced with
>  > something else then we'll need them (is this too short sighted?). Also
> 
> while it _can_ be typed by a user, i don't see this command set as
> being something anyone would _want_ to type.  given that there will
> probably always be some sort of UI in front of this, then requiring
> the addr= and value= keywords is probably the right thing to do.  the
> ambiguity you'll eliminate if you someday, say, decide to add "name=...",
> will make it worth it.

Actually the Agent (daemon) is not meant to be interfaced by the end
user. They're supposed to interface to the server, the Agent is supposed
to handle the communication between the servers and the device (many to
one). The reason for the ASCII interface and the commands are so the
programmer has an easier time of debugging. Having structured commands
(and int a certain order) should be an added burden to the programmer.

>  > we've planed for. Also can we have more than 1 value? Say we d/l some
>  > code to the Ocelot, we can first send the size of the file (make
>  > reserving memory easier) then the actual file, which can we a stream
>  > of 8 bit characters (no null or <CR> to worry about or escape).
> 
> this is a good idea.  but don't require sending the size of the file
> first.  or at least make it device-specific.  it's hard to do
> on-the-fly compression if you need to know the final size up front.

Why would we need to do on the fly compression? Please don't take this
as a negative comment. I do not see a reason for it, it could be stored
commpressed (which in my opinion would be better).

-- 
Linux Home Automation           Neil Cherry             nc...@ho...
http://members.home.net/ncherry                         (Text only)
http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
http://linuxha.sourceforge.net/				(SourceForge)

Re: [LHA-misc] Line parsing routine

[Jay H. <jh...@fa...>]

At 10:24 AM 11/29/00 -0500, you wrote:
> > >Just to clearify, before I implement, are the USER/PASS/QUIT optional if a
> > >agent does not want to handle logins? If using logins, I imagine it should
> > >actually validate the login on the host machine's users (ie: use the passwd
> > >file under linux)? Kind of pain for a simple agent. As far as security, it is
> > >probably better not to let the agent port past the firewall.
> > 
> > I don't see a problem not implementing it. Just need an error message
> > for 'not implemented' and make sure the server side understands (and
>
>wouldn't it be better to return success than failure?

Sorry. The "read between the lines" is "return a success message
that indicates not implemented."


>regarding the "addr=" and "value=" thread:
>
> > - It maintains uniformity across everything for parsing/building
> >     messages because nothing except the initial keyword is positional.
>
>i don't recall your document mentioning it -- but i think that point should
>be made explicit if it hasn't been already.  it's an important thing
>for an implementor to remember.

Good point. When I did the first pass I was concerned more with the
protocol, addressing, and exact functionality.

Since it appears the document is going to live you are very correct
that I need to get all the "implied" stuff in place.

My todo list (in order):
- Sample XML of the sub-device configuration (and virtual devices)
- Update Agent-1.0 protocol doc for this and other "implied" stuff
- Put both up for initial comments (probably my web page) and get
    a rough review then let Neil post the stuff to sourceforge as
    a working doc if he wants.

Jay

Re: [LHA-misc] Line parsing routine

[Paul F. <pg...@fo...>]

 > >Just to clearify, before I implement, are the USER/PASS/QUIT optional if a
 > >agent does not want to handle logins? If using logins, I imagine it should
 > >actually validate the login on the host machine's users (ie: use the passwd
 > >file under linux)? Kind of pain for a simple agent. As far as security, it is
 > >probably better not to let the agent port past the firewall.
 > 
 > I don't see a problem not implementing it. Just need an error message
 > for 'not implemented' and make sure the server side understands (and

wouldn't it be better to return success than failure?

regarding the "addr=" and "value=" thread:

 > - It maintains uniformity across everything for parsing/building
 >     messages because nothing except the initial keyword is positional.

i don't recall your document mentioning it -- but i think that point should
be made explicit if it hasn't been already.  it's an important thing
for an implementor to remember.

paul
=---------------------
  paul fox, pg...@fo... (arlington, ma, where it's 40.6 degrees)

Re: [LHA-misc] Line parsing routine

[Jay H. <jh...@fa...>]

At 07:02 AM 11/29/00 -0500, you wrote:
>Ok Jay, This is great stuff.
>
>Just to clearify, before I implement, are the USER/PASS/QUIT optional if a
>agent does not want to handle logins? If using logins, I imagine it should
>actually validate the login on the host machine's users (ie: use the passwd
>file under linux)? Kind of pain for a simple agent. As far as security, it is
>probably better not to let the agent port past the firewall.

I don't see a problem not implementing it. Just need an error message
for 'not implemented' and make sure the server side understands (and
teach people ctrl-]/close for getting out of telenet )


>I assume what you call Agents is what Neil is calling Deamons (maybe that's
>where I'm mixed up)?

An Agent is a type of Daemon - On the Daemon side you have some that talk
to physcial interfaces (I'm calling them Agents), some that gather and
manage info from Agents (loosely called a "server" or I think Neil has
also used 'lhad'), and some that use the info from the server to make
decisions and send requests back to the server (that then forwards them
back to the agents).


>On your OSI Model I always envisioned the UI always talking directly to the
>server. The server in turn talks to the individual agents. This way all the
>communications goes through the server and the server can track the current
>state of the devices.

I do agree with this model and it works very well for GUI/Web type
devices. It also works pretty well with the Leopard because it is
well defined how to handle screen changes, button presses are events,
etc.

I run into a problem that grays the lines on something like a NetMedia
LCD+ that has a keypad, 4x20 LCD, and 8 inputs. It is a physical
hardware-attached device with a protocol and has inputs. It fits the
definition of what an Agent would control.
- How do you send it lines to display?
- It needs interactive data from other "agents"
- Keypresses aren't mapped to a specific event like the Leopard
In short, it needs processing. It needs to feed the "server" like
an agent so everything can see the inputs but the program processing
it needs visibility to everything to.
On possibility is to attempt to define a set of commands for UI's
so generic stuff could be handled but at the moment I can't wrap
my arms around the definition of what will be on the other end for
UI's where the "raw input/output" side I could grasp at the moment.

I know I didn't answer your question but I hope you see the problem
I ran into. It also makes the "server" a little more complex at the
moment...


>One minor gripe, the SET command's (and 8000 message) syntax with ADDR= and
>VALUE= is a bit verbose (why can't it just be {noun} {verb} syntac like "SET
>0.x10.A1 ON"? Not a big deal though.

Thought about this but experience says go with the more verbose:
- Remember it is primarily programs using this, not humans.
- Tagged values make for forward/backward compatibility and allow the
    'SET' command to do more stuff later:
    'SET ADDR="..." Value="On" AT="9:30" length=":30"' to a *server*
- The tag method is 98% to XML/SOAP where I think this will end up:
    <SET><ADDR>...</ADDR><VALUE>ON</VALUE></SET>
- It maintains uniformity across everything for parsing/building
    messages because nothing except the initial keyword is positional.
    Going to XML nothing is positional (unless you count nesting
    as positional)

>Any chance of getting this turned into a working document on Sourceforge?

I don't have a problem with it - it is still on the rough side and I
don't know how to do much on sourceforge yet (Neil?).

There is also key stuff missing to make all this work like...
"Upon issuing a SET command, when confirmation is received that the
command has been executed or the command is executed in an environment
without confirmation, an 8XXX event will be issued to notify all
listeners of the change in state."

This enables you to hop on the alarm agent, set the alarm, then the
event will get fed back to the server so it can do all the HVAC
setbacks, turn off the TV's, lights, ...  and we don't have to
worry about things getting out-of-sync. It also lets somebody run
multiple "servers" of different flavors for testing/evaluating/...

>Guess this interface is called AGENT-1.0??

Had to come up with some sort of identification.


>Also, I noticed you talk about NAPCO 9600 (I have one too), do you have an
>Agent running? What do you have that speaks this language?

I have the basics of an agent working, capturing info, and the address/
domain structure laid out but not fully implemented. Do you have the HA
ROM so you get a full time event feed?

DISCLAIMER: Remember these are my opinions and don't represent the
thoughts of anybody but myself. This is Neil's project and we have
chatted about a lot of this but it does not represent his opinions.

Jay


>Thanks,
>H. Gunner
>
>
>
>
>
>In a message dated 11/28/00 12:51:05 PM Eastern Standard Time,
>jh...@fa... writes:
>
> > Not to get out-of-sync with the last posting (in response to another
> >  request) you have some good points (Q's) in here that I think need to
> >  be addressed:
> >
> >  At 12:14 AM 11/28/00 -0500, Neil Cherry wrote:
> >  >I've looked over what you've posted so far and it looks good. Now I have
> >  >few questions.
> >  >
> >  > >What I've got implemented as far as commands are:
> >  > >USER <user id>
> >  > >PASS <password>
> >  >
> >  >The login/password part is difficult, how does one let an LCD login?
> >  >We can allow a default/guest id that is active when they connect. This
> >  >would allow monitors to do their work.
> >
> >  LCD's/UI's fall in an interesting area (at least the way I think
> >  about them) because (1) they generate events and have a proprietary
> >  control interface (be it web, lcd, leopard, ...) but (2) they make use
> >  of information from other "agents" for display purposes.
> >
> >  I see 3 ways to implement them. I have an LCD+ sitting here, still
> >  waiting for an LCD-80 to appear (not holding my breath) so I am not
> >  to biased towards either of these methods but the nature of the Leopard
> >  being a primary agent interface (ADICON) and UI (or multiple UI) almost
> >  forces both methods be available.
> >
> >  OSI Model (selected pieces):
> >  ~3 - Agents
> >  ~4 - Server that dispenses events and plays "traffic cop"
> >  ~6 - UI handling
> >  ~7 - Actual interface (web, ...)
> >
>_______________________________________________
>http://lists.sourceforge.net/mailman/listinfo/linuxha-misc
>To unsubscribe, send "unsubscribe Linuxha-misc"
>in the body of a message to  Lin...@li...

Re: [LHA-misc] Sample Commands (was Line parsing routine)

[Jay H. <jh...@fa...>]

At 12:03 AM 11/29/00 -0500, you wrote:
>Jay Hogg wrote:
>
> > The same *exact* format carries to the ADICON RS-485 direct
> > devices except that address 0 is actually on the network and
> > not the Ocelot... or if they ever get the Ocelot to fully support
> > RS-485 access then you could have a Leopard or Ocelot at an
> > address on the ADICON RS-485 bus and all the X10/IR/KEYTOUCH
> > domains are valid.
> >
> > >  Also I noticed that each
> > >device has a seperate port are these seperate agents (daemons)?
> >
> > Yes. They can all be on one machine, separate machines, on
> > remote networks through firewalls...  It also makes adding/
> > changing or creating brand new interfaces real easy...
>
>Good, that is very clear now, the ADICON device threw me, I had
>considered it a subdevice of the Ocelot. I see that it is not in this
>case.
>
>Have you given any thoughts to the subdevices being described in XML?
>It would probably cause some difficulty but it is what XML is made to
>handle.

Yes but it is dated and needs to be revised. I did it before I had
gone to address.domain.point so domain was a separate tag. It was a
mess and is what forced a rethink of addressing.

That level obviously gets into nesting of "points" under an interface
and provides a "global symbolic name" to "interface:address.domain.point"
xref along with the all-important stuff like capabilities (on/off/abs/
rel/in/out), scaling, and some "nifty" things like "momentary" and
how long for things like garage door openers.

It is designed to be read by both the "server" and the "agent" (along
with interface.conf). Each uses the information that it cares about.

I'll try to get my stuff updated tonight to at least something
reasonable. I haven't done a DTD for XML yet...


>BTW, I purchased a book on XML "Teach Yourself XML" by Sandra E.Eddy &
>John E. Schnyder (IDG books). It was the only book on the shelves (that
>I could find) on XML. I also found a student guide on XML from some
>classes given in the building I'm in (at work).

I've got an XML book to but haven't found anything that I'm overly
impressed with either in print or online.


> > Always fun. I'm having to learn IE messages on ISDN PRI at the moment
> > and doing some advance work on G729 channel banks over ATM without
> > the hardware in hand. I can relate fully :)
>
>AH! I actually understand all of that! Hey how about ATM over BiSync over
>IP over frame over ATM (the first ATM being Automated Teller Machines at
>1200 :-).

You've got me beat. And you've got my sympathy ;)  I'm currently looking
at stuff the IP/Frame/ATM arena, but the closest I got was IBM Cash
registers via BiSync to System/34/36/38 18 years ago.  It was always
fun writing protocol level stuff on a machine designed only for report
generation (the screen was basically an interactive report).

Jay


>--
>Linux Home Automation           Neil Cherry             nc...@ho...
>http://members.home.net/ncherry                         (Text only)
>http://meltingpot.fortunecity.com/lightsey/52           (Graphics)
>http://linuxha.sourceforge.net/                         (SourceForge)
>_______________________________________________
>http://lists.sourceforge.net/mailman/listinfo/linuxha-misc
>To unsubscribe, send "unsubscribe Linuxha-misc"
>in the body of a message to  Lin...@li...

Re: [LHA-misc] Line parsing routine

[<HGu...@ao...>]

Ok Jay, This is great stuff.

Just to clearify, before I implement, are the USER/PASS/QUIT optional if a 
agent does not want to handle logins? If using logins, I imagine it should 
actually validate the login on the host machine's users (ie: use the passwd 
file under linux)? Kind of pain for a simple agent. As far as security, it is 
probably better not to let the agent port past the firewall.

I assume what you call Agents is what Neil is calling Deamons (maybe that's 
where I'm mixed up)?

On your OSI Model I always envisioned the UI always talking directly to the 
server. The server in turn talks to the individual agents. This way all the 
communications goes through the server and the server can track the current 
state of the devices.

One minor gripe, the SET command's (and 8000 message) syntax with ADDR= and 
VALUE= is a bit verbose (why can't it just be {noun} {verb} syntac like "SET 
0.x10.A1 ON"? Not a big deal though.

Any chance of getting this turned into a working document on Sourceforge? 
Guess this interface is called AGENT-1.0??

Also, I noticed you talk about NAPCO 9600 (I have one too), do you have an 
Agent running? What do you have that speaks this language?

Thanks,
H. Gunner





In a message dated 11/28/00 12:51:05 PM Eastern Standard Time, 
jh...@fa... writes:

> Not to get out-of-sync with the last posting (in response to another
>  request) you have some good points (Q's) in here that I think need to
>  be addressed:
>  
>  At 12:14 AM 11/28/00 -0500, Neil Cherry wrote:
>  >I've looked over what you've posted so far and it looks good. Now I have
>  >few questions.
>  >
>  > >What I've got implemented as far as commands are:
>  > >USER <user id>
>  > >PASS <password>
>  >
>  >The login/password part is difficult, how does one let an LCD login?
>  >We can allow a default/guest id that is active when they connect. This
>  >would allow monitors to do their work.
>  
>  LCD's/UI's fall in an interesting area (at least the way I think
>  about them) because (1) they generate events and have a proprietary
>  control interface (be it web, lcd, leopard, ...) but (2) they make use
>  of information from other "agents" for display purposes.
>  
>  I see 3 ways to implement them. I have an LCD+ sitting here, still
>  waiting for an LCD-80 to appear (not holding my breath) so I am not
>  to biased towards either of these methods but the nature of the Leopard
>  being a primary agent interface (ADICON) and UI (or multiple UI) almost
>  forces both methods be available.
>  
>  OSI Model (selected pieces):
>  ~3 - Agents
>  ~4 - Server that dispenses events and plays "traffic cop"
>  ~6 - UI handling
>  ~7 - Actual interface (web, ...)
>