hamlib-stationserver — Discussion of station server a possible successor to rigctld/rotctld

Re: [Hamlib-stationserver] Philosophy: Abstraction vs Rig Interface Artifacts

[Art B. <ac...@in...>]

Thanks, Nate. 

If I may make a general observation:  There are two common perspectives on a project like this, in my experience.  They're both valid and they're so similar that sometimes folks miss the subtle difference between them and wind up talking past each other.  I call those two approaches the "processor-centric" and the "network-centric" views.

The processor-centric view, as the name suggests, puts a computer in the middle of the mental model.  This is typically how software folks view problems.  The term "API" is a often a signature of the processor-centric perspective.

Telecommunications folks, OTOH, tend toward the network-centric view.  They talk about "messages," "protocols" and "exchanges" and tend to see the wire (or the cloud) as the center of things and the processors as nodes... leaves on the network tree... and largely treat them as black boxes with (hopefully) standardized interfaces.

Again, both views are valid in their own domains.  The trick I've had to learn is to recognize which perspective is being use at any point in time.  Although I've written a fair bit of code over the years I'm a comms guy at heart and thus my personal dominant mode is net-centric.  

That said...

On Mar 3, 2014, at 7:13 PM, Nate Bargmann <n0...@n0...> wrote:
> Perhaps one question would be, how should a radio "look" to a calling
> application?  By that I mean that it may not "look" much like a radio at
> all.

From a net-centric view I'm thinking it should look like an instance of an abstract, Platonic-ideal model of a radio... composed of building-block components... transmitters, receivers, filters and so on... that can be assembled to describe any particular device.

The "server" is a gateway between the IP network environment and the digital controls on a radio... and a "client" is a gateway between a human (or non-human) user's inputs and the IP network that's compatible with the server's capabilities.  Like a radio each is made up of internal components that are (again hopefully) highly modular.

And between those components we have a series of interfaces:

    USER <=> User Interface <=> CLIENT <=> Network Interface <=> SERVER <=> Device Interface (e.g., CAT) <=> RADIO

We achieve interoperability by eliminating unique or instance-specific features of the shared network interface... but at the edge interfaces (the client UI and the radio CAT connection) we can still accommodate diversity.

Which means that client user interfaces could, at least in principle, be customized to fit whatever paradigm the user knows or prefers.  E.g. a client could offer a RIT widget that looks and works just like RIT traditionally does... without necessarily implementing a literal "RIT" command on the network interface or even using that literal feature on the device interface.

In other words, taking the network interface to a higher level of abstraction doesn't mean not giving users an interface they understand.  It just makes interoperability a lot easier behind the curtain and on the wire.  And it gives more freedom to adapt as technology and user perspectives evolve over time.

My thought at this point, therefore, is that in the near term we'll be designing a network "facade" interface to the existing hamlib virtual rig and rig-specific backends, enabling a variety of stationserver-compatible clients to offer different UIs to different users and for different applications.

Once we've done that it may make sense to work back into the hamlib codebase and see if that can be streamlined or otherwise improved.  (I'm guessing that by taking a less literal approach to the individual rig interfaces it may be possible to simplify it.)  I'd expect that the individual rig interfaces would be the last thing to refactor, except in cases where there are serious shortcomings or, of course, when there are new radios to support!


- Art

Re: [Hamlib-stationserver] Philosophy: Abstraction vs Rig Interface Artifacts

[Nate B. <n0...@n0...>]

* On 2014 03 Mar 16:54 -0600, Art Botterell wrote:
> Started some very preliminary doodle of the object models, and looking
> at the rigctld man page it occured to me that a number of the commands
> are really pass-throughs of legacy UI mechanisms.  
> 
> E.g., there are a variety of options for RIT (and XIT), split and
> repeater shift... all of which are really just ways of making the
> transmit and receive frequencies different according to various rig
> interfaces and technology legacies. 

There is a, as we call it, a "virtual rig" sort of abstraction to the
Hamlib C language API, although this is poorly reflected by
rigctld/rotctld (from now on I'll refer to them as the "control
daemons") which are essentially a "flat" command namespace.
Unfortunately, there is no document available that outlines this
abstraction.  Each programmer using Hamlib's C API has had to figure it
out, often by reading the header and or source files.  Certainly, any
willing backend contributor has had to do the same.  This has greatly
limited Hamlib's currency with new rigs.

Also, Hamlib's virtual rig only supports three VFOs, A-C, for example,
which are certainly far too limiting for the modern rigs coming down the
pike.  Originally, the goal was to support all features of available
rigs in the virtual rig interface.  In 2000 when the project went public
this probably seemed like an attainable goal.  Today, not so much as it
is just not practical, IMO, to try and include the unique features of
every model.

> Seems like it might be cleaner to abstract all that out and simply
> model each receiver and each transmitter as having independent
> frequency and other settings that might or might not happen to be the
> same at any point in time.
> 
> Which might require adding some rig-specific shims to order up the
> required configuration in language each particular rig understands...
> and of course, familiar mechanisms such as RIT, split or shift could
> be reproduced in client UIs as folks desire.  But I'm thinking what
> we're really after is a universal abstraction layer... or at least as
> universal as we can make it... rather than just passing through every
> CAT command there might be.
> 
> However I suspect this isn't a new dilemma, so I'd appreciate any
> background or insight anyone can offer.

Perhaps one question would be, how should a radio "look" to a calling
application?  By that I mean that it may not "look" much like a radio at
all.

My reading of Hamlib over the years has led me to understand that its
API tended to mirror the way we humans approach a radio.  We have knobs
and buttons and the API reflects that through C functions that mimic our
interaction with the actual radio.  At first glance it makes sense, but
does it make any more sense than the earliest GUI programs that were often
patterned after a generic transceiver front panel?  Quite likely not.  A
different paradigm now exists for the UI for rig control so perhaps as
well, a new paradigm should be developed for the way we control it via
software.

I look forward to seeing the ideas here.

> Also, I'm not seeing a lot of support for control of various filtering
> options.  As that's been a competitive feature among rig vendors it
> may be another area where we have to decide whether to abstract or
> accept various proprietary styles.

Hamlib supports filter selection as part of setting the mode.  If a new
bandwidth is desired the mode is set with the new bandwidth.  It is up
to the backend to work out the details between the virtual rig and the
hardware.

My explanations of Hamlib's implementation are for the collective
reference and not to tilt the discussion one way or the other.

73, de Nate >>

-- 

"The optimist proclaims that we live in the best of all
possible worlds.  The pessimist fears this is true."

Ham radio, Linux, bikes, and more: http://www.n0nb.us

[Hamlib-stationserver] Sharing a radio between multiple applications

[Art B. <ac...@in...>]

Thanks, Tony, lot's to chew on, there.  Hope you don't mind if I tease out some individual threads.

For starters... I'm not sure I totally grasp what you have in mind about "sharing of a radio between multiple applications."  Do you mean multiple clients, or multiple server threads, or something else?

Could you help me with an example of what you mean?

Thanks!

- Art 

Re: [Hamlib-stationserver] Philosophy: Abstraction vs Rig Interface Artifacts

[Tony L. <vk...@gm...>]

On 4/03/2014 9:53 AM, Art Botterell wrote:
> Started some very preliminary doodle of the object models, and looking at the rigctld man page it occured to me that a number of the commands are really pass-throughs of legacy UI mechanisms.
>
> E.g., there are a variety of options for RIT (and XIT), split and repeater shift... all of which are really just ways of making the transmit and receive frequencies different according to various rig interfaces and technology legacies.
>
> Seems like it might be cleaner to abstract all that out and simply model each receiver and each transmitter as having independent frequency and other settings that might or might not happen to be the same at any point in time.
I like this Art.  Certainly for split, VFO A/B, etc, all these could be 
handled in one API.  We could try doing repeater shift the same way, and 
only use shims where necessary.
>
> Which might require adding some rig-specific shims to order up the required configuration in language each particular rig understands... and of course, familiar mechanisms such as RIT, split or shift could be reproduced in client UIs as folks desire.  But I'm thinking what we're really after is a universal abstraction layer... or at least as universal as we can make it... rather than just passing through every CAT command there might be.
Agree totally.  The only thing the clients should know about the 
connected radios is the capabilities.  For example, the client needs to 
know I can't dial up 14.200 MHz on my FT-736R, or 1296.100 on the 
IC-7000 (it should get an error if I attempt to do these things), but it 
doesn't need to know how I implement repeater shift, RIT, XIT, etc.  It 
just needs to request these functions and let the server work it out, 
since the server has the information it needs to best communicate with 
and configure the radios.
>
> However I suspect this isn't a new dilemma, so I'd appreciate any background or insight anyone can offer.
>
> Thanks!
>
> - Art
>
> Also, I'm not seeing a lot of support for control of various filtering options.  As that's been a competitive feature among rig vendors it may be another area where we have to decide whether to abstract or accept various proprietary styles.
Might need a bit of research to determine the best approach.  The 
IC-7000 brings out 3 bandwidth settings (called 1, 2 and 3, with 1 being 
the widest), but these in turn can be configured elsewhere in the 
menus.  Also need to handle errors for radios that report errors when 
trying to select nonexistent filters (or be able to configure this 
capability in the driver setup - nonexistent filters could cause 
confusion for remote base users).  I have no idea what other radios do.

I'm for abstraction where possible, and only going proprietary where 
there's no viable alternative.

-- 
73 de Tony VK3JED/VK3IRL
http://vkradio.com

Re: [Hamlib-stationserver] [with attachment] A very preliminary requirements draft for discussion

[Tony L. <vk...@gm...>]

On 4/03/2014 8:42 AM, Art Botterell wrote:
> Thanks.  We should probably pause briefly to see who accretes to the list... but in the meantime, I'd very much appreciate some help in the Background section.
>
> Without being hypercritical I'm thinking it might be useful to highlight some of the things we'd like to improve over the status quo, and that's really not my area of expertise.
Just took a quick look at the draft.  I think we're better off kicking 
it around in here before creating a document, but thanks for the stab at it.

For me, the status quo has a number of limitations:

1.  Most solutions are single platform and proprietary (Hamlib is an 
exception).

2.  There are no solutions that allow for the sharing of a radio between 
multiple applications.

3.  Because of #2, there is no API for requesting exclusive control of 
the radio and releasing it.

4.  No integration between remote base software radio control, meaning 
you can't share a remote base with local control - see #2 and #3 above 
(as well as making it easier to use your own radio, could also be a 
useful training and PR tool).

What I would like to see:

Having a networking background, I tend to think in layers, and this 
model is relevant here.

Layer 1 - Physical.  This is where the actual radio control takes 
place.  Here we define the ports and soundcard resources used, as well 
as the control protocol between the radio and server.  The physical 
layer should be defined by a "driver", which requires no code (should be 
a list of definitions that a relative non programmer can change or 
write) in a standardised format.  A tool could be written to help users 
and radio tinkerers to create their own driver files.  I'm thinking 
something like RigCAT XML files in concept (but use what is best).  The 
ability to use Hamlib drivers would also be useful to help facilitate 
rapid adoption, since many Hamlib drivers are fully functional.  Similar 
can be done for rotator control.

Layer 2 - server/abstraction.  Here the specific commands sets of the 
radios and rotators are abstracted into a generalised API.  The server 
knows (from the drivers) the capabilities of each device and is 
configured with the resources (serial/audio/network/etc) that each one 
uses.  The API is the means by which client software communicated to the 
radio(s) and rotator(s).  Space should be made in the API for other 
devices (amplifiers, remotely controlled switches, etc).  The API is 
also network aware, has minimal latency and only needs moderate 
security.  This API should NOT be used outside of a controlled 
environment such as a single machine or a LAN.  Given that it is now 
2014, the radio control API should work over both IPv4 and IPv6.  Also 
at this level comes the ability to request exclusive use of one or more 
resources, so other software accessing that resource can't inadvertently 
change settings such as VFO or mode, or rotate the antenna at an 
inappropriate time.

Layer 3 - Additional functionality.  Here we "bolt on" additional 
functionality that is not part of the core control system.  This 
additional functionality would be implemented in additional (possibly 
and most likely third party) modules which communicate with the server 
via the API.  Examples of add-on modules would include:

- Remote base.  This provides the functionality we are now familiar with 
to share your radios over the Internet.  The remote base module would 
incorporate the functions specifically needed for remote base operation, 
such as enhanced security, user authentication and validation, audio 
compression, etc.  Astute readers will notice that the remote base can 
be overridden by local access controls at the server level, in case you 
want to use your radio for something which keeping the remote base 
active (e.g. for remote listeners), but prevent it from changing any 
settings.

- Data transmission.  Modules for popular data modes could be linked to 
the server to perform the modularion and demodulation, and exchange 
baseband data and optional diagnostic information (S/N, summarised 
waterfall, etc) with a compatible data client.  Want to run PSK-31 
without a soundcard (on the client)? :)  This would potentially reduce 
the bandwidth used by data modes, especially for remote bases.

- Digital voice, D-STAR and other modes.  Modes such as D-STAR, FreeDV 
and others could be implemented as modules and effectively add to the 
command set of radios which have the appropriate capabilities (e.g. 9600 
bps capability for D-STAR).  Of course, a D-STAR module would need a 
DV-RPTR AMBE board or a DV Dongle.  The DV Dongle could be positioned at 
the server end (for sharing among multiple clients) or at the client end 
(for saving network bandwidth).  This type of add-on would effectively 
add extra mode buttons to your radios, with the magic happening behind 
the scenes. DV modules also may need to set certain radio parameters 
such as mode and 9600 bps.

- Legacy radio emulation.  It's going to take a while for developers of 
radio control software to adopt the server API.  Radio emulation 
presents a traditional (virtual) serial and soundcard interface, using a 
well known radio command set (e.g. late model Kenwood).  It may be 
possible to use existing virtual device emulators (e.g. VSPE and Virtual 
Audio Cable in the case of Windows, etc) for the sound and serial 
support, which would save development of these parts. The radio emulator 
would ideally be network aware (since it uses a network aware API), so 
it wouldn't need to reside on the same box as the server, which would 
make old client software instantly network capable.  A Hamlib module 
could also be developed to enable Hamlib capable applications to 
directly access the server API without having to go through serial port 
emulation.

- Infrastructure linking - This sort of module is related to the remote 
base module, and would allow the server to be linked to and controlled 
by other radio networks - e.g. IRLP, Echolink, D-STAR, etc.  For 
example, creating an IRLP remote base, which would require the module to 
interface with an IRLP node for audio and accept commands generated by 
scripts for the control side.  An Echolink remote base could accept 
commands via a text window.  The infrastructure linking could be 
delevoped in conjunction with other open source projects such as 
thelinkbox (but not share code, otherwise you'll end up being GPL, which 
we don't want at this stage).

Anyway, that's a few of my thoughts.  Ad this stage a wish list, but 
hopefully to give some idea.  For priorities, I'd say that layers 1 and 
2 are the priorities, as well as the radio emulator, as these are the 
things needed to make the system work, as well as encourage users of 
existing software to adopt the system.

-- 
73 de Tony VK3JED/VK3IRL
http://vkradio.com

[Hamlib-stationserver] Philosophy: Abstraction vs Rig Interface Artifacts

[Art B. <ac...@in...>]

Started some very preliminary doodle of the object models, and looking at the rigctld man page it occured to me that a number of the commands are really pass-throughs of legacy UI mechanisms.  

E.g., there are a variety of options for RIT (and XIT), split and repeater shift... all of which are really just ways of making the transmit and receive frequencies different according to various rig interfaces and technology legacies. 

Seems like it might be cleaner to abstract all that out and simply model each receiver and each transmitter as having independent frequency and other settings that might or might not happen to be the same at any point in time.  

Which might require adding some rig-specific shims to order up the required configuration in language each particular rig understands... and of course, familiar mechanisms such as RIT, split or shift could be reproduced in client UIs as folks desire.  But I'm thinking what we're really after is a universal abstraction layer... or at least as universal as we can make it... rather than just passing through every CAT command there might be.

However I suspect this isn't a new dilemma, so I'd appreciate any background or insight anyone can offer.

Thanks!

- Art

Also, I'm not seeing a lot of support for control of various filtering options.  As that's been a competitive feature among rig vendors it may be another area where we have to decide whether to abstract or accept various proprietary styles.

Re: [Hamlib-stationserver] [with attachment] A very preliminary requirements draft for discussion

[Tony L. <vk...@gm...>]

On 4/03/2014 8:42 AM, Art Botterell wrote:
> Thanks.  We should probably pause briefly to see who accretes to the list... but in the meantime, I'd very much appreciate some help in the Background section.
>
> Without being hypercritical I'm thinking it might be useful to highlight some of the things we'd like to improve over the status quo, and that's really not my area of expertise.
I can help with this big picture/wishlist stuff Art, since I'm one of 
those who raised this. :)  I've only just joined the list, so have 
missed the discussion.  Will take a peek in the archives.

-- 
73 de Tony VK3JED/VK3IRL
http://vkradio.com

Re: [Hamlib-stationserver] [with attachment] A very preliminary requirements draft for discussion

[Art B. <ac...@in...>]

Thanks.  We should probably pause briefly to see who accretes to the list... but in the meantime, I'd very much appreciate some help in the Background section.  

Without being hypercritical I'm thinking it might be useful to highlight some of the things we'd like to improve over the status quo, and that's really not my area of expertise.

- Art


On Mar 3, 2014, at 1:39 PM, Nate Bargmann <n0...@n0...> wrote:

> I've uploaded the document to:
> 
> https://sourceforge.net/projects/hamlib/files/stationserver/documents/requirements/
> 
> 73, de Nate >>
> 
> -- 
> 
> "The optimist proclaims that we live in the best of all
> possible worlds.  The pessimist fears this is true."
> 
> Ham radio, Linux, bikes, and more: http://www.n0nb.us
> 
> ------------------------------------------------------------------------------
> Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
> With Perforce, you get hassle-free workflows. Merge that actually works. 
> Faster operations. Version large binaries.  Built-in WAN optimization and the
> freedom to use Git, Perforce or both. Make the move to Perforce.
> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
> _______________________________________________
> Hamlib-stationserver mailing list
> Ham...@li...
> https://lists.sourceforge.net/lists/listinfo/hamlib-stationserver

Re: [Hamlib-stationserver] [with attachment] A very preliminary requirements draft for discussion

[Nate B. <n0...@n0...>]

I've uploaded the document to:

https://sourceforge.net/projects/hamlib/files/stationserver/documents/requirements/

73, de Nate >>

-- 

"The optimist proclaims that we live in the best of all
possible worlds.  The pessimist fears this is true."

Ham radio, Linux, bikes, and more: http://www.n0nb.us

Re: [Hamlib-stationserver] A very preliminary requirements draft for discussion

[Nate B. <n0...@n0...>]

* On 2014 03 Mar 15:14 -0600, Art Botterell wrote:
> Friends -
> 
> Here's a very rudimentary requirement document, mainly to illustrate the form.  
> 
> For now I'm hoping the list is configured to allow attachments.  However we probably should start a file repository ASAP.  Nate, can that be done on SourceForge?

I've corrected the attachment limit and queued your document to the
list.

File hosting is not quite in the manner that you're used to in the
Yahoo! groups.  There is a file section where we can upload updates and
eventually source code releases.

I've just created a dedicated directory for Station Server:

https://sourceforge.net/projects/hamlib/files/stationserver/

More directories can be created under that directory to organize things
such as spec documents and source code releases.

73, de Nate >>

-- 

"The optimist proclaims that we live in the best of all
possible worlds.  The pessimist fears this is true."

Ham radio, Linux, bikes, and more: http://www.n0nb.us

[Hamlib-stationserver] [with attachment] A very preliminary requirements draft for discussion

[Art B. <ac...@in...>]

Friends -

Here's a very rudimentary requirement document, mainly to illustrate the form.  

For now I'm hoping the list is configured to allow attachments.  However we probably should start a file repository ASAP.  Nate, can that be done on SourceForge?

I've formatted this as a document so we can assert the CC license right from the start.  And I've made it PDF deliberately in order to discourage direct editing and the sort of confusion that can result from premature "redlining."  

Instead, my recommendation would be that we discuss needed changes point-by-point in email and then roll our conclusions into an updated document occasionally (say, weekly, or maybe even faster at first.)  

That way we can control crosstalk among distinct issues and keep a record in the archive not just of the changes but also of the rationale for them.  That will become important later as new folks join the conversation, as they'll be able to review prior debate offline and (hopefully) only add new perspectives rather than going over old ground.

Of course we don't need to get the requirements anywhere near perfect, IMHO, and we'll surely revisit them from time to time, but I've found it's really helpful to have a touchstone to which we can refer to make sure we're still draining the same swamp.

73, and big ups to Nate for bringing this together!

- Art

[Hamlib-stationserver] A very preliminary requirements draft for discussion

[Art B. <ac...@in...>]

Friends -

Here's a very rudimentary requirement document, mainly to illustrate the form.  

For now I'm hoping the list is configured to allow attachments.  However we probably should start a file repository ASAP.  Nate, can that be done on SourceForge?

I've formatted this as a document so we can assert the CC license right from the start.  And I've made it PDF deliberately in order to discourage direct editing and the sort of confusion that can result from premature "redlining."  

Instead, my recommendation would be that we discuss needed changes point-by-point in email and then roll our conclusions into an updated document occasionally (say, weekly, or maybe even faster at first.)  

That way we can control crosstalk among distinct issues and keep a record in the archive not just of the changes but also of the rationale for them.  That will become important later as new folks join the conversation, as they'll be able to review prior debate offline and (hopefully) only add new perspectives rather than going over old ground.

Of course we don't need to get the requirements anywhere near perfect, IMHO, and we'll surely revisit them from time to time, but I've found it's really helpful to have a touchstone to which we can refer to make sure we're still draining the same swamp.

73, and big ups to Nate for bringing this together!

- Art