hamlib-developer

[Hamlib-developer] Re: gnuradio and Hamlib

[Matt E. <ma...@et...>]

>
> Well, not really. Hamlib can be seen as a utility library. It does not
> need to control the main loop. However, the user application linked
> against Hamlib may want to control it, e.g. grig or any GUI.
> That's why it would be nice if the gnuradio backend would not require
> to grab the main loop.
> And this is possible, provided gnuradio library is compiled with -DTHREAD.

....
> > > * questions: is gnuradio pthread-safe (I've seen -D_REENTRANT, what about
> > >   the code itself)? does gnuradio use pthread by itself in constructors?
> > >   It looks like yes (VrMultiTask), but it is working fine?
> >
> > some of it is thread-safe.  It should all be, eventually.
>
> Good.
> IMHO, we should use -DTHREAD for gnuradio, this would simply remove the
> need to call m->process(), no more main loop, and everyone is happy.

This is way more complex than you think.  Several Master's and PhD theses were
written on the scheduling and threading in GNURadio's progenitor,
Spectra/pspectra.  It was designed to be multithreaded, but certain operations
are single-threaded only, like the scheduler.  Only actual processing is
multithreaded.  Additionally, many of the processing blocks must run in order on
the data (like IIR filters), but not all (FIR filters do not suffer from this). 
At this point, the threading is disabled so there is one less thing to worry
about.

I guess the main point is that gnuradio is many things, to many people :)  We
don't want to change gnuradio internals to be able to use hamlib in one
gnuradio-based app.  Conversely, I wouldn't want you to change hamlib's
internals to support gnuradio.

> > I'd like to be able to [easily] have my gnuradio-based app receive commands
> > from, and return status to a hamlib-based frontend program, like GRig.  Basic
> > stuff like change frequency, volume, mode, etc.
>
> Alright, seems pretty feasible. Since you're going to be first "user",
> what is your FI source ? Is it the MC4020+microtune_eval_board ?

I'm not sure what FI stands for, but yes, I have the microtune and MC4020 combo.
 That would be a great demo.

> I understand that you will most probably not use only this source,
> and the backend will serve as a lab rat, but Hamlib needs at least
> to release some working skeleton. So which ever is fine. Then we'll
> be able to clone this "model" (i.e. source) to other models
> within the backend.
> If this terminology sounds obscure to you, don't worry, as soon as code
> will show up, this will make sense.

ok.  And yes, it does sound obscure to me :)

> Note: there's missing some checking against fftw existence in configure.in

Thanks.  i'll check into that.

> Okay, I'll try to put it another way.
> Imagine you have everything in binary form, the user application, the Hamlib
> (frontend + backend), the gnuradio library. Now, someone develops
> some code to support a new FI source. Does gnuradio has an abstraction
> interface (at binary level) to support this new FI by just dropping the new
> "lib" with recompiling everything.

[I'm assuming FI means some sort of IO device, like the MC4020 here...]  That's
a tough one.  I'm more of a DSP and comms guy, so I don't know the intricacies
of shared libraries, linking, plugins, etc.  I'm pretty sure it hasn't been done
yet, but I don't know how hard it would be.  There's not much motivation,
though, since the source code is free.

>
> Hmm. Please correct me if I'm wrong. IOW, you're not gonna use Grig right
> out-of-the-box. Your idea/need is to take Grig as a base, and hack it
> specifically for gnuradio extension, in such a way that may not work
> with a stock GRig. Right?
> (I'm a bit exagerating, but this is to better understand what interfaces
> will be needed in the backend).

My intention was that GRig could be used unmodified.  I'd like the user not to
be able to tell whether they are using a ft-1000 or gnuradio-based SDR.


Thanks again,
Matt

P.S.  If anyone is interested, I will be giving a talk and demos for gnuradio at
TAPR's DCC this year.  See http://www.tapr.org

[Hamlib-developer] Re: [Discuss-gnuradio] gnuradio and Hamlib

[Matt E. <ma...@et...>]

Just to clarify a little....

GNURadio is a framework, not really a particular application.  GNUradio is not
just about ham radio done in software.

My goal was to avoid reinventing the wheel (and to avoid writing GUI code...). 
What I was hoping to create was a way for an application to use both GNURadio
and Hamlib.  Hamlib would allow the reuse of user interface frontends, like
GRig.  GNURadio would perform the radio stuff.

The main difficulty, as I see it, is that both environments want to control the
main loop.

Stephane Fillod <f4...@fr...> said:
> Preliminary note: for me, everything that comes from gnuradio tar ball is
> "gnuradio". I don't know enough about it to distinguish between libs and
> stuff yet.

Stuff in the src/gnu/examples directory are basic apps which demonstrate
gnuradio capabilities. The rest is basically library.  Try to stay in the gnu
hierarchy.

> * question: what is your expection about gnuradio in hamlib?
>   IOW what do you want to do (goal to achieve)?
>   This will help in the design.

I'd like to be able to [easily] have my gnuradio-based app receive commands
from, and return status to a hamlib-based frontend program, like GRig.  Basic
stuff like change frequency, volume, mode, etc.

> * Can you show me sample code for init (with no GUI support)
>   is gnu/examples/fm_demod2_no_gui.cc a good start  ?

That is a good example of a program with no gui, but it requires hardware you
don't have.  The best code to try without special hardware would be audio_scope,
chirp_fft, and mixer.  All of those have gui code, though.

>   Something I can test myself would be great (FI source: file or sound
>   card).
>   All we need is objects initialization, source select: acquisition card or
>   sound card, set_freq/change_freq, set_mode_FM.
>   that's enough to start with.

I can try to make a simple example program.  The easiest would be something
llike the mixer program, without the gui.  A simple demo would be to have hamlib
change the frequency instead of the gui.

> Note about gnu/examples/fm_demod2_no_gui.cc: the second FM demodulation may
> appear as a SUB band controller?

Not sure of the question here.  The program can demodulate 2 different FM
stations at the same time.

>
> * Am I wrong if I assume the output of gnuradio under Hamlib will always be
>   a sound card?
>   Or do we have to be output agnostic (and let gnuradio manage that)?

I wouldn't worry about the output.  Sometimes output will be audio, sometimes
data, sometimes IF samples, etc.

> * as far as I understand, gnuradio is only a toolbox/framework. I mean,
>   there's no defined API to select say, FM mode.
>   You have to do it by yourself. So I guess this has to be done in the
>   gnuradio backend unless some third party library already exists?

Right.  GNURadio is just a framework and library for designing signal processing
applications.  Making a radio, in the conventional sense, out of it is only one
possible application.

> * Hamlib backend would be obviously limititative because of API, and no
>   access to fine-tune parameters (rig_set_conf may help though).
>   SDR will only be seen as regular radio (advantage: re-use existing apps)

That's all I'm really looking for -- setting frequencies, volumes, gains, modes,
etc.

> * Would it be possible to hide the different FI sources to Hamlib ?
>   I mean Is there an abstraction layer in gnuradio that hides these
>   details, or does the application code has to know about the source?
>   IOW, do you have to recompile gnuradio/app if you change source?

Not sure of the question.

> * questions: is gnuradio pthread-safe (I've seen -D_REENTRANT, what about
>   the code itself)? does gnuradio use pthread by itself in constructors?
>   It looks like yes (VrMultiTask), but it is working fine?

some of it is thread-safe.  It should all be, eventually.

> * Implementation hints 2:
>   - gnuradio will be the backend name.
>   - each different VrSource will be a "rig" model on their own, belonging to
> this backend.

This is not really what I was thinking.  I was thinking more along the lines of
I write a program which may use any number of sources, sinks, and other blocks. 
In the main loop, between iterations, I can call methods on those blocks to
change parameters, like frequency (see src/gnu/lib/gr/GrFreqXlatingFIRfilter.h
for example).  Somehow, the control from the hamlib program needs to cause a
call to these methods.

>     So far, can you help me list the available VrSource's?
> MC4020+microtune_eval_board
>     Is there already libaudiofile support (reading from file)?  Input from
> sound card?

The VrSource's are irrelevant to this.

> * Have you heard already about soundmodem?
>   This is a software modem by Thomas Sailor for use on AX.25 packet networks.
>   It can make use of MMX, sound input/output: OSS/Free under Linux,
>   /dev/audio under Solaris and DirectSound under Windows
>   http://www.baycom.org/~tom/ham/soundmodem/

Seen it. Very cool, but not nearly as big a scope as what we're doing.

> * note: is it possible to put your doxygen generated documentation on the web?

That would be a good idea.

> * cheap advice (easier to give than to do :)
>   get away from asm code: use some SIMD library (like libSIMD
>   http://libsimd.sf.net )

libSIMD is not at the point of usefulness yet -- not even close.  We have
implemented general C++ code for all functions.  MMX/SSE/3DNOW are only used in
specializations.  We're able to get a tap per cyle

Matt

[Hamlib-developer] Re: gnuradio and Hamlib

[Stephane F. <f4...@fr...>]

On Mon, Jul 01, 2002, Matt Ettus wrote:
> The main difficulty, as I see it, is that both environments want to control the
> main loop.

Well, not really. Hamlib can be seen as a utility library. It does not 
need to control the main loop. However, the user application linked
against Hamlib may want to control it, e.g. grig or any GUI.
That's why it would be nice if the gnuradio backend would not require 
to grab the main loop.
And this is possible, provided gnuradio library is compiled with -DTHREAD.

 
> > * question: what is your expection about gnuradio in hamlib?
> >   IOW what do you want to do (goal to achieve)?
> >   This will help in the design.
> 
> I'd like to be able to [easily] have my gnuradio-based app receive commands
> from, and return status to a hamlib-based frontend program, like GRig.  Basic
> stuff like change frequency, volume, mode, etc.

Alright, seems pretty feasible. Since you're going to be first "user",
what is your FI source ? Is it the MC4020+microtune_eval_board ?

I understand that you will most probably not use only this source, 
and the backend will serve as a lab rat, but Hamlib needs at least 
to release some working skeleton. So which ever is fine. Then we'll 
be able to clone this "model" (i.e. source) to other models
within the backend.  
If this terminology sounds obscure to you, don't worry, as soon as code 
will show up, this will make sense.

> > * Can you show me sample code for init (with no GUI support)
> >   is gnu/examples/fm_demod2_no_gui.cc a good start  ?
> 
> That is a good example of a program with no gui, but it requires hardware you
> don't have.  The best code to try without special hardware would be audio_scope,
> chirp_fft, and mixer.  All of those have gui code, though.

I've ran them, nice. The code is pretty simple, and I can see what's the
essential part. I should be able to make a initial draft out of it.

Note: there's missing some checking against fftw existence in configure.in

> > Note about gnu/examples/fm_demod2_no_gui.cc: the second FM demodulation may
> > appear as a SUB band controller?
> 
> Not sure of the question here.  The program can demodulate 2 different FM
> stations at the same time.

In high-end rigs, there can be two indepedant receivers, a MAIN controler 
and a SUB band controler. Hamlib has support for such rigs.
So the question is not really a question. gnuradio will fit nicely, 
cool :^)


> > * Am I wrong if I assume the output of gnuradio under Hamlib will always be
> >   a sound card?
> >   Or do we have to be output agnostic (and let gnuradio manage that)?
> 
> I wouldn't worry about the output.  Sometimes output will be audio, sometimes
> data, sometimes IF samples, etc.

okay, however this will have to be setup in Hamlib through some kind of
interface. And that's what I try to think forward.
Anyway, I see already how (set_conf).


> > * Would it be possible to hide the different FI sources to Hamlib ?
> >   I mean Is there an abstraction layer in gnuradio that hides these
> >   details, or does the application code has to know about the source?
> >   IOW, do you have to recompile gnuradio/app if you change source?
> 
> Not sure of the question.

Okay, I'll try to put it another way.
Imagine you have everything in binary form, the user application, the Hamlib
(frontend + backend), the gnuradio library. Now, someone develops
some code to support a new FI source. Does gnuradio has an abstraction
interface (at binary level) to support this new FI by just dropping the new 
"lib" with recompiling everything.

Actually, Hamlib is able to do this.

If you're still not sure about the question, don't worry. I'll find the
answer sooner or later by reading the code :)

Note: seeing  eb->set_RF_freq and carrier->setFrequency does not look
	promising :(

> > * questions: is gnuradio pthread-safe (I've seen -D_REENTRANT, what about
> >   the code itself)? does gnuradio use pthread by itself in constructors?
> >   It looks like yes (VrMultiTask), but it is working fine?
> 
> some of it is thread-safe.  It should all be, eventually.

Good.
IMHO, we should use -DTHREAD for gnuradio, this would simply remove the
need to call m->process(), no more main loop, and everyone is happy.


> > * Implementation hints 2:
> >   - gnuradio will be the backend name.
> >   - each different VrSource will be a "rig" model on their own, belonging to
> > this backend.
> 
> This is not really what I was thinking.  I was thinking more along the lines of
> I write a program which may use any number of sources, sinks, and other blocks. 
> In the main loop, between iterations, I can call methods on those blocks to
> change parameters, like frequency (see src/gnu/lib/gr/GrFreqXlatingFIRfilter.h
> for example).  Somehow, the control from the hamlib program needs to cause a
> call to these methods.

Hmm. Please correct me if I'm wrong. IOW, you're not gonna use Grig right 
out-of-the-box. Your idea/need is to take Grig as a base, and hack it
specifically for gnuradio extension, in such a way that may not work
with a stock GRig. Right?
(I'm a bit exagerating, but this is to better understand what interfaces
will be needed in the backend).


-- 
Stephane

Re: [Hamlib-developer] Re: gnuradio and Hamlib

[Stephane F. <f4...@fr...>]

Hi Matt!

On Wed, Jul 03, 2002, Matt Ettus wrote:
> > Alright, seems pretty feasible. Since you're going to be first "user",
> > what is your FI source ? Is it the MC4020+microtune_eval_board ?
> 
> I'm not sure what FI stands for, but yes, I have the microtune and MC4020 combo.
>  That would be a great demo.

Problem of vocabulary. When I say FI, I mean a tap, a sampled source. What
is the best word anyway?
About the microtune and MC4020, I'll need some specs, like the bandwidth
of the FI (around 5MHz?) and the frequency range of the tuner.

 
> > Hmm. Please correct me if I'm wrong. IOW, you're not gonna use Grig right
> > out-of-the-box. Your idea/need is to take Grig as a base, and hack it
> > specifically for gnuradio extension, in such a way that may not work
> > with a stock GRig. Right?
> > (I'm a bit exagerating, but this is to better understand what interfaces
> > will be needed in the backend).
> 
> My intention was that GRig could be used unmodified.  I'd like the user not to
> be able to tell whether they are using a ft-1000 or gnuradio-based SDR.

Okay, so we have a dilemma about the main loop. Either GRig, Hamlib, or
gnuradio will have to suffer a (little) modification. Explicit main loop
for GRig and Hamlib or -DTHREAD for gnuradio.
A temporary solution would be to put the main loop in a modified
rpc.rigd daemon. We'll see.


Allright Matt, I have good news for you. I have just checked in an initial
gnuradio backend. It does not MC4020 yet, but there're should be enough
for you to see if I'm on the right track.

From a fresh checkout, add the --with-gnuradio to autogen.sh (after
doing chmod +x on the script). For the build to work, you'll have to put
libgnuradio.a in the gnuradio subdirectory (sorry, libgnuradio.la is
still not suitable for use, I'll explain you why in a future mail),
and have gnuradio-0.3 in the same parent directory as your hamlib
checkout. Having gnuradio installed in /usr/include and /usr/lib should
work also. And of course, libfftw must be already installed too :)

Let me know if you encounter any build problem.
I've provided a simple gnuradio/testgr program to be hack. Right now, it
should display a GUI with a scope, but dies with segfault:

  (fillods@charybde:gnuradio)$ ./testgr
  could not load logo file '/dev/null'
  testgr:hello, I am your main() !
  rig:rig_init called
  rig: loading backend gnuradio
  gnuradio: _init called
  rig_register (2001)
  gr_init called
  rig:rig_open called
  gr_open called
  VrNullSink cachedSize = 131072
  VrNullSink.setup[0x80869e0] 131072 freq 5e+06 (0.026214 sec)
  GrFFTSink cachedSize = 65536
  GrFFTSink.setup[0x8087cc0] 65536 freq 5e+06 (0.013107 sec)
  VrSource.setup[0x8086950] 500000 freq 5e+06 (0.100000 sec)
  Warning: could not allocate more than 33554432 bytes for shared memory
  buffer.
  VrSource output bufferSize = 4194304 (0.838861 sec)
  gr_get_vfo called: VFOA
  Erreur de segmentation

If I comment out line 84 in testgr.cc priv->m->process(), then I get the
GUI, and set_my_freq is called correctly. I must have messed up
something in the source and sink wiring.
Matt, can you have a look and tell me what I've done wrong?
BTW, the "Warning: could not allocate more than 33554432 bytes ...."
looks suspicious.

As a final note, yes, for now, this backend does nothing useful. Source
and sink are fake. And some data structures have to be better tought
(esp. the gnuradio_priv_data).
This first release is only to devise the glue between Hamlib and gnuradio.


> P.S.  If anyone is interested, I will be giving a talk and demos for gnuradio at
> TAPR's DCC this year.  See http://www.tapr.org

Hey, that's great!
If only I were still in AZ, I would have loved to attend!


Cheers,
Stephane


P.S. and see you in the contest "Rallye des points hauts" (kind of field 
     day on 2m and up) for those in Europe!

Re: [Hamlib-developer] Re: gnuradio and Hamlib

[Matt E. <ma...@et...>]

Quoting Stephane Fillod <f4...@fr...>:

> Problem of vocabulary. When I say FI, I mean a tap, a sampled source. What
> is the best word anyway?
> About the microtune and MC4020, I'll need some specs, like the bandwidth
> of the FI (around 5MHz?) and the frequency range of the tuner.

I see.  It looks like you want the hamlib plugin to be the actual radio.  I
guess that's reasonable.

> Allright Matt, I have good news for you. I have just checked in an initial
> gnuradio backend. It does not MC4020 yet, but there're should be enough
> for you to see if I'm on the right track.

Great.  

> >From a fresh checkout, add the --with-gnuradio to autogen.sh (after
> doing chmod +x on the script). For the build to work, you'll have to put
> libgnuradio.a in the gnuradio subdirectory (sorry, libgnuradio.la is
> still not suitable for use, I'll explain you why in a future mail),
> and have gnuradio-0.3 in the same parent directory as your hamlib
> checkout. Having gnuradio installed in /usr/include and /usr/lib should
> work also. And of course, libfftw must be already installed too :)
> 
> Let me know if you encounter any build problem.
> I've provided a simple gnuradio/testgr program to be hack. Right now, it
> should display a GUI with a scope, but dies with segfault:

I checked out from cvs, but configure is missing.  I tried ./autogen.sh, but I get:

[matt@localhost hamlib]$ ./autogen.sh 
I am going to run ./configure with no arguments - if you wish 
to pass any to it, please specify them on the ./autogen.sh command line.
/usr/bin/m4: configure.in: No such file or directory
kylix/Makefile.am:1: invalid variable `dist_sharedstate_DATA'
autoconf: configure.in: No such file or directory
./autogen.sh: ./configure: No such file or directory




On 2 totally unrelated notes...   

1> It's been about 10 years since I did any RPC stuff...  How is security
handled?  I wouldn't want to put a rig or rotator on the net only to have it hacked.

2> Has there been any thought/work on alternate pathways for audio/signals?  I
guess I mean that while with my FT-1000 I might want all audio to go through the
standard speaker/mike,  what happens if I want to pass rig audio through the PC
soundcard for processing?  Or what if I wanted to send the audio from my remote
rig over ethernet back to the point where I am operating grig over rpc?

Matt

Re: [Hamlib-developer] Re: gnuradio and Hamlib

[Stephane F. <f4...@fr...>]

Quoting Matt Ettus <ma...@et...>:
> I checked out from cvs, but configure is missing.  I tried ./autogen.sh,
> but I get:
> 
> [matt@localhost hamlib]$ ./autogen.sh 
> I am going to run ./configure with no arguments - if you wish 
> to pass any to it, please specify them on the ./autogen.sh command
> line.
> /usr/bin/m4: configure.in: No such file or directory
               ^^^^^^^^^^^^
> kylix/Makefile.am:1: invalid variable `dist_sharedstate_DATA'
> autoconf: configure.in: No such file or directory
> ./autogen.sh: ./configure: No such file or directory

hmm, it seems like you have an old autoconf (like 2.13).
You can check it with "autoconf --version" and "automake --version".
autoconf must be at least 2.50 or higher, and automake at least 1.5.


> 1> It's been about 10 years since I did any RPC stuff...  How is security
> handled?  I wouldn't want to put a rig or rotator on the net only to
> have it hacked.

AFAIN, RPC security has been left in the state it was 10 years ago.
There are some session oriented management in RPC, but Hamlib does
not support it yet. This can change in the future.
Another cheap way to protect your rpc daemon is to use a tcp/udp wrapper,
that restricts acces based on client IP address.

Anyone knows how to secure RPC?

> 2> Has there been any thought/work on alternate pathways for audio/signals?  I
> guess I mean that while with my FT-1000 I might want all audio to go through 
the
> standard speaker/mike,  what happens if I want to pass rig audio through the 
PC
> soundcard for processing?  Or what if I wanted to send the audio from my  
remote
> rig over ethernet back to the point where I am operating grig over rpc?

I guess we'll be making some configuration (rig_set_conf) or "extra" parameter 
(rig_set_ext_parm) in the gnuradio backend, to tell it what the output
should be, provided it has support for it.
The difference between rig_set_conf and rig_set_ext_parm is that set_conf
can only be done before rig_open, whereas set_ext_parm can be done any time 
afterwards.
IOW, do you want to change the ouput sink dynamically while gnuradio is running?


Cheers,
Stephane

PS: I'll add support for the FT-1000 next month, after my vacation 
    in 5U (sri, not qrv).