NarrowBand DataTrans Files

"DATATRANS" project for Linux - V1.5:
------------------------------------------------------
V.Portet - 2017 - http://sourceforge.net/u/vincent95/profile/


1) Targetted audience:
------------------------------

* Short wave / long wave (narrow bandwidth) Amateur Radio enthousiasts
* Experimenters of vintage transmission systems
* People living in areas with absolutely no internet and no 3G/4G/Edge access (only analog telephone or CB/Ham radio).
  Peer-to-peer data sharing (or even broadcast to several peers at a time) are then made possible.
* Conditions where only a unidirectional audio link is available (no handshaking required)

It is assumed that the user is able to find / build specific audio cables connecting the computer audio sockets to his own transmission equipment.
It is also assumed that he's familiar with some terms like "common ground, audio channel, audio jack pinout, amplitude, attenuator, resistor, impedance, level"...
Otherwise, if interested anyway by the project, ask some help from an experimented friend.



2) Exact purpose:
------------------------

Sending and receiving through a narrow bandwidth transmission media (at least 4.5KHz) several kinds of data files having moderate sizes. The signal output to the transmitter is the sound card's main loudspeaker analog socket while the received signal taken from the receiver is fed to the soundcard analog input line.

The transmission program is "datarans" while the corresponding reception program is "datarec".

Specific modulation and encoding methods are selectable when calling "datatrans" depending on the file type.
This allow giving priority to one of these requirements: Error correction, speed, or non destructive encoding:

* Text file (no error correction, speed of 84 bytes/s): mode "-t"
* Error-sensitive data file (error detection and correction, but very low effective speed of 22 bytes/s): mode "-d"
* B/W 200x160 (automatically downsized) pictures in AM subcarrier analog SSTV (80 times faster than data mode): mode "-g"
* Color 200x160 (automatically downsized) pictures in AM subcarrier analog SSTV (80 times faster than data mode): mode "-c"
* B/W 200x160 short movies in AM subcarrier analog SSTV (80 times faster than data mode): mode "-y"
* Start/stop synchronized and slow-speed analog audio transmission: mode "-a" not available yet.

When "datarec" (called without arguments) catches the leading tone + the transmission identification sequence, it automatically selects the appropriate mode for decoding, as well as the file name (shortened if too long). 

Example:
-----------

The transmitting peer call datatrans to transmit a text file "readme.txt":
$ datatrans -t readme.txt

The receiving peer has already prepared the following call in the console (but has not pressed <Enter>) while listening to the incoming signal:
$ datarec
Then, as soon as the leading tone is heard, the receiving peer presses <Enter>.
The reception goes on and the text file is appropriately decoded and written to disk.


Current restrictions (improvements planned in further versions):
---------------------------------------------------------------------------------------

* Need of manually launching datarec exactly at the time where the short leading tone starts
* No built-in amplitude monitoring to set-up and adjust the receiving station
  (need to use for instance Audacity in record mode for settings before the first real use)
* Robustness of AM SSTV versus noise has to be improved


Last implemented features in this version:
-------------------------------------------------------

* Selectable pre-emphasis at transmission (improves reliability when media frequency response is not flat at all)
* Robustness of data/text decoding improved versus speed deviation.



3) Recommendations to get the best results:
------------------------------------------------------------

* Adjust the received signal level within 50-90% of the maximum input (clipping) level by processing to a preliminary test using for instance Audacity as a recorder. This will guarantee that the signal-to-noise ratio is high while no saturation occurs. This monitoring is also a mean of checking that the transmitting peer does not overmodulate (amplitude clipping even at low level).

* When replaying pre-generated broadcast scenarios with low-tech storage devices (e.g. cassette tapes), ensure that the speed difference at playing time does not deviate from more than +/-3.5% from the speed at recording time.

* If test recordings made on the received signal with Audacity show that the amplitude varies in big proportions within the stream (1:8 or higher) while in data mode or text mode, try to use the "tone" control, if available, on your radio receiver to reduce this effect. On the transmitter's side, make tests with preemphasis enabled (optional parameter "-p"). Example:
$ datatrans -t -p myfile.txt

* For advanced users (familiar with hardware DIY): If the frequency response cannot be corrected by a "tone" control and if preemphasis at transmission is not strong enough, a basic filter (1st or 2nd order high-pass / low-pass) can be installed on the radio/telephone receiver output, depending on the response profile. 


4) Compatibility with old machines and distros:
------------------------------------------------------------------

Even if built with Pulse Audio support (see chapter 8), the executables can fall back to the OSS sound server connection mode (also supported by ALSA) in case where Pulse Audio is not running on the machine. This gives the possibility to use very old machines where only old distros can run.


5) Simulation mode:
----------------------------

In case where the user wants to try the present project before doing any real situation test (hardware, connections...), it is possible to use a simulation feature.
When calling datatrans as well as datarec, this mode is selected by simply adding the option "-w" in the command line:

$ datatrans -t -w myfile.txt
==> generates the audio output into an audio file: "myfile.wav"  (44100 spl/s 16 bits stereo (*) )

$ datarec -w
==> Takes as audio input the content of the file: "input.wav" (most common settings: 44100 spl/s 16 bits stereo (*) )
Therefore, any generated "wav" file to be loaded by datarec has to be renamed "input.wav" .


This simulation mode can also be useful in case where the user wants to develop complex multifile conversion/transmission/reception scripts based on datarans/datarec executables, without the need of testing transmissions at real-time speeds with two machines!

(*) See next paragraph.


6) Connections in real transmission mode:
----------------------------------------------------------

Even if the "wav" files are stereo, no actual stereo transmission media is required (both channels are identical).

In real transmission mode: Only one channel can be taken from the transmitting computer output to feed the mono transmitter (1), while the output of the mono receiver can be connected to both audio channels of the receiving computer input (2).

(1)(2) In both cases the output and input levels shall be adapted (possible resistor/potentiometer based attenuators...). In case where an output signal is much too high, avoid setting the source output volume at almost zero and prefer using a simple resistor divider as attenuator. Otherwise the signal-to-noise ratio and the amplitude stability might too poor to guarantee a reliable operation.



7) Scripts provided to simplify the transmission of pictures and short B/W movies:
--------------------------------------------------------------------------------------------------------------------

Picture and movie files transmitted through AM subcarrier analog SSTV are raw video files with a resolution of 200x160. They cannot be directly handled by multimedia applications.
To make things easier, bash scripts are provided in the subfolder "useful-scripts", so that png files and various types of video files can be transmitted in a user-friendly way. Any necessary downsizing is automatically processed.

Example:
-----------

After having installed the scripts and executables at the appropriate locations (see paragraph 8):

* The transmitting peer launches:
$ trans_colorpic.sh mycolorpicture.png

* The receiving peer has already prepared the following call (but has not pressed <Enter>) while listening to the incoming signal:
$ rec_colorpic.sh

Then, as soon as the leading tone is heard, the receiving peer presses <Enter>.
After a successful reception of the raw color picture, a png file is rebuilded as output.

Requirements for using these scripts:
---------------------------------------------------

The FFMPEG package (http://ffmpeg.org/ or from your distro's package manager) must be installed on both machines: Sender and receiver.

NOTE:
-------

The scripts include by default the simulation mode option "-w" which has to be removed for real situation use.



8) Compiling and installing:
------------------------------------

The two bash scripts "buildrec.sh" and "buildtrans.sh" are provided for an easy and fast compilation by gcc of both single-file source projects.

If compiling on a distribution where the pulseaudio-devel packages cannot be installed, modify these both scripts as follows:
Replace the string "-D__PA_ENABLED__" by "-D__PA_DISABLED__"
In this case both programs will be unable to use anything else than OSS / ALSA OSS support as audio server.

In any case the oss-devel package will be required to compile, as well as pkg-config.

After a successful compilation, move the compiled executables as well as the content of "useful-scripts" in one of the following possible destinations:
/home/<your-user-homedir>/bin  (if exists. Preferred)
/usr/local/bin  (copied as super user, if path is active. Preferred to next location)
/usr/bin  (copied as super user, if no other binary folder registered as executable path does exist)

Ensure that all these executable files are defined as executable through their access rights.