GBTIDL Release Notes Version 2.10
Introduction
Getting Started
New Features
Bugs Fixed
New Contributed Code
Register as a GBTIDL user
Running GBTIDL in Green Bank, Socorro, or Charlottesville:
Running your own copy of GBTIDL
IDL License
Download and Installation of GBTIDL
Introduction
We are pleased to announce the release of GBTIDL, version 2.10.
As of 5 July, 2016, GBTIDL at all NRAO sites corresponds to version 2.10 and
includes the changes described in this note.
Getting Started
The starting point for GBTIDL documentation, bug reporting and tracking,
and discussion is http://gbtidl.nrao.edu/
A comprehensive User's Guide is available.
New Features
Along with changes in the sdfits tool released with this version of GBTIDL,
the known VEGAS spikes (spurs) produced at regular intervals can now be
handled transparently by GBTIDL. See ModificationRequest16Q414 for more
details.
sdfits now will either flag or interpolate across these spikes in VEGAS
data.
Flagging, the default, produces an associated flag file that GBTIDL
reads and applies to the data as the data are read from disk.
That means that the data in the SDFITS file remain unchanged
from their original values and GBTIDL replaces those flagged
locations with not-a-number (NaN) as the data are read from the
file. IDL usually ignores NaN values in all calculations. These
values are not displayed on the plotter and do not participate
in math operations. Other software outside of IDL may have
issues with NaN so users may chose to replace them with
interpolated values.
Interpolation. When "-spurinterp" is used with the sdfits tool,
the data values at the spike locations are replaced by the
average of the two adjacent channels.
The center spike has always been interpolated by the sdfits tool.
That behavior remains except that now the center spike is not
necessarily always at the center channel.
The SDFITS file now includes additional columns that can be used
(e.g. by GBTIDL) to determine spike locations even when the flag
file is not present. These values are now found in the data
containers in GBTIDL for use by GBTIDL.
GBTIDL has 3 new routines to aid users in working with VEGAS data
containing spikes. These will only work with SDFITS files filled by
sdfits version 1.21 or later. They will not work with older SDFITS
files because the necessary information is not present.
spurinterp and dcspurinterp work out where the spikes are and
replaces the data at those locations with the average of the two
adjacent channels. This works on the default buffer or any of
the numbered data buffers. Data containers can be passed to
dcspurinterp.
flagspurs This regenerates all of the VEGAS spur flags using the
information contained in the SDFITS file. Any existing spur
flags (an idstring of "VEGAS_SPUR") are removed (unflagged)
before the new flags are set. This is useful if the flag file
has gotten corrupt or there is a desire to start flagging over
from scratch.
spurshow This draws vertical lines on the plotter at all of the
VEGAS spur locations for the currently display spectrum. Note
that these vertical lines (vline) are persistent in GBTIDL and
must be erased explicitly either using clear to clear the entire
display or clearvlines to clear just the vertical lines.
GBTIDL calibration and data retrieval routines will use the flag
information unless instructed not to. Use the /skipflag option
to either ignore all flags or just the vegas spur flags, which
have an idstring of "VEGAS_SPUR".
getfs, fold, dcfold The behavior of the fold routine, used by getfs, in
the presence of user-flagged channels was changed. fold is used with
frequency switched data to align and average the two reduced switched
states used to produce the final result. The previous behavior was to do
nothing special, meaning that in that final average if one value was a
NaN and the other was a finite value the final average would be that
finite value. It was realized when processing VEGAS data with spikes
flagged that if that finite value was not close to the surrounding
average values (e.g. the baselines differed noticeably in the two
switched states after alignment) that the result would be an apparent
spike (positive or negative) at the location of the flagged channel -
effectively the spike that was removed by flagging would reappear. The
new behavior is to make sure that the result of averaging a NaN with a
finite channel is a NaN. This is described in ModificationRequest2Q316.
In general this behavior is not desired, e.g. when flagging intermittent
RFI one generally wants the final average to be the average at that
channel for all of the unflagged integrations where no RFI was flagged.
Only fold was modified to behave this way.
The default values for IFNUM, PLNUM, and FDNUM have been changed to make
working with data more straightforward when the previous default
(all 0s) does not exist for that scan. In short, the calibration
routines will always proceed (unless there is no data) with reasonable
default values for the given scan(s). The actual values used are now
printed as the routines finish. See ModificationRequest5Q214 for more
details.
awv, gmeasure Karen Masters submitted a revised amv that adds a 4th
fitting mode that fits a polynomial to the two sides of the galaxy
profile. This mode is now available in gmeasure.
The nchans field in the structure returned by scan_info now has an element
for each sampler in order to support the full range of VEGAS
configurations.
scan_info and the data selection used by the calibration routines are
faster.
The re-indexing code that merges multiple indices into a directory index
so that a directory of SDFITS files can be used as a single data source
(e.g. VEGAS data) was rewritten so that the result more closely matches
that produced by sdfits. This allowed the speed-ups in scan_info and
data selection.
The index file now supports up to 10^7 rows by making use of existing space
(no new index version is necessary). Data selection is very slow at
that upper limit and there are no current plans to extend that limit or
improve the index design to speed data selection. Users should use
sdfits to split their data into multiple SDFITS files if the total
number of spectra to be filled exceeds that limit.
Bugs Fixed
accum and related were not ignoring data where Tsys was invalid (not a
number, NaN).
fold The optional ftol keyword was not being passed to dcfold, making that
parameter useless.
gettp The optional subref selection was broken.
chdoppler A fix was submitted by Carl Heiles that addressed a problem with
the routine was used to process a vector of coordinates. That feature is
not used within GBTIDL but may be used by GBTIDL users.
The data retrieval code used by all calibration routines no longer assumes
that all possible values of PLNUM and FDNUM are present within a given
scan.
Flagging involving an SDFITS directory (e.g. VEGAS) would sometimes fail.
Re-indexing a Zpectrometer FITS file in GBTIDL would fail.
New Contributed Code
sdextract Extracts a subregion by channel range from every spectra in an
SDFITS File, producing a new SDFITS file with fewer channels. The
extracted data may also be boxcar smoothed and the number of channels
appropriately reduced.
Register as a GBTIDL user
Use the on-line registration form.
Running GBTIDL in Green Bank, Socorro, or Charlottesville:
At the Unix prompt, simply type gbtidl
Running your own copy of GBTIDL
IDL License
You must have an IDL license to run GBTIDL. IDL version 6.3 and later is
supported. If you don't have a license you can run GBTIDL remotely, using the
computers in CV or GB. See the GBTIDL homepage for details.
Download and Installation of GBTIDL
To obtain GBTIDL, download the files here
Create a directory for the installation and unzip the tar ball:
mkdir mygbtidl
mv gbtidl-2.10.tar.gz mygbtidl/.
cd mygbtidl
gzip -d gbtidl-2.10.tar.gz
tar -xvf gbtidl-2.10.tar
go to the newly created installation directory, 'gbtidl':
cd gbtidl
copy the file 'run_gbtidl' to 'gbtidl':
cp run_gbtidl gbtidl
edit the 'gbtidl' script to use the correct path to your installation of
idl. For example:
LOCAL_IDL=/usr/local/bin/idl
edit the 'gbtidl' script to replace "PLACE_INSTALLATION_DIR_HERE" with the
full path to the gbtidl executable script. It should then look something
like this:
export GBT_IDL_DIR=/home/mygbtidl/gbtidl
start GBTIDL:
./gbtidl
You may wish to add the installation directory to your Unix path.
