#135 Using -cyclic with LOFAR data

[Sophia]

Hello! I'm trying to use dspsr to perform cyclic spectroscopy on LOFAR data with 20 sub-bands. I'm using a container that Vlad Kondratiev created (https://git.astron.nl/rd/pulp2-cwl-folding) which contains the commit 93ff8b39 of dspsr and includes the LOFAR_DAL backend. I tried to running it using something like:

$ dspsr -t 8 -K -d 1 -b 128 -cyclic 512 -L 60 -E /data/B0329+54.par -O /data/output_cyclic /data/L2054541_SAP000_B000_S0_P000_bf.h5

but I got the following error message:

dsp::UnloaderShare::unload error unloading division 0
Error::stack
    dsp::Archiver::unload
    dsp::Archiver::set Pulsar::Archive
    dsp::Archiver::set Pulsar::Integration
    dsp::Observation::get_unswapped_ichan
Error::InvalidState
Error::message
    nsub_swap=10240 is not less than nchan=10240

I thought that this was a problem with how I sat up my cyclic parameters, so I tried with -cyclic 1024, -cyclic 512, -b 128, -b 64, with and without -a psrfits, and they all produce the same error.

I'm now starting to suspect that this is a problem with how nsub_swap is handled in -cyclic mode:

1. I think that the LOFAR_DAL backend applies the subband frequency reversal at read time: it corrects the inverted subbands as it ingests the raw data, before any processing happens. By the time the data reaches the archiver, all channels are already in the correct frequency order, so `nsub_swap=0` and the `get_unswapped_ichan` check is never triggered because `nsub_swap = 0 < 20 =nchan = N_subbands`.
2. In cyclic mode, the processing order seems to be different? The cyclic spectrum is created after coherent dedispersion, which means the channel structure is rebuilt, and the 20 physical subbands each become 512 (or whatever was used for `cyclic`) cyclic channels. This channel reconstruction happens after the LOFAR_DAL read, which means the frequency reversal that was applied at read time in non-cyclic mode doesn't carry correctly into the new channel structure. Instead, what I *think* is happening is that when the archiver receives the 10240 channel cyclic output, some part of the code recomputes `nsub_swap` by scaling the original subband inversion count by `N_cyclic`, giving 20 x 512 = 10240 instead of either leaving it at 0 or keeping it at 20 physical subbands. The result is `nsub_swap = nchan`, which breaks the strict < check.

Does this make sense? And is there a way of fixing this? Thank you so much in advance!

[Sophia]

I fixed it! Here's my complete bug report and patch submission:

Bug: when running dspsr in cyclic spectroscopy mode (-cyclic) on LOFAR data (lofar_dal backend), dspsr fails with:

  dsp::Observation::get_unswapped_ichan
  Error::InvalidState
  nsub_swap=10240 is not less than nchan=10240

I narrowed down the bug to Signal/Pulsar/CyclicFold.C, where set_nsub_swap is called with out->get_nchan() (the expanded cyclic output channel count, N_subbands x N_cyclic) instead of in->get_nchan() (the number of physical input subbands). Since get_unswapped_ichan requires nsub_swap < nchan and uses nsub_swap to compute the number of cyclic channels per physical subband, passing the full output nchan causes nsub_swap = nchan, which always fails the check.

Patch: The fix is a one-line change: replaceout->get_nchan() with in->get_nchan() in the set_nsub_swap call. With this fix, nsub_swap correctly equals the number of subbands (e.g. 20 for LOFAR), and get_unswapped_ichan correctly applies the frequency reversal within each block of N_cyclic channels.

Non-cyclic folding of LOFAR data is unaffected (a different code does the subband swap at read time). The bug only pops up when you use a combination of lofar_dal + cyclic mode.

I tested this on dspsr commit 93ff8b39 with 20-subband LOFAR HBA data at 141 MHz (PSR B0329+54). I'm attaching the diff file I created for CyclicFold.C using Git.