#147 IDG Performance

[Yuping Huang]

Hi,

I'm looking into using IDG for imaging for the OVRO-LWA array. We have about 260 antennas and ~2000 frequency channels. For a single snapshot on a host with 64G memory, doing 3 channels would use up all of the memory. Changing the image size and pixel size doesn't really change the memory usage, which probably suggests that I am doing something wrong.

I was wondering if you could provide more detailed/up-to-date performance details of idg with wsclean than https://sourceforge.net/p/wsclean/wiki/ImageDomainGridder/#performance
e.g. the number of baselines of the MWA set? I'll try to go from there and figure out the expected performance is for our dataset.

Thanks!
Yuping

[André Offringa]

Hi Yuping,

Did you try to use -channels-out to lower the memory requirement by IDG (it's described in the link you refer too) ?

Cheers,
André

[André Offringa]

the number of baselines of the MWA set? I'll try to go from there and figure out the expected performance is for our dataset.

The MWA has 128 antennas, so ~128^2/2 baselines.

[Yuping Huang]

Hi André, thanks for the reply. I did not put an idg.conf file in the working directory so idg was trying to allocate a 256xsnapshot buffer and ran out of memory there. Now the memory use seems fine after I set buffersize to 1. Feel free to close this ticket :)