Creating Synthetic Hyperspectral Imagery with Spectral Python

[Tom Fitzgerald]

My goal is to create synthetic datasets and fully retain all native Spectral Python capabilities such as band resampling, spectral angle, clustering, etc.

Specifically I would like to create synthetic AVIRIS images in the Spectral Python image format. I want to use ASTER spectral library spectra as synthetic data source.

Thus far I have read the ASTER data, I have created an ASTER SQLite database, I have queried that database, resampled the data to match the AVIRIS band centers, and saved the resulting spectra to an Envi Spectral Library.

I now want to create a blank or "empty" Spectral Python AVIRIS image that I can populate as desired with the Envi Spectral Library data. Can I get some ideas or guidance for an optimal path for this step. It seems to me that I should be able to use functionality that is embedded in Spectral Python already but not separately documented. What about creating blank Envi or Erdas image formats?

[Thomas Boggs]

(NOTE: For anyone wanting to perform a similar task, the details of creating a custom spectral library can be found in the Spectral Libraries section of the SPy User's Guide.)

The easiest way to create a new image file with Spectral Python (SPy) is to create the new image as a numpy array and save it to ENVI format. So first, create an empty numpy array:

myimage = numpy.empty((nrows, ncols, nbands), 'float32')

Unless you are doing some kind of dimensionality reduction, nbands should correspond the number of bands in the spectral library you created.

Next, populate the numpy array with image data. For example, if mylib is the spectral library you created and my_pixel_generator is some function you wrote that creates random pixels from a library:

for i in range(nrows):
    for j in range(ncols):
        myimage[i, j] = my_pixel_generator(mylib)

Finally, save the image data to ENVI format by calling envi.save_image, which will create both the image file and the associated ENVI header file.

envi.save_image('myimage.hdr', myimage)

The new file can then be opened as usual (i.e., with open_image).

You can also achieve this by writing to a memory-mapped image file created by calling envi.create_image but there are a few more details involved so unless you experience significant performance or memory issues, it is easier to just use the method described above.

[Tom Fitzgerald]

I used the approach suggested and it works. But I have discovered two more issues.
1. After saving the new file and then reopening as a SPy image I use the command

img.bands = mylib.bands
to set the img bands class. The class is set properly. I can see this is so by using view(img) and clicking on spectra of interest. The plotted data is appropriately band indexed. But when I try to resave img to write over the file I just read from I get a write protected error. Even when using "force = True".
2. When I save img to a new file name the file is properly created but the Hdr file does not contain the band information. That is not retained even after being properly set and checked prior to saving.

[Thomas Boggs]

Regarding the write-protected error, that appears to be due to trying to overwrite a file to which you currently have open file handles. I don't know of an easy way around that. I have confirmed that a file can be overwritten multiple times (with force=True) as long as the file is not already open.

There is a reason (which escapes me at the moment) that I do not include the band information directly from the image being saved. I think the reason may be because the image being saved could be a linear transformation, in which case the original band info is no longer applicable. I will probably add a keyword argument to save_image during the next update to include the image's band info. In the mean time, you can explicitly add the band info by doing the following:

md = img.metadata.copy()
md['wavelength'] = img.bands.centers
md['fwhm'] = img.bands.bandwidths
envi.save_image('newimg.hdr', img, metadata=md)

You can omit the fwhm line if you don't have that info. You can also start with an empty metadata dict (i.e., md = {}), if that data is not relevant.

[Tom Fitzgerald]

I executed the instructions as written. Checked that md properly held the metadata, and saved the file. When I opened it the metadata was still not present.

[Thomas Boggs]

I made a typo in my previous post (corrected now). I mistakenly had the string "bandwidth" where it should have been "wavelength". The corrected line is:

md['wavelength'] = img.bands.centers

Since you are setting it from your spectral library, you will probably want it to be

md['wavelength'] = mylib.bands.centers

[Tom Fitzgerald]

Thanks. Working now. I'd seen the typo and adjusted. The mistake was my looking inside an earlier HDR file in my directory structure.

[Tom Fitzgerald]

Is there a way to specify the output number format. The default seems to be "5" for double precision - 64 bit. Can I instead specify 16 bit signed or unsigned integer?

[Thomas Boggs]

Yes. You can specify the output data type via keyword argument. This is from the envi.save_image doc string:

Keyword Arguments:

`dtype` (numpy dtype or type string):

    The numpy data type with which to store the image. For example,
    to store the image in 16-bit unsigned integer format, the argument
    could be any of `numpy.uint16`, "u2", "uint16", or "H".