Batch Processing Directory of Files

[Cory Bethrant]

Hey everyone,

I am trying to automated a set of repetative data processing steps as it will greatly help speed up the R&D process.
Here is my code so far:

import gwy

plugin_menu = "/Batch Process Data"
plugin_type = "PROCESS"
def run():
    # Create undo point
    key = gwy.gwy_app_data_browser_get_current(gwy.APP_DATA_FIELD_KEY)
    gwy.gwy_app_undo_qcheckpointv(gwy.data, key)

    # Get active datafield and store it to 'd' variable. The variable is object
    # of type gwy.DataField
    d = gwy.gwy_app_data_browser_get_current(gwy.APP_DATA_FIELD)

    # Call functions
    d.flatten_base()
    d.grain_mark()
    d.fix_zero()
    d.GwyToolGrainMeasure()


    # Report data change to Gwyddion, this is required to update (redraw)
    # graphical presentation of datafield in application window.
    d.data_changed()

I know Python so the main things I'm trying to find out if these steps can actually automated, what perameters should be used, and also where to place the script so it shows up as an option in gwyddion as I am on Mac OS X with MacPorts.

[Cory Bethrant]

Also, if there is anyway I can automate this for an entire directory, it would be very useful.

Thanks Again,
Cory Bethrant
CCMR REU Software Engineer

[David Nečas]

You can write standalone python scripts, i.e. not run from within Gwyddion, which usually makes more sense when you want to do things such as process entire directories (a normal pygwy script really controls the running Gwyddion and tells it what to do; a standalone script can actually look quite similar but feels more appropriate for this...). Please see

https://sourceforge.net/p/gwyddion/mailman/message/30725548/

https://sourceforge.net/p/gwyddion/discussion/pygwy/thread/f634d66b/

and also

http://gwyddion.net/faq.php#faq028

There should be an even better example somewhere in the mailing list archive but I cannot find it now.

Concerning running various functions, if there is a DataField method you can use directly, it is usually less hassle.

You can invoke data process modules using gwy.gwy_process_func_run() as shown in the FAQ example. This does not work completely like normal function calls, you need to set the parameters beforehand in the settings (from which the module will then read them). This is briefly demonstrated in the FAQ example. Often the easiest way to find what the parameters look like is running the functions from the GUI and then invoking View Log from the right-click image window menu.

If you have questions to specific functions I will try to answer.

[Cory Bethrant]

Lastly, it'll probably be helpful to know the steps I am trying to do:

1. Flatten Base
2. Mark grain by threshold
3. Shift Minimum data value to zero
4. Measure individual grains

Then repeat.

[David Nečas]

Flatten Base is a simple module function that can invoked (after selecting the required data, see the links):

gwy_process_func_run('flatten_base', container, gwy.RUN_IMMEDIATE)

Marking by threshold and shifting is best done directly using DataField methods:

mask = gwy.DataField.new_alike(datafield, False)
datafield.grains_mark_height(mask, 10.0, False)

and

datafield.add(-datafield.get_min())

Grain analysis generally requires numbering the grains first:

grains = mask.number_grains()

as most functions take the grain numbers (grains) as argument.

[Patrick Harrison]

Apologies for reopening this.

I am trying to automatically extract partcicle sizes. My thresholding works fine, however number_grains() produces an unwanted result for me, and I am wondering how to go about it.

In congruence with example above, if I extract grain statistics:

pixel_area = datafield.grains_get_values(grains, gwy.GRAIN_VALUE_PIXEL_AREA)

where pixel_area is a list. My problem is that pixel_area contains the pixel area of the background too. (Often, but not always, multiple orders of magnitude larger than the particles).

Is there a way to programatically exclude the background from the grain analysis?

Of course, using manual Gwyddion functions, the background is ignored in the analysis. I would like to replicate this. (eg. mark by Otsu's -> Distributions)

Regards,
Patrick

[David Nečas]

The i-th value in the returned list correspond to grain numbered i. Grain numbers are positive, starting from 1; DataField.number_grains() puts 0s to the area outside grains. The zeroth element of the returned list should be simply ignored.

I could perhaps remove the zeroth element in the Python API but then there would be confusing mismatch with C and one would have to remember to add/subtract 1 when working with grain numbers. If you do not care about grain numbers and just do not want the zeroth element there then do

del pixel_area[0]