pflexible-users — Users of pflexible

[Pflexible-users] Problems reading the header - Flexpart

[LUISA F. V. <lui...@ho...>]

Hello!
I am using pflexible and I have a problem reading a header from one of my output flexpart simulation. I am using flexpart 90.02.Can anyone help me please?
Thanks a lot!Luisa
-----------------------------------------------------------------------------------------------------------This is the error message:
Traceback (most recent call last):  File "/usr/local/lib/python2.7/dist-packages/pflexible-0.9.1-py2.7-linux-x86_64.egg/pflexible.py", line 4746, in __init__    h = read_header(path, **readheader_ops)  File "/usr/local/lib/python2.7/dist-packages/pflexible-0.9.1-py2.7-linux-x86_64.egg/pflexible.py", line 1189, in read_header    h.first_date = available_dates_dt[0]IndexError: list index out of rangeTraceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/usr/local/lib/python2.7/dist-packages/pflexible-0.9.1-py2.7-linux-x86_64.egg/pflexible.py", line 4754, in __init__    Does the `header` file exist in path?\n{0}'''.format(path))IOError:             Could not set header variables.            Does the `header` file exist in path?

 		 	   		  

[Pflexible-users] Pflexible mailing list

[LUISA F. V. <lui...@ho...>]

lui...@ho... 		 	   		  

Re: [Pflexible-users] read flexpart outputs v9.02

[Albert J. P. <alb...@ic...>]

Hi John,

Thank you!

For sure I will.

On 27/11/14 11:50, John Burkhart wrote:
> Hello Albert,
>
> Please check out the latest default branch. We have fixed pflexible so 
> that it is reading headers V6 -> V9.2
>
> However, not yet for the netcdf output.
>
> please report any errors.
>
> --john
>
>
> On Wed, Nov 26, 2014 at 3:53 PM, Albert Jornet Puig 
> <alb...@ic... <mailto:alb...@ic...>> wrote:
>
>     Hi all,
>
>     Does someone knows if pflexible is able to read flexpart v9.02
>     outputs?
>       If that is the case, from which version is possible?
>
>     Thanks
>
>     ------------------------------------------------------------------------------
>     Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>     from Actuate! Instantly Supercharge Your Business Reports and
>     Dashboards
>     with Interactivity, Sharing, Native Excel Exports, App Integration
>     & more
>     Get technology previously reserved for billion-dollar
>     corporations, FREE
>     http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>     _______________________________________________
>     Pflexible-users mailing list
>     Pfl...@li...
>     <mailto:Pfl...@li...>
>     https://lists.sourceforge.net/lists/listinfo/pflexible-users
>
>

Re: [Pflexible-users] read flexpart outputs v9.02

[John B. <jfb...@gm...>]

Hello Albert,

Please check out the latest default branch. We have fixed pflexible so that
it is reading headers V6 -> V9.2

However, not yet for the netcdf output.

please report any errors.

--john


On Wed, Nov 26, 2014 at 3:53 PM, Albert Jornet Puig <alb...@ic...>
wrote:

> Hi all,
>
> Does someone knows if pflexible is able to read flexpart v9.02 outputs?
>   If that is the case, from which version is possible?
>
> Thanks
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> _______________________________________________
> Pflexible-users mailing list
> Pfl...@li...
> https://lists.sourceforge.net/lists/listinfo/pflexible-users
>

[Pflexible-users] read flexpart outputs v9.02

[Albert J. P. <alb...@ic...>]

Hi all,

Does someone knows if pflexible is able to read flexpart v9.02 outputs?
  If that is the case, from which version is possible?

Thanks

Re: [Pflexible-users] plot accumulative with log10 for multiple levels

[John B. <jfb...@gm...>]

Hello Albert,

It would be helpful if you could describe the differences a bit more.
However, one thing I see immediately is that level=0 should give you the
total column.

Why are you breaking the plot ranges into tuples? It looks like you are
making individual plots of each levels.

Also, you are titling your plots with minlvl and not i...

So I need to know if you are trying to plot each individual model level or
an average or summation of the minlvl to maxlvl, and then I could provide
some code.

--john


On Thu, Nov 13, 2014 at 6:11 PM, Albert Jornet Puig <alb...@ic...>
wrote:

>  Hi,
>
> I am having difficulties to plot flexpart outputs with pflexible. We have
> several script in Matlab and we would like to move to python.
> For this purpose, we are make things work with pflexible if possible.
>
> As an example, I want to plot levels from 0 to 4, 5 to 15 and the last one
> accumulated and then multiply values to log10 to be able to improve
> visualitzation. I have seen  a couple of example and I managed to get the
> code found below but it is not producing the same outputs as in Matlab.
>
> lvl_values = [(0,4),(5,15),(16,17)]
>
> data = H.C[(0,0)]
> data_range=[-1, 4]
>
> for minlvl, maxlvl in lvl_values:
>     TC = None
>     for i in range(minlvl, (maxlvl)):
>         print i
>         TC = pf.plot_at_level(H, H.C[(0,0)], level=i, map_region='Europe',
> overlay=True, FIGURE=TC)
>         filename = '%s_tc_%s_%d_%d_overlay.png' % (data.species,
> data.timestamp, minlvl, maxlvl)
>         TC.fig.savefig(filename)
>
>
> I also checked source code but it is not clear to me a the moment.
>
> Does somebody have any idea or any way to improve the code?
>
> Thanks
>
>
> ------------------------------------------------------------------------------
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
> _______________________________________________
> Pflexible-users mailing list
> Pfl...@li...
> https://lists.sourceforge.net/lists/listinfo/pflexible-users
>
>

[Pflexible-users] plot accumulative with log10 for multiple levels

[Albert J. P. <alb...@ic...>]

Hi,

I am having difficulties to plot flexpart outputs with pflexible. We 
have several script in Matlab and we would like to move to python.
For this purpose, we are make things work with pflexible if possible.

As an example, I want to plot levels from 0 to 4, 5 to 15 and the last 
one accumulated and then multiply values to log10 to be able to improve 
visualitzation. I have seen  a couple of example and I managed to get 
the code found below but it is not producing the same outputs as in Matlab.

    lvl_values = [(0,4),(5,15),(16,17)]

    data = H.C[(0,0)]
    data_range=[-1, 4]

    for minlvl, maxlvl in lvl_values:
         TC = None
         for i in range(minlvl, (maxlvl)):
             print i
             TC = pf.plot_at_level(H, H.C[(0,0)], level=i,
    map_region='Europe', overlay=True, FIGURE=TC)
             filename = '%s_tc_%s_%d_%d_overlay.png' % (data.species,
    data.timestamp, minlvl, maxlvl)
             TC.fig.savefig(filename)


I also checked source code but it is not clear to me a the moment.

Does somebody have any idea or any way to improve the code?

Thanks

Re: [Pflexible-users] pflexible example on introduction page

[Janusz M. <jan...@gm...>]

Hello Everyone,

I have been trying to get pflexible to run for almost a day and a half now
according to the example set out in:

http://pflexible.readthedocs.org/en/latest/getting_started.html

unsuccessfully unfortuantely. Could someone please help me with this...

This is what I did..

wget http://niflheim.nilu.no/~burkhart/sharing/pflexible_testdata.tgz (1 GB)

To start off I noticed that the above test data is not available... so I
downloaded

wget http://niflheim.nilu.no/~burkhart/sharing/pflexpart_testdata.tgz (1.7 GB)

I tried the following instructions with the Pure python version and the
FortFlex module:


import pflexible as pf

H = pf.Header('/home_local/jmichalik/example_data')

FD = pf.read_grid(H,time_ret=0,nspec_ret=0)


D=H.fill_backward(nspec=(0,1))

T = pf.read_trajectories(H)

for s,k in H.D:
   data = H.D[(s,k)]
   TC = pf.plot_totalcolumn(H,data,region='POLARCAT',FIGURE=TC)
   TC = pf.plot_trajectory(H,T,k,FIGURE=TC)

With Pure Python I get the following error after the for loop:

Traceback (most recent call last):


  File "<input>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pflexible-0.9.0-py2.7-linux-x86_64.egg/pflexible.py",
line 4667, in __ge
tattr__
    return self[attr]


KeyError: 'D'

With the FortFlex module I get the following error after

getting grid for:  ['20070202150000']
Assumed V8 Flexpart
720 180 3 0 0 0 1 88
20070202150000
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pflexible-0.9.0-py2.7-linux-x86_64.egg/pflexible.py",
line 1861, in read
_grid
    return readgridV8(H, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pflexible-0.9.0-py2.7-linux-x86_64.egg/pflexible.py",
line 2066, in read
gridV8
    OPS.scaledepo, OPS.scaleconc, H.decayconstant, npspec_int)
TypeError: Required argument 'scaleconc' (pos 11) not found

I did try scaleconc=1 as per issue 14 in bitbucket but I still get the same
error.

Any help is appreciated, I need to be able to postprocess binary Flexpart
data...

Thank you in advance.

Best,
Janusz

[Pflexible-users] commit/pflexible: jfburkhart: Refactoring of read_header. Cleaned up, and minor bug fixes. Added 'flexpart' field that captures the version string (13S) from the header.

[Bitbucket <com...@bi...>]

1 new commit in pflexible:


https://bitbucket.org/jfburkhart/pflexible/changeset/680cdb48ca74/
changeset:   680cdb48ca74
user:        jfburkhart
date:        2012-12-14 01:57:57
summary:     Refactoring of read_header. Cleaned up, and minor bug fixes. Added 'flexpart' field that captures the version string (13S) from the header.
affected #:  1 file

Repository URL: https://bitbucket.org/jfburkhart/pflexible/

--

This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.

Re: [Pflexible-users] Trivial bug in pflexible.py, and questions about bug reporting

[John B. <jfb...@gm...>]

Don,

First off. I think using bitbucket issue tracker is probably best. I should
look into whether I could forward those emails on to this list however...
that could be a good idea.

I chose to use the sourceforge.net mail list, as it was a pretty easy
solution.

I'll have to find some time to look into this bug (not 'trivial').. but it
won't likely be before the weekend.

Thank you for pointing it out!

--john


On Mon, Oct 29, 2012 at 3:11 AM, Don Morton <Don...@al...> wrote:

> Hello, this is an update to the nested flag issue.  The problem is more
> bizzare than I first reported, and I have updated the bitbucket issues.
>  Here, I copy/paste from the bitbucket report.  Again, if these is a
> preferred way for me to report such things, I will gladly abide :)
>
> Thanks,
>
> Don
>
> ===========================================================
>
> The nesting option for the read header methods is misleading, wrong and
> inconsistent.
> The documentation within pflexible.py for the various methods says to use
> a "nest=0" or "nest=1" argument to specify nesting. However, I have found
> that if "nest=0" or "nest=1" is specified, then it will always flag as no
> nesting, but if "nested=0" or "nested=1" is specified, then it will always
> flag as nesting.
>
> A sample code follows, along with output, to demonstrate the issue, is
> inserted in Comment section.
>
>
>
>
> #!/usr/bin/env python
>
> FLEXPARTOutputDir =
> '/home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output'
>
> # From NILU pflexible project
> # Users will need to add the location of pflexible to $PYTHONPATH
> import pflexible as pf
>
> # Read the general header information common to all files
> H = pf.Header(FLEXPARTOutputDir, nested=1)
> isNested = H.nested  # 0 = not nested, 1 = nested
> print 'isNested: ' + str(isNested)
>
> H = pf.Header(FLEXPARTOutputDir, nested=0)
> isNested = H.nested  # 0 = not nested, 1 = nested
> print 'isNested: ' + str(isNested)
>
> H = pf.Header(FLEXPARTOutputDir, nest=1)
> isNested = H.nested  # 0 = not nested, 1 = nested
> print 'isNested: ' + str(isNested)
>
> H = pf.Header(FLEXPARTOutputDir, nest=0)
> isNested = H.nested  # 0 = not nested, 1 = nested
> print 'isNested: ' + str(isNested)
>
>
>
>
>
>
> The resulting output is:
>
> Header read:
> /home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output/header_nest
> isNested: 1 Header read:
> /home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output/header_nest
> isNested: 1 Header read:
> /home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output/header
> isNested: 0 Header read:
> /home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output/header
> isNested: 0
>
> --
> Voice:  +1 907 450 8679
> Arctic Region Supercomputing Center
> http://weather.arsc.edu/
> http://people.arsc.edu/~morton/ <http://www.arsc.edu/%7Emorton/>
>
>
>
> ------------------------------------------------------------------------------
> The Windows 8 Center - In partnership with Sourceforge
> Your idea - your app - 30 days.
> Get started!
> http://windows8center.sourceforge.net/
> what-html-developers-need-to-know-about-coding-windows-8-metro-style-apps/
> _______________________________________________
> Pflexible-users mailing list
> Pfl...@li...
> https://lists.sourceforge.net/lists/listinfo/pflexible-users
>
>

Re: [Pflexible-users] Use of pflexible 0.9 to access basic 2D fields

[John B. <jfb...@gm...>]

Don, Delia, and others who may be interested.

First off, Don. Thank you for the feedback. It is greatly appreciated.

pflexible is something I've been building up primarily for my own research
needs at NILU. I've tried to get others to use it, but it seems most are
satisfied with their Matlab or otherwise solutions. To my knowledge, not
that many are actually using pflexible yet (maybe I'm wrong, and simply
just not aware...).

However, with the new projects at NILU (VAST and otherwise), I believe
there may be some interest in further development of pflexible.

There are some (ahem, probably many...) inconsistencies, and the entire
code base is in need of a 'refactoring'. The key pieces of which, you are
discovering.

I think the key is for those involved with VAST to make the decision
whether pflexible will be used. If so, we should discuss what key changes
are required for it to work more consistently. Certainly some tests should
be added to the code.

In the pflexible release, I tried to remove some features and 'step back',
but I still think we may need to go back further ;)

Some of my data handling may not be optimal, and we may want to think of
improved ways to deal with it.

Certainly, I'm open to feedback and ideas!

--john




On Sun, Oct 28, 2012 at 4:44 PM, Don Morton <Don...@al...> wrote:

> Dear Colleagues (Delia Arnold cc'd, because I don't know if she's on this
> list):
>
> Delia and I have been working together to create some easy to use scripts
> for FLEXPART outputs (also for SRS files generated by any model).  To date,
> we have been using someone's flex81.py module, but prefer to use something
> supported by the FLEXPART community, and perhaps something cleaner to use
> in the long run.  So, we have started exploring the recent pflexible v0.9.
>  Our immediate interest is simply in extracting the data from the FLEXPART
> output files (as we already do with the flex81.py module) and then use our
> own matplotlib routines for plotting.
>
> First, please let me offer my thanks to those of you who put this together
> over the summer.  I was trying to use v0.2 earlier, and although I got it
> working through some hacking, it wasn't pretty.  This version seems to be
> much cleaner and, what I really love is that if the Fortran shared lib
> isn't available, it automagically uses the native python binary class for
> extraction.
>
> So now the questions - I have sample script and stdout available in
>
> http://weather.arsc.edu/Miscellaneous/pflexibleTry1/
>
>
> I'm a relaitvely new user of FLEXPART, so there may be lots of underlying
> detail I'm ignorant of, but I find that if I want to pull out a basic
> concentration grid for processing and plotting, the following will not work:
>
> FD = pf.read_grid(H, date=theTimeStamp)
> print FD.shape
>
> Instead, I need to actually index the whole structure by a tuple, as
> follows:
>
> theTuple = (0, theTimeStamp)
> theGrid = FD[theTuple]
> print 'Shape: ' + str(theGrid.shape)
>
> This is fine, but it's not clear to me what the "0" in the tuple actually
> indexes.  Is it a species number?
>
> On the other hand, I can easily extract the grids for dry and wet
> deposition without passing in any kind of tuple:
>
>
> WD = pf.read_grid(H, date=theTimeStamp, getwet=True)
> print 'Shape: ' + str(WD.shape)
>
> My basic question is whether I should expect this inconsistency between
> extracting concentration (or the counterpart in backward mode) and
> deposition grids.  Obviously, we can work with it.
>
> Also, it's not clear to me whether the get_slabs() routine has been
> thoroughly checked out, and whether I might be doing something wrong.  If I
> try the following (this is also in the sample script I pointed you to):
>
> FD = pf.read_grid(H, date=theTimeStamp)
> ###########
> # The call to get_slabs() fails - there is a statement in there,
> # "grid_shape = G.shape" in which the shape attribute is not recognized.
> #grid = pf.get_slabs(H, FD)
>
> Then I run into the same error I ran into when trying to access FD as a
> grid, rather than a structure with a grid in it.  It seems that, in this
> case, maybe get_slabs() itself would need to form that tuple first, and
> then grab the slab.
>
> ---------------------------------------
>
> Thank you for your time.  As we move ahead with this, we would be happy to
> create simple "HOWTO's" that demonstrate the use of pflexible in
> "extraction" operations, and will even contribute our own scripts for
> plotting once they're a little more mature.  You can see one example at
>
> http://weather.arsc.edu/Miscellaneous/FLEXPART-FairbanksFire.png
>
> First, we just want to make sure we're using pflexible as intended :)
>
> Best Regards,
>
> Don Morton and Delia Arnold
>
> --
> Voice:  +1 907 450 8679
> Arctic Region Supercomputing Center
> http://weather.arsc.edu/
> http://people.arsc.edu/~morton/ <http://www.arsc.edu/%7Emorton/>
>
>
>
> ------------------------------------------------------------------------------
> WINDOWS 8 is here.
> Millions of people.  Your app in 30 days.
> Visit The Windows 8 Center at Sourceforge for all your go to resources.
> http://windows8center.sourceforge.net/
> join-generation-app-and-make-money-coding-fast/
> _______________________________________________
> Pflexible-users mailing list
> Pfl...@li...
> https://lists.sourceforge.net/lists/listinfo/pflexible-users
>
>

Re: [Pflexible-users] Trivial bug in pflexible.py, and questions about bug reporting

[Don M. <Don...@al...>]

Hello, this is an update to the nested flag issue.  The problem is more
bizzare than I first reported, and I have updated the bitbucket issues.
 Here, I copy/paste from the bitbucket report.  Again, if these is a
preferred way for me to report such things, I will gladly abide :)

Thanks,

Don

===========================================================

The nesting option for the read header methods is misleading, wrong and
inconsistent.
The documentation within pflexible.py for the various methods says to use a
"nest=0" or "nest=1" argument to specify nesting. However, I have found
that if "nest=0" or "nest=1" is specified, then it will always flag as no
nesting, but if "nested=0" or "nested=1" is specified, then it will always
flag as nesting.

A sample code follows, along with output, to demonstrate the issue, is
inserted in Comment section.




#!/usr/bin/env python

FLEXPARTOutputDir =
'/home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output'

# From NILU pflexible project
# Users will need to add the location of pflexible to $PYTHONPATH
import pflexible as pf

# Read the general header information common to all files
H = pf.Header(FLEXPARTOutputDir, nested=1)
isNested = H.nested  # 0 = not nested, 1 = nested
print 'isNested: ' + str(isNested)

H = pf.Header(FLEXPARTOutputDir, nested=0)
isNested = H.nested  # 0 = not nested, 1 = nested
print 'isNested: ' + str(isNested)

H = pf.Header(FLEXPARTOutputDir, nest=1)
isNested = H.nested  # 0 = not nested, 1 = nested
print 'isNested: ' + str(isNested)

H = pf.Header(FLEXPARTOutputDir, nest=0)
isNested = H.nested  # 0 = not nested, 1 = nested
print 'isNested: ' + str(isNested)






The resulting output is:

Header read:
/home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output/header_nest
isNested: 1 Header read:
/home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output/header_nest
isNested: 1 Header read:
/home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output/header
isNested: 0 Header read:
/home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output/header
isNested: 0

-- 
Voice:  +1 907 450 8679
Arctic Region Supercomputing Center
http://weather.arsc.edu/
http://people.arsc.edu/~morton/ <http://www.arsc.edu/%7Emorton/>

[Pflexible-users] Trivial bug in pflexible.py, and questions about bug reporting

[Don M. <Don...@al...>]

First, Delia, if you're not a member of this group, would you please join
so I don't have to remember to keep cc'ing you?! :)  I'll send you listserv
info later.

Before the bug report - I'm not clear on how development is being performed
for this software.  Would you prefer that issues be reported via this list,
or the issues portion of the bitbucket repository (I note that nobody but
me has used this for the pflexible software yet :)).  Or, would you even
prefer that we make our own changes to the source and submit them to
bitbucket version control for consideration?

OK, here's the minor bug.....

In trying to adapt some of Delia's "nesting" plots to pflexible data
extraction, I have found a discrepancy in the documentation for the read
header routines.  They say to specify that you wanted to use nested output
by inserting the "nest=" flag in function call, but, in fact, it seems like
it should be "nested=".  Here is what I wrote up on the bitbucket site:

---------------------------
The documentation for the various read header methods says to use "nest="
to specify nesting or no nesting.  However, the correct use should be
"nested="

The following code demonstrates that, if using "nest=1", the nested key
returned with the header dictionary is 0, but if using "nested=1" it is
correct:


#!/usr/bin/env python

FLEXPARTOutputDir =
'/home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output'

# From NILU pflexible project
# Users will need to add the location of pflexible to $PYTHONPATH
import pflexible as pf

# Read the general header information common to all files
H = pf.Header(FLEXPARTOutputDir, nested=1)
isNested = H.nested  # 0 = not nested, 1 = nested
print 'isNested: ' + str(isNested)

H = pf.Header(FLEXPARTOutputDir, nest=1)
isNested = H.nested  # 0 = not nested, 1 = nested
print 'isNested: ' + str(isNested)




The output of the above segment (which I forgot to post into the bitbucket
issues) is:

Header read:
/home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output/header_nest
isNested: 1
Header read:
/home/morton/Research/CTBTO/FLEXPART_Benchmarking/NestExperiments/OutputNestOnlyWithDryWetConv/output/header
isNested: 0




Thanks,

Don Morton
-- 
Voice:  +1 907 450 8679
Arctic Region Supercomputing Center
http://weather.arsc.edu/
http://people.arsc.edu/~morton/ <http://www.arsc.edu/%7Emorton/>

[Pflexible-users] Use of pflexible 0.9 to access basic 2D fields

[Don M. <Don...@al...>]

Dear Colleagues (Delia Arnold cc'd, because I don't know if she's on this
list):

Delia and I have been working together to create some easy to use scripts
for FLEXPART outputs (also for SRS files generated by any model).  To date,
we have been using someone's flex81.py module, but prefer to use something
supported by the FLEXPART community, and perhaps something cleaner to use
in the long run.  So, we have started exploring the recent pflexible v0.9.
 Our immediate interest is simply in extracting the data from the FLEXPART
output files (as we already do with the flex81.py module) and then use our
own matplotlib routines for plotting.

First, please let me offer my thanks to those of you who put this together
over the summer.  I was trying to use v0.2 earlier, and although I got it
working through some hacking, it wasn't pretty.  This version seems to be
much cleaner and, what I really love is that if the Fortran shared lib
isn't available, it automagically uses the native python binary class for
extraction.

So now the questions - I have sample script and stdout available in

http://weather.arsc.edu/Miscellaneous/pflexibleTry1/


I'm a relaitvely new user of FLEXPART, so there may be lots of underlying
detail I'm ignorant of, but I find that if I want to pull out a basic
concentration grid for processing and plotting, the following will not work:

FD = pf.read_grid(H, date=theTimeStamp)
print FD.shape

Instead, I need to actually index the whole structure by a tuple, as
follows:

theTuple = (0, theTimeStamp)
theGrid = FD[theTuple]
print 'Shape: ' + str(theGrid.shape)

This is fine, but it's not clear to me what the "0" in the tuple actually
indexes.  Is it a species number?

On the other hand, I can easily extract the grids for dry and wet
deposition without passing in any kind of tuple:


WD = pf.read_grid(H, date=theTimeStamp, getwet=True)
print 'Shape: ' + str(WD.shape)

My basic question is whether I should expect this inconsistency between
extracting concentration (or the counterpart in backward mode) and
deposition grids.  Obviously, we can work with it.

Also, it's not clear to me whether the get_slabs() routine has been
thoroughly checked out, and whether I might be doing something wrong.  If I
try the following (this is also in the sample script I pointed you to):

FD = pf.read_grid(H, date=theTimeStamp)
###########
# The call to get_slabs() fails - there is a statement in there,
# "grid_shape = G.shape" in which the shape attribute is not recognized.
#grid = pf.get_slabs(H, FD)

Then I run into the same error I ran into when trying to access FD as a
grid, rather than a structure with a grid in it.  It seems that, in this
case, maybe get_slabs() itself would need to form that tuple first, and
then grab the slab.

---------------------------------------

Thank you for your time.  As we move ahead with this, we would be happy to
create simple "HOWTO's" that demonstrate the use of pflexible in
"extraction" operations, and will even contribute our own scripts for
plotting once they're a little more mature.  You can see one example at

http://weather.arsc.edu/Miscellaneous/FLEXPART-FairbanksFire.png

First, we just want to make sure we're using pflexible as intended :)

Best Regards,

Don Morton and Delia Arnold

-- 
Voice:  +1 907 450 8679
Arctic Region Supercomputing Center
http://weather.arsc.edu/
http://people.arsc.edu/~morton/ <http://www.arsc.edu/%7Emorton/>

Re: [Pflexible-users] reading header

[John B. <jfb...@gm...>]

Marta,

Could you please send me the header file you are trying to read.

Thank you,
john



On 07/10/2012 02:49 PM, Martinez-Camara Marta wrote:
> Hello John,
>
> These are the messages:
>
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "pflexible.py", line 4796, in __init__
>      raise IOError('Could not set header variables. Does the path exist?')
> IOError: Could not set header variables. Does the path exist?
>
> It seems that it doesn't recognize the header...
>
> ________________________________________
> De: John Burkhart [jfb...@gm...]
> Enviado el: lunes, 09 de julio de 2012 19:59
> Para: Martinez-Camara Marta
> Cc: pfl...@li...
> Asunto: Re: [Pflexible-users] reading header
>
> Marta,
>
> Could you please provide some of the error messages?
>
> Thanks,
> john
>
> On 07/09/2012 05:56 PM, Martinez-Camara Marta wrote:
>> Hi,
>>
>> I already installed pflexible, and pf.Header  read correctly the header
>> of the test files. The problem now is to read the header from the output
>> of my flexpart simulations. pflexible can't read the header file. I am
>> using flexpart v8.2 and it seems that the simulations finished
>> successfully. Anyone has any idea of what could happen?
>>
>>    Thanks a lot in advance.
>>
>> Marta
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>
>>
>>
>> _______________________________________________
>> Pflexible-users mailing list
>> Pfl...@li...
>> https://lists.sourceforge.net/lists/listinfo/pflexible-users
>>
>

Re: [Pflexible-users] reading header

[John B. <jfb...@gm...>]

Marta,

Could you please provide some of the error messages?

Thanks,
john

On 07/09/2012 05:56 PM, Martinez-Camara Marta wrote:
> Hi,
>
> I already installed pflexible, and pf.Header  read correctly the header
> of the test files. The problem now is to read the header from the output
> of my flexpart simulations. pflexible can't read the header file. I am
> using flexpart v8.2 and it seems that the simulations finished
> successfully. Anyone has any idea of what could happen?
>
>   Thanks a lot in advance.
>
> Marta
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>
>
>
> _______________________________________________
> Pflexible-users mailing list
> Pfl...@li...
> https://lists.sourceforge.net/lists/listinfo/pflexible-users
>

[Pflexible-users] reading header

[Martinez-Camara M. <mar...@ep...>]

Hi,

I already installed pflexible, and pf.Header  read correctly the header of the test files. The problem now is to read the header from the output of my flexpart simulations. pflexible can't read the header file. I am using flexpart v8.2 and it seems that the simulations finished successfully. Anyone has any idea of what could happen?

 Thanks a lot in advance.

Marta

Re: [Pflexible-users] backward.py

[John B. <jfb...@gm...>]

Francesco,

I need to update the tutorials to try and create something more useful 
and relevant to the new API. However, please use the source code. I've 
done what I can in general to document the different functions.

In generaly, my workflow is something like this:

H = pf.Header('/path/to/output')
H.fill_backwards()

Then the H object will have a 'C' attribute. The 'C' attribute is a 
dictionary keyed by tuples of the release time and species.

E.g.

data_00 = H.C[(0, 0)]
data_00.keys()
['itime',
  'min',
  'timestamp',
  'gridfile',
  'rel_i',
  'shape',
  'spec_i',
  'grid',
  'max',
  'species',
  'slabs']

The key things are the 'grid' which is an array of all the data for that 
release, and the 'slabs', which is a dictionary again.

data_00.slabs.keys()
[0, 1, 2, 3]

data_00.grid.shape
(360, 180, 3)

Note that each slab is a reference to the grid array for each vertical 
level.

e.g.
data_00.slabs[1] is the same as data_00.grid[:,:,0]

data_00.slabs[0] is the total column (so np.sum(data_00.grid, axis=2))

Once you have that information, you can start to play around with the 
plotting routines. In general, they expect the data_00 object.

so:

FIG = pf.plot_totalcolumn(H, data_00)

Should work...

--john




On Fri 06 Jul 2012 11:12:26 AM CEST, Francesco Graziosi wrote:
> Hi John,
>
> I followed your suggestion, thanks, so finaly (in flex_wrk ) the
> program can read the example data and the the grid (with the command
> read_grid).
> From this point the tutarial (0.8.1 version) said to lanch the
> backward.py, but I don't have in my directory of pflexin. What shoud I
> do for continue to elaborate the data ?
>
> Best regards,
>
> Francesco Graziosi

Re: [Pflexible-users] nilu.pflexpart 0.8.1

[John B. <jfb...@gm...>]

Francesco,

First of all, the convention for pflexible is to now simply import it 
directly:

import pflexible as pf

Your approach works fine, but I think it is easier to import it as above.

Regarding your error, it is due to a missing requirement. You need to 
have the matplotlib basemap routines installed. On ubuntu it would be 
something like:

sudo apt-get install python-mpltoolkits.basemap

Otherwise, you'll have to install it via the instructions at the 
packages site:

http://matplotlib.github.com/basemap/users/installing.html

--john

On 07/04/2012 06:41 PM, Francesco Graziosi wrote:
> Hi John,
>
> thanks for the replay.
> I have istalled pflexible on my pc, I run the setup by typing "python
> setup.py install", and it work fine.
> I setting the variable pythonpath : "export
> PYTHONPATH=$HOME:/home/johnny/pflexible" ; and then when I try to
> customize all command for the 0.8 version by running following command :
>   >>> from pflexible import pflexible as pf
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "pflexible/__init__.py", line 1, in <module>
>      from pflexible import *
>    File "pflexible/pflexible.py", line 80, in <module>
>      import mapping as mp
>    File "pflexible/mapping.py", line 28, in <module>
>      from matplotlib.toolkits import Basemap
> ImportError: cannot import name Basemap.
>
> I obtain the error above, but I have already installed Basemap.py in :
> /usr/lib64/python2.5/site-packages/matplotlib/toolkits/basemap/
>
> can you help me to find the mistake ?
>
> Best regards,
>
> Francesco Graziosi
>
>

[Pflexible-users] Welcome

[John B. <jfb...@gm...>]

Welcome everyone to the pflexible-users list.

I'm happy to announce that I've finally found some time to put together 
a more suitable version of my software to have online and for everyone 
to collective work on.

The new software is called pflexible. At present, there is little 
difference from the earlier routines, but the goal is to make future 
modifications more generic to handle multiple types of transport models.

The software is presently hosted at bitbucket, and you can find out more 
here: https://bitbucket.org/jfburkhart/pflexible

This maillist has been set up through sourceforge, despite the code 
hosting at bitbucket. Please submit your issues to bitbucket or by 
emailing to this list.

My goal is to encourage participation, and to solicit contributions back 
to the code. If you've made some additions that you think would be of 
interest, please let us all know.

Over the next few days I'll work to put together a more detailed 
tutorial outlining the pflexible API and how to work with model data.

--john