Alpha 19

Alpha 19 is out. Changes:

Added new functions to pyp_network for identifying ancestors, identifying descendants, identifying immediate family members (defined as parents and offspring, and does not include siblings), and identifying influential progeny based on the number of progeny they produce.

Fixed to pyp_network/ped_to_graph() so that graphs are now ordered in the correct direction. Before this fix, the graph was ordered backwards, so that offspring preceded parents in the graph.

It looks like most of the stuff on pyp_demog can be moved into pyp_reporting. I'm going to think about this. The demographics reports may be moved. The code would be a lot cleaner if those reports were queries against the database rather than formed from walking the pedigree.

Added a new option, oid, to pyp_network/ped_to_graph().

There is now a simple document history in the GUI. The "View log" feature in pyp_gui has been rewritten so that it displays the logfile associated with the loaded pedigree rather than presenting the user with a file selection dialogue.

I decided to back out of using Wax and just stick with plain old wxPython for the GUI. After messing with it for a couple of days I have decided that (i) Wax has a lot of potential and (ii) it's just not quite there yet in terms of functionality and, more importantly, documentation. If I am going to spend hours in the wxPython docs anyway I'll just write with that tool. Despite the much-despised IDs that I have to assign, pass, and debug.

I have added a new PyPedal file, pyp_gui_metrics, which contains convenience functions for entries in the Metrics menu to reduce repetitive code in pyp_gui.

pyp_metrics/a_effective_founders_lacy() and pyp_metrics/effective_founders_lacy() now return a dictionary that contains summary statistics, including the effective founder number. This change breaks at least one example program and may break at least one unit test.

Added a new option, log_long_filenames, that indicates whether or not logfile names should include datestamps. The default is to not use them.

Added pyp_io/summary_inbreeding() which returns a string representation of the data contained in the 'metadata' dictionary contained in the output dictionary returned by pyp_nrm/pyp_inbreeding().

I've been messing with Wax and pyp_gui today. I have added a new PyPedal file, pyp_gui_graphs, to package the classes subclassed from Dialog, such as PyPedalGraphDialogInbreeding(). Oh, sure, I tried to have a single PyPedalGraphDialog() class that I could use for any graph by passing titles and filenames, but I could not get it to work correctly. If anyone wants to try and fix the dreaded-and-deadly "TypeError: __init__() got multiple values for keyword argument 'pgdTitle'" problem they are welcome to it. For now it is easiest to just have a subclass for each graph that I am going to draw.

Cleanups in pyp_db so that messages are only printed to STDOUT when the 'messages' option is set to 'verbose'.

pyp_nrm/inbreeding() now returns a dictionary that contains two dictionaries: 'metadata', which contains summary statistics for the CoI in the pedigree, and 'fx', which contains the actual CoI for each animal. This change breaks at least one example program and may break at least one unit test.

Added a new function, plot_line_xy(), to pyp_graphics. plot_line_xy produces a plot of the values in an input dictionary by levels of the keys in the dictionary. It can take the dictionary returned by pyp_reports/meanMetricBy() and produce a simple chart from it.

Cleaned up logging in pyp_db/loadPedigreeDatabase().

Automatically-generated logfile names now include datestamps.

Added a new function, pyp_datestamp(), to pyp_utils which returns a datestamp of the form YYYYMMDDHHMMSS.

Added a new option, log_ped_lines, that indicates how many lines of the pedigree file should be written to the logfile for debugging. The default is zero. Any value other than a non-negative integer is set to 0 and a warning is written to the log.

pyp_newclasses/NewPedigree.save() has been modified so that it will save CoI whenever they have been computed for a pedigree.

pyp_nrm/inbreeding_vanraden() now writes some summary statistics to the logfile, and the screen when requested, whenever a round of processing included at least 1% of the animals in the pedigree.

Modified pyp_nrm/inbreeding_vanraden() to stop overwriting known CoI in the fx{} dictionary unnecessarily. This may not buy much in terms of efficiency, but why make lots of writes when you don't need to?

pyp_nrm/inbreeding() now calls pyp_nrm/inbreeding_vanraden() for pedigrees of 1,000 animals or more rather than 10,000.

Modified pyp_nrm/inbreeding() to set the f_computed flag before returning when it has been successful.

Added a new option, f_computed, that indicates whether or not CoI have been computed for animals in the current pedigree. If the pedigree format string includes 'f' this will be set to 1; it is also set to 1 on a successful return from pyp_nrm/inbreeding().

Added some code in pyp_io/unpickle_pedigree() to prevent a the addition of a .pkl extension to filenames that already have extensions. This solves the dreaded-and-deadly Double Pickle Problem.

pyp_io/pickle_pedigree() and pyp_io/unpicle_pedigree() seem to work on Python 2.4.1 compiled from source for 64-bit processors. More investigation is needed to see why this works on the 366 but not the 440.

Fixed a typo in an option name in pyp_newclasses/NewAMatrix().

Added a new module, pyp_network, for experimenting with the NetworkX graph library for Python (https://networkx.lanl.gov/). This provides a way to represent pedigrees (for example) as algebraic graphs and may provide a nice way to get around some of the problems I am having trying to code routines such as pyp_metrics/num_traced_gens().

pyp_utils/assign_offspring() now checks the pedigree format string to see if animal sexes were provided. If they were, offspring are assigned to their parents' correct sons or daghters list rather than to the unknowns list.

pyp_nrm/fast_a_matrix() and pyp_nrm/fast_a_matrix_r() now take an optional argument, 'method', that indicates whether a dense ('dense') or sparse ('sparse') matrix should be used for storing the NRM. The sparse matrix support is provided by PySparse (http://pysparse.sourceforge.net/), and there are source code and binary (Python 2.4 for Windows 32) versions available for download. This should allow PyPedal to manipulate larger NRM than is possible with the dense matrices provided by Numarray.

Several routines that did not return any values before now return some result (dictionaries of summary statistics, 0/1 on failure/success, etc.). In addition, many routines were not guaranteed to return the value(s) specified in the docstrings. This has been fixed with the judicious use of try/except blocks and default values, such as initially empty lists and dictionaries. The end result should be more stability from the user's perspective due to fewer instances of behavior contrary to the documentation.

Almost all routines now write messages to the logfile when they are entered and exited. Exceptions are made for, e.g., pad_id() which would result in an entry being made for each animal in the pedigree. try/except blocks are used to make sure that things don't fail if no logfile has been created, for example if someone is using an odd PyPedal routine or two out-of-context as it were.

Cleaned-up the output-to-file code. There were lots of places where output was created as a string in one line and written to an output file on a second line. This needless separation of powers was eliminated.

Lots of work has been done on the documentation. LaTeX is the One True Way. Repent, non-believers, or Many Bad Things will befall you in the next life!

Added a new pedigree format code, Z, that can be used to skip columns when reading a pedigree file.

pyp_newclasses/NewPedigree.preprocess() now checks to see if the datalines read from the pedigree file contain the same number of columns as specified in the pedigree format string. If there is a mismatch an error message is written to the console and PyPedal halts.

Fixed a possible bug in pyp_nrm/inbreeding_vanraden() caused by recent change in the arguments expected by pyp_nrm/fast_a_matrix().

Added a new function, unpickle_pedigree(), to pyp_io that unserializes (unpickles) a pedigree using the cPickle module. It is used to load a pedigree from a pickled file created with the pickle_pedigree() function.

Added a new function, pickle_pedigree(), to pyp_io that serializes (pickles) pedigrees using the cPickle module. Unfortunately, the call to the dump function in cPickle is throwing an exception I cannot decipher. I have e-mailed David S. to bug him for ideas.

More work on pyp_db and pyp_reports.

Minor bugfixes in pyp_nrm.

Some minor work on pyp_db and pyp_reports.

Added a new function, string_to_table_name, to pyp_utils(). It is used to produce strings that are safe for use as SQLite table names.

Added two new options, database_name and dbtable_name, for support of the new SQLite/pyp_reports features.

Added new modules pyp_db and pyp_reports. pyp_db is an optional module that can use SQLite (http://www.sqlite.org/) and the Python bindings to SQLite, pysqlite (http://initd.org/tracker/pysqlite) to store (and retrieve) pedigrees in a simple database. pyp_reports uses pyp_db to prepare summary reports, produce figures, etc. Neither of these modules is much use without SQLite. The thinking here is that there are lots of reports that are pretty trivial to produce with SQL, but which require lots of looping over PyPedal pedigrees. So I decided to give this a whirl. SQLite, "a small C library that implements a self-contained, embeddable, zero-configuration SQL database engine" is in the public domain.

Posted by John B. Cole 2005-10-25

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks