Menu
▾
▴
pyphant-devel — Discussions about the inner workings and future developments.
You can subscribe to this list here.
| 2007 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(13) |
Nov
(50) |
Dec
(40) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2008 |
Jan
(49) |
Feb
(72) |
Mar
(18) |
Apr
(27) |
May
(40) |
Jun
(52) |
Jul
(26) |
Aug
(8) |
Sep
(12) |
Oct
(26) |
Nov
(13) |
Dec
(14) |
| 2009 |
Jan
(13) |
Feb
(7) |
Mar
(8) |
Apr
(11) |
May
(4) |
Jun
(2) |
Jul
(7) |
Aug
(1) |
Sep
(3) |
Oct
|
Nov
(1) |
Dec
(1) |
| 2010 |
Jan
(1) |
Feb
(8) |
Mar
(3) |
Apr
(9) |
May
(16) |
Jun
(8) |
Jul
(7) |
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
| 2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(6) |
Dec
(12) |
|
From: <zk...@us...> - 2010-07-07 10:24:33
|
Revision: 693
http://pyphant.svn.sourceforge.net/pyphant/?rev=693&view=rev
Author: zklaus
Date: 2010-07-07 10:24:27 +0000 (Wed, 07 Jul 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Fix: Fixes #38 for single dependencies.
Enh: Added logging to TestFMFLoader.py
Fix: This commit fixes #37
Enh: Adapted thickness correction as suggested by Kristian.
Fix: This commit fixes #45
Modified Paths:
--------------
trunk/src/pyphant/pyphant/quantities/ParseQuantities.py
trunk/src/workers/OSC/OSC/OscThicknessCorrection.py
trunk/src/workers/fmfile/fmfile/FMFLoader.py
trunk/src/workers/fmfile/fmfile/tests/TestFMFLoader.py
Modified: trunk/src/pyphant/pyphant/quantities/ParseQuantities.py
===================================================================
--- trunk/src/pyphant/pyphant/quantities/ParseQuantities.py 2010-07-04 21:19:39 UTC (rev 692)
+++ trunk/src/pyphant/pyphant/quantities/ParseQuantities.py 2010-07-07 10:24:27 UTC (rev 693)
@@ -73,14 +73,22 @@
unit = Quantity(unit.encode('utf-8'))
elif FMFversion=='1.0':
unit1_0 = PhysicalQuantity(unit.encode('utf-8'))
- unit_base = Quantity(str(unit1_0.inBaseUnits()))
- unit = Quantity(str(unit1_0))
- if unit_base != unit:
- unit = unit_base
+ unit = Quantity(str(unit1_0.inBaseUnits()))
+ try:
+ unit_new = Quantity(str(unit1_0))
+ if unit_new == unit:
+ unit = unit_new
+ else:
+ _logger.warn('Usage of old unit "%s" required '
+ 'conversion to base units.' % unit1_0)
+ except:
_logger.warn('Usage of old unit "%s" required '
'conversion to base units.' % unit1_0)
- except:
- unit = float(unit)
+ except Exception, e:
+ try:
+ unit = float(unit)
+ except:
+ raise e
return unit
def parseQuantity(value,FMFversion='1.1'):
Modified: trunk/src/workers/OSC/OSC/OscThicknessCorrection.py
===================================================================
--- trunk/src/workers/OSC/OSC/OscThicknessCorrection.py 2010-07-04 21:19:39 UTC (rev 692)
+++ trunk/src/workers/OSC/OSC/OscThicknessCorrection.py 2010-07-07 10:24:27 UTC (rev 693)
@@ -71,11 +71,11 @@
t.shortname='t_c'
return t
- def perform_print_correction(self, x, y, uncorrected_t):
+ def perform_print_correction(self, x, raw_y, uncorrected_t):
t = copy.deepcopy(uncorrected_t)
- d = 1.9*y.data**2 - 19.3*y.data + 247.9
- #d *= y.unit/t.unit
- t.data = t.data + d
+ y = raw_y.data
+ d = 1.9*y**2 + 19.3*y + 49
+ t.data = t.data - d
t.longname='thickness corrected for printing'
t.shortname='t_c'
return t
Modified: trunk/src/workers/fmfile/fmfile/FMFLoader.py
===================================================================
--- trunk/src/workers/fmfile/fmfile/FMFLoader.py 2010-07-04 21:19:39 UTC (rev 692)
+++ trunk/src/workers/fmfile/fmfile/FMFLoader.py 2010-07-07 10:24:27 UTC (rev 693)
@@ -255,14 +255,14 @@
return result
def reshapeField(field):
- if field.isIndependent():
+ if field.isIndependent() or len(field.dimensions)==1:
return field
dimData = [numpy.unique(d.data) for d in field.dimensions]
dimDicts = [dict([(data, index) for index, data in enumerate(dimdata)]) \
for dimdata in dimData]
fieldData = numpy.ones([len(d) for d in dimData]) * numpy.nan
- indicess = [map(lambda x: dimDicts[index][x], dim.data) \
- for index, dim in enumerate(field.dimensions)]
+ indicess = zip(*[map(lambda x: dimDicts[index][x], dim.data) \
+ for index, dim in enumerate(field.dimensions)])
for datum, indices in zip(field.data, indicess):
fieldData[indices] = datum
newDims = [ DataContainer.FieldContainer(dimData[i],
Modified: trunk/src/workers/fmfile/fmfile/tests/TestFMFLoader.py
===================================================================
--- trunk/src/workers/fmfile/fmfile/tests/TestFMFLoader.py 2010-07-04 21:19:39 UTC (rev 692)
+++ trunk/src/workers/fmfile/fmfile/tests/TestFMFLoader.py 2010-07-07 10:24:27 UTC (rev 693)
@@ -313,8 +313,16 @@
if __name__ == "__main__":
import sys
+ import logging
+ logger = logging.getLogger('pyphant')
+ hdlr = logging.StreamHandler(sys.stderr)
+ formatter = logging.Formatter('[%(name)s|%(levelname)s] %(message)s')
+ hdlr.setFormatter(formatter)
+ logger.addHandler(hdlr)
+ logger.setLevel(logging.DEBUG)
if len(sys.argv) == 1:
unittest.main()
else:
- suite = unittest.TestLoader().loadTestsFromTestCase(eval(sys.argv[1:][0]))
+ suite = unittest.TestLoader().loadTestsFromTestCase(
+ eval(sys.argv[1:][0]))
unittest.TextTestRunner().run(suite)
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-07-04 21:19:46
|
Revision: 692
http://pyphant.svn.sourceforge.net/pyphant/?rev=692&view=rev
Author: zklaus
Date: 2010-07-04 21:19:39 +0000 (Sun, 04 Jul 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Enh: Added "numb_edge" capability to MRA.
Enh: Added Correction for printed solar cells.
Enh: DCSource Workers display #remaining matches
Enh: This commit closes #39
Enh: ImageProcessing Reduce Worker
Modified Paths:
--------------
trunk/src/pyphant/pyphant/visualizers/__init__.py
trunk/src/pyphant/pyphant/wxgui2/paramvisualization/OneLineStringField.py
trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ParamVisReg.py
trunk/src/workers/ImageProcessing/ImageProcessing/AutoFocus.py
trunk/src/workers/ImageProcessing/ImageProcessing/MarkAF.py
trunk/src/workers/ImageProcessing/ImageProcessing/__init__.py
trunk/src/workers/OSC/OSC/MRA.py
trunk/src/workers/OSC/OSC/OscThicknessCorrection.py
trunk/src/workers/tools/tools/DCSource.py
trunk/src/workers/tools/tools/FCSource.py
trunk/src/workers/tools/tools/SCSource.py
Added Paths:
-----------
trunk/src/pyphant/pyphant/visualizers/SingleValueVisualizer.py
trunk/src/workers/ImageProcessing/ImageProcessing/FeatureRatio.py
trunk/src/workers/ImageProcessing/ImageProcessing/Reduce.py
Copied: trunk/src/pyphant/pyphant/visualizers/SingleValueVisualizer.py (from rev 691, trunk/src/pyphant/pyphant/wxgui2/paramvisualization/OneLineStringField.py)
===================================================================
--- trunk/src/pyphant/pyphant/visualizers/SingleValueVisualizer.py (rev 0)
+++ trunk/src/pyphant/pyphant/visualizers/SingleValueVisualizer.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -0,0 +1,56 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2006-2008, Rectorate of the University of Freiburg
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+# * Neither the name of the Freiburg Materials Research Center,
+# University of Freiburg nor the names of its contributors may be used to
+# endorse or promote products derived from this software without specific
+# prior written permission.
+#
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
+# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+u"""
+"""
+
+__id__ = "$Id$"
+__author__ = "$Author$"
+__version__ = "$Revision$"
+# $Source$
+
+
+from pyphant.core.Connectors import (TYPE_IMAGE, TYPE_ARRAY)
+from pyphant.wxgui2.DataVisReg import DataVisReg
+import wx
+
+class SingleValueVisualizer(object):
+ name='Single Value'
+ def __init__(self, DataContainer, show=True):
+ if show:
+ value = DataContainer.data[0] * DataContainer.unit
+ print value
+ wx.MessageBox(caption=DataContainer.longname,
+ message="%s = %s" % (DataContainer.shortname,
+ value))
+
+
+DataVisReg.getInstance().registerVisualizer(TYPE_IMAGE, SingleValueVisualizer)
Modified: trunk/src/pyphant/pyphant/visualizers/__init__.py
===================================================================
--- trunk/src/pyphant/pyphant/visualizers/__init__.py 2010-06-30 11:17:29 UTC (rev 691)
+++ trunk/src/pyphant/pyphant/visualizers/__init__.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -51,4 +51,5 @@
#for module in filter(lambda file: file[-3:]=='.py', files):
# if not module == '__init__.py':
# exec 'import ' + module[:-3]
-import ImageVisualizer, Chart, KMVisualizer, ConfigurablePlot, ZStackVisualizer
+import ImageVisualizer, Chart, KMVisualizer, ConfigurablePlot, \
+ ZStackVisualizer, SingleValueVisualizer
Modified: trunk/src/pyphant/pyphant/wxgui2/paramvisualization/OneLineStringField.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/paramvisualization/OneLineStringField.py 2010-06-30 11:17:29 UTC (rev 691)
+++ trunk/src/pyphant/pyphant/wxgui2/paramvisualization/OneLineStringField.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -38,6 +38,8 @@
# $Source$
import wx
+from pyphant.core.Param import (ParamChangeExpected,
+ VisualizerChangeValue)
class OLSF(wx.TextCtrl):
def __init__(self, parent, param, validator):
@@ -46,3 +48,20 @@
def getValue(self):
return self.GetValue()
+
+class InstantOLSF(OLSF):
+ def __init__(self, parent, param, validator):
+ OLSF.__init__(self, parent, param, validator)
+ self.param = param
+ self.Bind(wx.EVT_KILL_FOCUS, self.onInput)
+ param._eventDispatcher.registerExclusiveListener(
+ self.onVCV, VisualizerChangeValue)
+
+ def onInput(self, Event=None):
+ pce = ParamChangeExpected(self.param, expectedValue=self.getValue())
+ self.param._eventDispatcher.dispatchEvent(pce)
+ if not Event is None:
+ Event.Skip()
+
+ def onVCV(self, event):
+ self.SetValue(event.value)
Modified: trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ParamVisReg.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ParamVisReg.py 2010-06-30 11:17:29 UTC (rev 691)
+++ trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ParamVisReg.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -92,6 +92,7 @@
register = [
(0, None, BoundedIntegerTextField.BoundedIntegerTextField),
(" ", None, OneLineStringField.OLSF),
+ (" ", Connectors.SUBTYPE_INSTANT, OneLineStringField.InstantOLSF),
(" ", Connectors.SUBTYPE_FILE, FileButton.FileButton),
(True, None, CheckBox.CheckBox),
(True, Connectors.SUBTYPE_INSTANT, CheckBox.InstantCheckBox),
Modified: trunk/src/workers/ImageProcessing/ImageProcessing/AutoFocus.py
===================================================================
--- trunk/src/workers/ImageProcessing/ImageProcessing/AutoFocus.py 2010-06-30 11:17:29 UTC (rev 691)
+++ trunk/src/workers/ImageProcessing/ImageProcessing/AutoFocus.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -158,7 +158,6 @@
numpy.array(numpy.gradient(image.data)))))
label_data = self.get_label_data(grad_data, zsp, zsu)
slicess = ndimage.find_objects(label_data)
- focus_data = numpy.zeros(grad_data.shape)
inclusions[zind] = {}
for label, slices in enumerate(slicess, start=1):
bigslices = [slice(max(sli.start - zsp.grow, 0),
@@ -175,8 +174,6 @@
if area < zsp.ath:
inclusions[zind][label] = InvalidInclusion()
continue
- focus_data[slices] = numpy.where(islabel, focus,
- focus_data[slices])
new_inclusion = Inclusion(slices, focus, area)
if last_label is None:
inclusions[zind][label] = new_inclusion
@@ -188,7 +185,7 @@
ld_inclusions.append(new_inclusion)
self.invalidate_unfocused(ld_inclusions)
inclusions[zind][label] = new_inclusion
- return focus_data, label_data
+ return label_data
def make_fc(self, prototype, data, unit, prefix="",
shortname=None, attributes={}, emd5_list=[]):
@@ -250,7 +247,6 @@
inclusions = {}
focused_inclusions = []
sharp_iters = len(zstack['emd5'].data)
- focus_fcs = []
label_fcs = []
zsp = ZSParams(self)
dimss = {}
@@ -259,14 +255,11 @@
for zid, emd5 in enumerate(zstack['emd5'].data):
raw_image = self.kmanager.getDataContainer(emd5)
dimss[zid] = raw_image.dimensions
- focus_data, label_data = self.autofocus(
- raw_image, zsp, zid, inclusions, label_data)
- cdata = [[raw_image, focus_data, ZSUnits.FQUANT, 'Focus_', 'f',
- self.get_attributes('FocusImage', zstack.id), focus_fcs],
- [raw_image, label_data, 1, 'Label_', 'l',
- self.get_attributes('LabelImage', zstack.id), label_fcs]]
- for data in cdata:
- self.make_fc(*data)
+ label_data = self.autofocus(raw_image, zsp, zid, inclusions,
+ label_data)
+ self.make_fc(
+ raw_image, label_data, 1, 'Label_', 'l',
+ self.get_attributes('LabelImage', zstack.id), label_fcs)
subscriber %= ((zid + 1) * 90) / sharp_iters
# Extract statistics:
for zind, inc_dict in inclusions.iteritems():
@@ -278,21 +271,21 @@
focused_inclusions.append(info)
subscriber %= 90 + ((zind + 1) * 10) / sharp_iters
# Put all results into SampleContainers:
- columnss = [zip(zvals, focus_fcs), zip(zvals, label_fcs),
+ columnss = [zip(zvals, label_fcs),
focused_inclusions]
- columnlnss = [['z-value', 'emd5'], ['z-value', 'emd5'],
+ columnlnss = [['z-value', 'emd5'],
['z-pos', 'y-pos', 'x-pos', 'diameter',
'focus', 'label', 'z-index',
'y-slice-start', 'y-slice-stop',
'x-slice-start', 'x-slice-stop']]
- columnsnss = [['z', 'e'], ['z', 'e'],
+ columnsnss = [['z', 'e'],
['z', 'y', 'x', 'd', 'f', 'l',
- 'zi', 'yt', 'yp', 'xt', 'xp']]
- longnames = ['Focus_' + zstack.longname, 'Label_' + zstack.longname,
- "Statistics_" + zstack.longname]
- shortnames = ['f', 'l', 's']
+ 'zi', 'yt', 'yp', 'xt', 'xp']]
+ longnames = ['Label_' + zstack.longname,
+ 'Statistics_' + zstack.longname]
+ shortnames = ['l', 's']
attributess = [self.get_attributes(zstype, zstack.id) \
- for zstype in ['FocusSC', 'LabelSC', 'StatisticsSC']]
+ for zstype in ['LabelSC', 'StatisticsSC']]
return [self.make_sc(*data) for data in \
zip(columnss, columnlnss, columnsnss,
longnames, shortnames, attributess)]
@@ -327,13 +320,9 @@
return lookup
@Worker.plug(Connectors.TYPE_ARRAY)
- def get_focus_sc(self, zstack, subscriber=0):
- return self.get_result_sc(zstack, 'FocusSC', 0, subscriber)
-
- @Worker.plug(Connectors.TYPE_ARRAY)
def get_label_sc(self, zstack, subscriber=0):
- return self.get_result_sc(zstack, 'LabelSC', 1, subscriber)
+ return self.get_result_sc(zstack, 'LabelSC', 0, subscriber)
@Worker.plug(Connectors.TYPE_ARRAY)
def get_statistics_sc(self, zstack, subscriber=0):
- return self.get_result_sc(zstack, 'StatisticsSC', 2, subscriber)
+ return self.get_result_sc(zstack, 'StatisticsSC', 1, subscriber)
Copied: trunk/src/workers/ImageProcessing/ImageProcessing/FeatureRatio.py (from rev 691, trunk/src/workers/ImageProcessing/ImageProcessing/__init__.py)
===================================================================
--- trunk/src/workers/ImageProcessing/ImageProcessing/FeatureRatio.py (rev 0)
+++ trunk/src/workers/ImageProcessing/ImageProcessing/FeatureRatio.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -0,0 +1,68 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2008, Rectorate of the University of Freiburg
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+# * Neither the name of the Freiburg Materials Research Center,
+# University of Freiburg nor the names of its contributors may be used to
+# endorse or promote products derived from this software without specific
+# prior written permission.
+#
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
+# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+TODO
+"""
+
+__id__ = "$Id$"
+__author__ = "$Author$"
+__version__ = "$Revision$"
+# $Source$
+
+from pyphant.core import (Worker, Connectors,
+ Param)
+import numpy
+
+
+class FeatureRatio(Worker.Worker):
+ """
+ TODO
+ """
+ API = 2
+ VERSION = 1
+ REVISION = "$Revision$"[11:-1]
+ name = "FeatureRatio"
+ _params = [('bgc', 'Background color', 0, None)]
+ _sockets = [("image", Connectors.TYPE_IMAGE)]
+
+ @Worker.plug(Connectors.TYPE_IMAGE)
+ def getRatio(self, image, subscriber=0):
+ from pyphant.core.DataContainer import FieldContainer
+ bgc = self.paramBgc.value
+ ratio = float(numpy.where(image.data != bgc, 1, 0).sum()) /\
+ float(image.data.size)
+ data = numpy.array([ratio])
+ fc = FieldContainer(data,
+ longname='Feature Ratio of %s' % image.longname,
+ shortname='r')
+ fc.seal()
+ return fc
Modified: trunk/src/workers/ImageProcessing/ImageProcessing/MarkAF.py
===================================================================
--- trunk/src/workers/ImageProcessing/ImageProcessing/MarkAF.py 2010-06-30 11:17:29 UTC (rev 691)
+++ trunk/src/workers/ImageProcessing/ImageProcessing/MarkAF.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -30,7 +30,7 @@
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
-This module provides a dummy worker for batching algorithms
+TODO
"""
__id__ = "$Id$"
@@ -67,6 +67,10 @@
VERSION = 1
REVISION = "$Revision$"[11:-1]
name = "MarkAF"
+ _SETMASK = 'Set mask according to focused slices'
+ _CUTBUTLABELS = 'Cut everything but focused labels'
+ _params = [('method', 'Method',
+ [_SETMASK, _CUTBUTLABELS], None)]
_sockets = [("zstack", Connectors.TYPE_ARRAY),
("statistics", Connectors.TYPE_ARRAY)]
from pyphant.core.KnowledgeManager import KnowledgeManager
@@ -74,6 +78,7 @@
@Worker.plug(Connectors.TYPE_ARRAY)
def getMarked(self, zstack, statistics, subscriber=0):
+ method = self.paramMethod.value
from copy import deepcopy
output_sc = deepcopy(zstack)
emd5s = []
@@ -81,13 +86,18 @@
max_count = len(zstack['emd5'].data)
for count, emd5 in enumerate(zstack['emd5'].data, start=1):
image = self.kmanager.getDataContainer(emd5)
- emd5s.append(self.getMarkedEmd5(image, statistics,
- vmin, vmax))
+ if method == self._SETMASK:
+ emd5s.append(self.getMarkedEmd5(image, statistics,
+ vmin, vmax))
+ elif method == self._CUTBUTLABELS:
+ emd5s.append(self.getCutEmd5(image, statistics))
subscriber %= (100 * count) / max_count
output_sc['emd5'].data = numpy.array(emd5s)
- output_sc.longname = 'Marked_' + output_sc.longname
+ typename = {self._SETMASK:'Masked',
+ self._CUTBUTLABELS:'CutLabels'}[method]
+ output_sc.longname = '%s_%s' % (typename, output_sc.longname)
output_sc.shortname = 'm'
- output_sc.attributes.update({'ZStackType':'MarkedSC'})
+ output_sc.attributes.update({'ZStackType':'%sSC' % typename})
output_sc.seal()
return output_sc
@@ -109,3 +119,25 @@
output_img.seal()
self.kmanager.registerDataContainer(output_img, temporary=True)
return output_img.id
+
+ def getCutEmd5(self, image, statistics):
+ from copy import deepcopy
+ output_img = deepcopy(image)
+ output_img.data = numpy.zeros(image.data.shape,
+ dtype=image.data.dtype)
+ mf_z = int(image.attributes['zid'][-2:])
+ labelslicess = [(label, (slice(yt, yp), slice(xt, xp))) \
+ for label, yt, yp, xt, xp, z in zip(
+ statistics['label'].data,
+ statistics['yt'].data, statistics['yp'].data,
+ statistics['xt'].data, statistics['xp'].data,
+ statistics['z-index'].data) if z == mf_z]
+ for labelslices in labelslicess:
+ img_detail = image.data[labelslices[1]]
+ output_detail = output_img.data[labelslices[1]]
+ output_img.data[labelslices[1]] = numpy.where(
+ img_detail == labelslices[0],
+ img_detail, output_detail)
+ output_img.seal()
+ self.kmanager.registerDataContainer(output_img, temporary=True)
+ return output_img.id
Added: trunk/src/workers/ImageProcessing/ImageProcessing/Reduce.py
===================================================================
--- trunk/src/workers/ImageProcessing/ImageProcessing/Reduce.py (rev 0)
+++ trunk/src/workers/ImageProcessing/ImageProcessing/Reduce.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -0,0 +1,106 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2006-2007, Rectorate of the University of Freiburg
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+# * Neither the name of the Freiburg Materials Research Center,
+# University of Freiburg nor the names of its contributors may be used to
+# endorse or promote products derived from this software without specific
+# prior written permission.
+#
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
+# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+u"""
+This module provides the Reduce Worker that extracts a single
+image from a stack by reduction of a selectable methode (e.g. 'or')
+"""
+
+__id__ = "$Id$"
+__author__ = "$Author$"
+__version__ = "$Revision$"
+# $Source$
+
+from pyphant.core import Worker, Connectors,\
+ Param, DataContainer
+import ImageProcessing
+
+
+class BinaryMethod(object):
+ import numpy
+ mdict = {'or':numpy.bitwise_or,
+ 'and':numpy.bitwise_and,
+ 'xor':numpy.bitwise_xor}
+
+ def __init__(self, methodname):
+ self.func = self.mdict[methodname]
+
+ def calcStep(self, last, next):
+ from copy import deepcopy
+ import numpy
+ result = deepcopy(last)
+ result.data = self.func(last.data.astype(bool), next.data.astype(bool))
+ result.seal()
+ return result
+
+
+class iterateImages(object):
+ def __init__(self, zstack, subscriber=0):
+ self.zstack=zstack
+ from pyphant.core.KnowledgeManager import KnowledgeManager
+ self.kmanager = KnowledgeManager.getInstance()
+ self.subscriber = subscriber
+ self.steps = len(self.zstack['emd5'].data)
+
+ def __iter__(self):
+ return self.next()
+
+ def next(self):
+ for count, emd5 in enumerate(self.zstack['emd5'].data[1:]):
+ yield self.kmanager.getDataContainer(emd5)
+ self.subscriber %= 99 / self.steps * (count + 2) + 1
+
+ def first(self):
+ self.subscriber %= 99 / self.steps + 1
+ return self.kmanager.getDataContainer(self.zstack['emd5'].data[0])
+
+
+class Reduce(Worker.Worker):
+ API = 2
+ VERSION = 1
+ REVISION = "$Revision$"[11:-1]
+ name = "Reduce"
+ _sockets = [("zstack", Connectors.TYPE_ARRAY)]
+ _params = [('method', 'Method', ['or'], None)]
+ _methodPool = {'or' : BinaryMethod('or'),
+ 'and' : BinaryMethod('and'),
+ 'xor' : BinaryMethod('xor')}
+ from pyphant.core.KnowledgeManager import KnowledgeManager
+ kmanager = KnowledgeManager.getInstance()
+
+ @Worker.plug(Connectors.TYPE_IMAGE)
+ def getReducedImage(self, zstack, subscriber=0):
+ method = self._methodPool[self.paramMethod.value]
+ iterator = iterateImages(zstack, subscriber)
+ last = iterator.first()
+ for next in iterator:
+ last = method.calcStep(last, next)
+ return last
Modified: trunk/src/workers/ImageProcessing/ImageProcessing/__init__.py
===================================================================
--- trunk/src/workers/ImageProcessing/ImageProcessing/__init__.py 2010-06-30 11:17:29 UTC (rev 691)
+++ trunk/src/workers/ImageProcessing/ImageProcessing/__init__.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -63,7 +63,9 @@
"SliceSelector",
"ThresholdingWorker",
"UltimatePointsCalculator",
- "Watershed"
+ "Watershed",
+ "Reduce",
+ "FeatureRatio"
]
def isFeature(point):
Modified: trunk/src/workers/OSC/OSC/MRA.py
===================================================================
--- trunk/src/workers/OSC/OSC/MRA.py 2010-06-30 11:17:29 UTC (rev 691)
+++ trunk/src/workers/OSC/OSC/MRA.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -50,7 +50,7 @@
from scipy.special import ive
from scipy.signal import convolve
-def findMaxima(fieldData, lastExtrema=None):
+def findMaxima(fieldData, numb_edge, lastExtrema=None):
leftLess = fieldData[:-2] < fieldData[1:-1]
leftEqual = fieldData[:-2] == fieldData[1:-1]
rightLess = fieldData[2:] < fieldData[1:-1]
@@ -60,6 +60,9 @@
maxima_re = numpy.logical_and(rightLess, leftEqual)
maxima_e = numpy.logical_and(maxima_le[:-1], maxima_re[1:])
maxima = numpy.logical_or(maxima_c[1:], maxima_e).nonzero()[0]
+ edge = len(fieldData)/numb_edge
+ maxima = maxima[edge<maxima]
+ maxima = maxima[maxima<(len(fieldData)-edge)]
if lastExtrema==None or len(maxima)==0 or len(lastExtrema)==len(maxima):
return maxima
trackedMaxima = []
@@ -68,7 +71,7 @@
trackedMaxima.append(distance.argmin())
return maxima[trackedMaxima]
-def findMinima(fieldData, lastExtrema=None):
+def findMinima(fieldData, numb_edge, lastExtrema=None):
leftGreater = fieldData[:-2] > fieldData[1:-1]
leftEqual = fieldData[:-2] == fieldData[1:-1]
rightGreater = fieldData[2:] > fieldData[1:-1]
@@ -78,6 +81,9 @@
minima_re = numpy.logical_and(rightGreater, leftEqual)
minima_e = numpy.logical_and(minima_le[:-1], minima_re[1:])
minima = numpy.logical_or(minima_c[1:], minima_e).nonzero()[0]
+ edge = len(fieldData)/numb_edge
+ minima = minima[edge<minima]
+ minima = minima[minima<(len(fieldData)-edge)]
if lastExtrema==None or len(minima)==0 or len(lastExtrema)==len(minima):
return minima
trackedMinima = []
@@ -98,18 +104,18 @@
RuntimeError.__init__(self, msg)
self.convolvedField = convolvedField
-def mra1d(dim, field, n):
+def mra1d(dim, field, n, numb_edge):
sigmaSpace = numpy.linspace(n, 1, 10)
convolvedField = convolveMRA(field, sigmaSpace[0])
- firstMinima = lastMinima = findMinima(convolvedField, None)
- firstMaxima = lastMaxima = findMaxima(convolvedField, None)
+ firstMinima = lastMinima = findMinima(convolvedField, numb_edge, None)
+ firstMaxima = lastMaxima = findMaxima(convolvedField, numb_edge, None)
if len(firstMinima)==0 and len(firstMaxima)==0:
raise MraError("No Extrema found at sigma level %s"%sigmaSpace[0],
convolvedField)
for sigma in sigmaSpace[1:]:
convolvedField = convolveMRA(field, sigma)
- lastMinima = findMinima(convolvedField, lastMinima)
- lastMaxima = findMaxima(convolvedField, lastMaxima)
+ lastMinima = findMinima(convolvedField, numb_edge, lastMinima)
+ lastMaxima = findMaxima(convolvedField, numb_edge, lastMaxima)
pos_minima = dim.data[numpy.array(lastMinima)+1]
error_minima = numpy.abs(pos_minima - dim.data[numpy.array(firstMinima)+1])
pos_maxima = dim.data[numpy.array(lastMaxima)+1]
@@ -135,6 +141,7 @@
_sockets = [("field", Connectors.TYPE_IMAGE)]
_params = [("scale", u"Scale", "200 nm", None),
+ ("numb_edge", u"Width of edge to discard extrema in [%%]", 5, None),
("longname",u"Name of result",'default',None),
("symbol",u"Symbol of result",'default',None)]
@@ -145,6 +152,7 @@
scale = quantities.Quantity(self.paramScale.value.encode('utf-8'))
except:
scale = float(self.paramScale.value)
+ numb_edge = 100.0/self.paramNumb_edge.value
d = scipy.diff(dim.data)
numpy.testing.assert_array_almost_equal(d.min(), d.max(),4)
sigmaMax = scale/(d[0]*dim.unit)
@@ -154,7 +162,7 @@
acc = 0.
for field1d in field:
try:
- p_e.append(mra1d(dim, field1d, sigmaMax))
+ p_e.append(mra1d(dim, field1d, sigmaMax, numb_edge))
except MraError:
p_e.append((([],[]),([],[])))
acc += inc
@@ -165,7 +173,7 @@
dims_min = [DataContainer.generateIndex(0,n_min), field.dimensions[0]]
dims_max = [DataContainer.generateIndex(0,n_max), field.dimensions[0]]
else:
- (pos_min, err_min), (pos_max, err_max) = mra1d(dim, field, sigmaMax)
+ (pos_min, err_min), (pos_max, err_max) = mra1d(dim, field, sigmaMax, numb_edge)
dims_min = [DataContainer.generateIndex(0,len(pos_min))]
dims_max = [DataContainer.generateIndex(0,len(pos_max))]
subscriber %= 100.
Modified: trunk/src/workers/OSC/OSC/OscThicknessCorrection.py
===================================================================
--- trunk/src/workers/OSC/OSC/OscThicknessCorrection.py 2010-06-30 11:17:29 UTC (rev 691)
+++ trunk/src/workers/OSC/OSC/OscThicknessCorrection.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -53,22 +53,44 @@
name = "Correct Thickness"
_sockets = [("osc", Connectors.TYPE_ARRAY)]
- _params = [("max_correction", "Max correction", "30 nm", None)]
+ _params = [("method","Correct thickness for", ["Spin Coating", "Printing"], None),
+ ("max_correction", "Max correction", "30 nm", None)]
+
def inithook(self):
self._logger = logging.getLogger("pyphant")
- @Worker.plug(Connectors.TYPE_IMAGE)
- def correct(self, osc, subscriber=0):
- x = osc[u'x-position']
- y = osc[u'y-position']
- t = copy.deepcopy(osc[u'thickness'])
+ def perform_spincoat_correction(self, x, y, uncorrected_t):
+ t = copy.deepcopy(uncorrected_t)
r = numpy.sqrt(x.data**2+y.data**2)
r_min = r.min()
r_max = r.max()
correction = Quantity(self.paramMax_correction.value)/t.unit
t.data = t.data + correction*((r-r_min)/(r_max-r_min))
- t.longname='corrected thickness'
+ t.longname='thickness corrected for spin coating'
t.shortname='t_c'
- t.seal()
return t
+
+ def perform_print_correction(self, x, y, uncorrected_t):
+ t = copy.deepcopy(uncorrected_t)
+ d = 1.9*y.data**2 - 19.3*y.data + 247.9
+ #d *= y.unit/t.unit
+ t.data = t.data + d
+ t.longname='thickness corrected for printing'
+ t.shortname='t_c'
+ return t
+
+ @Worker.plug(Connectors.TYPE_IMAGE)
+ def correct(self, osc, subscriber=0):
+ x = osc[u'x-position']
+ y = osc[u'y-position']
+ t = osc[u'thickness']
+ method = self.paramMethod.value
+ if method == "Spin Coating":
+ corrected_t = self.perform_spincoat_correction(x, y, t)
+ elif method == "Printing":
+ corrected_t = self.perform_print_correction(x, y, t)
+ else:
+ raise RuntimeError, "Unknown correction method %s." % method
+ corrected_t.seal()
+ return corrected_t
Modified: trunk/src/workers/tools/tools/DCSource.py
===================================================================
--- trunk/src/workers/tools/tools/DCSource.py 2010-06-30 11:17:29 UTC (rev 691)
+++ trunk/src/workers/tools/tools/DCSource.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -52,10 +52,13 @@
def __init__(self, dc_type):
self.dc_type = dc_type
self.expectedValues = {}
+ self._remaining = -1
for name, param in self._params.iteritems():
if name == 'name':
continue
param.registerListener(self.onPCE, ParamChangeExpected)
+ if name in ['reset', 'remaining']:
+ continue
param.registerListener(self.onPO, ParamOverridden)
self.expectedValues[name] = param.value
@@ -64,6 +67,9 @@
self.refreshParams(update=False, reset=True)
vcv = VisualizerChangeValue(event.param, value=False)
event.param._eventDispatcher.dispatchEvent(vcv)
+ elif event.param.name == 'remaining':
+ vcv = VisualizerChangeValue(event.param, value=self.remaining)
+ event.param._eventDispatcher.dispatchEvent(vcv)
elif event.param.name != 'reset' and \
event.expectedValue != self.expectedValues[event.param.name]:
self.expectedValues[event.param.name] = event.expectedValue
@@ -82,7 +88,8 @@
search_dict.update(dict(
[self.getKeyValue(key, val, name) \
for key, val in self.expectedValues.iteritems() \
- if key not in ['name', 'reset', name] and val != ANYSTR]))
+ if key not in ['name', 'reset', 'remaining', name] \
+ and val != ANYSTR]))
if name == 'dim_of':
search_dict = {'type':'field', 'has_dim':search_dict}
elif name == 'col_of':
@@ -103,14 +110,15 @@
if update:
self.expectedValues = dict(
[(name, param.value) for name, param \
- in self._params.iteritems() if name not in ['name', 'reset']])
+ in self._params.iteritems() if name not in \
+ ['name', 'reset', 'remaining']])
elif reset:
self.expectedValues = dict(
[(name, ANYSTR) for name in self._params.iterkeys() \
- if name not in ['name', 'reset']])
+ if name not in ['name', 'reset', 'remaining']])
kmanager = KnowledgeManager.getInstance()
for name, param in self._params.iteritems():
- if name in ['name', 'reset']:
+ if name in ['name', 'reset', 'remaining']:
continue
search_dict = self.getSearchDict(name)
newEVs = [[ANYSTR]]
@@ -123,3 +131,25 @@
param._eventDispatcher.dispatchEvent(event)
if update:
param.possibleValues = newEVs
+ if name == 'id':
+ if self.expectedValues['id'] != ANYSTR:
+ self.remaining = 1
+ else:
+ self.remaining = len(newEVs) - 1
+ if update:
+ self.paramRemaining.value = self.remaining
+
+ def _getRemaining(self):
+ if self._remaining == 1:
+ return "1: You may click OK!"
+ elif self._remaining == -1:
+ return "unknown"
+ else:
+ return "%d" % self._remaining
+
+ def _setRemaining(self, rem):
+ self._remaining = rem
+ vcv = VisualizerChangeValue(self.paramRemaining, value=self.remaining)
+ self.paramRemaining._eventDispatcher.dispatchEvent(vcv)
+
+ remaining = property(_getRemaining, _setRemaining)
Modified: trunk/src/workers/tools/tools/FCSource.py
===================================================================
--- trunk/src/workers/tools/tools/FCSource.py 2010-06-30 11:17:29 UTC (rev 691)
+++ trunk/src/workers/tools/tools/FCSource.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -63,7 +63,8 @@
("machine", u"machine ==", [ANYSTR], SUBTYPE_INSTANT),
("creator", u"and creator ==", [ANYSTR], SUBTYPE_INSTANT),
("id", u"and emd5 ==", [ANYSTR], SUBTYPE_INSTANT),
- ('reset', u'clear all parameters', False, SUBTYPE_INSTANT)]
+ ('reset', u'clear all parameters', False, SUBTYPE_INSTANT),
+ ('remaining', u'# matches', '', SUBTYPE_INSTANT)]
def __init__(self, *args, **kargs):
Worker.Worker.__init__(self, *args, **kargs)
@@ -71,6 +72,9 @@
@Worker.plug(Connectors.TYPE_IMAGE)
def getFieldContainer(self, subscriber = 0):
- emd5 = self.paramId.value
+ if len(self.paramId.possibleValues) == 2:
+ emd5 = self.paramId.possibleValues[1]
+ else:
+ emd5 = self.paramId.value
kmanager = KnowledgeManager.getInstance()
return kmanager.getDataContainer(emd5)
Modified: trunk/src/workers/tools/tools/SCSource.py
===================================================================
--- trunk/src/workers/tools/tools/SCSource.py 2010-06-30 11:17:29 UTC (rev 691)
+++ trunk/src/workers/tools/tools/SCSource.py 2010-07-04 21:19:39 UTC (rev 692)
@@ -60,7 +60,8 @@
("machine", u"machine ==", [ANYSTR], SUBTYPE_INSTANT),
("creator", u"and creator ==", [ANYSTR], SUBTYPE_INSTANT),
("id", u"and emd5 ==", [ANYSTR], SUBTYPE_INSTANT),
- ('reset', u'clear all parameters', False, SUBTYPE_INSTANT)]
+ ('reset', u'clear all parameters', False, SUBTYPE_INSTANT),
+ ('remaining', u'# matches', '', SUBTYPE_INSTANT)]
def __init__(self, *args, **kargs):
Worker.Worker.__init__(self, *args, **kargs)
@@ -68,6 +69,9 @@
@Worker.plug(Connectors.TYPE_ARRAY)
def getSampleContainer(self, subscriber = 0):
- emd5 = self.paramId.value
+ if len(self.paramId.possibleValues) == 2:
+ emd5 = self.paramId.possibleValues[1]
+ else:
+ emd5 = self.paramId.value
kmanager = KnowledgeManager.getInstance()
return kmanager.getDataContainer(emd5)
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-06-30 11:17:36
|
Revision: 691
http://pyphant.svn.sourceforge.net/pyphant/?rev=691&view=rev
Author: zklaus
Date: 2010-06-30 11:17:29 +0000 (Wed, 30 Jun 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Enh: FMFLoader speedup
Modified Paths:
--------------
trunk/src/workers/fmfile/fmfile/FMFLoader.py
Modified: trunk/src/workers/fmfile/fmfile/FMFLoader.py
===================================================================
--- trunk/src/workers/fmfile/fmfile/FMFLoader.py 2010-06-29 12:01:34 UTC (rev 690)
+++ trunk/src/workers/fmfile/fmfile/FMFLoader.py 2010-06-30 11:17:29 UTC (rev 691)
@@ -74,6 +74,7 @@
else:
data[pixelName] = rawContainer
subscriber %= float(i+1)/total*100.0
+
return data,names
def collectAttributes(data,names):
@@ -145,7 +146,7 @@
indexError = 2
for i,element in enumerate(column):
if not type(element)==type((0,)):
- column[i]=(shortname,numpy.NaN,None)
+ column[i]=(shortname,numpy.NaN,None)
try:
data = [element[indexDatum] for element in column]
except:
@@ -257,35 +258,13 @@
if field.isIndependent():
return field
dimData = [numpy.unique(d.data) for d in field.dimensions]
- fieldData = numpy.ones([len(d) for d in dimData])*numpy.nan
- data = numpy.vstack([field.data]+[d.data for d in field.dimensions]).transpose()
- for row in data:
- try:
- fieldData[[numpy.argwhere(dimData[i]==v) for i,v in enumerate(row[1:])]] = row[0]
- except AttributeError:
- from pyphant.wxgui2.wxPyphantApplication import LOGDIR
- import os, os.path
- DEBDIR=os.path.join(LOGDIR, "pyphant_debug")
- if not os.path.exists(DEBDIR):
- os.mkdir(DEBDIR)
- for i,v in enumerate(row[1:]):
- try:
- numpy.argwhere(dimData[i]==v)
- except AttributeError, e:
- _logger.debug(u"AttributeError occured:",
- exc_info=True)
- f = open(os.path.join(DEBDIR, "deblog"), 'w')
- f.write("dShape: %s; vShape: %s\n\n"%(dimData[i].shape, v.shape))
- f.write("%s"%dimData[i])
- f.write("\n\nv:\n")
- f.write("%s"%v)
- f.write("\n")
- f.close()
- numpy.savetxt(os.path.join(DEBDIR, "dimData.txt"), dimData[i])
- numpy.savetxt(os.path.join(DEBDIR, "data.txt"), data)
- numpy.savetxt(os.path.join(DEBDIR, "row.txt"), row)
- numpy.savetxt(os.path.join(DEBDIR, "v.txt"), v)
- raise
+ dimDicts = [dict([(data, index) for index, data in enumerate(dimdata)]) \
+ for dimdata in dimData]
+ fieldData = numpy.ones([len(d) for d in dimData]) * numpy.nan
+ indicess = [map(lambda x: dimDicts[index][x], dim.data) \
+ for index, dim in enumerate(field.dimensions)]
+ for datum, indices in zip(field.data, indicess):
+ fieldData[indices] = datum
newDims = [ DataContainer.FieldContainer(dimData[i],
f.unit,
longname=f.longname,
@@ -381,7 +360,6 @@
return oldVal
def config2tables(preParsedData, config, FMFversion='1.1'):
-
if config.has_key('*table definitions'):
longnames = dict([(i,k) for k,i in config['*table definitions'].iteritems()])
del config['*table definitions']
@@ -399,7 +377,7 @@
preParsedData[shortname],
config[k],FMFversion))
del config[k]
- attributes = config.walk(lambda section,key:
+ attributes = config.walk(lambda section,key:
item2value(section[key],FMFversion))
for t in tables:
t.attributes = copy.deepcopy(attributes)
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-06-29 12:02:14
|
Revision: 690
http://pyphant.svn.sourceforge.net/pyphant/?rev=690&view=rev
Author: zklaus
Date: 2010-06-29 12:01:34 +0000 (Tue, 29 Jun 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Fix: Deals with the ParseQuantities part of #37
Enh: Test for emd5 consist. with FMF import
Fix: DCSource: removed AutoSelect feature
Enh: Reset button for DCSource workers
Cosm: Worker Rep now presented in TreeCtrl
Fix: DCSource
Modified Paths:
--------------
trunk/src/pyphant/pyphant/core/Param.py
trunk/src/pyphant/pyphant/core/Worker.py
trunk/src/pyphant/pyphant/core/WorkerRegistry.py
trunk/src/pyphant/pyphant/quantities/ParseQuantities.py
trunk/src/pyphant/pyphant/wxgui2/WorkerRepository.py
trunk/src/pyphant/pyphant/wxgui2/paramvisualization/CheckBox.py
trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ListSelect.py
trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ParamVisReg.py
trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py
trunk/src/workers/fmfile/fmfile/tests/TestFMFLoader.py
trunk/src/workers/tools/tools/DCSource.py
trunk/src/workers/tools/tools/FCSource.py
trunk/src/workers/tools/tools/SCSource.py
Added Paths:
-----------
trunk/src/workers/fmfile/fmfile/tests/resources/
trunk/src/workers/fmfile/fmfile/tests/resources/fmf/
trunk/src/workers/fmfile/fmfile/tests/resources/fmf/dep.fmf
Modified: trunk/src/pyphant/pyphant/core/Param.py
===================================================================
--- trunk/src/pyphant/pyphant/core/Param.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/pyphant/pyphant/core/Param.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -53,11 +53,11 @@
self.expectedValue = expectedValue
-class PossibleValuesChangeExpected(object):
- def __init__(self, param, expectedPVs, autoSelect=False):
+class VisualizerChangeValue(object):
+ def __init__(self, param, **kargs):
self.param = param
- self.expectedPVs = expectedPVs
- self.autoSelect = autoSelect
+ for key, value in kargs.iteritems():
+ setattr(self, key, value)
class ParamChangeRequested(object):
Modified: trunk/src/pyphant/pyphant/core/Worker.py
===================================================================
--- trunk/src/pyphant/pyphant/core/Worker.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/pyphant/pyphant/core/Worker.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -43,9 +43,10 @@
class WorkerInfo(object):
## __slots__ = ["name", "createWorker"] #not possible due to pickling restrictions.
- def __init__(self, name, createWorker):
- self.name=name
- self.createWorker=createWorker
+ def __init__(self, cls):
+ self.name = cls.name
+ self.toolBoxName = cls.__module__.split('.')[0]
+ self.createWorker = cls
def plug(returnType):
def setPlug(_plug):
@@ -69,7 +70,7 @@
cls._plugs.append((f, cdict[f]))
super(WorkerFactory, cls).__init__(name, bases, cdict)
if cls.__name__ != 'Worker':
- WorkerFactory.workerRegistry.registerWorker(WorkerInfo(cls.name,cls))
+ WorkerFactory.workerRegistry.registerWorker(WorkerInfo(cls))
class Worker(object):
API = 2
Modified: trunk/src/pyphant/pyphant/core/WorkerRegistry.py
===================================================================
--- trunk/src/pyphant/pyphant/core/WorkerRegistry.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/pyphant/pyphant/core/WorkerRegistry.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -42,20 +42,48 @@
import pkg_resources
+class ToolBoxInfo(object):
+ def __init__(self, name):
+ self.name = name
+ self.workerInfos = []
+ self._logger = logging.getLogger("pyphant")
+
+ def sortWorkerInfos(self):
+ self.workerInfos.sort(key=lambda x:x.name.lower())
+
+ def __eq__(self, other):
+ return self.name == other.name
+
+ def __ne__(self, other):
+ return self.name != other.name
+
+ def addWorkerInfo(self, workerInfo):
+ if not workerInfo in self.workerInfos:
+ self.workerInfos.append(workerInfo)
+ self._logger.info("Added worker %s from toolbox %s." \
+ % (workerInfo.name, self.name))
+
+
class WorkerRegistry(singletonmixin.Singleton):
def __init__(self):
- self._workerPool = []
+ self._toolBoxInfoList = []
+ self._toolBoxInfoDict = {}
self._logger = logging.getLogger("pyphant")
self._dirty = True
def registerWorker(self, workerInfo):
- if not workerInfo in self._workerPool:
- self._workerPool.append(workerInfo)
- self._logger.info("added worker")
- else:
- self._logger.info("did nothing")
+ tBI = ToolBoxInfo(workerInfo.toolBoxName)
+ if not tBI in self._toolBoxInfoList:
+ self._toolBoxInfoList.append(tBI)
+ self._toolBoxInfoDict[tBI.name] = tBI
+ self._toolBoxInfoDict[tBI.name].addWorkerInfo(workerInfo)
- def getWorkers(self):
+ def sortToolBoxInfos(self):
+ for tBI in self._toolBoxInfoList:
+ tBI.sortWorkerInfos()
+ self._toolBoxInfoList.sort(key=lambda x: x.name.lower())
+
+ def getToolBoxInfoList(self):
if self._dirty:
for worker in pkg_resources.iter_entry_points("pyphant.workers"):
wm = worker.load()
@@ -75,5 +103,6 @@
"worker archive " + worker.module_name \
+ " contains invalid worker " + module \
+ ": " + str(e))
+ self.sortToolBoxInfos()
self._dirty = False
- return self._workerPool
+ return self._toolBoxInfoList
Modified: trunk/src/pyphant/pyphant/quantities/ParseQuantities.py
===================================================================
--- trunk/src/pyphant/pyphant/quantities/ParseQuantities.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/pyphant/pyphant/quantities/ParseQuantities.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -73,7 +73,12 @@
unit = Quantity(unit.encode('utf-8'))
elif FMFversion=='1.0':
unit1_0 = PhysicalQuantity(unit.encode('utf-8'))
+ unit_base = Quantity(str(unit1_0.inBaseUnits()))
unit = Quantity(str(unit1_0))
+ if unit_base != unit:
+ unit = unit_base
+ _logger.warn('Usage of old unit "%s" required '
+ 'conversion to base units.' % unit1_0)
except:
unit = float(unit)
return unit
Modified: trunk/src/pyphant/pyphant/wxgui2/WorkerRepository.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/WorkerRepository.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/pyphant/pyphant/wxgui2/WorkerRepository.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -39,40 +39,28 @@
import wx
import cPickle
-import pyphant.core.WorkerRegistry
+from pyphant.core.WorkerRegistry import WorkerRegistry
-class WorkerRepository(wx.ScrolledWindow):
- def __init__(self, parent, id=-1,
- pos=wx.DefaultPosition,
- size=wx.DefaultSize,
- style=0):
- wx.ScrolledWindow.__init__(self, parent, id)
- workerRegistry = pyphant.core.WorkerRegistry.WorkerRegistry
- self._workerRegistry = workerRegistry.getInstance()
- self._boxSizer = wx.BoxSizer(wx.VERTICAL)
- self.initFoldPanelBar()
- self.SetSizer(self._boxSizer)
- self.SetScrollRate(1, 1)
- self._boxSizer.SetVirtualSizeHints(self)
- def initFoldPanelBar(self):
- map(self.addWorkerButton, self._workerRegistry.getWorkers())
+class WorkerRepository(wx.TreeCtrl):
+ def __init__(self, *args, **kargs):
+ wx.TreeCtrl.__init__(self, *args, **kargs)
+ tBIL = WorkerRegistry.getInstance().getToolBoxInfoList()
+ rootId = self.AddRoot('toolboxes')
+ for tBI in tBIL:
+ toolBoxId = self.AppendItem(rootId, tBI.name)
+ for workerInfo in tBI.workerInfos:
+ wIId = self.AppendItem(toolBoxId, workerInfo.name)
+ self.SetItemData(wIId, wx.TreeItemData(workerInfo))
+ self.Bind(wx.EVT_TREE_BEGIN_DRAG,
+ self.onDragInit, id=self.GetId())
- def addWorkerButton(self, workerInfo):
- btn = WorkerButton(self, workerInfo)
- self._boxSizer.Add(btn, 1, wx.EXPAND, wx.ALL, 10)
-
-
-class WorkerButton(wx.Button):
- def __init__(self, parent, workerInfo):
- name = workerInfo.name
- wx.Button.__init__(self, parent, label=name)
- self._workerInfo = workerInfo
- self.Bind(wx.EVT_LEFT_DOWN, self.onLeftDown)
-
- def onLeftDown(self, evt):
- dropSource = wx.DropSource(self)
- pickledObj = cPickle.dumps(self._workerInfo)
- data = wx.TextDataObject(pickledObj)
- dropSource.SetData(data)
- dragResult = dropSource.DoDragDrop(True)
+ def onDragInit(self, event):
+ workerInfo = self.GetItemData(event.Item).Data
+ if workerInfo is not None:
+ dropSource = wx.DropSource(self)
+ pickledObj = cPickle.dumps(workerInfo)
+ data = wx.TextDataObject(pickledObj)
+ dropSource.SetData(data)
+ dragResult = dropSource.DoDragDrop(True)
+ event.Skip()
Modified: trunk/src/pyphant/pyphant/wxgui2/paramvisualization/CheckBox.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/paramvisualization/CheckBox.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/pyphant/pyphant/wxgui2/paramvisualization/CheckBox.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -38,7 +38,10 @@
# $Source$
import wx
+from pyphant.core.Param import (
+ ParamChangeExpected, VisualizerChangeValue)
+
class CheckBox(wx.CheckBox):
def __init__(self, parent, param, validator):
wx.CheckBox.__init__(self, parent, validator=validator)
@@ -46,3 +49,19 @@
def getValue(self):
return self.GetValue()
+
+class InstantCheckBox(CheckBox):
+ def __init__(self, parent, param, validator):
+ CheckBox.__init__(self, parent, param, validator)
+ self.param = param
+ self.Bind(wx.EVT_CHECKBOX, self.onCheck)
+ param._eventDispatcher.registerExclusiveListener(
+ self.onVCV, VisualizerChangeValue)
+
+ def onCheck(self, Event):
+ pce = ParamChangeExpected(self.param, expectedValue=self.getValue())
+ self.param._eventDispatcher.dispatchEvent(pce)
+ Event.Skip()
+
+ def onVCV(self, event):
+ self.SetValue(event.value)
Modified: trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ListSelect.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ListSelect.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ListSelect.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -40,7 +40,7 @@
import wx
from wx import EVT_CHOICE
from pyphant.core.Param import (
- ParamChangeExpected, PossibleValuesChangeExpected)
+ ParamChangeExpected, VisualizerChangeValue)
class ListSelect(wx.Choice):
def __init__(self, parent, param, validator):
@@ -64,7 +64,7 @@
"""
This class dispatches the ParamChangeExpected event as soon as
the user selects a new value and listens for the
- PossibleValueChangeExpected event which should be raised
+ VisualizerChangeValue event which should be raised
if the possible values for the underlying ListSelect
need to be updated.
"""
@@ -74,23 +74,19 @@
self.param = param
self.possibleValues = param.possibleValues
param._eventDispatcher.registerExclusiveListener(
- self.onPVCE, PossibleValuesChangeExpected)
+ self.onVCV, VisualizerChangeValue)
- def onPVCE(self, event):
- value = self.data[self.GetStringSelection()]
- assert value in event.expectedPVs, "%s not in %s" % (value,
- event.expectedPVs)
- self.data = dict([(str(val), val) for val in event.expectedPVs])
- self.possibleValues = event.expectedPVs
- self.SetItems(map(str, event.expectedPVs))
- if event.autoSelect and len(event.expectedPVs) == 2:
- self.SetSelection(1)
- pce = ParamChangeExpected(
- self.param, expectedValue=self.data[self.GetStringSelection()])
- pce.norefresh = True
- self.param._eventDispatcher.dispatchEvent(pce)
+ def onVCV(self, event):
+ if not hasattr(event, 'value'):
+ value = self.data[self.GetStringSelection()]
else:
- self.SetValue(value)
+ value = event.value
+ assert value in event.possibleValues, "%s not in %s" \
+ % (value, event.possibleValues)
+ self.data = dict([(str(val), val) for val in event.possibleValues])
+ self.possibleValues = event.possibleValues
+ self.SetItems(map(str, event.possibleValues))
+ self.SetValue(value)
def onChoice(self, event):
event.Skip()
Modified: trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ParamVisReg.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ParamVisReg.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ParamVisReg.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -94,6 +94,7 @@
(" ", None, OneLineStringField.OLSF),
(" ", Connectors.SUBTYPE_FILE, FileButton.FileButton),
(True, None, CheckBox.CheckBox),
+ (True, Connectors.SUBTYPE_INSTANT, CheckBox.InstantCheckBox),
([], None, ListSelect.ListSelect),
([], Connectors.SUBTYPE_INSTANT, ListSelect.InstantSelect)
]
Modified: trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -60,7 +60,8 @@
import wx.aui
import sogl
import pyphant.wxgui2.paramvisualization.ParamVisReg as ParamVisReg
-import pyphant.core.PyTablesPersister
+from pyphant.core.PyTablesPersister import (loadRecipeFromHDF5File,
+ saveRecipeToHDF5File)
import WorkerRepository
import ConfigureFrame
import platform
@@ -128,43 +129,31 @@
self._statusBar = self.CreateStatusBar()
self._wxPyphantApp = _wxPyphantApp
self._initMenuBar()
- self._initSash()
+ self._initWorkerRep()
self.recipeState = None
self.onOpenCompositeWorker(None)
self._initAui()
- self._workerRepository.Bind(wx.EVT_SASH_DRAGGED_RANGE,
- self.onFoldPanelBarDrag,
- id=self.ID_WINDOW_TOP,
- id2=self.ID_WINDOW_BOTTOM)
- #self.Bind(wx.EVT_SIZE, self.onSize)
self.compositeWorkerStack = []
- def _initSash(self):
- self._workerRepository = wx.SashLayoutWindow(
- self, self.ID_WINDOW_RIGHT, wx.DefaultPosition, wx.Size(220,1000),
- wx.NO_BORDER)
- #self._workerRepository.SetDefaultSize(wx.Size(220,1000))
- #self._workerRepository.SetOrientation(wx.LAYOUT_VERTICAL)
- #self._workerRepository.SetAlignment(wx.LAYOUT_RIGHT)
- #self._workerRepository.SetSashVisible(wx.SASH_LEFT, True)
- #self._workerRepository.SetExtraBorderSize(10)
+ def _initWorkerRep(self):
try:
- WorkerRepository.WorkerRepository(self._workerRepository)
+ self._workerRepository = WorkerRepository.WorkerRepository(
+ self, self.ID_WINDOW_RIGHT, wx.DefaultPosition,
+ wx.Size(220, -1))
+ self._workerRepository.Expand(self._workerRepository.RootItem)
except:
import sys
- self._wxPyphantApp._logger.debug(u"An exception occured while "\
- "loading the toolboxes.",
- exc_info=sys.exc_info())
- wx.MessageBox("An error has occurred while importing "\
- "the toolboxes.\nPlease investigate the logfile "\
- "for further details.\nThe logfile is located at %s\n"\
- "You may also try to update and restart wxPyphant."\
- % os.path.join(LOGDIR, 'pyphant.log'),
- "Toolbox Error!")
- self._workerRepository = wx.SashLayoutWindow(
- self, self.ID_WINDOW_RIGHT, wx.DefaultPosition,
- wx.Size(220,1000),
- wx.NO_BORDER)
+ self._wxPyphantApp._logger.debug(
+ u"An exception occured while loading the toolboxes.",
+ exc_info=sys.exc_info())
+ wx.MessageBox(
+ "An error has occurred while importing "\
+ "the toolboxes.\nPlease investigate the logfile "\
+ "for further details.\nThe logfile is located at %s\n"\
+ "You may also try to update and restart wxPyphant."\
+ % os.path.join(LOGDIR, 'pyphant.log'),
+ "Toolbox Error!")
+ self._workerRepository = wx.TreeCtrl(self)
def _initAui(self):
self._auiManager = wx.aui.AuiManager(self)
@@ -175,10 +164,9 @@
def OnClick(self, event):
logbuffer.flush()
event.Skip()
- self._logpane = ClickableText(self, -1, "",
- wx.DefaultPosition, wx.Size(640, 200),
- wx.NO_BORDER | wx.TE_MULTILINE \
- | wx.TE_READONLY)
+ self._logpane = ClickableText(
+ self, -1, "", wx.DefaultPosition, wx.Size(640, 200),
+ wx.NO_BORDER | wx.TE_MULTILINE | wx.TE_READONLY)
class WxHandler(logging.Handler):
def __init__(self, ctrl):
logging.Handler.__init__(self)
@@ -201,25 +189,8 @@
'Worker Repository')
self._auiManager.AddPane(self._remainingSpace, wx.CENTER, 'Main')
wrpane = self._auiManager.GetPane(self._workerRepository)
- wrpane.Floatable(False)
- wrpane.Movable(False)
self._auiManager.Update()
- def onSize(self, event):
- wx.LayoutAlgorithm().LayoutWindow(self,self._remainingSpace)
- event.Skip()
-
- def onFoldPanelBarDrag(self, event):
- if event.GetDragStatus() == wx.SASH_STATUS_OUT_OF_RANGE:
- return
- if event.GetId() == self.ID_WINDOW_RIGHT:
- self._workerRepository.SetDefaultSize(
- wx.Size(event.GetDragRect().width, 1000))
- # Leaves bits of itself behind sometimes
- #wx.LayoutAlgorithm().LayoutWindow(self, self._remainingSpace)
- self._remainingSpace.Refresh()
- event.Skip()
-
def onOpenCompositeWorker(self, event):
if not self._wxPyphantApp.pathToRecipe:
if pltform == 'Linux' or pltform == 'Darwin':
@@ -250,20 +221,22 @@
if self._wxPyphantApp.pathToRecipe[-3:] == '.h5':
if os.path.exists(self._wxPyphantApp.pathToRecipe):
try:
- recipe = pyphant.core.PyTablesPersister.loadRecipeFromHDF5File(
+ recipe = loadRecipeFromHDF5File(
self._wxPyphantApp.pathToRecipe)
- self._remainingSpace = PyphantCanvas.PyphantCanvas(self, recipe)
+ self._remainingSpace = PyphantCanvas.PyphantCanvas(
+ self, recipe)
except:
- self._wxPyphantApp._logger.debug(u"An exception occured while "\
- "loading a recipe.",
- exc_info=sys.exc_info())
- wx.MessageBox("An error has occurred while opening "\
- "the recipe.\nRecipe has been set to an "\
- "empty file in order to prevent data loss.\n"\
- "Please investigate the logfile "\
- "for further details.\nThe logfile is located at %s"\
- % os.path.join(LOGDIR, 'pyphant.log'),
- "Recipe broken, unknown format or outdated!")
+ self._wxPyphantApp._logger.debug(
+ u"An exception occured while loading a recipe.",
+ exc_info=sys.exc_info())
+ wx.MessageBox(
+ "An error has occurred while opening "\
+ "the recipe.\nRecipe has been set to an "\
+ "empty file in order to prevent data loss.\n"\
+ "Please investigate the logfile "\
+ "for further details.\nThe logfile is located at %s"\
+ % os.path.join(LOGDIR, 'pyphant.log'),
+ "Recipe broken, unknown format or outdated!")
self._wxPyphantApp.pathToRecipe += ".error.h5"
self._remainingSpace = PyphantCanvas.PyphantCanvas(self)
else:
@@ -281,10 +254,9 @@
self.recipeState = 'dirty'
def onSaveCompositeWorker(self, event=None):
- pyphant.core.PyTablesPersister.saveRecipeToHDF5File(
- self._remainingSpace.diagram.recipe,
- self._wxPyphantApp.pathToRecipe,
- self._fileMenu.IsChecked(wx.ID_FILE4))
+ saveRecipeToHDF5File(self._remainingSpace.diagram.recipe,
+ self._wxPyphantApp.pathToRecipe,
+ self._fileMenu.IsChecked(wx.ID_FILE4))
self.recipeState = 'clean'
def onSaveAsCompositeWorker(self, event=None):
@@ -296,10 +268,9 @@
filename = dlg.GetPath()
if not filename.endswith(".h5"):
filename += ".h5"
- pyphant.core.PyTablesPersister.saveRecipeToHDF5File(
+ saveRecipeToHDF5File(
self._remainingSpace.diagram.recipe,
- filename,
- self._fileMenu.IsChecked(wx.ID_FILE4))
+ filename, self._fileMenu.IsChecked(wx.ID_FILE4))
self._wxPyphantApp.pathToRecipe = filename
self.recipeState = 'clean'
from pyphant.core.WebInterface import shorten
Modified: trunk/src/workers/fmfile/fmfile/tests/TestFMFLoader.py
===================================================================
--- trunk/src/workers/fmfile/fmfile/tests/TestFMFLoader.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/workers/fmfile/fmfile/tests/TestFMFLoader.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -41,7 +41,7 @@
from pyphant.core.DataContainer import FieldContainer,assertEqual
from pyphant.quantities import Quantity
from pyphant.quantities.ParseQuantities import str2unit
-
+
class FieldContainerCondenseDim(unittest.TestCase):
def setUp(self):
self.x = numpy.linspace(0,0.9,10)
@@ -135,13 +135,13 @@
created: 2010-03-16
place: ICE 676, Offenburg-Karlsruhe, Germany
[Mathematical and Physical Constants]
-Area of unit circle: pi = 3.1415926535897931
+Area of unit circle: pi = 3.1415926535897931
Speed of light: c = 299792458 m/s
Permeability of vacuum: \mu_0 = 4.e-7 pi*N/A**2
Permittivity of vacuum: \eps_0 = 1.0 1/mu0/c**2
-Gravitational constant: Grav = 6.67259e-11 m**3/kg/s**2
+Gravitational constant: Grav = 6.67259e-11 m**3/kg/s**2
Planck constant: hplanck = 6.6260755e-34 J*s
-Planck constant / 2pi: hbar = 0.5 hplanck/pi
+Planck constant / 2pi: hbar = 0.5 hplanck/pi
Elementary charge: e = 1.60217733e-19 C
Electron mass: m_e = 9.1093897e-31 kg
Proton mass: m_p = 1.6726231e-27 kg
@@ -163,7 +163,7 @@
Missing Value: V_m
Infinite Value: V_i
[*data: T]
-H_2 1 1. 1e1 1+0j nan inf
+H_2 1 1. 1e1 1+0j nan inf
O_2 2 .2 2E1 2+.1j NaN INF
O 2 2 .2 2E1 2.+2j NAN Inf
[*data definitions: M]
@@ -209,14 +209,14 @@
created: 2010-03-17
place: ICE 604, Offenburg-Karlsruhe, Germany
[Mathematical and Physical Constants]
-Area of unit circle: pi = 3.1415926535897931
+Area of unit circle: pi = 3.1415926535897931
Speed of light: c = 299792458 m/s
Permeability of vacuum: \mu_0 = 4.e-7 pi*N/A**2
Permittivity of vacuum: \eps_0 = 1.0 1/mu0/c**2
Faraday constant: Fa = 96485.3399 C/mol
-Gravitational constant: G = 6.67428e-11 m**3/kg/s**2
+Gravitational constant: G = 6.67428e-11 m**3/kg/s**2
Planck constant: h = 6.62606896e-34 J*s
-Planck constant / 2pi: hbar = 0.5 h/pi
+Planck constant / 2pi: hbar = 0.5 h/pi
Elementary charge: e = 1.602176487e-19 C
Electron mass: m_e = 9.10938215e-31 kg
Proton mass: m_p = 1.672621637e-27 kg
@@ -240,7 +240,7 @@
Missing Value: V_m
Infinite Value: V_i
[*data: T]
-H_2 1 1. 1e1 1+0j nan inf
+H_2 1 1. 1e1 1+0j nan inf
O_2 2 .2 2E1 2+.1j NaN INF
O 2 2 .2 2E1 2.+2j NAN Inf
[*data definitions: M]
@@ -272,7 +272,45 @@
self.assertEqual(consts[u'Parsec'][1],str2unit("1 pc",FMFversion="1.1"))
self.assertEqual(consts[u'US gallon'][1],str2unit("1 galUS",FMFversion="1.1"))
self.assertEqual(consts[u'Atomic mass units'][1],str2unit("1 u",FMFversion="1.1"))
-
+
+
+class Emd5ConsistencyTestCase(unittest.TestCase):
+ def setUp(self):
+ from fmfile import __path__ as path
+ import os
+ self.filename = os.path.join(path[0], 'tests', 'resources',
+ 'fmf','dep.fmf')
+
+ def testImportFMF(self):
+ from fmfile import FMFLoader
+ table = FMFLoader.loadFMFFromFile(self.filename)
+ print "Testing imported SampleContainer for consistency..."
+ for column in ['y0', 'y1', 'y2', 'y3', 'y4', 'y5', 'y6', 'y7', 'y8']:
+ self.assertEqual(table[column].dimensions[0].id,
+ table['x'].id)
+
+ def testRegisterFMF(self):
+ from pyphant.core.KnowledgeManager import KnowledgeManager
+ kmanager = KnowledgeManager.getInstance()
+ table_id = kmanager.registerFMF(self.filename, temporary=True)
+ table = kmanager.getDataContainer(table_id)
+ print "Testing registered SampleContainer for consistency..."
+ for column in ['y0', 'y1', 'y2', 'y3', 'y4', 'y5', 'y6', 'y7', 'y8']:
+ self.assertEqual(table[column].dimensions[0].id,
+ table['x'].id)
+
+ def testRegisterFMFSummary(self):
+ from pyphant.core.KnowledgeManager import KnowledgeManager
+ kmanager = KnowledgeManager.getInstance()
+ table_id = kmanager.registerFMF(self.filename, temporary=True)
+ table = kmanager.getDataContainer(table_id)
+ print "Testing registered SampleContainer summary for consistency..."
+ for column in ['y0', 'y1', 'y2', 'y3', 'y4', 'y5', 'y6', 'y7', 'y8']:
+ summary = kmanager.getSummary(table[column].id)
+ emd5 = summary['dimensions'][0]
+ self.assertEqual(emd5, table['x'].id)
+
+
if __name__ == "__main__":
import sys
if len(sys.argv) == 1:
Added: trunk/src/workers/fmfile/fmfile/tests/resources/fmf/dep.fmf
===================================================================
--- trunk/src/workers/fmfile/fmfile/tests/resources/fmf/dep.fmf (rev 0)
+++ trunk/src/workers/fmfile/fmfile/tests/resources/fmf/dep.fmf 2010-06-29 12:01:34 UTC (rev 690)
@@ -0,0 +1,41 @@
+; -*- fmf-version: 1.0; delimiter: whitespace -*-
+[*reference]
+title: Import Test
+creator: Alexander Held
+created: 2010-25-06 14:15:00
+place: Uni Freiburg FMF
+
+[*data definitions]
+x: x [m]
+y0: y0(x) [m]
+y1: y1(x) [m]
+y2: y2(x) [m]
+y3: y3(x) [m]
+y4: y4(x) [m]
+y5: y5(x) [m]
+y6: y6(x) [m]
+y7: y7(x) [m]
+y8: y8(x) [m]
+
+[*data]
+0 0 1 2 3 4 5 6 7 8
+1 0 1 2 3 4 5 6 7 8
+2 0 1 2 3 4 5 6 7 8
+3 0 1 2 3 4 5 6 7 8
+;4 0 1 2 3 4 5 6 7 8
+5 0 1 2 3 4 5 6 7 8
+6 0 1 2 3 4 5 6 7 8
+7 0 1 2 3 4 5 6 7 8
+8 0 1 2 3 4 5 6 7 8
+9 0 1 2 3 4 5 6 7 8
+10 0 1 2 3 4 5 6 7 8
+11 0 1 2 3 4 5 6 7 8
+12 0 1 2 3 4 5 6 7 8
+13 0 1 2 3 4 5 6 7 8
+14 0 1 2 3 4 5 6 7 8
+15 0 1 2 3 4 5 6 7 8
+16 0 1 2 3 4 5 6 7 8
+;17 0 1 2 3 4 5 6 7 8
+18 0 1 2 3 4 5 6 7 8
+19 0 1 2 3 4 5 6 7 8
+20 0 1 2 3 4 5 6 7 8
Modified: trunk/src/workers/tools/tools/DCSource.py
===================================================================
--- trunk/src/workers/tools/tools/DCSource.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/workers/tools/tools/DCSource.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -40,7 +40,7 @@
from pyphant.core.KnowledgeManager import KnowledgeManager
from pyphant.core.Param import (
- ParamChangeExpected, PossibleValuesChangeExpected, ParamOverridden)
+ ParamChangeExpected, VisualizerChangeValue, ParamOverridden)
ANYSTR = u"-- any --"
@@ -53,18 +53,21 @@
self.dc_type = dc_type
self.expectedValues = {}
for name, param in self._params.iteritems():
- if name in ["name"]:
+ if name == 'name':
continue
param.registerListener(self.onPCE, ParamChangeExpected)
param.registerListener(self.onPO, ParamOverridden)
self.expectedValues[name] = param.value
def onPCE(self, event):
- if event.expectedValue != self.expectedValues[event.param.name]:
+ if event.param.name == 'reset' and event.expectedValue:
+ self.refreshParams(update=False, reset=True)
+ vcv = VisualizerChangeValue(event.param, value=False)
+ event.param._eventDispatcher.dispatchEvent(vcv)
+ elif event.param.name != 'reset' and \
+ event.expectedValue != self.expectedValues[event.param.name]:
self.expectedValues[event.param.name] = event.expectedValue
- if not hasattr(event, 'norefresh'):
- autoSelect = event.expectedValue != ANYSTR
- self.refreshParams(update=False, autoSelect=autoSelect)
+ self.refreshParams(update=False)
def onPO(self, event):
self.expectedValues[event.param.name] = event.newValue
@@ -79,7 +82,7 @@
search_dict.update(dict(
[self.getKeyValue(key, val, name) \
for key, val in self.expectedValues.iteritems() \
- if key not in ['name', name] and val != ANYSTR]))
+ if key not in ['name', 'reset', name] and val != ANYSTR]))
if name == 'dim_of':
search_dict = {'type':'field', 'has_dim':search_dict}
elif name == 'col_of':
@@ -96,21 +99,27 @@
else:
return name
- def refreshParams(self, subscriber=None, update=True, autoSelect=True):
+ def refreshParams(self, subscriber=None, update=True, reset=False):
if update:
self.expectedValues = dict(
[(name, param.value) for name, param \
- in self._params.iteritems() if name != 'name'])
+ in self._params.iteritems() if name not in ['name', 'reset']])
+ elif reset:
+ self.expectedValues = dict(
+ [(name, ANYSTR) for name in self._params.iterkeys() \
+ if name not in ['name', 'reset']])
kmanager = KnowledgeManager.getInstance()
for name, param in self._params.iteritems():
- if name == 'name':
+ if name in ['name', 'reset']:
continue
search_dict = self.getSearchDict(name)
newEVs = [[ANYSTR]]
newEVs.extend(kmanager.search(
[self.getResKey(name)], search_dict=search_dict, distinct=True))
newEVs = [newEV[0] for newEV in newEVs]
- param._eventDispatcher.dispatchEvent(
- PossibleValuesChangeExpected(param, newEVs, autoSelect))
+ event = VisualizerChangeValue(param, possibleValues=newEVs)
+ if reset:
+ event.value = ANYSTR
+ param._eventDispatcher.dispatchEvent(event)
if update:
param.possibleValues = newEVs
Modified: trunk/src/workers/tools/tools/FCSource.py
===================================================================
--- trunk/src/workers/tools/tools/FCSource.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/workers/tools/tools/FCSource.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -53,16 +53,17 @@
VERSION = 1
REVISION = "$Revision$"[11:-1]
name = "FieldContainer"
- _params = [("machine", u"machine ==", [ANYSTR], SUBTYPE_INSTANT),
- ("creator", u"and creator ==", [ANYSTR], SUBTYPE_INSTANT),
- ("longname", u"and longname ==", [ANYSTR], SUBTYPE_INSTANT),
+ _params = [("longname", u"and longname ==", [ANYSTR], SUBTYPE_INSTANT),
("shortname", u"and shortname ==", [ANYSTR], SUBTYPE_INSTANT),
("unit", u"and unit is compatible to", [ANYSTR],
SUBTYPE_INSTANT),
("has_dim", u"and has dimension", [ANYSTR], SUBTYPE_INSTANT),
("col_of", u"and is column of", [ANYSTR], SUBTYPE_INSTANT),
("dim_of", u"and is dimension of", [ANYSTR], SUBTYPE_INSTANT),
- ("id", u"and emd5 ==", [ANYSTR], SUBTYPE_INSTANT)]
+ ("machine", u"machine ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("creator", u"and creator ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("id", u"and emd5 ==", [ANYSTR], SUBTYPE_INSTANT),
+ ('reset', u'clear all parameters', False, SUBTYPE_INSTANT)]
def __init__(self, *args, **kargs):
Worker.Worker.__init__(self, *args, **kargs)
Modified: trunk/src/workers/tools/tools/SCSource.py
===================================================================
--- trunk/src/workers/tools/tools/SCSource.py 2010-06-22 10:09:24 UTC (rev 689)
+++ trunk/src/workers/tools/tools/SCSource.py 2010-06-29 12:01:34 UTC (rev 690)
@@ -53,13 +53,14 @@
VERSION = 1
REVISION = "$Revision$"[11:-1]
name = "SampleContainer"
- _params = [("machine", u"machine ==", [ANYSTR], SUBTYPE_INSTANT),
- ("creator", u"and creator ==", [ANYSTR], SUBTYPE_INSTANT),
- ("longname", u"and longname ==", [ANYSTR], SUBTYPE_INSTANT),
+ _params = [("longname", u"and longname ==", [ANYSTR], SUBTYPE_INSTANT),
("shortname", u"and shortname ==", [ANYSTR], SUBTYPE_INSTANT),
("has_col", u"and has column", [ANYSTR], SUBTYPE_INSTANT),
("col_of", u"and is column of", [ANYSTR], SUBTYPE_INSTANT),
- ("id", u"and emd5 ==", [ANYSTR], SUBTYPE_INSTANT)]
+ ("machine", u"machine ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("creator", u"and creator ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("id", u"and emd5 ==", [ANYSTR], SUBTYPE_INSTANT),
+ ('reset', u'clear all parameters', False, SUBTYPE_INSTANT)]
def __init__(self, *args, **kargs):
Worker.Worker.__init__(self, *args, **kargs)
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-06-22 10:09:30
|
Revision: 689
http://pyphant.svn.sourceforge.net/pyphant/?rev=689&view=rev
Author: zklaus
Date: 2010-06-22 10:09:24 +0000 (Tue, 22 Jun 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Fix: Remove bad conversion to baseunits for older fmf files.
Fix: WebInterface
Fix: DCSource
Modified Paths:
--------------
trunk/src/pyphant/pyphant/core/WebInterface.py
trunk/src/pyphant/pyphant/quantities/ParseQuantities.py
trunk/src/workers/tools/tools/DCSource.py
Modified: trunk/src/pyphant/pyphant/core/WebInterface.py
===================================================================
--- trunk/src/pyphant/pyphant/core/WebInterface.py 2010-06-18 10:08:13 UTC (rev 688)
+++ trunk/src/pyphant/pyphant/core/WebInterface.py 2010-06-22 10:09:24 UTC (rev 689)
@@ -268,7 +268,7 @@
class HTMLFCScheme(object):
def __init__(self, fc_id, kn):
self.dom = kn.km.search(['shortname', 'latex_unit'],
- {'type':'field', 'dim_of':fc_id})
+ {'type':'field', 'dim_of':{'id':fc_id}})
self.rng = kn.km.search(['shortname', 'latex_unit'],
{'type':'field', 'id':fc_id})[0]
self.latex = '$%s$(%s)[%s]'
@@ -286,7 +286,8 @@
class HTMLSCScheme(object):
def __init__(self, sc_id, kn):
- columns = kn.km.search(['id'], {'type':'field', 'col_of':sc_id})
+ columns = kn.km.search(['id'], {'type':'field',
+ 'col_of':{'id':sc_id}})
self.fc_schemes = [HTMLFCScheme(col[0], kn) for col in columns]
self.shortname = kn.km.search(['shortname'], {'type':'sample',
'id':sc_id})[0][0]
@@ -306,7 +307,7 @@
def __init__(self, dc_id, kn):
child = cond(dc_id.endswith('field'), ('dim_of', 'col_of'))
result = kn.km.search(
- ['id', 'longname'], {'type':'field', child:dc_id})
+ ['id', 'longname'], {'type':'field', child:{'id':dc_id}})
rows = [[HTMLSummaryLink(res) for res in result]]
HTMLTable.__init__(self, rows, headings=False)
Modified: trunk/src/pyphant/pyphant/quantities/ParseQuantities.py
===================================================================
--- trunk/src/pyphant/pyphant/quantities/ParseQuantities.py 2010-06-18 10:08:13 UTC (rev 688)
+++ trunk/src/pyphant/pyphant/quantities/ParseQuantities.py 2010-06-22 10:09:24 UTC (rev 689)
@@ -73,7 +73,7 @@
unit = Quantity(unit.encode('utf-8'))
elif FMFversion=='1.0':
unit1_0 = PhysicalQuantity(unit.encode('utf-8'))
- unit = Quantity(str(unit1_0.inBaseUnits()))
+ unit = Quantity(str(unit1_0))
except:
unit = float(unit)
return unit
Modified: trunk/src/workers/tools/tools/DCSource.py
===================================================================
--- trunk/src/workers/tools/tools/DCSource.py 2010-06-18 10:08:13 UTC (rev 688)
+++ trunk/src/workers/tools/tools/DCSource.py 2010-06-22 10:09:24 UTC (rev 689)
@@ -63,7 +63,8 @@
if event.expectedValue != self.expectedValues[event.param.name]:
self.expectedValues[event.param.name] = event.expectedValue
if not hasattr(event, 'norefresh'):
- self.refreshParams(update=False)
+ autoSelect = event.expectedValue != ANYSTR
+ self.refreshParams(update=False, autoSelect=autoSelect)
def onPO(self, event):
self.expectedValues[event.param.name] = event.newValue
@@ -95,7 +96,7 @@
else:
return name
- def refreshParams(self, subscriber=None, update=True):
+ def refreshParams(self, subscriber=None, update=True, autoSelect=True):
if update:
self.expectedValues = dict(
[(name, param.value) for name, param \
@@ -110,6 +111,6 @@
[self.getResKey(name)], search_dict=search_dict, distinct=True))
newEVs = [newEV[0] for newEV in newEVs]
param._eventDispatcher.dispatchEvent(
- PossibleValuesChangeExpected(param, newEVs, autoSelect=True))
+ PossibleValuesChangeExpected(param, newEVs, autoSelect))
if update:
param.possibleValues = newEVs
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-06-18 10:08:20
|
Revision: 688
http://pyphant.svn.sourceforge.net/pyphant/?rev=688&view=rev
Author: zklaus
Date: 2010-06-18 10:08:13 +0000 (Fri, 18 Jun 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Fix: Flatten some index arrays to account for old numpy versions.
Modified Paths:
--------------
trunk/src/workers/OSC/OSC/OscAbsorption.py
Modified: trunk/src/workers/OSC/OSC/OscAbsorption.py
===================================================================
--- trunk/src/workers/OSC/OSC/OscAbsorption.py 2010-06-17 18:18:00 UTC (rev 687)
+++ trunk/src/workers/OSC/OSC/OscAbsorption.py 2010-06-18 10:08:13 UTC (rev 688)
@@ -103,7 +103,9 @@
dim = Abso.dimensions[-1]
minVal = quantities.Quantity('654nm')/dim.unit
maxVal = quantities.Quantity('660nm')/dim.unit
- lamp_interval = numpy.intersect1d(numpy.argwhere(minVal < dim.data), numpy.argwhere(dim.data<maxVal))
+ high_part = numpy.argwhere(minVal<dim.data).flatten()
+ low_part = numpy.argwhere(dim.data<maxVal).flatten()
+ lamp_interval = numpy.intersect1d(high_part, low_part)
min_index = lamp_interval[0]
max_index = lamp_interval[-1]
steps = max_index-min_index
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-06-17 18:18:07
|
Revision: 687
http://pyphant.svn.sourceforge.net/pyphant/?rev=687&view=rev
Author: zklaus
Date: 2010-06-17 18:18:00 +0000 (Thu, 17 Jun 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Enh: Remove bad prints.
Modified Paths:
--------------
trunk/src/workers/OSC/OSC/OscAbsorption.py
Modified: trunk/src/workers/OSC/OSC/OscAbsorption.py
===================================================================
--- trunk/src/workers/OSC/OSC/OscAbsorption.py 2010-06-17 18:05:17 UTC (rev 686)
+++ trunk/src/workers/OSC/OSC/OscAbsorption.py 2010-06-17 18:18:00 UTC (rev 687)
@@ -111,8 +111,6 @@
min_value = row[min_index]
max_value = row[max_index]
step_size = (max_value-min_value)/steps
- print min_value, max_value, step_size, steps, min_index, max_index
- print (lamp_interval-min_index)*step_size
row[lamp_interval] = min_value + (lamp_interval-min_index)*step_size
Abso.seal()
return Abso
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-06-17 18:05:24
|
Revision: 686
http://pyphant.svn.sourceforge.net/pyphant/?rev=686&view=rev
Author: zklaus
Date: 2010-06-17 18:05:17 +0000 (Thu, 17 Jun 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Enh: Added possible masking of lamp peak to Absorption calculator.
Enh: More parameters for FCSource and DCSource
Enh: New keywords for DataContainer search
Modified Paths:
--------------
trunk/src/pyphant/pyphant/core/KnowledgeManager.py
trunk/src/pyphant/pyphant/core/Param.py
trunk/src/pyphant/pyphant/core/SQLiteWrapper.py
trunk/src/pyphant/pyphant/tests/TestKnowledgeManager.py
trunk/src/pyphant/pyphant/tests/TestSQLiteWrapper.py
trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ListSelect.py
trunk/src/workers/OSC/OSC/OscAbsorption.py
trunk/src/workers/tools/tools/FCSource.py
trunk/src/workers/tools/tools/SCSource.py
trunk/src/workers/tools/tools/__init__.py
Added Paths:
-----------
trunk/src/workers/tools/tools/DCSource.py
Modified: trunk/src/pyphant/pyphant/core/KnowledgeManager.py
===================================================================
--- trunk/src/pyphant/pyphant/core/KnowledgeManager.py 2010-06-17 16:17:44 UTC (rev 685)
+++ trunk/src/pyphant/pyphant/core/KnowledgeManager.py 2010-06-17 18:05:17 UTC (rev 686)
@@ -435,52 +435,7 @@
def search(self, result_keys, search_dict={}, order_by=None,
order_asc=True, limit=-1, offset=0, distinct=False):
"""
- returns a list of tuples filled with values of the result keys
- matching the constraints of search_dict.
- Arguments:
- - result_keys: List (of length >= 1) of keys to include in the
- result tuples.
- - search_dict: Dict mapping keys to constraint values.
- Use empty dict for no constraints at all
- possible keys: values (used relational operator[, type constraint]):
- 'longname': str types (==)
- 'shortname': str types (==)
- 'machine': str types (==)
- 'creator: str types (==)
- 'date_from:' str types:
- YYYY[-MM[-DD[_hh:[mm:[ss[.s[s[s[s[s[s]]]]]]]]]]] (>=)
- 'date_to:' str types:
- YYYY[-MM[-DD[_hh:[mm:[ss[.s[s[s[s[s[s]]]]]]]]]]] (<)
- 'hash': str types (==)
- 'id': str types: emd5 (==)
- 'type': 'field' or 'sample' (==)
- 'attributes': dict mapping attr. key to attr. value (==)
- use (SQLiteWrapper instance).any_value
- or (KM instance).any_value to skip value check
- 'storage': str types (==)
- 'unit': PhysicalUnit or number or Quantity (==, FC only)
- 'dimensions': list of FC search dicts
- (see above definitions, FC only)
- 'columns': list of FC search dicts (see above definitions, SC only)
- - order_by: element of result_keys to order the results by
- or None for no special ordering
- - order_asc: whether to order ascending
- - limit: maximum number of results to return,
- set to -1 for no limit, default: -1
- - offset: number of search results to skip, default: 0
- - distinct: flag that indicates whether the result list
- should only contain distinct tuples.
- Usage Examples:
- Get list of all longnames:
- get_andsearch_result(['longname'], distinct=True)
- --> [('name1', ), ('name2', ), ...]
- Get id and shortname of all FCs that are parametrized by
- a time dimension along the primary axis:
- tunit = Quantity(1, 's')
- get_andsearch_result(['id', 'shortname'],
- {'type':'field',
- 'dimensions':[{'unit':tunit}]})
- --> [('emd5_1', 'name_1'), ('emd5_2', 'name_2'), ...]
+ see SQLiteWrapper.get_andsearch_result
"""
with SQLiteWrapper(self.dbase) as wrapper:
return wrapper.get_andsearch_result(
Modified: trunk/src/pyphant/pyphant/core/Param.py
===================================================================
--- trunk/src/pyphant/pyphant/core/Param.py 2010-06-17 16:17:44 UTC (rev 685)
+++ trunk/src/pyphant/pyphant/core/Param.py 2010-06-17 18:05:17 UTC (rev 686)
@@ -54,10 +54,10 @@
class PossibleValuesChangeExpected(object):
- def __init__(self, param, expectedPVs, update=False):
+ def __init__(self, param, expectedPVs, autoSelect=False):
self.param = param
self.expectedPVs = expectedPVs
- self.update = update
+ self.autoSelect = autoSelect
class ParamChangeRequested(object):
Modified: trunk/src/pyphant/pyphant/core/SQLiteWrapper.py
===================================================================
--- trunk/src/pyphant/pyphant/core/SQLiteWrapper.py 2010-06-17 16:17:44 UTC (rev 685)
+++ trunk/src/pyphant/pyphant/core/SQLiteWrapper.py 2010-06-17 18:05:17 UTC (rev 686)
@@ -187,11 +187,11 @@
one_to_one_search_keys = ['longname', 'shortname', 'machine',
'creator', 'hash', 'storage']
one_to_one_result_keys = one_to_one_search_keys + ['date', 'id', 'type']
- common_search_keys = one_to_one_search_keys + ['id', 'attributes',
- 'date_from', 'date_to']
+ common_search_keys = one_to_one_search_keys + \
+ ['id', 'attributes', 'date_from', 'date_to', 'col_of']
fc_search_keys = common_search_keys + ['unit', 'dimensions',
- 'dim_of', 'col_of']
- sc_search_keys = common_search_keys + ['columns']
+ 'dim_of', 'has_dim']
+ sc_search_keys = common_search_keys + ['columns', 'has_col']
sortable_keys = common_keys + ['storage', 'type']
any_value = AnyValue()
@@ -506,6 +506,59 @@
expr += ' AND '
return (expr[:-5] + ')', new_value, True)
+ def translate_parent_search(self, key, dc_search_dict, type):
+ id_str = replace_type('%s_id', type)
+ if dc_search_dict.has_key('type'):
+ check = dc_search_dict.pop('type')
+ else:
+ check = None
+ if key == 'col_of':
+ assert check in [None, 'sample']
+ parent_type = 'sample'
+ parent_id_str = 'sc_id'
+ child_id_str = 'fc_id'
+ table = 'km_sc_columns'
+ elif key == 'dim_of':
+ assert check in [None, 'field']
+ parent_type = 'field'
+ parent_id_str = 'fc_id'
+ child_id_str = 'dim_id'
+ table = 'km_fc_dimensions'
+ dc_query, dc_values = self.get_andsearch_query(
+ parent_type, ['id'], dc_search_dict, False)
+ expr = "(%s IN (SELECT %s FROM %s WHERE %s IN (%s)))" \
+ % (id_str, child_id_str, table, parent_id_str, dc_query)
+ return (expr, dc_values, True)
+
+ def translate_child_search(self, key, child_search_dict):
+ if key == 'has_col':
+ if child_search_dict.has_key('type'):
+ child_type = child_search_dict.pop('type')
+ child_qry, child_values = self.get_andsearch_query(
+ child_type, ['id'], child_search_dict, False)
+ else:
+ fc_child_qry, fc_child_values = self.get_andsearch_query(
+ 'field', ['id'], child_search_dict, False)
+ sc_child_qry, sc_child_values = self.get_andsearch_query(
+ 'sample', ['id'], child_search_dict, False)
+ child_qry = "%s UNION ALL %s"
+ child_qry = child_qry % (fc_child_qry, sc_child_qry)
+ child_values = fc_child_values + sc_child_values
+ id_str = 'sc_id'
+ child_id_str = 'fc_id'
+ table = 'km_sc_columns'
+ elif key == 'has_dim':
+ if child_search_dict.has_key('type'):
+ assert child_search_dict.pop('type') == 'field'
+ child_qry, child_values = self.get_andsearch_query(
+ 'field', ['id'], child_search_dict, False)
+ id_str = 'fc_id'
+ child_id_str = 'dim_id'
+ table = 'km_fc_dimensions'
+ expr = "(%s IN (SELECT %s FROM %s WHERE %s IN (%s)))" \
+ % (id_str, id_str, table, child_id_str, child_qry)
+ return (expr, child_values, True)
+
def translate_search_dict(self, type, search_dict):
where = ''
values = []
@@ -528,19 +581,19 @@
elif key == 'columns' or key == 'dimensions':
expr, value, extend = self.translate_list_search(
key, value, type)
- elif key == 'dim_of':
- expr = '(fc_id IN (SELECT dim_id FROM km_fc_dimensions '\
- 'WHERE fc_id=?))'
- elif key == 'col_of':
- expr = '(fc_id IN (SELECT fc_id FROM km_sc_columns '\
- 'WHERE sc_id=?))'
+ elif key in ['dim_of', 'col_of']:
+ expr, value, extend = self.translate_parent_search(
+ key, value, type)
+ elif key in ['has_dim', 'has_col']:
+ expr, value, extend = self.translate_child_search(
+ key, value)
else:
raise NotImplementedError(key)
where += expr + " AND "
- if extend:
+ if not extend:
+ values.append(value)
+ elif value is not None:
values.extend(value)
- else:
- values.append(value)
return where[:-5], values
def get_andsearch_query(self, type, result_keys, search_dict, distinct):
@@ -595,9 +648,11 @@
'unit': PhysicalUnit or number or Quantity (==, FC only)
'dimensions': list of FC search dicts
(see above definitions, FC only)
- 'dim_of': str types: emd5 of parent FC (==, FC only)
- 'col_of': str types: emd5 of parent SC (==, FC only)
+ 'dim_of': FC search dict (see above definitions, FC only)
+ 'col_of': SC search dict (see above definitions)
'columns': list of FC search dicts (see above definitions, SC only)
+ 'has_col': DC search dict (see above defs, SC only)
+ 'had_dim': FC search dict (see above definitions, FC only)
- order_by: element of result_keys to order the results by
or None for no special ordering
- order_asc: whether to order ascending
@@ -623,11 +678,8 @@
else:
assert order_by in result_keys
assert order_by in self.sortable_keys
- order = ' ORDER BY %s' % order_by
- if order_asc:
- order += ' ASC'
- else:
- order += ' DESC'
+ order = ' ORDER BY %s %s' \
+ % (order_by, {True:'ASC', False:'DESC'}[order_asc])
assert isinstance(limit, int)
assert isinstance(offset, int)
if not search_dict.has_key('type'):
@@ -665,7 +717,7 @@
query, values = self.get_andsearch_query(
search_dict['type'], result_keys, mod_search_dict, distinct)
query = "%s%s LIMIT %d OFFSET %d" % (query, order, limit, offset)
- if mod_search_dict != {}:
+ if values is not None and len(values) > 0:
self.cursor.execute(query, values)
else:
self.cursor.execute(query)
Modified: trunk/src/pyphant/pyphant/tests/TestKnowledgeManager.py
===================================================================
--- trunk/src/pyphant/pyphant/tests/TestKnowledgeManager.py 2010-06-17 16:17:44 UTC (rev 685)
+++ trunk/src/pyphant/pyphant/tests/TestKnowledgeManager.py 2010-06-17 18:05:17 UTC (rev 686)
@@ -45,7 +45,7 @@
CACHE_MAX_SIZE,
CACHE_MAX_NUMBER)
import pyphant.core.PyTablesPersister as ptp
-from pyphant.core.DataContainer import FieldContainer
+from pyphant.core.DataContainer import (FieldContainer, SampleContainer)
import numpy as N
import tables
import urllib
@@ -104,6 +104,21 @@
km_fc = km.getDataContainer(self._fc.id)
self.assertEqual(self._fc, km_fc)
+ def testSCwithSCColumn(self):
+ fc_child1 = FieldContainer(longname='fc_child1', data=N.ones((10, 10)))
+ fc_child2 = FieldContainer(longname='fc_child2', data=N.ones((20, 20)))
+ sc_child = SampleContainer(longname='sc_child', columns=[fc_child1])
+ sc_parent = SampleContainer(longname='sc_parent', columns=[sc_child,
+ fc_child2])
+ sc_parent.seal()
+ km = KnowledgeManager.getInstance()
+ km.registerDataContainer(sc_parent, temporary=True)
+ lnlist = km.search(['longname'], {'col_of':{'longname':'sc_parent'}})
+ lnlist = [entry[0] for entry in lnlist]
+ assert len(lnlist) == 2
+ assert 'fc_child2' in lnlist
+ assert 'sc_child' in lnlist
+
def testExceptions(self):
km = KnowledgeManager.getInstance()
#TODO:
Modified: trunk/src/pyphant/pyphant/tests/TestSQLiteWrapper.py
===================================================================
--- trunk/src/pyphant/pyphant/tests/TestSQLiteWrapper.py 2010-06-17 16:17:44 UTC (rev 685)
+++ trunk/src/pyphant/pyphant/tests/TestSQLiteWrapper.py 2010-06-17 18:05:17 UTC (rev 686)
@@ -193,10 +193,11 @@
rowwrapper = self.wrapper[id]
assert rowwrapper['dimensions'] == self.summary['dimensions']
search_result = self.wrapper.get_andsearch_result(
- ['longname'], {'type':'field', 'dim_of':id})
+ ['longname'], {'type':'field', 'dim_of':{'id':id}})
assert search_result == [(im_summary['longname'], )]
search_result = self.wrapper.get_andsearch_result(
- ['shortname'], {'type':'field', 'col_of':self.sc_summary['id']})
+ ['shortname'], {'type':'field',
+ 'col_of':{'id':self.sc_summary['id']}})
assert search_result == [(self.summary['shortname'], )]
Modified: trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ListSelect.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ListSelect.py 2010-06-17 16:17:44 UTC (rev 685)
+++ trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ListSelect.py 2010-06-17 18:05:17 UTC (rev 686)
@@ -73,29 +73,27 @@
self.Bind(EVT_CHOICE, self.onChoice)
self.param = param
self.possibleValues = param.possibleValues
- self.selected = param.value
param._eventDispatcher.registerExclusiveListener(
self.onPVCE, PossibleValuesChangeExpected)
- def setSelected(self, value):
- self._selected = value
-
- def getSelected(self):
- return self._selected
- selected = property(getSelected, setSelected)
-
def onPVCE(self, event):
- value = self.selected
+ value = self.data[self.GetStringSelection()]
assert value in event.expectedPVs, "%s not in %s" % (value,
event.expectedPVs)
self.data = dict([(str(val), val) for val in event.expectedPVs])
self.possibleValues = event.expectedPVs
self.SetItems(map(str, event.expectedPVs))
- self.SetValue(value)
+ if event.autoSelect and len(event.expectedPVs) == 2:
+ self.SetSelection(1)
+ pce = ParamChangeExpected(
+ self.param, expectedValue=self.data[self.GetStringSelection()])
+ pce.norefresh = True
+ self.param._eventDispatcher.dispatchEvent(pce)
+ else:
+ self.SetValue(value)
def onChoice(self, event):
event.Skip()
- self.selected = self.data[self.GetStringSelection()]
self.param._eventDispatcher.dispatchEvent(
ParamChangeExpected(
self.param, expectedValue=self.data[self.GetStringSelection()]))
Modified: trunk/src/workers/OSC/OSC/OscAbsorption.py
===================================================================
--- trunk/src/workers/OSC/OSC/OscAbsorption.py 2010-06-17 16:17:44 UTC (rev 685)
+++ trunk/src/workers/OSC/OSC/OscAbsorption.py 2010-06-17 18:05:17 UTC (rev 686)
@@ -80,7 +80,8 @@
name = "Compute Absorption"
_sockets = [("osc", Connectors.TYPE_ARRAY)]
- _params = [("clipping", "Clipping", 1, None)]
+ _params = [("clipping", "Clipping", 1, None),
+ ("mask_lamp", "Mask lamp", 0, None)]
def inithook(self):
self._logger = logging.getLogger("pyphant")
@@ -98,6 +99,21 @@
longname=u'absorption',
shortname=ur'\tilde{A}')
Abso.dimensions[-1] = I.dimensions[-1]
+ if self.paramMask_lamp.value==1:
+ dim = Abso.dimensions[-1]
+ minVal = quantities.Quantity('654nm')/dim.unit
+ maxVal = quantities.Quantity('660nm')/dim.unit
+ lamp_interval = numpy.intersect1d(numpy.argwhere(minVal < dim.data), numpy.argwhere(dim.data<maxVal))
+ min_index = lamp_interval[0]
+ max_index = lamp_interval[-1]
+ steps = max_index-min_index
+ for row in Abso.data:
+ min_value = row[min_index]
+ max_value = row[max_index]
+ step_size = (max_value-min_value)/steps
+ print min_value, max_value, step_size, steps, min_index, max_index
+ print (lamp_interval-min_index)*step_size
+ row[lamp_interval] = min_value + (lamp_interval-min_index)*step_size
Abso.seal()
return Abso
Copied: trunk/src/workers/tools/tools/DCSource.py (from rev 685, trunk/src/workers/tools/tools/FCSource.py)
===================================================================
--- trunk/src/workers/tools/tools/DCSource.py (rev 0)
+++ trunk/src/workers/tools/tools/DCSource.py 2010-06-17 18:05:17 UTC (rev 686)
@@ -0,0 +1,115 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2008, Rectorate of the University of Freiburg
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+# * Neither the name of the Freiburg Materials Research Center,
+# University of Freiburg nor the names of its contributors may be used to
+# endorse or promote products derived from this software without specific
+# prior written permission.
+#
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
+# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+This module provides a base class for FCSource and SCSource
+"""
+
+__id__ = "$Id$"
+__author__ = "$Author$"
+__version__ = "$Revision$"
+# $Source$
+
+from pyphant.core.KnowledgeManager import KnowledgeManager
+from pyphant.core.Param import (
+ ParamChangeExpected, PossibleValuesChangeExpected, ParamOverridden)
+ANYSTR = u"-- any --"
+
+
+class DCSource(object):
+ """
+ This is the base class for SCSource and FCSource. It has to
+ be subclassed before use.
+ """
+ def __init__(self, dc_type):
+ self.dc_type = dc_type
+ self.expectedValues = {}
+ for name, param in self._params.iteritems():
+ if name in ["name"]:
+ continue
+ param.registerListener(self.onPCE, ParamChangeExpected)
+ param.registerListener(self.onPO, ParamOverridden)
+ self.expectedValues[name] = param.value
+
+ def onPCE(self, event):
+ if event.expectedValue != self.expectedValues[event.param.name]:
+ self.expectedValues[event.param.name] = event.expectedValue
+ if not hasattr(event, 'norefresh'):
+ self.refreshParams(update=False)
+
+ def onPO(self, event):
+ self.expectedValues[event.param.name] = event.newValue
+
+ def getKeyValue(self, key, value, name):
+ if key in ['col_of', 'dim_of', 'has_dim', 'has_col']:
+ value = {'longname':value}
+ return (key, value)
+
+ def getSearchDict(self, name):
+ search_dict = {'type':self.dc_type}
+ search_dict.update(dict(
+ [self.getKeyValue(key, val, name) \
+ for key, val in self.expectedValues.iteritems() \
+ if key not in ['name', name] and val != ANYSTR]))
+ if name == 'dim_of':
+ search_dict = {'type':'field', 'has_dim':search_dict}
+ elif name == 'col_of':
+ search_dict = {'type':'sample', 'has_col':search_dict}
+ elif name == 'has_dim':
+ search_dict = {'type':'field', 'dim_of':search_dict}
+ elif name == 'has_col':
+ search_dict = {'col_of':search_dict}
+ return search_dict
+
+ def getResKey(self, name):
+ if name in ['col_of', 'dim_of', 'has_col', 'has_dim']:
+ return 'longname'
+ else:
+ return name
+
+ def refreshParams(self, subscriber=None, update=True):
+ if update:
+ self.expectedValues = dict(
+ [(name, param.value) for name, param \
+ in self._params.iteritems() if name != 'name'])
+ kmanager = KnowledgeManager.getInstance()
+ for name, param in self._params.iteritems():
+ if name == 'name':
+ continue
+ search_dict = self.getSearchDict(name)
+ newEVs = [[ANYSTR]]
+ newEVs.extend(kmanager.search(
+ [self.getResKey(name)], search_dict=search_dict, distinct=True))
+ newEVs = [newEV[0] for newEV in newEVs]
+ param._eventDispatcher.dispatchEvent(
+ PossibleValuesChangeExpected(param, newEVs, autoSelect=True))
+ if update:
+ param.possibleValues = newEVs
Modified: trunk/src/workers/tools/tools/FCSource.py
===================================================================
--- trunk/src/workers/tools/tools/FCSource.py 2010-06-17 16:17:44 UTC (rev 685)
+++ trunk/src/workers/tools/tools/FCSource.py 2010-06-17 18:05:17 UTC (rev 686)
@@ -39,16 +39,12 @@
__version__ = "$Revision$"
# $Source$
-from pyphant.core import (Worker, Connectors,
- Param)
+from pyphant.core import (Worker, Connectors, Param)
+from DCSource import (ANYSTR, DCSource)
from pyphant.core.KnowledgeManager import KnowledgeManager
-from pyphant.core.Param import (
- ParamChangeExpected, PossibleValuesChangeExpected, ParamOverridden)
from pyphant.core.Connectors import SUBTYPE_INSTANT
-ANYSTR = u"-- any --"
-
-class FCSource(Worker.Worker):
+class FCSource(DCSource, Worker.Worker):
"""
This worker provides instantly updated dropdown lists for selecting
a FieldContainer from the KnowledgeManager.
@@ -57,55 +53,23 @@
VERSION = 1
REVISION = "$Revision$"[11:-1]
name = "FieldContainer"
- _params = [("machine", u"Machine", [ANYSTR], SUBTYPE_INSTANT),
- ("creator", u"Creator", [ANYSTR], SUBTYPE_INSTANT),
- ("longname", u"Longname", [ANYSTR], SUBTYPE_INSTANT),
- ("shortname", u"Shortname", [ANYSTR], SUBTYPE_INSTANT),
- ("id", u"emd5", [ANYSTR], SUBTYPE_INSTANT)]
+ _params = [("machine", u"machine ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("creator", u"and creator ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("longname", u"and longname ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("shortname", u"and shortname ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("unit", u"and unit is compatible to", [ANYSTR],
+ SUBTYPE_INSTANT),
+ ("has_dim", u"and has dimension", [ANYSTR], SUBTYPE_INSTANT),
+ ("col_of", u"and is column of", [ANYSTR], SUBTYPE_INSTANT),
+ ("dim_of", u"and is dimension of", [ANYSTR], SUBTYPE_INSTANT),
+ ("id", u"and emd5 ==", [ANYSTR], SUBTYPE_INSTANT)]
def __init__(self, *args, **kargs):
Worker.Worker.__init__(self, *args, **kargs)
- self.expectedValues = {}
- for name, param in self._params.iteritems():
- if name in ["name"]:
- continue
- param.registerListener(self.onPCE, ParamChangeExpected)
- param.registerListener(self.onPO, ParamOverridden)
- self.expectedValues[name] = param.value
+ DCSource.__init__(self, 'field')
- def onPCE(self, event):
- if event.expectedValue != self.expectedValues[event.param.name]:
- self.expectedValues[event.param.name] = event.expectedValue
- self.refreshParams(update=False)
-
- def onPO(self, event):
- self.expectedValues[event.param.name] = event.newValue
-
- def refreshParams(self, subscriber=None, update=True):
- if update:
- self.expectedValues = dict(
- [(name, param.value) for name, param \
- in self._params.iteritems() if name != 'name'])
- kmanager = KnowledgeManager.getInstance()
- for name, param in self._params.iteritems():
- if name == 'name':
- continue
- search_dict = {'type':'field'}
- update_dict = dict(
- [(key, val) for key, val in self.expectedValues.iteritems() \
- if key not in ['name', name] and val != ANYSTR])
- search_dict.update(update_dict)
- newEVs = [[ANYSTR]]
- newEVs.extend(kmanager.search([name], search_dict=search_dict,
- distinct=True))
- newEVs = [newEV[0] for newEV in newEVs]
- param._eventDispatcher.dispatchEvent(
- PossibleValuesChangeExpected(param, newEVs))
- if update:
- param.possibleValues = newEVs
-
@Worker.plug(Connectors.TYPE_IMAGE)
- def getDataContainer(self, subscriber = 0):
+ def getFieldContainer(self, subscriber = 0):
emd5 = self.paramId.value
kmanager = KnowledgeManager.getInstance()
return kmanager.getDataContainer(emd5)
Modified: trunk/src/workers/tools/tools/SCSource.py
===================================================================
--- trunk/src/workers/tools/tools/SCSource.py 2010-06-17 16:17:44 UTC (rev 685)
+++ trunk/src/workers/tools/tools/SCSource.py 2010-06-17 18:05:17 UTC (rev 686)
@@ -39,16 +39,12 @@
__version__ = "$Revision$"
# $Source$
-from pyphant.core import (Worker, Connectors,
- Param)
+from pyphant.core import (Worker, Connectors, Param)
+from DCSource import (ANYSTR, DCSource)
from pyphant.core.KnowledgeManager import KnowledgeManager
-from pyphant.core.Param import (
- ParamChangeExpected, PossibleValuesChangeExpected, ParamOverridden)
from pyphant.core.Connectors import SUBTYPE_INSTANT
-ANYSTR = u"-- any --"
-
-class SCSource(Worker.Worker):
+class SCSource(DCSource, Worker.Worker):
"""
This worker provides instantly updated dropdown lists for selecting
a SampleContainer from the KnowledgeManager.
@@ -57,55 +53,20 @@
VERSION = 1
REVISION = "$Revision$"[11:-1]
name = "SampleContainer"
- _params = [("machine", u"Machine", [ANYSTR], SUBTYPE_INSTANT),
- ("creator", u"Creator", [ANYSTR], SUBTYPE_INSTANT),
- ("longname", u"Longname", [ANYSTR], SUBTYPE_INSTANT),
- ("shortname", u"Shortname", [ANYSTR], SUBTYPE_INSTANT),
- ("id", u"emd5", [ANYSTR], SUBTYPE_INSTANT)]
+ _params = [("machine", u"machine ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("creator", u"and creator ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("longname", u"and longname ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("shortname", u"and shortname ==", [ANYSTR], SUBTYPE_INSTANT),
+ ("has_col", u"and has column", [ANYSTR], SUBTYPE_INSTANT),
+ ("col_of", u"and is column of", [ANYSTR], SUBTYPE_INSTANT),
+ ("id", u"and emd5 ==", [ANYSTR], SUBTYPE_INSTANT)]
def __init__(self, *args, **kargs):
Worker.Worker.__init__(self, *args, **kargs)
- self.expectedValues = {}
- for name, param in self._params.iteritems():
- if name in ["name"]:
- continue
- param.registerListener(self.onPCE, ParamChangeExpected)
- param.registerListener(self.onPO, ParamOverridden)
- self.expectedValues[name] = param.value
+ DCSource.__init__(self, 'sample')
- def onPCE(self, event):
- if event.expectedValue != self.expectedValues[event.param.name]:
- self.expectedValues[event.param.name] = event.expectedValue
- self.refreshParams(update=False)
-
- def onPO(self, event):
- self.expectedValues[event.param.name] = event.newValue
-
- def refreshParams(self, subscriber=None, update=True):
- if update:
- self.expectedValues = dict(
- [(name, param.value) for name, param \
- in self._params.iteritems() if name != 'name'])
- kmanager = KnowledgeManager.getInstance()
- for name, param in self._params.iteritems():
- if name == 'name':
- continue
- search_dict = {'type':'sample'}
- update_dict = dict(
- [(key, val) for key, val in self.expectedValues.iteritems() \
- if key not in ['name', name] and val != ANYSTR])
- search_dict.update(update_dict)
- newEVs = [[ANYSTR]]
- newEVs.extend(kmanager.search([name], search_dict=search_dict,
- distinct=True))
- newEVs = [newEV[0] for newEV in newEVs]
- param._eventDispatcher.dispatchEvent(
- PossibleValuesChangeExpected(param, newEVs))
- if update:
- param.possibleValues = newEVs
-
@Worker.plug(Connectors.TYPE_ARRAY)
- def getDataContainer(self, subscriber = 0):
+ def getSampleContainer(self, subscriber = 0):
emd5 = self.paramId.value
kmanager = KnowledgeManager.getInstance()
return kmanager.getDataContainer(emd5)
Modified: trunk/src/workers/tools/tools/__init__.py
===================================================================
--- trunk/src/workers/tools/tools/__init__.py 2010-06-17 16:17:44 UTC (rev 685)
+++ trunk/src/workers/tools/tools/__init__.py 2010-06-17 18:05:17 UTC (rev 686)
@@ -40,7 +40,7 @@
# $Source$
workers=[
- "Emd5Src",
+ #"Emd5Src",
"FCSource",
"SCSource"
]
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-06-17 16:17:53
|
Revision: 685
http://pyphant.svn.sourceforge.net/pyphant/?rev=685&view=rev
Author: zklaus
Date: 2010-06-17 16:17:44 +0000 (Thu, 17 Jun 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Enh: rich comparison for Quantity
Enh: Added ZStack Visualizer
Enh: Refactored ZStack and AutoFocus
Enh: Emd5Src
Fix: SQLiteWrapper, SliceSelector
Fix: Removed unused workers
Enh: Feedback during update
Fix: WebInterface order_by
Cosm: gitignore
Fix: Handle broken toolboxes
Fix: Typo
Enh: NDImage Worker
Enh: Improved ZStack
Modified Paths:
--------------
trunk/.gitignore
trunk/src/pyphant/pyphant/cli/pyphant_exec.py
trunk/src/pyphant/pyphant/core/Connectors.py
trunk/src/pyphant/pyphant/core/EventDispatcher.py
trunk/src/pyphant/pyphant/core/Param.py
trunk/src/pyphant/pyphant/core/PyTablesPersister.py
trunk/src/pyphant/pyphant/core/SQLiteWrapper.py
trunk/src/pyphant/pyphant/core/WebInterface.py
trunk/src/pyphant/pyphant/core/ZStackManager.py
trunk/src/pyphant/pyphant/quantities/__init__.py
trunk/src/pyphant/pyphant/tests/TestQuantities.py
trunk/src/pyphant/pyphant/tests/TestZStacks.py
trunk/src/pyphant/pyphant/visualizers/__init__.py
trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ListSelect.py
trunk/src/pyphant/pyphant/wxgui2/paramvisualization/ParamVisReg.py
trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py
trunk/src/workers/ImageProcessing/ImageProcessing/AutoFocus.py
trunk/src/workers/ImageProcessing/ImageProcessing/NDImageWorker.py
trunk/src/workers/ImageProcessing/ImageProcessing/SliceSelector.py
trunk/src/workers/ImageProcessing/ImageProcessing/__init__.py
trunk/src/workers/tools/tools/Emd5Src.py
trunk/src/workers/tools/tools/__init__.py
trunk/src/workers/tools/tools/tests/TestEmd5Source.py
Added Paths:
-----------
trunk/src/pyphant/pyphant/visualizers/ZStackVisualizer.py
trunk/src/workers/ImageProcessing/ImageProcessing/MarkAF.py
trunk/src/workers/tools/tools/FCSource.py
trunk/src/workers/tools/tools/SCSource.py
Removed Paths:
-------------
trunk/src/pyphant/pyphant/core/RecipeAlg.py
trunk/src/pyphant/pyphant/tests/TestAutoFocus.py
trunk/src/pyphant/pyphant/tests/resources/recipes/AutoFocus.h5
trunk/src/pyphant/pyphant/tests/resources/recipes/MeasureFocus.h5
trunk/src/pyphant/pyphant/tests/resources/recipes/pre-MF-no-median.h5
trunk/src/workers/ImageProcessing/ImageProcessing/MeasureFocus.py
trunk/src/workers/ImageProcessing/ImageProcessing/ZStacks.py
trunk/src/workers/tools/tools/BatchExtractor.py
trunk/src/workers/tools/tools/BatchHead.py
trunk/src/workers/tools/tools/BatchTail.py
trunk/src/workers/tools/tools/ParameterRun.py
Modified: trunk/.gitignore
===================================================================
--- trunk/.gitignore 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/.gitignore 2010-06-17 16:17:44 UTC (rev 685)
@@ -17,3 +17,4 @@
*.out
*.bst
*.bbl
+.ropeproject/
Modified: trunk/src/pyphant/pyphant/cli/pyphant_exec.py
===================================================================
--- trunk/src/pyphant/pyphant/cli/pyphant_exec.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/cli/pyphant_exec.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -59,10 +59,10 @@
w = recipe.getWorker(sSpec[0])
s = getattr(w, sSpec[-1])
src = Emd5Src.Emd5Src(recipe)
- src.paramDc.value=emd5
+ src.paramEmd5.value=emd5
if s.isFull():
s.pullPlug()
- s.insert(src.plugLoad)
+ s.insert(src.plugGetDataContainer)
pSpec = order[1][0].split('.')
d = recipe.getWorker(pSpec[0])
plug = getattr(d, pSpec[1])
Modified: trunk/src/pyphant/pyphant/core/Connectors.py
===================================================================
--- trunk/src/pyphant/pyphant/core/Connectors.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/core/Connectors.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -48,6 +48,7 @@
TYPE_INT = type(2)
TYPE_STRING = type("")
SUBTYPE_FILE = "filename"
+SUBTYPE_INSTANT = "instant"
TYPE_INACTIVE = type(None)
TYPE_BOOL = type(True)
DEFAULT_DATA_TYPE = TYPE_ARRAY
Modified: trunk/src/pyphant/pyphant/core/EventDispatcher.py
===================================================================
--- trunk/src/pyphant/pyphant/core/EventDispatcher.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/core/EventDispatcher.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -47,6 +47,14 @@
except KeyError:
self._listeners[eventType] = [listener]
+ # The following method is a quick workaround for the shortcoming
+ # of this EventDispatcher which offers no control, in which
+ # order listeners for the same event are called and the
+ # event interface is not specified at all, so there is no
+ # functionality like event.skip()
+ def registerExclusiveListener(self, listener, eventType):
+ self._listeners[eventType] = [listener]
+
def unregisterListener(self, listener, eventType=None):
if eventType:
self._listeners[eventType].remove( listener )
Modified: trunk/src/pyphant/pyphant/core/Param.py
===================================================================
--- trunk/src/pyphant/pyphant/core/Param.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/core/Param.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -47,18 +47,40 @@
return Param(worker, paramName, displayName, values, subtype)
+class ParamChangeExpected(object):
+ def __init__(self, param, expectedValue):
+ self.param = param
+ self.expectedValue = expectedValue
+
+
+class PossibleValuesChangeExpected(object):
+ def __init__(self, param, expectedPVs, update=False):
+ self.param = param
+ self.expectedPVs = expectedPVs
+ self.update = update
+
+
class ParamChangeRequested(object):
def __init__(self, param, oldValue, newValue):
self.param = param
self.oldValue = oldValue
self.newValue = newValue
+
class ParamChanged(object):
def __init__(self, param, oldValue, newValue):
self.param = param
self.oldValue = oldValue
self.newValue = newValue
+
+class ParamOverridden(object):
+ def __init__(self, param, oldValue, newValue):
+ self.param = param
+ self.oldValue = oldValue
+ self.newValue = newValue
+
+
class VetoParamChange(ValueError):
def __init__(self, paramChangeEvent):
ValueError.__init__(self, "Veto: change %s from %s to %s." %
@@ -67,6 +89,7 @@
paramChangeEvent.newValue))
self.paramChangeEvent = paramChangeEvent
+
class Param(Connectors.Socket):
def __getValue(self):
if self.isFull():
@@ -75,7 +98,10 @@
return self._value
def overrideValue(self, value):
+ oldValue = self._value
self._value = value
+ self._eventDispatcher.dispatchEvent(
+ ParamOverridden(self, oldValue, value))
def __setValue(self, value):
oldValue = self.value
Modified: trunk/src/pyphant/pyphant/core/PyTablesPersister.py
===================================================================
--- trunk/src/pyphant/pyphant/core/PyTablesPersister.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/core/PyTablesPersister.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -68,7 +68,6 @@
from tables import StringCol, Col
from pyphant.quantities import Quantity
from pyphant.quantities import Quantity as PhysicalQuantity
-from ImageProcessing.AutoFocus import FocusSlice # For loading FCs...
import scipy
import logging
_logger = logging.getLogger("pyphant")
Deleted: trunk/src/pyphant/pyphant/core/RecipeAlg.py
===================================================================
--- trunk/src/pyphant/pyphant/core/RecipeAlg.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/core/RecipeAlg.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -1,148 +0,0 @@
-#!/usr/bin/env python2.5
-# -*- coding: utf-8 -*-
-
-# Copyright (c) 2006-2009, Rectorate of the University of Freiburg
-# All rights reserved.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# * Redistributions of source code must retain the above copyright
-# notice, this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright
-# notice, this list of conditions and the following disclaimer in the
-# documentation and/or other materials provided with the distribution.
-# * Neither the name of the Freiburg Materials Research Center,
-# University of Freiburg nor the names of its contributors may be used to
-# endorse or promote products derived from this software without specific
-# prior written permission.
-#
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
-# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
-# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
-# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
-# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-u"""Provides ZStackManager
-"""
-
-__id__ = "$Id$".replace('$','')
-__author__ = "$Author$".replace('$','')
-__version__ = "$Revision$".replace('$','')
-# $Source$
-
-from pyphant.core.KnowledgeManager import KnowledgeManager
-from pyphant.core.PyTablesPersister import loadRecipeFromHDF5File
-from tools import Emd5Src
-import os
-kmanager = KnowledgeManager.getInstance()
-
-
-class RecipeAlg(object):
- def __init__(self, recipe_fname, result_worker,
- result_plug, params={}):
- self.recipe_fname = recipe_fname
- self.recipe = loadRecipeFromHDF5File(recipe_fname)
- self.params = params
- self.rwn = result_worker
- self.rpn = result_plug
- self.result_plug = self.recipe.getWorker(
- result_worker).getPlug(result_plug)
-
- def get_result(self, input_ids, id_only=False, temporary=False):
- from pyphant.core.Helpers import utf82uc
- search_dict = {'attributes':{
- 'recipe':utf82uc(self.recipe_fname),
- 'result_worker':utf82uc(self.rwn),
- 'result_plug':utf82uc(self.rpn),
- 'applied_to':utf82uc(input_ids.__repr__()),
- 'params':utf82uc(self.params.__repr__())}}
- result_ids = kmanager.search(['id'], search_dict)
- if len(result_ids) > 0 and not temporary:
- if id_only:
- return result_ids[0][0]
- else:
- return kmanager.getDataContainer(result_ids[0][0])
- else:
- # Fill open sockets:
- for wname, sockets in input_ids.iteritems():
- worker = self.recipe.getWorker(wname)
- for sname, emd5 in sockets.iteritems():
- socket = worker.getSocket(sname)
- dummy = Emd5Src.Emd5Src()
- dummy.paramSelectby.value = u"enter emd5"
- dummy.paramEnteremd5.value = emd5
- socket.insert(dummy.getPlugs()[0])
- # Set parameters:
- for wname, wparams in self.params.iteritems():
- for pname, pvalue in wparams.iteritems():
- self.recipe.getWorker(wname).getParam(pname).value = pvalue
- # Get Result
- result_dc = self.result_plug.getResult()
- # Pull plugs
- for wname, sockets in input_ids.iteritems():
- worker = self.recipe.getWorker(wname)
- for sname, emd5 in sockets.iteritems():
- worker.getSocket(sname).pullPlug()
- result_dc.attributes['recipe'] = utf82uc(self.recipe_fname)
- result_dc.attributes['applied_to'] = utf82uc(input_ids.__repr__())
- result_dc.attributes['params'] = utf82uc(self.params.__repr__())
- result_dc.attributes['result_plug'] = utf82uc(self.rpn)
- result_dc.attributes['result_worker'] = utf82uc(self.rwn)
- kmanager.registerDataContainer(result_dc, temporary=temporary)
- if id_only:
- return result_dc.id
- else:
- return result_dc
-
- def get_batch_result_id(self, input_ids, column='emd5', update_attrs={},
- subscriber=0, start=1, end=100, temporary=False):
- subscriber %= start
- from pyphant.core.Helpers import utf82uc
- search_dict = {'attributes':{
- 'batch_recipe':utf82uc(self.recipe_fname),
- 'result_worker':utf82uc(self.rwn),
- 'result_plug':utf82uc(self.rpn),
- 'applied_to':utf82uc(input_ids.__repr__()),
- 'params':utf82uc(self.params.__repr__())}}
- result_ids = kmanager.search(['id'], search_dict)
- if len(result_ids) > 0:
- subscriber %= end
- return result_ids[0][0]
- import copy
- input_scs = dict([(wname, dict([(sname, kmanager.getDataContainer(emd5)) \
- for sname, emd5 in sockets.iteritems()])) \
- for wname, sockets in input_ids.iteritems()])
- out_column = []
- ref_sc = input_scs.values()[0].values()[0]
- output_sc = copy.deepcopy(ref_sc)
- import numpy
- sharpruns = len(ref_sc[column].data)
- import math
- for index in xrange(sharpruns):
- subscriber %= 1 + math.floor(start + float((end - start) * index) \
- / sharpruns)
- ids = dict([(wname, dict([(sname, unicode(isc[column].data[index])) \
- for sname, isc in sockets.iteritems()])) \
- for wname, sockets in input_scs.iteritems()])
- out_column.append(self.get_result(ids, id_only=True))
- output_sc[column].data = numpy.array(out_column)
- output_sc.longname = "%s_%s" % (os.path.basename(self.recipe_fname),
- ref_sc.longname)
- output_sc.attributes['batch_recipe'] = utf82uc(self.recipe_fname)
- output_sc.attributes['applied_to'] = utf82uc(input_ids.__repr__())
- output_sc.attributes['result_plug'] = utf82uc(self.rpn)
- output_sc.attributes['result_worker'] = utf82uc(self.rwn)
- output_sc.attributes['params'] = utf82uc(self.params.__repr__())
- output_sc.attributes.update(update_attrs)
- output_sc.seal()
- kmanager.registerDataContainer(output_sc, temporary=temporary)
- subscriber %= end
- return output_sc.id
Modified: trunk/src/pyphant/pyphant/core/SQLiteWrapper.py
===================================================================
--- trunk/src/pyphant/pyphant/core/SQLiteWrapper.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/core/SQLiteWrapper.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -189,9 +189,10 @@
one_to_one_result_keys = one_to_one_search_keys + ['date', 'id', 'type']
common_search_keys = one_to_one_search_keys + ['id', 'attributes',
'date_from', 'date_to']
- fc_search_keys = common_search_keys + ['unit', 'dimensions', 'dim_of', 'col_of']
+ fc_search_keys = common_search_keys + ['unit', 'dimensions',
+ 'dim_of', 'col_of']
sc_search_keys = common_search_keys + ['columns']
- sortable_keys = common_keys + ['id', 'storage', 'type']
+ sortable_keys = common_keys + ['storage', 'type']
any_value = AnyValue()
def __init__(self, database, timeout=60.0):
Modified: trunk/src/pyphant/pyphant/core/WebInterface.py
===================================================================
--- trunk/src/pyphant/pyphant/core/WebInterface.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/core/WebInterface.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -593,6 +593,8 @@
+ [key for key in common_keys \
if qry[key] == self.anystr] \
+ ['id']
+ if not order_by in missing_keys:
+ order_by = 'date'
search_result = self.kn.km.search(
missing_keys, search_dict, order_by=order_by,
order_asc=order_asc, limit=limit, offset=offset)
Modified: trunk/src/pyphant/pyphant/core/ZStackManager.py
===================================================================
--- trunk/src/pyphant/pyphant/core/ZStackManager.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/core/ZStackManager.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -45,7 +45,6 @@
import scipy
import Image
from pyphant.core.KnowledgeManager import KnowledgeManager
-from pyphant.core.RecipeAlg import RecipeAlg
import os
kmanager = KnowledgeManager.getInstance()
@@ -53,8 +52,8 @@
class ZStack(object):
def __init__(self, sc_id=None, name=None, xml_file=None, temporary=False):
"""Initializes a ZStack from an existing id or a local source"""
- self.temporary = temporary
assert (sc_id is None) is not (xml_file is None)
+ self.temporary = temporary
self._recipe_path = None
if sc_id is not None:
self.repr_sc = kmanager.getDataContainer(sc_id)
@@ -137,11 +136,13 @@
'zvi_filename':img_meta['zvi_filename'],
'zvalue':zvalue,
'timestamp':img_meta['timestamp'],
- 'zid':img_meta['zid']}
+ 'zid':img_meta['zid'],
+ 'ZStackType':'RawImage',
+ 'vmin':0, 'vmax':255}
img_fc = FieldContainer(data=data,
longname=os.path.basename(
img_meta['img_filename']),
- shortname="img",
+ shortname="i",
dimensions=dimensions, attributes=fcattr)
img_fc.seal()
kmanager.registerDataContainer(img_fc, temporary=temporary)
@@ -153,7 +154,8 @@
shortname='z', unit=Quantity(1.0, 'mum'))
filefc = FieldContainer(scipy.array(files), longname='filename',
shortname='f')
- emd5fc = FieldContainer(scipy.array(emd5s), longname='emd5', shortname='i')
+ emd5fc = FieldContainer(scipy.array(emd5s), longname='emd5',
+ shortname='i')
if pdial is not None:
pdial.Destroy()
return zfc, filefc, emd5fc
@@ -175,55 +177,16 @@
return None
attributes = {}
attributes['ztol'] = ZStack._estimate_ztol(zfc)
- attributes['isZStack'] = 'yes'
+ attributes['ZStackType'] = 'RawSC'
ssc = SampleContainer([zfc, filefc, emd5fc],
name,
- "z-stack",
+ "z",
attributes)
ssc.seal()
kmanager.registerDataContainer(ssc, temporary=self.temporary)
return ssc
- def _set_recipe_path(self, rpath):
- self._recipe_path = os.path.realpath(rpath)
- def _get_recipe_path(self):
- return self._recipe_path
- recipe_path = property(_get_recipe_path, _set_recipe_path)
-
- def _get_mf_id(self, gradient_alg, label_alg, human, subscriber=0):
- assert self._recipe_path is not None
- uattr = {'isZStack':'no'}
- in_ids = {'grey_invert':{'image':self.repr_sc.id}}
- gradient_id = gradient_alg.get_batch_result_id(
- in_ids, update_attrs=uattr, subscriber=subscriber,
- start=1, end=30, temporary=self.temporary)
- in_ids = {'threshold':{'image':gradient_id}}
- label_id = label_alg.get_batch_result_id(
- in_ids, update_attrs=uattr, subscriber=subscriber,
- start=31, end=66, temporary=self.temporary)
- mf_alg = RecipeAlg(os.path.join(self.recipe_path, 'MeasureFocus.h5'),
- 'MeasureFocus', 'measure_focus',
- {'MeasureFocus':{'humanOutput':human}})
- in_ids = {'MeasureFocus':{'image':gradient_id,
- 'labels':label_id}}
- return mf_alg.get_batch_result_id(
- in_ids, update_attrs=uattr, subscriber=subscriber,
- start=67, end=100, temporary=self.temporary)
-
- def get_statistics(self, gradient_alg, label_alg, subscriber=0):
- mf_id = self._get_mf_id(gradient_alg, label_alg, False, subscriber)
- af_alg = RecipeAlg(os.path.join(self.recipe_path, 'AutoFocus.h5'),
- 'AutoFocus', 'AutoFocusWorker')
- in_ids = {'AutoFocus':{'focusSC':mf_id}}
- af_sc = af_alg.get_result(in_ids, temporary=self.temporary)
- return af_sc
-
- def get_human_imgs(self, gradient_alg, label_alg, subscriber=0):
- mf_id = self._get_mf_id(gradient_alg, label_alg, True, subscriber)
- return kmanager.getDataContainer(mf_id)
-
-
class ZStackManager(object):
def addZStack(self, zstack):
"""Adds a given zstack to the pool"""
@@ -231,8 +194,17 @@
def getZStacks(self):
"""Returns a list of all ZStacks in the pool"""
- search_dict = {'type':'sample', 'attributes':{'isZStack':'yes'}}
+ search_dict = {'type':'sample', 'attributes':{'ZStackType':'RawSC'}}
search_result = kmanager.search(['id'], search_dict)
if len(search_result) == 0:
return []
return [ZStack(zs_id[0]) for zs_id in search_result]
+
+ def getZStackByName(self, name):
+ search_dict = {'type':'sample', 'attributes':{'ZStackType':'RawSC'},
+ 'longname':name}
+ sresult = kmanager.search(['id'], search_dict)
+ if sresult == []:
+ raise ValueError("There is no ZStack called %s!" \
+ % name)
+ return ZStack(sc_id=sresult[0][0])
Modified: trunk/src/pyphant/pyphant/quantities/__init__.py
===================================================================
--- trunk/src/pyphant/pyphant/quantities/__init__.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/quantities/__init__.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -241,6 +241,46 @@
def __rsub__(self, other):
return self._sum(other, -1, 1)
+ def __eq__(self, other):
+ if isQuantity(other) and self.isCompatible(other.unit):
+ return self.__cmp__(other) == 0
+ else:
+ return False
+
+ def __ne__(self, other):
+ if isQuantity(other) and self.isCompatible(other.unit):
+ return self.__cmp__(other) != 0
+ else:
+ return True
+
+ def __le__(self, other):
+ if isQuantity(other) and self.isCompatible(other.unit):
+ return self.__cmp__(other) <= 0
+ else:
+ return False
+
+ def __ge__(self, other):
+ if isQuantity(other) and self.isCompatible(other.unit):
+ return self.__cmp__(other) >= 0
+ elif other is None:
+ return True
+ else:
+ return False
+
+ def __lt__(self, other):
+ if isQuantity(other) and self.isCompatible(other.unit):
+ return self.__cmp__(other) < 0
+ else:
+ return False
+
+ def __gt__(self, other):
+ if isQuantity(other) and self.isCompatible(other.unit):
+ return self.__cmp__(other) > 0
+ elif other is None:
+ return True
+ else:
+ return False
+
def __cmp__(self, other):
normed = self.inBaseUnits() # Minimizing numerical errors
diff = normed._sum(other, 1, -1)
Deleted: trunk/src/pyphant/pyphant/tests/TestAutoFocus.py
===================================================================
--- trunk/src/pyphant/pyphant/tests/TestAutoFocus.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/tests/TestAutoFocus.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -1,229 +0,0 @@
-#!/usr/bin/env python2.5
-# -*- coding: utf-8 -*-
-
-# Copyright (c) 2006-2010, Rectorate of the University of Freiburg
-# Copyright (c) 2009-2010, Andreas W. Liehr (li...@us...)
-# All rights reserved.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# * Redistributions of source code must retain the above copyright
-# notice, this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright
-# notice, this list of conditions and the following disclaimer in the
-# documentation and/or other materials provided with the distribution.
-# * Neither the name of the Freiburg Materials Research Center,
-# University of Freiburg nor the names of its contributors may be used to
-# endorse or promote products derived from this software without specific
-# prior written permission.
-#
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
-# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
-# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
-# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
-# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-u"""Provides unittest classes for AutoFocus worker.
-"""
-
-__id__ = "$Id$".replace('$','')
-__author__ = "$Author$".replace('$','')
-__version__ = "$Revision$".replace('$','')
-# $Source$
-
-import unittest
-import pkg_resources
-pkg_resources.require("pyphant")
-from pyphant.quantities import Quantity
-from pyphant.core.DataContainer import FieldContainer, SampleContainer
-from ImageProcessing import AutoFocus as AF
-import numpy
-from pyphant.core.KnowledgeManager import KnowledgeManager
-
-
-class CubeTestCase(unittest.TestCase):
- def setUp(self):
- self.cube1 = AF.Cube([slice(0, 10),
- slice(0, 10),
- slice(0, 10)])
- self.cube2 = AF.Cube([slice(3, 5),
- slice(4, 6),
- slice(-5, 7)])
- def tearDown(self):
- pass
-
- def testEq(self):
- cube1c = AF.Cube(self.cube1.slices)
- assert self.cube1 == cube1c
- assert not self.cube1.__eq__(self.cube2)
-
- def testAnd(self):
- expected = AF.Cube([slice(3, 5), slice(4, 6), slice(0, 7)])
- assert self.cube1 & self.cube2 == expected
-
- def testOr(self):
- expected = AF.Cube([slice(0, 10), slice(0, 10), slice(-5, 10)])
- assert self.cube1 | self.cube2 == expected
-
- def testVolume(self):
- assert self.cube1.getVolume() == 1000
- assert self.cube2.getVolume() == 48
- assert AF.Cube([slice(0, 0),
- slice(0, 1000),
- slice(0, 1000)]).getVolume() == 0
-
- def testSubCube(self):
- expected = AF.Cube([slice(3,5 ), slice(-5, 7)])
- assert self.cube2.getSubCube([0, 2]) == expected
-
- def testGetEdgeLength(self):
- assert self.cube2.getEdgeLength(0) == 2
- assert self.cube2.getEdgeLength(1) == 2
- assert self.cube2.getEdgeLength(2) == 12
-
- def testSub(self):
- expected = AF.Cube([slice(-3, 7),
- slice(-4, 6),
- slice(5, 15)])
- assert self.cube1 - self.cube2 == expected
-
- def testCenter(self):
- expected = (5.0, 5.0, 5.0)
- assert self.cube1.getCenter() == expected
-
-
-class ZTubeTestCase(unittest.TestCase):
- def setUp(self):
- slices = [slice(0, 10), slice(0, 10)]
- mask = numpy.ones((20, 20), dtype=bool)
- mask_fc = FieldContainer(mask)
- mask_fc.seal()
- km = KnowledgeManager.getInstance()
- km.registerDataContainer(mask_fc, temporary=True)
- mask_parent = mask_fc.id
- fslice = AF.FocusSlice(slices, 10.0, mask_parent, slices, 1, 1)
- self.ztube = AF.ZTube(fslice, 0, 1, 0.5, 0.5)
- testslices1 = [slice(3, 12), slice(2, 9)]
- mask1 = numpy.ones((9, 7), dtype=bool)
- self.testfslice1 = AF.FocusSlice(testslices1, 12.0, mask_parent,
- testslices1, 1, 1)
- testslices2 = [slice(7, 17), slice(8, 16)]
- mask2 = numpy.ones((10, 8), dtype=bool)
- self.testfslice2 = AF.FocusSlice(testslices2, 8.0, mask_parent,
- testslices2, 1, 1)
-
- def tearDown(self):
- pass
-
- def testMatching(self):
- assert self.ztube.match(self.testfslice1, 1)
- assert not self.ztube.match(self.testfslice2, 1.01)
- expectedyx = AF.Cube([slice(0, 12),
- slice(0, 10)])
- expectedz = AF.Cube([slice(-1, 2)])
- assert self.ztube.yxCube == expectedyx
- assert self.ztube.zCube == expectedz
- assert self.ztube.focusedFSlice == self.testfslice1
- assert self.ztube.focusedZ == 1
-
-
-class AutoFocusTestCase(unittest.TestCase):
- def setUp(self):
- from pyphant.core.KnowledgeManager import KnowledgeManager
- km = KnowledgeManager.getInstance()
- sl1 = [slice(Quantity('1.0mm'),
- Quantity('2.0mm')),
- slice(Quantity('1.5mm'),
- Quantity('3.5mm'))]
- sl2 = [slice(Quantity('0.8mm'),
- Quantity('1.9mm')),
- slice(Quantity('1.7mm'),
- Quantity('3.4mm'))]
- mask = numpy.ones((11, 20), dtype=bool)
- mask_fc = FieldContainer(mask)
- mask_fc.seal()
- mask_parent = mask_fc.id
- km = KnowledgeManager.getInstance()
- km.registerDataContainer(mask_fc, temporary=True)
- fsl1 = AF.FocusSlice(sl1, Quantity('10.0mm**-3'), mask_parent,
- [slice(0, 10), slice(0, 20)],
- Quantity('1.0mm'),
- Quantity('1.0mm'))
- self.fsl2 = AF.FocusSlice(sl2, Quantity('12.0mm**-3'),
- mask_parent, [slice(0, 11), slice(0, 17)],
- Quantity('0.1mm'), Quantity('0.1mm'))
- fc1 = FieldContainer(numpy.array([fsl1]))
- fc2 = FieldContainer(numpy.array([self.fsl2]))
- fc1.seal()
- fc2.seal()
- km.registerDataContainer(fc1, temporary=True)
- km.registerDataContainer(fc2, temporary=True)
- columns = [FieldContainer(numpy.array([.5, 1.0]),
- unit=Quantity('1.0mm'),
- longname='z-value'),
- FieldContainer(numpy.array([fc1.id, fc2.id]),
- longname="emd5")]
- attributes = {u'ztol': Quantity('0.5mm')}
- self.inputSC = SampleContainer(columns, attributes=attributes)
- self.inputSC.seal()
-
- def tearDown(self):
- pass
-
- def testAutofocus(self):
- columns = AF.autofocus(self.inputSC, 0.5, 0.75)
- inclusionSC = SampleContainer(columns,
- "AutoFocus")
- for fc in inclusionSC.columns:
- assert fc.data.shape == (1, )
- zfc, yfc, xfc, dfc, ffc = inclusionSC.columns
- assert zfc.data[0] * zfc.unit == Quantity('1.0mm')
- assert (yfc.data[0] * yfc.unit,
- xfc.data[0] * xfc.unit) == self.fsl2.getCenter()
- assert ffc.data[0] * ffc.unit == Quantity('12.0mm**-3')
-
-
-class FocusSliceTestCase(unittest.TestCase):
- def setUp(self):
- pass
-
- def tearDown(self):
- pass
-
- def testSaveLoadFocusSlice(self):
- km = KnowledgeManager.getInstance()
- mask = numpy.ones((100, 150), dtype=bool)
- mask_fc = FieldContainer(mask)
- mask_fc.seal()
- mask_parent = mask_fc.id
- km.registerDataContainer(mask_fc, temporary=True)
- slices = [slice(Quantity('100mm'),
- Quantity('200mm')),
- slice(Quantity('150mm'),
- Quantity('350mm'))]
- fslice = AF.FocusSlice(slices, Quantity('10mm**-3'),
- mask_parent, [slice(0, 100), slice(0, 150)],
- Quantity('1mm'), Quantity('1mm'))
- fc = FieldContainer(numpy.array([fslice for xr in xrange(1000)]))
- fc.seal()
- km.registerDataContainer(fc, temporary=True)
- returnfc = km.getDataContainer(fc.id, use_cache=False)
- assert returnfc.data[0].slices[0].start == fc.data[0].slices[0].start
-
-
-if __name__ == "__main__":
- import sys
- if len(sys.argv) == 1:
- unittest.main()
- else:
- suite = unittest.TestLoader().loadTestsFromTestCase(
- eval(sys.argv[1:][0]))
- unittest.TextTestRunner().run(suite)
Modified: trunk/src/pyphant/pyphant/tests/TestQuantities.py
===================================================================
--- trunk/src/pyphant/pyphant/tests/TestQuantities.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/tests/TestQuantities.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -45,6 +45,15 @@
(Quantity(731814.5,'d'), None)
"""
class TestQuantity(unittest.TestCase):
+ def setUp(self):
+ self.quants = []
+ self.quants.append(Quantity("1000 m"))
+ self.quants.append(Quantity("1 km"))
+ self.quants.append(Quantity("1000.1 m"))
+ self.quants.append(0)
+ self.quants.append(None)
+ self.quants.append(Quantity("1000 s"))
+
def testTextualQuantitySpecification(self):
self.assertEqual(Quantity('1V'),
Quantity(1.0,'V')
@@ -58,6 +67,90 @@
Quantity(1.0,'V')
)
+ def compare(self, quant0, cop, quant1, result):
+ if cop == '==':
+ expression = quant0 == quant1
+ elif cop == '!=':
+ expression = quant0 != quant1
+ elif cop == '<=':
+ expression = quant0 <= quant1
+ elif cop == '>=':
+ expression = quant0 >= quant1
+ elif cop == '<':
+ expression = quant0 < quant1
+ elif cop == '>':
+ expression = quant0 > quant1
+ if expression != result:
+ print "%s %s %s = %s, should be %s" % (quant0, cop, quant1,
+ expression, result)
+ self.assertEqual(expression, result)
+
+ def matrixTest(self, matrix, cop):
+ for index0, quant0 in enumerate(self.quants):
+ for index1, quant1 in enumerate(self.quants):
+ self.compare(quant0, cop, quant1, matrix[index0][index1])
+
+ def testComparisonOperationEqual(self):
+ matrix = [[1, 1, 0, 0, 0, 0],
+ [1, 1, 0, 0, 0, 0],
+ [0, 0, 1, 0, 0, 0],
+ [0, 0, 0, 1, 0, 0],
+ [0, 0, 0, 0, 1, 0],
+ [0, 0, 0, 0, 0, 1]]
+ self.matrixTest(matrix, '==')
+
+ def testComparisonOperationNotEqual(self):
+ matrix = [[0, 0, 1, 1, 1, 1],
+ [0, 0, 1, 1, 1, 1],
+ [1, 1, 0, 1, 1, 1],
+ [1, 1, 1, 0, 1, 1],
+ [1, 1, 1, 1, 0, 1],
+ [1, 1, 1, 1, 1, 0]]
+ self.matrixTest(matrix, '!=')
+
+ def testComparisonOperationLessOrEqual(self):
+ matrix = [[1, 1, 1, 0, 0, 0],
+ [1, 1, 1, 0, 0, 0],
+ [0, 0, 1, 0, 0, 0],
+ [0, 0, 0, 1, 0, 0],
+ [1, 1, 1, 1, 1, 1],
+ [0, 0, 0, 0, 0, 1]]
+ self.matrixTest(matrix, '<=')
+
+ def testComparisonOperationGreaterOrEqual(self):
+ matrix = [[1, 1, 0, 0, 1, 0],
+ [1, 1, 0, 0, 1, 0],
+ [1, 1, 1, 0, 1, 0],
+ [0, 0, 0, 1, 1, 0],
+ [0, 0, 0, 0, 1, 0],
+ [0, 0, 0, 0, 1, 1]]
+ self.matrixTest(matrix, '>=')
+
+ def testComparisonOperationLess(self):
+ matrix = [[0, 0, 1, 0, 0, 0],
+ [0, 0, 1, 0, 0, 0],
+ [0, 0, 0, 0, 0, 0],
+ [0, 0, 0, 0, 0, 0],
+ [1, 1, 1, 1, 0, 1],
+ [0, 0, 0, 0, 0, 0]]
+ self.matrixTest(matrix, '<')
+
+ def testComparisonOperationGreater(self):
+ matrix = [[0, 0, 0, 0, 1, 0],
+ [0, 0, 0, 0, 1, 0],
+ [1, 1, 0, 0, 1, 0],
+ [0, 0, 0, 0, 1, 0],
+ [0, 0, 0, 0, 0, 0],
+ [0, 0, 0, 0, 1, 0]]
+ self.matrixTest(matrix, '>')
+
+ def testList(self):
+ self.assertEqual(self.quants[1] in self.quants, True)
+ self.assertEqual(self.quants[0] in self.quants[1:], True)
+ self.assertEqual(self.quants[2] in self.quants[3:], False)
+ self.assertEqual(self.quants[1] in [], False)
+
+
if __name__ == "__main__":
import sys
if len(sys.argv) == 1:
Modified: trunk/src/pyphant/pyphant/tests/TestZStacks.py
===================================================================
--- trunk/src/pyphant/pyphant/tests/TestZStacks.py 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/tests/TestZStacks.py 2010-06-17 16:17:44 UTC (rev 685)
@@ -51,19 +51,6 @@
def tearDown(self):
pass
- def get_rp_ga_la(self, ppath):
- import os
- from pyphant.core.RecipeAlg import RecipeAlg
- gradient_recipe = "pre-MF-no-median.h5"
- label_recipe = "pre-MF-no-median.h5"
- rpath = os.path.join(ppath, "tests", "resources", "recipes")
- gradient_alg = RecipeAlg(os.path.join(rpath, gradient_recipe),
- 'gradient', 'gradientWorker',
- {'median':{'size':5}})
- label_alg = RecipeAlg(os.path.join(rpath, label_recipe),
- 'label', 'ndimage')
- return (rpath, gradient_alg, label_alg)
-
def check(self, rng, value):
assert value >= rng[0] - rng[1] and value <= rng[0] + rng[1], \
"Value %f not in range (%f, %f)" % (value, rng[0] - rng[1],
@@ -81,22 +68,21 @@
temporary=True)
print "Done."
print "Calculating ZStack-statistics..."
- rpath, gradient_alg, label_alg = self.get_rp_ga_la(ppath[0])
- zstack.recipe_path = rpath
- statistics = zstack.get_statistics(gradient_alg, label_alg)
+ from ImageProcessing.AutoFocus import AutoFocus
+ afw = AutoFocus()
+ statistics = afw.get_statistics_sc(zstack.repr_sc)
print "Done."
assert len(statistics['diameter'].data) == 2
imax = statistics['diameter'].data.argmax()
imin = statistics['diameter'].data.argmin()
- self.check((200.0, 1.0), statistics['x-value'].data[imax])
- self.check((200.0, 1.0), statistics['y-value'].data[imax])
- self.check((300.0, 0.0), statistics['z-value'].data[imax])
- self.check((20.5, 1.0), statistics['diameter'].data[imax])
- self.check((53.0, 1.0), statistics['x-value'].data[imin])
- self.check((53.0, 1.0), statistics['y-value'].data[imin])
- self.check((300.0, 0.0), statistics['z-value'].data[imin])
- self.check((6.5, 1.0), statistics['diameter'].data[imin])
- print statistics['diameter'].data
+ self.check((200.0, 1.0), statistics['x-pos'].data[imax])
+ self.check((200.0, 1.0), statistics['y-pos'].data[imax])
+ self.check((300.0, 0.0), statistics['z-pos'].data[imax])
+ self.check((20.7, 1.0), statistics['diameter'].data[imax])
+ self.check((53.0, 1.0), statistics['x-pos'].data[imin])
+ self.check((53.0, 1.0), statistics['y-pos'].data[imin])
+ self.check((300.0, 0.0), statistics['z-pos'].data[imin])
+ self.check((7.0, 1.0), statistics['diameter'].data[imin])
if __name__ == "__main__":
Deleted: trunk/src/pyphant/pyphant/tests/resources/recipes/AutoFocus.h5
===================================================================
--- trunk/src/pyphant/pyphant/tests/resources/recipes/AutoFocus.h5 2010-05-26 20:45:06 UTC (rev 684)
+++ trunk/src/pyphant/pyphant/tests/resources/recipes/AutoFocus.h5 2010-06-17 16:17:44 UTC (rev 685)
@@ -1,18 +0,0 @@
-\x89HDF
-
- |
|
From: Rolf W. <ro...@di...> - 2010-06-14 19:44:40
|
Hi all ... Today I've splitted of to subprojects concerning FMF to an fmf project. The subversion repository is located at: https://bone.digitalis.org/fmf (gast, Bitte) The corresponding trac installation is located at: https://bone.digitalis.org/FMF where you can set up you own account using "register". The repository contains two packages: simplefmf - which is an first step towards an (simple) python class and can write fmf files ;) FMF - which is an perl package and can read fmf files ;) At least the simplefmf class is somewhat stable but should undergo some testing so we can fix errors and made enhancements (there will be many) prior releasing the 0.1 versions (and go to sf-net) which we should do in time sync with the pyphant releases. Kind regards, Rolf -- Security is an illusion - Datasecurity twice Rolf Würdemann - ro...@di... GnuPG fingerprint: 7383 348F 67D1 CD27 C90F DDD0 86A3 31B6 67F0 D02F jabber: ro...@di... ECF127C7 EAB85F87 BC75ACB5 2EC646D4 9211A31 |
|
From: <zk...@us...> - 2010-05-26 20:45:13
|
Revision: 684
http://pyphant.svn.sourceforge.net/pyphant/?rev=684&view=rev
Author: zklaus
Date: 2010-05-26 20:45:06 +0000 (Wed, 26 May 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Chng: Modified correction formula according to discussion with Kristian.
Fix: ZStacks
Cosm: Cleaned up some Code
Enh: Import of ZStacks in wxPyphant
Enh: Z-Stack diameter estimation
Fix: Zeiss meta import
Undo: Changes to OscAbsorption
Fix: KMVisualizer
Enh: Minor changes to ZStackManager
Fix: SQLiteWrapper
Enh: First working state of ZStackManager
Enh: Started implementing ZStackManager
Enh: Added z-axis to OSCMapper
Modified Paths:
--------------
trunk/src/pyphant/pyphant/core/H5FileHandler.py
trunk/src/pyphant/pyphant/core/KnowledgeManager.py
trunk/src/pyphant/pyphant/core/SQLiteWrapper.py
trunk/src/pyphant/pyphant/core/WebInterface.py
trunk/src/pyphant/pyphant/core/WorkerRegistry.py
trunk/src/pyphant/pyphant/tests/TestAutoFocus.py
trunk/src/pyphant/pyphant/visualizers/KMVisualizer.py
trunk/src/pyphant/pyphant/wxgui2/WorkerRepository.py
trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py
trunk/src/workers/ImageProcessing/ImageProcessing/AutoFocus.py
trunk/src/workers/ImageProcessing/ImageProcessing/MeasureFocus.py
trunk/src/workers/ImageProcessing/ImageProcessing/NDImageWorker.py
trunk/src/workers/ImageProcessing/ImageProcessing/__init__.py
trunk/src/workers/OSC/OSC/OscThicknessCorrection.py
Added Paths:
-----------
trunk/src/pyphant/pyphant/core/RecipeAlg.py
trunk/src/pyphant/pyphant/core/ZStackManager.py
trunk/src/pyphant/pyphant/tests/TestZStacks.py
trunk/src/pyphant/pyphant/tests/resources/
trunk/src/pyphant/pyphant/tests/resources/recipes/
trunk/src/pyphant/pyphant/tests/resources/recipes/AutoFocus.h5
trunk/src/pyphant/pyphant/tests/resources/recipes/MeasureFocus.h5
trunk/src/pyphant/pyphant/tests/resources/recipes/pre-MF-no-median.h5
trunk/src/pyphant/pyphant/tests/resources/zstack/
trunk/src/pyphant/pyphant/tests/resources/zstack/TestZStack_z00.tif
trunk/src/pyphant/pyphant/tests/resources/zstack/TestZStack_z01.tif
trunk/src/pyphant/pyphant/tests/resources/zstack/TestZStack_z02.tif
trunk/src/pyphant/pyphant/tests/resources/zstack/TestZStack_z03.tif
trunk/src/pyphant/pyphant/tests/resources/zstack/TestZStack_z04.tif
trunk/src/pyphant/pyphant/tests/resources/zstack/TestZStack_z05.tif
trunk/src/pyphant/pyphant/tests/resources/zstack/TestZStack_z06.tif
trunk/src/pyphant/pyphant/tests/resources/zstack/_meta.xml
trunk/src/workers/ImageProcessing/ImageProcessing/SliceSelector.py
trunk/src/workers/ImageProcessing/ImageProcessing/ZStacks.py
Modified: trunk/src/pyphant/pyphant/core/H5FileHandler.py
===================================================================
--- trunk/src/pyphant/pyphant/core/H5FileHandler.py 2010-05-25 20:33:33 UTC (rev 683)
+++ trunk/src/pyphant/pyphant/core/H5FileHandler.py 2010-05-26 20:45:06 UTC (rev 684)
@@ -41,6 +41,7 @@
from pyphant.core import DataContainer
from tables import StringCol
from pyphant.quantities import Quantity
+PhysicalQuantity = Quantity
import scipy
import logging
import os
Modified: trunk/src/pyphant/pyphant/core/KnowledgeManager.py
===================================================================
--- trunk/src/pyphant/pyphant/core/KnowledgeManager.py 2010-05-25 20:33:33 UTC (rev 683)
+++ trunk/src/pyphant/pyphant/core/KnowledgeManager.py 2010-05-26 20:45:06 UTC (rev 684)
@@ -225,6 +225,24 @@
has_entry = wrapper.has_entry(dcid)
return has_entry
+ def isTemporary(self, dcid):
+ """
+ Returns whether the given DC is stored temporarily.
+ """
+ with SQLiteWrapper(self.dbase) as wrapper:
+ is_tmp = wrapper.is_temporary(dcid)
+ return is_tmp
+
+ def setTemporary(self, dcid, temporary):
+ """
+ Sets the given entry to temporary, which means it will be
+ deleted upon restart or removes the temporary flag.
+ - dcid: emd5 of DataContainer
+ - temporary: boolean
+ """
+ with SQLiteWrapper(self.dbase) as wrapper:
+ wrapper.set_temporary(dcid, temporary)
+
def getH5FileHandler(self, filename, mode='r'):
"""
Returns an H5FileHandler for the given filename to perform IO
Added: trunk/src/pyphant/pyphant/core/RecipeAlg.py
===================================================================
--- trunk/src/pyphant/pyphant/core/RecipeAlg.py (rev 0)
+++ trunk/src/pyphant/pyphant/core/RecipeAlg.py 2010-05-26 20:45:06 UTC (rev 684)
@@ -0,0 +1,148 @@
+#!/usr/bin/env python2.5
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2006-2009, Rectorate of the University of Freiburg
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+# * Neither the name of the Freiburg Materials Research Center,
+# University of Freiburg nor the names of its contributors may be used to
+# endorse or promote products derived from this software without specific
+# prior written permission.
+#
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
+# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+u"""Provides ZStackManager
+"""
+
+__id__ = "$Id$".replace('$','')
+__author__ = "$Author$".replace('$','')
+__version__ = "$Revision$".replace('$','')
+# $Source$
+
+from pyphant.core.KnowledgeManager import KnowledgeManager
+from pyphant.core.PyTablesPersister import loadRecipeFromHDF5File
+from tools import Emd5Src
+import os
+kmanager = KnowledgeManager.getInstance()
+
+
+class RecipeAlg(object):
+ def __init__(self, recipe_fname, result_worker,
+ result_plug, params={}):
+ self.recipe_fname = recipe_fname
+ self.recipe = loadRecipeFromHDF5File(recipe_fname)
+ self.params = params
+ self.rwn = result_worker
+ self.rpn = result_plug
+ self.result_plug = self.recipe.getWorker(
+ result_worker).getPlug(result_plug)
+
+ def get_result(self, input_ids, id_only=False, temporary=False):
+ from pyphant.core.Helpers import utf82uc
+ search_dict = {'attributes':{
+ 'recipe':utf82uc(self.recipe_fname),
+ 'result_worker':utf82uc(self.rwn),
+ 'result_plug':utf82uc(self.rpn),
+ 'applied_to':utf82uc(input_ids.__repr__()),
+ 'params':utf82uc(self.params.__repr__())}}
+ result_ids = kmanager.search(['id'], search_dict)
+ if len(result_ids) > 0 and not temporary:
+ if id_only:
+ return result_ids[0][0]
+ else:
+ return kmanager.getDataContainer(result_ids[0][0])
+ else:
+ # Fill open sockets:
+ for wname, sockets in input_ids.iteritems():
+ worker = self.recipe.getWorker(wname)
+ for sname, emd5 in sockets.iteritems():
+ socket = worker.getSocket(sname)
+ dummy = Emd5Src.Emd5Src()
+ dummy.paramSelectby.value = u"enter emd5"
+ dummy.paramEnteremd5.value = emd5
+ socket.insert(dummy.getPlugs()[0])
+ # Set parameters:
+ for wname, wparams in self.params.iteritems():
+ for pname, pvalue in wparams.iteritems():
+ self.recipe.getWorker(wname).getParam(pname).value = pvalue
+ # Get Result
+ result_dc = self.result_plug.getResult()
+ # Pull plugs
+ for wname, sockets in input_ids.iteritems():
+ worker = self.recipe.getWorker(wname)
+ for sname, emd5 in sockets.iteritems():
+ worker.getSocket(sname).pullPlug()
+ result_dc.attributes['recipe'] = utf82uc(self.recipe_fname)
+ result_dc.attributes['applied_to'] = utf82uc(input_ids.__repr__())
+ result_dc.attributes['params'] = utf82uc(self.params.__repr__())
+ result_dc.attributes['result_plug'] = utf82uc(self.rpn)
+ result_dc.attributes['result_worker'] = utf82uc(self.rwn)
+ kmanager.registerDataContainer(result_dc, temporary=temporary)
+ if id_only:
+ return result_dc.id
+ else:
+ return result_dc
+
+ def get_batch_result_id(self, input_ids, column='emd5', update_attrs={},
+ subscriber=0, start=1, end=100, temporary=False):
+ subscriber %= start
+ from pyphant.core.Helpers import utf82uc
+ search_dict = {'attributes':{
+ 'batch_recipe':utf82uc(self.recipe_fname),
+ 'result_worker':utf82uc(self.rwn),
+ 'result_plug':utf82uc(self.rpn),
+ 'applied_to':utf82uc(input_ids.__repr__()),
+ 'params':utf82uc(self.params.__repr__())}}
+ result_ids = kmanager.search(['id'], search_dict)
+ if len(result_ids) > 0:
+ subscriber %= end
+ return result_ids[0][0]
+ import copy
+ input_scs = dict([(wname, dict([(sname, kmanager.getDataContainer(emd5)) \
+ for sname, emd5 in sockets.iteritems()])) \
+ for wname, sockets in input_ids.iteritems()])
+ out_column = []
+ ref_sc = input_scs.values()[0].values()[0]
+ output_sc = copy.deepcopy(ref_sc)
+ import numpy
+ sharpruns = len(ref_sc[column].data)
+ import math
+ for index in xrange(sharpruns):
+ subscriber %= 1 + math.floor(start + float((end - start) * index) \
+ / sharpruns)
+ ids = dict([(wname, dict([(sname, unicode(isc[column].data[index])) \
+ for sname, isc in sockets.iteritems()])) \
+ for wname, sockets in input_scs.iteritems()])
+ out_column.append(self.get_result(ids, id_only=True))
+ output_sc[column].data = numpy.array(out_column)
+ output_sc.longname = "%s_%s" % (os.path.basename(self.recipe_fname),
+ ref_sc.longname)
+ output_sc.attributes['batch_recipe'] = utf82uc(self.recipe_fname)
+ output_sc.attributes['applied_to'] = utf82uc(input_ids.__repr__())
+ output_sc.attributes['result_plug'] = utf82uc(self.rpn)
+ output_sc.attributes['result_worker'] = utf82uc(self.rwn)
+ output_sc.attributes['params'] = utf82uc(self.params.__repr__())
+ output_sc.attributes.update(update_attrs)
+ output_sc.seal()
+ kmanager.registerDataContainer(output_sc, temporary=temporary)
+ subscriber %= end
+ return output_sc.id
Modified: trunk/src/pyphant/pyphant/core/SQLiteWrapper.py
===================================================================
--- trunk/src/pyphant/pyphant/core/SQLiteWrapper.py 2010-05-25 20:33:33 UTC (rev 683)
+++ trunk/src/pyphant/pyphant/core/SQLiteWrapper.py 2010-05-26 20:45:06 UTC (rev 684)
@@ -43,9 +43,10 @@
import time
from pyphant.core.Helpers import (utf82uc, uc2utf8, emd52dict)
from pyphant.quantities import (Quantity,PhysicalUnit,_base_units)
+PhysicalQuantity = Quantity
from types import (FloatType, IntType, LongType, StringTypes)
-DBASE_VERSION = 1 #increment if there have been structural changes to the dbase!
+DBASE_VERSION = 2 #increment if there have been structural changes to the dbase!
def quantity2powers(quantity):
numberOfBaseUnits = len(_base_units)
@@ -325,6 +326,11 @@
(id, ))
return self.cursor.fetchone() != None
+ def is_temporary(self, id):
+ exe = self.cursor.execute
+ exe("SELECT dc_id FROM km_temporary WHERE dc_id=?", (id, ))
+ return self.cursor.fetchone() != None
+
def _set_fc_keys(self, insert_dict, summary):
exe = self.cursor.execute
insert_dict['fc_id'] = summary['id']
@@ -374,6 +380,8 @@
attr_query = "INSERT INTO km_attributes VALUES (?, ?, ?)"
for key, value in summary['attributes'].iteritems():
assert isinstance(key, StringTypes)
+ if isinstance(value, StringTypes):
+ value = utf82uc(value)
exe(attr_query, (summary['id'], key, value.__repr__()))
if type == 'fc':
self._set_fc_keys(insert_dict, summary)
@@ -396,6 +404,24 @@
insert_query = insert_query % (type, key_query, value_query)
exe(insert_query, tuple(value_list))
+ def set_temporary(self, entry_id, temporary):
+ """Sets the given entry to temporary, which means it will be
+ deleted upon restart or removes the temporary flag.
+ - entry_id: emd5 of DataContainer
+ - temporary: boolean"""
+ if not self.has_entry(entry_id):
+ return
+ exe = self.cursor.execute
+ if temporary:
+ try:
+ exe("INSERT OR ABORT INTO km_temporary VALUES (?)",
+ (entry_id,))
+ except sqlite3.IntegrityError:
+ pass # Already set to temporary!
+ else:
+ exe("DELETE FROM km_temporary WHERE dc_id=?",
+ (entry_id,))
+
def get_emd5_list(self):
self.cursor.execute("SELECT fc_id FROM km_fc")
emd5_list = self.cursor.fetchall()
@@ -445,6 +471,8 @@
new_value.append(attr_key)
else:
value_expr = ' AND value=?'
+ if isinstance(attr_value, StringTypes):
+ attr_value = utf82uc(attr_value)
new_value.extend([attr_key, attr_value.__repr__()])
expr += '(%s IN (SELECT dc_id FROM km_attributes '\
'WHERE key=?%s))' \
Modified: trunk/src/pyphant/pyphant/core/WebInterface.py
===================================================================
--- trunk/src/pyphant/pyphant/core/WebInterface.py 2010-05-25 20:33:33 UTC (rev 683)
+++ trunk/src/pyphant/pyphant/core/WebInterface.py 2010-05-26 20:45:06 UTC (rev 684)
@@ -521,7 +521,7 @@
qry['jump'] == 'True',
(""" onload="window.location.hash='result_view';" """, ""))
offset = max(int(qry['offset']), 0)
- limit = 10
+ limit = 500
search_dict = dict([(key, qry[key]) for key in common_keys \
if qry[key] != self.anystr])
if qry['date_from'] != '':
@@ -542,10 +542,16 @@
attr_post.append(str(last_index + 1))
qry['attr_key' + attr_post[-1]] = ''
qry['attr_value' + attr_post[-1]] = ''
+ from pyphant.core.Helpers import utf82uc
+ def testany(str1):
+ str2 = utf82uc(str1)
+ if str2 == u"":
+ str2 = self.kn.km.any_value
+ return str2
search_dict['attributes'] = dict([(qry['attr_key' + apost],
- qry['attr_value' + apost]) \
+ testany(qry['attr_value' + apost])) \
for apost in attr_post])
- print search_dict
+ #print search_dict
# --- common search keys ---
optionss = [[(self.anystr, )] \
+ self.kn.km.search([key], search_dict, distinct=True) \
@@ -590,9 +596,11 @@
search_result = self.kn.km.search(
missing_keys, search_dict, order_by=order_by,
order_asc=order_asc, limit=limit, offset=offset)
- rows = [order_bar(missing_keys[:-1], order_by, order_asc) + ['details']]
+ rows = [order_bar(missing_keys[:-1], order_by, order_asc)\
+ + ['details', 'tmp']]
rows.extend([nice(srow[:-1], missing_keys[:-1], do_shorten) \
- + (HTMLSummaryLink((srow[-1], 'click')), ) \
+ + (HTMLSummaryLink((srow[-1], 'click')),
+ self.kn.km.isTemporary(srow[-1])) \
for srow in search_result])
bbar = HTMLBrowseBar(offset, limit).getHTML()
result = bbar + "<br />" + HTMLTable(rows).getHTML() + "<br />" + bbar
Modified: trunk/src/pyphant/pyphant/core/WorkerRegistry.py
===================================================================
--- trunk/src/pyphant/pyphant/core/WorkerRegistry.py 2010-05-25 20:33:33 UTC (rev 683)
+++ trunk/src/pyphant/pyphant/core/WorkerRegistry.py 2010-05-26 20:45:06 UTC (rev 684)
@@ -41,18 +41,19 @@
import singletonmixin
import pkg_resources
+
class WorkerRegistry(singletonmixin.Singleton):
def __init__(self):
- self._workerPool=[]
- self._logger=logging.getLogger("pyphant")
+ self._workerPool = []
+ self._logger = logging.getLogger("pyphant")
self._dirty = True
def registerWorker(self, workerInfo):
if not workerInfo in self._workerPool:
self._workerPool.append(workerInfo)
- self._logger.info( "added worker" )
+ self._logger.info("added worker")
else:
- self._logger.info( "did nothing" )
+ self._logger.info("did nothing")
def getWorkers(self):
if self._dirty:
@@ -60,17 +61,19 @@
wm = worker.load()
for module in wm.workers:
try:
- moduleName = worker.module_name+"."+module
- self._logger.info("Trying to import "+moduleName)
- exec 'import '+moduleName
+ moduleName = worker.module_name + "." + module
+ self._logger.info("Trying to import " + moduleName)
+ exec 'import ' + moduleName
try:
- version=eval(moduleName+".__version__")
+ version = eval(moduleName + ".__version__")
except:
- version="unknown"
- self._logger.info("Import module %s in version %s" %(moduleName, version))
+ version = "unknown"
+ self._logger.info("Import module %s in version %s" \
+ %(moduleName, version))
except ImportError, e:
- self._logger.warning("worker archive " + worker.module_name
- + " contains invalid worker "+module+": " + str(e))
+ self._logger.warning(
+ "worker archive " + worker.module_name \
+ + " contains invalid worker " + module \
+ + ": " + str(e))
self._dirty = False
return self._workerPool
-
Added: trunk/src/pyphant/pyphant/core/ZStackManager.py
===================================================================
--- trunk/src/pyphant/pyphant/core/ZStackManager.py (rev 0)
+++ trunk/src/pyphant/pyphant/core/ZStackManager.py 2010-05-26 20:45:06 UTC (rev 684)
@@ -0,0 +1,238 @@
+#!/usr/bin/env python2.5
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2006-2009, Rectorate of the University of Freiburg
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+# * Neither the name of the Freiburg Materials Research Center,
+# University of Freiburg nor the names of its contributors may be used to
+# endorse or promote products derived from this software without specific
+# prior written permission.
+#
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
+# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+u"""Provides ZStackManager
+"""
+
+__id__ = "$Id$".replace('$','')
+__author__ = "$Author$".replace('$','')
+__version__ = "$Revision$".replace('$','')
+# $Source$
+
+
+from pyphant.core.XMLHandler import getXMLRoot
+from pyphant.quantities import Quantity
+from pyphant.core.DataContainer import (FieldContainer, SampleContainer)
+import scipy
+import Image
+from pyphant.core.KnowledgeManager import KnowledgeManager
+from pyphant.core.RecipeAlg import RecipeAlg
+import os
+kmanager = KnowledgeManager.getInstance()
+
+
+class ZStack(object):
+ def __init__(self, sc_id=None, name=None, xml_file=None, temporary=False):
+ """Initializes a ZStack from an existing id or a local source"""
+ self.temporary = temporary
+ assert (sc_id is None) is not (xml_file is None)
+ self._recipe_path = None
+ if sc_id is not None:
+ self.repr_sc = kmanager.getDataContainer(sc_id)
+ else:
+ assert name is not None
+ self.repr_sc = self._import_zstack(name,
+ os.path.realpath(xml_file))
+
+ @staticmethod
+ def _get_images_meta(xml_file):
+ def getFloat(xmlelement):
+ return float(xmlelement.content.strip().replace(',', '.'))
+
+ def getMum(xmlelement):
+ return Quantity(getFloat(xmlelement), 'mum')
+
+ base_path = os.path.dirname(xml_file)
+ xml_root = getXMLRoot(xml_file)
+ images_meta = []
+ for ztag in [value for key, value in xml_root.children.iteritems()\
+ if key.startswith('z')]:
+ meta = {}
+ fname = xml_root['Tags']['V5'].content.strip()
+ root, ext = os.path.splitext(fname)
+ fname = "%s_%s%s" % (root, ztag.name, ext)
+ meta['img_filename'] = os.path.join(base_path, fname)
+ meta['zvi_filename'] = os.path.join(
+ base_path, xml_root['Tags']['V150'].content.strip())
+ meta['xml_filename'] = xml_file
+ meta['zid'] = ztag.name
+ ztag = ztag['Tags']
+ meta['timestamp'] = ztag['V44'].content.strip()
+ meta['width'] = int(ztag['V3'].content.strip())
+ meta['height'] = int(ztag['V4'].content.strip())
+ meta['x-pos'] = getMum(ztag['V15'])
+ meta['y-pos'] = getMum(ztag['V16'])
+ meta['z-pos'] = getMum(ztag['V93'])
+ meta['pixel_width'] = getMum(xml_root['Scaling']['Factor_0'])
+ meta['pixel_height'] = getMum(xml_root['Scaling']['Factor_1'])
+ images_meta.append(meta)
+ images_meta.sort(key = lambda x: x['z-pos'].value)
+ return images_meta
+
+ @staticmethod
+ def _get_image_fcs(images_meta, temporary=False):
+ zvalues = []
+ emd5s = []
+ files = []
+ from wx import (ProgressDialog, PyNoAppError)
+ try:
+ pdial = ProgressDialog('Importing ZStack...', ' ' * 50,
+ maximum=len(images_meta))
+ except PyNoAppError:
+ pdial = None
+ count = 1
+ for img_meta in images_meta:
+ if pdial is not None:
+ pdial.Update(count, os.path.basename(img_meta['img_filename']))
+ count += 1
+ files.append(img_meta['xml_filename'])
+ img = Image.open(img_meta['img_filename'])
+ data = scipy.misc.fromimage(img, flatten=True)
+ zvalue = img_meta['z-pos']
+ ydata = scipy.arange(img_meta['height']) \
+ * img_meta['pixel_height'].inUnitsOf('mum').value \
+ + img_meta['y-pos'].inUnitsOf('mum').value
+ xdata = scipy.arange(img_meta['width']) \
+ * img_meta['pixel_width'].inUnitsOf('mum').value \
+ + img_meta['x-pos'].inUnitsOf('mum').value
+ dimensions = [FieldContainer(ydata,
+ unit=Quantity(1.0, 'mum'),
+ longname=u'y-axis',
+ shortname=u'y'),
+ FieldContainer(xdata,
+ unit=Quantity(1.0, 'mum'),
+ longname=u'x-axis',
+ shortname=u'x')]
+ fcattr = {'img_filename':img_meta['img_filename'],
+ 'xml_filename':img_meta['xml_filename'],
+ 'zvi_filename':img_meta['zvi_filename'],
+ 'zvalue':zvalue,
+ 'timestamp':img_meta['timestamp'],
+ 'zid':img_meta['zid']}
+ img_fc = FieldContainer(data=data,
+ longname=os.path.basename(
+ img_meta['img_filename']),
+ shortname="img",
+ dimensions=dimensions, attributes=fcattr)
+ img_fc.seal()
+ kmanager.registerDataContainer(img_fc, temporary=temporary)
+ emd5s.append(img_fc.id)
+ zvalues.append(zvalue.inUnitsOf('mum').value)
+ if len(zvalues) == 0:
+ return None, None, None
+ zfc = FieldContainer(scipy.array(zvalues), longname='z-value',
+ shortname='z', unit=Quantity(1.0, 'mum'))
+ filefc = FieldContainer(scipy.array(files), longname='filename',
+ shortname='f')
+ emd5fc = FieldContainer(scipy.array(emd5s), longname='emd5', shortname='i')
+ if pdial is not None:
+ pdial.Destroy()
+ return zfc, filefc, emd5fc
+
+ @staticmethod
+ def _estimate_ztol(zfc):
+ zmums = [(float(zvalue) * zfc.unit).inUnitsOf('mum').value \
+ for zvalue in zfc.data]
+ zmums.sort()
+ diffs = []
+ for index in xrange(len(zmums) - 1):
+ diffs.append(zmums[index + 1] - zmums[index])
+ return Quantity(2.0 * sum(diffs) / float(len(diffs)), 'mum')
+
+ def _import_zstack(self, name, xml_file):
+ images_meta = ZStack._get_images_meta(xml_file)
+ zfc, filefc, emd5fc = ZStack._get_image_fcs(images_meta, self.temporary)
+ if zfc == None:
+ return None
+ attributes = {}
+ attributes['ztol'] = ZStack._estimate_ztol(zfc)
+ attributes['isZStack'] = 'yes'
+ ssc = SampleContainer([zfc, filefc, emd5fc],
+ name,
+ "z-stack",
+ attributes)
+ ssc.seal()
+ kmanager.registerDataContainer(ssc, temporary=self.temporary)
+ return ssc
+
+ def _set_recipe_path(self, rpath):
+ self._recipe_path = os.path.realpath(rpath)
+
+ def _get_recipe_path(self):
+ return self._recipe_path
+ recipe_path = property(_get_recipe_path, _set_recipe_path)
+
+ def _get_mf_id(self, gradient_alg, label_alg, human, subscriber=0):
+ assert self._recipe_path is not None
+ uattr = {'isZStack':'no'}
+ in_ids = {'grey_invert':{'image':self.repr_sc.id}}
+ gradient_id = gradient_alg.get_batch_result_id(
+ in_ids, update_attrs=uattr, subscriber=subscriber,
+ start=1, end=30, temporary=self.temporary)
+ in_ids = {'threshold':{'image':gradient_id}}
+ label_id = label_alg.get_batch_result_id(
+ in_ids, update_attrs=uattr, subscriber=subscriber,
+ start=31, end=66, temporary=self.temporary)
+ mf_alg = RecipeAlg(os.path.join(self.recipe_path, 'MeasureFocus.h5'),
+ 'MeasureFocus', 'measure_focus',
+ {'MeasureFocus':{'humanOutput':human}})
+ in_ids = {'MeasureFocus':{'image':gradient_id,
+ 'labels':label_id}}
+ return mf_alg.get_batch_result_id(
+ in_ids, update_attrs=uattr, subscriber=subscriber,
+ start=67, end=100, temporary=self.temporary)
+
+ def get_statistics(self, gradient_alg, label_alg, subscriber=0):
+ mf_id = self._get_mf_id(gradient_alg, label_alg, False, subscriber)
+ af_alg = RecipeAlg(os.path.join(self.recipe_path, 'AutoFocus.h5'),
+ 'AutoFocus', 'AutoFocusWorker')
+ in_ids = {'AutoFocus':{'focusSC':mf_id}}
+ af_sc = af_alg.get_result(in_ids, temporary=self.temporary)
+ return af_sc
+
+ def get_human_imgs(self, gradient_alg, label_alg, subscriber=0):
+ mf_id = self._get_mf_id(gradient_alg, label_alg, True, subscriber)
+ return kmanager.getDataContainer(mf_id)
+
+
+class ZStackManager(object):
+ def addZStack(self, zstack):
+ """Adds a given zstack to the pool"""
+ kmanager.registerDataContainer(zstack.repr_sc)
+
+ def getZStacks(self):
+ """Returns a list of all ZStacks in the pool"""
+ search_dict = {'type':'sample', 'attributes':{'isZStack':'yes'}}
+ search_result = kmanager.search(['id'], search_dict)
+ if len(search_result) == 0:
+ return []
+ return [ZStack(zs_id[0]) for zs_id in search_result]
Modified: trunk/src/pyphant/pyphant/tests/TestAutoFocus.py
===================================================================
--- trunk/src/pyphant/pyphant/tests/TestAutoFocus.py 2010-05-25 20:33:33 UTC (rev 683)
+++ trunk/src/pyphant/pyphant/tests/TestAutoFocus.py 2010-05-26 20:45:06 UTC (rev 684)
@@ -109,16 +109,16 @@
km = KnowledgeManager.getInstance()
km.registerDataContainer(mask_fc, temporary=True)
mask_parent = mask_fc.id
- fslice = AF.FocusSlice(slices, 10.0, mask_parent, slices)
+ fslice = AF.FocusSlice(slices, 10.0, mask_parent, slices, 1, 1)
self.ztube = AF.ZTube(fslice, 0, 1, 0.5, 0.5)
testslices1 = [slice(3, 12), slice(2, 9)]
mask1 = numpy.ones((9, 7), dtype=bool)
self.testfslice1 = AF.FocusSlice(testslices1, 12.0, mask_parent,
- testslices1)
+ testslices1, 1, 1)
testslices2 = [slice(7, 17), slice(8, 16)]
mask2 = numpy.ones((10, 8), dtype=bool)
self.testfslice2 = AF.FocusSlice(testslices2, 8.0, mask_parent,
- testslices2)
+ testslices2, 1, 1)
def tearDown(self):
pass
@@ -154,9 +154,12 @@
km = KnowledgeManager.getInstance()
km.registerDataContainer(mask_fc, temporary=True)
fsl1 = AF.FocusSlice(sl1, Quantity('10.0mm**-3'), mask_parent,
- [slice(0, 10), slice(0, 20)])
+ [slice(0, 10), slice(0, 20)],
+ Quantity('1.0mm'),
+ Quantity('1.0mm'))
self.fsl2 = AF.FocusSlice(sl2, Quantity('12.0mm**-3'),
- mask_parent, [slice(0, 11), slice(0, 17)])
+ mask_parent, [slice(0, 11), slice(0, 17)],
+ Quantity('0.1mm'), Quantity('0.1mm'))
fc1 = FieldContainer(numpy.array([fsl1]))
fc2 = FieldContainer(numpy.array([self.fsl2]))
fc1.seal()
@@ -207,7 +210,8 @@
slice(Quantity('150mm'),
Quantity('350mm'))]
fslice = AF.FocusSlice(slices, Quantity('10mm**-3'),
- mask_parent, [slice(0, 100), slice(0, 150)])
+ mask_parent, [slice(0, 100), slice(0, 150)],
+ Quantity('1mm'), Quantity('1mm'))
fc = FieldContainer(numpy.array([fslice for xr in xrange(1000)]))
fc.seal()
km.registerDataContainer(fc, temporary=True)
Added: trunk/src/pyphant/pyphant/tests/TestZStacks.py
===================================================================
--- trunk/src/pyphant/pyphant/tests/TestZStacks.py (rev 0)
+++ trunk/src/pyphant/pyphant/tests/TestZStacks.py 2010-05-26 20:45:06 UTC (rev 684)
@@ -0,0 +1,109 @@
+#!/usr/bin/env python2.5
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2006-2010, Rectorate of the University of Freiburg
+# Copyright (c) 2009-2010, Andreas W. Liehr (li...@us...)
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+# * Neither the name of the Freiburg Materials Research Center,
+# University of Freiburg nor the names of its contributors may be used to
+# endorse or promote products derived from this software without specific
+# prior written permission.
+#
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
+# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+u"""Provides unittest classes for ZStack feature.
+"""
+
+__id__ = "$Id$".replace('$','')
+__author__ = "$Author$".replace('$','')
+__version__ = "$Revision$".replace('$','')
+# $Source$
+
+import unittest
+import pkg_resources
+pkg_resources.require("pyphant")
+
+
+class ZStackTestCase(unittest.TestCase):
+ def setUp(self):
+ pass
+
+ def tearDown(self):
+ pass
+
+ def get_rp_ga_la(self, ppath):
+ import os
+ from pyphant.core.RecipeAlg import RecipeAlg
+ gradient_recipe = "pre-MF-no-median.h5"
+ label_recipe = "pre-MF-no-median.h5"
+ rpath = os.path.join(ppath, "tests", "resources", "recipes")
+ gradient_alg = RecipeAlg(os.path.join(rpath, gradient_recipe),
+ 'gradient', 'gradientWorker',
+ {'median':{'size':5}})
+ label_alg = RecipeAlg(os.path.join(rpath, label_recipe),
+ 'label', 'ndimage')
+ return (rpath, gradient_alg, label_alg)
+
+ def check(self, rng, value):
+ assert value >= rng[0] - rng[1] and value <= rng[0] + rng[1], \
+ "Value %f not in range (%f, %f)" % (value, rng[0] - rng[1],
+ rng[0] + rng[1])
+
+ def testZStack(self):
+ import os
+ from pyphant.core import KnowledgeManager
+ from pyphant import __path__ as ppath
+ from pyphant.core.ZStackManager import ZStack
+ print "Importing ZStack..."
+ zstack = ZStack(name="TestCase_ZStack",
+ xml_file=os.path.join(ppath[0], "tests", "resources",
+ "zstack", "_meta.xml"),
+ temporary=True)
+ print "Done."
+ print "Calculating ZStack-statistics..."
+ rpath, gradient_alg, label_alg = self.get_rp_ga_la(ppath[0])
+ zstack.recipe_path = rpath
+ statistics = zstack.get_statistics(gradient_alg, label_alg)
+ print "Done."
+ assert len(statistics['diameter'].data) == 2
+ imax = statistics['diameter'].data.argmax()
+ imin = statistics['diameter'].data.argmin()
+ self.check((200.0, 1.0), statistics['x-value'].data[imax])
+ self.check((200.0, 1.0), statistics['y-value'].data[imax])
+ self.check((300.0, 0.0), statistics['z-value'].data[imax])
+ self.check((20.5, 1.0), statistics['diameter'].data[imax])
+ self.check((53.0, 1.0), statistics['x-value'].data[imin])
+ self.check((53.0, 1.0), statistics['y-value'].data[imin])
+ self.check((300.0, 0.0), statistics['z-value'].data[imin])
+ self.check((6.5, 1.0), statistics['diameter'].data[imin])
+ print statistics['diameter'].data
+
+
+if __name__ == "__main__":
+ import sys
+ if len(sys.argv) == 1:
+ unittest.main()
+ else:
+ suite = unittest.TestLoader().loadTestsFromTestCase(
+ eval(sys.argv[1:][0]))
+ unittest.TextTestRunner().run(suite)
Added: trunk/src/pyphant/pyphant/tests/resources/recipes/AutoFocus.h5
===================================================================
--- trunk/src/pyphant/pyphant/tests/resources/recipes/AutoFocus.h5 (rev 0)
+++ trunk/src/pyphant/pyphant/tests/resources/recipes/AutoFocus.h5 2010-05-26 20:45:06 UTC (rev 684)
@@ -0,0 +1,18 @@
+\x89HDF
+
+ |
|
From: <zk...@us...> - 2010-05-25 20:33:39
|
Revision: 683
http://pyphant.svn.sourceforge.net/pyphant/?rev=683&view=rev
Author: zklaus
Date: 2010-05-25 20:33:33 +0000 (Tue, 25 May 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Enh: Added linear correction for round solar cells.
Modified Paths:
--------------
trunk/src/workers/OSC/OSC/__init__.py
Added Paths:
-----------
trunk/src/workers/OSC/OSC/OscThicknessCorrection.py
Copied: trunk/src/workers/OSC/OSC/OscThicknessCorrection.py (from rev 682, trunk/src/workers/OSC/OSC/__init__.py)
===================================================================
--- trunk/src/workers/OSC/OSC/OscThicknessCorrection.py (rev 0)
+++ trunk/src/workers/OSC/OSC/OscThicknessCorrection.py 2010-05-25 20:33:33 UTC (rev 683)
@@ -0,0 +1,73 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2007-2009, Rectorate of the University of Freiburg
+# Copyright (c) 2009, Andreas W. Liehr (li...@us...)
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+# * Neither the name of the Freiburg Materials Research Center,
+# University of Freiburg nor the names of its contributors may be used to
+# endorse or promote products derived from this software without specific
+# prior written permission.
+#
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
+# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+u"""
+"""
+
+__id__ = "$Id$"
+__author__ = "$Author$"
+__version__ = "$Revision$"
+# $Source$
+
+import numpy
+from pyphant.core import (Worker, Connectors,
+ Param, DataContainer)
+
+import scipy.interpolate
+from pyphant.quantities import Quantity
+import logging, copy, math
+
+class OscThicknessCorrector(Worker.Worker):
+ API = 2
+ VERSION = 1
+ REVISION = "$Revision$"[11:-1]
+ name = "Correct Thickness"
+
+ _sockets = [("osc", Connectors.TYPE_ARRAY)]
+ _params = [("max_correction", "Max correction", "30 nm", None)]
+
+ def inithook(self):
+ self._logger = logging.getLogger("pyphant")
+
+ @Worker.plug(Connectors.TYPE_IMAGE)
+ def correct(self, osc, subscriber=0):
+ x = osc[u'x-position']
+ y = osc[u'y-position']
+ t = copy.deepcopy(osc[u'thickness'])
+ r = numpy.sqrt(x.data**2+y.data**2)
+ r_max = r.max()
+ correction = Quantity(self.paramMax_correction.value)/t.unit
+ t.data = t.data + correction*(r/r_max-1)
+ t.longname='corrected thickness'
+ t.shortname='t_c'
+ t.seal()
+ return t
Modified: trunk/src/workers/OSC/OSC/__init__.py
===================================================================
--- trunk/src/workers/OSC/OSC/__init__.py 2010-05-25 14:11:56 UTC (rev 682)
+++ trunk/src/workers/OSC/OSC/__init__.py 2010-05-25 20:33:33 UTC (rev 683)
@@ -49,6 +49,7 @@
"MRA",
"OscAbsorption",
"OscCurrent",
+ "OscThicknessCorrection",
"OscVisualisers",
"Slicing",
"Smoother",
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-05-25 14:12:02
|
Revision: 682
http://pyphant.svn.sourceforge.net/pyphant/?rev=682&view=rev
Author: zklaus
Date: 2010-05-25 14:11:56 +0000 (Tue, 25 May 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Enh: Added recipe path to title bar
Modified Paths:
--------------
trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py
Modified: trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py 2010-05-25 09:11:09 UTC (rev 681)
+++ trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py 2010-05-25 14:11:56 UTC (rev 682)
@@ -117,7 +117,8 @@
ID_VIEW_LOGFILE = wx.NewId()
def __init__(self, _wxPyphantApp):
- wx.Frame.__init__(self, None, -1, "wxPyphant %s" % __version__,
+ self.titleStr = "wxPyphant %s | Recipe: %s" % (__version__, "%s")
+ wx.Frame.__init__(self, None, -1, self.titleStr % "None",
size=(640,480))
import PyphantCanvas
self._statusBar = self.CreateStatusBar()
@@ -237,6 +238,9 @@
self._remainingSpace = PyphantCanvas.PyphantCanvas(self, recipe)
else:
self._remainingSpace = PyphantCanvas.PyphantCanvas(self)
+ from pyphant.core.WebInterface import shorten
+ self.SetTitle(self.titleStr \
+ % shorten(self._wxPyphantApp.pathToRecipe, 30, 30))
else:
raise IOError('Unknown file format in file "%s"'\
% self._wxPyphantApp.pathToRecipe)
@@ -275,6 +279,8 @@
self._fileMenu.IsChecked(wx.ID_FILE4))
self._wxPyphantApp.pathToRecipe = filename
self.recipeState = 'clean'
+ from pyphant.core.WebInterface import shorten
+ self.SetTitle(self.titleStr % shorten(filename, 30, 30))
else:
dlg.Destroy()
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-05-25 09:11:16
|
Revision: 681
http://pyphant.svn.sourceforge.net/pyphant/?rev=681&view=rev
Author: zklaus
Date: 2010-05-25 09:11:09 +0000 (Tue, 25 May 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Enh: Improve ConfigurablePlot.
Fix: Minor Bug in ConfigureFrame for parameterless workers.
Modified Paths:
--------------
trunk/src/pyphant/pyphant/visualizers/ConfigurablePlot.py
trunk/src/pyphant/pyphant/wxgui2/ConfigureFrame.py
Modified: trunk/src/pyphant/pyphant/visualizers/ConfigurablePlot.py
===================================================================
--- trunk/src/pyphant/pyphant/visualizers/ConfigurablePlot.py 2010-05-22 19:29:18 UTC (rev 680)
+++ trunk/src/pyphant/pyphant/visualizers/ConfigurablePlot.py 2010-05-25 09:11:09 UTC (rev 681)
@@ -39,6 +39,7 @@
# $Source$
import matplotlib
+import matplotlib.transforms
import wx
import copy
@@ -99,69 +100,117 @@
x_key=0
y_key=1
c_key=2
+ radius=Quantity('0.4mm')
+ vmin = None
+ vmax = None
def __init__(self, parent, dataContainer):
self.dataContainer = dataContainer
PlotPanel.__init__(self, parent)
def draw(self):
- x = self.dataContainer[self.x_key]
- y = self.dataContainer[self.y_key]
- c = self.dataContainer[self.c_key]
- vmin = None
- vmax = None
- if vmin is not None:
- vmin /= c.unit
- if vmax is not None:
- vmax /= c.unit
+ self.x = self.dataContainer[self.x_key]
+ self.y = self.dataContainer[self.y_key]
+ self.c = self.dataContainer[self.c_key]
+ if self.vmin is not None:
+ self.vmin /= self.c.unit
+ if self.vmax is not None:
+ self.vmax /= self.c.unit
self.figure.clear()
self.ax = self.figure.add_subplot(111)
self.figure.subplots_adjust(hspace=0.8)
+ aspect = self.y.unit/self.x.unit
+ if not isinstance(aspect, Quantity):
+ self.ax.set_aspect(aspect)
try:
- scat = self.ax.scatter(x.data, y.data, c=c.data, vmin=vmin, vmax=vmax)
- self.figure.colorbar(scat, format=F(c), ax=self.ax)
- except:
- pass
- self.ax.set_xlabel(x.label)
- self.ax.set_ylabel(y.label)
+ self.scat = self.ax.scatter(self.x.data, self.y.data,
+ s=numpy.pi*(self.radius/self.x.unit)**2,
+ c=self.c.data,
+ vmin=self.vmin, vmax=self.vmax)
+ self.colorbar = self.figure.colorbar(self.scat, format=F(self.c),
+ ax=self.ax)
+ self.rescale(self.ax)
+ except Exception, e:
+ print e
+ self.ax.set_xlabel(self.x.label)
+ self.ax.set_ylabel(self.y.label)
try:
self.ax.set_title(self.dataContainer.attributes['title'])
except KeyError,TypeError:
pass
+ self.ax.callbacks.connect('xlim_changed', self.rescale)
+ self.ax.callbacks.connect('ylim_changed', self.rescale)
self.canvas.draw()
+ def rescale(self, ax):
+ scale = self.ax.transData.get_affine().get_matrix().copy()
+ scale[0,2]=0
+ scale[1,2]=0
+ scale[1,1]=scale[0,0]
+ self.scat.set_transform(matplotlib.transforms.Affine2D(scale))
+
+
class ConfigurationPanel(wx.PyPanel):
def __init__( self, parent, dataContainer, plot_panel):
wx.Panel.__init__( self, parent)
self.dataContainer = dataContainer
self.plot_panel = plot_panel
- self.columns = self.dataContainer.longnames.keys()
- sizer = wx.FlexGridSizer(2, 3)
- sizer.Add(wx.StaticText(self, label="Independent variable: "))
- sizer.Add(wx.StaticText(self, label="Dependent variable: "))
+ self.columns = [name for name in self.dataContainer.longnames.keys()
+ if isinstance(self.dataContainer[name],
+ DataContainer.FieldContainer)]
+ sizer = wx.FlexGridSizer(2, 6)
+ sizer.Add(wx.StaticText(self, label="x axis: "))
+ sizer.Add(wx.StaticText(self, label="y axis: "))
sizer.Add(wx.StaticText(self, label="Color variable: "))
+ sizer.Add(wx.StaticText(self, label="Minimal value: "))
+ sizer.Add(wx.StaticText(self, label="Maximal value: "))
+ sizer.Add(wx.StaticText(self, label="Radius: "))
self.independentCombo = wx.ComboBox(self,
style=wx.CB_READONLY | wx.CB_SORT,
choices=self.columns)
- self.independentCombo.SetStringSelection(self.dataContainer[0].longname)
+ self.independentCombo.SetStringSelection(self.columns[0])
+ longest = ''
+ for s in self.columns:
+ if len(s)>len(longest):
+ longest = s
+ offset = self.independentCombo.Size \
+ - self.independentCombo.GetTextExtent(longest)
+ maxSize = wx.Size(*self.independentCombo.GetTextExtent("White Reference Count somemore"))
+ maxSize += offset
+ self.independentCombo.MinSize = maxSize
sizer.Add(self.independentCombo)
self.dependentCombo = wx.ComboBox(self,
style=wx.CB_READONLY | wx.CB_SORT,
choices=self.columns)
- self.dependentCombo.SetStringSelection(self.dataContainer[1].longname)
+ self.dependentCombo.SetStringSelection(self.columns[1])
+ self.dependentCombo.MinSize = maxSize
sizer.Add(self.dependentCombo)
self.colorCombo = wx.ComboBox(self,
style=wx.CB_READONLY | wx.CB_SORT,
choices=self.columns)
- self.colorCombo.SetStringSelection(self.dataContainer[2].longname)
+ self.colorCombo.SetStringSelection(self.columns[2])
+ self.colorCombo.MinSize = maxSize
sizer.Add(self.colorCombo)
+ self.vsize = wx.Size(*self.colorCombo.GetTextExtent('1234567890 nm')) + offset
+ self.vmin_text = wx.TextCtrl(self, size=self.vsize,
+ style=wx.TE_PROCESS_ENTER)
+ sizer.Add(self.vmin_text)
+ self.vmax_text = wx.TextCtrl(self, size=self.vsize,
+ style=wx.TE_PROCESS_ENTER)
+ sizer.Add(self.vmax_text)
+ self.radius_text = wx.TextCtrl(self, size=self.vsize,
+ style=wx.TE_PROCESS_ENTER)
+ self.radius_text.Value = '0.4 mm'
+ sizer.Add(self.radius_text)
self.SetSizer(sizer)
self.Centre()
+ self.Bind(wx.EVT_COMBOBOX, self.changeColumns, self.independentCombo)
+ self.Bind(wx.EVT_COMBOBOX, self.changeColumns, self.dependentCombo)
+ self.Bind(wx.EVT_COMBOBOX, self.changeColumns, self.colorCombo)
+ self.Bind(wx.EVT_TEXT_ENTER, self.changeColumns, self.vmin_text)
+ self.Bind(wx.EVT_TEXT_ENTER, self.changeColumns, self.vmax_text)
+ self.Bind(wx.EVT_TEXT_ENTER, self.changeColumns, self.radius_text)
- self.Bind(wx.EVT_COMBOBOX, self.updatePlot, self.independentCombo)
- self.Bind(wx.EVT_COMBOBOX, self.updatePlot, self.dependentCombo)
- self.Bind(wx.EVT_COMBOBOX, self.updatePlot, self.colorCombo)
-
- def updatePlot(self, event=None):
+ def changeColumns(self, event=None):
independentVariable = self.independentCombo.GetValue()
dependentVariable = self.dependentCombo.GetValue()
colorVariable = self.colorCombo.GetValue()
@@ -171,7 +220,16 @@
return
self.plot_panel.x_key = independentVariable
self.plot_panel.y_key = dependentVariable
- self.plot_panel.c_key = colorVariable
+ if self.plot_panel.c_key != colorVariable:
+ self.plot_panel.c_key = colorVariable
+ field = self.dataContainer[colorVariable]
+ vmin = field.data.min().round()*field.unit
+ self.vmin_text.Value=str(vmin)
+ vmax = field.data.max().round()*field.unit
+ self.vmax_text.Value=str(vmax)
+ self.plot_panel.vmin = Quantity(self.vmin_text.Value)
+ self.plot_panel.vmax = Quantity(self.vmax_text.Value)
+ self.plot_panel.radius = Quantity(self.radius_text.Value)
self.plot_panel.draw()
self.GetSizer().Fit(self)
Modified: trunk/src/pyphant/pyphant/wxgui2/ConfigureFrame.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/ConfigureFrame.py 2010-05-22 19:29:18 UTC (rev 680)
+++ trunk/src/pyphant/pyphant/wxgui2/ConfigureFrame.py 2010-05-25 09:11:09 UTC (rev 681)
@@ -61,14 +61,14 @@
computer.join()
pl = worker.getParamList()
if len(pl) <= 0:
- self.Add(wx.StaticText("Nothing to be configured"))
+ self.Add(wx.StaticText(self, label="Nothing to be configured"))
else:
sizer = wx.FlexGridSizer(len(pl),3,5,5)
- sizer.Add(wx.StaticText(self,label="Label"))
- sizer.Add(wx.StaticText(self,label="Value"))
- sizer.Add(wx.StaticText(self,label="is external"))
+ sizer.Add(wx.StaticText(self, label="Label"))
+ sizer.Add(wx.StaticText(self, label="Value"))
+ sizer.Add(wx.StaticText(self, label="is external"))
for param in pl:
- sizer.Add(wx.StaticText(self,label=param.displayName))
+ sizer.Add(wx.StaticText(self, label=param.displayName))
vis=self._paramVisReg.createVisualizerFor(self, param)
sizer.Add(vis)
checkBox=wx.CheckBox(self)
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-05-22 19:29:25
|
Revision: 680
http://pyphant.svn.sourceforge.net/pyphant/?rev=680&view=rev
Author: zklaus
Date: 2010-05-22 19:29:18 +0000 (Sat, 22 May 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Fix: Bottle MS Windows
Fix: Windows KM bugs
Modified Paths:
--------------
trunk/src/pyphant/pyphant/core/KnowledgeManager.py
trunk/src/pyphant/pyphant/core/bottle.py
trunk/src/pyphant/pyphant/tests/TestKnowledgeManager.py
Modified: trunk/src/pyphant/pyphant/core/KnowledgeManager.py
===================================================================
--- trunk/src/pyphant/pyphant/core/KnowledgeManager.py 2010-05-22 17:33:17 UTC (rev 679)
+++ trunk/src/pyphant/pyphant/core/KnowledgeManager.py 2010-05-22 19:29:18 UTC (rev 680)
@@ -66,10 +66,12 @@
"""
Returns a unique filename for the given emd5.
"""
- emd5list = urlparse(dcId + '.h5')[2][2:].split('/')
+ emd5list = (dcId + '.h5')[7:].split('/')
emd5path = os.path.join(
*(emd5list[:-2] + [emd5list[-2][:10],
emd5list[-2][11:] + '.' + emd5list[-1]]))
+ #needed for windows version of pytables:
+ emd5path = emd5path.replace(':', '.')
directory = os.path.dirname(emd5path)
filename = os.path.basename(emd5path)
if temporary:
@@ -265,10 +267,14 @@
tmp_extension = ''
if temporary:
tmp_extension = 'tmp'
+ try:
+ remote_path = parsed.path
+ except AttributeError:
+ remote_path = parsed[2]
directory = os.path.join(KM_PATH, tmp_extension,
- 'registered', parsed[1])
+ 'registered')
filename = os.path.join(getPyphantPath(directory),
- os.path.basename(parsed[2]))
+ os.path.basename(remote_path))
if os.path.exists(filename):
i = 0
directory = os.path.dirname(filename)
Modified: trunk/src/pyphant/pyphant/core/bottle.py
===================================================================
--- trunk/src/pyphant/pyphant/core/bottle.py 2010-05-22 17:33:17 UTC (rev 679)
+++ trunk/src/pyphant/pyphant/core/bottle.py 2010-05-22 19:29:18 UTC (rev 680)
@@ -470,8 +470,8 @@
def send_file(filename, root, guessmime = True, mimetype = None):
""" Aborts execution and sends a static files as response. """
- root = os.path.abspath(root) + '/'
- filename = os.path.abspath(os.path.join(root, filename.strip('/')))
+ root = os.path.abspath(root)
+ filename = os.path.abspath(os.path.join(root, filename))
if not filename.startswith(root):
abort(401, "Access denied.")
Modified: trunk/src/pyphant/pyphant/tests/TestKnowledgeManager.py
===================================================================
--- trunk/src/pyphant/pyphant/tests/TestKnowledgeManager.py 2010-05-22 17:33:17 UTC (rev 679)
+++ trunk/src/pyphant/pyphant/tests/TestKnowledgeManager.py 2010-05-22 19:29:18 UTC (rev 680)
@@ -70,7 +70,12 @@
ptp.saveResult(self._fc, h5)
h5.close()
km = KnowledgeManager.getInstance()
- km.registerURL('file://' + h5name, temporary=True)
+ from urllib import pathname2url
+ url = pathname2url(h5name)
+ if not url.startswith('///'):
+ url = '//' + url
+ url = 'file:' + url
+ km.registerURL(url, temporary=True)
km_fc = km.getDataContainer(self._fc.id)
self.assertEqual(self._fc, km_fc)
os.remove(h5name)
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-05-22 17:33:23
|
Revision: 679
http://pyphant.svn.sourceforge.net/pyphant/?rev=679&view=rev
Author: zklaus
Date: 2010-05-22 17:33:17 +0000 (Sat, 22 May 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Enh: Improve ConfigurablePlot (read: make it work).
Enh: Notify the user if a result cannot be generated.
Added ConfigurablePlot for more flexible plotting.
Modified Paths:
--------------
trunk/src/pyphant/pyphant/visualizers/__init__.py
trunk/src/pyphant/pyphant/wxgui2/PyphantDiagram.py
Added Paths:
-----------
trunk/src/pyphant/pyphant/visualizers/ConfigurablePlot.py
Added: trunk/src/pyphant/pyphant/visualizers/ConfigurablePlot.py
===================================================================
--- trunk/src/pyphant/pyphant/visualizers/ConfigurablePlot.py (rev 0)
+++ trunk/src/pyphant/pyphant/visualizers/ConfigurablePlot.py 2010-05-22 17:33:17 UTC (rev 679)
@@ -0,0 +1,210 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2006-2008, Rectorate of the University of Freiburg
+# Copyright (c) 2009, Andreas W. Liehr (li...@us...)
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+# * Neither the name of the Freiburg Materials Research Center,
+# University of Freiburg nor the names of its contributors may be used to
+# endorse or promote products derived from this software without specific
+# prior written permission.
+#
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
+# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+u"""
+"""
+
+__id__ = "$Id$"
+__author__ = "$Author$"
+__version__ = "$Revision$"
+# $Source$
+
+import matplotlib
+import wx
+
+import copy
+import pylab
+import numpy
+import pyphant.core.Connectors
+from pyphant.core import DataContainer
+from pyphant.wxgui2.DataVisReg import DataVisReg
+from pyphant.quantities import isQuantity, Quantity
+
+
+class PlotPanel (wx.PyPanel):
+ """The PlotPanel has a Figure and a Canvas. OnSize events simply set a
+ flag, and the actual resizing of the figure is triggered by an Idle event."""
+ def __init__( self, parent, color=None, dpi=None, **kwargs ):
+ from matplotlib.backends.backend_wxagg import (FigureCanvasWxAgg,
+ NavigationToolbar2WxAgg)
+ from matplotlib.figure import Figure
+
+ # initialize Panel
+ if 'id' not in kwargs.keys():
+ kwargs['id'] = wx.ID_ANY
+ if 'style' not in kwargs.keys():
+ kwargs['style'] = wx.NO_FULL_REPAINT_ON_RESIZE
+ wx.Panel.__init__( self, parent, **kwargs )
+ self.parent = parent
+
+ # initialize matplotlib stuff
+ self.figure = Figure( None, dpi )
+ self.canvas = FigureCanvasWxAgg( self, -1, self.figure )
+ self.ax = self.figure.add_subplot(111)
+ self.figure.subplots_adjust(hspace=0.8)
+ self.toolbar = NavigationToolbar2WxAgg(self.canvas)
+
+ vSizer = wx.BoxSizer(wx.VERTICAL)
+ vSizer.Add(self.canvas, 1, wx.LEFT | wx.TOP | wx.GROW)
+ vSizer.Add(self.toolbar, 0, wx.EXPAND)
+ self.draw()
+ self.SetAutoLayout(True)
+ self.SetSizer(vSizer)
+ self.Layout()
+
+ def draw(self):
+ pass # abstract, to be overridden by child classes
+
+
+class F(pylab.Formatter):
+ def __init__(self, container, *args, **kwargs):
+ self.container=container
+ def __call__(self, x, pos=None):
+ try:
+ return str(x*self.container.unit)
+ except IndexError, error:
+ return str(x)
+
+
+class OscPlotPanel(PlotPanel):
+ x_key=0
+ y_key=1
+ c_key=2
+ def __init__(self, parent, dataContainer):
+ self.dataContainer = dataContainer
+ PlotPanel.__init__(self, parent)
+
+ def draw(self):
+ x = self.dataContainer[self.x_key]
+ y = self.dataContainer[self.y_key]
+ c = self.dataContainer[self.c_key]
+ vmin = None
+ vmax = None
+ if vmin is not None:
+ vmin /= c.unit
+ if vmax is not None:
+ vmax /= c.unit
+ self.figure.clear()
+ self.ax = self.figure.add_subplot(111)
+ self.figure.subplots_adjust(hspace=0.8)
+ try:
+ scat = self.ax.scatter(x.data, y.data, c=c.data, vmin=vmin, vmax=vmax)
+ self.figure.colorbar(scat, format=F(c), ax=self.ax)
+ except:
+ pass
+ self.ax.set_xlabel(x.label)
+ self.ax.set_ylabel(y.label)
+ try:
+ self.ax.set_title(self.dataContainer.attributes['title'])
+ except KeyError,TypeError:
+ pass
+ self.canvas.draw()
+
+class ConfigurationPanel(wx.PyPanel):
+ def __init__( self, parent, dataContainer, plot_panel):
+ wx.Panel.__init__( self, parent)
+ self.dataContainer = dataContainer
+ self.plot_panel = plot_panel
+ self.columns = self.dataContainer.longnames.keys()
+ sizer = wx.FlexGridSizer(2, 3)
+ sizer.Add(wx.StaticText(self, label="Independent variable: "))
+ sizer.Add(wx.StaticText(self, label="Dependent variable: "))
+ sizer.Add(wx.StaticText(self, label="Color variable: "))
+ self.independentCombo = wx.ComboBox(self,
+ style=wx.CB_READONLY | wx.CB_SORT,
+ choices=self.columns)
+ self.independentCombo.SetStringSelection(self.dataContainer[0].longname)
+ sizer.Add(self.independentCombo)
+ self.dependentCombo = wx.ComboBox(self,
+ style=wx.CB_READONLY | wx.CB_SORT,
+ choices=self.columns)
+ self.dependentCombo.SetStringSelection(self.dataContainer[1].longname)
+ sizer.Add(self.dependentCombo)
+ self.colorCombo = wx.ComboBox(self,
+ style=wx.CB_READONLY | wx.CB_SORT,
+ choices=self.columns)
+ self.colorCombo.SetStringSelection(self.dataContainer[2].longname)
+ sizer.Add(self.colorCombo)
+ self.SetSizer(sizer)
+ self.Centre()
+
+ self.Bind(wx.EVT_COMBOBOX, self.updatePlot, self.independentCombo)
+ self.Bind(wx.EVT_COMBOBOX, self.updatePlot, self.dependentCombo)
+ self.Bind(wx.EVT_COMBOBOX, self.updatePlot, self.colorCombo)
+
+ def updatePlot(self, event=None):
+ independentVariable = self.independentCombo.GetValue()
+ dependentVariable = self.dependentCombo.GetValue()
+ colorVariable = self.colorCombo.GetValue()
+ if (u''==independentVariable
+ or u''==dependentVariable
+ or u''==colorVariable):
+ return
+ self.plot_panel.x_key = independentVariable
+ self.plot_panel.y_key = dependentVariable
+ self.plot_panel.c_key = colorVariable
+ self.plot_panel.draw()
+ self.GetSizer().Fit(self)
+
+class PlotFrame(wx.Frame):
+ name='Plot frame'
+ def __init__(self, dataContainer, show=True):
+ wx.Frame.__init__(self, None, -1, "Visualize "+dataContainer.longname)
+ self.sizer = wx.BoxSizer(wx.VERTICAL)
+ self.plot_panel = OscPlotPanel(self, dataContainer)
+ self.configuration_panel = ConfigurationPanel(self, dataContainer,
+ self.plot_panel)
+ self.sizer.Add(self.configuration_panel, 0, wx.EXPAND)
+ self.sizer.Add(self.plot_panel, 1, wx.LEFT | wx.TOP | wx.GROW)
+ self.SetSizer(self.sizer)
+ self.Centre()
+ self.Show(True)
+
+ def createVisualizerList(self):
+ ids={}
+ if isinstance(self.dataContainer, DataContainer.SampleContainer):
+ data_type = pyphant.core.Connectors.TYPE_ARRAY
+ elif isinstance(self.dataContainer, DataContainer.FieldContainer):
+ data_type = pyphant.core.Connectors.TYPE_IMAGE
+ for visualizer in DataVisReg.getInstance().getVisualizers(data_type):
+ id=wx.NewId()
+ ids[id]=visualizer
+ #self.GetCanvas().Bind(wx.EVT_MENU, self.visualize, id=id)
+ return ids
+
+ def OnExit(self, event):
+ self.Close()
+
+
+
+
+DataVisReg.getInstance().registerVisualizer(pyphant.core.Connectors.TYPE_ARRAY, PlotFrame)
Modified: trunk/src/pyphant/pyphant/visualizers/__init__.py
===================================================================
--- trunk/src/pyphant/pyphant/visualizers/__init__.py 2010-05-22 12:34:31 UTC (rev 678)
+++ trunk/src/pyphant/pyphant/visualizers/__init__.py 2010-05-22 17:33:17 UTC (rev 679)
@@ -51,4 +51,4 @@
#for module in filter(lambda file: file[-3:]=='.py', files):
# if not module == '__init__.py':
# exec 'import ' + module[:-3]
-import ImageVisualizer, Chart, KMVisualizer
+import ImageVisualizer, Chart, KMVisualizer, ConfigurablePlot
Modified: trunk/src/pyphant/pyphant/wxgui2/PyphantDiagram.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/PyphantDiagram.py 2010-05-22 12:34:31 UTC (rev 678)
+++ trunk/src/pyphant/pyphant/wxgui2/PyphantDiagram.py 2010-05-22 17:33:17 UTC (rev 679)
@@ -264,9 +264,14 @@
progress.Destroy()
if computer.isAlive():
computer.join()
- self._ids[event.GetId()](computer.result)
+ result = computer.result
else:
- self._ids[event.GetId()](self._plug._result)
+ result = self._plug._result
+ if not result == None:
+ self._ids[event.GetId()](result)
+ else:
+ wx.MessageBox(u"No result is available.\nPlease check the logs.",
+ u"No Result", wx.ICON_ERROR)
def OnRightClick(self, x, y, keys, attachments):
if not hasattr(self, '_menu'):
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <zk...@us...> - 2010-05-22 12:34:37
|
Revision: 678
http://pyphant.svn.sourceforge.net/pyphant/?rev=678&view=rev
Author: zklaus
Date: 2010-05-22 12:34:31 +0000 (Sat, 22 May 2010)
Log Message:
-----------
Merge branch 'master' into svn-trunk
* master:
Enh: Auto index rebuilding
Fix: global exception handling
Modified Paths:
--------------
trunk/src/pyphant/pyphant/core/KnowledgeManager.py
trunk/src/pyphant/pyphant/core/KnowledgeNode.py
trunk/src/pyphant/pyphant/core/SQLiteWrapper.py
trunk/src/pyphant/pyphant/wxgui2/PyphantCanvas.py
trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py
Modified: trunk/src/pyphant/pyphant/core/KnowledgeManager.py
===================================================================
--- trunk/src/pyphant/pyphant/core/KnowledgeManager.py 2010-04-19 18:15:08 UTC (rev 677)
+++ trunk/src/pyphant/pyphant/core/KnowledgeManager.py 2010-05-22 12:34:31 UTC (rev 678)
@@ -161,8 +161,6 @@
else:
self.dbase = KM_DBASE
self.any_value = AnyValue()
- with SQLiteWrapper(self.dbase) as wrapper:
- wrapper.setup_dbase()
self.node = None # for hooking up a KnowledgeNode
self.uuid = uuid1().urn
tmpdir = getPyphantPath(os.path.join(KM_PATH, 'tmp'))
@@ -172,7 +170,51 @@
rmtree(tmpdir)
except OSError:
self.logger.warn("Could not delete '%s'." % tmpdir)
+ with SQLiteWrapper(self.dbase) as wrapper:
+ rebuild = wrapper.dbase_broken()
+ if rebuild:
+ self.logger.info("dbase needs rebuild")
+ self.rebuildIndex()
+ else:
+ with SQLiteWrapper(self.dbase) as wrapper:
+ wrapper.setup_sqlite()
+ def rebuildIndex(self):
+ self.logger.info("rebuilding dbase...")
+ oldname = self.dbase + ".bak"
+ if os.path.exists(oldname):
+ os.remove(oldname)
+ os.rename(self.dbase, oldname)
+ with SQLiteWrapper(self.dbase) as wrapper:
+ wrapper.setup_dbase()
+ self.updateIndex()
+ self.logger.info("done rebuilding dbase")
+
+ def updateIndex(self):
+ file_list = []
+ def accumulate_files(flist, directory, files):
+ for fname in [afname for afname in files if afname.endswith('.h5')]:
+ flist.append(os.path.realpath(os.path.join(directory, fname)))
+ os.path.walk(getPyphantPath(KM_PATH), accumulate_files, file_list)
+ from wx import (ProgressDialog, PyNoAppError)
+ try:
+ pdial = ProgressDialog('Rebuilding index...', ' ' * 100,
+ maximum=len(file_list))
+ except PyNoAppError:
+ pdial = None
+ count = 1
+ for realname in file_list:
+ if pdial is not None:
+ pdial.Update(count, os.path.basename(realname))
+ count += 1
+ try:
+ self.registerH5(realname)
+ except:
+ self.logger.warn("Could not extract meta data from '%s'."\
+ % realname)
+ if pdial is not None:
+ pdial.Destroy()
+
def hasDataContainer(self, dcid):
"""
Returns whether the given DC is stored locally.
Modified: trunk/src/pyphant/pyphant/core/KnowledgeNode.py
===================================================================
--- trunk/src/pyphant/pyphant/core/KnowledgeNode.py 2010-04-19 18:15:08 UTC (rev 677)
+++ trunk/src/pyphant/pyphant/core/KnowledgeNode.py 2010-05-22 12:34:31 UTC (rev 678)
@@ -228,7 +228,7 @@
try:
columns = [('host', 'TEXT'), ('port', 'INT'), ('status', 'INT'),
('', 'UNIQUE(host, port)')]
- create_table('kn_remotes', columns, cursor)
+ create_table('kn_remotes', columns, cursor, ignore_exists=True)
cursor.execute("SELECT * FROM kn_remotes")
self.remotes = [RemoteKN(host, port, status) \
for host, port, status in cursor]
Modified: trunk/src/pyphant/pyphant/core/SQLiteWrapper.py
===================================================================
--- trunk/src/pyphant/pyphant/core/SQLiteWrapper.py 2010-04-19 18:15:08 UTC (rev 677)
+++ trunk/src/pyphant/pyphant/core/SQLiteWrapper.py 2010-05-22 12:34:31 UTC (rev 678)
@@ -45,6 +45,8 @@
from pyphant.quantities import (Quantity,PhysicalUnit,_base_units)
from types import (FloatType, IntType, LongType, StringTypes)
+DBASE_VERSION = 1 #increment if there have been structural changes to the dbase!
+
def quantity2powers(quantity):
numberOfBaseUnits = len(_base_units)
if isinstance(quantity, Quantity):
@@ -142,8 +144,11 @@
wc += ')'
return wc
-def create_table(table_name, columns, cursor):
- query = "CREATE TABLE IF NOT EXISTS %s (" % (table_name, )
+def create_table(table_name, columns, cursor, ignore_exists=False):
+ if ignore_exists:
+ query = "CREATE TABLE IF NOT EXISTS %s (" % (table_name, )
+ else:
+ query = "CREATE TABLE %s (" % (table_name, )
for name, type in columns:
query += name + " " + type + ", "
query = query[:-2] + ")"
@@ -151,7 +156,7 @@
def create_trigger(trigger_name, action, table_name,
statements, cursor):
- query = "CREATE TRIGGER IF NOT EXISTS %s AFTER %s ON %s "\
+ query = "CREATE TRIGGER %s AFTER %s ON %s "\
"FOR EACH ROW BEGIN %s END"
st_query = ''
for st in statements:
@@ -227,9 +232,13 @@
return SCRowWrapper(emd5, self.cursor)
raise KeyError(emd5)
- def setup_dbase(self):
+ def setup_sqlite(self):
sqlite3.register_converter('QUANTITY', dbase2quantity)
sqlite3.register_converter('LATEX', dbase2latex)
+ #clean tmp:
+ self.cursor.execute("DELETE FROM km_temporary")
+
+ def setup_dbase(self):
#create tables:
columns = [('sc_id', 'TEXT PRIMARY KEY UNIQUE NOT NULL'),
('longname', 'TEXT'),
@@ -279,6 +288,10 @@
('bit', 'INT'),
('', 'UNIQUE(m, g, s, A, K, mol, cd, rad, sr, EUR,bit)')]
create_table('km_base_units', columns, self.cursor)
+ columns = [('version', 'INT')]
+ create_table('db_info', columns, self.cursor)
+ self.cursor.execute("INSERT INTO db_info (version) VALUES (?)",
+ (DBASE_VERSION, ))
#create triggers:
create_trigger('trigger_del_fc', 'DELETE', 'km_fc',
['DELETE FROM km_attributes WHERE dc_id=OLD.fc_id',
@@ -292,9 +305,16 @@
['DELETE FROM km_fc WHERE fc_id=OLD.dc_id',
'DELETE FROM km_sc WHERE sc_id=OLD.dc_id'],
self.cursor)
- #clean tmp:
- self.cursor.execute("DELETE FROM km_temporary")
+ self.setup_sqlite()
+ def dbase_broken(self):
+ try:
+ self.cursor.execute("SELECT version FROM db_info")
+ version = self.cursor.fetchone()[0]
+ return version is not DBASE_VERSION
+ except sqlite3.OperationalError:
+ return True
+
def has_entry(self, id):
exe = self.cursor.execute
if id == 'IndexMarker':
Modified: trunk/src/pyphant/pyphant/wxgui2/PyphantCanvas.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/PyphantCanvas.py 2010-04-19 18:15:08 UTC (rev 677)
+++ trunk/src/pyphant/pyphant/wxgui2/PyphantCanvas.py 2010-05-22 12:34:31 UTC (rev 678)
@@ -46,7 +46,7 @@
class PyphantCanvas(sogl.ShapeCanvas):
def __init__(self,parent,recipe=CompositeWorker.CompositeWorker(), id=-1,pos=wx.DefaultPosition,
- size=wx.DefaultSize,style=wx.SUNKEN_BORDER,name="PyphantCanvas"):
+ size=wx.DefaultSize,style=wx.NO_BORDER,name="PyphantCanvas"):
sogl.ShapeCanvas.__init__(self,parent, id, pos, size, style, name)
#self.SetVirtualSize((1100,1000))
self.SetScrollRate(20,20)
Modified: trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py
===================================================================
--- trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py 2010-04-19 18:15:08 UTC (rev 677)
+++ trunk/src/pyphant/pyphant/wxgui2/wxPyphantApplication.py 2010-05-22 12:34:31 UTC (rev 678)
@@ -41,6 +41,7 @@
from pyphant.core.Helpers import getPyphantPath
LOGDIR = getPyphantPath()
import logging
+import logging.handlers
# logging.basicConfig(level=logging.DEBUG)
#else:
logging.basicConfig(level=logging.DEBUG,
@@ -50,10 +51,13 @@
"d:%(module)s.%(funcName)s(l %(lineno)d):%(message)s")
console = logging.StreamHandler()
console.setLevel(logging.WARNING)
+logbuffer = logging.handlers.MemoryHandler(1000)
logging.getLogger('').addHandler(console)
+logging.getLogger('').addHandler(logbuffer)
import sys
import wx
+import wx.aui
import sogl
import pyphant.wxgui2.paramvisualization.ParamVisReg as ParamVisReg
import pyphant.core.PyTablesPersister
@@ -74,7 +78,6 @@
if not wx.PySimpleApp.OnInit(self):
return False
self._logger = logging.getLogger("pyphant")
- self._excframe = wx.Frame(None, -2, "")
sys.excepthook = self.excepthook
sogl.SOGLInitialize()
self._knowledgeNode = None
@@ -87,16 +90,6 @@
self._logger.debug(u"An unhandled exception occured.",
exc_info=(type, value, trace))
sys.__excepthook__(type, value, trace)
- try:
- cpt = type.__name__
- #import traceback
- #traceStr = ''.join(traceback.format_exception(type, value, trace))
- msg = "Additional information:\n%s\n\n" % value
- dlg = wx.MessageDialog(self._excframe, msg, cpt, wx.OK)
- dlg.ShowModal()
- except:
- # avoid loop if displaying error message fails
- self._logger.debug(u"Failed to display error message in wxPyphant.")
def getMainFrame(self):
return self._frame
@@ -120,6 +113,8 @@
ID_WINDOW_BOTTOM = 103
ID_CLOSE_COMPOSITE_WORKER = wx.NewId()
ID_UPDATE_PYPHANT = wx.NewId()
+ ID_VIEW_WORKERREP = wx.NewId()
+ ID_VIEW_LOGFILE = wx.NewId()
def __init__(self, _wxPyphantApp):
wx.Frame.__init__(self, None, -1, "wxPyphant %s" % __version__,
@@ -131,11 +126,12 @@
self._initSash()
self.recipeState = None
self.onOpenCompositeWorker(None)
+ self._initAui()
self._workerRepository.Bind(wx.EVT_SASH_DRAGGED_RANGE,
self.onFoldPanelBarDrag,
id=self.ID_WINDOW_TOP,
id2=self.ID_WINDOW_BOTTOM)
- self.Bind(wx.EVT_SIZE, self.onSize)
+ #self.Bind(wx.EVT_SIZE, self.onSize)
self.compositeWorkerStack = []
wx.MessageBox("Located log directory at %s.\n"
"Logging will go to %s." %
@@ -144,15 +140,53 @@
def _initSash(self):
self._workerRepository = wx.SashLayoutWindow(
- self, self.ID_WINDOW_RIGHT, wx.DefaultPosition, wx.Size(200,1000),
- wx.NO_BORDER | wx.SW_3D | wx.CLIP_CHILDREN)
- self._workerRepository.SetDefaultSize(wx.Size(220,1000))
- self._workerRepository.SetOrientation(wx.LAYOUT_VERTICAL)
- self._workerRepository.SetAlignment(wx.LAYOUT_RIGHT)
- self._workerRepository.SetSashVisible(wx.SASH_LEFT, True)
- self._workerRepository.SetExtraBorderSize(10)
+ self, self.ID_WINDOW_RIGHT, wx.DefaultPosition, wx.Size(220,1000),
+ wx.NO_BORDER)
+ #self._workerRepository.SetDefaultSize(wx.Size(220,1000))
+ #self._workerRepository.SetOrientation(wx.LAYOUT_VERTICAL)
+ #self._workerRepository.SetAlignment(wx.LAYOUT_RIGHT)
+ #self._workerRepository.SetSashVisible(wx.SASH_LEFT, True)
+ #self._workerRepository.SetExtraBorderSize(10)
WorkerRepository.WorkerRepository(self._workerRepository)
+ def _initAui(self):
+ self._auiManager = wx.aui.AuiManager(self)
+ class ClickableText(wx.TextCtrl):
+ def __init__(self, *args, **kwargs):
+ wx.TextCtrl.__init__(self, *args, **kwargs)
+ self.Bind(wx.EVT_LEFT_DOWN, self.OnClick)
+ def OnClick(self, event):
+ logbuffer.flush()
+ event.Skip()
+ self._logpane = ClickableText(self, -1, "",
+ wx.DefaultPosition, wx.Size(640, 200),
+ wx.NO_BORDER | wx.TE_MULTILINE \
+ | wx.TE_READONLY)
+ class WxHandler(logging.Handler):
+ def __init__(self, ctrl):
+ logging.Handler.__init__(self)
+ self.ctrl = ctrl
+ def emit(self, record):
+ try:
+ self.ctrl.AppendText(self.format(record) + "\n")
+ except wx.PyDeadObjectError:
+ pass
+ handler = WxHandler(self._logpane)
+ handler.setFormatter(logging.Formatter(
+ "%(asctime)s - %(levelname)s:%(name)s:%(thread)"\
+ "d:%(module)s.%(funcName)s(l %(lineno)d):%(message)s"))
+ logbuffer.setTarget(handler)
+ logbuffer.flush()
+ self._auiManager.AddPane(self._logpane, wx.BOTTOM, 'Logfile '\
+ '(click into text to update)')
+ self._auiManager.AddPane(self._workerRepository, wx.RIGHT,
+ 'Worker Repository')
+ self._auiManager.AddPane(self._remainingSpace, wx.CENTER, 'Main')
+ wrpane = self._auiManager.GetPane(self._workerRepository)
+ wrpane.Floatable(False)
+ wrpane.Movable(False)
+ self._auiManager.Update()
+
def onSize(self, event):
wx.LayoutAlgorithm().LayoutWindow(self,self._remainingSpace)
event.Skip()
@@ -164,7 +198,7 @@
self._workerRepository.SetDefaultSize(
wx.Size(event.GetDragRect().width, 1000))
# Leaves bits of itself behind sometimes
- wx.LayoutAlgorithm().LayoutWindow(self, self._remainingSpace)
+ #wx.LayoutAlgorithm().LayoutWindow(self, self._remainingSpace)
self._remainingSpace.Refresh()
event.Skip()
@@ -195,16 +229,24 @@
self._wxPyphantApp.pathToRecipe = path
dlg.Destroy()
import PyphantCanvas
- if self._wxPyphantApp.pathToRecipe[-3:] == '.h5':
- if os.path.exists(self._wxPyphantApp.pathToRecipe):
- recipe = pyphant.core.PyTablesPersister.loadRecipeFromHDF5File(
- self._wxPyphantApp.pathToRecipe)
- self._remainingSpace = PyphantCanvas.PyphantCanvas(self, recipe)
+ try:
+ if self._wxPyphantApp.pathToRecipe[-3:] == '.h5':
+ if os.path.exists(self._wxPyphantApp.pathToRecipe):
+ recipe = pyphant.core.PyTablesPersister.loadRecipeFromHDF5File(
+ self._wxPyphantApp.pathToRecipe)
+ self._remainingSpace = PyphantCanvas.PyphantCanvas(self, recipe)
+ else:
+ self._remainingSpace = PyphantCanvas.PyphantCanvas(self)
else:
- self._remainingSpace = PyphantCanvas.PyphantCanvas(self)
- else:
- raise IOError("Unknown file format in file \"%\""\
- % self._wxPyphantApp.pathToRecipe)
+ raise IOError('Unknown file format in file "%s"'\
+ % self._wxPyphantApp.pathToRecipe)
+ except:
+ wx.MessageBox("An error has occurred while opening "\
+ "the recipe.\nPlease investigate the logfile "\
+ "for further details.\nThe logfile is located at %s"\
+ % os.path.join(LOGDIR, 'pyphant.log'),
+ "Recipe broken, unknown format or outdated!")
+ raise
self.recipeState = 'clean'
self._remainingSpace.diagram.recipe.registerListener(self.recipeChanged)
@@ -257,6 +299,8 @@
"&Close Composite Worker")
self._updateMenu = self.createUpdateMenu()
self._menuBar.Append(self._updateMenu, "&Update")
+ self._viewMenu = self.createViewMenu()
+ self._menuBar.Append(self._viewMenu, "&View")
self.SetMenuBar(self._menuBar)
self._menuBar.EnableTop(1, False)
#self.Bind(wx.EVT_MENU, self.onCreateNew, id=wx.ID_NEW)
@@ -285,10 +329,28 @@
self.Bind(wx.EVT_MENU, self.onUpdatePyphant, id=nId)
return updateMenu
+ def createViewMenu(self):
+ viewMenu = wx.Menu()
+ viewMenu.Append(self.ID_VIEW_WORKERREP, "&Worker Repository")
+ viewMenu.Append(self.ID_VIEW_LOGFILE, "&Logfile")
+ self.Bind(wx.EVT_MENU, self.onWorkerRep, id=self.ID_VIEW_WORKERREP)
+ self.Bind(wx.EVT_MENU, self.onLogfile, id=self.ID_VIEW_LOGFILE)
+ return viewMenu
+
def onUpdatePyphant(self, event):
import pyphant.core.UpdateManager
pyphant.core.UpdateManager.updatePackage(self.updateIds[event.Id])
+ def onWorkerRep(self, event):
+ wrpane = self._auiManager.GetPane(self._workerRepository)
+ wrpane.Show(True)
+ self._auiManager.Update()
+
+ def onLogfile(self, event):
+ logpane = self._auiManager.GetPane(self._logpane)
+ logpane.Show(True)
+ self._auiManager.Update()
+
def onQuit(self,event):
self.Close()
@@ -309,7 +371,7 @@
self._wxPyphantApp._knowledgeNode.stop()
except AttributeError:
pass
- self._wxPyphantApp._excframe.Destroy()
+ self._auiManager.UnInit()
self.Destroy()
def editCompositeWorker(self, worker):
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: Rolf W. <ro...@di...> - 2010-05-06 22:10:02
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Klaus Zimmermann schrieb: > Hello all, > > > Am Freitag, den 30.04.2010, 21:40 +0200 schrieb Rolf Wuerdemann: >> On Thu, 29 Apr 2010 07:34:09 +0200, "Dr. Andreas W. Liehr" >> <ob...@fm...> wrote: >>> [...] > > So what do you think? We had discussed this topic some times .... My choice would be some hybride ... A set of hooks with pre-defined functions. Users can use our supplied functions and get the data in an language approbiate manner (whatever this means). They can also supply their own functions and store the data in any way they likes to (but are also responsible for the storage) ... Why I would choose this way? For the most applications an "In Memory" storage would be nice (esp. for smaller applications) and if we force user to set up their own data storage they can (and will) also write their own libs and perhaps will use other formats (csv?) I see the big usage of fmf - and from this point I like to take the entry point for using fmf as low as possible ... from this point of view I would recommend the above mentioned hybride - which so can use in a way he or she wants - and no exclusive-or solution. > > Cheers, > Klaus > > Kind regards, Rolf - -- Security is an illusion - Datasecurity twice Rolf Würdemann - ro...@di... GnuPG fingerprint: 7383 348F 67D1 CD27 C90F DDD0 86A3 31B6 67F0 D02F jabber: ro...@di... ECF127C7 EAB85F87 BC75ACB5 2EC646D4 99211A31 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvjPjAACgkQhqMxtmfw0C8t+QCgmPPrj9q7kbiymw/vLLYvu65O 3m4An2TK117JGyiBokOrOyaHj18bhR6V =aWyW -----END PGP SIGNATURE----- |
|
From: Rolf W. <ro...@di...> - 2010-05-06 21:19:54
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dr. Andreas W. Liehr schrieb: > Dear *, Dear *.* ;) At first many thanks to Klaus for clearifying the situation - now the zip-file issue get's a little clearer ;) > >> So what do you think: >> 1) Do we need a specification for aggregate data? > Yes. Good question - it will give researchers the possibilty to inform people which parameters they like to get changed and which parameters changed during time (to x,y should change, but how about temperature ;) For processing it might be nicer to let the user choose this .... > >> 2) Is zip a good means for that? > Yes. Zip has the property to be wide spread - other than tar.gz or .arj > >> 3) Can we get proper inspiration from prior art? What about jar files, >> eggs? > Perhaps. But our starting concept is quite good. If we don't use language specific files (that we don't do) an look on other formats may help us - for our current problem an 'secured' section '*volatile' or '*series' which defines the volatile parameters might help us. So we can define on which fields changes should be tracked (as variables) and which fields should get ignored. > > There is another point: What if you want to document a bunch of images > from experiments? We could allow for an index.fmf file within a ZIP > archive, which relates image filenames to the relevant metadata. This > could be the proper way to document Alexanders's image batches. I keep Alex out of this - just to think plain about fmf and not fmf as input-system for batch processing with the pyphant ;) Open wider - don't think on "usual images" - take stm data during an grow analysis or wavefunctions for example - the base system is prepared in the same manner over the complete data set and it might be nice to have an file which contains the experimental setup and links to the individual measurements .... So I would say yes - use the fmf also this way - but on the other hand I see the problem to overload the fmf > > All the best, > Andreas > > Kind regards, Rolf >[...] - -- Security is an illusion - Datasecurity twice Rolf Würdemann - ro...@di... GnuPG fingerprint: 7383 348F 67D1 CD27 C90F DDD0 86A3 31B6 67F0 D02F jabber: ro...@di... ECF127C7 EAB85F87 BC75ACB5 2EC646D4 99211A31 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvjMmwACgkQhqMxtmfw0C9gjgCfW67sTXoRx4xzaIMPIYYA5dOW EfUAoLSdiRfEoaFYk2UoR6nSEmLHpHaY =3BJc -----END PGP SIGNATURE----- |
|
From: Klaus Z. <kla...@fm...> - 2010-05-04 09:40:01
|
Dear *, we are working on it. So far all major bugs seem to have been fixed. But you can take a look at the open issues over at the redmine. Also macports integration is almost complete and the work with Kristian is progressing nicely. So stay tuned and file bugs in the tracker. Cheers, Klaus On 04.05.10 06:37, Dr. Andreas W. Liehr wrote: > Dear *, > > so let's publish a new Pyphant release. Are there any issues, which > have to be resolved before? What are the next steps? > > All the best, > Andreas > > Am 02.05.10 14:16, schrieb Klaus Zimmermann: >> Hello all, >> >> first of all thanks for the discussion! It is good to see this list put >> to use. >> >> Before I answer to individual points, two general remarks: >> >> 1) I am absolutely in favor of splitting pyphant and fmf. >> I am however not entirely sure, if this should also be done by >> separating the projects in sourceforge or merely by disentangling the >> code base, possibly into two separate repositories. Anyway I want to >> drop svn and completely switch over to git, which in the mean time is >> also offered by sf. With this, keep in mind that a git repository >> corresponds more closely to a svn module and less to a svn repository. >> This is also why, in contrast to other scm, most notably svn, sf >> supports multiple git repositories under the hood of one project. >> >> 2) No matter how the separation occurs, I would very much like to push >> out one last stable release of the pyphant as it is with all its flaws. >> Than we can keep the svn repo around for bugfixing in the pyphant 1 >> series and make a fresh start in a set of git repos for a pyphant 2 >> series. However, since Rolf needs some fmf functionality for other >> projects pretty much right now, of course we can start the new repos >> already. >> >> Rolf wrote: >>> Btw: my idea behind the fork of fmf was to give other the >>> possibility to use the fmf without having to install the whole pyphant. >>> And also to give some libs in other languages - so people from other >>> software projects can use it easily. >> I totally second this. However the fmf parser will need significant work >> to make it useful without the pyphant. For more details see the other >> thread entitled "New fmf library interface". >> >> I had written some answers to the individual points. Then I noticed they >> got rather lengthy and were not really related to one another. So I >> decided to split them off to separate threads. Please see the respective >> e-mails. >> >> Cheers, >> Klaus >> > > > > ------------------------------------------------------------------------------ > > > > _______________________________________________ > Pyphant-devel mailing list > Pyp...@li... > https://lists.sourceforge.net/lists/listinfo/pyphant-devel > |
|
From: Dr. A. W. L. <ob...@fm...> - 2010-05-04 04:35:39
|
Dear *,
so let's publish a new Pyphant release. Are there any issues, which have
to be resolved before? What are the next steps?
All the best,
Andreas
Am 02.05.10 14:16, schrieb Klaus Zimmermann:
> Hello all,
>
> first of all thanks for the discussion! It is good to see this list put
> to use.
>
> Before I answer to individual points, two general remarks:
>
> 1) I am absolutely in favor of splitting pyphant and fmf.
> I am however not entirely sure, if this should also be done by
> separating the projects in sourceforge or merely by disentangling the
> code base, possibly into two separate repositories. Anyway I want to
> drop svn and completely switch over to git, which in the mean time is
> also offered by sf. With this, keep in mind that a git repository
> corresponds more closely to a svn module and less to a svn repository.
> This is also why, in contrast to other scm, most notably svn, sf
> supports multiple git repositories under the hood of one project.
>
> 2) No matter how the separation occurs, I would very much like to push
> out one last stable release of the pyphant as it is with all its flaws.
> Than we can keep the svn repo around for bugfixing in the pyphant 1
> series and make a fresh start in a set of git repos for a pyphant 2
> series. However, since Rolf needs some fmf functionality for other
> projects pretty much right now, of course we can start the new repos
> already.
>
> Rolf wrote:
>> Btw: my idea behind the fork of fmf was to give other the
>> possibility to use the fmf without having to install the whole pyphant.
>> And also to give some libs in other languages - so people from other
>> software projects can use it easily.
> I totally second this. However the fmf parser will need significant work
> to make it useful without the pyphant. For more details see the other
> thread entitled "New fmf library interface".
>
> I had written some answers to the individual points. Then I noticed they
> got rather lengthy and were not really related to one another. So I
> decided to split them off to separate threads. Please see the respective
> e-mails.
>
> Cheers,
> Klaus
>
--
===============================================================
Dr. Andreas W. Liehr
Assoziiertes Mitglied
Freiburger Materialforschungszentrum
Stefan-Meier-Str. 21
D-79104 Freiburg i. Br.
Tel.: 0175 6568390
===============================================================
|
|
From: Dr. A. W. L. <ob...@fm...> - 2010-05-04 04:34:49
|
Dear *,
> So what do you think:
> 1) Do we need a specification for aggregate data?
Yes.
> 2) Is zip a good means for that?
Yes.
> 3) Can we get proper inspiration from prior art? What about jar files,
> eggs?
Perhaps. But our starting concept is quite good.
There is another point: What if you want to document a bunch of images
from experiments? We could allow for an index.fmf file within a ZIP
archive, which relates image filenames to the relevant metadata. This
could be the proper way to document Alexanders's image batches.
All the best,
Andreas
|
|
From: Dr. A. W. L. <ob...@fm...> - 2010-05-03 04:22:27
|
Dear *,
great ideas. But remember our experiments with ANTLR. Chopping the file
by means of a grammar extremely slowed down the parsing extremely,
espescially for large data sections. Therefore we had decided to
preparse the FMF files, such that the parsing of [*data]-section is
accomplished by numpys IO-functions and all other sections by a INI-Parser.
All the best,
Andreas
Am 02.05.10 15:02, schrieb Klaus Zimmermann:
> Hello all,
>
>
> Am Freitag, den 30.04.2010, 21:40 +0200 schrieb Rolf Wuerdemann:
>> On Thu, 29 Apr 2010 07:34:09 +0200, "Dr. Andreas W. Liehr"
>> <ob...@fm...> wrote:
>>> There is no doubt, that the quantities module should belong to the
>>> FMF package. But, what about DataContainers? Pyphant as such, does
>>> not play any restrictions on the data objects to be transfered
>>> between the workers, but on the other hand uses DataContainers
>>> heavily for Visualisation and the Knowledge Cloud.
>>>
>>> Of course a FMF module could return a dictionary of dictionary
>>> containing the metadata and a set of numpy arrays, containing the
>>> data columns.
> [...]
>> Klaus also pronounced the very good idea of using an sax-like
>> approach where we generate events wo help people parsing the files.
>> I would suggest to give people both possibilities: "In Memory"
>> (dict type) and "event driven" (sax type).
>
> To formalize my proposal a little more:
> Say someone wanted to read in the simple fmf file
>
> ---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---
> [*reference]
> author: Max Mustermann
> place: Musterstadt
>
> [*data definition]
> wavelength: \lambda [nm]
> absorption: A(\lambda)
>
> [*data]
> 500 .2
> 510 .4
> 520 .6
> 530 .4
> 540 .2
> ---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---
>
> She would give it to the parser. The parser would start forming tokens
> in a stream like model. It would then call (in order)
>
> ---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---
> start_section("*reference")
> item("author", "Max Mustermann")
> item("place", "Musterstadt")
> start_data_definition("")
> column("wavelength", "\lambda", Quantity("1 nm"))
> column("absorption", "A", 1)
> start_data("")
> cell(500)
> cell(.2)
> new_row()
> cell(510)
> cell(.4)
> .
> .
> .
> ---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---
>
> This model, while at first glance perhaps a bit complicated has some
> intriguing features:
> 1) No dependence on any data structure.
> While there are associative arrays in many languages, like dicts in
> python, in equally many languages these are available only through non
> standard libraries or not at all. Moreover arbitrary data structures may
> be used to store the information retrieved by the parser without the
> need for copying of stuff in memory.
> 2) Applicability to many languages.
> While data structures may vary, many languages have the concept of
> procedures and strings.
> 3) I/O Symmetry.
> Want to write a fmf file? Just call the same sequence on the writer!
> This also facilitates easy use as a network streaming protocol.
> 4) Extensibility.
> The functions do not have to be hard coded. Even the names can be
> allowed to very. Just provide the parser with a list of function
> pointers. If you want to add a new item type provide a dispatch
> function. A set of item types could be comfortably managed with the use
> of regular expressions for example.
> 5) Back and forward compatibility.
> If one of the above mentioned mechanisms is exploited to provide an
> extension this would not break compatibility on a lower level. At the
> very least every parser would still be capable of extracting pairs of
> strings, equating to level 0.
>
> How does all of this relate to the entrance question of Data containers
> and stuff? Such a parser, if available in a fmf library could still do
> most of the difficult stuff like converting the various entries to
> appropriate objects in the respective language. It would however have a
> greatly reduced set of dependencies, thus further simplifying its use
> and provide a powerful means for the construction of application
> specific front ends, be it a vba macro to feed the information into an
> excel sheet or an embedded c library to configure a micro controller on
> the fly without ever storing the information transmitted.
>
> So what do you think?
>
> Cheers,
> Klaus
>
>
>
>
> ------------------------------------------------------------------------------
>
>
>
> _______________________________________________
> Pyphant-devel mailing list
> Pyp...@li...
> https://lists.sourceforge.net/lists/listinfo/pyphant-devel
|
|
From: Klaus Z. <kla...@fm...> - 2010-05-02 13:02:44
|
Hello all,
Am Freitag, den 30.04.2010, 21:40 +0200 schrieb Rolf Wuerdemann:
> On Thu, 29 Apr 2010 07:34:09 +0200, "Dr. Andreas W. Liehr"
> <ob...@fm...> wrote:
> > There is no doubt, that the quantities module should belong to the
> > FMF package. But, what about DataContainers? Pyphant as such, does
> > not play any restrictions on the data objects to be transfered
> > between the workers, but on the other hand uses DataContainers
> > heavily for Visualisation and the Knowledge Cloud.
> >
> > Of course a FMF module could return a dictionary of dictionary
> > containing the metadata and a set of numpy arrays, containing the
> > data columns.
[...]
> Klaus also pronounced the very good idea of using an sax-like
> approach where we generate events wo help people parsing the files.
> I would suggest to give people both possibilities: "In Memory"
> (dict type) and "event driven" (sax type).
To formalize my proposal a little more:
Say someone wanted to read in the simple fmf file
---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---
[*reference]
author: Max Mustermann
place: Musterstadt
[*data definition]
wavelength: \lambda [nm]
absorption: A(\lambda)
[*data]
500 .2
510 .4
520 .6
530 .4
540 .2
---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---
She would give it to the parser. The parser would start forming tokens
in a stream like model. It would then call (in order)
---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---
start_section("*reference")
item("author", "Max Mustermann")
item("place", "Musterstadt")
start_data_definition("")
column("wavelength", "\lambda", Quantity("1 nm"))
column("absorption", "A", 1)
start_data("")
cell(500)
cell(.2)
new_row()
cell(510)
cell(.4)
.
.
.
---8<---8<---8<---8<---8<---8<---8<---8<---8<---8<---
This model, while at first glance perhaps a bit complicated has some
intriguing features:
1) No dependence on any data structure.
While there are associative arrays in many languages, like dicts in
python, in equally many languages these are available only through non
standard libraries or not at all. Moreover arbitrary data structures may
be used to store the information retrieved by the parser without the
need for copying of stuff in memory.
2) Applicability to many languages.
While data structures may vary, many languages have the concept of
procedures and strings.
3) I/O Symmetry.
Want to write a fmf file? Just call the same sequence on the writer!
This also facilitates easy use as a network streaming protocol.
4) Extensibility.
The functions do not have to be hard coded. Even the names can be
allowed to very. Just provide the parser with a list of function
pointers. If you want to add a new item type provide a dispatch
function. A set of item types could be comfortably managed with the use
of regular expressions for example.
5) Back and forward compatibility.
If one of the above mentioned mechanisms is exploited to provide an
extension this would not break compatibility on a lower level. At the
very least every parser would still be capable of extracting pairs of
strings, equating to level 0.
How does all of this relate to the entrance question of Data containers
and stuff? Such a parser, if available in a fmf library could still do
most of the difficult stuff like converting the various entries to
appropriate objects in the respective language. It would however have a
greatly reduced set of dependencies, thus further simplifying its use
and provide a powerful means for the construction of application
specific front ends, be it a vba macro to feed the information into an
excel sheet or an embedded c library to configure a micro controller on
the fly without ever storing the information transmitted.
So what do you think?
Cheers,
Klaus
|
|
From: Klaus Z. <kla...@fm...> - 2010-05-02 12:20:06
|
Hello all, first of all thanks for the discussion! It is good to see this list put to use. Before I answer to individual points, two general remarks: 1) I am absolutely in favor of splitting pyphant and fmf. I am however not entirely sure, if this should also be done by separating the projects in sourceforge or merely by disentangling the code base, possibly into two separate repositories. Anyway I want to drop svn and completely switch over to git, which in the mean time is also offered by sf. With this, keep in mind that a git repository corresponds more closely to a svn module and less to a svn repository. This is also why, in contrast to other scm, most notably svn, sf supports multiple git repositories under the hood of one project. 2) No matter how the separation occurs, I would very much like to push out one last stable release of the pyphant as it is with all its flaws. Than we can keep the svn repo around for bugfixing in the pyphant 1 series and make a fresh start in a set of git repos for a pyphant 2 series. However, since Rolf needs some fmf functionality for other projects pretty much right now, of course we can start the new repos already. Rolf wrote: > Btw: my idea behind the fork of fmf was to give other the > possibility to use the fmf without having to install the whole pyphant. > And also to give some libs in other languages - so people from other > software projects can use it easily. I totally second this. However the fmf parser will need significant work to make it useful without the pyphant. For more details see the other thread entitled "New fmf library interface". I had written some answers to the individual points. Then I noticed they got rather lengthy and were not really related to one another. So I decided to split them off to separate threads. Please see the respective e-mails. Cheers, Klaus |