netepi-discuss — Discussion of the NetEpi tools

[NetEpi-discuss] NetEpi v1.5.0 now available

[CHURCHES, T. <tc...@do...>]

A tarball for v1.5.0 of NetEpi Collection is now available from http://code.google.com/p/netepi/

A list of the extensive changes for v1.5.0 is available at http://code.google.com/p/netepi/wiki/CHANGES_v_1_5_0

The README file for v1.5.0 is available at http://code.google.com/p/netepi/wiki/README_v1_5_0

Please note that the v1.5.0 tarball has not yet been uploaded to SourceForge due to technical difficulties with that site.

Updated end-user and administrator manuals for v1.5.0 are in production and news about their release will be posted to this mailing list next week.

Thanks to Andrew McNamara for his hard work on this release.

Regards,

Tim C


______________________________________________________________________
Disclaimer: This message is intended for the addressee named and may contain confidential information. If you are not the intended recipient, please delete it and notify the sender. Views expressed in this message are those of the individual sender, and are not necessarily the views of NSW Health.
______________________________________________________________________
This email has been scanned for the NSW Department of Health by the MessageLabs Email Security System. The Department regularly monitors emails and attachments to ensure compliance with its Electronic Messaging Policy.
_____________________________________________________________________

Re: [NetEpi-discuss] newbie

[CHURCHES, T. <tc...@do...>]

Anthony,

Many thanks. Please feel free to suggest enhancements to the README file or perhaps a more detailed set of installation instructions. Also, in the distribution tarball there is a directory called LiveCD which contains a makeile script to create a bootable NetEpi Ubuntu CD, although for an ancient version of Ubuntu (7.10 I think). We intend to update it, but contained in the script are lots of useful things.

Tim C

From: Anthony Kong [mailto:ant...@gm...]
Sent: Thursday, 10 December 2009 3:08 AM
To: kit sumabat
Cc: Net...@li...
Subject: Re: [NetEpi-discuss] newbie

I am a python developer myself (not of netepi team though), so if it is a general installation issue, I may be able to help.

What problem did you encountered during installation? e.g. pgsql driver or mx?

Cheers


On Thu, Dec 10, 2009 at 3:01 AM, kit sumabat <kit...@gm...<mailto:kit...@gm...>> wrote:
Hi Tim,

Hope everything is fine. We finally got an Ubuntu desktop running on
Karmic. We're having difficulty with the NetEpi installation. Hope you
can give us a hand. We'll be happy to "pay-it-forward" by making a
screencast of the installation and uploading it online.

Thanks in advance.

Kit


________________
This email has been scanned for the NSW Department of Health by the MessageLabs Email Security System. The Department regularly monitors emails and attachments to ensure compliance with its Electronic Messaging Policy.
_____________________________________________________________________

______________________________________________________________________
Disclaimer: This message is intended for the addressee named and may contain confidential information. If you are not the intended recipient, please delete it and notify the sender. Views expressed in this message are those of the individual sender, and are not necessarily the views of NSW Health.
______________________________________________________________________
This email has been scanned for the NSW Department of Health by the MessageLabs Email Security System. The Department regularly monitors emails and attachments to ensure compliance with its Electronic Messaging Policy.
_____________________________________________________________________

Re: [NetEpi-discuss] newbie

[CHURCHES, T. <tc...@do...>]

Kit,

Yes, we are more than happy to assist. Note that we plan to release v1.5.0 next Tuesday, 15 Dec, and you should definitely install that version, which has a lot of improvements based on our experience in using NetEpi for the Human swine flu (Influenza A H1N1 09) response during our winter here in NSW earlier this year. Version 1.6 will be out in late Feb or early March 2010 with further improvements. A screencast of your experience would be great - it can go up on the NetEpi site at Google Code.

First step: what sort of help would you like? Have you encountered a specific problem or do you need general help? Is email OK? We could use Google Chat messaging, or Skype, but let's try email and chat first.

I'm a bit busy today but Andrew McNamara may be able to answer specific questions. Tomorrow I have more time to assist. I also have a newly installed Karmic on a laptop, so I can go through the installation step-by-step on that.
 
Regards,

Tim C

> -----Original Message-----
> From: kit sumabat [mailto:kit...@gm...]
> Sent: Thursday, 10 December 2009 3:02 AM
> To: net...@li...
> Subject: Re: [NetEpi-discuss] newbie
> 
> Hi Tim,
> 
> Hope everything is fine. We finally got an Ubuntu desktop running on
> Karmic. We're having difficulty with the NetEpi installation. Hope you
> can give us a hand. We'll be happy to "pay-it-forward" by making a
> screencast of the installation and uploading it online.
> 
> Thanks in advance.
> 
> Kit
> 
> On Wed, Jul 29, 2009 at 9:37 PM, kit sumabat
> <kit...@gm...> wrote:
> > Hi!
> > I'm a newbie here. I want to have some idea on how to maximize
> NetEpi. Can
> > anyone help me with the installation?
> > Thank you.
> >
> > --
> > Kit Sumabat, RN, DNE
> > Health Informatics Specialist
> > Diabetes Nurse Educator
> >
> 
> 
> 
> --
> Kit Sumabat, RN, DNE, MSHI (cand)
> Health Informatics Specialist
> Diabetes Nurse Educator
> 
> Master of Science in Health Informatics
> University of the Philippine-College of Medicine
> 
> Certified Diabetes Nurse Educator
> Philippine Association of Diabetes Educators
> 
> Registered Linux User #: 499522
> Since Nov 2009
> 
> -----------------------------------------------------------------------
> -------
> Return on Information:
> Google Enterprise Search pays you back
> Get the facts.
> http://p.sf.net/sfu/google-dev2dev
> _______________________________________________
> NetEpi-discuss mailing list
> Net...@li...
> https://lists.sourceforge.net/lists/listinfo/netepi-discuss
> 
> ________________
> This email has been scanned for the NSW Department of Health by the
> MessageLabs Email Security System. The Department regularly monitors
> emails and attachments to ensure compliance with its Electronic
> Messaging Policy.
> _____________________________________________________________________

______________________________________________________________________
Disclaimer: This message is intended for the addressee named and may contain confidential information. If you are not the intended recipient, please delete it and notify the sender. Views expressed in this message are those of the individual sender, and are not necessarily the views of NSW Health.
______________________________________________________________________
This email has been scanned for the NSW Department of Health by the MessageLabs Email Security System. The Department regularly monitors emails and attachments to ensure compliance with its Electronic Messaging Policy.
_____________________________________________________________________

Re: [NetEpi-discuss] newbie

[Anthony K. <ant...@gm...>]

I am a python developer myself (not of netepi team though), so if it is a
general installation issue, I may be able to help.

What problem did you encountered during installation? e.g. pgsql driver or
mx?

Cheers



On Thu, Dec 10, 2009 at 3:01 AM, kit sumabat <
kit...@gm...> wrote:

> Hi Tim,
>
> Hope everything is fine. We finally got an Ubuntu desktop running on
> Karmic. We're having difficulty with the NetEpi installation. Hope you
> can give us a hand. We'll be happy to "pay-it-forward" by making a
> screencast of the installation and uploading it online.
>
> Thanks in advance.
>
> Kit
>
>

Re: [NetEpi-discuss] newbie

[kit s. <kit...@gm...>]

Hi Tim,

Hope everything is fine. We finally got an Ubuntu desktop running on
Karmic. We're having difficulty with the NetEpi installation. Hope you
can give us a hand. We'll be happy to "pay-it-forward" by making a
screencast of the installation and uploading it online.

Thanks in advance.

Kit

On Wed, Jul 29, 2009 at 9:37 PM, kit sumabat
<kit...@gm...> wrote:
> Hi!
> I'm a newbie here. I want to have some idea on how to maximize NetEpi. Can
> anyone help me with the installation?
> Thank you.
>
> --
> Kit Sumabat, RN, DNE
> Health Informatics Specialist
> Diabetes Nurse Educator
>



-- 
Kit Sumabat, RN, DNE, MSHI (cand)
Health Informatics Specialist
Diabetes Nurse Educator

Master of Science in Health Informatics
University of the Philippine-College of Medicine

Certified Diabetes Nurse Educator
Philippine Association of Diabetes Educators

Registered Linux User #: 499522
Since Nov 2009

Re: [NetEpi-discuss] newbie

[CHURCHES, T. <tc...@do...>]

Hi Kit,

Sorry for the delay in replying - we are still a bit busy with the response to swine flu here in Australia (NetEpi is being used extensively and successfully for this). We can respond in a bit more detail over the forthcoming weekend, but as a start, could you describe the setting you are working in, what you want to use NetEpi for, and what sort of Web server(s) you have access to.

Regards,

Tim C

From: kit sumabat [mailto:kit...@gm...]
Sent: Wednesday, 29 July 2009 11:38 PM
To: net...@li...
Subject: [NetEpi-discuss] newbie

Hi!

I'm a newbie here. I want to have some idea on how to maximize NetEpi. Can anyone help me with the installation?

Thank you.

--
Kit Sumabat, RN, DNE
Health Informatics Specialist
Diabetes Nurse Educator

________________
This email has been scanned for the NSW Department of Health by the MessageLabs Email Security System. The Department regularly monitors emails and attachments to ensure compliance with its Electronic Messaging Policy.
_____________________________________________________________________

______________________________________________________________________
Disclaimer: This message is intended for the addressee named and may contain confidential information. If you are not the intended recipient, please delete it and notify the sender. Views expressed in this message are those of the individual sender, and are not necessarily the views of NSW Health.
______________________________________________________________________
This email has been scanned for the NSW Department of Health by the MessageLabs Email Security System. The Department regularly monitors emails and attachments to ensure compliance with its Electronic Messaging Policy.
_____________________________________________________________________

[NetEpi-discuss] newbie

[kit s. <kit...@gm...>]

Hi!
I'm a newbie here. I want to have some idea on how to maximize NetEpi. Can
anyone help me with the installation?

Thank you.

-- 
Kit Sumabat, RN, DNE
Health Informatics Specialist
Diabetes Nurse Educator

Re: [NetEpi-discuss] Installation Problem of NetEpi Analysis

[Andrew M. <an...@ob...>]

>When I run python test.py, it displayed as following:
>
>[root@pc-4e13-0 NetEpi-Analysis]# python test.py
>Traceback (most recent call last):
>  File "test.py", line 55, in <module>
>    test_suite = unittest.defaultTestLoader.loadTestsFromNames(tests)
>  File "/usr/lib/python2.5/unittest.py", line 565, in loadTestsFromNames
>    suites = [self.loadTestsFromName(name, module) for name in names]
>  File "/usr/lib/python2.5/unittest.py", line 533, in loadTestsFromName
>    module = __import__('.'.join(parts_copy))
>  File "tests/indirect_std_SAS.py", line 26, in <module>
>    from SOOMv0.Sources.CSV import CSVDataSource
>ImportError: No module named CSV
>
>After check the \NetEpi-Analysis\SOOMv0\Sources direction, however, there is
>a CSV.py in this direction.
>
>Is there any suggestion for this?

No, that's a very odd problem and I'm not sure what to suggest. Clearly
something is going awry in the python import machinery. The only thing I
can suggest is repeating the steps by hand - importing SOOMv0.Sources,
printing it's __file__ attribute, and so on. For example, this is what
I get:

    $ python
    Python 2.5.2 (r252:60911, Nov 14 2008, 19:46:32) 
    [GCC 4.3.2] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import SOOMv0.Sources
    >>> print SOOMv0.Sources.__file__
    SOOMv0/Sources/__init__.pyc
    >>> from SOOMv0.Sources import CSV
    >>> print CSV.__file__
    SOOMv0/Sources/CSV.pyc
    >>>

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

[NetEpi-discuss] Installation Problem of NetEpi Analysis

[Min L. <inu...@gm...>]

Hi, everyone.

I have a installation trouble with NetEpi Analysis under RedHat linux.
According to the readme file in the NetEpi Analysis, there are several steps
to run NetEpi Analysis, such as Prerequisites, Installation, Installing the
Web interface, etc.

In the Prerequisites stage, I have installed all items. Howverer, in the
next stage (installation stage), I suffered an error in Unit tests section.
When I run python test.py, it displayed as following:

[root@pc-4e13-0 NetEpi-Analysis]# python test.py
Traceback (most recent call last):
  File "test.py", line 55, in <module>
    test_suite = unittest.defaultTestLoader.loadTestsFromNames(tests)
  File "/usr/lib/python2.5/unittest.py", line 565, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python2.5/unittest.py", line 533, in loadTestsFromName
    module = __import__('.'.join(parts_copy))
  File "tests/indirect_std_SAS.py", line 26, in <module>
    from SOOMv0.Sources.CSV import CSVDataSource
ImportError: No module named CSV

After check the \NetEpi-Analysis\SOOMv0\Sources direction, however, there is
a CSV.py in this direction.

Is there any suggestion for this?


thanks a lot

Lee

[NetEpi-discuss] NetEpi Collection Version 1.3.0 released

[Tim C. <tc...@op...>]

Version 1.3.0 of NetEpi Collection (formerly known as NetEpi Case
Manager) is now available from SourceForge at
http://sourceforge.net/projects/netepi/
or from Google Code at http://code.google.com/p/netepi/

Following is a list of cumulative changes since v1.2.0.

A roadmap for future versions, which may be subject to change,
also appears below. We expect to release Version 1.4.0 in June 2008.

Regards,

Tim Churches and Andrew McNamara
22nd May 2008

SUMMARY OF CHANGES IN VERSION 1.3.0 (LATEST) RELATIVE TO VERSION 1.2.0
======================================================================

The following changes have been made in this version:

* The ability to import demographic data from CSV or other delimited or
   column-formatted data files has been enhanced and several bugs in the
   previous version of this facility have been fixed. A name can now be
   assigned to each source of data to be imported, and if the "Local ID"
   field is assigned, this is assumed to be a unique key which can be
   used to update records that have already been imported. This means that
   updated versions of an imported demographic data can be re-imported and
   the changes in that updated file will be applied to existing records,
   rather than a duplicate set of records being created. Demographic data
   imported from external sources is marked as read-only, although users can
   "take ownership" of such records if they need to make changes in NetEpi
   (and such changes will not be over-written by future data imports from that
   data source). The use-case (motivation) for this enhancement is the need to
   be able to repeatedly import batches of demographic data from sources such
   as aircraft passenger manifest files, or from Disaster Victim Registration
   (DVR) databases, which are often operated by Police or other agencies in
   disaster and emergency situations. Note that deletion of imported records
   which have disappeared from subsequent import data files does not occur
   (this may be added in a future version).

* The batch duplicate person/case/contact detection facility has been made
   more robust and now runs on demand "in the background", rather than
   interactively, and the results of the last batch run of the duplicate
   detector are displayed. This means that the duplicate detection system is
   now able to deal with tens of thousands of records if necessary.

* Installation options were added to allow customised login page and banner
   logos to be used (see the "install_logo" and "install_logo_small" options
   described later in this file).

* The installer is now smarter and doesn't reset previously specified options
   (with a few exceptions - see the installation notes later in this file).
   This makes the installation of upgrades and bug-fixes easier and less error-
   prone.

* The administrator interface for configuring demographic fields has been
   improved, making it much quicker to use.

* The tabbed interface to demographic fields can be disabled by the system
   administrator if required - for example, if only a small number of
   demographic fields have been enabled (see the "tabbed_demogfields_threshold"
   option described later in this file).

* A few minor and obscure bugs were fixed, including a problem with the
   pop-up calendar when used with Microsoft Internet Explorer 6 and 7 web
   browsers.

* A utility has been added to create scatterplot graphs of the RTT (round-trip
   time) performance data collected by NetEpi as it operates. These data give
   an indication of the response times of individual end-users of a NetEpi
   installation. This allows problems with their Internet or other network
   access to the NetEpi web site to be monitored and addressed pro-actively.
   The script which extracts the RTT data from the web server logs is now
   smarter and can filter the data in various ways. Further details are given
   later in this file under the heading "Client interactivity reporting". Some
   example scatterplots are included in the tools/ directory of the
   NetEpi-Collection distribution tarball.

* An example of a separate, simple "labsurv" web application designed to
   collect weekly, aggregate surveillance data on the numbers and results of
   tests for common respiratory viruses has been included under the labsurv/
   directory of the NetEpi-Collection distribution tarball. Currently, NetEpi
   Collection is designed to collect unit-record data about persons (cases or
   contacts of a disease or syndrome), and although it can be (ab)used to
   collect aggregate data about "things", it is far from optimal for such
   purposes. The simple "labsurv" application, which uses the same infra-
   structure as NetEpi Collection (i.e. Python, the Albatross web framework,
   PostgreSQL) has been provided as both a template for similar use-cases else-
   where, and as a test-bed for features and design approaches to be integrated
   into future versions of NetEpi Collection. It should be possible for a
   programmer familiar with Python to customise the labsurv application for
   similar purposes in just a few days. In the future, we hope to provide the
   ability for NetEpi administrators to be able to  set up similar aggregate
   and/or "thing"-related (as opposed to person-related) data collection
   facilities with the usual NetEpi point-and-click interface.

ROADMAP FOR THE NEXT FEW VERSIONS
=================================

The following information may be subject to change, but at the time of
release of version 1.3.0, the following features were planned for the next
few versions, which we hope to release in the months preceding July 2008:

* v1.4.x: further improvements to reporting and analysis capabilities,
   including integration of NetEpi Analysis.

* v1.5.x: integration of Australian-specific and international geocoding
   facilities, and display of data on maps; user database and passwords
   shared between NetEpi instances on the same web server or cluster of web
   servers.

* v1.6.x: further enhancements to the demographic data import facility;
   addition of support for authorisation, tracking and reporting on lab
   specimens.

[NetEpi-discuss] NetEpi Collection Version 1.2.0 released

[Tim C. <tc...@op...>]

Version 1.2.0 of NetEpi Collection (formerly known as NetEpi Case
Manager) is now available from SourceForge at
http://sourceforge.net/projects/netepi/
or from Google Code at http://code.google.com/p/netepi/

Following is a list of cumulative changes since v1.1.0.

Versions 1.3.0, 1.4.0 and perhaps 1.5.0 will be released in May and June 
2008.

Regards,

Tim Churches and Andrew McNamara
5th April 2008

SUMMARY OF CHANGES IN VERSION 1.2.0 (LATEST) RELATIVE TO VERSION 1.1.0
======================================================================

The following changes have been made in this version:

* The ability to import demographic data from CSV or other delimited or
   column-formatted data files has been added. A "point-and-click"
   interface to "map" columns in the file to be imported to target
   demographic fields in NetEpi Collection is provided, and a range of
   transformations of data values can also be specified. Transformed data
   can be previewed before it is imported. The parameters used to import
   a particular data file can be saved and re-used on subsequent
   occasions.

* Several new fields were added to the core "demographics" fields:

   - a third set of address fields, so there are now address fields
     for residential address, an alternate address, and a work or
     school address (although the field labels on all of these can
     be customised as previously, so they can be used for other types
     of addresses). [1]

   - a Country field has been added to each of the three sets of address
     fields.

   - an Occupation field was added, grouped with the Work/School address
     fields. [2]

   - a second set of Passport Number and Passport Country fields were
     added to accommodate people with dual nationality. The default
     label on the Passport Country fields was changed to "Passport
     country/Nationality".

   - fields to capture the name and contact details of the notifier of a
     case or contact were added (these details are associated with the
     case or contact and are thus stored on the underlying CASES table)

   - an "Other Information" text box field was added to record other,
     miscellaneous information about each person.

   As previously, all of these fields and the other demographics fields
   are configurable, and can be hidden or disable or set to appear in
   only certain parts of the application.

* In order to better accommodate the increased number of demographics
   core fields, a "tabbed" display for some of these fields has
   been provided, which helps to avoid the need for too much vertical
   scrolling. The tabbed field display is also provided on the search
   parameter pages.

* Two new data collection "input" types were added: a pull-down list
   input for "Countries", which lists all current countries, and an
   equivalent one for "Languages", which lists major languages.

* Disabled input boxes are no longer used for the "read-only" display of
   demographic data in certain places in the application; instead, an
   easier-to-read and more compact format is used.

* Searching for existing cases and contacts has been improved by
   providing given name and nickname substitution when the "Phonetic" box
   is checked: for example, a search for "Tim" will also find "Timothy"
   and a search for "Margaret" will find "Peggy" and so on.

* The duplicate person matching algorithm has been improved (it now uses
   trigrams instead of bigrams) and has been made faster.

* Case and contact details are now colour-coded on the search results
   pages in order to make visual scanning and parsing of the results
   easier.

* "<<Back" buttons have been added to the top (in addition to the foot)
   of most pages.

* The display of available contact definitions on the case contacts page
   has been made consistent with the way in which case and contact
   definitions are displayed on the home page.

* The page is now scrolled down to any "Unsaved changes" warning
   messages to avoid any confusion on long pages.

* An "login_helpdesk_contact" configuration option has been added. The
   option specifies an contact message that is used on the login and
   new-user registration pages. This allows an alternate message to be
   specified in these (potentially) less secure contexts.

* Several other minor cosmetic and layout improvements have been made.

* Several bug fixes and some refactoring to improve the robustness and
   maintainability of the program code.

[1] We know that an unlimited number of addresses, including a complete
address history, should ideally be stored in a separate ADDRESS table,
but that requires fairly major refactoring of core parts of the
application and will have to wait until version 2. The current 
arrangements seem to be adequate in the meantime (and changes to 
addresses are captured in the audit logs, although we understand that 
that is not a substitute for a full address history).

[2] The Occupation column was added to the PERSONS table, for reasons
of expedience. Ideally, it should be on the CASES table and reside
with other CASE data (as should some addresses and other data current 
stored in the PERSONS table). These structural changes are planned for 
version 2, but in the meantime, the current arrangements are unlikely to 
cause major problems with most acute outbreak investigations or other 
relatively short-term epidemiological data collections, although we 
recognise that the current data model is not ideal for use by, say, a 
chronic disease registry.

ROADMAP FOR THE NEXT FEW VERSIONS
=================================

The following information may be subject to change, but at the time of 
release of version 1.2.0, the following features were planned for the 
next few versions, which we hope to release in the months preceding July 
2008:

* v1.3.x: user database and passwords shared between NetEpi instances on
   the same web server or cluster of web servers; further improvements to
   reporting and analysis capabilities, including integration of NetEpi
   Analysis.

* v1.4.x: further enhancements to the demographic data import facility,
   including the ability to import updates to already-imported records;
   possible addition of support for authorisation and tracking of lab
   specimens.

* v1.5.x: integration of Australian-specific and international geocoding
   facilities, and display of data on maps.

[NetEpi-discuss] NetEpi Collection Version 1.1.0 released

[Tim C. <tc...@op...>]

Version 1.1.0 of NetEpi Collection (formerly known as NetEpi Case
Manager) is now available from SourceForge at
http://sourceforge.net/projects/netepi/
or from Google Code at http://code.google.com/p/netepi/

The last announced release as version 1.0.2. Version 1.0.3 was never 
released, and version 1.0.4 was released but never announced. We then 
changed the release numbering scheme, so that version 1.1.0 is in fact 
the next version after v1.0.4.

Following is a list of cumulative changes since v1.0.2.

Version 1.2.0 is due to be released in mid March 2008, followed by 
versions 1.3.0 and 1.4.0 in May and June 2008 respectively.

Regards,

Tim Churches and Andrew McNamara
26th February, 2008

SUMMARY OF CHANGES IN VERSION 1.1.0 RELATIVE TO VERSION 1.0.4
===============================================================

* The login page has been re-styled to be rather more attractive.

* The previously separate administration application has been merged
   into the main end-user application. Now there is only one URL for each
   NetEpi  Collection instance, and if a logged-in user has administrator
   rights, then an additional menu item appears which allows them to
   access the administration functions that were previously accessible
   only from a separate admin URL. Apart from making things easier for
   administrators, this change has enabled a reduction in otherwise
   duplicated code - thus the total line count for the project has gone
   down substantially.

* An internal "notification" mechanism has been added so that changes
   made in the administration interface are immediately reflected in all
   sessions for that NetEpi instance. Previously, due to caching,
   administrative changes took up to ten minutes to propagate through to
   all user sessions, which could be confusing for users and
   administrators alike. See also the notes in the README file on
   "Stand-alone Notification Daemon" regarding the need to disable this
   facility in certain unusual server configurations.

* Administrators can now specify the order in which
   syndromes/case/contact definitions are listed on the home page.

* Each instance of a data collection form is now assigned a unique ID,
   which incorporates a check-digit, and this ID number is shown on
   screen and appears in print-outs of forms and in data export files.

* Users can specify a form instance ID number in the search page, and if
   it is a valid form ID number and the user has permission to access
   that case or contact, the form editing page for that form instance
   (and thus for the correct person and case or contact record) will be
   displayed immediately.

* A simple "epi curve" charting facility has been added to the built-in
   reports. An "epi curve" is a frequency histogram by onset date
   (reporting date can also be be chosen, or dual charts for both can be
   drawn). Various date and time "binning" options are provided to
   control the degree of temporal aggregation on the x-axis. Further
   features will be added to this chart in later releases, and other
   built-in charts and graphs will be added.  See also the additional
   installation prerequisites listed in the README file which relate to
   this new feature.

* The search parameters which a user enters are now shown on all search
   results pages. This is particularly important when adding a new case
   or contact, because these search parameters provide context to the
   "Create a new case" or "Create a new contact" buttons.

* Users with "task queue admin" rights can now set up task queues
   specific to their unit (or shared with other units or specific users)
   without needing to ask a central administrator to do it for them.

* Administrators can see summary statistics about the tasks allocated to
   each task queue.

* The refresh button on the end-user task listing page has been removed
   (provided that Javascript is available and enabled in the user's
   browser).

* Task filtering and display parameters for each user are automatically
   restored and saved each time a user visits the task list page.

* Built-in reporting parameters for each user are automatically restored
   and saved, on a per case/contact definition basis, each time a user
   visits the built-in reports facility.

* User accounts are now only logically deleted, thus preserving entries
   in the audit trail relating to deleted users. User accounts can also
   be easily undeleted. The enabled/disabled user flag remains,
   unchanged, so that user accounts can still be temporally disabled if
   necessary, and self-registered user accounts are still created with
   the disabled flag set and need to be deliberately enabled by an
   administrator.

* Additional checks are now made before a unit can be deleted, and units
   with enabled users assigned to them can't be deleted. Previously, if
   units with active users were deleted, the users could be left with no
   unit membership, which prevented them from logging in.

* Deleting work queues is now possible. Database integrity constraints
   previously prevented this except in the case of work queues that had
   never had any tasks associated with them. Work queues can now be
   deleted (after confirmation), provided that all associated tasks have
   been completed.

* Previously, there were a number of places in the application in which
   administrators could accidentally exit an admin page without being
   warned they had unsaved changes. This has now been addressed.

* More useful configuration and debug information is included in
   tracebacks if an application exception occurs.

* Authenticated user id, unit id and the NetEpi instance name are now
   included in the RTT (round-trip-time) "perceived performance" data
   written to the web server log for each user interaction. This makes
   identification of network performance issues affecting individual
   users or groups of users at a particular location much easier.

* After much experimentation, problems with unexpected behavior when
   using the browser back buttons on the Opera and Konqueror web browsers
   have now been addressed. The browser back button now behaves as
   expected (that is, the same way that it behaves when Gecko-based
   browsers such as Mozilla and Firefox, or Microsoft Internet Explorer
   version 6 or 7 are used). In the Safari browser, the  browser back
   button is now effectively disabled when accessing NetEpi. This is an
   improvement on the unexpected behaviour of the browser back button in
   Safari browser when accessing previous versions of NetEpi. It should
   be noted that many web applications have major difficulties
   with the non-standard behaviour of the browser back button in the
   Safari browser (and possibly in the related WebKit browser engine). Of
   course, Safari users can still use the "<<Back" buttons provided in
   the NetEpi application: these work exactly as expected in Safari and
   in all other Web browsers.

* Numerous minor bug fixes and other minor cosmetic adjustments have
   been made.

* Updates to technical and system documentation.

SUMMARY OF CHANGES IN VERSION 1.0.4 RELATIVE TO VERSION 1.0.3
===============================================================

Version 1.0.4 provides the ability to set the preferred date input and
output format on an instance-wide basis, and/or by individual users. 
Three date formats can be specified: DMY (DD/MM/YYYY format), MDY 
(MM/DD/YYYY format) or ISO (YYYY-MM-DD format). All dates and date/time
values are stored in the database in a format-neutral manner, thus there
is no problem if different users of the same NetEpi database instance 
use different preferences for the date format, and individual users can
change their preferred date format on-the-fly without problems.

This feature, together with customisable state/territory/region fields
introduced in Version 1.0.3 and the ability to customise the labels of
the demographic fields added to Version 0.94, should allow NetEpi
Collection to be more easily adapted for use in English-speaking locales 
outside Australia. A fully internationalised version accommodating 
non-English locales is possible but collaborators are required - the 
current NetEpi team cannot create such a fully-internationalised version 
without assistance, although they would very much like to do so.

Additionally, Version 1.0.4 fixes several minor bugs - a list of
fixes can be found in the doc/CHANGELOG file. Support for a new
Python-to-PostgreSQL adaptor being developed by Object Craft Pty Ltd
(http://www.object-craft.com.au) has been added. The free, open-source
Object Craft adaptor is still be tested but when released, NetEpi
Collection installations running Version 1.0.4 or later will 
automatically make use of it if it is installed.  The release of the 
Object Craft Python-to-PostgreSQL adaptor, which promises to be 
considerably faster than existing Python-to-PostgreSQL adaptors, will be 
announced on the netepi-discuss mailing list as well as elsewhere.

Finally, a PDF version of a presentation on NetEpi Collection and
NetEpi Analysis, given at the School of Information Technologies
at the University of Sydney in June 2007 has been included in
documentation section of the NetEpi page on SourceForge and Google Code 
- see http://sourceforge.net/docman/?group_id=123700 or 
http://code.google.com/p/netepi/downloads/list

SUMMARY OF CHANGES IN VERSION 1.0.3 RELATIVE TO VERSION 1.0.2
===============================================================

Version 1.0.3 provides the ability to customise the list of geographical
states, territories and regions in the address part of the demographics
section.  The global list of states and territories for an entire 
instance of NetEpi Collection can be customised, and this can be further 
customised on a per-syndrome or per-case-definition basis if required. 
The customised lists of states and territories are automatically taken 
into account when performing general searches.

Additionally, Version 1.0.3 fixes a small number of minor but annoying
bugs - a list of fixes can be found in the doc/CHANGELOG file. Special
thanks to Stefan Stirzaker, Office of Health Protection, Surveillance
Branch, Surveillance Policy & Systems Section, Population Health 
Division, Australian Government Department of Health and Ageing, for 
reporting the bugs in question.

Re: [NetEpi-discuss] documentation and a few questions

[Christophe C. <cc...@go...>]

Tim Churches a écrit :
> 
> OK, I'll start to investigate Zope 3 for use as a portal.
> 

Good idea, you will be welcome on the zope3-users list
and on the #zope3-dev channel!

> 
> OK. Dave Cole of Object Craft wrote the python-sybase adaptor and also
> wrote the set functions C code used in SOOM...

it seems the world is small :)

> 
> Time between events tends to obey a triangular or hyper-geometric
> distribution. Statistics based on normal distributions and the central
> limit theorem may not be of much use. Look at the SPC (statistical
> process control) literature for clues on how to handle time-between-events.

Ok, after several hours upgrading on the subject, I understand that I need to 
draw inverse sampling control charts for rare events, and classical ones for 
frequent events (those which happen everyday). I've seen there is already some R 
functions to do g-type charts. I need to investigate more.

thanks,
Christophe

Re: [NetEpi-discuss] documentation and a few questions

[Tim C. <tc...@op...>]

Christophe Combelles wrote:
> Thanks for your answers, all this will help me a lot
> 
> Tim Churches a écrit :
>> There is no built-in security model or access control model in NetEpi
>> Analysis - you can restrict access to each instance of it, via apache
>> etc, but not more fine-grained than that. In theory adding access
>> control at the dataset and/or column level would be quite possible, but
>> we have not done that yet. Row-level access control would not be
>> feasible.
> 
> Ok I'll try to manage permissions inside zope, by creating persistent objects 
> that represents datasets, and attributes for columns or such
> 
>>         
>>> I'm ok on the web part of the work, I already know Zope3 to manage 
>>> authentication, templating, object database, etc.
>> It is good to know that Zope is still a popular choice in France. We are
>> looking at Plone as a portal for NetEpi Collection and NetEpi Analysis
>> instances.
>>
>> Is Zope3 ready for production use? I thought it was still being
>> developed.
> 
> Actually it's already used for production, although there is few advertisement. 
> The schooltool project uses it, as well as the Launchpad from Canonical/ubuntu, 
> some sites like http://www.lovelybooks.com or http://kelpi.com, and my last 
> project http://eztranet.gorfou.fr
> It's a great framework, very well designed, all made of interfaces and adapters 
> and much more pluggable and flexible than zope2 and plone. There are many 
> packages for it, but they are often hidden in svn repositories. And and there is 
> an excellent book to learn it http://worldcookery.com that details everything 
> from concepts to deployment in production.

OK, I'll start to investigate Zope 3 for use as a portal.

>> Have a look at PyTables as well.
> 
> thanks for this, there is a full doc, but there is no stats or graphing 
> included. But that could be an option.
> 
>>> I've seen that the demo load some cpickled data with dsload, but how
>>> should I manually fill the dataset, is it a standard python mapping? 
>>> dataset['column'][??] = ??
>> Can you describe what your source data looks like, and where it is
>> coming from? There are standard loaders for data files in CSV or other
>> delimited form, in fixed-width columnar format, in XML and for SQL
>> databases. Are your source data in any of these forms? Or you can load
>> data directly into columns - there is an .addcolumnfromseq() method for
>> dataset objects - have a look at demo/loaders/whopop.py for an example.
>> Others of the demo loading scripts illustrate how to create derived
>> columns as well. If you need an example of loading from an SQL database,
>> let us know and we can send one to you (it is not included in the
>> current distribution tarball).
> 
> this is an old windows-based Sybase database I can access with sybase-python and 
> freetds from linux. It is large but not huge (just below 1M rows)

OK. Dave Cole of Object Craft wrote the python-sybase adaptor and also
wrote the set functions C code used in SOOM...

>>> Would you have any kind of advice, to allow me to finish on time?
>> Can you describe the scope of your project in a bit more detail?
>> Off-list if you prefer.
> 
> I must retrieve several fields on a daily basis, which include the date of the 
> call, the date of intervention, the date of end, the symptom, diagnosis, 
> geographic area, sex and so on.
> The goal is to draw two types of graph:
> - the number of events (to choose) by time unit (and its N-day mean)
> - and statistics based on the standard deviation of the time between events (not 
> sure what to use)

Time between events tends to obey a triangular or hyper-geometric
distribution. Statistics based on normal distributions and the central
limit theorem may not be of much use. Look at the SPC (statistical
process control) literature for clues on how to handle time-between-events.

> Are there already methods for this kind of calculations?
> 
> Then, I must allow to define an alert when there is an abnormal change in one of 
> the graph.

We've spent over a year working out how to do this, and simulating it.
It is a difficult problem. There is now quite a large literature on
signal detection in disease surveillance data, but there is no consensus
on a single right way to do it - it also depends a lot on what diseases
are being monitored and in how large a population.

> So I must query the database everyday, and append new data to a dataset. I plan 
> to use a single sql query, and the dataset columns must be based on the result 
> fields.
> What is the method you're speaking about for sql query in netepi? How does it 
> support appending data?

No, you can use an SQL query to extract data from a database for loading
into NetEpi.

> I'm currently making it remember both the "last fetched primary key" of the 
> source table, and the "modification date" field to add a limiter to the query 
> and not fetch everything every day.

NetEpi doesn't currently support appending - you need to reload the
entire dataset. It does support dataset versioning so you can reload a
new dataset in the background. We do plan to support appending and
updates but it is not a high priority. Loading a million rows of a few
columns should not take very long, so you could easily just reload the
entire dataset each day.

> If you create a derived column and then you append new data, how do you 
> calculate the derived fields of the new data?

See above.

> Then I should probably create one dataset for each type of graph I have to draw,
> and chain updates between datasets, unless the graphs can be drawn very quickly 
> on demand.

Should not take more than a few seconds to summarise the data and draw
each graph.

If you only have a million or so rows and a few columns of data, then
you could just do everything in R and hold all your data in memory. See
the RSOAP and Rserver projects and use them via RPy.

Tim C

Re: [NetEpi-discuss] documentation and a few questions

[Christophe C. <cc...@go...>]

Thanks for your answers, all this will help me a lot

Tim Churches a écrit :
> There is no built-in security model or access control model in NetEpi
> Analysis - you can restrict access to each instance of it, via apache
> etc, but not more fine-grained than that. In theory adding access
> control at the dataset and/or column level would be quite possible, but
> we have not done that yet. Row-level access control would not be
> feasible.

Ok I'll try to manage permissions inside zope, by creating persistent objects 
that represents datasets, and attributes for columns or such

>         
>> I'm ok on the web part of the work, I already know Zope3 to manage 
>> authentication, templating, object database, etc.
> 
> It is good to know that Zope is still a popular choice in France. We are
> looking at Plone as a portal for NetEpi Collection and NetEpi Analysis
> instances.
> 
> Is Zope3 ready for production use? I thought it was still being
> developed.

Actually it's already used for production, although there is few advertisement. 
The schooltool project uses it, as well as the Launchpad from Canonical/ubuntu, 
some sites like http://www.lovelybooks.com or http://kelpi.com, and my last 
project http://eztranet.gorfou.fr
It's a great framework, very well designed, all made of interfaces and adapters 
and much more pluggable and flexible than zope2 and plone. There are many 
packages for it, but they are often hidden in svn repositories. And and there is 
an excellent book to learn it http://worldcookery.com that details everything 
from concepts to deployment in production.


> 
> Have a look at PyTables as well.

thanks for this, there is a full doc, but there is no stats or graphing 
included. But that could be an option.

>> I've seen that the demo load some cpickled data with dsload, but how
>> should I manually fill the dataset, is it a standard python mapping? 
>> dataset['column'][??] = ??
> 
> Can you describe what your source data looks like, and where it is
> coming from? There are standard loaders for data files in CSV or other
> delimited form, in fixed-width columnar format, in XML and for SQL
> databases. Are your source data in any of these forms? Or you can load
> data directly into columns - there is an .addcolumnfromseq() method for
> dataset objects - have a look at demo/loaders/whopop.py for an example.
> Others of the demo loading scripts illustrate how to create derived
> columns as well. If you need an example of loading from an SQL database,
> let us know and we can send one to you (it is not included in the
> current distribution tarball).

this is an old windows-based Sybase database I can access with sybase-python and 
freetds from linux. It is large but not huge (just below 1M rows)

>> Would you have any kind of advice, to allow me to finish on time?
> 
> Can you describe the scope of your project in a bit more detail?
> Off-list if you prefer.

I must retrieve several fields on a daily basis, which include the date of the 
call, the date of intervention, the date of end, the symptom, diagnosis, 
geographic area, sex and so on.
The goal is to draw two types of graph:
- the number of events (to choose) by time unit (and its N-day mean)
- and statistics based on the standard deviation of the time between events (not 
sure what to use)

Are there already methods for this kind of calculations?

Then, I must allow to define an alert when there is an abnormal change in one of 
the graph.

So I must query the database everyday, and append new data to a dataset. I plan 
to use a single sql query, and the dataset columns must be based on the result 
fields.
What is the method you're speaking about for sql query in netepi? How does it 
support appending data?
I'm currently making it remember both the "last fetched primary key" of the 
source table, and the "modification date" field to add a limiter to the query 
and not fetch everything every day.

If you create a derived column and then you append new data, how do you 
calculate the derived fields of the new data?

Then I should probably create one dataset for each type of graph I have to draw,
and chain updates between datasets, unless the graphs can be drawn very quickly 
on demand.

Christophe

Re: [NetEpi-discuss] documentation and a few questions

[Tim C. <tc...@it...>]

On Thu, 2007-07-26 at 00:54 +0200, Christophe Combelles wrote:
> I've discovered NetEpi today, while looking for projects similar to
> the one I have to create. It looks great and I would like to know if
> there is some documentation available?
> I've looked at the test demo and it gives some useful information, but
> I'm not sure I will understand everything quickly without some
> reference. I'm mostly interested in **NetEpi Analysis**.

No, there is not any additional documentation for NetEpi Analysis at
this stage - it is still at version 0.8.x - although we use it almost
every day and it is quite stable. We have been busy working on Version
1.0.x of NetEpi Collection, including comprehensive documentation, but
have almost finished that and will get back to working on (and
documenting) NetEpi Analysis soon.

However, we are very happy to answer any questions, even fairly general
ones, about how to use it.

> To say it fast, I have only one month to create a (GPL or so) web
> application that:
>- collects data from a database of emergency interventions
>- allow to build queries/filters on the data
>- draws some statistical graphs
>- fires some alerts on some user-defineable criterions
>- All this with user acccounts and permissions.

NetEpi Analyis can currently do all but the last two of those - there is
no facility for user-definable trigger criteria, although we are about
to start work on integration of a sophisticated outbreak/signal
detection facility using NetEpi Analysis and R, which uses control chart
techniques on the residuals of iteratively-fitted one-step ahead EWMA
models (with lots of adjustments etc).

There is no built-in security model or access control model in NetEpi
Analysis - you can restrict access to each instance of it, via apache
etc, but not more fine-grained than that. In theory adding access
control at the dataset and/or column level would be quite possible, but
we have not done that yet. Row-level access control would not be
feasible.
        
> I'm ok on the web part of the work, I already know Zope3 to manage 
> authentication, templating, object database, etc.

It is good to know that Zope is still a popular choice in France. We are
looking at Plone as a portal for NetEpi Collection and NetEpi Analysis
instances.

Is Zope3 ready for production use? I thought it was still being
developed.

> I was beginning to implement a data model to store large sets of data
> and to do some operations on it, but I've discovered that SOOM is
> exactly what I needed!
>
> So I just have to thank you for this piece of software.

Have a look at PyTables as well.
        
> Is there some reference documentation for SOOM ?

Not yet, but we can answer your questions.

> I've seen that the demo load some cpickled data with dsload, but how
> should I manually fill the dataset, is it a standard python mapping? 
> dataset['column'][??] = ??

Can you describe what your source data looks like, and where it is
coming from? There are standard loaders for data files in CSV or other
delimited form, in fixed-width columnar format, in XML and for SQL
databases. Are your source data in any of these forms? Or you can load
data directly into columns - there is an .addcolumnfromseq() method for
dataset objects - have a look at demo/loaders/whopop.py for an example.
Others of the demo loading scripts illustrate how to create derived
columns as well. If you need an example of loading from an SQL database,
let us know and we can send one to you (it is not included in the
current distribution tarball).

> In which module can I find the statistical operations?

The main ones are in SOOMv0/Stats.py

> What is the role of the R language in NetEpi Analysis, apart from
> graphing?

R is also used for some of the statistical calculations in the Stats.py
module, but we try to do as much as possible using Numeric Python
because it is faster and more memory-efficient. But things like fitting
generalised linear models will all be done in R.

> Would you have any kind of advice, to allow me to finish on time?

Can you describe the scope of your project in a bit more detail?
Off-list if you prefer.

Tim C

[NetEpi-discuss] documentation and a few questions

[Christophe C. <cc...@go...>]

Hello,

I've discovered NetEpi today, while looking for projects similar to the one I 
have to create. It looks great and I would like to know if there is some 
documentation available?
I've looked at the test demo and it gives some useful information, but I'm not 
sure I will understand everything quickly without some reference.
I'm mostly interested in **NetEpi Analysis**.

To say it fast, I have only one month to create a (GPL or so) web application that:
- collects data from a database of emergency interventions
- allow to build queries/filters on the data
- draws some statistical graphs
- fires some alerts on some user-defineable criterions
All this with user acccounts and permissions.

I'm ok on the web part of the work, I already know Zope3 to manage 
authentication, templating, object database, etc.
I was beginning to implement a data model to store large sets of data and to do 
some operations on it, but I've discovered that SOOM is exactly what I needed!

So I just have to thank you for this piece of software.

Is there some reference documentation for SOOM ?
I've seen that the demo load some cpickled data with dsload, but how should I 
manually fill the dataset, is it a standard python mapping? 
dataset['column'][??] = ??
In which module can I find the statistical operations?
What is the role of the R language in NetEpi Analysis, apart from graphing?
Would you have any kind of advice, to allow me to finish on time?

thanks,
Christophe Combelles

[NetEpi-discuss] NetEpi Collection (formerly Case Manager) Version 1.0.2 released

[Tim C. <tc...@op...>]

Version 1.0.2 of NetEpi Collection (formerly known as NetEpi Case
Manager) is now available from SourceForge at
http://sourceforge.net/projects/netepi/

Existing users are urged to upgrade to this new release, which fixes
all known bugs. Special thanks to Francois Marsan for finding and then
carefully reporting a number of bugs which we had overlooked, and to
Stefan Stirzaker for his work on migrating the data and metadata from a
very early installation of NetEpi to a V1.0 installation, which also
revealed a bug in the current version.

We will make a further release of V1.0 by the end of July
2007, which will be accompanied by updated user and administrator
manuals. Work on updating the manuals is approaching completion, at last.

Regards,

Tim Churches and Andrew McNamara
19th July 2007

[NetEpi-discuss] NetEpi Collection (formerly Case Manager) Version 1.0.1 released

[Tim C. <tc...@op...>]

Version 1.0.1 of NetEpi Collection (formerly known as NetEpi Case
Manager) is now available from SourceForge at
http://sourceforge.net/projects/netepi/

Existing users are urged to upgrade to this new release, which fixes
several bugs. We will make a further release of V1.0 by the end of July
2007, which will be accompanied by updated user and administrator
manuals. These are taking me a lot longer than expected to complete.

Regards,

Tim Churches and Andrew McNamara
7th July 2007

Re: [NetEpi-discuss] version 1.0 bug

[Andrew M. <an...@ob...>]

>When I tried to ad a form on the " case/contact definition page , I get the
>following message.
[...]
>Sorry - an error has occured in this application
[...]
>Template traceback (most recent call last):
[...]
>  File "admin_pages/search_pt.html", line 44, in al-if
>          <al-if expr="s.index() < len(s) - 1">
>Traceback (most recent call last):
[...]
>  File "<albatross>", line 1, in <module>
>AttributeError: ListIteratorLite instance has no attribute '__len__'

I've checked in a fix for this that will appear in the next release. The
problem was triggered by a change in Albatross, but the fix is in
NetEpi Collection. Here's the diff, if you want to try it before the
next release:

Index: admin/pages/search_pt.html
===================================================================
--- admin/pages/search_pt.html  (revision 2647)
+++ admin/pages/search_pt.html  (working copy)
@@ -41,7 +41,7 @@
           </al-if>
          </td>
          <td width="14">
-          <al-if expr="s.index() < len(s) - 1">
+          <al-if expr="s.index() < len(pt_search) - 1">
            <al-input type="image" height="10" width="14" alt="dn"
              srcexpr="appath('images/button-down.png')"
              nameexpr="'%s:move_dn:%s' % (pt_search.name, s.value()[0])" />


-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

Re: [NetEpi-discuss] version 1.0 bug

[Tim C. <tc...@op...>]

francois marsan wrote:
> Hello,
> 
> When I tried to ad a form on the " case/contact definition page , I get the
> following message.
> 
>  NetEpi Collection
> Sorry - an error has occured in this application
> 
> Details of the error have been forwarded to the system maintainers
> 
> You will now be logged out of the application to clear the error. You are
> welcome to log back in.
> LOGIN<https://scpicsl.mlnfm.com/cgi-bin/collection/admin.py?>
> 
> Is this a bug in the program or it's a problem with my install?
> 
> Using Kubuntu 7.04 with all the packages for postgresql, python and
> albatross from the distribution.

Sorry for the delay in replying. To properly determine teh cause of the
problem, it would be helpful if you could send us part of the Apache
error log - On Kubuntu 7.04 it should be in /var/log/apache2 - not sure
which log file, probably error.log. just scroll through until you see a
Python traceback - send us the tracebacks or the entire log (send
directly to me, not to this mailing list as there may be IP addresses or
other information in the logs which you don't want to be sent to everyone).

However, the problem may be related to Python 2.5 - we have discovered a
subtle change in the way that Python 2.5 handles exception objects. We
already have a fix and can upload a new tarball or send you a patch -
but we would like to see the traceback in the apache2 log first to be
sure of the cause.

The problem is that our testing was done with Python 2.3 and 2.4, and
when Python 2.5 came out we failed to test under that. We won't make
that mistake again. Andrew: do we need to add some additional unit tests
for the Python 2.5 exception handling changes?

Tim C

[NetEpi-discuss] version 1.0 bug

[francois m. <ma...@gm...>]

Hello,

When I tried to ad a form on the " case/contact definition page , I get the
following message.

  NetEpi Collection
Sorry - an error has occured in this application

Details of the error have been forwarded to the system maintainers

You will now be logged out of the application to clear the error. You are
welcome to log back in.
LOGIN<https://scpicsl.mlnfm.com/cgi-bin/collection/admin.py?>

Is this a bug in the program or it's a problem with my install?

Using Kubuntu 7.04 with all the packages for postgresql, python and albatros
from the distribution.

Re: [NetEpi-discuss] installation issue

[Tim C. <tc...@op...>]

francois marsan <ma...@gm...> wrote:
> 
> Hi,
> 
> I want to know when the windows version is gonna be available?

An MS-Windows version of NetEpi Collection is perfectly possible as all the software infrastructure which it needs (Python, Apache, PostgreSQL) all run under MS-Windows. However there is a modest amount of work in adapting a few aspects to allow it to run on MS-Windows and to create an installer - perhaps one or two weeks work for someone familiar with such work. At this stage we have not scheduled such work but we would love to hear from anyone willing to take it on as a project - we will provide whatever technical support and guidance is necessary to assist with the MS-Windows version.

Alternatively, we have a LiveCD version which is almost ready. You can use this to boot Ubuntu Linux with NetEpi pre-installed on any recent (less than 4 years old) laptop or desktop computer, and data can be saved on a USB memory stick or external hard disc drive. You will be able to download the ISO file for this liveCD by the end of May (we'll announce it on this list), or I will send a physical CD-ROM to interetsed parties if you provide your physical mailing address (via a private email to tc...@op... please).

> I tried to install on a unbuntu 7.04 machine but it's a pain in the 
> neck. I get the front page with the message that the session server is not 
> working. What does it mean??? Is it the Albatross software that is not working??? 
> The configuration of apache???
> I follow all the install instructions.

OK, you need to start the Albatross session server. The best thing is to add an entry to /etc/init.d to start it automatically. And easy way to do this is to save the follow shell script to a file and run it as root - this should work without chnages on an Ubuntu 7.04 machine (it worked on mine). The paths may need to be changed for other Linux distributions.

#########################
# Init script for Albatross session daemon
if grep -q '^alsession:' /etc/passwd ; then true
else
    groupadd -g 123 alsession
    useradd -u 123 -g 123 alsession
fi
cat > /etc/init.d/al-session <<"EOF"
#!/bin/sh -e
### BEGIN INIT INFO
# Provides:             al-session
# Required-Start:       
# Required-Stop:       
# Should-Start:         
# Should-Stop:         
# Default-Start:        2 3 4 5
# Default-Stop:         S 0 1 6
# Short-Description:    Albatross Session Daemon
### END INIT INFO

DAEMON=/usr/bin/al-session-daemon
USER=alsession
LOGFILE=/var/log/al-session-daemon.log
PIDFILE=/var/run/albatross/session-daemon.pid
	
test -x ${DAEMON} || exit 0
	
umask 077

case "$1" in
    start)
        echo "Starting albatross session daemon"
        piddir=`dirname ${PIDFILE}`
        touch ${LOGFILE}
        if [ ! -d ${piddir} ]; then
                mkdir -p ${piddir}
        fi
        chown ${USER} ${LOGFILE} ${piddir}
        su ${USER} -c "${DAEMON} -l ${LOGFILE} -k ${PIDFILE} start"
        ;;

    stop)
        echo "Stopping albatross session daemon"
        su ${USER} -c "${DAEMON} -l ${LOGFILE} -k ${PIDFILE} stop"
        ;;

    *)
        echo "Usage: $* {start|stop}"
        exit 1
        ;;
esac
EOF
chmod a+rx /etc/init.d/al-session
update-rc.d al-session \
    start 21 2 3 4 5 . \
    stop 79 S 0 1 6 .
####################

> I already have a windows server with apache, posgresql, php, which 
> running a platform for medical use. It wasn't so heavy to install.

Yes, we do want to have an MS-Windows server version of NetEpi, but we don't have any experience of creating MS-Windows installers etc - we need help!

> I'm working to find a software to help me in my job for prevention of
> diseases.

Feel free to describe what sort of work you do and what your software requirements are on this list and we can advise whether NetEpi is likely to help you.

Tim C

[NetEpi-discuss] installation issue

[francois m. <ma...@gm...>]

Hi,

I want to know when the windows version is gonna be available?
I tried to install on a unbuntu 7.04 machine but it's a pain in the neck. I
get the front page with the message that the session server is not working.
What does it mean??? Is it the Albatross software that is not working??? The
configuration of apache???
I follow all the install instructions.

I already have a windows server with apache, posgresql, php, which running a
platform for medical use. It wasn't so heavy to install.

I'm working to find a software to help me in my job for prevention of
diseases.

Any one can help?

F. Marsan

[NetEpi-discuss] NetEpi Collection (formerly Case Manager) Version 1.0beta released

[Tim C. <tc...@op...>]

Version 1.0beta of NetEpi Collection (formerly known as NetEpi Case Manager) is
now available from SourceForge at
http://sourceforge.net/projects/netepi/

Existing or prospective users are urged to test this new release. We
expect to make a final release of V1.0 at the end or March or early April 2007.

Following are some notes on the changes in Version 1.0beta relative to the
previous releases back to Version 0.90, which was released in
June 2005.

Please consult the README file at
http://sourceforge.net/docman/display_doc.php?docid=46810&group_id=123700
for brief instructions on installing the software (on Linux, Apple Mac
OS X or other Unix servers only at this stage, although it can be
accessed from any kind of computer running any relatively recent Web
browser).

End-user and administrator manuals and training material will be updated
prior to the final Version 1.0 release - currently the manuals and help
files reflect the Version 0.90 release from June 2005.

Regards,

Tim Churches and Andrew McNamara
7th December 2006

NetEpi Collection Version 1.0beta README
========================================

LICENCE
=======

All materials associated with "NetEpi Collection" are Copyright (C) 2004-2007
Health Administration Corporation (New South Wales Department of Health) and
others (see the CONTRIBUTORS file for details).

NetEpi Collection is licensed under the terms of the Health Administration
Corporation Open Source Licence Version 1.2 (HACOS Licence V1.2), the full text
of which can be found in the LICENCE file provided with NetEpi Collection.

SUMMARY OF CHANGES IN VERSION 1.0beta RELATIVE TO VERSION 0.99
==============================================================

The major addition to Version 1.0beta is the inclusion of a comprehensive
record merging facility. This provides a mechanism to identify potentially
duplicated PERSON records (using a bigram-based fuzzy matching algorithm,
although arbitrary pairs of records can also be merged), and the user can
then selectively merge demographic details and associated case, contact
and task records, for each pair of duplicated person records. This may
result in duplicated case or contact records for the resulting "merged"
person, so there is an additional facility which can be used to merge
case or contact records for a given person. Similarly for each case or
contact record, there is a facility to merge data form records of the
same type. In these merge facilities, each of which can be accessed
independently of the others, the user can select whether to retain
data from "record A" or "record B", on a data-item-by-data-item basis,
or may chose to include information from both A and B records and edit
the resulting combined data field.  Much effort was put into designing a
simple user interface to these merge facilities which make the complex
merging task as quick and easy as possible for the end user. Unlike
deletions, merges of records cannot be undone, but full details of
all merge actions are captured in the audit trail log, allowing manual
separation of merged records if required subsequently.

Another major change in this version is the renaming of the application
from "NetEpi Case Manager" to "NetEpi Collection", which better reflects
the fairly generic role it plays, while also sounding less clinical.

Other significant enhancements to Version 1.0beta include:

* Provision of a read-only mode, in which the privileges of selected
  users can be restricted so that they can only view data, but not
  edit or delete it. The usual fine-grained access control mechanisms
  still apply to such read-only users.
* Pull-down menus have replaced a plethora of buttons on the case
  and contact editing pages. The result is a cleaner look with no loss
  in usability (we feel, feedback welcome).
* Labelling of cases versus contacts has been made smarter in order to
  reduce the occasional semantic confusion that was evident in parts
  of the application.
* On certain pages more contextual information with respect to the
  current user or current case or contact has been provided to assist
  users in regaining their "bearings" after being interrupted while using
  the system.
* Visual feedback is now provided each time a record of any type is updated.
* Most error messages and warnings now appear at the top of the page,
  where they are harder to overlook.
* Access to the built-in line-listing/reports facility is now restricted
  to those users who have been granted the "bulk export" privilege.
* The "Next" button on search pages is now the default, so that the
  Enter key can be used to cause a search to be performed (thus
  improving end user ergonomics).
* The "round-trip-time" facility, which measures and captures the
  response time (including network-related delays and latency) of NetEpi
  from the point-of-view of the end user, has been enhanced to work
  correctly when the Microsoft Internet Explorer browser is used, and now
  also records a timestamp for each response time captured in the web server
  log. Utilities to extract these response times from the web server log
  for analysis and presentation in statistical or other visualisation
  packages have also been added, including a script to import the data
  into NetEpi Analysis.
* Another utility, called "httpinteract", has been included. It is
  a simple tool which can be used to acquire time-series data about
  network performance from end-user's workstations or access points.
  We have found it useful in pinpointing when and where network bandwidth,
  latency or congestion may be inadequate for satisfactory use of NetEpi,
  so that appropriate remedial action can be taken - such network problems
  can almost always be solved.
* The deletion status of records is clearly shown when they are printed
  out.
* Facilities to browse all cases for a given person, and to jump directly
  to the parent case of a contact, were added.
* Unit tests and some Selenium functional tests were updated to work
  correctly with the current version.
* System documentation was brought up-to-date.

Finally, a lot of work has also gone into the creation of a "live CD"
for demonstration purposes. This "live CD", which is based on the popular
Ubuntu Linux distribution, allows most recent (that is, less than four
or five years old) desktop or laptop computers to be booted directly
from the live CD into a cut-down version of Ubuntu Linux, with a full,
working copy of NetEpi Collection installed and ready to use. Data can be
stored between sessions on a USB memory stick or external hard drive. No
software or files are actually installed on the host computer - it runs
entirely from the CD-ROM. The demonstration system includes a full Web
server and can thus be accessed over a network by multiple users, as well
as locally.  Introductory "screencasts" and other background material will
be included on the final NetEpi live CD. However, at this stage the live
CD is still undergoing testing, but we have included the scripts needed
to build the live CD in the NetEpi Collection distribution tarball for
those who wish to create their own customised version. However, these
scripts are complex and dangerous, and should only be run and modified
by experienced system administrators and developers who know exactly
what they are doing. They are only needed to create new versions of a
live CD and should be completely ignored by NetEpi users and most system
administrators. An .iso file of a demo live CD will be made available
for download (and then burning onto a CD-ROM) at the time of the final
release of Version 1.0.


SUMMARY OF CHANGES IN VERSION 0.99 RELATIVE TO VERSION 0.98
===========================================================

Apart from several minor bug fixes, some general cleaning up of the code
base and some additional unit tests, the very big addition to Version
0.99 is a built-in "line-listing" reporting facility.

This facility allows tabular "line-listing" reports to be produced
through a simple and easy-to-use interface. Filters and sorting orders
based on demographic data and on data items contained in syndrome/case
definition-specific forms can be defined, and the resulting record sets
appear in the report or can be used as a "browse list" to page through
case or contact records in the edit screens, one at a time. Summary
or user-selectable fields (columns) from forms can also be included
in the reports.

The (still experimental) case-to-contact relationship visualisation tool
has also been integrated into the reporting framework to take advantage
of the filtering facilities it provides.

Report definitions (including the filtering parameters) can be saved
for re-use later, and these saved report definitions can be shared
with other members of the user's business unit or with all users
on the system.


SUMMARY OF CHANGES IN VERSION 0.98 RELATIVE TO VERSION 0.97
===========================================================

Version 0.98 brings the following (in no particular order):

* A (global) count of active cases of each syndrome is now
  shown on the home page.

* An indication of number of contacts is now shown on the
  main case edit page.

* The ability to examine the details of users in each
  business unit from task assignment and ACL (access control
  list) pages.

* All current and enabled syndromes/case definitions are now
  shown on the main page, not just those the unit (or rather
  users in that unit) has access to, but the "add" button is
  suppressed if the unit does not have edit rights for that
  syndrome/case definition.

* A facility to allow NetEpi administrators to "clear" a
  syndrome/case definition of data - this deletes all cases,
  contacts, form instances (but not form definitions!) and
  tasks associated with a syndrome/case definition (without
  deleting the syndrome/case definition itself). This is very
  useful for clearing out training or test data from a
  syndrome/case definition prior to "production" or real-life
  use. An additional confirmation was added because this
  facility permanently deletes data.

* A facility by which a NetEpi administrator can use the web
  interface to permanently delete a syndrome/case definition
  and all associated cases, forms, tasks and log entries. This
  allows redundant, obsolete or test/dummy syndromes/case
  definitions to be removed from an existing NetEpi instance,
  or for a NetEpi instance to be cloned (using PostgreSQL
  database back-up and restore utilities) and then unwanted
  syndromes/case definitions removed. An additional confirmation
  was added because this facility permanently deletes data.

* No longer indicate nature of login failures to improve
  security.

* Add additional demographic fields for one extra set of
  alternative address and other contact information (which
  can be relabelled for each case definition/syndrome)

* Improvements to form definition importing (form definition
  import name matching a little smarter, make form import seed
  the "forms" table if necessary).

* A new command line facility to import and export form definitions
  in batches from a Collection instance.

* "Onset date" and "Notification date" error messages now
  honour syndrome-specific labelling for these fields.

* The ability to view logs associated with a contact.

* The main application structure graph was updated, and an admin
  application structure graph was added.

SUMMARY OF CHANGES IN VERSION 0.97 RELATIVE TO VERSION 0.96
===========================================================

The major additions to Version 0.97 are:

* Ability to add a contact record and post-hoc associate it with
  a case, and/or dissociate it from a case and re-assign to a different
  case, and handle the merging of ACL (access rights) necessitated
  by such changes.
* Improvements to the form data roll-forward facility so that
  incompatible changes to form filed data types that would lead
  to data loss or truncation can be readily identified and corrected.
* Ability to print the data associated with a case or contact as a
  "report" (rather like a set a already-filled-in forms)
* Ability to optionally include or exclude deleted records from
  data exports
* Restriction of ability to export data to only some users who
  have been explicitly granted the right to do so.
* Improvements to the information captured in the audit log,
  particularly with respect to contact records and their association
  with case records.
* Require confirmation when undeleting cases or contacts.
* Require confirmation needed when deleting a user in the admin
  application.
* Fixed bug when a users from a disabled unit tried to log in.
* Simplification of conditional question skip/enable subsystem.
* Additional guards against inadvertent failure to save changes in
  admin interface

In addition, numerous minor cosmetic changes and bug fixes have been
implemented.

SUMMARY OF CHANGES IN VERSION 0.96 RELATIVE TO VERSION 0.95
===========================================================

The major addition to Version 0.96 is a conditional form input skip/enable
feature, which allows parts of questions, whole questions or multiple
questions on a form to be selectively enabled or disabled based on
categorical data elsewhere on the same form. Skip instructions are
automatically generated for printed forms or where Javascript is
unavailable or disabled.

A large improvement to usability is also provided by some clever
Javascript which captures the "Back" button on the user's browser and
makes it behave like the "<<Back"" navigation buttons in the application
itself. This effectively removes a major source of user frustration by
avoiding browser-generated "re-post" dialogue boxes appearing on screen.

In addition, a large number of bug fixes have been made and the interface
to parts of the admin application has been made more consistent and
foolproof - it is now very hard to inadvertently forget to save changes
made in the admin application. Also, the question editor pages in the
admin form definition editor facility has been revised.


SUMMARY OF CHANGES IN VERSION 0.95 RELATIVE TO VERSION 0.94
===========================================================

The major addition to Version 0.95 is a simple (but we hope effective)
workflow subsystem, which allows "tasks", such as completing a case or
contact follow-up form, to be assigned to individual users, business
units or arbitrary "task queues" to which users and business units can
be subscribed.  These tasks can be scheduled for immediate or delayed
attention, and users have access to a list of tasks assigned to them
or otherwise available to them through their membership of a business
unit or subscription to a task queue.  Clicking on an assigned task
automatically loads the correct case or contact record and opens the
correct data form to be completed. Additional instructions can be added
to tasks if necessary.

This subsystem is intended to allow tasks generated by a large number
of cases or contacts of cases of, say, a novel strain of influenza to
be distributed to many workers located in many business units across
the health system. It should also be useful for many other purposes,
including reminders to oneself to follow-up missing data or check
responses and so on.

Other features added to Version 0.95 include better visual feedback when
data in forms is updated, better indication of the deletion status of
records, improvements to the form editor, and various minor cosmetic
and labelling improvements.

SUMMARY OF CHANGES IN VERSION 0.94 RELATIVE TO VERSION 0.90
===========================================================

Following is a summary, in no particular order, of the major changes
visible to or of relevance to end-users and/or administrators, in
NetEpi Collection Version 0.94, relative to NetEpi Collection
Version 0.90 which was released in June 2005. Note that Versions 0.91
and 0.92 were intermediate versions which were not released publicly
and Version 0.93 was quickly superseded by Version 0.94.

In addition, there has been considerable refactoring and general tidying-up
of the underlying programme code, as well as several bug fixes and security
improvements.

 * Problem with download of exported data files via Microsoft Internet
   Explorer fixed.

 * Added a new class of user (called "unit admins") who can administer
   users in their unit via a new page accessible from the end-user (not
   admin) application. This removes a potential bottleneck of admins
   having to remotely vet and approve large numbers of new user accounts
   in a short time during an emergency.

 * Added JavaScript to track and report client's submit RTT
   (round-trip-time), added code to Request object to log reported
   RTT. This allows the total response time, as perceived by the end
   user, to be monitored centrally - in particular, problems related to
   network congestion can be detected, as well as problems due excessive
   server load.

 * Added the ability to print selectable sets of blank forms, on
   a per-syndrome basis, for use if the system is offline, down or
   otherwise unavailable.

 * Ability to add certain rights to groups of users, and also ability
   to add certain rights to individual users. These include the right
   to see or edit all records.

 * Version information added to login screens to help avoid confusion
   during support calls.

 * Ability to save user preferences, including preferences for phonetic
   searching, results per page and pop-up calendars (see below).

 * Added client-side sorting of various tables in the admin application.

 * Javascript used to to scroll to first error in forms.

 * Form definitions are now stored in XML form in the main database, not
   as Python files in the Web server file system. This improves security
   and makes it much easier to back-up the entire state of the system,
   without having to interrupt use of the system in any way.

 * Added ability to export and import form definitions in XML form, which
   lays the foundations for a shared and/or distributed form definition
   "library".

 * Form inputs that are required (mandatory) are now visually identified.

 * Added "definition last updated" and other useful metadata to form
   definitions.

 * Merge demogfields-branch

 * Added additional demographic/ID fields: mobile_phone, fax_phone,
   e_mail, passport_number and passport_country, interpreter
   required/language, to make the system better suited for use in border
   screening at airports etc.

 * Added ability to relabel or hide demographic/ID fields on a
   per-syndrome basis.

 * Added per-syndrome case status field, renamed field to just "status".

 * Storage of case and contact data unified. This allows a great deal
   more flexibility, including the ability to specify multiple "contact
   syndromes" for different types of contacts of cases.

 * Added interactive pop-up date entry widget (calendar).

 * Allowed allow a person to be contact of a case for more than one
   contact syndrome; included contact type in search results; revised
   styling of search results for cases and contacts to make them slightly
   clearer.

 * Simplified application navigation slightly (removed user details
   link from page banner, use tools instead), carry more context across
   new/edit/search screens.

 * Added paging to contacts list to allow cases with large numbers of
   contacts to be more easily managed.

 * Added experimental Case/Contact relationship visualisation using
   GraphViz.

 * Changes to allow PostgreSQL versions 8.0 and 8.1 to be used, as well
   as versions 7.3 and 7.4 as supported previously.

 * Deleting/hiding of cases and contacts now fully implemented.

 * Major speed improvements to phonetic searches.

 * Updated Collection-ER documentation to reflect schema changes.

 * Added a confirmation dialog when admins enable a user, to ensure that
   they have checked the new user's bona fides.

 * Add logging of case access (ACL) control changes.

 * Added support for use of Trac-style wiki mark-up for text used in
   various places in the application (see doc/NetEpi_wiki_markup.html ).


A NOTE FOR THOSE WISHING TO UPGRADE EARLIER VERSIONS OF NetEpi Collection
=========================================================================

Users of early versions of NetEpi Collection - that is, versions released
prior to May 2005 (Version 0.90) - should consult the developers via email
for advice on how to proceed with upgrading existing installations.

Users of Version 0.90 or later should carefully read the advice provided
below (especially the INSTALLATION section). If in doubt, please do not
hesitate to contact the developers via the netepi-discuss mailing list
(see below) for advice on upgrading.

An exhaustive list of the changes made since the inception of the project
appears in the CHANGELOG document in doc/ directory contained in this
distribution.

DOCUMENTATION AND MANUALS
=========================

Please note that the end-user and administrator manuals for NetEpi
Collection, which are available from SourceForge in a separate file,
have NOT been updated to reflect new features and changes made since
version 0.90 was released. These documents will be updated before the
final release of version 1.0.

Some interim documentation of the wiki mark-up feature, which was
introduced in Version 0.94, appears in the doc directory.

BUG REPORTS, FEATURE REQUESTS AND GENERAL DISCUSSION
====================================================

Please report all bugs, problems, feature requests and ideas to the
NetEpi-discuss mailing list. You need to subscribe to this list in
order to post messages to it - see the list management Web page at:

    http://lists.sourceforge.net/mailman/listinfo/netepi-discuss