#261 Async processing of inbound HL7 results

[MD Support]

This commit implements async processing of inbound HL7 results from all partners.

Current logic fetches and processes HL7 files from all partner SFTP sites every time list_reports.php is invoked. This step can take several seconds (minutes in case of multiple partners or if one of the sites is down) before sending any output. This patch will make the process asynchronous.

By default, this script produces report of all results received to date since the dates default to blank. If a site has processed 1000s of results, creation and transmission of this large list delays output. This patch sets today's date as the default - effectively limiting the output.

[Brady Miller]

Placed some comments on github.

Hopefully Rod can weigh in on this since he has more experience with this module.

-brady

[Rod Roark]

Maybe I'm missing something but don't see how this can work properly. The $info array is modified by the polling function and used to send back patient matching requests and such that the user must respond to. How can that work when the request happens asynchronously?

[MD Support]

Users see the report page immediately and if they stay on the page, 'pollmsg' area is populated with a lag - similar to clinical reminders appearing bit later when patient demographics is viewed.

fetch_data.php was part of our earlier modifications to hl7 business process when $info wasn't needed. One can now include $info in response as well. However for our number of partners and volumes, we need always on background processes. So we will shortly handle hl7s outside this application.

Let's close this ticket. Thx.

[Rod Roark]

OK so do I understand correctly that as it is right now, this change would break manual patient matching?

I think what is really needed is a background process that will store lab results that need manual attention into a database table, and a GUI to allow the user to make the corrections and get them processed.

Thanks.

[MD Support]

That's correct. As it stands, it will report the error messages but not $info. If you need $info stuff to be presented, we will need to make error resolution as common functions in receiving for correct response in 'pollmsg'.

We currently use fetch_data.php as a background service running every hour. Manual matches are done with a small script that shows name, dob and specimen collection date from hl7 files and choices of open orders with no results.