I just posted a new Supplementary Topic documentation in the 4.2 user guide ("Review: EDI History Module" at the bottom) and I'm eager to hear any feedback and/ or questions people might have.
Briefly, Kevin McCormick's EDI History Module lets you read the contents of most EDI files, which is helpful in troubleshooting billing issues. The module has been in OpenEMR since 4.2.0, I believe, and even though it isn't fully implemented it is still very useful in its present state. Being not fully implemented contributes to the overall length and complexity of the docs, but if you read it through you will then know how to use the much more convenient 'Quick Reference' at the top.
Give it a whirl and maybe if enough people think it's useful Kevin will buff it up and release it for real.
Regards- Harley
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for the kind words. I have been working on a version 2. Present code is at https://github.com/kevmccor/openemr/tree/edihistory_v2 There is a lot of extra cruft, and all the files will be renamed, but most of the nuts and bolts is near acceptable. The user interface is what I am starting to work on and there is the issue of having a working openemr installation for testing.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Lot of code in there :)
Is this ready for a review or still a work in progress?
Are the filenames with _v2 in them going to overwrite their companion v1 files?
Still a work in progress. The v2 files are not direct replacements since I have revised the processing flow. The idea is the v2 files will replace the v1 files as a whole. Users should not notice any major change, but I do want the user nterface to be better, so v2 files under /interface/billing/ will replace v1 files.
Question: is there a script in OpenEMR for auto ftp connect and upload/download?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes, that part of the script looks promising. Probably all I can do is put together an outline of code and a wish list.
Another question: the x12 278 authorization HI segment seems to require the LOINC set of health care codes. LOINC These come with a script for creating a mysql database. Is this in OpenEMR somewhere? Are there license issues? Is the same thing in SNOMED?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
i'm pretty sure it's not in the openemr database but i don't think any payers actually use LOINC
you can find a discussion of the licensing under Administration->Other->External Data Loads when you click on SNOMED and hover over the question mark and then follow the link to ihtsdo.org
Last edit: Stephen Waite 2016-04-04
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We use a CLI script for background SFTP transfer - submitted as code review ticket #234, Never got around to converting it to background service. You are welcome to check that approach and bring it in standard codebase.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have read over the phpseclib code, but I am not at the level of beng able to understand it without a lot of fiddling, which is hard with this type of operation. Does it interact with php $FILES in any way? There is a $dir_to variable in the script, which is presumably the download directory. At this time, the edi_history requires the user to download edi files to their home directory and then upload them in the New Files tab. This would be the process to automate.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
$FILES requires interactive processing. As noted in comments to Brady, we needed this for background file transfers to a specified directory on the server without any user action. In your case you could schedule server to server file transfers. Then the files could get processed automatically or when user calls edi_history, it can process all files in the directory.
Our cronjob command examples for FTP from various laboratory sites and remote backup storage are:
FTP for all records configured in 'Procedures/Sevices Provider" or you could specify a specific record id
actn=get
FTP action (get or put)
ldir=xxx
Local directory on server (e.g. ldir=/var/shared/lab)
sub=yyy
Subdirectory token (e.g. sub=ppid will result in files from partner 1 to be placed in/read from server directory /var/shared/lab/1). Comments in the code specify other tokens available for sub directory specification.
pdir=pxx
Directory on partner's FTP site
rhost=a.b.c.d
FTP partner's ip/dns name
rport=22
FTP port for the partner.
ruser=rusr rpass=rpp
FTP Authentication
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have a dummy file which goes over a way to tie sftp into edi_history. Nothing on sftp itself, just a way the sftp could be structured and the downloaded files uploaded into the /edi/history/filetype directories and subequently parsed. test_edih_sftp_files.php. I think someone else will have to code the actual sftp process.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I believe version 2 is getting very close to being ready for review. There are still a couple of issues in the code flow, but my main problem is jquery, jquery-ui, and css. You can check it out at edihistory_v2 branch
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think all the functionality and operation of version 2 is ready. I am currently investigating a minimize button in the juery-UI dialog titlebar, but that will not affect anything else. I asked for people to download and test the code in the developer forum, but no luck. The test process is to download the branch edihistory_v2. Then install it as a test version of OpenEMR, i.e. different path in your /var/www/htdocs/ or wherever your httpd server gets web pages from. Once OpenEMR is set up with an admin or user with billing/accounting permissions, you simply go to EDI History in the left nav and upload edi files in the New Files tab. With some files uploaded, you can check all the other tabs, especially the Archive tab. I think version 2 is much better than version 1, so I hope someone will test it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
hi Kevin, thank you for version 2. I keep receiving a popup stating "Archive already performed!" after I first chose files older than 24m and there weren't any. "Archive: type f835 archive found no files older than 24m"
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Many thanks for the feedback. The "Archive" action uses the "one" jQuery action, instead of "on", meaning it will run only once in a session (/interface/edih_view.php ~ line 1053). I did this to prevent inadvertent over-archiving, basically asuming the archive period would be carefully chosen (possibly a wrong assumption). The /library/edih_archive.php ~ line 997, does check if the archive file already exists. If you re-load the EDI History from the left nav it should work again.
I should have mentioned that the 837 batch claim files should be copied directly to the testing /sites/default/edi directory. This is where OpenEMR puts them after they are generated. The response files (997/999, 277, or 835) are the ones that have to be loaded manually.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I just made a commit that cleaned up /interface/billing/edih_view.php and some edits towards using the jQuery-UI accordion in /library/edihistory/edih_277_html.php. With regard Stephen Waite's comments, I changed the jQuery for the Archive tab (edih_view.php) so the alert was not triggered, but an existing archive would not be re-done. I think the code is pretty well complete unless there is an error I have missed, but cleanup would not hurt.
Looking through the comments in your commit, I see a lot about translations. IMHO, translations are good for the actual user interface, edih_view.php and places in edih_main.php, but we should not translate any edi file contents. I made a comment on this, but I think whatever language issues there are would be handled by the insurance company or clearinghouse. I think the x12 edi is really a US standard and there is a different format in other countries (who are not so tied into insurance companies). When presenting x12 edi file contents, I simply take whatever characters are between the delimiters and put them in their spot, without making any inferences or interpretations. The idea is to display all the actual contents in a readable format so people can see what is in the files.
Thanks, I appreciate your efforts.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Translations for this type of thing are obviously murky and will defer to you on where it should be used. Note there is always the possibility that somebody in the US will be more fluent in another language (such as Spanish or Chinese); for example, an Acupuncture and Chinese Medicine clinic that bills to US insurance but are only fluent in Chinese (and English insurance lingo).
It was a little tricky for me to get the jQuery libraries to all work together, which is why I ended up with what I have. It should be just a matter of changing the paths in the /interface/billing/edih_view.php as that is the only place they are used. How would I do that with my github branch?
It might be a good idea to have one set of jQuery libraries, plugins, and css all together that interact correctly because it now appears to be a hodgpodge of javascript libraries in /library/js. I found I had to use the latest jQuery DataTables because the older versions were not really available and then I had to use the more recent jQuery-UI and the jQuery it was based on. I don't think the "latest and greatest" is the main issue with jQuery, just having the whole thing working seems more important. I'm not a javascript expert by any measure, so I could not explain why the older versions did not work or why it ended up working the way I did it. Even with "firebug" coding javascript is a total pain, IMHO.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just let me know when you have a break in coding. I'll then rebase your code into 1 commit on top of most recent codebase. Then you can work from that branch.
Brady, I think you should just continue with your review and change links to javascript libraries to the locations you like. Also any translations you need. This will all be in the file /interface/edih_view.php. The only thing I was wanting to change was to extend the jquery-ui dialog to add a minimize button in the titlebar, but that is taking too long and I have other things I need to do now. It is not that important for the user, but I thought it would be a nice touch and I will submit it when I finally get it done. I don't want to hold things up for this. When I submitted the first version, you did all this for me then, too. Tie ins to the OpenEMR libraries will be in /interface/edih_main.php, but I think that is all there.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Please test it and let me know that it is working and looking fine. After confirm that it is working, then I would be happy to help you get this project past the finish line.
Hi Folks-
I just posted a new Supplementary Topic documentation in the 4.2 user guide ("Review: EDI History Module" at the bottom) and I'm eager to hear any feedback and/ or questions people might have.
Briefly, Kevin McCormick's EDI History Module lets you read the contents of most EDI files, which is helpful in troubleshooting billing issues. The module has been in OpenEMR since 4.2.0, I believe, and even though it isn't fully implemented it is still very useful in its present state. Being not fully implemented contributes to the overall length and complexity of the docs, but if you read it through you will then know how to use the much more convenient 'Quick Reference' at the top.
Give it a whirl and maybe if enough people think it's useful Kevin will buff it up and release it for real.
Regards- Harley
thanks Harley, i'll check it out, here's the link
Last edit: Stephen Waite 2016-01-18
that's a nice summary of Kevin's tremendous work, let's hope the force awakens
minor alterations on my instance include the addition of .835 to the regular expression here for importing office ally era
there's also an easy way to ignore newly introduced segments like MIA at this location
Hi Stephen,
Just checking to see if you think your commit above should go into the main codebase?
thanks,
-brady
OpenEMR
Thanks for the kind words. I have been working on a version 2. Present code is at https://github.com/kevmccor/openemr/tree/edihistory_v2 There is a lot of extra cruft, and all the files will be renamed, but most of the nuts and bolts is near acceptable. The user interface is what I am starting to work on and there is the issue of having a working openemr installation for testing.
Hi,
Lot of code in there :)
Is this ready for a review or still a work in progress?
Are the filenames with _v2 in them going to overwrite their companion v1 files?
-brady
OpenEMR
Still a work in progress. The v2 files are not direct replacements since I have revised the processing flow. The idea is the v2 files will replace the v1 files as a whole. Users should not notice any major change, but I do want the user nterface to be better, so v2 files under /interface/billing/ will replace v1 files.
Question: is there a script in OpenEMR for auto ftp connect and upload/download?
Rod references this object.
Yes, that part of the script looks promising. Probably all I can do is put together an outline of code and a wish list.
Another question: the x12 278 authorization HI segment seems to require the LOINC set of health care codes. LOINC These come with a script for creating a mysql database. Is this in OpenEMR somewhere? Are there license issues? Is the same thing in SNOMED?
i'm pretty sure it's not in the openemr database but i don't think any payers actually use LOINC
you can find a discussion of the licensing under Administration->Other->External Data Loads when you click on SNOMED and hover over the question mark and then follow the link to ihtsdo.org
Last edit: Stephen Waite 2016-04-04
We use a CLI script for background SFTP transfer - submitted as code review ticket #234, Never got around to converting it to background service. You are welcome to check that approach and bring it in standard codebase.
I have read over the phpseclib code, but I am not at the level of beng able to understand it without a lot of fiddling, which is hard with this type of operation. Does it interact with php $FILES in any way? There is a $dir_to variable in the script, which is presumably the download directory. At this time, the edi_history requires the user to download edi files to their home directory and then upload them in the New Files tab. This would be the process to automate.
$FILES requires interactive processing. As noted in comments to Brady, we needed this for background file transfers to a specified directory on the server without any user action. In your case you could schedule server to server file transfers. Then the files could get processed automatically or when user calls edi_history, it can process all files in the directory.
Our cronjob command examples for FTP from various laboratory sites and remote backup storage are:
I have a dummy file which goes over a way to tie sftp into edi_history. Nothing on sftp itself, just a way the sftp could be structured and the downloaded files uploaded into the /edi/history/filetype directories and subequently parsed. test_edih_sftp_files.php. I think someone else will have to code the actual sftp process.
I believe version 2 is getting very close to being ready for review. There are still a couple of issues in the code flow, but my main problem is jquery, jquery-ui, and css. You can check it out at edihistory_v2 branch
I think all the functionality and operation of version 2 is ready. I am currently investigating a minimize button in the juery-UI dialog titlebar, but that will not affect anything else. I asked for people to download and test the code in the developer forum, but no luck. The test process is to download the branch edihistory_v2. Then install it as a test version of OpenEMR, i.e. different path in your /var/www/htdocs/ or wherever your httpd server gets web pages from. Once OpenEMR is set up with an admin or user with billing/accounting permissions, you simply go to EDI History in the left nav and upload edi files in the New Files tab. With some files uploaded, you can check all the other tabs, especially the Archive tab. I think version 2 is much better than version 1, so I hope someone will test it.
hi Kevin, thank you for version 2. I keep receiving a popup stating "Archive already performed!" after I first chose files older than 24m and there weren't any. "Archive: type f835 archive found no files older than 24m"
Many thanks for the feedback. The "Archive" action uses the "one" jQuery action, instead of "on", meaning it will run only once in a session (/interface/edih_view.php ~ line 1053). I did this to prevent inadvertent over-archiving, basically asuming the archive period would be carefully chosen (possibly a wrong assumption). The /library/edih_archive.php ~ line 997, does check if the archive file already exists. If you re-load the EDI History from the left nav it should work again.
I should have mentioned that the 837 batch claim files should be copied directly to the testing /sites/default/edi directory. This is where OpenEMR puts them after they are generated. The response files (997/999, 277, or 835) are the ones that have to be loaded manually.
Hi Kevin,
I rebased your branch into 1 commit a couple days ago and Ray has been reviewing the code there. Here is the branch:
https://github.com/bradymiller/openemr/commits/rebase-kevmccor-edihistory_v2
On the commit, search for 'ophthal' to go through Ray's comments.
-brady
OpenEMR
I just made a commit that cleaned up /interface/billing/edih_view.php and some edits towards using the jQuery-UI accordion in /library/edihistory/edih_277_html.php. With regard Stephen Waite's comments, I changed the jQuery for the Archive tab (edih_view.php) so the alert was not triggered, but an existing archive would not be re-done. I think the code is pretty well complete unless there is an error I have missed, but cleanup would not hurt.
Looking through the comments in your commit, I see a lot about translations. IMHO, translations are good for the actual user interface, edih_view.php and places in edih_main.php, but we should not translate any edi file contents. I made a comment on this, but I think whatever language issues there are would be handled by the insurance company or clearinghouse. I think the x12 edi is really a US standard and there is a different format in other countries (who are not so tied into insurance companies). When presenting x12 edi file contents, I simply take whatever characters are between the delimiters and put them in their spot, without making any inferences or interpretations. The idea is to display all the actual contents in a readable format so people can see what is in the files.
Thanks, I appreciate your efforts.
Hi Kevin,
Translations for this type of thing are obviously murky and will defer to you on where it should be used. Note there is always the possibility that somebody in the US will be more fluent in another language (such as Spanish or Chinese); for example, an Acupuncture and Chinese Medicine clinic that bills to US insurance but are only fluent in Chinese (and English insurance lingo).
Would it be possible to use the libraries in public/assets instead(and drop the ones you are bringing in)? I added the libaries that you are using there in the main codebase:
https://github.com/openemr/openemr/commit/40e62de244119d1e1d70d92c6359cd0252cb6960
-brady
OpenEMR
It was a little tricky for me to get the jQuery libraries to all work together, which is why I ended up with what I have. It should be just a matter of changing the paths in the /interface/billing/edih_view.php as that is the only place they are used. How would I do that with my github branch?
It might be a good idea to have one set of jQuery libraries, plugins, and css all together that interact correctly because it now appears to be a hodgpodge of javascript libraries in /library/js. I found I had to use the latest jQuery DataTables because the older versions were not really available and then I had to use the more recent jQuery-UI and the jQuery it was based on. I don't think the "latest and greatest" is the main issue with jQuery, just having the whole thing working seems more important. I'm not a javascript expert by any measure, so I could not explain why the older versions did not work or why it ended up working the way I did it. Even with "firebug" coding javascript is a total pain, IMHO.
Hi Kevin,
Just let me know when you have a break in coding. I'll then rebase your code into 1 commit on top of most recent codebase. Then you can work from that branch.
As an aside, there is a ongoing project in an attempt to standardize these libraries(first step is simply organizing them):
http://www.open-emr.org/wiki/index.php/Bower
-brady
OpenEMR
Brady, I think you should just continue with your review and change links to javascript libraries to the locations you like. Also any translations you need. This will all be in the file /interface/edih_view.php. The only thing I was wanting to change was to extend the jquery-ui dialog to add a minimize button in the titlebar, but that is taking too long and I have other things I need to do now. It is not that important for the user, but I thought it would be a nice touch and I will submit it when I finally get it done. I don't want to hold things up for this. When I submitted the first version, you did all this for me then, too. Tie ins to the OpenEMR libraries will be in /interface/edih_main.php, but I think that is all there.
Hi Kevin,
First step is to migrate to the standard assets, which I have just done. I placed the work on this branch(your work is in the top commit there and note there are no more libraries in there :) ):
https://github.com/bradymiller/openemr/commits/rebase-kevmccor-edihistory_v5
Please test it and let me know that it is working and looking fine. After confirm that it is working, then I would be happy to help you get this project past the finish line.
thanks,
-brady
OpenEMR