Menu
▾
▴
#1974 Any msys utility called from Apache->cgi->python freezes
assigned
Support
none
Unknown
False
2013-05-18
2013-05-17
No
#!/Python27/python -u
# File name: test.py
import subprocess
import datetime
def log(strIn):
t = str(datetime.datetime.now())
print(t + ": " + strIn)
def myTest():
print "Content-type: text/html"
print
print "<html><body><pre>"
log("start")
process = subprocess.Popen(["c:/Program Files/msys/1.0/bin/date.exe"], stdout = subprocess.PIPE)
# process = subprocess.Popen(["c:/temp/wbash/date.exe"], stdout = subprocess.PIPE)
log("#1")
print process.communicate()[0]
log("end")
print "</pre></body></html>"
myTest()
I tried to use MSYS unix utilities called from apache within python cgi script. The script is shown above. It doesn't matter i call bash, sed, date using Popen in that example. Called utility "freezes" for 14 s, in process explorer i always see it has 4 threads and no CPU consumption.
When i call python script above manually outside of apache script, everything works in expected way.
When I use other commands then MSYS commands, everything goes well too (within apache cgi/python). I migrated from MSYS to Windows bash (cygwin) to achieve solution.
Called from python (no cgi script solution), everything lasts 1/10 of second:
ActivePython 2.7.2.5 (ActiveState Software Inc.) based on
Python 2.7.2 (default, Jun 24 2011, 12:21:10) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
import test
Content-type: text/html
<html><body><pre>
2013-05-17 13:20:23.203000: start
2013-05-17 13:20:23.234000: #1
Fri May 17 13:20:23 SEAST 2013
2013-05-17 13:20:23.312000: end
</pre></body></html>
Python cgi script called from apache lasts 14 seconds, doesn't take CPU:
2013-05-17 13:20:58.656000: start
2013-05-17 13:20:58.671000: #1
Fri May 17 13:21:22 SEAST 2013
2013-05-17 13:21:22.734000: end
My configuration:
Windows 7 Ultimate/Oracle VirtualBox/Windows 7 Ultimate
ActivePython 2.7.2.5
Apache httpd-2.0.64-win32-x86-no_ssl.msi
MSYS-1.0.11.exe
Upgrade and then report back. Issue reports against an old version do not fly well with any development community.
Hello, I can't use mingw-get since my connection is limited and there's a lot of files being downloaded. However I started from a scratch in c:/MinGW, downloaded manually msys 1.0.18 and other fresh files needed (you can see below), run pi.bat, edited PATH, restarted Windows. Result is very same like described before.
Your use case isn't something we support much, it is well beyond the requirements for the Minimal SYStem mission statement. But it is worth declaring an issue; however, Cesar has little time to look. Alexey is working on MSYS2 based on more current release of Cygwin so eventually this may work but not anytime soon.
One thing that might help is to make sure that the pipes are flushed in your Python script. I'm guessing that perhaps Apache hasn't seen the end of communication to process the data.
I've no idea. Yesterday i stumbled upon this link: 'Running subprocess.Popen under apache+mod_wsgi is always returning an error'. Perhaps there's a chance it is related bug.
But then i tried other binaries, eg. from windows/system32 or from cygwin and it is being processed well. Therefore i tend to believe something in MinGW is broken.
Apache server I use is not seen from Internet, Windows is restarted, output of script and script itself is minimal. And the result is always stable.
I doubt that the link that you provided has any bearing on this issue. There may indeed be a problem in MSYS but your use case scenario is under supported.
I suggest flushing buffers because of a known issue which causes pty issues (which are implemented using pipes) when a non MSYS executable is involved. I do not know if that will resolve the issue but you'll need to try it to determine if that will help.
Last edit: Earnie Boyd 2013-05-17
Thank you for response. I was playing with flushing. I was playing again with using files instead of writing to stdout/stderr/PIPEs. The result's always the same: MSYS app freezes when called by means of Apache.
Could it be something that MSYS app is looking for stdin and can't handle it? Because I can play with sys.stdin file descriptor while i'm in python. When python script's called from apache, it says stdin: bad file descriptor.
I don't press to resolve the bug, I just regret there's such bug and even bug which is very difficult to figure out, therefore someone else can get stuck too.
Last edit: Miroslav Stibor 2013-05-18
BTW, Keith is working on reducing the download of the catalogue files so that ``mingw-get update'' will only download those that have a new timestamp. That should help eliminate the bulk downloads.
That's already working, in the latest snapshot on my project page. However, it doesn't help for the first time 'mingw-get update', because the aggregate catalogue furnished by mingw-get-inst.exe lacks the necessary issue time stamp keys in the package list entries.
I'm currently working on mingw-get-setup.exe, which will supplant mingw-get-inst.exe; (I don't plan to update mingw-get-inst.exe ever again). mingw-get-setup.exe will furnish an aggregate catalogue as a free standing tarball, (bundled with an additional temporary DLL, for post-install use by mingw-get-setup.exe itself). Would it help if I were to post an advance copy of that?
The release often method might help stamp out the bugs. I know you're trying to prevent a stampede of crying users but if you're close enough to actually being slightly useful then go ahead and pre-release it.
We'll see if Cesar can get to looking into it before MSYS2 is out. I'll doubt if that will happen though.