Re: [Webware-discuss] reloading source modules

[Jason H. <ja...@pe...>]

On Wed, 2002-09-25 at 15:02, Tavis Rudd wrote:
> On September 25, 2002 12:56 pm, Jason Hildebrand wrote:
> > Cool!  It's basically the same idea that I had
> > (see http://webware.colorstudy.net/twiki/bin/view/Webware/OneShot), but
> > it's the application server that restarts itself, not some external
> > entity, which makes it much cleaner and more general (don't have to
> > worry about which user the appserver runs under, because it stays the
> > same).
> 
> Another big difference is that it will find changes in any module imported by 
> Webware or your application, rather than just changes in your application 
> path subtree.

Exactly -- it doesn't require any configuration (of application paths),
which is another win.

Here are a couple of ideas for improving the implementation, though.

One procedure I do fairly often is to 'cvs update' my application then
restart the server.  I'm not sure if your implementation will work
flawlessly in the situation where multiple files change -- there is a
race condition which I see:

1) File A is updated
2) Appserver notices that A has changed, shuts down and begins to
restart
3) In restarting, Appserver imports file B at time t.
4) File B is updated at time t+1.
5) The file monitor thread notices that File B has been imported,
records the mtime as t+1.

So the appserver is running using an out-of-date File B.  

One way to fix this would be to hook into Python's import routine to
record the mtime somewhere before the file is actually
loaded/initialized.  A search just turned up the standard module
ihooks.py, which looks promising.

Also, in the situtation where multiple files are changing, it might be
useful for the server _not_ to restart immediately, but for it to wait
until the files stop changing (think again of a 'cvs update' which
updates 100 source files over a one-minute period -- we'd want to
prevent the app server from restarting continuously during that minute,
and just do it once at the end).  This might be look like: 
(take note of the need_restart and saw_changed_file flags)


    def _fileMonitorThreadLoop(self,
                               getmtime=os.path.getmtime):

        monitoredFiles = self._monitoredFiles
        monitoredModules = self._monitoredModules
        need_restart = 0

        while self._autoReload:
            time.sleep(1)               #@@TR: softcode this later
            for mod in sys.modules.values():
                if not mod or mod in monitoredModules:
                    continue
                monitoredModules.append(mod)

                f2 = getattr(mod, '__orig_file__', False) # @@TR might
rename 
__orig_file__, this is used for cheetah and psp mods
                f = getattr(mod, '__file__', False)

                if f2 and f2 not in monitoredFiles.keys():
                    try:
                        monitoredFiles[f2] = getmtime(f2)
                    except OSError:
                        pass
                elif f and f not in monitoredFiles.keys():
                    try:
                        monitoredFiles[f] = getmtime(f)
                    except OSError:
                        pass

            saw_changed_file = 0
            for f, mtime in monitoredFiles.items():
                try:
                    if mtime < getmtime(f):
                        print '*** The file', f, 'has changed.  The
server is restarting now.'
                        need_restart = 1
                        saw_changed_file = 1
                except OSError:
                    print '*** The file', f, 'is no longer accessible 
The server is restarting now.'
                    need_restart = 1
                    saw_changed_file = 1


            if need_restart and not saw_changed_file:
                sys.stdout.flush()
                sys.stderr.flush()
                self._autoReload = False
                return self.restart()


Increasing the 1-second sleep time would improve this aspect, too, but
is a trade-off against response time.  I'm for making it configurable,
and letting the user decide (a 1-second default is probably fine).

I'll try merging your code into "stock" Webware and keep you posted.

-- 
Jason D. Hildebrand
ja...@pe...