Re: [Ffnet-users] ffnet question

[Marek W. <mwojc@p.lodz.pl>]

Dnia wtorek 11 listopad 2008, napisałeś:
> Dear Mr Wojciechowski,
>
> Please excuse the unsolicited contact, I found your email address in the
> copyright notice of (the excellent) ffnet.f which I downloaded via
> scipy.org ..
>
> I am evaluating neural network implementations for a private project and
> have a question which I would very much appreciate your answer to.
>
> I have a requirement to have an unseeen validation set alongside the
> training set.  Periodically during training I need to compute the error
> against the unseeen validation set to test whether the network has true
> predictive capabilities as a safeguard against over-fitting.  The objective
> is to select the network which gives the best balance between the overall
> error and the error against the unseen data.
>
> I have not yet found a network implementation which allows this and am
> contemplating creating it for myself.  In practice all I think I need is
> the ability to provide a callback and a parameter which governs how many
> training iterations pass between calls.  My question is whether you think
> it is possible to extend the implementation you have provided to have this
> additional feature, and if so, where would it best be placed?
>
> My preliminary investigation suggests that it would require alteration to
> _ffnet.f.  Not being a Fortran expert I am not sure whether the language
> even supports callbacks, but if it does I am sure I can figure out how to
> do it.
>
> I would appreciate any guidance you feel able to provide and look forward
> to hearing from you.
>
> Kind regards
> Simon Palmer

Hallo!
You can do this externally in python, and i think this is the best way to do 
so:

validation_error = 1e20
best_weights = net.weights.copy()
for iter in xrange(0, maxiter, 50):
    net.train_tnc(input, target, maxiter = iter, messages = 1)
    verr = net.sqerror(validation_input, validation_target)
    if verr < validation_error:
        best_weights = net.weights.copy()
        validation_error = verr
        print "Validation OK"
   else:
       net.weights = best_weights.copy()
       print "Validation failed. Keeped weights from previous step."
       break

Obviously this is raw recipe and it can be extended. Maybe you're right i 
should add something like that in ffnet...

Greetings,
-- 
Marek Wojciechowski

P.S. I invite you at ffn...@li...