Topic: OpenOpt does not give consistant results

I have tried to use OpenOpt but I have some troubles because it does not always give the same result even though the input is exactly the same. Is there some kind of random initial guess build into the optimizer or am I missing something.

Below I have an example

from matplotlib.pylab import *
from openopt import NLP
import numpy as np
from numpy import linspace
from scipy import interpolate

def residual(p, r, y):
    tck = interpolate.splrep(np.concatenate(([r[0]], p[0:len(p)/2], [r[-1]])), np.concatenate(([y[0]], p[len(p)/2::], [y[-1]])))
    yFit = interpolate.splev(r, tck)

    resid = sum((y-yFit)**2)
      return resid

def FitDistribution(r, y):
    rP = linspace(r[0], r[-1], 12)
    yP = np.interp(rP, r, y)
    
    p0 = np.concatenate((rP[1:-1], yP[1:-1]))
        
        # form box-bound constraints lb <= x <= ub
    lb = np.concatenate((np.zeros(len(rP)-2), np.ones(len(rP)-2)*(-np.inf))) # lower bound
    ub = np.concatenate((np.ones(len(rP)-2)*r[-1], np.ones(len(rP)-2)*(np.inf))) # upper bound
    
    ftol = 10e-5
    
    f = lambda p: residual(p, r, y)        
           
    p = NLP(f, p0, lb=lb, ub=ub, maxIter=5500, iprint=20, ftol=ftol)
    solution = p.solve('ralg', plot = False)    

    pvec = solution.xf
    
        tck = interpolate.splrep(np.concatenate(([r[0]], pvec[0:len(pvec)/2], [r[-1]])), np.concatenate(([y[0]], pvec[len(pvec)/2::], [y[-1]])))
    yFit = interpolate.splev(r, tck)
    
    return yFit

r = linspace(1,100)
y = r**2

yFit = FitDistribution(r, y)

plot(r, y, label = 'func')
plot(r, yFit, label = 'fit')
legend(loc=0)
grid()
show()

In this case as you can see in the example I am trying to represent a curve with a cubic spline. As input to the spline I have 12 points – two end points and 10 inner points. I am trying to optimize these 10 inner points (both in x and y direction) in order to represent the curve as good as possible with this spline. But as said it does not always ends up with the same result. Sometimes you can run the code several times with the same result but suddenly the result differ.

What can be the reason for this?

Kind Regards,
Casper Skovby

Re: OpenOpt does not give consistant results

I have run your code for 10 times and obtained the same results:
  121  7.289e-05            -100.00
istop: 4 (|| F[k] - F[k-1] || < ftol)
Solver:   Time Elapsed = 1.17     CPU Time Elapsed = 1.07
objFunValue: 7.2891882e-05 (feasible, MaxResidual = 0)
Which OS do you use? Let me guess by myself... Windows, right? In Linux my code always return same results, while in Windows results (for example for my /examples/nlp_1.py) sometimes differ. I don't know what's the deuce, I thought somewhere arrays created by numpy.empty() are not correctly filled, but reviewing OO kernel  code haven't revealed any places like this, and would they be present, it would sometimes yield similar differences in Linux, while I haven't seen them for a long time (several years) since last bug of this kind was fixed.

--------------------------------------------------(Signature)--------------------------------------------------------
Currently I'm busy with a paid work and thus have a limited time for OpenOpt suite support on the forum.

Re: OpenOpt does not give consistant results

Thank you for your quick reply. Yes it is windows XP. I just tried to run it again. I had to run it 6 times before I got different results: I got these two results:

93  7.451e-04            -100.00
istop: 4 (|| F[k] - F[k-1] || < ftol)
Solver:   Time Elapsed = 0.72   CPU Time Elapsed = 0.720413895787
objFunValue: 0.00074507397 (feasible, MaxResidual = 0)

113  7.538e-05            -100.00
istop: 4 (|| F[k] - F[k-1] || < ftol)
Solver:   Time Elapsed = 0.88   CPU Time Elapsed = 0.869201182364
objFunValue: 7.5376233e-05 (feasible, MaxResidual = 0)

In this example the result is quite similar but making the problem more complex and adding constraints the results can differ quite a lot. And it is not always only two solutions it jumps between. I will try to run it in Linux but at some point the code is intended to be used on a Windows platform so I think I have to solve the problem somehow. Do you have any ideas?

Kind Regards,
Casper

Re: OpenOpt does not give consistant results

Unfortunately, I have no ideas yet and busy with other work to be done. Have you tried using other than ralg solvers?

--------------------------------------------------(Signature)--------------------------------------------------------
Currently I'm busy with a paid work and thus have a limited time for OpenOpt suite support on the forum.

Re: OpenOpt does not give consistant results

Yes, I have tried most of the available ones. It also give inconsistant results.

Kind Regards,
Casper

Re: OpenOpt does not give consistant results

have you tried gradient-free solvers like scipy_cobyla or bobyqa?

--------------------------------------------------(Signature)--------------------------------------------------------
Currently I'm busy with a paid work and thus have a limited time for OpenOpt suite support on the forum.

Re: OpenOpt does not give consistant results

I tried scipy_cobyla which also lead to inconsistant results. bobyqa requires nlOpt and I have some troubles installing this so I have not tried this yet.

Re: OpenOpt does not give consistant results

Could you try running scipy.optimize cobyla directly, w/o OpenOpt API?

--------------------------------------------------(Signature)--------------------------------------------------------
Currently I'm busy with a paid work and thus have a limited time for OpenOpt suite support on the forum.

Re: OpenOpt does not give consistant results

I just  running scipy.optimize cobyla directly. I got the same problems. What does that indicate?

Re: OpenOpt does not give consistant results

Probably something with NumPy build on Windows (less likely something with CPython).
You could point numpy-user or scipy-dev mail list to the discussion, maybe they know what it is due to.

--------------------------------------------------(Signature)--------------------------------------------------------
Currently I'm busy with a paid work and thus have a limited time for OpenOpt suite support on the forum.

Re: OpenOpt does not give consistant results

Now I have tried to run the code on a Linux platform. I got the same problem when using OpenOpt but running with scipy.optimize.cobyla I do not have the problem. Can you try to run the code below? It is almost similar to the previous - only the start guess is different. When I run it either in Windows or Linux it gives more different results than the previous.

from matplotlib.pylab import *
from openopt import NLP
import numpy as np
from numpy import linspace
from scipy import interpolate


def residual(p, r, y):
    tck = interpolate.splrep(np.concatenate(([r[0]], p[0:len(p)/2], [r[-1]])), np.concatenate(([y[0]], p[len(p)/2::], [y[-1]])))
    yFit = interpolate.splev(r, tck)

    resid = sum((y-yFit)**2)
      return resid

def FitDistribution(r, y):
    rP = linspace(r[0], r[-1], 12)
    yP = np.interp(rP, r, y)*10
    
    p0 = np.concatenate((rP[1:-1], yP[1:-1]))
        
        # form box-bound constraints lb <= x <= ub
    lb = np.concatenate((np.zeros(len(rP)-2), np.ones(len(rP)-2)*(-np.inf))) # lower bound
    ub = np.concatenate((np.ones(len(rP)-2)*r[-1], np.ones(len(rP)-2)*(np.inf))) # upper bound
    
    ftol = 10e-5
    
    f = lambda p: residual(p, r, y)        
    
    def    constraint(x):
        return 1
           
    p = NLP(f, p0, lb=lb, ub=ub, maxIter=5500, iprint=20, ftol=ftol)
    solution = p.solve('ralg', plot = False)    
    pvec = solution.xf
        
    
    tck = interpolate.splrep(np.concatenate(([r[0]], pvec[0:len(pvec)/2], [r[-1]])), np.concatenate(([y[0]], pvec[len(pvec)/2::], [y[-1]])))
    yFit = interpolate.splev(r, tck)
    
    return yFit

r = linspace(1,100)
y = r**2

yFit = FitDistribution(r, y)

plot(r, y, label = 'func')
plot(r, yFit, label = 'fit')
legend(loc=0)
grid()
show()

Re: OpenOpt does not give consistant results

With ralg I got each time on iter 607

 
The debugged program raised the exception unhandled ValueError
" Error on input data"
File: /usr/lib/python2.7/dist-packages/scipy/interpolate/fitpack.py, Line: 467

With other solvers I got the same error even on first iterations.

--------------------------------------------------(Signature)--------------------------------------------------------
Currently I'm busy with a paid work and thus have a limited time for OpenOpt suite support on the forum.

Re: OpenOpt does not give consistant results

My overall impression: it's a OS-dependend bug, probably related to ctypes, cython or f2py, since it triggers in both
* ralg
* scipy.optimize bobyqa (M.Powell solver with fortran code), which doesn't seem to use numpy arrays in it's code

As for scipy.interpolate, I have mentioned in FD documentation that its quality seems to be quite premature.

--------------------------------------------------(Signature)--------------------------------------------------------
Currently I'm busy with a paid work and thus have a limited time for OpenOpt suite support on the forum.

Re: OpenOpt does not give consistant results

Probably it is. I also tried to run OpenOpt with scipy_cobyla and lincher in Linux which turned out to be stable. Do you have any suggestions of how to solve this bug. If you do not have the time to look deeper into this problem who can I then contact?

Re: OpenOpt does not give consistant results

Have you tried it with mma, auglag? It would better locate where the bug is hiding.

You could contact to scipy-dev or numpy-user mail lists (visit scipy.org).

Since scipy_cobyla works different in Windows and Linux, probably something with f2py is wrong (or something in libraries it involves). Despite scipy.interpolate is quite premature, bug is hardly there, since it triggers with code that doesn't involve the module.

--------------------------------------------------(Signature)--------------------------------------------------------
Currently I'm busy with a paid work and thus have a limited time for OpenOpt suite support on the forum.

Re: OpenOpt does not give consistant results

In Linux mma and auglag give consistant results (they both give the same result...) What does that imply? I have not managed to install nlopt in Windows yet.

I will try to contact scipy-dev.