Counne says "tightened NLP is infeasible"

[svigerske]

Issue created by migration from Trac.

Original creator: thofrei

Original creation time: 2015-05-18 12:39:37

Assignee: @merraksh

Version:

Keywords: infeasible Couenne tightened NLP

Dear CoinOr

I'm trying to run a model in pyomo. One version of the model is using binary variables, which is why I'd like to use Couenne to solve it. In a second version of the model I have predefined these variables as parameters. I can solve the second version with Ipopt or Couenne and get a optimal solution. When I try to run the first version (which is basically the same model) with Couenne, however, I get the message:

Warning, tightened NLP is infeasible problem infeasible
ERROR: Unexpected exception while running model. [Errno 2] No such file or directory: '/tmp/tmpGAAINb.pyomo.sol'

Do you know, what this means and how I can get my model to work? I am using Couenne 0.4.

Thank you

[svigerske]

Comment by @merraksh created at 2015-05-18 23:18:32

Hi,

I assume your second version has those parameters fixed to a given value. The output you get seems to suggest that the bound reduction procedure indeed finds (incorrectly) that the problem is infeasible. If the problem is not too large, you may try using an option file couenne.opt (which has to reside in the same directory from which you run Couenne) that contains one or more of the following lines:

feasibility_bt  no
redcost_bt      no
aggressive_fbbt no
optimality_bt   no

However, in order to solve the issue, it might be useful if you attach the problem.

Regards,

Pietro

[svigerske]

Attachment Gas_Model_free_dir_EG.py by thofrei created at 2015-05-19 07:44:33

Abstract Model

[svigerske]

Attachment Belgium_EntsoG.dat by thofrei created at 2015-05-19 07:44:49

Data for model

[svigerske]

Comment by thofrei created at 2015-05-19 07:48:15

Hi Pietro

Thank you very much for your answer. Unfortunately, this didn't solve the problem. I put a simplified version of the model and the input file in the attachment. The variables that are causing the problem call m.a_dir and m.p_dir. When I specify them manually as Parameters ipopt and couenne give me a result, so there should be a solution that is feasible. I hope that we can solve this

[svigerske]

Comment by @merraksh created at 2015-06-04 00:33:12

Hi,

unfortunately I am not very familiar with pyomo. Have you got a .mod or a .nl copy of the model, so that I can submit it to Couenne? I couldn't see any link between the Python file and the .dat file you sent.

[svigerske]

Comment by thofrei created at 2015-06-04 19:36:07

Hi Pietro

Unfortunately, I don't have these files and since I am very new to all this I don't know how to get them. I changed a lot on the model in the mean time and don't get this specific error message anymore. But it's still the case that I get an optimal solution with ipopt, but couenne tells me that the problem is infeasible. The only difference in the couenne case is that I change two parameters to binary variables. So the solution should be reached if the variables get to the right values. Something strange that I realized is that with couenne I get errors such as "can't evaluate pow(0,0.5)" or /0 errors if I don't initialize the values although I specified the model as such that these cases shouldn't be possible. But as I said, the problem of infeasibility remains even if I make the initialization....

Do you have an other idea, what I could try?

[svigerske]

Comment by @merraksh created at 2015-06-04 23:33:18

Hi,

the fact that no .sol file was produced suggests that infeasibility was proven at bound tightening, which is weird since the options I gave before turn off all bound tightening. You could try another option to relax the feasibility tolerance from 1e-6 to 1e-4, for instance:

feas_tolerance 1e-4

Finally, since Couenne only accepts .nl file I'm thinking that Pyomo must have a procedure to convert instances to that format. See for example

https://projects.coin-or.org/Coopr/browser/coopr.pyomo/trunk/coopr/pyomo/scripting/convert.py?rev=9082

If you could somewhat modify the source file to use the convert function and produce a .nl file, I would certainly be able to help more.

[svigerske]

Comment by thofrei created at 2015-06-05 07:18:19

Hi Pietro

I found out how to get the .nl!! I hope we can figure out the problem with that. If you need more data please let me know

[svigerske]

Attachment Gas_Model_Couenne.nl by thofrei created at 2015-06-05 07:49:24

Model for solving with couenne

[svigerske]

Comment by @merraksh created at 2015-06-06 14:54:42

Hi,

it seems there is a bug in strong branching that reports an infeasible problem. Turning off strong branching seems to allow Couenne to continue, though the problem is quite large and I could not find a solution for a few hours. I'll try to fix this, but in the meantime you can try the following options:

feasibility_bt  yes
redcost_bt      yes
aggressive_fbbt yes
optimality_bt   yes

iterative_rounding_heuristic yes

output_level 2

bb_log_level 1
nlp_log_level 0
lp_log_level 0

log_num_obbt_per_level 1
log_num_abt_per_level 1
log_num_local_optimization_per_level 10

branching_object var_obj
variable_selection osi-simple

The last line is to turn off strong branching, while the rest is to increase the frequency of heuristics. Being a large problem, though, finding a solution won't be easy.

[svigerske]

Comment by thofrei created at 2015-06-07 10:14:13

Hi Pietro

Unfortunately, I get another error message with those options:

WARNING: converting to the 'pyomo solve' subcommand
[    0.00] Setting up Pyomo environment
[    0.00] Applying Pyomo preprocessing actions
[    0.08] Creating model
[    0.51] Applying solver
Couenne 0.4 --  an Open-Source solver for Mixed Integer Nonlinear Optimization
Mailing list: couenne`@`list.coin-or.org
Instructions: http://www.coin-or.org/Couenne
Error reading value for tag variable_selection from file.

Reformulation: 0.2 seconds
Restoration phase is called at point that is almost feasible,
  with constraint violation 3.455015e-11. Abort.
Restoration phase is called at point that is almost feasible,
  with constraint violation 3.455015e-11. Abort.
Loaded instance "/tmp/tmppfHVf7.pyomo.nl"
Constraints:          549
Variables:            651 (84 integer)
Auxiliaries:         2089 (165 integer)

Cloning TMINLP2TNLPQuadCuts.
NLP Heuristic: Optimality Based BT: 26 improved bounds
Probing: 20 improved bounds
NLP Heuristic: Launching IterativeRounding with parameters:
Max rounding iter: 10
Max feas point: 5
Base lbrhs: 15
Omega: 0.2
Max time firstcall: 300
starting feasibility IR
Exception of type: FAILED_INITIALIZATION in file "../../../../Ipopt/src/Algorithm/IpIpoptAlg.cpp" at line 547:
 Exception message: retval evaluated false: Error while obtaining initial iterates.

couenne: Error encountered in optimization.

        "Aborted"
A failure has occured but no starting point exists
Ipopt exited with error code -100 Unrecoverable exception
[   78.58] Processing results
    Number of solutions: 0
    Solver results file: results.yml
Problem loading solver results
ERROR: Unexpected exception while running model Gas_Model_Free_h_EG.py:
        Cannot load a SolverResults object with bad status: error

Do you know how to solve this?

[svigerske]

Comment by thofrei created at 2015-06-08 14:18:29

Sorry, I realized that it was due to my version of couenne. It runs now without this error message, but unfortunately still without getting me any results. Even if I reduce the number of the binary variables it calculates for a very long time

[svigerske]

Comment by @merraksh created at 2015-06-12 01:20:58

I do observe the same problem. Unfortunately it seems that it's hard to find a feasible solution. Couenne has a feasibility pump heuristic that, although quite computationally expensive, might help. You can activate it with options

feas_pump_heuristic yes
feas_pump_level 1

The second option is to limit the amount of calls to the heuristic, given that it can really take a lot of time, especially on such a large problem. Make sure SCIP is installed.

[svigerske]

Comment by thofrei created at 2015-06-19 12:34:43

Hi Pietro

I changed a lot of things on my model in the mean time (removing binary variables etc.) and now can solve some simple cases with couenne in a reasonable time. As soon as the cases get a bit larger, however couenne tries to reformulate the problem and then seems to be stuck. I will attache one of these cases. Can you please take a look at this?

Best,
Thomas

[svigerske]

Attachment IT_Model.nl by thofrei created at 2015-06-19 12:35:10

[svigerske]

Comment by @merraksh created at 2015-06-20 16:24:53

Hi,

Reformulation actually takes a short time. However, the problem is quite large and Couenne can't explore many BB nodes. Below is the output with stable/0.4. Strong branching indeed seems to slow it down a bit.

Couenne 0.4 --  an Open-Source solver for Mixed Integer Nonlinear Optimization
Mailing list: couenne`@`list.coin-or.org
Instructions: http://www.coin-or.org/Couenne

Reformulating problem: 0.2 seconds
Loaded instance "IT_Model.nl"
Constraints:         1066
Variables:           1074 (0 integer)
Auxiliaries:          273 (77 integer)

NLP Heuristic: Probing: 39 improved bounds
NLP Heuristic: Cbc0031I 59 added rows had average density of 2.7627119
Cbc0013I At root node, 59 cuts changed objective from 3.8639469 to 3.8639469 in 10 passes
Cbc0014I Cut generator 0 (Couenne convexifier cuts) - 415 row cuts average 2.7 elements, 1 column cuts (60 active)
Cbc0010I After 0 nodes, 0 on tree, 1e+50 best solution, best possible 3.8639469 (38.89 seconds)
[...]

[svigerske]

Comment by thofrei created at 2015-06-23 21:45:00

Hi Pietro

Sorry to bother you again, but I was wondering: have you had a look at strong branching? Is there a bug in the solver and could you figure it out? Because I tried all different options you gave me with different versions of couenne and unfortunately still have the result with some cases that Ipopt can give me an optimal solution, but couenne tells me that it is infeasible (even with feas_tolerance 1e-2)

[svigerske]

Comment by @merraksh created at 2015-07-03 22:46:22

Hi,

apologies for the late reply. I'm afraid I won't be able to do much before the second half of July, due to other projects.

Regards,
Pietro