CONOPT4 and MOSEK option files

Solver related questions
fermar
User
User
Posts: 7
Joined: 5 years ago

CONOPT4 and MOSEK option files

Post by fermar » 2 weeks ago

I'm running a model all in conic forms. My colleagues using CONOPT4 (and the options below) are getting sensible results (calibration and appropriate scenarios responses). But as I switch solvers to MOSEK (using exactly the same model), results are not looking good at all ( no scenarios responses, e. g.).
Thus, which options should I use in MOSEK that are equivalent to those below?

* -------------------------------------------------
*
* Option file for Solving Models
*
* -------------------------------------------------
* Maximum and minimum jacobian element
rtmaxj = 1.00e+20
* internal value of infinity
rtmaxv = 1.00e+30
* scaling
lsscal = t
lfscal = 20
* limit for slow progress
*lfnicr = 3000
lfnicr = 4000
* limit for superbasics
lfnsup = 10000
* iteration log frequency
lfilog = 100
* iteration log frequency for SLP iterations
lfilos = 100
*
rtnwtr = 1.00e-5
rtnwmi = 1.00e-8
*rtnwmi = 2.50e-5
* increases the number of stalled iterations
*lfstal = 200
lfstal = 500
*workfactor=2

User avatar
dirkse
Moderator
Moderator
Posts: 194
Joined: 6 years ago
Location: Fairfax, VA

Re: CONOPT4 and MOSEK option files

Post by dirkse » 2 weeks ago

Fermar,

You did not include your model, or any sort of GAMS code to use to reproduce your results, so it's difficult to say exactly what is needed. Looking at your CONOPT options, the most striking thing to me is that you have

Code: Select all

* Maximum and minimum jacobian element
rtmaxj = 1.00e+20
* internal value of infinity
rtmaxv = 1.00e+30
Probably you set these because CONOPT suggested you do so, which it would do if your model was very poorly scaled. In such a case, you would be better off asking how you can properly scale your model so that these options are not needed. You can read about this here:

https://www.gams.com/41/docs/UG_NLP_Goo ... ns_Scaling
https://www.gams.com/41/docs/S_CONOPT.h ... PT_SCALING

It's tempting to think or hope that different solvers will always produce the same results when given the same inputs, but this is simply not the case. It often happens that we reach this happy situation - aided by a convex model, good scaling, a good initial point, etc. - but it is not to be assumed.

HTH,

-Steve

eda
User
User
Posts: 10
Joined: 5 years ago
Location: Copenhagen
Contact:

Re: CONOPT4 and MOSEK option files

Post by eda » 2 weeks ago

The question is cross-posted at

https://groups.google.com/g/mosek/c/-H4QZptSo80

(My best guess is that the conversion of the model to conic form is not correct.)

Erling
Erling @ mosek.com

fermar
User
User
Posts: 7
Joined: 5 years ago

Re: CONOPT4 and MOSEK option files

Post by fermar » 2 weeks ago

Hi, thanks for the reply. I attach the MOSEK specification only as I don't have the CONOPT license, it was run in a different computer.

On line 244 is the start of the first stage of positive math programming to get marginals for calibration constraints. On line 395 is the start of the second stage to construct the exponential cost curve.

On line 523 is the third stage where the exponential cost curve is entered in the OF, I calibrate for the number of cows. And it calibrates.

The problems start on line 716 for the policy model where I enter the sim_price_methane which is a proxy for emissions pricing (methane). The number of cows variable does not react at all, if I increase the price the reactions are wild for some units but not for others. After extensive exploration of scaling problems, I can’t find the source .

I appreciate every feedback

M



PS: the gdx file is for 10 units only, the original file is more than 10 thousand
NOM_Input_v2_gf.gdx
(9.88 KiB) Downloaded 14 times
NOM_16112022_gf.gms
(37.46 KiB) Downloaded 19 times

eda
User
User
Posts: 10
Joined: 5 years ago
Location: Copenhagen
Contact:

Re: CONOPT4 and MOSEK option files

Post by eda » 2 weeks ago

I posted a reply in the Mosek Google Group thread.
Erling @ mosek.com

User avatar
bussieck
Moderator
Moderator
Posts: 859
Joined: 6 years ago

Re: CONOPT4 and MOSEK option files

Post by bussieck » 2 weeks ago

Erling, here are the log and task files. One is with explicit user scaling while the other one is without user scaling. -Michael
scaled.log
(7.57 KiB) Downloaded 15 times
unscaled.log
(8.96 KiB) Downloaded 20 times
tasks.zip
(12.94 KiB) Downloaded 15 times

eda
User
User
Posts: 10
Joined: 5 years ago
Location: Copenhagen
Contact:

Re: CONOPT4 and MOSEK option files

Post by eda » 2 weeks ago

The problem you have is kind of nasty and v10.0 struggles with it. It does not solve your problem. One thing I can see is that the norm of the primal solution is quite large i.e. 2e7 and that is bad.

However, the current development version of Mosek solves the problem quite OK as you can see below. Most likely it is some additional stuff in the presolve that makes your problem solvable.
I am looking into that and maybe that can tell us what it is in your problem that makes it hard.

26 8.8e-11 1.6e-14 6.4e-17 1.00e+00 1.025095051e+07 1.025095051e+07 1.5e-16 0.03
Optimizer terminated. Time: 0.03

Open file 'out\jobs\runmosek\stat\rue\default-scaled.stat'

Interior-point solution summary
Problem status : PRIMAL_AND_DUAL_FEASIBLE
Solution status : OPTIMAL
Primal. obj: 1.0250950506e+07 nrm: 2e+07 Viol. con: 6e-11 var: 0e+00 cones: 0e+00
Dual. obj: 1.0250950505e+07 nrm: 8e+04 Viol. con: 0e+00 var: 1e-06 cones: 0e+00
Erling @ mosek.com

eda
User
User
Posts: 10
Joined: 5 years ago
Location: Copenhagen
Contact:

Re: CONOPT4 and MOSEK option files

Post by eda » 2 weeks ago

Btw Mosek chooses to solve the dual problem which seems to cause issues in 10.0. If you force Mosek to solve the primal problem it seems you get a decent solution.

So you should set the parameter

MSK_IPAR_SIM_SOLVE_FORM

to force Mosek to solve the primal problem.

See

https://www.gams.com/latest/docs/S_MOSE ... EK_OPTIONS

for details.
Erling @ mosek.com

fermar
User
User
Posts: 7
Joined: 5 years ago

Re: CONOPT4 and MOSEK option files

Post by fermar » 2 weeks ago

Thanks for that.
My MOsek version does solve the model. But the simulation in the fourth stage does not produce adequate responses.
Any hint on the major scaling issues coming from?

eda wrote:
2 weeks ago
The problem you have is kind of nasty and v10.0 struggles with it. It does not solve your problem. One thing I can see is that the norm of the primal solution is quite large i.e. 2e7 and that is bad.

However, the current development version of Mosek solves the problem quite OK as you can see below. Most likely it is some additional stuff in the presolve that makes your problem solvable.
I am looking into that and maybe that can tell us what it is in your problem that makes it hard.

26 8.8e-11 1.6e-14 6.4e-17 1.00e+00 1.025095051e+07 1.025095051e+07 1.5e-16 0.03
Optimizer terminated. Time: 0.03

Open file 'out\jobs\runmosek\stat\rue\default-scaled.stat'

Interior-point solution summary
Problem status : PRIMAL_AND_DUAL_FEASIBLE
Solution status : OPTIMAL
Primal. obj: 1.0250950506e+07 nrm: 2e+07 Viol. con: 6e-11 var: 0e+00 cones: 0e+00
Dual. obj: 1.0250950505e+07 nrm: 8e+04 Viol. con: 0e+00 var: 1e-06 cones: 0e+00

User avatar
bussieck
Moderator
Moderator
Posts: 859
Joined: 6 years ago

Re: CONOPT4 and MOSEK option files

Post by bussieck » 2 weeks ago

Erling,

MOSEK succeeds with the third model in fermar's GAMS file when using MSK_IPAR_SIM_SOLVE_FORM MSK_SOLVE_PRIMAL. The fourth model still fails (I tried both primal and dual). I have attached log and task.

-Michael
FOurthStage.zip
(8.65 KiB) Downloaded 15 times

Post Reply