Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0

Problems with syntax of GAMS
Post Reply
10mayerj
User
User
Posts: 4
Joined: 4 years ago

Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0

Post by 10mayerj »

Dear community,

I've now quite some experience in MPSGE modelling but currently encounter a problem that I've never had before.

First, I correctly calibrated a model to a small open economy SAM with one representative agent.
Afterwards, I disaggregated the SAM data of the private household side (expenditures and income) into 12 types of households using satellite data.
Then, I adjusted the MPSGE code and checked the benchmark with iterlim=0.
Now, I increased the iteration limit but surprisingly GAMS is having hard times to find a solution...

Has someone ever had a similar situation? Can anyone suggest a way out? (I tried setting starting levels for household variables, which does not help though).

Thanks in advance, JM
User avatar
Renger
Posts: 639
Joined: 7 years ago

Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0

Post by Renger »

Hi

You write that disaggregating the data causes problem when solving. If you set the iteration limit to zero, how big is the infeasibility of the disaggregated model? Could you attach the model?
Cheers
Renger
____________________________________
Enjoy modeling even more: Read my blog on modeling at The lazy economist
10mayerj
User
User
Posts: 4
Joined: 4 years ago

Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0

Post by 10mayerj »

Good morning Renger (here it is 8 a.m.),

first of all, thanks for your reply!

Regarding your comment, where can I find out the size of the infeasibility?

Unfortunately I can't attach the model due to institutional rules, but I can show you what I changed.

With one representative agent, the demand block of the model looks as follows:

$DEMAND:CONS s:0
D:PW Q:(sam("Output","CONS"))
D:PX("INV") Q:((sam("depr","CONS")+sam("save","CONS")))
E:PL Q:((-sam("L","CONS")))
E:PK Q:((-sam("K","CONS")))
E:PTR Q:((-sam("GOV_in","CONS")))

The agent is endowed with labour, capital and receives some public transfers (GOV_in is the public agent in the model) and uses income for consumption (PW is a bundle of various goods) and savings/investments.

With several households (using the disaggregated SAM and the set of household "phh"), the demand block now looks as follows:

$DEMAND:CONS(phh) s:0
D:PW(phh) Q:(sam("Output",phh))
D:PX("INV") Q:((sam("depr",phh)+sam("save",phh)))
E:PL Q:((-sam("L",phh)))
E:PK Q:((-sam("K",phh)))
E:PTR Q:((-sam("GOV_in",phh)))

Each agent "phh" consumes a specific bundle of goods "PW(phh)", the rest should be obvious.

Thanks again!
JM
User avatar
Renger
Posts: 639
Joined: 7 years ago

Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0

Post by Renger »

Hi

I would be 100% sure that the sum over the household demands is equal to the total, e.g.

Code: Select all

parameter check(*);

check("GOV_in") = sum(phh, sam("GOV_in",phh)) - sam("GOV_in","CONS");
...
and see if you assgined the values correctly.

You see the infeasibility in your output screen:

Code: Select all

 ** EXIT - iteration limit.
o
Major Iterations. . . . 0
Minor Iterations. . . . 0
Restarts. . . . . . . . 0
Crash Iterations. . . . 0
Gradient Steps. . . . . 0
Function Evaluations. . 1
Gradient Evaluations. . 1
Basis Time. . . . . . . 0.000000
Total Time. . . . . . . 0.187000
Residual. . . . . . . . 4.516419e-06
Or you can grab it with
solveinfo = yourmodelname.objval;

To check for infeasiblities with mpsge, see my blog entry on how to use a simple python script to display the infeasibilities (https://blog.modelworks.ch/looking-for- ... s-in-gams/).

Cheers
Renger

BTW. You can add code with </> in the menu (it will be nicely formatted). You could also send your model in a private message to me.
____________________________________
Enjoy modeling even more: Read my blog on modeling at The lazy economist
10mayerj
User
User
Posts: 4
Joined: 4 years ago

Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0

Post by 10mayerj »

Thanks again for your suggestions. I'm 100% sure that the SAM is balanced and GAMS reads in everything correctly from Excel.

The infeasibility output looks as follows:

Code: Select all

 ** EXIT - iteration limit.
o
Major Iterations. . . . 0
Minor Iterations. . . . 0
Restarts. . . . . . . . 0
Crash Iterations. . . . 0
Gradient Steps. . . . . 0
Function Evaluations. . 1
Gradient Evaluations. . 1
Total Time. . . . . . . 1.918000
Residual. . . . . . . . 3.366843e-04
When I run your python script for my .lst file, I get the following:

Code: Select all

runfile('T:/GitProjects/Py_LookForINFES/Py_LookForINFES.py', wdir='T:/GitProjects/Py_LookForINFES')
19796
Single-value variables: 

  Variable  Level   Marginal
0       FX  1.000  -5.587E-5
1     WGOV  1.000  3.3982E-5
2      PFX  1.000  2.3890E-5
3    PWGOV  1.000  6.6292E-5
4       PK  1.000  6.6743E-5


Multiple-value variables: 

   Variable Element  Level   Marginal
0         X    MACA  1.000  1.2137E-5
1         X    ELYs  1.000  3.7000E-5
2         X  ROADFT  1.000  -1.513E-5
3         X    ACCO  1.000  -1.027E-5
4         X    REAL  1.000  -5.252E-5
5         X    EDUC  1.000  1.1349E-5
6         X     INV  1.000  2.6750E-5
7         D    META  1.000  2.0000E-5
8         D    MACA  1.000  2.0000E-5
9         D    MAVE  1.000  1.1000E-5
10        D    ELYs  1.000  -5.400E-5
11        D    BUIL  1.000  -1.400E-5
12        D    CONT  1.000  -3.600E-5
13        D    TRWH  1.000  -4.000E-5
14        D    ACCO  1.000  3.0000E-5
15        D    SFIN  1.000  4.0000E-5
16        D    PUBL  1.000  -3.000E-5
17        D    EDUC  1.000  -2.500E-5
18        G    FEXT  1.000  5.0000E-5
19        G    FOOD  1.000  5.0000E-5
20        G    META  1.000  4.0000E-5
21        G    MAME  1.000  -3.000E-5
22        G    MACA  1.000  -2.000E-5
23        G    MAVE  1.000  -1.100E-5
24        G    ELYs  1.000  5.3000E-5
25        G    BUIL  1.000  5.5000E-5
26        G    CONT  1.000  -1.300E-5
27        G    TRWH  1.000  7.0000E-5
28        G    TRRE  1.000  -4.700E-5
29        G    ACCO  1.000  -4.000E-5
..      ...     ...    ...        ...
32        G    EDUC  1.000  5.6000E-5
33        G    HEAL  1.000  -4.500E-5
34        W  ThQ_ur  1.000  -3.542E-5
35        W  FoQ_ur  1.000  1.8903E-5
36        W  SeQ_pe  1.000  2.9799E-5
37        W  ThQ_pe  1.000  -4.375E-5
38        W  FoQ_pe  1.000  2.3299E-5
39       PX     INV  1.000  -4.636E-5
40       PG    FEXT  1.000  -2.898E-5
41       PG    FOOD  1.000  -4.609E-5
42       PG    META  1.000  -4.048E-5
43       PG    MAME  1.000  3.0705E-5
44       PG    MAVE  1.000  -3.051E-5
45       PG    ELYs  1.000  -3.186E-5
46       PG    BUIL  1.000  -7.447E-5
47       PG    CONT  1.000  4.5641E-5
48       PG    TRWH  1.000  -4.136E-5
49       PG    TRRE  1.000  4.8829E-5
50       PG    ACCO  1.000  1.8086E-5
51       PG    SFIN  1.000  4.0791E-5
52       PG    REAL  1.000  2.5735E-5
53       PG    EDUC  1.000  -5.523E-5
54       PG    HEAL  1.000  3.6881E-5
55       PW  ThQ_ur  1.000  2.8911E-5
56       PW  FoQ_ur  1.000  -6.015E-5
57       PW  SeQ_su  1.000  1.6000E-5
58       PW  FoQ_su  1.000  -1.600E-5
59       PW  SeQ_pe  1.000  -4.103E-5
60       PW  ThQ_pe  1.000  5.0219E-5
61       PW  FoQ_pe  1.000  -4.608E-5

[62 rows x 4 columns]
Correct me if I'm wrong, but marginals seem to be low enough. If I increase the iterlim, GAMS aborts after a while and I get now 'other error' with message 'locally infeasible'. I really don't understand why GAMS is not able to find a solution...

Best, JM
User avatar
gamsadmin
Site Admin
Site Admin
Posts: 16
Joined: 7 years ago

Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0

Post by gamsadmin »

Hi JM

It seems your SAM is not perfectly balanced. 1E-4 shows that there are balance problems. The solver tries to find a solution with the maximum infeasibility below the tolerance level (something like 1E-7), so it can't find a solution because the imbalances are too big and a few imbalances can't be resolved.
Did you scale your SAM to numbers around 1 by dividing your SAM accordingly? If your model solved correctly using the aggregated SAM with an infeasibility around 10^-6.

I would suggest the following:
- Scale your SAM, so the infeasibility is around 1E-6 and see if it solves.
- If your SAM is already scaled, I would use a least-square procedure to get the SAM balanced to higher precision.
- Another option is to set the tolerance level to 1E-3, but if that is necessary in my models, I would use the LS-approach and balance the SAM.
Cheers
Renger
10mayerj
User
User
Posts: 4
Joined: 4 years ago

Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0

Post by 10mayerj »

Hi,

thanks a lot for your hints. I finally understand where the problems emerge!

And yes, I scaled the SAM already right from the beginning (by a factor of 1000). When I increase scaling by another order of magnitude, the solver returns an optimal solution (but it takes ~2 minutes). I'm now thinking about rebalancing by least-square methods (because I'm also skeptical about increasing tolerance levels). Do you have a suggestion (or a reference) for doing so "quickly"?

Thanks again,
cheers,
JM
User avatar
Renger
Posts: 639
Joined: 7 years ago

Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0

Post by Renger »

Hi
You could have a look at the paper by Robinson (they also provide the code). .
Cheers
Renger
____________________________________
Enjoy modeling even more: Read my blog on modeling at The lazy economist
Post Reply