Dear community,
I've now quite some experience in MPSGE modelling but currently encounter a problem that I've never had before.
First, I correctly calibrated a model to a small open economy SAM with one representative agent.
Afterwards, I disaggregated the SAM data of the private household side (expenditures and income) into 12 types of households using satellite data.
Then, I adjusted the MPSGE code and checked the benchmark with iterlim=0.
Now, I increased the iteration limit but surprisingly GAMS is having hard times to find a solution...
Has someone ever had a similar situation? Can anyone suggest a way out? (I tried setting starting levels for household variables, which does not help though).
Thanks in advance, JM
Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0
Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0
Hi
You write that disaggregating the data causes problem when solving. If you set the iteration limit to zero, how big is the infeasibility of the disaggregated model? Could you attach the model?
Cheers
Renger
You write that disaggregating the data causes problem when solving. If you set the iteration limit to zero, how big is the infeasibility of the disaggregated model? Could you attach the model?
Cheers
Renger
____________________________________
Enjoy modeling even more: Read my blog on modeling at The lazy economist
Enjoy modeling even more: Read my blog on modeling at The lazy economist
Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0
Good morning Renger (here it is 8 a.m.),
first of all, thanks for your reply!
Regarding your comment, where can I find out the size of the infeasibility?
Unfortunately I can't attach the model due to institutional rules, but I can show you what I changed.
With one representative agent, the demand block of the model looks as follows:
$DEMAND:CONS s:0
D:PW Q:(sam("Output","CONS"))
D:PX("INV") Q:((sam("depr","CONS")+sam("save","CONS")))
E:PL Q:((-sam("L","CONS")))
E:PK Q:((-sam("K","CONS")))
E:PTR Q:((-sam("GOV_in","CONS")))
The agent is endowed with labour, capital and receives some public transfers (GOV_in is the public agent in the model) and uses income for consumption (PW is a bundle of various goods) and savings/investments.
With several households (using the disaggregated SAM and the set of household "phh"), the demand block now looks as follows:
$DEMAND:CONS(phh) s:0
D:PW(phh) Q:(sam("Output",phh))
D:PX("INV") Q:((sam("depr",phh)+sam("save",phh)))
E:PL Q:((-sam("L",phh)))
E:PK Q:((-sam("K",phh)))
E:PTR Q:((-sam("GOV_in",phh)))
Each agent "phh" consumes a specific bundle of goods "PW(phh)", the rest should be obvious.
Thanks again!
JM
first of all, thanks for your reply!
Regarding your comment, where can I find out the size of the infeasibility?
Unfortunately I can't attach the model due to institutional rules, but I can show you what I changed.
With one representative agent, the demand block of the model looks as follows:
$DEMAND:CONS s:0
D:PW Q:(sam("Output","CONS"))
D:PX("INV") Q:((sam("depr","CONS")+sam("save","CONS")))
E:PL Q:((-sam("L","CONS")))
E:PK Q:((-sam("K","CONS")))
E:PTR Q:((-sam("GOV_in","CONS")))
The agent is endowed with labour, capital and receives some public transfers (GOV_in is the public agent in the model) and uses income for consumption (PW is a bundle of various goods) and savings/investments.
With several households (using the disaggregated SAM and the set of household "phh"), the demand block now looks as follows:
$DEMAND:CONS(phh) s:0
D:PW(phh) Q:(sam("Output",phh))
D:PX("INV") Q:((sam("depr",phh)+sam("save",phh)))
E:PL Q:((-sam("L",phh)))
E:PK Q:((-sam("K",phh)))
E:PTR Q:((-sam("GOV_in",phh)))
Each agent "phh" consumes a specific bundle of goods "PW(phh)", the rest should be obvious.
Thanks again!
JM
Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0
Hi
I would be 100% sure that the sum over the household demands is equal to the total, e.g.
and see if you assgined the values correctly.
You see the infeasibility in your output screen:
Or you can grab it with
solveinfo = yourmodelname.objval;
To check for infeasiblities with mpsge, see my blog entry on how to use a simple python script to display the infeasibilities (https://blog.modelworks.ch/looking-for- ... s-in-gams/).
Cheers
Renger
BTW. You can add code with </> in the menu (it will be nicely formatted). You could also send your model in a private message to me.
I would be 100% sure that the sum over the household demands is equal to the total, e.g.
Code: Select all
parameter check(*);
check("GOV_in") = sum(phh, sam("GOV_in",phh)) - sam("GOV_in","CONS");
...
You see the infeasibility in your output screen:
Code: Select all
** EXIT - iteration limit.
o
Major Iterations. . . . 0
Minor Iterations. . . . 0
Restarts. . . . . . . . 0
Crash Iterations. . . . 0
Gradient Steps. . . . . 0
Function Evaluations. . 1
Gradient Evaluations. . 1
Basis Time. . . . . . . 0.000000
Total Time. . . . . . . 0.187000
Residual. . . . . . . . 4.516419e-06
solveinfo = yourmodelname.objval;
To check for infeasiblities with mpsge, see my blog entry on how to use a simple python script to display the infeasibilities (https://blog.modelworks.ch/looking-for- ... s-in-gams/).
Cheers
Renger
BTW. You can add code with </> in the menu (it will be nicely formatted). You could also send your model in a private message to me.
____________________________________
Enjoy modeling even more: Read my blog on modeling at The lazy economist
Enjoy modeling even more: Read my blog on modeling at The lazy economist
Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0
Thanks again for your suggestions. I'm 100% sure that the SAM is balanced and GAMS reads in everything correctly from Excel.
The infeasibility output looks as follows:
When I run your python script for my .lst file, I get the following:
Correct me if I'm wrong, but marginals seem to be low enough. If I increase the iterlim, GAMS aborts after a while and I get now 'other error' with message 'locally infeasible'. I really don't understand why GAMS is not able to find a solution...
Best, JM
The infeasibility output looks as follows:
Code: Select all
** EXIT - iteration limit.
o
Major Iterations. . . . 0
Minor Iterations. . . . 0
Restarts. . . . . . . . 0
Crash Iterations. . . . 0
Gradient Steps. . . . . 0
Function Evaluations. . 1
Gradient Evaluations. . 1
Total Time. . . . . . . 1.918000
Residual. . . . . . . . 3.366843e-04
Code: Select all
runfile('T:/GitProjects/Py_LookForINFES/Py_LookForINFES.py', wdir='T:/GitProjects/Py_LookForINFES')
19796
Single-value variables:
Variable Level Marginal
0 FX 1.000 -5.587E-5
1 WGOV 1.000 3.3982E-5
2 PFX 1.000 2.3890E-5
3 PWGOV 1.000 6.6292E-5
4 PK 1.000 6.6743E-5
Multiple-value variables:
Variable Element Level Marginal
0 X MACA 1.000 1.2137E-5
1 X ELYs 1.000 3.7000E-5
2 X ROADFT 1.000 -1.513E-5
3 X ACCO 1.000 -1.027E-5
4 X REAL 1.000 -5.252E-5
5 X EDUC 1.000 1.1349E-5
6 X INV 1.000 2.6750E-5
7 D META 1.000 2.0000E-5
8 D MACA 1.000 2.0000E-5
9 D MAVE 1.000 1.1000E-5
10 D ELYs 1.000 -5.400E-5
11 D BUIL 1.000 -1.400E-5
12 D CONT 1.000 -3.600E-5
13 D TRWH 1.000 -4.000E-5
14 D ACCO 1.000 3.0000E-5
15 D SFIN 1.000 4.0000E-5
16 D PUBL 1.000 -3.000E-5
17 D EDUC 1.000 -2.500E-5
18 G FEXT 1.000 5.0000E-5
19 G FOOD 1.000 5.0000E-5
20 G META 1.000 4.0000E-5
21 G MAME 1.000 -3.000E-5
22 G MACA 1.000 -2.000E-5
23 G MAVE 1.000 -1.100E-5
24 G ELYs 1.000 5.3000E-5
25 G BUIL 1.000 5.5000E-5
26 G CONT 1.000 -1.300E-5
27 G TRWH 1.000 7.0000E-5
28 G TRRE 1.000 -4.700E-5
29 G ACCO 1.000 -4.000E-5
.. ... ... ... ...
32 G EDUC 1.000 5.6000E-5
33 G HEAL 1.000 -4.500E-5
34 W ThQ_ur 1.000 -3.542E-5
35 W FoQ_ur 1.000 1.8903E-5
36 W SeQ_pe 1.000 2.9799E-5
37 W ThQ_pe 1.000 -4.375E-5
38 W FoQ_pe 1.000 2.3299E-5
39 PX INV 1.000 -4.636E-5
40 PG FEXT 1.000 -2.898E-5
41 PG FOOD 1.000 -4.609E-5
42 PG META 1.000 -4.048E-5
43 PG MAME 1.000 3.0705E-5
44 PG MAVE 1.000 -3.051E-5
45 PG ELYs 1.000 -3.186E-5
46 PG BUIL 1.000 -7.447E-5
47 PG CONT 1.000 4.5641E-5
48 PG TRWH 1.000 -4.136E-5
49 PG TRRE 1.000 4.8829E-5
50 PG ACCO 1.000 1.8086E-5
51 PG SFIN 1.000 4.0791E-5
52 PG REAL 1.000 2.5735E-5
53 PG EDUC 1.000 -5.523E-5
54 PG HEAL 1.000 3.6881E-5
55 PW ThQ_ur 1.000 2.8911E-5
56 PW FoQ_ur 1.000 -6.015E-5
57 PW SeQ_su 1.000 1.6000E-5
58 PW FoQ_su 1.000 -1.600E-5
59 PW SeQ_pe 1.000 -4.103E-5
60 PW ThQ_pe 1.000 5.0219E-5
61 PW FoQ_pe 1.000 -4.608E-5
[62 rows x 4 columns]
Best, JM
Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0
Hi JM
It seems your SAM is not perfectly balanced. 1E-4 shows that there are balance problems. The solver tries to find a solution with the maximum infeasibility below the tolerance level (something like 1E-7), so it can't find a solution because the imbalances are too big and a few imbalances can't be resolved.
Did you scale your SAM to numbers around 1 by dividing your SAM accordingly? If your model solved correctly using the aggregated SAM with an infeasibility around 10^-6.
I would suggest the following:
- Scale your SAM, so the infeasibility is around 1E-6 and see if it solves.
- If your SAM is already scaled, I would use a least-square procedure to get the SAM balanced to higher precision.
- Another option is to set the tolerance level to 1E-3, but if that is necessary in my models, I would use the LS-approach and balance the SAM.
Cheers
Renger
It seems your SAM is not perfectly balanced. 1E-4 shows that there are balance problems. The solver tries to find a solution with the maximum infeasibility below the tolerance level (something like 1E-7), so it can't find a solution because the imbalances are too big and a few imbalances can't be resolved.
Did you scale your SAM to numbers around 1 by dividing your SAM accordingly? If your model solved correctly using the aggregated SAM with an infeasibility around 10^-6.
I would suggest the following:
- Scale your SAM, so the infeasibility is around 1E-6 and see if it solves.
- If your SAM is already scaled, I would use a least-square procedure to get the SAM balanced to higher precision.
- Another option is to set the tolerance level to 1E-3, but if that is necessary in my models, I would use the LS-approach and balance the SAM.
Cheers
Renger
Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0
Hi,
thanks a lot for your hints. I finally understand where the problems emerge!
And yes, I scaled the SAM already right from the beginning (by a factor of 1000). When I increase scaling by another order of magnitude, the solver returns an optimal solution (but it takes ~2 minutes). I'm now thinking about rebalancing by least-square methods (because I'm also skeptical about increasing tolerance levels). Do you have a suggestion (or a reference) for doing so "quickly"?
Thanks again,
cheers,
JM
thanks a lot for your hints. I finally understand where the problems emerge!
And yes, I scaled the SAM already right from the beginning (by a factor of 1000). When I increase scaling by another order of magnitude, the solver returns an optimal solution (but it takes ~2 minutes). I'm now thinking about rebalancing by least-square methods (because I'm also skeptical about increasing tolerance levels). Do you have a suggestion (or a reference) for doing so "quickly"?
Thanks again,
cheers,
JM
Re: Benchmark checked with iterlim=0 but GAMS does not find solution with iterlim>0
____________________________________
Enjoy modeling even more: Read my blog on modeling at The lazy economist
Enjoy modeling even more: Read my blog on modeling at The lazy economist