Page 1 of 1

### Manipulatiing parameters - Assignment problem due to different dimension

Posted: Sun Mar 17, 2019 10:59 pm
Hi,
I'm having trouble assigning values. For example I have several files and every file represents a sation which meassures windspeed. These files look like this:

Code: Select all

Station;Date;Time;  SP
34;2017-09-12;0000;    2.5
34;2017-09-12;0010;    6.1
34;2017-09-12;0020;    3.0
...
Then I have other files where power is assigned to speed depending on the generator type

Code: Select all

type;Speed;Power
1;       0	;0
1;       0.1	;0
...
1;      2.9	;0
1;      3	;100
1;      3.1	;211
...
1;      34.9	;4000
1;      35	;0
1;      35.1	;0
...
Now I want to calculate the power output for every station at each day and time and for a specific typ, so the gdx file should look something like this:

Code: Select all

i;j;k;t; value
34;2017-09-12;0000; 1;   0
34;2017-09-12;0010; 1;   1050
34;2017-09-12;0020; 1;   100
...
34;2017-09-12;0000; 2;   0
34;2017-09-12;0010; 2;   2111
34;2017-09-12;0020; 2;   300
My code is attached, but it won't work (error - dimension different).

Jason

PS: If someone happen to know if there is an easy way to read several input files(or could point me in the right direction) I would appreciate it too. I've read about the Put Writing Facility, but at first impression it was not very intuitive. I was thinking more of a simple loop, but that doesn't seem to work or at least I dont know how.

### Re: Manipulatiing parameters - Assignment problem due to different dimension

Posted: Mon Mar 18, 2019 1:00 pm
Hi,

If I understand you correctly, you somehow need to compare the set elements of

Code: Select all

set    w 'windspeed';
to the parameter values of

Code: Select all

Parameter speed(i,j,k)         'Speed';
In your approach you basically try to replace set w with parameter speed(i,j,k) which - as you already noticed - does not work that way. However, if you know that set elements represent numerical values (such as the the elements of w), you can access those values via set attribute .val (https://www.gams.com/latest/docs/UG_Set ... Attributes). The following should do the trick:

Code: Select all

power_output(i,j,k,t) = sum(w\$[abs(w.val-speed(i,j,k))<1e-6], power_wind(t,w));
In the example I check whether the absolute difference between w.val and parameter speed is < 1e-6.

It seems that if you have stations 1,2,...,N you want to read files station1.txt, station2.txt, ..., stationN.txt. put_utulity, as you already suspected, allows you to do that. If you share a small executable example that illustrates what you want to do, other users might use that example and modify it. That would make helping you much easier than constructing examples from a few code snippets.

One more remark: it Seems that for example in station*.txt, the actual station is not only part of the filename but also defined via the first column of the file. Given that, I wonder what's the reason of having the data in N files instead of having it in just one file.

Best,
Fred

### Re: Manipulatiing parameters - Assignment problem due to different dimension

Posted: Wed Mar 20, 2019 10:20 pm
Hi Fred,

thank you very much for your reply. Your code example did what I wanted, thanks! I never would have guessed. There is not an easier solution to such a problem? I don't mean that in a bad way, I'm just curious, because I thought you can solve it with a simple assignment also with the ulterior motive that this would be faster too (Speed is not an issue yet, but it took GAMS 30s to execute this step - 160.000 entries were created in this step, in comparison reading 4 files with 80.000 entries each took like 1s)
It seems that if you have stations 1,2,...,N you want to read files station1.txt, station2.txt, ..., stationN.txt. put_utulity, as you already suspected, allows you to do that. If you share a small executable example that illustrates what you want to do, other users might use that example and modify it. That would make helping you much easier than constructing examples from a few code snippets.
I'm not sure if I understand you correctly, but I wasn't able to get into the put utulity and came up with a solution or at least something close to a solution, yet. Or do you mean such examples?
Generator1.txt
Station1.txt
One more remark: it Seems that for example in station*.txt, the actual station is not only part of the filename but also defined via the first column of the file. Given that, I wonder what's the reason of having the data in N files instead of having it in just one file.
I get the data from an external source, so I don't have any influence on it. Maybe it would be simply too much data for a regular user who just needs data from one station, but I dont know.

Greetings,
Jason

### Re: Manipulatiing parameters - Assignment problem due to different dimension

Posted: Wed Apr 10, 2019 1:50 pm
Jason,

Seems that we are discussing two points

1. Slow execution of assignment
Let me try to give you some background. You use different data types for wind speed.
a) set w; This set contains different wind speeds in the form of a discrete set 0.0, 0.1, 0.2, 0.3, ... Set elements are treated as strings (but if you know they have a numerical value, you can access it via .val)
b) parameter speed(i,j,k); This parameter also contains different wind speeds but as numerical values 0.0, 0.1, 0.2, 0.3, ...

Since you already treat wind speeds as set elements, I wonder whether you actually need parameter speed. You could read the wind speeds as sets from the station*.txt files. The advantage is that this allows you to get rid of the comparison (abs(w.val-speed(i,j,k))<1e-6) in the assignment to power_output(i,j,k,t) and do sth. like

Code: Select all

...
set speed(i,j,k,w)
...
power_output(i,j,k,t) = sum(speed(i,j,k,w), power_wind(t,w));
...
This is considerably faster.

2. reading enumerated files with identical filestem (station1.txt, station2.txt, ...)
There are many ways to reduce the amount of code. The attached examples illustrate two alternatives. Note that your recent station.txt is slightly different to the on from post #1. I read only some of the columns of station*.txt and do not read generator*.txt but I guess you can adopt the examples as needed.

a) batinclude: Write a little GAMS script that takes the number of the file as argument and does the reading.

Code: Select all

...
\$call csv2gdx station%1.txt output=station%1.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y
\$ife errorlevel<>0 \$abort problem with csv2gdx station%1
\$gdxIn station%1.gdx
\$gdxIn
\$offecho

\$onMulti
...
\$onecho ... writes the file readStationData.gms which us subesequently (bat)included. The argument of the batinclude replaces the internal placeholder (%1). You can read more about that in the documentations

b) put_ultility: Write a script where in the script you loop over all the files that need to be read.

Code: Select all

...
set f files / 1*%NBFILES%/;
loop(f,
put_utility 'exec' / 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y'
abort\$(errorlevel<>0) 'probem running csv2gdx via put utility'
put load '\$gdxin station' f.tl:0 '.gdx' /
'\$gdxin'                       /;
);
putclose;

\$offecho

\$ife errorlevel<>0 \$abort problem running readStationData.gms
\$onMulti
...
You pass the number of files to read (--NBFILES) as an argument to the script which loops over the txt files, creates the gdx files, and writes a list of gdxin instructions to include later. Note that you have to outsource this put_utility approach to a script because the put_utility command is carried out at execution time while you (probably) want the data to be available at compile time. The trick is to call the script at compile time via \$call. You can read more about the two phases (compilation, execution) in the documentation: https://www.gams.com/latest/docs/UG_Gam ... ll_TwoPass

I hope this helps!

Fred

### Re: Manipulatiing parameters - Assignment problem due to different dimension

Posted: Thu Apr 11, 2019 8:07 pm
Fred, you are awesome. Thank you so much for your detailed answer!!

I used the put ultility and I got it working

Not that important, but did you use this

Code: Select all

put_utility 'exec' / 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y'

Code: Select all

put load 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y' /
on purpose?
Because both works, but I wonder if there is a significant difference between them or is the latter maybe just a bad coding style?

### Re: Manipulatiing parameters - Assignment problem due to different dimension

Posted: Fri Apr 12, 2019 8:17 am
jason_s wrote:
6 months ago
Fred, you are awesome.
Thanks
jason_s wrote:
6 months ago
Not that important, but did you use this

Code: Select all

put_utility 'exec' / 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y'

Code: Select all

put load 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y' /
on purpose?
Because both works, but I wonder if there is a significant difference between them or is the latter maybe just a bad coding style?
In fact there is significant difference and I used put_utility on purpose.

With put_utility the csv2gdx command is executed from readStationData.gms which you call at compile time of your main file --> Data is available at compile time in your main file and you can do \$gdxin, \$load etc.

With put the csv2gdx command is written to file loadGDX.inc which is included in your main file. In general, you could do that. Then you need to run the csv2gdx command at compile time via \$call. In your example, it seems that you don't run it at all, so you should get an error. The following should work:

Code: Select all

[...]
set f files / 1*%NBFILES%/;
loop(f,
put load '\$call csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y' /
'\$ife errorlevel<>0 \$abort problem with csv2gdx station'f.tl:0 /
'\$gdxin station' f.tl:0 '.gdx' /
'\$gdxin'                       /;
);
putclose;
\$offecho