Manipulatiing parameters - Assignment problem due to different dimension

Problems with syntax of GAMS
Post Reply
jason_s
User
User
Posts: 6
Joined: 6 months ago

Manipulatiing parameters - Assignment problem due to different dimension

Post by jason_s » 6 months ago

Hi,
I'm having trouble assigning values. For example I have several files and every file represents a sation which meassures windspeed. These files look like this:

Code: Select all

Station;Date;Time;  SP
       34;2017-09-12;0000;    2.5
       34;2017-09-12;0010;    6.1
       34;2017-09-12;0020;    3.0
       ...
Then I have other files where power is assigned to speed depending on the generator type

Code: Select all

type;Speed;Power
1;       0	;0
1;       0.1	;0
...
 1;      2.9	;0
 1;      3	;100
 1;      3.1	;211
 ...
 1;      34.9	;4000
 1;      35	;0
 1;      35.1	;0
...
Now I want to calculate the power output for every station at each day and time and for a specific typ, so the gdx file should look something like this:

Code: Select all

i;j;k;t; value
       34;2017-09-12;0000; 1;   0
       34;2017-09-12;0010; 1;   1050
       34;2017-09-12;0020; 1;   100
       ...
       34;2017-09-12;0000; 2;   0
       34;2017-09-12;0010; 2;   2111
       34;2017-09-12;0020; 2;   300
My code is attached, but it won't work (error - dimension different).

Thanks in advanced for your help.
Jason

PS: If someone happen to know if there is an easy way to read several input files(or could point me in the right direction) I would appreciate it too. I've read about the Put Writing Facility, but at first impression it was not very intuitive. I was thinking more of a simple loop, but that doesn't seem to work or at least I dont know how.
Attachments
power.gms
(1.11 KiB) Downloaded 18 times

Fred
Posts: 158
Joined: 3 years ago

Re: Manipulatiing parameters - Assignment problem due to different dimension

Post by Fred » 6 months ago

Hi,

If I understand you correctly, you somehow need to compare the set elements of

Code: Select all

set    w 'windspeed';
to the parameter values of

Code: Select all

Parameter speed(i,j,k)         'Speed';
In your approach you basically try to replace set w with parameter speed(i,j,k) which - as you already noticed - does not work that way. However, if you know that set elements represent numerical values (such as the the elements of w), you can access those values via set attribute .val (https://www.gams.com/latest/docs/UG_Set ... Attributes). The following should do the trick:

Code: Select all

power_output(i,j,k,t) = sum(w$[abs(w.val-speed(i,j,k))<1e-6], power_wind(t,w));
In the example I check whether the absolute difference between w.val and parameter speed is < 1e-6.

Regarding your question on how to read multiple input files:
It seems that if you have stations 1,2,...,N you want to read files station1.txt, station2.txt, ..., stationN.txt. put_utulity, as you already suspected, allows you to do that. If you share a small executable example that illustrates what you want to do, other users might use that example and modify it. That would make helping you much easier than constructing examples from a few code snippets.

One more remark: it Seems that for example in station*.txt, the actual station is not only part of the filename but also defined via the first column of the file. Given that, I wonder what's the reason of having the data in N files instead of having it in just one file.

Best,
Fred

jason_s
User
User
Posts: 6
Joined: 6 months ago

Re: Manipulatiing parameters - Assignment problem due to different dimension

Post by jason_s » 6 months ago

Hi Fred,

thank you very much for your reply. Your code example did what I wanted, thanks! I never would have guessed. There is not an easier solution to such a problem? I don't mean that in a bad way, I'm just curious, because I thought you can solve it with a simple assignment also with the ulterior motive that this would be faster too (Speed is not an issue yet, but it took GAMS 30s to execute this step - 160.000 entries were created in this step, in comparison reading 4 files with 80.000 entries each took like 1s)
It seems that if you have stations 1,2,...,N you want to read files station1.txt, station2.txt, ..., stationN.txt. put_utulity, as you already suspected, allows you to do that. If you share a small executable example that illustrates what you want to do, other users might use that example and modify it. That would make helping you much easier than constructing examples from a few code snippets.
I'm not sure if I understand you correctly, but I wasn't able to get into the put utulity and came up with a solution or at least something close to a solution, yet. Or do you mean such examples?
Generator1.txt
(10.53 KiB) Downloaded 16 times
Station1.txt
(1.77 MiB) Downloaded 16 times
One more remark: it Seems that for example in station*.txt, the actual station is not only part of the filename but also defined via the first column of the file. Given that, I wonder what's the reason of having the data in N files instead of having it in just one file.
I get the data from an external source, so I don't have any influence on it. Maybe it would be simply too much data for a regular user who just needs data from one station, but I dont know.

Greetings,
Jason

Fred
Posts: 158
Joined: 3 years ago

Re: Manipulatiing parameters - Assignment problem due to different dimension

Post by Fred » 5 months ago

Jason,

Sorry for the late reply.
Seems that we are discussing two points

1. Slow execution of assignment
Let me try to give you some background. You use different data types for wind speed.
a) set w; This set contains different wind speeds in the form of a discrete set 0.0, 0.1, 0.2, 0.3, ... Set elements are treated as strings (but if you know they have a numerical value, you can access it via .val)
b) parameter speed(i,j,k); This parameter also contains different wind speeds but as numerical values 0.0, 0.1, 0.2, 0.3, ...

Since you already treat wind speeds as set elements, I wonder whether you actually need parameter speed. You could read the wind speeds as sets from the station*.txt files. The advantage is that this allows you to get rid of the comparison (abs(w.val-speed(i,j,k))<1e-6) in the assignment to power_output(i,j,k,t) and do sth. like

Code: Select all

...
set speed(i,j,k,w)
...
power_output(i,j,k,t) = sum(speed(i,j,k,w), power_wind(t,w));
...
This is considerably faster.


2. reading enumerated files with identical filestem (station1.txt, station2.txt, ...)
There are many ways to reduce the amount of code. The attached examples illustrate two alternatives. Note that your recent station.txt is slightly different to the on from post #1. I read only some of the columns of station*.txt and do not read generator*.txt but I guess you can adopt the examples as needed.

a) batinclude: Write a little GAMS script that takes the number of the file as argument and does the reading.

Code: Select all

...
$onechoV > readStationData.gms
$call csv2gdx station%1.txt output=station%1.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y
$ife errorlevel<>0 $abort problem with csv2gdx station%1
$gdxIn station%1.gdx
$load speed
$gdxIn
$offecho

$onMulti
$batinclude readStationData.gms 1
$batinclude readStationData.gms 2
...
$onecho ... writes the file readStationData.gms which us subesequently (bat)included. The argument of the batinclude replaces the internal placeholder (%1). You can read more about that in the documentations


b) put_ultility: Write a script where in the script you loop over all the files that need to be read.

Code: Select all

...
$onechoV > readStationData.gms
file load / loadGDX.inc /;
set f files / 1*%NBFILES%/;
loop(f,
  put_utility 'exec' / 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y'
  abort$(errorlevel<>0) 'probem running csv2gdx via put utility'
  put load '$gdxin station' f.tl:0 '.gdx' /
           '$load speed'                  /
           '$gdxin'                       /;
);
putclose;

$offecho

$call gams readStationData.gms lo=%gams.lo% --NBFILES=5
$ife errorlevel<>0 $abort problem running readStationData.gms
$onMulti
$include loadGDX.inc
...
You pass the number of files to read (--NBFILES) as an argument to the script which loops over the txt files, creates the gdx files, and writes a list of gdxin instructions to include later. Note that you have to outsource this put_utility approach to a script because the put_utility command is carried out at execution time while you (probably) want the data to be available at compile time. The trick is to call the script at compile time via $call. You can read more about the two phases (compilation, execution) in the documentation: https://www.gams.com/latest/docs/UG_Gam ... ll_TwoPass


I hope this helps!

Fred
Attachments
jason.zip
(772.41 KiB) Downloaded 17 times

jason_s
User
User
Posts: 6
Joined: 6 months ago

Re: Manipulatiing parameters - Assignment problem due to different dimension

Post by jason_s » 5 months ago

Fred, you are awesome. Thank you so much for your detailed answer!!

I used the put ultility and I got it working :-)

Not that important, but did you use this

Code: Select all

put_utility 'exec' / 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y'
instead of

Code: Select all

put load 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y' /
on purpose?
Because both works, but I wonder if there is a significant difference between them or is the latter maybe just a bad coding style?

Fred
Posts: 158
Joined: 3 years ago

Re: Manipulatiing parameters - Assignment problem due to different dimension

Post by Fred » 5 months ago

jason_s wrote:
5 months ago
Fred, you are awesome.
Thanks :-)
jason_s wrote:
5 months ago
Not that important, but did you use this

Code: Select all

put_utility 'exec' / 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y'
instead of

Code: Select all

put load 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y' /
on purpose?
Because both works, but I wonder if there is a significant difference between them or is the latter maybe just a bad coding style?
In fact there is significant difference and I used put_utility on purpose.

With put_utility the csv2gdx command is executed from readStationData.gms which you call at compile time of your main file --> Data is available at compile time in your main file and you can do $gdxin, $load etc.

With put the csv2gdx command is written to file loadGDX.inc which is included in your main file. In general, you could do that. Then you need to run the csv2gdx command at compile time via $call. In your example, it seems that you don't run it at all, so you should get an error. The following should work:

Code: Select all

[...]
$onechoV > readStationData.gms
file load / loadGDX.inc /;
set f files / 1*%NBFILES%/;
loop(f,
  put load '$call csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y' /
           '$ife errorlevel<>0 $abort problem with csv2gdx station'f.tl:0 /
           '$gdxin station' f.tl:0 '.gdx' /
           '$load speed'                  /
           '$gdxin'                       /;
);
putclose;
$offecho

$call gams readStationData.gms lo=%gams.lo% --NBFILES=5
$ife errorlevel<>0 $abort problem running readStationData.gms
$onMulti
$include loadGDX.inc
[...]
Best,
Fred

jason_s
User
User
Posts: 6
Joined: 6 months ago

Re: Manipulatiing parameters - Assignment problem due to different dimension

Post by jason_s » 5 months ago

Alright thanks again for the clarification. I get it now. I had the put_utility command and the put command not differentiated, for me both fell under the category put_utility.
Best regards,
Jason

Post Reply