csv2gdx - detecting failures during execution rather than compilation Topic is solved

[AndrewC]

I am new to csv2gdx so I am trying to get my head around how it operates when the input csv is bad.

I want to generate some gdx files from csv inputs. But sometimes those input csv files might be bad.

When in use GDXXRW to read from Excel, if the data is bad it just doesn't generate the GDX output, and I detect the absence of this file within the GAMS code to write a little helpful error message and then exit the code.

However it seems that when the csv2gdx encounters bad data it makes an empty gdx. [Or is there an option to NOT do this???] So I cannot look for this file being missing.
I found this post viewtopic.php?t=10524 where @bussieck checks the errorlevel and aborts.
$call csv2gdx DC03092018_utf8.csv id=timeSeries useheader=y index=(1,2) values=(3..lastcol)
$if errorlevel 1 $abort problems with csv2gdx

However if I understand it correctly this is a compilation time check so it doesn't give me the chance to write an error message to the log file that I write as the code executes. Writting the error message would allow me to tell the user which of the input csv files is bad.

Is there a way to detect the gdx file is bad during EXECUTION? So I can write a helpful message and abort cleanly.

All help appreciated.
Cheers

[bussieck]

Hi, relying on the file being there or not is not a good indicator, even for gdxxrw. It depends when the error happens. Both tools open the GDX file and write to it. They might not close it though (in case of an error) and you have an incomplete=bad GDX file. For all tools you should rely on the return code which you can query from GAMS. At compile time after a "$call xyz" you do that with "$if errorLevel 1 $abort problem with tool xyz" or "$ifE errorLevel<>0 $abort problem with tool xyz" (if the tool gives also negative return codes for bad runs). At execution time, you can check the errorLevel function after running an external program via "execute 'xyz';" or "put_utility 'exec' / 'xyz';" and also abort: "abort$errorLevel 'problem with tool xyz';". The easiest is to use the .checkErrorLevel variant of $call and execute. This internally checks the return code and stops the further compilation and/or execution of the program if the return code is not 0 from the tool: "$call.checkErrorLevel xyz" and "execute.checkErrorLevel 'xyz';" or "put_utility 'exec.checkErrorLevel' / 'xyz';".

-Michael

[AndrewC]

Thanks, a lot fo rme to unpack. Ive never really delved deeply into all this stuff because I mainly run my own models so can figure out crashes when they happen. With our models moving more and more into the clients hands they tend to screw up the data more regularly, so I am going to have to learn this stuff.

Thankis for the pointers.

[bussieck]

Yes, the default mode is the error mode. -Michael