Non-ASCII characters in .gdx created with Python API

Questions on using the GAMS APIs (Python, .NET, etc.)
Post Reply
bfsantoro
User
User
Posts: 6
Joined: 4 years ago

Non-ASCII characters in .gdx created with Python API

Post by bfsantoro »

I am struggling to work with the Python API when there are some interactions with .gdx files containing Non-ASCII characters. Of course one possible workaround would be to simply avoid such type of characters, but this is not the solution that I am looking for because in my application the .gdx files come from the result of an optimization problem defined by an external user, and (s)he may use Portuguese names of cities or other words.

A minimum working example of the issue is provided in the attached file. It is a simple modification of the example transport4.py, including a Non-ASCII character in one of the cities' names. To run this file it is first necessary to change its extension to .py and also to create a folder called tmp inside the folder where the .py is located. The result is that variable x is printed in Python as

x(São Paulo,New-York): level=0.0 marginal=5e-324
x(São Paulo,Chicago): level=300.0 marginal=0.0
x(São Paulo,Topeka): level=0.0 marginal=0.036000000000000004
x(San-Diego,New-York): level=325.0 marginal=0.0
x(San-Diego,Chicago): level=0.0 marginal=0.009000000000000008
x(San-Diego,Topeka): level=275.0 marginal=0.0

but the .gdx of the corresponding run does not properly present the "ã" character.
gdx_Issue.png
PS: I don't know if this is related to the problem, but the release notes of version 31.2.0 mentions that some issues with Non-ASCII characters were addressed in .NET and C# APIs, and nothing was said about Python. I am using GAMS version 33.0 but the problem seems to be present in previous versions as well.
transport4_PortugueseAccentIssue.txt
(3.3 KiB) Downloaded 282 times
User avatar
bussieck
Moderator
Moderator
Posts: 1033
Joined: 7 years ago

Re: Non-ASCII characters in .gdx created with Python API

Post by bussieck »

Your text file is encoded in utf-8 and you look at the GDX file with the IDE which can only do the "system" encoding that shows the utf-8 encoded ã incorrectly. The GDX file is actually fine. For example, if you dump into a text file and look at it with an editor that encodes with utf-8 then things look good:

Code: Select all

gdxdump tmp\_gams_py_gdb1.gdx > x.txt
image.png
image.png (6.14 KiB) Viewed 4649 times
If you switch to GAMS Studio (which understand UTF-8) the GDX viewer also shows this nicely:
image.png
So it depends in what encoding you get your non-ASCII characters. GAMS does not do any conversion of encodings, it takes your labels as a byte array.

-Michael
bfsantoro
User
User
Posts: 6
Joined: 4 years ago

Re: Non-ASCII characters in .gdx created with Python API

Post by bfsantoro »

Thank you very much for the support! I just kept using the traditional IDE instead of GAMS Studio - old habits die hard, right?
Post Reply