out of memory

Post Reply
SAID09
User
User
Posts: 7
Joined: 2 months ago
Location: Morocco

out of memory

Post by SAID09 » 2 months ago

Hello everybody

I have a problem when I run a model on GAMS. More precisely, the model does not finish the execution to output the results, because of a memory problem. More concretely, the following message comes out to me:

"

*** Out of memory
*** HeapLimit = 1E299 MB HeapSize = 28568.9 MB

".

What is it about?
And what do you recommend I do to fix this problem?

Thank you so much

User avatar
dirkse
Moderator
Moderator
Posts: 126
Joined: 3 years ago
Location: Fairfax, VA

Re: out of memory

Post by dirkse » 2 months ago

Hi,

The message indicates that GAMS/Base has allocated more than 28GB of memory to run your GAMS code, and there is certainly some additional memory used elsewhere in GAMS/Base. GAMS needs more memory to run your code, but when it requests more, it cannot get any.

Sometimes, you are really just out of memory. You must either get a bigger machine or reduce the size of your problem or the amount of memory used. Less often there can be an issue where the machine has memory but it isn't letting a process use more than a certain amount.

You don't say anything about the machine you run this on or the version of GAMS you are using so it's hard to say more.

-Steve

SAID09
User
User
Posts: 7
Joined: 2 months ago
Location: Morocco

Re: out of memory

Post by SAID09 » 2 months ago

dirkse wrote:
2 months ago
Hi,

The message indicates that GAMS/Base has allocated more than 28GB of memory to run your GAMS code, and there is certainly some additional memory used elsewhere in GAMS/Base. GAMS needs more memory to run your code, but when it requests more, it cannot get any.

Sometimes, you are really just out of memory. You must either get a bigger machine or reduce the size of your problem or the amount of memory used. Less often there can be an issue where the machine has memory but it isn't letting a process use more than a certain amount.

You don't say anything about the machine you run this on or the version of GAMS you are using so it's hard to say more.

-Steve
Hello Steve
thank you very much for your answer. I will tell you the characteristics of my computer. Indeed, yes, I was running a "big" model (microsimulated model) using and calling data from a household survey containing information on almost 16,000 households.
image.png

SAID09
User
User
Posts: 7
Joined: 2 months ago
Location: Morocco

Re: out of memory

Post by SAID09 » 2 months ago

Hello Steve
Thank you very much for your answer.

I will tell you the characteristics of my computer.

Indeed, yes, I was running a "big" model (microsimulated model) using and calling data from a household survey containing information on almost 16,000 households.

- Said-
about my machine.png

User avatar
dirkse
Moderator
Moderator
Posts: 126
Joined: 3 years ago
Location: Fairfax, VA

Re: out of memory

Post by dirkse » 2 months ago

Hi,

Your machine has 8GB of RAM. For a GAMS model/computation such as yours that requires 28+ GB that is not nearly enough RAM. Windows tries to compensate by using the "disk" (hopefully an SSD) as virtual memory, but this will be very slow compared to using actually RAM. In general, you want to avoid using virtual memory for GAMS or any of its solvers.

-Steve

SAID09
User
User
Posts: 7
Joined: 2 months ago
Location: Morocco

Re: out of memory

Post by SAID09 » 2 months ago

Hello Steve

Thank you very much for your answer.

Thank you very much (also) for the precision and clarity of your answer.
So I'm going to manage to increase this capacity.
I will keep you posted soon after I resolve this issue.

Happy to be part of this community of GAMS users.

My greetings

-Said-

SAID09
User
User
Posts: 7
Joined: 2 months ago
Location: Morocco

Re: out of memory

Post by SAID09 » 2 months ago

Hi Steve

I come back to you

I just added memory in my pc in D: and I ran the model which took time and which continued with more memory.
However, the program stopped running before finish with the message below (attached).
I must have understood that I was missing a file called GAMSCMEX ...

Did I understand right? IF yes how to download it? if not what the problem is if you look at the screenshot of the output message I took.

thank you in advance for your response

-Said-
image.png

User avatar
dirkse
Moderator
Moderator
Posts: 126
Joined: 3 years ago
Location: Fairfax, VA

Re: out of memory

Post by dirkse » 2 months ago

Said,

The computation does get farther, it seems. And instead of running out of memory at around 32 GB , it is happening at 64 GB.

The executable or module that does the primary work of GAMS/Base (reading, compiling, and executing the GAMS source) is gamscmex.exe, aka GAMSCMEX. You don't need to download this - it is definitely there and running as intended, or you would never have gotten as far as you have. But you are still running out of memory.

Instead of adding yet more memory or running this model on a machine in the cloud with a lot of memory, you should try to reduce the size of the model to see what is possible and at what cost and with what memory requirement. There is almost always a way to do this: e.g. you can use smaller data sets, or aggregate parts of the data set to make it smaller. Or maybe fix some of the variables if that is a possibility. You don't get the full problem generality you might hope for that way, but you could get a problem that is easier to solve. If you do this, check the documentation on using holdfixed:

https://www.gams.com/latest/docs/UG_Gam ... Oholdfixed

You are running out of memory generating the model. Typically, the solver uses (much) more memory to solve than GAMS uses to generate, so you may have to reduce size or complexity substantially to a) generate the model and b) get a solution.

HTH,

-Steve

SAID09
User
User
Posts: 7
Joined: 2 months ago
Location: Morocco

Re: out of memory

Post by SAID09 » 1 month ago

Hello
I come back to you with great pleasure.

I have a (large) set of elements which I will note H.

I split this set (H) into 3 subsets which I will name H1(H), H2(H) and H3(H).
Card(H) = Card(H1) + Card(H2) + Card(H3)

Of course, the elements of these subsets are scattered throughout the large set according to their value D(H).
That is, the values D(H1) (of the elements of the subset H1), D(H2) (of the elements of the subset H2) and D(H3) (of the elements of the subset H3 ) are not ordered.

My request / question =
"I would like to know if there is a command on GAMS which allows me to order or sort the elements of each subset so as to have subsets (say T(H1), T(H2) and T(H3)) containing the elements but sorted according to the value of the variable D (*). "

The objective being to be able to use the command ord (T(H1)), ord(T(H2)) and ord (T(H3)).

I hope that I did not get too complicated and that I was not too long.
Thank you so much.

SAID09
User
User
Posts: 7
Joined: 2 months ago
Location: Morocco

About sorting elements of some subsets of a large set

Post by SAID09 » 1 month ago

Example to illustrate my last request on sorting a subset
image.png

Post Reply