Page 1 of 1

Delete files during execution not in compile phase

Posted: Fri Apr 17, 2020 9:22 pm
by jgrimm
Hi all,

I am using the usercutcall und userIncbCall functionality by the BCH facility.

The logic is as follows:
- userincbcall is called once incumbent solution has been determined during branch and bound (/cut) process. This might happen several times in a sequence (i.e. several incumbents are found in a series, each time the program is executed). The called program is identifying some relevant cuts and rejects the current incumbent solution if it is infeasible. Delta logic for cuts is working fine, means all identified cuts by several execution of the program incbcall.inc are kept in a file called 'nextcuts.gdx'.
- usercutcall is used to determine cuts as well and to (finally) add all cuts which have been identified since the last execution. This also works fine as the bchin.gdx file is automatically created by GAMS once execution of the program for usercutcall has finished.

Currently, I am struggling with two things:
- after completion of the usercutcall program cutcall.inc I would like to delete the file 'nextcuts.gdx' as it is not needed anymore and the file is getting bigger and bigger as new userincbcalls are following which are reading the already given cuts and adding them again to the cut pool. The command $call rm -f 'nextcuts.gdx' in the program cutcall.inc is (in my understanding) not working as it is executed during compile (not execution) phase and will drop the file before the cuts are added to the cut pool.
- the bchin.gdx file is getting quite big as the usercutcall is doing some quite intense logic and all parameters, sets and scalars are exported automatically. Is there a way to export manually? I have tried to create bchin.gdx file on my own but the file has been overwritten.

Kind regards,
Jonathan

Re: Delete files during execution not in compile phase

Posted: Sat Apr 18, 2020 7:12 am
by Renger
Hi Jonathan

Why don't you use the following: write a batch file that deletes the file (e.g. test.bat):

Code: Select all

rm myfile.txt -f
In your Gams program call the batch file:

Code: Select all

execute 'test.bat';
.

Your other question with regard to gdx: I assume that somewhere in your code you have a command like Execute_Unload 'bchin.gdx.gdx';.
Why don't you send only the parameters you want to save:

Code: Select all

execute_unload 'bchin.gdx', myparam, myohterparam;
I hope this helps
Cheers
Renger

Re: Delete files during execution not in compile phase

Posted: Sat Apr 18, 2020 8:01 am
by bussieck
Hi,

1) As Renger suggests you can call rm at run time via "execute 'rm -f nextcuts.gdx';". No need to first create a batch script.
2) The bchin.gdx in created when GAMS/CPLEX calls the usercutcall (via the gdx command line parameter). What you can do is to clear the data that is not relevant for the cut communication at the end of your program. You can clear everything but domain sets with "option clear=sym;". While this does not remove a symbol it removes the data and that takes up space in the bchin.gdx.

Hope this helps,
-Michael

Re: Delete files during execution not in compile phase

Posted: Wed May 06, 2020 10:55 pm
by jgrimm
Hi all,

thanks for the hints and sorry for my late response. It works perfectly! Thank you :)

Currently I am facing some kind of other question using BCH facility. I am using userincbcall to identify cuts. So what happens (sample log entries):
userincbcall executed: incumbent rejected (-> because program has found e.g. 5 cuts)
userincbcall executed: incumbent rejected (-> because program has found e.g. 4 cuts)
...
usercutcall executed: 25 cuts added

Coding works like that:
- Each time userincbcall program is executed a file nextcuts.gdx is opened and the cuts are added.
- If file nextcuts.gdx is already there, the new cuts are added on top to the already given ones.
- Once usercutcall program is executed the nextcuts.gdx file is dropped by using 'execute rm -f nextcuts.gdx' (see previous issue I was dealing with)

So far so good. I have implemented this kind of "adding feature" in the userincbcall program via this statement which is always executed during compile phase:

Code: Select all

$ifThen exist 'nextcuts.gdx'
$GDXIN 'nextcuts.gdx'
$LOAD numcuts
$eval maxnumberofnewcuts numcuts + 10000
set cc 'cuts' /1*%maxnumberofnewcuts%/;
Set ss subsets / ss1*ss%maxnumberofnewcuts% / ;
$GDXIN
$else
set cc 'cuts' /1*10000/;
Set ss subsets / ss1*ss10000 / ;
numcuts=0;
$endif
in a later statement all already given cuts are read in, and new cuts are added by numcuts+1. So far so good.

So I am having those two questions:
- I am asking myself if this is the right way to do it as each time a "re-compile" of the userincbcall program is required.
- I am searching for a solution how the program could not be limited to the number of 10000 cuts to be put on top. Why is that? (In my understanding) It is not under control how often userincbcall programm is executed in a series, so there is no control about the final number of cuts given by checking incumbents until a usercutcall is executed.

Last but no least, I am also wondering about this:

Code: Select all

* Cplex Option file to enable the cut generation call at every new integer point
$onecho     > cplex.opt
usercutcall cutcall.inc --DEBUG=0 o=cutcall.lst lf=cutcall.log
userIncbCall incbcall.inc --DEBUG=0 o=incbcall.lst lf=incbcall.log
$offecho
Is there a reason why *.gms files are not accepted and *.inc files have to be used? And is there a way to push parameters through for *.inc files? I have recognized that parameter DEBUG is not passed through to the usercutcall and userincbcall programs.

May do have some tricks/advice for me? Please just let me know if a coding example should be provided (I did avoid this so far as the behavior is difficult to show in a "real" mini example).

Kind Regards,
Jonathan

Re: Delete files during execution not in compile phase

Posted: Thu May 07, 2020 8:42 am
by bussieck
  1. Don't worry about the recompile. The BCH programs executed by the solver are always recompiled. This goes super fast.
  2. You need to prepare the data from the cuts so you need a cut index. This needs some kind finite number. You made it 10000 but you can increase this but it needs to be defined. Don't make it giantic because GAMS will store all the elements in the cut index set. That is the GAMS data model. There are no open ended lists or arrays everything is indexed by sets and their members need to be defined at compile time.
  3. No need to have your BCH programs with extension .inc you can use .gms or what ever you like.
  4. The --DEBUG=val should be passed on. Do a $log %DEBUG% in your BCH program (together with a lo=2 on the calling line) and inspect the GMSbch.log. (I have attached the modified bchmknap with the log file to do exactly this)
2forum.zip
(3.59 KiB) Downloaded 194 times
-Michael