Re: Excessive copies of set elements in GMPL

And here is the output of the the script bellow:

Memory usage:

ubuntu glpsol package : 1,422,160 KB => 4.20s elapsed

glpk with my changes: 429,000 KB => 1.37s elapsed

=====

/usr/bin/time glpsol -m test-nested-sets.mod
GLPSOL: GLPK LP/MIP Solver, v4.65
Parameter(s) specified in the command line:
  -m test-nested-sets.mod
Reading model section from test-nested-sets.mod...
test-nested-sets.mod:12: warning: unexpected end of file; missing end 
statement inserted
12 lines were read
A count: 100
B count: 200
C count: 300
D count: 100
Display statement at line 12
D:
    (1,2,4)
    (2,3,5)
    (3,4,6)
    (4,5,7)
    (5,6,8)
    (6,7,9)
    (7,8,10)
...

Time used:   0.0 secs
Memory used: 1383.2 Mb (1450389029 bytes)
3.67user 0.53system 0:04.20elapsed 99%CPU (0avgtext+0avgdata 
1422160maxresident)k
0inputs+0outputs (0major+354938minor)pagefaults 0swaps

/usr/bin/time myglpsol -m test-nested-sets.mod
GLPSOL: GLPK LP/MIP Solver, v4.65
Parameter(s) specified in the command line:
  -m test-nested-sets.mod
Reading model section from test-nested-sets.mod...
test-nested-sets.mod:12: warning: unexpected end of file; missing end 
statement inserted
12 lines were read
A count: 100
B count: 200
C count: 300
D count: 100
Display statement at line 12
D:
    (1,2,4)
    (2,3,5)
    (3,4,6)
    (4,5,7)
    (5,6,8)
    (6,7,9)
    (7,8,10)
...

Time used:   0.0 secs
Memory used: 415.5 Mb (435706477 bytes)
1.18user 0.18system 0:01.37elapsed 99%CPU (0avgtext+0avgdata 
429000maxresident)k
0inputs+0outputs (0major+106673minor)pagefaults 0swaps

=====

=====

set A := {1..100};
set B := {1..200};
set C := {1..300};

set D := { (a,b,c) in A cross B cross C : b=a+1 and c=b+2 };

printf "A count: %d\n", card(A);
printf "B count: %d\n", card(B);
printf "C count: %d\n", card(C);
printf "D count: %d\n", card(D);

display D;

=====

On 24/7/20 9:05, Domingo Alvarez Duarte wrote:
> Here is the output of the script from the previous message:
>
> Memory usage:
>
> ubuntu glpsol package : 4,544 KB
>
> glpk with my changes: 3,268 KB
>
> ====
>
> /usr/bin/time glpsol -m test-nested-sets.mod
> GLPSOL: GLPK LP/MIP Solver, v4.65
> Parameter(s) specified in the command line:
>  -m test-nested-sets.mod
> Reading model section from test-nested-sets.mod...
> test-nested-sets.mod:12: warning: unexpected end of file; missing end 
> statement inserted
> 12 lines were read
> A count: 10
> B count: 20
> C count: 30
> D count: 10
> Display statement at line 12
> D:
>    (1,2,4)
>    (2,3,5)
>    (3,4,6)
>    (4,5,7)
>    (5,6,8)
>    (6,7,9)
>    (7,8,10)
>    (8,9,11)
>    (9,10,12)
>    (10,11,13)
> Model has been successfully generated
> GLPK Simplex Optimizer, v4.65
> 0 rows, 0 columns, 0 non-zeros
> ~     0: obj =   0.000000000e+00  infeas =  0.000e+00
> OPTIMAL SOLUTION FOUND
> Time used:   0.0 secs
> Memory used: 1.5 Mb (1543589 bytes)
> 0.00user 0.00system 0:00.01elapsed 80%CPU (0avgtext+0avgdata 
> 4544maxresident)k
> 0inputs+0outputs (0major+516minor)pagefaults 0swaps
>
> /usr/bin/time myglpsol -m test-nested-sets.mod
> GLPSOL: GLPK LP/MIP Solver, v4.65
> Parameter(s) specified in the command line:
>  -m test-nested-sets.mod
> Reading model section from test-nested-sets.mod...
> test-nested-sets.mod:12: warning: unexpected end of file; missing end 
> statement inserted
> 12 lines were read
> A count: 10
> B count: 20
> C count: 30
> D count: 10
> Display statement at line 12
> D:
>    (1,2,4)
>    (2,3,5)
>    (3,4,6)
>    (4,5,7)
>    (5,6,8)
>    (6,7,9)
>    (7,8,10)
>    (8,9,11)
>    (9,10,12)
>    (10,11,13)
> Model has been successfully generated
> GLPK Simplex Optimizer, v4.65
> 0 rows, 0 columns, 0 non-zeros
> ~     0: obj =   0.000000000e+00  infeas =  0.000e+00
> OPTIMAL SOLUTION FOUND
> Time used:   0.0 secs
> Memory used: 0.5 Mb (507469 bytes)
> 0.00user 0.00system 0:00.00elapsed 100%CPU (0avgtext+0avgdata 
> 3268maxresident)k
> 0inputs+0outputs (0major+214minor)pagefaults 0swaps
>
> ====
>
> On 24/7/20 8:58, Domingo Alvarez Duarte wrote:
>> Hello Michael !
>>
>> What kind of problem do you expect ? Other than memory size 
>> requirements ?
>>
>> Do you mean something like this:
>>
>> ====
>>
>> set A := {1..10};
>> set B := {1..20};
>> set C := {1..30};
>>
>> set D := { (a,b,c) in A cross B cross C : b=a+1 and c=b+2 };
>>
>> printf "A count: %d\n", card(A);
>> printf "B count: %d\n", card(B);
>> printf "C count: %d\n", card(C);
>> printf "D count: %d\n", card(D);
>>
>> display D;
>>
>> ====
>>
>> A count: 10
>> B count: 20
>> C count: 30
>> D count: 10
>> Display statement at line 12
>> D:
>>    (1,2,4)
>>    (2,3,5)
>>    (3,4,6)
>>    (4,5,7)
>>    (5,6,8)
>>    (6,7,9)
>>    (7,8,10)
>>    (8,9,11)
>>    (9,10,12)
>>    (10,11,13)
>>
>> ====
>>
>> Cheers !
>>
>> On 23/7/20 18:40, Michael Hennebry wrote:
>>> On Tue, 21 Jul 2020, Domingo Alvarez Duarte wrote:
>>>
>>>> Do you have any example doing what you are talking about ?
>>>>
>>>> In theory it should work.
>>>>
>>>> Right now it process all models in the "examples" folder and the 
>>>> output is identical to the original GLPK/GMPL except that uses less 
>>>> memory and is slightly faster.
>>>
>>> Things like the GMPL equivalent of { (a,b,c) in A x B x C : b=a+1, 
>>> c=b+2 }
>>> My understanding is that the current GMPL processor will form A x B x C
>>> before applying the filtering.
>>> If A, B and C have 100 items each, the inefficiency can probably be 
>>> lived with.
>>> If they have 1,000 items each, it will be grindly slow, assuming it 
>>> finishes.
>>> If they have 10,000 items each,
>>> the set will be to big for the handler to store.
>>>
>>> Of course, done right, a set of 10,000 3-tuples should not be a 
>>> problem.
>>>
>>>> On 21/7/20 15:24, Michael Hennebry wrote:
>>>>> Are these changes supposed to deal with things like nested set 
>>>>> iterations?
>>>