Primal task, trying to get simple baseline working #31

kushaltirumala · 2021-07-13T08:54:35Z

kushaltirumala
Jul 13, 2021

I'm trying to establish a simple baseline in the primal task by just running SCIP for 1 second in the agents __call__ function. I extract the variable values as specified by action_set and return those values as the action.

I print the primal bound after calling optimize() in the __call__ function and the objective is reasonably lower than the initial primal bound. I also print the primal bound in the extract function of TimeLimitPrimalIntegral where it seems that the primal bound is the same as the initial primal bound (and the cumulated reward is a little worse than initial_primal_bound * -1). In other words, it does not seem that the environment is considering the action the agent gives.

My code in the Policy function is:

    def __call__(self, action_set, observation):
        old_model = observation

        var_ids = action_set

        new_model = Model(sourceModel=old_model, globalcopy=True)

        # set time limit manually
        new_model.setRealParam('limits/time', 1.0)
        new_model.setHeuristics(pyscipopt.scip.PY_SCIP_PARAMSETTING.DEFAULT)

        # optimize model
        new_model.optimize()

        # get best solution
        bestsol = new_model.getBestSol()
        bestsolvalue = new_model.getSolObjVal(bestsol)
        print(f"best solution value is {bestsolvalue}")

        new_vars = new_model.getVars(transformed=True)

        # extract solution
        final_sol = np.array([new_model.getSolVal(bestsol, new_vars[var_id]) for var_id in var_ids])

        # return assignments for just the variables
        action = (var_ids, final_sol)

        return action

where the observation is given by:

    def extract(self, model, done):
        if done:
            return None

        m = model.as_pyscipopt()

        observation = (m)

        return observation

I've gone through and made sure that the index of each variable (getIndex()) matches between the variable list of the input model and the variable list of new_model so it seems that the variables in new_model do correspond to the same variables in old_model (as in, the order of the output in getVars() is the same for both old_model and new_model).

I'm not sure what else could be going wrong, and any pointers would be really appreciated.

Answered by gasse

Jul 23, 2021

Dear all,

I figured the issue, the SCIPcopy() method actually copies the transformed problem from the source SCIP instance into the original problem of the target SCIP instance. So it suffices to extract solution values for the original variables in the copies instance and then index those values with the var_ids (variable indexes in the transformed problem for the environment's SCIP model, which is equivalent to variable indexes in the original problem for the agent's copied SCIP model). The following code seems to work. I think this baseline might be valuable to other participants as well, so maybe what we'll do now is polish it a little and then expose it to other participants by distr…

View full answer

LoganZhao1997 · 2021-07-14T23:37:21Z

LoganZhao1997
Jul 14, 2021

Hi! This is because running SCIP for 1 second cannot get any feasible solution. You can try new_model.getNSols() to get the number of feasible solutions. It should be 0. In this way, the action you give is basically nothing. And it cannot improve the primal bound.

There are two potential methods that come to me. First, you can simply set a longer time limit. Second, you can use an event handler https://imada.sdu.dk/~marco/DM871/PySCIPOpt/event_8pxi.html#details (see the documents here). However, the documents of the event handler are not clearly stated. I have not succeeded yet.

4 replies

gasse Jul 15, 2021
Maintainer

Thank you @LoganZhao1997 for providing this information :)

Let me point out that there was an issue with Python's GIL when event handlers were registered through PySCIPOpt but later called during the execution of Ecole. If you want to use event handlers for the purpose of collecting your observations, you should use the latest version of PySCIPOpt from github which has the partial fix merged, and then have a look at this comment that explains an additional step that has to be taken care of (keeping a pointer to the PySCIPOpt model somewhere).

@LoganZhao1997 @kushaltirumala if you go in that direction, let us know if it works.

Best,
Maxime

kushaltirumala Jul 16, 2021
Author

@LoganZhao1997 I don't think this is true — if I print the number of solutions it is above 0, and the primal bound found is definitely lower than the initial primal bound. My code is:

    def __call__(self, action_set, observation):
        old_model = observation

        var_ids = action_set

        new_model = Model(sourceModel=old_model, globalcopy=True)

        # set time limit manually
        new_model.setRealParam('limits/time', 1.0)
        new_model.setHeuristics(pyscipopt.scip.PY_SCIP_PARAMSETTING.DEFAULT)

        # optimize model
        new_model.optimize()

        # get best solution
        nsols = new_model.getNSols()  # the number of primal solutions stored within SCIP
        if nsols > 0:

            bestsol = new_model.getBestSol()
            bestsolvalue = new_model.getSolObjVal(bestsol)
            print(f"number of solutions: {nsols}")
            print(f"best solution value: {bestsolvalue}")

        new_vars = new_model.getVars(transformed=True)

        # extract solution
        final_sol = np.array([new_model.getSolVal(bestsol, new_vars[var_id]) for var_id in var_ids])

        # return assignments for just the variables
        action = (var_ids, final_sol)

        return action

and when I run python3 ../../common/evaluate.py primal item_placement -t 1, the first few instances' output is:

Instance item_placement_9964.mps.gz
  seed: 0
  initial primal bound: 544.289773029999
  initial dual bound: 2.781939907000245
  objective offset: 0
number of solutions: 15
best solution value: 53.023449416999966
primal bound at reward time is 544.289773029999
  cumulated reward (to be maximized): -557.8739857340031

Instance item_placement_9919.mps.gz
  seed: 1
  initial primal bound: 664.7459609090025
  initial dual bound: 0.2329074640000051
  objective offset: 0
number of solutions: 12
best solution value: 201.526414409
primal bound at reward time is 664.7459609090025
  cumulated reward (to be maximized): -680.7634781650396

...

th-yoon Jul 19, 2021

@gasse I tried with the latest pyscipopt version with GIL issue fixed, but I'm facing the same issue. I read the feasible solution and make it into action with its corresponding variable indices, but the environment does not seem to reflect the given action - primal bound does not change over iteration and the number of lp iteration does not change either. If I manipulate the environment model with pyscipopt functions at higher level module, say evaluate.py, the environment reflects the feasible solution. I wonder what causes this problem.
Thank you.

th-yoon Jul 20, 2021

I observed that the probing node is cutoff after fixing variables in probing mode. Trying to figure out why it's cutoff.

zhejiangzhoulei · 2021-07-19T11:59:18Z

zhejiangzhoulei
Jul 19, 2021

hi, @kushaltirumala , did you check the variable names?

I've gone through and made sure that the index of each variable (getIndex()) matches between the variable list of the input model and the variable list of new_model so it seems that the variables in new_model do correspond to the same variables in old_model (as in, the order of the output in getVars() is the same for both old_model and new_model).

I also try to solve the model with SCIP in the agents __call__ function, and meet the same problem.
I run my code, and here is my output. It seems that the names of variables do not match.

Instance item_placement_9900.mps.gz
  seed: 0
  initial primal bound: 518.8728455659981
  initial dual bound: 0.19746730300009405
  objective offset: 0
	len(vars):1083
	i,var_name,index
	0	t_place_4_9	1133
	1	t_place_59_5	1679
	2	t_place_4_1	1125
	3	t_place_37_7	1461
	4	t_place_65_4	1738
	5	t_place_99_0	2074
	6	t_place_98_4	2068
	7	t_place_14_6	1230
	8	t_place_104_4	2128
	9	t_place_30_5	1389
	10	t_place_24_5	1329
	11	t_place_38_9	1473
copy model...
2021-07-19 19:36:37.264017
2021-07-19 19:36:37.291136
	number of solutions: 1
	best solution value: 541.4726949959999
	len(vars):1083
	i,var_name,index,var_val
	0	t_t_place_48_0	1133	0.0
	1	t_t_place_2_7	1679	0.0
	2	t_t_place_91_0	1125	1.0
	3	t_t_place_69_7	1461	0.0
	4	t_t_place_63_6	1738	0.0
	5	t_t_place_51_7	2074	0.0
	6	t_t_place_50_0	2068	0.0
	7	t_t_place_14_0	1230	0.0
	8	t_t_place_25_8	2128	0.0
	9	t_t_place_16_1	1389	0.0
	10	t_t_place_5_4	1329	0.0
	11	t_t_place_27_3	1473	0.0

3 replies

th-yoon Jul 19, 2021

@zhejiangzhoulei Did you happen to call getVars(transformed=True) to get variables? I fixed that issue by calling pyscipoptmodel.getVars(transformed=False).

zhejiangzhoulei Jul 20, 2021

@th-yoon yes, you are right. It should be pyscipoptmodel.getVars(transformed=False). Thank you!

kushaltirumala Jul 21, 2021
Author

@zhejiangzhoulei yes I did look at variable names. The main problem I was running into (and still not sure how to deal with) is that the names/unique indices of the variables seem to change after calling model.optimize i.e I call model.getVars(transformed=True) before and after running model.optimize, and the var.name and var.getIndex() are different

gasse · 2021-07-21T08:30:32Z

gasse
Jul 21, 2021
Maintainer

Hi all.

Thanks for the discussion ! First of all let me say that this baseline does make a lot of sense, and it will be very helpful to beginners I am sure, so thank you for sharing it :)

I would suspect that there is a discrepancy between the ordering of the variables in the duplicated model and in the environment's model, as some of you have pointed out. But it might be also that there is a bug on our side.

I will have to dig into it more, but unfortunately I won't be able to do that until Friday. Meanwhile, I would suggest to thoroughly double-check variable names and orderings, as this seems like the obvious explanation. I am not sure though why PySCIPOpt would change the ordering when copying the original model.

Best,
Maxime

1 reply

kushaltirumala Jul 21, 2021
Author

@gasse I have figured out a few things:

the order of the variables seem to change before and calling model.optimize, not just because we duplicate the model.
You can check this by passing the variables given in the ObservationFunction i.e:

    def extract(self, model, done):
        if done:
            return None

        m = model.as_pyscipopt()
        variables = m.getVars(transformed=True)

        observation = (m, variables)

        return observation

and in the Policy:

    def __call__(self, action_set, observation):
        old_model, old_variables = observation

        new_model = Model(sourceModel=old_model, globalcopy=True)
        new_model_vars = new_model.getVars(transformed=True)

        new_model_names = [var.name for var in new_model_vars]
        new_model_indices = [var.getIndex() for var in new_model_vars]
        old_model_names = [var.name for var in old_variables]
        old_model_indices = [var.getIndex() for var in old_variables]
        print(new_model_names[:5])
        print(new_model_indices[:5])
        print(old_model_names[:5])
        print(old_model_indices[:5])

and see that the names match (but the indices do not), so maybe copying the model only changes the unique variable index. On the other hand, if you look at the list of variables before and after calling new_model.optimize() and print out the corresponding names/indices, nothing seems to match

If we want to stick to the original order i.e the order of variables that action_set agrees with, we want to consider the order of variables of the input model passed into the ObservationFunction; it seems like we can do this with something like:

    def __call__(self, action_set, observation):
        old_model, old_variables = observation

        new_model = Model(sourceModel=old_model, globalcopy=True)

        # set time limit manually
        new_model.setRealParam('limits/time', 1.0)
        new_model.setHeuristics(pyscipopt.scip.PY_SCIP_PARAMSETTING.DEFAULT)

        new_model.optimize()

        assert(new_model.getNSols() > 0)
        bestsol = new_model.getBestSol()
        bestsolvalue = new_model.getSolObjVal(bestsol)
        print(f"best solution value: {bestsolvalue}")

        # extract solution
        final_sol_old_var = np.array([new_model.getSolVal(bestsol, old_variables[var_id]) for var_id in action_set])

        action = (action_set, final_sol_old_var)

        return action

where we directly index into the original models variables via old_variables[var_id] and just lookup that Variable's value in the best solution in new_model. This actually is able to run i.e getSolVal is somehow able to correspond variables in a model to a copied/optimized model (not sure how), but the same problem occurs where the primal bound of the model does not change at reward time.

details for the above: I print the best primal bound above, and add a print statement in the extract function of TimeLimitPrimalIntegral as:

and the output when I run python3 ../../common/evaluate.py primal item_placement -t 1 look as:

Evaluating the primal task agent.
Processing instances from /Users/kushaltirumala/Desktop/code/ml4co_generaleyes/instances/1_item_placement/valid
Saving results to /Users/kushaltirumala/Desktop/code/ml4co_generaleyes/submissions/scip/results/primal/1_item_placement.csv

Instance item_placement_9964.mps.gz
  seed: 0
  initial primal bound: 544.289773029999
  initial dual bound: 2.781939907000245
  objective offset: 0
best solution value: 53.023449416999966
primal bound at reward time is 544.289773029999
  cumulated reward (to be maximized): -557.9036446280253

Instance item_placement_9919.mps.gz
  seed: 1
  initial primal bound: 664.7459609090025
  initial dual bound: 0.2329074640000051
  objective offset: 0
best solution value: 201.526414409
primal bound at reward time is 664.7459609090025
  cumulated reward (to be maximized): -679.7918828095912

Instance item_placement_9907.mps.gz
  seed: 2
  initial primal bound: 583.3833282300018
  initial dual bound: 2.253966259000072
  objective offset: 0
best solution value: 122.78176479400003
primal bound at reward time is 583.3833282300018
  cumulated reward (to be maximized): -596.4639585497081
...

so I'm not entirely sure the ordering is the problem. Is there an intended way to identify a variable in a problem after calling model.optimize? (perhaps the way getSolVal does the identifying is wrong and that is why the returned solution does not lower the primal bound at reward time).

gasse · 2021-07-23T20:50:17Z

gasse
Jul 23, 2021
Maintainer

Dear all,

I figured the issue, the SCIPcopy() method actually copies the transformed problem from the source SCIP instance into the original problem of the target SCIP instance. So it suffices to extract solution values for the original variables in the copies instance and then index those values with the var_ids (variable indexes in the transformed problem for the environment's SCIP model, which is equivalent to variable indexes in the original problem for the agent's copied SCIP model). The following code seems to work. I think this baseline might be valuable to other participants as well, so maybe what we'll do now is polish it a little and then expose it to other participants by distributing it as a baseline for the primal task.

import ecole as ec
import numpy as np
import pyscipopt


class ObservationFunction():

    def __init__(self, problem):
        # called once for each problem benchmark
        self.problem = problem  # to devise problem-specific observations
        self.m_copy = None

    def seed(self, seed):
        # called before each episode
        # use this seed to make your code deterministic
        pass

    def before_reset(self, model):
        # called when a new episode is about to start
        self.m_copy = None

    def extract(self, model, done):
        if done:
            return None

        m = model.as_pyscipopt()
        remaining_time_budget = m.getParam("limits/time") - m.getSolvingTime()

        # extract (copy) the model only once
        if self.m_copy is None:
            self.m_copy = pyscipopt.Model(sourceModel=m)  # copy the model into a new SCIP instance
            m_isfresh = True  # freshly created model
        else:
            m_isfresh = False  # not freshly created model

        observation = (self.m_copy, remaining_time_budget, m_isfresh)
            
        return observation


class Policy():

    def __init__(self, problem):
        # called once for each problem benchmark
        self.rng = np.random.RandomState()
        self.problem = problem  # to devise problem-specific policies

    def seed(self, seed):
        # called before each episode
        # use this seed to make your code deterministic
        self.rng = np.random.RandomState(seed)

    def __call__(self, action_set, observation):
        m, max_time_budget, m_isfresh = observation 
        var_ids = action_set

        # reset solution improvement and solving time counters for freshly created models
        if m_isfresh:
            self.sol_improvements_consumed = 0
            self.solving_time_consumed = 0

        m.setPresolve(pyscipopt.scip.PY_SCIP_PARAMSETTING.OFF)  # presolve has already been done
        m.setHeuristics(pyscipopt.scip.PY_SCIP_PARAMSETTING.AGGRESSIVE)  # we want to focus on finding feasible solutions
        m.setParam('limits/bestsol', self.sol_improvements_consumed + 1)  # stop after a new primal bound improvement is found
        m.setParam('limits/time', self.solving_time_consumed + max(max_time_budget - 0.01, 0))  # stop the agent before the environment times out
        m.optimize()

        # keep track of best sol improvements in the copied model to be able to set a limit
        if m.getStatus() == 'bestsollimit':
            self.sol_improvements_consumed += 1

        # keep track of solving time already spedn in the model to be able to increment it appropriately
        self.solving_time_consumed = m.getSolvingTime()

        assert m.getNSols() > 0

        print(f"{m.getSolvingTime()} seconds spend searching, best solution so far {m.getObjVal()}")

        sol = m.getBestSol()
        sol_vals = np.asarray([m.getSolVal(sol, var) for var in m.getVars(transformed=False)])

        action = (var_ids, sol_vals[var_ids])

        return action

14 replies

harshadkhadilkar Sep 15, 2021

Hi @gasse , is there a way to rebuild the model (constraints and objective coefficients) in the policy class? If I run commands like getLPRowsData() on the model in the env class, it returns all the constraints. However if I try it on the model in your baseline (both in ObservationFunction and in Policy), I get a null response.

harshadkhadilkar Sep 15, 2021

Update: I found that the original model 'm' has this information, but m_copy doesn't. Issue resolved.

gasse Sep 15, 2021
Maintainer

Great, glad you've found a solution :)

harshadkhadilkar Sep 28, 2021

@gasse I have a follow-up question about the "action" variable that is communicated in each step. Is this action actually implemented (variable values are set) in the model by the environment? This does not seem to be happening.
I am using the baseline that you have provided above, for the item placement problem. Just for experimentation, I figure out what the current mapping of objects to bins is (in the latest model). Then I move any one object from one bin another by setting the "place" variables. For example, change place_0_8 from 1 to 0 and place_0_6 from 0 to 1 in order to move object 0 from bin 8 to bin 6. However when I check the model received in the next step (observation class), I still see place_0_8 = 1 and place_0_6 = 0.

gasse Oct 12, 2021
Maintainer

Hi @harshadkhadilkar ,

Sorry for the late reply, I had missed your message somehow. I think what might happen is the following: with an action, you suggest an feasible solution to SCIP. SCIP then simply considers that solution, and evaluates two things:

is the proposed solution feasible (satisfies the MILP constraints) ?
does the proposed solution improves upon the current best solution (better objective value) ?

If those two conditions are not met, then SCIP will disregard the action (proposed solution), and in the next iteration if you ask the current solution to SCIP it will not return your proposed solution, but the current best one.

Hope that helps,

Best,
Maxime

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Primal task, trying to get simple baseline working #31

{{title}}

Replies: 4 comments 22 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Primal task, trying to get simple baseline working #31

Replies: 4 comments · 22 replies

gasse Jul 15, 2021 Maintainer

kushaltirumala Jul 16, 2021 Author

kushaltirumala Jul 21, 2021 Author

gasse Jul 21, 2021 Maintainer

kushaltirumala Jul 21, 2021 Author

gasse Jul 23, 2021 Maintainer

gasse Sep 15, 2021 Maintainer

gasse Oct 12, 2021 Maintainer

Replies: 4 comments 22 replies

gasse Jul 15, 2021
Maintainer

kushaltirumala Jul 16, 2021
Author

kushaltirumala Jul 21, 2021
Author

gasse
Jul 21, 2021
Maintainer

kushaltirumala Jul 21, 2021
Author

gasse
Jul 23, 2021
Maintainer

gasse Sep 15, 2021
Maintainer

gasse Oct 12, 2021
Maintainer