How to retrieve variables in the MsgOutput_*.gdx file

[cheolhung]

Dear developers,

I attempted to retrieve variables from a MsgOutput_*.gdx file using Python. For example, I tried running scenario.var('ACT_UP').query('Level>0'), but encountered the following error message:
RuntimeError: unhandled Java exception: There is no variable 'ACT_LO'!

Is there a general method or best practice for retrieving any variable (such as COMMODITY_USE) that exists in a MsgOutput_*.gdx file?

Thank you in advance for your help.

[glatterf42]

Hi @cheolhung, thanks for your question :)
In general, when you set up a MESSAGE Scenario and run Scenario.solve(), the variables and equations defined in the Scenario will be loaded into the Scenario object after a solution has been found. That is to say, you generally do not need to access the .gdx file manually, and your method of retrieving the variable data via scenario.var("ACT_UP") looks correct to me.
If you encounter any issue with that workflow, please provide more details as to what exactly is going on. Ideally, we need the information from message-ix show-versions, all commands that lead to the error message, and the full error traceback. For example, with what you provide, I'm not sure why the error message would refer to ACT_LO when you requested ACT_UP.

That being said, you can study .gdx files using GAMS' unfortunately proprietary software. Within our team, people regularly use GAMS Studio to do that.
Alternatively, you can use the freely available gamsapi Python package to load the data from a .gdx file into Python code.

[glatterf42]

One likely possibility for why your error occurs (thanks to @khaeru for pointing that out): message_ix has a lot of variables, see this list, but not all of them are loaded from a MsgOutput_*.gdx file back to the Scenario since you might not always be interested in all of them and it takes more time and memory to load the whole list. Instead, only a small subset of variables is loaded per default, defined through this list. Please note that this list does not include ACT_LO and ACT_UP.
You can overwrite this list (or rather: add to it) by specifying the parameter var_list in Scenario.solve(): this parameter should be a list of variable names that you want to load into the Scenario after it is solved in addition to the default set. See e.g.

message_ix/message_ix/tests/test_soft_constraints.py

Line 29 in d3cf5bd

s.solve(var_list=["ACT_UP"])

for an example of how to use it.