SpineOpt Doesn't Execute

[rputris]

Hello,

I get the message below from the Julia console when trying to execute the SpineOpt block.
What should I do to get it to work correctly?

Regarding the upgrade, how can I do this? I tried to upgrade from the settings but this does not seem to do anything since it still gives me this error. Thank you for your assistance.

julia> # Running 'julia run_spineopt.jl http://127.0.0.1:60820 http://127.0.0.1:60821'
2-element Vector{String}:
"http://127.0.0.1:60820"
"http://127.0.0.1:60821"
Running SpineOpt for http://127.0.0.1:60820...└ @ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:99

┌ Warning: The data structure is not the latest version.│ SpineOpt might still be able to run, but results aren't guaranteed.│ Please use run_spineopt(url_in; upgrade=true) to upgrade.

Initializing data structure from db...│ SpineOpt might still be able to run, but otherwise you'd need to check your input database.│ connection__to_node__user_constraint│ node__user_constraint│ connection__from_node__user_constraint│ unit__user_constraint

││ Missing item list follows:││ object classes user_constraint│││ relationship classes connection__user_constraint│ unit__to_node__user_constraint

┌ Warning: Some items are missing from the input database.│ We'll assume sensitive defaults for any missing parameter definitions, and empty collections for any missing classes.

│ unit__from_node__user_constraint│
│ parameters min_node_pressure
│ cyclic_condition
│ is_non_spinning
│ unit_start_flow
│ max_voltage_angle
│ has_pressure
│ storages_invested_available_coefficient
│ connections_invested_available_coefficient
│ connection_linepack_constant
│ fix_node_pressure
│ fix_connection_intact_flow
│ units_on_non_anticipativity_time
│ min_voltage_angle
│ has_voltage_angle
│ units_on_cost
│ has_binary_gas_flow
│ storages_invested_coefficient
│ compression_factor
│ overwrite_results_on_rolling
│ fixed_pressure_constant_1
│ representative_periods_mapping
│ output_resolution
│ unit_idle_heat_rate
│ units_invested_available_coefficient
│ units_invested_coefficient
│ connection_reactance_base
│ connections_invested_coefficient
│ big_m
│ fix_node_voltage_angle
│ unit_incremental_heat_rate
│ weight
│ max_node_pressure
│ fixed_pressure_constant_0
│ fix_binary_gas_connection_flow
│
└ @ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:116

11.169548 seconds (29.81 M allocations: 1.585 GiB, 3.30% gc time, 98.10% compilation time)
Preprocessing data structure... 6.511100 seconds (15.07 M allocations: 825.869 MiB, 3.46% gc time, 99.52% compilation time)
Checking data structure... 1.285851 seconds (2.39 M allocations: 131.385 MiB, 99.12% compilation time)
Creating temporal structure... 3.574960 seconds (8.33 M allocations: 455.413 MiB, 3.74% gc time, 99.43% compilation time)
Creating stochastic structure... 3.132785 seconds (6.84 M allocations: 383.535 MiB, 2.12% gc time, 99.48% compilation time)
Adding variables...
ERROR: LoadError: BoundsError: attempt to access 0-element Vector{Any} at index [1]
Stacktrace:
[1] getindex
@ .\array.jl:861 [inlined]
[2] first(a::Vector{Any})
@ Base .\abstractarray.jl:398
[3] _rep_ind(m::JuMP.Model, ind::NamedTuple{(:unit, :stochastic_scenario, :t), Tuple{SpineInterface.Object, SpineInterface.Object, SpineInterface.TimeSlice}}, indices::Function)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\variables\variable_common.jl:69
[4] (::SpineOpt.var"#716#717"{JuMP.Model, Dict{Any, Any}, typeof(units_on_indices)})(ind::NamedTuple{(:unit, :stochastic_scenario, :t), Tuple{SpineInterface.Object, SpineInterface.Object, SpineInterface.TimeSlice}})
@ SpineOpt .\none:0
[5] iterate
@ .\generator.jl:47 [inlined]
[6] _all(f::Base.var"#318#320", itr::Base.Generator{Vector{NamedTuple{(:unit, :stochastic_scenario, :t), Tuple{SpineInterface.Object, SpineInterface.Object, SpineInterface.TimeSlice}}}, SpineOpt.var"#716#717"{JuMP.Model, Dict{Any, Any}, typeof(units_on_indices)}}, #unused#::Colon)
@ Base .\reduce.jl:1156
[7] all
@ .\reduce.jl:1152 [inlined]
[8] Dict(kv::Base.Generator{Vector{NamedTuple{(:unit, :stochastic_scenario, :t), Tuple{SpineInterface.Object, SpineInterface.Object, SpineInterface.TimeSlice}}}, SpineOpt.var"#716#717"{JuMP.Model, Dict{Any, Any}, typeof(units_on_indices)}})
@ Base .\dict.jl:131
[9] _representative_periods_mapping(m::JuMP.Model, var::Dict{Any, Any}, indices::typeof(units_on_indices))
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\variables\variable_common.jl:86
[10] add_variable!(m::JuMP.Model, name::Symbol, indices::typeof(units_on_indices); lb::SpineOpt.var"#823#824", ub::Nothing, bin::typeof(SpineOpt.units_on_bin), int::typeof(SpineOpt.units_on_int), fix_value::Nothing, non_anticipativity_time::Nothing)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\variables\variable_common.jl:57
[11] add_variable_units_available!
@ C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\variables\variable_units_available.jl:26 [inlined]
[12] macro expansion
@ .\timing.jl:220 [inlined]
[13] macro expansion
@ C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:50 [inlined]
[14] add_variables!(m::JuMP.Model; add_user_variables::SpineOpt.var"#2#8", log_level::Int64)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt_sp.jl:94
[15] macro expansion
@ .\timing.jl:220 [inlined]
[16] macro expansion
@ C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:50 [inlined]
[17] init_model!(m::JuMP.Model; add_user_variables::Function, add_constraints::Function, log_level::Int64)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt_sp.jl:317
[18] rerun_spineopt_sp(url_out::String; mip_solver::Nothing, lp_solver::Nothing, add_user_variables::Function, add_constraints::Function, update_constraints::Function, log_level::Int64, optimize::Bool, use_direct_model::Bool)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt_sp.jl:42
[19] invokelatest(f::Any, args::Any; kwargs::Base.Pairs{Symbol, Any, NTuple{8, Symbol}, NamedTuple{(:mip_solver, :lp_solver, :add_user_variables, :add_constraints, :update_constraints, :log_level, :optimize, :use_direct_model), Tuple{Nothing, Nothing, SpineOpt.var"#2#8", SpineOpt.var"#3#9", SpineOpt.var"#4#10", Int64, Bool, Bool}}})
@ Base .\essentials.jl:718
[20] rerun_spineopt(url_out::String; mip_solver::Nothing, lp_solver::Nothing, add_user_variables::Function, add_constraints::Function, update_constraints::Function, log_level::Int64, optimize::Bool, use_direct_model::Bool)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:154
[21] run_spineopt(url_in::String, url_out::String; upgrade::Bool, mip_solver::Nothing, lp_solver::Nothing, cleanup::Bool, add_user_variables::Function, add_constraints::Function, update_constraints::Function, log_level::Int64, optimize::Bool, use_direct_model::Bool)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:126
[22] run_spineopt(url_in::String, url_out::String)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:95
[23] top-level scope
@ C:\Users\rputr\OneDrive\Documents\GitHub\spine-cs-a5\run_spineopt.jl:3
[24] include(fname::String)
@ Base.MainInclude .\client.jl:451
[25] top-level scope
@ none:1
in expression starting at C:\Users\rputr\OneDrive\Documents\GitHub\spine-cs-a5\run_spineopt.jl:3

caused by: BoundsError: attempt to access 0-element Vector{Any} at index [1]
Stacktrace:
[1] getindex
@ .\array.jl:861 [inlined]
[2] first(a::Vector{Any})
@ Base .\abstractarray.jl:398
[3] _rep_ind(m::JuMP.Model, ind::NamedTuple{(:unit, :stochastic_scenario, :t), Tuple{SpineInterface.Object, SpineInterface.Object, SpineInterface.TimeSlice}}, indices::Function)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\variables\variable_common.jl:69
[4] (::SpineOpt.var"#716#717"{JuMP.Model, Dict{Any, Any}, typeof(units_on_indices)})(ind::NamedTuple{(:unit, :stochastic_scenario, :t), Tuple{SpineInterface.Object, SpineInterface.Object, SpineInterface.TimeSlice}})
@ SpineOpt .\none:0
[5] iterate
@ .\generator.jl:47 [inlined]
[6] grow_to!(dest::Dict{Any, Any}, itr::Base.Generator{Vector{NamedTuple{(:unit, :stochastic_scenario, :t), Tuple{SpineInterface.Object, SpineInterface.Object, SpineInterface.TimeSlice}}}, SpineOpt.var"#716#717"{JuMP.Model, Dict{Any, Any}, typeof(units_on_indices)}})
@ Base .\dict.jl:140
[7] dict_with_eltype
@ .\abstractdict.jl:547 [inlined]
[8] Dict(kv::Base.Generator{Vector{NamedTuple{(:unit, :stochastic_scenario, :t), Tuple{SpineInterface.Object, SpineInterface.Object, SpineInterface.TimeSlice}}}, SpineOpt.var"#716#717"{JuMP.Model, Dict{Any, Any}, typeof(units_on_indices)}})
@ Base .\dict.jl:129
[9] _representative_periods_mapping(m::JuMP.Model, var::Dict{Any, Any}, indices::typeof(units_on_indices))
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\variables\variable_common.jl:86
[10] add_variable!(m::JuMP.Model, name::Symbol, indices::typeof(units_on_indices); lb::SpineOpt.var"#823#824", ub::Nothing, bin::typeof(SpineOpt.units_on_bin), int::typeof(SpineOpt.units_on_int), fix_value::Nothing, non_anticipativity_time::Nothing)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\variables\variable_common.jl:57
[11] add_variable_units_available!
@ C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\variables\variable_units_available.jl:26 [inlined]
[12] macro expansion
@ .\timing.jl:220 [inlined]
[13] macro expansion
@ C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:50 [inlined]
[14] add_variables!(m::JuMP.Model; add_user_variables::SpineOpt.var"#2#8", log_level::Int64)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt_sp.jl:94
[15] macro expansion
@ .\timing.jl:220 [inlined]
[16] macro expansion
@ C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:50 [inlined]
[17] init_model!(m::JuMP.Model; add_user_variables::Function, add_constraints::Function, log_level::Int64)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt_sp.jl:317
[18] rerun_spineopt_sp(url_out::String; mip_solver::Nothing, lp_solver::Nothing, add_user_variables::Function, add_constraints::Function, update_constraints::Function, log_level::Int64, optimize::Bool, use_direct_model::Bool)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt_sp.jl:42
[19] invokelatest(f::Any, args::Any; kwargs::Base.Pairs{Symbol, Any, NTuple{8, Symbol}, NamedTuple{(:mip_solver, :lp_solver, :add_user_variables, :add_constraints, :update_constraints, :log_level, :optimize, :use_direct_model), Tuple{Nothing, Nothing, SpineOpt.var"#2#8", SpineOpt.var"#3#9", SpineOpt.var"#4#10", Int64, Bool, Bool}}})
@ Base .\essentials.jl:718
[20] rerun_spineopt(url_out::String; mip_solver::Nothing, lp_solver::Nothing, add_user_variables::Function, add_constraints::Function, update_constraints::Function, log_level::Int64, optimize::Bool, use_direct_model::Bool)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:154
[21] run_spineopt(url_in::String, url_out::String; upgrade::Bool, mip_solver::Nothing, lp_solver::Nothing, cleanup::Bool, add_user_variables::Function, add_constraints::Function, update_constraints::Function, log_level::Int64, optimize::Bool, use_direct_model::Bool)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:126
[22] run_spineopt(url_in::String, url_out::String)
@ SpineOpt C:\Users\rputr.julia\packages\SpineOpt\AMGEb\src\run_spineopt.jl:95
[23] top-level scope
@ C:\Users\rputr\OneDrive\Documents\GitHub\spine-cs-a5\run_spineopt.jl:3
[24] include(fname::String)
@ Base.MainInclude .\client.jl:451
[25] top-level scope
@ none:1

[soininen]

Looks like the database is out-of-date. If you're running this on Spine Toolbox, you can upgrade the databases by double-clicking the 'input' and 'output' Data Store items on Tolbox' Design view. This should open a dialog that asks you to upgrade the databases. Note, that you need to upgrade both databases.

[rputris]

I have clicked on upgrade and this message was after I clicked upgrade. Also, now when I double-click on the input and output data store items in the design view the db editor opens and no dialog box comes up. The SpineOpt still gives me an error. Is there a way to upgrade the whole program or the version on my computer so I don't encounter these issues with any other example models? I have gotten this on another example model as well.

[soininen]

Alright, then I misunderstood the problem. Maybe somebody more familiar with SpineOpt can help you here? @manuelma @mihlema

[rputris]

I have deleted the files from github and redownloaded them. I opened the project with Spine Toolbox and did exactly as you have said and upgraded both the input and out data store items but when I try to execute the project it gives me the same error. I have a feeling something is wrong with my whole Spine toolbox or SpineOpt installation on my computer but I don't know what it is unfortunately. The reason I say this is because I am experiencing similar problems with other examples.

[Tasqu]

Based on the traceback, it would seem that the SpineOpt database template is out of date as well. You should be able to update it following the instructions in the Importing the SpineOpt database template section of the tutorial.

Unfortunately, I don't think these case studies have been kept up to date with latest changes to SpineOpt after the project ended, so there's probably no guarantee they will work as is. @manuelma or @mihlema might know better.

[manuelma]

Based on the traceback, it would seem that the SpineOpt database template is out of date as well. You should be able to update it following the instructions in the Importing the SpineOpt database template section of the tutorial.

I wouldn't recomend to do this. The db is upgraded automatically by run_spineopt.

[manuelma]

@rputris did you notice this line in the warning from spineopt?

┌ Warning: The data structure is not the latest version.│ SpineOpt might still be able to run, but results aren't guaranteed.│ Please use run_spineopt(url_in; upgrade=true) to upgrade.

This is trying to tell you what to do. Basically you need to modify the program that runs spine opt. Double click on the Tool (hammer icon) and you will see the code that is running. Modify the line run_spineopt(...) with run_spineopt(...; upgrade=true)

[DillonJ]

run_spineopt(...; upgrade=true)

Do we think that the SpineOpt plugin should be updated to have upgrade=true by default?

[manuelma]

Do we think that the SpineOpt plugin should be updated to have upgrade=true by default?

If we do that, I don't see why we would keep upgrade=false as default in the definition of the run_spineopt function in the SpineOpt package itself. We can set it to true, I don't think it's a very big deal. Purists will say that the application should never modify the user's data without the user telling it explicitely to do so, but again I don't think that claim is relevant here. The only drawback of upgrade=true is the database cannot be used by previous versions of SpineOpt...

[rputris]

@rputris did you notice this line in the warning from spineopt?

┌ Warning: The data structure is not the latest version.│ SpineOpt might still be able to run, but results aren't guaranteed.│ Please use run_spineopt(url_in; upgrade=true) to upgrade.

This is trying to tell you what to do. Basically you need to modify the program that runs spine opt. Double click on the Tool (hammer icon) and you will see the code that is running. Modify the line run_spineopt(...) with run_spineopt(...; upgrade=true)

Hi Manuel, yes I did notice the warning but I was not sure what to do about it (or how to do it). I am very new to all of this. I have tried what you suggested and it just keeps trying to upgrade without getting anywhere. It has been running for a very long time (over half an hour then I pressed stop). Here is a screenshot of what is happening. Image attached below.

In general I have the following questions which I need you help with (one of the main reasons for me trying to get this example to work):

1. How can one change water head in a tank (I suppose a tank can be represented by a node)?
2. How can one change temperature of a node?
3. How can one change speed of water flow in a pipe (I am guessing a pipe would be a connection?)?
4. How can one change speed of a motor? Can a 'unit' be used as a motor?
   Essentially, what are the mechanisms which one can use to change the state of a node for example? Would it be done by placing a unit between two nodes of the same commodity type (say water) and adding parameters to the unit to change the state in the next node?

Thank you for your time and assistance.