Fast(er) direct solver interfaces

[whart222]

I think users would generally expect direct solver interfaces to be faster, since we're avoiding file I/O. However, I think this is not always the case. For example, see #515

We need to critique our direct solver interfaces and assess whether we can setup/analyze problems more effectively. This activity could involve:

1. Identifying bottlenecks in existing solver interfaces
2. Exploring strategies for avoiding canonicalization in Pyomo (e.g. directly generating solver expressions from Pyomo expression trees)
3. Interfacing with POEK/COEK to see if their interfaces can resolve known bottlenecks.

NOTE: Aside from #51 and #1169, I only have anecdotal evidence that direct solver interfaces aren't faster. I would recommend confirming that they are faster before closing this issue.

[emma58]

I'm not sure whether @michaelbynum's recent enhancements change this, but the other thing that can make the persistent interface slower is hacking around functionality that's not yet in the persistent interface. The obvious one is changing the right-hand side of a constraint: The only way I know how to do this now is to remove and add the constraint, but I've never found this strategy to be worth the time. I just point this out because I wonder if there aren't other cases where the potential performance enhancements might be in what is not there rather than what is.

[qtothec]

I have been talking to Steve about making the GAMS direct "solver" faster by constructing in-memory GMO objects rather than writing a file, as well.

[qtothec]

As a follow-up to this, Steve Dirkse sent me an example for building up and solving an NLP using in-memory GMO and GEV.

[carldlaird]

I am not sure that #515 is the correct issue to reference. Does anyone know the issue number that shows a case where the direct solvers are slower than the indirect solvers?

[qtothec]

Well, for the GAMS case, I have this: #215 (comment)

I don't know about the MIP solver cases.

[whart222]

@carldlaird I updated the references to related issues. Both of these concern the CPLEX python solver interface.

[jsiirola]

Archived on the master Performance Proposals Issue (#1430). Closing this performance proposal until active development has begun.

[staadecker]

Fyi I know that for the large models I'm working with the direct gurobi solver interfaces is slower than the IO gurobi solver interface