Subspace expansion

[b-kloss]

This adds subspace expansion for dynamical adaptation of bond-dimension in 1-site TDVP algorithm. The subspace expansion scheme is similar to an expansion for uniform MPS as used within the VUMPS algorithm, see https://doi.org/10.48550/arXiv.1701.07035.

Currently, the subspace expansion is performed once before for every timestep (of a certain order, not per half-sweep).

parameters for the subspace expansion include:
atol - the tolerance for determining the nullspace of a site-tensor
cutoff - the cutoff for singular values of the gradient
cutoff_compress - SVD cutoff in initial two-site tensor SVD in subspace expansion (allows for redistributing bond-dimension between different QN sectors even after the maximum bond-dimension is reached, but introduces truncation error into 1-site TDVP)

Issues:

• The subspace expansion doesn't work for non-QN ITensors (a fix would be to add a dummy QN, but this is not currently done.)
• The subspace expansion fails for product states with QN ITensors (by design). A workaround is one or a few steps of two-site TDVP initially to grow the bond-dimension away from 1.
• cutoff in subspace expansion for 1-site TDVP is not easily related to cutoff in two-site TDVP (even after rescaling with timestep). To get similar bond-dimension increase, we need to choose many orders of magnitude larger cutoff for the subspace expansion (see test).
• The formal scaling of this implementation of the subspace expansion is that of 2-site TDVP. A direct solution can be implemented, with scaling identical to 1-site TDVP. This implementation was started (src/subspace_expansion_krylov.jl), but is not functional yet.

In the future we may want to include more control over when to perform subspace expansion (for example performing it only if there is less than a certain number of singular values below a threshold, in order to reduce the computation effort).

[mtfishman]

@b-kloss ITensor/ITensors.jl#929 is merged and registered in ITensors 0.3.16 so you can remove the nullspace implementation from this PR and set the compat entry of ITensors to 0.3.16 here:

ITensorTDVP.jl/Project.toml

Line 14 in 867a15a

ITensors = "0.3.3"

.

[mtfishman]

@b-kloss we've decided to move the functionality in this repository to https://github.com/mtfishman/ITensorNetworks.jl, please move this PR over there if you plan to continue working on this.

[mtfishman]

This should get reimplemented in https://github.com/ITensor/ITensorMPS.jl since the MPS solver code will be developed there from now one. I'll close for now and we can track it in ITensor/ITensorMPS.jl#41.

[emstoudenmire]

Agreed