[NDTensorsGPUArraysCoreExt] Fix nonuniform Diag-Dense contractions on GPU

[mtfishman]

This fixes:

using ITensors: Index, random_itensor, diag_itensor
using Metal: mtl
device = mtl
d = 3
i, j = Index.((d, d))
a = device(random_itensor(i, j))
d = device(diag_itensor(randn(d), i, j))
a * d

whereas before that contraction failed with a scalar indexing error. The fix is fairly crude and is based on converting the Diag tensor to Dense, similar to the fix for contracting uniform Diag with Dense in #1453.

I've also added some definitions for mapreduce for Tensors with storage types Dense, Diag, and BlockSparse, and defined sum(::ITensor) and prod(::ITensor) which make use of those definitions and therefore work on GPU and take advantage of sparsity.

This fixes the issues raised in https://itensor.discourse.group/t/gpu-issues-scalar-indexing-is-disallowed-on-sum-itensor-and-traces-of-products-involving-diagonal-itensors/1828.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.21%. Comparing base (7d5ecf9) to head (c631b00).

❗ Current head c631b00 differs from pull request most recent head c308634

Please upload reports for the commit c308634 to get more accurate results.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1511   +/-   ##
=======================================
  Coverage   79.21%   79.21%           
=======================================
  Files         148      148           
  Lines        9658     9658           
=======================================
  Hits         7651     7651           
  Misses       2007     2007           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mtfishman]

@kmp5VT I also went ahead and started using NDTensors.map_diag! to implement the definition of that function for ITensor, as a followup to #1497.

[mtfishman]

@kmp5VT another thing I added in this PR is a generic definition of == for Tensors which avoids scalar indexing and therefore works on GPU. I switched over some tests to use that new definition. It looks like we could remove a few @allowscalar calls in the tests now that we have that.

[kmp5VT]

@kmp5VT another thing I added in this PR is a generic definition of == for Tensors which avoids scalar indexing and therefore works on GPU. I switched over some tests to use that new definition. It looks like we could remove a few @allowscalar calls in the tests now that we have that.

This is great, thanks!!