M2 Ultra GPU not working

[arielt11]

Hi, I previously had the older VITmatte working fine in Nuke 15 (it appeared as a green node). After upgrading to the latest version of VITmatte, I’m encountering an issue where, with GPU usage enabled, the screen turns black with red horizontal lines. When I disable the GPU, everything works correctly. Other than that, thank you so much for these tools, they are GAME CHANGING!!!

[arielt11]

edit *** it works in 15.0v3 but it doesn't work in 15.1 v2

[magnoborgo]

Can confirm this on M1 chipset too. OSX 14.6.1
Nuke 15.0v4 works fine.

Nuke 15.1v1 wont work with the following message below
There's something funky going on because even if you disable "use GPU if available" it won't allow for a cpu processing of the node.

UNSUPPORTED (log once): POSSIBLE ISSUE: unit 0 GLD_TEXTURE_INDEX_2D is unloadable and bound to sampler type (Float) - using zero texture because texture unloadable
Exception caught processing model: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
  File .../__torch__.py, line 13, in forward
    trimap = torch.slice(_1, 1, 3, 4, 1)
    image_and_trimap = {"image": image, "trimap": trimap}
    _2 = (self.model).forward(image_and_trimap, )
          ~~~~~~~~~~~~~~~~~~~ <--- HERE
    _3 = torch.contiguous(_2["phas"], memory_format=0)
    return _3
  File .../vitmatte.py, line 16, in forward
    _0 = (self).preprocess_inputs(batched_inputs, )
    images, H, W, = _0
    features = (self.backbone).forward(images, )
                ~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
    outputs = (self.decoder).forward(features, images, )
    _1 = torch.slice(outputs["phas"], 0, 0, 9223372036854775807, 1)
  File .../vit.py, line 19, in forward
    _2 = self.pretrain_use_cls_token
    _3 = ((torch.size(x0))[1], (torch.size(x0))[2])
    x1 = torch.add(x0, _0(_1, _2, _3, ), alpha=1)
                       ~~ <--- HERE
    _4 = self.blocks
    _5 = getattr(_4, "0")
  File .../utils.py, line 34, in get_abs_pos
  if _3:
    _5 = torch.reshape(abs_pos0, [1, size, size, -1])
    new_abs_pos = _0(torch.permute(_5, [0, 3, 1, 2]), [h, w], None, "bicubic", False, None, )
                  ~~ <--- HERE
    _6 = torch.permute(new_abs_pos, [0, 2, 3, 1])
    _4 = _6
  File .../___torch_mangle_36.py, line 236, in interpolate
                                  _89 = scale_factor_list0[0]
                                  _90 = scale_factor_list0[1]
                                  _91 = torch.upsample_bicubic2d(input, output_size, align_corners10, _89, _90)
                                        ~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
                                  _87 = _91
                                else:

Traceback of TorchScript, original code (most recent call last):
  File .../vit.py, line 496, in forward
        x = self.patch_embed(x)
        if self.pos_embed is not None:
            x = x + get_abs_pos(
                    ~~~~~~~~~~~ <--- HERE
                self.pos_embed, self.pretrain_use_cls_token, (x.shape[1], x.shape[2])
            )
  File .../utils.py, line 166, in get_abs_pos

    if size != h or size != w:
        new_abs_pos = F.interpolate(
                      ~~~~~~~~~~~~~ <--- HERE
            abs_pos.reshape(1, size, size, -1).permute(0, 3, 1, 2),
            size=(h, w),
  File .../functional.py, line 3175, in interpolate
    elif input.dim() == 4 and mode == 'bicubic':
        assert align_corners is not None
        return torch._C._nn.upsample_bicubic2d(input, output_size, align_corners, sfl[0], sfl[1])
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
    else:
        raise NotImplementedError("Input Error: Only 3D, 4D and 5D input Tensors supported"
RuntimeError: "compute_indices_weights_cubic" not implemented for 'Half'

[magnoborgo]

Found a workaround. Inside the Gizmo, disable "optimize for speed and memory" checkbox on the Inference1 node.

[magnoborgo]

in fact looks like the "cpu" render issue is also connected to the "optimize for speed and memory" checkbox.
Looks like that even if hidden when "use gpu" is off , if the checkbox is checked it will throw gpu errors on cpu render.
Maybe a bug/bad design by foundry on the inference node.