Recreate Towhee image embedding with timm #2215
-
In order to fully understand how Towhee works I wanted to recreate image embeddings with timm.
Towhee code
This gives me a result like below
While running towhee I looked into ~/.cache/torch/hub/checkpoint for the model weights that were downloaded so that I could use the same.
Timm codeNow I tried to recreating the same result purely in Timm.
The result I get however is different
I would like to understand how I can recreate the same result as produce by Towhee (the same vector). |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 7 replies
-
https://towhee.io/image-embedding/timm/src/branch/main/timm_image.py import timm
import torch
from PIL import Image as PILImage
from timm.data.transforms_factory import create_transform
from timm.data import resolve_data_config
from timm.models.factory import create_model
model_name = 'resnet50'
model = create_model(model_name, pretrained=True, num_classes=1000)
model.eval()
config = resolve_data_config({}, model=model)
tfms = create_transform(**config)
read_image = PILImage.open(EXAMPLE_IMG)
inputs = torch.stack([tfms(read_image)])
features = model.forward_features(inputs)
global_pool = torch.nn.AdaptiveAvgPool2d(1)
features = global_pool(features)
features = features.flatten(1)
vecs = features.squeeze(0).detach().numpy()
print(vecs) |
Beta Was this translation helpful? Give feedback.
-
@junjiejiangjjj
If I read in 15 images and compare the results to towhee with 15 images, the results are identical. I've been investigating whole afternoon to understand why but for the life of my I can't understand why with multiple images at some point the rounding starts to change. |
Beta Was this translation helpful? Give feedback.
https://towhee.io/image-embedding/timm/src/branch/main/timm_image.py
If use timm: