Wrong aspect ratio for VLMPlanner screenshot

[fayeli]

I noticed that VLMPlanner is consistently doing screenshot at a wrong aspect ratio for my secondary monitor

My monitor is a 3440 x 1440 ultrawide, so this may be a rare edge case.

ShowUI, on the other hand, is doing the screenshot at the correct aspect ratio. The mouse coordinate calculation seems to be fine for the next step and the VLMPlanner screenshot issue doesn't seem to affect the model action, at least for my particular test case.

Anyways, great work Showlab team! I'm really enjoying seeing the development and just logging this minor issue here in case it is useful to you!

[yyyang-2019]

Hi @fayeli,
Thanks for your feedback! You're right, we actually want the screenshots passed to vlm_planner to be after resized, e.g. to 1920x1080, in order to control the inference time and token usage. We will consider visualize the image before resized (i.e. its original size) in chat interface so that it doesn't confuse the user. Thanks again for your valuable findings :)