Skip to content

Commit

Permalink
Update Python VLM example in README (#1178)
Browse files Browse the repository at this point in the history
Existing example uses an undefined "read_image" function, and using
max_new_tokens in pipe.generate() resulted in an error with latest
nightly.

I updated the example to work out of the box. Makes it a bit longer, but
this section is hidden by default in the README, so it doesn't add to
visual clutter for people just visiting the repo. Also added links to
the relevant samples.

---------

Co-authored-by: Vladimir Zlobin <[email protected]>
  • Loading branch information
helena-intel and Wovchena authored Nov 21, 2024
1 parent cd05c8e commit 89865c3
Showing 1 changed file with 21 additions and 4 deletions.
25 changes: 21 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,17 +117,34 @@ optimum-cli export openvino --model openbmb/MiniCPM-V-2_6 --trust-remote-code --

### Run generation using VLMPipeline API in Python

See [Visual Language Chat](https://github.com/openvinotoolkit/openvino.genai/tree/master/samples/python/visual_language_chat) for a demo application.

Run the following command to download a sample image:

```sh
curl -O "https://storage.openvinotoolkit.org/test_data/images/dog.jpg"
```

```python
import numpy as np
import openvino as ov
import openvino_genai as ov_genai
#Will run model on CPU, GPU is a possible option
from PIL import Image

# Choose GPU instead of CPU in the line below to run the model on Intel integrated or discrete GPU
pipe = ov_genai.VLMPipeline("./MiniCPM-V-2_6/", "CPU")
rgb = read_image("cat.jpg")
print(pipe.generate(prompt, image=rgb, max_new_tokens=100))

image = Image.open("dog.jpg")
image_data = np.array(image.getdata()).reshape(1, image.size[1], image.size[0], 3).astype(np.uint8)
image_data = ov.Tensor(image_data)

prompt = "Can you describe the image?"
print(pipe.generate(prompt, image=image_data, max_new_tokens=100))
```

### Run generation using VLMPipeline in C++

Code below requires installation of C++ compatible package (see [here](https://docs.openvino.ai/2024/get-started/install-openvino/install-openvino-genai.html#archive-installation) for more details)
Code below requires installation of C++ compatible package (see [here](https://docs.openvino.ai/2024/get-started/install-openvino/install-openvino-genai.html#archive-installation) for more details). See [Visual Language Chat](https://github.com/openvinotoolkit/openvino.genai/tree/master/samples/cpp/visual_language_chat) for a demo application.

```cpp
#include "load_image.hpp"
Expand Down

0 comments on commit 89865c3

Please sign in to comment.