Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix/plot crop #2592

Merged
merged 5 commits into from
Apr 4, 2024
Merged

Bugfix/plot crop #2592

merged 5 commits into from
Apr 4, 2024

Conversation

timtmok
Copy link
Contributor

@timtmok timtmok commented Apr 1, 2024

Intent

Address #502
Original PR: github.com/posit-dev/positron-python/pull/432

Approach

The plot is requested with a size that will fill the plot pane. The resulting image was cropped and being displayed at 100% scale so it doesn't fill the pane. Calling format_display_figure was setting options that results in a cropped image.

Rendering the figure to a buffer and base64 encoding it is doing the same as format_display_figure but doesn't seem to crop the image. I don't know why it gets cropped or which option that causes it.

Another issue was setting BASE_DPI to 96 (100 is the matplotlib default) that caused a slight change in the resulting image size. Using 96dpi would convert a requested image size of 100x100 to 96x96@96dpi. I think this is a result of how matplotlib requires the image to be set in inches. We convert the resolution into inches but also changed the DPI. Since we are setting a lower DPI, there are less pixels per inch (aka DPI) and the image is smaller than requested.

QA Notes

This can be reproduced with a dynamic plot using code like this:

import numpy as np
import matplotlib.pyplot as plt

v = np.arange(365, 0, -1)

fig, ax = plt.subplots()
ax.set_xlabel('Day of the year')
ax.set_ylabel('Days until New Year\'s Eve')

plt.xlim(0, 365)
plt.ylim(1, 365)

plt.plot(v)

Switching the sizing to Fill should fill the Plot pane image area (minus space at the top of the image reserved for the progress bar; about 2 pixels).

The current issue looks like this:
image

When rendered with the fix:
image

It's most noticeable in the Fill mode but other modes also show how the image doesn't fill in the expected direction.

@timtmok timtmok force-pushed the bugfix/plot-crop branch from 94c34e9 to f88f4b2 Compare April 1, 2024 20:58
@timtmok timtmok requested a review from seeM April 2, 2024 15:48
timtmok added 2 commits April 2, 2024 11:50
Render without bbox=tight option that format_display_data() uses
@timtmok timtmok force-pushed the bugfix/plot-crop branch from f88f4b2 to 96d4643 Compare April 2, 2024 15:50
@timtmok timtmok force-pushed the bugfix/plot-crop branch from 96d4643 to 20403ba Compare April 2, 2024 15:55
Copy link
Contributor

@seeM seeM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, once the type errors are fixed. Thanks for figuring this out!

@@ -197,14 +199,19 @@ def _resize_pickled_figure(

figure.set_size_inches(width_in, height_in)

format_dict, md_dict = format_display_data(figure, include=formats, exclude=[]) # type: ignore
# Render the figure to a buffer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need the figure.set_dpi(dpi)above given that we pass it to savefig too?

@@ -197,14 +199,19 @@ def _resize_pickled_figure(

figure.set_size_inches(width_in, height_in)

format_dict, md_dict = format_display_data(figure, include=formats, exclude=[]) # type: ignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove the format_display_data import too?


plt.close(figure)

if was_interactive:
plt.ion()

return (format_dict, md_dict)
return format_dict
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error is because this function's return annotation is still Tuple[dict, dict], but it should be Dict[str, str], I think.

format_dict, md_dict = self._resize_pickled_figure(
pickled, width_px, height_px, pixel_ratio
)
format_dict = self._resize_pickled_figure(pickled, width_px, height_px, pixel_ratio)
data = format_dict["image/png"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this error is also due to the incorrect return type annotation on _resize_pickled_figure.

data = format_dict["image/png"]
output = PlotResult(data=data, mime_type="image/png").dict()
figure_comm.send_result(data=output, metadata=md_dict)
figure_comm.send_result(data=output, metadata={"mime_type": "image/png"})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know if this mime_type key is used anywhere?

Copy link
Collaborator

@petetronic petetronic Apr 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought this is needed by the Plot Client to construct a suitable data uri for the image?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is currently used and I was thinking to remove it entirely from the comm definition. It seems that R might use this though? I had changes in Amalthea when I tried regenerating using generate-comms.ts.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or did @seeM mean the metadata property here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I was referring to the metadata={"mime_type": "image/png"} .


# Serialize the reference figure as a base64-encoded image
data_ref, _ = format_display_data(fig_ref, include=["image/png"], exclude=[]) # type: ignore
# data_ref, _ = format_display_data(fig_ref, include=["image/png"], exclude=[]) # type: ignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# data_ref, _ = format_display_data(fig_ref, include=["image/png"], exclude=[]) # type: ignore

@petetronic
Copy link
Collaborator

Thanks for the additional fixes @timtmok I tried out the branch and your examples in Fill mode and it looks good to me!

Copy link
Contributor

@seeM seeM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@timtmok timtmok merged commit 6fa4fef into main Apr 4, 2024
23 checks passed
@timtmok timtmok deleted the bugfix/plot-crop branch April 4, 2024 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants