Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the Masking method for inpainting #16

Open
maeriil opened this issue Oct 15, 2023 · 0 comments
Open

Change the Masking method for inpainting #16

maeriil opened this issue Oct 15, 2023 · 0 comments
Labels
bug Something isn't working enhancement New feature or request help wanted Extra attention is needed

Comments

@maeriil
Copy link
Owner

maeriil commented Oct 15, 2023

We want to change the way we mask the original text from the image in order to remove it from the image. Our current approach is as follows:

  1. Crop the image section which contains the merged text section
  2. Grab its dimensions and generate an array of size height - border_size * 2 and width - border_size * 2 with each cell containing 255 value, (white colour)
  3. Add a border around the array of size border_size with each cell containing 0 value, (black colour)
  4. Inpaint the cropped section using the mask with cv2.inpaint() with inpaint radius of 7 and INPAINT_NS.
  5. Replace the cropped section in the original image with this new inpainted section

There are few issues with this method.

  • What is a good inpainting radius? Is 7 really the best possible value in our cases?
  • What is a good inpainting algorithm. cv2.INPAINT_NS is the fluid mechanic inpainting algorithm from the 2001 paper. Is this really the "optimal" algorithm for us? Is there other algorithm that we can use that is better than this?
  • The merged text section may contain other smaller text section. However, in doing so, there are empty spaces that does not contain texts. In our current approach, even these empty spaces will be replaced by the inpainting algorithm since the mask simply draws a rectangular dimension based on the merged text section. We want to change this so that we only mask on rectangular areas where the text section contains. This means that we must loop over merged text section's children and mask them individually.
  • This inpainting method works mostly on text bubbles. However, it does not work properly at all if the text is not inside a text bubble, but is in front of some other background. After the section has been inpainted, the image's backgrounds on that area will be completely ruined. We want to be able to support restoring it so we don't lose meaning of the image. This concept is similar to of Photoshop's Generative AI.
@maeriil maeriil added bug Something isn't working enhancement New feature or request help wanted Extra attention is needed labels Oct 15, 2023
@maeriil maeriil added this to the Second Version Release milestone Oct 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant