High quality OBJ 3D model export option

[niftyflora]

The preview videos look better in terms of mesh quality and also got even textures with transparency, for example when you input a jellyfish image.

I was wondering if there is any way we could export a high quality 3D model with a quad polygon mesh and also the texture and uv's instead of a low quality glb, that would be amazing to be able to export either the low quality game ready asset or the high quality 3D model. Also I would love to be able to input more than one image sometimes.

Thank you so much in advanced TRELLIS is an amazing technology and it's really a game changer.

[scottmudge]

I added a hacky way to do this on my fork:

scottmudge@fb97d50

It requires installing the pymeshlab library. On windows, there's some issues with the way pymeshlab generates texture images. I had to implement a few work-arounds to get it to properly generate an OBJ+MTL and PNG texture. Not sure if these workarounds will work (or are necessary) on linux, so keep that in mind.

I also changed the app.py behavior to save to the <local_repo>/output directory, rather than C:\tmp, and now all the generated files are in their individual subdirectory, based on the uuid (trial_id) generated when you load an input image.

Also expanded the range of the simplification step slider so you can have higher resolution meshes, and the texture size slider as well.

A bit hacky, but seems to generate valid OBJ+MTL files.

Enabling OBJ export (new checkbox) also breaks the mesh viewer in the Gradio UI, which doesn't seem to like OBJ files, but you can just use MeshLab or some other OBJ viewer to view the mesh.

[blackalice]

@scottmudge Works great for me on Windows. Cheers for this!

[niftyflora]

I added a hacky way to do this on my fork:

scottmudge@fb97d50

It requires installing the pymeshlab library. On windows, there's some issues with the way pymeshlab generates texture images. I had to implement a few work-arounds to get it to properly generate an OBJ+MTL and PNG texture. Not sure if these workarounds will work (or are necessary) on linux, so keep that in mind.

I also changed the app.py behavior to save to the <local_repo>/output directory, rather than C:\tmp, and now all the generated files are in their individual subdirectory, based on the uuid (trial_id) generated when you load an input image.

Also expanded the range of the simplification step slider so you can have higher resolution meshes, and the texture size slider as well.

A bit hacky, but seems to generate valid OBJ+MTL files.

Enabling OBJ export (new checkbox) also breaks the mesh viewer in the Gradio UI, which doesn't seem to like OBJ files, but you can just use MeshLab or some other OBJ viewer to view the mesh.

thank you so much for this! this way we can export much higher quality models the only downside is these are still triangle meshes (no subdiv possible) and the obj textures are kind of messed up with weird artifacts

[wes-kay]

I added a hacky way to do this on my fork:

scottmudge@fb97d50

It requires installing the pymeshlab library. On windows, there's some issues with the way pymeshlab generates texture images. I had to implement a few work-arounds to get it to properly generate an OBJ+MTL and PNG texture. Not sure if these workarounds will work (or are necessary) on linux, so keep that in mind.

I also changed the app.py behavior to save to the <local_repo>/output directory, rather than C:\tmp, and now all the generated files are in their individual subdirectory, based on the uuid (trial_id) generated when you load an input image.

Also expanded the range of the simplification step slider so you can have higher resolution meshes, and the texture size slider as well.

A bit hacky, but seems to generate valid OBJ+MTL files.

Enabling OBJ export (new checkbox) also breaks the mesh viewer in the Gradio UI, which doesn't seem to like OBJ files, but you can just use MeshLab or some other OBJ viewer to view the mesh.

Does this support transparency, glass, etc?

[niftyflora]

the uv's get scrambled around when exporting high polly objects in gbl and also in obj for some reason

[scottmudge]

Yeah I mean if they're 'scrambled' in the gbl, the conversion to obj isn't going to help. I haven't really dove into how it's generating/mapping the UVs internally, but it does look like there is very little border region between UV islands, and it's possible if the density is too high and the texture is too small, the UVs get messed up. Or it could be some other issue internal to the Trellis code, I'll have to dig into it a bit deeper later.

[bitplane]

Apologies if I'm stating the obvious here, and without knowing the full picture, but I think the best workflow is to use meshlab to interactively figure out the steps to produce a good mesh (a lot of trial and error, and no undo function), then automate it all with pymeshlab.

[scottmudge]

Well that was the goal of what I was doing on my fork. I noticed using the GLB meshes, the meshes themselves weren't manifold (the vertices weren't always joined), so I ended up having to merge close vertices and removing isolated pieces in Meshlab. Thus, automating it in pymeshlab as part of the Trellis example app.

I'm sure there's more processing that could be done, but recovering valid UV mapping from an existing mesh+texture is usually something you have to do by hand. Meshlab does allow remapping vertex colors to a texture, but it's very slow, and your texture resolution is ultimately limited to the spatial resolution/vertex density of your mesh.

There's other mesh filtering that can obviously be done... smoothing, hole filling, etc., but that (in my experience) is more case-/mesh-specific. On the whole, after close-vertex merging and removing isolated pieces, the meshes produced by Trellis are far superior to other image-to-mesh options currently. Nothing is going to be perfect out-of-the-box.

[niftyflora]

and any way to export quad meshes with uvs from trellis? that would be the dream

[niftyflora]

update: the scrambling is only happening at the viewport preview, the renders are fine