Added train by epoch for Trainer and added support for texts

[MarcusLoppe]

Trainer

• Added train function that will train with epochs instead of steps
  - Added option to display graph (maybe remove since it requires matlib?)
• Added checkpoint_every_epoch to init so checkpoints can be saved every X.

Data.py

• Modified custom_collate so it wont pad if the data isn't a tensor

MeshAutoencoder

• Was crashing if texts was in the args, added dummy parameterto prevent this.

Setup.py

• The setup file was missing a comma

[lucidrains]

this is awesome Marcus! will take a look at it tomorrow morning and get it merged!

[adeerAI]

Awesome work, thanks for including your suggestions to the main, this allows better understanding on the user's side.

[lucidrains]

ah, so the text is actually only conditioned through the transformer stage through cross attention

basically the autoencoder is given the job of only compressing meshes

[MarcusLoppe]

Yes I know :) But if you pass it a dict with texts it will give a error since the arg doesnt exist.
So then you would need two dataset classes.

Either replace the model(**forward_args) so it uses the prarameters directly:
model(vertices = data["vertices"], faces = data["faces"])

Or just implement a dummy texts :) There is probably a better solution

[lucidrains]

oh got it! yea, i can take care of that within the trainer class (just scrub out the text and text_embed keys)

[MarcusLoppe]

I don't think that will work, I'm not 100% since the dataloader passes the data and maybe copies it(?).

But it won't work if you access it without copying it since the dataset is returning the data and not copying/cloning, when you do del on a key, it will remove it completely from the dataset.
So if you train the encoder and then want to train a transformer, you'll need to recreate the dataset since the texts key is removed.

[fire]

I prefer if the dataset returns the text with each vertices and faces.

[lucidrains]

small typo here diplay

[lucidrains]

so i haven't documented this, but you can already use wandb.ai experiment tracker

you just have to do

trainer = Trainer(..., use_wandb_tracking = True)

with trainer.trackers('meshgpt', 'one-experiment-name'):
  trainer.train()

[MarcusLoppe]

Btw since I don't really think grad_accum_every is very useful I removed it from the train function, what is your option?

I forgot and left grad_accum_every in the loss function, so if it wont be used in the train function it should be removed from:

self.accelerator.backward(loss / self.grad_accum_every)

[lucidrains]

Btw since I don't really think grad_accum_every is very useful I removed it from the train function, what is your option?

I forgot and left grad_accum_every in the loss function, so if it wont be used in the train function it should be removed from:

self.accelerator.backward(loss / self.grad_accum_every)

i'm sure researchers will want to stretch to the next level if this approach pans out (multiple meshes, scenes etc)

probably good to keep it for the gpu poor

[MarcusLoppe]

Another thing :) I'm not very experienced in using github forks but it seems like the pull request added later commits then when I made the request.

I made bit of a error and replaced entire meshgpt_pytorch.py since there was some weird stash thing and I wanted to ensure it was up to date. I reverted but it seems like that stash thing messed it up bit, please double check if this is the case

[Ginjing-Yuan]

Hi! Thanks for your amazing job! May I ask why in MeshGPT_dem.ipynb has a
AttributeError: module 'meshgpt_pytorch.mesh_render' has no attribute 'combind_mesh_with_rows'