Added v0 of prometheus lm-buddy entrypoint

[aittalam]

Here's the first version of the lm-buddy prometheus entrypoint!

I tried to keep the script relatively close to the original evaluation script, as I'd like to use it as a use case for a "conversion" of some existing tool (a tutorial is being written...)

I feel it still needs some massaging as I am not 100% sure of the implementation choices I took (format of configs, using one method or another to get artifacts, etc). Happy to receive some early feedback and iterate on it soon!

[sfriedowitz]

Looking good overall with the format of the new job! A handful of comments, some bigger than others. Happy to discuss more or chat offline if anything didn't make sense.

[aittalam]

Many thanks for your comments! I will go through them (marked all of the actionable ones with a 👍) and also fix the linting issues before asking for approval 🙏

[sfriedowitz]

I'll approve this for now as most comments have been addressed. I left a few other smaller nit comments you may want to tackle as well in this PR.

In a followup, I think it would be good to take another iteration at the following:

• Refining how we store generated results to disk, and the format of the W&B artifacts. E.g., do we log cumulative metrics from the eval as a Run Table for easy visualization, in addition to the file artifact?
• Adding test coverage, both for ser/de issues on the new configs, and for the job entrypoint as a whole.

[veekaybee]

🚀