[TRACKER] SHARK Studio Roadmap

[monorimet]

SHARK Studio Roadmap

This project establishes and tracks a plan for phased releases of the SHARK Studio WebUI.

There are three objectives of this roadmap:

• Define product features, supported OS, and devices.
• Scheduling of features.

Product Goal

• Demonstrate SHARK/IREE ML acceleration framework capabilities
• Support common generative AI workloads for AMD users, as a standalone application.
• Example application showing how to integrate with SHARK/IREE inference solutions.

---

Product Definition

OS: Windows 11 and Ubuntu 24.04
Device Targets:

• 7900xtx
• Strix iGPUs
• MI300x
• MI325
• MI250
• LLVMCPU (x86)
  Serving:
• shortfin

Stable Diffusion (SDXL, SDXL-Turbo, SD3):
Features:

• Text to Image: given a textual prompt, generate an image.
• Custom Checkpoints/Weights: bring your own fine-tuned model weights as .safetensors
• LoRA Embeddings: bring your own embeddings for generating specific characters, objects, or styles.
• Output Image Size: 128x128 - 1024x1024, multiples of 8. (model restrictions apply)

Chatbot (LLaMa3):
Features:

• Text to Text: Have a conversation with text as input.
• Ability to query user-submitted document.

ML Perf Models (TBD)

Product Architecture

---

Schedule

Feature	Release
Device: MI300, OS: Linux, Models: SDXL, LLama3, Features except LoRA, document chatbot input	Nov 2024
Device: MI300x, Strix, OS: Win, Linux, Models: SDXL (+Turbo), SD3, Features: All	Jan 2025
Device: MI300x, Strix, OS: Win, Linux, Models: SDXL (+Turbo), SD3, LLaMa3, Features: All	Mar 2025

Nov. 2024 task tracking:

Task	Date	Assignees	PR
Shortfin SD serving (Python) + SDXL precompiled serving test added to e2eshark	Oct. 25	@monorimet	nod-ai/shark-ai#265
Sharktank model exports, uploads + pytorch-sourced component testing in e2eshark	Nov. 1	@monorimet @saienduri
WebUI (SDXL, LLaMa3) (fixed set of config parameters) + webui test in e2eshark	Nov. 8	@gpetters-amd
QA + bugfixes, unknowns	Nov. 15	@monorimet @gpetters-amd @pdhirajkumarprasad

Building Blocks

Component	Source
CLIP	turbine-models (SHARK-ModelDev)
Unet	sharktank
VAE	turbine-models (SHARK-ModelDev)
Scheduler	turbine-models (SHARK-ModelDev)

[gpetters-amd]

I'm happy to jump in on the WebUI and CI stuff once I'm done with the internal work.

[kumardeepakamd]

Super good diagram, schedule and clearly called out tasks with target dates @monorimet !!

[VeteranXT]

What works here? I heard Shark is broken?