Skip to content

Release v0.5.2
Compare
Choose a tag to compare
@wangxingjun778 wangxingjun778 released this 09 Aug 13:20
· 28 commits to release/0.5 since this release
v0.5.2
138c994
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

Highlight features

  • Support Multi-modal models evaluation (VLM Eval)
  • Transform the synchronous API to asynchronous for OpenAI's API format, speed up the evaluation process up to 10x .
  • Support installation with format: pip install evalscope[opencompass] or pip install evalscope[vlmeval]

Breaking Changes

None

What's Changed

  1. Support Multi-modal models evaluation (VLM Eval)
  2. Transform the synchronous API to asynchronous for OpenAI's API format, speed up the evaluation process up to 10x .
  3. Support installation with format: pip install evalscope[opencompass] or pip install evalscope[vlmeval]
  4. Update README
  5. Add UT cases for VLM eval
  6. Update examples for OpenCompass and VLMEval eval backends
  7. Update version restrictions for ms-opencompass and ms-vlmeval dependencies.
Assets 2
Loading