Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add evaluation module and update Gradio v5 #4

Open
wants to merge 65 commits into
base: main
Choose a base branch
from

Conversation

JGrothoff
Copy link
Contributor

@JGrothoff JGrothoff commented Nov 8, 2024

This PR mainly adds features to evaluate against AASX packages and updates gradio to version 5.x.
Alongside some necessary and helpful features as well as fixes are added supporting the evaluation and webui demo.
The evaluation sources and dependencies are added as optional module.
Some fixes to the basyx-python-sdk are brought in via a fork (PR on original repo is pending).
The PDF2AAS core conversion routine was revised and used in the evaluation class.
Many small fixes are introduced using mypy for static type checking.
Static type checking and tests for python version 3.10, 3.11 are added to the CI.
Some tests were added to increase the coverage.

Features

Core

  • select preprocessor based on extension and support chaining
  • allow AAS as dictionary and return properties
  • use tabulate for table formating in preprocessor and extractor

Evaluation

  • log only preferred language definition name
  • add class_ids from datasheets
  • preprocess and cut datasheet on article load
  • make eval path optional
  • add optional evaluation modul

Demo

  • add aas template filter
  • add preprocessor selection
  • use extensive prompt on batch size 1
  • adapt to gradio 5.0

Model

  • add method to get preferred definition
  • add function to get definition name with preferred language

Preprocessor

  • add pdfplumber text preprocessor
  • change camelot to pdfplumber

Extractor

  • add prompt order option for prompt caching

Generator

  • add overwrite option for embedded data spec
  • add search for datasheets in aas templates
  • add filter options for aas template properties

@JGrothoff JGrothoff force-pushed the dev branch 2 times, most recently from 9e07c4c to 5cdd540 Compare November 28, 2024 12:53
@JGrothoff JGrothoff marked this pull request as ready for review November 28, 2024 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant