0.70.0

The ZenML 0.70.0 release includes a significant number of database schema changes and migrations, which means upgrading to this version will require extra caution. As always, please make sure to make a copy of your production database before upgrading.

Key Changes

• Artifact Versioning Improvements: The handling of artifact versions has been improved, including the API improvements like the ability to batch artifact version requests to improve the execution times and more types for the step input/output artifacts, including multiple versions of the same artifact (e.g. model checkpoints), to improve the UX using ZenML UI or while working directly with the API.
• Scalability Enhancements: Various scalability improvements have been made, such as reducing unnecessary server requests and incrementing artifact versions server-side. These enhancements are expected to provide significant speed and scale improvements for ZenML users.
• Metadata management: Now, all the metadata-creating functions are gathered under one method called log_metadata. It is possible to call this method with different inputs to log run metadata for artifact versions, model versions, steps, and runs.
• The oneof filtering: This allows to filter entities using a new operator called oneof. You can use this with IDs (UUID type) or tags (or other string-typed attributes) like this PipelineRunFilter(tag='oneof:["cats", "dogs"]').
• Documentation Improvements: The ZenML documentation has been restructured and expanded, including the addition of new sections on finetuning and LLM/ML engineering resources.
• Bug Fixes: This release includes several bug fixes, including issues with in-process main module source loading, and more.

Caution: Make sure to back up your data before upgrading!

While this release brings many valuable improvements, the database schema changes and migrations pose a potential risk to users. It is strongly recommended that users:

• Test the upgrade on a non-production environment: Before upgrading a production system, test the upgrade process in a non-production environment to identify and address any issues.
• Back up your data: Ensure that you have a reliable backup of your ZenML data before attempting the upgrade.

What's Changed

• Optimizing the CI workflows by @bcdurak in #3145
• Adding 0.68.0 to the migration tests by @bcdurak in #3144
• Move step durations to body by @schustmi in #3046
• Docs on ZenML setup by @strickvl in #3100
• Remove wrongly set Model.was_created_in_this_run attribute by @schustmi in #3129
• Allow specifying run tags in pipeline configuration by @schustmi in #3130
• Fix materializer type compatibility check during loading by @schustmi in #3105
• [docs] Add icons to headers in docs by @wjayesh in #3149
• fix icons and remove redundant file by @wjayesh in #3150
• Merge 0.68.1 release into develop by @schustmi in #3153
• Allow filtering pipeline runs by stack component by @schustmi in #3142
• Allow artifact response as step input by @schustmi in #3134
• Filter component by user name by @schustmi in #3126
• [docs] Restructure how-to section to make it more readable by @wjayesh in #3147
• ZenML Pro web login implementation by @stefannica in #3141
• Scalability improvements: Reduce misc/hydration server requests by @schustmi in #3093
• Fix in-process main module source loading by @schustmi in #3119
• Catch assertion in GH library by @schustmi in #3160
• Enable cache precomputation for run templates by @schustmi in #3156
• Add LLM and ML engineering books to README by @htahir1 in #3159
• Add helper method to quickly create run template from pipeline by @schustmi in #3155
• Add CLI command to export stack requirements by @schustmi in #3158
• Scalability improvements: Increment artifact version server side by @schustmi in #3095
• Update OpenAI integration by @safoinme in #3163
• Remove deprecated torch version constraint by @safoinme in #3166
• vLLM model deployer by @dudeperf3ct in #3032
• Don't initialize client during flavor sync by @schustmi in #3168
• Cleanup materializer temporary directories after step execution by @schustmi in #3162
• Fix langchain in API docs by @avishniakov in #3171
• Finetuning guide by @strickvl in #3157
• Fix mypy issue vllm evidently by @safoinme in #3169
• Add artifact version batch request by @schustmi in #3164
• Add missing section links by @strickvl in #3172
• Fix uvloop mypy by @avishniakov in #3174
• Multiple output versions for a step outputs by @avishniakov in #3072
• Simplify Metadata handling by @AlexejPenner in #3096
• assign value to component_name in preset stack registration by @hirekk in #3178
• Updating the template versions with zenml login by @bcdurak in #3177
• Better input artifacts typing by @avishniakov in #3099
• Refactor environment setup and caching by @safoinme in #3077
• Fix spelling errors by @safoinme in #3181
• Prevent some race conditions by @schustmi in #3167
• Update stack deployments with latest features by @stefannica in #3183
• Terraform best practices by @htahir1 in #3131
• Fix sagemaker pipeline URLs by @stefannica in #3176
• Fix lightning orchestrator for multi-step pipelines by @wjayesh in #3170
• Port bugfixes from #2497 by @avishniakov in #3179
• Removing the enable_cache from the config files by @bcdurak in #3184
• Don't pass tags to step config by @schustmi in #3186
• New log_metadata function, new oneof filtering, additional run_metadata filtering by @bcdurak in #3182

New Contributors

• @hirekk made their first contribution in #3178

Full Changelog: 0.68.1...0.70.0