Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: dv3-imagination notebook #290

Merged
merged 5 commits into from
May 20, 2024
Merged

fix: dv3-imagination notebook #290

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
306125d
fix: dv3-imagination notebook
michele-milesi May 20, 2024
6d7ef9b
remove notebook outputs
michele-milesi May 20, 2024
accdfdf
remove jupyter clear output cell hook
michele-milesi May 20, 2024
dcd849c
remove nbconvert from dependencies
michele-milesi May 20, 2024
7100bb1
fix: set to in the dv3 imagination notebook
michele-milesi May 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
322 changes: 14 additions & 308 deletions examples/model_manager.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,133 +81,9 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Experiment: <Experiment: artifact_location='mlflow-artifacts:/242317125620601262', creation_time=1701949559261, experiment_id='242317125620601262', last_update_time=1701949559261, lifecycle_stage='active', name='mlflow_example', tags={}>\n",
"Experiment (242317125620601262) runs:\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>run_id</th>\n",
" <th>experiment_id</th>\n",
" <th>status</th>\n",
" <th>artifact_uri</th>\n",
" <th>start_time</th>\n",
" <th>end_time</th>\n",
" <th>metrics.Loss/entropy_loss</th>\n",
" <th>metrics.Test/cumulative_reward</th>\n",
" <th>metrics.Info/ent_coef</th>\n",
" <th>metrics.Info/learning_rate</th>\n",
" <th>...</th>\n",
" <th>params.algo/gae_lambda</th>\n",
" <th>params.env/action_repeat</th>\n",
" <th>params.env/grayscale</th>\n",
" <th>params.metric/aggregator/metrics/Loss/policy_loss/sync_on_compute</th>\n",
" <th>params.metric/log_level</th>\n",
" <th>tags.mlflow.user</th>\n",
" <th>tags.mlflow.source.type</th>\n",
" <th>tags.mlflow.runName</th>\n",
" <th>tags.mlflow.source.name</th>\n",
" <th>tags.mlflow.log-model.history</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1e453cf2114d43f28410803df985598a</td>\n",
" <td>242317125620601262</td>\n",
" <td>FINISHED</td>\n",
" <td>mlflow-artifacts:/242317125620601262/1e453cf21...</td>\n",
" <td>2023-12-07 11:45:59.641000+00:00</td>\n",
" <td>2023-12-07 11:46:10.350000+00:00</td>\n",
" <td>-0.687031</td>\n",
" <td>48.0</td>\n",
" <td>0.0</td>\n",
" <td>0.001</td>\n",
" <td>...</td>\n",
" <td>0.95</td>\n",
" <td>1</td>\n",
" <td>False</td>\n",
" <td>False</td>\n",
" <td>1</td>\n",
" <td>mmilesi</td>\n",
" <td>LOCAL</td>\n",
" <td>ppo_CartPole-v1_2023-12-07_12-45-58</td>\n",
" <td>/home/mmilesi/miniconda3/envs/sheeprl/lib/pyth...</td>\n",
" <td>[{\"run_id\": \"1e453cf2114d43f28410803df985598a\"...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>1 rows × 130 columns</p>\n",
"</div>"
],
"text/plain": [
" run_id experiment_id status \\\n",
"0 1e453cf2114d43f28410803df985598a 242317125620601262 FINISHED \n",
"\n",
" artifact_uri \\\n",
"0 mlflow-artifacts:/242317125620601262/1e453cf21... \n",
"\n",
" start_time end_time \\\n",
"0 2023-12-07 11:45:59.641000+00:00 2023-12-07 11:46:10.350000+00:00 \n",
"\n",
" metrics.Loss/entropy_loss metrics.Test/cumulative_reward \\\n",
"0 -0.687031 48.0 \n",
"\n",
" metrics.Info/ent_coef metrics.Info/learning_rate ... \\\n",
"0 0.0 0.001 ... \n",
"\n",
" params.algo/gae_lambda params.env/action_repeat params.env/grayscale \\\n",
"0 0.95 1 False \n",
"\n",
" params.metric/aggregator/metrics/Loss/policy_loss/sync_on_compute \\\n",
"0 False \n",
"\n",
" params.metric/log_level tags.mlflow.user tags.mlflow.source.type \\\n",
"0 1 mmilesi LOCAL \n",
"\n",
" tags.mlflow.runName \\\n",
"0 ppo_CartPole-v1_2023-12-07_12-45-58 \n",
"\n",
" tags.mlflow.source.name \\\n",
"0 /home/mmilesi/miniconda3/envs/sheeprl/lib/pyth... \n",
"\n",
" tags.mlflow.log-model.history \n",
"0 [{\"run_id\": \"1e453cf2114d43f28410803df985598a\"... \n",
"\n",
"[1 rows x 130 columns]"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"import mlflow\n",
"\n",
Expand All @@ -229,26 +105,9 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Name: mlflow_example_agent\n",
"Description: # MODEL CHANGELOG\n",
"## **Version 1**\n",
"### Author: mmilesi\n",
"### Date: 07/12/2023 12:46:10 CET\n",
"### Description: \n",
"PPO Agent in CartPole-v1 Environment\n",
"\n",
"Tags: {}\n",
"Latest Version: 1\n"
]
}
],
"outputs": [],
"source": [
"from sheeprl.utils.mlflow import MlflowModelManager\n",
"from lightning import Fabric\n",
Expand Down Expand Up @@ -315,33 +174,9 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2023/12/07 12:47:03 WARNING mlflow.utils.requirements_utils: The following packages were not found in the public PyPI package index as of 2023-10-28; if these packages are not present in the public PyPI index, you must install them manually before loading your model: {'sheeprl'}\n",
"Registered model 'mlflow_example_agent' already exists. Creating a new version of this model...\n",
"2023/12/07 12:47:04 INFO mlflow.store.model_registry.abstract_store: Waiting up to 300 seconds for model version to finish creation. Model name: mlflow_example_agent, version 2\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Registered model mlflow_example_agent with version 2\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Created version '2' of model 'mlflow_example_agent'.\n"
]
}
],
"outputs": [],
"source": [
"from sheeprl.cli import registration\n",
"\n",
Expand Down Expand Up @@ -374,31 +209,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Name: mlflow_example_agent\n",
"Description: # MODEL CHANGELOG\n",
"## **Version 1**\n",
"### Author: mmilesi\n",
"### Date: 07/12/2023 12:46:10 CET\n",
"### Description: \n",
"PPO Agent in CartPole-v1 Environment\n",
"## **Version 2**\n",
"### Author: mmilesi\n",
"### Date: 07/12/2023 12:47:04 CET\n",
"### Description: \n",
"New PPO Agent version trained in CartPole-v1 environment\n",
"\n",
"Tags: {}\n",
"Latest Version: 2\n"
]
}
],
"outputs": [],
"source": [
"model_info = mlflow.search_registered_models(filter_string=f\"name='{model_name}'\")[-1]\n",
"print(\"Name:\", model_info.name)\n",
Expand All @@ -418,32 +231,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Transitioning model mlflow_example_agent version 2 from None to staging\n"
]
},
{
"data": {
"text/plain": [
"<ModelVersion: aliases=[], creation_timestamp=1701949624027, current_stage='Staging', description=('# MODEL CHANGELOG\\n'\n",
" '## **Version 2**\\n'\n",
" '### Author: mmilesi\\n'\n",
" '### Date: 07/12/2023 12:47:04 CET\\n'\n",
" '### Description: \\n'\n",
" 'New PPO Agent version trained in CartPole-v1 environment\\n'), last_updated_timestamp=1701949660778, name='mlflow_example_agent', run_id='eefbe09e8815463eaa83c6542cbc36c7', run_link='', source='mlflow-artifacts:/242317125620601262/eefbe09e8815463eaa83c6542cbc36c7/artifacts/agent', status='READY', status_message='', tags={}, user_id='', version='2'>"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"model_manager.transition_model(\n",
" model_name=\"mlflow_example_agent\",\n",
Expand Down Expand Up @@ -489,42 +279,9 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Successfully registered model 'ppo_agent_cartpole_best_reward'.\n",
"2023/12/07 12:47:55 INFO mlflow.store.model_registry.abstract_store: Waiting up to 300 seconds for model version to finish creation. Model name: ppo_agent_cartpole_best_reward, version 1\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Registered model ppo_agent_cartpole_best_reward with version 1\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Created version '1' of model 'ppo_agent_cartpole_best_reward'.\n"
]
},
{
"data": {
"text/plain": [
"{'agent': <ModelVersion: aliases=[], creation_timestamp=1701949675859, current_stage='None', description='', last_updated_timestamp=1701949675859, name='ppo_agent_cartpole_best_reward', run_id='eefbe09e8815463eaa83c6542cbc36c7', run_link='', source='mlflow-artifacts:/242317125620601262/eefbe09e8815463eaa83c6542cbc36c7/artifacts/agent', status='READY', status_message='', tags={}, user_id='', version='1'>}"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"models_info = {\n",
" \"agent\": {\n",
Expand All @@ -547,60 +304,9 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model named mlflow_example_agent with version 1 does not exist\n"
]
},
{
"data": {
"text/plain": [
"<RegisteredModel: aliases={}, creation_timestamp=1701949570369, description=('# MODEL CHANGELOG\\n'\n",
" '## **Version 1**\\n'\n",
" '### Author: mmilesi\\n'\n",
" '### Date: 07/12/2023 12:46:10 CET\\n'\n",
" '### Description: \\n'\n",
" 'PPO Agent in CartPole-v1 Environment\\n'\n",
" '## **Version 2**\\n'\n",
" '### Author: mmilesi\\n'\n",
" '### Date: 07/12/2023 12:47:04 CET\\n'\n",
" '### Description: \\n'\n",
" 'New PPO Agent version trained in CartPole-v1 environment\\n'\n",
" '## **Transition:**\\n'\n",
" '### Version 2 from None to Staging\\n'\n",
" '### Author: mmilesi\\n'\n",
" '### Date: 07/12/2023 12:47:40 CET\\n'\n",
" '### Description: \\n'\n",
" 'Staging Model for demo\\n'\n",
" '## **Deletion:**\\n'\n",
" '### Version 1 from stage: None\\n'\n",
" '### Author: mmilesi\\n'\n",
" '### Date: 07/12/2023 12:48:36 CET\\n'\n",
" '### Description: \\n'\n",
" 'Delete model version 1\\n'), last_updated_timestamp=1701949716092, latest_versions=[<ModelVersion: aliases=[], creation_timestamp=1701949624027, current_stage='Staging', description=('# MODEL CHANGELOG\\n'\n",
" '## **Version 2**\\n'\n",
" '### Author: mmilesi\\n'\n",
" '### Date: 07/12/2023 12:47:04 CET\\n'\n",
" '### Description: \\n'\n",
" 'New PPO Agent version trained in CartPole-v1 environment\\n'\n",
" '## **Transition:**\\n'\n",
" '### Version 2 from None to Staging\\n'\n",
" '### Author: mmilesi\\n'\n",
" '### Date: 07/12/2023 12:47:40 CET\\n'\n",
" '### Description: \\n'\n",
" 'Staging Model for demo\\n'), last_updated_timestamp=1701949660803, name='mlflow_example_agent', run_id='eefbe09e8815463eaa83c6542cbc36c7', run_link='', source='mlflow-artifacts:/242317125620601262/eefbe09e8815463eaa83c6542cbc36c7/artifacts/agent', status='READY', status_message='', tags={}, user_id='', version='2'>], name='mlflow_example_agent', tags={}>"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"model_manager.delete_model(\n",
" model_name, int(latest_version.version) - 1, f\"Delete model version {int(latest_version.version)-1}\"\n",
Expand Down
Loading
Loading