A_1a.Scaler_choice.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<summary>Table of Contents</summary>\n",
    "\n",
    "- [1. Standard Scaler Informer ](#1-standard-scaler-informer)\n",
    "- [2. Standard Scaler PatchTST](#2-standard-scaler-patchtst)\n",
    "- [3. MinMax Scaler Informer](#3-minmax-scaler-informer)\n",
    "- [4. MinMax Scaler PatchTST](#4-minmax-scaler-patchtst)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we perform a check on **Germany** dataset to confirm choice of scaler for our data.\n",
    "I kept here only the most important ones. You can also test e. g. Robust Scaler and other loss functions (e. g. LogCosh etc.).\n",
    "\n",
    "This script is to run the models. Final results are in the notebook \"Comparison\". \n",
    "\n",
    "Please note, the cell content is almost identical. However, when duplicating code and changing some arguments, it becomes easier to store and read results (especially if you want to experiment with 1 subpart) and split long running time into subprocesses. \n",
    "\n",
    "**For Standard Scaler and MinMax we tried learning rates: 0.0001, 0.00001, 0.000001.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "import subprocess\n",
    "import shutil\n",
    "from utils.helper import extract_metrics_from_output, convert_results_into_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Standard Scaler Informer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "cuda_device = \"1\"\n",
    "\n",
    "# Paths to files and data\n",
    "data_path = os.getcwd() + \"/datasets/\"\n",
    "\n",
    "script_path = \"./PatchTST-main/PatchTST_supervised/run_longExp.py\"\n",
    "\n",
    "# Arguments that will be used also for file names\n",
    "model = \"Informer\"\n",
    "dataset = 'DE_data.csv'\n",
    "losses = [\"MSE\"]\n",
    "country = dataset[:2]\n",
    "\n",
    "log_dir = f\"logs/scaler_choice/standard\"\n",
    "if not os.path.exists(log_dir):\n",
    "    os.makedirs(log_dir)\n",
    "\n",
    "log_file_path = f\"{log_dir}/{model}_{country}.log\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== Starting experiments for loss function: MSE ===\n",
      "\n",
      "\n",
      "=== Starting experiments for pred_len: 24 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_96_24_scaler_choice_for_DE', model='Informer', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='standard', if_relu=False, channel_mixing=0, seq_len=96, label_len=48, pred_len=24, inverse=True, loss_fnc='MSE', fc_dropout=0.05, head_dropout=0.0, patch_len=16, stride=8, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=512, n_heads=8, e_layers=2, d_layers=1, d_ff=2048, moving_avg=25, factor=5, distil=True, dropout=0.1, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_96_24_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl24_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 29017\n",
      "val 6217\n",
      "test 6217\n",
      "\titers: 100, epoch: 1 | loss: 0.9054202\n",
      "\tspeed: 0.0831s/iter; left time: 1497.1919s\n",
      "\titers: 200, epoch: 1 | loss: 0.8467745\n",
      "\tspeed: 0.0506s/iter; left time: 906.3395s\n",
      "\titers: 300, epoch: 1 | loss: 0.7909207\n",
      "\tspeed: 0.0508s/iter; left time: 904.9544s\n",
      "\titers: 400, epoch: 1 | loss: 0.7597419\n",
      "\tspeed: 0.0504s/iter; left time: 893.0498s\n",
      "\titers: 500, epoch: 1 | loss: 0.6280295\n",
      "\tspeed: 0.0509s/iter; left time: 896.2880s\n",
      "\titers: 600, epoch: 1 | loss: 0.5933272\n",
      "\tspeed: 0.0505s/iter; left time: 884.8268s\n",
      "\titers: 700, epoch: 1 | loss: 0.7665436\n",
      "\tspeed: 0.0502s/iter; left time: 875.2451s\n",
      "\titers: 800, epoch: 1 | loss: 0.5834797\n",
      "\tspeed: 0.0508s/iter; left time: 879.4119s\n",
      "\titers: 900, epoch: 1 | loss: 0.4690230\n",
      "\tspeed: 0.0504s/iter; left time: 868.1177s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:46.77s\n",
      "Steps: 906 | Train Loss: 0.7496831 Vali Loss: 0.6970405 Test Loss: 0.8205147\n",
      "Validation loss decreased (inf --> 0.697040).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.3835326\n",
      "\tspeed: 0.1078s/iter; left time: 1845.2750s\n",
      "\titers: 200, epoch: 2 | loss: 0.6076186\n",
      "\tspeed: 0.0506s/iter; left time: 860.2722s\n",
      "\titers: 300, epoch: 2 | loss: 0.4275496\n",
      "\tspeed: 0.0498s/iter; left time: 843.1521s\n",
      "\titers: 400, epoch: 2 | loss: 0.3615565\n",
      "\tspeed: 0.0506s/iter; left time: 850.1729s\n",
      "\titers: 500, epoch: 2 | loss: 0.3696667\n",
      "\tspeed: 0.0503s/iter; left time: 839.9591s\n",
      "\titers: 600, epoch: 2 | loss: 0.3259069\n",
      "\tspeed: 0.0508s/iter; left time: 843.2657s\n",
      "\titers: 700, epoch: 2 | loss: 0.3578444\n",
      "\tspeed: 0.0505s/iter; left time: 833.7609s\n",
      "\titers: 800, epoch: 2 | loss: 0.3353435\n",
      "\tspeed: 0.0507s/iter; left time: 831.7564s\n",
      "\titers: 900, epoch: 2 | loss: 0.2856516\n",
      "\tspeed: 0.0508s/iter; left time: 828.2208s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:46.01s\n",
      "Steps: 906 | Train Loss: 0.3706561 Vali Loss: 0.4825672 Test Loss: 0.5174062\n",
      "Validation loss decreased (0.697040 --> 0.482567).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.3838990\n",
      "\tspeed: 0.1084s/iter; left time: 1757.6983s\n",
      "\titers: 200, epoch: 3 | loss: 0.3096487\n",
      "\tspeed: 0.0501s/iter; left time: 807.5974s\n",
      "\titers: 300, epoch: 3 | loss: 0.2905636\n",
      "\tspeed: 0.0504s/iter; left time: 807.4197s\n",
      "\titers: 400, epoch: 3 | loss: 0.2842057\n",
      "\tspeed: 0.0507s/iter; left time: 807.0088s\n",
      "\titers: 500, epoch: 3 | loss: 0.2628434\n",
      "\tspeed: 0.0497s/iter; left time: 785.6448s\n",
      "\titers: 600, epoch: 3 | loss: 0.2647751\n",
      "\tspeed: 0.0505s/iter; left time: 794.0872s\n",
      "\titers: 700, epoch: 3 | loss: 0.2294270\n",
      "\tspeed: 0.0505s/iter; left time: 789.0309s\n",
      "\titers: 800, epoch: 3 | loss: 0.3252172\n",
      "\tspeed: 0.0505s/iter; left time: 783.3723s\n",
      "\titers: 900, epoch: 3 | loss: 0.2749336\n",
      "\tspeed: 0.0504s/iter; left time: 777.1800s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:45.91s\n",
      "Steps: 906 | Train Loss: 0.2914009 Vali Loss: 0.4601912 Test Loss: 0.5169730\n",
      "Validation loss decreased (0.482567 --> 0.460191).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.2458618\n",
      "\tspeed: 0.1081s/iter; left time: 1653.5856s\n",
      "\titers: 200, epoch: 4 | loss: 0.3059410\n",
      "\tspeed: 0.0508s/iter; left time: 771.8703s\n",
      "\titers: 300, epoch: 4 | loss: 0.2638421\n",
      "\tspeed: 0.0508s/iter; left time: 766.7216s\n",
      "\titers: 400, epoch: 4 | loss: 0.2703234\n",
      "\tspeed: 0.0507s/iter; left time: 760.0418s\n",
      "\titers: 500, epoch: 4 | loss: 0.3354372\n",
      "\tspeed: 0.0504s/iter; left time: 751.0311s\n",
      "\titers: 600, epoch: 4 | loss: 0.2871089\n",
      "\tspeed: 0.0499s/iter; left time: 738.6732s\n",
      "\titers: 700, epoch: 4 | loss: 0.3001944\n",
      "\tspeed: 0.0495s/iter; left time: 727.5349s\n",
      "\titers: 800, epoch: 4 | loss: 0.2543492\n",
      "\tspeed: 0.0505s/iter; left time: 737.0724s\n",
      "\titers: 900, epoch: 4 | loss: 0.1759309\n",
      "\tspeed: 0.0504s/iter; left time: 731.2258s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:45.86s\n",
      "Steps: 906 | Train Loss: 0.2598502 Vali Loss: 0.4622266 Test Loss: 0.4885969\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.2145841\n",
      "\tspeed: 0.1050s/iter; left time: 1511.0383s\n",
      "\titers: 200, epoch: 5 | loss: 0.1950129\n",
      "\tspeed: 0.0507s/iter; left time: 724.5225s\n",
      "\titers: 300, epoch: 5 | loss: 0.2457206\n",
      "\tspeed: 0.0507s/iter; left time: 719.6788s\n",
      "\titers: 400, epoch: 5 | loss: 0.2917987\n",
      "\tspeed: 0.0504s/iter; left time: 711.1431s\n",
      "\titers: 500, epoch: 5 | loss: 0.2300464\n",
      "\tspeed: 0.0498s/iter; left time: 697.3280s\n",
      "\titers: 600, epoch: 5 | loss: 0.2677855\n",
      "\tspeed: 0.0507s/iter; left time: 703.9680s\n",
      "\titers: 700, epoch: 5 | loss: 0.2723212\n",
      "\tspeed: 0.0508s/iter; left time: 701.5264s\n",
      "\titers: 800, epoch: 5 | loss: 0.2420058\n",
      "\tspeed: 0.0506s/iter; left time: 693.6491s\n",
      "\titers: 900, epoch: 5 | loss: 0.2596651\n",
      "\tspeed: 0.0505s/iter; left time: 686.9408s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:46.06s\n",
      "Steps: 906 | Train Loss: 0.2310203 Vali Loss: 0.4663165 Test Loss: 0.5086617\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.2156895\n",
      "\tspeed: 0.1049s/iter; left time: 1414.5355s\n",
      "\titers: 200, epoch: 6 | loss: 0.1715872\n",
      "\tspeed: 0.0495s/iter; left time: 662.5453s\n",
      "\titers: 300, epoch: 6 | loss: 0.2209537\n",
      "\tspeed: 0.0496s/iter; left time: 659.6886s\n",
      "\titers: 400, epoch: 6 | loss: 0.2260698\n",
      "\tspeed: 0.0497s/iter; left time: 655.0991s\n",
      "\titers: 500, epoch: 6 | loss: 0.2170567\n",
      "\tspeed: 0.0504s/iter; left time: 659.7348s\n",
      "\titers: 600, epoch: 6 | loss: 0.1648284\n",
      "\tspeed: 0.0506s/iter; left time: 656.9608s\n",
      "\titers: 700, epoch: 6 | loss: 0.1816590\n",
      "\tspeed: 0.0506s/iter; left time: 651.7248s\n",
      "\titers: 800, epoch: 6 | loss: 0.2155903\n",
      "\tspeed: 0.0506s/iter; left time: 646.5935s\n",
      "\titers: 900, epoch: 6 | loss: 0.1569917\n",
      "\tspeed: 0.0502s/iter; left time: 636.5855s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:45.74s\n",
      "Steps: 906 | Train Loss: 0.2036204 Vali Loss: 0.5005665 Test Loss: 0.5533465\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.1784568\n",
      "\tspeed: 0.1044s/iter; left time: 1313.3225s\n",
      "\titers: 200, epoch: 7 | loss: 0.1661351\n",
      "\tspeed: 0.0467s/iter; left time: 582.8447s\n",
      "\titers: 300, epoch: 7 | loss: 0.1797834\n",
      "\tspeed: 0.0502s/iter; left time: 621.8578s\n",
      "\titers: 400, epoch: 7 | loss: 0.2041640\n",
      "\tspeed: 0.0503s/iter; left time: 618.3347s\n",
      "\titers: 500, epoch: 7 | loss: 0.2167150\n",
      "\tspeed: 0.0505s/iter; left time: 614.8572s\n",
      "\titers: 600, epoch: 7 | loss: 0.1264821\n",
      "\tspeed: 0.0507s/iter; left time: 612.5563s\n",
      "\titers: 700, epoch: 7 | loss: 0.1789241\n",
      "\tspeed: 0.0503s/iter; left time: 602.4008s\n",
      "\titers: 800, epoch: 7 | loss: 0.1474319\n",
      "\tspeed: 0.0503s/iter; left time: 597.3079s\n",
      "\titers: 900, epoch: 7 | loss: 0.1750308\n",
      "\tspeed: 0.0504s/iter; left time: 593.5016s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:45.47s\n",
      "Steps: 906 | Train Loss: 0.1804432 Vali Loss: 0.5005258 Test Loss: 0.5421864\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.1498513\n",
      "\tspeed: 0.1044s/iter; left time: 1219.5589s\n",
      "\titers: 200, epoch: 8 | loss: 0.1800361\n",
      "\tspeed: 0.0503s/iter; left time: 582.9008s\n",
      "\titers: 300, epoch: 8 | loss: 0.1367881\n",
      "\tspeed: 0.0508s/iter; left time: 582.5728s\n",
      "\titers: 400, epoch: 8 | loss: 0.1441986\n",
      "\tspeed: 0.0503s/iter; left time: 571.8579s\n",
      "\titers: 500, epoch: 8 | loss: 0.1415319\n",
      "\tspeed: 0.0503s/iter; left time: 567.8285s\n",
      "\titers: 600, epoch: 8 | loss: 0.2070452\n",
      "\tspeed: 0.0502s/iter; left time: 561.7354s\n",
      "\titers: 700, epoch: 8 | loss: 0.1567003\n",
      "\tspeed: 0.0502s/iter; left time: 556.2228s\n",
      "\titers: 800, epoch: 8 | loss: 0.1627762\n",
      "\tspeed: 0.0506s/iter; left time: 555.5349s\n",
      "\titers: 900, epoch: 8 | loss: 0.1653901\n",
      "\tspeed: 0.0498s/iter; left time: 541.3539s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:45.80s\n",
      "Steps: 906 | Train Loss: 0.1589905 Vali Loss: 0.5030414 Test Loss: 0.5421357\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_96_24_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl24_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6217\n",
      "Scaled mse:0.5169051885604858, rmse:0.7189611792564392, mae:0.5058607459068298, rse:0.5690123438835144\n",
      "Original data scale mse:21333574.0, rmse:4618.828125, mae:3114.30859375, rse:0.22965744137763977\n",
      "\n",
      "=== Starting experiments for pred_len: 96 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_96_96_scaler_choice_for_DE', model='Informer', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='standard', if_relu=False, channel_mixing=0, seq_len=96, label_len=48, pred_len=96, inverse=True, loss_fnc='MSE', fc_dropout=0.05, head_dropout=0.0, patch_len=16, stride=8, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=512, n_heads=8, e_layers=2, d_layers=1, d_ff=2048, moving_avg=25, factor=5, distil=True, dropout=0.1, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_96_96_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl96_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 28945\n",
      "val 6145\n",
      "test 6145\n",
      "\titers: 100, epoch: 1 | loss: 1.0533408\n",
      "\tspeed: 0.0836s/iter; left time: 1502.3302s\n",
      "\titers: 200, epoch: 1 | loss: 0.9727684\n",
      "\tspeed: 0.0507s/iter; left time: 906.9818s\n",
      "\titers: 300, epoch: 1 | loss: 0.9566609\n",
      "\tspeed: 0.0508s/iter; left time: 903.2582s\n",
      "\titers: 400, epoch: 1 | loss: 0.8496140\n",
      "\tspeed: 0.0507s/iter; left time: 897.1543s\n",
      "\titers: 500, epoch: 1 | loss: 0.8372245\n",
      "\tspeed: 0.0508s/iter; left time: 893.0012s\n",
      "\titers: 600, epoch: 1 | loss: 0.7771551\n",
      "\tspeed: 0.0485s/iter; left time: 848.6755s\n",
      "\titers: 700, epoch: 1 | loss: 0.7536500\n",
      "\tspeed: 0.0508s/iter; left time: 883.2970s\n",
      "\titers: 800, epoch: 1 | loss: 0.7995765\n",
      "\tspeed: 0.0506s/iter; left time: 874.0471s\n",
      "\titers: 900, epoch: 1 | loss: 0.7616984\n",
      "\tspeed: 0.0508s/iter; left time: 872.7320s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:46.54s\n",
      "Steps: 904 | Train Loss: 0.8822277 Vali Loss: 0.9879580 Test Loss: 1.2488667\n",
      "Validation loss decreased (inf --> 0.987958).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.6864626\n",
      "\tspeed: 0.1138s/iter; left time: 1943.1433s\n",
      "\titers: 200, epoch: 2 | loss: 0.6349216\n",
      "\tspeed: 0.0507s/iter; left time: 860.1031s\n",
      "\titers: 300, epoch: 2 | loss: 0.6167125\n",
      "\tspeed: 0.0507s/iter; left time: 856.1975s\n",
      "\titers: 400, epoch: 2 | loss: 0.6698883\n",
      "\tspeed: 0.0505s/iter; left time: 847.0912s\n",
      "\titers: 500, epoch: 2 | loss: 0.6122433\n",
      "\tspeed: 0.0507s/iter; left time: 846.1800s\n",
      "\titers: 600, epoch: 2 | loss: 0.5006495\n",
      "\tspeed: 0.0507s/iter; left time: 841.2208s\n",
      "\titers: 700, epoch: 2 | loss: 0.4905136\n",
      "\tspeed: 0.0507s/iter; left time: 835.9750s\n",
      "\titers: 800, epoch: 2 | loss: 0.5182813\n",
      "\tspeed: 0.0508s/iter; left time: 831.6126s\n",
      "\titers: 900, epoch: 2 | loss: 0.5702390\n",
      "\tspeed: 0.0504s/iter; left time: 820.8246s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:46.03s\n",
      "Steps: 904 | Train Loss: 0.5981796 Vali Loss: 0.7245603 Test Loss: 0.8359482\n",
      "Validation loss decreased (0.987958 --> 0.724560).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.5490867\n",
      "\tspeed: 0.1171s/iter; left time: 1894.6441s\n",
      "\titers: 200, epoch: 3 | loss: 0.4005232\n",
      "\tspeed: 0.0508s/iter; left time: 816.4272s\n",
      "\titers: 300, epoch: 3 | loss: 0.4725600\n",
      "\tspeed: 0.0512s/iter; left time: 817.4598s\n",
      "\titers: 400, epoch: 3 | loss: 0.3812432\n",
      "\tspeed: 0.0512s/iter; left time: 812.6986s\n",
      "\titers: 500, epoch: 3 | loss: 0.4037902\n",
      "\tspeed: 0.0514s/iter; left time: 810.6093s\n",
      "\titers: 600, epoch: 3 | loss: 0.4054582\n",
      "\tspeed: 0.0514s/iter; left time: 806.0092s\n",
      "\titers: 700, epoch: 3 | loss: 0.4894638\n",
      "\tspeed: 0.0514s/iter; left time: 801.1864s\n",
      "\titers: 800, epoch: 3 | loss: 0.4026382\n",
      "\tspeed: 0.0515s/iter; left time: 796.6557s\n",
      "\titers: 900, epoch: 3 | loss: 0.4768871\n",
      "\tspeed: 0.0514s/iter; left time: 789.6003s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:46.49s\n",
      "Steps: 904 | Train Loss: 0.4435317 Vali Loss: 0.7027565 Test Loss: 0.9081082\n",
      "Validation loss decreased (0.724560 --> 0.702756).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.4925367\n",
      "\tspeed: 0.1137s/iter; left time: 1735.7425s\n",
      "\titers: 200, epoch: 4 | loss: 0.3639595\n",
      "\tspeed: 0.0511s/iter; left time: 774.9436s\n",
      "\titers: 300, epoch: 4 | loss: 0.4272591\n",
      "\tspeed: 0.0509s/iter; left time: 766.9412s\n",
      "\titers: 400, epoch: 4 | loss: 0.3669769\n",
      "\tspeed: 0.0513s/iter; left time: 768.5216s\n",
      "\titers: 500, epoch: 4 | loss: 0.3679672\n",
      "\tspeed: 0.0509s/iter; left time: 756.2749s\n",
      "\titers: 600, epoch: 4 | loss: 0.3602479\n",
      "\tspeed: 0.0512s/iter; left time: 756.5878s\n",
      "\titers: 700, epoch: 4 | loss: 0.3785375\n",
      "\tspeed: 0.0509s/iter; left time: 746.5856s\n",
      "\titers: 800, epoch: 4 | loss: 0.3032719\n",
      "\tspeed: 0.0511s/iter; left time: 744.0180s\n",
      "\titers: 900, epoch: 4 | loss: 0.3070508\n",
      "\tspeed: 0.0507s/iter; left time: 733.0774s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:46.33s\n",
      "Steps: 904 | Train Loss: 0.3769783 Vali Loss: 0.7373617 Test Loss: 0.8897136\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.3028362\n",
      "\tspeed: 0.1101s/iter; left time: 1581.7087s\n",
      "\titers: 200, epoch: 5 | loss: 0.2859340\n",
      "\tspeed: 0.0509s/iter; left time: 726.1004s\n",
      "\titers: 300, epoch: 5 | loss: 0.3439472\n",
      "\tspeed: 0.0514s/iter; left time: 727.9976s\n",
      "\titers: 400, epoch: 5 | loss: 0.4062416\n",
      "\tspeed: 0.0513s/iter; left time: 721.4178s\n",
      "\titers: 500, epoch: 5 | loss: 0.2793270\n",
      "\tspeed: 0.0516s/iter; left time: 720.8064s\n",
      "\titers: 600, epoch: 5 | loss: 0.3637531\n",
      "\tspeed: 0.0509s/iter; left time: 705.0881s\n",
      "\titers: 700, epoch: 5 | loss: 0.3885027\n",
      "\tspeed: 0.0502s/iter; left time: 691.6410s\n",
      "\titers: 800, epoch: 5 | loss: 0.2939192\n",
      "\tspeed: 0.0504s/iter; left time: 688.7527s\n",
      "\titers: 900, epoch: 5 | loss: 0.2735432\n",
      "\tspeed: 0.0510s/iter; left time: 691.9412s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:46.32s\n",
      "Steps: 904 | Train Loss: 0.3230915 Vali Loss: 0.7553846 Test Loss: 0.9410699\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.2577593\n",
      "\tspeed: 0.1105s/iter; left time: 1487.3036s\n",
      "\titers: 200, epoch: 6 | loss: 0.2776139\n",
      "\tspeed: 0.0506s/iter; left time: 676.6582s\n",
      "\titers: 300, epoch: 6 | loss: 0.2787929\n",
      "\tspeed: 0.0513s/iter; left time: 679.8927s\n",
      "\titers: 400, epoch: 6 | loss: 0.3177966\n",
      "\tspeed: 0.0509s/iter; left time: 669.6442s\n",
      "\titers: 500, epoch: 6 | loss: 0.2897230\n",
      "\tspeed: 0.0511s/iter; left time: 667.4972s\n",
      "\titers: 600, epoch: 6 | loss: 0.2846471\n",
      "\tspeed: 0.0507s/iter; left time: 657.0266s\n",
      "\titers: 700, epoch: 6 | loss: 0.2582789\n",
      "\tspeed: 0.0510s/iter; left time: 656.4837s\n",
      "\titers: 800, epoch: 6 | loss: 0.2577143\n",
      "\tspeed: 0.0513s/iter; left time: 654.1525s\n",
      "\titers: 900, epoch: 6 | loss: 0.2568874\n",
      "\tspeed: 0.0512s/iter; left time: 648.4883s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:46.33s\n",
      "Steps: 904 | Train Loss: 0.2781142 Vali Loss: 0.7626354 Test Loss: 0.9717472\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.2460101\n",
      "\tspeed: 0.1112s/iter; left time: 1396.9419s\n",
      "\titers: 200, epoch: 7 | loss: 0.2457341\n",
      "\tspeed: 0.0517s/iter; left time: 643.5054s\n",
      "\titers: 300, epoch: 7 | loss: 0.2472993\n",
      "\tspeed: 0.0517s/iter; left time: 638.4168s\n",
      "\titers: 400, epoch: 7 | loss: 0.2687068\n",
      "\tspeed: 0.0516s/iter; left time: 633.0348s\n",
      "\titers: 500, epoch: 7 | loss: 0.2189786\n",
      "\tspeed: 0.0517s/iter; left time: 628.2070s\n",
      "\titers: 600, epoch: 7 | loss: 0.2400969\n",
      "\tspeed: 0.0517s/iter; left time: 622.9928s\n",
      "\titers: 700, epoch: 7 | loss: 0.2581077\n",
      "\tspeed: 0.0517s/iter; left time: 617.8954s\n",
      "\titers: 800, epoch: 7 | loss: 0.2199688\n",
      "\tspeed: 0.0517s/iter; left time: 612.5414s\n",
      "\titers: 900, epoch: 7 | loss: 0.2432709\n",
      "\tspeed: 0.0514s/iter; left time: 604.3955s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:46.87s\n",
      "Steps: 904 | Train Loss: 0.2452642 Vali Loss: 0.7647664 Test Loss: 0.9858596\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.2065662\n",
      "\tspeed: 0.1112s/iter; left time: 1296.2836s\n",
      "\titers: 200, epoch: 8 | loss: 0.2027520\n",
      "\tspeed: 0.0515s/iter; left time: 594.7783s\n",
      "\titers: 300, epoch: 8 | loss: 0.2258547\n",
      "\tspeed: 0.0510s/iter; left time: 583.9710s\n",
      "\titers: 400, epoch: 8 | loss: 0.1931566\n",
      "\tspeed: 0.0509s/iter; left time: 577.3654s\n",
      "\titers: 500, epoch: 8 | loss: 0.2032788\n",
      "\tspeed: 0.0501s/iter; left time: 564.1364s\n",
      "\titers: 600, epoch: 8 | loss: 0.2519874\n",
      "\tspeed: 0.0511s/iter; left time: 569.4750s\n",
      "\titers: 700, epoch: 8 | loss: 0.2206146\n",
      "\tspeed: 0.0510s/iter; left time: 563.8247s\n",
      "\titers: 800, epoch: 8 | loss: 0.1822949\n",
      "\tspeed: 0.0507s/iter; left time: 555.4663s\n",
      "\titers: 900, epoch: 8 | loss: 0.2111897\n",
      "\tspeed: 0.0509s/iter; left time: 552.4796s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:46.29s\n",
      "Steps: 904 | Train Loss: 0.2182645 Vali Loss: 0.7549810 Test Loss: 0.9785702\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_96_96_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl96_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6145\n",
      "Scaled mse:0.9081009030342102, rmse:0.9529432654380798, mae:0.7101311087608337, rse:0.7558014988899231\n",
      "Original data scale mse:40752580.0, rmse:6383.77490234375, mae:4414.4140625, rse:0.31791427731513977\n",
      "\n",
      "=== Starting experiments for pred_len: 168 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_96_168_scaler_choice_for_DE', model='Informer', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='standard', if_relu=False, channel_mixing=0, seq_len=96, label_len=48, pred_len=168, inverse=True, loss_fnc='MSE', fc_dropout=0.05, head_dropout=0.0, patch_len=16, stride=8, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=512, n_heads=8, e_layers=2, d_layers=1, d_ff=2048, moving_avg=25, factor=5, distil=True, dropout=0.1, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_96_168_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl168_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 28873\n",
      "val 6073\n",
      "test 6073\n",
      "\titers: 100, epoch: 1 | loss: 0.9286664\n",
      "\tspeed: 0.0816s/iter; left time: 1464.5088s\n",
      "\titers: 200, epoch: 1 | loss: 1.0142589\n",
      "\tspeed: 0.0484s/iter; left time: 863.3774s\n",
      "\titers: 300, epoch: 1 | loss: 0.8930692\n",
      "\tspeed: 0.0488s/iter; left time: 865.9525s\n",
      "\titers: 400, epoch: 1 | loss: 0.9096307\n",
      "\tspeed: 0.0498s/iter; left time: 879.2560s\n",
      "\titers: 500, epoch: 1 | loss: 0.8646913\n",
      "\tspeed: 0.0504s/iter; left time: 883.5959s\n",
      "\titers: 600, epoch: 1 | loss: 0.9256495\n",
      "\tspeed: 0.0467s/iter; left time: 813.8432s\n",
      "\titers: 700, epoch: 1 | loss: 0.8651773\n",
      "\tspeed: 0.0454s/iter; left time: 787.8946s\n",
      "\titers: 800, epoch: 1 | loss: 0.8035756\n",
      "\tspeed: 0.0479s/iter; left time: 825.9306s\n",
      "\titers: 900, epoch: 1 | loss: 0.8942996\n",
      "\tspeed: 0.0476s/iter; left time: 816.4609s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:44.47s\n",
      "Steps: 902 | Train Loss: 0.9075228 Vali Loss: 1.0300285 Test Loss: 1.3209174\n",
      "Validation loss decreased (inf --> 1.030028).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.8145160\n",
      "\tspeed: 0.1191s/iter; left time: 2029.8837s\n",
      "\titers: 200, epoch: 2 | loss: 0.6879573\n",
      "\tspeed: 0.0454s/iter; left time: 769.7022s\n",
      "\titers: 300, epoch: 2 | loss: 0.7609401\n",
      "\tspeed: 0.0475s/iter; left time: 799.1779s\n",
      "\titers: 400, epoch: 2 | loss: 0.7600501\n",
      "\tspeed: 0.0493s/iter; left time: 824.5790s\n",
      "\titers: 500, epoch: 2 | loss: 0.6520451\n",
      "\tspeed: 0.0453s/iter; left time: 753.0851s\n",
      "\titers: 600, epoch: 2 | loss: 0.5897188\n",
      "\tspeed: 0.0416s/iter; left time: 688.5606s\n",
      "\titers: 700, epoch: 2 | loss: 0.5740029\n",
      "\tspeed: 0.0416s/iter; left time: 683.5938s\n",
      "\titers: 800, epoch: 2 | loss: 0.6271383\n",
      "\tspeed: 0.0419s/iter; left time: 685.2820s\n",
      "\titers: 900, epoch: 2 | loss: 0.5293421\n",
      "\tspeed: 0.0424s/iter; left time: 688.5103s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:40.51s\n",
      "Steps: 902 | Train Loss: 0.7022764 Vali Loss: 0.8072095 Test Loss: 0.9590882\n",
      "Validation loss decreased (1.030028 --> 0.807209).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.5540977\n",
      "\tspeed: 0.1226s/iter; left time: 1977.8249s\n",
      "\titers: 200, epoch: 3 | loss: 0.5439396\n",
      "\tspeed: 0.0507s/iter; left time: 813.5923s\n",
      "\titers: 300, epoch: 3 | loss: 0.4930642\n",
      "\tspeed: 0.0495s/iter; left time: 788.8431s\n",
      "\titers: 400, epoch: 3 | loss: 0.5340113\n",
      "\tspeed: 0.0483s/iter; left time: 764.7269s\n",
      "\titers: 500, epoch: 3 | loss: 0.4971465\n",
      "\tspeed: 0.0480s/iter; left time: 755.5026s\n",
      "\titers: 600, epoch: 3 | loss: 0.4557942\n",
      "\tspeed: 0.0469s/iter; left time: 733.8316s\n",
      "\titers: 700, epoch: 3 | loss: 0.4284682\n",
      "\tspeed: 0.0428s/iter; left time: 665.0511s\n",
      "\titers: 800, epoch: 3 | loss: 0.4483950\n",
      "\tspeed: 0.0423s/iter; left time: 652.4409s\n",
      "\titers: 900, epoch: 3 | loss: 0.4715318\n",
      "\tspeed: 0.0419s/iter; left time: 642.2768s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:42.33s\n",
      "Steps: 902 | Train Loss: 0.4862072 Vali Loss: 0.7372412 Test Loss: 0.9522145\n",
      "Validation loss decreased (0.807209 --> 0.737241).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.4853356\n",
      "\tspeed: 0.1218s/iter; left time: 1856.0667s\n",
      "\titers: 200, epoch: 4 | loss: 0.3664374\n",
      "\tspeed: 0.0480s/iter; left time: 726.0307s\n",
      "\titers: 300, epoch: 4 | loss: 0.3703359\n",
      "\tspeed: 0.0470s/iter; left time: 706.4142s\n",
      "\titers: 400, epoch: 4 | loss: 0.4330317\n",
      "\tspeed: 0.0499s/iter; left time: 744.9604s\n",
      "\titers: 500, epoch: 4 | loss: 0.3426898\n",
      "\tspeed: 0.0456s/iter; left time: 675.8362s\n",
      "\titers: 600, epoch: 4 | loss: 0.4110135\n",
      "\tspeed: 0.0423s/iter; left time: 623.5987s\n",
      "\titers: 700, epoch: 4 | loss: 0.3811014\n",
      "\tspeed: 0.0464s/iter; left time: 679.0446s\n",
      "\titers: 800, epoch: 4 | loss: 0.3654564\n",
      "\tspeed: 0.0469s/iter; left time: 681.2357s\n",
      "\titers: 900, epoch: 4 | loss: 0.3581348\n",
      "\tspeed: 0.0424s/iter; left time: 612.1843s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:42.03s\n",
      "Steps: 902 | Train Loss: 0.4010302 Vali Loss: 0.7843490 Test Loss: 0.9448935\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.3105854\n",
      "\tspeed: 0.1191s/iter; left time: 1706.6761s\n",
      "\titers: 200, epoch: 5 | loss: 0.3586848\n",
      "\tspeed: 0.0474s/iter; left time: 674.4268s\n",
      "\titers: 300, epoch: 5 | loss: 0.3422247\n",
      "\tspeed: 0.0466s/iter; left time: 658.0568s\n",
      "\titers: 400, epoch: 5 | loss: 0.3572131\n",
      "\tspeed: 0.0470s/iter; left time: 659.5067s\n",
      "\titers: 500, epoch: 5 | loss: 0.3113481\n",
      "\tspeed: 0.0461s/iter; left time: 643.0047s\n",
      "\titers: 600, epoch: 5 | loss: 0.3084607\n",
      "\tspeed: 0.0467s/iter; left time: 645.4020s\n",
      "\titers: 700, epoch: 5 | loss: 0.3185560\n",
      "\tspeed: 0.0466s/iter; left time: 639.6139s\n",
      "\titers: 800, epoch: 5 | loss: 0.3485337\n",
      "\tspeed: 0.0462s/iter; left time: 630.3358s\n",
      "\titers: 900, epoch: 5 | loss: 0.3253001\n",
      "\tspeed: 0.0480s/iter; left time: 650.0606s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:42.65s\n",
      "Steps: 902 | Train Loss: 0.3401620 Vali Loss: 0.8171146 Test Loss: 1.0580167\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.3090743\n",
      "\tspeed: 0.1190s/iter; left time: 1597.7347s\n",
      "\titers: 200, epoch: 6 | loss: 0.3268548\n",
      "\tspeed: 0.0480s/iter; left time: 640.1741s\n",
      "\titers: 300, epoch: 6 | loss: 0.3081989\n",
      "\tspeed: 0.0436s/iter; left time: 576.4673s\n",
      "\titers: 400, epoch: 6 | loss: 0.3361266\n",
      "\tspeed: 0.0419s/iter; left time: 550.7125s\n",
      "\titers: 500, epoch: 6 | loss: 0.3081892\n",
      "\tspeed: 0.0444s/iter; left time: 578.3739s\n",
      "\titers: 600, epoch: 6 | loss: 0.2805355\n",
      "\tspeed: 0.0475s/iter; left time: 613.8483s\n",
      "\titers: 700, epoch: 6 | loss: 0.2381649\n",
      "\tspeed: 0.0484s/iter; left time: 621.1702s\n",
      "\titers: 800, epoch: 6 | loss: 0.3035874\n",
      "\tspeed: 0.0498s/iter; left time: 633.8717s\n",
      "\titers: 900, epoch: 6 | loss: 0.2967162\n",
      "\tspeed: 0.0487s/iter; left time: 615.5673s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:42.59s\n",
      "Steps: 902 | Train Loss: 0.2951901 Vali Loss: 0.8042919 Test Loss: 1.0290098\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.2680151\n",
      "\tspeed: 0.1190s/iter; left time: 1490.4400s\n",
      "\titers: 200, epoch: 7 | loss: 0.2647797\n",
      "\tspeed: 0.0505s/iter; left time: 627.5789s\n",
      "\titers: 300, epoch: 7 | loss: 0.2809945\n",
      "\tspeed: 0.0481s/iter; left time: 593.4913s\n",
      "\titers: 400, epoch: 7 | loss: 0.2389415\n",
      "\tspeed: 0.0510s/iter; left time: 624.0371s\n",
      "\titers: 500, epoch: 7 | loss: 0.2740278\n",
      "\tspeed: 0.0510s/iter; left time: 617.9890s\n",
      "\titers: 600, epoch: 7 | loss: 0.2321251\n",
      "\tspeed: 0.0518s/iter; left time: 622.5261s\n",
      "\titers: 700, epoch: 7 | loss: 0.2788318\n",
      "\tspeed: 0.0515s/iter; left time: 613.8967s\n",
      "\titers: 800, epoch: 7 | loss: 0.2496115\n",
      "\tspeed: 0.0436s/iter; left time: 515.1710s\n",
      "\titers: 900, epoch: 7 | loss: 0.2870681\n",
      "\tspeed: 0.0476s/iter; left time: 558.1169s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:44.74s\n",
      "Steps: 902 | Train Loss: 0.2594971 Vali Loss: 0.8561326 Test Loss: 1.0475022\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.2252344\n",
      "\tspeed: 0.1212s/iter; left time: 1409.3320s\n",
      "\titers: 200, epoch: 8 | loss: 0.2587204\n",
      "\tspeed: 0.0426s/iter; left time: 491.2111s\n",
      "\titers: 300, epoch: 8 | loss: 0.2553909\n",
      "\tspeed: 0.0478s/iter; left time: 546.6388s\n",
      "\titers: 400, epoch: 8 | loss: 0.2135368\n",
      "\tspeed: 0.0489s/iter; left time: 553.4641s\n",
      "\titers: 500, epoch: 8 | loss: 0.2249764\n",
      "\tspeed: 0.0498s/iter; left time: 559.6468s\n",
      "\titers: 600, epoch: 8 | loss: 0.2209131\n",
      "\tspeed: 0.0501s/iter; left time: 557.8507s\n",
      "\titers: 700, epoch: 8 | loss: 0.2222484\n",
      "\tspeed: 0.0499s/iter; left time: 550.2823s\n",
      "\titers: 800, epoch: 8 | loss: 0.2131357\n",
      "\tspeed: 0.0469s/iter; left time: 512.1331s\n",
      "\titers: 900, epoch: 8 | loss: 0.2140368\n",
      "\tspeed: 0.0489s/iter; left time: 529.6782s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:44.05s\n",
      "Steps: 902 | Train Loss: 0.2304840 Vali Loss: 0.8534287 Test Loss: 1.0468464\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_96_168_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl168_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6073\n",
      "Scaled mse:0.9519778490066528, rmse:0.9756935238838196, mae:0.7115634679794312, rse:0.7729219794273376\n",
      "Original data scale mse:42312484.0, rmse:6504.8046875, mae:4406.45849609375, rse:0.3241006135940552\n"
     ]
    }
   ],
   "source": [
    "# Dynamic variables\n",
    "pred_lens = [\"24\", \"96\", \"168\"]\n",
    "seq_len = \"96\"\n",
    "lr = \"0.0001\"\n",
    "itr = 1  \n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = cuda_device\n",
    "\n",
    "# Lists to store the results\n",
    "informer_results_scaled, informer_results_unscaled = [], []\n",
    "\n",
    "# Log file\n",
    "with open(log_file_path, \"w\") as log_file:\n",
    "    for loss in losses:\n",
    "        statement_1 = f\"\\n=== Starting experiments for loss function: {loss} ===\\n\"\n",
    "        log_file.write(statement_1)\n",
    "        print(statement_1)  # Print to .ipynb output cell\n",
    "\n",
    "        for pred_len in pred_lens:\n",
    "            statement_2 = f\"\\n=== Starting experiments for pred_len: {pred_len} ===\\n\"\n",
    "            log_file.write(statement_2)\n",
    "            print(statement_2) \n",
    "            model_id = f\"{country}_{seq_len}_{pred_len}_scaler_choice_for_{country}\"\n",
    "            \n",
    "            # Arguments for the command\n",
    "            command = f\"\"\"\n",
    "            python {script_path} \\\n",
    "              --random_seed 2021 \\\n",
    "              --is_training 1 \\\n",
    "              --root_path \"{data_path}\" \\\n",
    "              --data_path \"{dataset}\" \\\n",
    "              --model_id {model_id} \\\n",
    "              --model \"{model}\" \\\n",
    "              --data \"custom\" \\\n",
    "              --features M \\\n",
    "              --seq_len {seq_len} \\\n",
    "              --label_len 48 \\\n",
    "              --pred_len {pred_len} \\\n",
    "              --e_layers 2 \\\n",
    "              --d_layers 1 \\\n",
    "              --factor 5 \\\n",
    "              --enc_in 5 \\\n",
    "              --dec_in 5 \\\n",
    "              --c_out 5 \\\n",
    "              --dropout 0.1 \\\n",
    "              --des 'Exp' \\\n",
    "              --train_epochs 20 \\\n",
    "              --patience 5 \\\n",
    "              --overlapping_windows \\\n",
    "              --inverse \\\n",
    "              --loss_fnc \"{loss}\" \\\n",
    "              --scaler_type standard \\\n",
    "              --itr {itr} --batch_size 32 --learning_rate \"{lr}\"\n",
    "            \"\"\"\n",
    "\n",
    "            # Run the command and capture the output\n",
    "            process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)\n",
    "\n",
    "            # Capture the output in real-time\n",
    "            output = []\n",
    "            for line in process.stdout:\n",
    "                output.append(line)\n",
    "                print(line, end='')  # Print in the .ipynb cell\n",
    "                log_file.write(line)  # Write to the log file\n",
    "\n",
    "            # Wait for the process to complete\n",
    "            process.wait()\n",
    "            \n",
    "            # Delete the checkpoints folder and all its contents\n",
    "            shutil.rmtree('./checkpoints' )\n",
    "\n",
    "            # Extract metrics for each iteration\n",
    "            iteration_metrics = extract_metrics_from_output(output, itr)\n",
    "            iteration_metrics_unscaled = extract_metrics_from_output(output, itr, if_scaled=False)\n",
    "\n",
    "            # Log the extracted metrics and save them\n",
    "            for iteration, (scaled_metrics, unscaled_metrics) in enumerate(zip(iteration_metrics, iteration_metrics_unscaled), start=1):\n",
    "                log_file.write(f\"\\nExtracted Metrics for {country}, pred_len={pred_len}, iteration={iteration}:\\n\")\n",
    "                log_file.write(f\"Scaled Metrics - MSE: {scaled_metrics[0]}, RMSE: {scaled_metrics[1]}, MAE: {scaled_metrics[2]}, RSE: {scaled_metrics[3]}\\n\")\n",
    "                log_file.write(f\"Unscaled Metrics - MSE: {unscaled_metrics[0]}, RMSE: {unscaled_metrics[1]}, MAE: {unscaled_metrics[2]}, RSE: {unscaled_metrics[3]}\\n\")\n",
    "\n",
    "                # Append the results to the informer_results lists\n",
    "                metrics_data = [(informer_results_scaled, scaled_metrics), (informer_results_unscaled, unscaled_metrics)]\n",
    "\n",
    "                for result_list, metrics in metrics_data:\n",
    "                    result_list.append({\n",
    "                        'Loss_function': loss,\n",
    "                        'Pred_len': pred_len,\n",
    "                        'Iteration': iteration,\n",
    "                        'MSE': metrics[0],\n",
    "                        'RMSE': metrics[1],\n",
    "                        'MAE': metrics[2],\n",
    "                        'RSE': metrics[3]\n",
    "                    })"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>MSE</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>RSE</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Loss_function</th>\n",
       "      <th>Iteration</th>\n",
       "      <th>Pred_len</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">MSE</th>\n",
       "      <th rowspan=\"3\" valign=\"top\">1</th>\n",
       "      <th>24</th>\n",
       "      <td>0.5169</td>\n",
       "      <td>0.7190</td>\n",
       "      <td>0.5059</td>\n",
       "      <td>0.5690</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>0.9081</td>\n",
       "      <td>0.9529</td>\n",
       "      <td>0.7101</td>\n",
       "      <td>0.7558</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>0.9520</td>\n",
       "      <td>0.9757</td>\n",
       "      <td>0.7116</td>\n",
       "      <td>0.7729</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     MSE    RMSE     MAE     RSE\n",
       "Loss_function Iteration Pred_len                                \n",
       "MSE           1         24        0.5169  0.7190  0.5059  0.5690\n",
       "                        96        0.9081  0.9529  0.7101  0.7558\n",
       "                        168       0.9520  0.9757  0.7116  0.7729"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path_dir = './results/scaler_choice'\n",
    "\n",
    "if not os.path.exists(path_dir):\n",
    "    os.makedirs(path_dir)\n",
    "\n",
    "csv_name_scaled = 'informer_scaler_results_scaled_default.csv'\n",
    "csv_name_unscaled = 'informer_scaler_results_unscaled_default.csv'\n",
    "\n",
    "# Convert the results into a DataFrame and save as CSV\n",
    "informer_df_scaled = convert_results_into_df(informer_results_scaled, path_dir, csv_name_scaled)\n",
    "informer_df_unscaled = convert_results_into_df(informer_results_unscaled, path_dir, csv_name_unscaled)\n",
    "informer_df_scaled.round(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>MSE</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>RSE</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Loss_function</th>\n",
       "      <th>Iteration</th>\n",
       "      <th>Pred_len</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">MSE</th>\n",
       "      <th rowspan=\"3\" valign=\"top\">1</th>\n",
       "      <th>24</th>\n",
       "      <td>21333574.0</td>\n",
       "      <td>4618.8281</td>\n",
       "      <td>3114.3086</td>\n",
       "      <td>0.2297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>40752580.0</td>\n",
       "      <td>6383.7749</td>\n",
       "      <td>4414.4141</td>\n",
       "      <td>0.3179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>42312484.0</td>\n",
       "      <td>6504.8047</td>\n",
       "      <td>4406.4585</td>\n",
       "      <td>0.3241</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         MSE       RMSE        MAE     RSE\n",
       "Loss_function Iteration Pred_len                                          \n",
       "MSE           1         24        21333574.0  4618.8281  3114.3086  0.2297\n",
       "                        96        40752580.0  6383.7749  4414.4141  0.3179\n",
       "                        168       42312484.0  6504.8047  4406.4585  0.3241"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "informer_df_unscaled.round(4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Standard Scaler PatchTST"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== Starting experiments for loss function: MSE ===\n",
      "\n",
      "\n",
      "=== Starting experiments for pred_len: 24 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_336_24_scaler_choice_for_DE', model='PatchTST', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='standard', if_relu=False, channel_mixing=0, seq_len=336, label_len=48, pred_len=24, inverse=True, loss_fnc='MSE', fc_dropout=0.2, head_dropout=0.0, patch_len=32, stride=16, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=128, n_heads=16, e_layers=3, d_layers=1, d_ff=256, moving_avg=25, factor=1, distil=True, dropout=0.2, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_336_24_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl24_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 28777\n",
      "val 6217\n",
      "test 6217\n",
      "\titers: 100, epoch: 1 | loss: 1.0400249\n",
      "\tspeed: 0.0573s/iter; left time: 1024.7867s\n",
      "\titers: 200, epoch: 1 | loss: 0.8410228\n",
      "\tspeed: 0.0306s/iter; left time: 543.2734s\n",
      "\titers: 300, epoch: 1 | loss: 0.6238504\n",
      "\tspeed: 0.0267s/iter; left time: 472.4196s\n",
      "\titers: 400, epoch: 1 | loss: 0.6429235\n",
      "\tspeed: 0.0257s/iter; left time: 452.2713s\n",
      "\titers: 500, epoch: 1 | loss: 0.5652475\n",
      "\tspeed: 0.0260s/iter; left time: 454.6983s\n",
      "\titers: 600, epoch: 1 | loss: 0.4985600\n",
      "\tspeed: 0.0259s/iter; left time: 450.6812s\n",
      "\titers: 700, epoch: 1 | loss: 0.4658573\n",
      "\tspeed: 0.0263s/iter; left time: 453.6535s\n",
      "\titers: 800, epoch: 1 | loss: 0.4743358\n",
      "\tspeed: 0.0270s/iter; left time: 463.1550s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:24.97s\n",
      "Steps: 899 | Train Loss: 0.6452772 Vali Loss: 0.5301473 Test Loss: 0.5831411\n",
      "Validation loss decreased (inf --> 0.530147).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.4451234\n",
      "\tspeed: 0.0736s/iter; left time: 1249.6904s\n",
      "\titers: 200, epoch: 2 | loss: 0.3722398\n",
      "\tspeed: 0.0247s/iter; left time: 417.1971s\n",
      "\titers: 300, epoch: 2 | loss: 0.3746257\n",
      "\tspeed: 0.0267s/iter; left time: 448.7213s\n",
      "\titers: 400, epoch: 2 | loss: 0.2649629\n",
      "\tspeed: 0.0257s/iter; left time: 428.0488s\n",
      "\titers: 500, epoch: 2 | loss: 0.3193873\n",
      "\tspeed: 0.0265s/iter; left time: 439.2134s\n",
      "\titers: 600, epoch: 2 | loss: 0.2182506\n",
      "\tspeed: 0.0246s/iter; left time: 405.6439s\n",
      "\titers: 700, epoch: 2 | loss: 0.2198727\n",
      "\tspeed: 0.0256s/iter; left time: 419.7349s\n",
      "\titers: 800, epoch: 2 | loss: 0.2759670\n",
      "\tspeed: 0.0269s/iter; left time: 437.4140s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:23.24s\n",
      "Steps: 899 | Train Loss: 0.3393205 Vali Loss: 0.4146878 Test Loss: 0.4560336\n",
      "Validation loss decreased (0.530147 --> 0.414688).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.2500065\n",
      "\tspeed: 0.0740s/iter; left time: 1189.4849s\n",
      "\titers: 200, epoch: 3 | loss: 0.4109719\n",
      "\tspeed: 0.0257s/iter; left time: 410.2101s\n",
      "\titers: 300, epoch: 3 | loss: 0.3107229\n",
      "\tspeed: 0.0261s/iter; left time: 415.0464s\n",
      "\titers: 400, epoch: 3 | loss: 0.2700932\n",
      "\tspeed: 0.0186s/iter; left time: 293.4410s\n",
      "\titers: 500, epoch: 3 | loss: 0.3280052\n",
      "\tspeed: 0.0200s/iter; left time: 313.2449s\n",
      "\titers: 600, epoch: 3 | loss: 0.2846107\n",
      "\tspeed: 0.0215s/iter; left time: 335.0689s\n",
      "\titers: 700, epoch: 3 | loss: 0.2958635\n",
      "\tspeed: 0.0155s/iter; left time: 239.5660s\n",
      "\titers: 800, epoch: 3 | loss: 0.3225683\n",
      "\tspeed: 0.0118s/iter; left time: 181.5314s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:17.90s\n",
      "Steps: 899 | Train Loss: 0.3053767 Vali Loss: 0.4059440 Test Loss: 0.4458437\n",
      "Validation loss decreased (0.414688 --> 0.405944).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.3483703\n",
      "\tspeed: 0.0637s/iter; left time: 967.3512s\n",
      "\titers: 200, epoch: 4 | loss: 0.2848158\n",
      "\tspeed: 0.0270s/iter; left time: 407.2280s\n",
      "\titers: 300, epoch: 4 | loss: 0.3461615\n",
      "\tspeed: 0.0212s/iter; left time: 318.3870s\n",
      "\titers: 400, epoch: 4 | loss: 0.2665159\n",
      "\tspeed: 0.0219s/iter; left time: 325.8781s\n",
      "\titers: 500, epoch: 4 | loss: 0.2355973\n",
      "\tspeed: 0.0308s/iter; left time: 455.7833s\n",
      "\titers: 600, epoch: 4 | loss: 0.2504388\n",
      "\tspeed: 0.0291s/iter; left time: 427.6594s\n",
      "\titers: 700, epoch: 4 | loss: 0.2402427\n",
      "\tspeed: 0.0274s/iter; left time: 399.8690s\n",
      "\titers: 800, epoch: 4 | loss: 0.2469124\n",
      "\tspeed: 0.0276s/iter; left time: 399.4491s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:23.83s\n",
      "Steps: 899 | Train Loss: 0.2945943 Vali Loss: 0.4021743 Test Loss: 0.4439670\n",
      "Validation loss decreased (0.405944 --> 0.402174).  Saving model ...\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.4255674\n",
      "\tspeed: 0.0720s/iter; left time: 1028.2502s\n",
      "\titers: 200, epoch: 5 | loss: 0.3238305\n",
      "\tspeed: 0.0250s/iter; left time: 353.9714s\n",
      "\titers: 300, epoch: 5 | loss: 0.2379073\n",
      "\tspeed: 0.0264s/iter; left time: 371.4142s\n",
      "\titers: 400, epoch: 5 | loss: 0.2654204\n",
      "\tspeed: 0.0229s/iter; left time: 319.8187s\n",
      "\titers: 500, epoch: 5 | loss: 0.3443564\n",
      "\tspeed: 0.0265s/iter; left time: 368.3646s\n",
      "\titers: 600, epoch: 5 | loss: 0.2426864\n",
      "\tspeed: 0.0272s/iter; left time: 374.4569s\n",
      "\titers: 700, epoch: 5 | loss: 0.3454446\n",
      "\tspeed: 0.0251s/iter; left time: 343.3158s\n",
      "\titers: 800, epoch: 5 | loss: 0.2666938\n",
      "\tspeed: 0.0239s/iter; left time: 324.6951s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:22.63s\n",
      "Steps: 899 | Train Loss: 0.2860920 Vali Loss: 0.4000861 Test Loss: 0.4442415\n",
      "Validation loss decreased (0.402174 --> 0.400086).  Saving model ...\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.2948705\n",
      "\tspeed: 0.0691s/iter; left time: 924.9994s\n",
      "\titers: 200, epoch: 6 | loss: 0.2781022\n",
      "\tspeed: 0.0234s/iter; left time: 311.4674s\n",
      "\titers: 300, epoch: 6 | loss: 0.2162359\n",
      "\tspeed: 0.0223s/iter; left time: 293.5593s\n",
      "\titers: 400, epoch: 6 | loss: 0.2867737\n",
      "\tspeed: 0.0249s/iter; left time: 326.2131s\n",
      "\titers: 500, epoch: 6 | loss: 0.2306262\n",
      "\tspeed: 0.0242s/iter; left time: 314.4151s\n",
      "\titers: 600, epoch: 6 | loss: 0.2209586\n",
      "\tspeed: 0.0256s/iter; left time: 330.1298s\n",
      "\titers: 700, epoch: 6 | loss: 0.3297862\n",
      "\tspeed: 0.0265s/iter; left time: 338.2312s\n",
      "\titers: 800, epoch: 6 | loss: 0.2745198\n",
      "\tspeed: 0.0252s/iter; left time: 320.1837s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:22.03s\n",
      "Steps: 899 | Train Loss: 0.2817403 Vali Loss: 0.3886921 Test Loss: 0.4340410\n",
      "Validation loss decreased (0.400086 --> 0.388692).  Saving model ...\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.3817490\n",
      "\tspeed: 0.0699s/iter; left time: 873.3374s\n",
      "\titers: 200, epoch: 7 | loss: 0.2235909\n",
      "\tspeed: 0.0285s/iter; left time: 353.4115s\n",
      "\titers: 300, epoch: 7 | loss: 0.4264503\n",
      "\tspeed: 0.0216s/iter; left time: 265.1778s\n",
      "\titers: 400, epoch: 7 | loss: 0.3386434\n",
      "\tspeed: 0.0149s/iter; left time: 181.6357s\n",
      "\titers: 500, epoch: 7 | loss: 0.2938150\n",
      "\tspeed: 0.0140s/iter; left time: 169.3909s\n",
      "\titers: 600, epoch: 7 | loss: 0.2654440\n",
      "\tspeed: 0.0221s/iter; left time: 264.6809s\n",
      "\titers: 700, epoch: 7 | loss: 0.2186387\n",
      "\tspeed: 0.0223s/iter; left time: 264.5428s\n",
      "\titers: 800, epoch: 7 | loss: 0.3766811\n",
      "\tspeed: 0.0138s/iter; left time: 162.2066s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:17.79s\n",
      "Steps: 899 | Train Loss: 0.2761739 Vali Loss: 0.3919950 Test Loss: 0.4359834\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.3090265\n",
      "\tspeed: 0.0648s/iter; left time: 750.4122s\n",
      "\titers: 200, epoch: 8 | loss: 0.2475936\n",
      "\tspeed: 0.0236s/iter; left time: 271.5569s\n",
      "\titers: 300, epoch: 8 | loss: 0.2571255\n",
      "\tspeed: 0.0303s/iter; left time: 344.9057s\n",
      "\titers: 400, epoch: 8 | loss: 0.3174316\n",
      "\tspeed: 0.0285s/iter; left time: 322.0612s\n",
      "\titers: 500, epoch: 8 | loss: 0.2695107\n",
      "\tspeed: 0.0272s/iter; left time: 304.0489s\n",
      "\titers: 600, epoch: 8 | loss: 0.2561891\n",
      "\tspeed: 0.0272s/iter; left time: 301.8968s\n",
      "\titers: 700, epoch: 8 | loss: 0.3343061\n",
      "\tspeed: 0.0270s/iter; left time: 297.0818s\n",
      "\titers: 800, epoch: 8 | loss: 0.2148796\n",
      "\tspeed: 0.0273s/iter; left time: 297.1939s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:24.08s\n",
      "Steps: 899 | Train Loss: 0.2729226 Vali Loss: 0.3913986 Test Loss: 0.4382161\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 5.904900000000001e-05\n",
      "\titers: 100, epoch: 9 | loss: 0.2765048\n",
      "\tspeed: 0.0726s/iter; left time: 775.7129s\n",
      "\titers: 200, epoch: 9 | loss: 0.3702734\n",
      "\tspeed: 0.0267s/iter; left time: 283.1123s\n",
      "\titers: 300, epoch: 9 | loss: 0.2977566\n",
      "\tspeed: 0.0227s/iter; left time: 237.8996s\n",
      "\titers: 400, epoch: 9 | loss: 0.2833446\n",
      "\tspeed: 0.0224s/iter; left time: 233.0584s\n",
      "\titers: 500, epoch: 9 | loss: 0.2557055\n",
      "\tspeed: 0.0258s/iter; left time: 265.0572s\n",
      "\titers: 600, epoch: 9 | loss: 0.1928890\n",
      "\tspeed: 0.0274s/iter; left time: 278.9905s\n",
      "\titers: 700, epoch: 9 | loss: 0.3351396\n",
      "\tspeed: 0.0275s/iter; left time: 277.3917s\n",
      "\titers: 800, epoch: 9 | loss: 0.2815662\n",
      "\tspeed: 0.0304s/iter; left time: 303.1803s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 9\n",
      "Cost time: 00h:00m:23.53s\n",
      "Steps: 899 | Train Loss: 0.2695688 Vali Loss: 0.3937631 Test Loss: 0.4410180\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 5.3144100000000005e-05\n",
      "\titers: 100, epoch: 10 | loss: 0.2011720\n",
      "\tspeed: 0.0757s/iter; left time: 741.2722s\n",
      "\titers: 200, epoch: 10 | loss: 0.2157880\n",
      "\tspeed: 0.0266s/iter; left time: 257.7937s\n",
      "\titers: 300, epoch: 10 | loss: 0.2906202\n",
      "\tspeed: 0.0278s/iter; left time: 266.5616s\n",
      "\titers: 400, epoch: 10 | loss: 0.2441356\n",
      "\tspeed: 0.0274s/iter; left time: 259.9374s\n",
      "\titers: 500, epoch: 10 | loss: 0.2988858\n",
      "\tspeed: 0.0272s/iter; left time: 255.8389s\n",
      "\titers: 600, epoch: 10 | loss: 0.2837629\n",
      "\tspeed: 0.0274s/iter; left time: 254.3644s\n",
      "\titers: 700, epoch: 10 | loss: 0.2925743\n",
      "\tspeed: 0.0288s/iter; left time: 264.2411s\n",
      "\titers: 800, epoch: 10 | loss: 0.2568387\n",
      "\tspeed: 0.0262s/iter; left time: 238.2360s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 10\n",
      "Cost time: 00h:00m:24.66s\n",
      "Steps: 899 | Train Loss: 0.2668552 Vali Loss: 0.3891308 Test Loss: 0.4404971\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 4.782969000000001e-05\n",
      "\titers: 100, epoch: 11 | loss: 0.2423147\n",
      "\tspeed: 0.0698s/iter; left time: 620.2746s\n",
      "\titers: 200, epoch: 11 | loss: 0.3319949\n",
      "\tspeed: 0.0240s/iter; left time: 210.9403s\n",
      "\titers: 300, epoch: 11 | loss: 0.1871923\n",
      "\tspeed: 0.0239s/iter; left time: 208.0102s\n",
      "\titers: 400, epoch: 11 | loss: 0.2175258\n",
      "\tspeed: 0.0239s/iter; left time: 205.2691s\n",
      "\titers: 500, epoch: 11 | loss: 0.1908453\n",
      "\tspeed: 0.0233s/iter; left time: 198.2557s\n",
      "\titers: 600, epoch: 11 | loss: 0.3060406\n",
      "\tspeed: 0.0247s/iter; left time: 207.3150s\n",
      "\titers: 700, epoch: 11 | loss: 0.2779558\n",
      "\tspeed: 0.0274s/iter; left time: 226.9199s\n",
      "\titers: 800, epoch: 11 | loss: 0.2580988\n",
      "\tspeed: 0.0274s/iter; left time: 224.0587s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 11\n",
      "Cost time: 00h:00m:22.19s\n",
      "Steps: 899 | Train Loss: 0.2639363 Vali Loss: 0.3874193 Test Loss: 0.4386898\n",
      "Validation loss decreased (0.388692 --> 0.387419).  Saving model ...\n",
      "Updating learning rate to 4.304672100000001e-05\n",
      "\titers: 100, epoch: 12 | loss: 0.2868238\n",
      "\tspeed: 0.0681s/iter; left time: 544.1661s\n",
      "\titers: 200, epoch: 12 | loss: 0.2886868\n",
      "\tspeed: 0.0260s/iter; left time: 205.4625s\n",
      "\titers: 300, epoch: 12 | loss: 0.2990099\n",
      "\tspeed: 0.0264s/iter; left time: 205.7893s\n",
      "\titers: 400, epoch: 12 | loss: 0.2739601\n",
      "\tspeed: 0.0249s/iter; left time: 191.8072s\n",
      "\titers: 500, epoch: 12 | loss: 0.2173029\n",
      "\tspeed: 0.0249s/iter; left time: 188.9945s\n",
      "\titers: 600, epoch: 12 | loss: 0.2563137\n",
      "\tspeed: 0.0250s/iter; left time: 186.9442s\n",
      "\titers: 700, epoch: 12 | loss: 0.2434747\n",
      "\tspeed: 0.0255s/iter; left time: 188.2998s\n",
      "\titers: 800, epoch: 12 | loss: 0.2823516\n",
      "\tspeed: 0.0238s/iter; left time: 173.8940s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 12\n",
      "Cost time: 00h:00m:22.64s\n",
      "Steps: 899 | Train Loss: 0.2616180 Vali Loss: 0.3917998 Test Loss: 0.4411685\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 3.874204890000001e-05\n",
      "\titers: 100, epoch: 13 | loss: 0.2045501\n",
      "\tspeed: 0.0721s/iter; left time: 511.4581s\n",
      "\titers: 200, epoch: 13 | loss: 0.2363232\n",
      "\tspeed: 0.0264s/iter; left time: 184.9023s\n",
      "\titers: 300, epoch: 13 | loss: 0.2339737\n",
      "\tspeed: 0.0246s/iter; left time: 169.4408s\n",
      "\titers: 400, epoch: 13 | loss: 0.2628110\n",
      "\tspeed: 0.0239s/iter; left time: 162.1100s\n",
      "\titers: 500, epoch: 13 | loss: 0.1901602\n",
      "\tspeed: 0.0271s/iter; left time: 181.3621s\n",
      "\titers: 600, epoch: 13 | loss: 0.2568354\n",
      "\tspeed: 0.0270s/iter; left time: 178.0211s\n",
      "\titers: 700, epoch: 13 | loss: 0.2101955\n",
      "\tspeed: 0.0278s/iter; left time: 180.7397s\n",
      "\titers: 800, epoch: 13 | loss: 0.2755387\n",
      "\tspeed: 0.0262s/iter; left time: 167.4379s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 13\n",
      "Cost time: 00h:00m:23.75s\n",
      "Steps: 899 | Train Loss: 0.2591560 Vali Loss: 0.3925091 Test Loss: 0.4400021\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 3.486784401000001e-05\n",
      "\titers: 100, epoch: 14 | loss: 0.2326988\n",
      "\tspeed: 0.0731s/iter; left time: 452.7965s\n",
      "\titers: 200, epoch: 14 | loss: 0.2823396\n",
      "\tspeed: 0.0249s/iter; left time: 151.7473s\n",
      "\titers: 300, epoch: 14 | loss: 0.2489126\n",
      "\tspeed: 0.0247s/iter; left time: 148.0352s\n",
      "\titers: 400, epoch: 14 | loss: 0.2651447\n",
      "\tspeed: 0.0247s/iter; left time: 145.3299s\n",
      "\titers: 500, epoch: 14 | loss: 0.2933319\n",
      "\tspeed: 0.0246s/iter; left time: 142.6954s\n",
      "\titers: 600, epoch: 14 | loss: 0.3006517\n",
      "\tspeed: 0.0249s/iter; left time: 141.5792s\n",
      "\titers: 700, epoch: 14 | loss: 0.2710021\n",
      "\tspeed: 0.0252s/iter; left time: 140.7322s\n",
      "\titers: 800, epoch: 14 | loss: 0.2862627\n",
      "\tspeed: 0.0263s/iter; left time: 144.6066s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 14\n",
      "Cost time: 00h:00m:22.76s\n",
      "Steps: 899 | Train Loss: 0.2578452 Vali Loss: 0.3906174 Test Loss: 0.4387840\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 3.138105960900001e-05\n",
      "\titers: 100, epoch: 15 | loss: 0.2404376\n",
      "\tspeed: 0.0704s/iter; left time: 372.9500s\n",
      "\titers: 200, epoch: 15 | loss: 0.2348949\n",
      "\tspeed: 0.0262s/iter; left time: 135.9196s\n",
      "\titers: 300, epoch: 15 | loss: 0.3137814\n",
      "\tspeed: 0.0221s/iter; left time: 112.4048s\n",
      "\titers: 400, epoch: 15 | loss: 0.2542456\n",
      "\tspeed: 0.0263s/iter; left time: 131.2210s\n",
      "\titers: 500, epoch: 15 | loss: 0.2748001\n",
      "\tspeed: 0.0297s/iter; left time: 145.5582s\n",
      "\titers: 600, epoch: 15 | loss: 0.2599124\n",
      "\tspeed: 0.0285s/iter; left time: 136.6495s\n",
      "\titers: 700, epoch: 15 | loss: 0.2621993\n",
      "\tspeed: 0.0296s/iter; left time: 138.8805s\n",
      "\titers: 800, epoch: 15 | loss: 0.2698059\n",
      "\tspeed: 0.0263s/iter; left time: 120.8254s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 15\n",
      "Cost time: 00h:00m:24.53s\n",
      "Steps: 899 | Train Loss: 0.2555122 Vali Loss: 0.3956259 Test Loss: 0.4446132\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 2.824295364810001e-05\n",
      "\titers: 100, epoch: 16 | loss: 0.2437944\n",
      "\tspeed: 0.0821s/iter; left time: 360.7822s\n",
      "\titers: 200, epoch: 16 | loss: 0.2546386\n",
      "\tspeed: 0.0257s/iter; left time: 110.2595s\n",
      "\titers: 300, epoch: 16 | loss: 0.2157390\n",
      "\tspeed: 0.0260s/iter; left time: 109.2382s\n",
      "\titers: 400, epoch: 16 | loss: 0.2680039\n",
      "\tspeed: 0.0260s/iter; left time: 106.6336s\n",
      "\titers: 500, epoch: 16 | loss: 0.3324822\n",
      "\tspeed: 0.0294s/iter; left time: 117.4177s\n",
      "\titers: 600, epoch: 16 | loss: 0.2465593\n",
      "\tspeed: 0.0279s/iter; left time: 108.6205s\n",
      "\titers: 700, epoch: 16 | loss: 0.2933993\n",
      "\tspeed: 0.0272s/iter; left time: 103.3292s\n",
      "\titers: 800, epoch: 16 | loss: 0.2289654\n",
      "\tspeed: 0.0272s/iter; left time: 100.4892s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 16\n",
      "Cost time: 00h:00m:24.42s\n",
      "Steps: 899 | Train Loss: 0.2535785 Vali Loss: 0.3939591 Test Loss: 0.4425671\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_336_24_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl24_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6217\n",
      "Scaled mse:0.4386899173259735, rmse:0.6623367071151733, mae:0.4260389804840088, rse:0.5241976380348206\n",
      "Original data scale mse:16448487.0, rmse:4055.67333984375, mae:2475.6689453125, rse:0.2016562670469284\n",
      "\n",
      "=== Starting experiments for pred_len: 96 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_336_96_scaler_choice_for_DE', model='PatchTST', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='standard', if_relu=False, channel_mixing=0, seq_len=336, label_len=48, pred_len=96, inverse=True, loss_fnc='MSE', fc_dropout=0.2, head_dropout=0.0, patch_len=32, stride=16, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=128, n_heads=16, e_layers=3, d_layers=1, d_ff=256, moving_avg=25, factor=1, distil=True, dropout=0.2, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_336_96_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl96_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 28705\n",
      "val 6145\n",
      "test 6145\n",
      "\titers: 100, epoch: 1 | loss: 1.0027357\n",
      "\tspeed: 0.0534s/iter; left time: 951.9109s\n",
      "\titers: 200, epoch: 1 | loss: 0.8645256\n",
      "\tspeed: 0.0252s/iter; left time: 447.1526s\n",
      "\titers: 300, epoch: 1 | loss: 0.7880483\n",
      "\tspeed: 0.0235s/iter; left time: 414.5383s\n",
      "\titers: 400, epoch: 1 | loss: 0.6831477\n",
      "\tspeed: 0.0257s/iter; left time: 450.4435s\n",
      "\titers: 500, epoch: 1 | loss: 0.7151287\n",
      "\tspeed: 0.0257s/iter; left time: 447.6665s\n",
      "\titers: 600, epoch: 1 | loss: 0.8562070\n",
      "\tspeed: 0.0259s/iter; left time: 448.4100s\n",
      "\titers: 700, epoch: 1 | loss: 0.7484691\n",
      "\tspeed: 0.0264s/iter; left time: 454.6193s\n",
      "\titers: 800, epoch: 1 | loss: 0.7798867\n",
      "\tspeed: 0.0255s/iter; left time: 436.8954s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:23.47s\n",
      "Steps: 897 | Train Loss: 0.7828709 Vali Loss: 0.7323778 Test Loss: 0.8487975\n",
      "Validation loss decreased (inf --> 0.732378).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.4915271\n",
      "\tspeed: 0.0807s/iter; left time: 1366.8929s\n",
      "\titers: 200, epoch: 2 | loss: 0.5145357\n",
      "\tspeed: 0.0256s/iter; left time: 431.1613s\n",
      "\titers: 300, epoch: 2 | loss: 0.3691760\n",
      "\tspeed: 0.0257s/iter; left time: 431.1580s\n",
      "\titers: 400, epoch: 2 | loss: 0.4813964\n",
      "\tspeed: 0.0261s/iter; left time: 434.9634s\n",
      "\titers: 500, epoch: 2 | loss: 0.4832175\n",
      "\tspeed: 0.0282s/iter; left time: 467.2770s\n",
      "\titers: 600, epoch: 2 | loss: 0.4980557\n",
      "\tspeed: 0.0282s/iter; left time: 463.0039s\n",
      "\titers: 700, epoch: 2 | loss: 0.4301472\n",
      "\tspeed: 0.0281s/iter; left time: 459.6368s\n",
      "\titers: 800, epoch: 2 | loss: 0.6801674\n",
      "\tspeed: 0.0279s/iter; left time: 453.4603s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:24.48s\n",
      "Steps: 897 | Train Loss: 0.5522313 Vali Loss: 0.6380516 Test Loss: 0.7696040\n",
      "Validation loss decreased (0.732378 --> 0.638052).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.4713278\n",
      "\tspeed: 0.0841s/iter; left time: 1349.9565s\n",
      "\titers: 200, epoch: 3 | loss: 0.5758635\n",
      "\tspeed: 0.0262s/iter; left time: 417.6076s\n",
      "\titers: 300, epoch: 3 | loss: 0.4972477\n",
      "\tspeed: 0.0254s/iter; left time: 402.8270s\n",
      "\titers: 400, epoch: 3 | loss: 0.5455269\n",
      "\tspeed: 0.0258s/iter; left time: 405.6505s\n",
      "\titers: 500, epoch: 3 | loss: 0.3950365\n",
      "\tspeed: 0.0257s/iter; left time: 401.6173s\n",
      "\titers: 600, epoch: 3 | loss: 0.4550214\n",
      "\tspeed: 0.0258s/iter; left time: 400.9559s\n",
      "\titers: 700, epoch: 3 | loss: 0.4795676\n",
      "\tspeed: 0.0254s/iter; left time: 392.4026s\n",
      "\titers: 800, epoch: 3 | loss: 0.4493077\n",
      "\tspeed: 0.0249s/iter; left time: 382.5920s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:23.29s\n",
      "Steps: 897 | Train Loss: 0.5143333 Vali Loss: 0.6387923 Test Loss: 0.7787517\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.4750580\n",
      "\tspeed: 0.0795s/iter; left time: 1204.7957s\n",
      "\titers: 200, epoch: 4 | loss: 0.5045003\n",
      "\tspeed: 0.0255s/iter; left time: 383.0586s\n",
      "\titers: 300, epoch: 4 | loss: 0.5793609\n",
      "\tspeed: 0.0258s/iter; left time: 385.6434s\n",
      "\titers: 400, epoch: 4 | loss: 0.3765642\n",
      "\tspeed: 0.0257s/iter; left time: 382.2303s\n",
      "\titers: 500, epoch: 4 | loss: 0.4283560\n",
      "\tspeed: 0.0257s/iter; left time: 378.4563s\n",
      "\titers: 600, epoch: 4 | loss: 0.5350238\n",
      "\tspeed: 0.0256s/iter; left time: 375.5096s\n",
      "\titers: 700, epoch: 4 | loss: 0.6713233\n",
      "\tspeed: 0.0252s/iter; left time: 366.6216s\n",
      "\titers: 800, epoch: 4 | loss: 0.4180396\n",
      "\tspeed: 0.0261s/iter; left time: 377.2194s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:23.35s\n",
      "Steps: 897 | Train Loss: 0.4982431 Vali Loss: 0.6376016 Test Loss: 0.7837718\n",
      "Validation loss decreased (0.638052 --> 0.637602).  Saving model ...\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.4218901\n",
      "\tspeed: 0.0814s/iter; left time: 1160.0554s\n",
      "\titers: 200, epoch: 5 | loss: 0.5042915\n",
      "\tspeed: 0.0276s/iter; left time: 391.2688s\n",
      "\titers: 300, epoch: 5 | loss: 0.4950100\n",
      "\tspeed: 0.0277s/iter; left time: 389.9538s\n",
      "\titers: 400, epoch: 5 | loss: 0.4343377\n",
      "\tspeed: 0.0263s/iter; left time: 366.4357s\n",
      "\titers: 500, epoch: 5 | loss: 0.4829984\n",
      "\tspeed: 0.0277s/iter; left time: 384.2488s\n",
      "\titers: 600, epoch: 5 | loss: 0.4440160\n",
      "\tspeed: 0.0251s/iter; left time: 344.9378s\n",
      "\titers: 700, epoch: 5 | loss: 0.5267438\n",
      "\tspeed: 0.0251s/iter; left time: 342.8261s\n",
      "\titers: 800, epoch: 5 | loss: 0.5847322\n",
      "\tspeed: 0.0256s/iter; left time: 347.3060s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:24.19s\n",
      "Steps: 897 | Train Loss: 0.4789489 Vali Loss: 0.6580500 Test Loss: 0.8409826\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.4827521\n",
      "\tspeed: 0.0816s/iter; left time: 1089.8803s\n",
      "\titers: 200, epoch: 6 | loss: 0.4522935\n",
      "\tspeed: 0.0275s/iter; left time: 363.9848s\n",
      "\titers: 300, epoch: 6 | loss: 0.4309393\n",
      "\tspeed: 0.0281s/iter; left time: 369.0568s\n",
      "\titers: 400, epoch: 6 | loss: 0.4970303\n",
      "\tspeed: 0.0267s/iter; left time: 348.2981s\n",
      "\titers: 500, epoch: 6 | loss: 0.5253251\n",
      "\tspeed: 0.0274s/iter; left time: 354.7181s\n",
      "\titers: 600, epoch: 6 | loss: 0.3221878\n",
      "\tspeed: 0.0272s/iter; left time: 349.8006s\n",
      "\titers: 700, epoch: 6 | loss: 0.4974601\n",
      "\tspeed: 0.0254s/iter; left time: 323.9042s\n",
      "\titers: 800, epoch: 6 | loss: 0.4453829\n",
      "\tspeed: 0.0256s/iter; left time: 324.3540s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:24.00s\n",
      "Steps: 897 | Train Loss: 0.4615198 Vali Loss: 0.6463689 Test Loss: 0.8351710\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.4842026\n",
      "\tspeed: 0.0809s/iter; left time: 1008.0995s\n",
      "\titers: 200, epoch: 7 | loss: 0.3841623\n",
      "\tspeed: 0.0273s/iter; left time: 336.8221s\n",
      "\titers: 300, epoch: 7 | loss: 0.5438702\n",
      "\tspeed: 0.0275s/iter; left time: 336.6430s\n",
      "\titers: 400, epoch: 7 | loss: 0.3777486\n",
      "\tspeed: 0.0278s/iter; left time: 338.1276s\n",
      "\titers: 500, epoch: 7 | loss: 0.4276087\n",
      "\tspeed: 0.0280s/iter; left time: 338.2015s\n",
      "\titers: 600, epoch: 7 | loss: 0.4295527\n",
      "\tspeed: 0.0269s/iter; left time: 322.1109s\n",
      "\titers: 700, epoch: 7 | loss: 0.4393570\n",
      "\tspeed: 0.0258s/iter; left time: 306.1280s\n",
      "\titers: 800, epoch: 7 | loss: 0.4118882\n",
      "\tspeed: 0.0264s/iter; left time: 310.5476s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:24.50s\n",
      "Steps: 897 | Train Loss: 0.4452508 Vali Loss: 0.6458729 Test Loss: 0.8272748\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.4742603\n",
      "\tspeed: 0.0791s/iter; left time: 914.7147s\n",
      "\titers: 200, epoch: 8 | loss: 0.3971712\n",
      "\tspeed: 0.0262s/iter; left time: 300.0183s\n",
      "\titers: 300, epoch: 8 | loss: 0.4236944\n",
      "\tspeed: 0.0263s/iter; left time: 298.7510s\n",
      "\titers: 400, epoch: 8 | loss: 0.4196156\n",
      "\tspeed: 0.0262s/iter; left time: 294.5263s\n",
      "\titers: 500, epoch: 8 | loss: 0.4691446\n",
      "\tspeed: 0.0271s/iter; left time: 302.0165s\n",
      "\titers: 600, epoch: 8 | loss: 0.3321491\n",
      "\tspeed: 0.0278s/iter; left time: 308.0055s\n",
      "\titers: 700, epoch: 8 | loss: 0.4951756\n",
      "\tspeed: 0.0282s/iter; left time: 308.8675s\n",
      "\titers: 800, epoch: 8 | loss: 0.4140143\n",
      "\tspeed: 0.0279s/iter; left time: 302.5336s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:24.28s\n",
      "Steps: 897 | Train Loss: 0.4307546 Vali Loss: 0.6581133 Test Loss: 0.8462913\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 5.904900000000001e-05\n",
      "\titers: 100, epoch: 9 | loss: 0.4991306\n",
      "\tspeed: 0.0807s/iter; left time: 860.3381s\n",
      "\titers: 200, epoch: 9 | loss: 0.5075079\n",
      "\tspeed: 0.0265s/iter; left time: 279.7415s\n",
      "\titers: 300, epoch: 9 | loss: 0.4385278\n",
      "\tspeed: 0.0267s/iter; left time: 279.1178s\n",
      "\titers: 400, epoch: 9 | loss: 0.4002862\n",
      "\tspeed: 0.0259s/iter; left time: 267.9980s\n",
      "\titers: 500, epoch: 9 | loss: 0.4016561\n",
      "\tspeed: 0.0268s/iter; left time: 274.6432s\n",
      "\titers: 600, epoch: 9 | loss: 0.4614063\n",
      "\tspeed: 0.0257s/iter; left time: 261.6523s\n",
      "\titers: 700, epoch: 9 | loss: 0.4610348\n",
      "\tspeed: 0.0272s/iter; left time: 273.3372s\n",
      "\titers: 800, epoch: 9 | loss: 0.3421496\n",
      "\tspeed: 0.0272s/iter; left time: 271.1107s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 9\n",
      "Cost time: 00h:00m:24.15s\n",
      "Steps: 897 | Train Loss: 0.4177802 Vali Loss: 0.6611531 Test Loss: 0.8501814\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_336_96_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl96_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6145\n",
      "Scaled mse:0.7837718725204468, rmse:0.8853089213371277, mae:0.6171771287918091, rse:0.70215904712677\n",
      "Original data scale mse:32140268.0, rmse:5669.23876953125, mae:3649.434326171875, rse:0.28233012557029724\n",
      "\n",
      "=== Starting experiments for pred_len: 168 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_336_168_scaler_choice_for_DE', model='PatchTST', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='standard', if_relu=False, channel_mixing=0, seq_len=336, label_len=48, pred_len=168, inverse=True, loss_fnc='MSE', fc_dropout=0.2, head_dropout=0.0, patch_len=32, stride=16, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=128, n_heads=16, e_layers=3, d_layers=1, d_ff=256, moving_avg=25, factor=1, distil=True, dropout=0.2, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_336_168_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl168_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 28633\n",
      "val 6073\n",
      "test 6073\n",
      "\titers: 100, epoch: 1 | loss: 1.0979578\n",
      "\tspeed: 0.0529s/iter; left time: 940.9306s\n",
      "\titers: 200, epoch: 1 | loss: 0.7812335\n",
      "\tspeed: 0.0256s/iter; left time: 452.7226s\n",
      "\titers: 300, epoch: 1 | loss: 0.7692299\n",
      "\tspeed: 0.0265s/iter; left time: 465.2399s\n",
      "\titers: 400, epoch: 1 | loss: 0.7496105\n",
      "\tspeed: 0.0254s/iter; left time: 444.8828s\n",
      "\titers: 500, epoch: 1 | loss: 0.8643950\n",
      "\tspeed: 0.0238s/iter; left time: 413.2807s\n",
      "\titers: 600, epoch: 1 | loss: 0.8250254\n",
      "\tspeed: 0.0214s/iter; left time: 369.6401s\n",
      "\titers: 700, epoch: 1 | loss: 0.8314230\n",
      "\tspeed: 0.0208s/iter; left time: 357.0463s\n",
      "\titers: 800, epoch: 1 | loss: 0.6790564\n",
      "\tspeed: 0.0262s/iter; left time: 448.0596s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:22.39s\n",
      "Steps: 894 | Train Loss: 0.8208767 Vali Loss: 0.7704461 Test Loss: 0.9012668\n",
      "Validation loss decreased (inf --> 0.770446).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.6493724\n",
      "\tspeed: 0.0820s/iter; left time: 1383.9009s\n",
      "\titers: 200, epoch: 2 | loss: 0.9108220\n",
      "\tspeed: 0.0251s/iter; left time: 421.8505s\n",
      "\titers: 300, epoch: 2 | loss: 0.5538403\n",
      "\tspeed: 0.0247s/iter; left time: 412.4497s\n",
      "\titers: 400, epoch: 2 | loss: 0.8628698\n",
      "\tspeed: 0.0244s/iter; left time: 405.1340s\n",
      "\titers: 500, epoch: 2 | loss: 0.5896612\n",
      "\tspeed: 0.0227s/iter; left time: 373.4424s\n",
      "\titers: 600, epoch: 2 | loss: 0.6238002\n",
      "\tspeed: 0.0232s/iter; left time: 380.0868s\n",
      "\titers: 700, epoch: 2 | loss: 0.7019249\n",
      "\tspeed: 0.0253s/iter; left time: 411.7858s\n",
      "\titers: 800, epoch: 2 | loss: 0.5525599\n",
      "\tspeed: 0.0257s/iter; left time: 416.4883s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:22.60s\n",
      "Steps: 894 | Train Loss: 0.6056082 Vali Loss: 0.6934643 Test Loss: 0.8506795\n",
      "Validation loss decreased (0.770446 --> 0.693464).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.6137096\n",
      "\tspeed: 0.0826s/iter; left time: 1320.2756s\n",
      "\titers: 200, epoch: 3 | loss: 0.5163989\n",
      "\tspeed: 0.0266s/iter; left time: 422.8676s\n",
      "\titers: 300, epoch: 3 | loss: 0.4615364\n",
      "\tspeed: 0.0266s/iter; left time: 419.3756s\n",
      "\titers: 400, epoch: 3 | loss: 0.5898119\n",
      "\tspeed: 0.0260s/iter; left time: 408.2545s\n",
      "\titers: 500, epoch: 3 | loss: 0.6073261\n",
      "\tspeed: 0.0259s/iter; left time: 404.1179s\n",
      "\titers: 600, epoch: 3 | loss: 0.5879161\n",
      "\tspeed: 0.0257s/iter; left time: 398.4728s\n",
      "\titers: 700, epoch: 3 | loss: 0.5766014\n",
      "\tspeed: 0.0282s/iter; left time: 434.6851s\n",
      "\titers: 800, epoch: 3 | loss: 0.5917500\n",
      "\tspeed: 0.0255s/iter; left time: 389.8110s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:23.83s\n",
      "Steps: 894 | Train Loss: 0.5631121 Vali Loss: 0.6926853 Test Loss: 0.8304730\n",
      "Validation loss decreased (0.693464 --> 0.692685).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.4691502\n",
      "\tspeed: 0.0821s/iter; left time: 1239.0119s\n",
      "\titers: 200, epoch: 4 | loss: 0.4433632\n",
      "\tspeed: 0.0276s/iter; left time: 414.7032s\n",
      "\titers: 300, epoch: 4 | loss: 0.5941539\n",
      "\tspeed: 0.0277s/iter; left time: 411.9591s\n",
      "\titers: 400, epoch: 4 | loss: 0.4265172\n",
      "\tspeed: 0.0261s/iter; left time: 386.2362s\n",
      "\titers: 500, epoch: 4 | loss: 0.5626983\n",
      "\tspeed: 0.0274s/iter; left time: 402.4388s\n",
      "\titers: 600, epoch: 4 | loss: 0.4693358\n",
      "\tspeed: 0.0273s/iter; left time: 397.8674s\n",
      "\titers: 700, epoch: 4 | loss: 0.5190641\n",
      "\tspeed: 0.0264s/iter; left time: 382.5743s\n",
      "\titers: 800, epoch: 4 | loss: 0.5175924\n",
      "\tspeed: 0.0258s/iter; left time: 371.7917s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:24.26s\n",
      "Steps: 894 | Train Loss: 0.5415466 Vali Loss: 0.7129731 Test Loss: 0.8530972\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.4650689\n",
      "\tspeed: 0.0794s/iter; left time: 1127.7913s\n",
      "\titers: 200, epoch: 5 | loss: 0.4794911\n",
      "\tspeed: 0.0255s/iter; left time: 359.9036s\n",
      "\titers: 300, epoch: 5 | loss: 0.5183489\n",
      "\tspeed: 0.0257s/iter; left time: 360.3564s\n",
      "\titers: 400, epoch: 5 | loss: 0.5231012\n",
      "\tspeed: 0.0274s/iter; left time: 381.5473s\n",
      "\titers: 500, epoch: 5 | loss: 0.5858033\n",
      "\tspeed: 0.0266s/iter; left time: 366.7485s\n",
      "\titers: 600, epoch: 5 | loss: 0.4935581\n",
      "\tspeed: 0.0262s/iter; left time: 359.1830s\n",
      "\titers: 700, epoch: 5 | loss: 0.5506232\n",
      "\tspeed: 0.0265s/iter; left time: 360.6072s\n",
      "\titers: 800, epoch: 5 | loss: 0.4523300\n",
      "\tspeed: 0.0270s/iter; left time: 364.7186s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:23.84s\n",
      "Steps: 894 | Train Loss: 0.5151335 Vali Loss: 0.7148864 Test Loss: 0.8561559\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.5759728\n",
      "\tspeed: 0.0811s/iter; left time: 1079.8728s\n",
      "\titers: 200, epoch: 6 | loss: 0.4942687\n",
      "\tspeed: 0.0257s/iter; left time: 339.6821s\n",
      "\titers: 300, epoch: 6 | loss: 0.4928923\n",
      "\tspeed: 0.0257s/iter; left time: 337.5245s\n",
      "\titers: 400, epoch: 6 | loss: 0.4270805\n",
      "\tspeed: 0.0265s/iter; left time: 345.0668s\n",
      "\titers: 500, epoch: 6 | loss: 0.4687174\n",
      "\tspeed: 0.0271s/iter; left time: 349.2471s\n",
      "\titers: 600, epoch: 6 | loss: 0.4441013\n",
      "\tspeed: 0.0266s/iter; left time: 341.3914s\n",
      "\titers: 700, epoch: 6 | loss: 0.5473166\n",
      "\tspeed: 0.0269s/iter; left time: 341.9699s\n",
      "\titers: 800, epoch: 6 | loss: 0.4409388\n",
      "\tspeed: 0.0266s/iter; left time: 335.9697s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:23.88s\n",
      "Steps: 894 | Train Loss: 0.4924809 Vali Loss: 0.7261763 Test Loss: 0.8750628\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.5669965\n",
      "\tspeed: 0.0804s/iter; left time: 998.2584s\n",
      "\titers: 200, epoch: 7 | loss: 0.4517164\n",
      "\tspeed: 0.0263s/iter; left time: 324.5506s\n",
      "\titers: 300, epoch: 7 | loss: 0.4270353\n",
      "\tspeed: 0.0272s/iter; left time: 332.4367s\n",
      "\titers: 400, epoch: 7 | loss: 0.4793290\n",
      "\tspeed: 0.0259s/iter; left time: 313.7076s\n",
      "\titers: 500, epoch: 7 | loss: 0.4417716\n",
      "\tspeed: 0.0266s/iter; left time: 319.8051s\n",
      "\titers: 600, epoch: 7 | loss: 0.4892814\n",
      "\tspeed: 0.0257s/iter; left time: 306.8141s\n",
      "\titers: 700, epoch: 7 | loss: 0.4100447\n",
      "\tspeed: 0.0262s/iter; left time: 310.1018s\n",
      "\titers: 800, epoch: 7 | loss: 0.4764896\n",
      "\tspeed: 0.0255s/iter; left time: 298.4330s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:23.54s\n",
      "Steps: 894 | Train Loss: 0.4731314 Vali Loss: 0.7361413 Test Loss: 0.8779513\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.4797615\n",
      "\tspeed: 0.0798s/iter; left time: 919.3834s\n",
      "\titers: 200, epoch: 8 | loss: 0.4817539\n",
      "\tspeed: 0.0276s/iter; left time: 314.8894s\n",
      "\titers: 300, epoch: 8 | loss: 0.4858697\n",
      "\tspeed: 0.0286s/iter; left time: 323.7980s\n",
      "\titers: 400, epoch: 8 | loss: 0.4218687\n",
      "\tspeed: 0.0270s/iter; left time: 302.8280s\n",
      "\titers: 500, epoch: 8 | loss: 0.3941292\n",
      "\tspeed: 0.0261s/iter; left time: 290.0680s\n",
      "\titers: 600, epoch: 8 | loss: 0.4460558\n",
      "\tspeed: 0.0277s/iter; left time: 304.8953s\n",
      "\titers: 700, epoch: 8 | loss: 0.4927359\n",
      "\tspeed: 0.0272s/iter; left time: 297.2919s\n",
      "\titers: 800, epoch: 8 | loss: 0.4673576\n",
      "\tspeed: 0.0267s/iter; left time: 289.0995s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:24.41s\n",
      "Steps: 894 | Train Loss: 0.4583363 Vali Loss: 0.7625589 Test Loss: 0.8916399\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_336_168_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl168_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6073\n",
      "Scaled mse:0.8304733037948608, rmse:0.9113031029701233, mae:0.6477203965187073, rse:0.7219134569168091\n",
      "Original data scale mse:34333580.0, rmse:5859.486328125, mae:3842.8037109375, rse:0.29194775223731995\n"
     ]
    }
   ],
   "source": [
    "# Dynamic + default variables\n",
    "pred_lens = [\"24\", \"96\", \"168\"]\n",
    "seq_len = \"336\"\n",
    "lr = \"0.0001\"\n",
    "model = \"PatchTST\"\n",
    "itr = 1  \n",
    "n_heads = \"16\"\n",
    "d_model = \"128\"\n",
    "d_ff = \"256\"\n",
    "dropout = \"0.2\"\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = cuda_device\n",
    "\n",
    "# New log file path\n",
    "log_file_path = f\"{log_dir}/{model}_{country}.log\"\n",
    "\n",
    "patchtst_results_scaled, patchtst_results_unscaled = [], []\n",
    "\n",
    "with open(log_file_path, \"w\") as log_file:\n",
    "    for loss in losses:\n",
    "        statement_1 = f\"\\n=== Starting experiments for loss function: {loss} ===\\n\"\n",
    "        log_file.write(statement_1)\n",
    "        print(statement_1)  # Print to notebook\n",
    "\n",
    "        for pred_len in pred_lens:\n",
    "            statement_2 = f\"\\n=== Starting experiments for pred_len: {pred_len} ===\\n\"\n",
    "            log_file.write(statement_2)\n",
    "            print(statement_2) \n",
    "            model_id = f\"{country}_{seq_len}_{pred_len}_scaler_choice_for_{country}\"\n",
    "                \n",
    "            # Command arguments\n",
    "            command = f\"\"\"\n",
    "            python {script_path} \\\n",
    "              --random_seed 2021 \\\n",
    "              --is_training 1 \\\n",
    "              --root_path \"{data_path}\" \\\n",
    "              --data_path \"{dataset}\" \\\n",
    "              --model_id {model_id} \\\n",
    "              --model \"{model}\" \\\n",
    "              --data \"custom\" \\\n",
    "              --features M \\\n",
    "              --seq_len {seq_len} \\\n",
    "              --pred_len {pred_len} \\\n",
    "              --e_layers 3 \\\n",
    "              --factor 1 \\\n",
    "              --enc_in 5 \\\n",
    "              --dec_in 5 \\\n",
    "              --c_out 5 \\\n",
    "              --des 'Exp' \\\n",
    "              --train_epochs 20 \\\n",
    "              --patience 5 \\\n",
    "              --n_heads {n_heads} \\\n",
    "              --d_model {d_model} \\\n",
    "              --d_ff {d_ff} \\\n",
    "              --dropout {dropout} \\\n",
    "              --fc_dropout {dropout} \\\n",
    "              --patch_len 32 \\\n",
    "              --stride 16 \\\n",
    "              --overlapping_windows \\\n",
    "              --inverse \\\n",
    "              --scaler_type standard \\\n",
    "              --loss_fnc \"{loss}\" \\\n",
    "              --itr {itr} --batch_size 32 --learning_rate \"{lr}\"\n",
    "            \"\"\"\n",
    "\n",
    "            # Run the command and capture the output\n",
    "            process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)\n",
    "\n",
    "            # Capture the output in real-time\n",
    "            output = []\n",
    "            for line in process.stdout:\n",
    "                output.append(line)\n",
    "                print(line, end='')  # Print in the .ipynb cell\n",
    "                log_file.write(line)  # Write to the log file\n",
    "\n",
    "            # Wait for the process to complete\n",
    "            process.wait()\n",
    "\n",
    "            # Delete the checkpoints folder and all its contents\n",
    "            shutil.rmtree('./checkpoints' )\n",
    "\n",
    "            # Extract metrics for each iteration\n",
    "            iteration_metrics_scaled = extract_metrics_from_output(output, itr)\n",
    "            iteration_metrics_unscaled = extract_metrics_from_output(output, itr, if_scaled=False)\n",
    "\n",
    "            # Log the extracted metrics and save them\n",
    "            for iteration, (scaled_metrics, unscaled_metrics) in enumerate(zip(iteration_metrics_scaled, iteration_metrics_unscaled), start=1):\n",
    "                log_file.write(f\"\\nExtracted Metrics for {country}, pred_len={pred_len}, iteration={iteration}:\\n\")\n",
    "                log_file.write(f\"Scaled Metrics - MSE: {scaled_metrics[0]}, RMSE: {scaled_metrics[1]}, MAE: {scaled_metrics[2]}, RSE: {scaled_metrics[3]}\\n\")\n",
    "                log_file.write(f\"Unscaled Metrics - MSE: {unscaled_metrics[0]}, RMSE: {unscaled_metrics[1]}, MAE: {unscaled_metrics[2]}, RSE: {unscaled_metrics[3]}\\n\")\n",
    "\n",
    "                # Append the results to the informer_results lists\n",
    "                metrics_data = [(patchtst_results_scaled, scaled_metrics), (patchtst_results_unscaled, unscaled_metrics)]\n",
    "\n",
    "                for result_list, metrics in metrics_data:\n",
    "                    result_list.append({\n",
    "                        'Loss_function': loss,\n",
    "                        'Pred_len': pred_len,\n",
    "                        'Iteration': iteration,\n",
    "                        'MSE': metrics[0],\n",
    "                        'RMSE': metrics[1],\n",
    "                        'MAE': metrics[2],\n",
    "                        'RSE': metrics[3]\n",
    "                    })"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>MSE</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>RSE</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Loss_function</th>\n",
       "      <th>Iteration</th>\n",
       "      <th>Pred_len</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">MSE</th>\n",
       "      <th rowspan=\"3\" valign=\"top\">1</th>\n",
       "      <th>24</th>\n",
       "      <td>0.4387</td>\n",
       "      <td>0.6623</td>\n",
       "      <td>0.4260</td>\n",
       "      <td>0.5242</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>0.7838</td>\n",
       "      <td>0.8853</td>\n",
       "      <td>0.6172</td>\n",
       "      <td>0.7022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>0.8305</td>\n",
       "      <td>0.9113</td>\n",
       "      <td>0.6477</td>\n",
       "      <td>0.7219</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     MSE    RMSE     MAE     RSE\n",
       "Loss_function Iteration Pred_len                                \n",
       "MSE           1         24        0.4387  0.6623  0.4260  0.5242\n",
       "                        96        0.7838  0.8853  0.6172  0.7022\n",
       "                        168       0.8305  0.9113  0.6477  0.7219"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path_dir = './results/scaler_choice'\n",
    "csv_name_scaled = 'patchtst_scaler_results_scaled_default.csv'\n",
    "csv_name_unscaled = 'patchtst_scaler_results_unscaled_default.csv'\n",
    "patchtst_df_scaled = convert_results_into_df(patchtst_results_scaled, path_dir, csv_name_scaled)\n",
    "patchtst_df_unscaled = convert_results_into_df(patchtst_results_unscaled, path_dir, csv_name_unscaled)\n",
    "patchtst_df_scaled.round(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>MSE</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>RSE</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Loss_function</th>\n",
       "      <th>Iteration</th>\n",
       "      <th>Pred_len</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">MSE</th>\n",
       "      <th rowspan=\"3\" valign=\"top\">1</th>\n",
       "      <th>24</th>\n",
       "      <td>16448487.0</td>\n",
       "      <td>4055.6733</td>\n",
       "      <td>2475.6689</td>\n",
       "      <td>0.2017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>32140268.0</td>\n",
       "      <td>5669.2388</td>\n",
       "      <td>3649.4343</td>\n",
       "      <td>0.2823</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>34333580.0</td>\n",
       "      <td>5859.4863</td>\n",
       "      <td>3842.8037</td>\n",
       "      <td>0.2919</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         MSE       RMSE        MAE     RSE\n",
       "Loss_function Iteration Pred_len                                          \n",
       "MSE           1         24        16448487.0  4055.6733  2475.6689  0.2017\n",
       "                        96        32140268.0  5669.2388  3649.4343  0.2823\n",
       "                        168       34333580.0  5859.4863  3842.8037  0.2919"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "patchtst_df_unscaled.round(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "shutil.rmtree(\"results_loss_scaled\") # we do not need this directory and results anymore. If you need - comment this line\n",
    "\n",
    "# Rename folder\n",
    "os.rename(\"results_loss_unscaled\", 'standard_unscaled')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. MinMax Scaler Informer\n",
    "\n",
    "We can use now \"ReLU\" activation function due to MinMax Scaler.\n",
    "\n",
    "With BS 1036, ReLU - results are bad. (as twice as bad as with 32!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Paths to files and data\n",
    "data_path = os.getcwd() + \"/datasets/\"\n",
    "\n",
    "script_path = \"./PatchTST-main/PatchTST_supervised/run_longExp.py\"\n",
    "\n",
    "# Arguments that will be used also for file names\n",
    "model = \"Informer\"\n",
    "dataset = 'DE_data.csv'\n",
    "losses = [\"MSE\"]\n",
    "country = dataset[:2]\n",
    "\n",
    "log_dir = f\"logs/scaler_choice/min_max\"\n",
    "if not os.path.exists(log_dir):\n",
    "    os.makedirs(log_dir)\n",
    "\n",
    "log_file_path = f\"{log_dir}/{model}_{country}.log\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== Starting experiments for loss function: MSE ===\n",
      "\n",
      "\n",
      "=== Starting experiments for pred_len: 24 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_96_24_scaler_choice_for_DE', model='Informer', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='minmax', if_relu=True, channel_mixing=0, seq_len=96, label_len=48, pred_len=24, inverse=True, loss_fnc='MSE', fc_dropout=0.05, head_dropout=0.0, patch_len=16, stride=8, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=512, n_heads=8, e_layers=2, d_layers=1, d_ff=2048, moving_avg=25, factor=5, distil=True, dropout=0.1, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_96_24_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl24_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 29017\n",
      "val 6217\n",
      "test 6217\n",
      "\titers: 100, epoch: 1 | loss: 0.0844291\n",
      "\tspeed: 0.0882s/iter; left time: 1589.2462s\n",
      "\titers: 200, epoch: 1 | loss: 0.0693888\n",
      "\tspeed: 0.0497s/iter; left time: 890.3039s\n",
      "\titers: 300, epoch: 1 | loss: 0.0655493\n",
      "\tspeed: 0.0498s/iter; left time: 887.5420s\n",
      "\titers: 400, epoch: 1 | loss: 0.0564399\n",
      "\tspeed: 0.0496s/iter; left time: 878.8175s\n",
      "\titers: 500, epoch: 1 | loss: 0.0501456\n",
      "\tspeed: 0.0494s/iter; left time: 870.5523s\n",
      "\titers: 600, epoch: 1 | loss: 0.0479440\n",
      "\tspeed: 0.0499s/iter; left time: 874.7438s\n",
      "\titers: 700, epoch: 1 | loss: 0.0540418\n",
      "\tspeed: 0.0488s/iter; left time: 850.5009s\n",
      "\titers: 800, epoch: 1 | loss: 0.0417317\n",
      "\tspeed: 0.0491s/iter; left time: 850.0658s\n",
      "\titers: 900, epoch: 1 | loss: 0.0365065\n",
      "\tspeed: 0.0494s/iter; left time: 850.1748s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:46.55s\n",
      "Steps: 906 | Train Loss: 0.0620887 Vali Loss: 0.0407531 Test Loss: 0.0445114\n",
      "Validation loss decreased (inf --> 0.040753).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.0251532\n",
      "\tspeed: 0.1115s/iter; left time: 1908.8361s\n",
      "\titers: 200, epoch: 2 | loss: 0.0322412\n",
      "\tspeed: 0.0500s/iter; left time: 850.4026s\n",
      "\titers: 300, epoch: 2 | loss: 0.0245991\n",
      "\tspeed: 0.0497s/iter; left time: 840.0318s\n",
      "\titers: 400, epoch: 2 | loss: 0.0171704\n",
      "\tspeed: 0.0497s/iter; left time: 836.3840s\n",
      "\titers: 500, epoch: 2 | loss: 0.0195076\n",
      "\tspeed: 0.0498s/iter; left time: 832.0382s\n",
      "\titers: 600, epoch: 2 | loss: 0.0165113\n",
      "\tspeed: 0.0496s/iter; left time: 824.4478s\n",
      "\titers: 700, epoch: 2 | loss: 0.0190108\n",
      "\tspeed: 0.0490s/iter; left time: 808.6833s\n",
      "\titers: 800, epoch: 2 | loss: 0.0165992\n",
      "\tspeed: 0.0488s/iter; left time: 801.8536s\n",
      "\titers: 900, epoch: 2 | loss: 0.0153506\n",
      "\tspeed: 0.0487s/iter; left time: 794.3668s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:45.21s\n",
      "Steps: 906 | Train Loss: 0.0211746 Vali Loss: 0.0227766 Test Loss: 0.0249225\n",
      "Validation loss decreased (0.040753 --> 0.022777).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.0191735\n",
      "\tspeed: 0.0970s/iter; left time: 1573.0170s\n",
      "\titers: 200, epoch: 3 | loss: 0.0155110\n",
      "\tspeed: 0.0406s/iter; left time: 654.6689s\n",
      "\titers: 300, epoch: 3 | loss: 0.0156315\n",
      "\tspeed: 0.0406s/iter; left time: 650.5093s\n",
      "\titers: 400, epoch: 3 | loss: 0.0145043\n",
      "\tspeed: 0.0406s/iter; left time: 646.1526s\n",
      "\titers: 500, epoch: 3 | loss: 0.0143492\n",
      "\tspeed: 0.0406s/iter; left time: 641.4356s\n",
      "\titers: 600, epoch: 3 | loss: 0.0139118\n",
      "\tspeed: 0.0406s/iter; left time: 638.0416s\n",
      "\titers: 700, epoch: 3 | loss: 0.0120604\n",
      "\tspeed: 0.0406s/iter; left time: 633.6134s\n",
      "\titers: 800, epoch: 3 | loss: 0.0160756\n",
      "\tspeed: 0.0425s/iter; left time: 658.6520s\n",
      "\titers: 900, epoch: 3 | loss: 0.0143501\n",
      "\tspeed: 0.0405s/iter; left time: 624.4087s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:37.23s\n",
      "Steps: 906 | Train Loss: 0.0150597 Vali Loss: 0.0232971 Test Loss: 0.0252371\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.0124714\n",
      "\tspeed: 0.1022s/iter; left time: 1563.2316s\n",
      "\titers: 200, epoch: 4 | loss: 0.0135723\n",
      "\tspeed: 0.0486s/iter; left time: 738.6894s\n",
      "\titers: 300, epoch: 4 | loss: 0.0133752\n",
      "\tspeed: 0.0492s/iter; left time: 743.3027s\n",
      "\titers: 400, epoch: 4 | loss: 0.0128325\n",
      "\tspeed: 0.0490s/iter; left time: 734.7768s\n",
      "\titers: 500, epoch: 4 | loss: 0.0170870\n",
      "\tspeed: 0.0500s/iter; left time: 745.5419s\n",
      "\titers: 600, epoch: 4 | loss: 0.0159228\n",
      "\tspeed: 0.0493s/iter; left time: 729.1818s\n",
      "\titers: 700, epoch: 4 | loss: 0.0160074\n",
      "\tspeed: 0.0491s/iter; left time: 721.2973s\n",
      "\titers: 800, epoch: 4 | loss: 0.0128487\n",
      "\tspeed: 0.0495s/iter; left time: 723.3591s\n",
      "\titers: 900, epoch: 4 | loss: 0.0095744\n",
      "\tspeed: 0.0498s/iter; left time: 722.1532s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:44.87s\n",
      "Steps: 906 | Train Loss: 0.0137407 Vali Loss: 0.0219377 Test Loss: 0.0240274\n",
      "Validation loss decreased (0.022777 --> 0.021938).  Saving model ...\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.0123596\n",
      "\tspeed: 0.1068s/iter; left time: 1536.9268s\n",
      "\titers: 200, epoch: 5 | loss: 0.0123557\n",
      "\tspeed: 0.0493s/iter; left time: 704.4004s\n",
      "\titers: 300, epoch: 5 | loss: 0.0114830\n",
      "\tspeed: 0.0488s/iter; left time: 692.9050s\n",
      "\titers: 400, epoch: 5 | loss: 0.0166790\n",
      "\tspeed: 0.0495s/iter; left time: 697.7833s\n",
      "\titers: 500, epoch: 5 | loss: 0.0130099\n",
      "\tspeed: 0.0496s/iter; left time: 694.3132s\n",
      "\titers: 600, epoch: 5 | loss: 0.0162379\n",
      "\tspeed: 0.0496s/iter; left time: 689.7491s\n",
      "\titers: 700, epoch: 5 | loss: 0.0147405\n",
      "\tspeed: 0.0492s/iter; left time: 679.1198s\n",
      "\titers: 800, epoch: 5 | loss: 0.0135671\n",
      "\tspeed: 0.0492s/iter; left time: 674.0310s\n",
      "\titers: 900, epoch: 5 | loss: 0.0163193\n",
      "\tspeed: 0.0491s/iter; left time: 667.6958s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:44.96s\n",
      "Steps: 906 | Train Loss: 0.0126119 Vali Loss: 0.0216679 Test Loss: 0.0243225\n",
      "Validation loss decreased (0.021938 --> 0.021668).  Saving model ...\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.0116733\n",
      "\tspeed: 0.1070s/iter; left time: 1443.4834s\n",
      "\titers: 200, epoch: 6 | loss: 0.0101081\n",
      "\tspeed: 0.0498s/iter; left time: 667.4018s\n",
      "\titers: 300, epoch: 6 | loss: 0.0103430\n",
      "\tspeed: 0.0502s/iter; left time: 666.7518s\n",
      "\titers: 400, epoch: 6 | loss: 0.0111628\n",
      "\tspeed: 0.0495s/iter; left time: 652.7870s\n",
      "\titers: 500, epoch: 6 | loss: 0.0122635\n",
      "\tspeed: 0.0493s/iter; left time: 645.7760s\n",
      "\titers: 600, epoch: 6 | loss: 0.0087395\n",
      "\tspeed: 0.0488s/iter; left time: 633.4605s\n",
      "\titers: 700, epoch: 6 | loss: 0.0120656\n",
      "\tspeed: 0.0493s/iter; left time: 635.7807s\n",
      "\titers: 800, epoch: 6 | loss: 0.0125768\n",
      "\tspeed: 0.0491s/iter; left time: 627.8910s\n",
      "\titers: 900, epoch: 6 | loss: 0.0099931\n",
      "\tspeed: 0.0491s/iter; left time: 623.0495s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:45.03s\n",
      "Steps: 906 | Train Loss: 0.0116739 Vali Loss: 0.0211125 Test Loss: 0.0254185\n",
      "Validation loss decreased (0.021668 --> 0.021113).  Saving model ...\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.0096771\n",
      "\tspeed: 0.1063s/iter; left time: 1337.5533s\n",
      "\titers: 200, epoch: 7 | loss: 0.0094396\n",
      "\tspeed: 0.0489s/iter; left time: 609.9079s\n",
      "\titers: 300, epoch: 7 | loss: 0.0114610\n",
      "\tspeed: 0.0488s/iter; left time: 604.9412s\n",
      "\titers: 400, epoch: 7 | loss: 0.0119442\n",
      "\tspeed: 0.0487s/iter; left time: 598.4630s\n",
      "\titers: 500, epoch: 7 | loss: 0.0132686\n",
      "\tspeed: 0.0491s/iter; left time: 598.3323s\n",
      "\titers: 600, epoch: 7 | loss: 0.0079299\n",
      "\tspeed: 0.0494s/iter; left time: 597.3615s\n",
      "\titers: 700, epoch: 7 | loss: 0.0124909\n",
      "\tspeed: 0.0494s/iter; left time: 592.3409s\n",
      "\titers: 800, epoch: 7 | loss: 0.0076778\n",
      "\tspeed: 0.0495s/iter; left time: 588.6138s\n",
      "\titers: 900, epoch: 7 | loss: 0.0091967\n",
      "\tspeed: 0.0492s/iter; left time: 580.1843s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:44.84s\n",
      "Steps: 906 | Train Loss: 0.0107831 Vali Loss: 0.0222866 Test Loss: 0.0263801\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.0086997\n",
      "\tspeed: 0.1042s/iter; left time: 1216.6490s\n",
      "\titers: 200, epoch: 8 | loss: 0.0115932\n",
      "\tspeed: 0.0493s/iter; left time: 571.3079s\n",
      "\titers: 300, epoch: 8 | loss: 0.0087272\n",
      "\tspeed: 0.0507s/iter; left time: 581.8266s\n",
      "\titers: 400, epoch: 8 | loss: 0.0111578\n",
      "\tspeed: 0.0499s/iter; left time: 567.4768s\n",
      "\titers: 500, epoch: 8 | loss: 0.0074500\n",
      "\tspeed: 0.0505s/iter; left time: 569.7127s\n",
      "\titers: 600, epoch: 8 | loss: 0.0114486\n",
      "\tspeed: 0.0492s/iter; left time: 550.2863s\n",
      "\titers: 700, epoch: 8 | loss: 0.0086220\n",
      "\tspeed: 0.0493s/iter; left time: 545.9020s\n",
      "\titers: 800, epoch: 8 | loss: 0.0097864\n",
      "\tspeed: 0.0508s/iter; left time: 557.6993s\n",
      "\titers: 900, epoch: 8 | loss: 0.0122829\n",
      "\tspeed: 0.0499s/iter; left time: 543.2821s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:45.51s\n",
      "Steps: 906 | Train Loss: 0.0098266 Vali Loss: 0.0233274 Test Loss: 0.0263052\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 5.904900000000001e-05\n",
      "\titers: 100, epoch: 9 | loss: 0.0102292\n",
      "\tspeed: 0.1044s/iter; left time: 1124.2814s\n",
      "\titers: 200, epoch: 9 | loss: 0.0096972\n",
      "\tspeed: 0.0496s/iter; left time: 529.5387s\n",
      "\titers: 300, epoch: 9 | loss: 0.0081635\n",
      "\tspeed: 0.0494s/iter; left time: 522.8030s\n",
      "\titers: 400, epoch: 9 | loss: 0.0104667\n",
      "\tspeed: 0.0495s/iter; left time: 518.5379s\n",
      "\titers: 500, epoch: 9 | loss: 0.0084879\n",
      "\tspeed: 0.0496s/iter; left time: 514.9702s\n",
      "\titers: 600, epoch: 9 | loss: 0.0097039\n",
      "\tspeed: 0.0492s/iter; left time: 505.8134s\n",
      "\titers: 700, epoch: 9 | loss: 0.0099554\n",
      "\tspeed: 0.0495s/iter; left time: 503.2534s\n",
      "\titers: 800, epoch: 9 | loss: 0.0107502\n",
      "\tspeed: 0.0494s/iter; left time: 497.3192s\n",
      "\titers: 900, epoch: 9 | loss: 0.0077211\n",
      "\tspeed: 0.0492s/iter; left time: 490.4062s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 9\n",
      "Cost time: 00h:00m:45.06s\n",
      "Steps: 906 | Train Loss: 0.0090420 Vali Loss: 0.0235619 Test Loss: 0.0281686\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 5.3144100000000005e-05\n",
      "\titers: 100, epoch: 10 | loss: 0.0089565\n",
      "\tspeed: 0.1021s/iter; left time: 1007.2380s\n",
      "\titers: 200, epoch: 10 | loss: 0.0086324\n",
      "\tspeed: 0.0495s/iter; left time: 483.6238s\n",
      "\titers: 300, epoch: 10 | loss: 0.0090554\n",
      "\tspeed: 0.0494s/iter; left time: 477.9484s\n",
      "\titers: 400, epoch: 10 | loss: 0.0098368\n",
      "\tspeed: 0.0494s/iter; left time: 472.7586s\n",
      "\titers: 500, epoch: 10 | loss: 0.0081708\n",
      "\tspeed: 0.0492s/iter; left time: 466.0368s\n",
      "\titers: 600, epoch: 10 | loss: 0.0068844\n",
      "\tspeed: 0.0496s/iter; left time: 464.6630s\n",
      "\titers: 700, epoch: 10 | loss: 0.0071657\n",
      "\tspeed: 0.0494s/iter; left time: 457.6230s\n",
      "\titers: 800, epoch: 10 | loss: 0.0073063\n",
      "\tspeed: 0.0492s/iter; left time: 451.1310s\n",
      "\titers: 900, epoch: 10 | loss: 0.0070751\n",
      "\tspeed: 0.0490s/iter; left time: 444.4632s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 10\n",
      "Cost time: 00h:00m:44.96s\n",
      "Steps: 906 | Train Loss: 0.0083348 Vali Loss: 0.0225941 Test Loss: 0.0268508\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 4.782969000000001e-05\n",
      "\titers: 100, epoch: 11 | loss: 0.0080009\n",
      "\tspeed: 0.1041s/iter; left time: 932.5755s\n",
      "\titers: 200, epoch: 11 | loss: 0.0075677\n",
      "\tspeed: 0.0489s/iter; left time: 433.5443s\n",
      "\titers: 300, epoch: 11 | loss: 0.0058150\n",
      "\tspeed: 0.0489s/iter; left time: 428.2408s\n",
      "\titers: 400, epoch: 11 | loss: 0.0062782\n",
      "\tspeed: 0.0488s/iter; left time: 422.6513s\n",
      "\titers: 500, epoch: 11 | loss: 0.0092325\n",
      "\tspeed: 0.0488s/iter; left time: 417.3940s\n",
      "\titers: 600, epoch: 11 | loss: 0.0077828\n",
      "\tspeed: 0.0492s/iter; left time: 416.4174s\n",
      "\titers: 700, epoch: 11 | loss: 0.0077392\n",
      "\tspeed: 0.0493s/iter; left time: 411.8546s\n",
      "\titers: 800, epoch: 11 | loss: 0.0067178\n",
      "\tspeed: 0.0492s/iter; left time: 406.3749s\n",
      "\titers: 900, epoch: 11 | loss: 0.0054118\n",
      "\tspeed: 0.0492s/iter; left time: 401.2608s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 11\n",
      "Cost time: 00h:00m:44.72s\n",
      "Steps: 906 | Train Loss: 0.0075969 Vali Loss: 0.0233432 Test Loss: 0.0285462\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_96_24_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl24_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6217\n",
      "Scaled mse:0.02549596317112446, rmse:0.15967455506324768, mae:0.10630149394273758, rse:0.5638949871063232\n",
      "Original data scale mse:21864558.0, rmse:4675.955078125, mae:2984.4697265625, rse:0.2324979156255722\n",
      "\n",
      "=== Starting experiments for pred_len: 96 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_96_96_scaler_choice_for_DE', model='Informer', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='minmax', if_relu=True, channel_mixing=0, seq_len=96, label_len=48, pred_len=96, inverse=True, loss_fnc='MSE', fc_dropout=0.05, head_dropout=0.0, patch_len=16, stride=8, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=512, n_heads=8, e_layers=2, d_layers=1, d_ff=2048, moving_avg=25, factor=5, distil=True, dropout=0.1, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_96_96_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl96_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 28945\n",
      "val 6145\n",
      "test 6145\n",
      "\titers: 100, epoch: 1 | loss: 0.0908792\n",
      "\tspeed: 0.0826s/iter; left time: 1485.8324s\n",
      "\titers: 200, epoch: 1 | loss: 0.0826019\n",
      "\tspeed: 0.0508s/iter; left time: 907.9116s\n",
      "\titers: 300, epoch: 1 | loss: 0.0739485\n",
      "\tspeed: 0.0507s/iter; left time: 902.3200s\n",
      "\titers: 400, epoch: 1 | loss: 0.0652045\n",
      "\tspeed: 0.0507s/iter; left time: 896.9798s\n",
      "\titers: 500, epoch: 1 | loss: 0.0608115\n",
      "\tspeed: 0.0507s/iter; left time: 892.0873s\n",
      "\titers: 600, epoch: 1 | loss: 0.0582726\n",
      "\tspeed: 0.0508s/iter; left time: 887.6143s\n",
      "\titers: 700, epoch: 1 | loss: 0.0530457\n",
      "\tspeed: 0.0508s/iter; left time: 882.2590s\n",
      "\titers: 800, epoch: 1 | loss: 0.0556172\n",
      "\tspeed: 0.0508s/iter; left time: 877.3740s\n",
      "\titers: 900, epoch: 1 | loss: 0.0533727\n",
      "\tspeed: 0.0507s/iter; left time: 871.5903s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:46.76s\n",
      "Steps: 904 | Train Loss: 0.0693695 Vali Loss: 0.0569431 Test Loss: 0.0726330\n",
      "Validation loss decreased (inf --> 0.056943).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.0445395\n",
      "\tspeed: 0.1149s/iter; left time: 1962.9346s\n",
      "\titers: 200, epoch: 2 | loss: 0.0347430\n",
      "\tspeed: 0.0504s/iter; left time: 855.1274s\n",
      "\titers: 300, epoch: 2 | loss: 0.0310663\n",
      "\tspeed: 0.0506s/iter; left time: 853.8889s\n",
      "\titers: 400, epoch: 2 | loss: 0.0340965\n",
      "\tspeed: 0.0507s/iter; left time: 851.3432s\n",
      "\titers: 500, epoch: 2 | loss: 0.0340284\n",
      "\tspeed: 0.0508s/iter; left time: 846.5637s\n",
      "\titers: 600, epoch: 2 | loss: 0.0291398\n",
      "\tspeed: 0.0508s/iter; left time: 841.9530s\n",
      "\titers: 700, epoch: 2 | loss: 0.0287280\n",
      "\tspeed: 0.0507s/iter; left time: 836.1920s\n",
      "\titers: 800, epoch: 2 | loss: 0.0285814\n",
      "\tspeed: 0.0507s/iter; left time: 831.0109s\n",
      "\titers: 900, epoch: 2 | loss: 0.0318606\n",
      "\tspeed: 0.0506s/iter; left time: 824.2511s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:46.08s\n",
      "Steps: 904 | Train Loss: 0.0336223 Vali Loss: 0.0370270 Test Loss: 0.0475181\n",
      "Validation loss decreased (0.056943 --> 0.037027).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.0267080\n",
      "\tspeed: 0.1147s/iter; left time: 1855.3909s\n",
      "\titers: 200, epoch: 3 | loss: 0.0226177\n",
      "\tspeed: 0.0508s/iter; left time: 816.0968s\n",
      "\titers: 300, epoch: 3 | loss: 0.0249391\n",
      "\tspeed: 0.0508s/iter; left time: 811.1428s\n",
      "\titers: 400, epoch: 3 | loss: 0.0226962\n",
      "\tspeed: 0.0508s/iter; left time: 806.0078s\n",
      "\titers: 500, epoch: 3 | loss: 0.0232413\n",
      "\tspeed: 0.0508s/iter; left time: 801.0491s\n",
      "\titers: 600, epoch: 3 | loss: 0.0222054\n",
      "\tspeed: 0.0508s/iter; left time: 795.5253s\n",
      "\titers: 700, epoch: 3 | loss: 0.0253964\n",
      "\tspeed: 0.0508s/iter; left time: 790.3800s\n",
      "\titers: 800, epoch: 3 | loss: 0.0199290\n",
      "\tspeed: 0.0507s/iter; left time: 785.2387s\n",
      "\titers: 900, epoch: 3 | loss: 0.0254071\n",
      "\tspeed: 0.0507s/iter; left time: 780.0190s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:46.16s\n",
      "Steps: 904 | Train Loss: 0.0236946 Vali Loss: 0.0336329 Test Loss: 0.0415587\n",
      "Validation loss decreased (0.037027 --> 0.033633).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.0275110\n",
      "\tspeed: 0.1144s/iter; left time: 1747.2428s\n",
      "\titers: 200, epoch: 4 | loss: 0.0187502\n",
      "\tspeed: 0.0508s/iter; left time: 770.0094s\n",
      "\titers: 300, epoch: 4 | loss: 0.0265851\n",
      "\tspeed: 0.0508s/iter; left time: 765.6966s\n",
      "\titers: 400, epoch: 4 | loss: 0.0211057\n",
      "\tspeed: 0.0508s/iter; left time: 760.6529s\n",
      "\titers: 500, epoch: 4 | loss: 0.0217084\n",
      "\tspeed: 0.0508s/iter; left time: 754.7654s\n",
      "\titers: 600, epoch: 4 | loss: 0.0198707\n",
      "\tspeed: 0.0508s/iter; left time: 749.6991s\n",
      "\titers: 700, epoch: 4 | loss: 0.0226085\n",
      "\tspeed: 0.0508s/iter; left time: 744.6725s\n",
      "\titers: 800, epoch: 4 | loss: 0.0182402\n",
      "\tspeed: 0.0503s/iter; left time: 732.2430s\n",
      "\titers: 900, epoch: 4 | loss: 0.0187977\n",
      "\tspeed: 0.0505s/iter; left time: 730.3424s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:46.07s\n",
      "Steps: 904 | Train Loss: 0.0206394 Vali Loss: 0.0352910 Test Loss: 0.0430563\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.0173709\n",
      "\tspeed: 0.1112s/iter; left time: 1597.5864s\n",
      "\titers: 200, epoch: 5 | loss: 0.0166946\n",
      "\tspeed: 0.0508s/iter; left time: 724.3053s\n",
      "\titers: 300, epoch: 5 | loss: 0.0180556\n",
      "\tspeed: 0.0507s/iter; left time: 718.7875s\n",
      "\titers: 400, epoch: 5 | loss: 0.0225047\n",
      "\tspeed: 0.0508s/iter; left time: 714.5338s\n",
      "\titers: 500, epoch: 5 | loss: 0.0165685\n",
      "\tspeed: 0.0508s/iter; left time: 708.9317s\n",
      "\titers: 600, epoch: 5 | loss: 0.0213444\n",
      "\tspeed: 0.0507s/iter; left time: 703.5361s\n",
      "\titers: 700, epoch: 5 | loss: 0.0211287\n",
      "\tspeed: 0.0507s/iter; left time: 698.4840s\n",
      "\titers: 800, epoch: 5 | loss: 0.0154284\n",
      "\tspeed: 0.0507s/iter; left time: 693.4432s\n",
      "\titers: 900, epoch: 5 | loss: 0.0167327\n",
      "\tspeed: 0.0507s/iter; left time: 688.3171s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:46.09s\n",
      "Steps: 904 | Train Loss: 0.0181818 Vali Loss: 0.0360216 Test Loss: 0.0486139\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.0162390\n",
      "\tspeed: 0.1107s/iter; left time: 1490.5268s\n",
      "\titers: 200, epoch: 6 | loss: 0.0149850\n",
      "\tspeed: 0.0508s/iter; left time: 678.2410s\n",
      "\titers: 300, epoch: 6 | loss: 0.0161773\n",
      "\tspeed: 0.0507s/iter; left time: 672.6271s\n",
      "\titers: 400, epoch: 6 | loss: 0.0161455\n",
      "\tspeed: 0.0508s/iter; left time: 667.9827s\n",
      "\titers: 500, epoch: 6 | loss: 0.0159038\n",
      "\tspeed: 0.0505s/iter; left time: 659.7605s\n",
      "\titers: 600, epoch: 6 | loss: 0.0155437\n",
      "\tspeed: 0.0508s/iter; left time: 658.1906s\n",
      "\titers: 700, epoch: 6 | loss: 0.0135126\n",
      "\tspeed: 0.0508s/iter; left time: 652.7216s\n",
      "\titers: 800, epoch: 6 | loss: 0.0161070\n",
      "\tspeed: 0.0506s/iter; left time: 645.8676s\n",
      "\titers: 900, epoch: 6 | loss: 0.0155187\n",
      "\tspeed: 0.0506s/iter; left time: 640.1595s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:46.06s\n",
      "Steps: 904 | Train Loss: 0.0158869 Vali Loss: 0.0358567 Test Loss: 0.0487426\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.0146865\n",
      "\tspeed: 0.1106s/iter; left time: 1388.8532s\n",
      "\titers: 200, epoch: 7 | loss: 0.0132540\n",
      "\tspeed: 0.0508s/iter; left time: 632.2943s\n",
      "\titers: 300, epoch: 7 | loss: 0.0136056\n",
      "\tspeed: 0.0507s/iter; left time: 626.5165s\n",
      "\titers: 400, epoch: 7 | loss: 0.0151962\n",
      "\tspeed: 0.0507s/iter; left time: 620.8245s\n",
      "\titers: 500, epoch: 7 | loss: 0.0126808\n",
      "\tspeed: 0.0507s/iter; left time: 616.9596s\n",
      "\titers: 600, epoch: 7 | loss: 0.0143357\n",
      "\tspeed: 0.0504s/iter; left time: 607.9358s\n",
      "\titers: 700, epoch: 7 | loss: 0.0140345\n",
      "\tspeed: 0.0506s/iter; left time: 605.1632s\n",
      "\titers: 800, epoch: 7 | loss: 0.0124623\n",
      "\tspeed: 0.0505s/iter; left time: 599.3026s\n",
      "\titers: 900, epoch: 7 | loss: 0.0136355\n",
      "\tspeed: 0.0506s/iter; left time: 594.9575s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:46.02s\n",
      "Steps: 904 | Train Loss: 0.0139272 Vali Loss: 0.0363626 Test Loss: 0.0491312\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.0122056\n",
      "\tspeed: 0.1112s/iter; left time: 1296.3692s\n",
      "\titers: 200, epoch: 8 | loss: 0.0113194\n",
      "\tspeed: 0.0507s/iter; left time: 586.2115s\n",
      "\titers: 300, epoch: 8 | loss: 0.0139114\n",
      "\tspeed: 0.0508s/iter; left time: 581.5550s\n",
      "\titers: 400, epoch: 8 | loss: 0.0119940\n",
      "\tspeed: 0.0507s/iter; left time: 575.2979s\n",
      "\titers: 500, epoch: 8 | loss: 0.0131242\n",
      "\tspeed: 0.0507s/iter; left time: 570.9575s\n",
      "\titers: 600, epoch: 8 | loss: 0.0141879\n",
      "\tspeed: 0.0508s/iter; left time: 566.0321s\n",
      "\titers: 700, epoch: 8 | loss: 0.0130758\n",
      "\tspeed: 0.0508s/iter; left time: 561.0712s\n",
      "\titers: 800, epoch: 8 | loss: 0.0103310\n",
      "\tspeed: 0.0508s/iter; left time: 556.0683s\n",
      "\titers: 900, epoch: 8 | loss: 0.0104962\n",
      "\tspeed: 0.0508s/iter; left time: 551.1141s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:46.14s\n",
      "Steps: 904 | Train Loss: 0.0124156 Vali Loss: 0.0373577 Test Loss: 0.0492265\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_96_96_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl96_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6145\n",
      "Scaled mse:0.04156292974948883, rmse:0.20386987924575806, mae:0.14678193628787994, rse:0.7219445109367371\n",
      "Original data scale mse:38921600.0, rmse:6238.7177734375, mae:4206.650390625, rse:0.31069037318229675\n",
      "\n",
      "=== Starting experiments for pred_len: 168 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_96_168_scaler_choice_for_DE', model='Informer', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='minmax', if_relu=True, channel_mixing=0, seq_len=96, label_len=48, pred_len=168, inverse=True, loss_fnc='MSE', fc_dropout=0.05, head_dropout=0.0, patch_len=16, stride=8, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=512, n_heads=8, e_layers=2, d_layers=1, d_ff=2048, moving_avg=25, factor=5, distil=True, dropout=0.1, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_96_168_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl168_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 28873\n",
      "val 6073\n",
      "test 6073\n",
      "\titers: 100, epoch: 1 | loss: 0.0861944\n",
      "\tspeed: 0.0793s/iter; left time: 1423.5149s\n",
      "\titers: 200, epoch: 1 | loss: 0.0816551\n",
      "\tspeed: 0.0487s/iter; left time: 869.5722s\n",
      "\titers: 300, epoch: 1 | loss: 0.0688623\n",
      "\tspeed: 0.0492s/iter; left time: 872.9741s\n",
      "\titers: 400, epoch: 1 | loss: 0.0678409\n",
      "\tspeed: 0.0500s/iter; left time: 882.0423s\n",
      "\titers: 500, epoch: 1 | loss: 0.0633130\n",
      "\tspeed: 0.0497s/iter; left time: 871.8937s\n",
      "\titers: 600, epoch: 1 | loss: 0.0644796\n",
      "\tspeed: 0.0482s/iter; left time: 841.4349s\n",
      "\titers: 700, epoch: 1 | loss: 0.0581975\n",
      "\tspeed: 0.0474s/iter; left time: 821.2102s\n",
      "\titers: 800, epoch: 1 | loss: 0.0562265\n",
      "\tspeed: 0.0509s/iter; left time: 877.2640s\n",
      "\titers: 900, epoch: 1 | loss: 0.0581804\n",
      "\tspeed: 0.0503s/iter; left time: 861.4141s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:45.20s\n",
      "Steps: 902 | Train Loss: 0.0709426 Vali Loss: 0.0584743 Test Loss: 0.0763278\n",
      "Validation loss decreased (inf --> 0.058474).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.0473072\n",
      "\tspeed: 0.1224s/iter; left time: 2085.1556s\n",
      "\titers: 200, epoch: 2 | loss: 0.0385538\n",
      "\tspeed: 0.0496s/iter; left time: 839.8896s\n",
      "\titers: 300, epoch: 2 | loss: 0.0370987\n",
      "\tspeed: 0.0499s/iter; left time: 839.6438s\n",
      "\titers: 400, epoch: 2 | loss: 0.0379711\n",
      "\tspeed: 0.0501s/iter; left time: 838.4181s\n",
      "\titers: 500, epoch: 2 | loss: 0.0324598\n",
      "\tspeed: 0.0481s/iter; left time: 800.4128s\n",
      "\titers: 600, epoch: 2 | loss: 0.0304995\n",
      "\tspeed: 0.0475s/iter; left time: 785.4522s\n",
      "\titers: 700, epoch: 2 | loss: 0.0285279\n",
      "\tspeed: 0.0498s/iter; left time: 818.2353s\n",
      "\titers: 800, epoch: 2 | loss: 0.0329162\n",
      "\tspeed: 0.0515s/iter; left time: 841.9227s\n",
      "\titers: 900, epoch: 2 | loss: 0.0283897\n",
      "\tspeed: 0.0480s/iter; left time: 780.0359s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:44.75s\n",
      "Steps: 902 | Train Loss: 0.0370804 Vali Loss: 0.0431616 Test Loss: 0.0524194\n",
      "Validation loss decreased (0.058474 --> 0.043162).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.0306641\n",
      "\tspeed: 0.1256s/iter; left time: 2027.1892s\n",
      "\titers: 200, epoch: 3 | loss: 0.0285753\n",
      "\tspeed: 0.0508s/iter; left time: 814.6521s\n",
      "\titers: 300, epoch: 3 | loss: 0.0272127\n",
      "\tspeed: 0.0467s/iter; left time: 744.6532s\n",
      "\titers: 400, epoch: 3 | loss: 0.0306366\n",
      "\tspeed: 0.0465s/iter; left time: 735.7337s\n",
      "\titers: 500, epoch: 3 | loss: 0.0275743\n",
      "\tspeed: 0.0479s/iter; left time: 753.1672s\n",
      "\titers: 600, epoch: 3 | loss: 0.0266086\n",
      "\tspeed: 0.0469s/iter; left time: 733.1439s\n",
      "\titers: 700, epoch: 3 | loss: 0.0248030\n",
      "\tspeed: 0.0468s/iter; left time: 727.5312s\n",
      "\titers: 800, epoch: 3 | loss: 0.0238694\n",
      "\tspeed: 0.0468s/iter; left time: 722.7657s\n",
      "\titers: 900, epoch: 3 | loss: 0.0242699\n",
      "\tspeed: 0.0479s/iter; left time: 735.1091s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:43.37s\n",
      "Steps: 902 | Train Loss: 0.0274520 Vali Loss: 0.0379027 Test Loss: 0.0459632\n",
      "Validation loss decreased (0.043162 --> 0.037903).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.0232967\n",
      "\tspeed: 0.1228s/iter; left time: 1870.7689s\n",
      "\titers: 200, epoch: 4 | loss: 0.0210634\n",
      "\tspeed: 0.0475s/iter; left time: 718.3268s\n",
      "\titers: 300, epoch: 4 | loss: 0.0214336\n",
      "\tspeed: 0.0481s/iter; left time: 723.5955s\n",
      "\titers: 400, epoch: 4 | loss: 0.0200201\n",
      "\tspeed: 0.0481s/iter; left time: 718.0637s\n",
      "\titers: 500, epoch: 4 | loss: 0.0191740\n",
      "\tspeed: 0.0485s/iter; left time: 719.5579s\n",
      "\titers: 600, epoch: 4 | loss: 0.0223153\n",
      "\tspeed: 0.0483s/iter; left time: 711.4590s\n",
      "\titers: 700, epoch: 4 | loss: 0.0197920\n",
      "\tspeed: 0.0476s/iter; left time: 697.3343s\n",
      "\titers: 800, epoch: 4 | loss: 0.0204711\n",
      "\tspeed: 0.0481s/iter; left time: 699.4091s\n",
      "\titers: 900, epoch: 4 | loss: 0.0179707\n",
      "\tspeed: 0.0487s/iter; left time: 702.6970s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:43.71s\n",
      "Steps: 902 | Train Loss: 0.0220058 Vali Loss: 0.0428637 Test Loss: 0.0473708\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.0184987\n",
      "\tspeed: 0.1236s/iter; left time: 1771.0205s\n",
      "\titers: 200, epoch: 5 | loss: 0.0188701\n",
      "\tspeed: 0.0519s/iter; left time: 738.2737s\n",
      "\titers: 300, epoch: 5 | loss: 0.0192380\n",
      "\tspeed: 0.0511s/iter; left time: 722.3904s\n",
      "\titers: 400, epoch: 5 | loss: 0.0198872\n",
      "\tspeed: 0.0519s/iter; left time: 727.9460s\n",
      "\titers: 500, epoch: 5 | loss: 0.0168683\n",
      "\tspeed: 0.0515s/iter; left time: 717.1147s\n",
      "\titers: 600, epoch: 5 | loss: 0.0176144\n",
      "\tspeed: 0.0509s/iter; left time: 704.1869s\n",
      "\titers: 700, epoch: 5 | loss: 0.0178311\n",
      "\tspeed: 0.0517s/iter; left time: 710.5328s\n",
      "\titers: 800, epoch: 5 | loss: 0.0204910\n",
      "\tspeed: 0.0511s/iter; left time: 696.2440s\n",
      "\titers: 900, epoch: 5 | loss: 0.0178536\n",
      "\tspeed: 0.0498s/iter; left time: 673.9684s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:46.60s\n",
      "Steps: 902 | Train Loss: 0.0191415 Vali Loss: 0.0384142 Test Loss: 0.0487048\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.0186351\n",
      "\tspeed: 0.1207s/iter; left time: 1621.4348s\n",
      "\titers: 200, epoch: 6 | loss: 0.0176800\n",
      "\tspeed: 0.0492s/iter; left time: 656.0734s\n",
      "\titers: 300, epoch: 6 | loss: 0.0170789\n",
      "\tspeed: 0.0486s/iter; left time: 643.2095s\n",
      "\titers: 400, epoch: 6 | loss: 0.0206994\n",
      "\tspeed: 0.0478s/iter; left time: 628.1321s\n",
      "\titers: 500, epoch: 6 | loss: 0.0190296\n",
      "\tspeed: 0.0501s/iter; left time: 653.1714s\n",
      "\titers: 600, epoch: 6 | loss: 0.0164793\n",
      "\tspeed: 0.0477s/iter; left time: 616.3863s\n",
      "\titers: 700, epoch: 6 | loss: 0.0139499\n",
      "\tspeed: 0.0463s/iter; left time: 594.6939s\n",
      "\titers: 800, epoch: 6 | loss: 0.0153553\n",
      "\tspeed: 0.0458s/iter; left time: 582.6827s\n",
      "\titers: 900, epoch: 6 | loss: 0.0163441\n",
      "\tspeed: 0.0459s/iter; left time: 579.9547s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:43.50s\n",
      "Steps: 902 | Train Loss: 0.0168664 Vali Loss: 0.0399395 Test Loss: 0.0500806\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.0141258\n",
      "\tspeed: 0.1184s/iter; left time: 1482.8575s\n",
      "\titers: 200, epoch: 7 | loss: 0.0150832\n",
      "\tspeed: 0.0458s/iter; left time: 569.6457s\n",
      "\titers: 300, epoch: 7 | loss: 0.0154001\n",
      "\tspeed: 0.0464s/iter; left time: 571.7985s\n",
      "\titers: 400, epoch: 7 | loss: 0.0129656\n",
      "\tspeed: 0.0461s/iter; left time: 563.4069s\n",
      "\titers: 500, epoch: 7 | loss: 0.0154558\n",
      "\tspeed: 0.0478s/iter; left time: 579.6743s\n",
      "\titers: 600, epoch: 7 | loss: 0.0135689\n",
      "\tspeed: 0.0488s/iter; left time: 587.2020s\n",
      "\titers: 700, epoch: 7 | loss: 0.0144427\n",
      "\tspeed: 0.0480s/iter; left time: 572.6353s\n",
      "\titers: 800, epoch: 7 | loss: 0.0146086\n",
      "\tspeed: 0.0494s/iter; left time: 583.9122s\n",
      "\titers: 900, epoch: 7 | loss: 0.0156404\n",
      "\tspeed: 0.0514s/iter; left time: 602.2894s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:43.48s\n",
      "Steps: 902 | Train Loss: 0.0148521 Vali Loss: 0.0405050 Test Loss: 0.0519974\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.0135467\n",
      "\tspeed: 0.1206s/iter; left time: 1402.1046s\n",
      "\titers: 200, epoch: 8 | loss: 0.0147633\n",
      "\tspeed: 0.0494s/iter; left time: 569.4895s\n",
      "\titers: 300, epoch: 8 | loss: 0.0138894\n",
      "\tspeed: 0.0499s/iter; left time: 569.7456s\n",
      "\titers: 400, epoch: 8 | loss: 0.0129612\n",
      "\tspeed: 0.0484s/iter; left time: 548.2164s\n",
      "\titers: 500, epoch: 8 | loss: 0.0127115\n",
      "\tspeed: 0.0515s/iter; left time: 577.7243s\n",
      "\titers: 600, epoch: 8 | loss: 0.0125061\n",
      "\tspeed: 0.0517s/iter; left time: 575.5734s\n",
      "\titers: 700, epoch: 8 | loss: 0.0132125\n",
      "\tspeed: 0.0515s/iter; left time: 567.3860s\n",
      "\titers: 800, epoch: 8 | loss: 0.0115913\n",
      "\tspeed: 0.0515s/iter; left time: 562.6933s\n",
      "\titers: 900, epoch: 8 | loss: 0.0115792\n",
      "\tspeed: 0.0488s/iter; left time: 528.6753s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:45.61s\n",
      "Steps: 902 | Train Loss: 0.0132287 Vali Loss: 0.0421177 Test Loss: 0.0539093\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_96_168_scaler_choice_for_DE_Informer_custom_ftM_sl96_ll48_pl168_dm512_nh8_el2_dl1_df2048_fc5_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6073\n",
      "Scaled mse:0.04590200260281563, rmse:0.21424752473831177, mae:0.15394257009029388, rse:0.7590144276618958\n",
      "Original data scale mse:42592736.0, rmse:6526.31103515625, mae:4415.55029296875, rse:0.32517215609550476\n"
     ]
    }
   ],
   "source": [
    "# Dynamic variables\n",
    "pred_lens = [\"24\", \"96\", \"168\"]\n",
    "seq_len = \"96\"\n",
    "lr = \"0.0001\"\n",
    "itr = 1  \n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = cuda_device\n",
    "\n",
    "# Lists to store the results\n",
    "informer_results_scaled, informer_results_unscaled = [], []\n",
    "\n",
    "# Log file\n",
    "with open(log_file_path, \"w\") as log_file:\n",
    "    for loss in losses:\n",
    "        statement_1 = f\"\\n=== Starting experiments for loss function: {loss} ===\\n\"\n",
    "        log_file.write(statement_1)\n",
    "        print(statement_1)  # Print to .ipynb output cell\n",
    "\n",
    "        for pred_len in pred_lens:\n",
    "            statement_2 = f\"\\n=== Starting experiments for pred_len: {pred_len} ===\\n\"\n",
    "            log_file.write(statement_2)\n",
    "            print(statement_2) \n",
    "            model_id = f\"{country}_{seq_len}_{pred_len}_scaler_choice_for_{country}\"\n",
    "\n",
    "            # Arguments for the command\n",
    "            command = f\"\"\"\n",
    "            python {script_path} \\\n",
    "              --random_seed 2021 \\\n",
    "              --is_training 1 \\\n",
    "              --root_path \"{data_path}\" \\\n",
    "              --data_path \"{dataset}\" \\\n",
    "              --model_id {model_id} \\\n",
    "              --model \"{model}\" \\\n",
    "              --data \"custom\" \\\n",
    "              --features M \\\n",
    "              --seq_len {seq_len} \\\n",
    "              --label_len 48 \\\n",
    "              --pred_len {pred_len} \\\n",
    "              --e_layers 2 \\\n",
    "              --d_layers 1 \\\n",
    "              --factor 5 \\\n",
    "              --enc_in 5 \\\n",
    "              --dec_in 5 \\\n",
    "              --c_out 5 \\\n",
    "              --des 'Exp' \\\n",
    "              --train_epochs 20 \\\n",
    "              --patience 5 \\\n",
    "              --dropout 0.1 \\\n",
    "              --overlapping_windows \\\n",
    "              --inverse \\\n",
    "              --scaler_type minmax \\\n",
    "              --if_relu \\\n",
    "              --loss_fnc \"{loss}\" \\\n",
    "              --itr {itr} --batch_size 32 --learning_rate \"{lr}\"\n",
    "            \"\"\"\n",
    "\n",
    "            # Run the command and capture the output\n",
    "            process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)\n",
    "\n",
    "            # Capture the output in real-time\n",
    "            output = []\n",
    "            for line in process.stdout:\n",
    "                output.append(line)\n",
    "                print(line, end='')  # Print in the .ipynb cell\n",
    "                log_file.write(line)  # Write to the log file\n",
    "\n",
    "            # Wait for the process to complete\n",
    "            process.wait()\n",
    "            \n",
    "            # Delete the checkpoints folder and all its contents\n",
    "            shutil.rmtree('./checkpoints' )\n",
    "\n",
    "            # Extract metrics for each iteration\n",
    "            iteration_metrics = extract_metrics_from_output(output, itr)\n",
    "            iteration_metrics_unscaled = extract_metrics_from_output(output, itr, if_scaled=False)\n",
    "\n",
    "            # Log the extracted metrics and save them\n",
    "            for iteration, (scaled_metrics, unscaled_metrics) in enumerate(zip(iteration_metrics, iteration_metrics_unscaled), start=1):\n",
    "                log_file.write(f\"\\nExtracted Metrics for {country}, pred_len={pred_len}, iteration={iteration}:\\n\")\n",
    "                log_file.write(f\"Scaled Metrics - MSE: {scaled_metrics[0]}, RMSE: {scaled_metrics[1]}, MAE: {scaled_metrics[2]}, RSE: {scaled_metrics[3]}\\n\")\n",
    "                log_file.write(f\"Unscaled Metrics - MSE: {unscaled_metrics[0]}, RMSE: {unscaled_metrics[1]}, MAE: {unscaled_metrics[2]}, RSE: {unscaled_metrics[3]}\\n\")\n",
    "\n",
    "                # Append the results to the informer_results lists\n",
    "                metrics_data = [(informer_results_scaled, scaled_metrics), (informer_results_unscaled, unscaled_metrics)]\n",
    "\n",
    "                for result_list, metrics in metrics_data:\n",
    "                    result_list.append({\n",
    "                        'Loss_function': loss,\n",
    "                        'Pred_len': pred_len,\n",
    "                        'Iteration': iteration,\n",
    "                        'MSE': metrics[0],\n",
    "                        'RMSE': metrics[1],\n",
    "                        'MAE': metrics[2],\n",
    "                        'RSE': metrics[3]\n",
    "                    })"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>MSE</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>RSE</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Loss_function</th>\n",
       "      <th>Iteration</th>\n",
       "      <th>Pred_len</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">MSE</th>\n",
       "      <th rowspan=\"3\" valign=\"top\">1</th>\n",
       "      <th>24</th>\n",
       "      <td>0.0255</td>\n",
       "      <td>0.1597</td>\n",
       "      <td>0.1063</td>\n",
       "      <td>0.5639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>0.0416</td>\n",
       "      <td>0.2039</td>\n",
       "      <td>0.1468</td>\n",
       "      <td>0.7219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>0.0459</td>\n",
       "      <td>0.2142</td>\n",
       "      <td>0.1539</td>\n",
       "      <td>0.7590</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     MSE    RMSE     MAE     RSE\n",
       "Loss_function Iteration Pred_len                                \n",
       "MSE           1         24        0.0255  0.1597  0.1063  0.5639\n",
       "                        96        0.0416  0.2039  0.1468  0.7219\n",
       "                        168       0.0459  0.2142  0.1539  0.7590"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path_dir = './results/scaler_choice'\n",
    "csv_name_scaled = 'informer_scaler_results_scaled_minmax_default.csv'\n",
    "csv_name_unscaled = 'informer_scaler_results_unscaled_minmax_default.csv'\n",
    "\n",
    "# Convert the results into a DataFrame and save as CSV\n",
    "informer_df_scaled = convert_results_into_df(informer_results_scaled, path_dir, csv_name_scaled)\n",
    "informer_df_unscaled = convert_results_into_df(informer_results_unscaled, path_dir, csv_name_unscaled)\n",
    "informer_df_scaled.round(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>MSE</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>RSE</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Loss_function</th>\n",
       "      <th>Iteration</th>\n",
       "      <th>Pred_len</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">MSE</th>\n",
       "      <th rowspan=\"3\" valign=\"top\">1</th>\n",
       "      <th>24</th>\n",
       "      <td>21864558.0</td>\n",
       "      <td>4675.9551</td>\n",
       "      <td>2984.4697</td>\n",
       "      <td>0.2325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>38921600.0</td>\n",
       "      <td>6238.7178</td>\n",
       "      <td>4206.6504</td>\n",
       "      <td>0.3107</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>42592736.0</td>\n",
       "      <td>6526.3110</td>\n",
       "      <td>4415.5503</td>\n",
       "      <td>0.3252</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         MSE       RMSE        MAE     RSE\n",
       "Loss_function Iteration Pred_len                                          \n",
       "MSE           1         24        21864558.0  4675.9551  2984.4697  0.2325\n",
       "                        96        38921600.0  6238.7178  4206.6504  0.3107\n",
       "                        168       42592736.0  6526.3110  4415.5503  0.3252"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "informer_df_unscaled.round(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>MSE</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>RSE</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pred_len</th>\n",
       "      <th>Loss_function</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">24</th>\n",
       "      <th>MAE</th>\n",
       "      <td>0.0257</td>\n",
       "      <td>0.1603</td>\n",
       "      <td>0.1028</td>\n",
       "      <td>0.5661</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSE</th>\n",
       "      <td>0.0249</td>\n",
       "      <td>0.1578</td>\n",
       "      <td>0.1062</td>\n",
       "      <td>0.5572</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">96</th>\n",
       "      <th>MAE</th>\n",
       "      <td>0.0427</td>\n",
       "      <td>0.2066</td>\n",
       "      <td>0.1433</td>\n",
       "      <td>0.7315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSE</th>\n",
       "      <td>0.0410</td>\n",
       "      <td>0.2024</td>\n",
       "      <td>0.1447</td>\n",
       "      <td>0.7169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">168</th>\n",
       "      <th>MAE</th>\n",
       "      <td>0.0503</td>\n",
       "      <td>0.2243</td>\n",
       "      <td>0.1545</td>\n",
       "      <td>0.7947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSE</th>\n",
       "      <td>0.0487</td>\n",
       "      <td>0.2207</td>\n",
       "      <td>0.1547</td>\n",
       "      <td>0.7818</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           MSE    RMSE     MAE     RSE\n",
       "Pred_len Loss_function                                \n",
       "24       MAE            0.0257  0.1603  0.1028  0.5661\n",
       "         MSE            0.0249  0.1578  0.1062  0.5572\n",
       "96       MAE            0.0427  0.2066  0.1433  0.7315\n",
       "         MSE            0.0410  0.2024  0.1447  0.7169\n",
       "168      MAE            0.0503  0.2243  0.1545  0.7947\n",
       "         MSE            0.0487  0.2207  0.1547  0.7818"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Average the iterations\n",
    "informer_scaled = pd.read_csv(os.path.join(path_dir, csv_name_scaled))\n",
    "informer_unscaled = pd.read_csv(os.path.join(path_dir, csv_name_unscaled))\n",
    "\n",
    "inf_res_scaled = informer_scaled.groupby(['Pred_len', 'Loss_function']).mean().drop('Iteration', axis=1)\n",
    "inf_res_unscaled = informer_unscaled.groupby(['Pred_len', 'Loss_function']).mean().sort_index().drop('Iteration', axis=1)\n",
    "inf_res_scaled.round(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>MSE</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>RSE</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pred_len</th>\n",
       "      <th>Loss_function</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">24</th>\n",
       "      <th>MAE</th>\n",
       "      <td>22094459.0</td>\n",
       "      <td>4700.1877</td>\n",
       "      <td>2883.2867</td>\n",
       "      <td>0.2337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSE</th>\n",
       "      <td>21203669.0</td>\n",
       "      <td>4603.6091</td>\n",
       "      <td>2981.2959</td>\n",
       "      <td>0.2289</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">96</th>\n",
       "      <th>MAE</th>\n",
       "      <td>39982774.0</td>\n",
       "      <td>6322.3682</td>\n",
       "      <td>4100.9425</td>\n",
       "      <td>0.3149</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSE</th>\n",
       "      <td>37952808.0</td>\n",
       "      <td>6160.5803</td>\n",
       "      <td>4132.8708</td>\n",
       "      <td>0.3068</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">168</th>\n",
       "      <th>MAE</th>\n",
       "      <td>47602690.0</td>\n",
       "      <td>6896.9163</td>\n",
       "      <td>4429.5474</td>\n",
       "      <td>0.3436</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSE</th>\n",
       "      <td>46652354.0</td>\n",
       "      <td>6830.2322</td>\n",
       "      <td>4464.8701</td>\n",
       "      <td>0.3403</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                               MSE       RMSE        MAE     RSE\n",
       "Pred_len Loss_function                                          \n",
       "24       MAE            22094459.0  4700.1877  2883.2867  0.2337\n",
       "         MSE            21203669.0  4603.6091  2981.2959  0.2289\n",
       "96       MAE            39982774.0  6322.3682  4100.9425  0.3149\n",
       "         MSE            37952808.0  6160.5803  4132.8708  0.3068\n",
       "168      MAE            47602690.0  6896.9163  4429.5474  0.3436\n",
       "         MSE            46652354.0  6830.2322  4464.8701  0.3403"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# ALL 0.00001 - from 96 - BAD\n",
    "# # 24 lr=0.000001; 96, 168 lr=0.00001 - BAD\n",
    "\n",
    "inf_res_unscaled.round(4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. MinMax Scaler PatchTST"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "log_dir = f\"logs/scaler_choice/min_max\"\n",
    "if not os.path.exists(log_dir):\n",
    "    os.makedirs(log_dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== Starting experiments for loss function: MSE ===\n",
      "\n",
      "\n",
      "=== Starting experiments for pred_len: 24 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_336_24_scaler_choice_for_DE', model='PatchTST', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='minmax', if_relu=True, channel_mixing=0, seq_len=336, label_len=48, pred_len=24, inverse=True, loss_fnc='MSE', fc_dropout=0.2, head_dropout=0.0, patch_len=32, stride=16, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=128, n_heads=16, e_layers=3, d_layers=1, d_ff=256, moving_avg=25, factor=1, distil=True, dropout=0.2, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_336_24_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl24_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 28777\n",
      "val 6217\n",
      "test 6217\n",
      "\titers: 100, epoch: 1 | loss: 0.0473328\n",
      "\tspeed: 0.0529s/iter; left time: 946.5807s\n",
      "\titers: 200, epoch: 1 | loss: 0.0381640\n",
      "\tspeed: 0.0285s/iter; left time: 507.2045s\n",
      "\titers: 300, epoch: 1 | loss: 0.0281555\n",
      "\tspeed: 0.0287s/iter; left time: 507.0725s\n",
      "\titers: 400, epoch: 1 | loss: 0.0290323\n",
      "\tspeed: 0.0273s/iter; left time: 479.5858s\n",
      "\titers: 500, epoch: 1 | loss: 0.0260658\n",
      "\tspeed: 0.0260s/iter; left time: 454.2162s\n",
      "\titers: 600, epoch: 1 | loss: 0.0224518\n",
      "\tspeed: 0.0260s/iter; left time: 452.6706s\n",
      "\titers: 700, epoch: 1 | loss: 0.0212832\n",
      "\tspeed: 0.0275s/iter; left time: 475.3529s\n",
      "\titers: 800, epoch: 1 | loss: 0.0216284\n",
      "\tspeed: 0.0286s/iter; left time: 491.7646s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:25.03s\n",
      "Steps: 899 | Train Loss: 0.0292834 Vali Loss: 0.0252810 Test Loss: 0.0276644\n",
      "Validation loss decreased (inf --> 0.025281).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.0204496\n",
      "\tspeed: 0.0833s/iter; left time: 1414.7993s\n",
      "\titers: 200, epoch: 2 | loss: 0.0171114\n",
      "\tspeed: 0.0274s/iter; left time: 463.0959s\n",
      "\titers: 300, epoch: 2 | loss: 0.0169130\n",
      "\tspeed: 0.0267s/iter; left time: 447.3440s\n",
      "\titers: 400, epoch: 2 | loss: 0.0124452\n",
      "\tspeed: 0.0257s/iter; left time: 429.2668s\n",
      "\titers: 500, epoch: 2 | loss: 0.0146587\n",
      "\tspeed: 0.0264s/iter; left time: 437.4406s\n",
      "\titers: 600, epoch: 2 | loss: 0.0100212\n",
      "\tspeed: 0.0250s/iter; left time: 412.7649s\n",
      "\titers: 700, epoch: 2 | loss: 0.0099339\n",
      "\tspeed: 0.0261s/iter; left time: 427.7330s\n",
      "\titers: 800, epoch: 2 | loss: 0.0125124\n",
      "\tspeed: 0.0257s/iter; left time: 418.7484s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:24.05s\n",
      "Steps: 899 | Train Loss: 0.0154529 Vali Loss: 0.0197008 Test Loss: 0.0215191\n",
      "Validation loss decreased (0.025281 --> 0.019701).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.0113480\n",
      "\tspeed: 0.0811s/iter; left time: 1305.0193s\n",
      "\titers: 200, epoch: 3 | loss: 0.0180866\n",
      "\tspeed: 0.0274s/iter; left time: 437.4723s\n",
      "\titers: 300, epoch: 3 | loss: 0.0138883\n",
      "\tspeed: 0.0251s/iter; left time: 398.2313s\n",
      "\titers: 400, epoch: 3 | loss: 0.0121179\n",
      "\tspeed: 0.0254s/iter; left time: 401.5858s\n",
      "\titers: 500, epoch: 3 | loss: 0.0143962\n",
      "\tspeed: 0.0255s/iter; left time: 400.5763s\n",
      "\titers: 600, epoch: 3 | loss: 0.0132282\n",
      "\tspeed: 0.0252s/iter; left time: 393.2552s\n",
      "\titers: 700, epoch: 3 | loss: 0.0134729\n",
      "\tspeed: 0.0281s/iter; left time: 435.4498s\n",
      "\titers: 800, epoch: 3 | loss: 0.0145609\n",
      "\tspeed: 0.0283s/iter; left time: 435.5865s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:23.84s\n",
      "Steps: 899 | Train Loss: 0.0139160 Vali Loss: 0.0192382 Test Loss: 0.0209741\n",
      "Validation loss decreased (0.019701 --> 0.019238).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.0159280\n",
      "\tspeed: 0.0781s/iter; left time: 1186.1689s\n",
      "\titers: 200, epoch: 4 | loss: 0.0130239\n",
      "\tspeed: 0.0269s/iter; left time: 406.0593s\n",
      "\titers: 300, epoch: 4 | loss: 0.0162055\n",
      "\tspeed: 0.0267s/iter; left time: 400.6703s\n",
      "\titers: 400, epoch: 4 | loss: 0.0123946\n",
      "\tspeed: 0.0257s/iter; left time: 382.1770s\n",
      "\titers: 500, epoch: 4 | loss: 0.0110941\n",
      "\tspeed: 0.0233s/iter; left time: 345.0520s\n",
      "\titers: 600, epoch: 4 | loss: 0.0114620\n",
      "\tspeed: 0.0229s/iter; left time: 335.6499s\n",
      "\titers: 700, epoch: 4 | loss: 0.0112462\n",
      "\tspeed: 0.0231s/iter; left time: 336.7550s\n",
      "\titers: 800, epoch: 4 | loss: 0.0116232\n",
      "\tspeed: 0.0233s/iter; left time: 337.8737s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:22.30s\n",
      "Steps: 899 | Train Loss: 0.0134562 Vali Loss: 0.0192192 Test Loss: 0.0210943\n",
      "Validation loss decreased (0.019238 --> 0.019219).  Saving model ...\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.0193589\n",
      "\tspeed: 0.0758s/iter; left time: 1082.4173s\n",
      "\titers: 200, epoch: 5 | loss: 0.0149708\n",
      "\tspeed: 0.0264s/iter; left time: 374.3920s\n",
      "\titers: 300, epoch: 5 | loss: 0.0108379\n",
      "\tspeed: 0.0251s/iter; left time: 353.0749s\n",
      "\titers: 400, epoch: 5 | loss: 0.0123415\n",
      "\tspeed: 0.0256s/iter; left time: 357.3830s\n",
      "\titers: 500, epoch: 5 | loss: 0.0160639\n",
      "\tspeed: 0.0247s/iter; left time: 342.2999s\n",
      "\titers: 600, epoch: 5 | loss: 0.0113621\n",
      "\tspeed: 0.0248s/iter; left time: 342.2857s\n",
      "\titers: 700, epoch: 5 | loss: 0.0159059\n",
      "\tspeed: 0.0251s/iter; left time: 343.0513s\n",
      "\titers: 800, epoch: 5 | loss: 0.0128038\n",
      "\tspeed: 0.0268s/iter; left time: 364.7191s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:23.38s\n",
      "Steps: 899 | Train Loss: 0.0130951 Vali Loss: 0.0191006 Test Loss: 0.0210062\n",
      "Validation loss decreased (0.019219 --> 0.019101).  Saving model ...\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.0140222\n",
      "\tspeed: 0.0802s/iter; left time: 1073.5079s\n",
      "\titers: 200, epoch: 6 | loss: 0.0131318\n",
      "\tspeed: 0.0271s/iter; left time: 359.4035s\n",
      "\titers: 300, epoch: 6 | loss: 0.0100519\n",
      "\tspeed: 0.0267s/iter; left time: 352.6060s\n",
      "\titers: 400, epoch: 6 | loss: 0.0133216\n",
      "\tspeed: 0.0231s/iter; left time: 301.8650s\n",
      "\titers: 500, epoch: 6 | loss: 0.0105777\n",
      "\tspeed: 0.0245s/iter; left time: 318.7624s\n",
      "\titers: 600, epoch: 6 | loss: 0.0102108\n",
      "\tspeed: 0.0272s/iter; left time: 351.0172s\n",
      "\titers: 700, epoch: 6 | loss: 0.0152710\n",
      "\tspeed: 0.0255s/iter; left time: 326.0758s\n",
      "\titers: 800, epoch: 6 | loss: 0.0125583\n",
      "\tspeed: 0.0252s/iter; left time: 319.4878s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:23.18s\n",
      "Steps: 899 | Train Loss: 0.0129245 Vali Loss: 0.0186772 Test Loss: 0.0206865\n",
      "Validation loss decreased (0.019101 --> 0.018677).  Saving model ...\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.0171648\n",
      "\tspeed: 0.0803s/iter; left time: 1002.4518s\n",
      "\titers: 200, epoch: 7 | loss: 0.0108009\n",
      "\tspeed: 0.0256s/iter; left time: 316.9500s\n",
      "\titers: 300, epoch: 7 | loss: 0.0195854\n",
      "\tspeed: 0.0258s/iter; left time: 317.0514s\n",
      "\titers: 400, epoch: 7 | loss: 0.0157668\n",
      "\tspeed: 0.0260s/iter; left time: 317.3439s\n",
      "\titers: 500, epoch: 7 | loss: 0.0137457\n",
      "\tspeed: 0.0259s/iter; left time: 312.6628s\n",
      "\titers: 600, epoch: 7 | loss: 0.0118021\n",
      "\tspeed: 0.0259s/iter; left time: 310.2787s\n",
      "\titers: 700, epoch: 7 | loss: 0.0101876\n",
      "\tspeed: 0.0264s/iter; left time: 313.7015s\n",
      "\titers: 800, epoch: 7 | loss: 0.0168224\n",
      "\tspeed: 0.0298s/iter; left time: 351.7481s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:24.01s\n",
      "Steps: 899 | Train Loss: 0.0126926 Vali Loss: 0.0187632 Test Loss: 0.0207133\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.0144183\n",
      "\tspeed: 0.0803s/iter; left time: 930.5142s\n",
      "\titers: 200, epoch: 8 | loss: 0.0122153\n",
      "\tspeed: 0.0273s/iter; left time: 313.7107s\n",
      "\titers: 300, epoch: 8 | loss: 0.0122645\n",
      "\tspeed: 0.0267s/iter; left time: 304.3336s\n",
      "\titers: 400, epoch: 8 | loss: 0.0146948\n",
      "\tspeed: 0.0267s/iter; left time: 300.9024s\n",
      "\titers: 500, epoch: 8 | loss: 0.0122112\n",
      "\tspeed: 0.0287s/iter; left time: 320.7388s\n",
      "\titers: 600, epoch: 8 | loss: 0.0120254\n",
      "\tspeed: 0.0297s/iter; left time: 329.3877s\n",
      "\titers: 700, epoch: 8 | loss: 0.0151615\n",
      "\tspeed: 0.0268s/iter; left time: 294.6135s\n",
      "\titers: 800, epoch: 8 | loss: 0.0103575\n",
      "\tspeed: 0.0274s/iter; left time: 298.5715s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:24.94s\n",
      "Steps: 899 | Train Loss: 0.0125645 Vali Loss: 0.0188330 Test Loss: 0.0208373\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 5.904900000000001e-05\n",
      "\titers: 100, epoch: 9 | loss: 0.0126411\n",
      "\tspeed: 0.0822s/iter; left time: 878.3923s\n",
      "\titers: 200, epoch: 9 | loss: 0.0168661\n",
      "\tspeed: 0.0266s/iter; left time: 282.0225s\n",
      "\titers: 300, epoch: 9 | loss: 0.0139193\n",
      "\tspeed: 0.0268s/iter; left time: 281.2254s\n",
      "\titers: 400, epoch: 9 | loss: 0.0133854\n",
      "\tspeed: 0.0265s/iter; left time: 275.0796s\n",
      "\titers: 500, epoch: 9 | loss: 0.0118623\n",
      "\tspeed: 0.0262s/iter; left time: 269.3423s\n",
      "\titers: 600, epoch: 9 | loss: 0.0094803\n",
      "\tspeed: 0.0260s/iter; left time: 265.3325s\n",
      "\titers: 700, epoch: 9 | loss: 0.0153165\n",
      "\tspeed: 0.0260s/iter; left time: 262.6180s\n",
      "\titers: 800, epoch: 9 | loss: 0.0125253\n",
      "\tspeed: 0.0257s/iter; left time: 256.5976s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 9\n",
      "Cost time: 00h:00m:24.11s\n",
      "Steps: 899 | Train Loss: 0.0124251 Vali Loss: 0.0189148 Test Loss: 0.0211062\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 5.3144100000000005e-05\n",
      "\titers: 100, epoch: 10 | loss: 0.0089099\n",
      "\tspeed: 0.0785s/iter; left time: 768.1605s\n",
      "\titers: 200, epoch: 10 | loss: 0.0097364\n",
      "\tspeed: 0.0262s/iter; left time: 254.0085s\n",
      "\titers: 300, epoch: 10 | loss: 0.0132611\n",
      "\tspeed: 0.0254s/iter; left time: 243.9232s\n",
      "\titers: 400, epoch: 10 | loss: 0.0113564\n",
      "\tspeed: 0.0261s/iter; left time: 247.9161s\n",
      "\titers: 500, epoch: 10 | loss: 0.0131991\n",
      "\tspeed: 0.0281s/iter; left time: 263.7433s\n",
      "\titers: 600, epoch: 10 | loss: 0.0127616\n",
      "\tspeed: 0.0248s/iter; left time: 230.5914s\n",
      "\titers: 700, epoch: 10 | loss: 0.0134150\n",
      "\tspeed: 0.0274s/iter; left time: 251.7594s\n",
      "\titers: 800, epoch: 10 | loss: 0.0114996\n",
      "\tspeed: 0.0292s/iter; left time: 265.4118s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 10\n",
      "Cost time: 00h:00m:24.24s\n",
      "Steps: 899 | Train Loss: 0.0123267 Vali Loss: 0.0186323 Test Loss: 0.0209611\n",
      "Validation loss decreased (0.018677 --> 0.018632).  Saving model ...\n",
      "Updating learning rate to 4.782969000000001e-05\n",
      "\titers: 100, epoch: 11 | loss: 0.0112268\n",
      "\tspeed: 0.0774s/iter; left time: 688.4077s\n",
      "\titers: 200, epoch: 11 | loss: 0.0148988\n",
      "\tspeed: 0.0281s/iter; left time: 247.4033s\n",
      "\titers: 300, epoch: 11 | loss: 0.0087550\n",
      "\tspeed: 0.0278s/iter; left time: 241.7586s\n",
      "\titers: 400, epoch: 11 | loss: 0.0099113\n",
      "\tspeed: 0.0295s/iter; left time: 253.5057s\n",
      "\titers: 500, epoch: 11 | loss: 0.0091522\n",
      "\tspeed: 0.0302s/iter; left time: 256.6972s\n",
      "\titers: 600, epoch: 11 | loss: 0.0134615\n",
      "\tspeed: 0.0296s/iter; left time: 248.4159s\n",
      "\titers: 700, epoch: 11 | loss: 0.0131008\n",
      "\tspeed: 0.0261s/iter; left time: 216.1875s\n",
      "\titers: 800, epoch: 11 | loss: 0.0116711\n",
      "\tspeed: 0.0284s/iter; left time: 232.3308s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 11\n",
      "Cost time: 00h:00m:25.07s\n",
      "Steps: 899 | Train Loss: 0.0122084 Vali Loss: 0.0185785 Test Loss: 0.0209226\n",
      "Validation loss decreased (0.018632 --> 0.018579).  Saving model ...\n",
      "Updating learning rate to 4.304672100000001e-05\n",
      "\titers: 100, epoch: 12 | loss: 0.0133459\n",
      "\tspeed: 0.0830s/iter; left time: 663.4895s\n",
      "\titers: 200, epoch: 12 | loss: 0.0130178\n",
      "\tspeed: 0.0263s/iter; left time: 207.6775s\n",
      "\titers: 300, epoch: 12 | loss: 0.0139347\n",
      "\tspeed: 0.0260s/iter; left time: 202.2060s\n",
      "\titers: 400, epoch: 12 | loss: 0.0128065\n",
      "\tspeed: 0.0261s/iter; left time: 200.5727s\n",
      "\titers: 500, epoch: 12 | loss: 0.0100467\n",
      "\tspeed: 0.0261s/iter; left time: 198.2955s\n",
      "\titers: 600, epoch: 12 | loss: 0.0114552\n",
      "\tspeed: 0.0261s/iter; left time: 195.8816s\n",
      "\titers: 700, epoch: 12 | loss: 0.0114155\n",
      "\tspeed: 0.0262s/iter; left time: 193.7605s\n",
      "\titers: 800, epoch: 12 | loss: 0.0128516\n",
      "\tspeed: 0.0261s/iter; left time: 190.6179s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 12\n",
      "Cost time: 00h:00m:23.73s\n",
      "Steps: 899 | Train Loss: 0.0121030 Vali Loss: 0.0187584 Test Loss: 0.0210183\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 3.874204890000001e-05\n",
      "\titers: 100, epoch: 13 | loss: 0.0097026\n",
      "\tspeed: 0.0797s/iter; left time: 565.0621s\n",
      "\titers: 200, epoch: 13 | loss: 0.0107862\n",
      "\tspeed: 0.0256s/iter; left time: 179.2801s\n",
      "\titers: 300, epoch: 13 | loss: 0.0115946\n",
      "\tspeed: 0.0274s/iter; left time: 189.1019s\n",
      "\titers: 400, epoch: 13 | loss: 0.0120168\n",
      "\tspeed: 0.0289s/iter; left time: 196.4114s\n",
      "\titers: 500, epoch: 13 | loss: 0.0089159\n",
      "\tspeed: 0.0259s/iter; left time: 173.0580s\n",
      "\titers: 600, epoch: 13 | loss: 0.0114414\n",
      "\tspeed: 0.0288s/iter; left time: 190.1343s\n",
      "\titers: 700, epoch: 13 | loss: 0.0096415\n",
      "\tspeed: 0.0299s/iter; left time: 193.9663s\n",
      "\titers: 800, epoch: 13 | loss: 0.0126724\n",
      "\tspeed: 0.0297s/iter; left time: 189.8782s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 13\n",
      "Cost time: 00h:00m:25.25s\n",
      "Steps: 899 | Train Loss: 0.0119989 Vali Loss: 0.0187238 Test Loss: 0.0209651\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 3.486784401000001e-05\n",
      "\titers: 100, epoch: 14 | loss: 0.0114613\n",
      "\tspeed: 0.0820s/iter; left time: 508.0268s\n",
      "\titers: 200, epoch: 14 | loss: 0.0136278\n",
      "\tspeed: 0.0259s/iter; left time: 157.8619s\n",
      "\titers: 300, epoch: 14 | loss: 0.0107881\n",
      "\tspeed: 0.0262s/iter; left time: 156.9468s\n",
      "\titers: 400, epoch: 14 | loss: 0.0120697\n",
      "\tspeed: 0.0264s/iter; left time: 155.5809s\n",
      "\titers: 500, epoch: 14 | loss: 0.0137393\n",
      "\tspeed: 0.0265s/iter; left time: 153.5013s\n",
      "\titers: 600, epoch: 14 | loss: 0.0128277\n",
      "\tspeed: 0.0264s/iter; left time: 150.4922s\n",
      "\titers: 700, epoch: 14 | loss: 0.0130124\n",
      "\tspeed: 0.0261s/iter; left time: 145.7566s\n",
      "\titers: 800, epoch: 14 | loss: 0.0131269\n",
      "\tspeed: 0.0267s/iter; left time: 146.9355s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 14\n",
      "Cost time: 00h:00m:23.88s\n",
      "Steps: 899 | Train Loss: 0.0119450 Vali Loss: 0.0186224 Test Loss: 0.0209009\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 3.138105960900001e-05\n",
      "\titers: 100, epoch: 15 | loss: 0.0115028\n",
      "\tspeed: 0.0829s/iter; left time: 438.7445s\n",
      "\titers: 200, epoch: 15 | loss: 0.0109354\n",
      "\tspeed: 0.0275s/iter; left time: 142.6942s\n",
      "\titers: 300, epoch: 15 | loss: 0.0138472\n",
      "\tspeed: 0.0261s/iter; left time: 132.9185s\n",
      "\titers: 400, epoch: 15 | loss: 0.0113534\n",
      "\tspeed: 0.0272s/iter; left time: 135.8764s\n",
      "\titers: 500, epoch: 15 | loss: 0.0118323\n",
      "\tspeed: 0.0267s/iter; left time: 130.8881s\n",
      "\titers: 600, epoch: 15 | loss: 0.0121059\n",
      "\tspeed: 0.0271s/iter; left time: 130.0918s\n",
      "\titers: 700, epoch: 15 | loss: 0.0127937\n",
      "\tspeed: 0.0269s/iter; left time: 126.4823s\n",
      "\titers: 800, epoch: 15 | loss: 0.0129534\n",
      "\tspeed: 0.0275s/iter; left time: 126.5332s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 15\n",
      "Cost time: 00h:00m:24.85s\n",
      "Steps: 899 | Train Loss: 0.0118477 Vali Loss: 0.0189005 Test Loss: 0.0212427\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 2.824295364810001e-05\n",
      "\titers: 100, epoch: 16 | loss: 0.0115596\n",
      "\tspeed: 0.0833s/iter; left time: 366.1943s\n",
      "\titers: 200, epoch: 16 | loss: 0.0116916\n",
      "\tspeed: 0.0262s/iter; left time: 112.5201s\n",
      "\titers: 300, epoch: 16 | loss: 0.0107063\n",
      "\tspeed: 0.0260s/iter; left time: 109.0024s\n",
      "\titers: 400, epoch: 16 | loss: 0.0118408\n",
      "\tspeed: 0.0257s/iter; left time: 105.2768s\n",
      "\titers: 500, epoch: 16 | loss: 0.0156944\n",
      "\tspeed: 0.0262s/iter; left time: 104.6811s\n",
      "\titers: 600, epoch: 16 | loss: 0.0109108\n",
      "\tspeed: 0.0271s/iter; left time: 105.7373s\n",
      "\titers: 700, epoch: 16 | loss: 0.0144436\n",
      "\tspeed: 0.0275s/iter; left time: 104.5131s\n",
      "\titers: 800, epoch: 16 | loss: 0.0107780\n",
      "\tspeed: 0.0291s/iter; left time: 107.7053s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 16\n",
      "Cost time: 00h:00m:24.40s\n",
      "Steps: 899 | Train Loss: 0.0117654 Vali Loss: 0.0188331 Test Loss: 0.0212083\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_336_24_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl24_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6217\n",
      "Scaled mse:0.020922627300024033, rmse:0.144646555185318, mae:0.09112025797367096, rse:0.5108232498168945\n",
      "Original data scale mse:16405032.0, rmse:4050.3125, mae:2453.593017578125, rse:0.20138971507549286\n",
      "\n",
      "=== Starting experiments for pred_len: 96 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_336_96_scaler_choice_for_DE', model='PatchTST', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='minmax', if_relu=True, channel_mixing=0, seq_len=336, label_len=48, pred_len=96, inverse=True, loss_fnc='MSE', fc_dropout=0.2, head_dropout=0.0, patch_len=32, stride=16, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=128, n_heads=16, e_layers=3, d_layers=1, d_ff=256, moving_avg=25, factor=1, distil=True, dropout=0.2, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_336_96_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl96_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 28705\n",
      "val 6145\n",
      "test 6145\n",
      "\titers: 100, epoch: 1 | loss: 0.0453740\n",
      "\tspeed: 0.0514s/iter; left time: 917.8242s\n",
      "\titers: 200, epoch: 1 | loss: 0.0397385\n",
      "\tspeed: 0.0250s/iter; left time: 443.9314s\n",
      "\titers: 300, epoch: 1 | loss: 0.0354433\n",
      "\tspeed: 0.0242s/iter; left time: 427.7342s\n",
      "\titers: 400, epoch: 1 | loss: 0.0307970\n",
      "\tspeed: 0.0254s/iter; left time: 445.6868s\n",
      "\titers: 500, epoch: 1 | loss: 0.0320921\n",
      "\tspeed: 0.0259s/iter; left time: 451.9446s\n",
      "\titers: 600, epoch: 1 | loss: 0.0379598\n",
      "\tspeed: 0.0257s/iter; left time: 445.7856s\n",
      "\titers: 700, epoch: 1 | loss: 0.0334983\n",
      "\tspeed: 0.0260s/iter; left time: 448.1844s\n",
      "\titers: 800, epoch: 1 | loss: 0.0350917\n",
      "\tspeed: 0.0261s/iter; left time: 447.7980s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:23.48s\n",
      "Steps: 897 | Train Loss: 0.0353902 Vali Loss: 0.0347500 Test Loss: 0.0400146\n",
      "Validation loss decreased (inf --> 0.034750).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.0227104\n",
      "\tspeed: 0.0853s/iter; left time: 1445.6256s\n",
      "\titers: 200, epoch: 2 | loss: 0.0235286\n",
      "\tspeed: 0.0267s/iter; left time: 449.2187s\n",
      "\titers: 300, epoch: 2 | loss: 0.0171380\n",
      "\tspeed: 0.0254s/iter; left time: 424.7758s\n",
      "\titers: 400, epoch: 2 | loss: 0.0217260\n",
      "\tspeed: 0.0256s/iter; left time: 425.6560s\n",
      "\titers: 500, epoch: 2 | loss: 0.0216705\n",
      "\tspeed: 0.0255s/iter; left time: 421.7453s\n",
      "\titers: 600, epoch: 2 | loss: 0.0226315\n",
      "\tspeed: 0.0254s/iter; left time: 418.3501s\n",
      "\titers: 700, epoch: 2 | loss: 0.0197355\n",
      "\tspeed: 0.0255s/iter; left time: 417.3579s\n",
      "\titers: 800, epoch: 2 | loss: 0.0305562\n",
      "\tspeed: 0.0254s/iter; left time: 411.9473s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:23.39s\n",
      "Steps: 897 | Train Loss: 0.0248613 Vali Loss: 0.0302856 Test Loss: 0.0361761\n",
      "Validation loss decreased (0.034750 --> 0.030286).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.0214383\n",
      "\tspeed: 0.0813s/iter; left time: 1305.3030s\n",
      "\titers: 200, epoch: 3 | loss: 0.0260556\n",
      "\tspeed: 0.0267s/iter; left time: 426.4218s\n",
      "\titers: 300, epoch: 3 | loss: 0.0223418\n",
      "\tspeed: 0.0266s/iter; left time: 420.9714s\n",
      "\titers: 400, epoch: 3 | loss: 0.0244385\n",
      "\tspeed: 0.0262s/iter; left time: 413.0811s\n",
      "\titers: 500, epoch: 3 | loss: 0.0181675\n",
      "\tspeed: 0.0260s/iter; left time: 407.0276s\n",
      "\titers: 600, epoch: 3 | loss: 0.0206798\n",
      "\tspeed: 0.0261s/iter; left time: 406.0530s\n",
      "\titers: 700, epoch: 3 | loss: 0.0213475\n",
      "\tspeed: 0.0263s/iter; left time: 406.6171s\n",
      "\titers: 800, epoch: 3 | loss: 0.0205706\n",
      "\tspeed: 0.0257s/iter; left time: 394.7053s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:23.76s\n",
      "Steps: 897 | Train Loss: 0.0231707 Vali Loss: 0.0302562 Test Loss: 0.0365081\n",
      "Validation loss decreased (0.030286 --> 0.030256).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.0216257\n",
      "\tspeed: 0.0836s/iter; left time: 1265.7855s\n",
      "\titers: 200, epoch: 4 | loss: 0.0232036\n",
      "\tspeed: 0.0264s/iter; left time: 396.6728s\n",
      "\titers: 300, epoch: 4 | loss: 0.0271958\n",
      "\tspeed: 0.0269s/iter; left time: 402.7046s\n",
      "\titers: 400, epoch: 4 | loss: 0.0175405\n",
      "\tspeed: 0.0265s/iter; left time: 393.1570s\n",
      "\titers: 500, epoch: 4 | loss: 0.0194984\n",
      "\tspeed: 0.0264s/iter; left time: 389.2683s\n",
      "\titers: 600, epoch: 4 | loss: 0.0239931\n",
      "\tspeed: 0.0259s/iter; left time: 379.3989s\n",
      "\titers: 700, epoch: 4 | loss: 0.0287811\n",
      "\tspeed: 0.0254s/iter; left time: 369.1801s\n",
      "\titers: 800, epoch: 4 | loss: 0.0192671\n",
      "\tspeed: 0.0255s/iter; left time: 368.8914s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:23.71s\n",
      "Steps: 897 | Train Loss: 0.0225274 Vali Loss: 0.0301724 Test Loss: 0.0363412\n",
      "Validation loss decreased (0.030256 --> 0.030172).  Saving model ...\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.0186760\n",
      "\tspeed: 0.0809s/iter; left time: 1153.3924s\n",
      "\titers: 200, epoch: 5 | loss: 0.0230048\n",
      "\tspeed: 0.0270s/iter; left time: 382.6375s\n",
      "\titers: 300, epoch: 5 | loss: 0.0221688\n",
      "\tspeed: 0.0262s/iter; left time: 368.3140s\n",
      "\titers: 400, epoch: 5 | loss: 0.0198810\n",
      "\tspeed: 0.0269s/iter; left time: 374.7197s\n",
      "\titers: 500, epoch: 5 | loss: 0.0223420\n",
      "\tspeed: 0.0269s/iter; left time: 372.1788s\n",
      "\titers: 600, epoch: 5 | loss: 0.0207522\n",
      "\tspeed: 0.0273s/iter; left time: 375.5484s\n",
      "\titers: 700, epoch: 5 | loss: 0.0239369\n",
      "\tspeed: 0.0264s/iter; left time: 360.1203s\n",
      "\titers: 800, epoch: 5 | loss: 0.0264241\n",
      "\tspeed: 0.0290s/iter; left time: 393.6877s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:24.77s\n",
      "Steps: 897 | Train Loss: 0.0217860 Vali Loss: 0.0310060 Test Loss: 0.0383260\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.0211753\n",
      "\tspeed: 0.0848s/iter; left time: 1132.9668s\n",
      "\titers: 200, epoch: 6 | loss: 0.0210677\n",
      "\tspeed: 0.0268s/iter; left time: 355.5773s\n",
      "\titers: 300, epoch: 6 | loss: 0.0206664\n",
      "\tspeed: 0.0270s/iter; left time: 355.3279s\n",
      "\titers: 400, epoch: 6 | loss: 0.0222670\n",
      "\tspeed: 0.0275s/iter; left time: 358.3952s\n",
      "\titers: 500, epoch: 6 | loss: 0.0233414\n",
      "\tspeed: 0.0295s/iter; left time: 381.5749s\n",
      "\titers: 600, epoch: 6 | loss: 0.0156066\n",
      "\tspeed: 0.0244s/iter; left time: 313.9034s\n",
      "\titers: 700, epoch: 6 | loss: 0.0221044\n",
      "\tspeed: 0.0276s/iter; left time: 352.3996s\n",
      "\titers: 800, epoch: 6 | loss: 0.0202908\n",
      "\tspeed: 0.0267s/iter; left time: 337.6884s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:24.48s\n",
      "Steps: 897 | Train Loss: 0.0210544 Vali Loss: 0.0305621 Test Loss: 0.0383751\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.0232950\n",
      "\tspeed: 0.0843s/iter; left time: 1049.7594s\n",
      "\titers: 200, epoch: 7 | loss: 0.0179229\n",
      "\tspeed: 0.0269s/iter; left time: 332.8372s\n",
      "\titers: 300, epoch: 7 | loss: 0.0254590\n",
      "\tspeed: 0.0268s/iter; left time: 328.0850s\n",
      "\titers: 400, epoch: 7 | loss: 0.0177850\n",
      "\tspeed: 0.0264s/iter; left time: 321.0640s\n",
      "\titers: 500, epoch: 7 | loss: 0.0192334\n",
      "\tspeed: 0.0265s/iter; left time: 319.2506s\n",
      "\titers: 600, epoch: 7 | loss: 0.0190366\n",
      "\tspeed: 0.0294s/iter; left time: 352.0229s\n",
      "\titers: 700, epoch: 7 | loss: 0.0204799\n",
      "\tspeed: 0.0283s/iter; left time: 335.2642s\n",
      "\titers: 800, epoch: 7 | loss: 0.0181169\n",
      "\tspeed: 0.0263s/iter; left time: 309.2359s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:24.64s\n",
      "Steps: 897 | Train Loss: 0.0203314 Vali Loss: 0.0306304 Test Loss: 0.0383281\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.0224937\n",
      "\tspeed: 0.0827s/iter; left time: 956.2683s\n",
      "\titers: 200, epoch: 8 | loss: 0.0187863\n",
      "\tspeed: 0.0264s/iter; left time: 302.6567s\n",
      "\titers: 300, epoch: 8 | loss: 0.0188222\n",
      "\tspeed: 0.0262s/iter; left time: 297.8886s\n",
      "\titers: 400, epoch: 8 | loss: 0.0184973\n",
      "\tspeed: 0.0258s/iter; left time: 290.7332s\n",
      "\titers: 500, epoch: 8 | loss: 0.0211402\n",
      "\tspeed: 0.0260s/iter; left time: 289.7246s\n",
      "\titers: 600, epoch: 8 | loss: 0.0148417\n",
      "\tspeed: 0.0268s/iter; left time: 295.9921s\n",
      "\titers: 700, epoch: 8 | loss: 0.0227680\n",
      "\tspeed: 0.0271s/iter; left time: 297.2651s\n",
      "\titers: 800, epoch: 8 | loss: 0.0181889\n",
      "\tspeed: 0.0268s/iter; left time: 290.7681s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:24.00s\n",
      "Steps: 897 | Train Loss: 0.0196504 Vali Loss: 0.0312679 Test Loss: 0.0393807\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 5.904900000000001e-05\n",
      "\titers: 100, epoch: 9 | loss: 0.0209411\n",
      "\tspeed: 0.0817s/iter; left time: 870.8851s\n",
      "\titers: 200, epoch: 9 | loss: 0.0221004\n",
      "\tspeed: 0.0260s/iter; left time: 274.5577s\n",
      "\titers: 300, epoch: 9 | loss: 0.0205826\n",
      "\tspeed: 0.0259s/iter; left time: 271.4415s\n",
      "\titers: 400, epoch: 9 | loss: 0.0186236\n",
      "\tspeed: 0.0265s/iter; left time: 274.8035s\n",
      "\titers: 500, epoch: 9 | loss: 0.0185294\n",
      "\tspeed: 0.0266s/iter; left time: 272.9911s\n",
      "\titers: 600, epoch: 9 | loss: 0.0209287\n",
      "\tspeed: 0.0267s/iter; left time: 271.2725s\n",
      "\titers: 700, epoch: 9 | loss: 0.0221294\n",
      "\tspeed: 0.0275s/iter; left time: 276.9713s\n",
      "\titers: 800, epoch: 9 | loss: 0.0158577\n",
      "\tspeed: 0.0282s/iter; left time: 281.1991s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 9\n",
      "Cost time: 00h:00m:24.25s\n",
      "Steps: 897 | Train Loss: 0.0191172 Vali Loss: 0.0310483 Test Loss: 0.0395106\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_336_96_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl96_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6145\n",
      "Scaled mse:0.03634120151400566, rmse:0.19063368439674377, mae:0.13006821274757385, rse:0.675072431564331\n",
      "Original data scale mse:31287554.0, rmse:5593.52783203125, mae:3559.01708984375, rse:0.27855971455574036\n",
      "\n",
      "=== Starting experiments for pred_len: 168 ===\n",
      "\n",
      "Args in experiment:\n",
      "Namespace(random_seed=2021, is_training=1, model_id='DE_336_168_scaler_choice_for_DE', model='PatchTST', data='custom', root_path='/vol/fob-vol3/nebenf24/riabchuv/my_work-1/datasets/', data_path='DE_data.csv', features='M', target='OT', freq='h', checkpoints='./checkpoints/', overlapping_windows=True, scaler_type='minmax', if_relu=True, channel_mixing=0, seq_len=336, label_len=48, pred_len=168, inverse=True, loss_fnc='MSE', fc_dropout=0.2, head_dropout=0.0, patch_len=32, stride=16, padding_patch='end', revin=1, affine=0, subtract_last=0, decomposition=0, kernel_size=25, individual=0, embed_type=0, enc_in=5, dec_in=5, c_out=5, d_model=128, n_heads=16, e_layers=3, d_layers=1, d_ff=256, moving_avg=25, factor=1, distil=True, dropout=0.2, embed='timeF', activation='gelu', output_attention=False, do_predict=False, num_workers=10, itr=1, train_epochs=20, batch_size=32, patience=5, learning_rate=0.0001, des='Exp', lradj='type3', pct_start=0.3, use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0,1,2,3', test_flop=False)\n",
      "Use GPU: cuda:0\n",
      ">>>>>>>start training : DE_336_168_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl168_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>>\n",
      "train 28633\n",
      "val 6073\n",
      "test 6073\n",
      "\titers: 100, epoch: 1 | loss: 0.0497348\n",
      "\tspeed: 0.0559s/iter; left time: 994.4222s\n",
      "\titers: 200, epoch: 1 | loss: 0.0354503\n",
      "\tspeed: 0.0288s/iter; left time: 509.7851s\n",
      "\titers: 300, epoch: 1 | loss: 0.0345030\n",
      "\tspeed: 0.0287s/iter; left time: 504.9834s\n",
      "\titers: 400, epoch: 1 | loss: 0.0339950\n",
      "\tspeed: 0.0279s/iter; left time: 488.3170s\n",
      "\titers: 500, epoch: 1 | loss: 0.0386137\n",
      "\tspeed: 0.0280s/iter; left time: 487.3617s\n",
      "\titers: 600, epoch: 1 | loss: 0.0367431\n",
      "\tspeed: 0.0298s/iter; left time: 514.5732s\n",
      "\titers: 700, epoch: 1 | loss: 0.0371684\n",
      "\tspeed: 0.0302s/iter; left time: 518.4818s\n",
      "\titers: 800, epoch: 1 | loss: 0.0305217\n",
      "\tspeed: 0.0305s/iter; left time: 520.2620s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 1\n",
      "Cost time: 00h:00m:26.50s\n",
      "Steps: 894 | Train Loss: 0.0370499 Vali Loss: 0.0365673 Test Loss: 0.0423967\n",
      "Validation loss decreased (inf --> 0.036567).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 2 | loss: 0.0287805\n",
      "\tspeed: 0.0881s/iter; left time: 1487.4337s\n",
      "\titers: 200, epoch: 2 | loss: 0.0402144\n",
      "\tspeed: 0.0302s/iter; left time: 506.6349s\n",
      "\titers: 300, epoch: 2 | loss: 0.0245465\n",
      "\tspeed: 0.0294s/iter; left time: 490.4191s\n",
      "\titers: 400, epoch: 2 | loss: 0.0382835\n",
      "\tspeed: 0.0136s/iter; left time: 225.8184s\n",
      "\titers: 500, epoch: 2 | loss: 0.0266117\n",
      "\tspeed: 0.0132s/iter; left time: 217.9853s\n",
      "\titers: 600, epoch: 2 | loss: 0.0278356\n",
      "\tspeed: 0.0231s/iter; left time: 378.7404s\n",
      "\titers: 700, epoch: 2 | loss: 0.0313929\n",
      "\tspeed: 0.0300s/iter; left time: 488.3750s\n",
      "\titers: 800, epoch: 2 | loss: 0.0247499\n",
      "\tspeed: 0.0302s/iter; left time: 489.5293s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 2\n",
      "Cost time: 00h:00m:22.70s\n",
      "Steps: 894 | Train Loss: 0.0271346 Vali Loss: 0.0327959 Test Loss: 0.0396321\n",
      "Validation loss decreased (0.036567 --> 0.032796).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 3 | loss: 0.0268525\n",
      "\tspeed: 0.0843s/iter; left time: 1348.2970s\n",
      "\titers: 200, epoch: 3 | loss: 0.0233329\n",
      "\tspeed: 0.0275s/iter; left time: 437.4324s\n",
      "\titers: 300, epoch: 3 | loss: 0.0208040\n",
      "\tspeed: 0.0287s/iter; left time: 453.0351s\n",
      "\titers: 400, epoch: 3 | loss: 0.0260136\n",
      "\tspeed: 0.0281s/iter; left time: 440.7868s\n",
      "\titers: 500, epoch: 3 | loss: 0.0270874\n",
      "\tspeed: 0.0296s/iter; left time: 461.3814s\n",
      "\titers: 600, epoch: 3 | loss: 0.0264719\n",
      "\tspeed: 0.0283s/iter; left time: 438.7070s\n",
      "\titers: 700, epoch: 3 | loss: 0.0255002\n",
      "\tspeed: 0.0275s/iter; left time: 422.9911s\n",
      "\titers: 800, epoch: 3 | loss: 0.0263106\n",
      "\tspeed: 0.0277s/iter; left time: 423.4618s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 3\n",
      "Cost time: 00h:00m:25.39s\n",
      "Steps: 894 | Train Loss: 0.0252257 Vali Loss: 0.0325355 Test Loss: 0.0391449\n",
      "Validation loss decreased (0.032796 --> 0.032535).  Saving model ...\n",
      "Updating learning rate to 0.0001\n",
      "\titers: 100, epoch: 4 | loss: 0.0212919\n",
      "\tspeed: 0.0826s/iter; left time: 1246.5770s\n",
      "\titers: 200, epoch: 4 | loss: 0.0205656\n",
      "\tspeed: 0.0277s/iter; left time: 414.9825s\n",
      "\titers: 300, epoch: 4 | loss: 0.0260505\n",
      "\tspeed: 0.0274s/iter; left time: 407.7456s\n",
      "\titers: 400, epoch: 4 | loss: 0.0197661\n",
      "\tspeed: 0.0278s/iter; left time: 411.5574s\n",
      "\titers: 500, epoch: 4 | loss: 0.0253953\n",
      "\tspeed: 0.0284s/iter; left time: 417.4660s\n",
      "\titers: 600, epoch: 4 | loss: 0.0212008\n",
      "\tspeed: 0.0279s/iter; left time: 407.7749s\n",
      "\titers: 700, epoch: 4 | loss: 0.0230240\n",
      "\tspeed: 0.0269s/iter; left time: 389.5897s\n",
      "\titers: 800, epoch: 4 | loss: 0.0235370\n",
      "\tspeed: 0.0257s/iter; left time: 369.9903s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 4\n",
      "Cost time: 00h:00m:24.77s\n",
      "Steps: 894 | Train Loss: 0.0243229 Vali Loss: 0.0333974 Test Loss: 0.0400419\n",
      "EarlyStopping counter: 1 out of 5\n",
      "Updating learning rate to 9e-05\n",
      "\titers: 100, epoch: 5 | loss: 0.0211974\n",
      "\tspeed: 0.0851s/iter; left time: 1208.4167s\n",
      "\titers: 200, epoch: 5 | loss: 0.0214492\n",
      "\tspeed: 0.0276s/iter; left time: 388.6521s\n",
      "\titers: 300, epoch: 5 | loss: 0.0234056\n",
      "\tspeed: 0.0273s/iter; left time: 382.6770s\n",
      "\titers: 400, epoch: 5 | loss: 0.0236356\n",
      "\tspeed: 0.0276s/iter; left time: 384.0422s\n",
      "\titers: 500, epoch: 5 | loss: 0.0267114\n",
      "\tspeed: 0.0259s/iter; left time: 357.5170s\n",
      "\titers: 600, epoch: 5 | loss: 0.0224755\n",
      "\tspeed: 0.0273s/iter; left time: 373.7695s\n",
      "\titers: 700, epoch: 5 | loss: 0.0251154\n",
      "\tspeed: 0.0293s/iter; left time: 398.8807s\n",
      "\titers: 800, epoch: 5 | loss: 0.0205094\n",
      "\tspeed: 0.0254s/iter; left time: 342.5754s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 5\n",
      "Cost time: 00h:00m:24.47s\n",
      "Steps: 894 | Train Loss: 0.0232769 Vali Loss: 0.0335537 Test Loss: 0.0404431\n",
      "EarlyStopping counter: 2 out of 5\n",
      "Updating learning rate to 8.1e-05\n",
      "\titers: 100, epoch: 6 | loss: 0.0261952\n",
      "\tspeed: 0.0782s/iter; left time: 1040.5452s\n",
      "\titers: 200, epoch: 6 | loss: 0.0221056\n",
      "\tspeed: 0.0280s/iter; left time: 369.8862s\n",
      "\titers: 300, epoch: 6 | loss: 0.0231464\n",
      "\tspeed: 0.0264s/iter; left time: 346.1522s\n",
      "\titers: 400, epoch: 6 | loss: 0.0196449\n",
      "\tspeed: 0.0274s/iter; left time: 356.5527s\n",
      "\titers: 500, epoch: 6 | loss: 0.0214147\n",
      "\tspeed: 0.0263s/iter; left time: 340.1258s\n",
      "\titers: 600, epoch: 6 | loss: 0.0199187\n",
      "\tspeed: 0.0262s/iter; left time: 336.2384s\n",
      "\titers: 700, epoch: 6 | loss: 0.0246073\n",
      "\tspeed: 0.0257s/iter; left time: 326.4358s\n",
      "\titers: 800, epoch: 6 | loss: 0.0200236\n",
      "\tspeed: 0.0257s/iter; left time: 324.2194s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 6\n",
      "Cost time: 00h:00m:23.84s\n",
      "Steps: 894 | Train Loss: 0.0224271 Vali Loss: 0.0337346 Test Loss: 0.0411209\n",
      "EarlyStopping counter: 3 out of 5\n",
      "Updating learning rate to 7.290000000000001e-05\n",
      "\titers: 100, epoch: 7 | loss: 0.0264276\n",
      "\tspeed: 0.0816s/iter; left time: 1012.7889s\n",
      "\titers: 200, epoch: 7 | loss: 0.0209202\n",
      "\tspeed: 0.0251s/iter; left time: 308.7540s\n",
      "\titers: 300, epoch: 7 | loss: 0.0200401\n",
      "\tspeed: 0.0254s/iter; left time: 310.3734s\n",
      "\titers: 400, epoch: 7 | loss: 0.0221033\n",
      "\tspeed: 0.0256s/iter; left time: 310.2546s\n",
      "\titers: 500, epoch: 7 | loss: 0.0198155\n",
      "\tspeed: 0.0257s/iter; left time: 308.7248s\n",
      "\titers: 600, epoch: 7 | loss: 0.0217914\n",
      "\tspeed: 0.0258s/iter; left time: 307.8416s\n",
      "\titers: 700, epoch: 7 | loss: 0.0182190\n",
      "\tspeed: 0.0261s/iter; left time: 308.9942s\n",
      "\titers: 800, epoch: 7 | loss: 0.0214543\n",
      "\tspeed: 0.0265s/iter; left time: 310.3960s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 7\n",
      "Cost time: 00h:00m:23.78s\n",
      "Steps: 894 | Train Loss: 0.0217058 Vali Loss: 0.0347749 Test Loss: 0.0408930\n",
      "EarlyStopping counter: 4 out of 5\n",
      "Updating learning rate to 6.561e-05\n",
      "\titers: 100, epoch: 8 | loss: 0.0212449\n",
      "\tspeed: 0.0806s/iter; left time: 928.5749s\n",
      "\titers: 200, epoch: 8 | loss: 0.0207765\n",
      "\tspeed: 0.0271s/iter; left time: 309.9068s\n",
      "\titers: 300, epoch: 8 | loss: 0.0217745\n",
      "\tspeed: 0.0282s/iter; left time: 319.3277s\n",
      "\titers: 400, epoch: 8 | loss: 0.0210134\n",
      "\tspeed: 0.0223s/iter; left time: 250.7213s\n",
      "\titers: 500, epoch: 8 | loss: 0.0186592\n",
      "\tspeed: 0.0265s/iter; left time: 294.9955s\n",
      "\titers: 600, epoch: 8 | loss: 0.0210895\n",
      "\tspeed: 0.0279s/iter; left time: 307.1084s\n",
      "\titers: 700, epoch: 8 | loss: 0.0229552\n",
      "\tspeed: 0.0231s/iter; left time: 252.7012s\n",
      "\titers: 800, epoch: 8 | loss: 0.0213346\n",
      "\tspeed: 0.0232s/iter; left time: 251.0718s\n",
      "-------------------------------------------------------------------------------------\n",
      "Epoch: 8\n",
      "Cost time: 00h:00m:22.82s\n",
      "Steps: 894 | Train Loss: 0.0211006 Vali Loss: 0.0355346 Test Loss: 0.0419611\n",
      "EarlyStopping counter: 5 out of 5\n",
      "Early stopping\n",
      "-------------------------------------------------------------------------------------\n",
      ">>>>>>>testing : DE_336_168_scaler_choice_for_DE_PatchTST_custom_ftM_sl336_ll48_pl168_dm128_nh16_el3_dl1_df256_fc1_ebtimeF_dtTrue_lossMSE_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\n",
      "test 6073\n",
      "Scaled mse:0.03914487361907959, rmse:0.19785062968730927, mae:0.13802330195903778, rse:0.7009251713752747\n",
      "Original data scale mse:34331184.0, rmse:5859.28173828125, mae:3804.841552734375, rse:0.29193758964538574\n"
     ]
    }
   ],
   "source": [
    "# Dynamic variables\n",
    "losses = [\"MSE\"]\n",
    "pred_lens = [\"24\", \"96\", \"168\"]\n",
    "seq_len = \"336\"\n",
    "lr = \"0.0001\"\n",
    "model = \"PatchTST\"\n",
    "itr = 1 \n",
    "n_heads = \"16\"\n",
    "d_model = \"128\"\n",
    "d_ff = \"256\"\n",
    "dropout = \"0.2\"\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = cuda_device\n",
    "\n",
    "# New log file path\n",
    "log_file_path = f\"{log_dir}/{model}_{country}.log\"\n",
    "\n",
    "patchtst_results_scaled, patchtst_results_unscaled = [], []\n",
    "\n",
    "with open(log_file_path, \"w\") as log_file:\n",
    "    for loss in losses:\n",
    "        statement_1 = f\"\\n=== Starting experiments for loss function: {loss} ===\\n\"\n",
    "        log_file.write(statement_1)\n",
    "        print(statement_1)  # Print to notebook\n",
    "\n",
    "        for pred_len in pred_lens:\n",
    "            statement_2 = f\"\\n=== Starting experiments for pred_len: {pred_len} ===\\n\"\n",
    "            log_file.write(statement_2)\n",
    "            print(statement_2) \n",
    "            model_id = f\"{country}_{seq_len}_{pred_len}_scaler_choice_for_{country}\"\n",
    "\n",
    "            # Command arguments\n",
    "            command = f\"\"\"\n",
    "            python {script_path} \\\n",
    "              --random_seed 2021 \\\n",
    "              --is_training 1 \\\n",
    "              --root_path \"{data_path}\" \\\n",
    "              --data_path \"{dataset}\" \\\n",
    "              --model_id {model_id} \\\n",
    "              --model \"{model}\" \\\n",
    "              --data \"custom\" \\\n",
    "              --features M \\\n",
    "              --seq_len {seq_len} \\\n",
    "              --pred_len {pred_len} \\\n",
    "              --e_layers 3 \\\n",
    "              --factor 1 \\\n",
    "              --enc_in 5 \\\n",
    "              --dec_in 5 \\\n",
    "              --c_out 5 \\\n",
    "              --des 'Exp' \\\n",
    "              --train_epochs 20 \\\n",
    "              --patience 5 \\\n",
    "              --n_heads {n_heads} \\\n",
    "              --d_model {d_model} \\\n",
    "              --d_ff {d_ff} \\\n",
    "              --dropout {dropout} \\\n",
    "              --fc_dropout {dropout} \\\n",
    "              --patch_len 32 \\\n",
    "              --stride 16 \\\n",
    "              --overlapping_windows \\\n",
    "              --inverse \\\n",
    "              --scaler_type minmax \\\n",
    "              --if_relu \\\n",
    "              --loss_fnc \"{loss}\" \\\n",
    "              --itr {itr} --batch_size 32 --learning_rate \"{lr}\"\n",
    "            \"\"\"\n",
    "\n",
    "            # Run the command and capture the output\n",
    "            process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)\n",
    "\n",
    "            # Capture the output in real-time\n",
    "            output = []\n",
    "            for line in process.stdout:\n",
    "                output.append(line)\n",
    "                print(line, end='')  # Print in the .ipynb cell\n",
    "                log_file.write(line)  # Write to the log file\n",
    "\n",
    "            # Wait for the process to complete\n",
    "            process.wait()\n",
    "\n",
    "            # Delete the checkpoints folder and all its contents\n",
    "            shutil.rmtree('./checkpoints' )\n",
    "\n",
    "            # Extract metrics for each iteration\n",
    "            iteration_metrics_scaled = extract_metrics_from_output(output, itr)\n",
    "            iteration_metrics_unscaled = extract_metrics_from_output(output, itr, if_scaled=False)\n",
    "\n",
    "            # Log the extracted metrics and save them\n",
    "            for iteration, (scaled_metrics, unscaled_metrics) in enumerate(zip(iteration_metrics_scaled, iteration_metrics_unscaled), start=1):\n",
    "                log_file.write(f\"\\nExtracted Metrics for {country}, pred_len={pred_len}, iteration={iteration}:\\n\")\n",
    "                log_file.write(f\"Scaled Metrics - MSE: {scaled_metrics[0]}, RMSE: {scaled_metrics[1]}, MAE: {scaled_metrics[2]}, RSE: {scaled_metrics[3]}\\n\")\n",
    "                log_file.write(f\"Unscaled Metrics - MSE: {unscaled_metrics[0]}, RMSE: {unscaled_metrics[1]}, MAE: {unscaled_metrics[2]}, RSE: {unscaled_metrics[3]}\\n\")\n",
    "\n",
    "                # Append the results to the informer_results lists\n",
    "                metrics_data = [(patchtst_results_scaled, scaled_metrics), (patchtst_results_unscaled, unscaled_metrics)]\n",
    "\n",
    "                for result_list, metrics in metrics_data:\n",
    "                    result_list.append({\n",
    "                        'Loss_function': loss,\n",
    "                        'Pred_len': pred_len,\n",
    "                        'Iteration': iteration,\n",
    "                        'MSE': metrics[0],\n",
    "                        'RMSE': metrics[1],\n",
    "                        'MAE': metrics[2],\n",
    "                        'RSE': metrics[3]\n",
    "                    })"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>MSE</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>RSE</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Loss_function</th>\n",
       "      <th>Iteration</th>\n",
       "      <th>Pred_len</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">MSE</th>\n",
       "      <th rowspan=\"3\" valign=\"top\">1</th>\n",
       "      <th>24</th>\n",
       "      <td>0.0209</td>\n",
       "      <td>0.1446</td>\n",
       "      <td>0.0911</td>\n",
       "      <td>0.5108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>0.0363</td>\n",
       "      <td>0.1906</td>\n",
       "      <td>0.1301</td>\n",
       "      <td>0.6751</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>0.0391</td>\n",
       "      <td>0.1979</td>\n",
       "      <td>0.1380</td>\n",
       "      <td>0.7009</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     MSE    RMSE     MAE     RSE\n",
       "Loss_function Iteration Pred_len                                \n",
       "MSE           1         24        0.0209  0.1446  0.0911  0.5108\n",
       "                        96        0.0363  0.1906  0.1301  0.6751\n",
       "                        168       0.0391  0.1979  0.1380  0.7009"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path_dir = './results/scaler_choice'\n",
    "csv_name_scaled = 'patchtst_scaler_results_scaled_minmax_default.csv'\n",
    "csv_name_unscaled = 'patchtst_scaler_results_unscaled_minmax_default.csv'\n",
    "\n",
    "# Convert the results into a DataFrame and save as CSV\n",
    "patchtst_df_scaled = convert_results_into_df(patchtst_results_scaled, path_dir, csv_name_scaled)\n",
    "patchtst_df_unscaled = convert_results_into_df(patchtst_results_unscaled, path_dir, csv_name_unscaled)\n",
    "patchtst_df_scaled.round(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>MSE</th>\n",
       "      <th>RMSE</th>\n",
       "      <th>MAE</th>\n",
       "      <th>RSE</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Loss_function</th>\n",
       "      <th>Iteration</th>\n",
       "      <th>Pred_len</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">MSE</th>\n",
       "      <th rowspan=\"3\" valign=\"top\">1</th>\n",
       "      <th>24</th>\n",
       "      <td>16405032.0</td>\n",
       "      <td>4050.3125</td>\n",
       "      <td>2453.5930</td>\n",
       "      <td>0.2014</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>31287554.0</td>\n",
       "      <td>5593.5278</td>\n",
       "      <td>3559.0171</td>\n",
       "      <td>0.2786</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>34331184.0</td>\n",
       "      <td>5859.2817</td>\n",
       "      <td>3804.8416</td>\n",
       "      <td>0.2919</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         MSE       RMSE        MAE     RSE\n",
       "Loss_function Iteration Pred_len                                          \n",
       "MSE           1         24        16405032.0  4050.3125  2453.5930  0.2014\n",
       "                        96        31287554.0  5593.5278  3559.0171  0.2786\n",
       "                        168       34331184.0  5859.2817  3804.8416  0.2919"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "patchtst_results_scaled = pd.read_csv(os.path.join(path_dir, csv_name_scaled))\n",
    "patchtst_df_unscaled.round(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "shutil.rmtree(\"results_loss_scaled\") # we do not need this directory and results anymore. If you need - comment this line\n",
    "\n",
    "# Rename folder\n",
    "os.rename(\"results_loss_unscaled\", 'minmax')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "val",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}