Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Google Tensorflow 2.14 on 13900K Dual NVIDIA RTX-4090 GPUs #35

Open
obriensystems opened this issue Dec 1, 2024 · 3 comments
Open

Google Tensorflow 2.14 on 13900K Dual NVIDIA RTX-4090 GPUs #35

obriensystems opened this issue Dec 1, 2024 · 3 comments
Assignees
@obriensystems

Comments

@obriensystems
Copy link
Member

obriensystems commented Dec 1, 2024 •
edited
Loading

see #34
see #33
running
https://hub.docker.com/layers/tensorflow/tensorflow/2.14.0-gpu/images/sha256-64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206?context=explore
FROM tensorflow/tensorflow:2.14.0-gpu

batch_size

dual GPU
128, 26%/26% TDP, 51ms
256, 27%/27% TDP, 52 ms
512, 31%/31% TDP, 53 ms
1024, 37%/37% TDP, 59 ms
2048, 45%/45% TDP, 75%, 69 ms
4096, 55%/55% TDP, 78%, 104 ms
5120, 60%/60% TDP, 87%, 127 ms, coil wine
6144, 65%/65% TDP, 90%, 142ms, coil
8192, 70%/70% TDP, 92%, 75% mem, 172 ms, coil
16384, 75%/75% TDP, 99%, 70% mem, 292 ms, coil

GPU 0 (lower lower temp)
22274MB
128, 36% TDP, 45%, 10% mem, 20 ms
256, 47% TDP, 61/16%, 20 ms
512, 59% TDP, 76/27$, 25 ms
1024, 65% TDP, 85/47%, 40 ms
2048, 71% TDP, 92/65%, 72 ms
4096, 76% TDP, 95/73%, 137 ms
5120, 75% TDP, 97/78%, 181 ms
6144, 74% TDP, 99/77%, 202 ms
8192, 76% TDP, 99/83%, 259 ms
16384, n/a

GPU 1 (upper higher temp)
23517MB
128, 36% TDP, 44/10, 22 ms
256, 27% TDP, ms
512, 31% TDP, ms
1024, 37% TDP, ms
2048, 45% TDP, 75%, ms
4096, 55% TDP, 78%, ms
5120, 60% TDP, 87%, ms, coil wine
6144, 65% TDP, 90%, ms, coil
8192, 70% TDP, 92%, 75% mem, ms, coil
16384, n/a
@obriensystems obriensystems self-assigned this Dec 1, 2024
@obriensystems obriensystems changed the title Google Tensorflow 2.14 on 13900K Dual RTX-4090 GPUs Google Tensorflow 2.14 on 13900K Dual NVIDIA RTX-4090 GPUs Dec 1, 2024
@obriensystems
Copy link
Member Author 

michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
[+] Building 1.2s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.9s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 3.61kB                                                                                                                                                         0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => [3/3] COPY /src/tflow.py .                                                                                                                                                              0.0s
 => exporting to image                                                                                                                                                                      0.1s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:653b6994370c319550b17deb55bf6638311685c7ff0b96c365d8e3c270bd63d3                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 15:29:36.064526: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 15:29:36.251740: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 15:29:36.251766: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 15:29:36.252844: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 15:29:36.336719: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 15:29:38.192608: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.192646: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.205811: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.205846: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.205856: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.205864: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.429494: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.429529: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.429541: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.429548: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.429556: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.429577: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.436268: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.436352: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.436384: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.436389: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:29:38.436399: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.436412: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:29:38.436432: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.436446: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 15:29:38.436939: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:29:38.436958: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 4s 0us/step
Epoch 1/25
2024-12-01 15:30:02.287422: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:30:02.294821: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:30:04.431518: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f0553ed45d0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 15:30:04.431543: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:30:04.431546: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:30:04.437219: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 15:30:04.503179: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
391/391 [==============================] - 45s 53ms/step - loss: 4.7545 - accuracy: 0.0672
Epoch 2/25
391/391 [==============================] - 20s 52ms/step - loss: 4.2115 - accuracy: 0.0981
Epoch 3/25
391/391 [==============================] - 20s 51ms/step - loss: 3.7802 - accuracy: 0.1541
Epoch 4/25
391/391 [==============================] - 20s 51ms/step - loss: 3.6664 - accuracy: 0.1855
Epoch 5/25
391/391 [==============================] - 21s 53ms/step - loss: 4.1901 - accuracy: 0.0955
Epoch 6/25
391/391 [==============================] - 21s 53ms/step - loss: 4.3061 - accuracy: 0.0790
Epoch 7/25
391/391 [==============================] - 21s 54ms/step - loss: 4.1252 - accuracy: 0.0948
Epoch 8/25
391/391 [==============================] - 21s 54ms/step - loss: 3.9576 - accuracy: 0.1167
Epoch 9/25
391/391 [==============================] - 20s 52ms/step - loss: 3.8180 - accuracy: 0.1347
Epoch 10/25
391/391 [==============================] - 20s 52ms/step - loss: 3.7038 - accuracy: 0.1526
Epoch 11/25
391/391 [==============================] - 20s 52ms/step - loss: 3.7394 - accuracy: 0.1520
Epoch 12/25
391/391 [==============================] - 20s 52ms/step - loss: 3.6223 - accuracy: 0.1634
Epoch 13/25
391/391 [==============================] - 20s 51ms/step - loss: 3.3519 - accuracy: 0.2069
Epoch 14/25
391/391 [==============================] - 20s 51ms/step - loss: 3.1753 - accuracy: 0.2341
Epoch 15/25
391/391 [==============================] - 21s 53ms/step - loss: 3.0678 - accuracy: 0.2533
Epoch 16/25
391/391 [==============================] - 20s 51ms/step - loss: 2.9156 - accuracy: 0.2809
Epoch 17/25
391/391 [==============================] - 20s 51ms/step - loss: 2.8118 - accuracy: 0.2982
Epoch 18/25
391/391 [==============================] - 20s 52ms/step - loss: 2.7719 - accuracy: 0.3078
Epoch 19/25
391/391 [==============================] - 20s 52ms/step - loss: 2.5601 - accuracy: 0.3468
Epoch 20/25
391/391 [==============================] - 20s 51ms/step - loss: 2.4264 - accuracy: 0.3754
Epoch 21/25
391/391 [==============================] - 20s 51ms/step - loss: 2.2701 - accuracy: 0.4075
Epoch 22/25
391/391 [==============================] - 20s 52ms/step - loss: 2.1148 - accuracy: 0.4378
Epoch 23/25
391/391 [==============================] - 21s 53ms/step - loss: 1.9359 - accuracy: 0.4778
Epoch 24/25
391/391 [==============================] - 20s 52ms/step - loss: 1.8513 - accuracy: 0.4983
Epoch 25/25
391/391 [==============================] - 21s 53ms/step - loss: 1.5693 - accuracy: 0.5663
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
[+] Building 0.7s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.5s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 3.61kB                                                                                                                                                         0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => [3/3] COPY /src/tflow.py .                                                                                                                                                              0.0s
 => exporting to image                                                                                                                                                                      0.1s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:1e168ac6217970b101bb9aefa35e9021ae879dd7a1d3252b492f2ab174e7f850                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 15:41:23.019421: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 15:41:23.037913: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 15:41:23.037950: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 15:41:23.037963: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 15:41:23.041702: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 15:41:23.896145: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:23.896183: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:23.898611: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:23.898643: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:23.898653: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:23.898661: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.094940: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.094973: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.094983: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.094989: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.094995: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.095001: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.100916: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.101009: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.101053: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.101059: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:41:24.101069: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.101082: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:41:24.101103: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.101128: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 15:41:24.101622: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:41:24.101643: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 5s 0us/step
Epoch 1/25
2024-12-01 15:41:48.664651: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:41:48.674170: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:41:50.533896: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f377a4153b0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 15:41:50.533915: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:41:50.533944: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:41:50.537223: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 15:41:50.593178: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
196/196 [==============================] - 36s 59ms/step - loss: 4.4174 - accuracy: 0.0850
Epoch 2/25
196/196 [==============================] - 10s 52ms/step - loss: 3.8822 - accuracy: 0.1527
Epoch 3/25
196/196 [==============================] - 10s 52ms/step - loss: 3.5778 - accuracy: 0.1930
Epoch 4/25
196/196 [==============================] - 10s 53ms/step - loss: 3.6797 - accuracy: 0.1806
Epoch 5/25
196/196 [==============================] - 10s 53ms/step - loss: 3.6930 - accuracy: 0.1955
Epoch 6/25
196/196 [==============================] - 10s 52ms/step - loss: 3.5007 - accuracy: 0.2034
Epoch 7/25
196/196 [==============================] - 10s 53ms/step - loss: 3.3586 - accuracy: 0.2297
Epoch 8/25
196/196 [==============================] - 10s 53ms/step - loss: 2.9995 - accuracy: 0.2762
Epoch 9/25
196/196 [==============================] - 10s 53ms/step - loss: 2.9760 - accuracy: 0.2886
Epoch 10/25
196/196 [==============================] - 10s 53ms/step - loss: 2.8610 - accuracy: 0.3078
Epoch 11/25
196/196 [==============================] - 10s 53ms/step - loss: 2.6190 - accuracy: 0.3515
Epoch 12/25
196/196 [==============================] - 10s 52ms/step - loss: 2.5177 - accuracy: 0.3695
Epoch 13/25
196/196 [==============================] - 10s 53ms/step - loss: 2.3116 - accuracy: 0.4111
Epoch 14/25
196/196 [==============================] - 10s 53ms/step - loss: 2.1925 - accuracy: 0.4363
Epoch 15/25
196/196 [==============================] - 11s 56ms/step - loss: 1.9779 - accuracy: 0.4823
Epoch 16/25
196/196 [==============================] - 11s 57ms/step - loss: 1.9467 - accuracy: 0.4884
Epoch 17/25
196/196 [==============================] - 11s 56ms/step - loss: 1.8068 - accuracy: 0.5296
Epoch 18/25
196/196 [==============================] - 11s 56ms/step - loss: 1.6428 - accuracy: 0.5629
Epoch 19/25
196/196 [==============================] - 11s 56ms/step - loss: 1.3729 - accuracy: 0.6289
Epoch 20/25
196/196 [==============================] - 11s 55ms/step - loss: 1.5040 - accuracy: 0.6096
Epoch 21/25
196/196 [==============================] - 11s 55ms/step - loss: 1.2415 - accuracy: 0.6640
Epoch 22/25
196/196 [==============================] - 11s 54ms/step - loss: 0.9934 - accuracy: 0.7266
Epoch 23/25
196/196 [==============================] - 11s 54ms/step - loss: 0.9116 - accuracy: 0.7459
Epoch 24/25
196/196 [==============================] - 11s 54ms/step - loss: 0.6941 - accuracy: 0.8036
Epoch 25/25
196/196 [==============================] - 11s 55ms/step - loss: 0.7163 - accuracy: 0.8108
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
[+] Building 0.8s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.6s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 3.61kB                                                                                                                                                         0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => [3/3] COPY /src/tflow.py .                                                                                                                                                              0.0s
 => exporting to image                                                                                                                                                                      0.1s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:6552aa8cc1dfaa5eda06554b9d32e623ab6680636713819c56a7d6a2236d8811                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 15:50:58.253087: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 15:50:58.271736: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 15:50:58.271776: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 15:50:58.271787: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 15:50:58.275343: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 15:50:59.128552: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.128591: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.131079: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.131116: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.131125: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.131133: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.325393: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.325429: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.325441: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.325448: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.325454: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.325461: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.331764: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.331859: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.331895: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.331901: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:50:59.331919: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.331923: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:50:59.331938: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.331952: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 15:50:59.332522: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:50:59.332552: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 3s 0us/step
Epoch 1/25
2024-12-01 15:51:22.144457: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:51:22.153981: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:51:24.231274: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5e8978d0c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 15:51:24.231297: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:51:24.231299: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:51:24.234963: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 15:51:24.288497: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
98/98 [==============================] - 31s 69ms/step - loss: 4.4659 - accuracy: 0.0765
Epoch 2/25
98/98 [==============================] - 5s 55ms/step - loss: 3.6156 - accuracy: 0.1711
Epoch 3/25
98/98 [==============================] - 6s 57ms/step - loss: 3.2186 - accuracy: 0.2361
Epoch 4/25
98/98 [==============================] - 5s 54ms/step - loss: 3.3893 - accuracy: 0.2170
Epoch 5/25
98/98 [==============================] - 5s 54ms/step - loss: 3.0986 - accuracy: 0.2622
Epoch 6/25
98/98 [==============================] - 5s 54ms/step - loss: 2.8655 - accuracy: 0.3112
Epoch 7/25
98/98 [==============================] - 5s 54ms/step - loss: 2.5071 - accuracy: 0.3704
Epoch 8/25
98/98 [==============================] - 5s 54ms/step - loss: 2.4072 - accuracy: 0.3954
Epoch 9/25
98/98 [==============================] - 5s 55ms/step - loss: 2.3805 - accuracy: 0.4016
Epoch 10/25
98/98 [==============================] - 5s 54ms/step - loss: 2.1766 - accuracy: 0.4400
Epoch 11/25
98/98 [==============================] - 5s 55ms/step - loss: 1.8503 - accuracy: 0.5117
Epoch 12/25
98/98 [==============================] - 5s 54ms/step - loss: 2.7644 - accuracy: 0.3747
Epoch 13/25
98/98 [==============================] - 5s 53ms/step - loss: 2.6842 - accuracy: 0.3591
Epoch 14/25
98/98 [==============================] - 5s 53ms/step - loss: 2.3684 - accuracy: 0.4187
Epoch 15/25
98/98 [==============================] - 5s 54ms/step - loss: 1.9617 - accuracy: 0.5011
Epoch 16/25
98/98 [==============================] - 5s 54ms/step - loss: 1.5381 - accuracy: 0.5914
Epoch 17/25
98/98 [==============================] - 5s 55ms/step - loss: 1.2321 - accuracy: 0.6630
Epoch 18/25
98/98 [==============================] - 5s 55ms/step - loss: 0.9855 - accuracy: 0.7305
Epoch 19/25
98/98 [==============================] - 5s 55ms/step - loss: 0.8516 - accuracy: 0.7721
Epoch 20/25
98/98 [==============================] - 5s 55ms/step - loss: 0.7908 - accuracy: 0.7795
Epoch 21/25
98/98 [==============================] - 5s 55ms/step - loss: 0.5266 - accuracy: 0.8594
Epoch 22/25
98/98 [==============================] - 5s 54ms/step - loss: 0.4616 - accuracy: 0.8755
Epoch 23/25
98/98 [==============================] - 5s 54ms/step - loss: 0.7672 - accuracy: 0.7893
Epoch 24/25
98/98 [==============================] - 5s 54ms/step - loss: 0.5442 - accuracy: 0.8464
Epoch 25/25
98/98 [==============================] - 5s 54ms/step - loss: 0.3228 - accuracy: 0.9266
(base)

@obriensystems
Copy link
Member Author 

michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
[+] Building 0.6s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.3s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 3.61kB                                                                                                                                                         0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => [3/3] COPY /src/tflow.py .                                                                                                                                                              0.0s
 => exporting to image                                                                                                                                                                      0.1s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:239da08ca38f1b47938fb901b027c3372be33750818273c0d91a3af5e862db82                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 15:54:44.253773: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 15:54:44.272725: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 15:54:44.272771: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 15:54:44.272783: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 15:54:44.276439: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 15:54:45.120692: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.120728: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.123144: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.123187: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.123196: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.123205: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.340706: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.340738: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.340748: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.340755: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.340760: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.340767: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.346896: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.346984: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.347014: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.347029: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:54:45.347041: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.347044: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:54:45.347059: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.347083: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 15:54:45.347821: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:54:45.347846: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 4s 0us/step
Epoch 1/25
2024-12-01 15:55:09.326818: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:55:09.336495: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:55:11.491964: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f86e4d5cc10 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 15:55:11.491989: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:55:11.491992: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:55:11.495545: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 15:55:11.551982: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
49/49 [==============================] - 29s 92ms/step - loss: 4.8119 - accuracy: 0.0600
Epoch 2/25
49/49 [==============================] - 3s 59ms/step - loss: 3.8926 - accuracy: 0.1392
Epoch 3/25
49/49 [==============================] - 3s 61ms/step - loss: 3.8155 - accuracy: 0.1533
Epoch 4/25
49/49 [==============================] - 3s 60ms/step - loss: 3.4287 - accuracy: 0.2158
Epoch 5/25
49/49 [==============================] - 3s 61ms/step - loss: 3.2803 - accuracy: 0.2342
Epoch 6/25
49/49 [==============================] - 3s 61ms/step - loss: 3.1009 - accuracy: 0.2646
Epoch 7/25
49/49 [==============================] - 3s 60ms/step - loss: 3.0373 - accuracy: 0.2814
Epoch 8/25
49/49 [==============================] - 3s 60ms/step - loss: 2.7357 - accuracy: 0.3329
Epoch 9/25
49/49 [==============================] - 3s 61ms/step - loss: 2.5178 - accuracy: 0.3764
Epoch 10/25
49/49 [==============================] - 3s 60ms/step - loss: 2.3803 - accuracy: 0.4047
Epoch 11/25
49/49 [==============================] - 3s 61ms/step - loss: 2.0930 - accuracy: 0.4681
Epoch 12/25
49/49 [==============================] - 3s 60ms/step - loss: 1.8126 - accuracy: 0.5267
Epoch 13/25
49/49 [==============================] - 3s 59ms/step - loss: 1.6059 - accuracy: 0.5784
Epoch 14/25
49/49 [==============================] - 3s 58ms/step - loss: 2.0562 - accuracy: 0.4750
Epoch 15/25
49/49 [==============================] - 3s 59ms/step - loss: 1.5365 - accuracy: 0.5783
Epoch 16/25
49/49 [==============================] - 3s 58ms/step - loss: 1.2546 - accuracy: 0.6494
Epoch 17/25
49/49 [==============================] - 3s 58ms/step - loss: 0.9489 - accuracy: 0.7278
Epoch 18/25
49/49 [==============================] - 3s 58ms/step - loss: 0.6872 - accuracy: 0.8028
Epoch 19/25
49/49 [==============================] - 3s 58ms/step - loss: 0.5358 - accuracy: 0.8443
Epoch 20/25
49/49 [==============================] - 3s 58ms/step - loss: 0.3975 - accuracy: 0.8849
Epoch 21/25
49/49 [==============================] - 3s 57ms/step - loss: 0.4887 - accuracy: 0.8567
Epoch 22/25
49/49 [==============================] - 3s 58ms/step - loss: 0.4382 - accuracy: 0.8716
Epoch 23/25
49/49 [==============================] - 3s 61ms/step - loss: 0.2882 - accuracy: 0.9194
Epoch 24/25
49/49 [==============================] - 3s 62ms/step - loss: 0.2461 - accuracy: 0.9330
Epoch 25/25
49/49 [==============================] - 3s 62ms/step - loss: 0.3109 - accuracy: 0.9116
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
[+] Building 0.7s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.5s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 57B                                                                                                                                                            0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => CACHED [3/3] COPY /src/tflow.py .                                                                                                                                                       0.0s
 => exporting to image                                                                                                                                                                      0.0s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:239da08ca38f1b47938fb901b027c3372be33750818273c0d91a3af5e862db82                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 15:57:21.468860: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 15:57:21.487036: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 15:57:21.487071: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 15:57:21.487083: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 15:57:21.490761: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 15:57:22.366364: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.366401: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.368852: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.368893: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.368902: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.368909: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.580292: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.580321: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.580332: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.580339: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.580345: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.580351: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.586294: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.586387: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.586428: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.586441: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:57:22.586452: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.586454: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:57:22.586468: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.586480: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 15:57:22.587021: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:57:22.587049: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 4s 0us/step
Epoch 1/25
2024-12-01 15:57:46.179518: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:57:46.189280: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:57:48.534752: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f23d96acff0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 15:57:48.534776: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:57:48.534778: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:57:48.538478: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 15:57:48.591374: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
49/49 [==============================] - 30s 92ms/step - loss: 4.6395 - accuracy: 0.0595
Epoch 2/25
49/49 [==============================] - 3s 60ms/step - loss: 3.6285 - accuracy: 0.1611
Epoch 3/25
49/49 [==============================] - 3s 60ms/step - loss: 3.1562 - accuracy: 0.2405
Epoch 4/25
49/49 [==============================] - 3s 60ms/step - loss: 2.8581 - accuracy: 0.2968
Epoch 5/25
21/49 [===========>..................] - ETA: 1s - loss: 2.4861 - accuracy: 0.3744Traceback (most recent call last):
  File "/src/tflow.py", line 73, in <module>
    parallel_model.fit(x_train, y_train, epochs=25, batch_size=1024)#256)#7168)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/keras/src/utils/traceback_utils.py", line 65, in error_handler
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/keras/src/engine/training.py", line 1783, in fit
    tmp_logs = self.train_function(iterator)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/util/traceback_utils.py", line 150, in error_handler
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 831, in __call__
    result = self._call(*args, **kwds)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 867, in _call
    return tracing_compilation.call_function(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py", line 139, in call_function
    return function._call_flat(  # pylint: disable=protected-access
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/eager/polymorphic_function/concrete_function.py", line 1264, in _call_flat
    return self._inference_function.flat_call(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/eager/polymorphic_function/atomic_function.py", line 217, in flat_call
    flat_outputs = self(*args)
                   ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/eager/polymorphic_function/atomic_function.py", line 252, in __call__
    outputs = self._bound_context.call_function(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/eager/context.py", line 1479, in call_function
    outputs = execute.execute(
              ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
[+] Building 0.7s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.4s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 3.61kB                                                                                                                                                         0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => [3/3] COPY /src/tflow.py .                                                                                                                                                              0.0s
 => exporting to image                                                                                                                                                                      0.1s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:4ff56c07c6229bb33fa750eeb9828b7c6f8205d8fe59ede5f09bba20faf9c139                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 15:58:18.764427: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 15:58:18.783564: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 15:58:18.783610: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 15:58:18.783623: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 15:58:18.787488: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 15:58:19.622960: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.622996: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.625510: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.625546: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.625556: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.625567: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.858396: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.858429: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.858441: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.858448: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.858455: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.858474: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.864584: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.864691: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.864729: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.864734: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:58:19.864744: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.864755: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 15:58:19.864774: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.864787: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 15:58:19.865136: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 15:58:19.865162: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 4s 0us/step
Epoch 1/25
2024-12-01 15:58:42.893958: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:58:42.904436: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 15:58:45.703703: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fad305cdf40 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 15:58:45.703727: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:58:45.703729: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 15:58:45.708116: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 15:58:45.764793: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
25/25 [==============================] - 29s 142ms/step - loss: 5.0307 - accuracy: 0.0407
Epoch 2/25
25/25 [==============================] - 2s 70ms/step - loss: 3.7985 - accuracy: 0.1228
Epoch 3/25
25/25 [==============================] - 2s 70ms/step - loss: 3.3469 - accuracy: 0.2027
Epoch 4/25
25/25 [==============================] - 2s 70ms/step - loss: 2.9406 - accuracy: 0.2773
Epoch 5/25
25/25 [==============================] - 2s 70ms/step - loss: 2.5887 - accuracy: 0.3454
Epoch 6/25
25/25 [==============================] - 2s 69ms/step - loss: 2.4306 - accuracy: 0.3789
Epoch 7/25
25/25 [==============================] - 2s 69ms/step - loss: 2.2856 - accuracy: 0.4165
Epoch 8/25
25/25 [==============================] - 2s 69ms/step - loss: 1.8341 - accuracy: 0.5111
Epoch 9/25
25/25 [==============================] - 2s 71ms/step - loss: 1.9082 - accuracy: 0.5059
Epoch 10/25
25/25 [==============================] - 2s 71ms/step - loss: 2.2201 - accuracy: 0.4202
Epoch 11/25
25/25 [==============================] - 2s 69ms/step - loss: 1.4970 - accuracy: 0.5927
Epoch 12/25
25/25 [==============================] - 2s 69ms/step - loss: 0.9931 - accuracy: 0.7173
Epoch 13/25
25/25 [==============================] - 2s 70ms/step - loss: 0.7213 - accuracy: 0.7931
Epoch 14/25
25/25 [==============================] - 2s 69ms/step - loss: 0.6466 - accuracy: 0.8084
Epoch 15/25
25/25 [==============================] - 2s 71ms/step - loss: 0.5503 - accuracy: 0.8334
Epoch 16/25
25/25 [==============================] - 2s 70ms/step - loss: 0.4349 - accuracy: 0.8725
Epoch 17/25
25/25 [==============================] - 2s 69ms/step - loss: 0.3532 - accuracy: 0.8953
Epoch 18/25
25/25 [==============================] - 2s 70ms/step - loss: 0.3178 - accuracy: 0.9048
Epoch 19/25
25/25 [==============================] - 2s 72ms/step - loss: 0.7592 - accuracy: 0.7819
Epoch 20/25
25/25 [==============================] - 2s 70ms/step - loss: 0.8643 - accuracy: 0.7478
Epoch 21/25
25/25 [==============================] - 2s 72ms/step - loss: 0.3979 - accuracy: 0.8800
Epoch 22/25
25/25 [==============================] - 2s 71ms/step - loss: 0.2674 - accuracy: 0.9216
Epoch 23/25
25/25 [==============================] - 2s 69ms/step - loss: 0.3516 - accuracy: 0.9055
Epoch 24/25
25/25 [==============================] - 2s 71ms/step - loss: 0.4831 - accuracy: 0.8848
Epoch 25/25
25/25 [==============================] - 2s 69ms/step - loss: 1.1186 - accuracy: 0.6726
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
2024/12/01 11:00:25 http2: server: error reading preface from client //./pipe/dockerDesktopLinuxEngine: file has already been closed
[+] Building 0.5s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.3s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 3.61kB                                                                                                                                                         0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => CACHED [3/3] COPY /src/tflow.py .                                                                                                                                                       0.0s
 => exporting to image                                                                                                                                                                      0.0s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:f4f33156922489e81eb5136a5c4c6fed49ebea99b774b7386d54c55709dbc4fa                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 16:00:27.140713: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 16:00:27.159562: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 16:00:27.159598: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 16:00:27.159610: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 16:00:27.163423: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 16:00:28.015442: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.015477: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.017879: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.017913: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.017922: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.017930: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.240676: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.240709: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.240721: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.240728: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.240735: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.240756: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.246586: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.246692: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.246775: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.246789: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:00:28.246800: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.246810: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:00:28.246829: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.246851: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 16:00:28.247322: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:00:28.247342: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 4s 0us/step
Epoch 1/25
2024-12-01 16:00:51.598760: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:00:51.610766: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:00:55.443430: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f43daa1c0e0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 16:00:55.443454: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:00:55.443456: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:00:55.447950: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 16:00:55.502469: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
13/13 [==============================] - 30s 261ms/step - loss: 5.4356 - accuracy: 0.0266
Epoch 2/25
13/13 [==============================] - 1s 104ms/step - loss: 4.2198 - accuracy: 0.0711
Epoch 3/25
13/13 [==============================] - 1s 104ms/step - loss: 3.7814 - accuracy: 0.1301
Epoch 4/25
13/13 [==============================] - 1s 104ms/step - loss: 3.4190 - accuracy: 0.1892
Epoch 5/25
13/13 [==============================] - 1s 104ms/step - loss: 3.1379 - accuracy: 0.2454
Epoch 6/25
13/13 [==============================] - 1s 105ms/step - loss: 2.7448 - accuracy: 0.3208
Epoch 7/25
13/13 [==============================] - 1s 106ms/step - loss: 2.3393 - accuracy: 0.4037
Epoch 8/25
13/13 [==============================] - 1s 107ms/step - loss: 2.0334 - accuracy: 0.4686
Epoch 9/25
13/13 [==============================] - 1s 106ms/step - loss: 1.7193 - accuracy: 0.5408
Epoch 10/25
13/13 [==============================] - 1s 108ms/step - loss: 1.4711 - accuracy: 0.6004
Epoch 11/25
13/13 [==============================] - 1s 108ms/step - loss: 1.3995 - accuracy: 0.6207
Epoch 12/25
13/13 [==============================] - 1s 106ms/step - loss: 1.1924 - accuracy: 0.6770
Epoch 13/25
13/13 [==============================] - 1s 107ms/step - loss: 0.9293 - accuracy: 0.7343
Epoch 14/25
13/13 [==============================] - 1s 106ms/step - loss: 0.8229 - accuracy: 0.7577
Epoch 15/25
13/13 [==============================] - 1s 106ms/step - loss: 0.6891 - accuracy: 0.8003
Epoch 16/25
13/13 [==============================] - 1s 108ms/step - loss: 0.5356 - accuracy: 0.8474
Epoch 17/25
13/13 [==============================] - 1s 107ms/step - loss: 0.4675 - accuracy: 0.8623
Epoch 18/25
13/13 [==============================] - 1s 108ms/step - loss: 0.4357 - accuracy: 0.8712
Epoch 19/25
13/13 [==============================] - 1s 108ms/step - loss: 0.3915 - accuracy: 0.8858
Epoch 20/25
13/13 [==============================] - 1s 107ms/step - loss: 0.3640 - accuracy: 0.8940
Epoch 21/25
13/13 [==============================] - 1s 110ms/step - loss: 0.3244 - accuracy: 0.9082
Epoch 22/25
13/13 [==============================] - 1s 108ms/step - loss: 0.2628 - accuracy: 0.9233
Epoch 23/25
13/13 [==============================] - 1s 106ms/step - loss: 0.2220 - accuracy: 0.9379
Epoch 24/25
13/13 [==============================] - 1s 105ms/step - loss: 0.2351 - accuracy: 0.9317
Epoch 25/25
13/13 [==============================] - 1s 107ms/step - loss: 0.2382 - accuracy: 0.9347
(base)

@obriensystems
Copy link
Member Author 

michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
[+] Building 0.7s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.4s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 3.61kB                                                                                                                                                         0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => [3/3] COPY /src/tflow.py .                                                                                                                                                              0.0s
 => exporting to image                                                                                                                                                                      0.1s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:0cd2207ce62b4d32573d9d8b23ee0b523fbbeea0e974741f875d1e4a70872ce1                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 16:02:30.479098: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 16:02:30.497640: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 16:02:30.497673: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 16:02:30.497687: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 16:02:30.501329: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 16:02:31.370282: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.370318: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.372840: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.372872: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.372892: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.372913: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.579979: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.580010: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.580020: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.580026: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.580032: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.580038: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.586200: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.586289: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.586320: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.586325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:02:31.586335: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.586348: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:02:31.586367: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.586384: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 16:02:31.586591: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:02:31.586618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 3s 0us/step
Epoch 1/25
2024-12-01 16:02:54.529471: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:02:54.538881: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:02:58.936839: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7efc0b9b0880 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 16:02:58.936907: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:02:58.936911: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:02:58.940744: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 16:02:58.997247: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
10/10 [==============================] - 32s 513ms/step - loss: 5.7753 - accuracy: 0.0194
Epoch 2/25
10/10 [==============================] - 1s 129ms/step - loss: 4.4960 - accuracy: 0.0447
Epoch 3/25
10/10 [==============================] - 1s 130ms/step - loss: 4.1686 - accuracy: 0.0786
Epoch 4/25
10/10 [==============================] - 1s 130ms/step - loss: 3.8292 - accuracy: 0.1302
Epoch 5/25
10/10 [==============================] - 1s 129ms/step - loss: 3.4992 - accuracy: 0.1812
Epoch 6/25
10/10 [==============================] - 1s 127ms/step - loss: 3.1657 - accuracy: 0.2399
Epoch 7/25
10/10 [==============================] - 1s 127ms/step - loss: 2.8361 - accuracy: 0.2997
Epoch 8/25
10/10 [==============================] - 1s 132ms/step - loss: 2.4991 - accuracy: 0.3713
Epoch 9/25
10/10 [==============================] - 1s 128ms/step - loss: 2.1742 - accuracy: 0.4503
Epoch 10/25
10/10 [==============================] - 1s 127ms/step - loss: 1.7884 - accuracy: 0.5370
Epoch 11/25
10/10 [==============================] - 1s 132ms/step - loss: 1.3445 - accuracy: 0.6414
Epoch 12/25
10/10 [==============================] - 1s 133ms/step - loss: 1.0112 - accuracy: 0.7305
Epoch 13/25
10/10 [==============================] - 1s 132ms/step - loss: 0.7582 - accuracy: 0.8016
Epoch 14/25
10/10 [==============================] - 1s 136ms/step - loss: 0.5313 - accuracy: 0.8633
Epoch 15/25
10/10 [==============================] - 1s 130ms/step - loss: 0.3854 - accuracy: 0.8976
Epoch 16/25
10/10 [==============================] - 1s 132ms/step - loss: 0.3243 - accuracy: 0.9145
Epoch 17/25
10/10 [==============================] - 1s 133ms/step - loss: 0.2753 - accuracy: 0.9284
Epoch 18/25
10/10 [==============================] - 1s 134ms/step - loss: 0.2642 - accuracy: 0.9343
Epoch 19/25
10/10 [==============================] - 1s 135ms/step - loss: 0.2341 - accuracy: 0.9364
Epoch 20/25
10/10 [==============================] - 1s 136ms/step - loss: 0.2305 - accuracy: 0.9413
Epoch 21/25
10/10 [==============================] - 1s 133ms/step - loss: 0.2204 - accuracy: 0.9425
Epoch 22/25
10/10 [==============================] - 1s 130ms/step - loss: 0.2355 - accuracy: 0.9387
Epoch 23/25
10/10 [==============================] - 1s 133ms/step - loss: 0.2615 - accuracy: 0.9259
Epoch 24/25
10/10 [==============================] - 1s 136ms/step - loss: 0.2328 - accuracy: 0.9331
Epoch 25/25
10/10 [==============================] - 1s 129ms/step - loss: 0.2391 - accuracy: 0.9318
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
[+] Building 0.5s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.3s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 3.61kB                                                                                                                                                         0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => CACHED [3/3] COPY /src/tflow.py .                                                                                                                                                       0.0s
 => exporting to image                                                                                                                                                                      0.0s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:e70483c27e438524d264e3228504825e1efeb927ade051c9f91136def3419872                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 16:04:19.567248: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 16:04:19.586226: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 16:04:19.586265: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 16:04:19.586276: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 16:04:19.589987: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 16:04:20.418690: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.418730: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.421115: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.421155: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.421164: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.421172: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.628293: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.628326: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.628337: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.628343: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.628348: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.628354: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.634482: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.634562: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.634585: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.634590: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:04:20.634597: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.634614: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:04:20.634629: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.634651: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 16:04:20.635138: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:04:20.635169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 4s 0us/step
Epoch 1/25
2024-12-01 16:04:43.885582: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:04:43.897847: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:04:48.882113: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f4ea0a77db0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 16:04:48.882137: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:04:48.882140: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:04:48.886119: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 16:04:48.941198: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
9/9 [==============================] - 31s 374ms/step - loss: 5.8951 - accuracy: 0.0202
Epoch 2/25
9/9 [==============================] - 1s 146ms/step - loss: 4.4757 - accuracy: 0.0559
Epoch 3/25
9/9 [==============================] - 1s 151ms/step - loss: 4.0624 - accuracy: 0.0865
Epoch 4/25
9/9 [==============================] - 1s 148ms/step - loss: 3.8697 - accuracy: 0.1175
Epoch 5/25
9/9 [==============================] - 1s 152ms/step - loss: 3.6101 - accuracy: 0.1507
Epoch 6/25
9/9 [==============================] - 1s 148ms/step - loss: 3.4423 - accuracy: 0.1833
Epoch 7/25
9/9 [==============================] - 1s 149ms/step - loss: 3.2824 - accuracy: 0.2127
Epoch 8/25
9/9 [==============================] - 1s 149ms/step - loss: 3.0320 - accuracy: 0.2544
Epoch 9/25
9/9 [==============================] - 1s 145ms/step - loss: 2.7815 - accuracy: 0.3042
Epoch 10/25
9/9 [==============================] - 1s 148ms/step - loss: 2.5588 - accuracy: 0.3532
Epoch 11/25
9/9 [==============================] - 1s 147ms/step - loss: 2.2764 - accuracy: 0.4146
Epoch 12/25
9/9 [==============================] - 1s 150ms/step - loss: 2.0394 - accuracy: 0.4730
Epoch 13/25
9/9 [==============================] - 1s 146ms/step - loss: 1.8234 - accuracy: 0.5244
Epoch 14/25
9/9 [==============================] - 1s 144ms/step - loss: 1.6274 - accuracy: 0.5658
Epoch 15/25
9/9 [==============================] - 1s 147ms/step - loss: 1.4785 - accuracy: 0.6037
Epoch 16/25
9/9 [==============================] - 1s 145ms/step - loss: 1.1880 - accuracy: 0.6725
Epoch 17/25
9/9 [==============================] - 1s 151ms/step - loss: 1.1013 - accuracy: 0.7005
Epoch 18/25
9/9 [==============================] - 1s 152ms/step - loss: 0.8656 - accuracy: 0.7565
Epoch 19/25
9/9 [==============================] - 1s 145ms/step - loss: 0.7259 - accuracy: 0.7908
Epoch 20/25
9/9 [==============================] - 1s 145ms/step - loss: 0.6433 - accuracy: 0.8191
Epoch 21/25
9/9 [==============================] - 1s 150ms/step - loss: 0.5595 - accuracy: 0.8426
Epoch 22/25
9/9 [==============================] - 1s 144ms/step - loss: 0.4944 - accuracy: 0.8572
Epoch 23/25
9/9 [==============================] - 1s 148ms/step - loss: 0.6380 - accuracy: 0.8117
Epoch 24/25
9/9 [==============================] - 1s 151ms/step - loss: 0.5736 - accuracy: 0.8359
Epoch 25/25
9/9 [==============================] - 1s 151ms/step - loss: 0.4493 - accuracy: 0.8720
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
[+] Building 0.7s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.5s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 57B                                                                                                                                                            0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => CACHED [3/3] COPY /src/tflow.py .                                                                                                                                                       0.0s
 => exporting to image                                                                                                                                                                      0.0s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:e70483c27e438524d264e3228504825e1efeb927ade051c9f91136def3419872                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 16:34:11.733611: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 16:34:11.902440: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 16:34:11.902470: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 16:34:11.903623: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 16:34:11.985945: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 16:34:13.647697: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.647734: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.661033: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.661064: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.661072: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.661078: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.955866: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.955899: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.955909: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.955915: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.955920: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.955926: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.962495: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.962584: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.962616: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.962621: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:34:13.962631: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.962644: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:34:13.962663: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.962678: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 16:34:13.963167: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:34:13.963196: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 4s 0us/step
Epoch 1/25
2024-12-01 16:34:37.824731: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:34:37.831294: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:34:43.076845: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f0a74aea0f0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 16:34:43.076871: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:34:43.076874: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:34:43.082821: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 16:34:43.145852: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
9/9 [==============================] - 31s 389ms/step - loss: 5.9035 - accuracy: 0.0201
Epoch 2/25
9/9 [==============================] - 1s 153ms/step - loss: 4.3402 - accuracy: 0.0630
Epoch 3/25
9/9 [==============================] - 1s 151ms/step - loss: 3.9343 - accuracy: 0.1027
Epoch 4/25
9/9 [==============================] - 1s 152ms/step - loss: 3.6462 - accuracy: 0.1482
Epoch 5/25
9/9 [==============================] - 1s 154ms/step - loss: 3.3885 - accuracy: 0.1946
Epoch 6/25
9/9 [==============================] - 1s 150ms/step - loss: 3.1260 - accuracy: 0.2412
Epoch 7/25
9/9 [==============================] - 1s 150ms/step - loss: 2.8548 - accuracy: 0.2933
Epoch 8/25
9/9 [==============================] - 1s 150ms/step - loss: 3.0553 - accuracy: 0.2611
Epoch 9/25
9/9 [==============================] - 1s 147ms/step - loss: 2.8153 - accuracy: 0.3038
Epoch 10/25
9/9 [==============================] - 1s 147ms/step - loss: 2.4767 - accuracy: 0.3735
Epoch 11/25
9/9 [==============================] - 1s 154ms/step - loss: 2.1397 - accuracy: 0.4506
Epoch 12/25
9/9 [==============================] - 1s 142ms/step - loss: 1.8181 - accuracy: 0.5227
Epoch 13/25
9/9 [==============================] - 1s 148ms/step - loss: 1.5448 - accuracy: 0.5874
Epoch 14/25
9/9 [==============================] - 1s 147ms/step - loss: 1.2849 - accuracy: 0.6488
Epoch 15/25
9/9 [==============================] - 1s 154ms/step - loss: 1.0770 - accuracy: 0.7018
Epoch 16/25
9/9 [==============================] - 1s 149ms/step - loss: 0.9445 - accuracy: 0.7369
Epoch 17/25
9/9 [==============================] - 1s 150ms/step - loss: 2.7559 - accuracy: 0.3252
Epoch 18/25
9/9 [==============================] - 1s 151ms/step - loss: 2.8289 - accuracy: 0.3016
Epoch 19/25
9/9 [==============================] - 1s 147ms/step - loss: 2.4519 - accuracy: 0.3773
Epoch 20/25
9/9 [==============================] - 1s 150ms/step - loss: 2.0530 - accuracy: 0.4629
Epoch 21/25
9/9 [==============================] - 1s 147ms/step - loss: 1.6613 - accuracy: 0.5569
Epoch 22/25
9/9 [==============================] - 1s 148ms/step - loss: 1.3448 - accuracy: 0.6379
Epoch 23/25
9/9 [==============================] - 1s 150ms/step - loss: 1.0147 - accuracy: 0.7133
Epoch 24/25
9/9 [==============================] - 1s 147ms/step - loss: 0.7676 - accuracy: 0.7859
Epoch 25/25
9/9 [==============================] - 1s 145ms/step - loss: 0.6042 - accuracy: 0.8318
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
[+] Building 0.6s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.3s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 3.61kB                                                                                                                                                         0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => [3/3] COPY /src/tflow.py .                                                                                                                                                              0.1s
 => exporting to image                                                                                                                                                                      0.1s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:eb166471386d598f9dbac62895e5e8c79b64a9f41c850f957cb9b0f0bc8240d2                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 16:37:23.927066: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 16:37:24.105544: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 16:37:24.105571: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 16:37:24.106557: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 16:37:24.188942: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 16:37:25.820421: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:25.820455: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:25.833223: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:25.833255: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:25.833265: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:25.833273: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.043896: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.043932: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.043945: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.043953: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.043960: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.043981: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.051178: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.051286: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.051320: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.051325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:37:26.051346: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.051351: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:37:26.051379: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.051394: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 16:37:26.051876: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:37:26.051905: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 3s 0us/step
Epoch 1/25
2024-12-01 16:37:48.164957: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:37:48.172077: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:37:54.605328: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f276443f630 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 16:37:54.605350: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:37:54.605353: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:37:54.611412: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 16:37:54.675029: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
7/7 [==============================] - 32s 497ms/step - loss: 5.7852 - accuracy: 0.0162
Epoch 2/25
7/7 [==============================] - 1s 172ms/step - loss: 4.6236 - accuracy: 0.0423
Epoch 3/25
7/7 [==============================] - 1s 175ms/step - loss: 4.2625 - accuracy: 0.0703
Epoch 4/25
7/7 [==============================] - 1s 186ms/step - loss: 3.9680 - accuracy: 0.1016
Epoch 5/25
7/7 [==============================] - 1s 182ms/step - loss: 3.7609 - accuracy: 0.1339
Epoch 6/25
7/7 [==============================] - 1s 177ms/step - loss: 3.5476 - accuracy: 0.1721
Epoch 7/25
7/7 [==============================] - 1s 195ms/step - loss: 3.3920 - accuracy: 0.2017
Epoch 8/25
7/7 [==============================] - 1s 187ms/step - loss: 3.2717 - accuracy: 0.2226
Epoch 9/25
7/7 [==============================] - 1s 182ms/step - loss: 3.1309 - accuracy: 0.2504
Epoch 10/25
7/7 [==============================] - 1s 192ms/step - loss: 2.8985 - accuracy: 0.2951
Epoch 11/25
7/7 [==============================] - 1s 189ms/step - loss: 2.6478 - accuracy: 0.3443
Epoch 12/25
7/7 [==============================] - 1s 180ms/step - loss: 2.7442 - accuracy: 0.3266
Epoch 13/25
7/7 [==============================] - 1s 178ms/step - loss: 2.6449 - accuracy: 0.3435
Epoch 14/25
7/7 [==============================] - 1s 182ms/step - loss: 2.3661 - accuracy: 0.4041
Epoch 15/25
7/7 [==============================] - 1s 181ms/step - loss: 2.1068 - accuracy: 0.4669
Epoch 16/25
7/7 [==============================] - 1s 188ms/step - loss: 1.7842 - accuracy: 0.5330
Epoch 17/25
7/7 [==============================] - 1s 188ms/step - loss: 1.5464 - accuracy: 0.5913
Epoch 18/25
7/7 [==============================] - 1s 181ms/step - loss: 1.8864 - accuracy: 0.5099
Epoch 19/25
7/7 [==============================] - 1s 183ms/step - loss: 1.7786 - accuracy: 0.5323
Epoch 20/25
7/7 [==============================] - 1s 180ms/step - loss: 1.4850 - accuracy: 0.6114
Epoch 21/25
7/7 [==============================] - 1s 183ms/step - loss: 1.1787 - accuracy: 0.6816
Epoch 22/25
7/7 [==============================] - 1s 183ms/step - loss: 1.0279 - accuracy: 0.7200
Epoch 23/25
7/7 [==============================] - 1s 194ms/step - loss: 0.9220 - accuracy: 0.7545
Epoch 24/25
7/7 [==============================] - 1s 181ms/step - loss: 0.8037 - accuracy: 0.7768
Epoch 25/25
7/7 [==============================] - 1s 179ms/step - loss: 0.7027 - accuracy: 0.8091
(base)
michael@13900b MINGW64 /c/wse_github/obrienlabsdev/machine-learning (main)
$ ./build.sh
2024/12/01 11:39:25 http2: server: error reading preface from client //./pipe/dockerDesktopLinuxEngine: file has already been closed
[+] Building 0.8s (8/8) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                        0.0s
 => => transferring dockerfile: 511B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/tensorflow/tensorflow:2.14.0-gpu                                                                                                                 0.5s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                             0.0s
 => [1/3] FROM docker.io/tensorflow/tensorflow:2.14.0-gpu@sha256:64602abcd8cc4f4bdd6268ca0abc39e6d37113d700886afd15f6dd151210b206                                                           0.0s
 => [internal] load build context                                                                                                                                                           0.0s
 => => transferring context: 3.61kB                                                                                                                                                         0.0s
 => CACHED [2/3] WORKDIR /src                                                                                                                                                               0.0s
 => [3/3] COPY /src/tflow.py .                                                                                                                                                              0.1s
 => exporting to image                                                                                                                                                                      0.1s
 => => exporting layers                                                                                                                                                                     0.0s
 => => writing image sha256:8d4ab67fdebb7842e4d6e545cadf444092121196102c7856c1ab2c0b015b6a48                                                                                                0.0s
 => => naming to docker.io/library/ml-tensorflow-win                                                                                                                                        0.0s

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview
2024-12-01 16:39:27.594237: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-01 16:39:27.613693: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-12-01 16:39:27.613733: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-12-01 16:39:27.613747: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-01 16:39:27.617352: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-01 16:39:28.459484: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.459520: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.461976: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.462009: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.462019: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.462027: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.663827: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.663854: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.663864: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.663870: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.663876: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.663882: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.669837: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.669932: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.669974: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.669988: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:39:28.670001: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.670003: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1977] Could not identify NUMA node of platform GPU id 1, defaulting to 0.  Your kernel may not have been built with NUMA support.
2024-12-01 16:39:28.670017: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.670029: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21458 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:01:00.0, compute capability: 8.9
2024-12-01 16:39:28.670524: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:880] could not open file to read NUMA node: /sys/bus/pci/devices/0000:02:00.0/numa_node
Your kernel may have been built without NUMA support.
2024-12-01 16:39:28.670545: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1886] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 21458 MB memory:  -> device: 1, name: NVIDIA GeForce RTX 4090, pci bus id: 0000:02:00.0, compute capability: 8.9
Downloading data from https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz
169001437/169001437 [==============================] - 4s 0us/step
Epoch 1/25
2024-12-01 16:39:52.428385: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:39:52.455445: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:442] Loaded cuDNN version 8600
2024-12-01 16:39:58.788377: W tensorflow/tsl/framework/bfc_allocator.cc:296] Allocator (GPU_0_bfc) ran out of memory trying to allocate 8.02GiB with freed_by_count=0. The caller indicates that this is not a failure, but this may mean that there could be performance gains if more memory were available.
2024-12-01 16:39:58.825028: W tensorflow/tsl/framework/bfc_allocator.cc:296] Allocator (GPU_1_bfc) ran out of memory trying to allocate 8.02GiB with freed_by_count=0. The caller indicates that this is not a failure, but this may mean that there could be performance gains if more memory were available.
2024-12-01 16:40:02.991547: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fc5732a44a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-12-01 16:40:02.991573: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:40:02.991576: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (1): NVIDIA GeForce RTX 4090, Compute Capability 8.9
2024-12-01 16:40:02.995422: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2024-12-01 16:40:03.049758: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
4/4 [==============================] - 36s 897ms/step - loss: 6.2996 - accuracy: 0.0136
Epoch 2/25
4/4 [==============================] - 1s 292ms/step - loss: 4.9917 - accuracy: 0.0237
Epoch 3/25
4/4 [==============================] - 1s 290ms/step - loss: 4.6501 - accuracy: 0.0377
Epoch 4/25
4/4 [==============================] - 1s 307ms/step - loss: 4.4486 - accuracy: 0.0528
Epoch 5/25
4/4 [==============================] - 1s 303ms/step - loss: 4.2761 - accuracy: 0.0654
Epoch 6/25
4/4 [==============================] - 1s 298ms/step - loss: 4.1253 - accuracy: 0.0831
Epoch 7/25
4/4 [==============================] - 1s 287ms/step - loss: 3.9839 - accuracy: 0.1019
Epoch 8/25
4/4 [==============================] - 1s 303ms/step - loss: 3.8343 - accuracy: 0.1208
Epoch 9/25
4/4 [==============================] - 1s 288ms/step - loss: 3.7115 - accuracy: 0.1383
Epoch 10/25
4/4 [==============================] - 1s 314ms/step - loss: 3.5881 - accuracy: 0.1579
Epoch 11/25
4/4 [==============================] - 1s 294ms/step - loss: 3.4612 - accuracy: 0.1790
Epoch 12/25
4/4 [==============================] - 1s 297ms/step - loss: 3.3691 - accuracy: 0.2008
Epoch 13/25
4/4 [==============================] - 1s 314ms/step - loss: 3.2355 - accuracy: 0.2237
Epoch 14/25
4/4 [==============================] - 1s 308ms/step - loss: 3.1157 - accuracy: 0.2506
Epoch 15/25
4/4 [==============================] - 1s 294ms/step - loss: 3.0441 - accuracy: 0.2638
Epoch 16/25
4/4 [==============================] - 1s 297ms/step - loss: 2.9439 - accuracy: 0.2823
Epoch 17/25
4/4 [==============================] - 1s 302ms/step - loss: 2.8458 - accuracy: 0.3030
Epoch 18/25
4/4 [==============================] - 1s 316ms/step - loss: 2.7565 - accuracy: 0.3259
Epoch 19/25
4/4 [==============================] - 1s 290ms/step - loss: 2.6267 - accuracy: 0.3444
Epoch 20/25
4/4 [==============================] - 1s 299ms/step - loss: 2.5415 - accuracy: 0.3639
Epoch 21/25
4/4 [==============================] - 1s 329ms/step - loss: 2.5037 - accuracy: 0.3837
Epoch 22/25
4/4 [==============================] - 1s 283ms/step - loss: 2.3973 - accuracy: 0.4048
Epoch 23/25
4/4 [==============================] - 1s 293ms/step - loss: 2.3483 - accuracy: 0.4067
Epoch 24/25
4/4 [==============================] - 1s 286ms/step - loss: 2.3516 - accuracy: 0.4097
Epoch 25/25
4/4 [==============================] - 1s 304ms/step - loss: 2.2700 - accuracy: 0.4228

This was referenced Dec 1, 2024
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@obriensystems obriensystems

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@obriensystems