From 07c0131068fba8b1393d09834b9e269cd24a65a2 Mon Sep 17 00:00:00 2001 From: Spinachboul Date: Thu, 21 Dec 2023 14:17:04 +0530 Subject: [PATCH 01/12] Improved GEKPLS Function --- docs/pages.jl | 1 + docs/src/Improvedgekpls.md | 56 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) create mode 100644 docs/src/Improvedgekpls.md diff --git a/docs/pages.jl b/docs/pages.jl index 24d4d3a7..4d00f6c4 100644 --- a/docs/pages.jl +++ b/docs/pages.jl @@ -15,6 +15,7 @@ pages = ["index.md" "Variable Fidelity" => "variablefidelity.md", "Gradient Enhanced Kriging" => "gek.md", "GEKPLS" => "gekpls.md", + "Improved GEKPLS" => "Improvedgekpls.md" "MOE" => "moe.md", "Parallel Optimization" => "parallel.md", ] diff --git a/docs/src/Improvedgekpls.md b/docs/src/Improvedgekpls.md new file mode 100644 index 00000000..195c79bf --- /dev/null +++ b/docs/src/Improvedgekpls.md @@ -0,0 +1,56 @@ +# GEKPLS Function + +Gradient Enhanced Kriging with Partial Least Squares Method (GEKPLS) is a surrogate modelling technique that brings down computation time and returns improved accuracy for high-dimensional problems. The Julia implementation of GEKPLS is adapted from the Python version by [SMT](https://github.com/SMTorg) which is based on this [paper](https://arxiv.org/pdf/1708.02663.pdf). + +# Modifications for Improved GEKPLS Function: + +To enhance the GEKPLS function, sampling method was changed from ```SobolSample()``` to ```HaltonSample()```. + + +```@example gekpls_water_flow + +using Surrogates +using Zygote + +function water_flow(x) + r_w = x[1] + r = x[2] + T_u = x[3] + H_u = x[4] + T_l = x[5] + H_l = x[6] + L = x[7] + K_w = x[8] + log_val = log(r/r_w) + return (2*pi*T_u*(H_u - H_l))/ ( log_val*(1 + (2*L*T_u/(log_val*r_w^2*K_w)) + T_u/T_l)) +end + +n = 1000 +lb = [0.05,100,63070,990,63.1,700,1120,9855] +ub = [0.15,50000,115600,1110,116,820,1680,12045] +x = sample(n,lb,ub,HaltonSample()) +grads = gradient.(water_flow, x) +y = water_flow.(x) +n_test = 100 +x_test = sample(n_test,lb,ub,GoldenSample()) +y_true = water_flow.(x_test) +n_comp = 2 +delta_x = 0.0001 +extra_points = 2 +initial_theta = [0.01 for i in 1:n_comp] +g = GEKPLS(x, y, grads, n_comp, delta_x, lb, ub, extra_points, initial_theta) +y_pred = g.(x_test) +rmse = sqrt(sum(((y_pred - y_true).^2)/n_test)) #root mean squared error +println(rmse) #0.0347 +``` + +
+
+ + + +| **Sampling Method** | **RMSE** | **Differences** | +|----------------------|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **Sobol Sampling** | 0.021472963465423097 | Utilizes digital nets to generate quasi-random numbers, offering low-discrepancy points for improved coverage. - Requires careful handling, especially in higher dimensions. | +| **Halton Sampling** | 0.02144270998045834 | Uses a deterministic sequence based on prime numbers to generate points, allowing for quasi-random, low-discrepancy sampling. - Simpler to implement but may exhibit correlations in some dimensions affecting coverage. | + From 4358bee565c1e98406d8883492fb41170af37dd5 Mon Sep 17 00:00:00 2001 From: MRIDUL JAIN <105979087+Spinachboul@users.noreply.github.com> Date: Thu, 21 Dec 2023 15:26:04 +0530 Subject: [PATCH 02/12] Update Improvedgekpls.md --- docs/src/Improvedgekpls.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/src/Improvedgekpls.md b/docs/src/Improvedgekpls.md index 195c79bf..52ac943e 100644 --- a/docs/src/Improvedgekpls.md +++ b/docs/src/Improvedgekpls.md @@ -51,6 +51,5 @@ println(rmse) #0.0347 | **Sampling Method** | **RMSE** | **Differences** | |----------------------|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| **Sobol Sampling** | 0.021472963465423097 | Utilizes digital nets to generate quasi-random numbers, offering low-discrepancy points for improved coverage. - Requires careful handling, especially in higher dimensions. | -| **Halton Sampling** | 0.02144270998045834 | Uses a deterministic sequence based on prime numbers to generate points, allowing for quasi-random, low-discrepancy sampling. - Simpler to implement but may exhibit correlations in some dimensions affecting coverage. | - +| **Sobol Sampling** | 0.021472963465423097 | Utilizes digital nets to generate quasi-random numbers, offering low discrepancy points for improved coverage. - Requires careful handling, especially in higher dimensions. | +| **Halton Sampling** | 0.02144270998045834 | Uses a deterministic sequence based on prime numbers to generate points, allowing for quasi-random, low-discrepancy sampling. - Simpler to implement but may exhibit correlations in some dimensions affecting coverage. From ef4c12b6d3f720cd216aa070f7d333f5bb93c793 Mon Sep 17 00:00:00 2001 From: MRIDUL JAIN <105979087+Spinachboul@users.noreply.github.com> Date: Thu, 21 Dec 2023 15:26:52 +0530 Subject: [PATCH 03/12] Update Improvedgekpls.md From da68799586b94d9849c6a09c064f46675fb7aca2 Mon Sep 17 00:00:00 2001 From: MRIDUL JAIN <105979087+Spinachboul@users.noreply.github.com> Date: Thu, 21 Dec 2023 15:55:44 +0530 Subject: [PATCH 04/12] Update pages.jl --- docs/pages.jl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/pages.jl b/docs/pages.jl index 4d00f6c4..455a9702 100644 --- a/docs/pages.jl +++ b/docs/pages.jl @@ -15,7 +15,7 @@ pages = ["index.md" "Variable Fidelity" => "variablefidelity.md", "Gradient Enhanced Kriging" => "gek.md", "GEKPLS" => "gekpls.md", - "Improved GEKPLS" => "Improvedgekpls.md" + "Improved GEKPLS" => "Improvedgekpls.md", "MOE" => "moe.md", "Parallel Optimization" => "parallel.md", ] From 2e519c996098e35890dfb459fa067491d3f80c11 Mon Sep 17 00:00:00 2001 From: MRIDUL JAIN <105979087+Spinachboul@users.noreply.github.com> Date: Thu, 28 Dec 2023 16:51:00 +0530 Subject: [PATCH 05/12] Update tensor_prod.md --- docs/src/tensor_prod.md | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/docs/src/tensor_prod.md b/docs/src/tensor_prod.md index 832603cc..df89333a 100644 --- a/docs/src/tensor_prod.md +++ b/docs/src/tensor_prod.md @@ -1,6 +1,9 @@ # Tensor product function -The tensor product function is defined as: -``f(x) = \prod_{i=1}^d \cos(a\pi x_i)`` +A tensor product function combines multiple functions or vectors using the tensor product operation. The tensor product is a mathematical operation that takes two vectors and produces another vector space, capturing their joint behavior across multiple dimensions. + +For instance, consider a tensor product function defined as follows: + +```\[ f(x) = ∏ᵢ=₁ᵈ cos(aπxᵢ) \]``` Let's import Surrogates and Plots: ```@example tensor @@ -38,3 +41,16 @@ plot!(xs,f.(xs), label="True function", legend=:top) plot!(xs, loba_1.(xs), label="Lobachevsky", legend=:top) plot!(xs, krig.(xs), label="Kriging", legend=:top) ``` + +## Kriging Plot + +![kriging](https://github.com/Spinachboul/Surrogates.jl/assets/105979087/906e6688-db47-48be-90d1-ea471aacac16) + +## Lobachevsky Plot + +![lobachevsky](https://github.com/Spinachboul/Surrogates.jl/assets/105979087/678cfc13-0aec-4488-8e4d-39649853ecdd) + +## Combined Plot + +![combined_plot](https://github.com/Spinachboul/Surrogates.jl/assets/105979087/46762f0d-50c5-4d6c-961a-236fd9fb3ad5) + From c54684073bc168acd229cf7ad8bf137b7ca64c49 Mon Sep 17 00:00:00 2001 From: MRIDUL JAIN <105979087+Spinachboul@users.noreply.github.com> Date: Sun, 31 Dec 2023 15:40:34 +0530 Subject: [PATCH 06/12] Update docs/src/tensor_prod.md Co-authored-by: Sathvik Bhagavan <35105271+sathvikbhagavan@users.noreply.github.com> --- docs/src/tensor_prod.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/tensor_prod.md b/docs/src/tensor_prod.md index df89333a..53e5b6b7 100644 --- a/docs/src/tensor_prod.md +++ b/docs/src/tensor_prod.md @@ -1,5 +1,5 @@ # Tensor product function -A tensor product function combines multiple functions or vectors using the tensor product operation. The tensor product is a mathematical operation that takes two vectors and produces another vector space, capturing their joint behavior across multiple dimensions. +A tensor product combines multiple functions or vectors. It is a mathematical operation that takes two vector spaces and produces another vector space, capturing their joint behavior across multiple dimensions. For instance, consider a tensor product function defined as follows: From dc946d4e2e43c8ce224842a567b54d253c955753 Mon Sep 17 00:00:00 2001 From: MRIDUL JAIN <105979087+Spinachboul@users.noreply.github.com> Date: Sun, 31 Dec 2023 17:20:58 +0530 Subject: [PATCH 07/12] Update pages.jl --- docs/pages.jl | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/pages.jl b/docs/pages.jl index 455a9702..24d4d3a7 100644 --- a/docs/pages.jl +++ b/docs/pages.jl @@ -15,7 +15,6 @@ pages = ["index.md" "Variable Fidelity" => "variablefidelity.md", "Gradient Enhanced Kriging" => "gek.md", "GEKPLS" => "gekpls.md", - "Improved GEKPLS" => "Improvedgekpls.md", "MOE" => "moe.md", "Parallel Optimization" => "parallel.md", ] From 63a72d3d021a9d891e88670a406d7e8daa443203 Mon Sep 17 00:00:00 2001 From: MRIDUL JAIN <105979087+Spinachboul@users.noreply.github.com> Date: Thu, 4 Jan 2024 13:18:02 +0530 Subject: [PATCH 08/12] Update tensor_prod.md --- docs/src/tensor_prod.md | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-) diff --git a/docs/src/tensor_prod.md b/docs/src/tensor_prod.md index 53e5b6b7..2cd30afd 100644 --- a/docs/src/tensor_prod.md +++ b/docs/src/tensor_prod.md @@ -1,9 +1,6 @@ # Tensor product function -A tensor product combines multiple functions or vectors. It is a mathematical operation that takes two vector spaces and produces another vector space, capturing their joint behavior across multiple dimensions. -For instance, consider a tensor product function defined as follows: - -```\[ f(x) = ∏ᵢ=₁ᵈ cos(aπxᵢ) \]``` +The tensor product function is defined as: ```\[ f(x) = ∏ᵢ=₁ᵈ cos(aπxᵢ) \]``` Let's import Surrogates and Plots: ```@example tensor @@ -42,15 +39,3 @@ plot!(xs, loba_1.(xs), label="Lobachevsky", legend=:top) plot!(xs, krig.(xs), label="Kriging", legend=:top) ``` -## Kriging Plot - -![kriging](https://github.com/Spinachboul/Surrogates.jl/assets/105979087/906e6688-db47-48be-90d1-ea471aacac16) - -## Lobachevsky Plot - -![lobachevsky](https://github.com/Spinachboul/Surrogates.jl/assets/105979087/678cfc13-0aec-4488-8e4d-39649853ecdd) - -## Combined Plot - -![combined_plot](https://github.com/Spinachboul/Surrogates.jl/assets/105979087/46762f0d-50c5-4d6c-961a-236fd9fb3ad5) - From 670ecc77461becb54e67a656b5209495be796fcd Mon Sep 17 00:00:00 2001 From: MRIDUL JAIN <105979087+Spinachboul@users.noreply.github.com> Date: Thu, 4 Jan 2024 13:18:41 +0530 Subject: [PATCH 09/12] Update tensor_prod.md --- docs/src/tensor_prod.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/src/tensor_prod.md b/docs/src/tensor_prod.md index 2cd30afd..d2b0e114 100644 --- a/docs/src/tensor_prod.md +++ b/docs/src/tensor_prod.md @@ -1,5 +1,4 @@ # Tensor product function - The tensor product function is defined as: ```\[ f(x) = ∏ᵢ=₁ᵈ cos(aπxᵢ) \]``` Let's import Surrogates and Plots: From 02aabfbbd21746a90f444bd513a79d9bf74bf32d Mon Sep 17 00:00:00 2001 From: MRIDUL JAIN <105979087+Spinachboul@users.noreply.github.com> Date: Thu, 4 Jan 2024 13:20:12 +0530 Subject: [PATCH 10/12] Update Improvedgekpls.md --- docs/src/Improvedgekpls.md | 19 +------------------ 1 file changed, 1 insertion(+), 18 deletions(-) diff --git a/docs/src/Improvedgekpls.md b/docs/src/Improvedgekpls.md index 52ac943e..df0ce0d5 100644 --- a/docs/src/Improvedgekpls.md +++ b/docs/src/Improvedgekpls.md @@ -1,12 +1,5 @@ # GEKPLS Function -Gradient Enhanced Kriging with Partial Least Squares Method (GEKPLS) is a surrogate modelling technique that brings down computation time and returns improved accuracy for high-dimensional problems. The Julia implementation of GEKPLS is adapted from the Python version by [SMT](https://github.com/SMTorg) which is based on this [paper](https://arxiv.org/pdf/1708.02663.pdf). - -# Modifications for Improved GEKPLS Function: - -To enhance the GEKPLS function, sampling method was changed from ```SobolSample()``` to ```HaltonSample()```. - - ```@example gekpls_water_flow using Surrogates @@ -28,7 +21,7 @@ end n = 1000 lb = [0.05,100,63070,990,63.1,700,1120,9855] ub = [0.15,50000,115600,1110,116,820,1680,12045] -x = sample(n,lb,ub,HaltonSample()) +x = sample(n,lb,ub,SobolSample()) grads = gradient.(water_flow, x) y = water_flow.(x) n_test = 100 @@ -43,13 +36,3 @@ y_pred = g.(x_test) rmse = sqrt(sum(((y_pred - y_true).^2)/n_test)) #root mean squared error println(rmse) #0.0347 ``` - -
-
- - - -| **Sampling Method** | **RMSE** | **Differences** | -|----------------------|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| **Sobol Sampling** | 0.021472963465423097 | Utilizes digital nets to generate quasi-random numbers, offering low discrepancy points for improved coverage. - Requires careful handling, especially in higher dimensions. | -| **Halton Sampling** | 0.02144270998045834 | Uses a deterministic sequence based on prime numbers to generate points, allowing for quasi-random, low-discrepancy sampling. - Simpler to implement but may exhibit correlations in some dimensions affecting coverage. From fa8164a0514b38d62a5c13d49cf05978cf8714c0 Mon Sep 17 00:00:00 2001 From: MRIDUL JAIN <105979087+Spinachboul@users.noreply.github.com> Date: Thu, 4 Jan 2024 15:56:11 +0530 Subject: [PATCH 11/12] Delete docs/src/Improvedgekpls.md --- docs/src/Improvedgekpls.md | 38 -------------------------------------- 1 file changed, 38 deletions(-) delete mode 100644 docs/src/Improvedgekpls.md diff --git a/docs/src/Improvedgekpls.md b/docs/src/Improvedgekpls.md deleted file mode 100644 index df0ce0d5..00000000 --- a/docs/src/Improvedgekpls.md +++ /dev/null @@ -1,38 +0,0 @@ -# GEKPLS Function - -```@example gekpls_water_flow - -using Surrogates -using Zygote - -function water_flow(x) - r_w = x[1] - r = x[2] - T_u = x[3] - H_u = x[4] - T_l = x[5] - H_l = x[6] - L = x[7] - K_w = x[8] - log_val = log(r/r_w) - return (2*pi*T_u*(H_u - H_l))/ ( log_val*(1 + (2*L*T_u/(log_val*r_w^2*K_w)) + T_u/T_l)) -end - -n = 1000 -lb = [0.05,100,63070,990,63.1,700,1120,9855] -ub = [0.15,50000,115600,1110,116,820,1680,12045] -x = sample(n,lb,ub,SobolSample()) -grads = gradient.(water_flow, x) -y = water_flow.(x) -n_test = 100 -x_test = sample(n_test,lb,ub,GoldenSample()) -y_true = water_flow.(x_test) -n_comp = 2 -delta_x = 0.0001 -extra_points = 2 -initial_theta = [0.01 for i in 1:n_comp] -g = GEKPLS(x, y, grads, n_comp, delta_x, lb, ub, extra_points, initial_theta) -y_pred = g.(x_test) -rmse = sqrt(sum(((y_pred - y_true).^2)/n_test)) #root mean squared error -println(rmse) #0.0347 -``` From 688fee1c3784a940b38de40b86053e2b98c32b78 Mon Sep 17 00:00:00 2001 From: MRIDUL JAIN <105979087+Spinachboul@users.noreply.github.com> Date: Fri, 5 Jan 2024 16:07:50 +0530 Subject: [PATCH 12/12] Update tensor_prod.md --- docs/src/tensor_prod.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/src/tensor_prod.md b/docs/src/tensor_prod.md index d2b0e114..70d74d7d 100644 --- a/docs/src/tensor_prod.md +++ b/docs/src/tensor_prod.md @@ -1,5 +1,6 @@ # Tensor product function -The tensor product function is defined as: ```\[ f(x) = ∏ᵢ=₁ᵈ cos(aπxᵢ) \]``` +The tensor product function is defined as: +```\[ f(x) = ∏ᵢ=₁ᵈ cos(aπxᵢ) \]``` Let's import Surrogates and Plots: ```@example tensor @@ -37,4 +38,3 @@ plot!(xs,f.(xs), label="True function", legend=:top) plot!(xs, loba_1.(xs), label="Lobachevsky", legend=:top) plot!(xs, krig.(xs), label="Kriging", legend=:top) ``` -