Skip to content

Commit

Permalink
[KYUUBI #6653] Persist spark pod name conf into batch metadata
Browse files Browse the repository at this point in the history
# 🔍 Description
## Issue References 🔗

As title, save the pod name conf into batch metadata, it is very meaningful.
## Describe Your Solution 🔧
For spark on k8s batch app, persist the spark pod name configs into batch metadata.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6653 from turboFei/pod_name_conf.

Closes #6653

2bfa0aa [Wang, Fei] save spark pod name conf into metadata

Authored-by: Wang, Fei <[email protected]>
Signed-off-by: Wang, Fei <[email protected]>
  • Loading branch information
turboFei committed Sep 2, 2024
1 parent e1e7772 commit dff40e8
Showing 1 changed file with 4 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,10 @@ class KyuubiBatchSession(
Map(KyuubiConf.KUBERNETES_CONTEXT.key -> context)
}.getOrElse(Map.empty) ++ appMgrInfo.kubernetesInfo.namespace.map { namespace =>
Map(KyuubiConf.KUBERNETES_NAMESPACE.key -> namespace)
}.getOrElse(Map.empty)
}.getOrElse(Map.empty) ++ (batchJobSubmissionOp.builder match {
case builder: SparkProcessBuilder => builder.appendPodNameConf(optimizedConf)
case _ => Map.empty[String, String]
})
}

(metadata, fromRecovery) match {
Expand Down

0 comments on commit dff40e8

Please sign in to comment.