[bitnami/spark] Spark-UI stages and other tabs not accessible when reverse-proxy is enabled

[imsharadmishra]

Name and Version

bitnami/spark - 6.0.0

What architecture are you using?

None

What steps will reproduce the bug?

Kubernetes Version: 1.25
Spark: 3.4.2
Helm chart: 6.0.0

When enabled reverse proxy in master and worker configOptions. We're not able to access different tabs available in spark UI e.g.(stages, environment, storage etc.)

Steps to reproduce:
After installing the chart Spark Cluster(Master and worker) UI is available at:
https://spark.staging.abc.com/

We are able to access running application by click on applicationID under Running Applications link:

We can access spark UI by clicking Application Detail UI:

We are taken to jobs tab when we click on Application Detail UI
URL looks like: https://spark.staging.abc.com/proxy/app-20240208103209-0030/stages/

When we click any of the tab from spark UI e.g. stages or environment etc, it takes us back to spark cluster UI page

We noticed that endpoint changes to
https://spark.staging.abc.com/stages/
instead of
https://spark.staging.abc.com/proxy/app-20240208103209-0030/stages/

Are you using any custom parameters or values?

Configurations set in values.yaml

master:
  configOptions:
    -Dspark.ui.reverseProxy=true
    -Dspark.ui.reverseProxyUrl=https://spark.staging.abc.com

worker:
  configOptions:
    -Dspark.ui.reverseProxy=true
    -Dspark.ui.reverseProxyUrl=https://spark.staging.abc.com

service:
  type: ClusterIP
  ports:
    http: 8080
    https: 443
    cluster: 7077

ingress:

  enabled: true
  pathType: ImplementationSpecific
  apiVersion: ""
  hostname: spark.staging.abc.com
  ingressClassName: "staging"
  path: /

What is the expected behavior?

Expected behaviour is that when I click on stages tab, instead of taking me to
https://spark.staging.abc.com/stages/
it should take me to following URL:
https://spark.staging.abc.com/proxy/app-20240208103209-0030/stages/

What do you see instead?

current behaviour is it takes me to URL: https://spark.staging.abc.com/stages/ , which shows spark cluster UI with master and worker details

Additional information

No response

[javsalgar]

Hi,

Looking at the issue, it seems that you were able to set the proper Spark settings, but these do not work as expected. I'm not sure this is an issue with the Bitnami packaging of Spark but in Spark configuration itself. Did you check with the upstream devs?

[imsharadmishra]

Hi @javsalgar , No I haven't checked with upstream devs, wanted to make sure, I'm setting it up properly and not missing any configuration, which is causing it. One other observation that I noticed is that once application is completed/killed and available in spark-history server, all the URL tabs work as expected.

[javsalgar]

That's strange. That leads me to think that the issue may be in Spark, so my advice would be to open a ticket in upstream.

[github-actions]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[imsharadmishra]

opened a ticket with Spark: https://issues.apache.org/jira/browse/SPARK-47232

[github-actions]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[github-actions]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

[vlieftinkmrdm]

I ran into the same problem, but I think I know what's happening here.
You need to make sure that the driver ( the client that actually submits the job ) also has the proxy settings applied to their Spark-config:

spark.ui.reverseProxy true
spark.ui.reverseProxyUrl https://spark.staging.abc.com

It's not a Apache Spark nor a Bitnami problem.

[javsalgar]

Thanks for letting us know!

[haarvester]

Hi All. The issue still persist same as @imsharadmishra.
I set
spark.ui.reverseProxy true
spark.ui.reverseProxyUrl https://my.domain.com But paths /stages /executors and etc. still redirect me to main details page