Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop SparkSession In Object Metorikku #425

Open
drinks5 opened this issue May 28, 2021 · 2 comments
Open

Stop SparkSession In Object Metorikku #425

drinks5 opened this issue May 28, 2021 · 2 comments

Comments

@drinks5
Copy link

drinks5 commented May 28, 2021

Should we stop the sparksession?
when I submit the command in yarn,
spark-submit --class com.yotpo.metorikku.Metorikku --master yarn --deploy-mode client ~/metorikku/metorikku.jar -c examples/movies.yaml

metorikku app has finished the job but still wait to close the sparksession with a loop, log shows as follow,
image

when I stop the sparkSession, the above problem solved
session.sparkSession.stop()

@lyogev
Copy link
Contributor

lyogev commented Jun 2, 2021

@drinks5
Copy link
Author

drinks5 commented Jun 5, 2021

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants