Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to open log file: open("/databricks/driver/hyperd.log", 66): Permission denied #64

Open
harrywatts-hov opened this issue Dec 30, 2020 · 0 comments

Comments

@harrywatts-hov
Copy link

Appears to try to access databricks root directory during hyper extract creation, which is denied.
Example below:

"query" is accessing a delta table in Databricks Hive

hf = HyperFile("TPCH_Extract", query, is_dbfs_enabled=True, creation_mode='copy')


HyperException Traceback (most recent call last)
HyperException: The Hyper server process exited during startup with exit code: 1
Command-line: "/databricks/python/lib/python3.7/site-packages/tableauhyperapi/bin/hyper/hyperd run --date-style=MDY --date-style-lenient=false --init-user=tableau_internal_user --language=en_US --log-config=file,json,all,hyperd,0 --log-dir=/databricks/driver --no-password=true --skip-license=true --listen-connection tab.domain:///tmp/domain/auto --callback-connection tab.domain:///tmp/domain/31f6c5fb534749aa9c43e8b4afc17f98"
Child process' stderr:
Unable to open log file: open("/databricks/driver/hyperd.log", 66): Permission denied
FileListener in error state after initial rotate()

Context: 0x13cead20

The above exception was the direct cause of the following exception:

HyperException Traceback (most recent call last)
in
3 # Step 1: Create a Hyper File from Spark SQL
4
----> 5 hf = HyperFile("TPCH_Extract", query, is_dbfs_enabled=True, creation_mode='copy')

/local_disk0/spark-bfb001e1-256e-4232-a0ea-1d515d18a094/userFiles-5f9eb004-70b6-4fdd-a74b-097f172c405f/addedFile76068253217714208702c2c0_09f1_4d07_a0cf_5f1648f667c2_hyperleaup_0_0_5_py3_7_a9c46-53e02.egg/hyperleaup/hyper_file.py in init(self, name, sql, df, is_dbfs_enabled, creation_mode, null_values_replacement)
42 self.is_dbfs_enabled,
43 self.creation_mode,
---> 44 self.null_values_replacement).create()
45 self.luid = None
46

/local_disk0/spark-bfb001e1-256e-4232-a0ea-1d515d18a094/userFiles-5f9eb004-70b6-4fdd-a74b-097f172c405f/addedFile76068253217714208702c2c0_09f1_4d07_a0cf_5f1648f667c2_hyperleaup_0_0_5_py3_7_a9c46-53e02.egg/hyperleaup/creator.py in create(self)
217 # COPY data into a Tableau .hyper file
218 logging.info("Copying data into Hyper File...")
--> 219 database_path = copy_data_into_hyper_file(csv_path, self.name, table_def)
220
221 elif self.creation_mode.upper() == CreationMode.INSERT.value:

/local_disk0/spark-bfb001e1-256e-4232-a0ea-1d515d18a094/userFiles-5f9eb004-70b6-4fdd-a74b-097f172c405f/addedFile76068253217714208702c2c0_09f1_4d07_a0cf_5f1648f667c2_hyperleaup_0_0_5_py3_7_a9c46-53e02.egg/hyperleaup/creator.py in copy_data_into_hyper_file(csv_path, name, table_def)
117 """Helper function that copies data from a CSV file to a .hyper file."""
118 hyper_database_path = f"/tmp/hyperleaup/{name}/{name}.hyper"
--> 119 with HyperProcess(telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU) as hp:
120 with Connection(endpoint=hp.endpoint,
121 database=Path(hyper_database_path),

/databricks/python/lib/python3.7/site-packages/tableauhyperapi/hyperprocess.py in init(self, telemetry, user_agent, hyper_path, parameters)
90 Error.check(hapi.hyper_instance_create(InteropUtil.string_to_char_p(hyper_path),
91 telemetry == Telemetry.SEND_USAGE_DATA_TO_TABLEAU,
---> 92 native_params.cdata, pp))
93 self.__cdata = pp[0]
94

/databricks/python/lib/python3.7/site-packages/tableauhyperapi/impl/dllutil.py in check(p)
108 # this will free the error when it goes out of scope
109 errp = Error(p)
--> 110 raise errp.to_exception()
111
112 @staticmethod

HyperException: Failed to start a new Hyper instance.
Context: 0x86a93465

Caused by:
The Hyper server process exited during startup with exit code: 1
Command-line: "/databricks/python/lib/python3.7/site-packages/tableauhyperapi/bin/hyper/hyperd run --date-style=MDY --date-style-lenient=false --init-user=tableau_internal_user --language=en_US --log-config=file,json,all,hyperd,0 --log-dir=/databricks/driver --no-password=true --skip-license=true --listen-connection tab.domain:///tmp/domain/auto --callback-connection tab.domain:///tmp/domain/31f6c5fb534749aa9c43e8b4afc17f98"
Child process' stderr:
Unable to open log file: open("/databricks/driver/hyperd.log", 66): Permission denied
FileListener in error state after initial rotate()

Context: 0x13cead20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant