Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: [sc-216078] [Google Drive] Add option to retrieve googlesheets as xlsx files #12

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

feat: [sc-216078] [Google Drive] Add option to retrieve googlesheets as xlsx files #12

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
cdea354
Add option to retrieve google sheets as xlsx documents
alexbourret Nov 25, 2024
43e7242
update changelog
alexbourret Nov 25, 2024
0afb1ac
version update
alexbourret Nov 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

## [Version 1.1.5](https://github.com/dataiku/dss-plugin-googledrive/releases/tag/v1.1.5) - Fix release - 2024-11-25

- Add option to import GoogleSheets document as xlsx instead of csv file, allowing import of multitab worksheets

## [Version 1.1.4](https://github.com/dataiku/dss-plugin-googledrive/releases/tag/v1.1.4) - Fix release - 2022-11-23

- Fix SSO for Google WebApps
Expand Down
2 changes: 1 addition & 1 deletion plugin.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"id": "googledrive",
"version": "1.1.4",
"version": "1.1.5",
"meta": {
"label": "Google Drive",
"description": "Read and write data from/to your Google Drive account",
Expand Down
6 changes: 6 additions & 0 deletions python-fs-providers/googledrive-googledrive/fs-provider.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,12 @@
"description": "Found after the last / of the shared folder's URL",
"type": "STRING"
},
{
"name": "output_google_sheets_as_xlsx",
"label": "Read parameter",
"description": "Convert Google Sheets to .xlsx",
"type": "BOOLEAN"
},
{
"name": "googledrive_write_as_google_doc",
"label": "Write parameter",
Expand Down
14 changes: 12 additions & 2 deletions python-lib/dku_googledrive/googledrive_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ class GoogleDriveUtils(object):
SPREADSHEET = "application/vnd.google-apps.spreadsheet"
GOOGLE_DOCUMENT = "application/vnd.google-apps.document"
CSV = "text/csv"
XLSX = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
GOOGLE_APPS = "google-apps"
BINARY_STREAM = "binary/octet-stream"
LIST_FIELDS = "nextPageToken, files(id, name, size, parents, mimeType, createdTime, modifiedTime)"
Expand All @@ -34,6 +35,12 @@ class GoogleDriveUtils(object):
"application/vnd.google-apps.drawing": "image/svg+xml",
"application/vnd.google-apps.presentation": "application/vnd.openxmlformats-officedocument.presentationml.presentation"
}
GOOGLE_DOC_MIME_EQUIVALENCE_MULTISHEETS = {
SPREADSHEET: XLSX,
GOOGLE_DOCUMENT: "text/plain",
"application/vnd.google-apps.drawing": "image/svg+xml",
"application/vnd.google-apps.presentation": "application/vnd.openxmlformats-officedocument.presentationml.presentation"
}
DEFAULT_MIME_TYPE = CSV

@staticmethod
Expand Down Expand Up @@ -116,8 +123,11 @@ def get_google_doc_type(file):
return file[GoogleDriveUtils.MIME_TYPE]

@staticmethod
def get_google_doc_mime_equivalence(gdoc_type):
return GoogleDriveUtils.GOOGLE_DOC_MIME_EQUIVALENCE.get(gdoc_type, GoogleDriveUtils.DEFAULT_MIME_TYPE)
def get_google_doc_mime_equivalence(gdoc_type, output_google_sheets_as_xlsx):
if output_google_sheets_as_xlsx:
return GoogleDriveUtils.GOOGLE_DOC_MIME_EQUIVALENCE_MULTISHEETS.get(gdoc_type, GoogleDriveUtils.DEFAULT_MIME_TYPE)
else:
return GoogleDriveUtils.GOOGLE_DOC_MIME_EQUIVALENCE.get(gdoc_type, GoogleDriveUtils.DEFAULT_MIME_TYPE)

@staticmethod
def file_size(item):
Expand Down
6 changes: 5 additions & 1 deletion python-lib/dku_googledrive/session.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ def __init__(self, config, plugin_config):
connection = plugin_config.get("googledrive_connection")
self.auth_type = config.get("auth_type")
self.write_as_google_doc = config.get("googledrive_write_as_google_doc")
self.output_google_sheets_as_xlsx = config.get("output_google_sheets_as_xlsx", False)
self.nodir_mode = False # Future development

if self.auth_type == "oauth":
Expand Down Expand Up @@ -94,7 +95,10 @@ def googledrive_download(self, item, stream):
document_type = gdu.get_google_doc_type(item)
data = self.drive.files().export_media(
fileId=gdu.get_id(item),
mimeType=gdu.get_google_doc_mime_equivalence(document_type)
mimeType=gdu.get_google_doc_mime_equivalence(
document_type,
self.output_google_sheets_as_xlsx
)
).execute()
file_handle = BytesIO()
file_handle.write(data)
Expand Down