Use REDASH_CSV_WRITER_ENCODING for downloading CSV #5826
Conversation
redash/utils/__init__.py
Outdated
| @@ -135,42 +135,6 @@ def build_url(request, host, path): | |||
| return "{}://{}{}".format(request.scheme, host, path) | |||
|
|
|||
|
|
|||
| class UnicodeWriter: | |||
There was a problem hiding this comment.
UnicodeWriter no longer appears to be in use.
redash/handlers/query_results.py
Outdated
| @@ -37,6 +38,7 @@ | |||
| serialize_job, | |||
| ) | |||
|
|
|||
| WRITER_ENCODING = os.environ.get("REDASH_CSV_WRITER_ENCODING", "utf-8") | |||
There was a problem hiding this comment.
We should put it in the settings.
There was a problem hiding this comment.
@junnplus I guess you mean the list of environment variable strings?
https://redash.io/help/open-source/admin-guide/env-vars-settings
If so, then definitely yes. 😄
There was a problem hiding this comment.
not website, jun mean have to change to file https://github.com/getredash/redash/blob/7f4ade5f1ff95f4357f60acb07b0655a33527972/redash/settings/__init__.py
junnplus
force-pushed
the
fix_using_redash_csv_writer_encoding
branch
from
7a5530b
to
ff188a6
Compare
|
@kaibadash thank you for your contribution! Would you mind resolving the merge conflicts, as well as adding the env var in the documentation here? |
kaibadash
force-pushed
the
fix_using_redash_csv_writer_encoding
branch
2 times, most recently
from
36060bd
to
73c25bd
Compare
kaibadash
force-pushed
the
fix_using_redash_csv_writer_encoding
branch
from
73c25bd
to
017b6bd
Compare
|
Thank you for reviewing. I resolved conflicts.
I created a pull request. |
|
@kaibadash Could you add tests ? |
|
You need to run |
redash/handlers/query_results.py
Outdated
| @@ -37,6 +38,7 @@ | |||
| serialize_job, | |||
| ) | |||
|
|
|||
| WRITER_ENCODING = os.environ.get("REDASH_CSV_WRITER_ENCODING", "utf-8") | |||
There was a problem hiding this comment.
not website, jun mean have to change to file https://github.com/getredash/redash/blob/7f4ade5f1ff95f4357f60acb07b0655a33527972/redash/settings/__init__.py
…badash/redash into fix_using_redash_csv_writer_encoding
Codecov Report
@@ Coverage Diff @@
## master #5826 +/- ##
=======================================
Coverage 61.26% 61.26%
=======================================
Files 158 158
Lines 12889 12890 +1
Branches 1755 1755
=======================================
+ Hits 7896 7897 +1
Misses 4743 4743
Partials 250 250
|
|
I tried to add tests in class TestMakeCsvResponse(BaseTestCase):
# TODO: How can I mock CSV_WRITER_ENCODING ?
def test_make_csv_response_with_cp932_encoding(self):
# TODO: How can I get query_result with multi byte string?
query_result = self.factory.create_query_result()
response = self.make_csv_response(query_result)
# TODO: How can I get result ?
self.assertEqual(response["TODO"], "こんにちは means hello in Japanese".encode("cp932")) |
|
@guidopetri Do you have time to help out here? 😄 |
What type of PR is this?
Description
REDASH_CSV_WRITER_ENCODINGenvironment variable is ignored and always downloads as UTF-8.We, engineers may not understand what the problem is that.
The administrative staffs in Japan like to edit the CSV directly in Excel and upload it to another system. Excel needs to be encoded in CP932(Shift-JIS) to edit it directly, so we need to be able to set the character encoding.
I think all be solved if Excel supported UTF-8, but Microsoft never do that and Japanese administrative staffs can only use Excel... 😫
How is this tested?
Related Tickets & Documents
Sorry for the Japanese documentation. There are people who are having the same problem as I am.
https://qiita.com/koike_moyashi/items/d0d5dc37a93b398f0aaa
Mobile & Desktop Screenshots/Recordings (if there are UI changes)