Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Encoding::UndefinedConversionError #205

Open
beydogan opened this issue Jan 2, 2015 · 4 comments
Open

Encoding::UndefinedConversionError #205

beydogan opened this issue Jan 2, 2015 · 4 comments

Comments

@beydogan
Copy link

beydogan commented Jan 2, 2015

I'm getting this exception on markdown render requests for some repos.

Exception message is: ""\xC5" from ASCII-8BIT to UTF-8"

I'm trying to render readme files of repos;
ruby/ruby is working fine but
airbnb/ruby is not working.

Sample Code:

res = github_api.repos.contents.readme "airbnb", "ruby"
readme_base64 = res.content
decoded = Base64.decode64 readme_base64
readme = github_api.markdown.render text: decoded

Any solutions?
@piotrmurach
Copy link
Owner

Hi @beydogan, Could you please provide whole exception trace? The obvious question is why the response encoding is different from the character set contained by the markdown? Without spending time on it I cannot provide any answers. There are docs for extra parameters, maybe they will help.

@beydogan
Copy link
Author

beydogan commented Jan 2, 2015

Hi @peter-murach

I think, Base64.decode64 cant decode some UTF-8 characters. Is there a way to get parsed readme? With custom media types as mentioned here: https://developer.github.com/v3/repos/contents/#custom-media-types

Here is full-trace:

activesupport (4.1.6) lib/active_support/core_ext/object/json.rb:34:in `encode'
activesupport (4.1.6) lib/active_support/core_ext/object/json.rb:34:in `to_json'
activesupport (4.1.6) lib/active_support/core_ext/object/json.rb:34:in `to_json_with_active_support_encoder'
activesupport (4.1.6) lib/active_support/json/encoding.rb:56:in `to_json'
/home/mehmet/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/json/common.rb:223:in `generate'
/home/mehmet/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/json/common.rb:223:in `generate'
activesupport (4.1.6) lib/active_support/json/encoding.rb:96:in `stringify'
activesupport (4.1.6) lib/active_support/json/encoding.rb:34:in `encode'
activesupport (4.1.6) lib/active_support/json/encoding.rb:21:in `encode'
activesupport (4.1.6) lib/active_support/core_ext/object/json.rb:37:in `to_json_with_active_support_encoder'
multi_json (1.10.1) lib/multi_json/adapters/json_common.rb:21:in `dump'
multi_json (1.10.1) lib/multi_json/adapter.rb:24:in `dump'
multi_json (1.10.1) lib/multi_json.rb:136:in `dump'
github_api (0.12.2) lib/github_api/request/jsonize.rb:28:in `encode_body'
github_api (0.12.2) lib/github_api/request/jsonize.rb:16:in `call'
faraday (0.9.0) lib/faraday/rack_builder.rb:139:in `build_response'
faraday (0.9.0) lib/faraday/connection.rb:377:in `run_request'
faraday (0.9.0) lib/faraday/connection.rb:177:in `post'
github_api (0.12.2) lib/github_api/request.rb:68:in `call'
github_api (0.12.2) lib/github_api/request/verbs.rb:35:in `post_request'
github_api (0.12.2) lib/github_api/client/markdown.rb:34:in `render'
github_api (0.12.2) lib/github_api/api.rb:224:in `block in execute'
github_api (0.12.2) lib/github_api/api.rb:211:in `run_callbacks'
github_api (0.12.2) lib/github_api/api.rb:223:in `execute'
github_api (0.12.2) lib/github_api/api.rb:183:in `block in method_added'
actionpack (4.1.6) lib/action_controller/metal/implicit_render.rb:4:in `send_action'
actionpack (4.1.6) lib/abstract_controller/base.rb:189:in `process_action'
actionpack (4.1.6) lib/action_controller/metal/rendering.rb:10:in `process_action'
actionpack (4.1.6) lib/abstract_controller/callbacks.rb:20:in `block in process_action'
activesupport (4.1.6) lib/active_support/callbacks.rb:113:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:113:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
activesupport (4.1.6) lib/active_support/callbacks.rb:229:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:229:in `block in halting'
activesupport (4.1.6) lib/active_support/callbacks.rb:229:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:229:in `block in halting'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `block in halting'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `block in halting'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `block in halting'
activesupport (4.1.6) lib/active_support/callbacks.rb:86:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:86:in `run_callbacks'
actionpack (4.1.6) lib/abstract_controller/callbacks.rb:19:in `process_action'
actionpack (4.1.6) lib/action_controller/metal/rescue.rb:29:in `process_action'
actionpack (4.1.6) lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
activesupport (4.1.6) lib/active_support/notifications.rb:159:in `block in instrument'
activesupport (4.1.6) lib/active_support/notifications/instrumenter.rb:20:in `instrument'
activesupport (4.1.6) lib/active_support/notifications.rb:159:in `instrument'
actionpack (4.1.6) lib/action_controller/metal/instrumentation.rb:30:in `process_action'
actionpack (4.1.6) lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
activerecord (4.1.6) lib/active_record/railties/controller_runtime.rb:18:in `process_action'
actionpack (4.1.6) lib/abstract_controller/base.rb:136:in `process'
actionview (4.1.6) lib/action_view/rendering.rb:30:in `process'
actionpack (4.1.6) lib/action_controller/metal.rb:196:in `dispatch'
actionpack (4.1.6) lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
actionpack (4.1.6) lib/action_controller/metal.rb:232:in `block in action'
actionpack (4.1.6) lib/action_dispatch/routing/route_set.rb:82:in `call'
actionpack (4.1.6) lib/action_dispatch/routing/route_set.rb:82:in `dispatch'
actionpack (4.1.6) lib/action_dispatch/routing/route_set.rb:50:in `call'
actionpack (4.1.6) lib/action_dispatch/journey/router.rb:73:in `block in call'
actionpack (4.1.6) lib/action_dispatch/journey/router.rb:59:in `each'
actionpack (4.1.6) lib/action_dispatch/journey/router.rb:59:in `call'
actionpack (4.1.6) lib/action_dispatch/routing/route_set.rb:678:in `call'
omniauth (1.2.2) lib/omniauth/strategy.rb:186:in `call!'
omniauth (1.2.2) lib/omniauth/strategy.rb:164:in `call'
warden (1.2.3) lib/warden/manager.rb:35:in `block in call'
warden (1.2.3) lib/warden/manager.rb:34:in `catch'
warden (1.2.3) lib/warden/manager.rb:34:in `call'
rack (1.5.2) lib/rack/etag.rb:23:in `call'
rack (1.5.2) lib/rack/conditionalget.rb:25:in `call'
rack (1.5.2) lib/rack/head.rb:11:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/params_parser.rb:27:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/flash.rb:254:in `call'
rack (1.5.2) lib/rack/session/abstract/id.rb:225:in `context'
rack (1.5.2) lib/rack/session/abstract/id.rb:220:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/cookies.rb:560:in `call'
activerecord (4.1.6) lib/active_record/query_cache.rb:36:in `call'
activerecord (4.1.6) lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
activerecord (4.1.6) lib/active_record/migration.rb:380:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
activesupport (4.1.6) lib/active_support/callbacks.rb:82:in `run_callbacks'
actionpack (4.1.6) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/reloader.rb:73:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
railties (4.1.6) lib/rails/rack/logger.rb:38:in `call_app'
railties (4.1.6) lib/rails/rack/logger.rb:20:in `block in call'
activesupport (4.1.6) lib/active_support/tagged_logging.rb:68:in `block in tagged'
activesupport (4.1.6) lib/active_support/tagged_logging.rb:26:in `tagged'
activesupport (4.1.6) lib/active_support/tagged_logging.rb:68:in `tagged'
railties (4.1.6) lib/rails/rack/logger.rb:20:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/request_id.rb:21:in `call'
rack (1.5.2) lib/rack/methodoverride.rb:21:in `call'
rack (1.5.2) lib/rack/runtime.rb:17:in `call'
activesupport (4.1.6) lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
rack (1.5.2) lib/rack/lock.rb:17:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/static.rb:64:in `call'
rack (1.5.2) lib/rack/sendfile.rb:112:in `call'
railties (4.1.6) lib/rails/engine.rb:514:in `call'
railties (4.1.6) lib/rails/application.rb:144:in `call'
rack (1.5.2) lib/rack/lock.rb:17:in `call'
rack (1.5.2) lib/rack/content_length.rb:14:in `call'
rack (1.5.2) lib/rack/handler/webrick.rb:60:in `service'
/home/mehmet/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:138:in `service'
/home/mehmet/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:94:in `run'
/home/mehmet/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/server.rb:295:in `block in start_thread'

@beydogan
Copy link
Author

beydogan commented Jan 2, 2015

I tried following request;

github_api.repos.contents.readme "airbnb", "ruby", media: 'html'

and got:

MultiJson::ParseError in ReposController#show

795: unexpected token at '<div .. LOTS OF HTML here'

Trace:

/home/mehmet/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/json/common.rb:155:in `parse'
/home/mehmet/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/json/common.rb:155:in `parse'
multi_json (1.10.1) lib/multi_json/adapters/json_common.rb:16:in `load'
multi_json (1.10.1) lib/multi_json/adapter.rb:20:in `load'
multi_json (1.10.1) lib/multi_json.rb:119:in `load'
github_api (0.12.2) lib/github_api/jsonable.rb:12:in `decode'
github_api (0.12.2) lib/github_api/response/jsonize.rb:13:in `block in <class:Jsonize>'
github_api (0.12.2) lib/github_api/response/jsonize.rb:25:in `call'
github_api (0.12.2) lib/github_api/response/jsonize.rb:25:in `parse'
faraday (0.9.0) lib/faraday/response.rb:16:in `on_complete'
faraday (0.9.0) lib/faraday/response.rb:9:in `block in call'
faraday (0.9.0) lib/faraday/response.rb:57:in `on_complete'
faraday (0.9.0) lib/faraday/response.rb:8:in `call'
faraday (0.9.0) lib/faraday/response.rb:8:in `call'
faraday (0.9.0) lib/faraday/request/url_encoded.rb:15:in `call'
faraday (0.9.0) lib/faraday/request/multipart.rb:14:in `call'
github_api (0.12.2) lib/github_api/request/jsonize.rb:23:in `call'
faraday (0.9.0) lib/faraday/rack_builder.rb:139:in `build_response'
faraday (0.9.0) lib/faraday/connection.rb:377:in `run_request'
faraday (0.9.0) lib/faraday/connection.rb:140:in `get'
github_api (0.12.2) lib/github_api/request.rb:68:in `call'
github_api (0.12.2) lib/github_api/request/verbs.rb:20:in `get_request'
github_api (0.12.2) lib/github_api/client/repos/contents.rb:30:in `readme'
github_api (0.12.2) lib/github_api/api.rb:224:in `block in execute'
github_api (0.12.2) lib/github_api/api.rb:211:in `run_callbacks'
github_api (0.12.2) lib/github_api/api.rb:223:in `execute'
github_api (0.12.2) lib/github_api/api.rb:183:in `block in method_added'
actionpack (4.1.6) lib/action_controller/metal/implicit_render.rb:4:in `send_action'
actionpack (4.1.6) lib/abstract_controller/base.rb:189:in `process_action'
actionpack (4.1.6) lib/action_controller/metal/rendering.rb:10:in `process_action'
actionpack (4.1.6) lib/abstract_controller/callbacks.rb:20:in `block in process_action'
activesupport (4.1.6) lib/active_support/callbacks.rb:113:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:113:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
activesupport (4.1.6) lib/active_support/callbacks.rb:229:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:229:in `block in halting'
activesupport (4.1.6) lib/active_support/callbacks.rb:229:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:229:in `block in halting'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `block in halting'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `block in halting'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:166:in `block in halting'
activesupport (4.1.6) lib/active_support/callbacks.rb:86:in `call'
activesupport (4.1.6) lib/active_support/callbacks.rb:86:in `run_callbacks'
actionpack (4.1.6) lib/abstract_controller/callbacks.rb:19:in `process_action'
actionpack (4.1.6) lib/action_controller/metal/rescue.rb:29:in `process_action'
actionpack (4.1.6) lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
activesupport (4.1.6) lib/active_support/notifications.rb:159:in `block in instrument'
activesupport (4.1.6) lib/active_support/notifications/instrumenter.rb:20:in `instrument'
activesupport (4.1.6) lib/active_support/notifications.rb:159:in `instrument'
actionpack (4.1.6) lib/action_controller/metal/instrumentation.rb:30:in `process_action'
actionpack (4.1.6) lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
activerecord (4.1.6) lib/active_record/railties/controller_runtime.rb:18:in `process_action'
actionpack (4.1.6) lib/abstract_controller/base.rb:136:in `process'
actionview (4.1.6) lib/action_view/rendering.rb:30:in `process'
actionpack (4.1.6) lib/action_controller/metal.rb:196:in `dispatch'
actionpack (4.1.6) lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
actionpack (4.1.6) lib/action_controller/metal.rb:232:in `block in action'
actionpack (4.1.6) lib/action_dispatch/routing/route_set.rb:82:in `call'
actionpack (4.1.6) lib/action_dispatch/routing/route_set.rb:82:in `dispatch'
actionpack (4.1.6) lib/action_dispatch/routing/route_set.rb:50:in `call'
actionpack (4.1.6) lib/action_dispatch/journey/router.rb:73:in `block in call'
actionpack (4.1.6) lib/action_dispatch/journey/router.rb:59:in `each'
actionpack (4.1.6) lib/action_dispatch/journey/router.rb:59:in `call'
actionpack (4.1.6) lib/action_dispatch/routing/route_set.rb:678:in `call'
omniauth (1.2.2) lib/omniauth/strategy.rb:186:in `call!'
omniauth (1.2.2) lib/omniauth/strategy.rb:164:in `call'
warden (1.2.3) lib/warden/manager.rb:35:in `block in call'
warden (1.2.3) lib/warden/manager.rb:34:in `catch'
warden (1.2.3) lib/warden/manager.rb:34:in `call'
rack (1.5.2) lib/rack/etag.rb:23:in `call'
rack (1.5.2) lib/rack/conditionalget.rb:25:in `call'
rack (1.5.2) lib/rack/head.rb:11:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/params_parser.rb:27:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/flash.rb:254:in `call'
rack (1.5.2) lib/rack/session/abstract/id.rb:225:in `context'
rack (1.5.2) lib/rack/session/abstract/id.rb:220:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/cookies.rb:560:in `call'
activerecord (4.1.6) lib/active_record/query_cache.rb:36:in `call'
activerecord (4.1.6) lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
activerecord (4.1.6) lib/active_record/migration.rb:380:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
activesupport (4.1.6) lib/active_support/callbacks.rb:82:in `run_callbacks'
actionpack (4.1.6) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/reloader.rb:73:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
railties (4.1.6) lib/rails/rack/logger.rb:38:in `call_app'
railties (4.1.6) lib/rails/rack/logger.rb:20:in `block in call'
activesupport (4.1.6) lib/active_support/tagged_logging.rb:68:in `block in tagged'
activesupport (4.1.6) lib/active_support/tagged_logging.rb:26:in `tagged'
activesupport (4.1.6) lib/active_support/tagged_logging.rb:68:in `tagged'
railties (4.1.6) lib/rails/rack/logger.rb:20:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/request_id.rb:21:in `call'
rack (1.5.2) lib/rack/methodoverride.rb:21:in `call'
rack (1.5.2) lib/rack/runtime.rb:17:in `call'
activesupport (4.1.6) lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
rack (1.5.2) lib/rack/lock.rb:17:in `call'
actionpack (4.1.6) lib/action_dispatch/middleware/static.rb:64:in `call'
rack (1.5.2) lib/rack/sendfile.rb:112:in `call'
railties (4.1.6) lib/rails/engine.rb:514:in `call'
railties (4.1.6) lib/rails/application.rb:144:in `call'
rack (1.5.2) lib/rack/lock.rb:17:in `call'
rack (1.5.2) lib/rack/content_length.rb:14:in `call'
rack (1.5.2) lib/rack/handler/webrick.rb:60:in `service'
/home/mehmet/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:138:in `service'
/home/mehmet/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/httpserver.rb:94:in `run'
/home/mehmet/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/webrick/server.rb:295:in `block in start_thread'

@piotrmurach
Copy link
Owner

How about before you Base64.decode64 check what is the res.content.encoding to make sure it's valid utf-8 and encode the string in utf-8 if not?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@piotrmurach @beydogan