invalid byte sequence in UTF-8

[dbarison]

problem with rails 2.3.8 and ruby 1.9.1

[mclark]

It looks like I'm having the same issue with rails 2.3.8 and ruby 1.9.2.pre2. Here is the stack trace to reproduce:

invalid byte sequence in UTF-8

/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:135:in `scan'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:135:in `block in extract_files'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:134:in `each'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:134:in `inject'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:134:in `extract_files'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:15:in `files'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate/keys.rb:6:in `files'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate_controller.rb:37:in `initialize_keys'
/home/mclark/projects/tunstall-2.0/vendor/plugins/translate/lib/translate_controller.rb:12:in `index'
/home/mclark/.rvm/gems/ruby-1.9.2-head/gems/actionpack-2.3.8/lib/action_controller/base.rb:1331:in `perform_action'
/home/mclark/.rvm/gems/ruby-1.9.2-head/gems/actionpack-2.3.8/lib/action_controller/filters.rb:617:in `call_filters'

[mclark]

For the record, this was caused by most source files in my app being in iso-8859-1 encoding, while the extract_files method loads the file assuming it is in utf-8 by default (not sure where this comes from, it could be a system default. I've fixed the issue locally by just changing the line to the following. I don't know enough about this stuff to make a proper fix, but this should do for me, for now.

IO.read(file, :encoding => 'iso-8859-1' ).scan(i18n_lookup_pattern).flatten.map(&:to_sym).each do |key|