Support for either pluggable HTTP clients or passing in Hackney options

[sparkertime]

Right now, there's no way to pass additional options through to hackney that I can find. This means I cannot use a proxy in my HTTP requests. That's required in my environment so Waffle is not usable at the moment.

ExAws gets around this in two ways:

• first, it lets you configure the HttpClient through a defined behavior.
• second, if you choose to use hackney, it supports hackney_opts which get included in each request

I'd love to see the former but the latter would probably be simpler to implement and would also help.

[achempion]

Hi,

Waffle uses :hackney only to implement remote downloads and you can configure the headers.

Could you, please, provide more context for your issue?

We use ExAws and you can override the default configuration.

[sparkertime]

Sure. In order to use an HTTP proxy, you need to pass in Hackney options (not headers) as described here in the Hackney docs.

While I can override ExAws to use a proxy, I cannot do that for Waffle itself since I can only configure the headers, not the Hackney options. Since remote downloads fail in my environment without using an outbound HTTP proxy, that means I can't use Waffle even though ExAws works fine.

[sparkertime]

You can see an example of how ExAws supports this in Hackney through hackney_opts here - https://github.com/ex-aws/ex_aws/blob/6e6cb2655e74ac0fe88dea55257235efe90152b9/lib/ex_aws/request/hackney.ex#L18

[achempion]

Ok, I get it, you want to use custom options for remote downloads.

Good point, we definitely need that. As a workaround you could download the image upfront and then pass it as binary/path to the Waffle.

[sparkertime]

Ok, I get it, you want to use custom options for remote downloads.

Yup, that's correct. I think the best option would be making the HTTP backend for Waffle configurable like ExAws - it was pretty painless to write my own which used my proxied Finch setup instead of Hackney.

However, that's probably more work. Just providing a :hackney_opts option would be a great solution too.

[achempion]

Yes, indeed.

It should be relatively straight forward to implement, similar to remote_file_headers/1 method.

We could introduce something like remote_file_options/1 to merge it with the current options list here.

Probably, better to allow per-uploader configuration with dynamic method call for more flexibility instead of static option in configuration file.