Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proxy does not get set #224

Open
n1xn opened this issue Jan 21, 2023 · 0 comments
Open

Proxy does not get set #224

n1xn opened this issue Jan 21, 2023 · 0 comments

Comments

@n1xn
Copy link

n1xn commented Jan 21, 2023

I am trying to use cuprite to test my scrappers, therefore I have a base test that is checking if it can connect to my browserless instance via url and then checks if the proxy is applied. The Proxy connected test is currently failling on purpose to obtain an image of the ip in use. The proxy with this setup is not applied at all.

Down below is an example of my scrapper using Ferrum and having proxy successfully applied.

Recreation:
docker run -p 3000:3000 browserless/chrome
BROWSERLESS_URL=http://localhost:3000

cuprite_helper.rb

# frozen_string_literal: true

require 'capybara/cuprite'

# Default settings
Capybara.default_driver = :browserless
Capybara.javascript_driver = :browserless

# Main Browserless driver
Capybara.register_driver(:browserless) do |app|
  driver_options = {
    url: ENV.fetch('BROWSERLESS_URL', nil),
    browser_options: { 'no-sandbox': nil }
  }

  Capybara::Cuprite::Driver.new(app, driver_options).tap do |driver|

    # DOES NOT WORK
    #
    # driver.set_proxy(ENV.fetch('BROWSERLESS_PROXY_HOST', nil),
    #                  ENV.fetch('BROWSERLESS_PROXY_PORT', nil),
    #                  ENV.fetch('BROWSERLESS_PROXY_USERNAME', nil),
    #                  ENV.fetch('BROWSERLESS_PROXY_PASSWORD', nil),
    #                  Capybara.app_host)
  end
end

# This is a helper to run a block of code which
# url is not pointing to the current project.
# This is useful for testing external websites.
# @param [String] root_url The root url of the external site
def external(root_url)
  # Unsetting default Capybara settings for the block
  previous_app_host = Capybara.app_host
  Capybara.always_include_port = false
  Capybara.app_host = root_url

  yield

  # Resetting default Capybara settings
  Capybara.app_host = previous_app_host
  Capybara.always_include_port = true
end

browser_less_test.rb

# frozen_string_literal: true

require 'application_system_test_case'

class BrowserLessTest < ApplicationSystemTestCase
  include Capybara::DSL

  test 'Remote browser connected' do
    assert_not page.driver.browser.nil?
  end

  test 'Proxy connected' do
    external 'https://whatismyipaddress.com' do
     
      # DOES NOT WORK
      page.driver.set_proxy(ENV.fetch('BROWSERLESS_PROXY_HOST', nil),
                            ENV.fetch('BROWSERLESS_PROXY_PORT', nil),
                            ENV.fetch('BROWSERLESS_PROXY_USERNAME', nil),
                            ENV.fetch('BROWSERLESS_PROXY_PASSWORD', nil),
                            Capybara.app_host)
      visit '/'
      sleep 10
      assert page.has_content?('you passed')
    end
  end

Scrapper Example (working proxy)

# frozen_string_literal: true

module ScrapServices
  class MyClass
    def initialize
      @browser = Ferrum::Browser.new(browser_options)
    end

    def scrap
      @browser.go_to('/')
      cleanup
    end

    private

      # Browser options for connecting to remote
      # chrome instance.
      def browser_options
        {
          url: ENV.fetch('BROWSERLESS_URL', nil),
          base_url: 'https://whatismyipaddress.com',
          proxy: browser_proxy
        }
      end

      # Proxy options for browser instance
      def browser_proxy
        {
          host: ENV.fetch('BROWSERLESS_PROXY_HOST', nil),
          port: ENV.fetch('BROWSERLESS_PROXY_PORT', nil),
          user: ENV.fetch('BROWSERLESS_PROXY_USERNAME', nil),
          password: ENV.fetch('BROWSERLESS_PROXY_PASSWORD', nil)
        }
      end

      # Quits the browser
      def cleanup
        @browser.reset
        @browser.quit
      end
  end
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant