Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception while setting custom ja3 parameter on requests.get() for certain cipher suite integer values #443

Open
3 tasks done
charliedelta02 opened this issue Nov 26, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@charliedelta02
Copy link

charliedelta02 commented Nov 26, 2024

Please check the following items before reporting a bug, otherwise it may be closed immediately.

  • This is NOT a site-related "bugs", e.g. some site blocks me when using curl_cffi,
    UNLESS it has been verified that the reason is missing pieces in the impersonation.
  • A code snippet that can reproduce this bug is provided, even if it's a one-liner.
  • Version information will be pasted as below.

Describe the bug

Cipher Exception:

Exception while setting custom ja3 parameter on requests.get() for certain cipher suite integer values
The exception observed is:

[2024-11-26 11:25:42,203] ERROR in curl_cffi_script: 49324
[2024-11-26 11:25:42,209] ERROR in curl_cffi_script:   File "/Users/Documents/scripts/curl_cffi_script.py", line 258, in crawl_url
    response = await s.get(
  File "/Users/python_virtual_env/python3/lib/python3.9/site-packages/curl_cffi/requests/session.py", line 1253, in request
    req, buffer, header_buffer, q, header_recved, quit_now = self._set_curl_options(
  File "/Users/python_virtual_env/python3/lib/python3.9/site-packages/curl_cffi/requests/session.py", line 654, in _set_curl_options
    self._set_ja3_options(c, ja3, permute=permute)
  File "/Users/python_virtual_env/python3/lib/python3.9/site-packages/curl_cffi/requests/session.py", line 304, in _set_ja3_options
    cipher_name = TLS_CIPHER_NAME_MAP[cipher_id]

Cipher 49324 translates to a Hex value: 0xC0,0xAC which is the cipher TLS_ECDHE_ECDSA_WITH_AES_128_CCM as per https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-4

Similar issue observed with Cipher values: 65, 64, 49202, 49167, 49194, 106 and a few more.

To Reproduce

async with AsyncSession() as s:
    # This is the ja3n text for Safari v18 on a Mac obtained from: https://tls.browserleaks.com/json
    ja3n_str = "771,49199-49171-49191-49200-49172-49192-156-47-60-157-53-61-65-132-49195-49324-49326-49161-49187-49196-49325-49327-49162-49188-158-51-103-159-57-107-69-136-4865-4866-255,0-10-11-13-23-43-51,23-29-24-256-257-258,0"
    # Another ja3n string that fails
    # ja3n_str = "771,4866-4865-4867-49196-49195-52393-49200-52392-49199-159-52394-163-158-162-49188-49192-49187-49191-107-106-103-64-49198-49202-49197-49201-49190-49194-49189-49193-49162-49172-49161-49171-57-56-51-50-49157-49167-49156-49166-157-156-61-60-53-47-255,0-5-10-11-13-17-23-43-45-50-51,29-23-24-25-30-256-257-258-259-260,0"
    response = await s.get(
        url,
        timeout=30,
        ja3=ja3n_str
    )

Expected behavior
The valid custom ja3n fingerprint string must be used upon requests.get().

Versions

  • OS: Mac Sonoma
  • curl_cffi version 0.7.3

Additional context

  • Which session are you using? async or sync? Async
@charliedelta02 charliedelta02 added the bug Something isn't working label Nov 26, 2024
@lexiforest lexiforest added enhancement New feature or request and removed bug Something isn't working labels Dec 2, 2024
@lexiforest
Copy link
Owner

lexiforest commented Dec 2, 2024

These values are not available in boringssl, thus there is no way to use them unless we patch boringssl, which requires a significant amount of investigation. Please expect an extended time to fix.

@charliedelta02
Copy link
Author

These values are not available in boringssl, thus there is no way to use them unless we patch boringssl, which requires a significant amount of investigation. Please expect an extend time to fix.

Thank you. I shall keep this ticket open as a reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants