Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] master from curl:master #279

Open
wants to merge 339 commits into
base: master
Choose a base branch
from
Open

[pull] master from curl:master #279

wants to merge 339 commits into from

Conversation

pull[bot]
Copy link

@pull pull bot commented Oct 5, 2024

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

vszakats and others added 18 commits October 15, 2024 01:40
Add a build-only cmake job with system mbedTLS package and `pkg-config`
enabled. Ubuntu 24.04 comes with mbedTLS 2.28.8 which supports
`pkg-config`.

Follow-up to 7bab201 #15193
Closes #15286
Make sure we use functions with the correct prototype.

Closes #15289
It makes the callbacks get different signnatures when used from within
libcurl vs outside of it by libcurl-using applications (such as the
libtests) and this triggers UndefinedBehaviorSanitizer errors.

Closes #15289
Make test applications use the correct prototypes for callbacks.

Closes #15289
Fix new issues found by `proselint`.

Also:
- silence this technical warning:
  ```
  :0: DeprecationWarning: /home/runner/.proselintrc was found instead of a JSON file. Rename to /home/runner/.proselintrc.json.
  ```
- fix an input filename.
  `proselints` fails now if an input file is missing.

Reported-by: Jay Satiro
Bug: #15291 (comment)
Closes #15293
- update Mbed-TLS/mbedtls to v3.6.2
- update libressl-portable/portable to v4
- update github/codeql-action digest to f779452

Closes #15301
Closes #15299
Closes #15188
... instead of separate malloc() calls:

- removes two mallocs (and associated error handling paths)
- makes cleanup easier

Also reduce maximum SFTP file path lengths to 1024 bytes universally
everywhere. Using the system's own MAX_PATH did not make sense since
this is mostly about getting a remote file name.

Closes #15285
Avoids having to use a big stack buffer for this.

Closes #15285
```
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c: In function 'msghdr_get_udp_gro':
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:344: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  344 | #pragma clang diagnostic push
      |
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:345: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  345 | #pragma clang diagnostic ignored "-Wsign-compare"
      |
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:346: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  346 | #pragma clang diagnostic ignored "-Wcast-align"
      |
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vquic/vquic.c:350: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
  350 | #pragma clang diagnostic pop
      |
```
https://github.com/curl/curl-for-win/actions/runs/11356281008/job/31587180874#step:3:9534

Follow-up to a571afc #14012
Closes #15303
Follow-up to facf59c

Reported-by: Viktor Szakats
Bug: #15285 (comment)
Closes #15309
Seen with curl-for-win linux-musl-from-mac build with gcc 9.2.0.

```
n file included from /Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:136:
/Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/tool_ca_embed.c:4:28: warning: redundant redeclaration of 'curl_ca_embed' [-Wredundant-decls]
    4 | extern const unsigned char curl_ca_embed[];
      |                            ^~~~~~~~~~~~~
In file included from /Users/runner/work/curl-for-win/curl-for-win/curl/_x64-linux-musl-bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:88:
/Users/runner/work/curl-for-win/curl-for-win/curl/src/tool_operate.c:107:28: note: previous declaration of 'curl_ca_embed' was here
  107 | extern const unsigned char curl_ca_embed[];
      |                            ^~~~~~~~~~~~~
```
https://github.com/curl/curl-for-win/actions/runs/11192203640/job/31116070669#step:3:4894

Follow-up to 8a3740b #14059
Closes #15307
vsftpd 3.0 at least writes its version number to stdin (!) instead of
stderr. This works due for backwards compatibility reasons in UNIX, so
we must check stdin for anything written there to reliably parse the
version string.

Closes #15278
Commit 8ea120f added --break-system-packages which works in Ubuntu
24.04 but not 22.04, so explicitly specify that version in the runner
instead of relying on ubuntu-latest to provide it.  Some runners have
regressed back to 22.04 for ubuntu-latest, resulting in build failures.
To match other workflows and to avoid repetition in rules.

Also:
- fix build example step for cmake. update a job to use it.
- use `cmake` to invoke the builds (instead of ninja directly).
- extend test 2100 exclusion to more jobs.
  It fails with all `!debug gcc-12` jobs with autotools.
  With cmake this only happened for gcc-12 Secure Transport jobs
  for some reason.

Closes #15312
- Add a new section explaining that a build can be cleaned by adding the
  keyword "clean" to the build command.

- Add an example of using the "x64 Native Tools" prompt to the VS
  command prompt section.

- Update the Legacy Windows section's lack-of-cipher support warning to
  say "Windows 8 and earlier" instead of "Windows XP and earlier".

Ref: #15277

Closes #15291
- improve descriptions
- require http/2, not h2c, since they are done over HTTPS

Closes #15317
Override the system default config in test_17_09, since we want to check
all TLS versions. Provide own, empty config file to gnutls, so that any
system wide file has no effect.

The latest ubunu image in GH CI disables TLS 1.0 and 1.1
system wide for GnuTLS. Good intentions.

Closes #15310
talregev and others added 30 commits November 26, 2024 19:15
Enable ECH feature on wolfSSL

Closes #15649
`getch()` is deprecated according to MSDN:
https://learn.microsoft.com/cpp/c-runtime-library/reference/getch
"The Microsoft-specific function name `getch` is a deprecated alias
for the `_getch` function."

Ref: https://learn.microsoft.com/cpp/c-runtime-library/reference/getch-getwch

Follow-up to 72edb22 #15638
Closes #15642
- GHA/windows: un-ignore tests 472 1299 1613.
  They were ignored for the mingw-w64 c-ares U job.
  They do run fine now:
  https://github.com/curl/curl/actions/runs/12032875421/job/33547724780?pr=15644

- globally re-enable test 2086.
  Comment says it only affected Windows.
  Seems to be running fine now. Example:
  Windows:
  https://github.com/curl/curl/actions/runs/12032875421/job/33547718309?pr=15644#step:13:3856
  Linux:
  https://github.com/curl/curl/actions/runs/12032875397/job/33545739712#step:41:3650

- update comments for disabled tests 1184, 1801.
  They affect all operating systems, likely all CIs.

  FAIL 1801: 'HTTP/2 upgrade with lying server' HTTP, HTTP/2
  https://github.com/curl/curl/actions/runs/12032362497/job/33544053001#step:14:4265
  FAIL 1184: 'HTTP 1.1 CONNECT with redirect and set -H user-agent' HTTP, HTTP CONNECT, HTTP proxy, proxytunnel
  https://github.com/curl/curl/actions/runs/12032362497/job/33544051415#step:14:5252

Closes #15644
The ECH feature cannot be built without HTTPS RR.

ECH automatically implied HTTPS RR in `./configure` but not in CMake,
winbuild, documentation.

Also update documentation and CI configs.

Follow-up to a362962 #11922
Closes #15648
The MSVC UWP job in CI did not actually enable UWP. Fix this and
the fallouts discovered after enabling it.

- GHA/windows: make sure to enable UWP in MSVC vcpkg UWP job.
  Use the CMake options and C flags already used for mingw-w64, but use
  `WINAPI_FAMILY_PC_APP` instead of the deprecated `WINAPI_FAMILY_APP`.
  (The former is not supported by mingw-w64, so leave it there as-is.)
  Follow-up to cb22cfc #14077

- GHA/windows: by default the MSVC UWP job became 2x-3x slower than
  others after actually enabling UWP. Most of it is caused by
  CMake/MSBuild automatically building full APPX containers for each
  `.exe` target. This includes 21 CMake feature detections. Each
  detection app is built into a 15MB APPX project, with code signing,
  logos, etc. Example:
    https://github.com/curl/curl/actions/runs/12056968170/job/33620610958
  Disable this overhead for curl build targets via custom
  `CMAKE_VS_GLOBALS` options. I've found no way to apply them to feature
  detection targets, so those remain slow.

- cmake: automatically enable Unicode for UWP builds. It's required.
  Also stop enabling it manually in the existing CI job.

- tests: fix `getpid()` use for Windows UWP:
  ```
  tests\server\util.c(281,21): warning C4013: 'getpid' undefined; assuming extern returning int
  ```
  Ref: https://github.com/curl/curl/actions/runs/12061215311/job/33632904249#step:11:38

- src/tool_doswin: disable `GetLoadedModulePaths()` for UWP.
  mingw-w64 UWP was okay with this, but MS SDK headers are not.
  This makes `--dump-module-paths` return empty for UWP builds.
  ```
  src\tool_doswin.c(620,3): error C2065: 'MODULEENTRY32': undeclared identifier
  src\tool_doswin.c(626,11): warning C4013: 'CreateToolhelp32Snapshot' undefined; assuming extern returning int
  src\tool_doswin.c(626,36): error C2065: 'TH32CS_SNAPMODULE': undeclared identifier
  src\tool_doswin.c(632,7): warning C4013: 'Module32First' undefined; assuming extern returning int
  ```
  Ref: https://github.com/curl/curl/actions/runs/12055081933/job/33614629930#step:9:35

- examples: fix `websocket.c` to include `winsock2.h` before `windows.h`
  to make it build with MSVC UWP:
  ```
  include\curl\curl.h(143,16): error C2061: syntax error: identifier 'curl_socket_t'
  include\curl\curl.h(143,16): error C2059: syntax error: ';'
  include\curl\curl.h(417,52): error C2146: syntax error: missing ')' before identifier 'curlfd'
  include\curl\curl.h(417,38): error C2081: 'curl_socket_t': name in formal parameter list illegal
  ```
  Ref: https://github.com/curl/curl/actions/runs/12055317910/job/33615644427#step:14:126

- GHA/windows: silence linker warning with MSVC UWP builds:
  ```
  LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/OPT:ICF' specification
  ```
  Ref: https://github.com/curl/curl/actions/runs/12055696808/job/33616629610#step:11:38

- GHA/windows: set `/INCREMENTAL:NO` for all MSVC jobs to improve
  performance a little.

- cmake: show `UWP` platform flag.

Ref: #15652
Closes #15657
This was once supported in CMake 2.x and in current 3.x versions is
ignored.

Closes #15661
This makes `runtests.pl` run the final executables directly.
Before this patch it called the autotools/libtool wrapper tool, which
then called the final executables.

This solution was already used for `curl.exe`.

Applies to tests run in the `mingw, AM x86_64 c-ares U` job, which still
shows unexplained flakiness.

Also makes tests finish 45 seconds faster.

Ref: #14854
Follow-up to 1a2d38c #15437
Closes #15662
This provides better information on the length of the job and when it
completed.
It was already done in cmake jobs, but not in autotools ones.

Follow-up to 1a2d38c #15437
Follow-up to 04184d4 #15662

Closes #15663
- Restore some necessary options for builds without HTTP and MQTT.

The logic to turn off a segment of options in builds without HTTP and
MQTT was too expansive. Those builds (such as FTP-only builds) could not
use options such as CURLOPT_URL or CURLOPT_USERNAME etc.

Prior to this change 30da1f5 (precedes 8.11.0) refactored the options
processing and caused this issue.

Reported-by: Yoshimasa Ohno

Fixes #15634
Closes #15640
Reported-by: SuperStormer on github
Fixes #15658
Closes #15660
Allowing both just creates a transfer with behaviors no user can
properly anticipate so better just deny the combo.

Fixes #15646
Reported-by: Harry Sintonen
Closes #15666
Add test_02_33 to run with various values for the multi option
CURLMOPT_MAX_HOST_CONNECTIONS and CURLOPT_FRESH_CONNECT to trigger
connection pool limit handling code.

Closes #15494
Fix regression that no longer printed the error messages about expired
certificates in openssl. Add test case for openssl/gnutls/wolfssl.

Fixes #15612
Reported-by: hiimmat on github
Closes #15613
Timestamps in trace logs used a mix of realtime and monotonic time
sources, leading to fractional seconds carrying wrong values. Use
realtime only, so the correct nanoseconds are printed.

Fixes #15614
Reported-by: jethrogb on github
Closes #15641
Test 481 verifies

Fixes #15645
Reported-by: Harry Sintonen
Closes #15668
Test 482 verifies

Fixes #15645
Reported-by: Harry Sintonen
Closes #15668
Support async sftp upload for curl built with libssh.

Closes #15625
- ngtcp2/ngtcp2 to v1.9.1
- github/codeql-action digest to f09c1c0
- rustls/rustls-ffi to v0.14.1
- awslabs/aws-lc to v1.40.0

Closes #15616
Closes #15619
Closes #15629
Closes #15651
It is a prefix already taken and is used by OpenSSL

Closes #15673
... and make it static. As it is not used anywhere else.

Closes #15672
For MD4, MD5, and DES

Assisted-by: Viktor Szakats
Closes #15650
Pointed out by CodeSonar

Closes #15679
This moves argument parsing logic for a number of options into sub
functions to reduce the overall complexity of the single getparameter()
function. pmccabe says it takes complexity down from 234 to 147.

The command line options that now has dedicated parser funtions are:

 --continue-at, --ech, --header, --localport, --output, --quote, --range
 --remote-name, --time-cond, --upload-file, --url, --verbose, --writeout

These parsers were selected for thise because they had more than 15
lines of logic in the main switch(). Detected like this:

 git grep -hn 'case C_' tool_getparam.c |
  cut -d: -f1 |
  awk '{if(($1 - prev) > 15) { printf "%d\n", prev;} prev = $1;}'

Closes #15680
Other programs (Postman, Chrome, Python request) use a 16 byte cnonce
and there are instances of server-side implementations that don't
support the larger lengths curl used previously.

Fixes #15653
Reported-by: Florian Eckert
Closes #15670
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.