fix(deps): update dependency crawlee to v3.12.1

[renovate]

This PR contains the following updates:

Package	Change	Age	Adoption	Passing	Confidence
crawlee (source)	3.9.1 -> 3.12.1

---

Release Notes

apify/crawlee (crawlee)

v3.12.1

Compare Source

Bug Fixes

• log status message timeouts to debug level (55ee44a)
• social: support new URL formats for Facebook, YouTube and X (#​2758) (4c95847), closes #​525

Features

• tieredProxyUrls accept null for switching the proxy off (#​2743) (82f4ea9), closes #​2740

v3.12.0

Compare Source

Bug Fixes

• .trim() urls from pretty-printed sitemap.xml files (#​2709) (802a6fe), closes #​2698
• ensure correct column order in CSV export (#​2734) (b66784f), closes #​2718
• ignore errors from iframe content extraction (#​2714) (627e5c2), closes #​2708
• update fingerprintGeneratorOptions types (#​2705) (fcb098d), closes #​2703

Features

• allow using other HTTP clients (#​2661) (568c655), closes #​2659

3.11.5 (2024-10-04)

Bug Fixes

• forefront request fetching in RQv2 (#​2689) (03951bd), closes #​2669
• prolong- and deleteRequestLock forefront option (#​2690) (cba8da3), closes #​2681 #​2689 #​2669
• check .isFinished() before RequestList reads (#​2695) (6fa170f)
• core: accept UInt8Array in KVS.setValue() (#​2682) (8ef0e60)
• core: trigger errorHandler for session errors (#​2683) (7d72bcb), closes #​2678
• decode special characters in proxy username and password (#​2696) (0f0fcc5)
• http-crawler: avoid crashing when gotOptions.cache is on (#​2686) (1106d3a)
• puppeteer: rename ignoreHTTPSErrors to acceptInsecureCerts to support v23 (#​2684) (f3927e6)
• respect forefront option in MemoryStorage's RequestQueue (#​2681) (b0527f9), closes #​2669

3.11.4 (2024-09-23)

Bug Fixes

• SitemapRequestList.teardown() doesn't break persistState calls (#​2673) (fb2c5cd), closes /github.com/apify/crawlee/blob/f3eb99d9fa9a7aa0ec1dcb9773e666a9ac14fb76/packages/core/src/storages/sitemap_request_list.ts#L446 #​2672

3.11.3 (2024-09-03)

Bug Fixes

• improve FACEBOOK_REGEX to match older style page URLs (#​2650) (a005e69), closes #​2216
• RequestQueueV2: reset recently handled cache too if the queue is pending for too long (#​2656) (51a69bc)

3.11.2 (2024-08-28)

Bug Fixes

• RequestQueueV2: remove inProgress cache, rely solely on locked states (#​2601) (57fcb08)
• use namespace imports for cheerio to be compatible with v1 (#​2641) (f48296f)
• Use the correct mutex in memory storage RequestQueueClient (#​2623) (2fa8a29)

Features

• globs & regexps for SitemapRequestList (#​2631) (b5fd3a9)
• resilient sitemap loading (#​2619) (1dd7660)

3.11.1 (2024-07-24)

Bug Fixes

• use getHTML in the shadow root expansion (#​2587) (a244d62), closes #​2583

v3.11.5

Compare Source

Bug Fixes

• forefront request fetching in RQv2 (#​2689) (03951bd), closes #​2669
• prolong- and deleteRequestLock forefront option (#​2690) (cba8da3), closes #​2681 #​2689 #​2669
• check .isFinished() before RequestList reads (#​2695) (6fa170f)
• core: accept UInt8Array in KVS.setValue() (#​2682) (8ef0e60)
• core: trigger errorHandler for session errors (#​2683) (7d72bcb), closes #​2678
• decode special characters in proxy username and password (#​2696) (0f0fcc5)
• http-crawler: avoid crashing when gotOptions.cache is on (#​2686) (1106d3a)
• puppeteer: rename ignoreHTTPSErrors to acceptInsecureCerts to support v23 (#​2684) (f3927e6)
• respect forefront option in MemoryStorage's RequestQueue (#​2681) (b0527f9), closes #​2669

v3.11.4

Compare Source

Bug Fixes

• SitemapRequestList.teardown() doesn't break persistState calls (#​2673) (fb2c5cd), closes /github.com/apify/crawlee/blob/f3eb99d9fa9a7aa0ec1dcb9773e666a9ac14fb76/packages/core/src/storages/sitemap_request_list.ts#L446 #​2672

v3.11.3

Compare Source

Bug Fixes

• improve FACEBOOK_REGEX to match older style page URLs (#​2650) (a005e69), closes #​2216
• RequestQueueV2: reset recently handled cache too if the queue is pending for too long (#​2656) (51a69bc)

v3.11.2

Compare Source

Bug Fixes

• RequestQueueV2: remove inProgress cache, rely solely on locked states (#​2601) (57fcb08)
• use namespace imports for cheerio to be compatible with v1 (#​2641) (f48296f)
• Use the correct mutex in memory storage RequestQueueClient (#​2623) (2fa8a29)

Features

• globs & regexps for SitemapRequestList (#​2631) (b5fd3a9)
• resilient sitemap loading (#​2619) (1dd7660)

v3.11.1

Compare Source

Bug Fixes

• use getHTML in the shadow root expansion (#​2587) (a244d62), closes #​2583

v3.11.0

Compare Source

Features

• add iframe expansion to parseWithCheerio in browsers (#​2542) (328d085), closes #​2507
• add ignoreIframes opt-out from the Cheerio iframe expansion (#​2562) (474a8dc)
• Sitemap-based request list implementation (#​2498) (7bf8f0b)

3.10.5 (2024-06-12)

Bug Fixes

• allow creating new adaptive crawler instance without any parameters (9b7f595)
• declare missing peer dependencies in @crawlee/browser package (#​2532) (3357c7f)
• fix detection of HTTP site when using the useState in adaptive crawler (#​2530) (7e195c1)
• mark context.request.loadedUrl and id as required inside the request handler (#​2531) (2b54660)

3.10.4 (2024-06-11)

Bug Fixes

• add waitForAllRequestsToBeAdded option to enqueueLinks helper (925546b), closes #​2318
• add missing useState implementation into crawling context (eec4a71)
• make crawler.log publicly accessible (#​2526) (3e9e665)
• playwright: allow passing new context options in launchOptions on type level (0519d40), closes #​1849
• respect crawler.log when creating child logger for Statistics (0a0d75d), closes #​2412

3.10.3 (2024-06-07)

Bug Fixes

• adaptive-crawler: log only once for the committed request handler execution (#​2524) (533bd3f)
• increase timeout for retiring inactive browsers (#​2523) (195f176)
• respect implicit router when no requestHandler is provided in AdaptiveCrawler (#​2518) (31083aa)
• revert the scaling steps back to 5% (5bf32f8)

Features

• add waitForSelector context helper + parseWithCheerio in adaptive crawler (#​2522) (6f88e73)
• log desired concurrency in the default status message (9f0b796)

3.10.2 (2024-06-03)

Bug Fixes

• Autodetect sitemap filetype from content (#​2497) (62a9f40), closes #​2461
• improve fix for double extension in KVS with HTML files (#​2505) (157927d), closes #​2419

Features

• Loading sitemaps from string (#​2496) (38ed0d6), closes #​2460

3.10.1 (2024-05-23)

Bug Fixes

• adjust URL_NO_COMMAS_REGEX regexp to allow single character hostnames (#​2492) (ec802e8), closes #​2487
• investigate and temp fix for possible 0-concurrency bug in RQv2 (#​2494) (4ebe820)
• provide URLs to the error snapshot (#​2482) (7f64145), closes /github.com/apify/apify-sdk-js/blob/master/packages/apify/src/key_value_store.ts#L25

v3.10.5

Compare Source

Bug Fixes

• allow creating new adaptive crawler instance without any parameters (9b7f595)
• declare missing peer dependencies in @crawlee/browser package (#​2532) (3357c7f)
• fix detection of HTTP site when using the useState in adaptive crawler (#​2530) (7e195c1)
• mark context.request.loadedUrl and id as required inside the request handler (#​2531) (2b54660)

v3.10.4

Compare Source

Bug Fixes

• add waitForAllRequestsToBeAdded option to enqueueLinks helper (925546b), closes #​2318
• add missing useState implementation into crawling context (eec4a71)
• make crawler.log publicly accessible (#​2526) (3e9e665)
• playwright: allow passing new context options in launchOptions on type level (0519d40), closes #​1849
• respect crawler.log when creating child logger for Statistics (0a0d75d), closes #​2412

v3.10.3

Compare Source

Bug Fixes

• adaptive-crawler: log only once for the committed request handler execution (#​2524) (533bd3f)
• increase timeout for retiring inactive browsers (#​2523) (195f176)
• respect implicit router when no requestHandler is provided in AdaptiveCrawler (#​2518) (31083aa)
• revert the scaling steps back to 5% (5bf32f8)

Features

• add waitForSelector context helper + parseWithCheerio in adaptive crawler (#​2522) (6f88e73)
• log desired concurrency in the default status message (9f0b796)

v3.10.2

Compare Source

Bug Fixes

• Autodetect sitemap filetype from content (#​2497) (62a9f40), closes #​2461
• improve fix for double extension in KVS with HTML files (#​2505) (157927d), closes #​2419

Features

• Loading sitemaps from string (#​2496) (38ed0d6), closes #​2460

v3.10.1

Compare Source

Bug Fixes

• adjust URL_NO_COMMAS_REGEX regexp to allow single character hostnames (#​2492) (ec802e8), closes #​2487
• investigate and temp fix for possible 0-concurrency bug in RQv2 (#​2494) (4ebe820)
• provide URLs to the error snapshot (#​2482) (7f64145), closes /github.com/apify/apify-sdk-js/blob/master/packages/apify/src/key_value_store.ts#L25

v3.10.0

Compare Source

Bug Fixes

• EnqueueStrategy.All erroring with links using unsupported protocols (#​2389) (8db3908)
• conversion between tough cookies and browser pool cookies (#​2443) (74f73ab)
• fire local SystemInfo events every second (#​2454) (1fa9a66)
• use createSessionFunction when loading Session from persisted state (#​2444) (3c56b4c)
• do not drop statistics on migration/resurrection/resume (#​2462) (8ce7dd4)
• double tier decrement in tiered proxy (#​2468) (3a8204b)
• Fixed double extension for screenshots (#​2419) (e8b39c4), closes #​1980
• malformed sitemap url when sitemap index child contains querystring (#​2430) (e4cd41c)
• return true when robots.isAllowed returns undefined (#​2439) (6f541f8), closes #​2437
• sitemap content-type check breaks on content-type parameters (#​2442) (db7d372)

Features

• add FileDownload "crawler" (#​2435) (d73756b)
• implement ErrorSnapshotter for error context capture (#​2332) (e861dfd), closes #​2280
• make RequestQueue v2 the default queue, see more on Apify blog (#​2390) (41ae8ab), closes #​2388

Performance Improvements

• improve scaling based on memory (#​2459) (2d5d443)
• optimize RequestList memory footprint (#​2466) (12210bd)
• optimize adding large amount of requests via crawler.addRequests() (#​2456) (6da86a8)

3.9.2 (2024-04-17)

Bug Fixes

• break up growing stack in AutoscaledPool.notify (#​2422) (6f2e6b0), closes #​2421
• don't call notify in addRequests() (#​2425) (c4d5446), closes #​2421

Features

• sitemap: Support CDATA in sitemaps (#​2424) (635f046)

3.9.1 (2024-04-11)

Features

• browserPerProxy browser launch option (#​2418) (df57b29)

v3.9.2

Compare Source

Bug Fixes

• break up growing stack in AutoscaledPool.notify (#​2422) (6f2e6b0), closes #​2421
• don't call notify in addRequests() (#​2425) (c4d5446), closes #​2421

Features

• sitemap: Support CDATA in sitemaps (#​2424) (635f046)

---

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[codesandbox]

Review or Edit in CodeSandbox

Open the branch in Web Editor • VS Code • Insiders

Open Preview