cURL error 28 (timeout) when using pagereres for creating screenshots

[ramon-roca]

Describe the bug
get error:
cURL error 28: Operation timed out after 10000 milliseconds with 0 bytes received (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for ... (url)

To Reproduce
Steps to reproduce the behavior (on ubuntu 22.04 running ob proxmox VM, using starlink satellite internet):

1. Install pageres-cli as snap
2. Create bookmark
3. See error "cURL error 28..." at cron execution for bookmarks application at the nexcloud log reader. No screenshot has been generated.

Expected behavior
Get the screenshot for the bookmark just added

Desktop (please complete the following information):

• n/a
  Server (please complete the following information):
• OS: [Ubuntu 22.04 running on Proxmox VM, usint starlink satellite internet]
• PHP version: [8.1]
• Nextcloud version: [29.0.2]
• Bookmarks app version: [14.2.2]
• Activated Nextcloud Apps:

sudo -u www-data php /var/www/nextcloud/occ app:list
Enabled:
 - activity: 2.21.1
 - admin_audit: 1.19.0
 - announcementcenter: 6.8.1
 - appointments: 2.1.4
 - bookmarks: 14.2.2
 - calendar: 4.7.6
 - calendar_resource_management: 0.7.0
 - circles: 29.0.0-dev
 - cloud_federation_api: 1.12.0
 - collectives: 2.12.0
 - comments: 1.19.0
 - contacts: 6.0.0
 - contactsinteraction: 1.10.0
 - dashboard: 7.9.0
 - dav: 1.30.1
 - deck: 1.13.0
 - drawio: 3.0.2
 - external: 5.4.0
 - federatedfilesharing: 1.19.0
 - federation: 1.19.0
 - files: 2.1.0
 - files_accesscontrol: 1.19.1
 - files_downloadlimit: 2.0.0
 - files_external: 1.21.0
 - files_mindmap: 0.0.30
 - files_pdfviewer: 2.10.0
 - files_reminders: 1.2.0
 - files_sharing: 1.21.0
 - files_trashbin: 1.19.0
 - files_versions: 1.22.0
 - fileslibreofficeedit: 1.1.0
 - firstrunwizard: 2.18.0
 - forms: 4.2.4
 - groupfolders: 17.0.1
 - ldap_write_support: 1.11.0
 - logreader: 2.14.0
 - lookup_server_connector: 1.17.0
 - mail: 3.7.1
 - memories: 7.3.1
 - money: 0.26.6
 - nextcloud_announcements: 1.18.0
 - notes: 4.10.0
 - notifications: 2.17.0
 - oauth2: 1.17.0
 - password_policy: 1.19.0
 - passwords: 2024.5.10
 - photos: 2.5.0
 - polls: 7.1.1
 - privacy: 1.13.0
 - provisioning_api: 1.19.0
 - quota_warning: 1.19.0
 - recommendations: 2.1.0
 - related_resources: 1.4.0
 - richdocuments: 8.4.3
 - serverinfo: 1.19.0
 - settings: 1.12.0
 - sharebymail: 1.19.0
 - side_menu: 3.12.0
 - spreed: 19.0.3
 - support: 1.12.0
 - survey_client: 1.17.0
 - systemtags: 1.19.0
 - text: 3.10.0
 - theming: 2.4.0
 - twofactor_backupcodes: 1.18.0
 - updatenotification: 1.19.1
 - user_ldap: 1.20.0
 - user_status: 1.9.0
 - viewer: 2.3.0
 - weather_status: 1.9.0
 - workflowengine: 2.11.0
Disabled:
 - bruteforcesettings: 2.9.0
 - encryption: 2.17.0
 - files_retention: 1.18.0 (installed 1.18.0)
 - files_rightclick: 0.15.1 (installed 1.6.0)
 - groupfolders-12.0.3: 12.0.3
 - impersonate: 1.16.0 (installed 1.16.0)
 - ncdownloader: 1.0.20 (installed 1.0.20)
 - oidc_login: 3.1.1 (installed 3.1.1)
 - richdocumentscode: 24.4.201 (installed 24.4.201)
 - sociallogin: 5.6.5 (installed 5.6.5)
 - suspicious_login: 7.0.0
 - twofactor_totp: 11.0.0-dev
 - user_saml: 6.1.3 (installed 6.1.3)

• Nextcloud configuration:

sudo -u www-data php /var/www/nextcloud/occ config:list system
{
   "system": {
       "passwordsalt": "***REMOVED SENSITIVE VALUE***",
       "secret": "***REMOVED SENSITIVE VALUE***",
       "trusted_domains": [
           "localhost",
           "kalohan.redesperansa.local",
           "cloud.redesperansa.local",
           "kalohan.redesperansa.tl",
           "cloud.redesperansa.tl",
           "192.168.88.8"
       ],
       "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
       "datadirectory": "***REMOVED SENSITIVE VALUE***",
       "dbtype": "mysql",
       "version": "29.0.2.2",
       "overwrite.cli.url": "http:\/\/localhost",
       "overwriteprotocol": "https",
       "default_phone_region": "TL",
       "dbname": "***REMOVED SENSITIVE VALUE***",
       "dbhost": "***REMOVED SENSITIVE VALUE***",
       "dbport": "",
       "dbtableprefix": "oc_",
       "mysql.utf8mb4": true,
       "dbuser": "***REMOVED SENSITIVE VALUE***",
       "dbpassword": "***REMOVED SENSITIVE VALUE***",
       "installed": true,
       "instanceid": "***REMOVED SENSITIVE VALUE***",
       "allow_local_remote_servers": true,
       "memcache.local": "\\OC\\Memcache\\APCu",
       "filelocking.enabled": "true",
       "memcache.locking": "\\OC\\Memcache\\Redis",
       "redis": {
           "host": "***REMOVED SENSITIVE VALUE***",
           "port": 0,
           "dbindex": 0,
           "password": "***REMOVED SENSITIVE VALUE***",
           "timeout": 1.5
       },
       "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
       "maintenance": false,
       "mail_from_address": "***REMOVED SENSITIVE VALUE***",
       "mail_smtpmode": "smtp",
       "mail_sendmailmode": "smtp",
       "mail_domain": "***REMOVED SENSITIVE VALUE***",
       "mail_smtpauthtype": "LOGIN",
       "mail_smtpauth": 1,
       "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
       "mail_smtpport": "587",
       "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
       "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
       "mail_smtpsecure": "tls",
       "ncd_admin_settings": {
           "ncd_yt_binary": "\/usr\/bin\/yt-dlp",
           "focusVisibleAdded": ""
       },
       "global_aria2_config": [],
       "lost_password_link": "https:\/\/identities.redesperansa.tl\/realms\/master\/login-actions\/reset-credentials",
       "loglevel": 2,
       "maintenance_window_start": 1,
       "app_install_overwrite": [
           "files_mindmap"
       ],
       "memories.db.triggers.fcu": true,
       "memories.exiftool": "\/var\/www\/nextcloud\/apps\/memories\/bin-ext\/exiftool-amd64-glibc",
       "memories.vod.path": "\/var\/www\/nextcloud\/apps\/memories\/bin-ext\/go-vod-amd64",
       "memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg",
       "memories.vod.ffprobe": "\/usr\/bin\/ffprobe"
   }
}

• Nextcloud external user backend: LDAP/Keycloak

Nextcloud log (nextcloud/data/nextcloud.log)

{"reqId":"Xb1X57d9GDcUTwVYYKuA","level":2,"time":"2024-06-22T03:45:13+00:00","remoteAddr":"","user":"--","app":"bookmarks","method":"","url":"--","message":"cURL error 28: Operation timed out after 10000 milliseconds with 0 bytes received (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for **BOOKMARK_URL**/","userAgent":"--","version":"29.0.2.2","data":{"app":"bookmarks"}}

[github-actions]

Hello 👋

Thank you for taking the time to open this issue with the bookmarks app. I know it's frustrating when software
causes problems. You have made the right choice to come here and open an issue to make sure your problem gets looked at
and if possible solved.
I'm Marcel and have been maintaining this software the last few years. I currently work for Nextcloud but maintain this app
in my free time, because it is not an official Nextcloud product. My day job at Nextcloud is pretty awesome but sadly leaves me with
less time for side projects like this one than I used to have.
I still try to answer all issues and if possible fix all bugs here, but it sometimes takes a while until I get to it.
Until then, please be patient.
Note also that GitHub is a place where people meet to make software better together. Nobody here is under any obligation
to help you, solve your problems or deliver on any expectations or demands you may have, but if enough people come together we can
collaborate to make this software better. For everyone.
Thus, if you can, you could also look at other issues to see whether you can help other people with your knowledge
and experience. If you have coding experience it would also be awesome if you could step up to dive into the code and
try to fix the odd bug yourself. Everyone will be thankful for extra helping hands!
One last word: If you feel, at any point, like you need to vent, this is not the place for it; you can go to the forum,
to twitter or somewhere else. But this is a technical issue tracker, so please make sure to
focus on the tech and keep your opinions to yourself. (Also see our Code of Conduct. Really.)

I look forward to working with you on this issue
Cheers 💙

[marcelklehr]

I think this might be unrelated to pageres. It seems that the app simply cannot connect to the bookmarked URL

[ramon-roca]

Hi Marcel,

Thanks for your kind comments, sure I understand the circumstances and very happy to be patient and if somehow I can help/contribute/cooperate on improving this brilliant code/software.

In the meantime and after reading your comments, I've seen that pageres is able to connect to the bookmarked url by executing pageres from the terminal, I do see that the image is being produced correctly as expected, but never in less than 10 seconds, always 20 secs or even more. Typically betweeen 20 and 30 secs.

time pageres https://google.com

✔ Generated 1 screenshot from 1 url and 1 size

real	0m21.376s
user	0m1.501s
sys	0m0.442s

So looks just a timeout issue because of having a fixed 10 secs.

The servers are located in a very remote location (a school in a rural area at southeast Asia) with very poor connectivity, so using satellite. That might explain higher-than-expected execution time.

So maybe the question is just about that timeout for that cURL being set to 10 seconds... where it comes from? There is any way to change/customize it?.

[marcelklehr]

So, there's two issues here:

1. Timeout when requesting meta data from a URL
2. Timeout when trying to take screenshot using pageres (pageres is not using curl)

I will look into both.