Non-deterministic timeout during read(8192) from linux electron app

[cgc]

I'm mostly opening this issue to document things and explain some of my reasoning. When I started doing uploads from a linux-based computer (voxel2), I ran into this issue:

[uwsgi-body-read] Timeout reading 7476 bytes. Content-Length: 15773325 consumed: 17100 left: 15756225
scitran.api                  base.py   288:ERRO Traceback (most recent call last):
  File "./api/upload.py", line 161, in upload
    return process_upload(self.request, strategy, origin=self.origin, context=context)
  File "./api/upload.py", line 74, in process_upload
    form, tempdir = files.process_form(request)
  File "./api/files.py", line 110, in process_form
    fp=request.body_file, environ=env, keep_blank_values=True
  File "/usr/lib/python2.7/cgi.py", line 507, in __init__
    self.read_multi(environ, keep_blank_values, strict_parsing)
...
IOError: timeout during read(8192) on wsgi.input

While the issue would happen non-deterministically (a majority of uploads would result in a timeout), every single timeout would occur at consumed: 17100. While uploading large numbers of files from a mac, I also saw timeouts, but they didn't end at byte 17100 and the timeouts occurred for what seemed like 1 in 10 uploads.

Organizer makes HTTP requests using the node.js library request. request doesn't have browser support, so it seems like it's making use of the http node.js module which is accessible to renderer processes in electron. On a bit of a whim, I decided to see if changing from this node.js http-based library to the Fetch API would help things. It seems like it's completely fixed the issue. It also now shows network requests in the network tab of dev tools, which helps a lot with debugging. You can see the diff for the change here (many properties retain their name, so the diff really only deals with multipart POST body): 506f417

[cgc]

I tried this simple approximate replication of this issue, but was unable to reproduce this issue on voxel2: szwacz/electron-boilerplate@master...cgc:org-74 While my working theory had been that this is a fundamental issue with the way Node's http library is accessible to browser processes in linux, this seems to make that unlikely. It really seems like the issue is at the intersection of a handful of things, potentially a pretty juicy bug!

I also tried upgrading organizer to latest electron, which also didn't help.

other approaches to investigate this issue:
Does a toy python server using the same framework as scitran/core have the same issue?
- if not, then I'd try to replicate the issue with a scitran/core with no nginx in front.
- if that doesn't work, then issue is nginx
Do other linux machines have the same issue?
unclear how I'd do this: can we get OS X or windows to have this issue by more closely replicating linux HTTP requests?