Broken response streams

[drewvolz]

Reopening from #901

Background

Checking on the changes pushed to the routes, we’re still dealing with streams being consumed more than once issue. Is it at an intercepted or middleware layer like we thought about before? docker-compose logs and the client app confirm we're still hitting these issues.

Issue

Response objects in ccc are being consumed more than once without being cloned first. If we are using the response body multiple times, which is a stream, it can't be read again, so we have issue.

$ docker-compose logs

TypeError: Response.clone: Body has already been consumed.
    at webidl.errors.exception (node:internal/deps/undici/undici:3384:14)
    at _Response.clone (node:internal/deps/undici/undici:8883:31)
    at result.<computed> [as json] (file:///app/node_modules/ky/distribution/core/Ky.js:55:48)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async getOlafAtoZ (file:///app/dist/source/ccci-stolaf-college/v1/a-z.js:23:41)
    at async atoz (file:///app/dist/source/ccci-stolaf-college/v1/a-z.js:49:24)
    at async bodyParser (/app/node_modules/koa-bodyparser/index.js:78:5)
    at async etag (/app/node_modules/koa-etag/index.js:27:5)
    at async /app/node_modules/koa-conditional-get/index.js:15:5
    at async compressMiddleware (/app/node_modules/koa-compress/lib/index.js:56:5)

The assumption being that this could be triggered by another request, logging, middleware, or most likely our mem caching.

The issue being the memoized response will return the same Response object, leading to an error if we are trying to access something on it.

This is what I am thinking is happening

1. GET(url) makes the http request and returns a Response object
2. .json() consumes the body of the Response
3. Memoized GET(url) --> subsequent calls to GET(url).json() reuse the same Response object from first http call

I can point my physical phone at a local version of ccc but I haven't replicated the issue when running local ccc so far.

Endpoints (* means we've seen it error, ✔ means we've fixed it)

Preview Give feedback

1. A-to-z *
2. Calendar
3. Contacts *
4. Dictionary *
5. FAQs *
6. Hours
7. Jobs
8. Menu * (in our smoke test)
9. News
10. Orgs
11. Routes [dev only] *
12. Streams *
13. Transport