Get Socket.io working in SSR environment

[marshallswain]

This is an epic for keeping track of all the moving parts it takes to get Socket.io working in SSR.

It's in the early Proof of Concept / planning stage right now. After talking with @BigAB, @matthewp, and @daffl about the moving parts, it looks to me like this could very well be the best solution for implementing SSR in a Feathers-connected app. If it works out, setting up your client app would be a two step process:

1. Add the zone plugin to your DoneSSR config.
2. Use steal-socket.io instead of socket.io-client.

Everything else would be exactly the same as if you were creating the app without SSR.

Basic Idea

SSR for apps that use XHRs is quite simple because the connection isn't stateful. Currently, with feathers-socketio and feathers-authentication, the socket auth is stateful in order to identify which socket connections should receive server-side events. Since SSR won't need those, we can implement a stateless, shared socket for all connected clients.

Also important to note is that while it's possible to directly access the Feathers app if the SSR server and the Feathers server are on the same machine, I'm purposely ignoring that solution. I favor a setup that that will work when the API server and SSR server are separate, which will technically still work if they're on the same machine.

Why it's a good idea

• Clean, simpler client app setup (as mentioned earlier). There are fewer moving parts necessary than the solution we've been working towards (which swaps out socket.io requests for XHRs).
• Quicker SSR page response times due to
  • Less overhead: Instead of setting up a socket for every request (as is done with XHR), a single socket is shared. Resting memory usage will be higher than when using XHR, but only very slightly. (By "resting" I mean when there are no active requests.) Possibly more important is that memory usage while requests are in progress will be lower.
  • Faster requests: @daffl did some basic testing on a remote Heroku service and the response times were approximately 100x faster. Most of this is probably due to having lower overhead, though some could be from browser limits on simultaneous XHRs.

Caveats

• It will only work with "stateless" sockets, which should be fine in any SSR scenario that uses JWT.
• Apps should not call feathersClient.authenticate() during the SSR loading process, or it will statefully authenticate the socket. A shared, authenticated socket could potentially return private data to the wrong user. While we could figure out a way to prevent feathersClient.authenticate() from being called on the SSR server. the developer should take care of this. It just needs to be clearly documented.

Making it work

• Update can-connect-feathers to use Session.get() instead of Session.create()
  • Update this if statement in can-connect-feathers to check for !window.doneSsr.
• Create a steal-socket.io can-zone plugin similar to the XHR zone plugin in Done-SSR that
  • accepts a list of approved domains that should receive the socket. (Maybe not necessary?)
  • reads the JWT from a named cookie.
  • monkey patches socket.emit to approved domains so that the payload includes the JWT token on a special param.
  • populates the WS_CACHE global to send in the initial page load HTML, similar to the XHR_CACHE.
• Make steal-socket.io check the WS_CACHE. That's a better place for it than in can-connect-feathers or another can-connect behavior, since steal-socket.io's main purpose is to assist with SSR.
• Create a server-side Feathers hook that reads the special param for auth and removes it from the payload.
• Update DoneSSR to accept an array of zone plugins.
• Update done-serve to allow a config property that points to a file containing the configuration with the extra zones. done-serve isn't "production ready", but this would make setting up the development environment much easier.

For consideration once it's working

• Address deploying to production. Since done-serve isn't production ready. Maybe one of these options:
  • Audit done-serve and get it production ready. @matthewp seemed to have concerns about possible unknown security issues. We would want to clear those up. Since we're creating every DoneJS app with it built in, it would be fantastic if it were ready for production.
  • Make an easy way to add an Express server for production. Maybe add it to the donejs-feathers generator.
  • Just document the required process. There's a lot that can go into making something production ready, so we might want to just cover the basics of getting it to work with Express reliably. Again, the developer is responsible to make things secure.

[DesignByOnyx]

the developer is responsible to make things secure

This is practical in theory but not in practice. Grab a random developer and ask him to explain something as basic as click jacking or csrf to you - they can't. Asking them to make sure their app is secure is a hefty request. Wordpress suffered from a bunch of security issues for a long time, and it's a hard thing to recover from. You lose trust, and a large majority of sites/apps are not actively maintained and would remain vulnerable even after a patch is out.

[marshallswain]

@DesignByOnyx I felt your words even as I typed the original phrase. :)

[DesignByOnyx]

Yeah - I'm sure. I don't know how to solve it. So let me get this straight - user-specific requests will be going across a single socket connection during SSR? If so, coming up with a solution might pave a way to make socket communication more secure even on clients. For example, instead of emitting "update" events, we emit instead [some-hard-to-get-token]-update events - each client would have it's own token somehow. This would prevent a malicious script on a page from being able to snoop on a connection as easily - they would have to get the token somehow. I don't know, I'm not a security expert.

[matthewp]

@DesignByOnyx We tell people not to use done-serve in production for precisely these types of concerns (security the big one, and to some degree performance as well). Maintaining a production quality server is a huge project, and we want to make it clear that done-serve isn't such a project.

That doesn't mean developers need to do it all themselves. They can use Express or Hapi or some other framework where the maintainers do put in the level of care that's needed. It's not DoneJS' mission to be full-stack, we provide SSR b/c that is necessary for good user experience.

[BigAB]

@matthewp What are we suggesting people use for server side rendering of their production donejs-apps then?

[matthewp]

done-ssr or done-ssr-middleware. You can use either with Express or Feathers or Hapi or anything else. done-serve, the cli, is the only thing that's not for production.

[BigAB]

Oh I misunderstood, again, thanks

[frank-dspeed]

I think my Deploy donejs applications in production guide could address and solve that.

as done ssr is production ready but it needs some Application Delivery Controller and on large deployments also a special dns layer as extra layer on top.

This is current work in progress

i try to port some of my success fullest Production Setups as easy use able examples
This Configuration is powering some of the biggest porn sites in the world with over 10 mio uniq visitors a day so its really proven.
https://github.com/direktspeed/donejs-production-guide

[frank-dspeed]

@marshallswain @matthewp canjs/can-connect-feathers#104
a PR from me that adds conditional loading and async feathers client connection support without changing existing api Enables SSR Support for feathers.