lazy loading

[jancborchardt]

Currently, all posts are loaded in the stream, which makes the site go very slow. Instead, only the latest active ~10 posts should be loaded (including comments) and then when you scroll down the next ones should be displayed seamlessly (no pressing »more«). Like on Twitter or Facebook.

[johlrogge]

We have been discussing a X/Y approach where X could be like 200 and Y could be like 100 (just as an example)

You have the latest messages under a "current key", when you have X messages under the current key you take the Y oldest messages and move them to a new key, then you let the current key point to the new key in it's tail.

It has so far been considered less important than other issues but perhaps now that more than 6 people are using the system it is becoming more prioritized.

I have not profiled the application but is it the amount of data or the number of requests that is the biggest issue? If it is the number of requests then an adaptive poll interval algorithm in combination with local storage caching could maybe be more prioritized?

[jancborchardt]

Not sure if it’s the number of requests or the amount of data. I suppose the length of the page alone could be very much the reason. In any case, it’s probably good to reduce both.

[brolund]

I don't think the issue is the amount of data fetched (yet), I think it is the amount of data constantly (re)rendered by knockout.

So for now, I'd expect good improvements by just not rendering more than the user can see.

[johlrogge]

sounds like a reasonable assumption Daniel. Of course all of adaptive polling, chunked fetches, partial rendering etc will eventually be important. We just need to do them in the right order.

[brolund]

I have implemented lazy display with a "get more updates"-link (temporary solution). It displays a week of data at a time, which I found worked fine on my computer. We can of course change that to fewer days. It would be more difficult to show a certain number of updates, since you'd have to do some pretty magic stuff to filter the streams from the different users. Now I just set a cut-off date.

I also profiled the application and found a couple of issues that I fixed. If you can, please try the develop version and see what you think performance-wise.

[brolund]

I'm gonna set it to three days instead. :-)

[jancborchardt]

Nice, thank you so much! Works way better on my computer now.

As soon as loading is automatic without the button, it doesn’t even matter that we have the technical limitation of only being able to limit by time and not by number of updates.

[brolund]

I'll close this. Well probably get back to refining this at several occasions.

[jancborchardt]

Then the issue should stay open though. :)

[brolund]

So, when do we close it? :-)

[jancborchardt]

As soon as it loads additional updates lazily – automatically as you scroll down, without the need to click a button.