Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce page weight by ~65MB #5

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Conversation

slowe
Copy link

@slowe slowe commented Sep 10, 2024

I noticed on the intro page a note saying "there may be a slight delay in loading the interactive map depending on the device you are using". Looking at the interactive map page I noticed that it has a 97MB page load (uncompressed) of which 89MB is in the HTML. Looking at that I noticed most of this is in inefficient JSON. In this pull request I've:

  1. Truncated coordinates from 15 decimal places (which is equivalent to atom-level precision) to 5 decimal places (around 1.1m at worst which should be fine for 200x200m boxes). That saves approximately ~21MB;
  2. Converted RGB values into integers instead of floating point values. This saves another ~1.7MB.

These were a couple of "quick win" improvements. However there are is potential for a lot more reduction:

  1. There seem to be 195,395 cells so the variables accessible_population, id, population, proximity_population, transport_performance, uc name and within_urban_centre are repeated that many times. Are they used in the visualisation? If not, they could be removed entirely saving another 27MB. If they are, could they be shorter?
  2. The quadrilaterals that make each cell are currently sent as GeoJSON in the JSON. That requires 10 values to be sent for each cell (lat/lon for 5 vertices). With a little re-engineering you could send a more concise format to the browser. You could then use JS to construct the GeoJSON representation before sending it to the map. I suspect that 195,000 operations may be much quicker than the many MB of HTML required.

Truncate precision to 5 decimal places.
Use integers for RGB.
To reduce the page size by another 11MB we can remove the fifth vertex, trim the geometry object and then rebuild it in the browser. The rebuild in the browser takes under a second.
We can save another 15MB by truncating variable names. This takes us down to under 40MB from almost 90MB.
@slowe
Copy link
Author

slowe commented Sep 10, 2024

I've done the two extra points:

  1. saved the polygons as 4 vertices (rather than 5) in geom and rebuilt them as geometry objects in the browser - saves 11MB
  2. trimmed variable names - saves 15MB

@slowe slowe changed the title Reduce page weight by 22.5MB (with suggestions for more) Reduce page weight by ~50MB Sep 10, 2024
By converting the data for each layer into a CSV format we can save another 15MB.
@slowe slowe changed the title Reduce page weight by ~50MB Reduce page weight by ~65MB Sep 10, 2024
@slowe
Copy link
Author

slowe commented Sep 10, 2024

I've converted the data variable for each layer from JSON to a CSV format. This saves another 15MB as we don't need to repeat variable names. There is a little bit of Javascript to parse this back into the original format on page load.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant