Land Technologies
< all posts

How we Scaled our Mapping Tool with Mapbox & Google

Posted by DanielM on February 01, 2019 · 4 min read open sourcescalingperformance

At the heart of the LandInsight product is a map showing a variety of datasets. Users can switch between map layers and filter features within each layer. Our customers - property developers - use this data to find and purchase land for new homes. Ultimately we’re trying to help solve the UK Housing Crisis (and yes, we’re hiring).

That’s all you need to know about the company, so on with the story…

Screenshot

The problem

Back in 2016, when we had an MVP used by just a handful of customers, we used a third party service (CartoDB1) to host our mapping data and generate images for map tiles. Initially this worked great - we just had to upload our data once a month, and then it was available for use within Google Maps in the browser. But as we grew, we quickly realised that there was no way this solution was going to scale if we wanted to continue allowing our customers to filter their maps on the fly. It was also proving painful to sync our data into the third party’s database at a reasonable speed. And costs were very high already.

We realised that if we wanted to scale to larger numbers of users and to enable them to filter things freely, then we really needed to shift the map rendering down to the client - the server would provide tiles of vector data, and the browser would render it into images as per the user’s requirements. Fortunately, the web platform has an API for this kind of heavy rendering job - WebGL - and it is almost ubiquitously available. And, also somewhat fortunately, one of the team already had some experience with WebGL. As it turned out, we didn’t need to build anything from scratch as Mapbox had recently open sourced a great library that does exactly this kind of vector map rendering. The only problem was that we couldn’t put it on top of a Google Map, which we really needed to do if we were to keep the strong location search, satellite imagery, and StreetView offered by Google (and not break any licensing agreements!).

The solution

So, after much head-scratching we decided we would fork the Mapbox code. There are two parts to the solution: firstly, unlike the upstream project which lets you control a whole map, our fork gives you control of rendering individual vector tiles within a canvas; and secondly, we implement a custom Google Map layer that uses the Mapbox code to render canvas tiles on the fly. Our fork is open source, and the documentation is here, code here, and demos here and here. We believe that this will be of interest to many others using google maps, and we also hope to push something upstream to Mapbox at some point.

Screenshot Demo screenshot showing fine-grained control of vector tile rendering.

Building this solution was definitely a challenge, especially at the start when we were rushing to replace the core parts of the old system and we were still a tiny startup team. But, after a couple of iterations, we now have something that works well from the user’s perspective (it is fast and feature rich), and something that is easy to deal with from the engineer’s perspective - we have a lot more control over how data gets updated and it’s much faster to do these updates. As a result we now support many hundred users on our platform, and provide them with up-to-date data, without an exorbitant per-user cost or development overhead. Everyone wins!

Postscript

Having this working in production was a relief, but as it was a fairly cutting-edge use of the web, we encountered a couple of regressions in Chrome over the months. Both of them were dealt with by the Chromium team very quickly, so hats off to them for showing us how to handle bugs efficiently! One outstanding gotcha relates to the performance of drawImage on different sizes of canvas. There seem to be two different code paths used based on the size of the canvas - it turns out that drawing to a 256x257px canvas is much, much, faster than drawing to a 256x256px canvas. Bigger is better. On that note, a reminder that we’re hiring for a variety of roles.


  1. It turns out that CartoDB ended up using Mapbox themselves.


open sourcescalingperformance
Land Technologies

We are the engineers behind LandInsight and LandEnhance. We’re helping property professionals build more houses, one line of code at a time. We're based in London, and yes, we're hiring!