How to run supercluster as server side clustering

[jpnathannew]

I am creating server side clustering for my map. When I use PHP and Mysql for create the clustering it is taking more time. So I am planning to use this project
I have few doubts

How to run this project as server?
If I pass My lat and lang points into this project this will convert everything to cluster?
How to expose API for markers to cluster?

Please help me. I am waiting for your response

[mourner]

Duplicate of #79

[jpnathannew]

Duplicate of #79

It is not a duplicate of #79. Please guide me to use this supercluster

[xlcrr]

hi @jpnathannew did you make any progress? I am working on the same problem. Maybe we can help each other?

[kityan]

hi @xlcrr! What is the problem you trying to solve? You can follow these steps with nodejs app:

• load points (lat, lng) from source (database, file, etc)
• convert data to a suitable format (GeoJSON Point)
• build index with new Supercluster(options).load(points)
  When I was developing such application, I've wanted to store clusters, but the best solution was to serve directly from nodejs app using (see getClusters method).

[xlcrr]

Hi @kityan thanks for the message

I am using Laravel PHP framework on the backend.

Once the client has the data, processing it with supercluster is incredibly fast (100's ms). The slowest part seems to be getting the data from the table into the browser (9-10s for 100,000 points and growing on virtual machine) therefore I would like to save the clusters to a table and update this with a script on the backend.

I am still trying to figure out how to do this but I am thinking about passing the data into a javascript file and execute it on the backend with $response = exec('node my_script.js'), and store this data to a table to allow for faster lookup, or maybe using a microservice. (Both of which I am new to).

[kityan]

@xlcrr the main idea is not to send 100k points to client for clusterization. For my project I have splitted front end code: for some zoom levels (e.g. 9-18) it requests backend for points (here you can use directly your php backend). For other zoom levels (e.g. 1-8) it requests backend for CLUSTERS not for POINTS. And here nodejs app with Supercluster comes in. You can proxy with web-server (nginx, etc) or via your php code.
Of course loading 100k points from db may be slow, but it will be done rarely. You nodejs app (with Supercluster) should do this operation (rebuild index) on some events in db: e.g. point add, point remove, point update. So, you need to make some kind of communication between your php code (here are methods for point add/remove/update) and nodejs app. Or if you use MongoDB or any other db with watch features, you can place such watchers directly in nodejs app (handler function will get all points again and rebuild index)

[kityan]

@xlcrr My advice: do not store clusters index to db (I am not really sure that Supercluster's index is serializable).
Build a standalone microservice using nodejs+Supercluster and let it store and rebuild clusters index in memory.
You will be able easily request this microservice to give clusters for BBox (see getClusters method)
Yes, you should be familiar with long-live server side (nodejs vs php). But it's worth it.

In this case your $response = exec('node my_script.js') is not suitable. You should have standalone nodejs app and communicate with it (directly from web front, or via php code (here you can use sockets, websockets, etc))

P.S. Moreover, you should expect that number of clusters on higher zooms can be very close to number of points (depends on cluster radius). So you could have a lot of data. It is also a reason no to try storing clusters to database on each clusters index rebuild but to serve them from nodejs app.

[xlcrr]

@kityan thank you very much for the reply. I will do some research and report back

[xlcrr]

@kityan I decided to save zoom levels to DB

see https://openlittermap.com/global :-)

[kityan]

@xlcrr, well done! I don't think that it's the best solution (to store clusters to DB and not to serve them from standalone app), especially if you have to reindex clusters frequently. But, anyway. My backend is always nodejs, so it's more convenient to me.
P.S. I've remembered your project from angel.co, we've chatted there. If you will need any advice on my stack (see the link in github profile) — let me know.

[kityan]

@xlcrr P.S. A little more about pros and cons. When I use standalone app to handle frontend requests I can use getClusterExpansionZoom() to improve UX. I recieve bbox and zoom level from frontend. Then I get clusters with getClusters() and then patch data with info from getClusterExpansionZoom() for each cluster. Also you can decrease response size. GeoJSON is a bit verbose and if you communicate only with your own frontend, you can simplify json.
But I think you can patch data also before storing in database. Anyway, as a result you can make clusters clickable (jump to zoom level the cluster expands). It's much more user friendly.

[xlcrr]

@kityan I have recently launched OpenLitterMap as open source. You can see my backend PHP implementation here. Can't believe we chatted before that's great! Sounds like an awesome implementation, well done! We will see what the emerging OLM community can recommend as refactoring this atm is currently outside my timeframe

[yudeqang]

Have any postgis implementation?