How to support interpolation for Berlin/Germany?

[jengeb]

Dear Pelias maintainers,

I use the dockerfiles repository and would actually like to set up a local instance of Pelias with the data of Berlin/Germany. I've already searched for possibilities to adjust the data pathes and files in a config file called pelias.json.
The interpolation part for Portland, Oregon is the following:

"interpolation": {
  "download": {
    "tiger": {
      "datapath": "/data/tiger",
      "states": [
        {
          "state_code": 41
        }
      ]
    }
  }
}

Until now I've found no way to configure this part for Berlin.

But I executed the following code of your readme and got this error message (I also tried using sudo):

$ curl -s http://missinglink.files.s3.amazonaws.com/berlin.gz | gzip -d > /tmp/data/berlin.0sv
$ bash: /tmp/data/berlin.0sv: Permission denied

Do you know a way to configure the interpolation for Berlin? I would be very happy if anyone could help me.

[otbutz]

Please check the permissions of /tmp/data:

ls -lah /tmp

Your user should have write permissions. This most likely the reason why you receive the Permission denied error.

[orangejulius]

Hi @jengeb,
TIGER is a dataset that is only relevant for the US. I'm not 100% sure but you might be able to remove the "tiger" section completely. @missinglink can confirm.

[jengeb]

Thank you very much for your answers. I am still a bit confused how to apply interpolation for a non US region or city. Starting with the pelias.json config file, I searched for and adjusted the imported datasets for openaddresses, polyline etc. I thought I could do the same with interpolation.
I read the How to guide, but only found instructions for using tiger and US data. Is there another guide explaining how to use interpolation with non US data?

[otbutz]

Is there another guide explaining how to use interpolation with non US data?

Here you go: https://github.com/pelias/interpolation#building-the-databases

tl;dr:

• create street.db from your polyline data
• use your street.db and your openaddresses data to create address.db
• calculate vertices using both address.db and street.db
• start your interpolation service

btw: OSM vs Openaddresses. I don't know how good the openaddresses dataset is for Berlin. I'd grab an OSM extract from geofabrik if you run into any problems.

[missinglink]

Hi @jengeb sorry for the late reply on this one.

Julian is correct in saying that the TIGER block range data comes from the US CENSUS and so is only relevant in the USA.

I'm the author of the interpolation library and I'm also based in Berlin, so you'll actually find a few test cases for Berlin in the test directory.

I saw you opened a few other issues and PRs across the Pelias org, if you're interested in contributing more to the project I'd be happy to meet you for a coffee to talk about your experience of installing/running the code and the requirements for your project.

[jengeb]

Hi @missinglink,

Thanks for your response. I set up a local instance of Pelias only including data for Berlin and Brandenburg for my project. The goal is to have a self-hosted version of Pelias running as a geocoder for a map application. This app will only use the endpoints for searching and autocompleting addresses.

After getting docker and the following command running (see #117 ) I could build the databases as described. Thanks @otbutz!

$ cat /data/brandenburg.polylines | docker run -i -v /data:/data pelias/interpolation polyline /data/street.db

I am using pelias.json of the pelias/dockerfiles repo as a configuration file. I still wonder if it is possible to just configure the interpolation part of pelias.json for my needs (Berlin, Brandenburg datasets) - all examples I could find use tiger and US datasets. So I guess I should leave the interpolation part of pelias.json empty after running the steps to build the databases. Do these steps actually import data and populate the elastic (or the db where all the other datasets specified in pelias.json are stored) with information to make use of interpolation by requesting the API?

Best,
Jenny

[nilsnolde]

Hi @jengeb,
I'm currently wondering the same thing. Actually the exact same use case: Berlin, docker-compose images, wanting to use oa/osm interpolation. It's testing really, will set up a proper Pelias instance once the full capabilities become clear to me. Still bugging me though..

Did you get interpolation to work within the docker-compose pelias.json?

[missinglink]

Hi @jengeb, Sorry I didn't see your question.

The interpolation service is a standalone service from Pelias, so it's built separately and runs in a different process.

There are two configuration sections in the ~/pelas.json file which relate to interpolation, the imports.interpolation section (at the bottom of the file) relates to build-specific settings.

Since I designed it to be standalone from pelias, that section doesn't really have a lot of options, it's assumed that you follow the directions in the readme for pelias/interpolation to build the index.

There is one setting which made it in there and it relates to TIGER, which is US-only, so you can leave that whole thing empty.

The second related section is: https://github.com/pelias/dockerfiles/blob/master/pelias.json#L29-L31

This tells the pelias/api code (the HTTP server) where it can find the running pelias/interpolation server on the network. If you have this enabled it can be used to improve matching accuracy.

It is currently only enabled for one query type:

• If a user requests an address and elasticsearch returns a street, send a second request to the interpolation service for a result. If successful, use the interpolated result, otherwise use the street centroid.

This also only applies to the /v1/search endpoint and not to the autocomplete endpoint at this stage.

We had plans to further develop the integration between the two services but when Mapzen was shut down, those plans were put on hold.

The interpolation service offers a bunch of other APIs such as nearest street (with projected accuracy) and other functions which return more accurate results than elasticsearch, which only works off centroids.

[missinglink]

here's some examples of what the service can do:

• https://interpolation.synergy.io/search/geojson?lat=52.500545&lon=13.313151&number=140&street=kurfursten%20damm
• https://interpolation.synergy.io/street/near/geojson?lat=52.498010244522554&lon=13.295942957047375

note: this dev server is running off a toaster laptop in my kitchen

[jengeb]

@missinglink: Thanks a lot for the detailed explanation.

@nilsnolde: I've left the interpolation part of the imports section in pelias.json out and built the index for interpolation as described here.