Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

importVenues option do not work #515

Open
geopamplona opened this issue Jan 15, 2020 · 6 comments
Open

importVenues option do not work #515

geopamplona opened this issue Jan 15, 2020 · 6 comments

Comments

@geopamplona
Copy link

Hi


Here's what I did 😇

I have osm importer with this config:

{
    "logger": {
        "level": "warning",
        "timestamp": false
    },
    "esclient": {
        "apiVersion": "5.6",
        "hosts": [
            {
                "host": "elasticsearch"
            }
        ]
    },
    "elasticsearch": {
        "settings": {
            "index": {
                "refresh_interval": "10s",
                "number_of_replicas": "0",
                "number_of_shards": "1"
            }
        }
    },
    "api": {
        "autocomplete": {
            "exclude_address_length": 2
        },
        "targets": {
            "auto_discover": true
        },
        "services": {
            "pip": {
                "url": "http://pip:4200"
            },
            "libpostal": {
                "url": "http://libpostal:4400"
            },
            "placeholder": {
                "url": "http://placeholder:4100"
            }
        },
        "defaultParameters": {
            "focus.point.lat": 41.40019,
            "focus.point.lon": 2.17448
        }
    },
    "imports": {
        "adminLookup": {
            "enabled": true
        },
        "openstreetmap": {
            "leveldbpath": "/tmp",
            "datapath": "/code",
            "importVenues": false,
            "import": [
                {
                    "filename": "spain.pbf",
                    "importVenues": false
                }]
        },
        "whosonfirst": {
            "sqlite": true,
            "datapath": "/mnt/pelias/whosonfirst",
            "importVenues": false,
            "importPostalcodes": true
        },
        "services": {
            "pip": {
                "url": "http://pip:4200"
            }
        }
    }
}

I am using docker images
pelias/openstreetmap:master-2019-09-13-b1a69740ee6d34b98290d5554d80b692beddb7ad
pelias/elasticsearch:5.6.12

And use pip with whosonfirst data from Spain:
pelias/pip-service:master-2019-12-18-24a6e31f42b07adec218a6f5a1aebf45105d681e


Here's what I got 🙀

I see that venues and streets are loaded in elasticsearch; i am using a elasticsearch client to see data

imagen


Here's what I was expecting ✨

I config importVenues to false; but there are data from venues.
I want to do not load venues data; I dont want to use this data in Pelias.


Here's what I think could be improved 🏆

I dont know, maybe importVenues option do not work.

@stefanocudini
Copy link

I have same problem: Error: "imports.whosonfirst.importVenues" is not allowed

@stefanocudini
Copy link

is deprecated: pelias/whosonfirst#503

@orangejulius
Copy link
Member

orangejulius commented Sep 23, 2020

To clarify, imports.whosonfirst.importVenues and imports.openstreetmap.importVenues are two different config options, and only the first is deprecated.

Background on WOF venue data

The venue data in WOF comes from the open-sourcing of proprietary venue data collected by a long-gone company called SimpleGeo. The data didn't seem amazing to begin with, but it's gotten less and less valuable as time goes on as it gets out of date. It's from 2011 or earlier, so pretty old now.

The venue data in OSM is, of course, continuously kept up to date by the community. That venue data is pretty core to OSM, so the default is to import that data.

importVenues in OSM

To be honest, we consider not importing OSM venue data to be a pretty rare need, especially since you can always filter out that data at query time, and there aren't that many venues. But back in 2018 we merged a PR that came in to allow disabling that venue import (#382 and #467).

It looks like we have at least some tests for the behavior around the imports.openstreetmap.importVenues flag, but it's possible it doesn't work. We're open to confirmations of this original bug report or pull requests to fix any issues that come up. Let us know!

@stefanocudini
Copy link

stefanocudini commented Sep 24, 2020

@orangejulius thanks for the clarification,
I didn't understand where is a single spot to find all the updated pelias plugin options, it's a bit tricky every time to figure it out.
for example in this file there is no openstreetmap option you are talking about:
https://github.com/pelias/config/blob/master/config/defaults.json#L100

@missinglink
Copy link
Member

There is no single place documenting all the config options, I think this would be nice for users but it would also be difficult for us to maintain.

Each section of the config file defines the variables for a specific part of Pelias, so if you'd like to find the documentation for the imports.openstreetmap section then you'll find the information in the README for the OpenStreetMap repo here https://github.com/pelias/openstreetmap#configuration

The project is maintained by only three developers so we try to avoid duplicating documentation as it tends to mean more work for us to maintain.

@missinglink
Copy link
Member

@geopamplona I can see that maybe this line is causing the problem https://github.com/pelias/openstreetmap/blob/master/stream/pbf.js#L48

...but the edge case I'm considering would only be under these conditions:

  • you had multiple values listed under import.openstreetmap.import
  • the first entry either defines importVenues: true or importVenues: undefined

is it possible that your config isn't exactly as you posted above?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants