due to deprecation of /api/instr-list in dispatcher, oda-api can't get list of instruments

[dsavchenko]

No description provided.

[volodymyrss]

I thought we made a redirect? We want if possible to make old versions compatible.
It does not work with old-style url? Both should.

[dsavchenko]

I doesn't, there is a strict check for the response code 200.
I will adapt then

[burnout87]

Indeed, both versions should work.

[volodymyrss]

requests should be able to follow redirects, there is even some test https://github.com/oda-hub/oda_api/blob/master/tests/test_remote.py

[dsavchenko]

That's weird, I definitely encountered problems with, but now I rechecked manually again everything is OK.
Probably, I was on the wrong branch somewhere...
Sorry to bother.

[dsavchenko]

I found the problem, which I actually encountered. Dispatcher redirects /api/instr-list to the absolute url, constructed using products_url config option, which "should coincide with ingress for frontend" according to a comment there.
So everything works fine with production.
But when trying it locally, without running frontend, it fails, of course. I just didn't read the error message carefully.

It's probably an unwanted behaviour of dispatcher, which add ties to the frontend configuration in the place where it should not exist. Please make issue if you agree

[volodymyrss]

Dispatcher needs to send full urls in emails, so it needs to know the frontend url. There is not necessarily a way for dispatcher to learn about this url (it might come in headers or might not).

It would be best if the redirect did not depend on this though, I'd think relative url redicrect might even work. But there was some issue with that. If you want, give it a try. Also maybe it's good if @burnout87 comments, I forgot the details.

[burnout87]

In the code, in case the product_url is set and is a valid url, this is used. In case it's not, the request.url is used.

I encountered the second option within my development setup, more specifically when running the various tests, because in that case the products_url is either not set or an invalid url https://github.com/oda-hub/dispatcher-app/blob/5f901f8831d09ba618bfd1513c648a956f1a3657/cdci_data_analysis/pytest_fixtures.py#L366-L395. I also developed a number of tests using different possible values to assign to it.

So I am curios to know which value of products_url you have set, in this way we can extend also to your case.

[dsavchenko]

It is set to my local front-end URL, which is on the other port of localhost. So if it runs, everything works. If not, it fails.
In standard installation it's ok and in installation without frontend, this url may be set accordingly. But imagine there is installation with frontend but also with direct url of dispatcher, then in case of frontend fail, dispatcher becomes also badly functional.
But probably there are other places with such coupling.

[dsavchenko]

With products_url = None, this problem doesn't appear, but I'm not completely sure, if it will not affect any frontend-unrelated functionality

[burnout87]

It is set to my local front-end URL, which is on the other port of localhost.

So an example could be: localhost:1234 ? To have an idea so that I could try also this case

[dsavchenko]

Yes. The example is http://localhost:1234/mmoda/
Works well, until frontend there is down for some reason. Dispatcher still accessible and functional at e.g. http://localhost:8000 except for this endpoint.

My only point here is that this redirection should not depend on frontend by any means

[volodymyrss]

My only point here is that this redirection should not depend on frontend by any means

It would be best if the redirection works with relative path. I thought I would, but somehow it did not workout in this case, not sure, right @burnout87 ? If you want to propose, go ahead. Or maybe later if needed.

[dsavchenko]

Probably related thing I thought about: if we have several frontends, each requesting the same dispatcher, what is the proper value for product_url option?

[dsavchenko]

Getting back to this.
When testing dispatcher on localhost without any frontend available (hence no proxy with url of the form */dispatch-data) I set product_url = None in config for the redirection to be relative to the request https://github.com/oda-hub/dispatcher-app/blob/fe7aaf5fee8037050ba35c5a646b2c2278f88042/cdci_data_analysis/flask_app/app.py#L103.

But then there is exception at https://github.com/oda-hub/dispatcher-app/blob/fe7aaf5fee8037050ba35c5a646b2c2278f88042/cdci_data_analysis/analysis/instrument.py#L473 trying to do None + string

In general, hardcoding /dispatch-data subpath in the dispatcher code is not good. We can have it inside the product_url parameter. Then everything can be configured consistently, even for the installation without frontend.

[burnout87]

Thanks for the detailed explanation.

But then there is exception at https://github.com/oda-hub/dispatcher-app/blob/fe7aaf5fee8037050ba35c5a646b2c2278f88042/cdci_data_analysis/analysis/instrument.py#L473 trying to do None + string

In general, hardcoding /dispatch-data subpath in the dispatcher code is not good. We can have it inside the product_url parameter. Then everything can be configured consistently, even for the installation without frontend.

At this point, I'd rather have the dispatch-data subpath in a dedicated config entry (eg dispatch-data-path). Then, check for None the back_end_query.config.products_url, and if that is the case, simply assign it an empty string. Same principle can be applied to the dispatch-data-path config if we decide to introduce it.

My question would be, are there situations where we need the products_url config without the dispatch-data subpath? I already took a look at the email_helper and over there we use it frequently without adding the dispatch-data subpath. Instead, by keeping those two parts separated into two different config entries we won't have problems in situations like those.

But yeah, then there is the added work to do replace every dispatch-data usage with the dedicated config entry.

What do you think?