Bad input metrics if bidder is disabled and stored impression still contains bidder

[muuki88]

Hi,

We found an unfortunate behaviour when it comes down to enabling/disabling bidders. The scenario is the following

1. A bidder is enabled
2. Bidder configuration is added to stored impressions
3. The bidder is disabled via the config file, but the stored impressions are left unchanged

We observed that this triggers the bad-input metrics in prebid server, which is not false, but also super annoying. If we disable a bidder, it's okay that the stored impressions that still contain the bidder config, may be "bad input", because the bidder is disabled.

Why is this important? Disabling a bidder should not trigger an entire reconfiguration of all stored impressions. This makes it very hard to turn of a bidder, maybe because it's having latency issues or other technical issues.

[bsardo]

@muuki88 We discussed this in backlog meeting. We believe this is behaving as intended. The incoming request is merged with the stored request to form the final incoming request and if that final incoming request contains information for a bidder that is disabled we believe it should be considered bad input. A couple of options were discussed:

1. We could analyze the stored request prior to merging with the incoming request to see if the stored request contains any bidders that are disabled. This would have a performance impact, at least on the PBS-Go side, since the merge operation is done on raw JSON.
2. Perhaps it would be easier to just analyze all of your stored requests in your backend storage and make adjustments there a single time rather than doing it per request?
3. We could make modifications to the stored request just prior to loading it into your local cache stripping out any disabled bidder information. This option seems a bit unorthodox. It'll be hiding the issue.

Thoughts?

[muuki88]

Hi @bsardo

Thanks for the detailed response ☺️

We believe this is behaving as intended.

IIRC prebid.js acts similarly by logging a warning. The tricky thing is that this can be intentional or unintentional.

And I get that it's better to have a noisy warning, because this can potentially hurt revenue.

We could analyze the stored request prior to merging with the incoming request to see if the stored request contains any bidders that are disabled.

I agree that this is not worth the performance cost

Perhaps it would be easier to just analyze all of your stored requests in your backend storage and make adjustments there a single time rather than doing it per request?

Sure. The thing is that this would mean that we have to change thousands of stored impressions when we want to disable a bidder. It would be okay if a bidder is removed permanent, but for short time disabling, this would be tricky.

We could make modifications to the stored request just prior to loading it into your local cache stripping out any disabled bidder information. This option seems a bit unorthodox. It'll be hiding the issue.

This is a good point. Maybe I'm mixing up two different use cases. The first one being "remove this bidder permanently". This should update all stored impressions.

The other being "disable this bidder temporarily for reasons, e.g. bad behavior, technical issues or even AB testing". This is more a "blocking bidder" feature and there's already a well written draft by Bret ( on my phone right now, not easy to find a link )

[bretg]

When we start removing bidders for efficiency reasons, we won't want false metrics. Will bring this up in a future PMC.

[bretg]

Discussed alert when bidder requests are being created.

Scenarios:

• seeing a request with an explicitly disabled bidder - we propose shifting this to a different metric. Instead of bad-input, shift this to a new disabled-bidder metric. This is a per-adapter metric.
• seeing a request with an unknown bidder. This has been changed recently with Stop hard-failing on unknown imp.ext values prebid-server#3735. It's now a warning -- no metrics.
• seeing a request with a bidder that was removed by a module. This isn't valid as the bidder won't exist since the module removed it.

@muuki88 - this was approved by the committee. Community PR welcome.