Conventions and controls for adapters responding with multiple biddercodes

[bretg]

The Prebid.js committee has been working through scenarios where a given bid adapter can return bids for biddercodes not defined in the original AdUnit. This issue describes them as "extra" bids -- prebid/Prebid.js#8129

Use cases:

• bid adapters that own multiple exchanges (e.g. triplelift, grid)
• bid adapters that multiplex other bidders behind them

PBJS is proposing that two fields can be used to control and track these scenarios:

1. biddercode - this is the entity making the bid. Currently the PBS 'seat' is used to fill this field.
2. adaptercode - indicates the bidder code that was in the adunit. This might be an alias of an underlying bid adapter. e.g. it would be "pubmatic" and not "groupm". This is a new field that would be set to the label used by the adunit.

First phase

1. We extend the metadata object to include an adaptercode field, but have PBS-core fill it rather than relying on the adapter. PBS-core fills seatbid.bid.ext.prebid.meta.adaptercode with the biddercode as defined in imp[].ext.prebid.bidder.BIDDERCODE. Even if it's an alias, that's fine.
2. PBS-Core defines a way for adapters to set the seat (seatbid.seat) and does not initially validate what "seat" is set by the adapter. If the seat is not set, then PBS-core should set it to the biddercode.
3. Targeting values created for the includebidderkeys scenario always use biddercode for the suffix. The value of the hb_bidder key for the overall winner is also the biddercode.

Second phase

There is concern over SPO in these cases (e.g. bid jamming). Or some kind of fraud, e.g. adapterA falsely bidding for adapterB to confuse algorithms. So host companies and publishers may want to control which adapters are trusted to do this, and possibly which biddercodes are returned from each.

PBJS defines parameters for controlling this behavior, so PBS could as well. The proposal is to make it part of account config and default account config:

account.alternatebiddercodes.enabled: true/false // the default for adapters not specifically named below
account.alternatebiddercodes.bidders.BIDDER.enabled: true/false
account.alternatebiddercodes.bidders.BIDDER.allowedbiddercodes: ["bidderX", "bidderY"] // defaults to all

1. if the bid response seat is different than the expected adapter code:

• check account.alternatebiddercodes.enabled. If it exists:
  • If false, reject. (see below)
  • if true, then check account.alternatebiddercodes.bidders.BIDDER.allowedbiddercodes.
    • If doesn't exist or ["*"], then allow.
    • Else, check that biddercode is on the allowedbiddercodes list. If true, allow. If false, reject. If anything is rejected, create a warning when in debug mode. Log a warning at N% and emit a metric for "alerts.response"
• check account.alternatebiddercodes.bidders.BIDDER.enabled. If it exists, do the same as above.

2. Update the bid adjustment factors feature. If request.ext.prebid.bidadjustmentfactors exists:

• check for the existence of the seat-defined-bidderCode in the bidadjustmentfactors object. If it exists, apply the adjustment and we're done.
• otherwise, check for the existence of the adaptercode (seatbid.bid.ext.prebid.meta.adaptercode) in the bidadjustmentfactors object. If it exists, apply the adjustment

If a bidder tries to set a seat that's different than the biddercode in the imp but is not allowed to do so, reject it. Log a debug error and an N% sampled log entry and metric: adapter.BIDDER.response.validation.seat. The log entry should mention the account ID, the adaptercode, and the biddercode rejected.

See the PBJS issue for test cases.

For example, let's say bidderA returned 3 bids: one for bidderA, one for bidderB, one for bidderC.

1. account.alternatebiddercodes.bidders.bidderA.enabled is true
2. account.alternatebiddercodes.bidders.bidderA.allowedbiddercodes: ["bidderA", "bidderB"]
3. the bidderA adapter returns the 3 bid responses with different seats for each.
4. PBS-core examines the bid responses and throws out the bid for bidderC based on the config. This rejection is logged to ext.warnings. It emits an alerts.responses error and N% of the time it also sends to the log.
5. The final set of bid responses includes two entries -- seatbid[].seat: "bidderA" and seatbid[].seat: "bidderB". Both contain seatbid[].bid.ext.prebid.meta.adaptercode: "bidderA"

[bretg]

Discussed in PBS committee.

[bretg]

Updated to reflect recent clarifications on the PBJS issue.

[pm-nilesh-chate]

Hi @bretg,

Could you please help me with a few doubts mentioned below?. I have started working on the implementation and need clarity on a few things.

1. Shouldn't the example in the description have adapterCode value as "pubmatic" and not "groupm"?. As, Pubmatic is the incoming/requester adapter here who makes the request and groupm is the the bidder-name that we intent to use for a selected bid(s)?

2. Does this example correctly depicts the expected changes in the PBS response?.

{
    "id": "test-request-id",
    "seatbid": [
        {
            "bid": [
                {
                    ...
                    "ext": {
                        ...
                        "prebid": {
                            "meta": {
                                "adapterCode": "pubmatic"
                            }
                            ...
                        }
                    }
                }
            ],
            "seat": "groupm"
        },
        {
            "bid": [
                {
                    ...
                    "ext": {
                        ...
                        "prebid": {
                            "meta": {
                                "adapterCode": "pubmatic"
                            }
                            ...
                        }
                    }
                }
            ],
            "seat": "pubmatic"
        },
        {
            "bid": [...],
            "seat": "appnexus"
        }
        ...
    ],
    "cur": "USD"
}

Here:

• Adapter sets preferred seat for each bid. Ex. PubMatic sets groupm as seat for a few bids.
• PBS-Core validates if groupm as bidder-name is allowed and then splits the pubmatic adapter bids into two seatbids, one for pubmatic bids and other for groupm.
• PBS-Core updates the adapterCode for each bid . Ex. adapterCode for bids under seat pubmatic and groupm is pubmatic.

3. How do we deal with targeting key for such bids?. Would they be considered part of adapter pubmatic or do we introduce a runtime dummy adapter called groupm?.
4. This feature would also need changes at Prebid-JS to parse these fields, wouldn't it?

[bretg]

@pm-nilesh-chate

Shouldn't the example in the description have adapterCode value as "pubmatic" and not "groupm"?

Agreed. Sorry. Fixed.

Does this example correctly depicts the expected changes in the PBS response?

Almost. 'adapterCode' should be all lowercase... 'adaptercode'. We're trying to be consistent about all new extension values being strictly lowercases.

How do we deal with targeting key for such bids?

Updated the description to define that targeting keys are always based on the biddercode.

This feature would also need changes at Prebid-JS to parse these fields, wouldn't it?

As noted in prebid/Prebid.js#8129

This means that for now, the existing 'allowUnknownBidderCodes' is how pubs can validate/enforce what comes back from PBS. In the future, once PBS reports both seatbid.seat and seatbid.bid.ext.prebid.adapterCode, the PBS bid adapter could implement the allowAlternateBidderCodes logic.

[bsardo]

@bretg,

If account.alternateBidderCodes.enabled does not exist and account.alternateBidderCodes.adapters.BIDDER.enabled does not exist, should we default to allowing all alternate bidder codes for the bidder?

Also, from some of the comments in the Second Phase snippet above, it appears account.alternateBidderCodes.adapters.BIDDER.enabled takes precedence over account.alternateBidderCodes.enabled if they both exist. Is that correct?

Here is how I see the logic based on description above:

if account.alternateBidderCodes.enabled does not exist and 
   account.alternateBidderCodes.adapters.BIDDER.enabled does not exist:
         return allow
if account.alternateBidderCodes.adapters.BIDDER.enabled exists and 
   account.alternateBidderCodes.adapters.BIDDER.enabled = false:
        return disallow
if account.alternateBidderCodes.adapters.BIDDER.enabled exists and 
   account.alternateBidderCodes.adapters.BIDDER.enabled = true:
        return ValidBidderCodeHelper(bidder, code)
if account.alternateBidderCodes.enabled exists and 
   account.alternateBidderCodes.enabled = false:
        return disallow
return allow   // account.alternateBidderCodes.enabled exists and = true:

func ValidBidderCodeHelper:
    if account.alternateBidderCodes.adapters.BIDDER.allowedBidderCodes does not exist:
        return allow
    if account.alternateBidderCodes.adapters.BIDDER.allowedBidderCodes contains "*":
        return allow
    if account.alternateBidderCodes.adapters.BIDDER.allowedBidderCodes[bidderCode] is found:
        return allow
    return disallow

[pm-nilesh-chate]

Hi @bsardo,

I had the initial code changes similar to your explanation but later updated those after discussion with @hhhjort. Refer thread #2257 (comment).

If we assume that an empty AllowedBidderCodes set means all bidder codes are allowed, which is a change from today's behavior. This requires PBS hosts to make a positive entry for every adapter to disable the new behavior, and continue creating a new entry every time a new adapter is released/enabled. I think we want to require a positive change for each adapter, like setting "*", as enabling this functionality requires additional trust in the behavior of an adapter that was not needed before. Or possibly allowing a default allowBidderCodes that determines what to do with an empty map.

To summarise, we should keep the PBS behaviour as is for the publishers after they update their PBS (not break reports, etc).
Anyone who wants to use extra-bid feature should explicitly enable it.

[bsardo]

Ah thanks @pm-nilesh-chate for pointing me to that thread. So the multi-bid feature in PBS-Go should be opt-in for various reasons including backwards compatibility and security. Sounds good to me.

[bretg]

From Prebid Slack:

Nilesh writes:

PBS host can now define a list of extra-bidders that an adapter is allowed to return along with its own bids. However, the SSPs are unaware of these extra bidders that are allowed by the PBS host. Here, the best SSPs can do is send extra extra-bids in every response hoping those would be consumed. Instead, would it be a good idea, if SSPs are aware of these extra-bidders? so that SSPs can make a wise decision to send extra bids only if they are allowed.
Ex. PubMatic SSP would send extra bids for bidder 'groupm' only if the request had groupm bidder in the altenatebiddercode list.

The proposal is to pass alternatebiddercodes.adapters into ExtraRequestInfo

[bretg]

Lots of interest in this feature lately. Besides adapters wanting to know about the alternatebiddercode config, there's now interest in being able to pass this info on the request. e.g. if PBJS allowUnknownBidderCodes is true.

So expanding this proposal so the config becomes formally part of the Prebid ORTB extensions. Here's a proposal:

1. incoming ORTB could contain:
   ext.prebid.alternatebiddercodes.enabled: true/false
   ext.prebid.alternatebiddercodes.bidders.BIDDER.enabled: true/false
   ext.prebid.alternatebiddercodes.bidders.BIDDER.allowedbiddercodes: [array, of, strings]

2. This would override the account-level config

3. If the request doesn't specify these values, then PBS-core would merge the account config into these values

4. Before sending to each adapter, PBS-core would need to remove entries that don't belong to that bidder. i.e. they would only receive ext.prebid.alternatebiddercodes.bidders.ME

5. Everything else is up to the adapter. They can ignore or use at will.

Heads up @pm-nilesh-chate

[pm-nilesh-chate]

Current behaviour in PBS-Go

account.alternateBidderCodes.enabled: true/false is the feature toggle.

• If false, reject the alternate bidder. (default would be false to keep things running as is after upgrade)
• If true, check value of account.alternateBidderCodes.adapters.BIDDER.allowedBidderCodes:
  - * would allow all the extra bids.
  - Empty/not-present would reject all the alternate bidder.
  - Presence of bidder array would allow bids only from those bidders.

Example pbs.yaml:

account_defaults:
  alternatebiddercodes:
    enabled: true
    adapters:
      pubmatic:
        allowedbiddercodes: [groupm]
      appnexus:
        allowedbiddercodes: [*]

[bsardo]

Ah yes, we kept the top level alternatebiddercodes.enabled flag but figured it wasn't necessary to have an enabled flag per adapter since we can just look at allowedbiddercodes.

IMO, I found it a bit verbose to have to say I want to enable the feature for a given adapter when I think that is implied if bidder codes have been specified but perhaps there's a scenario I'm missing where it would be of value.

[bretg]

The config here was meant to mirror what was done in Prebid.js. Those of us that work with both platforms prefer consistency where possible.

[bretg]

FYI - as discussed in last week's committee meeting, the config has been changed from

account.alternateBidderCodes.adapters.BIDDER.enabled

to

account.alternateBidderCodes.bidders.BIDDER.enabled

[bsardo]

@bretg from the description:

if true, then check account.alternateBidderCodes.bidders.BIDDER.allowedBidderCodes.

• If doesn't exist or ["*"], then allow.
• Else, check that bidderCode is on the allowedAlternateBidderCodes list. If true, allow. If false, reject. If anything is rejected, create a warning when in debug mode. Log a warning at N% and emit a metric for "alerts.response"

If I'm reading this correctly, the absence of allowedBidderCodes would result in an allow while an empty allowedBidderCodes would result in a rejection correct?

[bretg]

correct

[bsardo]

Lots of interest in this feature lately. Besides adapters wanting to know about the alternatebiddercode config, there's now interest in being able to pass this info on the request. e.g. if PBJS allowUnknownBidderCodes is true.

So expanding this proposal so the config becomes formally part of the Prebid ORTB extensions. Here's a proposal:

1. incoming ORTB could contain:
   ext.prebid.alternatebiddercodes.enabled: true/false
   ext.prebid.alternatebiddercodes.bidders.BIDDER.enabled: true/false
   ext.prebid.alternatebiddercodes.bidders.BIDDER.allowedbiddercodes: [array, of, strings]
2. This would override the account-level config
3. If the request doesn't specify these values, then PBS-core would merge the account config into these values
4. Before sending to each adapter, PBS-core would need to remove entries that don't belong to that bidder. i.e. they would only receive ext.prebid.alternatebiddercodes.bidders.ME
5. Everything else is up to the adapter. They can ignore or use at will.

@bretg regarding the above requirements described here I'd just like to clarify a couple of things:

1. If alternateBidderCodes isn’t set in the account or on the original request, do you think we should omit alternateBidderCodes from the bid requests rather than passing an alternateBidderCodes object with alternateBidderCodes.enabled = false?
2. It appears from your use of the word merge that alternateBidderCodes for a given bidder request should match what was on the incoming request with the only adjustment being that the other bidders were removed, is that correct? Perhaps there is value in a bidder knowing the exact configuration received by PBS? I ask this because there are multiple ways to represent the same information to a bidder and theoretically we could express the intent to a bidder in a different way from how it was expressed in the original request. For example, there are multiple ways we could convey that all alternate bidder codes are allowed for a given bidder in a bidder request:
   a) top level enabled = true with no bidders defined
   b) top level enabled = true with the bidder defined with a wildcard for the acceptable alternate bidder codes
   c) top level enabled = true with the bidder defined with missing alternate bidder codes array

Thanks in advance

[bretg]

If alternateBidderCodes isn’t set in the account or on the original request, do you think we should omit alternateBidderCodes from the bid requests rather than passing an alternateBidderCodes object with alternateBidderCodes.enabled = false?

Omit is correct. Honestly I'm not sure why this is a question. I didn't suggest PBS should create this field, but anyhow, we're on the same page.

It appears from your use of the word merge that alternateBidderCodes for a given bidder request should match what was on the incoming request with the only adjustment being that the other bidders were removed, is that correct?

Correct.

[bsardo]

Closing as this is complete in PBS-Go. Alternate bidder codes are supported in the account config and on the request with priority given to the request. Bid adjustment factors take alternate bidder codes into account by first attempting to using any adjustment factor specified for the seat before falling back to any adjustment factor associated with the underlying adapter.