test item metadata on the master

[RonnyPfannschmidt]

to sanely address #17 and #18 its very helpful if the master can know about fixtures and fixture scopes

[nicoddemus]

Here's an idea:

Create a hook that is called after collection on the slaves, something like:

def pytest_xdist_collect_modify_meta(item, meta):
    """called during collection for each test item. Plugins can
    then update the `meta` dictionary with information from the collected item
    that will be sent back to the master worker.

    keys and values in the `meta` dictionary can contain only 
    builtin types (strings, ints, tuples, dicts, lists).
    """

Each worker after collection (probably in pytest_collection_modifyitems) would keep a dict of testid(str) -> meta(dict), and call the hook above in all registered plugins.

xdist would have an implementation which puts fixture-related information into the meta dictionary, but other plugins have the chance to add more information as well that can then be used later by the master worker to decide on how to divide the tests into chunks.

The master worker would then call a separate hook to divide the collected test into chunks:

@hookspec(firstresult=True)
def pytest_xdist_divide_chunks(items, meta_dict):
    """called by the master worker, plugins can override to split the list of 
    items into chunks. 

    `meta_dict` is a dict of `item-nodeid(str) -> meta(dict)`

    This hook should return a list of chunks of test items. Each chunk is 
    guaranteed to execute in the same slave. As an special case, a "chunk"
    might be an item instead of a list.

    For example, suppose a hook receives a list of items as such (assume the 
    numbers are actually nodeid strings): 

        [0, 1, 2, 3, 4, 5]

    And wants tests 0, 1, and 2 to execute in the same slave, it should return:

        [[0, 1, 2], 3, 4, 5]                
    """

xdist default implementation would by default separate tests according to their fixture's scope as proposed in #18, but other plugins might want to override that behavior.

The idea here is to add some flexibility to xdist on how to collect meta-data and divide tests into chunks, so others can override the default behavior if needed.

A few more considerations:

• Perhaps as optimization pytest_xdist_collect_modify_meta could be called
  only from one of the workers (say, the fist worker) since they are all supposed
  to produce the same meta-data information.
• Hook names, parameters and terms are all open for suggestion.

cc @nchammas @hpk42

[RonnyPfannschmidt]

Good work on the draft

All workers should collect all metadata,
As it might differ between hosts

The hook should not be allowed to modify, only add