numGroups / numVarDataFields support (SBE spec v2)

[estaban]

The 1.0 specification mentions that the following attributes are part of message headers

Number of repeating groups
A count of repeating groups at the root level of the message. The count does not include nested repeating groups.

Number of variable-length fields
A count of the variable-length fields at the root level of the message. The count does not include variable-length fields within repeating groups.

When generating Java encoder/decoder for this schema using 1.33, numGroups and numVarDataFields do not seem to have any special significance.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<sbe:messageSchema xmlns:sbe="http://fixprotocol.io/2016/sbe"
                   package="uk.co.real_logic.sbe.benchmarks"
                   id="1"
                   version="1"
                   semanticVersion="5.2"
                   description="Example schema"
                   byteOrder="littleEndian">
    <types>
        <composite name="messageHeader" description="Message identifiers and length of message root">
            <type name="blockLength" primitiveType="uint16"/>
            <type name="templateId" primitiveType="uint16"/>
            <type name="schemaId" primitiveType="uint16"/>
            <type name="version" primitiveType="uint16"/>
            <type name="numGroups" primitiveType="uint16" />
            <type name="numVarDataFields" primitiveType="uint16" />
        </composite>
    </types>

    <sbe:message name="Car" id="1" description="Description of a basic Car">
        <field name="serialNumber" id="1" type="uint32"/>
    </sbe:message>
</sbe:messageSchema>

Generated Java code:

    public CarEncoder wrapAndApplyHeader(
        final MutableDirectBuffer buffer, final int offset, final MessageHeaderEncoder headerEncoder)
    {
        headerEncoder
            .wrap(buffer, offset)
            .blockLength(BLOCK_LENGTH)
            .templateId(TEMPLATE_ID)
            .schemaId(SCHEMA_ID)
            .version(SCHEMA_VERSION);

        return wrap(buffer, offset + MessageHeaderEncoder.ENCODED_LENGTH);
    }

Are numGroups / numVarDataFields not supported ?

[ZachBray]

I believe the specification repository was modified after version 1 was "cut". The modifying commit refers to changes since v1. It is a bit confusing.

EDIT: The published 2020 version of the SBE v1 specification does include these header fields.

AFAIK, the specification for SBE v2 is still in the release candidate phase. It would be useful to adopt these header fields (and changes to repeating group headers), and adjust our implementation, as they permit less restricted schema extension (w.r.t. groups and variable length data). However, adopting these changes is not currently on our roadmap.

[ZachBray]

Actually, I was more correct before the edit, i.e., it is part of the v2 specification. Please see this issue: FIXTradingCommunity/fix-simple-binary-encoding#157