Tuple field map in read view

[nshy]

Reviewers

• Main reviewer: @locker
• Team Lead: @sergepetrenko
• CTO: @sergos

Problem statement

We don't have field map in read views and as a result accessing tuple fields is slow (see also #10429). The issue is read views use in_gc field of struct memtx_tuple which overlaps with format_id of struct tuple.

tuple structs

struct PACKED tuple
{
        uint8_t local_refs;
        uint8_t flags;
        uint16_t format_id;
       <snip>
};
struct PACKED memtx_tuple {
        union {
                struct {
                        uint32_t version;
                        struct tuple base;
                };
                struct stailq_entry in_gc;
        };
};

Solution

Use 48bit (6-byte) pointer for in_gc. What?

48 bit limit is quite large (256 TiB) to address memtx memory. So we can make memtx arena a single chunk of memory and then deal with offset inside arena instead of dealing with pointers for linking tuples in in_gc.

Currently memtx arena is not a single chunk in general. After first box.cfg{} it is a single chunk of memtx_size but if we increase memtx_size after then this property is lost. We can allocate for memtx arena a chunk of size near to available physical memory on box creation and only increase quota if memtx memory is increased. It is possible in terms of OS memory management API.

The solution is suggested by @locker.

Alternatives

1. Increase memtx tuple size on 2 bytes

memtx_tuple changes

                        uint32_t version;
+                       uint16_t padding;
                        struct tuple base;

Unfortunately the memory usage is increased. I tested insertion of 1M tuples consisting of 1, 2, 4 or 8 32-bit values and also tuples consisting of 32-bit value and a string of length from 10 to 100 bytes. The memory usage is recorded on fresh Tarantool start for each variant.

Test results.

tuple	current memory usage, bytes	new memory usage, bytes	differences
1 x 32-bit	15996744	23992856	+50%
2 x 32-bit	23993048	23993048	0%
4 x 32-bit	31988456	39984360	+25%
8 x 32-bit	55972056	55972056	0%
32 bit + str	76126512	78053232	+3 %

test script

local fiber = require('fiber')
box.cfg{
    wal_mode = 'none',
    memtx_memory = 1024 * 1024 * 1024,
    log_level = 'error',
}
print('started')
local s = box.schema.create_space('test')
s:create_index('pk')

local mem_usage = function()
    return box.slab.info().items_used
end

local rand_uint = function()
    return math.floor(math.random() * 2^30)
end

local before = mem_usage()
box.begin()
local max = 1000000
for i=1,max do
    -- s:replace{rand_uint()}
    -- s:replace{rand_uint(), rand_uint()}
    -- s:replace{rand_uint(), rand_uint(), rand_uint(), rand_uint()}
    -- s:replace{rand_uint(), rand_uint(), rand_uint(), rand_uint(),
    --           rand_uint(), rand_uint(), rand_uint(), rand_uint()}
    s:replace{i, string.rep('a', math.random(10, 100))}
    if i % 1000 == 0 then
        box.commit()
        fiber.yield()
        box.begin()
    end
end
box.commit()
print(mem_usage() - before)
os.exit()

2. Do not use intrusive linking for tuples in read views

We can use a mempool to store a pair of pointer to tuple with a link for stailq.

Pros and cons:

• +memory usage is increased only for read views

• -overhead is 16 bytes per tuple

[sergos]

I personally don't like the pre-allocation obligation that imposed by the proposed approach. It means the <engine>_memory option become static, while so far one can extend the arena size at runtime.