Why we should not be using in-memory cache in the runtime (in most cases)

[MaksymZavershynskyi]

Current state

Currently, Contract Runtime team is working on the following changes:

• Making Wasm contract compilation happening during the deployment and the state sync. And them making compiled Wasm contract to be stored on the disk and retrieved from it during function calls;
• Improving the time for retrieving Wasm contract from disk and recomputing function_call_cost so that it reflects the actual cost of retrieving the contract from the disk. The new function_call_cost cannot be larger than the current one, since we cannot be increasing the gas costs on our blockchain, or otherwise we can accidentally break some contracts. Therefore if the new cost turns out to be larger, then we need to work on optimizing loading of Wasm modules until it does not;

Also, while we are working on these changes, we have temporary added in-memory cache for contract compilations. This allows us to hide the problem that function_call_cost is underestimating the time it takes to load the contract, which leads to the slow-down of block production when we have full blocks.

Proposal

Once these changes are implemented we should remove any in-memory cache for compiled contracts. We need to do it because of the two fundamental runtime rules:

• Our gas fees are estimated based on the worst-case scenario.
• Also, our TPS is determined by the gas fees. Specifically, the maximum theoretical limit for Wasm function calls is 5*10^14/(function_call_cost. send_not_sir+function_call_cost.execution).

It seems attractive to increase execution speed of certain contracts by either adding a LRU cache or having some Wasm contracts (like EVM) white-listed to be permanently in cache. This however, does not decrease our fees, and therefore it does not increase TPS. This however unfortunately hides any issues that we might potentially have if we have underestimated the costs of certain operations and assigned fees that are too low. The reason why we have noticed that function_call_cost is too low, is because we did not have in-memory cache and it slowed down our network. With in-memory cache the network might not have been slowed down, but it still would've been vulnerable to cold-cache attacks.

[MaksymZavershynskyi]

To clarify, I am not saying we should deprioritize other tasks like adding diagnostics. However, if it turns out that after switching to Wasmer 1.0 and removing in-memory cache for contract compilation our new function_call_cost is higher than our current function_call_cost, then we should definitely work with Wasmer or by ourselves on improving it. Otherwise, we are vulnerable to cold-cache attacks.

[ailisp]

Related work status:

• (wip) deploy time contract compilation:
  Cache contract on deployment. #3859
• (not started) contract compilation during state sync
• Improving the time for retrieving Wasm contract from disk and recomputing function_call_cost
  • (wip) by a different execution model: pre-deserialize/prepare contract in another thread: Concurrent preloader for contracts #3985
  • by optimization on wasmer side:
    • (not started by wasmer) Todo items for wasmer: https://docs.google.com/document/d/17fBMmurSqkeIlQsfxouhrxRoAool9eQRTC6F6OgBTjQ/edit
    • (wip) Use borsh to deserialize JIT Artifact wasmerio/wasmer#2119
    • (not started) use Abomonation crate instead of borsh, raw memmap maybe faster

[bowenwang1996]

I want to mention a couple things here:

• With the introduction of on-disk cache, we realized that deserialization from disk becomes a bottleneck. Even with the optimization that @ailisp has done, it still takes 3-4ms to deserialize a contract that is ~300kb (which is the size of a typical contract we have today) and that is quite suboptimal and it means that if we measure the gas cost correctly (i.e, taking this deserialization cost into consideration), we will end up not being able to do more than 300 function calls per block, even if they are in fact very cheap function calls. Therefore, it is desirable if we can find ways to speed this up.
• I definitely agree that gas cost and therefore tps depends on the worst case scenario and that is exactly what I told @olonho. However, during our discussion @olonho and I came up with an idea: we can potentially always prefetch contracts before the chunk is applied if we can guarantee that the contract is always in memory when the receipt is processed, then we can in fact rely on in memory cache when we compute the gas cost as well. However, this is just an idea and requires more investigation on its feasibility.

[ailisp]

Yeah, pre-fetch contract sounds a solution, in which, the contract is considered "compiled, ready in memory" when execution and our current cost would be good. It's WIP in #3985. Alternative approach is wasmer does better optimization, which we put some optimistic but cannot fully count on

[MaksymZavershynskyi]

With pre-fetching we need to take care of the following nuances:

• When we run sharded node, the runtimes do not compete for the disk during aggressive fetching;
• It works in the worst case scenario where the chunk is full of contact calls that touch unique cold contracts;
• If it is multithreaded, it should not consume more CPU than what is dedicated to a single shard.

[bowenwang1996]

the runtimes do not compete for the disk during aggressive fetching

Rocksdb supports parallel reads from multiple different threads. We don't touch trie here at all and it is simply some database reads, so I don't think it will be an issue.

[MaksymZavershynskyi]

Rocksdb is not an issue. AFAIK SSD has a physical limitation on the number of parallel reads.