Storage node Piecestore Improvements (Hashstore)

[iglesiasbrandon]

Description:

We discovered some improvements could be made to the storage node piece store.

What is the problem/pain point?

Storage nodes are unable to keep up with a high number of read/ writes while also cleaning up trash and TTL files.

What is the impact?

Rewriting the piece store to be more efficient will allow each spindle (storage node) to perform more operations per second because of how the metadata for pieces will be stored. This means each storage node will be able to serve more customer requests while also being able to perform background functions such as deleting trash pieces and taking care of TTL pieces.

Why now?

Because what storage node doesn't want to be more efficient...

Links:

https://review.dev.storj.io/q/storagenode/hashstore

[iglesiasbrandon]

@mniewrzal @profclems can you all update the description of this issue to detail the improvements we are making.

[shaupt131]

Sprint 51 update:

https://docs.google.com/presentation/d/10nVuLgWosrNEUVERryBsyYeWsoFMga3rCb6ja0Ib2nU/edit?usp=sharing

[shaupt131]

Sprint 52 update:

https://docs.google.com/presentation/d/14HWaSaNDjt9WH8pWDDqzS90wydtgjjpOkY33nWCO9mA/edit?usp=sharing