Skip to content

Meta layer API as flowchart

Youngjin Yu edited this page Jan 3, 2023 · 7 revisions

Total

flowchart TD
    subgraph META
        IMetaUpdater --> IJournalWriter --> IJournalLogBuffer
        IJournalLogBuffer --> MetaFileIntf
        MetaFileIntf --> MetaFsIoScheduler
        MetaFsIoScheduler --> MioHandler
        MioHandler --> MetaStorageSubsystem    
        CheckpointSubmission --> CheckpointManager
        CheckpointManager --> CheckpointHandler
        CheckpointHandler --> |FlushDirtyMaps|Mapper
        CheckpointHandler --> |FlushContexts|ContextManager
        Mapper --> |FlushTouchedPages|StripeMapManager
        Mapper --> |FlushTouchedPages|VSAMapManager
        StripeMapManager --> MetaFileIntf
        VSAMapManager --> MetaFileIntf
        ReverseMapManager --> MetaFileIntf
        ContextManager --> MetaFileIntf
        BlockManager --> |AllocateFreeSegment|ContextManager
        SegmentCtx --> |stored by|ContextManager
        IWBStripeAllocator
        SegmentContextUpdater
        Stripe
    end
    MetaStorageSubsystem --> |Enqueue\nIssueWriteEvent|EventScheduler
    MetaStorageSubsystem --> |Dest != SSD|IIOSubmitHandler
    IJournalWriter --> |CheckpointSubmission\nconditionally|EventScheduler
    EventScheduler -.-> |Execute\nCheckpointSubmission|CheckpointSubmission
    
    subgraph IOPath
    
        subgraph WriteSubmission
            BlockMapUpdateRequest --> |UpdateBlockMap|IMetaUpdater
            Translator --> |GetVSAs|Mapper
            Translator --> |GetRandomVSA|Mapper
            Translator --> |GetLSA|StripeMapManager
            Translator --> |ReferLsidCnt|IWBStripeAllocator
            VsaRangeMaker --> |GetVSAs|Mapper      
        end
        WriteSubmission --> |Submit VolumeIo|IODispatcher
        WriteSubmission --> |AllocateWriteBufferBlks|BlockManager
        WriteSubmission --> |IsProhibitedUserBlkAlloc|BlockManager
        StripePutEvent --> |FreeWBStripeId|IWBStripeAllocator
        WriteCompletion --> |GetStripe|IWBStripeAllocator
        WriteCompletion --> |Flush|Stripe
        WriteCompletion --> |DecreseBlksRemaining|Stripe
        ReadCompletion --> |DereferLsidCnt|IWBStripeAllocator
        ReadCompletionForPartialWrite --> |DereferLsidCnt|IWBStripeAllocator
        subgraph FlushSubmission
            StripeMapUpdateRequest --> |UpdateStripeMap|IMetaUpdater
            StripeMapUpdateRequest --> |GetLSA|StripeMapManager  
        end
        
        FlushCompletion --> |GetLSA|StripeMapManager
    
        EventScheduler -.-> |Execute\nIssueWriteEvent|IIOSubmitHandler
        IIOSubmitHandler --> IODispatcher
        IODispatcher --> IODispatcherSubmission
        IODispatcherSubmission --> UnvmeSsd
        UnvmeSsd --> UnvmeDrv
        UnvmeDrv --> UnvmeCmd
    end
    
    UnvmeCmd --> |spdk_nvme_ns_cmd_*|SPDK
    SPDK --> SSD 
    SSD[(Database)]
    
    
    subgraph SRM
        GcFlushCompletion --> GcMapUpdateRequest
        GcFlushCompletion --> |Flush|Stripe
        GcFlushSubmission --> |AllocateGcDestStripe|BlockManager
        GcFlushSubmission --> |UpdateReverseMapEntry|Stripe
        GcFlushSubmission --> |GetReverseMapEntry|Stripe
        GcFlushSubmission --> |UpdateVictimVsa|Stripe
        GcMapUpdateRequest --> |UpdateGcMap|IMetaUpdater
        GcMapUpdateRequest --> |GetVSAInternal|Mapper
        GcMapUpdateRequest --> |GetReverseMapEntry|Stripe
        GcMapUpdateCompletion --> |GetReverseMapEntry|Stripe
        VictimStripe --> |GetVSAInternal|Mapper
        VictimStripe --> |GetLSA|StripeMapManager
        VictimStripe --> |Load|ReverseMapManager
        VictimStripe --> |AllocReverseMapPack|ReverseMapManager
        StripeCopier --> VictimStripe
        StripeCopySubmission --> VictimStripe
        Copier --> |AllocateGCVictimSegment|ContextManager
        Copier --> |MoveToFreeState|SegmentCtx
        Copier --> |ResetInfos|SegmentContextUpdater
        SegmentBasedRebuild --> |AllocateRebuildTargetSegment|ContextManager
    end
Loading

Part: Meta & SRM Only

flowchart TD
    subgraph META
        IMetaUpdater --> IJournalWriter --> IJournalLogBuffer
        IJournalLogBuffer --> MetaFileIntf
        MetaFileIntf --> MetaFsIoScheduler
        MetaFsIoScheduler --> MioHandler
        MioHandler --> MetaStorageSubsystem    
        CheckpointSubmission --> CheckpointManager
        CheckpointManager --> CheckpointHandler
        CheckpointHandler --> |FlushDirtyMaps|Mapper
        CheckpointHandler --> |FlushContexts|ContextManager
        Mapper --> |FlushTouchedPages|StripeMapManager
        Mapper --> |FlushTouchedPages|VSAMapManager
        StripeMapManager --> MetaFileIntf
        VSAMapManager --> MetaFileIntf
        ReverseMapManager --> MetaFileIntf
        ContextManager --> MetaFileIntf
        BlockManager --> |AllocateFreeSegment|ContextManager
        SegmentCtx --> |stored by|ContextManager
        IWBStripeAllocator
        SegmentContextUpdater
        Stripe
    end
    MetaStorageSubsystem --> |Enqueue\nIssueWriteEvent|EventScheduler
    MetaStorageSubsystem --> |Dest != SSD|IIOSubmitHandler
    IJournalWriter --> |CheckpointSubmission\nconditionally|EventScheduler
    EventScheduler -.-> |Execute\nCheckpointSubmission|CheckpointSubmission
        
    
    subgraph SRM
        GcFlushCompletion --> GcMapUpdateRequest
        GcFlushCompletion --> |Flush|Stripe
        GcFlushSubmission --> |AllocateGcDestStripe|BlockManager
        GcFlushSubmission --> |UpdateReverseMapEntry|Stripe
        GcFlushSubmission --> |GetReverseMapEntry|Stripe
        GcFlushSubmission --> |UpdateVictimVsa|Stripe
        GcMapUpdateRequest --> |UpdateGcMap|IMetaUpdater
        GcMapUpdateRequest --> |GetVSAInternal|Mapper
        GcMapUpdateRequest --> |GetReverseMapEntry|Stripe
        GcMapUpdateCompletion --> |GetReverseMapEntry|Stripe
        VictimStripe --> |GetVSAInternal|Mapper
        VictimStripe --> |GetLSA|StripeMapManager
        VictimStripe --> |Load|ReverseMapManager
        VictimStripe --> |AllocReverseMapPack|ReverseMapManager
        StripeCopier --> VictimStripe
        StripeCopySubmission --> VictimStripe
        Copier --> |AllocateGCVictimSegment|ContextManager
        Copier --> |MoveToFreeState|SegmentCtx
        Copier --> |ResetInfos|SegmentContextUpdater
        SegmentBasedRebuild --> |AllocateRebuildTargetSegment|ContextManager
    end
Loading

Part: Meta & IOPath only

flowchart TD
    subgraph META
        IMetaUpdater --> IJournalWriter --> IJournalLogBuffer
        IJournalLogBuffer --> MetaFileIntf
        MetaFileIntf --> MetaFsIoScheduler
        MetaFsIoScheduler --> MioHandler
        MioHandler --> MetaStorageSubsystem    
        CheckpointSubmission --> CheckpointManager
        CheckpointManager --> CheckpointHandler
        CheckpointHandler --> |FlushDirtyMaps|Mapper
        CheckpointHandler --> |FlushContexts|ContextManager
        Mapper --> |FlushTouchedPages|StripeMapManager
        Mapper --> |FlushTouchedPages|VSAMapManager
        StripeMapManager --> MetaFileIntf
        VSAMapManager --> MetaFileIntf
        ReverseMapManager --> MetaFileIntf
        ContextManager --> MetaFileIntf
        BlockManager --> |AllocateFreeSegment|ContextManager
        SegmentCtx --> |stored by|ContextManager
        IWBStripeAllocator
        SegmentContextUpdater
        Stripe
    end
    MetaStorageSubsystem --> |Enqueue\nIssueWriteEvent|EventScheduler
    MetaStorageSubsystem --> |Dest != SSD|IIOSubmitHandler
    IJournalWriter --> |CheckpointSubmission\nconditionally|EventScheduler
    EventScheduler -.-> |Execute\nCheckpointSubmission|CheckpointSubmission
    
    subgraph IOPath_WriteSubmission
            BlockMapUpdateRequest --> |UpdateBlockMap|IMetaUpdater
            Translator --> |GetVSAs|Mapper
            Translator --> |GetRandomVSA|Mapper
            Translator --> |GetLSA|StripeMapManager
            Translator --> |ReferLsidCnt|IWBStripeAllocator
            VsaRangeMaker --> |GetVSAs|Mapper          
    end
        IOPath_WriteSubmission --> |Submit VolumeIo|IODispatcher
        IOPath_WriteSubmission --> |AllocateWriteBufferBlks|BlockManager
        IOPath_WriteSubmission --> |IsProhibitedUserBlkAlloc|BlockManager
    
    subgraph IOPath_FlushSubmission
            StripeMapUpdateRequest --> |UpdateStripeMap|IMetaUpdater
            StripeMapUpdateRequest --> |GetLSA|StripeMapManager      
    end
    
    subgraph IOPath_WriteCompletion
        WriteCompletion --> |GetStripe|IWBStripeAllocator
        WriteCompletion --> |Flush|Stripe
        WriteCompletion --> |DecreseBlksRemaining|Stripe    
    end
    
    subgraph IOPath_Remaining
        StripePutEvent --> |FreeWBStripeId|IWBStripeAllocator
        ReadCompletion --> |DereferLsidCnt|IWBStripeAllocator
        ReadCompletionForPartialWrite --> |DereferLsidCnt|IWBStripeAllocator
        
        FlushCompletion --> |GetLSA|StripeMapManager
    
        EventScheduler -.-> |Execute\nIssueWriteEvent|IIOSubmitHandler
        IIOSubmitHandler --> IODispatcher
        IODispatcher --> IODispatcherSubmission
        IODispatcherSubmission --> UnvmeSsd
        UnvmeSsd --> UnvmeDrv
        UnvmeDrv --> UnvmeCmd
    end
    
    
    
    UnvmeCmd --> |spdk_nvme_ns_cmd_*|SPDK
    SPDK --> SSD 
    SSD[(Database)]    
Loading