Add replay verify on archive related workflows

[areshand]

Description

add auto snapshot provisioning and replay-on-archive to workflows

How Has This Been Tested?

Tested on testnet for replay-verify https://github.com/aptos-labs/aptos-core/actions/runs/11903312271/job/33170576296

testnet auto provision snapshot https://github.com/aptos-labs/aptos-core/actions/runs/11898590978/job/33155418244?pr=15272

Key Areas to Review

Type of Change

• New feature
• Bug fix
• Breaking change
• Performance improvement
• Refactoring
• Dependency update
• Documentation update
• Tests

Which Components or Systems Does This Change Impact?

• Validator Node
• Full Node (API, Indexer, etc.)
• Move/Aptos Virtual Machine
• Aptos Framework
• Aptos CLI/SDK
• Developer Infrastructure
• Move Compiler
• Other (specify)

Checklist

• I have read and followed the CONTRIBUTING doc
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I identified and added all stakeholders and component owners affected by this change as reviewers
• I tested both happy and unhappy path of the functionality
• I have made corresponding changes to the documentation

[trunk-io]

⏱️ 22h 45m total CI duration on this PR

Slowest 15 Jobs	Cumulative Duration	Recent Runs
rust-cargo-deny	2h 17m	🟩 🟩 🟩 🟩 🟩 (+77 more)
check-dynamic-deps	2h	🟩 🟩 🟩 🟩 🟩 (+78 more)
replay-testnet / run-replay-verify	1h 59m	🟩 🟥 🟩
replay-testnet / provision	1h 14m	🟥 🟥 🟥 🟩 ⬜ (+35 more)
replay-mainnet / provision	1h 5m	🟥 🟥 🟥 🟩 ⬜ (+35 more)
test-target-determinator	46m	🟩 🟩 🟩 🟩 🟩 (+10 more)
semgrep/ci	41m	🟩 🟩 🟩 🟩 🟩 (+78 more)
general-lints	38m	🟩 🟩 🟩 🟩 🟩 (+77 more)
provision-testnet / provision	37m	🟥 🟥 🟩 ⬜ 🟩 (+1 more)
provision-mainnet / provision	35m	🟥 🟥 🟥 🟩 ⬜ (+2 more)
replay-testnet / prepare	24m	⬜
replay-testnet / run-replay-verify	23m	⬜
replay-mainnet / run-replay-verify	20m	⬜ ⬜ 🟥
file_change_determinator	15m	🟩 🟩 🟩 🟩 🟩 (+78 more)
rust-move-tests	13m	🟩

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

[github-actions]

✅ Forge suite realistic_env_max_load success on 481c594a9d4c945dc41d255cc674c154ede24a9a

two traffics test: inner traffic : committed: 14474.96 txn/s, latency: 2746.75 ms, (p50: 2700 ms, p70: 2700, p90: 2900 ms, p99: 3600 ms), latency samples: 5503880
two traffics test : committed: 99.92 txn/s, latency: 1607.84 ms, (p50: 1400 ms, p70: 1500, p90: 1600 ms, p99: 10400 ms), latency samples: 1780
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.022, avg: 1.558", "ConsensusProposalToOrdered: max: 0.324, avg: 0.293", "ConsensusOrderedToCommit: max: 0.378, avg: 0.364", "ConsensusProposalToCommit: max: 0.675, avg: 0.657"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 1.07s no progress at version 2774761 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.63s no progress at version 2774759 (avg 8.63s) [limit 15].
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite compat success on 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> 481c594a9d4c945dc41d255cc674c154ede24a9a

Compatibility test results for 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> 481c594a9d4c945dc41d255cc674c154ede24a9a (PR)
1. Check liveness of validators at old version: 010570d3b7aa20889fb5ad0e5b23800aa33f5634
compatibility::simple-validator-upgrade::liveness-check : committed: 15405.28 txn/s, latency: 2208.25 ms, (p50: 2100 ms, p70: 2100, p90: 2400 ms, p99: 5400 ms), latency samples: 495160
2. Upgrading first Validator to new version: 481c594a9d4c945dc41d255cc674c154ede24a9a
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 7070.49 txn/s, latency: 3843.36 ms, (p50: 4200 ms, p70: 4800, p90: 5300 ms, p99: 5400 ms), latency samples: 126580
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 7605.25 txn/s, latency: 4214.97 ms, (p50: 4400 ms, p70: 4500, p90: 6200 ms, p99: 6400 ms), latency samples: 251140
3. Upgrading rest of first batch to new version: 481c594a9d4c945dc41d255cc674c154ede24a9a
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7114.30 txn/s, latency: 3715.36 ms, (p50: 4200 ms, p70: 4300, p90: 4400 ms, p99: 4600 ms), latency samples: 144560
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 7246.43 txn/s, latency: 4379.25 ms, (p50: 4600 ms, p70: 4700, p90: 6700 ms, p99: 7000 ms), latency samples: 242080
4. upgrading second batch to new version: 481c594a9d4c945dc41d255cc674c154ede24a9a
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 11497.39 txn/s, latency: 2365.22 ms, (p50: 2400 ms, p70: 2500, p90: 3500 ms, p99: 3600 ms), latency samples: 200140
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 11043.00 txn/s, latency: 2824.33 ms, (p50: 2600 ms, p70: 2800, p90: 5100 ms, p99: 6300 ms), latency samples: 359700
5. check swarm health
Compatibility test for 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> 481c594a9d4c945dc41d255cc674c154ede24a9a passed
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite framework_upgrade success on 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> 481c594a9d4c945dc41d255cc674c154ede24a9a

Compatibility test results for 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> 481c594a9d4c945dc41d255cc674c154ede24a9a (PR)
Upgrade the nodes to version: 481c594a9d4c945dc41d255cc674c154ede24a9a
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1363.46 txn/s, submitted: 1367.13 txn/s, failed submission: 3.67 txn/s, expired: 3.67 txn/s, latency: 2348.75 ms, (p50: 2100 ms, p70: 2400, p90: 3700 ms, p99: 4800 ms), latency samples: 118980
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1371.77 txn/s, submitted: 1374.73 txn/s, failed submission: 2.96 txn/s, expired: 2.96 txn/s, latency: 2250.45 ms, (p50: 1800 ms, p70: 2400, p90: 3900 ms, p99: 5700 ms), latency samples: 120440
5. check swarm health
Compatibility test for 010570d3b7aa20889fb5ad0e5b23800aa33f5634 ==> 481c594a9d4c945dc41d255cc674c154ede24a9a passed
Upgrade the remaining nodes to version: 481c594a9d4c945dc41d255cc674c154ede24a9a
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1304.71 txn/s, submitted: 1308.91 txn/s, failed submission: 4.20 txn/s, expired: 4.20 txn/s, latency: 2394.53 ms, (p50: 2100 ms, p70: 2400, p90: 4300 ms, p99: 5700 ms), latency samples: 118060
Test Ok

• Grafana dashboard
• Humio Logs
• Axiom Logs
• Validator CPU Profile
• Fullnode CPU Profile
• Test runner output
• Test run is land-blocking