Skip to content

order book benchmark with multi-market#17206

Merged
igor-aptos merged 1 commit into
mainfrom
igor/benchmark_with_multi_market
Jul 31, 2025
Merged

order book benchmark with multi-market#17206
igor-aptos merged 1 commit into
mainfrom
igor/benchmark_with_multi_market

Conversation

@igor-aptos

Copy link
Copy Markdown
Contributor

Description

How Has This Been Tested?

Key Areas to Review

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

@igor-aptos igor-aptos force-pushed the igor/benchmark_with_multi_market branch from 1dc454a to f5096b5 Compare July 30, 2025 17:01
@igor-aptos igor-aptos requested a review from a team as a code owner July 30, 2025 17:01

@manudhundi manudhundi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Igor, there are some test failures

@igor-aptos igor-aptos force-pushed the igor/benchmark_with_multi_market branch 2 times, most recently from b6963b1 to 3c62064 Compare July 30, 2025 20:39
@igor-aptos igor-aptos added CICD:run-execution-performance-test Run execution performance test CICD:run-execution-performance-full-test Run execution performance test (full version) labels Jul 30, 2025
@igor-aptos igor-aptos force-pushed the igor/benchmark_with_multi_market branch from 3c62064 to b2fce70 Compare July 30, 2025 21:28
@igor-aptos igor-aptos removed CICD:run-execution-performance-test Run execution performance test CICD:run-execution-performance-full-test Run execution performance test (full version) labels Jul 30, 2025
@igor-aptos igor-aptos force-pushed the igor/benchmark_with_multi_market branch from b2fce70 to 4b4e566 Compare July 30, 2025 22:33
@igor-aptos igor-aptos enabled auto-merge (squash) July 30, 2025 22:34
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Forge suite compat success on 05f28bf5ae80954a0402d0f1bc1a510224a82e59 ==> 4b4e5666e98c742bed0b31b648df1baa3eb31afb

Compatibility test results for 05f28bf5ae80954a0402d0f1bc1a510224a82e59 ==> 4b4e5666e98c742bed0b31b648df1baa3eb31afb (PR)
1. Check liveness of validators at old version: 05f28bf5ae80954a0402d0f1bc1a510224a82e59
compatibility::simple-validator-upgrade::liveness-check : committed: 7409.76 txn/s, latency: 4262.50 ms, (p50: 3500 ms, p70: 4400, p90: 8600 ms, p99: 10100 ms), latency samples: 247480
2. Upgrading first Validator to new version: 4b4e5666e98c742bed0b31b648df1baa3eb31afb
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 2441.56 txn/s, latency: 13861.34 ms, (p50: 15200 ms, p70: 15700, p90: 16300 ms, p99: 16300 ms), latency samples: 91900
3. Upgrading rest of first batch to new version: 4b4e5666e98c742bed0b31b648df1baa3eb31afb
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 2452.51 txn/s, latency: 13765.53 ms, (p50: 15100 ms, p70: 15600, p90: 16100 ms, p99: 16200 ms), latency samples: 92400
4. upgrading second batch to new version: 4b4e5666e98c742bed0b31b648df1baa3eb31afb
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 4279.23 txn/s, latency: 7936.49 ms, (p50: 8800 ms, p70: 9100, p90: 9300 ms, p99: 9500 ms), latency samples: 151340
5. check swarm health
Compatibility test for 05f28bf5ae80954a0402d0f1bc1a510224a82e59 ==> 4b4e5666e98c742bed0b31b648df1baa3eb31afb passed
Test Ok

@github-actions

Copy link
Copy Markdown
Contributor

✅ Forge suite realistic_env_max_load success on 4b4e5666e98c742bed0b31b648df1baa3eb31afb

Forge report malformed: Expecting property name enclosed in double quotes: line 4 column 1 (char 23)
'{\n  "metrics": [\n    {\n[2025-07-30T23:08:30Z INFO  aptos_forge::report] Test Ok\n      "test_name": "two traffics test: inner traffic",\n      "metric": "submitted_txn",\n      "value": 4337120.0\n    },\n    {\n      "test_name": "two traffics test: inner traffic",\n      "metric": "expired_txn",\n      "value": 20.0\n    },\n    {\n      "test_name": "two traffics test: inner traffic",\n      "metric": "avg_tps",\n      "value": 11406.731576399485\n    },\n    {\n      "test_name": "two traffics test: inner traffic",\n      "metric": "avg_latency",\n      "value": 3305.029451937931\n    },\n    {\n      "test_name": "two traffics test: inner traffic",\n      "metric": "p50_latency",\n      "value": 3200.0\n    },\n    {\n      "test_name": "two traffics test: inner traffic",\n      "metric": "p90_latency",\n      "value": 3600.0\n    },\n    {\n      "test_name": "two traffics test: inner traffic",\n      "metric": "p99_latency",\n      "value": 5100.0\n    },\n    {\n      "test_name": "two traffics test",\n      "metric": "submitted_txn",\n      "value": 48620.0\n    },\n    {\n      "test_name": "two traffics test",\n      "metric": "expired_txn",\n      "value": 0.0\n    },\n    {\n      "test_name": "two traffics test",\n      "metric": "avg_tps",\n      "value": 99.98166513933411\n    },\n    {\n      "test_name": "two traffics test",\n      "metric": "avg_latency",\n      "value": 905.0714285714286\n    },\n    {\n      "test_name": "two traffics test",\n      "metric": "p50_latency",\n      "value": 800.0\n    },\n    {\n      "test_name": "two traffics test",\n      "metric": "p90_latency",\n      "value": 1100.0\n    },\n    {\n      "test_name": "two traffics test",\n      "metric": "p99_latency",\n      "value": 2600.0\n    }\n  ],\n  "text": "two traffics test: inner traffic : committed: 11406.73 txn/s, submitted: 11406.78 txn/s, expired: 0.05 txn/s, latency: 3305.03 ms, (p50: 3200 ms, p70: 3300, p90: 3600 ms, p99: 5100 ms), latency samples: 4337100\\ntwo traffics test : committed: 99.98 txn/s, latency: 905.07 ms, (p50: 800 ms, p70: 900, p90: 1100 ms, p99: 2600 ms), latency samples: 1960\\nLatency breakdown for phase 0: [\\"MempoolToBlockCreation: max: 2.648, avg: 2.447\\", \\"ConsensusProposalToOrdered: max: 0.165, avg: 0.163\\", \\"ConsensusOrderedToCommit: max: 0.235, avg: 0.207\\", \\"ConsensusProposalToCommit: max: 0.398, avg: 0.370\\"]\\nMax non-epoch-change gap was: 2 rounds at version 3942478 (avg 0.00) [limit 4], 2.21s no progress at version 3942478 (avg 0.07s) [limit 15].\\nMax epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.36s no progress at version 1839245 (avg 0.35s) [limit 16].\\nTest Ok"\n}'
Trailing Log Lines:
networkchaos.chaos-mesh.org "4-gcp--as-southeast1-to-3-gcp--us-east4-netem" deleted
test CompositeNetworkTest ... ok
Test Statistics: 
two traffics test: inner traffic : committed: 11406.73 txn/s, submitted: 11406.78 txn/s, expired: 0.05 txn/s, latency: 3305.03 ms, (p50: 3200 ms, p70: 3300, p90: 3600 ms, p99: 5100 ms), latency samples: 4337100
two traffics test : committed: 99.98 txn/s, latency: 905.07 ms, (p50: 800 ms, p70: 900, p90: 1100 ms, p99: 2600 ms), latency samples: 1960
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.648, avg: 2.447", "ConsensusProposalToOrdered: max: 0.165, avg: 0.163", "ConsensusOrderedToCommit: max: 0.235, avg: 0.207", "ConsensusProposalToCommit: max: 0.398, avg: 0.370"]
Max non-epoch-change gap was: 2 rounds at version 3942478 (avg 0.00) [limit 4], 2.21s no progress at version 3942478 (avg 0.07s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.36s no progress at version 1839245 (avg 0.35s) [limit 16].
Test Ok

=== BEGIN JUNIT ===
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="forge" tests="1" failures="0" errors="0" uuid="d2538347-b355-43ef-8d11-72f5c35ebbfe">
    <testsuite name="local" tests="1" disabled="0" errors="0" failures="0">
        <testcase name="CompositeNetworkTest(network:multi-region-network-emulation(two traffics test)) with ">
        </testcase>
    </testsuite>
</testsuites>
=== END JUNIT ===
[2025-07-30T23:08:30Z INFO  aptos_forge::backend::k8s::cluster_helper] Deleting namespace forge-e2e-pr-17206: Some(NamespaceStatus { conditions: None, phase: Some("Terminating") })
[2025-07-30T23:08:30Z INFO  aptos_forge::backend::k8s::cluster_helper] aptos-node resources for Forge removed in namespace: forge-e2e-pr-17206

test result: ok. 1 passed; 0 failed; 0 filtered out

Debugging output:
NAME                                         READY   STATUS      RESTARTS   AGE
aptos-node-0-fullnode-eforge5a4d41b0-0       1/1     Running     0          13m
aptos-node-0-validator-0                     1/1     Running     0          13m
aptos-node-1-fullnode-eforge5a4d41b0-0       1/1     Running     0          13m
aptos-node-1-validator-0                     1/1     Running     0          13m
aptos-node-2-fullnode-eforge5a4d41b0-0       1/1     Running     0          13m
aptos-node-2-validator-0                     1/1     Running     0          13m
aptos-node-3-fullnode-eforge5a4d41b0-0       1/1     Running     0          13m
aptos-node-3-validator-0                     1/1     Running     0          13m
aptos-node-4-fullnode-eforge5a4d41b0-0       1/1     Running     0          13m
aptos-node-4-validator-0                     1/1     Running     0          13m
aptos-node-5-validator-0                     1/1     Running     0          13m
aptos-node-6-validator-0                     1/1     Running     0          13m
forge-testnet-deployer-dgqrw                 0/1     Completed   0          13m
genesis-aptos-genesis-eforge5a4d41b0-kkk7f   0/1     Completed   0          13m

@github-actions

Copy link
Copy Markdown
Contributor

✅ Forge suite framework_upgrade success on 05f28bf5ae80954a0402d0f1bc1a510224a82e59 ==> 4b4e5666e98c742bed0b31b648df1baa3eb31afb

Compatibility test results for 05f28bf5ae80954a0402d0f1bc1a510224a82e59 ==> 4b4e5666e98c742bed0b31b648df1baa3eb31afb (PR)
Upgrade the nodes to version: 4b4e5666e98c742bed0b31b648df1baa3eb31afb
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1683.01 txn/s, submitted: 1689.20 txn/s, failed submission: 6.19 txn/s, expired: 6.19 txn/s, latency: 1756.86 ms, (p50: 1800 ms, p70: 1800, p90: 2400 ms, p99: 2900 ms), latency samples: 152242
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1023.69 txn/s, submitted: 1027.47 txn/s, failed submission: 3.78 txn/s, expired: 3.78 txn/s, latency: 2907.47 ms, (p50: 1900 ms, p70: 2200, p90: 9100 ms, p99: 12700 ms), latency samples: 92041
5. check swarm health
Compatibility test for 05f28bf5ae80954a0402d0f1bc1a510224a82e59 ==> 4b4e5666e98c742bed0b31b648df1baa3eb31afb passed
Upgrade the remaining nodes to version: 4b4e5666e98c742bed0b31b648df1baa3eb31afb
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1633.80 txn/s, submitted: 1640.47 txn/s, failed submission: 6.67 txn/s, expired: 6.67 txn/s, latency: 1846.23 ms, (p50: 1800 ms, p70: 2000, p90: 2400 ms, p99: 3300 ms), latency samples: 146900
Test Ok

@igor-aptos igor-aptos merged commit 570ec4a into main Jul 31, 2025
80 of 95 checks passed
@igor-aptos igor-aptos deleted the igor/benchmark_with_multi_market branch July 31, 2025 01:13
fengjy73 pushed a commit to fengjy73/aptos-core that referenced this pull request Aug 20, 2025
faculerena pushed a commit to CoinFabrik/aptos-core that referenced this pull request Aug 25, 2025
jgcrosta pushed a commit to CoinFabrik/aptos-core that referenced this pull request Sep 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants