Skip to content

feat: add --order option to run tests in random order#6144

Open
kirthi-b wants to merge 1 commit into
mochajs:mainfrom
kirthi-b:feat/order-random
Open

feat: add --order option to run tests in random order#6144
kirthi-b wants to merge 1 commit into
mochajs:mainfrom
kirthi-b:feat/order-random

Conversation

@kirthi-b

Copy link
Copy Markdown

PR Checklist

Overview

Implements the --order option from #902, following the design approved in
#902 (comment) and the
option shape from the issue body: --order default, --order random and
--order random:<seed>.

How it works:

  • Sibling suites and sibling tests are shuffled (Fisher-Yates) within their
    parent suite, per 🚀 Feature: --order option for random test order #902 (comment).
    The hierarchy is preserved, so before/after/beforeEach/afterEach run
    exactly as often as in a normal run.
  • Every randomized run reports its seed: Mocha: test order randomized with seed 42; use --order random:42 to reproduce. The seed goes to STDERR, so
    machine-readable reporters on STDOUT (json, xunit) are unaffected. This was
    the main open question on the old reference PR feat: randomization support [WIP REFERENCE - DO NOT MERGE] #2301.
  • The shuffle happens in Runner#run's prepare step, so it also works with
    --delay and inside parallel workers. In parallel mode the main process
    bakes the generated seed into the options before they are serialized, so all
    workers shuffle from the same seed and the seed is printed once.
  • --sort and --order random error out as mutually exclusive, same as the
    existing --parallel and --sort pairing.
  • Also available programmatically via mocha.order('random:42') and the
    order constructor option.

Includes unit tests, integration tests (fixed seed order, reproduce-from-
reported-seed round trip, error cases) and docs for the new flag.

Notes for reviewers:

  1. The 2024 comment suggests targeting the randomization branch, but that
    branch no longer exists on the remote, so this targets main.
  2. RNG: the approval said seedrandom "or an equivalent". This uses a small
    inline seeded PRNG (mulberry32) to avoid adding a runtime dependency;
    identical seeds give identical orders on any platform. Happy to switch to
    seedrandom if you prefer the dependency.
  3. Seed reporting currently writes to STDERR (console.log in browsers). If you
    would rather surface it through reporters or a runner event, I can rework
    that.
  4. Only random and default are implemented; the random-suites variant
    floated mid-thread was left out to keep scope to the approved design.

Closes #902

@linux-foundation-easycla

linux-foundation-easycla Bot commented Jul 16, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: kirthi-b / name: kirthi_b (de942cc)

@mark-wiemer

Copy link
Copy Markdown
Member

Looks OK, pretty big and low-priority, but I'll add it to my backlog :)

@mark-wiemer

Copy link
Copy Markdown
Member

FYI @JoshuaKGoldberg the backing issue has 80+ likes!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🚀 Feature: --order option for random test order

2 participants