@fluid-internal/mocha-test-setup

IMPORTANT: This package is intended strictly as an implementation detail of the Fluid Framework and is not intended for public consumption. We make no stability guarantees regarding its APIs.

This package has a few main purposes:

Expose/generate a default .mocharc.cjs configuration for running mocha tests, which other packages can extend.
Map paths for required packages/modules to account for Lerna's dependency hoisting.
Add mocha beforeAll, beforeEach and afterEach root hook plugins to add some special behavior when we run tests.

Base `mocharc.cjs` configuration

To leverage the base mocha configuration exposed by this package, first add it as a devDependency to your package.json (the rest of the file has been omitted):

{
    "devDependencies": {
        "@fluid-internal/mocha-test-setup": "version-that-matches-the-rest-of-the-release-group"
    }
}

Then put this in a .mocharc.cjs file at the root of your package:

"use strict";

const getFluidTestMochaConfig = require("@fluid-internal/mocha-test-setup/mocharc-common");

const config = getFluidTestMochaConfig(__dirname);
module.exports = config;

If you need extra configuration, make changes to the config object before assigning it to module.exports.

Mocha will use this file by default if no path to a configuration file is provided explicitly.

The default configuration generated by getFluidTestMochaConfig() looks like this:

{
  "recursive": true,
  "require": ["@fluid-internal/mocha-test-setup", "source-map-support/register"], // can be extended by providing additionalRequiredModules to getFluidTestMochaConfig()
  "unhandled-rejections": "strict",
  "node-option": ["conditions=allow-ff-test-exports", "expose-gc"],
  "ignore": [ "**/*.tool.{js,cjs,mjs}" ], // Ignore "tools" which are scripts intended to be run, not part of the test suite.
  "spec": "lib/test" // Defaults to "lib/test" where esm tests typically are. Can be overridden via MOCHA_SPEC.
};

It is recommended that the source for tests files use a file name ending in ".spec.ts", but this is not enforced: no file named based filtering of test files is done by default, except for the ignoring of "**/*.tool.{js,cjs,mjs}" noted above.

Note: some packages do explicitly filter test files to *.spec.js: in such packages test files with other names will be silently ignored.

Environment Variables

In package.json scripts, environment variables can be set using cross-env, like "cross-env MOCHA_SPEC=lib/mocha/test/*/.spec.*js mocha".

The configuration will have additional settings if the following environment variables are present:

FLUID_TEST_TIMEOUT

If it exists, the default configuration will also include this:

{
    "timeout": "<value-of-FLUID_TEST_TIMEOUT>"
}

FLUID_TEST_FORBID_ONLY

If it exists, the default configuration will also include this:

{
    "forbid-only": "true"
}

FLUID_LOGGER_PROPS

In case there is a need to override telemetry metrics, one can make use of an environment variable FLUID_LOGGER_PROPS, that to override them during execution time:

FLUID_LOGGER_PROPS='{ "hostName": "Benchmark" }'

FLUID_TEST_LOGGER_PKG_SPECIFIER

Injects implementation of createTestLogger. We use it in our pipelines to submit telemetry to internal engineering systems. Probably not useful outside of that scenario, except when using it locally to test something related to its use in the pipelines.

FLUID_TEST_MODULE_SYSTEM

When set to "CJS", current run is setup to be for CommonJS coverage:

"CJS" is noted as a variant and shown as prefix to test case names.
Coverage file will be prefixed with "CJS-".
Default spec is changed from lib/test to dist/test; see more below.

MOCHA_SPEC

Select the "spec". Note that unlike Mocha's built in MOCHA_OPTIONS environment variable, this will replace our default spec instead of adding to the spec (See https://mochajs.org/next/running/configuring/#merging). Also unlike Mocha's default behavior (see https://mochajs.org/next/running/cli/), this configuration defaults spec to lib/test (where we place our esm tests) instead of test.

Mapping of package paths to account for Lerna hoisting

The way we use Lerna to manage our monorepo, package dependencies are sometimes moved out of the node_modules folder of the package that declares them, and into a node_modules folder at the root of the release group. This applies to external dependencies (i.e. not between packages in our repo) that are present in more than one of our packages, including mocha itself as well as other test runners, which means that their path when they execute might not be the one you'd normally expect. This can cause issues when test scripts point to other dependencies of the package being tested (e.g. with the -r/--require flag) when running tests. This utility maps the paths of packages/modules passed to the getFluidTestMochaConfig() function, to ensure they are imported from whichever location they end up at.

NOTE: once we move to pnpm this functionality might become irrelevant because pnpm keeps all dependencies in the expected paths and creates the necessary symlinks, instead of changing where dependency folders are located.

Considerations for non-package/module arguments

Sometimes you'll want to provide a package/module/path as an argument when executing your tests but outside of the required packages that you can pass to getFluidTestMochaConfig(). For example, the path to a reporter file:

mocha --require @fluid-internal/mocha-test-setup --reporter @fluid-tools/benchmark/dist/MochaReporter.js

Depending on where that file is coming from, you'll need to be careful with how that path is specified. E.g. @fluid-tools/benchmark, while being part of the repository, is not part of any release group, so it will always be downloaded from the public npm feed and treated as an external dependency. Since several packages declare it as a dependency, Lerna will hoist it to node_modules at the root of the release group. In general, the safest solution is to specify it starting with the package name (including scope if applicable). This might break in a few edge cases as long as we use Lerna with hoisting, so we'll have to deal with them individually.

A particular way of specifying a package/module/path (e.g. starting with the package name vs starting with node_modules/) might behave differently locally and in CI builds, so that's another factor to keep in mind.

Special behavior

Suppress console output by default

console.log(), console.warn() and console.error() are disabled by default when using this package. They can be re-enabled by setting the FLUID_TEST_VERBOSE environment variable to any non-empty value, e.g. FLUID_TEST_VERBOSE=1.

Telemetry events

A couple of telemetry events fluid:telemetry:Test_start and fluid:telemetry:Test_end will be generated for each test. The Test_end event includes fields with the test's state (passing/failed), its duration, if it timed out. If a FLUID_TEST_VARIANT environment variable exists, the testVariant field in the events will have its value.

Support for a custom logger

This utility supports using a custom logger implementation. This can be useful to send the operational telemetry generated by the FluidFramework while the tests are running, to a particular log sink.

It is expected that the package with the custom logger implementation adds a getTestLogger() => ITelemetryBufferedLogger function to the global object, which this package will call in order to obtain an instance of the custom logger.

If a custom logger implementation will be used when executing tests in a package that leverages this utility, the path to the module with that implementation should not be specified with the --require/-r flag, but through the FLUID_TEST_LOGGER_PKG_PATH environment variable. This package has logic that needs the custom logger to be imported/executed at a particular time, and we ensure that by requiring that the path be provided through that environment variable.

Contribution Guidelines

There are many ways to contribute to Fluid.

Participate in Q&A in our GitHub Discussions.
Submit bugs and help us verify fixes as they are checked in.
Review the source code changes.
Contribute bug fixes.

Detailed instructions for working in the repo can be found in the Wiki.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

This project may contain Microsoft trademarks or logos for Microsoft projects, products, or services. Use of these trademarks or logos must follow Microsoft’s Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.

Help

Not finding what you're looking for in this README? Check out fluidframework.com.

Still not finding what you're looking for? Please file an issue.

Thank you!

Trademark

This project may contain Microsoft trademarks or logos for Microsoft projects, products, or services.

Use of these trademarks or logos must follow Microsoft's Trademark & Brand Guidelines.

Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.

Détail du package