Auto merge of #10726 - weihanglo:test-doc-enhance, r=epage

Enhance documentation of testing

Author: bors

src/doc/man/cargo-bench.md

@@ -28,10 +28,11 @@ similarly named benchmarks like `foobar`):
 
     cargo bench -- foo --exact
 
-Benchmarks are built with the `--test` option to `rustc` which creates an
-executable with a `main` function that automatically runs all functions
-annotated with the `#[bench]` attribute. Cargo passes the `--bench` flag to
-the test harness to tell it to run only benchmarks.
+Benchmarks are built with the `--test` option to `rustc` which creates a
+special executable by linking your code with libtest. The executable
+automatically runs all functions annotated with the `#[bench]` attribute.
+Cargo passes the `--bench` flag to the test harness to tell it to run
+only benchmarks.
 
 The libtest harness may be disabled by setting `harness = false` in the target
 manifest settings, in which case your code will need to provide its own `main`
@@ -81,6 +82,8 @@ them from being benchmarked by default. Target selection options that take a
 target by name ignore the `bench` flag and will always benchmark the given
 target.
 
+{{> options-targets-bin-auto-built }}
+
 {{> options-targets }}
 
 {{> section-features }}

src/doc/man/cargo-build.md

@@ -23,6 +23,8 @@ When no target selection options are given, `cargo build` will build all
 binary and library targets of the selected packages. Binaries are skipped if
 they have `required-features` that are missing.
 
+{{> options-targets-bin-auto-built }}
+
 {{> options-targets }}
 
 {{> section-features }}

src/doc/man/cargo-rustc.md

@@ -35,6 +35,8 @@ To pass flags to all compiler processes spawned by Cargo, use the `RUSTFLAGS`
 When no target selection options are given, `cargo rustc` will build all
 binary and library targets of the selected package.
 
+{{> options-targets-bin-auto-built }}
+
 {{> options-targets }}
 
 {{> section-features }}

src/doc/man/cargo-test.md

@@ -12,7 +12,7 @@ cargo-test - Execute unit and integration tests of a package
 
 ## DESCRIPTION
 
-Compile and execute unit and integration tests.
+Compile and execute unit, integration, and documentation tests.
 
 The test filtering argument `TESTNAME` and all the arguments following the two
 dashes (`--`) are passed to the test binaries and thus to _libtest_ (rustc's
@@ -27,20 +27,34 @@ on 3 threads in parallel:
 
     cargo test foo -- --test-threads 3
 
-Tests are built with the `--test` option to `rustc` which creates an
-executable with a `main` function that automatically runs all functions
-annotated with the `#[test]` attribute in multiple threads. `#[bench]`
-annotated functions will also be run with one iteration to verify that they
-are functional.
+Tests are built with the `--test` option to `rustc` which creates a special
+executable by linking your code with libtest. The executable automatically
+runs all functions annotated with the `#[test]` attribute in multiple threads.
+`#[bench]` annotated functions will also be run with one iteration to verify
+that they are functional.
+
+If the package contains multiple test targets, each target compiles to a
+special executable as aforementioned, and then is run serially.
 
 The libtest harness may be disabled by setting `harness = false` in the target
 manifest settings, in which case your code will need to provide its own `main`
 function to handle running tests.
 
+### Documentation tests
+
 Documentation tests are also run by default, which is handled by `rustdoc`. It
-extracts code samples from documentation comments and executes them. See the
-[rustdoc book](https://doc.rust-lang.org/rustdoc/) for more information on
-writing doc tests.
+extracts code samples from documentation comments of the library target, and
+then executes them.
+
+Different from normal test targets, each code block compiles to a doctest
+executable on the fly with `rustc`. These executables run in parallel in
+separate processes. The compilation of a code block is in fact a part of test
+function controlled by libtest, so some options such as `--jobs` might not
+take effect. Note that this execution model of doctests is not guaranteed
+and may change in the future; beware of depending on it.
+
+See the [rustdoc book](https://doc.rust-lang.org/rustdoc/) for more information
+on writing doc tests.
 
 ## OPTIONS
 
@@ -73,13 +87,7 @@ ignore the `test` flag and will always test the given target.
 Doc tests for libraries may be disabled by setting `doctest = false` for the
 library in the manifest.
 
-Binary targets are automatically built if there is an integration test or
-benchmark. This allows an integration test to execute the binary to exercise
-and test its behavior. The `CARGO_BIN_EXE_<name>`
-[environment variable](../reference/environment-variables.html#environment-variables-cargo-sets-for-crates)
-is set when the integration test is built so that it can use the
-[`env` macro](https://doc.rust-lang.org/std/macro.env.html) to locate the
-executable.
+{{> options-targets-bin-auto-built }}
 
 {{> options-targets }}
 

src/doc/man/generated_txt/cargo-bench.txt

@@ -23,10 +23,11 @@ DESCRIPTION
 
            cargo bench -- foo --exact
 
-       Benchmarks are built with the --test option to rustc which creates an
-       executable with a main function that automatically runs all functions
-       annotated with the #[bench] attribute. Cargo passes the --bench flag to
-       the test harness to tell it to run only benchmarks.
+       Benchmarks are built with the --test option to rustc which creates a
+       special executable by linking your code with libtest. The executable
+       automatically runs all functions annotated with the #[bench] attribute.
+       Cargo passes the --bench flag to the test harness to tell it to run only
+       benchmarks.
 
        The libtest harness may be disabled by setting harness = false in the
        target manifest settings, in which case your code will need to provide
@@ -117,6 +118,15 @@ OPTIONS
        options that take a target by name ignore the bench flag and will always
        benchmark the given target.
 
+       Binary targets are automatically built if there is an integration test
+       or benchmark being selected to benchmark. This allows an integration
+       test to execute the binary to exercise and test its behavior. The
+       CARGO_BIN_EXE_<name> environment variable
+       <https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates>
+       is set when the integration test is built so that it can use the env
+       macro <https://doc.rust-lang.org/std/macro.env.html> to locate the
+       executable.
+
        Passing target selection flags will benchmark only the specified
        targets.
 

src/doc/man/generated_txt/cargo-build.txt

@@ -51,6 +51,15 @@ OPTIONS
        binary and library targets of the selected packages. Binaries are
        skipped if they have required-features that are missing.
 
+       Binary targets are automatically built if there is an integration test
+       or benchmark being selected to build. This allows an integration test to
+       execute the binary to exercise and test its behavior. The
+       CARGO_BIN_EXE_<name> environment variable
+       <https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates>
+       is set when the integration test is built so that it can use the env
+       macro <https://doc.rust-lang.org/std/macro.env.html> to locate the
+       executable.
+
        Passing target selection flags will build only the specified targets.
 
        Note that --bin, --example, --test and --bench flags also support common

src/doc/man/generated_txt/cargo-rustc.txt

@@ -42,6 +42,15 @@ OPTIONS
        When no target selection options are given, cargo rustc will build all
        binary and library targets of the selected package.
 
+       Binary targets are automatically built if there is an integration test
+       or benchmark being selected to build. This allows an integration test to
+       execute the binary to exercise and test its behavior. The
+       CARGO_BIN_EXE_<name> environment variable
+       <https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates>
+       is set when the integration test is built so that it can use the env
+       macro <https://doc.rust-lang.org/std/macro.env.html> to locate the
+       executable.
+
        Passing target selection flags will build only the specified targets.
 
        Note that --bin, --example, --test and --bench flags also support common

src/doc/man/generated_txt/cargo-test.txt

@@ -7,7 +7,7 @@ SYNOPSIS
        cargo test [options] [testname] [-- test-options]
 
 DESCRIPTION
-       Compile and execute unit and integration tests.
+       Compile and execute unit, integration, and documentation tests.
 
        The test filtering argument TESTNAME and all the arguments following the
        two dashes (--) are passed to the test binaries and thus to libtest
@@ -23,20 +23,34 @@ DESCRIPTION
 
            cargo test foo -- --test-threads 3
 
-       Tests are built with the --test option to rustc which creates an
-       executable with a main function that automatically runs all functions
-       annotated with the #[test] attribute in multiple threads. #[bench]
-       annotated functions will also be run with one iteration to verify that
-       they are functional.
+       Tests are built with the --test option to rustc which creates a special
+       executable by linking your code with libtest. The executable
+       automatically runs all functions annotated with the #[test] attribute in
+       multiple threads. #[bench] annotated functions will also be run with one
+       iteration to verify that they are functional.
+
+       If the package contains multiple test targets, each target compiles to a
+       special executable as aforementioned, and then is run serially.
 
        The libtest harness may be disabled by setting harness = false in the
        target manifest settings, in which case your code will need to provide
        its own main function to handle running tests.
 
+   Documentation tests
        Documentation tests are also run by default, which is handled by
-       rustdoc. It extracts code samples from documentation comments and
-       executes them. See the rustdoc book <https://doc.rust-lang.org/rustdoc/>
-       for more information on writing doc tests.
+       rustdoc. It extracts code samples from documentation comments of the
+       library target, and then executes them.
+
+       Different from normal test targets, each code block compiles to a
+       doctest executable on the fly with rustc. These executables run in
+       parallel in separate processes. The compilation of a code block is in
+       fact a part of test function controlled by libtest, so some options such
+       as --jobs might not take effect. Note that this execution model of
+       doctests is not guaranteed and may change in the future; beware of
+       depending on it.
+
+       See the rustdoc book <https://doc.rust-lang.org/rustdoc/> for more
+       information on writing doc tests.
 
 OPTIONS
    Test Options
@@ -116,9 +130,9 @@ OPTIONS
        the library in the manifest.
 
        Binary targets are automatically built if there is an integration test
-       or benchmark. This allows an integration test to execute the binary to
-       exercise and test its behavior. The CARGO_BIN_EXE_<name> environment
-       variable
+       or benchmark being selected to test. This allows an integration test to
+       execute the binary to exercise and test its behavior. The
+       CARGO_BIN_EXE_<name> environment variable
        <https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-crates>
        is set when the integration test is built so that it can use the env
        macro <https://doc.rust-lang.org/std/macro.env.html> to locate the

src/doc/man/includes/options-targets-bin-auto-built.md

@@ -0,0 +1,8 @@
+Binary targets are automatically built if there is an integration test or
+benchmark being selected to {{lower actionverb}}. This allows an integration
+test to execute the binary to exercise and test its behavior. 
+The `CARGO_BIN_EXE_<name>`
+[environment variable](../reference/environment-variables.html#environment-variables-cargo-sets-for-crates)
+is set when the integration test is built so that it can use the
+[`env` macro](https://doc.rust-lang.org/std/macro.env.html) to locate the
+executable.

src/doc/src/commands/cargo-bench.md

@@ -28,10 +28,11 @@ similarly named benchmarks like `foobar`):
 
     cargo bench -- foo --exact
 
-Benchmarks are built with the `--test` option to `rustc` which creates an
-executable with a `main` function that automatically runs all functions
-annotated with the `#[bench]` attribute. Cargo passes the `--bench` flag to
-the test harness to tell it to run only benchmarks.
+Benchmarks are built with the `--test` option to `rustc` which creates a
+special executable by linking your code with libtest. The executable
+automatically runs all functions annotated with the `#[bench]` attribute.
+Cargo passes the `--bench` flag to the test harness to tell it to run
+only benchmarks.
 
 The libtest harness may be disabled by setting `harness = false` in the target
 manifest settings, in which case your code will need to provide its own `main`
@@ -138,6 +139,16 @@ them from being benchmarked by default. Target selection options that take a
 target by name ignore the `bench` flag and will always benchmark the given
 target.
 
+Binary targets are automatically built if there is an integration test or
+benchmark being selected to benchmark. This allows an integration
+test to execute the binary to exercise and test its behavior. 
+The `CARGO_BIN_EXE_<name>`
+[environment variable](../reference/environment-variables.html#environment-variables-cargo-sets-for-crates)
+is set when the integration test is built so that it can use the
+[`env` macro](https://doc.rust-lang.org/std/macro.env.html) to locate the
+executable.
+
+
 Passing target selection flags will benchmark only the specified
 targets.