Skip to content

Commit 34e7e5f

Browse files
committed
tokens.md: move the link reference definitions to the end of the file
1 parent 785325f commit 34e7e5f

File tree

1 file changed

+37
-40
lines changed

1 file changed

+37
-40
lines changed

src/tokens.md

Lines changed: 37 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -430,8 +430,6 @@ Note that the Rust syntax considers `-1i8` as an application of the [unary minus
430430
operator] to an integer literal `1i8`, rather than
431431
a single integer literal.
432432

433-
[unary minus operator]: expressions/operator-expr.md#negation-operators
434-
435433
#### Tuple index
436434

437435
> **<sup>Lexer</sup>**\
@@ -542,8 +540,6 @@ Lifetime parameters and [loop labels] use LIFETIME_OR_LABEL tokens. Any
542540
LIFETIME_TOKEN will be accepted by the lexer, and for example, can be used in
543541
macros.
544542

545-
[loop labels]: expressions/loop-expr.md
546-
547543
## Punctuation
548544

549545
Punctuation symbol tokens are listed here for completeness. Their individual
@@ -609,6 +605,41 @@ them are referred to as "token trees" in [macros]. The three types of brackets
609605
| `[` `]` | Square brackets |
610606
| `(` `)` | Parentheses |
611607

608+
## Reserved prefixes
609+
610+
> **<sup>Lexer 2021+</sup>**\
611+
> RESERVED_TOKEN_DOUBLE_QUOTE : ( IDENTIFIER_OR_KEYWORD <sub>_Except `b` or `r` or `br`_</sub> | `_` ) `"`\
612+
> RESERVED_TOKEN_SINGLE_QUOTE : ( IDENTIFIER_OR_KEYWORD <sub>_Except `b`_</sub> | `_` ) `'`\
613+
> RESERVED_TOKEN_POUND : ( IDENTIFIER_OR_KEYWORD <sub>_Except `r` or `br`_</sub> | `_` ) `#`
614+
615+
Some lexical forms known as _reserved prefixes_ are reserved for future use.
616+
617+
Source input which would otherwise be lexically interpreted as a non-raw identifier (or a keyword or `_`) which is immediately followed by a `#`, `'`, or `"` character (without intervening whitespace) is identified as a reserved prefix.
618+
619+
Note that raw identifiers, raw string literals, and raw byte string literals may contain a `#` character but are not interpreted as containing a reserved prefix.
620+
621+
Similarly the `r`, `b`, and `br` prefixes used in raw string literals, byte literals, byte string literals, and raw byte string literals are not interpreted as reserved prefixes.
622+
623+
> **Edition Differences**: Starting with the 2021 edition, reserved prefixes are reported as an error by the lexer (in particular, they cannot be passed to macros).
624+
>
625+
> Before the 2021 edition, a reserved prefixes are accepted by the lexer and interpreted as multiple tokens (for example, one token for the identifier or keyword, followed by a `#` token).
626+
>
627+
> Examples accepted in all editions:
628+
> ```rust
629+
> macro_rules! lexes {($($_:tt)*) => {}}
630+
> lexes!{a #foo}
631+
> lexes!{continue 'foo}
632+
> lexes!{match "..." {}}
633+
> lexes!{r#let#foo} // three tokens: r#let # foo
634+
> ```
635+
>
636+
> Examples accepted before the 2021 edition but rejected later:
637+
> ```rust,edition2018
638+
> macro_rules! lexes {($($_:tt)*) => {}}
639+
> lexes!{a#foo}
640+
> lexes!{continue'foo}
641+
> lexes!{match"..." {}}
642+
> ```
612643
613644
[Inferred types]: types/inferred.md
614645
[Range patterns]: patterns.md#range-patterns
@@ -636,6 +667,7 @@ them are referred to as "token trees" in [macros]. The three types of brackets
636667
[if let]: expressions/if-expr.md#if-let-expressions
637668
[keywords]: keywords.md
638669
[lazy-bool]: expressions/operator-expr.md#lazy-boolean-operators
670+
[loop labels]: expressions/loop-expr.md
639671
[machine types]: types/numeric.md
640672
[macros]: macros-by-example.md
641673
[match]: expressions/match-expr.md
@@ -656,42 +688,7 @@ them are referred to as "token trees" in [macros]. The three types of brackets
656688
[tuple structs]: items/structs.md
657689
[tuple variants]: items/enumerations.md
658690
[tuples]: types/tuple.md
691+
[unary minus operator]: expressions/operator-expr.md#negation-operators
659692
[use declarations]: items/use-declarations.md
660693
[use wildcards]: items/use-declarations.md
661694
[while let]: expressions/loop-expr.md#predicate-pattern-loops
662-
663-
## Reserved prefixes
664-
665-
> **<sup>Lexer 2021+</sup>**\
666-
> RESERVED_TOKEN_DOUBLE_QUOTE : ( IDENTIFIER_OR_KEYWORD <sub>_Except `b` or `r` or `br`_</sub> | `_` ) `"`\
667-
> RESERVED_TOKEN_SINGLE_QUOTE : ( IDENTIFIER_OR_KEYWORD <sub>_Except `b`_</sub> | `_` ) `'`\
668-
> RESERVED_TOKEN_POUND : ( IDENTIFIER_OR_KEYWORD <sub>_Except `r` or `br`_</sub> | `_` ) `#`
669-
670-
Some lexical forms known as _reserved prefixes_ are reserved for future use.
671-
672-
Source input which would otherwise be lexically interpreted as a non-raw identifier (or a keyword or `_`) which is immediately followed by a `#`, `'`, or `"` character (without intervening whitespace) is identified as a reserved prefix.
673-
674-
Note that raw identifiers, raw string literals, and raw byte string literals may contain a `#` character but are not interpreted as containing a reserved prefix.
675-
676-
Similarly the `r`, `b`, and `br` prefixes used in raw string literals, byte literals, byte string literals, and raw byte string literals are not interpreted as reserved prefixes.
677-
678-
> **Edition Differences**: Starting with the 2021 edition, reserved prefixes are reported as an error by the lexer (in particular, they cannot be passed to macros).
679-
>
680-
> Before the 2021 edition, a reserved prefixes are accepted by the lexer and interpreted as multiple tokens (for example, one token for the identifier or keyword, followed by a `#` token).
681-
>
682-
> Examples accepted in all editions:
683-
> ```rust
684-
> macro_rules! lexes {($($_:tt)*) => {}}
685-
> lexes!{a #foo}
686-
> lexes!{continue 'foo}
687-
> lexes!{match "..." {}}
688-
> lexes!{r#let#foo} // three tokens: r#let # foo
689-
> ```
690-
>
691-
> Examples accepted before the 2021 edition but rejected later:
692-
> ```rust,edition2018
693-
> macro_rules! lexes {($($_:tt)*) => {}}
694-
> lexes!{a#foo}
695-
> lexes!{continue'foo}
696-
> lexes!{match"..." {}}
697-
> ```

0 commit comments

Comments
 (0)