Fix parsing logic in `proc_macro::quote` #148209

moatom · 2025-10-28T13:41:55Z

library/proc_macro/src/quote.rs

rustbot · 2025-10-30T15:15:04Z

rustbot has assigned @petrochenkov.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

moatom · 2025-10-30T15:16:19Z

r? @tgross35

rustbot · 2025-10-30T15:16:23Z

tgross35 is currently at their maximum review capacity.
They may take a while to respond.

tgross35 · 2025-10-30T18:07:40Z

David knows this much better than I do

r? @dtolnay

moatom · 2025-10-31T00:04:57Z

@bors r+

bors · 2025-10-31T00:05:00Z

@moatom: 🔑 Insufficient privileges: Not in reviewers

tgross35 · 2025-10-31T04:15:46Z

@bors r+

@moatom what are you trying to do here? This is the command to approve PRs.

moatom · 2025-10-31T04:51:16Z

@tgross35 thx. I wanted to restart CI just in case because this PR compiles locally.

tgross35 · 2025-10-31T06:09:44Z

I restarted it. For future reference, you can either force push something or close+reopen to restart CI (bors doesn’t interact with PR CI)

moatom · 2025-11-01T09:32:30Z

@dtolnay I've resolved all of the problems. Could you please confirm when you have a moment?

dtolnay · 2025-11-04T18:25:06Z

library/proc_macro/src/quote.rs

 match tree {
- TokenTree::Group(tt) => {
+ TokenTree::Group(ref tt) => {
 // Handles repetition by expanding `$( CONTENTS ) SEP_OPT *` to `{ REP_EXPANDED }`.


The repetition syntax is supposed to be $(CONTENTS) SEP *, like inside macro_rules. Not $[CONTENTS] SEP * and not ${CONTENTS} SEP *, and especially not $ CONTENTS SEP * with Delimiter::None. Please add tests for these.

Please add tests for these.

I think it is impossible to prepare it for Delimiter::None.

It can be tested by calling quote! from a macro_rules macro.

macro_rules! do_quote { ($dollar:tt $content:expr) => { proc_macro::quote!($dollar $content *) }; } let arr = [0; 2]; eprintln!("{}", do_quote!($ f!($arr)));

On current nightly this nonsensically prints f! (0i32) f! (0i32).

dtolnay · 2025-11-04T18:26:54Z

library/proc_macro/src/quote.rs

- ]);
-
- minimal_quote!((@ TokenTree::Group(Group::new(Delimiter::Brace, rep_expanded)))).to_tokens(&mut tokens);
+ consume_dollar_group_sep_star(contents.clone(), &mut iter)


Suggested change

consume_dollar_group_sep_star(contents.clone(), &mut iter)

consume_dollar_group_sep_star(contents, &mut iter)

This has not been addressed. You made a change on the same line but it is not the suggested one. There is no reason to clone the value of tt.stream(), which is already an owned TokenStream.

tests/ui/proc-macro/quote/auxiliary/basic.rs

library/proc_macro/src/quote.rs

moatom · 2025-11-06T15:31:07Z

@dtolnay
Thank you for the review!
I've addressed your comments and left the fix commit separate for traceability.
Please let me know if it's okay to squash.

dtolnay · 2025-11-10T05:42:40Z

tests/ui/proc-macro/quote/not-quotable-repetition-group-bracket.rs

+
+fn main() {
+ let arr = [1, 2, 3];
+ let _ = quote! { ${$arr}* }; //~ ERROR the trait bound `[{integer}; 3]: ToTokens` is not satisfied [E0277]


Suggested change

let _ = quote! { ${$arr}* }; //~ ERROR the trait bound `[{integer}; 3]: ToTokens` is not satisfied [E0277]

let _ = quote! { $[$arr]* }; //~ ERROR the trait bound `[{integer}; 3]: ToTokens` is not satisfied [E0277]

dtolnay · 2025-11-10T05:55:57Z

library/proc_macro/src/quote.rs

- match &tree {
- TokenTree::Punct(tt) if tt.as_char() == '$' => {
- if let Some(TokenTree::Ident(id)) = iter.peek() {
+ while let Some(ref tree) = iter.next() {


This is going to miss adjacent metavariables. For example this works:

let a = ['a']; let b = ['b']; eprintln!("{}", proc_macro::quote!($($a . $b)*));

but this does not:

eprintln!("{}", proc_macro::quote!($($a $b)*));

[note] This was caused by an extra iter.next();: b2ae84c#diff-6ebaa6be050b7cea1d542579c11cde3cba0942a3417c53298001f9db02a8d06dL407

dtolnay · 2025-11-10T06:26:39Z

library/proc_macro/src/quote.rs

 }

+/// Consume a `$( CONTENTS ) SEP *` accordingly. It handles repetition by expanding `$( CONTENTS ) SEP *` to `{ REP_EXPANDED }`.
+fn consume_dollar_group_sep_star(


This is accepting too many tokens in the "separator" in some cases.

let arr = ['a', 'b']; eprintln!("{}", proc_macro::quote!($($arr) $$ x .. false [] *));

'a' $x .. false [] 'b'

I introduced is_valid_sep_prefix to resolve this, but there might be a more elegant solution.

moatom · 2025-11-18T18:00:20Z

r? @dtolnay

rustbot · 2025-11-18T18:00:22Z

Requested reviewer is already assigned to this pull request.

Please choose another assignee.

rustbot · 2025-11-26T16:59:31Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

moatom · 2025-11-26T17:04:06Z

@dtolnay

I've addressed your comments and left the fix commit separate for traceability.
Please let me know if it's okay to squash.

I've now rebased and squashed the commits for smoother communication.
If you still need the detailed history, please take a look at moatom:stabilize-proc-macro-saved.

moatom · 2025-11-26T17:12:34Z

library/proc_macro/src/quote.rs

+ consume_dollar_group_sep_star(tt.stream(), &mut iter).to_tokens(&mut tokens);
+ continue;
+ }
+ TokenTree::Group(_) => {


output_dollar_group_sep seems available here too.

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Oct 28, 2025

This comment has been minimized.

Sign in to view

bjorn3 reviewed Oct 28, 2025

View reviewed changes

library/proc_macro/src/quote.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

moatom force-pushed the stabilize-proc-macro branch from 114e740 to 02f04b0 Compare October 30, 2025 14:12

This comment has been minimized.

Sign in to view

moatom force-pushed the stabilize-proc-macro branch from 02f04b0 to 0e341e2 Compare October 30, 2025 15:13

moatom changed the title ~~WIP: Fix parseing in proc_macro::quote~~ Fix parsing in proc_macro::quote Oct 30, 2025

moatom marked this pull request as ready for review October 30, 2025 15:14

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 30, 2025

rustbot assigned petrochenkov Oct 30, 2025

rustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Oct 30, 2025

rustbot assigned tgross35 and unassigned petrochenkov Oct 30, 2025

moatom changed the title ~~Fix parsing in proc_macro::quote~~ Fix parsing logic in proc_macro::quote Oct 30, 2025

moatom force-pushed the stabilize-proc-macro branch 2 times, most recently from 8dc03cd to 36c13e9 Compare October 30, 2025 15:27

This comment has been minimized.

Sign in to view

rustbot assigned dtolnay and unassigned tgross35 Oct 30, 2025

moatom force-pushed the stabilize-proc-macro branch from 36c13e9 to 900f0e2 Compare November 1, 2025 08:06

This comment has been minimized.

Sign in to view

dtolnay requested changes Nov 4, 2025

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 4, 2025

moatom requested a review from dtolnay November 6, 2025 15:31

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 6, 2025

dtolnay reviewed Nov 10, 2025

View reviewed changes

moatom requested a review from dtolnay November 18, 2025 17:36

moatom force-pushed the stabilize-proc-macro branch from abc4763 to ae0617b Compare November 18, 2025 17:57

This comment has been minimized.

Sign in to view

moatom force-pushed the stabilize-proc-macro branch from ae0617b to 93fd70a Compare November 19, 2025 00:18

This comment has been minimized.

Sign in to view

moatom force-pushed the stabilize-proc-macro branch 3 times, most recently from 88f4fbe to 3613f22 Compare November 19, 2025 00:50

Fix parsing logic in proc_macro::quote

62534aa

moatom force-pushed the stabilize-proc-macro branch from 3613f22 to 62534aa Compare November 26, 2025 16:59

moatom commented Nov 26, 2025

View reviewed changes

	consume_dollar_group_sep_star(contents.clone(), &mut iter)
	consume_dollar_group_sep_star(contents, &mut iter)

	let _ = quote! { ${$arr}* }; //~ ERROR the trait bound `[{integer}; 3]: ToTokens` is not satisfied [E0277]
	let _ = quote! { $[$arr]* }; //~ ERROR the trait bound `[{integer}; 3]: ToTokens` is not satisfied [E0277]

Fix parsing logic in proc_macro::quote #148209

Are you sure you want to change the base?

Fix parsing logic in proc_macro::quote #148209

Uh oh!

Conversation

moatom commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

This comment has been minimized.

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rustbot commented Oct 30, 2025

moatom commented Oct 30, 2025

rustbot commented Oct 30, 2025

This comment has been minimized.

tgross35 commented Oct 30, 2025

moatom commented Oct 31, 2025

bors commented Oct 31, 2025

tgross35 commented Oct 31, 2025

moatom commented Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

tgross35 commented Oct 31, 2025

This comment has been minimized.

moatom commented Nov 1, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

moatom commented Nov 6, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moatom commented Nov 18, 2025

rustbot commented Nov 18, 2025

This comment has been minimized.

This comment has been minimized.

rustbot commented Nov 26, 2025

moatom commented Nov 26, 2025

Choose a reason for hiding this comment

Labels

8 participants

Fix parsing logic in `proc_macro::quote` #148209

Fix parsing logic in `proc_macro::quote` #148209

moatom commented Oct 28, 2025 •

edited

Loading

moatom commented Oct 31, 2025 •

edited

Loading