fix removing literal suffixes from vec![10u32, 20u32, 30u32] #1368
Conversation
ahomescu
force-pushed
the
ahomescu/fix_reorganize_definitions
branch
from
3b6b204 to
0f281b4
Compare
|
This fixes the issue? Was a macro invocation also the problem in the transpiled code? If the problem is |
| // Regression input for the token-stream mismatch caused by editing vec! literals. | ||
|
|
||
| fn main() { | ||
| let ints = vec![10u32, 20u32, 30u32]; |
There was a problem hiding this comment.
Are we adding, or deleting a token stream here?
There was a problem hiding this comment.
My impression is that, because we're editing the tokens, the cache of tokens is invalidated and rustc generates a new token stream. That mismatch in None token stream and Some(TokenStream) is the structural mismatch that caused a panic before and this changes.
This fixes the issue?
yes
If the problem is LazyTokenStream, the problem might be that we're missing an attribute for the matcher for that node
I'm not sure how to address this question. I think the crash was happening before any token stream comparison happened, i.e. before it recursed into Some(TokenStream). Are you saying there could be some attribute that results in comparing two Some(_) values rather than (None, Some) such that a crash wouldn't have happened?
There was a problem hiding this comment.
Are you saying there could be some attribute that results in comparing two Some(_) values rather than (None, Some) such that a crash wouldn't have happened?
It shouldn't be doing the comparison in the first place, can you figure out where it's happening? We have LazyTokenStream set to ignore
#[equiv_mode=ignore] #[match=ignore] #[rewrite_ignore]
flag LazyTokenStream;
so the two streams should always compare as equal.
When running the refactor tool for removing literal suffixes on a
vec![10u32, 20u32, 30u32]for example, the refactor tool would crash because the collapse pipeline code inCollectMacroswas expecting structural equivalence in unexpanded/expanded ASTs, but by rewriting the macro contents, we introduced aLazyTokenStreamin the expansion.This treats such a scenario as expected, but still panics for the other non-token-stream causes of structural mismatch.