If you want to delve into the code straight, you can check it out here.

clippy::nonstandard_macro_braces is a Clippy lint that catches macro calls with the wrong braces.

In Rust, if you want to instantiate an empty vector, you can call the vec macro vec![]. However, there’s technically nothing stopping you from doing vec!(), or even vec! {"???"}.

However, everyone hates that. Everyone hates println! {}.

So, Clippy has the perfect lint to avoid you becoming the noobie in your community. It’s the forementioned clippy::nonstandard_macro_braces, which has some hard-coded macros and their idiomatic braces.

Now, Clippy runs after macro expansion. This means that, for example, instead of seeing println! {"..."}, Clippy sees:

std::io::_print(std::format_args_nl!("Hello, world!"));

Take a minute to look at it.

Judging by that code snippet, where exactly are the braces stored? I’m sure you’re very smart, that’s why you are WRONG

There’s no way that we can know what braces were used, not in a pre-expansion lint anyways. Rust does not have a real defined macro callmap, and that might be the greatest pain-point that Clippy has to deal with.

So we first start out investigation by, you guessed it, checking if we’re in a macro expansion. Rust knows if the current tokens comes from expansion, sometimes.

Okay, now the only thing’s left is going up the chain with the following formula:

Now we’ll do some source text trickery, we get the source text for the span according to the macro expansion in the hygiene data.

So, we have the following:

std::io::_print(std::format_args_nl!("Hello, world!"));
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
           |
           |- Check with the static "session globals" for the hygiene data, get the span "src/main.rs:1:16 to src/main:1:32"

Looking at the same session globals, which holds the hygiene data, we can go look at the span “file src/main.rs, line 1, column 16” to “file src/main.rs, line 1, column 32”. That line corresponds to this string:

println! {"..."}

And, if we where to do some string manipulation (split the string by !, trim the second block and look if the first character is [, ( or {). We have finally found out what the brace is!

So, we’ve called the hygiene data functions twice, locked the symbol interner (a system which transforms code identifiers into their string formats, for easier comparing), and we’ve engaged in a recursive loop.

FOR EVERY SINGLE EXPRESSION.

You’re suspicion is correct, we were calling all this code for every single expression, statment and item, in your whole codebase.

That means that we’re calling the session globals (effectively blocking every single other thing in the compiler) AND locking the symbol interner (Which slows down everything else in the compiler), for pretty much EVERY LOGIC UNIT IN YOUR CODE.

I want to be clear here, this wasn’t an error that one person did because of their malpractice or malevolence. Blaming the contributor is never the right call.

This is on us, the maintainers. Our role is to make having bad code as hard as possible.

Rust has done the hard part for us, we can omit thinking about buffer overflows or use-after-free errors (or at least, not having it very present unless we’re doing quirky stuff).

Yes, that sometimes means writing more code. Yes, that means not using as many macros. And that definitely means not abstracting away your way into madness, so that a seemingly inocuous function (one that you’ve probably seen a hundred times by now as a contributor) is taking 25% of your Clippy runtime.

We have three duties are open source maintainers:

  1. Complain about feature request. The most important one.
  2. Do not mess with the user’s workflow, that’s sacred.
  3. Reduce bugs.

It seems that, if we code slipped by, we haven’t been doing our jobs.

The fix? Less than 200 lines of code. I simply rewrote the problematic function from a post-expansion lint into a pre-expansion one.

No more having to get the source text and calculate the bracket span with math. No more locking the symbol interner, and no more locking the session globals.

sigh~, I know, if you know anything about Clippy is that lints are run post-expansion. Pre-expansion code is not trusted, it deceives you. In the end, it’s a hack. But it’s a hack that’s saving hundreds of thousands of computing dollars.

Minimizing performance issues in the future

Clippy now has a benchmarking server. Yes, I’ve been trying to build one for close to 4 years now, well, it’s now done. Thanks to the Rust Foundation contracting me, I’ve been able to scale tremendously my operations. And we now have a benchmarking server with 200 days of memory.

It’s all self-hosted, at least to an extent. And it’s also home-benchmarked, so the numbers will probably be very close to real user experience (I have a very common CPU architecture.)

If you want to hear more about the setup, send me an email.

Thanks for reading, see you next time Clippy is revolutionized (or maybe something else…)