IDE re-indexes the same IC sources. Side effect of using Gradle's "transform" cache?

[jonathanlermitage]

What happened?

Sometimes, when working on a plugin project, the IDE starts indexing IDE's sources again, even if it has been indexed before. I'm talking about the same sources, per example idea:ideaC:win:2024.1.2.
I did not clear Gradle's cache. The indexing issue happens randomly, on project loading, or when I update & reload my build.gradle.kts.
It happened with projects using intellij platform plugin beta 4 and 5. I did not pay attention to this when using older betas.

I guess this is a side effect of using the Gradle's transform directory for caching IDEs sources. It did not happen with plugin v1 because it was using the cache directory. Please look at the screenshots: the transform directory seems a bit unstable, which could explain the issue:

• Two days ago:
  
• Today:
  

As you can see, the ID in ...transform-4/ID/transformed/... path changed. Maybe IDE sources should not be managed this way, or IDE should be able to adapt.
It seems random. I faced the issue two days ago and today, but nothing yesterday.

Using IntelliJ Ultimate 2024.1.2.

Relevant log output or stack trace

No response

Steps to reproduce

Reload project or edit build.gradle.kts. Please not this is random, so you may need time, and luck, to reproduce this issue.

Gradle IntelliJ Plugin version

2.0.0-beta5

Gradle version

8.7

Operating System

Windows

Link to build, i.e. failing GitHub Action job

No response

[jonathanlermitage]

I noticed IdeaC 2024.1.2 is stored twice in my transforms:

If you look at the paths, we have ...transforms-4/d747... and ...transforms-4/cec.... This is what is also displayed in the two first screenshots. I compared some files (like build.txt and product-info.json), and it seems that these two directories contains the exact same files. This would mean the same IDE has been cached multiple times (and not at the same time).
I don't know if it helps...

[hsz]

Hello, Jonathan! I am very well aware of this problem. Let me explain to you the state of the current implementation:

Adding the IntelliJ Platform dependency to the project resolves it from the IJ Maven Repository or CDN (download.jetbrains.com).
Both sources provide the IntelliJ Platform archive — Maven gives ZIP and CDN, depending on your OS: DMG, ZIP, or TAR.GZ.
Gradle fetches such an archive into the cache directory, like ~/.gradle/caches/modules-2/files-2.1/....

The Gradle IntelliJ Plugin 1.x is extracting content next to the archive, polluting the cache, but it works.
With IntelliJ Platform Gradle Plugin 2.0, I decided to do it correctly and involve the artifact transformers mechanism provided by Gradle. The extracted content goes into a dedicated ~/.gradle/caches/transforms-4/[HASH] location, and the dependency can be correctly formed using native Gradle features.

After the implementation, it turned out, that Gradle calculates [HASH] using the project build classpath. This means that this hash changes whenever you update the IntelliJ Platform Gradle Plugin plugin or any other Gradle plugin in your project (or have a buildSrc local setup that changes).
As a side effect, the IntelliJ Platform archive gets extracted again, the cache grows, and the IDE reindexes the IntelliJ Platform dependency again.

I already had a chat with Gradle folks about this issue, but there's no solution possible to keep relying on the artifact transformers feature.
I'm currently trying to figure out another solution, and most likely, we'll introduce a custom cache location, such as ~/.intellijPlatform/ides/, to keep just a single copy of the extracted IDE.
Keep your fingers crossed!