IntelliSense indexing is very slow on RHEL & CentOS clusters

[djairao]

Hello,

I am using VS-Code (C/C++ extension version 1.5.1) for remote C++ programming (via SSH) on a HPC cluster running RHEL (7.9) using GCC 8.4. IntelliSense indexing is very slow on the RHEL cluster. I tried a different HPC cluster running CentOS (7.8) and IntelliSense indexing was slow there too. In both cases the small flame (:fire:) icon shows up on the bottom and stays there for about 6 minutes. Then the indexing starts (I have enabled debug level logging) and the tag parser runs (showing the cylinder icon in the bottom) for hours slowly parsing about 20,000 headers, one/few headers at a time (logs show up in blocks), recursively from /usr/include.

However, the same operations run very fast (in less than a minute) on Ubuntu and Fedora Linux servers (same GCC 8.4 compiler). So there is some setup, albeit consistent, on the two HPC servers that cause IntelliSense to run slowly. Unfortunately, there isn't sufficient information in the logs (even in Debug mode) to troubleshoot this performance issue.

Of course, limiting the number of files to index definitely helps. Here is my settings.json file (all other settings are defaults that come out of the box) that limits the folders to speed up the process but only helps marginally (I would actually like to index more headers):

// Temporary settings to help reduce the number of headers to make IntelliSense somewhat tolerable
{
    "remote.autoForwardPortsSource": "output",
    "C_Cpp.default.cppStandard": "c++17",
    "C_Cpp.default.systemIncludePath": [ "/usr/include/*",
        "/usr/local/gnu/8.4.0/**", 
        "/usr/local/boost/gnu/8.4/mvapich2/2.3/1.72.0/**",
        "/opt/mvapich2/gnu/8.4/2.3.3/**",
        "${workspaceFolder}/**",
    ],
}

I am wondering if there are any suggestions or tips that I can explore to improve IntelliSense performance? I will have 100s of users trying to use this on the cluster very soon. Thank you, in advance, for any help/troubleshooting tips.

[djairao]

Summary: It was disk I/O that was slowing things down

After much struggle and spending hours trying to strace and analyze the cpptols binary, I ran perf and it hinted hotspots with __libc_lseek64 and sqlite3VdbeExec etc. That led me to think maybe the issue was with the mounts --

• On the Linux machines where IntelliSense was running fast, the mounts were on local disk (ext4 mounts)
• On the Linux machines where IntelliSense was running low, the mounts were remote/NAS mounts (nfs4 or gpfs mounts)

That led me to explore the possibility of using a local disk for storing the ~/.vscode-server directory. Unfortunately, there isn't a clean option in the VS-Code UI to specify an install directory (if there is one, I could not find it). So I deleted the existing folder and then created a directory in /tmp and symlinked it to ~/.vscode-server directory. Then I reinstalled the cpptools extension and viola! The indexing just blazed through.

So in summary, the performance of IntelliSense is heavily dependent on the underlying disk. If you are using any kind of network-attached storage, then IntelliSense is going to be very slow.

[djairao]

A simpler/better solution is to set up the databaseFilename in code-server/User/settings.json as shown below. It works well for me use case.

{
    "C_Cpp.default.compilerPath": "/opt/mvapich2/gnu/8.4/2.3.3/bin/mpicxx",
    "C_Cpp.default.cppStandard": "c++17",
    "C_Cpp.commentContinuationPatterns": [

        "/**"
    ],
    "C_Cpp.default.browse.databaseFilename": "/tmp/${USER}/cpp-vscode.db"
}