Spyder 6 significantly slower working with projects on entwork drives than 5.5

[gb119]

Problem Description

I often work with projects that are stored on a network drive (that is known not to be super fast since it's an antiquated samba server), however I'm seeing significant regressions in general speed when comparing Spyder 6 to Spyder 5.5.6 (tested on the same machine at the same time with two different conda environments and different configuration directories). Certainly general project related operations (opening/closing projects) are much slower on 6 than 5.5 with long periods where the UI becomes unresponsive - particularly on project load or initial application startup with an open project.

I've had similar issues with Spyder 6 speed on two different machines - although I've only done the 'head-to-head' comparison on one of those - and done things like conda update --all on the envuronments.

Possibly related, about 40% of the time Spyder 6's Ipython console announces than "conda run" had failed - I don't believe this is real and suspect that there is some timeout happening as the other 60% of the time it manages.

I'm also seeing some zmq errors in the internal console:

self.stream = zmqstream.ZMQStream(self.socket, self.ioloop)
C:\ProgramData\anaconda3\envs\py312\Lib\site-packages\jupyter_client\threaded.py:73: RuntimeWarning: ZMQStream only supports the base zmq.Socket class.

            Use zmq.Socket(shadow=other_socket)
            or `ctx.socket(zmq.SUB, socket_class=zmq.Socket)`
            to create a base zmq.Socket object,
            no matter what other kind of socket your Context creates.

self.stream = zmqstream.ZMQStream(self.socket, self.ioloop)

Sorry, not to be more specific.

Mandatory:

aiohttp >=3.9.3 : 3.10.5 (OK)
asyncssh >=2.14.0,<3.0.0 : 2.17.0 (OK)
atomicwrites >=1.2.0 : 1.4.0 (OK)
chardet >=2.0.0 : 4.0.0 (OK)
cloudpickle >=0.5.0 : 3.0.0 (OK)
cookiecutter >=1.6.0 : 2.6.0 (OK)
diff_match_patch >=20181111 : 20200713 (OK)
github >=2.3.0 : 2.4.0 (OK)
importlib_metadata >=4.6.0 : 7.0.1 (OK)
intervaltree >=3.0.2 : 3.1.0 (OK)
IPython >=8.13.0,<9.0.0,!=8.17.1 : 8.27.0 (OK)
jedi >=0.17.2,<0.20.0 : 0.19.1 (OK)
jellyfish >=0.7 : 1.0.1 (OK)
jsonschema >=3.2.0 : 4.19.2 (OK)
keyring >=17.0.0 : 24.3.1 (OK)
nbconvert >=4.0 : 7.16.4 (OK)
numpydoc >=0.6.0 : 1.7.0 (OK)
parso >=0.7.0,<0.9.0 : 0.8.3 (OK)
pexpect >=4.4.0 : 4.8.0 (OK)
pickleshare >=0.4 : 0.7.5 (OK)
psutil >=5.3 : 5.9.0 (OK)
pygments >=2.0 : 2.15.1 (OK)
pylint >=3.1,<4 : 3.2.7 (OK)
pylint_venv >=3.0.2 : 3.0.3 (OK)
pyls_spyder >=0.4.0 : 0.4.0 (OK)
pylsp >=1.12.0,<1.13.0 : 1.12.0 (OK)
pylsp_black >=2.0.0,<3.0.0 : 2.0.0 (OK)
pyuca >=1.2 : 1.2 (OK)
qdarkstyle >=3.2.0,<3.3.0 : 3.2.3 (OK)
qstylizer >=0.2.2 : 0.2.2 (OK)
qtawesome >=1.3.1,<1.4.0 : 1.3.1 (OK)
qtconsole >=5.6.0,<5.7.0 : 5.6.0 (OK)
qtpy >=2.4.0 : 2.4.1 (OK)
rtree >=0.9.7 : 1.0.1 (OK)
setuptools >=49.6.0 : 72.1.0 (OK)
sphinx >=0.6.6 : 7.3.7 (OK)
spyder_kernels >=3.0.0,<3.1.0 : 3.0.0 (OK)
superqt >=0.6.2,<1.0.0 : 0.6.7 (OK)
textdistance >=4.2.0 : 4.2.1 (OK)
three_merge >=0.1.1 : 0.1.1 (OK)
watchdog >=0.10.3 : 4.0.1 (OK)
yarl >=1.9.4 : 1.11.0 (OK)
zmq >=24.0.0 : 25.1.2 (OK)

Optional:

cython >=0.21 : None (NOK)
matplotlib >=3.0.0 : 3.9.2 (OK)
numpy >=1.7 : 1.26.4 (OK)
pandas >=1.1.1 : 2.2.2 (OK)
scipy >=0.17.0 : 1.12.0 (OK)
sympy >=0.7.3 : 1.13.2 (OK)

[gb119]

As a further data point, this happens on 6.0.1 as well, but it seems to be associated with loading a Project from a network drive. In the following "fast" means a few seconds, "slow" means more than a minute:
Fast:

• Loading spyder with no project, but editor pane with several files (doesn't make a difference whether the files are local or network based)
• Loading Spyder with a local project, ediutor pane with several local files
• Switching between local projects in an open Spyder

Slow:

• Loading Spyder with a project from a network drive (doesn't seem to make a different how m,any files are open in the editor pane).
• Switching between projects on network drives.

It seems that the longest part of the delay happens after the editor pane loads the files from the new project and before the console panel has restarted with the new project.

Should add, this is all on Windows x86_64 11. I'll see if I can get a working Spyder 6 in WSL which is the best I can do for checking other platforms.

[ccordoba12]

Hey @gb119, thanks for reporting. We did two major changes to projects in Spyder 6 that could be causing this:

• Show project files in our file switcher with the fzf package. You can disable that by going here:

  spyder/spyder/plugins/projects/widgets/main_widget.py

  Line 174 in ad17193

  self._fzf = find_program('fzf')

  and changing that line to self._fzf = None.

• Using a simpler file system watcher. You can disable it by commenting this line:

  spyder/spyder/plugins/projects/widgets/main_widget.py

  Line 410 in ad17193

  self.watcher.start(path)

Let us know if those changes work for you. With that info we could add a way disable one or the other in Preferences.

[gb119]

So it appears that I need both suggested fixes to avoid Spyder 6 (6.0.0, I hadn't bumped the one I was testing to 6.0.1) becoming unresponsive.
I was still having persistent problems with it thinking conda run wasn't a command though - is there a point where it does a test for installed conda that might be timing out on me? It stops the Ipython console from working.

What do the things I've disabled do for me?

Happy to do further testing, but it's the start of our academic year so I'm little busy (why do I always report these things at the start of term!?)

[ccordoba12]

So it appears that I need both suggested fixes to avoid Spyder 6 (6.0.0, I hadn't bumped the one I was testing to 6.0.1) becoming unresponsive.

Thanks for the confirmation @gb119. We'll try to address this problem in 6.0.3 or 6.0.4, both to be released before the end of the year.

I was still having persistent problems with it thinking conda run wasn't a command though - is there a point where it does a test for installed conda that might be timing out on me? It stops the Ipython console from working.

I don't understand this comment very well. Is the IPython console not working for you? Or are you getting an error similar to the one reported in #22554?

What do the things I've disabled do for me?

• No fzf: project files won't be listed in the file switcher to easily opening them in the Editor.
• No watcher: linting won't be updated automatically for project files open in the Editor when another project file is modified.

Happy to do further testing, but it's the start of our academic year so I'm little busy (why do I always report these things at the start of term!?)

No problem, I think your last comment has all the info we need.

[gb119]

I was still having persistent problems with it thinking conda run wasn't a command though - is there a point where it does a test for installed conda that might be timing out on me? It stops the Ipython console from working.

I don't understand this comment very well. Is the IPython console not working for you? Or are you getting an error similar to the one reported in #22554?

So sometimes when starting Spyder (with or without the edits to the projects widget), the Ipython shell comes up looking like this:

, but if I simply restart Spyder then it comes up as I would expect:

When it is in the error condition then restarting just the console or unloading and loading projects does not fix the issue and it always comes back saying that there is no command "conda run". Needless to say there is a "conda run" in the environment and most of the time spyder seems to find it ok! I've tried start spyder from the command line with the debugging output on but I can't (easily) see where it decides that conda run doesn't exist.
From the "sticky" nature of the problem and the fact it is intermittent (and not predictable as far as I can tell) makes me think that there must be a race condition or a timeout or similar at the point where spyder is deciding it is working in a conda environment - but I've not looked at the code enough to know where to look and what to look for. Sorry, that's not the most useful bug report!