direnv loads too late sometimes

[peterhoeg]

direnv doesn't add itself to the front of the post-command-hook meaning other things get to run before direnv has set up the environment.

One example is a haskell/stack project directory, where flycheck cannot find the executables because it runs before direnv has updated the PATH.

Cc: @bkchr

[bkchr]

That is the way I solved it for setup.

[wbolster]

this packages uses:

  (add-hook 'post-command-hook #'direnv--maybe-update-environment)

which puts it at the beginning:

(add-hook HOOK FUNCTION &optional APPEND LOCAL)
...
FUNCTION is added (if necessary) at the beginning of the hook list

but packages loaded later may use the same pattern, causing direnv to get push further down.

perhaps this package could try to move its own hook to the front all the time.

[peterhoeg]

Thanks for the pointer @bkchr and that does indeed solve the flycheck issue but it is a workaround specific to flycheck and a proper fix for direnv would be very neat.

[bkchr]

Yeah a proper fix would be better :) I already do this "hack" for flycheck and the lsp-mode.

[wbolster]

actually i am not convinced what the right "fix" is tbh, since i do not fully understand the actual issue here.

for instance, i use direnv with python and virtualenvs which also contain linters (flake8) and i have never experienced issues even though the flycheck linter requires the PATH to be set up for the project.

[bkchr]

The right fix would something that runs direnv directly after switching to the buffer, before any minor/major mode hooks are called. For example the lsp mode starts are server after entering a buffer and to start the server it needs to know the location of the executable. So, the executable of the server is searched when the major-mode is loaded and that is before direnv has setup the environment.

[wbolster]

ok. if you know the right trick/hook (if it exists at all) to get direnv to run at the right time, lemme know, because i have no idea apart from post-command-hook which is already a bit aggressive...

[bkchr]

Maybe change-major-mode-hook could be used? I think that post-command-hook is still required for buffers without a major mode, but I'm not sure.

[wbolster]

but that hook is definitely not good enough to cover the main use case of this package: when switching between files (buffers, windows, frames) the env needs to keep "in sync" with the current file.

perhaps both could work? can you try adding direnv--maybe-update-environment to change-major-mode-hook and see how that works out?

[wbolster]

fwiw, for flycheck it seems this does the trick:

  (add-hook 'flycheck-before-syntax-check-hook 'direnv-update-environment)

[adelbertc]

Hey all, I've been playing with emacs-direnv + Nix + Haskell and hit this same issue. @wbolster with your hook suggestion:

(use-package direnv
  :hook (flycheck-before-syntax-check . direnv-update-environment)
  :config
  (direnv-mode))

(use-package flycheck
  :init
  (global-flycheck-mode))

(use-package flycheck-haskell
  :hook (haskell-mode . flycheck-haskell-setup))

(use-package haskell)

I get something like a No such file or directory: runghc error. However using @bkchr 's workaround:

(use-package direnv
  :config
  (direnv-mode))

(use-package flycheck
  :init
  (global-flycheck-mode)
  :config
  (setq flycheck-executable-find
        (lambda (cmd) (direnv-update-environment default-directory) (executable-find cmd))))

(use-package flycheck-haskell
  :hook (haskell-mode . flycheck-haskell-setup))

(use-package haskell)

flycheck-haskell works as expected.

[wbolster]

what happens when you also add direnv to find-file-hook? (without "patching" flycheck)?

[adelbertc]

Same issue (No such file or directory: runghc):

(use-package direnv
  :hook ((find-file flycheck-before-syntax-check) . direnv-update-environment)
  :config
  (direnv-mode))

(use-package flycheck
  :init
  (global-flycheck-mode))

(use-package flycheck-haskell
  :hook (haskell-mode . flycheck-haskell-setup))

(use-package haskell)

[peterhoeg]

In case any of you are using this with spacemacs, the trick to get everything to pay nice was to modify
dotspacemacs/user-env

from:

(spacemacs/load-spacemacs-env)
to

(require 'dotenv)
(dotenv-update-environment)

Then you end up with a proper environment.

[jethrokuan]

@peterhoeg what's dotenv and what does (dotenv-update-environment) look like?

[peterhoeg]

(direnv-update-environment), sorry...

[terlar]

If anyone have issues with .dir-locals.el file being executed before the direnv I used the following snippet to work around this issue:

(add-hook 'before-hack-local-variables-hook #'direnv-update-environment)

[hakanserce]

I was having the same issue with lsp-mode and python recently and what I did was to add a before advice as follows:

(use-package direnv
 :config
 (direnv-mode)
 (advice-add 'python-mode :before #'direnv-update-environment))

You can probably use prog-mode to make it work with all programming modes.

Now python-mode and lsp-mode use the right env variables.

[peterhoeg]

Shouldn't this work then?

(use-package direnv
  :init
  (add-hook 'prog-mode-hook #'direnv-update-environment)
  :config
  (direnv-mode))

[wbolster]

@hakanserce @peterhoeg would before-hack-local-variables-hook also work?

(add-hook 'before-hack-local-variables-hook #'direnv-update-environment)

if so, perhaps that should be default for direnv-mode?

[peterhoeg]

Ohhh, didn't know about that one. Looks much more appropriate!

[wbolster]

i've opened #54 to use before-hack-local-variables-hook from direnv-mode by default. please try it out and report back.

[what-the-functor]

Based on @wbolster's suggestion, this works for me:

(use-package direnv
  ;; Ensures that external dependencies are available before they are called.
  :hook (before-hack-local-variables . #'direnv-update-environment)
  :config
  (add-to-list 'direnv-non-file-modes 'vterm-mode)
  (direnv-mode 1))

[Ekleog]

FWIW, none of the above solutions (tried with both (advice-add 'python-mode :before #'direnv-update-environment) and (add-hook 'before-hack-local-variables-hook #'direnv-update-environment)) worked for me, and I just resorted to using this for loading lsp:

(add-hook 'prog-mode-hook
          (lambda () (progn (direnv-update-environment) (lsp))))

[kenranunderscore]

I encountered the problem for the first time today in a Haskell project that uses direnv and lorri. The HIE LSP suddenly started too early and used a wrong compiler version.

The fixes above did not work for me, but I found another (very strange) workaround. Instead of starting my emacs session (I use the daemon and only tested it with the TTY emacsclient) like this from the project directory:

emacsclient --some-flags src/MyFile.hs

I found that just starting emacs from within my home directory like this solves the problem:

emacsclient --some-flags my-projects/blafoo/src/MyFile.hs

I don't know why, but simply starting from another directory might help other people too. I'd love an explanation though :D

Edit: Another (probably simpler) workaround for me was just opening a non-Haskell file inside the project directory first.

[peterhoeg]

I'm guessing you have .envrc file somewhere between $HOME and blafoo that gives you a new environment.

[kenranunderscore]

No, I just started using direnv and the .envrc file in this particular project is in fact the single .envrc file on this machine (except for some clones of it inside the nix store at /nix/....).
But I'm fine with it now since the workarounds are so easy.

[wbolster]

with #54 merged i will close this issue...

let's see how it works out for the many people who commented here 😬 🤞

[deifactor]

I'm running into this issue where lsp-mode isn't playing nice, with the before-hack-local-variables-hook solution not working. Specifically, in my situation, I only have the analyzer installed inside the direnv, which means that lsp-mode tries to install it, which I don't want.

It looks like that runs so early that (current-buffer) is still nil, which means direnv can't figure out where the direnv is. I was able to solve this via (advice-add 'lsp :before #'direnv-update-environment). Unfortunately, I don't know enough about the guts of emacs to be able to say where the right place to set up direnv is here.

[psionic-k]

I found a use case and that this issue is still a problem. When providing the language server with per-project resources using nix, the shell entry needs to be done, as others have identified, when direnv can succeed at updating, but before lsp starts.

The exact case is rust-analyzer needing to know RUST_SRC_PATH to find the per-project rust-src component. I used to have a working advice on rustic's startup, but it appears to have regressed. Currently direnv-update-environment and then lsp-workspace-restart is my only working method at this time. Will continue looking for a reliable entry point to either advise or add a hook onto. Didn't try everything in this issue thread yet.

[psionic-k]

(use-package direnv ; direnv integration
  :after lsp
  :delight 'direnv-mode
  :config
  ;; Ensures that external dependencies are available before they are called.
  (add-hook 'prog-mode-hook #'direnv--maybe-update-environment)
  (setq direnv-always-show-summary nil)
  (direnv-mode 1))

I switched from a Rustic advice to a hook and got back rolling. I like the use of prog-mode-hook since it will fix a lot of issues with similar mechanism

[acowley]

The prog-mode-hook approach doesn't seem to work for me (tested with rustic and rust-analyzer), but the advice-add on lsp does. I'm on GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.24, cairo version 1.16.0) (built yesterday).

[peterhoeg]

I must admit that I've taken the easy way out - disabled emacs-direnv and then simply launch an emacs instance per project. Regular file editing goes via the daemon but everything else is a project specific instance.