Support $XDG_DATA_DIRS or equivalent for templates and lua filters

[pbsds]

Describe your proposed improvement and the problem it solves.

To avoid tailoring defaults.yaml to my specific environment using absolute paths, I'd rather configure my pandoc setup to search various paths for lua filters. defaults.yaml does not support using environment variables to my knowledge, overriding $XDG_DATA_HOME is a very heavy-handed (i run other things as well) and it only supports configuring one path making such a solution not scale.

For this reason I request support for $XDG_DATA_DIRS, which is the natural conclusion of #3582. Optionally an environment variable such as $PANDOC_DATA_DIRS that supports multiple entries could also do the trick.

It could be the solution to #6635. For me, it would allow me to use the nixpkgs package pandoc-lua-filters in a nix-shell environment, which simply dumps the lua filters in a read-only folder:

$ nix-build '<nixpkgs>' -A pandoc
/nix/store/x58dmn3qj75vnrlhvv347kwb2hb465k2-pandoc-2.14.0.3
$ nix-build '<nixpkgs>' -A pandoc-lua-filters
/nix/store/ac0fx53idm0ywln5zi0g6p11g2s4w7sd-pandoc-lua-filters-2020-11-30
$ ls `nix-build '<nixpkgs>' -A pandoc-lua-filters`/share/pandoc/filters
abstract-to-meta.lua    diagram-generator.lua   lilypond.lua                 pagebreak.lua           scrlttr2.lua        table-short-captions.lua
author-info-blocks.lua  include-code-files.lua  lua-debug-example.lua        pandoc-quotes.lua       section-refs.lua    track-changes.lua
bibexport.lua           include-files.lua       minted.lua                   revealjs-codeblock.lua  short-captions.lua  wordcount.lua
cito.lua                latex-hyphen.lua        multiple-bibliographies.lua  scholarly-metadata.lua  spellcheck.lua

Tangentially related due to the discussion: #6760

[jgm]

See https://pandoc.org/MANUAL.html#defaults-files
You can interpolate environment variables in paths in defaults files.

[pbsds]

Great to know as it solves my immediate problem, but what is your opinion on supporting $XDG_DATA_DIRS?

[jgm]

I don't know. I'd like to understand better what this would allow us to do that we can't do now, and why it's important.

[pbsds]

It would allow one to compose multiple directories with templates and filters using only environ variables. It would allow package managers like nix, and modules (commonly found in multi-user cluster environments) to work its magic.

It would also allow one to wrap pandoc like the following:

#!/bin/sh
# this file is in PATH
export XDG_DATA_DIRS=/some/lua/collection:"$XDG_DATA_DIRS"
exec /not/in/PATH/pandoc "$@"

[alerque]

The argument for enabling nix to be a usable way to scope project development environments is a good point here.

[pbsds]

My shell.nix currently looks like this:

{ pkgs ? import <nixpkgs> {} }:

with pkgs; mkShell {
  buildInputs = [
    graphviz
    (pkgs.writeShellScriptBin "pandoc" ''
      exec env XDG_DATA_HOME=${pandoc-lua-filters}/share ${pkgs.pandoc}/bin/pandoc "$@"
    '')
    pandoc-imagine
    nodePackages.mermaid-cli
    haskellPackages.pandoc-crossref
    pandoc-plantuml-filter
    ( texlive.combine {
      inherit (texlive)
        scheme-small
        titlesec
        fontaxes
        supertabular
        xtab
        atkinson;
    }
  )
  ];
}

It is possible to join the lua filter with other filters using symlinkJoin, but using this solution makes it so any user-defined filters in $HOME will not be found.

[jgm]

Adding link to specification:
https://specifications.freedesktop.org/basedir-spec/0.6/ar01s03.html