add support for loading dynamic plugins for GHC >= 9.4

[mantkiew]

Supersedes #582.

This is a port of PR #582 on top of the current master. On Arch/CachyOS, all Haskell packages are dynamic-only. Users can conveniently just install pacman -S gitit but this version does not allow them loading plugins.

Tested on CachyOS, GHC 9.6.7, and claferwiki plugin compiled dynamically and installed into system.

Here's an explanation by Claude

How the gitit plugin-loading fix works

Background: build modes and the in-process loader

GHC can compile Haskell code in different build modes (GHC calls them "ways"). The two that matter here are the static mode, which produces .hi interface files and .o/.a objects, and the dynamic mode, which produces .dyn_hi and .dyn_o/.so. The files of each mode only work with that mode.

gitit loads a plugin by starting a GHC session inside its own process. It runs runGhc, sets the compiler flags, and then compiles and links the plugin module into the running program. The code that's loaded has to match how gitit itself was linked: a dynamically linked gitit can only load dynamic code.

On Arch/CachyOS everything is dynamic. The system Haskell packages ship only .dyn_hi and .so files, and your cabal.config (-dynamic, library-vanilla: False) builds dynamic-only too.

Why loading failed

gitit's original code does:

dflags <- getSessionDynFlags
setSessionDynFlags dflags

That gives the session GHC's default flags, which caused two problems:

1. The session compiles for the static mode. It looks for .hi files, which don't exist, so you got "Could not find module Prelude… There are files missing in the base-4.18.3.0 package".
2. Package environment files are ignored. The ghc program reads GHC_ENVIRONMENT and ~/.ghc/<arch>-<ver>/environments/default in parseDynamicFlags, which only it calls. setSessionDynFlags doesn't read them. So packages added with cabal install --lib are invisible, and you got "Could not find module Network.Gitit.Plugin.ClaferWiki".

Why the PR wasn't enough

PR #582 sets Opt_BuildDynamicToo (-dynamic-too). That flag makes GHC write dynamic output alongside the static output when it compiles, but the static mode stays primary. GHC still reads .hi files and still expects static dependencies, so on a dynamic-only system it fails the same way. It might help when a plugin is compiled from a source file, but here the plugin is an installed package, and your first run of the PR-only build still failed on Prelude.

The fix: three changes in loadPlugin, used on GHC ≥ 9.4

logger  <- getLogger
dflags' <- liftIO $ interpretPackageEnv logger dflags            -- (1)
let exposeGitit = ExposePackage "-package gitit" (PackageArg "gitit")
                    (ModRenaming True [])
    dflags'' = dflags' { packageFlags = exposeGitit : packageFlags dflags' }  -- (2)
setSessionDynFlags $
  if hostIsDynamic
     then dflags'' { targetWays_ = addWay WayDyn (targetWays_ dflags'') }     -- (3)
     else dflags''

1. interpretPackageEnv reads the environment file the way ghc does. It honours GHC_ENVIRONMENT, or finds the default environment, and adds its package databases and package-id entries to the session. That's what lets cabal install --lib all make plugins loadable.

2. Expose gitit. An environment file exposes only the packages it lists. Once (1) is in place, gitit itself becomes hidden, which caused your last error: "member of the hidden package gitit-0.16.0.2". Every plugin imports Network.Gitit.Interface, so gitit now always adds the equivalent of -package gitit to the session.

3. Dynamic mode when the host is dynamic.

   • hostIsDynamic asks the running runtime whether gitit itself is dynamically linked. If it is, the session switches its target mode to WayDyn, the equivalent of -dynamic. GHC then looks for .dyn_hi files and links .so files, which can be loaded into the running program.
   • GHCi does the same thing for the same reason.
   • When gitit is statically linked, as in a typical stack build, nothing changes and it keeps working as before.

On GHC versions older than 9.4 the code is unchanged, because several of these APIs moved or changed signature in earlier releases (to be exact, interpretPackageEnv changed since GHC 9.2 and hostIsDynamic is only available on GHC >= 9.2) .

Also, the original PR #582 did not completely solve the issue on fully dynamic systems like Arch/CachyOS, so that fix is not ported here for GHCs <= 9.2.

[jgm]

I just pushed a gitit.cabal change that means we only support ghc >= 9.6. That should simplify things.

[mantkiew]

This is confirmed to work on CachyOS binaries with GHC 9.6.7:

claferwiki plugin works in it (built as a dynamic library and cabal installed). During the build, gitit loads the plugin (Loading plugin 'Network.Gitit.Plugin.ClaferWiki'... then Finished loading plugins.), and a Clafer code block sent to /_preview comes back rendered by claferwiki.

Works on Ubuntu (static linking) as before. Ready to be merged.

[jgm]

I removed almost all the CPP from gitit.
It would be helpful if you could force-rebase this on top of current main.