Skip to content

Dev to release - #51

Merged
Zacgoose merged 8 commits into
mainfrom
dev
Oct 12, 2026
Merged

Zacgoose merged 8 commits into
mainfrom
dev

Conversation

@Zacgoose

Copy link
Copy Markdown
Contributor

Summary

Each PowerShell worker now uses much less memory and CPU. App script modules are parsed once per process and shared across workers, instead of each worker importing or cloning its own copy. The rest of the PR removes allocation hot spots found while profiling a CIPP cache collection and test run.

Changes

Shared script modules (6427f99, 7a0d3f4)

  • New SharedScriptModules: each app .psm1 is parsed once (Parser.ParseFile), and every worker builds it with New-Module | Import-Module. This uses public APIs only, with no reflection.
  • A module falls back to a per-worker import through the InitialSessionState if a single module file can't carry it. That covers a binary root, NestedModules/RequiredModules, type or format data, ScriptsToProcess, or top-level dot-sourcing. RequiredAssemblies become SessionStateAssemblyEntry.
  • Removes the SSFE clone path, ExportedModuleState and the module-name repair.
  • Shared modules are parsed under a runspace, so they have a language mode and & (Get-Module X) { } works. Worker:ModuleInjections depends on this.
  • New-Module runs from the module directory, so ModuleBase, Path and $PSScriptRoot point at the module.

Behaviour differences for shared modules (also in docs/configuration.md):

  • Every function in the .psm1 is exported, including ones FunctionsToExport leaves out. ModuleBuilder lists one function per file, and apps call helpers across modules.
  • Aliases and variables are not exported.
  • Version reads 0.0.

PowerShell runtime hygiene (4604a80, 67fc180)

  • Ships powershell.config.json (LogLevel: Warning) to $PSHOME. Without it, PowerShell formats a syslog entry for every command start and stop, which was about 11% of all allocation in a cache collection.
  • Fixes a race in the module function index: it is now published only once fully built, so concurrent first lookups see every function.
  • Cleanup() now clears $Error.
  • Global JsonPreloads are parsed by the first worker, and the same object is handed to the others.

Smaller allocation fixes (004e25f, 4e3975c)

  • The module function index reads each .psm1 as a stream and only keeps lines that can start a function. Before, it loaded the whole file as one string. Parity tests check it against FunctionBlockRegex.
  • The HTTP diagnostic listener no longer reads every non-GET request body to build a slow-request preview. That read buffered and copied every outgoing request, including Table $batch bodies. [HTTP-SLOW] now logs only the URL and elapsed time.

Logging config (45c107d, acb1fa8)

  • New App:FileLogging:PowerShellLogLevel (App__FileLogging__PowerShellLogLevel) controls Write-Verbose and Write-Debug capture separately from the .NET log level. It can only narrow LogLevel. If unset, it follows LogLevel as before.
  • Docs cover how modules load and the PowerShell logging config.

Tests

  • New: WorkerModuleSharingTests (module scope, injections, ModuleBase, unlisted helpers, Get-Help, the NestedModules fallback), ModuleFunctionIndexTests (race + parity), HttpDiagnosticListenerTests, PowerShellLogLevelTests and PowerShellWorkerRunspaceHygieneTests.
  • Removed: ClonedWorkerModuleIdentityTests, since the clone path is gone.
  • New e2e coverage in the perf harness: a PerfNested fixture module and run-e2e.ps1 checks.

… workers

- every worker instantiates the app's script modules from one parse per process with New-Module
- modules a module file alone cannot carry (binary root, nested/required modules, type or format data,
  ScriptsToProcess, top-level dot-sourcing) are imported per worker through the InitialSessionState
- removes the SSFE clone path and its module-name repair
…unction index race

- ship powershell.config.json with LogLevel Warning so PowerShell stops formatting a syslog entry per command
- publish the module function index only once it is built, so concurrent first lookups see every function
…duleBase

- parse shared module script blocks under a runspace so `& $module {}` and ModuleInjections work
- run New-Module from the module directory so ModuleBase, Path and $PSScriptRoot point at it
- unit and e2e coverage for module scope, injections, ModuleBase, unlisted helpers, Get-Help and NestedModules
- reads each .psm1 with a stream and only materialises lines that can start a function, instead of the whole file
- same names as FunctionBlockRegex: parity tests over edge cases and random text
- the HTTP diagnostic listener no longer reads every non-GET request body to build a preview, which buffered and copied each outgoing request (Table $batch bodies serialised twice)
- in-flight entries keep only the request URI and start time
… log level

- App:FileLogging:PowerShellLogLevel decides whether workers emit Write-Verbose (Trace) and Write-Debug (Debug)
- it can only narrow LogLevel; unset, it follows LogLevel as before
… JSON preloads between workers

- Cleanup clears $Error, which the global variable sweep keeps as a built-in
- global JsonPreloads are parsed by the first worker and the same object is given to the rest
@Zacgoose
Zacgoose requested a review from JohnDuprey October 12, 2026 07:48
@Zacgoose
Zacgoose merged commit 39a89ef into main Oct 12, 2026
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants