Skip to content

CompuMaster.EPPlus4: address inherited concurrency defects and document unsupported multithreaded use #26

Closed

Description

@jochenwezel

Current status

The targeted fixes and documentation are merged in PR #27 and published in release 2026.10.07.1. The follow-up investigations are complete. The remaining limitations are documented below and accepted for the current scope; further remediation is deferred unless a concrete use case requires it. Historical reproduction results refer to the pre-fix commit, not the released version.

Problem and scope

CompuMaster.EPPlus4 inherits concurrency defects from EPPlus 4.5.3.3. Concurrent access to the same ExcelPackage can lose or misplace cell values without an exception, corrupt shared style collections, and fail during formula calculation. Working on different worksheets of one workbook does not isolate the shared state.

Until a supported concurrency contract has been implemented and tested, use each package and its workbook, worksheets, ranges, and streams in one thread, including calculation, saving, and disposal. If an application must share a package, it must externally serialize all access with one stable lock per package/workbook. Locks per worksheet or per cell are insufficient. Multi-step read/modify/write sequences require one lock around the entire sequence.

This issue records both the implementation work (part 1) and the documentation work (part 2). It does not claim that every operation on independent packages fails, or that later EPPlus versions have been tested here.

Evidence baseline

  • Assessment date: 2026-10-07.
  • CompuMaster source: c18469edbe5662ebb41666219bbef6c1f5338f31.
  • Runtime probes: freshly built EPPlus45-FixCalcsEdition.MultiTarget.csproj, target net8.0, Windows. The local build succeeded with five existing warnings and no errors. Other target frameworks have not yet been runtime-validated.
  • Archived upstream baseline: JanKallman/EPPlus at 55c5ba6169eaa6ae7867fefb36a84cc0b28be85a, dated 2020-03-08.
  • CellStore.cs, ExcelStyleCollection.cs, ExcelStyles.cs, ExcelRange.cs, FormulaParsing/ParsingScopes.cs, FormulaParsing/Excel/Functions/Math/Rand.cs, and FormulaParsing/Excel/Operators/OperatorsDict.cs are identical to that upstream baseline after namespace and newline/BOM normalization. CalculateExtentions.cs is not identical as a whole; the CM implementation still shares the workbook formula parser between worksheet calculations.

Historical reports and maintainer guidance

  1. Codeplex issue 15154: confirmed historical stream corruption, already fixed upstream before 4.5. The 2015-02-06 fix states that non-thread-safe streams were corrupted while copying to the internal MemoryStream and introduces a static CopyStream lock. The CM edition retains that lock. The legacy Issue15154 test uses parallel construction/saving, is ignored, and depends on external files; it is not a durable current regression test. Do not count this historical fixed defect as an unresolved current constructor bug.
  2. Upstream issue #330, opened 2018-10-10, still open: Parallel.For over a shared worksheet was reported to cause NullReferenceException. Maintainer swmal's response suspected thread safety and retained it as an enhancement, without isolating or confirming the exact cause. A 2019 report adds InvalidCastException (Decimal to ExcelColumn) in ExcelStyles.SetStyleAddress / CellStore.SetRangeValueSpecial.
  3. EPPlusSoftware issue #894: different worksheets of one workbook were reported to cause exceptions. Jan Källman's 2023-06-06 response explicitly recommends avoiding multiple threads on one workbook. This is later-version guidance, not a direct EPPlus 4.5 runtime reproduction.
  4. EPPlusSoftware issue #1611, 2024: tracks better multithreading support for ranges, styles, and calculation and references #894. Do not infer EPPlus 4.5 support or a backported fix from this later-version roadmap.

Reproduced defects at the historical pre-fix baseline

1. CellStore: concurrent writes can lose data or throw

The probe writes 300 rows in each of 64 columns on one worksheet. Each write obtains a fresh ws.Cells[row, col], so this does not depend on sharing a cached range object.

  • Initial parallel run with maximum degree 16: 5 exceptions, including NullReferenceException in CellStore.GetPosition / SetValueSpecial; 5,578 wrong/missing cells out of 19,200. This mismatch count includes writes skipped after a worker's exception.
  • Second parallel run with default scheduling: 92 wrong/missing cells out of 19,200, with zero exceptions. This demonstrates silent incorrect results.
  • Serial control: zero mismatches and zero exceptions.

CellStore.SetValue and other writes lock _columnIndex. AddColumn replaces that array when it grows. Threads can therefore lock different objects for the same store. GetValue also reads without that lock. The exact interleaving behind each observed mismatch has not been isolated, but the changing lock identity is a concrete synchronization defect.

2. Shared styles: different worksheet locks do not protect workbook collections

The probe initializes workbook styles before parallel work, then sets 500 distinct number formats on each of eight worksheets.

  • Both parallel runs failed, with 8 worker exceptions. A representative exception is InvalidOperationException: concurrent changes to a non-concurrent collection corrupted its state.
  • Stack: Dictionary.ContainsKey -> ExcelStyleCollection.FindByID -> ExcelXfs.GetIdNumberFormat -> ExcelStyles.SetStyleAddress.
  • Mismatch counts were 3,923/4,000 and 3,830/4,000; these include unperformed operations after workers failed and must not all be described as silent corruption.
  • Serial control: zero mismatches and zero exceptions.

ExcelStyles.PropertyChange locks the individual worksheet's _values, while style collections are shared by the workbook. ExcelStyleCollection.Add changes a List and Dictionary without a common collection lock; ContainsKey plus Add and ID assignment are not atomic. Reads, additions, IDs, and enumeration must be considered together.

3. Calculation: different worksheets share mutable workbook parser state

The probe prepares eight worksheets with 300 formulas each (A<row>*2+1) and calculates the first worksheet before parallel work to initialize the parser.

  • Both parallel runs failed with 4 worker exceptions. Observed exceptions include FormatException (operator missing expression) and NullReferenceException during formula-token storage.
  • Wrong/missing result counts were 1,171/2,400 and 831/2,400, including incomplete calculations after exceptions.
  • Serial control: zero mismatches and zero exceptions.

Worksheet calculation uses worksheet.Workbook.FormulaParser. InitNewCalc resets provider state, and ParsingScopes uses an unprotected Stack. Parser/provider/graph state is shared. Runtime failure is confirmed; the complete causal interleaving remains to be isolated.

4. Cached ExcelRange: indexing changes the shared object's address

ExcelRange indexers mutate address fields and return this. A coordinated two-thread probe uses var cells = ws.Cells; thread A selects A1, thread B selects B1, and thread A then assigns through its earlier reference. The assignment lands in B1, leaving A1 empty.

ws.Cells itself creates a new ExcelRange on each property access. Distinguish this shared-range hazard from defect 1, which reproduces with fresh ranges. Locking the indexer and setter separately cannot protect the entire select-then-use sequence. Changing historical indexer semantics may break compatibility and must not be done silently.

Additional code observations at the historical pre-fix baseline

  • ExcelWorkbook.Styles, the workbook parser, and other lazy initialization paths need a synchronization audit; unprotected check-then-create can publish competing instances. The runtime probes above deliberately preinitialize styles/parser where relevant.
  • ExcelAddressCache.Add, GetNewId, and Clear lock, but Get performs unprotected ContainsKey then index access, and Count is unprotected. Concurrent reads/clear can race. No separate runtime reproduction was performed.
  • RAND() changes a static Seed without synchronization and uses millisecond-based seeding. Lost updates and correlated/repeated random sequences are plausible across independent packages. No statistical/runtime validation was performed.
  • The static ExcelPackage._id++ used for image URIs is not atomic. The image lock is package-local. A shared counter race does not by itself prove image-part collisions across independent packages because part namespaces are package-local; investigate without overstating impact.
  • OperatorsDict.Instance and several operator/validation singletons use unsynchronized lazy initialization. Duplicate initialization alone is not proof of incorrect results; the publicly exposed mutable dictionary needs contract review.
  • CopyStream uses a static global lock: affected stream copies on independent packages are serialized. This is a code-established scalability concern, not a newly confirmed correctness failure. Seek occurs outside that lock, so it is not a comprehensive shared-stream safety guarantee.
  • Concurrent save/dispose, structural changes, and nominally read-only access have not been comprehensively validated and must remain outside any new guarantee until explicitly reviewed/tested.

Independent-package control and limits

100 independently owned packages were created, formatted, calculated, saved to individual memory buffers, and reopened in parallel. Both runs had zero exceptions and zero result mismatches. The serial control also passed. This supports independent package/stream ownership as the safer concurrency boundary, but does not establish comprehensive thread safety for all features, target frameworks, images, encryption, RAND(), or first-use initialization.

Part 1: completed implementation and regression coverage

  • Give CellStore a stable monitor for competing reads/writes, resizing, traversal steps, and disposal. Cell traversal is not a snapshot under arbitrary concurrent structural edits.
  • Protect workbook-wide style updates and collection lookups/ID creation; protect core lazy style initialization and capture style-collection membership for enumeration.
  • Serialize workbook/worksheet/range/formula calculations, including dependency-chain and token preparation, and coordinate parser-manager operations under the package monitor.
  • Protect core package/workbook/worksheet/style/parser first-use publication and coordinate calculation, saving, and disposal. Arbitrary workbook edits still require caller coordination.
  • Add 20 durable synchronization regression cases, including serial controls, result/round-trip assertions, controlled interleavings, stable monitor identity, cached-range compatibility, and independent-package overlap.
  • Validate net48 and net8.0 at runtime: the selected suite has 99 passed and 2 skipped per runtime. Build all four library targets and pass CI on Windows, Linux, and macOS.
  • Preserve public signatures and cached-range semantics; document caller responsibility for complete select/read/modify/write sequences.
  • Fix address-cache read/clear races, operator-dictionary publication, and the non-atomic image counter; classify other observations as unresolved investigation or documented limitations.
  • Document the targeted synchronization scope and remaining unsupported operations without claiming general thread safety.
  • Merge PR Fix internal EPPlus4 races and document threading limits #27 and publish release 2026.10.07.1 with successful post-merge and release workflows.

Part 2: completed README and NuGet documentation

  • Add matching Known Issues sections after Special CompuMaster Edition in all three README entry points, including the packaged NuGet README.
  • Use the warning: Epplus4 (including its fork CompuMaster.EPPlus4) is not thread-safe.
  • Recommend one owner thread per package, link Jan Källman's guidance, and explain why worksheet/cell locks are insufficient.
  • Distinguish the fixed internal races from remaining cached-range, compound-operation, callback-reentry, cross-workbook/direct-mutation, and global stream-copy limitations.
  • Verify the released package README content, all four EPPlus4 target assemblies, and dependency versions. All eleven NuGet uploads were accepted.

Completed investigations and accepted remaining limitations

  • Investigate the shared RAND() seed across independent packages and determine whether a compatibility-preserving fix is needed. The released-assembly probes confirmed repeated output sequences even during sequential use; see the investigation comment for evidence and limits.
  • Audit remaining lazy initialization and publicly mutable operator/validation paths; classify any additional demonstrated defects and add targeted regression coverage if needed.
  • Evaluate safe recursive calculation/parser use from custom functions or loggers separately. Internal monitor reentrancy is implemented, but recursive calculation/save/dispose from callbacks remains unsupported; waiting for another thread requiring the same package can deadlock.

Follow-up investigation on the published 2026.10.7.1 assemblies confirmed RAND quality defects, Encryption/Properties first-use failures, and grouped-formula/logger callback reentry failures on net48 and net8.0. Detailed evidence and a self-contained harness are in the latest investigation comment. The checked investigation items above indicate completed investigation, not remediation. The possible follow-up changes below are deferred and are not active TODO items.

  • Deferred: improve RAND/RANDBETWEEN generator quality.
  • Deferred: protect Encryption/Properties publication and retained settings/metadata during unsupported shared-package concurrency.
  • Deferred: resolve remaining singleton publication and clarify the mutable global operator/token contract.
  • Deferred: guard unsupported callback/lifecycle reentry; true nested-context support remains a separate design decision.

Cached range mutation and complete caller-side ownership/coordination remain intentional compatibility constraints. The inherited global CopyStream lock remains a documented throughput limitation. Neither implies an outstanding promise to provide general multithreading support in this release. Benchmarks were explicitly excluded from this task.

The targeted implementation, documentation, release work, and follow-up investigations are complete. At the maintainer's request, this issue is closed and removed from the active TODO list. The remaining findings are retained as documented limitations; further fixes are deferred until a concrete use case warrants them. Each package should continue to have one owner thread, and the RAND limitation also applies to sequential use. Historical evidence is retained for traceability.

Self-contained reproduction

Save the following program as Program.cs in a net8.0 console project referencing the freshly built CompuMaster.EPPlus4 assembly. Run normally for the parallel probes and with serial for the serial controls. The cached-range probe deliberately coordinates competing accesses and runs only in parallel mode. Counts vary with scheduling. This is an exploratory probe, not a replacement for durable regression tests.

Program.cs
using CompuMaster.Epplus4;
using CompuMaster.Epplus4.FormulaParsing;
using System.Collections.Concurrent;

class Program
{
    static bool Serialized;
    static void For(int from, int to, Action<int> action) { if(Serialized) { for(int i=from;i<to;i++) action(i); } else Parallel.For(from,to,action); }
    static void Main(string[] args)
    {
        Serialized = args.Contains("serial"); Console.WriteLine("Serialized="+Serialized);
        Console.WriteLine("Assembly: " + typeof(ExcelPackage).Assembly.FullName);
        if(!Serialized) CachedRange();
        Values();
        Styles();
        Calculation();
        IndependentPackages();
    }
    static void Failure(string label, ConcurrentQueue<Exception> errors)
    {
        Console.WriteLine(label + " exceptions=" + errors.Count);
        if (errors.TryPeek(out var e)) Console.WriteLine(e.ToString());
    }
    static void CachedRange()
    {
        using var p = new ExcelPackage();
        var ws = p.Workbook.Worksheets.Add("Sheet");
        var cells = ws.Cells;
        using var selectedA = new ManualResetEventSlim();
        using var selectedB = new ManualResetEventSlim();
        var t = new Thread(() => { var a = cells[1, 1]; selectedA.Set(); selectedB.Wait(); a.Value = "A"; });
        t.Start(); selectedA.Wait(); var b = cells[1, 2]; selectedB.Set(); t.Join();
        Console.WriteLine("Cached range: expected A1=A; actual A1=" + (ws.Cells[1,1].Value ?? "<null>") + "; B1=" + ws.Cells[1,2].Value);
    }
    static void Values()
    {
        using var p = new ExcelPackage();
        var ws = p.Workbook.Worksheets.Add("Sheet");
        var errors = new ConcurrentQueue<Exception>();
        For(1, 65, col => {
            try { for (int row=1; row<=300; row++) ws.Cells[row,col].Value = row * 1000 + col; }
            catch(Exception e) { errors.Enqueue(e); }
        });
        int wrong = 0;
        for(int col=1;col<=64;col++) for(int row=1;row<=300;row++) if(!Equals(ws.Cells[row,col].Value, row * 1000 + col)) wrong++;
        Console.WriteLine("Same sheet fresh ranges: mismatches=" + wrong + "/19200"); Failure("Values", errors);
    }
    static void Styles()
    {
        using var p = new ExcelPackage();
        var sheets = Enumerable.Range(0,8).Select(i => p.Workbook.Worksheets.Add("Sheet" + i)).ToArray();
        var unused = p.Workbook.Styles;
        var errors = new ConcurrentQueue<Exception>();
        For(0,8, i => {
            try { for(int row=1;row<=500;row++) sheets[i].Cells[row,1].Style.Numberformat.Format = "0.00\"s" + i + "r" + row + "\""; }
            catch(Exception e) { errors.Enqueue(e); }
        });
        int wrong=0;
        for(int i=0;i<8;i++) for(int row=1;row<=500;row++) {
            try { if(sheets[i].Cells[row,1].Style.Numberformat.Format != "0.00\"s" + i + "r" + row + "\"") wrong++; }
            catch(Exception e) { errors.Enqueue(e); wrong++; }
        }
        Console.WriteLine("Different sheets same workbook: style mismatches=" + wrong + "/4000"); Failure("Styles",errors);
    }
    static void Calculation()
    {
        using var p = new ExcelPackage();
        var sheets=Enumerable.Range(0,8).Select(i=>p.Workbook.Worksheets.Add("Sheet"+i)).ToArray();
        for(int i=0;i<8;i++) for(int row=1;row<=300;row++) { sheets[i].Cells[row,1].Value = i+row; sheets[i].Cells[row,2].Formula = "A"+row+"*2+1"; }
        // Initialize parser before parallel use to isolate shared parsing state from lazy initialization.
        sheets[0].Calculate();
        var errors=new ConcurrentQueue<Exception>();
        For(0,8,i=>{ try{sheets[i].Calculate();} catch(Exception e){errors.Enqueue(e);} });
        int wrong=0;
        for(int i=0;i<8;i++) for(int row=1;row<=300;row++) {
            var value=sheets[i].Cells[row,2].Value;
            if(value is not double d || d != (i+row)*2+1) wrong++;
        }
        Console.WriteLine("Different sheets calculation: mismatches=" + wrong + "/2400"); Failure("Calculation",errors);
    }
    static void IndependentPackages()
    {
        var errors=new ConcurrentQueue<Exception>(); int wrong=0;
        For(0,100,i=>{
            try {
                using var p=new ExcelPackage(); var ws=p.Workbook.Worksheets.Add("Sheet");
                ws.Cells[1,1].Value=i; ws.Cells[1,2].Formula="A1*2+1"; ws.Cells[1,1].Style.Numberformat.Format="0.00";
                ws.Calculate(); var bytes=p.GetAsByteArray();
                using var p2=new ExcelPackage(new MemoryStream(bytes));
                if(Convert.ToDouble(p2.Workbook.Worksheets[0].Cells[1,2].Value) != i*2+1) Interlocked.Increment(ref wrong);
            } catch(Exception e){errors.Enqueue(e);}
        });
        Console.WriteLine("Independent packages roundtrip: mismatches="+wrong+"/100"); Failure("Independent packages",errors);
    }
}

Activity

  1. jochenwezel commented on Oct 7, 2026

    @jochenwezel
    MemberAuthor

    Implementation is available for review in draft PR #27 (commit 001141f). It fixes the reproduced cell-store, workbook-style, and concurrent calculation races, adds a shared package monitor for calculation/parser-manager operations/save/dispose, protects core first-use publication and address-cache reads, safely publishes OperatorsDict, and makes the image counter atomic.

    All 20 new regression cases pass on net48 and net8.0. Together with the selected existing EPPlus4 engine/data tests, each runtime has 99 passed and 2 skipped tests. All four library targets build. The packaged README was checked against its source and all three Known Issues sections match. No benchmarks or release/package publication were performed.

    The remaining scope is explicit in the READMEs: cached range indexers still mutate and return the same object; arbitrary workbook reads/writes and compound operations require caller coordination; callbacks must not recursively calculate/parse/save/dispose the same package or wait for another thread needing its monitor; cross-workbook style copying and direct XML/collection mutation require external coordination. Cell enumeration is not a snapshot. The inherited global CopyStream lock and shared RAND seed remain unchanged; other lazy/operator/validation paths are not comprehensively validated.

    The monitor supports internal nested method calls. It does not implement the checklist's broader proposal to make recursive calculation from custom functions safe; that remains unsupported and documented. Passing these regression cases establishes the targeted fixes, not general thread safety. This issue remains open pending review and integration, with the additional observations retained as follow-up items.

  2. jochenwezel commented on Oct 7, 2026

    @jochenwezel
    MemberAuthor

    PR #27 has been merged into main and released as 2026.10.07.1.

    The post-merge build/test workflow succeeded on Windows, Linux, and macOS. The release workflow also succeeded, attached six binary archives, and uploaded all eleven NuGet packages (normalized package version 2026.10.7.1). The generated packages contain the updated EPPlus4 dependency, all four EPPlus4 target assemblies, and the updated NuGet README. NuGet accepted the uploads; public package-index propagation is still pending at verification time.

    This issue remains open for the documented remaining limitations and investigation items. The release fixes the targeted internal races but does not establish general thread safety; packages still require one owner thread or complete external coordination.

  3. jochenwezel commented on Oct 7, 2026

    @jochenwezel
    MemberAuthor

    Results of the three remaining investigations

    Checked the actual published CompuMaster.EPPlus4 assemblies, version 2026.10.7.1, on Windows with .NET Framework 4.8 and .NET 8.0.31. No production code was changed. These are correctness probes, not performance benchmarks. Counts depending on thread scheduling are examples, not guaranteed frequencies.

    1. RAND: confirmed poor randomness, including independent packages

    Rand.Execute updates a process-wide static Seed without synchronization, then constructs a new Random(DateTime.Now.Millisecond + Seed) for every result. Sequential execution cycles through seed offsets 0, 5, ..., 55, so the sequential algorithm has at most 1,055 possible integer seed inputs. The parallel update reads the property multiple times and is not an atomic read/modify/write; lost-update frequencies were not isolated separately.

    Published assembly probe net8.0 net48
    Distinct RAND results out of 12,000 sequential calls 66 71
    Distinct RAND results out of 12,000 parallel calls 48 61
    Distinct RAND results from 32 independently owned packages / 4,096 cells 92 108

    All values stayed in [0, 1), but the extensive exact repetition demonstrates a correctness/quality issue already in sequential use. RANDBETWEEN delegates to Rand and inherits this generator weakness. Making only the small seed increment atomic would not solve the low diversity. A suitable fix is a stateful generator with adequate initial seeding and explicitly synchronized ownership, compatible with all supported frameworks.

    Source: Rand.cs.

    2. Remaining lazy initialization and mutable global objects

    Two concrete first-use defects were reproduced through normal public access on fresh packages, without reflection or cached-field resets:

    • Encryption configuration can be discarded. Eight threads obtain package.Encryption simultaneously; multiple returned instances can exist. After all threads finish, assigning a password to a returned instance that was overwritten in the cache does not configure the retained instance. In 100 fresh-package trials this happened 14 times on net8.0 and once on net48; those packages saved as ordinary unencrypted ZIP/XLSX data. The probe uses a synthetic password. This involves the already unsupported shared-package concurrency case, not correctly owned independent packages.
    • Workbook.Properties can throw during a getter and lose metadata. First access constructs OfficeProperties, creates package parts, and loads separate XML documents. Eight competing getters over 100 fresh packages produced 524/800 getter exceptions on net8.0 and 393/800 on net48. A representative exception is InvalidOperationException: Part already exist, from ZipPackage.CreatePart -> OfficeProperties.GetXmlDocument -> OfficeProperties constructor -> ExcelWorkbook.Properties. Updating a discarded returned object's Title was absent from the retained object in 57/100 and 96/100 trials respectively.
    • Serial controls and 100 independently owned packages processed in parallel passed on both runtimes with zero lost settings or exceptions.

    Source: Encryption getter, Properties getter.

    Additional cold-publication replay probes reset only private cached fields between rounds via reflection, to simulate repeated first-use races. They produced competing instances for Operator.Percent, ExpressionConverter.Instance, ExcelDataValidationType.List, workbook Protection/View, and package Compatibility. This is not evidence that all of them corrupt workbook results: operators/converter are effectively stateless, validation descriptors have private setters, and several wrappers refer to shared XML/package state. The independent-package list-validation probe (100 rounds, eight packages per round) passed; a wrong exported validation result was not reproduced. Validation descriptors also compare Type in Equals but use object identity in GetHashCode, and two internal List comparisons use reference identity; these are additional contract concerns when duplicate descriptors are published.

    OperatorsDict.Instance is safely initialized after PR #27, but its public dictionary remains mutable. Temporarily replacing its '+' mapping with Minus changes a parser's 2+3 result from 5 to -1. The default ITokenSeparatorProvider.Tokens exposes one process-wide dictionary containing mutable Token objects. Appending to the returned '+' token affected a second provider and changed 2+3 to 2 without an exception. These deliberate mutation probes demonstrate cross-instance shared state; they do not show failures in normal use with unchanged defaults. Mutation during parallel parsing remains unsupported. TokenSeparatorProvider's static constructor itself is safely published.

    Suggested fixes: protect remaining package/workbook first-use publication under the existing package monitor, beginning with Encryption and Properties; safely publish immutable singleton descriptors; explicitly define the global dictionary/token customization boundary without silently breaking existing public mutation APIs. This was a scoped audit of the identified Package/Workbook, FormulaParsing, and DataValidation paths, not a guarantee for every workbook feature or lazy path (for example VBA/load/dispose interactions).

    3. Callback reentry: confirmed calculation errors and lifecycle hazards

    On both runtimes:

    • Control formula (CALLBACK()+10)*2, with CALLBACK returning 7, produced 34 and its dependent cell produced 35.
    • When CALLBACK invoked the same workbook's FormulaParserManager.Parse("2+3"), Calculate(otherWorksheet, "2+3"), or Calculate(otherWorksheet), the outer result and dependent cell became #VALUE!. Inner calculation returned 5 where applicable.
    • A custom logger that invoked those operations once from LogFunction changed (SUM(1,2)+10)*2 from 26 to #VALUE!.
    • The simpler CALLBACK()+10 and SUM(CALLBACK(),10) controls and nested variants happened to pass. Do not extrapolate these passing cases into safe general reentry.
    • Saving from CALLBACK captured the in-progress workbook: reopened A1 had no cached result. Calling Dispose from CALLBACK caused NullReferenceException; calling save twice during CALLBACK()+SUM(1,2)+CALLBACK() produced #VALUE! in the outer calculation.
    • A callback started a dedicated second thread, waited until that thread attempted parser access, then waited with a bounded timeout. The second thread could not complete until the callback returned and released the package monitor. An unbounded synchronous wait creates the documented deadlock cycle; the probe intentionally did not hang permanently.

    Cause: monitor reentrancy permits the nested entry but does not isolate the parser, builder graph, compiler expression field, dependency tokens, or provider state. For example ExpressionGraphBuilder.Build resets the active graph, and ExpressionCompiler.Compile replaces shared compiler state.

    The narrow next step is an explicit, consistent guard/diagnostic for unsupported reentry and lifecycle operations during active calculation. Truly supporting nested calculations requires isolated calculation contexts and broader regression coverage. Waiting on another thread for the same package must remain prohibited by the ownership contract.

    Implementation work remaining

    • Replace the weak RAND generator and cover RAND/RANDBETWEEN quality and independent-package behavior with meaningful regression checks.
    • Synchronize remaining Encryption/Properties publication and verify retained configuration, encrypted save/reopen, metadata persistence, and serial compatibility.
    • Safely publish the remaining relevant singleton descriptors and decide/document the global mutable operator/token customization contract; avoid overstating harmless duplicate allocation as result corruption.
    • Add a consistent calculation callback/lifecycle reentry guard with regression cases for grouped formulas and custom loggers; keep the broader isolated-context design separate.

    The original three investigations are now completed at this scope. The fixes above are not implemented or released yet, so this issue should remain open. Existing one-thread-per-package guidance still applies, and RAND is an additional concern even under that guidance.

    Reproduction harness

    For a fresh reproduction, create a console project targeting net48/net8.0 with LangVersion latest and a PackageReference to CompuMaster.EPPlus4 version 2026.10.7.1, and use the following Program.cs. Run modes rand, encryption, properties, controls, mutable, validation, lazy, and logger. Run callback "(CALLBACK()+10)*2" for the grouped reentry failure; callback uses the simpler passing case. lazy and validation explicitly use reflection only to replay cached singleton first use; encryption, properties, RAND, and callback/logger failure probes do not. The watchdog exits a stalled probe after 30 seconds.

    Program.cs
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Reflection;
    using System.Threading;
    using System.Threading.Tasks;
    using CompuMaster.Epplus4;
    using CompuMaster.Epplus4.DataValidation;
    using CompuMaster.Epplus4.FormulaParsing;
    using CompuMaster.Epplus4.FormulaParsing.Excel.Functions;
    using CompuMaster.Epplus4.FormulaParsing.ExpressionGraph;
    using CompuMaster.Epplus4.FormulaParsing.Excel.Operators;
    using CompuMaster.Epplus4.FormulaParsing.LexicalAnalysis;
    using CompuMaster.Epplus4.FormulaParsing.Logging;
    using RandFunction = CompuMaster.Epplus4.FormulaParsing.Excel.Functions.Math.Rand;
    
    class Program
    {
        static string outerFormula = "CALLBACK()+10";
        static void Main(string[] args)
        {
            using (var watchdog = new Timer(_ => { Console.WriteLine("PROBE TIMEOUT"); Environment.Exit(3); }, null, TimeSpan.FromSeconds(30), Timeout.InfiniteTimeSpan))
            {
            Console.WriteLine("Runtime=" + Environment.Version + "; assembly=" + typeof(ExcelPackage).Assembly.FullName);
            if (args.Length > 1) outerFormula = args[1];
            switch (args[0])
            {
                case "rand": Randomness(); break;
                case "lazy": LazyAudit(); break;
                case "mutable": MutableGlobals(); break;
                case "callback": CallbackAudit(); break;
                case "validation": ValidationAudit(); break;
                case "logger": LoggerAudit(); break;
                case "encryption": EncryptionAudit(); break;
                case "properties": PropertiesAudit(); break;
                case "controls": SerialControls(); break;
            }
            }
        }
    
        static void Randomness()
        {
            foreach (bool parallel in new[] { false, true })
            {
                var results = new double[12000];
                Action<int> sample = i => results[i] = Convert.ToDouble(new RandFunction().Execute(Enumerable.Empty<FunctionArgument>(), null).Result);
                if (parallel) Parallel.For(0, results.Length, new ParallelOptions { MaxDegreeOfParallelism = 8 }, sample);
                else for (int i = 0; i < results.Length; i++) sample(i);
                Console.WriteLine("RAND direct parallel=" + parallel + "; samples=" + results.Length + "; distinct=" + results.Distinct().Count() + "; outside-range=" + results.Count(v => v < 0 || v >= 1));
            }
            var values = new double[4096];
            Parallel.For(0, 32, new ParallelOptions { MaxDegreeOfParallelism = 8 }, worker =>
            {
                using (var package = new ExcelPackage())
                {
                    var sheet = package.Workbook.Worksheets.Add("Sheet");
                    for (int row = 1; row <= 128; row++) sheet.Cells[row, 1].Formula = "RAND()";
                    CalculationExtension.Calculate(sheet);
                    for (int row = 1; row <= 128; row++) values[worker * 128 + row - 1] = Convert.ToDouble(sheet.Cells[row, 1].Value);
                }
            });
            Console.WriteLine("RAND independent packages=32; samples=" + values.Length + "; distinct=" + values.Distinct().Count());
        }
    
        static void LazyAudit()
        {
            Cold("validation List", () => ExcelDataValidationType.List, typeof(ExcelDataValidationType).GetField("_list", BindingFlags.Static | BindingFlags.NonPublic), null);
            Cold("operator Percent", () => Operator.Percent, typeof(Operator).GetField("_percent", BindingFlags.Static | BindingFlags.NonPublic), null);
            Cold("expression converter", () => ExpressionConverter.Instance, typeof(ExpressionConverter).GetField("_instance", BindingFlags.Static | BindingFlags.NonPublic), null);
            using (var package = new ExcelPackage())
            {
                package.Workbook.Worksheets.Add("Sheet");
                Cold("package Encryption", () => package.Encryption, typeof(ExcelPackage).GetField("_encryption", BindingFlags.Instance | BindingFlags.NonPublic), package);
                Cold("workbook Protection", () => package.Workbook.Protection, typeof(ExcelWorkbook).GetField("_protection", BindingFlags.Instance | BindingFlags.NonPublic), package.Workbook);
                Cold("workbook View", () => package.Workbook.View, typeof(ExcelWorkbook).GetField("_view", BindingFlags.Instance | BindingFlags.NonPublic), package.Workbook);
                Cold("workbook Properties", () => package.Workbook.Properties, typeof(ExcelWorkbook).GetField("_properties", BindingFlags.Instance | BindingFlags.NonPublic), package.Workbook);
                Cold("package Compatibility", () => package.Compatibility, typeof(ExcelPackage).GetField("_compatibility", BindingFlags.Instance | BindingFlags.NonPublic), package);
            }
        }
    
        // Reset only the cached field between rounds to replay cold publication; no library code is patched.
        static void Cold(string label, Func<object> getter, FieldInfo field, object owner)
        {
            const int workers = 8, rounds = 100;
            var references = new object[workers];
            var errors = new Exception[workers];
            int duplicates = 0, failures = 0, discarded = 0;
            using (var start = new Barrier(workers + 1))
            using (var finish = new Barrier(workers + 1))
            {
                var tasks = Enumerable.Range(0, workers).Select(i => Task.Factory.StartNew(() =>
                {
                    for (int round = 0; round < rounds; round++)
                    {
                        if (!start.SignalAndWait(TimeSpan.FromSeconds(10))) throw new TimeoutException();
                        try { references[i] = getter(); } catch (Exception ex) { errors[i] = ex; }
                        if (!finish.SignalAndWait(TimeSpan.FromSeconds(10))) throw new TimeoutException();
                    }
                }, CancellationToken.None, TaskCreationOptions.LongRunning, TaskScheduler.Default)).ToArray();
                for (int round = 0; round < rounds; round++)
                {
                    Array.Clear(references, 0, references.Length);
                    Array.Clear(errors, 0, errors.Length);
                    field.SetValue(owner, null);
                    if (!start.SignalAndWait(TimeSpan.FromSeconds(10))) throw new TimeoutException();
                    if (!finish.SignalAndWait(TimeSpan.FromSeconds(10))) throw new TimeoutException();
                    failures += errors.Count(e => e != null);
                    var first = references.FirstOrDefault(r => r != null);
                    if (references.Any(r => r != null && !ReferenceEquals(r, first))) duplicates++;
                    if (owner is ExcelPackage)
                    {
                        var stale = references.OfType<ExcelEncryption>().FirstOrDefault(r => !ReferenceEquals(r, getter()));
                        if (stale != null)
                        {
                            stale.Password = "probe-password";
                            if (((ExcelPackage)owner).Encryption.Password != "probe-password") discarded++;
                        }
                    }
                }
                if (!Task.WaitAll(tasks, TimeSpan.FromSeconds(10))) throw new TimeoutException();
            }
            Console.WriteLine("Cold " + label + "; rounds=" + rounds + "; competing-instances=" + duplicates + "; exceptions=" + failures + "; lost-password-settings=" + discarded);
        }
    
        static void ValidationAudit()
        {
            const int workers = 8, rounds = 100;
            var field = typeof(ExcelDataValidationType).GetField("_list", BindingFlags.Static | BindingFlags.NonPublic);
            int mismatch = 0, failure = 0;
            for (int round = 0; round < rounds; round++)
            {
                var packages = Enumerable.Range(0, workers).Select(_ => new ExcelPackage()).ToArray();
                try
                {
                    var sheets = packages.Select(p => p.Workbook.Worksheets.Add("Sheet")).ToArray();
                    var types = new ExcelDataValidationType[workers];
                    using (var barrier = new Barrier(workers))
                    {
                        field.SetValue(null, null);
                        var tasks = Enumerable.Range(0, workers).Select(i => Task.Factory.StartNew(() =>
                        {
                            if (!barrier.SignalAndWait(TimeSpan.FromSeconds(10))) throw new TimeoutException();
                            var validation = sheets[i].DataValidations.AddListValidation("A1");
                            validation.Formula.Values.Add("A");
                            types[i] = validation.ValidationType;
                            if (!barrier.SignalAndWait(TimeSpan.FromSeconds(10))) throw new TimeoutException();
                            try { validation.Validate(); }
                            catch (InvalidOperationException) { Interlocked.Increment(ref failure); }
                        }, CancellationToken.None, TaskCreationOptions.LongRunning, TaskScheduler.Default)).ToArray();
                        if (!Task.WaitAll(tasks, TimeSpan.FromSeconds(15))) throw new TimeoutException();
                        mismatch += types.Count(t => !ReferenceEquals(t, ExcelDataValidationType.List));
                    }
                }
                finally { foreach (var package in packages) package.Dispose(); }
            }
            Console.WriteLine("Validation independent packages; cold-rounds=" + rounds + "; validation objects with different singleton=" + mismatch + "; validation exceptions=" + failure);
        }
    
        // Fresh packages and normal public first-use access; no reflection/reset.
        static void EncryptionAudit()
        {
            int lost = 0, plaintext = 0;
            for (int round = 0; round < 100; round++)
            {
                using (var package = new ExcelPackage())
                using (var start = new Barrier(8))
                {
                    package.Workbook.Worksheets.Add("Sheet");
                    var references = new ExcelEncryption[8];
                    var tasks = Enumerable.Range(0, 8).Select(i => Task.Factory.StartNew(() =>
                    {
                        if (!start.SignalAndWait(TimeSpan.FromSeconds(5))) throw new TimeoutException();
                        references[i] = package.Encryption;
                    }, CancellationToken.None, TaskCreationOptions.LongRunning, TaskScheduler.Default)).ToArray();
                    if (!Task.WaitAll(tasks, TimeSpan.FromSeconds(10))) throw new TimeoutException();
                    var stale = references.FirstOrDefault(r => !ReferenceEquals(r, package.Encryption));
                    if (stale == null) continue;
                    stale.Password = "probe-password";
                    if (package.Encryption.Password != "probe-password") lost++;
                    var bytes = package.GetAsByteArray();
                    if (bytes[0] == (byte)'P' && bytes[1] == (byte)'K') plaintext++;
                }
            }
            Console.WriteLine("Natural Encryption first use; packages=100; lost password=" + lost + "; plaintext saves=" + plaintext);
        }
    
        static void LoggerAudit()
        {
            foreach (string mode in new[] { "control", "parse", "formula", "worksheet" })
            {
                using (var package = new ExcelPackage())
                {
                    var main = package.Workbook.Worksheets.Add("Main");
                    var other = package.Workbook.Worksheets.Add("Other");
                    other.Cells[1, 1].Formula = "2+3";
                    main.Cells[1, 1].Formula = "(SUM(1,2)+10)*2";
                    package.Workbook.FormulaParserManager.AttachLogger(new CallbackLogger(() =>
                    {
                        switch (mode)
                        {
                            case "parse": package.Workbook.FormulaParserManager.Parse("2+3"); break;
                            case "formula": CalculationExtension.Calculate(other, "2+3"); break;
                            case "worksheet": CalculationExtension.Calculate(other); break;
                        }
                    }));
                    try
                    {
                        CalculationExtension.Calculate(main);
                        Console.WriteLine("Logger " + mode + "; expected=26; actual=" + main.Cells[1, 1].Value);
                    }
                    catch (Exception ex) { Console.WriteLine("Logger " + mode + "; exception=" + ex.GetType().Name); }
                }
            }
        }
    
        static void PropertiesAudit()
        {
            int failures = 0, lostTitle = 0;
            Exception firstError = null;
            for (int round = 0; round < 100; round++)
            {
                using (var package = new ExcelPackage())
                using (var start = new Barrier(8))
                {
                    package.Workbook.Worksheets.Add("Sheet");
                    var references = new OfficeProperties[8];
                    var errors = new Exception[8];
                    var tasks = Enumerable.Range(0, 8).Select(i => Task.Factory.StartNew(() =>
                    {
                        if (!start.SignalAndWait(TimeSpan.FromSeconds(5))) throw new TimeoutException();
                        try { references[i] = package.Workbook.Properties; }
                        catch (Exception ex) { errors[i] = ex; }
                    }, CancellationToken.None, TaskCreationOptions.LongRunning, TaskScheduler.Default)).ToArray();
                    if (!Task.WaitAll(tasks, TimeSpan.FromSeconds(10))) throw new TimeoutException();
                    failures += errors.Count(e => e != null);
                    firstError = firstError ?? errors.FirstOrDefault(e => e != null);
                    var retained = package.Workbook.Properties;
                    var stale = references.FirstOrDefault(r => r != null && !ReferenceEquals(r, retained));
                    if (stale != null)
                    {
                        stale.Title = "probe-title";
                        if (retained.Title != "probe-title") lostTitle++;
                    }
                }
            }
            Console.WriteLine("Natural Properties first use; packages=100; getter exceptions=" + failures + "; lost title=" + lostTitle);
            if (firstError != null) Console.WriteLine("First Properties exception=" + firstError);
        }
    
        static void SerialControls()
        {
            for (int i = 0; i < 100; i++)
            {
                using (var package = new ExcelPackage())
                {
                    package.Workbook.Worksheets.Add("Sheet");
                    var encryption = package.Encryption;
                    encryption.Password = "probe-password";
                    if (!ReferenceEquals(encryption, package.Encryption) || package.Encryption.Password != "probe-password") throw new Exception("Serial encryption control failed");
                    var properties = package.Workbook.Properties;
                    properties.Title = "probe-title";
                    if (!ReferenceEquals(properties, package.Workbook.Properties) || package.Workbook.Properties.Title != "probe-title") throw new Exception("Serial properties control failed");
                }
            }
            Console.WriteLine("Serial Encryption/Properties controls; packages=100; failures=0");
            Parallel.For(0, 100, new ParallelOptions { MaxDegreeOfParallelism = 8 }, i =>
            {
                using (var package = new ExcelPackage())
                {
                    package.Workbook.Worksheets.Add("Sheet");
                    package.Encryption.Password = "probe-password";
                    package.Workbook.Properties.Title = "probe-title";
                    if (package.Encryption.Password != "probe-password" || package.Workbook.Properties.Title != "probe-title") throw new Exception("Owned-package control failed");
                }
            });
            Console.WriteLine("Independent owned-package Encryption/Properties controls; packages=100; failures=0");
        }
    
        class CallbackLogger : IFormulaParserLogger
        {
            readonly Action callback;
            int called;
            public CallbackLogger(Action callback) { this.callback = callback; }
            public void LogFunction(string func) { if (Interlocked.Exchange(ref called, 1) == 0) callback(); }
            public void LogFunction(string func, long milliseconds) { }
            public void Log(ParsingContext context, Exception ex) { }
            public void Log(ParsingContext context, string message) { }
            public void Log(string message) { }
            public void LogCellCounted() { }
            public void Dispose() { }
        }
    
        static void MutableGlobals()
        {
            using (var package = new ExcelPackage())
            {
                var manager = package.Workbook.FormulaParserManager;
                var original = OperatorsDict.Instance["+"];
                try
                {
                    Console.WriteLine("Operators control 2+3=" + manager.Parse("2+3"));
                    OperatorsDict.Instance["+"] = Operator.Minus;
                    Console.WriteLine("Operators edited global 2+3=" + manager.Parse("2+3"));
                }
                finally { OperatorsDict.Instance["+"] = original; }
                var tokens = ((ITokenSeparatorProvider)new TokenSeparatorProvider()).Tokens;
                string originalValue = tokens["+"].Value;
                try
                {
                    tokens["+"].Append("CORRUPTED");
                    Console.WriteLine("Separator value shared across providers=" + ((ITokenSeparatorProvider)new TokenSeparatorProvider()).Tokens["+"].Value);
                    try { Console.WriteLine("Separator edited global 2+3=" + manager.Parse("2+3")); }
                    catch (Exception ex) { Console.WriteLine("Separator edited global exception=" + ex.GetType().Name); }
                }
                finally { typeof(Token).GetProperty("Value").SetValue(tokens["+"], originalValue, null); }
            }
        }
    
        static void CallbackAudit()
        {
            foreach (string mode in new[] { "control", "parse", "formula", "worksheet", "save", "dispose", "other-thread" })
            {
                using (var package = new ExcelPackage())
                {
                    var sheet = package.Workbook.Worksheets.Add("Main");
                    var other = package.Workbook.Worksheets.Add("Other");
                    other.Cells[1, 1].Formula = "2+3";
                    Task waiter = null;
                    bool waitBlocked = false;
                    byte[] saved = null;
                    var manager = package.Workbook.FormulaParserManager;
                    manager.LoadFunctionModule(new CallbackModule(() =>
                    {
                        switch (mode)
                        {
                            case "parse": Console.WriteLine("  inner parse=" + manager.Parse("2+3")); break;
                            case "formula": Console.WriteLine("  inner formula=" + CalculationExtension.Calculate(other, "2+3")); break;
                            case "worksheet": CalculationExtension.Calculate(other); break;
                            case "save": saved = package.GetAsByteArray(); break;
                            case "dispose": package.Dispose(); break;
                            case "other-thread":
                                using (var attempting = new ManualResetEventSlim())
                                {
                                waiter = Task.Factory.StartNew(() => { attempting.Set(); manager.Parse("2+3"); }, CancellationToken.None, TaskCreationOptions.LongRunning, TaskScheduler.Default);
                                if (!attempting.Wait(TimeSpan.FromSeconds(2))) throw new TimeoutException();
                                waitBlocked = !waiter.Wait(TimeSpan.FromMilliseconds(300));
                                }
                                break;
                        }
                    }));
                    sheet.Cells[1, 1].Formula = outerFormula;
                    sheet.Cells[1, 2].Formula = "A1+1";
                    try
                    {
                        CalculationExtension.Calculate(sheet);
                        Console.WriteLine("Callback " + mode + "; A1=" + sheet.Cells[1, 1].Value + "; B1=" + sheet.Cells[1, 2].Value + "; other-thread-blocked=" + waitBlocked);
                    }
                    catch (Exception ex) { Console.WriteLine("Callback " + mode + "; exception=" + ex.GetType().Name + ": " + ex.Message); }
                    if (waiter != null && !waiter.Wait(TimeSpan.FromSeconds(5))) throw new TimeoutException("Waiter did not recover after callback returned");
                    if (saved != null)
                    {
                        using (var reopened = new ExcelPackage(new System.IO.MemoryStream(saved)))
                            Console.WriteLine("  callback save reopened A1=" + (reopened.Workbook.Worksheets[0].Cells[1, 1].Value ?? "<null>"));
                    }
                }
            }
        }
    
        class CallbackModule : FunctionsModule
        {
            public CallbackModule(Action callback) { Functions.Add("callback", new CallbackFunction(callback)); }
        }
        class CallbackFunction : ExcelFunction
        {
            readonly Action callback;
            public CallbackFunction(Action callback) { this.callback = callback; }
            public override CompileResult Execute(IEnumerable<FunctionArgument> arguments, ParsingContext context)
            {
                callback();
                return new CompileResult(7d, DataType.Decimal);
            }
        }
    }
  4. jochenwezel commented on Oct 7, 2026

    @jochenwezel
    MemberAuthor

    Closing at the maintainer's request to remove this issue from the active TODO list. The targeted fixes, regression coverage, README/NuGet warnings, release 2026.10.07.1, and follow-up investigations are complete. Remaining findings are retained as documented limitations; additional remediation is deferred until a concrete use case requires it. This closure does not indicate general thread safety or resolution of every finding. Continue to use each ExcelPackage in one owner thread; the RAND/RANDBETWEEN quality limitation also applies to sequential use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions