test: document problems with the current VT_I1 mapping in tlbparser.

[haoxuanjng-lang]

The current type-library parser maps VT_I1 to c_char. A signed scalar retval can therefore fail during automatic output allocation, and a generated signed-byte SAFEARRAY exposes unsigned tuple values or incorrect NumPy results. This PR records those problems and compares current and proposed interface definitions against the same native COM server.

The default tlbparser.COMTYPES mapping stays unchanged. schar_type is defined for later work, but is not selected by the parser. Mapping, warnings, feature flags and migration decisions remain in #969.

Changes:

• Add a small first-party C++ COM fixture for signed/unsigned byte scalars, explicit LPSTR, and plain character input/output pointers. Its IDL, implementation, factory and build registration are included in the existing CI server.
• Compare handwritten legacy and proposed COMMETHOD declarations, automatic/pointer/byref/array result helpers, literal return values and explicit TypeError/ArgumentError paths. Keep the freshly generated interface separate; assert it still matches the legacy declarations and exercise its actual calls.
• Retain the existing signed-byte SAFEARRAY smoke tests, add explicit mismatches against signed values, and record the current one-dimensional NumPy byte-string scalar and two-dimensional ValueError.
• Leave migration documentation out of this PR because no default behavior changes. No expected-failure decorators are used.

Observed validation on Windows x64, Python 3.13.5, NumPy 2.1.3 and pywin32 308:

• Focused native COM/SAFEARRAY/NumPy/VARIANT tests: 88 run, 79 passed, 9 skipped.
• Full unittest discovery: 494 run, 470 passed, 24 skipped.
• Focused tests without NumPy or pywin32: 63 run, 53 passed, 10 skipped.
• Ruff 0.6.9: complete CI format and lint commands pass in a clean validation worktree (151 files); git diff --check passes.
• The new current-generated-mapping assertion fails against the previous signed-byte parser default and passes with the unchanged upstream mapping. Each stage regenerated the native fixture wrappers; temporary per-user registry entries were removed afterward.

The unchanged native binary from the earlier fixture build was reused; its build was not rerun for this revision. These are local results, not a claim about new-head GitHub CI. The real-library investigation in #969 covers generated declarations and local records, not end-to-end SAPI, PortableDevice, CLR or event-source calls. This PR does not claim to fix the intermittent UIAutomation exception discussed elsewhere.

AI assistance: prepared and revised with OpenAI Codex; commands and outcomes above were observed locally.

Current GitHub CI for 379ef3e: 64 checks succeeded, one failed and 11 were cancelled. The failure is the existing TestLocalServer_win32com.test_get_name working-set assertion (Leaks 4096 bytes) on Python 3.14 x86 with third parties; all changed tests passed in that job. Root cause remains unresolved. Rerun was attempted but requires repository admin rights; awaiting maintainer rerun. Job: https://github.com/enthought/comtypes/actions/runs/39103259500/job/117603688961

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 99.24812% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.53%. Comparing base (91408c2) to head (379ef3e).

Files with missing lines	Patch %	Lines
...t/test_compatibility_tlbparser_VT_I1_assignment.py	99.19%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #962      +/-   ##
==========================================
+ Coverage   89.34%   89.53%   +0.18%     
==========================================
  Files         142      143       +1     
  Lines       13856    14122     +266     
==========================================
+ Hits        12380    12644     +264     
- Misses       1476     1478       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[junkmd]

Thank you for your continued contributions!

This is clearly an inconsistency, and it is normally not the correct behavior for software. However, I could not find any earlier bug reports about this specific behavior. This makes me wonder whether users have come to expect the current behavior over time, or whether no one has ever reached this code path.

So, let's move forward step by step, checking both the impact of this change and backward compatibility.

As before, let's start with the test structure.

[haoxuanjng-lang]

Added the reverse-direction probe and expanded the real native-call matrix. The separate component/interface/test module are now named ByteCompatibilityTest, and I merged current main (939341c), resolving the three native fixture conflicts while retaining your new string-SAFEARRAY-in-VARIANT component.

The output probe writes one character through a plain char*, with caller-owned storage. I used this shape because a plain character pointer is not a NUL-terminated string contract; an allocated char** prototype could not safely assume a terminator through the universal marshaler. The final tests avoid that unbounded read.

Observed with freshly generated wrappers against latest-main production versus the proposed mapping:

Call	Old mapping	Proposed mapping
ReadCharPointer(b"h")	104	ArgumentError
Same call with pointer, byref, or array of c_byte(104)	ArgumentError	104
Generated raw GetCharacterBytes(104, create_string_buffer(2)), then buffer .value	b"h"	ArgumentError
Same raw output call with numeric byte pointer/reference/array	ArgumentError	stored 104
High-level GetCharacterBytes(104)	TypeError allocating implicit STRING output	integer 104
High-level outputs for 128 / 255	same allocation TypeError	integers -128 / -1

This establishes the opposite direction's accepted-buffer change independently of SAFEARRAYs. Explicit LPSTR and unsigned scalar tables still pass both versions. I retained the two legacy input/output assertions as expected failures and added documentation for affected generated types, saved/handwritten interfaces, replacement arguments, signed outputs and explicit string handling.

Full local unittest: 474 run / 448 passed / 24 skipped / 2 expected compatibility failures. Focused suites: 70 / 59 / 9 / 2; without NumPy/pywin32: 51 / 41 / 8 / 2. Native build, full Ruff CI scope and strict Sphinx HTML build pass. Existing full docs doctest cannot get past its optional Excel type-library setup locally (WinError -2147319779); this is recorded as unvalidated, not passing. New-head upstream CI is pending.

The mapping itself is unchanged in this follow-up and the PR remains draft. These results and the documentation describe the compatibility cost; they do not resolve it or decide whether a more detailed mapping should be used.

[junkmd]

I opened #969 to record decisions about this change and provide a place for future support.

When referring to this mapping in code comments or documentation, please link to this issue.

[junkmd]

To understand real-world use cases of VT_I1, I first added a branch that emits a warning when parsing the libraries used in our tests.

You can review the implementation in a409eea.

The warnings alone are shown below.

path\to\comtypes\comtypes\tools\tlbparser.py:689: UserWarning: VT_I1 is used in '_IMSVidCtlEvents' method 'Error' parameter 'CancelDisplay', <libid='{B0EDF154-910A-11D2-B632-00C04F79498E}', lcid=0, major=1, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:680: UserWarning: VT_I1 is used in 'INumericSafearrayTest' method 'GetInt8Array' parameter 'result', <libid='{07D2AEE5-1DF8-4D2C-953A-554ADFD25F99}', lcid=0, major=1, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:680: UserWarning: VT_I1 is used in 'IByteCompatibilityTest' method 'EchoInt8' parameter 'value', <libid='{07D2AEE5-1DF8-4D2C-953A-554ADFD25F99}', lcid=0, major=1, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:680: UserWarning: VT_I1 is used in 'IByteCompatibilityTest' method 'EchoInt8' parameter 'result', <libid='{07D2AEE5-1DF8-4D2C-953A-554ADFD25F99}', lcid=0, major=1, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:680: UserWarning: VT_I1 is used in 'IByteCompatibilityTest' method 'ReadCharPointer' parameter 'value', <libid='{07D2AEE5-1DF8-4D2C-953A-554ADFD25F99}', lcid=0, major=1, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:680: UserWarning: VT_I1 is used in 'IByteCompatibilityTest' method 'GetCharacterBytes' parameter 'result', <libid='{07D2AEE5-1DF8-4D2C-953A-554ADFD25F99}', lcid=0, major=1, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:696: UserWarning: VT_I1 is used in '__MIDL___MIDL_itf_PortableDeviceApi_0001_0000_0001' field 'cVal', <libid='{1F001332-1A57-4934-BE31-AFFC99F4EE0A}', lcid=0, major=1, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:696: UserWarning: VT_I1 is used in '__MIDL_IOleAutomationTypes_0004' field 'cVal', <libid='{1F001332-1A57-4934-BE31-AFFC99F4EE0A}', lcid=0, major=1, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:696: UserWarning: VT_I1 is used in '__MIDL_IOleAutomationTypes_0004' field 'pcVal', <libid='{1F001332-1A57-4934-BE31-AFFC99F4EE0A}', lcid=0, major=1, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:676: UserWarning: VT_I1 is used in 'tagCAC' field 'pElems', <libid='{1F001332-1A57-4934-BE31-AFFC99F4EE0A}', lcid=0, major=1, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:696: UserWarning: VT_I1 is used in '__MIDL___MIDL_itf_PortableDeviceApi_0001_0000_0001' field 'pcVal', <libid='{1F001332-1A57-4934-BE31-AFFC99F4EE0A}', lcid=0, major=1, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:676: UserWarning: VT_I1 is used in 'SPPHRASERULE' field 'Confidence', <libid='{C866CA3A-32F7-11D2-9602-00C04F8EE628}', lcid=0, major=5, minor=4>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:676: UserWarning: VT_I1 is used in 'SPPHRASEPROPERTY' field 'Confidence', <libid='{C866CA3A-32F7-11D2-9602-00C04F8EE628}', lcid=0, major=5, minor=4>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:676: UserWarning: VT_I1 is used in 'SPPHRASEELEMENT' field 'RequiredConfidence', <libid='{C866CA3A-32F7-11D2-9602-00C04F8EE628}', lcid=0, major=5, minor=4>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:676: UserWarning: VT_I1 is used in 'SPPHRASEELEMENT' field 'ActualConfidence', <libid='{C866CA3A-32F7-11D2-9602-00C04F8EE628}', lcid=0, major=5, minor=4>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:706: UserWarning: VT_I1 is used in 'IConvertible' method 'ToSByte' parameter 'pRetVal', <libid='{BED7F4EA-1A96-11D2-8F08-00A0C9A6186D}', lcid=0, major=2, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:676: UserWarning: VT_I1 is used in 'SByte' field 'm_value', <libid='{BED7F4EA-1A96-11D2-8F08-00A0C9A6186D}', lcid=0, major=2, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.
path\to\comtypes\comtypes\tools\tlbparser.py:706: UserWarning: VT_I1 is used in 'IFormatterConverter' method 'ToSByte' parameter 'pRetVal', <libid='{BED7F4EA-1A96-11D2-8F08-00A0C9A6186D}', lcid=0, major=2, minor=0>. The generated types are planned to change: from c_char to c_byte for VT_I1, and from STRING to POINTER(c_byte) for pointers to VT_I1. See https://github.com/enthought/comtypes/issues/969 for details.

Please investigate the use cases of these interfaces and structures to determine whether the impact of changing the mapping is small, or whether switching to the new mapping would fix bugs.

[haoxuanjng-lang]

Pushed the requested test organization and explicit compatibility assertions:

• Renamed the module to test_compatibility_tlbparser_VT_I1_assignment.py and divided it into TestEchoInt8, TestEchoUint8, TestEchoLpStr, TestReadCharPointer and TestGetCharacterBytes, separating successful calls from TypeError and ArgumentError paths where those occur.
• Split the old interface factory into create_legacy_automatic_int8_output and create_legacy_buffered_int8_output. The latter owns the raw call, buffer and slicing; the explanation of the automatic STRING limitation is next to that override.
• Removed both expectedFailure decorators. Tests now assert the exact incompatible generated-interface calls raise ArgumentError, and compare literal numeric/bytes results explicitly. Passing tests do not mean those old calling conventions are preserved.
• Added links to Fix the tlbparser ctypes mapping for VT_I1. #969 in the mapping comment, test module and migration page. Investigated the four real Microsoft libraries mentioned above and will put the detailed findings and warning tradeoffs in Fix the tlbparser ctypes mapping for VT_I1. #969.

Observed Windows x64 / Python 3.13.5 / NumPy 2.1.3 / pywin32 308 results: focused 75 run, 66 passed, 9 skipped; full unittest discovery 481 run, 457 passed, 24 skipped. No expected failures remain. Old-mapping regeneration matches the legacy definitions; the proposed-definition assertion fails and the original native SAFEARRAY tests still fail (3 failures, 1 error). Ruff 0.6.9 passes the complete CI lint/format scope in an isolated worktree (151 files); the migration RST parses without warnings. Temporary HKCU registrations were cleaned up and the last focused run regenerated the proposed fixture wrappers.

The native binary is unchanged and was reused; I did not rerun its build, the optional-dependency-free environment or the Sphinx documentation build. Production behavior is unchanged from the previous PR revision. Draft status remains while the compatibility/transition plan is discussed; upstream CI for this new head is tracked separately.

AI assistance: OpenAI Codex.

[junkmd]

Thank you for the research and tests. The tests with the current test doubles are now much easier to read.

I have a suggestion. Could we leave the tlbparser.COMTYPES mapping as it is in this PR?
Adding the schar_type definition in tlbparser is fine.

The title of this PR could be something like "Add tests to show the problems with the current VT_I1 mapping in tlbparser."

Instead of trying to solve all the topics discussed in #969 in one PR, adding tests and features step by step would make this PR easier to merge.

Also, if you tested real calls using SAPI or PortableDevice, could we turn those tests into tests that run in GitHub Actions? Such tests would help answer a concern: we might fix the technical problem but still fail to solve a problem that users face in real use.

As discussed in #969 (comment), let's keep discussing warnings, feature flags, and documentation.

Thank you!

[haoxuanjng-lang]

Changed this PR to the incremental test scope you suggested. COMTYPES[VT_I1] is back to char_type; only the unused schar_type definition and the #969 comment remain in the parser diff. The title and description now describe the current mapping problems, and I removed the proposed migration page and its index entry.

The generated interface is now self.generated, separate from self.proposed_automatic, self.proposed_pointer, self.proposed_reference and self.proposed_array. It has its own declaration-parity and native-call tests, with no conditional exclusion of a generated entry from the proposed cases. Current generated calls demonstrate automatic-output TypeError, negative signed input rejection with an explicit output buffer, preserved bytes-pointer input and character-buffer output, and unchanged unsigned/LPSTR behavior. The handwritten proposed matrix remains available for comparing the different calling conventions.

The SAFEARRAY cases now record the current problems explicitly instead of requiring the unimplemented mapping change: unsigned tuple values differ from the signed fixture values; 1-D ndarray output is a byte-string scalar; 2-D ndarray conversion raises ValueError. No expectedFailure decorators are used. The current-generated-mapping assertion first failed with the previous signed-byte default, then passed after restoring the upstream mapping and regenerating wrappers.

Observed local results: focused 88 run / 79 passed / 9 skipped; full 494 / 470 / 24; without NumPy/pywin32 63 / 53 / 10. Full Ruff 0.6.9 CI lint/format scope passes in a clean validation worktree (151 files), and git diff --check passes. All temporary per-user registrations were cleaned up. I reused the unchanged native binary rather than rebuilding it; new-head CI is tracked separately.

For the real-library question: my earlier SAPI/PortableDevice investigation regenerated their actual type libraries and inspected declarations, field values and local counted buffers; it did not invoke an end-to-end speech-recognition/device COM path. I therefore have no such integration call to promote into an Actions test in this revision. The first-party fixture tests do make real out-of-process COM calls, and run in the existing Actions test job. A real-library test should additionally verify an affected user operation and its native allocation/lifetime contract before we claim that coverage.

I will keep the warning/feature-flag discussion in #969. AI assistance: OpenAI Codex.

[haoxuanjng-lang]

CI follow-up for 379ef3e538674530dc9403c0908765e77f4c6f1d: the Python 3.14 x86 / with third parties job failed in the existing test_comserver.TestLocalServer_win32com.test_get_name. Its _find_memleak(lambda: obj.name) assertion reported Leaks 4096 bytes. The job ran 540 tests with one failure and 33 skips; every test in test_compatibility_tlbparser_VT_I1_assignment and all four int8 code-generation cases passed in its log. The matrix then cancelled 11 other jobs. The formatter and 64 checks succeeded, so the overall matrix is still not green.

test_comserver.py and find_memleak.py are unchanged in this PR. The helper measures process working-set growth; one observation does not establish whether this is transient measurement variation or a reproducible leak. My local Python 3.13 x64 run does not reproduce that exact CI environment, so I am not claiming the failure is resolved or proven unrelated.

I attempted gh run rerun 39103259500 --failed, but GitHub returned Must have admin rights to Repository. Could a maintainer rerun the failed/cancelled jobs when convenient? I have kept the failure visible in the PR description and have not changed the existing memory test to suppress it.

AI assistance: OpenAI Codex.

[junkmd]

Could a maintainer rerun the failed/cancelled jobs when convenient?

I did this. When we have active PRs, I try to check them on a best effort basis, and if the CI fails, I click "rerun failed jobs".

The helper measures process working-set growth; one observation does not establish whether this is transient measurement variation or a reproducible leak.

I also don't know the cause yet. I cannot reproduce the failure in my local environment either.
This test fails unpredictably on CI, but my current workaround is to keep clicking the "rerun failed jobs" button manually until it eventually succeeds.

Although this is separate from the VT_I1 mapping issue, it would be a huge contribution to the quality assurance of this package if you could open an issue to investigate this matter and share the results of testing GHA on your fork.

[junkmd]

I think this test double and the test content are enough to show the current problems with tlbparser for now.
So, I will approve this PR now. Thank you very much!
If you are still interested in this project, please try the next steps below.

As you can see in my PoC commit for warnings, I think we also need to test fields in structs and unions.
To do this, we can add methods and structs to source/CppTestSrv/CoByteCompatibilityTest.*, and then add tests to test_compatibility_tlbparser_VT_I1_assignment.py.
The test double is for development and we do not publish it as an API for a specific application, so it is OK to change the interface.

Also, like SAFEARRAY, I think we need to study and test VT_I1 marshalling in VARIANT.
If there are problems like the tlbparser mapping this time and we need to fix production code to resolve the inconsistency, please open an issue.
If you only add regression tests, you do not need an issue—a PR is enough.