Skip to content

convert: read one dataset of a plain HDF5 file (#25) - #65

Merged
Claptar merged 3 commits into
mainfrom
feat/convert-hdf5
Oct 20, 2026
Merged

Claptar merged 3 commits into
mainfrom
feat/convert-hdf5

Conversation

@Claptar

@Claptar Claptar commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Closes #25 with the text PR below it (read_hdf). Stacked on the text PR; it retargets to main once that merges.

What it reads

One 2-D numeric dataset of a plain HDF5 file as X, as anndata's read_hdf:

  • --key names it; without it nothing is written and the refusal lists the 2-D numeric datasets (shape, dtype) and the 1-D ones that could hold names;
  • names from the root row_names / col_names (naming the dataset's rows and columns), or --obs-names-key / --var-names-key; a misfit length warns and numbers that axis;
  • --transpose for genes as rows; dtype kept; CSR, or --dense.

Recognised as .hdf5 / .he5 / .hdf by extension only (a content sniff would take h5ad files). A plain .h5 that is not 10x needs --from hdf5, and an HDF5 file no format reads now says exactly that.

How

The slab writer moves from loom.py to conversion/dense.py (write_dense_matrix, slab_size), shared by loom (genes x cells) and both orientations here: slabs of whole chunks along the cell axis. loom.SLAB_BYTES stays loom's knob.

Checks

  • tests/test_hdf5.py: listing refusal, names from root and flags, numbering with a warning, keys that are no matrix, transposed CSR and dense, slabs across chunks, --from hdf5 on .h5 while a 10x .h5 stays 10x, make-unique.
  • Integration: anndata.read_hdf (and .T) gives the same X, dtype and names.
  • Perf: work linear in cells both ways; write calls independent of them.
  • Loom tests unchanged and green.

🤖 Generated with Claude Code

anndata's read_hdf, streamed: --key names a 2-D numeric dataset, which
becomes X (CSR, or dense with --dense), its dtype kept; --transpose reads
one holding genes as rows. Names come from read_hdf's root row_names and
col_names, or --obs-names-key / --var-names-key.

- Without --key nothing is written: the refusal lists the 2-D numeric
  datasets and the 1-D ones that could hold names.
- No content sniff, which would take h5ad files: .hdf5, .he5 and .hdf by
  extension, a plain .h5 with --from hdf5. An HDF5 file no format reads
  now says to pass --from hdf5.
- The slab writer moves from loom.py to conversion/dense.py, shared by
  loom's genes x cells matrices and both orientations here: slabs of
  whole chunks along the cell axis.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 20, 2026 18:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-20T18:20:46.732879Z 9d8dd16 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9d8dd16af8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/adata/conversion/hdf5.py Outdated
Comment thread src/adata/conversion/hdf5.py
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (py3.13)

1 631 tests  +25   1 631 ✅ +25   3m 50s ⏱️ +12s
    1 suites ± 0       0 💤 ± 0 
    1 files   ± 0       0 ❌ ± 0 

Results for commit 31457bf. ± Comparison against base commit d8358ef.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Test Results (py3.12)

1 631 tests  +25   1 631 ✅ +25   4m 25s ⏱️ +12s
    1 suites ± 0       0 💤 ± 0 
    1 files   ± 0       0 ❌ ± 0 

Results for commit 31457bf. ± Comparison against base commit d8358ef.

♻️ This comment has been updated with latest results.

Claptar and others added 2 commits October 20, 2026 19:30
Found in review: a complex64 or complex128 dataset was refused as not
numeric, though anndata stores complex arrays and read_hdf reads them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Claptar
Claptar changed the base branch from feat/convert-text-tables to main October 20, 2026 18:57
@Claptar
Claptar merged commit 38b821e into main Oct 20, 2026
6 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Convert: generic HDF5 matrix and delimited text files

2 participants