Repository navigation
Conversation
fitsio-pure's compress module is a pure-Rust (no_std, wasm32) port of cfitsio's Rice encoder and tile quantization, checked against cfitsio in its CI. rice.rs and gzip.rs (461 lines) and the miniz_oxide dependency go. HCOMPRESS_1, the tile grid and the global f32 quantization step stay here and are passed through exactly, so every builder setting behaves as before. Checked against 1.0.0-pre6 on 17 codec x type x tiling cases, at 57x23 and at 26 MP: - Every file is the same size, and astropy decodes every case, quantized f32 included, to identical values. Metadata cards are unchanged. - Write time is within about 10% either way; u8 is faster. Lossless GZIP_1 floats now carry ZQUANTIZ = 'NONE', as cfitsio writes them, so the float_compresses_both_ways test checks for that.
Owner
|
Hey, I was actually looking at your crate and wondering about this myself. Thank you for doing the pull request! I will review and merge when I get a second. I would also really appreciate if you can go over the API surface and give some feedback. OrbitalCommons sounds interesting, thanks for the lead! |
Owner
|
I think my current issue is - I have to maintain a FITS writer inside refimage at all. I would love it if we could replace that entirely with fitsio-pure. Replacing just the encoders is not a huge win in my book. I am more than happy to contribute refimage to OrbitalCommons if it would be interesting to that audience. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hello nerdy friend.
I maintain fitsio-pure, a pure-Rust FITS library. Its new
compressmodule (0.21.0) writesRICE_1andGZIP_1tiles. It's a no_std, wasm-capable port of cfitsio's encoders, checked against cfitsio in CI, and your encoders here were its cross-check (credited in its docs). This PR swaps refimage'srice.rs/gzip.rsfor it so you don't have to maintain those codecs. It's a net −452 lines and dropsminiz_oxide.Nothing user-visible changes. HCOMPRESS, the tile grid and your global f32 quantization step all stay in refimage, and every builder setting behaves as before.
Tested against 1.0.0-pre6 on 17 codec/type/tiling cases, at 57×23 and at 26 MP:
ZQUANTIZ = 'NONE'on lossless GZIP floats, which is how cfitsio marks them.Happy to adjust, or to close it if you'd rather keep the encoders in-tree.
As an aside, OrbitalCommons is doing grants for astronomy maintainers that want to work in the Rust ecosystem, let me know if that is something of interest to you.