Repository navigation
Version 1.0: compiled kernels, TopoToolbox parity, and a GDAL installer - #12
Merged
Merged
Conversation
Repairs roughly 70 defects, several of which prevented the package from importing at all on current NumPy and SciPy, and replaces the recursive basin traversals -- which needed a million-frame recursion limit and still overflowed on real DEMs -- with iterative kernels. The public interface is unchanged: every class, method and keyword still exists and still means what it did. A handful of results differ because they were wrong; those are listed in CHANGELOG.md under "Results that change". Depression filling now uses the Priority-Flood algorithms of Barnes, Lehman and Mulla (2014), compiled to C++ and roughly 50x faster, with an equivalent NumPy implementation in TopoAnalysis.fastops that produces identical numbers when the extension cannot be built. The compiled kernels release the GIL. Adds TopoAnalysis.topotoolbox, which reproduces TopoToolbox 2 (MATLAB) function by function, and docs/topotoolbox_parity.md, which tabulates the differing conventions and says where exact agreement is not achievable. Adds an installer for GDAL, the one dependency pip cannot supply. There is no GDAL wheel on PyPI at any version for any platform -- only an sdist that compiles the bindings against a C library which must already exist -- so "install GDAL" is two jobs that have to agree. install.py and the topoanalysis-gdal command supply both halves, choosing between an existing C library, conda, a prebuilt Windows wheel, the system package manager, or a from-source build of PROJ and GDAL into the virtualenv that needs no administrator rights. Two failure modes are detected explicitly: bindings built without numpy, where osgeo imports cleanly and ReadAsArray fails at run time, and a stale PROJ_LIB inherited from another installation. Tests: 350 passing with the C++ kernels, 347 with the NumPy kernels. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Windows route is the only GDAL strategy with nothing behind it: there is no gdal-config and no source build on Windows, so a prebuilt wheel that bundles the C library is the whole plan. It had never been exercised anywhere. The new job installs GDAL through it and asserts that ReadAsArray actually works rather than that osgeo merely imports. The from-source job stays off the default path -- it is about forty minutes -- but a pull request labelled `test-source-build` now runs it, which is what a change to the build itself needs. Waiting for the weekly schedule means finding out on Monday. `labeled` joins the pull_request event types so applying the label is enough to start it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CHANGELOG.mdunder "Results that change".TopoAnalysis.topotoolbox, reproducing TopoToolbox 2 (MATLAB) function by function, plus a parity table saying where exact agreement is and is not achievable.Why GDAL needs its own installer
There is no GDAL wheel on PyPI at any version for any platform: every release is an sdist whose
setup.pyshells out togdal-configand compiles the bindings against a C library that must already exist.install.pyandtopoanalysis-gdalsupply both halves, choosing between an existing C library, conda, a prebuilt Windows wheel, the system package manager, or a from-source build of PROJ and GDAL into the virtualenv that needs no administrator rights.Two failure modes are detected explicitly:
osgeo.gdal_arrayis silently skipped —osgeoimports cleanly andReadAsArray()fails at run time. Ubuntu 22.04 (GDAL 3.4) and 24.04 (3.8) are both affected.PROJ_LIB/GDAL_DATA. Conda activation exports both, and they outrank the paths compiled into every other GDAL on the machine; a PROJ 9 library handed PROJ 8'sproj.dbrefuses every lookup.Test plan
Verified on macOS (Python 3.10):
python3 install.py→ exit 0, GeoTIFF round-trippython3 install.py --gdal-strategy source→ PROJ + GDAL built from source into the venv, exit 0, 159 driverspython3 install.py --no-gdal→ exit 0, library works without GDALNot yet exercised anywhere — this is what CI is for: Linux (the apt route,
lib64, loader paths), Windows (the prebuilt-wheel strategy), the conda and system strategies, and the Dockerfile. The workflow added here coversinstall.pyon all three platforms, an Ubuntu job that reproduces thegdal_arraytrap, and a scheduled from-source build.🤖 Generated with Claude Code