Repository navigation
Trustworthiness metric - #521
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Odd-sized datasets reject valid neighbor counts, and the new metric lacks a common embedding-metric interface.
3 open findings
What changed in this PR
Adds Trustworthiness scoring for evaluating dimensionality-reduction embeddings.
Changes:
- Implements and tests Trustworthiness scoring.
- Adds usage documentation and navigation.
- Adds benchmarking and changelog entries.
| File | Description |
|---|---|
src/CrossValidation/Metrics/Trustworthiness.php |
Implements the metric. |
tests/CrossValidation/Metrics/TrustworthinessTest.php |
Tests behavior and validation. |
benchmarks/CrossValidation/Metrics/TrustworthinessBench.php |
Benchmarks scoring. |
docs/cross-validation/metrics/trustworthiness.md |
Documents the metric. |
docs/cross-validation/metrics/api.md |
Documents embedding scoring. |
mkdocs.yml |
Adds documentation navigation. |
CHANGELOG.md |
Records the feature. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
apphp
requested changes
Oct 15, 2026
apphp
approved these changes
Oct 16, 2026
andrewdalpino
added a commit
that referenced
this pull request
Oct 20, 2026
* Ward clusterer (#520) * Intial commit * A little nicer * A little nicer * Remove AI slop * A little nicer * A little nicer * Update distance calculation in Ward.php Refactor distance calculation to use kernel method. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Initialize centroids and log training completion Initialize centroids, sizes, and losses when maxMerges is 0. Log training completion if logger is available. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Remove AI slop * Add max distance early stopping * Add Squared Euclidean kernel * Fix typo * Clean up a bit * Fix typo * Fix typo * Fix ordering * Periodically rebuild min heap * Appease stan * A little nicer * Fix coding style --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Update CHANGELOG * OPTICS clusterer (#519) * Bayesian Search (#509) * Cosine Scheduler (#491) * Ramp scheduler (#490) * Initial commit * Fix docs * A little nicer * Initial commit * Add candidate selection from combinations Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Fix calculation of spaceSize in BayesianSearch Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Change return type of base() to Estimator Update return type of base() method and clarify hyper-parameter retrieval. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Add Manhattan distance kernel for hyperparameter tuning Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * A little nicer * K-medoids (#493) * Initial commit * A little nicer * Clean up a bit * Nicer * A little nicer * Nicer * Assert kernel is symmetric * Clean up a bit * A little nicer * Nicer * Clean up a bit * Remove comment * Optimize PAM * A little nicer * Use set * Nicer * Stop if seed not in training set * Fix typo * Fix docs * Fix tests * Implement Iterative * Nicer * Fix test * ARFF Extractor (#498) * Intial commit * A little nicer * Even nicer * Check balanced buffer after stripping comments Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Validate integer values using filter_var Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Throw exception for unterminated quoted strings Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Modify categoricalPlaceholder type in ARFF documentation Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Convert to static members * Fix escaped apostrophe * Remove AI slop --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Shuffler extractor (#499) * Intial commit * Validate buffer size in Shuffler constructor Added validation for buffer size in constructor. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Update CHANGELOG * Ensure array keys are reindexed after uniqueness Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Remove defunct interface * Appease stan --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Power transformer (#505) * Cosine Scheduler (#491) * Ramp scheduler (#490) * Initial commit * Fix docs * A little nicer * Initial commit * Refactor yeoJohnson function for better performance Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * A little nicer * Optimize --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Word stemmer (#514) * Cosine Scheduler (#491) * Ramp scheduler (#490) * Initial commit * Fix docs * A little nicer * K-medoids (#493) * Initial commit * A little nicer * Clean up a bit * Nicer * A little nicer * Nicer * Assert kernel is symmetric * Clean up a bit * A little nicer * Nicer * Clean up a bit * Remove comment * Optimize PAM * A little nicer * Use set * Nicer * Stop if seed not in training set * Fix typo * Fix docs * Fix tests * Implement Iterative * Nicer * Fix test * ARFF Extractor (#498) * Intial commit * A little nicer * Even nicer * Check balanced buffer after stripping comments Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Validate integer values using filter_var Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Throw exception for unterminated quoted strings Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Modify categoricalPlaceholder type in ARFF documentation Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Convert to static members * Fix escaped apostrophe * Remove AI slop --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Shuffler extractor (#499) * Intial commit * Validate buffer size in Shuffler constructor Added validation for buffer size in constructor. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Update CHANGELOG * Intial commit * Save game * Save game * Clean up a bit * Rename language to stemmer * Preserve punctuation --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Consistency with Grid Search * Add setup method * Better logging * Add trial number to logs * Change order of arguments * Rename parameters * Better sampling * Make smoothing configurable * Optimize power transformer * Avoid stating the obvious * A little nicer * Initial commit * Better naming * Better benchmark * Optimize * Fix tests * Make transients null by default * A little nicer * Remove progress() * Use radius instead of coreDistance in range method Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Revert "Use radius instead of coreDistance in range method" This reverts commit 10168cb. * Fix * Fair comparison * Optimize * Remove transient properties from class definition * Fix OPTICS distance-label misalignment in range processing --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Trustworthiness metric (#521) * Intial commit * Remove AI slop * Nicer docs * Edit CHANGELOG * Fix neighbor count condition in Trustworthiness.php Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Rename test method for clarity Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Clean up AI slop * Save game * Clean up a bit * Clean up a bit * Quadratic to linear memory scaling * A little nicer --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * ROCAUC and Average Precision Metrics (#522) * Initial commit * Add average precision metric * Simplify condition for returning zero in AveragePrecision Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Clarify the docs * Align edge cases --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Update CHANGELOG * Add trustworthiness test * Add more trustworthiness tests * Update CHANGELOG * Softmax Ranks Features (#523) * Initial commit * Add test * Add interface to docs --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first metric for Embedders. Currently a standalone class - it does not implement the Metric interface or others. Delaying the API design until the future when we have more information at our disposal.