Skip to content

Decouple metric fetch errors from trial status in Orchestrator (#5287) - #5287

Open
saitcakmak wants to merge 1 commit into
facebook:mainfrom
saitcakmak:export-D119207750
Open

saitcakmak wants to merge 1 commit into
facebook:mainfrom
saitcakmak:export-D119207750

Conversation

@saitcakmak

@saitcakmak saitcakmak commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary:

Keep trial execution status authoritative to runner polling when metric fetching fails. Orchestrator now leaves completed trials completed, uses MetricAvailability to count incomplete optimization-config data in the existing failure-rate policy, and handles expanded multi-metric result keys without assuming they are registered experiment metrics.

This mirrors the behavior core Ax already has in the Orchestrator.

This implements the core design from the abandoned RFC D98741656 and fixes T287759832.

Reviewed By: esantorella

Differential Revision: D119207750

@meta-cla meta-cla Bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Sep 9, 2026
@meta-codesync

meta-codesync Bot commented Sep 9, 2026

Copy link
Copy Markdown

@saitcakmak has exported this pull request. If you are a Meta employee, you can view the originating Diff in D119207750.

@codecov-commenter

codecov-commenter commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.59%. Comparing base (1cb1f8a) to head (8a51f43).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5287      +/-   ##
==========================================
+ Coverage   96.57%   96.59%   +0.02%     
==========================================
  Files         621      621              
  Lines       70947    70958      +11     
==========================================
+ Hits        68518    68544      +26     
+ Misses       2429     2414      -15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@meta-codesync meta-codesync Bot changed the title Decouple metric fetch errors from trial status in Orchestrator Decouple metric fetch errors from trial status in Orchestrator (#5287) Sep 16, 2026
saitcakmak added a commit to saitcakmak/Ax that referenced this pull request Sep 16, 2026
…ook#5287)

Summary:

Keep trial execution status authoritative to runner polling when metric fetching fails. Orchestrator now leaves completed trials completed, uses MetricAvailability to count incomplete optimization-config data in the existing failure-rate policy, and handles expanded multi-metric result keys without assuming they are registered experiment metrics.

This mirrors the behavior core Ax already has in the Orchestrator.

This implements the core design from the abandoned RFC D98741656 and fixes T287759832.

Differential Revision: D119207750
…ook#5287)

Summary:

Keep trial execution status authoritative to runner polling when metric fetching fails. Orchestrator now leaves completed trials completed, uses MetricAvailability to count incomplete optimization-config data in the existing failure-rate policy, and handles expanded multi-metric result keys without assuming they are registered experiment metrics.

This mirrors the behavior core Ax already has in the Orchestrator.

This implements the core design from the abandoned RFC D98741656 and fixes T287759832.

Reviewed By: esantorella

Differential Revision: D119207750
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed Do not delete this pull request or issue due to inactivity. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants