fix(extraction): retain errors for empty LLM responses - #2228
Open
Kuang-xianxin wants to merge 1 commit into
Open
fix(extraction): retain errors for empty LLM responses#2228Kuang-xianxin wants to merge 1 commit into
Kuang-xianxin wants to merge 1 commit into
Conversation
Return empty-response diagnostics before successful-block normalization in both sync and async extraction, preserving section indices and token accounting.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Keep empty LLM completions marked as extraction errors in both
LLMExtractionStrategy.extractandaextract.When a provider returns
content=Noneor""(for example withfinish_reason="length"or"content_filter"), the existing code creates an error block and then overwrites itserrorflag withFalsein the successful-block loop. Whitespace-only content instead disappears as an empty list. Downstream code filtering successful records byerrorcan therefore accept diagnostics as extracted data or miss the failure entirely.Return the empty-response diagnostic before successful-result normalization, retain the actual section index and finish reason, and treat whitespace-only content as empty. Usage accounting still runs before this return. A valid model response containing
[]continues to mean successful extraction with no matching records.Files changed
crawl4ai/extraction_strategy.py: the same narrow empty-content guard in sync and async extraction.tests/unit/test_llm_extraction_empty_response.py: parameterized regressions using real LiteLLMModelResponseobjects with mocked completion boundaries; no model API calls.Validation
tests/unitsuite: 121 passed on Python 3.13.14 after installing the optionalpypdfdependency needed by existing PDF tests.--target-version py310), compilation of both changed Python files, andgit diff --checkpass. The existing source file was not reformatted wholesale.Command:
LITELLM_LOCAL_MODEL_COST_MAP=True uv run --no-sync pytest tests/unit -q.Based on and targeting
develop. Found by source inspection; no existing issue is claimed as resolved. Older #1450 addressed alternative response fields rather than this error-flag overwrite.No live browser crawl or model-provider validation is claimed.