Skip to content

Add source_resolver dispatch to download pipeline #1293

Closed

Description

@smoparth

Context

After the source field is activated (Phase 1), the download pipeline needs to dispatch to the new system when a package has source: config.

Parent epic: #1254 (Phase 3)

What needs to happen

Add a dispatch check in sources.download_source():

def download_source(*, ctx, req, version, download_url):
    pbi = ctx.package_build_info(req)

    if pbi.source_resolver is not None:
        # New path: source_resolver handles download
        path, download_kind = pbi.source_resolver.download(ctx, req, candidate)
        return path
    else:
        # Legacy path: unchanged
        source_path = overrides.find_and_invoke(
            req.name, "download_source", default_download_source, ...
        )
        return source_path

Open design question

The current download_source() signature takes download_url: str, but the new source_resolver.download() needs a Candidate object (which has url, version, name). The implementer needs to decide how to bridge this:

  • Option A: Thread Candidate through the pipeline from resolution to download (cleaner, larger change to callers)
  • Option B: Construct a minimal Candidate from download_url + version + req.name at the dispatch point (smaller, pragmatic)

Return value change

The new path returns (path, DownloadKind) tuple. The DownloadKind is needed by Phase 4 (prepare_source and build_sdist). The implementer should decide whether to:

  • Return the tuple from download_source() (breaking change to callers)
  • Store DownloadKind on context or pass it separately

Files

  • src/fromager/sources.py — download_source()
  • Callers in src/fromager/bootstrapper/ (may need signature changes)

Tests

  • Package with source: {provider: pypi-sdist} → source_resolver.download() called
  • Package without source: → legacy default_download_source() path used
  • DownloadKind is correctly propagated for Phase 4

Phase 3b follow-up

#1331 will replace the temporary (Path, DownloadKind) result with a typed result and implement download support for HookSDistResolver and HookPrebuiltResolver.

Hook compatibility is intentionally out of scope for this issue. #1331 depends on #1293 and must land before Phase 4 (#1205 and #1206).

Depends on

  • Phase 1: Activate source field + PBI property

Related

Activity

  1. added 2 commits that reference this issue on Sep 15, 2026
    1a38b8b
    ef28435
  2. added 2 commits that reference this issue on Oct 13, 2026
    f597f48
    8b94678
  3. LalatenduMohanty commented on Oct 13, 2026

    @LalatenduMohanty
    Member

    PR #1338 connects the new source: setting to the download step.

    When a package has source: configured, sources.download_source() now asks its resolver to download the artifact. It builds the resolver’s Candidate from the package name, resolved version, and URL.

    The function returns both the downloaded path and DownloadKind, so later steps can tell what kind of artifact they received. Packages without source: continue using the existing download hook. The PR also updates callers that need only the path and adds tests for both paths.

    This PR covers download dispatch. #1331 covers the typed result and hook downloads; #1205 and #1206 cover using the artifact kind during preparation and sdist building.

  4. LalatenduMohanty commented on Oct 13, 2026

    @LalatenduMohanty
    Member

    Closing as #1338 has merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions