KNOX-3478 - Add support to forward auth tokens in KNOX-AUTH-SERVICE

[moresandeep]

KNOX-3478 - Add support to forward auth tokens in KNOX-AUTH-SERVICE

What changes were proposed in this pull request?

Downstream services increasingly need the caller's JWT bearer token itself, not just the
resolved username — e.g. to re-validate it, to extract scopes, or to call a third service on
the user's behalf. There is currently no way to get it: the token is consumed and discarded
by JWTFederationFilter, and nothing carries it forward.

This PR fixes this by setting the new preauth.auth.header.auth.token.name service parameter on KNOX-AUTH-SERVICE and making auth/api/v1/pre and auth/api/v1/extauthz return the caller's token in the named response header, which the calling proxy then copies onto the downstream request.

The header name that enables this feature is preauth.auth.header.auth.token.name . By default this feature is OFF i.e. auth tokens are no propagated downstream by default.

How was this patch tested?

This patch was tested locally.

Topology snippet

<service>
        <role>KNOX-AUTH-SERVICE</role>
        <param>
            <name>preauth.auth.header.actor.id.name</name>
            <value>x-knox-actor-username</value>
        </param>
        <param>
            <name>preauth.auth.header.actor.groups.prefix</name>
            <value>x-knox-actor-groups</value>
        </param>
        <param>
            <name>preauth.group.filter.pattern</name>
            <value>[^\s]+</value>
        </param>

        <!-- KNOX-3478: return the caller's own JWT in this header -->
        <param>
            <name>preauth.auth.header.auth.token.name</name>
            <value>x-knox-auth-token</value>
        </param>
        <!-- optional; 6144 is the default. A longer token is omitted with a WARN. -->
        <param>
            <name>preauth.auth.header.auth.token.size.limit</name>
            <value>6144</value>
        </param>
    </service>

curl request output

curl -isk -H "Authorization: Bearer $TOKEN" \
  "$GW/gateway/authtoken/auth/api/v1/pre"
HTTP/1.1 200 OK
Date: Thu, 24 Sep 2026 00:27:29 GMT
x-knox-actor-username: guest
x-knox-auth-token: eyJqa3UiOiJodHRwczovL2xvY2FsaG9zdDo4NDQzL2dhdGV3YXkvYXV0aHRva2VuLW1pbnQva25veHRva2VuL2FwaS92MS9qd2tzLmpzb24iLCJraWQiOiJkVWZkbDRIUzFUNkZiNmlnYWs5MWpxREFmQ191YU0ySnN2bEtOeE04QmxzIiwidHlwIjoiSldUIiwiYWxnIjoiUlMyNTYifQ.eyJzdWIiOiJndWVzdCIsImprdSI6Imh0dHBzOi8vbG9jYWxob3N0Ojg0NDMvZ2F0ZXdheS9hdXRodG9rZW4tbWludC9rbm94dG9rZW4vYXBpL3YxL2p3a3MuanNvbiIsImtpZCI6ImRVZmRsNEhTMVQ2RmI2aWdhazkxanFEQWZDX3VhTTJKc3ZsS054TThCbHMiLCJpc3MiOiJLTk9YU1NPIiwiZXhwIjoxNzkwMjEzMTgxLCJtYW5hZ2VkLnRva2VuIjoiZmFsc2UiLCJpYXQiOjE3OTAyMDk1ODEsImtub3guaWQiOiJiNDUwOWJhOS03OGQ2LTQzYjEtODliYS1iMmNjNGRmMjlhM2IifQ.xP7KawCESwDICd2m6W_lzstB0v6bYy3C0NfmX6wbt8q3tX6YJU_qJ5IVI9RtwrnuhmotNe0HVMggMVF_uoQGx5k_hkYtU7cSHQfkgfOf9RmPM0bCYQmPdYC8EZhPahLAavgFJ4TrDygHA6n6S2xdfr86q06YG73J02CF0M9j7DYCOh21JwM28NghW1s2cvgoD9ybdHLHL9t8bl_OeGnG1Tc9e7DZ4nsRWXzdcuefdwUmkHzTebcHKgRlmiCl6FaUy1KrMl6eQJy6errCjhLIUhDWct7uue_h7t1tcj2G9QDHlUKvnBG9MzFjXuiJPjRnn97HPPp_Y8ju-hPQreaN6Q
Cache-Control: no-store
x-knox-actor-groups-1: forwarded-token-group
Content-Length: 0

Integration Tests

Added .github/workflows/tests/test_knoxauth_token_forwarding.py (11 tests) plus three topology fixtures, all picked up by the existing default tests service with no tests.yml or compose change — these topologies coexist with the base gateway config, so they don't need a dedicated step.

Topology	Scenario
knoxauthtoken.xml	parameter set ⇒ token forwarded
knoxauthtokenlimit.xml	size.limit=16, below any real JWT ⇒ omitted, still 200
knoxauthtokencollide.xml	token name == actor id name ⇒ token dropped, identity kept
knoxtoken.xml, knoxldap.xml	unchanged — negative controls proving the feature stays opt-in

All three new topologies configure group.principal.mapping = guest=forwarded-token-group. That is deliberate, not decoration: it makes groupsMapped true so every request goes through the identity-assertion re-wrap described above. That branch is the one a unit test can most easily pass while the real code path drops the token, so test_forwarded_token_survives_the_identity_assertion_rewrap asserts the mapped group and the token in the same response — if the mapping ever stops running, the test fails loudly instead of passing vacuously.

Coverage: byte-identical JWT with no Bearer prefix; sub naming the original caller alongside X-Knox-Actor-ID; Cache-Control: no-store present when a token is emitted and absent when it isn't; no header when the parameter is unset; none for a Basic-auth caller; oversized token omitted at 200; collision preserving the identity; 401 with no header for unauthenticated and for a tampered token.

[github-actions]

Test Results

  4 files    4 suites   43s ⏱️
127 tests 127 ✅ 0 💤 0 ❌
145 runs  145 ✅ 0 💤 0 ❌

Results for commit 6b15c3c.

♻️ This comment has been updated with latest results.

[smolnar82]

HadoopAuthPostFilter never captures the caller JWT; this feature silently dead for topologies using the HadoopAuth filter.

gateway-provider-security-hadoopauth/.../HadoopAuthPostFilter.java:91

doFilter() still calls the public single-arg createSubjectFromToken(String), which resolves to createSubjectFromToken(JWT, null). It was never updated to capture the caller's JWT as an AuthTokenCredential the way JWTFederationFilter.doFilter() now does.

Impact: On any topology using the HadoopAuth federation provider with support.jwt=true, no AuthTokenCredential is ever added to the Subject, so SubjectUtils.getAuthToken() returns null and the X-Knox-Auth-Token header is never emitted, even though the same caller gets it on a JWTProvider/SSOCookieProvider topology. The PR docs misattribute this to an inherent "innermost Subject.doAs wins" quirk, when it's actually a one-line oversight at this call site.

I approve this PR now and file a follow-up JIRA to fill this gap.

[lmccay]

I expected a bit more generic of an approach here. This is a rather large change for adding a single header to the AUTH-SERVICE API. When I saw the description I expected something more generic that would inform the API what things to pull from the security context. Since Principal name isn't likely enough since that is also the actual value of the Principal and they are more class type based. I would have expected an extension that you can interrogate with a getType() for a value that would match the param in the topology to add to a header. It would require a new convention of Principals and Credentials where this extension is used and maybe not be fully dynamic but maybe there is something more we can do to make it so.

[smolnar82]

Follow-up: https://issues.apache.org/jira/browse/KNOX-3483

[moresandeep]

HadoopAuthPostFilter never captures the caller JWT; this feature silently dead for topologies using the HadoopAuth filter.

gateway-provider-security-hadoopauth/.../HadoopAuthPostFilter.java:91

doFilter() still calls the public single-arg createSubjectFromToken(String), which resolves to createSubjectFromToken(JWT, null). It was never updated to capture the caller's JWT as an AuthTokenCredential the way JWTFederationFilter.doFilter() now does.

Impact: On any topology using the HadoopAuth federation provider with support.jwt=true, no AuthTokenCredential is ever added to the Subject, so SubjectUtils.getAuthToken() returns null and the X-Knox-Auth-Token header is never emitted, even though the same caller gets it on a JWTProvider/SSOCookieProvider topology. The PR docs misattribute this to an inherent "innermost Subject.doAs wins" quirk, when it's actually a one-line oversight at this call site.

I approve this PR now and file a follow-up JIRA to fill this gap.

You are right, I did not think about HadoopAuth path. My patch fixes it for SSO Cookie and JWT Provider. For future reference (to me) this is HadoopAuth where the gap is

<provider>
      <role>authentication</role>
      <name>HadoopAuth</name>
      <enabled>true</enabled>
      <param>
        <name>config.prefix</name>
        <value>hadoop.auth.config</value>
      </param>
      <param>
        <name>hadoop.auth.config.type</name>
        <value>simple</value>
      </param>
      <param>
        <name>hadoop.auth.config.simple.anonymous.allowed</name>
        <value>false</value>
      </param>
      <param>
        <name>hadoop.auth.config.signature.secret</name>
        <value>knox-signature-secret</value>
      </param>
      <!-- Routes Bearer JWTs to the inner JWTFederationFilter in both HadoopAuthFilter and HadoopAuthPostFilter -->
      <param>
        <name>support.jwt</name>
        <value>true</value>
      </param>
    </provider>

Looks like you created a JIRA for this https://issues.apache.org/jira/browse/KNOX-3483
Thanks @smolnar82 !

[moresandeep]

convention

I see, i misunderstood the intention. The intention for the feature was to have, say credential "types" and principal "types" that we can use getType() on to pull data from security context? example of such type include Credential (type, value), Principal(type, value)

e.g. Credential("token", "ey......"), Principal("UserPrincipal", "Tom"), Principal("X500Principal", "Acme") etc.

[pzampino]

Do we expect tokens to be this large?

[moresandeep]

We have had cases where JWT tokens were huge, this can be because they can include assertions, groups etc!
This is configurable.