feat: Support querying S2A Addresses from MDS #1400

@@ -143,6 +143,11 @@ private S2AConfig getS2AConfigFromMDS() {
      try {
        HttpResponse response = request.execute();
        if (!response.isSuccessStatusCode()) {
+          int statusCode = response.getStatusCode();


fyi: @lqiu96 @zhumin8 , after chatting with @xmenxk (authored the Go implementation: https://github.com/googleapis/google-cloud-go/blob/main/auth/internal/transport/s2a.go), he mentioned that we shouldn't retry if the endpoint doesn't exist. This logic exists in Go impl, so adding it here as well.

I am leaning toward switching to HttpRequest's built-in retry handlers, logic like to customize the retry behavior based on return code can go into HttpUnsuccessfulResponseHandler. See other comment for details.

we shouldn't retry if the endpoint doesn't exist

This means that we don't retry only on 404 status code, right?

I am leaning toward switching to HttpRequest's built-in retry handlers, logic like to customize the retry behavior based on return code can go into HttpUnsuccessfulResponseHandler. See other comment for details.

SG! I'll address your other comment.

This means that we don't retry only on 404 status code, right?

Yes, we want to retry on everything but 404.
Update: I've inspected the list of error codes returned by MDS, and updated the list of retriable error codes to include the ones that we want to retry on; we only want to retry on 5xx error codes. I'll ping you these references internally.

rmehta19 · 2024-10-24T17:33:19Z

Thanks for the second pass through @lqiu96! I've addressed the comments, please let me know of any followups / additional comments.

oauth2_http/java/com/google/auth/oauth2/S2A.java

zhumin8 · 2024-10-24T18:26:17Z

oauth2_http/java/com/google/auth/oauth2/S2A.java

+      return S2AConfig.createBuilder().build();
+    }
+
+    for (int i = 0; i < MAX_MDS_PING_TRIES; i++) {


I only mentioned ExponentialBackOff because that is used in one of the recent changes related to retry (https://github.com/googleapis/google-auth-library-java/pull/1452/files)
I suppose you can do similar, by using HttpRequest's built-in retry handlers and set request.setNumberOfRetries(OAuth2Utils.DEFAULT_NUMBER_OF_RETRIES);? If you do not need ExponentialBackOff, maybe something simple e.g.

// Set the number of retries request.setNumberOfRetries(OAuth2Utils.DEFAULT_NUMBER_OF_RETRIES); // Retry on specific status codes (you might want to adjust these) request.setUnsuccessfulResponseHandler( new HttpUnsuccessfulResponseHandler() { @Override public boolean handleResponse( HttpRequest request, HttpResponse response, boolean supportsRetry) throws IOException { return RETRYABLE_STATUS_CODES.contains(response.getStatusCode()); } });

About thread-safety, noticed HttpRequest is also marked as not safe here, likely because its member variable hold state information . But I don't see it a concern in this implementation, as executeAsync() is not used and getS2AConfigFromMDS() creates and executes a request each time.

zhumin8 · 2024-10-24T18:31:24Z

oauth2_http/java/com/google/auth/oauth2/S2A.java

@@ -143,6 +143,11 @@ private S2AConfig getS2AConfigFromMDS() {
      try {
        HttpResponse response = request.execute();
        if (!response.isSuccessStatusCode()) {
+          int statusCode = response.getStatusCode();


I am leaning toward switching to HttpRequest's built-in retry handlers, logic like to customize the retry behavior based on return code can go into HttpUnsuccessfulResponseHandler. See other comment for details.

zhumin8 · 2024-10-24T18:33:29Z

oauth2_http/java/com/google/auth/oauth2/S2AConfig.java

+
+    @CanIgnoreReturnValue
+    public Builder setPlaintextAddress(String plaintextAddress) {
+      this.plaintextAddress = plaintextAddress;


Can you add this explanation as javadoc comment here or to getS2AConfigFromMDS() for future references?

rmehta19 · 2024-10-25T00:42:54Z

Thanks for another review @zhumin8 and @lqiu96 ! I have addressed this round of comments, please let me know if there is anything else

lqiu96 · 2024-10-25T17:05:10Z

oauth2_http/java/com/google/auth/oauth2/S2A.java

+    try {
+      request = transportFactory.create().createRequestFactory().buildGetRequest(genericUrl);
+      request.setParser(parser);
+      request.getHeaders().set(METADATA_FLAVOR, GOOGLE);
+      request.setThrowExceptionOnExecuteError(false);
+      request.setNumberOfRetries(OAuth2Utils.DEFAULT_NUMBER_OF_RETRIES);
+
+      ExponentialBackOff backoff =
+          new ExponentialBackOff.Builder()
+              .setInitialIntervalMillis(OAuth2Utils.INITIAL_RETRY_INTERVAL_MILLIS)
+              .setRandomizationFactor(OAuth2Utils.RETRY_RANDOMIZATION_FACTOR)
+              .setMultiplier(OAuth2Utils.RETRY_MULTIPLIER)
+              .build();
+
+      // Retry on 5xx status codes.
+      request.setUnsuccessfulResponseHandler(
+          new HttpBackOffUnsuccessfulResponseHandler(backoff)
+              .setBackOffRequired(
+                  response -> RETRYABLE_STATUS_CODES.contains(response.getStatusCode())));
+      request.setIOExceptionHandler(new HttpBackOffIOExceptionHandler(backoff));
+    } catch (IOException e) {
+      return S2AConfig.createBuilder().build();
+    }


S2AConfig.createBuilder().build() sets plainTextAddress and mtlsS2AAddress to "" by default. Can we move this code in this try block into the one below (merging them)?

Done in ed681f5

oauth2_http/java/com/google/auth/oauth2/S2A.java

oauth2_http/javatests/com/google/auth/oauth2/MockMetadataServerTransport.java

lqiu96 · 2024-10-25T17:13:11Z

oauth2_http/javatests/com/google/auth/oauth2/MockMetadataServerTransport.java

+        // Create the JSON response
+        GenericJson content = new GenericJson();
+        content.setFactory(OAuth2Utils.JSON_FACTORY);
+        if (requestStatusCode < 400) {


nit: Can this be 200? Or are there other 2xx/ 3xx codes?

Yeah it can just be 200. We expect only 200 on successful responses. Done in 20825f7

lqiu96 · 2024-10-25T17:17:10Z

oauth2_http/javatests/com/google/auth/oauth2/MockMetadataServerTransport.java

+  private String plaintextS2AAddressJsonKey;
+
+  private String plaintextS2AAddress;
+
+  private String mtlsS2AAddressJsonKey;
+
+  private String mtlsS2AAddress;


nit: Can these values be stored as a map?

perhaps Map<String, String> s2aContentMap (or a different name)? there would be one setter setS2AContentMap() instead of 4?

Done in 594df7b

lqiu96

Thanks for working with us to address the comments. I think this mostly looks good to me given the context.

Can you also do two additional things:

Run mvn fmt:format on this PR?
Update the title to be feat: {title}? Something that describes the functionality feat: Support querying S2A Addresses from MDS or whatever you think aptly describes it.

rmehta19 · 2024-10-25T18:00:01Z

@lqiu96 , I've addressed your latest round of comments -- thanks for the feedback!

Run mvn fmt:format on this PR?

Done in 16fd964

Update the title to be feat: {title}? Something that describes the functionality feat: Support querying S2A Addresses from MDS or whatever you think aptly describes it.

Done.

lqiu96 · 2024-10-25T18:53:20Z

oauth2_http/javatests/com/google/auth/oauth2/S2ATest.java

+    transportFactory.transport.setS2AContentMap(
+        "plaintextS2AAddressJsonKey", S2A.S2A_PLAINTEXT_ADDRESS_JSON_KEY);
+    transportFactory.transport.setS2AContentMap("plaintextS2AAddress", S2A_PLAINTEXT_ADDRESS);
+    transportFactory.transport.setS2AContentMap(
+        "mtlsS2AAddressJsonKey", S2A.S2A_MTLS_ADDRESS_JSON_KEY);
+    transportFactory.transport.setS2AContentMap("mtlsS2AAddress", S2A_MTLS_ADDRESS);


Ah, I was going to suggest something like:
For valid data:

transportFactory.transport.setS2AContentMap( ImmutableMap.of(S2A.S2A_PLAINTEXT_ADDRESS_JSON_KEY, S2A_PLAINTEXT_ADDRESS, S2A.S2A_MTLS_ADDRESS_JSON_KEY, S2A_MTLS_ADDRESS));

For invalid data:

transportFactory.transport.setS2AContentMap( ImmutableMap.of(INVALID_JSON_KEY, S2A_PLAINTEXT_ADDRESS, S2A.S2A_MTLS_ADDRESS_JSON_KEY, S2A_MTLS_ADDRESS));

And the content would be populated via iteration through tkey k/v's of the map. Something like:

if (requestStatusCode == 200) { for (Map.Entry<String, String> entrySet: content.entrySet()) { content.put(entrySet.getKey(), entrySet.getValue()) } }

This is more succinct -- thanks. Done in 1e6c058

lqiu96

LGTM. Added a few nits. Please do let @zhumin8 to review and give an approval as well

rmehta19 · 2024-10-25T21:40:43Z

LGTM. Added a few nits. Please do let @zhumin8 to review and give an approval as well

Thanks for the review @lqiu96 ! @zhumin8 , please let me know if there is anything else to address, thanks!

westarle · 2024-10-28T13:06:15Z

oauth2_http/java/com/google/auth/oauth2/S2AConfig.java

+    return new Builder();
+  }
+
+  public String getPlaintextAddress() {


I'd like Javadocs for all these public things, pointing to public docs for MDS.

We don't have this documented in public MDS docs (e.g. https://cloud.google.com/compute/docs/metadata/predefined-metadata-keys). We do have an AIP: https://google.aip.dev/auth/4115 which discusses this autconfig endpoint and how it fits in the mTLS via S2A + bound tokens story. WDYT about 934679c?

sonarcloud · 2024-10-28T17:55:01Z

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
92.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

rmehta19 added 5 commits February 23, 2024 18:15

utils.

c96cb4a

formatted.

f90be0b

static mtls config.

0c64a0a

update autoconfig endpoint URL.

993663d

plaintext and mtls S2A address.

0f96e86

rmehta19 requested a review from a team as a code owner May 9, 2024 00:20

product-auto-label bot added the size: l Pull request size is large. label May 9, 2024

rmehta19 added 5 commits May 8, 2024 17:24

utils.

3d68cef

formatted.

6d75a4e

static mtls config.

d932e0c

update autoconfig endpoint URL.

6aa071b

plaintext and mtls S2A address.

ddac7aa

rmehta19 force-pushed the s2a-java-integration branch from 0f96e86 to ddac7aa Compare May 9, 2024 00:24

clundin25 requested a review from westarle May 13, 2024 18:36

xmenxk reviewed May 15, 2024

View reviewed changes

rmehta19 added 3 commits May 17, 2024 09:50

Merge remote-tracking branch 'refs/remotes/origin/s2a-java-integratio…

2c26736

…n' into s2a-java-integration

Use logic in ComputeEngineCredentials to get MDS URL.

67f9462

retry MDS request.

36d4cd1