The firebase-admin-java 9.4.0 does not work with NIO (Non-blocking I/O).

[stager0909]

[READ] Step 1: Are you in the right place?

• For issues or feature requests related to the code in this repository
  file a Github issue.
  • If this is a feature request make sure the issue title starts with "FR:".
• For general technical questions, post a question on StackOverflow
  with the firebase tag.
• For general Firebase discussion, use the firebase-talk
  google group.
• For help troubleshooting your application that does not fall under one
  of the above categories, reach out to the personalized
  Firebase support channel.

[REQUIRED] Step 2: Describe your environment

• Operating System version: �ubuntu
• Firebase SDK version: firebase-admin-java 9.4.0
• Library version: java 17
• Firebase Product: fcm

[REQUIRED] Step 3: Describe the problem

Hello,
I am testing the firebase-admin-java SDK 9.4.0 with the following configuration:

FirebaseOptions options = FirebaseOptions.builder()
    .setCredentials(googleCredentials(firebaseCredential, httpTransport))
    .setDatabaseUrl(firebaseDatabaseUrl)
    .setHttpTransport(httpTransport)
    .setConnectTimeout(FIREBASE_CONNECT_TIMEOUT)
    .setReadTimeout(FIREBASE_READ_TIMEOUT)
    .setThreadManager(new CustomFirebaseThreadManager(
        firebaseTaskExecutor.getThreadPoolExecutor()))
    .build();

The reason I explicitly added ThreadManager is to monitor the ThreadPoolExecutor. Without this configuration, the ThreadPoolExecutor generated by FirebaseThreadManagers.DefaultThreadManager.doInit() will be used, which has a queue size of Integer.MAX_VALUE. If FCM responses are delayed, the queue can pile up, potentially causing an OOM (OutOfMemoryError).

Below is the configuration for H2AsyncClientBuilder:

H2AsyncClientBuilder h2AsyncClientBuilder = H2AsyncClientBuilder.create()
    .setDefaultConnectionConfig(connectionConfig)
    .evictIdleConnections(TimeValue.of(Duration.ofMinutes(10)))
    .setDefaultRequestConfig(requestConfig())
    .setIOReactorConfig(IOReactorConfig.custom()
        .setSoTimeout(FIREBASE_READ_TIMEOUT, TimeUnit.MILLISECONDS)
        .build())
    .addRequestInterceptorFirst(
        (HttpRequest request, EntityDetails entity, HttpContext context) -> {
            activeStream.incrementAndGet();
            if (context != null) {
                context.setAttribute("startTime", System.nanoTime());
            }
        })
    .addResponseInterceptorFirst(
        (HttpResponse response, EntityDetails entity, HttpContext context) -> {
            if (context == null || context.getAttribute("startTime") == null) {
                return;
            }
            long startTime = Long.parseLong(context.getAttribute("startTime").toString());
            long endTime = System.nanoTime();
            if (startTime < endTime) {
                Timer.builder(FCM_RESPONSE_TIMER_NAME)
                    .publishPercentiles(0.50, 0.95, 0.99)
                    .tags(
                        Lists.newArrayList(
                            Tag.of("host", ((HttpRoute) context.getAttribute("http.route")).getTargetHost().getHostName()),
                            Tag.of("http_version", context.getProtocolVersion().toString())
                        )
                    )
                    .register(meterRegistry)
                    .record(endTime - startTime, TimeUnit.NANOSECONDS);
            }
            activeStream.decrementAndGet();
        })
    .disableCookieManagement()
    .setH2Config(H2Config.custom().setMaxConcurrentStreams(500 * 10).build())
    .disableRedirectHandling()
    .disableAutomaticRetries();

Here is the code for sending FCM messages. The postProcessExecutor is simply responsible for logging the send results:

ApiFuture<String> responseApiFuture = firebaseMessaging.sendAsync(fcmMessage);
responseApiFuture.addListener(() -> {
    try {
        responseApiFuture.get();
        message.setResponseCode(FcmResult.SUCCESS.getCode());
        postProcessor.postProcess(MessageStatus.SENT, message);
    } catch (Exception e) {
        log ~~~
    }
}, postProcessorExecutor);

When testing this, if I set a breakpoint inside addRequestInterceptorFirst (or addResponseInterceptorFirst) in the second code block, the thread executing addRequestInterceptorFirst is the httpclient-dispatch thread.
At this point, firebaseThreadPoolTask.getActiveCount() (as set in FirebaseOptions.builder().setThreadManager()) is 1.
In other words, it seems like the firebaseThread is staying active while the httpclient-dispatch thread is running.

I would like the behavior to work as follows:

The firebaseThread performs tasks (such as SDK validation) before the HTTP call.
After handing over to httpclient-dispatch, the firebaseThread is released.
The httpclient-dispatch makes the request to the FCM server, and the IO selector detects when a response is received.
Once the IO selector detects the response, it hands over to httpclient-dispatch, which processes operations like interceptors and then passes it to postProcessorExecutor.
This behavior should prevent the firebaseThread from getting blocked when the FCM server is down or latency is high.
If the firebaseThread gets blocked, it means that the queue in the thread pool fills up, which can lead to OOM or RejectedExecutionException.

If I have misunderstood anything, please let me know.
If my understanding is correct, how can I resolve this issue?

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.