Skip to content

Conversation

@sojanmathew
Copy link
Contributor

@sojanmathew sojanmathew commented Oct 20, 2025

Fix the bug where activities that return with the 'interrupted' flag set, were unable to report their completion to the server. Activities can now successfully report completion even when they have the 'interrupted' flag set.

Without this fix, activity results are not successfully reported to server ( gRPC calls in ActivityWorker.sendReply() would fail with CancellationException when they have the 'interrupted' flag set).
The fix clears the 'interrupted' flag before making gRPC calls in ActivityWorker.sendReply() and restores it afterward, ensuring:

  • Activity results are successfully reported to the server
  • Thread interruption semantics are preserved for worker shutdown
  • All activity completion scenarios work (success/failure/cancellation)

Fixes: #731
Related: #722

What was changed

Fix the bug where gRPC calls fail when the thread's 'interrupted' flag is set, hence unable to report their completion status to the server.

Why?

The fix ensures that activities report their completion status to the Temporal server, while maintaining proper thread interruption semantics for the worker shutdown process.
According to the issue description and PR #722, the issue was that when an activity has 'interrupted' flag set:
i.e,
1 Catches an InterruptedException
2 Restores the 'interrupted' flag (Thread.currentThread().interrupt())
3 Continues to run and returns a result

The subsequent gRPC calls in ActivityWorker.sendReply() would fail with CancellationException ( gRPC request was
cancelled because the 'interrupted' flag was still set), hence activity result were not reported to the server.

The CancellationException is thrown in GrpcSyncRetryer when reporting activity completion results:

      
  } catch (InterruptedException e) {                                                                                  
    Thread.currentThread().interrupt();                                                                               
    throw new CancellationException();                                                                                
  }

Solution
Modify ActivityWorker.sendReply() method to:

1 Check the 'interrupted' flag before making gRPC calls using Thread.interrupted() (which also clears the flag)
2 Temporarily clear the flag to allow gRPC calls to succeed
3 Make the gRPC call to report the activity result
4 Restore the 'interrupted' flag in a finally block if it was originally set

@Spikhalskiy

Checklist

  1. Closes Activities that return with interrupted flag should be successfully reported as Completed #731

  2. How was this tested:

    1 Created and ran a comprehensive test that reproduces the exact scenario described in the bug report
    2 Verified the test failed before the fix (confirming the bug existed)
    3 Verified the test passes after the fix (confirming the bug is resolved)
    4 Ran existing shutdown tests to ensure no regressions were introduced

  3. Any docs updates needed?
    No

@sojanmathew sojanmathew requested a review from a team as a code owner October 20, 2025 00:13
@CLAassistant
Copy link

CLAassistant commented Oct 20, 2025

CLA assistant check
All committers have signed the CLA.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Sojan Mathew seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

  Activities that catch InterruptedException, restore the interrupted flag,
  and continue execution were unable to report their completion due to gRPC call failures.
  This occurred because gRPC calls fail when the thread's interrupted flag is set.
  The fix clears the interrupted flag before making gRPC calls in ActivityWorker.sendReply() and restores it afterward, ensuring:
  - Activity results are successfully reported to the server
  - Thread interruption semantics are preserved for worker shutdown
  - All activity completion scenarios work (success/failure/cancellation)

  Additionally, extract executeGrpcCallWithInterruptHandling() method to eliminate code duplication across the three gRPC response calls,
  improving maintainability and reducing the risk of inconsistent implementations.

  Fixes: temporalio#731
  Related: temporalio#722
*/
private void executeGrpcCallWithInterruptHandling(Runnable grpcCall) {
// Check if the current thread is interrupted before making gRPC calls
// If it is, we need to clear the flag to allow gRPC calls to succeed,then restore it after reporting.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What part of the SDK relies on it being restored?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What part of the SDK relies on it being restored?
@Quinn-With-Two-Ns
The CancellationException is thrown in GrpcSyncRetryer when reporting activity completion results:

      
  } catch (InterruptedException e) {                                                                                  
    Thread.currentThread().interrupt();                                                                               
    throw new CancellationException();                                                                                
  }      

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Quinn-With-Two-Ns Are you good with above change, if so, could you please approve PR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sojanmathew no I still don't understand the logic here, why is the flag being restored?

Copy link
Contributor Author

@sojanmathew sojanmathew Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Quinn-With-Two-Ns

  • Why Restoration is Critical:

    • Java Best Practices: Interrupted state should be preserved to prevent "swallowing" interrupts
    • Contract Preservation: The gRPC operation should be transparent to interrupt state
    • Calling Code Expectations: Framework or user code may check interrupt status after completion

Hope the test helps to explain before / after situation.
In summary :

  • Activity sets interrupted flag → Store original flag → Clear flag → gRPC call succeeds → Restore original flag .
    Please let me know if it clarifies or still not clear.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Activities that return with interrupted flag should be successfully reported as Completed

3 participants