Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: confusion deadletter while ask timeout #668 #664

Merged
merged 10 commits into from
Sep 30, 2023
Merged

Conversation

Roiocam
Copy link
Member

@Roiocam Roiocam commented Sep 20, 2023

Motivation

Attach the necessary information before sending the message to DeadLetter to correctly display the destination in the DeadLetter.

References: #668

How does this PR fix work?

In classic Actors, when an Actor dies, its Mailbox is "swap" to a deadLetterMailbox Mailbox.

https://github.com/apache/incubator-pekko/blob/a3f48833c97501a0193f4135f99c5f6ea04a03dd/actor/src/main/scala/org/apache/pekko/dispatch/AbstractDispatcher.scala#L239-L245

When an Actor sends a message to another ActorRef, it essentially enqueues the message to that Actor's mailbox.

When an Actor sends a message to a dead Actor, it means that the message would enqueue to the "deadLetterMailbox",the dead mailbox will transform the message into a DeadLetter event and then publish it to the EventBus. This is how DeadLetters work in classic Actors.

In Typed actor, it basiclly same, but the sender variable cannot be passed implicitly.

In the Ask pattern, there are some differences. When an Actor uses ask to request another Actor, it essentially creates a temporary PromiseActorRef to receive the response from the target Actor.

When the PromiseActorRef receives a tell call, it checks if the current Future has already completed and, if so, emits a DeadLetter event. This is done to ensure consistent behavior with the tell method (i.e., sending DeadLetter events to the EventBus via DeadLetterActorRef).

However, the PromiseActorRef directly sends the message without transforming it into a DeadLetter event, resulting in distorted logs.

This PR converts the message into a DeadLetter event before sending it from PromiseActorRef to DeadLetterActorRef.

@@ -617,7 +617,7 @@ private[pekko] final class PromiseActorRef(

override def !(message: Any)(implicit sender: ActorRef = Actor.noSender): Unit = state match {
case Stopped | _: StoppedWithPath =>
provider.deadLetters ! message
provider.deadLetters ! DeadLetter(message, if (sender eq Actor.noSender) provider.deadLetters else sender, this)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, the sender will always be empty because in typed, the passing of the sender is changed from explicitly on method calling to optional choice as a field in the message, unless we can extract the value with the same field name (such as sender, replyTo) from the user's message and use it here.

Copy link
Contributor

@pjfanning pjfanning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks the Scala 3 build and possibly other builds.

actor/src/main/scala/org/apache/pekko/pattern/AskSupport.scala has been like this for years - changing it now seems like a bad idea.

I will not agree to unblock this change without major community support in favour of it.

@Roiocam
Copy link
Member Author

Roiocam commented Sep 20, 2023

actor/src/main/scala/org/apache/pekko/pattern/AskSupport.scala has been like this for years - changing it now seems like a bad idea.

have you read the original issue(29795) on akka yet? Even if it doesn't convert into a DeadLetter and send it, the DeadLetterActor will still perform the transform and have almost the same behavior.

Furthermore, in typed, this would result in ask from anywhere not showing the destination but instead showing deadLetterActor path. I don't think this is more helpful for troubleshooting.

@Roiocam
Copy link
Member Author

Roiocam commented Sep 20, 2023

@pjfanning
Copy link
Contributor

Could you at least fix the build issue? This PR does not compile when Scala 3 is used.

@pjfanning
Copy link
Contributor

have you read the original issue(29795) on akka yet? Even if it doesn't convert into a DeadLetter and send it, the DeadLetterActor will still perform the transform and have almost the same behavior.

Do not ask a Pekko developer to look at Akka. We could be sued be for looking at Akka code.

@Roiocam
Copy link
Member Author

Roiocam commented Sep 20, 2023

Could you at least fix the build issue? This PR does not compile when Scala 3 is used.

Of course, otherwise this PR would not have passed the checks. However, I need some time to set up my environment. It's already early morning in my timezone, so I will fix this PR during my free time tomorrow.

@Roiocam
Copy link
Member Author

Roiocam commented Sep 20, 2023

Do not ask a Pekko developer to look at Akka. We could be sued be for looking at Akka code.

I'm sorry, I am not aware of this.

@Roiocam Roiocam marked this pull request as draft September 20, 2023 17:22
@Roiocam Roiocam marked this pull request as ready for review September 20, 2023 18:24
@He-Pin
Copy link
Member

He-Pin commented Sep 20, 2023

@Roiocam Would you like to openup an issue here, thanks and welcome

@He-Pin
Copy link
Member

He-Pin commented Sep 21, 2023

The fix looks good but the tests need some update, and would be nice to have a issue to track the problem too.

@Roiocam Roiocam changed the title fix: keep typed ask deadletter same as classic fix: confusion deadletter while ask timeout #668 Sep 21, 2023
@Roiocam
Copy link
Member Author

Roiocam commented Sep 21, 2023

@He-Pin PTAL, and why pekko PR action requires approval from a maintainer.

@He-Pin
Copy link
Member

He-Pin commented Sep 21, 2023

@He-Pin PTAL, and why pekko PR action requires approval from a maintainer.

Because this is the first time, it was in Akka too.

import org.apache.pekko.actor.typed.scaladsl.AskPattern.{ schedulerFromActorSystem, Askable }
import org.apache.pekko.actor.typed.scaladsl.Behaviors
import org.apache.pekko.actor.typed.scaladsl.Behaviors._
import org.apache.pekko.util.Timeout
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mdedetrich How to format these imports as the style your suggested, is there any sbt command?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears that the contribution guide mentions this, but sortImports sort too many files... that not correct behavior.
https://github.com/apache/incubator-pekko/blob/main/CONTRIBUTING.md#scala-style

Copy link
Member

@He-Pin He-Pin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix looking good, just need updating a little about the test

@pjfanning
Copy link
Contributor

@Roiocam Could you fill in a CLA? With changes like this, it seems safer to have a CLA on file.

https://www.apache.org/licenses/contributor-agreements.html


class ManualTerminatedTestSetup(workerCnt: Int) {
implicit val timeout: Timeout = 10.millis
val workerLatch = new CountDownLatch(workerCnt)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is bad practice for unit tests, multiple tests in DeadLetterSpec may use this same immutable value.

@Roiocam
Copy link
Member Author

Roiocam commented Sep 23, 2023

@pjfanning I have signed my ICLA and sent it to [email protected], but I didn't receive any reply from it.
Does it need to be cc to the Pekko project or not?

Furthermore, those flaky test has been fixed, and need to trigger the CI run.

@pjfanning
Copy link
Contributor

@pjfanning I have signed my ICLA and sent it to [email protected], but I didn't receive any reply from it. Does it need to be cc to the Pekko project or not?

Thanks @Roiocam - it may take a few days for ASF Secretary to respond, especially since it is the weekend.

@Roiocam Roiocam requested a review from pjfanning September 23, 2023 09:28
@Roiocam
Copy link
Member Author

Roiocam commented Sep 23, 2023

Thanks @Roiocam - it may take a few days for ASF Secretary to respond, especially since it is the weekend.

Got it, Thanks. Plz disregard duplicate review requests.

@Roiocam Roiocam closed this Sep 28, 2023
@He-Pin
Copy link
Member

He-Pin commented Sep 29, 2023

@Roiocam HI, happy holidays, I just saw those Pull requests and issues are closed, is there any reason behind those?

I’m off this network because I back to my hometown, sorry for not reviewing it.

@Roiocam Roiocam reopened this Sep 30, 2023
@Roiocam
Copy link
Member Author

Roiocam commented Sep 30, 2023

@He-Pin I am sorry for that closed this PR for no reason, this PR is not only my contribution.

Copy link
Member

@He-Pin He-Pin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

@pjfanning
Copy link
Contributor

@mdedetrich what do you think? In the end, this is probably what the code should have always done. Seems safe enough for a v1.1.x change to me.

Copy link
Contributor

@mdedetrich mdedetrich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, is #664 (review) still a concern?

Copy link
Contributor

@pjfanning pjfanning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@pjfanning pjfanning merged commit 225701d into apache:main Sep 30, 2023
17 checks passed
@pjfanning pjfanning added the release notes Need to release note label Sep 30, 2023
@pjfanning pjfanning added this to the 1.1.0 milestone Sep 30, 2023
@He-Pin
Copy link
Member

He-Pin commented Oct 9, 2023

@Roiocam Thank you.

@pjfanning pjfanning removed the release notes Need to release note label Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants