Stream JavaDSL: start adding alternate methods that use Java function interfaces instead of the Pekko ones

[pjfanning]

org.apache.pekko.japi.function package is commonly used for functions in our javadsl.

We can't just change the existing methods to use the Java built-in Function interfaces. (java.util.function package)

@kerr says that we can't add overloaded methods because the Java compiler will not be able to distinguish between the 2 overloaded methods when you try to provide lamdba functions as inputs.

We have options like

• add new methods with V2 in the name. An example is Flow.statefulMap which takes a Pekko Creator function as its first param. We could add a statefulMapV2 that takes a Supplier instance instead.
• add an object to javadsl.Flow called juf (for java.util.function) and add some variant functions there (instead of using V2 suffix) - suggested by @nvollmar
• we could add a javadsl2 package but we would end up duplicating a lot of code - a lot of the methods in javadsl will not need to change in javadsl2
  • if someone has the time and the motivation, this still might be a tidier solution that adding some 'V2' methods to the existing javadsl

[pjfanning]

@kerr would you be able to provide some code examples with some of the existing javadsl functions where coding would be easier if the Pekko javadsl method took a java.util.function as input instead of an org.apache.pekko.japi.function ?

We could end up going down a big rabbit hole on this topic and so far, I haven't seen too many real world examples. I must admit to not using Java for functional programming much. I use Scala for FP and some times Java for legacy applications (that don't tend to be FP focused).

[He-Pin]

The core reason is that when a Java user calls these functions using new xFunction, they will see a strange japi.function.Function instead of the familiar java.util.function.* that they are used to. I believe this is the main issue, especially for beginners. As for @exception, I don't think it's a significant problem since Java streams have never provided it anyway.

[He-Pin]

Please understand me, I work at a Java company where there are many Java developers who often wonder why we don't use Java functions. In fact, in Java, we have many ways to avoid the inconvenience brought by @exception, such as @SneakyThrow. Using Lombok is not an issue, almost everyone uses it. This is why I believe the benefits of keeping consistent with libraries like reactor-core and making javadsl more java native.

[mdedetrich]

Java is also being used at the company where I work, lombok is never used there (in fact I haven't seen lombok used in any modern Java codebases).

This is such a small issue though, while a beginner may be confused about org.apache.pekko.japi.function.*, this confusion should be temporary and its also justified. Pekko (or even Akka's) goal is also not to the most familiar java dsl to beginners at all costs which is what is being asked.

If you want to blame anyone for the confusion you should blame the JDK dev's, they should have supplied @FunctionalInterface's that allow throwing exceptions. Pekko is forced to do things like put allocation of resources (i.e. creating a file) into a lambda/function because it has to be evaluated on demand (since streams are lazy) and JDK doesn't provide a nice way of doing this and Pekko is not the only Java API that has to deal with this (I have to deal with this problem in Java all of the time).

[nvollmar]

I've never really used the java API of Akka/Pekko.
Would it be a long-term goal to switch all java api functions over from japi.function to java.util.function?
That would question would probably have a big impact on the best way to solve this.

Personally I don't like method suffixes like V2 as they are not really descriptive.
Another option instead of another package would be a sub-object to place such methods. Simple Example:

object Flow {
  ...
  def statefulMap[S, T](
    create: function.Creator[S],
    f: function.Function2[S, Out, Pair[S, T]],
    onComplete: function.Function[S, Optional[T]]): javadsl.Flow[In, T, Mat] = ...

  object juf {
      def statefulMap[R, T](
        create: java.util.function.Supplier[R],
        f: java.util.function.BiFunction[R, Out, T],
        close: java.util.function.Function[R, Optional[T]]): javadsl.Flow[In, T, Mat] = ...
   }
}

So Flow.statefulMap() vs Flow.juf.statefulMap()

[pjfanning]

Thanks, an object called juf or v2 or something else would be a better approach than appending V2 to method names.

With changing the javadsl methods altogether to remove the use of japi.function - that would have to be a major Pekko release (2.0.0) and I don't think we are looking at that yet.

[He-Pin]

Thanks, but this would introduce much overhead.

[mdedetrich]

So lets lay things out here, the only perceived benefit I can see to either changing the current API to use java.lang.* (or providing an alternate API that uses java.lang.*) is

The core reason is that when a Java user calls these functions using new xFunction, they will see a strange japi.function.Function instead of the familiar java.util.function.* that they are used to. I believe this is the main issue, especially for beginners. As for @exception, I don't think it's a significant problem since Java streams have never provided it anyway.

Which is an incredibly weak reason to change the entire API (I am even ignoring the fact that using hacks like @SneakyThrows would be just as confusing to any Java beginner as would org.apache.pekko.japi.function.*, you are just trading inconvenience for another and @SneakyThrows is objectively worse here since its a hack). On the other hand, there is a very good technical and correct reason as to why org.apache.pekko.japi.function.* exists, its for lambda's that can legitimately throw an exception i.e. in a when supplying a lambda to create for creating a resource (such as a file) that lambda can throw an IOException. This is succinctly summarised as

• Supplied lambda should never throw (i.e. creating a collection)? Use java.util.*
• Supplied lambda could throw (i.e. allocating a resource such as a file)? Use org.apache.pekko.japi.function.*

When reading an API and programming against the Pekko API in java, this to me is CRITICAL as it CLEARLY tells me for the lambda I am providing whether it makes sense for exceptions to be thrown or not, i.e. if with this allocate parameter, since its a java.util.* @FunctionalInterface I know that if I am trying to provide a lambda and its throwing an exception I am doing something very wrong. This is also why I don't wan't to encourage hacks like lombok's @SneakyThrow because its actually teaching the wrong pattern, if you are using the current Pekko javadsl you should never have to use @SneakyThrow because if you are throwing an exception in one of the lambda's that use java.util.* you are almost certainly doing something wrong.

Ontop of this, the benefit we get (which is some small convenience for beginners) is greatly overshadowed by the inconvenience of either having to change everything for Pekko 2.0.x (we cannot do this before, it will break users) or creating an alternate v2/juf javadsl which to be honestly I would almost certainly -1 as this is much more confusing than any other option or the status quo. If your goal is not to confuse beginners, then having multiple java api's will achieve the exact opposite effect.

Another point to keep in mind is that minor beginner inconveniences will always get trumped by correctness/API design as this is one of the core tenets of Akka and hence Pekko. If you want something easy and quick then you probably shouldn't be looking at Akka/Pekko in the first place. That doesn't mean we should make it as Pekko approachable by beginners, but it doesn't mean that Pekko should cater to every single minor beginner inconvenience, this is a slippery slope that we should not go down.

Given that, there is one glaring hole which is the this distinction of @FunctionalInterface's in core java stdlib vs the ones in org.apache.pekko.japi.function.* doesn't appear to be generally documented (even though it is clearly documented in code), due to this it makes sense to document this in the Pekko general docs somewhere so that its clear to beginners why this is the case.

Also it does make sense to maybe contribute @FunctionalInterface to official JDK that do accept throwing exceptions because this is a big oversight from JDK's side, Pekko needs to use lambda's because the evaluation has to be lazy but in some cases that also mean creating things like files that can throw exceptions.

[He-Pin]

That's true and why vavr exists.

[He-Pin]

Perhaps the Java API could be written in Java to make it look better for Java users, at least it won't affect any binary compatibility. This is also one of the reasons why there are currently a lot of Java code in kalix-sdk.

And for the checkedFunction interface, a new JEP?

[mdedetrich]

Perhaps the Java API could be written in Java to make it look better for Java users,

Yes but not at the cost of correct API design which is whats being asked. This distinction between the different function types is there for a good reason, its not like someone designed the current API this way to waste peoples time or to be deliberately confusing

And for the checkedFunction interface, a new JEP?

Definitely in for this option.