Extension methods

[jackfirth]

Currently, namespaces can be extended, and classes double as namespaces. But that's not enough to support extension methods. I'd like this to work:

#lang rhombus
export: Foo // includes Foo.foo() method
class Foo()
method Foo.foo(): 42

[jackfirth]

To clarify, my use case is a Sequence.toList() method. Defining that on Sequence directly would introduce a cyclic dependency between Sequence and List. So instead, I'd like to define Sequence in rhombus/data/private/sequence, make List depend on that, then add a rhombus/data/sequence module where I import Sequence, add the toList() extension method, and then re-export Sequence. So I'd have something like this:

#lang rhombus

export:
  Sequence

import:
  rhombus/data/private/sequence.Sequence

method Sequence.toList():
  let iterator = iterate() // this is a reference to the Sequence.iterate() method
  let builder = List.builder()
  ... loop over the iterator, adding each element into the builder ...
  builder.build()

[benknoble]

This reminds me at least incidentally of Kotlin and C#'s extension methods, which have been great in their respective ecosystems for developing project- and domain- specific extensions to the standard library. In this vein I think it makes a great complement to Racket's language-oriented programming.

[jackfirth]

Kotlin's extension methods were exactly my inspiration here 🙂

[mflatt]

I see that Kotlin extension methods are static and resolved by scope, the same as Rhombus namespace extensions. Along those lines, it seems possible how to make static extension methods work — but it would mean that you can only call an extension method on an object with . when the right-handle side of . has the static information for the extended annotation. For example, dynamic(seq).to_list() would not work (where dynamic is an identity function that promises to have no static result information).

With private fields and methods, we have already ventured into into the space where some class members are accessible only via . with static information. This worries me, but requiring static information to access private members seems ok on the grounds that it's relatively local; code that needs access to private is in a place where static information can be reasonably maintained. Is similar constraint acceptable for extension methods?

[jackfirth]

Unfortunately that means changing a method from a regular method to an extension method can break programs, especially programs that don't use annotations much. But I think that drawback could be acceptable, given how useful extension methods are for untangling cyclic dependencies and for creating DSLs. I think we should give it a shot.

[jackfirth]

Oh, one more important point: extension methods should be available on subtypes too. So (xs :: Collection).toList() should work, since Collection extends Sequence.

[gus-massa]

I hope it was not discussed before, but I have a question:

I was recently looking at racket-lua looking for some low handling ideas for the optimization. It has "global" monkey patching, because the official version of lua has "global" monkey patching. My concern is that it's very unfriendly with the optimization, for example math.sqrt can be overwritten (for example to add debug output, or in a weird case with arbitrary code).

Is the idea here to add "global" monkey patching or "local" monkey patching? (I'm not sure about the correct technical term.) A "local" version can be optimizer friendly if it expands to a few macros under the hood (like functions with optional parameters in Racket).

Also, what happens with diamond inheritance?

[mflatt]

I don't think the idea here would count as monkey-patching. It's scope-based, and not mutation-based. If you don't import the module that re-exports Foo and its extension, then the extension isn't visible. It doesn't not provide a way to change the meaning of an existing binding like math.sqrt.

With extensible namespace as currently implemented, diamond importing works. If a namespace like Foo is re-exported with extensions from two different modules, those modules are still exporting the same original binding, and so the common part can be imported from both (without conflict) into a client module. The client module can also see both extensions; if there are conflicts between the extensions, then the conflict can be rejected the same as any name conflict on import. The inheritance facet of interfaces probably creates the potential for a new kind of conflict via inheritance, though, and an offhand guess is that canbe method calls that cannot be resolved unambiguously — so, an error later than checking import conflicts.

[mflatt]

I don't know how to make this work, after all.

It does work to extend the Sequence namespace with a toList function:

fun Sequence.toList(s :: Sequence) :: List:
  let iterator = s.iterate()
  .... // etc.

....
Sequence.toList(some_seq)

That approach does not make some_seq.toList() work, however.

The problem with supporting extension methods is not just dynamic dispatch. Even if some_seq has static information that points . at the Sequence definition, there's no path from the Sequence definition's binding (possibly in some other module) to the extension method's binding (possibly not only in a different module, but in a nested scope within the module).

Longer explanation: The macro system can find bindings based on symbols plus scopes, but the needed lookup here depends on what the binding's compile-time value refers to (i.e., find an external-method binding that refers to the same other binding as an annotation). That mapping turns out to be difficult to set up outside the module system, in part because there's no way to get control over what happens when a name is imported by import or require. (Adding something like would have its own issues; there are surely some trade-offs here.) Kotlin, in contrast, has a simpler binding story, and I expect that extension methods are built directly into the binding implementation; that is, binding knows about extension methods directly, and it setups up a suitable registration on import based on the type that's extended by the extension method.

Since it seems that we're far from making extension methods work in Rhombus, and since extension methods are not clearly a complete solution for the original problem (no overriding and no dynamic dispatch), I think we should probably look for other solutions to the original problem.

[jackfirth]

Hmm. Let's abandon this idea for now and keep it in mind as something to address in the future. For the collection APIs, I'm gonna just not bother adding a fluent method for copying sequences. Instead I'll add List.copyOf(seq) / Set.copyOf(seq) / etc. functions.