Scala Inheritance; Builder Trouble; Non-Generic IterableLike

Question

I'm trying to implement a simple design goal, but the complexity of Scala's type system gives me some headache. After a comparison of Traversable, Iterator, Iterable, Stream, View etc my decision is to define a custom trait (let's just call it Stream for brevity) that

• is non-generic (my stream semantically only makes sense as some Stream[StreamEntry] and I want to avoid meaningless types like Stream[Int])
• has similar usage to Iterable
• all members like take, drop, etc should return Stream and not the basic Iterable.

This is what I have tried so far:

Approach 1

To sketch the use case, a simple example (which violates the third design goal) would be:

case class StreamEntry(data: Double) // just a dummy

trait Stream extends Iterable[StreamEntry] {
  val metaInfo: String
}

// example use case
val s = new Stream {
  val metaInfo = "something"
  val iterator = StreamEntry(1) :: StreamEntry(2) :: StreamEntry(3) :: Nil toIterator
}

val t = s.take(1) // unfortunately, this is no longer a Stream

Approach 2

This third requirement calls for the usage of a template trait instead of the base trait (I hope this is the standard terminology to refer to either SomeCollection or SomeCollectionLike). This means I have to use IterableLike[StreamEntry, Stream] which redefines the return types of the representing collection just as Iterable extends IterableLike[A, Iterable[A]] to return Iterables. My idea was to do pretty much the same as Iterable does. This would be:

// this is exactly the way `Iterable` is defined, but non-generic
trait Stream extends Traversable[StreamEntry]
             with GenIterable[StreamEntry]
             with GenericTraversableTemplate[StreamEntry, Stream]
             with IterableLike[StreamEntry, Stream] {
  ...
}

Unfortunately, this does not compile because Stream appears as template argument to GenericTraversableTemplate and the compiler now requires a template argument (exactly one) for Stream itself, which makes sense.

Approach 3, 4, ...

Starting from here, I got lost in the type system. Just removing with GenericTraversableTemplate leads to an incompatible type of newBuilder and an illegal inheritance due to conflicts in the type parameters in GenericTraversableTemplate from GenInterable and Traversable.

Maybe the closest solution was the following:

trait Stream extends TraversableLike[StreamEntry, Stream] 
             with IterableLike[StreamEntry, Stream] {
  val metaInfo: String
  def seq = this
  def newBuilder: scala.collection.mutable.Builder[StreamEntry, Stream] = ???
}

This compiles but unfortunately I have no idea how to implement the Builder. Is it possible to reuse a generic Builder for my non-generic trait? Actually I though I can go without a Builder because I never actually want to build a new Stream from other collections. But currently I'm experiencing some strange runtime behavior with this approach, which I cannot fully understand. For instance:

val s = new Stream {
  val metaInfo = "something"
  val iterator = StreamEntry(1) :: StreamEntry(2) :: StreamEntry(3) :: Nil toIterator
}

// executing the following independently (not in sequence) results in:

s.take(1)    // throws: scala.NotImplementedError: an implementation is missing
             // seems to require a Builder :(
s.toArray    // works
s.toIterator // works
s.toIterable // throws: java.lang.ClassCastException: cannot be cast to scala.collection.Iterable

Now I feel somewhat lost in the depth of the Scala type system. Am I still on the right track with this last approach and is the Builder just the missing piece in this puzzle?

And how would a Builder implementation look like for a non-generic non-cached type? A straightforward idea to implement += would be to use some mutable Buffer, but this would be very much against the use Iterators in the first place... And how should I implement the to member if I don't know how to construct a class of that type? I guess all relevant code must be somewhere in the library, I just can't dig it out.

Answer 1

Wow! You've got a lot going on there...

Here are some things you should know or consider in solving this design problem...

Terminology:

• We don't refer to "template"s, we call them "generic" or "parameterized" types. The reason for this is that these types are not templates! That is, they're not filled in with their actual type parameters to create new classes each time they're used (as is the case in C++, which rightly uses the term "template"). Instead, only one class is created (*) and it serves every instantiation of that generic type with particular type parameters.

Design & Language Factors:

You say:

… is non-generic (my stream semantically only makes sense as some Stream[StreamEntry] and I want to avoid meaningless types like Stream[Int])

The requirement does not argue for a non-generic class. Rather it is the essence of what a "type bound" is. E.g.:

class Generic[T <: UpperBound](ctorArgs...) {
}

In this case, class Generic may only be instantiated with types that are subtypes of UpperBound. (Note that whenever we say "subtype" we mean a reflexive subtype relationship. In other words, every type is a subtype of itself under this definition.

The upshot:

I wonder what your "stream" class is or does or has that is not satisfied by an existing type in the Scala Standard Library? As you've discovered extending the standard library collection classes is not entirely trivial, though it certainly is doable. I think doing so is not an elementary exercise in Scala programming and probably shouldn't be attempted as one of your first forays into Scala.

(*) This is an oversimplification that suffices for the purpose of this explanation.