When are scala's for-comprehensions lazy?

Question

In Python, I can do something like this:

lazy = ((i,j) for i in range(0,10000) for j in range(0,10000))
sum((1 for i in lazy))

It will take a while, but the memory use is constant.

The same construct in scala:

(for(i<-0 to 10000; j<-i+1 to 10000) yield (i,j)).count((a:(Int,Int)) => true)

After a while, I get a java.lang.OutOfMemoryError, even though it should be evaluated lazily.

Ben James · Accepted Answer · 2011-08-09T13:02:05.673

26

Nothing's inherently lazy about Scala's for-comprehension; it's syntactic sugar* which won't change the fact that the combination of your two ranges will be eager.

If you work with lazy views of your ranges, the result of the comprehension will be lazy too:

scala> for(i<-(0 to 10000).view; j<-(i+1 to 10000).view) yield (i,j)
res0: scala.collection.SeqView[(Int, Int),Seq[_]] = SeqViewN(...)

scala> res0.count((a: (Int, Int)) => true)
res1: Int = 50005000

The laziness here is nothing to do with the for-comprehension, but because when flatMap or map (see below) are called on some type of container, you get back a result in the same type of container. So, the for-comprehension will just preserve the laziness (or lack of) of whatever you put in.

*for something like:

(0 to 10000).flatMap(i => (i+1 to 10000).map(j => (i, j)))

edited Aug 09 '11 at 13:02

answered Aug 09 '11 at 12:25

Ben James

121,135
26
193
155

"something like", but the latter expression seems to return 100020001 elements, not 50005000. Whiskey tango foxtrot? – Michael Lorton Aug 09 '11 at 12:48
@Malvolio thanks for noticing! `j` should have been from `i+1 to 10000`, not `1 to 10000`. Fixed now. – Ben James Aug 09 '11 at 13:03
D'oh. I should have noticed the solution, not just the problem. I was staring at the map functions, ignoring the arguments.. – Michael Lorton Aug 09 '11 at 13:11
Wasn't Int.to(Int) lazy in 2.7? – tstenner Aug 10 '11 at 07:14

score 13 · Answer 2 · answered Aug 09 '11 at 12:43

Laziness comes not from the for-comprehension, but from the collection itself. You should look into the strictness characteristics of the collection.

But, for the lazy :-), here's a summary: Iterator and Stream are non-strict, as are selected methods of the view of any collection. So, if you want laziness, be sure to .iterator, .view or .toStream your collection first.

When are scala's for-comprehensions lazy?

2 Answers2

Linked