Using a dynamically calculated 'step' inside 'xrange'

Question

Suppose I have:

def func(n):
    for i in range(1,100000,2*n+3):
        ...

It is obvious that the step = 2*n+3 part is calculated once.

But is this guaranteed to be the same for xrange?

According to this answer, xrange is a sequence object that evaluates lazily.

So the question is basically - which part evaluates lazily?

Is it only the start <= stop part, or also the step part?

I tried a simple test in order to determine the answer:

n = 1
for a in xrange(0,100,n):
    print a
    n += 1

This test clearly shows that n is not reevaluated at every iteration.

But I'm suspecting that perhaps the n inside the xrange expression "lives in a different scope" than that of the n declared before the xrange.

Thank you.

score 4 · Answer 1 · edited Jul 18 '17 at 12:15

4

Neither range() nor xrange() care how the step value was derived; the expression is executed and the result of the expression is passed to the call, be that range(), xrange() or any other callable object.

That's because (...) is an expression too; it is a call expression; the arguments passed into a call are all expressions that are evaluated before the result is passed in to the call. It doesn't matter what is being called here.

TLDR; the xrange() object is passed the outcome of the expression, not the expression itself. As long as that outcome is an integer object, it'll be stored by the object (as an immutable value) to base the virtual sequence of.

edited Jul 18 '17 at 12:15

Christian Dean

22,138
7
54
87

answered Jul 18 '17 at 12:03

Martijn Pieters

1,048,767
296
4,058
3,343

1

Is the `step` part inside the `xrange` guaranteed to be calculated once, or can it possibly be calculated at every iteration in some cases? – goodvibration Jul 18 '17 at 12:04
1

@goodvibration: it is calculated *before the object is created*. So just once, not for every step. – Martijn Pieters Jul 18 '17 at 12:05
I think I'm beginning to understand where my confusion derives from. I need to look at `range` as a function that returns an array, and at `xrange` as a function that returns an iterator. That pretty much solves everything. Thank you!!! – goodvibration Jul 18 '17 at 12:08
1

@goodvibration: exactly. They both are callable objects (functions are the most common callable object, but classes are callable too, and so is the `xrange()` *type*). A call expression produces another result; the returned value of the function, or the new instance of a type. – Martijn Pieters Jul 18 '17 at 12:09
@goodvibration: if you ever needed to create a range-like object where the step was to be dynamic, you can't just pass in an expression like that; you'd have to pass in a function or other callable that produces the step size each time you need it. The object would then call it each time. – Martijn Pieters Jul 18 '17 at 12:10

score 2 · Answer 2 · answered Jul 18 '17 at 12:05

The fact that xrange works lazily means that the arguments are evaluated at "construction" time of the xrange (in fact xrange never knew what the expressions were in the first place), but the elements that are emitted are generated lazily. So the step parameter is actually calculated before you even call xrange(..). As a result, xrange(..) does not know how the step was calcuated, and thus cannot ask to re-evaluate it.

Although xrange(..) is more complex than that (since it can work with negative steps, etc.), a very basic implementation would be:

def xrange(frm,to,step):
    i = frm
    while i < to:
        yield i
        i += step

Your depiction of `xrange` as a function made it perfectly clear to me, thank you! — goodvibration, Jul 18 '17 at 12:10

score 2 · Answer 3 · answered Jul 18 '17 at 12:09

2

To add on top of the other answers: A good way to test this would be to provide a function as the step argument. If it is to be evaluated multiple times then the function would need to be called multiple times.

def foo():
    print 'foo was called'
    return 25

for i in xrange(0, 100, foo()):
    print i

The above code outputs

foo was called
0
25
50
75

which shows that the step argument is evaluated exactly once.

answered Jul 18 '17 at 12:09

DeepSpace

78,697
11
109
154

Yes, it occurred to me shortly after I had posted the question. The answers here have been very helpful though. Thanks. – goodvibration Jul 18 '17 at 12:11

score 2 · Answer 4 · answered Jul 18 '17 at 12:30

when you pass the appropriate arguments into xrange(), Python will construct a generator object that will compute values "on the fly".

Once that generator object is constructed and returned, the object is set. It no longer cares whether a variable originally used in calculating its constructor arguments changes or not.

This same principal applies not only to xrange(), but to other callable objects as well. Once you pass arguments to a callable, Python calculates the expressions using the current value of any variables, and passes the result to the callable. If any variables that were used in calculating the arguments of the callable have their value changed, Python does not care because only the current value of the variable was used. Python will not keep recalculating the arguments of a function every time a variable used in an expression passed into it changes:

>>> n = 10
>>> xr = xrange(n)
>>> n += 1
>>> n
11
>>> list(xr)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>

score 1 · Answer 5 · answered Jul 18 '17 at 12:04

Lazy evaluation does not mean you can change the evaluation while it is going on; while iterating xrange you are dealing with a generator, that is created with the parameters you provide. It can not change its step during iteration, at least not with the built in one.

Using a dynamically calculated 'step' inside 'xrange'

5 Answers5