Why is this output variable in my LINQ expression NOT problematic?

Question

Given the following code:

var strings = Enumerable.Range(0, 100).Select(i => i.ToString());
int outValue = 0;
var someEnumerable = strings.Where(s => int.TryParse(s, out outValue))
                            .Select(s => outValue);
outValue = 3;
//enumerating over someEnumerable here shows ints from 0 to 99

I am able to see a "snapshot" of the out parameter for each iteration. Why does this work correctly instead of me seeing 100 3's (deferred execution) or 100 99's (access to modified closure)?

Answer 1

First you define a query, strings that knows how to generate a sequence of strings, when queried. Each time a value is asked for it will generate a new number and convert it to a string.

Then you declare a variable, outValue, and assign 0 to it.

Then you define a new query, someEnumerable, that knows how to, when asked for a value, get the next value from the query strings, try to parse the value and, if the value can be parsed, yields the value of outValue. Once again, we have defined a query that can do this, we have not actually done any of this.

You then set outValue to 3.

Then you ask someEnumerable for it's first value, you are asking the implementation of Select for its value. To compute that value it will ask the Where for its first value. The Where will ask strings. (We'll skip a few steps now.) The Where will get a 0. It will call the predicate on 0, specifically calling int.TryParse. A side effect of this is that outValue will be set to 0. TryParse returns true, so the item is yielded. Select then maps that value (the string 0) into a new value using its selector. The selector ignores the value and yields the value of outValue at that point in time, which is 0. Our foreach loop now does whatever with 0.

Now we ask someEnumerable for its second value, on the next iteration of the loop. It asks Select for a value, Select asks Where,Where asks strings, strings yields "1", Where calls the predicate, setting outValue to 1 as a side effect, Select yields the current value of outValue, which is 1. The foreach loop now does whatever with 1.

So the key point here is that due to the way in which Where and Select defer execution, performing their work only immediately when the values are needed, the side effect of the Where predicate ends up being called immediately before each projection in the Select. If you didn't defer execution, and instead performed all of the TryParse calls before any of the projections in Select, then you would see 100 for each value. We can actually simulate this easily enough. We can materialize the results of the Where into a collection, and then see the results of the Select be 100 repeated over and over:

var someEnumerable = strings.Where(s => int.TryParse(s, out outValue))
    .ToList()//eagerly evaluate the query up to this point
    .Select(s => outValue);

Having said all of that, the query that you have is not particularly good design. Whenever possible you should avoid queries that have side effects (such as your Where). The fact that the query both causes side effects, and observes the side effects that it creates, makes following all of this rather hard. The preferable design would be to rely on purely functional methods that aren't causing side effects. In this context the simplest way to do that is to create a method that tries to parse a string and returns an int?:

public static int? TryParse(string rawValue)
{
    int output;
    if (int.TryParse(rawValue, out output))
        return output;
    else
        return null;
}

This allows us to write:

var someEnumerable = from s in strings
    let n = TryParse(s)
    where n != null
    select n.Value;

Here there are no observable side effects in the query, nor is the query observing any external side effects. It makes the whole query far easier to reason about.

Answer 2

Because when you enumerate the value changes one at a time and changes the value of the variable on the fly. Due to the nature of LINQ the select for the first iteration is executed before the where for the second iteration. Basically this variable turns into a foreach loop variable of a kind.

This is what deferred execution buys us. Previous methods do not have to execute fully before the next method in the chain starts. One value moves through all the methods before the second goes in. This is very useful with methods like First or Take which stop the iteration early. Exceptions to the rule are methods that need to aggregate or sort like OrderBy (they need to look at all elements before finding out which is first). If you add an OrderBy before the Select the behavior will probably break.

Of course I wouldn't depend on this behavior in production code.

Answer 3

I don't understand what is odd for you?

If you write a loop on this enumerable like this

foreach (var i in someEnumerable)
{
     Console.WriteLine(outValue);
}

Because LINQ enumerates each where and select lazyly and yield each value, if you add ToArray

var someEnumerable = strings.Where(s => int.TryParse(s, out outValue))
                        .Select(s => outValue).ToArray();

Than in the loop you will see 99 s

Edit

The below code will print 99 s

var strings = Enumerable.Range(0, 100).Select(i => i.ToString());
int outValue = 0;
var someEnumerable = strings.Where(s => int.TryParse(s, out outValue))
                                    .Select(s => outValue).ToArray();
//outValue = 3;


foreach (var i in someEnumerable)
{
    Console.WriteLine(outValue);
}