What is the best way to modify a list in a 'foreach' loop?

Question

A new feature in C# / .NET 4.0 is that you can change your enumerable in a foreach without getting the exception. See Paul Jackson's blog entry An Interesting Side-Effect of Concurrency: Removing Items from a Collection While Enumerating for information on this change.

What is the best way to do the following?

foreach(var item in Enumerable)
{
    foreach(var item2 in item.Enumerable)
    {
        item.Add(new item2)
    }
}

Usually I use an IList as a cache/buffer until the end of the foreach, but is there better way?

Answer 1

The collection used in foreach is immutable. This is very much by design.

As it says on MSDN:

The foreach statement is used to iterate through the collection to get the information that you want, but can not be used to add or remove items from the source collection to avoid unpredictable side effects. If you need to add or remove items from the source collection, use a for loop.

The post in the link provided by Poko indicates that this is allowed in the new concurrent collections.

Answer 2

Make a copy of the enumeration, using an IEnumerable extension method in this case, and enumerate over it. This would add a copy of every element in every inner enumerable to that enumeration.

foreach(var item in Enumerable)
{
    foreach(var item2 in item.Enumerable.ToList())
    {
        item.Add(item2)
    }
}

Answer 3

To illustrate Nippysaurus's answer: If you are going to add the new items to the list and want to process the newly added items too during the same enumeration then you can just use for loop instead of foreach loop, problem solved :)

var list = new List<YourData>();
... populate the list ...

//foreach (var entryToProcess in list)
for (int i = 0; i < list.Count; i++)
{
    var entryToProcess = list[i];

    var resultOfProcessing = DoStuffToEntry(entryToProcess);

    if (... condition ...)
        list.Add(new YourData(...));
}

For runnable example:

void Main()
{
    var list = new List<int>();
    for (int i = 0; i < 10; i++)
        list.Add(i);

    //foreach (var entry in list)
    for (int i = 0; i < list.Count; i++)
    {
        var entry = list[i];
        if (entry % 2 == 0)
            list.Add(entry + 1);

        Console.Write(entry + ", ");
    }

    Console.Write(list);
}

Output of last example:

0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 1, 3, 5, 7, 9,

List (15 items)
0
1
2
3
4
5
6
7
8
9
1
3
5
7
9

Answer 4

As mentioned, but with a code sample:

foreach(var item in collection.ToArray())
    collection.Add(new Item...);

Answer 5

You should really use for() instead of foreach() in this case.

Answer 6

LINQ is very effective for juggling with collections.

Your types and structure are unclear to me, but I will try to fit your example to the best of my ability.

From your code it appears that, for each item, you are adding to that item everything from its own 'Enumerable' property. This is very simple:

foreach (var item in Enumerable)
{
    item = item.AddRange(item.Enumerable));
}

As a more general example, let's say we want to iterate a collection and remove items where a certain condition is true. Avoiding foreach, using LINQ:

myCollection = myCollection.Where(item => item.ShouldBeKept);

Add an item based on each existing item? No problem:

myCollection = myCollection.Concat(myCollection.Select(item => new Item(item.SomeProp)));

Answer 7

You can't change the enumerable collection while it is being enumerated, so you will have to make your changes before or after enumerating.

The for loop is a nice alternative, but if your IEnumerable collection does not implement ICollection, it is not possible.

Either:

1) Copy collection first. Enumerate the copied collection and change the original collection during the enumeration. (@tvanfosson)

or

2) Keep a list of changes and commit them after the enumeration.

Answer 8

To add to Timo's answer LINQ can be used like this as well:

items = items.Select(i => {

     ...
     //perform some logic adding / updating.

     return i / return new Item();
     ...

     //To remove an item simply have logic to return null.

     //Then attach the Where to filter out nulls

     return null;
     ...


}).Where(i => i != null);

Answer 9

Here's how you can do that (quick and dirty solution. If you really need this kind of behavior, you should either reconsider your design or override all IList<T> members and aggregate the source list):

using System;
using System.Collections.Generic;

namespace ConsoleApplication3
{
    public class ModifiableList<T> : List<T>
    {
        private readonly IList<T> pendingAdditions = new List<T>();
        private int activeEnumerators = 0;

        public ModifiableList(IEnumerable<T> collection) : base(collection)
        {
        }

        public ModifiableList()
        {
        }

        public new void Add(T t)
        {
            if(activeEnumerators == 0)
                base.Add(t);
            else
                pendingAdditions.Add(t);
        }

        public new IEnumerator<T> GetEnumerator()
        {
            ++activeEnumerators;

            foreach(T t in ((IList<T>)this))
                yield return t;

            --activeEnumerators;

            AddRange(pendingAdditions);
            pendingAdditions.Clear();
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            ModifiableList<int> ints = new ModifiableList<int>(new int[] { 2, 4, 6, 8 });

            foreach(int i in ints)
                ints.Add(i * 2);

            foreach(int i in ints)
                Console.WriteLine(i * 2);
        }
    }
}

Answer 10

The best approach from a performance perspective is probably to use a one or two arrays. Copy the list to an array, do operations on the array, and then build a new list from the array. Accessing an array element is faster than accessing a list item, and conversions between a List<T> and a T[] can use a fast "bulk copy" operation which avoids the overhead associated accessing individual items.

For example, suppose you have a List<string> and wish to have every string in the list which starts with T be followed by an item "Boo", while every string that starts with "U" is dropped entirely. An optimal approach would probably be something like:

int srcPtr,destPtr;
string[] arr;

srcPtr = theList.Count;
arr = new string[srcPtr*2];
theList.CopyTo(arr, theList.Count); // Copy into second half of the array
destPtr = 0;
for (; srcPtr < arr.Length; srcPtr++)
{
  string st = arr[srcPtr];
  char ch = (st ?? "!")[0]; // Get first character of string, or "!" if empty
  if (ch != 'U')
    arr[destPtr++] = st;
  if (ch == 'T')
    arr[destPtr++] = "Boo";
}
if (destPtr > arr.Length/2) // More than half of dest. array is used
{
  theList = new List<String>(arr); // Adds extra elements
  if (destPtr != arr.Length)
    theList.RemoveRange(destPtr, arr.Length-destPtr); // Chop to proper length
}
else
{
  Array.Resize(ref arr, destPtr);
  theList = new List<String>(arr); // Adds extra elements
}

It would have been helpful if List<T> provided a method to construct a list from a portion of an array, but I'm unaware of any efficient method for doing so. Still, operations on arrays are pretty fast. Of note is the fact that adding and removing items from the list does not require "pushing" around other items; each item gets written directly to its appropriate spot in the array.

Answer 11

I have written one easy step, but because of this performance will be degraded

Here is my code snippet:-

for (int tempReg = 0; tempReg < reg.Matches(lines).Count; tempReg++)
                            {
                                foreach (Match match in reg.Matches(lines))
                                {
                                    var aStringBuilder = new StringBuilder(lines);
                                    aStringBuilder.Insert(startIndex, match.ToString().Replace(",", " ");
                                    lines[k] = aStringBuilder.ToString();
                                    tempReg = 0;
                                    break;
                                }
                            }