Avoiding await in foreach loop

Question

I am trying to optimize this code to decrease the time taken to complete the forloop. In this case, CreateNotification() takes a long time and using async await does not improve performance as each asynchronous call is being awaited. I would like to use Task.WhenAll() to optimize the code. How can I do this?

foreach (var notification in notificationsInput.Notifications)
{
  try
  {
    var result = await CreateNotification(notification);
    notification.Result = result;          
  }
  catch (Exception exception)
  {
    notification.Result = null;
  }
  notifications.Add(notification);
}

You can probably use [Parallel.ForEach](https://msdn.microsoft.com/en-us/library/system.threading.tasks.parallel.foreach.aspx) instead. — Leandro, Jun 17 '18 at 05:25
CreateNotification is a lambda function. CreateNotification calls an endpoint to create a notification. i have edited the code to remove context — Ajit Goel, Jun 17 '18 at 05:25
@Leandro That doesn't really play so nicely with async methods :) — ProgrammingLlama, Jun 17 '18 at 05:25
https://stackoverflow.com/questions/15136542/parallel-foreach-with-asynchronous-lambda — Leandro, Jun 17 '18 at 05:38
just curious, if number of notifications is not big simply using thread pool is good idea or not! — Amit, Jun 17 '18 at 06:48

Douglas · Accepted Answer · 2018-06-17T09:04:01.460

8

You can call Select on the collection whose elements you want to process in parallel, passing an asynchronous delegate to it. This asynchronous delegate would return a Task for each element that's processed, so you could then call Task.WhenAll on all these tasks. The pattern is like so:

var tasks = collection.Select(async (x) => await ProcessAsync(x));
await Task.WhenAll(tasks);

For your example:

var tasks = notificationsInput.Notifications.Select(async (notification) =>
{
    try
    {
        var result = await CreateNotification(notification);
        notification.Result = result;          
    }
    catch (Exception exception)
    {
        notification.Result = null;
    }
});
await Task.WhenAll(tasks);

This assumes that CreateNotification is thread-safe.

edited Jun 17 '18 at 09:04

answered Jun 17 '18 at 08:56

Douglas

53,759
13
140
188

Does this execute in parallel? I think this still executes asynchronously. – David B Jun 30 '19 at 15:15
@DavidB: The asynchronous tasks will execute in parallel. At the `await` keyword, control will be returned to the caller; thus, the `Select` would continue iterating over the rest of the notifications whilst the previous `CreateNotification` tasks are still in flight. – Douglas Dec 15 '19 at 17:35

TheGeneral · Answer 2 · 2018-06-17T08:02:18.843

Update

You will need to install DataFlow to use this solution

https://www.nuget.org/packages/System.Threading.Tasks.Dataflow/

Depending on what CreateNotification is and whether you want to run this in parallel.

You could use a DataFlow ActionBlock, it will give you the best of both worlds if this is IO bound or Mix IO/CPU bound operations and let you run async and in parallel

public static async Task DoWorkLoads(NotificationsInput notificationsInput)
{
   var options = new ExecutionDataflowBlockOptions
                     {
                        MaxDegreeOfParallelism = 50
                     };

   var block = new ActionBlock<Something>(MyMethodAsync, options);

   foreach (var notification in notificationsInput.Notifications)
      block.Post(notification);

   block.Complete();
   await block.Completion;

}

...

public async Task MyMethodAsync(Notification notification)
{       
     var result = await CreateNotification(notification);
     notification.Result = result;    
}

Add pepper and salt to taste.

score 1 · Answer 3 · answered Jun 17 '18 at 05:35

I think this ought to be equivalent to your code:

var notifications = new ConcurrentBag<Notification>();
var tasks = new List<Task>();
foreach (var notification in notificationsInput.Notifications)
{
    var task = CreateNotification(notification)
                    .ContinueWith(t =>
                    {
                        if (t.Exception != null)
                        {
                            notification.Result = null;
                        }
                        else
                        {
                            notification.Result = t.Result;
                        }
                        notifications.Add(notification);
                    });
    tasks.Add(task);
}
await Task.WhenAll(tasks);

.ContinueWith( will receive the completed/failed task from CreateNotification(, and is itself a task. We add the ContinueWith task to a list and use that in the WhenAll(.

I'm using a ConcurrentBag for notifications so that you can add from multiple threads safely. If you want to turn this into a regular list, you can call var regularListNotifications = notifications.ToList(); (assuming you have a using for LINQ).

Avoiding await in foreach loop

3 Answers3