How to asynchronously make a web request without a callback

Question

I have a service that talks to a 3rd party on the internet, if i want this to scale i don't want my threads to be blocked waiting for the response when they could be handling new requests. One way around this is to use the httpRequest.BeginGetResponse method and pass in a callback. The issue with this is this method returns immediately and i have nothing to give back to the caller of my service. Once in the callback i cant do anything useful.

What i would like is something like this

public string CallServiceNonBlocking(string url) {

   var httpRequest = (HttpWebRequest)WebRequest.Create(url);

   HttpResponse res = httpRequest.FetchNonBlocking(); //this does the work but my thread should  be returned to the asp.net thread pool to do other work rather than blocking here

   return res.GetValue(); 

}

Note i don't have .Net 4.5 available to me only .NET 4. But for reference can i see a solution for both versions?

Your CallServiceNonBlocking would have to take a completion handler/callback for when your fetch is complete. You *can* use the TPL in .NET 4.0. — vcsjones, Mar 28 '14 at 15:01
You might want to check out: http://msdn.microsoft.com/en-us/library/dd997423(v=vs.100).aspx — PCG, Mar 28 '14 at 15:02
`i don't want my threads to be blocked waiting for the response when they could be handling new requests` Maybe just increase the worker thread count? That's a very simple solution up to a point. — usr, Mar 28 '14 at 16:06

score 2 · Answer 1 · edited May 23 '17 at 10:26

Coding this with .NET 4.5 and C# 5.0 async/await is very easy.

You can make it work asynchronously with .NET 4.0 / C# 4.0 too, if you have to. You'd need to derive your controller from AsyncController and use AsyncManager. The basic steps are described in "Using an Asynchronous Controller in ASP.NET MVC".

In this light, your code may look like this (untested):

static public Task<WebResponse> GetResponseTapAsync(this WebRequest request)
{
    return Task.Factory.FromAsync(
         (asyncCallback, state) =>
             request.BeginGetResponse(asyncCallback, state),
         (asyncResult) =>
             request.EndGetResponse(asyncResult), null);
}

// ...

public class YourController : AsyncController
{
    public void YourMethodAsyc(string url)
    {
        AsyncManager.OutstandingOperations.Increment();

        var request = (HttpWebRequest)WebRequest.Create(url);
        request.GetResponseTapAsync().ContinueWith(responseTask =>
        {
            try
            {
                var stream = responseTask.Result.GetResponseStream();
                using (var streamReader = new StreamReader(stream))
                {
                    // still blocking here, see notes below
                    var data = streamReader.ReadToEnd();

                    AsyncManager.Parameters["data"] = data;
                }
            }
            finally
            {
                AsyncManager.OutstandingOperations.Decrement();
            }
        }, TaskScheduler.FromCurrentSynchronizationContext());
    }

    public ActionResult YourMethodCompleted(string data)
    {
        return View("Data", new ViewModel
        {
            Data = data
        });
    }
}

You could take it further and implement ReadToEndAsync (as it is absent in .NET 4.0), but you won't be able to use using. Check this for some more details.

Ideally, if you need to target ASP.NET MVC with .NET 4.0, but develop with VS2012+ in C# 5.0, you still can use async/await, Microsoft provides Microsoft.Bcl.Async library for that. Then your code might look like this:

public class YourController : AsyncController
{
    async Task<string> YourMethodAsyncImpl(string url)
    {
        var request = (HttpWebRequest)WebRequest.Create(url);
        using (var response = await request.GetResponseAsync()
        using (var streamReader = new StreamReader(response.GetResponseStream())
            return await streamReader.ReadToEndAsync();
    }

    public void YourMethodAsyc(string url)
    {
        AsyncManager.OutstandingOperations.Increment();

        YourMethodAsyncImpl(url).ContinueWith(resultTask =>
        {
            try
            {
                AsyncManager.Parameters["data"] = resultTask.Result;
            }
            finally
            {
                AsyncManager.OutstandingOperations.Decrement();
            }
        }, TaskScheduler.FromCurrentSynchronizationContext());
    }

    public ActionResult YourMethodCompleted(string data)
    {
        return View("Data", new ViewModel
        {
            Data = data
        });
    }
}

score 0 · Answer 2 · answered Mar 29 '14 at 06:23

I'd use Microsoft's Reactive Framework for this. (NuGet "Rx-Main")

It's then super easy.

Define your function like this:

public IObservable<string> CallServiceNonBlocking(string url)
{
    return Observable.Start(() =>
    {
        var httpRequest = (HttpWebRequest)WebRequest.Create(url);
        HttpResponse res = httpRequest.FetchBlocking();
        return res.GetValue(); 
    });
}

Then call it like this:

CallServiceNonBlocking(url)
    .Subscribe(response =>
    {
        /* handle response here */
    });

It's easy to use an ObserveOn to make the subscription run on the UI thread, or current context, etc.

I don't think this would help the OP's web service to scale better, as it's still blocking at least one pool thread for `FetchBlocking`. — noseratio, Mar 29 '14 at 06:58

How to asynchronously make a web request without a callback

2 Answers2