I'm kinda surprised I haven't run into this situation before. I need to execute two web requests in series (because the second call depended on information from the first) against an array of values. Naturally, I wanted the array to perform in parallel, but how to get the second call to either execute automatically after the first call is done or force execution if requested before the second call is made.

Here's the pattern I came up with:

    Public Class TwoPartRequest
        Private req1, req2 As HttpWebRequest
        Private res1, res2 As IAsyncResult
        Private run1, run2 As Integer
        Private coolResult As CoolObject

        Public Sub New(ByVal url As String)
            req1 = WebRequest.Create(url)
            req1 = req1.BeginGetResponse(AddressOf Part1Complete, Nothing)
        End Sub

        Private Sub Part1Complete(ByVal ar As IAsyncResult)
            If System.Threading.Interlocked.Increment(run1) = 1 Then
                Using resp As HttpWebResponse = req1.EndGetResponse(ar)
                    Dim url As String = FigureOutNextUrlUsingResponse(resp)
                    req2 = WebRequest.Create(url)
                    res1 = req2.BeginGetResponse(AddressOf Part2Complete, Nothing)
                End Using
            End If
        End Sub

        Private Sub Part2Complete(ByVal ar As IAsyncResult)
            If System.Threading.Interlocked.Increment(run2) = 1 Then
                Using resp As HttpWebResponse = req2.EndGetResponse(ar)
                    coolResult = BuildResultUsingResponse(resp)
                End Using
            End If
        End Sub

        Public Function Result() As CoolObject
            Part1Complete(res1)
            Part2Complete(res2)
            Return coolResult
        End Function
    End Class

To use this, I have a list of these objects that I new up in a loop. In a second loop, I get all the results. If the target server can process requests in parallel well the number of iterations on the loop shouldn't matter.

Clearly, one could generalize the heck out of this, but I happy with it working.