nested async and sync in Julia-CodePudding

I want to do multi Task (A,B) pairs which have no connections with each other. Each Task A include multi Task a. Task A has to be done before B start in the same one (A,B) pair. So it's like

@async for loop do multi (A,B)s
    @sync do one (A,B)
        @async for loop do Task A
            do Task a
        do Task B

how can I achieve this?

I have tried:

b = Vector{String}(undef, 6)
ta = @async for i in range(1, length(b))
    a = Vector{String}(undef, 6)
    @async for j in range(1, length(a)) # task A
        a[j] = "hello "
        sleep(1) # task a
    end

    #task B
    b[i] = prod(a) * "world!"
    sleep(1)

end
@time wait(ta)

fail: Task B does not wait for task A in the same pair.

b = Vector{String}(undef, 6)
ta = @async for i in range(1, length(b))
    a = Vector{String}(undef, 6)
    for j in range(1, length(a)) # task A
        @async begin
            a[j] = "hello "
            sleep(1) # task a
        end
    end

    #task B
    b[i] = prod(a) * "world!"
    sleep(1)

end
@time wait(ta)

fail: Task B does not wait for task A in the same pair.

b = Vector{String}(undef, 6)
ta = @async for i in range(1, length(b))
    a = Vector{String}(undef, 6)
    taA= @async for j in range(1, length(a)) # task A
        a[j] = "hello "
        sleep(1) # task a
    end
    wait(taA)

    #task B
    b[i] = prod(a) * "world!"
    sleep(1)

end
@time wait(ta)

fail: multi (A,B) pairs not async to each other.

b = Vector{String}(undef, 6)
ta = @async for i in range(1, length(b))
    a = Vector{String}(undef, 6)
    taA = @async for j in range(1, length(a)) # task A
        a[j] = "hello "
        sleep(1) # task a
    end

    while !istaskdone(taA)
        sleep(0.5)
    end

    b[i] = prod(a) * "world!"
    sleep(1)

end
@time wait(ta)

fail: multi (A,B) pairs not async to each other.

CodePudding user response：

If I understand your question right you want to have:

@sync for (A, B) in ABs
           @async begin
               @sync for a in A
                   @async do_task_A(a)
               end
               do_task_B(B)
           end
       end

For an example consider the function:

function do_task(t, x)
   sleep(1/x)
   println("Done $t : $x")
   flush(stdout)
end

Lets run it!

julia> @time @sync for (A, B) in [([1,2], 100), ([3,4], 101),([5,6], 102)]
           @async begin
               @sync for a in A
                   @async do_task(:A, a)
               end
               do_task(:B, B)
           end
       end
Done A : 6
Done A : 5
Done B : 102
Done A : 4
Done A : 3
Done B : 101
Done A : 2
Done A : 1
Done B : 100
  1.079950 seconds (20.25 k allocations: 1020.699 KiB, 3.52% compilation time)

You can see that tasks got executed asynchronously exactly in the order you requested.

CodePudding user response：

All the answer from @Przemyslaw Szufel, I just sum it up as a note:

@sync for loop do task A
    @async do task(a)

do task(B)

then Task B will wait until Task A finishes. And Multi Task a in Task A will be async to each other.

It is the building block for do multi Task a and then do Task B.

if you want to nest it, then just:

@sync for loop do Task (A,B)
    @async begin
        @sync for loop do task A
            @async do task(a)

        do task(B)
    end

do task C

use the @sync macro before the for loop and then use the @async macro to wrap the content in the for loop. Then the multi little tasks in the for loop will run async to each other, but the code after the for loop will wait until the for loop finishes.

example code from @Przemyslaw Szufel

function do_task(t, x)
    println("start $t : $x")
    flush(stdout)
    sleep(1 / x)
    println("Done $t : $x")
    flush(stdout)
end

@time @sync for (A, B) in [([1, 2], 100), ([3, 4], 101), ([5, 6], 102)]
    @async begin
        @sync for a in A
            @async do_task(:A, a)
        end
        do_task(:B, B)
    end
end