Need help to execute sequential api calls in for-loop-CodePudding

I am working with Angular and NodeJs (with Axios) using RxJs, and currently find it very challenging to achieve this task. I will explain my problem based on a scenario.

I have an array of objects like this, with possibly even more than 100 objects:

let objArr = [{name: 'john', id: '123', country: 'usa'}, {name: 'doe', id: '456', country: 'china'}....]

Then I have another 4-5 validation APIs that can be called for different params e.g. id, name and country based on each object:

api_1 = "validate_step_1/:id"
api_2 = "validate_step_2/:id/:name"
api_3 = "validate_step_3/:id/:country"
api_4 = "validate_step_4/:id:/:name/:country"

These API calls should strictly happen one after another in a sequential pattern, e.g. api_2 should only be called if api_1 returns true and so on.

What I want:

I would like to execute for-loop on the array that should run in parallel, and each object should then sequentially validate itself based on these 4 API calls. Something like sequential API calls based on each item in for-loop in parallel for all 100 objects.

Is this even possible? Also, any solutions to achieve this on the Node side are also welcomed.

What I tried

Right now I am using this method, but it's very slow, even sometimes resulting in timeout errors in Axios:

  of(...this.objArr).pipe(
    concatMap((obj: any) => this.service.api_1(id)),
    concatMap((obj: any) => this.service.api_2(id, name)),
    concatMap((obj: any) => this.service.api_3(id, country)),
    concatMap((obj: any) => this.service.api_4(id, name, country)),
    catchError( error => this.handleError(error))
  ).subscribe(
    success => {
      console.log("validation succeed", success);
    },
    errorData => {
      console.log("validation failure: ", errorData);
    }
  )

CodePudding user response：

Switch map your data array into a stream of parrallel pipes that join together.

const dataArray: myData[];

of(this.dataArray).pipe(
  // switch and listen to the returned pipe
  switchMap(arrayOfObjects => {
    // a normal map (not an RxJS map), returns an array of pipes
    const nParrallelPipes = arrayOfObjects.map(data =>
      const test1 = this.service.api_1(data.id);
      const test2 = this.service.api_2(data.id, data.name);
      const test3 = this.service.api_3(data.id, data.country);
      const test4 = this.service.api_4(data.id, data.name, data.country);
      return concat(test1, test2, test3, test4).pipe(
        catchError(error => this.handleError(error))
      )
    );
    // listen to all pipes in parallel, and only return when all pipes have completed
    return forkJoin(nParrallelPipes) // or combineLatest(parrallelCalls).pipe(take(1))
  }
).subscribe(
  success => {
    console.log("validation succeed", success);
  },
  errorData => {
    console.log("validation failure: ", errorData);
  }
)

nParrallelPipes contains n synchronous sequential validation pipes, where n is the amount of data in your initial array. ForkJoining them together (or combineLatest) will fire them off in parallel.

I hope that helps.

Happy coding.

CodePudding user response：

you can use forkJoin operator to call multiple endpoints in case you're only interested in the final response of all the endpoints and no more future emissions, it takes an array of observables and gives tou an array of responses then completes...something like this:

const obs1$ = this.service.api1;
const obs2$ = this.service.api2;
const subscription = forkJoin([obs1$,obs2$].subscribe(([response1, response2]) => console.log(response1, response2))

but in case you'll be expecting more emissions from one or more observables you can use combineLatest or preferably zip operator.