Angular: Increase Query Loading Time in Firebase Database-CodePudding

I have an angular app where i am querying my firebase database as below:

constructor() {
  this.getData();
}

getData() {
this.projectSubscription$ = this.dataService.getAllProjects()
    .pipe(
      map((projects: any) => 
        projects.map(sc=> ({ key: sc.key, ...sc.payload.val() }))
      ),
      switchMap(appUsers => this.dataService.getAllAppUsers()
        .pipe(
          map((admins: any)  => 
          appUsers.map(proj =>{
              const match: any = admins.find(admin => admin.key === proj.admin);
              return {...proj, imgArr: this.mapObjectToArray(proj.images), adminUser: match.payload.val()}
            })
          )
        )
      )
    ).subscribe(res => {
      this.loadingState = false;
      this.projects = res.reverse();
    });
}

mapObjectToArray = (obj: any) => {
    const mappedDatas = [];
    for (const key in obj) {
        if (Object.prototype.hasOwnProperty.call(obj, key)) {
          mappedDatas.push({ ...obj[key], id: key }); 
        }
    }
    return mappedDatas;
};

And here is what I am querying inside dataService:

getAllProjects() {
    return this.afDatabase.list('/projects/', ref=>ref.orderByChild('createdAt')).snapshotChanges();
  }

getAllAppUsers() {
    return this.afDatabase.list('/appUsers/', ref=>ref.orderByChild('name')).snapshotChanges();
  }

The problem I am facing with this is I have 400 rows of data which I am trying to load and it is taking around 30seconds to load which is insanely high. Any idea how can I query this in a faster time?

CodePudding user response：

We have no way to know whether the 30s is reasonable, as that depends on the amount of data loaded, the connection latency and bandwidth of the client, and more factors we can't know/control.

But one thing to keep in mind is that you're performing 400 queries to get the users of each individual app, which is likely not great for performance.

Things you could consider:

Pre-load all the users once, and then use that list for each project.
Duplicate the name of each user into each project, so that you don't need to join any data at all.

If you come from a background in relational databases the latter may be counterintuitive, but it is actually very common in NoSQL data modeling and is one of the reasons NoSQL databases scale so well.

CodePudding user response：

I propose 3 solutions.

1. Pagination

Instead of returning all those documents on app load, limit them to just 10 and keep record of the last one. Then display the 10 (or any arbitrary base number)

Then make the UI in such a way that the user has to click next or when the user scrolls, you fetch the next set based on the previous last document's field's info.

I'm supposing you need to display all the fetched data in some table or list so having the UI paginate the data should make sense.

2. Loader

Show some loader UI on website load. Then when all the documents have fetched, you hide the loader and show the data as you want. You can use some custom stuff for loader, or choose from any of the abundant libraries out there, or use mat-progress-spinner from Angular Material

3. onCall Cloud Function

What if you try getting them through an onCall cloud function? It night be faster because it's just one request that the app will make and Firebase's Cloud Functions are very fast within Google's data centers.

Given that the user's network might be slow to iterate the documents but the cloud function will return all at once and that might give you what you want.

I guess you could go for this option only if you really really need to display all that data at once on website load.

... Note on cost

Fetching 400 or more documents every time a given website loads might be expensive. It'll be expensive if the website is visited very frequently by very many users. Firebase cost will increase as you are charged per document read too.

Check to see if you could optimise the data structure to avoid fetching this much.

This doesn't apply to you if this some admin dashboard or if fetching all users like this is done rarely making cost to not be high in that case.