How to handle thousands of messages in a real-time group chat with Firestore?-CodePudding

I need some help with Firestore building a chat app. I've looked at the documentation but I couldn't find the answer I need.

I'm building a real-time chat (many-to-many) that must handle thousands of messages, and those messages can also be edited, deleted and undeleted. The main problem is that loading all the messages once (as Firebase suggests) and then manage them on the FrontEnd side freezes my frontend application for the huge amount of messages. I tried to do that with the pagination API but I got some edge cases e.g.

let last = null;

// Execute the first query saving the last doc
const ref = db
 .collection('chat')
 .orderBy('timestamp')
 .limit(10)
 .onSnapshot(({docs}) => {
    last = docs[docs.length - 1]
   // my logic
 })

// When needed I'll execute the second query starting from the last doc
// of the previous query
const ref = db
 .collection('chat')
 .orderBy('timestamp')
 .limit(10)
 .startAfter(last)
 .onSnapshot(({docs}) => {
    last = docs[docs.length - 1]
    // my logic
 })

If the last document returned by the first query is deleted, I get the update only from the first query, e.g.

[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9] // original docs from the first query
[10,11,12,13,14,15,16,17,18,19] // original docs from the second query

Deleting the document 9 triggers the update from the first query so on my frontend I will have the document 10 duplicated.

[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 10] // updated docs from the first query
[10,11,12,13,14,15,16,17,18,19] // original docs from the second query

I'm guessing that there are also other edge cases that I hadn't considered. Do you have some suggestion aboutn how can I handle this big amount of messages with Firebase? Is there any error in my approach with the pagination? Or maybe this is the wrong approach?

CodePudding user response：

Your approach is fine, but it turns out that pagination and realtime updates with a cursor based API lead to some tricky edge-cases.

You'd have to either remove the duplicate documents based on their ID and then have pages with different sizes, or update the starting point of the second query (and then later queries as you have more pages).

There are more such edge cases, which is one of the reasons the FirestorePagingAdapter in FirebaseUI doesn't handle realtime updates.