Our Mongo replication servers experienced a crash because of to many traffic and a query that wasn't optimized. This all has been solved and everything is working correctly, we just have one problem we can't figure out.
While our primary server crashed, some items got added. That went ok, but the weird thing is that these items that where added in the crash window are now returned double in our aggregate queries.
If I'm going a find() query, it doesn't show up. How is this even possible?
Here's our aggregate query: `
[
{
'$match': { is_active: true, is_removed: { $ne: true }, _id: { $in: ['390195122352164864'] } }
},
{ '$sort': { 'list_ranking.default': -1 } },
{ '$limit': 48 },
{
'$lookup': {
from: 'items',
localField: 'parent_id',
foreignField: '_id',
as: 'parent'
}
},
{
'$lookup': {
from: 'items_type',
localField: 'item_type',
foreignField: 'key',
as: 'type'
}
},
{
'$lookup': {
from: 'items_rarity',
localField: 'item_rarity',
foreignField: 'key',
as: 'rarity'
}
},
{
'$lookup': {
from: 'items_serie',
localField: 'item_serie',
foreignField: 'key',
as: 'serie'
}
},
{
'$lookup': {
from: 'items_set',
localField: 'item_set',
foreignField: 'key',
as: 'sets'
}
},
{
'$lookup': {
from: 'items_introduction',
localField: 'item_introduction',
foreignField: 'key',
as: 'introduction'
}
},
{
'$lookup': {
from: 'items_background',
localField: 'item_background',
foreignField: 'key',
as: 'background'
}
},
{
'$lookup': {
from: 'items_set',
localField: 'parent.item_set',
foreignField: 'key',
as: 'parentSets'
}
},
{ '$unwind': { path: '$parent', preserveNullAndEmptyArrays: true } },
{ '$unwind': { path: '$type', preserveNullAndEmptyArrays: true } },
{ '$unwind': { path: '$rarity', preserveNullAndEmptyArrays: true } },
{ '$unwind': { path: '$serie', preserveNullAndEmptyArrays: true } },
{ '$unwind': { path: '$sets', preserveNullAndEmptyArrays: true } },
{
'$unwind': { path: '$introduction', preserveNullAndEmptyArrays: true }
},
{
'$unwind': { path: '$background', preserveNullAndEmptyArrays: true }
},
{
'$unwind': { path: '$parentSets', preserveNullAndEmptyArrays: true }
},
{
'$project': {
slug: 1,
'parent.slug': 1,
name: 1,
'parent.name': 1,
description: 1,
'parent.description': 1,
key: 1,
'parent.key': 1,
icon: 1,
'parent.icon': 1,
featured: 1,
'parent.featured': 1,
media_id: 1,
'parent.media_id': 1,
media_type: 1,
'parent.media_type': 1,
media_uploaded_at: 1,
'parent.media_uploaded_at': 1,
media_processed_at: 1,
'parent.media_processed_at': 1,
list_order: 1,
'parent.list_order': 1,
list_ranking: 1,
'parent.list_ranking': 1,
rating_good: 1,
'parent.rating_good': 1,
rating_bad: 1,
'parent.rating_bad': 1,
estimated_available_combos: 1,
'parent.estimated_available_combos': 1,
obtained_type: 1,
'parent.obtained_type': 1,
obtained_value: 1,
'parent.obtained_value': 1,
v3_itemid: 1,
'parent.v3_itemid': 1,
v3_itemkey: 1,
'parent.v3_itemkey': 1,
v3_mediaid: 1,
'parent.v3_mediaid': 1,
is_active: 1,
'parent.is_active': 1,
is_released: 1,
'parent.is_released': 1,
is_removed: 1,
'parent.is_removed': 1,
modified_at: 1,
'parent.modified_at': 1,
created_at: 1,
'parent.created_at': 1,
'type.slug': 1,
'type.name': 1,
'type.key': 1,
'rarity.slug': 1,
'rarity.name': 1,
'rarity.key': 1,
'rarity.color': 1,
'serie.slug': 1,
'serie.name': 1,
'serie.key': 1,
'serie.color': 1,
'sets.slug': 1,
'sets.name': 1,
'sets.key': 1,
'sets.is_active': 1,
'parentSets.slug': 1,
'parentSets.name': 1,
'parentSets.key': 1,
'parentSets.is_active': 1,
'background.slug': 1,
'background.name': 1,
'background.key': 1,
'introduction.slug': 1,
'introduction.name': 1,
'introduction.key': 1,
'introduction.chapter': 1,
'introduction.season': 1,
'parent._id': 1
}
}
]
`
And this is what we're getting back: `
[
{
_id: '390195122352164864',
slug: 'ffc-neymar-jr',
name: 'FFC Neymar Jr',
description: 'Knows a thing or two.',
key: 'character_redoasisgooseberry',
icon: 'b74a4677-e2ba-4f25-9e92-25756dafc9d2',
featured: 'b422fadb-6960-4122-9d3d-37cbc06501e8',
media_id: '04ddd281-309e-40e5-811d-767e52d84847',
media_type: 'video/mp4',
media_uploaded_at: null,
media_processed_at: 2022-12-02T07:34:43.371Z,
obtained_type: 'vbucks',
obtained_value: '1200',
rating_good: 101,
rating_bad: 6,
list_order: 6396,
list_ranking: {
default: 6807,
last_1_hr: 534,
last_24_hrs: 6807,
last_7_days: 7682
},
estimated_available_combos: 6,
is_released: true,
is_active: true,
is_removed: false,
modified_at: null,
created_at: 2022-11-30T17:36:06.643Z,
type: { slug: 'outfit', name: 'Outfit', key: 'AthenaCharacter' },
rarity: {
slug: 'rare',
name: 'Rare',
key: 'EFortRarity::Rare',
color: '28C4F2'
},
serie: {
slug: 'icon',
name: 'Icon Series',
key: 'CreatorCollabSeries',
color: '5DD6EA'
},
sets: {
is_active: false,
slug: 'set01',
name: 'Fortnite Football Club',
key: 'SphereKickGroup'
},
introduction: {
slug: 'chapter-3-season-4',
name: 'Introduced in Chapter 3, Season 4.',
key: 22,
chapter: '3',
season: '4'
}
},
{
_id: '390195122352164864',
slug: 'ffc-neymar-jr',
name: 'FFC Neymar Jr',
description: 'Knows a thing or two.',
key: 'character_redoasisgooseberry',
icon: 'b74a4677-e2ba-4f25-9e92-25756dafc9d2',
featured: 'b422fadb-6960-4122-9d3d-37cbc06501e8',
media_id: '04ddd281-309e-40e5-811d-767e52d84847',
media_type: 'video/mp4',
media_uploaded_at: null,
media_processed_at: 2022-12-02T07:34:43.371Z,
obtained_type: 'vbucks',
obtained_value: '1200',
rating_good: 101,
rating_bad: 6,
list_order: 6396,
list_ranking: {
default: 6807,
last_1_hr: 534,
last_24_hrs: 6807,
last_7_days: 7682
},
estimated_available_combos: 6,
is_released: true,
is_active: true,
is_removed: false,
modified_at: null,
created_at: 2022-11-30T17:36:06.643Z,
type: { slug: 'outfit', name: 'Outfit', key: 'AthenaCharacter' },
rarity: {
slug: 'rare',
name: 'Rare',
key: 'EFortRarity::Rare',
color: '28C4F2'
},
serie: {
slug: 'icon',
name: 'Icon Series',
key: 'CreatorCollabSeries',
color: '5DD6EA'
},
sets: {
slug: 'set01',
name: 'Fortnite Football Club',
key: 'SphereKickGroup',
is_active: false
},
introduction: {
slug: 'chapter-3-season-4',
name: 'Introduced in Chapter 3, Season 4.',
key: 22,
chapter: '3',
season: '4'
}
}
]
`
How is this possible?
Thanks, Sam
Tried to rebuild indexes, restarted all Mongo instances, restarted the api servers, removed the items and added them back.
CodePudding user response：
This is the expected outcome when using $unwind as if there is more than one matching document it will essentially duplicate the whole document, replacing the array with each item in your array as an object.
If you're only expecting a single document to be returned it sounds like the issue is caused by there being more than one matching document from a $lookup.
It could be caused by any one of the lookups, but from the output you posted the two sets are slightly different so I would start by searching items_set to see if there is more than one matching document. If there is then deleting the duplicate should solve the issue.
db.items_set.find({ key: 'SphereKickGroup' })