Home > database >  Django Shell indexing Queryset on any position returns the exact same result - DB Lazy Evaluation
Django Shell indexing Queryset on any position returns the exact same result - DB Lazy Evaluation

Time:03-13

This isn't maybe as much of a question as it is a confirmation and request for more insight.

When running a QuerySet request in python shell something like this:

qs = MyModel.objects.all() or qs = MyModel.objects.get_queryset() or qs = MyModel.objects.filter(field='something')

then followed by anything like this: qs[0] or qs[1] or qs[any_valid_int] it will always return the DB entry in position 0 of the returned QuerySet no matter what index you use. This mad me scratch my head at first. Now I understand QuerySets are generators in Django (?) and are evaluated lazy as documented here: Django Doc: Querysets Are Lazy

When you do anything (in the shell) that evaluates the returned qs like for example len(qs) and then use indexing again it will suddenly return the correct DB entries with qs[any_valid_int] behaving exactly as expected. I believe this corresponds with this Django bug report question from 6 years ago as well.

So am I on the right track here assuming that this has to do with lazy evaluation? And what is the best/quick way in Django shell to just get a QuerySet and directly index the one you want for testing without having to use something like len(qs) first? Is that even possible to use direct indexing like that or do you have iterate through them / filter the QuerySet further etc.? Thanks

CodePudding user response:

So am I on the right track here assuming that this has to do with lazy evaluation?

Yes. If you have an unevaluated queryset qs, and you query with qs[i], it will make a query that looks like:

SELECT …
FROM …
WHERE …
LIMIT 1 OFFSET i

But if you do not specify a ORDER BY …, then the database can return records in any order, and thus the item at position 0 in the first query, can be at position 3 in the second query, which thus can result in the fact that qs[0] and qs[1] return the same item.

If you however use len(qs), list(qs), or anything that loads the data into the queryset, then qs[0] will not make an extra query to the database: it already has the records in the cache, so that means it will return the item at a specific location in the cache of the queryset, and since the records then have been returned in a certain order, regardless what that order is, qs[0] and qs[1] are thus different records returned by the database.

  • Related