Django bulk load-CodePudding

I want to load my page faster in my Django project, so I know there has the "Bulk" option.

views.py:

class HRListView(Budgetlevel2Mixin, BudgetMixin, LoginRequiredMixin, ListView):
    model = HR
    template_name = "budget/hr_list.html"

models.py:

class HR(models.Model):
year_scenario_version = models.ForeignKey(
measure_name = models.ForeignKey(
employee_type = models.ForeignKey(
employee_level = models.ForeignKey(
subsidiary = models.ForeignKey(
department = models.ForeignKey(
currency = models.ForeignKey(

hr_list.html:

{% block thead %}
<tr>
  <th>ID</th>
  <th>HR Year Scenario</th>
  <th>Measure Name</th>
  <th>Employee Type</th>
  <th>Employee Level</th>
  <th>Subsidiary</th>
  <th>Department</th>
</tr>
{% endblock %}


{% block tbody %}
  {% for q in object_list %}
  <tr>
    <td>{{ q.id }}</td>
    <td>{{ q.year_scenario_version }}</td>
    <td>{{ q.measure_name }}</td>
    <td>{{ q.employee_type }}</td>
    <td>{{ q.employee_level }}</td>
    <td>{{ q.subsidiary }}</td>
    <td>{{ q.department }}</td>
  </tr>
  {% endfor %}
{% endblock %}

How can I improve the page loading time? It takes almost 10 sec for full loading, something around 1200 records.

Thanks a lot!

CodePudding user response：

Since all your HR fields are foreign keys and you did not prefetch them, accessing each field on each row is a BD hit.

Try adding to your ListView:

def get_queryset():
    return HR.objects.select_related("year_scenario_version", "measure_name", "employee_type", "employee_level", "subsidiary", "departement")

CodePudding user response：

You are indeed correct that bulk operations can speed up things considerably. You should:

Prefetch related fields [Django-doc] Use pagination, so that you do not load 1200 objects in memory at once Make use of database indices to speed up lookups Cache the page, so that repeated visits are faster Reduce the amount of logic in the template

So you can modify the view to: class HRListView(Budgetlevel2Mixin, BudgetMixin, LoginRequiredMixin, ListView): model = HR template_name = "budget/hr_list.html"

def get_queryset(self):
    return super().get_queryset().prefetch_related(
        'year_scenario_version', 'measure_name', 'employee_type',
        'employee_level', 'subsidiary', 'department', 'currency'
    )

and use pagination, for example with Django Paginator in the template. Furthermore you can make sure you have indices on:

HR.year_scenario_version_id HR.measure_name_id etc.

to speed up the lookups. Finally you can implement caching, for example with the cache template tag, or with django-cacheback, or by implementing cache headers.