[layerindex-web][PATCH 1/5] layerindex/views.py: fix StatsView statistics page timeout

Tim Orling <[email protected]>
Newsgroups org.yoctoproject.lists.yocto-patches
Message-ID <[email protected]>
The per-branch statistics query used a single ORM queryset with five
Count(..., distinct=True) annotations spanning deep reverse FK chains
(layerbranch__recipe, layerbranch__bbclass, etc.).  Django translates
this into one SQL query with multiple LEFT OUTER JOINs across all those
tables simultaneously.  At production data volumes (~541 layers, ~25k
recipes) the resulting query exhausts gunicorn worker memory/time,
causing a 504 Gateway Time-out.

Fix by iterating over each visible branch individually and issuing simple
COUNT queries per table.  This replaces one huge cross-join with N×5
small indexed WHERE-clause queries that are far cheaper and independently
cacheable.

The template is unchanged; Django's template engine resolves dict keys
and object attributes identically with dot notation.

[YOCTO #15391]

AI-Generated: Claude Cowork Sonnet 4.6
Signed-off-by: Tim Orling <[email protected]>
---
 layerindex/views.py | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/layerindex/views.py b/layerindex/views.py
index 3cf91d2..23bafd0 100644
--- a/layerindex/views.py
+++ b/layerindex/views.py
@@ -1522,12 +1522,23 @@ class StatsView(TemplateView):
         context['class_count_distinct'] = BBClass.objects.values('name').distinct().count()
         context['machine_count_distinct'] = Machine.objects.values('name').distinct().count()
         context['distro_count_distinct'] = Distro.objects.values('name').distinct().count()
-        context['perbranch'] = Branch.objects.filter(hidden=False).order_by('sort_priority').annotate(
-                layer_count=Count('layerbranch', distinct=True),
-                recipe_count=Count('layerbranch__recipe', distinct=True),
-                class_count=Count('layerbranch__bbclass', distinct=True),
-                machine_count=Count('layerbranch__machine', distinct=True),
-                distro_count=Count('layerbranch__distro', distinct=True))
+        # Compute per-branch counts with individual queries rather than a single
+        # annotated queryset. The multi-Count(distinct=True) approach generates one
+        # large SQL query with many LEFT JOINs that causes a gunicorn worker timeout
+        # at production data volumes. Simple per-branch COUNT queries are far cheaper.
+        # See: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15391
+        perbranch = []
+        for branch in Branch.objects.filter(hidden=False).order_by('sort_priority'):
+            perbranch.append({
+                'name': branch.name,
+                'updates_enabled': branch.updates_enabled,
+                'layer_count': LayerBranch.objects.filter(branch=branch).count(),
+                'recipe_count': Recipe.objects.filter(layerbranch__branch=branch).count(),
+                'class_count': BBClass.objects.filter(layerbranch__branch=branch).count(),
+                'machine_count': Machine.objects.filter(layerbranch__branch=branch).count(),
+                'distro_count': Distro.objects.filter(layerbranch__branch=branch).count(),
+            })
+        context['perbranch'] = perbranch
         return context
 
 
-- 
2.54.0
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.