HyperKitty consumes all CPU time after removing a mail list

After removing a Mailman list, HyperKitty, the mail list archive manager, starts using all available CPU time. What’s going on?

Looking at the mailman-web log file (/var/log/mailman3/web/mailman-web.log) will give us some clues:

21:35:41 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. : Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django_q/cluster.py", line 421, in worker
    res = f(*task["args"], **task["kwargs"])
  File "/usr/lib/python3/dist-packages/hyperkitty/tasks.py", line 79, in _rebuild_mailinglist_cache_for_month
    mlist = MailingList.objects.get(name=mlist_name)
  File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 406, in get
    raise self.model.DoesNotExist(
hyperkitty.models.mailinglist.MailingList.DoesNotExist: MailingList matching query does not exist.

Mailman tries to rebuild the cache of messages of a specific month (_rebuild_mailinglist_cache_for_month). As the list does not exists anymore, it fails, and tries again. And again…

To try to get out of this loop, let’s look at the code:

vim +79 /usr/lib/python3/dist-packages/hyperkitty/tasks.py

The rebuild function looks for the mail list, but does not check for a possible error, like, may be a deleted mail list?

def _rebuild_mailinglist_cache_for_month(mlist_name, year, month):
    mlist = MailingList.objects.get(name=mlist_name)
    mlist.cached_values["participants_count_for_month"].rebuild(year, month)

Let’s add some error checking code:

def _rebuild_mailinglist_cache_for_month(mlist_name, year, month):
    try:
        mlist = MailingList.objects.get(name=mlist_name)
        mlist.cached_values["participants_count_for_month"].rebuild(year, month)
    except MailingList.DoesNotExist:
        log.warning(
            "Cannot rebuild the mailinglist cache: mailinglist %s does not exist.",
            mlist_name)
        return

Then, we need to restart the mailman-web service. And check again the log file:

WARNING 2022-07-10 10:20:31,902 15517 hyperkitty.tasks Cannot rebuild the mailinglist cache: mailinglist xxx@example.org does not exist.
10:20:32 [Q] INFO Process-1:1 processing [rebuild_mailinglist_cache_for_month]
10:20:32 [Q] INFO Processed [rebuild_mailinglist_cache_for_month]

The tasks starts, then sees that the list is missing and handle the case correctly. Then, HyperKitty starts the following task. Issue closed!