Django, Modelforms and Long Dropdowns

Lets say you want to make a page to create or update a model that has a foreign key to 10k other records. If you are not careful, this can have detrimental consequences. For example, without caching, your app will need to pull 10k records from the db to make the drop-down. This post is my notes for dealing with this problem.

Cache the Values in a Fat Model

The goal is to create a form for creating and updating MyModel2 (see below). This model has a foreign key to MyModel1. Make a classmethod in Model1 for getting the drop down query set from cache. Over-ride the save method to keep the cache up to date.

from django.db import models
from django.core.cache import cache

class MyModel1(models):
    """Model with 50k records, 10k active."""
    afield = models.CharField()
    is_active = models.BooleanField()

    def __unicode__(self):
        return self.afield

    @classmethod
    def get_active(cls):
        cache_key = 'model1_cache'
        qs = cache.get(cache_key)
        if not qs:
            qs = cls.objects.filter(is_active=True)
            cache.set(cache_key, qs)
        return qs

    def save(self, **kwargs):
        cache.delete('model1_cache')
        return super(MyModel1, self).save(**kwargs)


class MyModel2(models):
    fk = models.ForeignKey(MyModel1)
    field2 = models.CharField()

    @property
    def fk_is_readonly(self):
        if self.field2 == 'xyz':
            return True
        else:
            return False

Use JQuery Chosen in Your Template

A standard HTML select sucks when you have lots of items. You will need an enhanced widget. Recently I tried using jquery autocomplete, but it required too many hacks to get it to work. The worst problem was limiting the user to the provided choices, while at the same time triggering an AJAX call once a valid value was entered. It was doable, but hacky and fragile.

I settled on Jquery Chosen. It’s easy to install and use. However some hacks were required when I needed it to be read-only. The hacks are in the Django form.

Make the ModelForm

If you use the standard methods for making the model form, Django will pull all the values from the database. You don’t want to do that. Below is what you want. The second line make sure the query set is pulled from cache and only has active items.

What happens if you are updating a MyModel2 record and the associated MyModel1 became inactive? We handle that by adding that instance to the query set in the __init__ method.

Finally, I often run into the case where fk needs to be “read-only”. I tried using the chosen “disable” function, but it removes the field from the POST data. I handle this by making a custom widget that’s a HiddenInput with text.

class MyModel2Form(forms.ModelForm):
    fk = forms.ModelChoiceField(required=True, queryset=MyModel1.get_active())

    class Meta:
        model = MyModel2
        fields = ('fk', 'field2')

    def __init__(self, *args, **kwargs):
        super(MyModel2Form, self).__init__(*args, **kwargs)

        if self.instance.id:
            # Make sure the current value is in the queryset. If that value became inactive since the record was
            # created, then it will not be in the query set. Without it, if the user saves without changing the
            # value they will get a form error saying its an invalid choice.
            new_qs = self.fields['fk'].queryset.filter(id=self.instance.fk.id)
            if not new_qs:
                self.fields['fk'].queryset = MyModel1.objects.filter(id=self.instance.fk.id) | self.fields['fk'].queryset
            self.fields['fk'].empty_label = None  # remove the empty label
            
            if self.instance.fk_is_readonly:
                self.fields['fk'].widget = HiddenInputWText(unicode(self.instance.fk))

Hidden Input with Text

Here is one way to make a field “read-only”. The field value will still be in the POST. Use CSS to alter the appearance to make it clear this is readonly.

from django import forms
from django.utils.safestring import mark_safe

class HiddenInputWText(forms.HiddenInput):
    """A widget with text following a hidden input. For making a field read-only."""
    def __init__(self, the_text, *args, **kwargs):
        super(HiddenInputWText, self).__init__(*args, **kwargs)
        self.the_text = the_text

    def render(self, name, value, attrs=None):
        html = super(HiddenInputWText, self).render(name, value, attrs=attrs)
        html += '<span class="hidden_input_text">{0}</span>'.format(self.the_text)
        return mark_safe(html)</pre>
<h2>The Javascript</h2>
Here is a javascript snippet for binding the chosen widget to the field and calling an event when the value changes. This accounts for the case when the field is a hidden input.
<pre>var fk_field = $("#id_fk");
if (fk_field[0].tagName === "SELECT"){
    fk_field.chosen().change(my_change_event);
};

Django Modelform with Many-to-Many

Django (1.5) Modelform is supposed to handle many-to-many fields. In my case, I was editing the Django User model with Django groups. Everything seemed to be working correctly. The correct group associations were automatically showing up in the many-to-many widget. The problem was the new associations were not being saved.

A quick look at the docs revealed a discussion of the problems that can happen with many-to-many when a save is done using commit=False. But I was not doing that.

Turns out the problem was in my multi-select widget. I am using the “Whitelabel” theme from revaxarts.com.It auto-magically replaces clunky widgets with better widgets. When I used a bare-bones HTML template, the many-to-many worked.

When I looked at the cleaned data right before the save command, I noticed that the Groups query set was empty. Adding the following to the form’s clean methods solved the problem:

def clean_groups(self):
    if 'groups[]' in self.data:
        group_ids = [int(x) for x in self.data['groups[]']]
        g = Group.objects.filter(id__in=group_ids)
    else:
        g = Group.objects.none()
    return g