Django Crispy Forms and Readonly Fields

I am constantly re-inventing the readonly field for crispy forms. Part of the reason for this is I often use TimeStampedModel, and I want to show the timestamps. I also frequently include a “created_by” field. All these fields are automatically set and not meant to be edited by the user.

One solution I have considered is to include the fields, but use the Django Crispy readonly attribute. Such as:

Field('api_token', readonly=True)

I like this solution because it does not require much extra coding and it is in the style of the form. The downside is it does not work well with model select widgets (i.e. created_by). In that case it renders a readonly widget, but it is still a selected, which kind of gives the impression its not readonly. Further, it does not work with TimeStampedModel. As of Django 1.7, forms.ModelForm cannot find the added fields: “created” or “modified”.

I have tried other solutions using the Crispy HTML function to generate all the markup of a pseudo-form field. It works, but feels hacky.

Then a voice of a Stackoverflow guru that has been nagging me for a while, finally made sense. When I was googling this a long time ago, I ended up on SO and this guru commented that making readonly form fields for fields the user never can edited is bad design for a user interface. Wow – that is so true in this case. So my DRY solution was to make a template include for these fields and put it above the form. Here is the include (it’s Bootstrap):

{%if object.created %}
    <div class="timestamp_include row">
        <div class="col-xs-3 col-xs-offset-2"><span>created: </span>{{ obj.created }}</div>
        <div class="col-xs-3"><span>modified: </span>{{ obj.modified }}</div>

        {% if obj.created_by %}
            <div class="col-xs-4"><span>created_by: </span>
                {% if obj.created_by.last_name or obj.created_by.first_name %}
                    {{ obj.created_by.first_name }} {{ obj.created_by.last_name }}
                {% else %}
                    {{ obj.created_by }}
                {% endif %}
            </div>
        {% endif %}
    </div>
{% endif %}

A little CSS and problem solved.

Advertisements

Django FileField and Invalid Forms

I was writing a form for uploading a CSV file. I had fields to let the user map their columns to my model fields and I had a forms.FileField() to get the filename. I even remembered to include “multipart/form-data” in the form tag.

Because it is likely my users will get confused between their excel files and their CSV files, I added some code in the form clean method to make sure the file is CSV-ish. Then I tested that code by trying to upload some excel file. This code worked as expected and the form was found to be invalid. So far so good.

Like all the rest of the form fields, I expected that when the invalid form was rendered, the name of the file would appear in the file upload form field. This was desirable because it would let the user see what file was invalid. The problem was no matter what I tried, the FileField widget showed “No file chosen”.

It turns out this was not an error, it was a feature. As mentioned here in StackOverflow,

HTML file input fields are never populated with existing data – this is a browser restriction, nothing to do with Django. It’s a security measure, to stop malicious sites tricking you into uploading arbitrary content from your computer.

What to do? Just add the filename to the error message.

Multisection Django Bootstrap Forms Using Crispy

I use Bootstrap in most of my projects. It is great for getting my Django projects working on all devices. One potential drawback of Bootstrap for forms is they require a lot of markup. More than I am willing to do manually. This is where django-crispy-forms comes in. Crispy knows how to take a Django form and render it in Bootstrap. So DRY. So crispy!

If you follow all the tutorials, etc… you will cruise along making beautiful web forms that work on the big screen as well as mobile. Until one day, you realize that having one input per row is often a waste of space. If only there were a way to create a single form, with multiple sections, without abandoning crispy. Here is one way to do it.

To keep the Bootstrap goodness, we will use the Bootstrap grid system. Go read those docs and then come back.

It is a form for customer intake. Here is the model.

# models.py
class Intake(TimeStampedModel):
    received_by = models.ForeignKey(ROIUser)
    client_first_name = models.CharField(max_length=256)
    client_last_name = models.CharField(max_length=256)
    notes = models.TextField()
    status = models.IntegerField(choices=[[1, 'Open'], [2, 'Close']])

Below is the Django Crispy form. Note how we incorporate the Bootstrap grid as crispy divs.

# forms.py
from crispy_forms.helper import FormHelper
from crispy_forms.layout import Layout, Submit, Div

class IntakeForm(forms.ModelForm):
    class Meta:
        model = Intake
        fields = ['received_by', 'client_first_name', 'client_last_name', 'notes', 'status']

    def __init__(self, *args, **kwargs):
        super(IntakeForm, self).__init__(*args, **kwargs)

        self.helper = FormHelper()
        self.helper.form_id = 'id_intake_form'
        self.helper.form_method = 'POST'
        self.helper.form_tag = True
        self.helper.layout = Layout(
            Div(
                Div('received_by', 'client_first_name', 'client_last_name', css_class='col-md-6'),
                Div('status', 'notes', css_class='col-md-6'), css_class='row'
            ),
            Div(
                Div(Submit('save', 'Save'), css_class='col-md-12'), css_class='row'
            )
        )

Here is the Django template, with the form nicely contained in a Bootstrap panel. I should mention that my {%block content %} is surrounded by a <div class=”container”></div>

{% extends 'base.html' %}
{% load crispy_forms_tags %}

{% block title %}Intake{% endblock %}

{% block content %}
    <div class="panel panel-primary">
      <div class="panel-heading">
        <h3 class="panel-title">Client Intake</h3>
      </div>
      <div class="panel-body">
        {% crispy form %}
      </div>
    </div>
{% endblock %}

Here is the result:

Screenshot from 2015-02-14 15:26:44

Here it is on mobile:

Screenshot from 2015-02-14 15:51:43

Pretty cool.

Django Dynamic ModelChoiceFields

In my projects, it is common to have select options that depend on other fields in the form. One seemingly obvious way to handle this is to attach an AJAX function to the onchange method of the independent fields and get the desired options for the select from the server. The problem with that approach is Django says the form is invalid if there are select options that were not there when the form was generated. What to do?

One solution is to include every possible select option in the Django form and then reduce those options as needed on the client side. When the form gets the POST data it tries to convert the result into a value by looking for the POST value as the first element of a choices tuple. The fact that some of the choices were not available on the client side does not matter.

Another solution is given here. Change your Django forms.ChoiceField into:

forms.CharField(widget=forms.Select())

A third option is to make a custom form field that solves this once and for all. It turns out the the problem is Django is trying to help you by trying to create a useful python object from the POST data. For the forms.ModelChoiceField, here is the Django source code:

class ModelChoiceField(ChoiceField):
    def to_python(self, value):
        if value in self.empty_values:
            return None
        try:
            key = self.to_field_name or 'pk'
            value = self.queryset.get(**{key: value})
        except (ValueError, self.queryset.model.DoesNotExist):
            raise ValidationError(self.error_messages['invalid_choice'], code='invalid_choice')
        return value

One way to get around that is to create a custom ModelChoiceField that overrides the to_python method. Here is one way to do that.

class AjaxModelChoiceField(forms.ModelChoiceField):
    def __init__(self, model_class, *args, **kwargs):
        queryset = model_class.objects.none()
        super(AjaxModelChoiceField, self).__init__(queryset, *args, **kwargs)
        self.model_class = model_class

    def to_python(self, value):
        if value in self.empty_values:
            return None
        try:
            key = self.to_field_name or 'pk'
            value = self.model_class.objects.get(**{key: value})
        except (ValueError, self.queryset.model.DoesNotExist):
            raise ValidationError(self.error_messages['invalid_choice'], code='invalid_choice')
        return value

# In your form
class MyForm(forms.Form):
    my_field = AjaxModelChoiceField(MyModel)

Django Forms, Choices and Caching

Some of the drop down menus on my forms require a bunch of database hits. To speed things up and reduce the load on my database, I use Django’s low level caching to hold those menus. To keep the cache fresh, I add code to clear the cache in the save method of the model. Nothing out of the ordinary here. So why weren’t my menus updating when the model changed???

My first thought was there was something wrong with my code to delete the cache. I was using the cache.delete(key) command. Some debug statements showed that was not the problem. They also showed that my method for generating the menu was not being called either. Odd…

Then I remembered I was setting the form choices when I was creating class variables. The class and those variables are created once when the server is started. Here was the offending code:

from django import forms

class MyForm(forms.Form):
    my_choice_field = forms.ChoiceField(choices=my_model.make_choices())

The problem was solved by moving the choices code to inside the __init__ method:

from django import forms

class MyForm(forms.Form):
    my_choice_field = forms.ChoiceField(choices=[])

    def __init__(self, *args, **kwargs):
        super(MyForm, self).__init__(*args, **kwargs)
        self.fields['my_choice_field'].choices = my_model.make_choices()

Django, Modelforms and Long Dropdowns

Lets say you want to make a page to create or update a model that has a foreign key to 10k other records. If you are not careful, this can have detrimental consequences. For example, without caching, your app will need to pull 10k records from the db to make the drop-down. This post is my notes for dealing with this problem.

Cache the Values in a Fat Model

The goal is to create a form for creating and updating MyModel2 (see below). This model has a foreign key to MyModel1. Make a classmethod in Model1 for getting the drop down query set from cache. Over-ride the save method to keep the cache up to date.

from django.db import models
from django.core.cache import cache

class MyModel1(models):
    """Model with 50k records, 10k active."""
    afield = models.CharField()
    is_active = models.BooleanField()

    def __unicode__(self):
        return self.afield

    @classmethod
    def get_active(cls):
        cache_key = 'model1_cache'
        qs = cache.get(cache_key)
        if not qs:
            qs = cls.objects.filter(is_active=True)
            cache.set(cache_key, qs)
        return qs

    def save(self, **kwargs):
        cache.delete('model1_cache')
        return super(MyModel1, self).save(**kwargs)


class MyModel2(models):
    fk = models.ForeignKey(MyModel1)
    field2 = models.CharField()

    @property
    def fk_is_readonly(self):
        if self.field2 == 'xyz':
            return True
        else:
            return False

Use JQuery Chosen in Your Template

A standard HTML select sucks when you have lots of items. You will need an enhanced widget. Recently I tried using jquery autocomplete, but it required too many hacks to get it to work. The worst problem was limiting the user to the provided choices, while at the same time triggering an AJAX call once a valid value was entered. It was doable, but hacky and fragile.

I settled on Jquery Chosen. It’s easy to install and use. However some hacks were required when I needed it to be read-only. The hacks are in the Django form.

Make the ModelForm

If you use the standard methods for making the model form, Django will pull all the values from the database. You don’t want to do that. Below is what you want. The second line make sure the query set is pulled from cache and only has active items.

What happens if you are updating a MyModel2 record and the associated MyModel1 became inactive? We handle that by adding that instance to the query set in the __init__ method.

Finally, I often run into the case where fk needs to be “read-only”. I tried using the chosen “disable” function, but it removes the field from the POST data. I handle this by making a custom widget that’s a HiddenInput with text.

class MyModel2Form(forms.ModelForm):
    fk = forms.ModelChoiceField(required=True, queryset=MyModel1.get_active())

    class Meta:
        model = MyModel2
        fields = ('fk', 'field2')

    def __init__(self, *args, **kwargs):
        super(MyModel2Form, self).__init__(*args, **kwargs)

        if self.instance.id:
            # Make sure the current value is in the queryset. If that value became inactive since the record was
            # created, then it will not be in the query set. Without it, if the user saves without changing the
            # value they will get a form error saying its an invalid choice.
            new_qs = self.fields['fk'].queryset.filter(id=self.instance.fk.id)
            if not new_qs:
                self.fields['fk'].queryset = MyModel1.objects.filter(id=self.instance.fk.id) | self.fields['fk'].queryset
            self.fields['fk'].empty_label = None  # remove the empty label
            
            if self.instance.fk_is_readonly:
                self.fields['fk'].widget = HiddenInputWText(unicode(self.instance.fk))

Hidden Input with Text

Here is one way to make a field “read-only”. The field value will still be in the POST. Use CSS to alter the appearance to make it clear this is readonly.

from django import forms
from django.utils.safestring import mark_safe

class HiddenInputWText(forms.HiddenInput):
    """A widget with text following a hidden input. For making a field read-only."""
    def __init__(self, the_text, *args, **kwargs):
        super(HiddenInputWText, self).__init__(*args, **kwargs)
        self.the_text = the_text

    def render(self, name, value, attrs=None):
        html = super(HiddenInputWText, self).render(name, value, attrs=attrs)
        html += '&lt;span class="hidden_input_text"&gt;{0}&lt;/span&gt;'.format(self.the_text)
        return mark_safe(html)</pre>
<h2>The Javascript</h2>
Here is a javascript snippet for binding the chosen widget to the field and calling an event when the value changes. This accounts for the case when the field is a hidden input.
<pre>var fk_field = $("#id_fk");
if (fk_field[0].tagName === "SELECT"){
    fk_field.chosen().change(my_change_event);
};

Speeding Up Django Forms

Lets say you have a model with a foreign key to another model that has thousands of records and you want to use forms.ModelForm to make a form. What could go wrong? Well the default is Django is going to build a dropdown with all those foreign key records. Amongst other things, this will put a huge load on your db. Especially if this form is used a lot. I have found two ways to deal with this.

Method 1: Shadow the Field

Lets say the field is “employees”. Use the modelform exclude variable to exclude employees. Then add another field like this:

class MyForm(forms.ModelForm):
    my_employee = forms.CharField(required=False, max_length=100, label='Employee')

    class Meta:
        model = MyModel
        exclude = ('employee', )

    def __init__(self, *args, **kwargs):
        super(MyForm, self).__init__(*args, **kwargs)

        if self.instance.id:
            self.fields['my_employee'].initial = self.instance.employee

    def clean_my_employee(self):
        """ I use jquery autocomplete so some cleaning is needed."""
        my_employee = self.cleaned_data['my_employee']
        # Clean it
        self.my_employee = x  #  some code to create a valid id for the foreign key
        return my_employee

    def save(self, commit=True):
        obj = super(MyForm, self).save(commit=False)
        obj.employee = self.my_employee
        return obj

I use jquery autocomplete for the field on the form. I cache the allowed values and pass them to javascript. Then I use additional javascript to require that the user select from the provided choices. Here is the javascript:

var limit_autocomplete= function (id_autocomplete, field_name, valid_choices){
 var ac_field = $('#'+id_autocomplete);
 var dialog_message = $('<div id="MyDialog"><p>You must select one of the available choices for ' + field_name + ' or clear this field.</p></div>');

 var OK_clicked = function(){
 $(this).dialog("close");
 ac_field.focus();
 ac_field.trigger('keydown');
 };

 var check_autocomplete = function(){
 var input_value = ac_field.val();
 var i = $.inArray(input_value, valid_choices);
 console.log(input_value + ' '+i);
 if (i === -1 && input_value.length>1){
 dialog_message.dialog({modal: true, buttons: [{text: "OK", click: OK_clicked}]});
 }
 };

 //ac_field.on("autocompletechange", check_autocomplete);
 ac_field.focusout(check_autocomplete);
};

I also use this javascript to prevent autocomplete from submitting the form when the enter key is pressed.

Method 2: Over-ride the Field

What if you still want a drop-down? For example, lets say that even though you have 5000 employees, you only want a small subset? If you just change the choices attribute on the field in __init__, it’s too late. Django has already hit your db and constructed the full drop down. I was at a loss at this point. Stackoverflow came to the rescue. Here is the solution:

class MyForm(forms.ModelForm):
    employees = forms.ModelChoiceField(required=False, queryset=Employees.objects.none())

    class Meta:
        model = Model_w_Employees

    def __init__(self, *args, **kwargs):
        super(MyForm, self).__init__(*args, **kwargs)

        self.fields['employee'].queryset = my_make_queryset()

Note that you can still use the same field name as in the model.