Django Form Invalid, Despite No Errors

I was using Django CBV FormView. The form was being flagged as invalid but when I inspected the form inside the form_invalid() method using the PyCharm debugger, there were no errors. When I traced deeper into the Django code, I found this line that is returned by the is_valid() method.:

return self.is_bound and not self.errors

It turned out my form was not bound. But why would that be???

The problem was I over-rode the get_form_kwargs() method and forgot to call super(). The POST data is one of the kwargs passed to the form. If you don’t call super(), the post data does not make it into the form kwargs. Hence the form remains unbound.

Advertisements

Django: Putting Form Values in the URL Query String

I used to avoid using URLs with long query strings. This was partially due to an essay I read by a Big Shot about how URLs should be beautiful. Not sure I agree any more.

My use case is the user enters some filter parameters, a database query is run and a report is generated. Putting the filter parameters in the query string allows users to return to the page at another time or to share the link.

Here is the code:

from django.views.generic.edit import FormView
from django.http import HttpResponseRedirect
from django.core.urlresolvers import reverse
from django.utils.http import urlencode


class MyFormView(FormView):
    def form_valid(self, form):
        base_url = reverse('filter_tickets_result')

        # Post process some fields
        if form.cleaned_data['job']:
            form.cleaned_data['job'] = form.cleaned_data['job'].id
        qs = urlencode(form.cleaned_data)
        return HttpResponseRedirect(base_url + '?' + qs)

Note, I am using the Django version of urlencode. This avoids issues with Python 2.X and 3.X.

Django StaticLiveServerTestCase Gotchas

I was trying to do some Django (1.11) tests using StaticLiveServerTestCase and Selenium, but for all pages, I was getting an HTTP 500 error. There was no obvious way to debug this. Eventually I discovered this:

StaticLiveServerTestCase sets DEBUG=False before running

To debug the StaticLiveServerTestCase I set DEBUG=False and ran the development server. This also produced the HTTP 500 error and generated an email that explained that the problem was:

Exception Value: The file 'images/favicon/favicon.ico' could not be found with <pipeline.storage.PipelineCachedStorage object at 0x7f82db7f2250>.

For my settings files, I have a base settings file and then a separate settings file for each location (e.g. dev, staging, production). The location settings file imports the base settings file and then adds and over-rides settings as needed.

No matter how you generate your dev settings file, in the end you want:

PIPELINE = {'PIPELINE_ENABLED': False}
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'

Note: after you make those changes, if you run the dev server with DEBUG=False, your static files will not be served. But if you run StaticLiveServerTestCase with DEBUG=False the static files will be served. This is because StaticLiveServerTestCase has a static file server built into it.

 

Django Site Works Most Places, Except the IPhone

I had a Django 1.11 site running with nginx and gunicorn. It worked on several devices, including my Ubuntu desktop, a Mac laptop, and an IPad. But it did not work on my Iphone.

On my IPhone, I got a “network connection failed” error.

On my server, there was no record of my Iphone page request in the nginx access log and there wa no record in the nginx error log either.

The clue was in the /var/log/nginx/error.log. This file had a ton of lines like:

2019/03/03 13:42:46 [alert] 4820#0: worker process 18707 exited on signal 11 (core dumped)

To fix it, I made a bunch of changes to my nginx config file. I am not sure which line was causing the crash. But it is working now.

 

Using Mock with Django

Django has lots of great tools for testing. All of them involve creating a test database before the tests are run. This can be slow.

Python Unittests are another important part of testing. They often are very fast. But what if you want to do unit tests that involve Django model instances? This is where mock comes in. Mock lets you create objects that function like model instances without constructing a database. This Celerity Blog post is a great way to get started.

One problem you may encounter while using mock.patch is this error:

AppRegistryNotReady: Apps aren't loaded yet.

This can be solved by adding this to your test:

import django
django.setup()

You might need to setup some paths and specify a settings file. I do not have to do that because I run the tests from within PyCharm and it does some of that stuff automatically.

Here is a sample of how you might test a Django request that accesses a Django model instance:

import unittest
import json

from django.http import JsonResponse

import mock

def get_contract_items_for_ticket(request):
    """
    AJAX function for loading contract item select based on job
    :param request: Django request object
    :return:
    """
    # Import here to avoid circular imports
    from data.models import Job

    job = Job.objects.get(id=int(request.POST['ticket_job_id']))
    ci_choices = get_contract_items(job.number)
    response = {'choices': ci_choices}
    return JsonResponse(response)


class MyTest(unittest.TestCase):
    import django
    django.setup()

    def setUp(self):
        # Setup code here

    def test(self):
        job_w_contract = mock.Mock()
        job_w_contract.number = '2222-22'
        job_w_contract.id = 22

        job_wo_contract = mock.Mock()
        job_wo_contract.number = '1111-11'
        job_wo_contract.id = 1

        # Function for mocking the .get() method
        def get_job(id=None):
            if id == job_w_contract.id:
                return job_w_contract
            else:
                return job_wo_contract

        # Mock the import statement and test the function
        with mock.patch('data.models.Job') as my_model_mock:
            my_model_mock.objects = mock.Mock()  # mocks Job.objects

            # Make Job.objects.get() use get_job()
            conf = {'get.side_effect': get_job}
            my_model_mock.objects.configure_mock(**conf)

            # Make request object for a job with contract items
            mock_request = mock.Mock()
            mock_request.POST = {'ticket_job_id': str(job_w_contract.id)}

            # Do a request that returns choices
            response = get_contract_items_for_ticket(mock_request)
            choices = json.loads(response.content)['choices']
            self.assertTrue(len(choices) > 1)

            # Do a request that does not return choices
            mock_request.POST = {'ticket_job_id': str(job_wo_contract.id)}
            response = get_contract_items_for_ticket(mock_request)
            choices = json.loads(response.content)['choices']
            self.assertTrue(len(choices) == 1)

Migrating django_manage from Ansible 1.9 to Ansible 2.5

Here is one of the errors (to help you get here from Google):

"msg": "\n:stderr: /usr/bin/env: python\r: No such file or directory\n"

I had an old Django manage.py file that worked for Ansible 1.9. It did not have a shebang (e.g. #!/usr/bin/env python). The error it was generating was:

"msg": "[Errno 8] Exec format error", "rc": 8

Reading the docs, I found this key tidbit:

As of ansible 2.x, your manage.py application must be executable (rwxr-xr-x), and must have a valid shebang, i.e. “#!/usr/bin/env python”, for invoking the appropriate Python interpreter.

That seemed easy enough. But I was not sure what shebang to use since I was using a virtualenv. The quickest way to find out seemed to be use nano to edit manage.py on the server, run the Ansible task that was failing. Repeat until it works.

Then things got weird. Every reasonable shebang lead to the error I started this post with. Also, for no obvious reason, nano started warning me about the file being in DOS format, which is strange since this project has always been in Linux. Maybe when I was cutting and pasting some shebangs into the file I ended up with some DOS whitespace. Running the command “dos2unix manage.py” fixed it.

The shebang that worked was:

#!/usr/bin/env python

UPDATE

It appears that the DOS format chars entered manage.py a long time ago. Maybe when the project was created. I ended up fixing the file in my repo.