Django Tests, Selenium, Ajax and PyCharm

If you need to test Django code that involves Ajax, Selenium is the way to go. If you use PyCharm, you probably use the debugger all the time. In fact, you might be inclined to put some break points in the Ajax callbacks and inspect some variables when execution stops at those break points. All pretty straight forward… except if your callback accesses the database.

When the debugger stops in the callback, it turns out it cannot access the test database. If your callback accesses the database, then what the debugger is showing you is not what is actually happening. I ended up solving the problem I was having by putting print statements in the callback.

Django, Selenium Headless Tests Fail

Running headless selenium tests is great. Except when the tests succeed when headed, then fail when headless. When that happens it’s hard not to let your imagination run wild with crazy javascript thoughts.

There are many ways to run headless. The method I use is described in this post. In a sense this is not headless because a normal browser is running; it’s just running on a hidden display. This makes the headless fails all the more perplexing.

In this case, the problem was the headless browser window was a different size compared to the headed version. I use the chosen widget a lot. It’s a fancy combo of an HTML select widget and auto-complete. Clicking on this widget creates a pop-up drop down. If the needed choice is outside of the browser, selenium auto scrolls the page so that the choice is visible. When it decides to scroll depends on the browser window dimensions.

In many cases, this auto scroll would no create problems. In my case, I also have a nav-bar locked to the top of the screen. Sometimes the desired choice would end up under the nav-bar after auto-scroll. When selenium moved to select the choice, it would get the nav bar instead. The form value would not be set and the form would fail. Although it did not happen to me, I supposed it’s possible selenium could have ended up on a different page. That could make some strange errors.

The first part of the fix is to explicitly set the browser window so the headed and headless versions are the same. Something like this:

if getattr(settings, 'HEADLESS_TESTS', False):
    self.vdisplay = Display(visible=False, size=(1600, 1000))
    self.vdisplay.start()
self.selenium = webdriver.Firefox()
self.selenium.set_window_size(1500, 900)

If all the problems go away, you are done. If not, then hopefully the headed tests and headless tests will now fail in the same way, making debugging easier.

In my case, I added code to scroll the page if the item was under the nav bar. Here is the code.

Django Headless Testing: Address already in use Error

Here is a great post for how to run Django tests headless. It uses xvfb.

Sometimes after a test crashes, the next time you run a test, you will get the error:

Traceback (most recent call last):
 File "/home/chuck/.virtualenvs/qdb7/local/lib/python2.7/site-packages/django/test/testcases.py", line 1189, in setUpClass
 raise cls.server_thread.error
error: [Errno 98] Address already in use

To get rid of this error, find the Xvbf process that was left running after the crash:

ps aux | grep Xv

Then run kill on the pid.

Mocking Django Timezone

I use django.utils.timezone.now() in a lot of my code. It returns the datetime in the timezone specified in settings.py. One side effect of using this function is that during testing, you may want “now()” to be a fixed date time. No worries. Mock to the rescue. Here is the code:

import datetime

from django.test import TestCase
from django.utils import timezone

import mock

# Make now() a constant
NOW_FOR_TESTING = datetime.datetime(2015, 10, 10, 10)


# This is the function that replaces django.utils.timezone.now()
def mocked_now():
    return NOW_FOR_TESTING


# This function shows that the mocking is in effect even outside of the TestMyTest scope.
def a_func():
    return timezone.now()


@mock.patch('django.utils.timezone.now', side_effect=mocked_now)
class TestMyTest(TestCase):
    def test_time_zone(self, *args):
        # After patching, mock passes in some extra vars. Put *args to handle them.
        self.assertEqual(timezone.now(), NOW_FOR_TESTING)
        self.assertEqual(timezone.now().date(), NOW_FOR_TESTING.date())
        self.assertEqual(mocked_now(), NOW_FOR_TESTING)

 

Sending Key Up in Selenium

I have a text input with a handler attached to the keyup event. The problem was that the Selenium send_keys method does not send a keyup event when it completes. There is no special Key for that. For example Key.Up does not solve the problem.

Action Chains has key_up and key_down methods. The problem is they are designed to be used with modifier keys, such as Control. It turns out you can get a keyup event by a key_down immediately followed by a key_up, like this:

ActionChains(driver).move_to_element(element).send_keys('sometext').\
    key_down(Keys.CONTROL).key_up(Keys.CONTROL).perform()

 

Django, Multiple Test Files and Database Not Resetting

As per the new options for organizing tests, I put my tests for an app into a folder called tests. When I ran each test individually, each test worked. But when I ran all of them for that app together, one failed because the database was not resetting.

It turns out that the problem was due to my used of the test class SimpleTestCase and using mommy to make model instances. As the docs clearly point out, SimpleTestCase should not be used if you are using the ORM. Switching to TestCase solved the problem.

Being a good Django citizen, I looked into opening a ticket for this. Looks like someone else already did it. I am using Django 1.7. This ticket was completed around June 2015, so it’s not in 1.7