Independent Django Apps, Migrations and Version Control

The Django docs recommend keeping project migrations in version control. This makes sense. Let say you are modifying a model in development. You get it working. Run your tests, etc… and now you are ready to put it into production. If your migrations are in version control, then to migrate production, all you have to do is push the new code, along with the migrations, to the production server and run Django migrate. Pretty simple and fool-proof.

But what if you are creating a re-usable Django app. When I develop a re-usable app, I put it in a simple Django project for testing during development and as a demo of the app. During the process, I often create migrations for the app. Should those migrations be in version control?

If your apps are all independent of each other, it does not matter. However, if your apps are not independent, then using the migrations can lead to problems. In my case, I have two apps: User Management (UM) and Events. Events have foreign keys to users. Thus they are not independent. The problem that arises is related to a model in UM that contains the field: models.FilePathField.

That field has a required attribute called path that is that absolute path to the folder to get choices from. During dev, it gets set to a string for a path for the sample project. This is hard-coded in. When you install the app in a different project and run migrate, migrate sees that the path is no longer correct and creates a migration to correct the path. The problem comes when I create a project that uses both these apps. The version of UM in version control does not have the migration that Events needs. This causes the project migration to fail.

One possible solution is to hack the offending migration. That’s not a very good solution because it would require hacking every time a new version of the app is uploaded to the project. It also is possible that the hack might not be done right, which could make a mess.

The solution I settled on was to the MIGRATION_MODULES setting to create app migrations inside the project. This will also ignore the migrations in the app package. But that’s what you really want anyway. The project migrations are for migrating the project database. The only reason the app migrations work is because often the project db and the app db are similar.

If you use this approach, be aware that you will need to run makemigrations for each app in MIGRATION_MODULES setting. Because it is not always obvious which apps need to be in MIGRATION_MODULES, I have created a custom command to set things up:

import os

from django.core.management.base import BaseCommand
from django.core.management import call_command

from django.conf import settings


def make_package(root):
    if os.path.exists(root):
        return

    os.mkdir(root)
    fp = open(os.path.join(root, '__init__.py'), 'wb')
    fp.close()


class Command(BaseCommand):
    """
    For putting all installed app migrations into a migrations folder in this project.

    This command also creates a settings file called migration_modules_settings.py. After that it
    runs initial migrations on each app. Finally it runs the migrate command.

    Before running this command, make sure your settings file has something like:

        try:
            from migrations.migration_modules_settings import MIGRATION_MODULES
        except:
            pass

    """
    help = 'Sets up project migrations and runs migrate.'

    def handle(self, *args, **options):
        manage_py_path = os.getcwd()
        migrations_dir = os.path.join(manage_py_path, 'migrations')
        make_package(migrations_dir)

        # For creating a settings file
        migration_modules = {}

        for full_app_name in settings.INSTALLED_APPS:
            app_name = full_app_name.split('.')[-1]
            root = os.path.join(migrations_dir, app_name)
            make_package(root)

            migration_modules[app_name] = 'migrations.' + app_name

        fp = open(os.path.join(migrations_dir, 'migration_modules_settings.py'), 'w')
        fp.write('MIGRATION_MODULES = {\n')
        for key in migration_modules:
            fp.write("    '{}': '{}',\n".format(key, migration_modules[key]))
        fp.write('}\n')
        fp.close()

        call_command('makemigrations', *migration_modules.keys())
        call_command('migrate')

Problems with the PyCharm Test Runner

I had some problems with the PyCharm test runner. This was strange since I could run the tests from the command line. It turns out the problem was due to the project root being pretty far down the list in sys.path. The PyCharm test runner was picking up migrations in other apps. Adding this code to the top of my settings file solved the problem:

import sys
sys.path.insert(0, 'PATH CONTAINING MIGRATION FOLDER')

 

Advertisements

4 thoughts on “Independent Django Apps, Migrations and Version Control

  1. I’m just heading this problem with ‘FilePathField’ and migrations. My problem is simpler BTW: I’m just worried by the fact that the absolute path is written in the migration, and is not necessary the same as the absolute path in the production server.

    In my code, the value of the ‘path’ argument of the ‘FilePathField’ is obtained from a setting. It’s value vary in every machine I deploy my project. But the migration writes textually whatever value is set when I issue the ‘makemigrations’ command, and I have to adjust them everywhere.

    I understand your approach to avoid this, but as it was posted 1 year ago, I wanted to ask you if you have figured out a cleaner solution for this?. I bet there should be a cleaner way to solve this.

    • Ok, I found an interesting solution I’m happy applying. The main idea is to define an object which implement the special method `__str__` as the value of the `path` argument. The problem is that you can’t put any object there because then `makemigrations` will not be able to serialize your model. So the object you put there should implement the `deconstruct`[1] method. You can use the `descontructible` decorator to obtain an implementation of that method in many cases. All the idea I found in this answer[2] to a similar question in the Internet.

      My code looks like this::

      @deconstructible
      class StringSettingsReference:
      def __init__(self, name):
      self.name = name

      def __str__(self):
      return getattr(settings, self.name)

      class MyModel(models.Model):

      a_file_path = models.FilePath(path=stringSettingsReference(‘A_SETTING_NAME’, default=”)

      So when I issue the `makemigrations`, the text that ends in the migration is something like the following::

      (‘a_file_path’, models.FilePathField(default=”, path=theapp.models.StringSettingsReference(‘CALLSLIST_XML_REPO’))),

      [1] https://docs.djangoproject.com/en/1.11/topics/migrations/#adding-a-deconstruct-method
      [2] http://jakzaprogramowac.pl/pytanie/20808,django-migrations-and-customizable-reusable-apps

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s