Deferred foreign keys with django-dfk

django-dfk is a project that I developed for a recent project to allow foreign keys to be declared on models without an explicit target ('dfk' stands for 'deferred foreign key'). It provides an API to 'point' these foreign keys to a concrete target at a later date, and also allows you to forcefully 'repoint' foreign keys that have already been set up. This last facility should be used with caution - it's essentially akin to monkey-patching.

You can use GenericForeignKeys for this, and these are slightly more flexible in that each model instance foreign key may point to a different model. However, there is a performance cost associated with them, and joining can be problematic.

(The project actually rarely uses django-dfk directly - instead, it uses it as a basis for abstract foreign keys, which have a greater awareness of the application environment - however, that's a topic for another day.)

Before we go much further though - rather than using this package, a better long-term investment would be to look at the app-loading branch that Arthur Koziel and Jannis Leidel have been working on - testing it, and helping them to get it into a state to be merged into trunk. I think that should provide a more holistic approach to solving this kind of problem. Until then...

Deferred foreign keys are useful in applications where you know that a model will require a foreign key, but don't know what the target will be at the time you're writing the application. Taking an example from the project that django-dfk was created for, let's say you have a Django app that is an online game. Users are entered into the game by way of an 'Entry' model, and the Entry contains a foreign key back to a model which contains user information.

However, you will have several instances of this game deployed, and each game may have its own source of player data - hence, its own Player model.

Let's say that there are two applications involved: one called 'core', which contains the core game logic, and one called 'mygame1', which contains models and logic specific to an individual game deployment.

core/models.py:
from django.db import models

class Entry(models.Model):
    created = models.DateTimeField(defaults=datetime.datetime.now)
    player = models.ForeignKey( …. uh, what do I type here?

Remember - the core application is going to be deployed in several places, and there are several possible places that FK might need to point.

One approach solving this would be to introduce a model which relates an entry to the game-specific Player model, resulting in models that look something like this:

core/models.py:
from django.db import models

class Entry(models.Model):
    created = models.DateTimeField(defaults=datetime.datetime.now)


mygame1/models.py:

from django.db import models

class MyPlayer(models.Model):
    name = models.CharField(max_length=50)
    entry = models.OneToOneField(Entry)

This works fine - however, there are some games which share player data with each other. This means that the key needs to live on the Entry model - but, we don't know which player model to point the FK at, as this model might be deployed in multiple places.

We can solve this using django-dfk.

core/models.py:
from django.db import models
from dfk.models import DeferredForeignKey

class Entry(models.Model):
    created = models.DateTimeField(defaults=datetime.datetime.now)
    player = DeferredForeignKey(unique=True)


mygame1/models.py:

from django.db import models
from dfk import point
from core import Entry

class MyPlayer(models.Model):
    name = models.CharField(max_length=50)

point(Entry, 'player', MyPlayer)

The first thing to notice is that our Entry model now sports a DeferredForeignKey instance. It's important to realise that this isn't a real field, it's just a placeholder. Any arguments (except for the special 'name' argument, more on this below) are simply stored.

The action happens during the call to 'point', in mygame1's models.py. As the name implies, this points the DFK called 'player' on Entry to the MyPlayer class. (Actually, under the hood, it simply replaces the DeferredForeignKey instance with a real ForeignKey instance complete with the arguments which were originally passed to the DFK). Note that we do this at the module level in models.py - all your pointing (and repointing) needs to be done before the application is ready to use to ensure that syncdb outputs the correct SQL.

After all this is done, the definition of Entry will effectively look like this (although of course your code won't have changed!):

class Entry(models.Model):
   created = models.DateTimeField(defaults=datetime.datetime.now)
   player = models.ForeignKey('mygame1.MyPlayer', unique=True)

Other game applications (say, mygame2 and mygame3) would point the foreign key to the Player model that is appropriate for their game.

It's quite common to need to point a number of these keys at once - there might be several models which refer to a player. Rather than writing lots of 'point' statements (and having to add to them if a new key is added), django-dfk allows deferred foreign keys to be named:

core/models.py
class Entry(models.Model):
   created = models.DateTimeField(defaults=datetime.datetime.now)
   player = DeferredForeignKey(name='Player', unique=True)

class StatusUpdate(models.Model):
   text = models.CharField(max_length=140)
   player = DeferredForeignKey(name='Player')


mygame1/models.py:

from django.db import models
from dfk import point
from core import Entry

class MyPlayer(models.Model):
   name = models.CharField(max_length=50)

point_named('core', 'Player', MyPlayer)

Roughly translated, this means 'point all the deferred foreign keys in all models in the core app which have the name 'Player' to the MyPlayer model'. This will affect both the 'Entry' and 'StatusUpdate' models above.

Finally, django-dfk also provides a 'repoint' function. This is a big hammer, and is not to be used lightly. 'point' only works on DeferredForeignKey instances, by design - it's meant to prevent you making mistakes. 'repoint' works on regular foreign keys too. It's useful if you've used 'point' to change the destination of a DFK, but later need to change it. (However, if you find yourself in this position, you should probably refactor your code to just use 'point'). Both 'point' and 'repoint' take care of cleaning up various internal Django caches to ensure things life filtering on related fields work properly after a point operation.

django-dfk can be found on PyPI, and the source is on github - forks, bug reports and patches with docs and tests are welcome. django-dfk is in production on several high-volume sites.