Note: the information in this entry is now out of date:
- Django no longer runs unit tests for 3rd party apps by default, so that issue goes away.
- Django's @skipIf decorator allows us to easily skip tests that depend on an external library that is not actually installed.
Sometimes your Django project will incorporate applications whose unit tests will never pass. Typical reasons can include:
- the application in question depends on a 3rd party library that is not actually installed on your development machine - for example, because of an expensive license, so only some members of the team have it installed.
- or you have installed a 3rd party application whose unit tests make certain assumptions, and these assumptions prove incorrect. For example, if you are writing a site that can only ever be accessed by authenticated users, this will break most 3rd party Django applications.
I first met the above problem when I was working for Hive Online. One of the developers there had quickly put together a script to fix this issue, and this was used instead of the 'test' command:
#!/usr/bin/env python
import sys
import test_settings
from django.core.management import call_command, setup_environ
setup_environ(test_settings)
APPS_FOR_TESTING = (
'app1',
'app2',
'etc...',
)
apps_for_testing = sys.argv[1:] or APPS_FOR_TESTING
call_command('test', settings=test_settings, *apps_for_testing)
I later moved on to another contract, this time working for Quru, and ran into exactly the same problem.
So I dug up the script for use in this new project. Of course, I'm a geek, so I couldn't just use it as is, I had to "improve" it. My first task was the application list: wouldn't it be better to list applications that should be excluded from testing? This way the script would require less maintenance (no need to modify it every time that you add a new application). This was a very easy change:
APPS_TO_NOT_RUN = (
'django_extensions',
'pagination',
# etc...
)
if sys.argv[2:]:
apps_for_testing = sys.argv[2:]
else:
apps_for_testing = [app for app in settings.INSTALLED_APPS
if not app in APPS_TO_NOT_RUN
and not app.startswith('django.')]
My second task was to make the above script work transparently - as a replacement to the 'test' command. While it is not a major pain to remember to run a separate script, it is non-standard, a trivial mental hoop that we have to jump through many times a day. If the project that we are working on has a large number of trivial mental hoops that we have to jump through in order to get anything done, we will become noticeably less productive - so I like to spend time on making things as 'standard' as possible.
Ok, so how to fix it? Well, in Django we can specify the test runner used by the 'test' command. While I get the impression that it's more aimed at people who want to use a totally different testing framework, there's nothing to stop us using it to slightly tweak the existing test runner:
from django.test.simple import DjangoTestSuiteRunner
from django.conf import settings
class ExcludeAppsTestSuiteRunner(DjangoTestSuiteRunner):
"""Override the default django 'test' command, exclude from testing
apps which we know will fail."""
def run_tests(self, test_labels, extra_tests=None, **kwargs):
if not test_labels:
# No appnames specified on the command line, so we run all
# tests, but remove those which we know are troublesome.
APPS_TO_NOT_RUN = (
'django_extensions',
'pagination',
# etc...
)
test_labels = [app for app in settings.INSTALLED_APPS
if not app in APPS_TO_NOT_RUN
and not app.startswith('django.')]
return super(ExcludeAppsTestSuiteRunner, self).\
run_tests(test_labels, extra_tests, **kwargs)
And we add the following to settings.py:
TEST_RUNNER = 'path.to.ExcludeAppsTestSuiteRunner'And that's it. A drop-in replacement for the test command.