June 24, 2022, 5:28 p.m.
A couple of months back I came across a template containing stages for conveyance matters for a legal firm. The legal firm wanted to track progress for the matters it was handling for its clients, who are mostly banks. There are five types of matters and each has a different number of stages. Some stages are common to all while some are common to a few and others are particular to a single type of matter. A conveyance matter between a bank and an individual could have one or more matters.
Making an app for tracking these matters seemed to be a daunting task since I needed to figure out how best to deal with the different data involved. I thought to myself, that this would be the best kind of scenario to build a Django API with Django Rest Framework that would be consumed with a Vue JS frontend.
Read on to find out how I tackled this fun project.
Please note that this is a three-part series of posts. The article is divided into the following:
The app was developed using:
When I started learning Python and Django, testing was difficult for me. At PyCon Namibia 2017, I ran a workshop on Testing with Django, having been introduced to it by Anna Balica in her talk at Django Under The Hood 2016. I still considered myself a beginner when it came to Python and Django, let alone testing. I missed the end of the conference but before I left, a friend of mine from Namibia, Gabriel-Tuhafeni Nhinda, looked for me to give me a book he had been asked by Harry Percival to pass on to me.
Harry had looked for me to give me a signed copy of his Test-Driven Development with Python but I had gone out. We later bumped into him and his family at the airport on our way back to Zimbabwe and I thanked him for the gift. I was so grateful and I still am very grateful, many years later. I haven’t finished the book cover to cover yet but I still refer to it a lot. Since I am reading a book he co-authored with Bob Gregory these days which also uses the TDD approach to architecture patterns with Python, it only makes sense that TDD is at the core of my latest projects.
For this project, I created the initial project and added continuous integration with GitHub Actions, CodeCov, and code formatting with black
. Once everything was working, I created another branch, add-backend
, which I developed the backend using TDD. After I thought I was done with the backend and all tests were passing, I created another branch, frontend
, and started working on the frontend code.
As it turned out, I was wrong about the backend. I was wrong about the structure of the Stage
class and BaseMatter
dataclass and I was certainly wrong about the structure of ConveyanceMatter
and Matter
serializers. I was able to create Matter
objects using stages.createcreate_conveyance_object
function but because the Stage
and BaseMatter
classes had issues, it was a garbage-in garbage-out (GIGO) scenario.
This also affected the views, but I also managed to figure out what I needed for the frontend while working on the frontend. My frontend needs became clearer while working on the frontend code. This made me realize how important it is for backend engineers to be able to work well with frontend engineers so they can be able to meet the needs of frontend engineers. I fixed all these issues without fixing tests at first until I could consume the backend from the Vue frontend. After the frontend was working, I then went back and refactored my backend tests and code where necessary to get everything to pass again.
However, for now, TDD was used on the backend and not frontend but I do hope to add tests for the frontend later.
The backend has two apps, the accounts
app for managing user authentication and the matters
app for managing conveyances matters. The conveyances
project also contains the project settings.py
and root urls.py
. Since I will need to render a Vue frontend with a Django project, I needed to create a templates
folder with two files, base.html
and index.html
to render the app.
I am not much of a frontend person so I used Startbootstrap SB Admin 2 template for my frontend. The CSS and JS files are loaded as static files in the templates/base.html
. The templates/index.html
renders the Vue JS app. Django Webpack Loader and Webpack Bundle Tracker handle tracking of changes within the Vue frontend
app without the need to reload the Django webserver (enabling Hot Module Replacement for Django server). The code for this file is shown below.
templates/index.html
{% extends "base.html" %}
{% load render_bundle from webpack_loader %}
{% block content %}
<div class="container-fluid">
<noscript>
<strong>We're sorry but <%= htmlWebpackPlugin.options.title %> doesn't work properly without JavaScript enabled.
Please enable it to continue.</strong>
</noscript>
<div id="app">
<!-- built files will be auto injected -->
{% render_bundle 'app' 'js' %}
</div>
</div>
{% endblock %}
For this project, not much work was done on user authentication. I decided to use the Django default authentication system. I only needed to define the User
model in accounts.models.py
which extends the django.contrib.auth.models.AbstractUser
and inherit everything from it. I also defined a LoginForm
in accounts.forms
which extends django.contrib.auth.forms.AuthenticationForm
.
I also only needed two URLs, for login and log out from django.contrib.auth.views
. I added these to accounts/urls.py
shown below. After adding urls.py, I added the app to the INSTALLED_APPS
in conveyances/settings.py
and included accounts.urls
in conveyances/urls.py
and I had a working app.
accounts/urls.py
from django.contrib.auth import views as auth_views
from django.urls import path
from .forms import LoginForm
app_name = "accounts"
urlpatterns = [
path(
"login/",
auth_views.LoginView.as_view(
template_name="accounts/login.html",
form_class=LoginForm,
extra_context={"title": "Login"},
),
name="login",
),
path("logout/", auth_views.logout_then_login, name="logout"),
]
The auth_views.logout_then_login
logs out the user and redirects them to the login page again. How cool is that!
I needed a way to be able to have a few models, one for storing Conveyance Matters
which could have one or two or more matters. The matters themselves needed to be in one model despite having different fields in their data, due to having a different number of stages which are also in a different order. Creating a base class and using inheritance wouldn't have worked so I opted for Python dataclasses.dataclass
(thanks to Google Tech Learning series for knowledge on how to use dataclasses for different data structures) which does not need a init() method and therefore can be created with a varying number of fields. This worked for me in matters/stages.py
shown below.
matters/stages.py
from dataclasses import dataclass
from typing import Iterable, List, Text
# I left out the transfer, mortgage_bond, lost_deed_application,
# mortgage_bond_other_lawyers and mortgage_bond_cancellation
# lists used in the matters dict. You can checkout the full file in the repo.
matters = {
"transfer": {"name": "Transfer", "stages": transfer},
"mortgage_bond": {"name": "Mortgage Bond", "stages": mortgage_bond},
"mortgage_bond_other_lawyers": {
"name": "Mortgage Bond with Other Lawyers Transfering",
"stages": mortgage_bond_other_lawyers,
},
"lost_deed_application": {
"name": "Lost Deed Application",
"stages": lost_deed_application,
},
" mortgage_bond_cancellation": {
"name": "Mortgage Bond Cancellation",
"stages": mortgage_bond_cancellation,
},
}
class Stage:
def __init__(self, step, stage):
self.stage = stage
self.step = step
self.comment: Text = None
self.done: bool = False
@dataclass
class BaseMatter:
name: Text
stages: List
def create_conveyance_object(name: str, stages: List) -> Iterable[BaseMatter]:
matter_stages = []
for i, stage in enumerate(stages):
matter_stages.append(vars(Stage(i + 1, stage)))
convenyance_matter = BaseMatter(name, matter_stages)
return convenyance_matter
For the matters app, I needed three models, Bank
, Matter
and ConveyanceMatter
. Bank
is a ForeignKey
field for the ConveyanceMatter
model, which makes it easy to filter for all matters for one bank (the banks are the clients for the law firm). Matter
has to be a ManyToManyField
since one ConveyanceMatter
can be linked to more than Matter
object.
I also used a uuid
field for all the models as the index
field so as not to expose the id
of a record to the public. The matters
field of the Matter model had to be a JSONField
to allow partial edits to the field. This worked out later when I used nested serializers with Django Rest Framework to edit this field from the ConveyanceModelSerializer
.
The complete matters/models.py
is shown below:
matters/models.py
import uuid as uuid_lib
from django.db import models
class Bank(models.Model):
uuid = models.UUIDField(db_index=True, default=uuid_lib.uuid4, editable=False)
name = models.CharField(max_length=255)
def __str__(self):
return self.name
class Matter(models.Model):
uuid = models.UUIDField(db_index=True, default=uuid_lib.uuid4, editable=False)
name = models.CharField(max_length=255, unique=True)
stages = models.JSONField()
created_at = models.DateTimeField(auto_now_add=True)
last_updated = models.DateTimeField(auto_now=True)
def __str__(self):
return self.name
class ConveyanceMatter(models.Model):
uuid = models.UUIDField(db_index=True, default=uuid_lib.uuid4, editable=False)
title = models.CharField(max_length=255)
matters = models.ManyToManyField("Matter")
created_at = models.DateTimeField(auto_now_add=True)
last_updated = models.DateTimeField(auto_now=True)
created_by = models.ForeignKey("accounts.User", on_delete=models.CASCADE)
bank = models.ForeignKey(Bank, on_delete=models.CASCADE)
complete = models.BooleanField(default=False)
comment = models.TextField(blank=True, null=True)
def __str__(self):
return self.title
Since matters
field is a ManyToManyField
for ConveyanceMatter
and it is the one field that requires constant updates to the Matter
object, this introduced the complexity of working with nested serializers. I needed to overwrite the create()
and update()
methods of the ConveyanceMatter
object.
The create()
method was easy enough but the update()
method was complicated such that after a few trials and errors I went back to the Django Rest Framework documentation on nested serializers which recommended using the drf_writable_nested
third party package, which is the route I ended up taking to make the partial edits and creating Matter
objects from the ConveyanceMatterSerializer
.
It also turned out that I needed a UserSerializer
after all so as to be able to display the name of the currently logged in user in the frontend so I added the serializer below the matters app serializers instead of creating a serializers.py
in accounts
app.
matters/serializers.py
from rest_framework import serializers
from drf_writable_nested.mixins import UniqueFieldsMixin
from drf_writable_nested.serializers import WritableNestedModelSerializer
from accounts.models import User
from .models import Bank, ConveyanceMatter, Matter
class BankSerializer(serializers.ModelSerializer):
class Meta:
model = Bank
fields = ("uuid", "id", "name")
read_only_fields = ("uuid", "id")
class MatterSerializer(UniqueFieldsMixin, serializers.ModelSerializer):
class Meta:
model = Matter
fields = ("uuid", "pk", "name", "stages", "created_at", "last_updated")
read_only_fields = ("created_at", "uuid", "last_updated")
extra_kwargs = {
"id": {"read_only": False},
}
class ConveyanceMatterSerializer(WritableNestedModelSerializer):
created_by = serializers.StringRelatedField()
bank = serializers.SlugRelatedField(
queryset=Bank.objects.all(), read_only=False, slug_field="name"
)
matters = MatterSerializer(many=True)
class Meta:
model = ConveyanceMatter
fields = (
"uuid",
"id",
"title",
"matters",
"created_at",
"last_updated",
"created_by",
"bank",
"complete",
"comment",
)
read_only_fields = ("created_at", "last_updated", "uuid", "pk")
class BaseMatterSerializer(serializers.Serializer):
name = serializers.CharField(max_length=255)
stages = serializers.JSONField()
class UserSerializer(serializers.ModelSerializer):
class Meta:
model = User
queryset = User.objects.all()
fields = ("id", "username", "first_name", "last_name")
read_only_fields = ("id",)
I added a little customization to the DRF pagination so I could be able to use the pagination on the frontend in matters/pagination.py
as shown below:
matters/pagination.py
from collections import OrderedDict
from rest_framework import pagination
from rest_framework.response import Response
class PageNumberPagination(pagination.PageNumberPagination):
page_size = 10
def get_paginated_response(self, data):
return Response(
OrderedDict(
[
("count", self.page.paginator.count),
("countItemsOnPage", self.page_size),
("current", self.page.number),
("next", self.get_next_link()),
("previous", self.get_previous_link()),
("total_pages", self.page.paginator.num_pages),
("results", data),
]
)
)
For the views, the CurrentUserView
and BaseMatterView
had to be APIViews
since they are static views, one for getting the current user and the other for generating empty matter objects for the user to use in creating matter objects. The BankViewSet
, MattersViewSet
and ConveyanceMatterViewSet
had to be viewsets so as to make the creation, update and viewing of the objects easier.
matters/views.py
from rest_framework import mixins, views, viewsets
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response
from .models import Bank, ConveyanceMatter, Matter
from .serializers import (
BankSerializer,
ConveyanceMatterSerializer,
BaseMatterSerializer,
MatterSerializer,
UserSerializer,
)
from .stages import create_conveyance_object, matters
class BankViewSet(
viewsets.GenericViewSet,
mixins.CreateModelMixin,
mixins.ListModelMixin,
mixins.RetrieveModelMixin,
mixins.UpdateModelMixin,
):
permission_classes = (IsAuthenticated,)
queryset = Bank.objects.all().order_by("id")
serializer_class = BankSerializer
lookup_field = "uuid"
def get_queryset(self):
name = self.request.query_params.get("name")
queryset = self.queryset
if name:
return queryset.filter(name__icontains=name).order_by("id")
return queryset
class ConveyanceMatterViewSet(
viewsets.GenericViewSet,
mixins.CreateModelMixin,
mixins.ListModelMixin,
mixins.RetrieveModelMixin,
mixins.UpdateModelMixin,
):
queryset = ConveyanceMatter.objects.all().order_by("-created_at")
serializer_class = ConveyanceMatterSerializer
permission_classes = (IsAuthenticated,)
lookup_field = "uuid"
def get_queryset(self):
bank = self.request.query_params.get("bank")
title = self.request.query_params.get("title")
queryset = self.queryset
if bank:
return queryset.filter(bank__name__icontains=bank).order_by("-created_at")
if title:
return queryset.filter(title__icontains=title).order_by("-created_at")
return queryset
def perform_create(self, serializer):
serializer.save(created_by=self.request.user)
class BaseMatterView(views.APIView):
permission_classes = (IsAuthenticated,)
def get(self, request):
conveyance_matters = []
for key in matters.keys():
conveyance_matter = create_conveyance_object(
matters[key]["name"], matters[key]["stages"]
)
conveyance_matters.append(conveyance_matter)
results = BaseMatterSerializer(conveyance_matters, many=True).data
return Response(results)
class MatterViewSet(
viewsets.GenericViewSet,
mixins.CreateModelMixin,
mixins.ListModelMixin,
mixins.RetrieveModelMixin,
mixins.UpdateModelMixin,
):
queryset = Matter.objects.all().order_by("-id")
permission_classes = (IsAuthenticated,)
serializer_class = MatterSerializer
lookup_field = "uuid"
class CurrentUserView(views.APIView):
def get(self, request, format=None):
current_user = self.request.user
results = UserSerializer(current_user).data
return Response(results)
The viewsets are handled by rest_framework.routers.DefaultRouter
while the APIViews are handled as class based views. The matters/urls.py
is shown below:
matters/urls.py
from django.urls import include, path
from rest_framework.routers import DefaultRouter
from . import views
app_name = "matters"
router = DefaultRouter()
router.register("conveyance_matters", views.ConveyanceMatterViewSet)
router.register("banks", views.BankViewSet)
router.register("matters", views.MatterViewSet)
urlpatterns = [
path("", include(router.urls)),
path("base_matters/", views.BaseMatterView.as_view(), name="base_matters"),
path("current_user/", views.CurrentUserView.as_view(), name="current_user"),
]
This concludes the backend architecture part. Read the next post to see how I handled the frontend and integration to Django.
Pytest equivalent of
unittest.TestCase.assertTrue()
is
assert some_condition
.
For example
self.assertTrue(form.is_valid())
using pytest would be
assert form.is_valid()
.
✨Magic ✨