Building a Conveyances system with Django, Django Rest Framework, and Vue Part 1

by Anna Makarudze


June 24, 2022, 5:28 p.m.



A couple of months back I came across a template containing stages for conveyance matters for a legal firm. The legal firm wanted to track progress for the matters it was handling for its clients, who are mostly banks. There are five types of matters and each has a different number of stages. Some stages are common to all while some are common to a few and others are particular to a single type of matter. A conveyance matter between a bank and an individual could have one or more matters.

Making an app for tracking these matters seemed to be a daunting task since I needed to figure out how best to deal with the different data involved. I thought to myself, that this would be the best kind of scenario to build a Django API with Django Rest Framework that would be consumed with a Vue JS frontend.

Read on to find out how I tackled this fun project.

Please note that this is a three-part series of posts. The article is divided into the following:

  • Part 1 - Backend Architecture
  • Part 2 - Frontend Architecture
  • Part 3 - Testing the Backend

Architecture

The app was developed using:

  • Python 3.9+
  • Django 4.0+
  • Django Rest Framework
  • Postgres 12
  • Vue 3
  • Node 16.15.0
  • npm 8.5.5
  • Webpack
  • Django Webpack Loader
  • Webpack Bundle Tracker

Test-Driven Development & Continuous Integration (CI)

When I started learning Python and Django, testing was difficult for me. At PyCon Namibia 2017, I ran a workshop on Testing with Django, having been introduced to it by Anna Balica in her talk at Django Under The Hood 2016. I still considered myself a beginner when it came to Python and Django, let alone testing. I missed the end of the conference but before I left, a friend of mine from Namibia, Gabriel-Tuhafeni Nhinda, looked for me to give me a book he had been asked by Harry Percival to pass on to me.

Harry had looked for me to give me a signed copy of his Test-Driven Development with Python but I had gone out. We later bumped into him and his family at the airport on our way back to Zimbabwe and I thanked him for the gift. I was so grateful and I still am very grateful, many years later. I haven’t finished the book cover to cover yet but I still refer to it a lot. Since I am reading a book he co-authored with Bob Gregory these days which also uses the TDD approach to architecture patterns with Python, it only makes sense that TDD is at the core of my latest projects.

For this project, I created the initial project and added continuous integration with GitHub Actions, CodeCov, and code formatting with black. Once everything was working, I created another branch, add-backend, which I developed the backend using TDD. After I thought I was done with the backend and all tests were passing, I created another branch, frontend, and started working on the frontend code.

As it turned out, I was wrong about the backend. I was wrong about the structure of the Stage class and BaseMatter dataclass and I was certainly wrong about the structure of ConveyanceMatter and Matter serializers. I was able to create Matter objects using stages.createcreate_conveyance_object function but because the Stage and BaseMatter classes had issues, it was a garbage-in garbage-out (GIGO) scenario.

This also affected the views, but I also managed to figure out what I needed for the frontend while working on the frontend. My frontend needs became clearer while working on the frontend code. This made me realize how important it is for backend engineers to be able to work well with frontend engineers so they can be able to meet the needs of frontend engineers. I fixed all these issues without fixing tests at first until I could consume the backend from the Vue frontend. After the frontend was working, I then went back and refactored my backend tests and code where necessary to get everything to pass again.

However, for now, TDD was used on the backend and not frontend but I do hope to add tests for the frontend later.

Backend

The backend has two apps, the accounts app for managing user authentication and the matters app for managing conveyances matters. The conveyances project also contains the project settings.py and root urls.py. Since I will need to render a Vue frontend with a Django project, I needed to create a templates folder with two files, base.html and index.html to render the app.

I am not much of a frontend person so I used Startbootstrap SB Admin 2 template for my frontend. The CSS and JS files are loaded as static files in the templates/base.html. The templates/index.html renders the Vue JS app. Django Webpack Loader and Webpack Bundle Tracker handle tracking of changes within the Vue frontend app without the need to reload the Django webserver (enabling Hot Module Replacement for Django server). The code for this file is shown below.

templates/index.html

{% extends "base.html" %}
{% load render_bundle from webpack_loader %}

{% block content %}
    <div class="container-fluid">
        <noscript>
            <strong>We're sorry but <%= htmlWebpackPlugin.options.title %> doesn't work properly without JavaScript enabled.
                Please enable it to continue.</strong>
        </noscript>

        <div id="app">
            <!-- built files will be auto injected -->
            {% render_bundle 'app' 'js' %}
        </div>
    </div>
{% endblock %}

Accounts App

For this project, not much work was done on user authentication. I decided to use the Django default authentication system. I only needed to define the User model in accounts.models.py which extends the django.contrib.auth.models.AbstractUser and inherit everything from it. I also defined a LoginForm in accounts.forms which extends django.contrib.auth.forms.AuthenticationForm.

I also only needed two URLs, for login and log out from django.contrib.auth.views. I added these to accounts/urls.py shown below. After adding urls.py, I added the app to the INSTALLED_APPS in conveyances/settings.py and included accounts.urls in conveyances/urls.py and I had a working app.

accounts/urls.py

from django.contrib.auth import views as auth_views
from django.urls import path

from .forms import LoginForm

app_name = "accounts"


urlpatterns = [
    path(
        "login/",
        auth_views.LoginView.as_view(
            template_name="accounts/login.html",
            form_class=LoginForm,
            extra_context={"title": "Login"},
        ),
        name="login",
    ),
    path("logout/", auth_views.logout_then_login, name="logout"),
]

The auth_views.logout_then_login logs out the user and redirects them to the login page again. How cool is that!

Matters App

I needed a way to be able to have a few models, one for storing Conveyance Matters which could have one or two or more matters. The matters themselves needed to be in one model despite having different fields in their data, due to having a different number of stages which are also in a different order. Creating a base class and using inheritance wouldn't have worked so I opted for Python dataclasses.dataclass (thanks to Google Tech Learning series for knowledge on how to use dataclasses for different data structures) which does not need a init() method and therefore can be created with a varying number of fields. This worked for me in matters/stages.py shown below.

matters/stages.py

from dataclasses import dataclass
from typing import Iterable, List, Text

# I left out the transfer, mortgage_bond, lost_deed_application,
# mortgage_bond_other_lawyers and mortgage_bond_cancellation
# lists used in the matters dict. You can checkout the full file in the repo.

matters = {
    "transfer": {"name": "Transfer", "stages": transfer},
    "mortgage_bond": {"name": "Mortgage Bond", "stages": mortgage_bond},
    "mortgage_bond_other_lawyers": {
        "name": "Mortgage Bond with Other Lawyers Transfering",
        "stages": mortgage_bond_other_lawyers,
    },
    "lost_deed_application": {
        "name": "Lost Deed Application",
        "stages": lost_deed_application,
    },
    " mortgage_bond_cancellation": {
        "name": "Mortgage Bond Cancellation",
        "stages": mortgage_bond_cancellation,
    },
}


class Stage:
    def __init__(self, step, stage):
        self.stage = stage
        self.step = step
        self.comment: Text = None
        self.done: bool = False


@dataclass
class BaseMatter:
    name: Text
    stages: List


def create_conveyance_object(name: str, stages: List) -> Iterable[BaseMatter]:
    matter_stages = []
    for i, stage in enumerate(stages):
        matter_stages.append(vars(Stage(i + 1, stage)))

    convenyance_matter = BaseMatter(name, matter_stages)
    return convenyance_matter

Models

For the matters app, I needed three models, Bank, Matter and ConveyanceMatter. Bank is a ForeignKey field for the ConveyanceMatter model, which makes it easy to filter for all matters for one bank (the banks are the clients for the law firm). Matter has to be a ManyToManyField since one ConveyanceMatter can be linked to more than Matter object.

I also used a uuid field for all the models as the index field so as not to expose the id of a record to the public. The matters field of the Matter model had to be a JSONField to allow partial edits to the field. This worked out later when I used nested serializers with Django Rest Framework to edit this field from the ConveyanceModelSerializer.

The complete matters/models.py is shown below:

matters/models.py

import uuid as uuid_lib
from django.db import models


class Bank(models.Model):
    uuid = models.UUIDField(db_index=True, default=uuid_lib.uuid4, editable=False)
    name = models.CharField(max_length=255)

    def __str__(self):
        return self.name


class Matter(models.Model):
    uuid = models.UUIDField(db_index=True, default=uuid_lib.uuid4, editable=False)
    name = models.CharField(max_length=255, unique=True)
    stages = models.JSONField()
    created_at = models.DateTimeField(auto_now_add=True)
    last_updated = models.DateTimeField(auto_now=True)

    def __str__(self):
        return self.name


class ConveyanceMatter(models.Model):
    uuid = models.UUIDField(db_index=True, default=uuid_lib.uuid4, editable=False)
    title = models.CharField(max_length=255)
    matters = models.ManyToManyField("Matter")
    created_at = models.DateTimeField(auto_now_add=True)
    last_updated = models.DateTimeField(auto_now=True)
    created_by = models.ForeignKey("accounts.User", on_delete=models.CASCADE)
    bank = models.ForeignKey(Bank, on_delete=models.CASCADE)
    complete = models.BooleanField(default=False)
    comment = models.TextField(blank=True, null=True)

    def __str__(self):
        return self.title

Serializers

Since matters field is a ManyToManyField for ConveyanceMatter and it is the one field that requires constant updates to the Matter object, this introduced the complexity of working with nested serializers. I needed to overwrite the create() and update() methods of the ConveyanceMatter object.

The create() method was easy enough but the update() method was complicated such that after a few trials and errors I went back to the Django Rest Framework documentation on nested serializers which recommended using the drf_writable_nested third party package, which is the route I ended up taking to make the partial edits and creating Matter objects from the ConveyanceMatterSerializer.

It also turned out that I needed a UserSerializer after all so as to be able to display the name of the currently logged in user in the frontend so I added the serializer below the matters app serializers instead of creating a serializers.py in accounts app.

matters/serializers.py

from rest_framework import serializers
from drf_writable_nested.mixins import UniqueFieldsMixin
from drf_writable_nested.serializers import WritableNestedModelSerializer

from accounts.models import User
from .models import Bank, ConveyanceMatter, Matter


class BankSerializer(serializers.ModelSerializer):
    class Meta:
        model = Bank
        fields = ("uuid", "id", "name")
        read_only_fields = ("uuid", "id")


class MatterSerializer(UniqueFieldsMixin, serializers.ModelSerializer):
    class Meta:
        model = Matter
        fields = ("uuid", "pk", "name", "stages", "created_at", "last_updated")
        read_only_fields = ("created_at", "uuid", "last_updated")
        extra_kwargs = {
            "id": {"read_only": False},
        }


class ConveyanceMatterSerializer(WritableNestedModelSerializer):
    created_by = serializers.StringRelatedField()
    bank = serializers.SlugRelatedField(
        queryset=Bank.objects.all(), read_only=False, slug_field="name"
    )
    matters = MatterSerializer(many=True)

    class Meta:
        model = ConveyanceMatter
        fields = (
            "uuid",
            "id",
            "title",
            "matters",
            "created_at",
            "last_updated",
            "created_by",
            "bank",
            "complete",
            "comment",
        )
        read_only_fields = ("created_at", "last_updated", "uuid", "pk")


class BaseMatterSerializer(serializers.Serializer):
    name = serializers.CharField(max_length=255)
    stages = serializers.JSONField()


class UserSerializer(serializers.ModelSerializer):
    class Meta:
        model = User
        queryset = User.objects.all()
        fields = ("id", "username", "first_name", "last_name")
        read_only_fields = ("id",)

Pagination

I added a little customization to the DRF pagination so I could be able to use the pagination on the frontend in matters/pagination.py as shown below:

matters/pagination.py

from collections import OrderedDict

from rest_framework import pagination
from rest_framework.response import Response


class PageNumberPagination(pagination.PageNumberPagination):

    page_size = 10

    def get_paginated_response(self, data):
        return Response(
            OrderedDict(
                [
                    ("count", self.page.paginator.count),
                    ("countItemsOnPage", self.page_size),
                    ("current", self.page.number),
                    ("next", self.get_next_link()),
                    ("previous", self.get_previous_link()),
                    ("total_pages", self.page.paginator.num_pages),
                    ("results", data),
                ]
            )
        )

Views

For the views, the CurrentUserView and BaseMatterView had to be APIViews since they are static views, one for getting the current user and the other for generating empty matter objects for the user to use in creating matter objects. The BankViewSet, MattersViewSet and ConveyanceMatterViewSet had to be viewsets so as to make the creation, update and viewing of the objects easier.

matters/views.py

from rest_framework import mixins, views, viewsets
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response

from .models import Bank, ConveyanceMatter, Matter
from .serializers import (
    BankSerializer,
    ConveyanceMatterSerializer,
    BaseMatterSerializer,
    MatterSerializer,
    UserSerializer,
)
from .stages import create_conveyance_object, matters


class BankViewSet(
    viewsets.GenericViewSet,
    mixins.CreateModelMixin,
    mixins.ListModelMixin,
    mixins.RetrieveModelMixin,
    mixins.UpdateModelMixin,
):
    permission_classes = (IsAuthenticated,)
    queryset = Bank.objects.all().order_by("id")
    serializer_class = BankSerializer
    lookup_field = "uuid"

    def get_queryset(self):
        name = self.request.query_params.get("name")
        queryset = self.queryset

        if name:
            return queryset.filter(name__icontains=name).order_by("id")
        return queryset


class ConveyanceMatterViewSet(
    viewsets.GenericViewSet,
    mixins.CreateModelMixin,
    mixins.ListModelMixin,
    mixins.RetrieveModelMixin,
    mixins.UpdateModelMixin,
):
    queryset = ConveyanceMatter.objects.all().order_by("-created_at")
    serializer_class = ConveyanceMatterSerializer
    permission_classes = (IsAuthenticated,)
    lookup_field = "uuid"

    def get_queryset(self):
        bank = self.request.query_params.get("bank")
        title = self.request.query_params.get("title")
        queryset = self.queryset

        if bank:
            return queryset.filter(bank__name__icontains=bank).order_by("-created_at")
        if title:
            return queryset.filter(title__icontains=title).order_by("-created_at")
        return queryset

    def perform_create(self, serializer):
        serializer.save(created_by=self.request.user)


class BaseMatterView(views.APIView):
    permission_classes = (IsAuthenticated,)

    def get(self, request):
        conveyance_matters = []
        for key in matters.keys():
            conveyance_matter = create_conveyance_object(
                matters[key]["name"], matters[key]["stages"]
            )
            conveyance_matters.append(conveyance_matter)
        results = BaseMatterSerializer(conveyance_matters, many=True).data
        return Response(results)


class MatterViewSet(
    viewsets.GenericViewSet,
    mixins.CreateModelMixin,
    mixins.ListModelMixin,
    mixins.RetrieveModelMixin,
    mixins.UpdateModelMixin,
):
    queryset = Matter.objects.all().order_by("-id")
    permission_classes = (IsAuthenticated,)
    serializer_class = MatterSerializer
    lookup_field = "uuid"


class CurrentUserView(views.APIView):
    def get(self, request, format=None):
        current_user = self.request.user
        results = UserSerializer(current_user).data
        return Response(results)

URLs

The viewsets are handled by rest_framework.routers.DefaultRouter while the APIViews are handled as class based views. The matters/urls.py is shown below:

matters/urls.py

from django.urls import include, path

from rest_framework.routers import DefaultRouter

from . import views

app_name = "matters"

router = DefaultRouter()
router.register("conveyance_matters", views.ConveyanceMatterViewSet)
router.register("banks", views.BankViewSet)
router.register("matters", views.MatterViewSet)

urlpatterns = [
    path("", include(router.urls)),
    path("base_matters/", views.BaseMatterView.as_view(), name="base_matters"),
    path("current_user/", views.CurrentUserView.as_view(), name="current_user"),
]

Conclusion

This concludes the backend architecture part. Read the next post to see how I handled the frontend and integration to Django.

Search
Coding Tips

Pytest assert True


Category: Django

Pytest equivalent of

unittest.TestCase.assertTrue()

is

assert some_condition.

For example

self.assertTrue(form.is_valid())

using pytest would be

assert form.is_valid().

✨Magic ✨