Introduction

Using Django Rest Framework we can create rest endpoints, which can be used to create, read, update and delete (CRUD) objects. However, those endpoints are limited to modifying one object at a time. This becomes a huge problem when we want to create and update multiple objects at a time.

In this article, we are going to call endpoints only once instead of thousands of calls to endpoints for thousands of objects. By the end of this article, you should be able to perform bulk operations in the most efficient way.

So without further ado let’s get started!!

Objectives

1. We will implement Create API using standard ListCreateView.

2. We will use ListSerializer class by customizing its behavior and bulk_create and bulk_update methods to handle multiple data input objects.

Project Setup

Create a new Django project named app, then start a new app called inventory.

    # Create the project directory  mkdir Inventory  cd Inventory  
      
    # Create a virtual environment to isolate our package dependencies locally  
    python3 -m venv venv  
    source venv/bin/activate # On Windows use `venv\Scripts\activate`  
      
    # Install Django and Django REST framework into the virtual environment  
    pip install django  
    pip install djangorestframework  
      
    # Set up a new project with a single application  
    django-admin startproject app . # Note the trailing '.' character  
    python manage.py startapp inventory  
      
    Add `'rest_framework' and 'inventory'` to your `INSTALLED_APPS` setting.  
    INSTALLED_APPS = [  
    ...  
    'rest_framework',  
    'inventory',  
    ]  

Implementation

We will implement the first approach as per what we mentioned in the objectives.

Data Model: models.py

class Product(models.Model):  
    title = models.CharField(max_length=255)  
    description = models.TextField(max_length=500)  
    price = models.IntegerField()  
    type = models.CharField(max_length=255)  
    visible = models.BooleanField(default=False)  
    discount = models.IntegerField()  
    created_at = models.DateTimeField(auto_now_add=True)  
    updated_at = models.DateTimeField(auto_now=True)  
  
    def __str__(self):  
        return self.title  

Serializer: serializers.py

1. Serializers allow complex data such as querysets and model instances to be converted to native Python datatypes that can then be easily rendered into JSON, XML, or other content types. Serializers also provide deserialization, allowing parsed data to be converted back into complex types, after first validating the incoming data.

2. For this example, we will create ProductSerializer which inherited from ModelSerializer.

    from rest_framework import serializers  
    from inventory import models  
      
    class ProductSerializer(serializers.ModelSerializer):  
        class Meta:  
            model = models.Product  
            fields = ["id", "title", "description", "price", "type", "visible", "discount", "created_at", "updated_at"]  

Views: views.py

1. The generic views provided by the REST framework allow you to quickly build API views that map closely to your database models.

2. If the generic views don't suit the needs of your API, you can drop down to using the regular APIView class, or reuse the mixins and base classes used by the generic views to compose your own set of reusable generic views.

3. For this example, we will create ProductView which inherited from ListCreateAPIView.

    from rest_framework import generics, status  
    from rest_framework.response import Response  
      
    from inventory import models, serializers  
      
    class ProductView(generics.ListCreateAPIView):  
        queryset = models.Product.objects.all()  
        serializer_class = serializers.ProductSerializer  
      
        def create(self, request, *args, **kwargs):  
            serializer = self.get_serializer(data=request.data, many=True)  
            serializer.is_valid(raise_exception=True)  
      
            try:  
                self.perform_create(serializer)  
                return Response(serializer.data, status=status.HTTP_201_CREATED)  
            except:  
                return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)  

URLs: urls.py

1. Create API URLs

    from django.urls import path, include  
    from inventory import views  
      
    urlpatterns = [  
        path('product-bulk-create-update/', views.ProductView.as_view(), name='product-bulk-create-update')  
    ]  

Result

1. In this approach, we overrided the create method and explicitly mentioned many=True in the serializer in order to handle multiple data input objects at the same time.

2. Even though we set many=True in the serializer, It took half a minute (531 milliseconds) to insert data consisting of 100 objects. In this approach, create method gets called 100 times to create 100 objects.

3. We have used Insomnia, an API client, to test our API endpoints.

4. We will implement the second approach using the ListSerializer class to speed up the performance.

5. The code inside models.py and urls.py remains the same, we don't need to modify it.

6. ListSerializer the class provides the behavior for serializing and validating multiple objects at once. The list serializer will allow you to submit a request for multiple creates in a single call.

7. To do this, let's override the get_serializer method of the ListCreateAPIView class to check whether the input data is list or not. If it is a list then set the property of kwargs["many"]=True. By setting this property it will tell the serializer that it should use the list_serilizer_class before calling individual creates for each object.

8. Now let's modify our ProductView inside of views.py.

    class ProductView(generics.ListCreateAPIView):  
        serializer_class = serializers.ProductSerializer  
      
        def get_serializer(self, *args, **kwargs):  
            if isinstance(kwargs.get("data", {}), list):  
                kwargs["many"] = True  
      
        return super(ProductView, self).get_serializer(*args, **kwargs)  

9. Customizing ListSerializer behavior

10. There are a few use cases when you might want to customize the ListSerializer behavior. For example:

11. You want to provide particular validation of the lists, such as checking that one element does not conflict with another element in a list.

12. You want to customize the create or update behavior of multiple objects.

13. For these cases, you can modify the class that is used when many=True is passed, by using the list_serializer_class option on the serializer Meta class.

14. The default implementation for multiple object creation is to simply call .create() for each item in the list. If you want to customize this behavior, you'll need to customize the .create() method on the ListSerializer class that is used when many=True is passed.

15. Let's modify the ProductSerializer inside of serializers.py

    class ProductBulkCreateUpdateSerializer(serializers.ListSerializer):  
        def create(self, validated_data):  
            product_data = [models.Product(**item) for item in validated_data]  
            return models.Product.objects.bulk_create(product_data)  
      
      
    class ProductSerializer(serializers.ModelSerializer):  
        class Meta:  
            model = models.Product  
            fields = ["id", "title", "description", "price", "type", "visible", "discount", "created_at", "updated_at"]  
            read_only_fields = ['id',]  
            list_serializer_class = ProductBulkCreateUpdateSerializer  

16. That's it. We have implemented efficient bulk creation by customizing ListSerializer class behavior.

Result

1. The main advantage of ListSerializer is that it uses bulk_create method instead of save on objects.

2. By using ListSerializer and bulk_create, It took less than 80 milliseconds to insert data consisting of 100 objects within a single POST request.

Summary

We have seen efficient bulk create by customizing the ListSerializer class behavior and how it can be used to increase our app performance. A full working project can be found on GitHub in the Django bulk create a repository.

That's all for today. Stay tuned for the next part where learn how to implement bulk updates using Django Rest Framework.