When getting ready for a final move to production, there are a few things one needs to review: performance and monitoring. This brought out the last few items that I needed to incorporate:
- Django Logging
- Django Cache
- React Performance
Django Logging
When you are first designing and developing your app, you are basically testing each line of code as you write it. Then there are oftentimes the leftover, slightly irritating yet comforting, echo/print statements that still show up in the code around those more problematic portions of code. It can help some of us programmers feel more comfortable to see that you haven’t broken it when adding other code at another time. But what about later on when it becomes too in the way, but we still need something for debugging? That’s what logging is for. And since Django is a fully developed back-end framework, the feature is already available. To enable lots of built-in log output, all that we need to do is add to the settings.py file:
# ==============================
# LOGGING CONFIGURATION
# ==============================
# Django logging setup that reads LOG_LEVEL from .env.
# Logs are stored in logs/django.log (ensure the 'logs/' directory exists).
# Adjust log level per environment: DEBUG (dev), INFO (staging), WARNING (prod).
LOG_LEVEL = os.getenv("LOG_LEVEL", "WARNING") # Default to WARNING if not set
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"verbose": {
"format": "{levelname} {asctime} {module} {message}",
"style": "{",
},
"simple": {
"format": "{levelname} {message}",
"style": "{",
},
},
"handlers": {
"file": {
"level": LOG_LEVEL,
"class": "logging.handlers.TimedRotatingFileHandler",
"filename": str(LOG_DIR / "django.log"),
"when": "midnight", # Rotate logs at midnight
"interval": 1, # Rotate every 1 day
'backupCount': 10, # Keep the last 10 days of logs
'formatter': 'verbose',
'delay': True,
# When the RotatingFileHandler is used without the delay option, it opens the log file
# as soon as the logging configuration is loaded, even before any logging takes place.
# This can result in the file being locked for the entire process, preventing log
# rotation (or renaming of the log file) because the file is still being held open by
# the logging handler.
# By setting delay: True, the RotatingFileHandler delays the opening of the log file
# until the first log message is written. This means the file is not immediately locked
# by the handler when the application starts, and it allows log rotation to proceed
# without any issues. Essentially, it avoids any conflicts between the log rotation
# process and any other process or thread that may want to access the log file
# (like Django’s development server, which could be holding the file open).
},
"console": {
"level": LOG_LEVEL,
"class": "logging.StreamHandler",
"formatter": "simple",
},
},
"loggers": {
# Log database queries
"django.db.backends": {
"level": LOG_LEVEL, # You can adjust this to INFO or ERROR as needed
"handlers": ["file", "console"], # Log to both file and console
"propagate": False, # Prevent it from propagating to the root logger
},
},
"root": {
"handlers": ["file", "console"],
"level": LOG_LEVEL,
},
}
Once this code is in, some Django functions automatically generate log output. There were a few minor snags when adding this block to the settings.py file:
- Initially, the log worked perfectly fine. But the next day, it all broke due to file permission issues. The core was stepping on itself trying to deal with the log files. Setting the delay value to true is counterproductive: why would you want to delay logging? Turning on this delay enables logging to be delayed by enough time for log files to be rotated.
- Another urge to fight when developing the code: LOG_LEVEL. It feels innocent enough to put in the string “DEBUG” and move on. Fight that urge from the beginning! The same goes for all those darn echo and print statements. You’ll notice the very first line is the os.getenv which enables the LOG_LEVEL to be driven by a system or environment variable. Turns out that for the variables that you want to be driven by docker, later on, using the .env system is ideal because the docker command line values will override the os.getenv values set in your .env file!
- Something I never even thought about until late in the design cycle when I moved the database externally: is there any database traffic? django.db.backends enables the core to automatically output all database traffic into the log.
Implementing my own log output was relatively simple:
import logging
logger = logging.getLogger(__name__)
// example of actually used error level
logger.error("Token refresh failed: %s", str(e))
// example of actually used debug level
logger.debug("Updating User Info: %s", dict(request.data))Django Cache
Enabling Cache proved a little more challenging. There are powerful cache systems out there, like Redis and Memcached. These all felt like overkill when the app really only needed just the job site table cached. So, I went with the simple and basic Django module:
# ==============================
# Django's caching configuration.
# We are using LocMemCache, an in-memory cache that is local to the Django process.
# This does not require an external caching service like Redis or Memcached.
# ==============================
CACHES = {
"default": {
"BACKEND": "django.core.cache.backends.locmem.LocMemCache", # Stores cached data in memory
"LOCATION": "unique-default-cache", # A unique identifier for this cache instance
"TIMEOUT": 86400, # Cache expiration time in seconds (1 day)
# OPTIONS allows us to control cache behavior:
"OPTIONS": {
"MAX_ENTRIES": 10000 # Limit the cache to approximately 10,000 entries
},
}
}The implementation was only in the job_site_views.py file. Perhaps the only thing I should have done differently was move the cache lifetime control out to the env file, but that was about it. The code itself, while scattered throughout the file, was pretty straightforward:
import logging
from rest_framework.response import Response
from rest_framework.decorators import api_view
from rest_framework import status
from django.shortcuts import get_object_or_404
from django.core.cache import cache
from job_search.job_site.job_site import JobSite
from job_search.job_site.job_site_serializer import (
JobSiteSerializer,
JobSiteListSerializer,
)
logger = logging.getLogger(__name__) # Set up logging for this module
@api_view(["GET", "POST"])
def job_site_list(request):
if not request.user.is_authenticated:
return Response({"detail": "Authentication credentials were not provided."},
status=status.HTTP_401_UNAUTHORIZED)
cache_key = f"job_sites_{request.user.id}" # Unique cache key per user
logger.info("Checking cache for key: %s", cache_key) # Log cache check
if request.method == "GET":
cached_data = cache.get(cache_key)
if cached_data is not None:
logger.info("Cache hit - returning cached job site data.") # Log cache hit
return Response(cached_data)
logger.info("Cache miss - querying database for job sites.") # Log cache miss
data = JobSite.objects.filter(user=request.user)
serializer = JobSiteListSerializer(data, context={"request": request}, many=True)
# ✅ Store data in cache
cache.set(cache_key, serializer.data, timeout=3600) # Cache for 1 hour
logger.info("Cached job site data for key: %s", cache_key)
return Response(serializer.data)
elif request.method == "POST":
serializer = JobSiteSerializer(data=request.data)
if serializer.is_valid():
serializer.save(user=request.user)
# Update the cache with the new data
data = JobSite.objects.filter(user=request.user)
updated_serializer = JobSiteListSerializer(data,
context={"request": request}, many=True)
cache.set(cache_key, updated_serializer.data, timeout=3600)
logger.info("Job site added - updated cache for key: %s", cache_key)
return Response(status=status.HTTP_201_CREATED)
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
@api_view(["GET", "PUT", "DELETE"])
def job_site_detail(request, pk):
if not request.user.is_authenticated:
return Response({"detail": "Authentication credentials were not provided."},
status=status.HTTP_401_UNAUTHORIZED)
cache_key = f"job_site_{pk}" # Unique cache key per job site
logger.info("Checking cache for key: %s", cache_key) # Log cache check
job_site = get_object_or_404(JobSite, pk=pk)
if job_site.user != request.user:
return Response({"detail": "You do not have permission to access this resource."},
status=status.HTTP_403_FORBIDDEN)
if request.method == "GET":
cached_data = cache.get(cache_key)
if cached_data is not None:
logger.info("Cache hit - returning cached job site detail.") # Log cache hit
return Response(cached_data)
logger.info("Cache miss - querying database for job site detail.") # Log cache miss
job_site = get_object_or_404(JobSite, pk=pk)
if job_site.user != request.user:
return Response({"detail": "You do not have permission to access this resource."},
status=status.HTTP_403_FORBIDDEN)
serializer = JobSiteSerializer(job_site, context={"request": request})
# Store data in cache for 1 hour (3600 seconds)
cache.set(cache_key, serializer.data, timeout=3600)
logger.info("Cached job site detail for key: %s", cache_key)
return Response(serializer.data)
elif request.method == "PUT":
job_site = get_object_or_404(JobSite, pk=pk)
if job_site.user != request.user:
return Response({"detail": "You do not have permission to access this resource."},
status=status.HTTP_403_FORBIDDEN)
serializer = JobSiteSerializer(job_site, data=request.data, context={"request": request})
if serializer.is_valid():
serializer.save()
# Update the cache with the updated job site data
updated_serializer = JobSiteSerializer(job_site, context={"request": request})
cache.set(cache_key, updated_serializer.data, timeout=3600)
logger.info("Job site updated - updated cache for key: %s", cache_key)
return Response(status=status.HTTP_204_NO_CONTENT)
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
elif request.method == "DELETE":
job_site = get_object_or_404(JobSite, pk=pk)
if job_site.user != request.user:
return Response({"detail": "You do not have permission to access this resource."},
status=status.HTTP_403_FORBIDDEN)
job_site.delete()
return Response(status=status.HTTP_204_NO_CONTENT)
React Performance
React is already a scalable high performing architecture, right? Well, as I was migrating my database out to the cloud, I noticed my local app performance took a huge hit due to the double-render nature of React. Not only was React rendering the components twice, but it was also waiting on database calls returning with data twice. Yikes! To fix this, I had to add a reference variable to track whether or not the fetch had already been sent. If it has already been sent, don’t go through the callback and make another call for it. I used the below code as a reference in all of my front-end react pages where I was making calls for data, including the Dashboard, Job Posting Edit, Job Posting List, Job Site List, Job Site View, Opportunity Details, and Opportunity List.
import React, { useEffect, useState, useCallback, useRef } from 'react';
const JobPostingList = () => {
const [jobPostings, setJobPostings] = useState([]);
const [filters, setFilters] = useState({
filteredJobPostings: [],
filterCompanyNameText: '',
filterPostingTitleText: ''
});
const { apiRequest } = useApiRequest();
const navigate = useNavigate();
const hasFetched = useRef(false); // Track if the request has already been made
const getJobPostings = useCallback(async () => {
if (hasFetched.current) return; // Prevent double fetch
hasFetched.current = true;
const data = await apiRequest(JOB_POSTING_API_URL, {method:'GET'});
if(data) {
setJobPostings(data);
setFilters((prevState) => ({
...prevState,
filteredJobPostings: data,
}))
} else {
console.error('Failed to fetch job postings');
}
}, [apiRequest]);Summary
With these three items added, I felt much more confident in the readiness to deploy my application to AWS!