Object detection research has hit its stride in 2021 with state-of-the-art frameworks including YOLOv5, Scaled YOLOv4, PP-YOLO, and now PP-YOLOv2 from Baidu. PP-YOLO2 builds upon the PP-YOLO framework, with some important improvements. The authors added Path Aggregation Network, mish activation function, increased image input size, and fine-tuned YOLO's loss function. That means minimizing data loss from overlapping bounding boxes. Maximize your object detection model by choosing the right YOLO algorithm framework for your model. Continue reading to learn about PP-YOLO and the YOLO algorithm for object detection. <h2>Introduction to PP-YOLO</h2> PP-YOLO (or PaddlePaddle YOLO) is a machine learning object detection framework based on the YOLO (You Only Look Once) object detection algorithm. PP-YOLO is not a new kind of object detection framework. Rather, PP-YOLO is a modified version of YOLOv4 with an improved inference speed and mAP score. These improvements are achieved by using a RESNET-50 backbone architecture and additional enhancements such as larger batch size, Dropblock, IOU Loss, and pre-trained models. In this article, we’ll explain fully what PP-YOLO is, why it is an improvement over YOLOv4, and show you how to use PP-YOLO for object detection.  <ul><li><a href="#what-is-object-detection">What is Object Detection?</a></li><li><a href="#pp-yolo-explained">YOLO and PP-YOLO Explained</a></li><li><a href="#structure">PP-YOLO: Structure</a></li><li><a href="#how-to-use">How to Use PP-YOLO</a></li><li><a href="#improvement">5 Reasons PP-YOLO is an Improvement Over YOLOv4</a></li><li><a href="#framerate-map-inf">Impact on Framerate, mAP, and Inference Time</a></li><li><a href="#summary">Summary</a></li></ul> <h2 id="what-is-object-detection">What is Object Detection?</h2> The human brain perceives the world and makes sense of its environment instantly. The two common tasks that the human visionary system does especially well are classifying and localizing objects. Imagine you are driving a car. Detecting objects in your surrounding environment is a trivial task. You have no trouble at all differentiating the cars and road signs around you from other objects and obstacles (unless, perhaps, you are a terrible driver). <b>AI researchers and computer vision scientists have been working hard to make machines understand the world visually in the way that we humans do – instantaneously and accurately.</b> This is what we refer to as object detection. <img class="wp-image-4385 size-large" src="https://wordpress.appsilon.com/wp-content/uploads/2020/01/yolo-1024x682.jpg" alt="Appsilon co-founder Damian Rodziewicz identified with the YOLO algorithm" width="1024" height="682" /> <em>Object detection in action with <a href="https://wordpress.appsilon.com">Appsilon</a> co-founder <a href="https://www.linkedin.com/in/drodziewicz/">Damian Rodziewicz</a>. The YOLO model predicts different types of objects in the photo with a high degree of accuracy.</em> <h2 id="pp-yolo-explained">YOLO Algorithm and PP-YOLO Explained</h2> In our last post on object detection and the YOLO algorithm, we discussed: <ul><li style="font-weight: 400;"><a href="https://appsilon.com/object-detection-yolo-algorithm/#object-detection">How Object Detection Works</a></li><li style="font-weight: 400;"><a href="https://appsilon.com/object-detection-yolo-algorithm/#yolo">Differences Between One-stage and Two-stage Object Detectors</a></li><li style="font-weight: 400;"><a href="https://appsilon.com/object-detection-yolo-algorithm/#yolo">Motivation Behind Using a One-stage Detector</a></li></ul> We then explored the darknet framework and applied the YOLO algorithm to a test image in order to predict the bounding box coordinates of various objects (two people, two cups, a chair, a laptop, and a dining table). <blockquote><strong>See more with Appsilon's <a href="https://appsilon.com/computer-vision/" target="_blank" rel="noopener noreferrer">Computer Vision and Custom ML Solutions</a>. </strong></blockquote> We observed that there is a trade-off between one-stage and two-stage detectors in terms of inference speed and accuracy. One-shot object detectors have been widely adopted in the computer vision community and variants of these detectors have been developed to make it run not only on powerful GPU-enabled computers but also on small embedded devices. Recently researchers at BAIDU worked on improving the performance of the original YOLO framework by adapting it a bit and have reached better options balancing the efficiency and effectiveness of the network. As you can see from Figure 1 below (figure from <a href="https://arxiv.org/abs/2007.12099" target="_blank" rel="noopener noreferrer">Long et al., 2020</a>) that <b>PP-YOLO outperforms the start-of-the-art YOLOv4 both in terms of FPS and mAP by a significant margin.</b> <img class="aligncenter wp-image-5246 size-full" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b021f740e2bdeffdb0f5fd_pp-yolo-fps.webp" alt="Comparison of pp-yolo, YOLOv4, and other frameworks for the YOLO algorithm" width="497" height="427" /> <p style="text-align: center;"><i> Source: Long et al., 2020</i></p> The algorithm called<strong> PP-YOLO</strong> or PADDLE-PADDLE YOLO is not a new object detection framework but a recipe to improve inference speed and the mAP score. <h2 id="structure">PP-YOLO: Structure</h2> A common paradigm amongst object detection frameworks has been to split the network into three essential components: <ul><li style="font-weight: 400;">Backbone Network</li><li style="font-weight: 400;">Neck</li><li style="font-weight: 400;">Detection Head</li></ul> <b>Backbone network</b>: Different variants of backbone networks have been explored in the last couple of years; the most efficient and popular being the <a href="https://arxiv.org/abs/1512.03385" target="_blank" rel="noopener noreferrer">ResNet</a> network (Deep Residual Learning for Image Recognition) which is quite a popular network architecture for object classification tasks. In the original <a href="https://arxiv.org/abs/1804.02767" target="_blank" rel="noopener noreferrer">YOLO V3 paper</a>, the Darknet-53 architecture was used. However, since ResNet has been widely studied and comes bundled with pre-trained networks, the authors decided to choose ResNet as a backbone network architecture which alone provided a boost in both effectiveness and efficiency. <b>Neck</b>: The neck of the network is where the network is made robust by making it scale and size invariant. Instead of augmenting the images, wouldn't it be great if the network itself could learn variances in scale? FPN (Feature Pyramid Network) is a common choice for a feature extractor addition which works by stacking up features at different scales in a bottom-up fashion. <b>Detection Head</b>: This is the final layer of the network which outputs the bounding box coordinates, class, and confidence score of the object. <img class="size-large wp-image-5301" src="https://wordpress.appsilon.com/wp-content/uploads/2020/09/appsilon_gathering_nicelabels-1024x669.png" alt="pp-yolo object detection demonstration" width="1024" height="669" /> <em><a href="https://wordpress.appsilon.com">Appsilon</a> team members demonstrate PP-YOLO object detection.</em> <div class="mceTemp"></div> <h2 id="how-to-use">How to Use PP-YOLO</h2> Now that we have seen how the architecture of the network is designed, let us explore how you can get started with experimenting with a trained model on a toy example.  Setting up your machine: In order to play around with PP-YOLO, you need to install the required dependencies and libraries. We will use a pre-built docker image provided by the authors of the Paddle Paddle deep learning framework. Start by running the following command in your terminal window: <pre><code class="language-r"> nvidia-docker pull hub.baidubce.com/paddlepaddle/paddle:1.8.4-gpu-cuda10.0-cudnn7 </code></pre> Once the docker image is pulled you can start using it. Run the following command to start the docker container: <pre><code class="language-r"> nvidia-docker run --name paddle -it -v $PWD:/paddle -v path_to_your_data:/data hub.baidubce.com/paddlepaddle/paddle:1.8.4-gpu-cuda10.0-cudnn7 /bin/bash </code></pre> You will have to change the path to your own image folder. Just replace the path_to_your_data with your path.  Now, execute the following command to get the predictions on a test image: <pre><code class="language-r"> CUDA_VISIBLE_DEVICES=0 python tools/infer.py -c configs/ppyolo/ppyolo.yml -o weights= https://paddlemodels.bj.bcebos.com/object_detection/ppyolo.pdparams --infer_img=/data/my_image.jpg </code></pre> &nbsp;   <p style="text-align: center;"><em>Original Image</em></p> <img class="aligncenter wp-image-5249 size-full" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b021f7e196d46b4e1f7df1_pp-yolo-prediction.webp" alt="pp-zebra-prediction" width="512" height="384" /> <p style="text-align: center;"><em>PP-YOLO Predicted Image</em></p> As you can see, the model was very accurate in classifying and localizing the three animals in the image. Two birds and one zebra. The model did not incorrectly identify the object behind the zebra as a bird or a group of birds. <h2 id="improvement">5 Reasons PP-YOLO Is An Improvement Over YOLOv4</h2> The authors of the paper experimented with 9 unique ways to tweak the base configuration to give a better performance. Let us explore 5 of these configurations: <p style="text-align: center;"><img class="aligncenter size-large wp-image-5276" src="https://wordpress.appsilon.com/wp-content/uploads/2020/09/Screen-Shot-2020-09-17-at-2.42.42-PM-1024x576.png" alt="PP-YOLO Configuration Settings" width="1024" height="576" /> <em>Configuration Settings for the PP-YOLO FRAMEWORK</em></p> <h4><strong>Batch Size</strong></h4> In order to improve the model performance, the authors of the paper experimented with a batch size of 192. Training the neural network with a larger batch size leads to a boost in the accuracy of the model by stabilizing the training process. <h4><b>Exponential Moving Average</b> </h4> Exponential moving average algorithms are quite common in the financial analytics domain. Using a similar methodology, the authors improved the performance of the model by smoothing out the weights during the training phase. This process is also very memory efficient since at every update step only the value of the smoothing factor needs to be kept in memory. This value was set to 0.9998 in the paper. <h4><b>Dropblock </b></h4> One way to improve the accuracy of the model and ensure that the model does not overfit the training data is to use regularization techniques like adding noise or using a dropout layer. Dropout layers randomly drop or ignore some nodes while training the neural network. This process can be made even more robust by dropping neurons in a continuous region as opposed to doing it independently. This ensures that the model can generalize well on unseen data. <h4><b>IoU Loss </b></h4> Neural networks have a loss function to make the network learn and adapt. In the original YOLOv3 paper only the L1 loss was used and adapted for estimating the bounding box coordinates. IoU or Intersection over union indicates what percentage of the predicted box coordinates overlaps with the ground-truth bounding box coordinates. The authors from the BAIDU research team experimented with both types of losses in the training process. The authors of the paper decided to stick with the basic version of the IoU loss since it gives good results in estimating the mAP metric. <h4><b>Pretrained Models </b></h4> Using pre-trained models and transferring the knowledge learned from one domain to another domain is a common trick to improve the speed of training the model and the effectiveness of the model. The authors used a better set of pre-trained weights for the backbone architecture which allowed them to improve the final detection accuracy even more. <h2 id="framerate-map-inf">Frame Rate, mAP, and Inference Time</h2> Let us now explore how the above methods impact the following:  <ol><li style="font-weight: 400;">Frame Rate</li><li style="font-weight: 400;">Mean Average Precision</li><li style="font-weight: 400;">Inference Time</li></ol> <h4><strong>Frame Rate</strong></h4> Computer vision applications using edge hardware need to have very low latency in processing the individual frames extracted from a live video stream. Changing the backbone architecture from Darknet-53 to ResNet leads to a significant increase in the number of frames that can be processed per second. The other configurations applied later do slow the algorithm slightly, however, it remains significantly faster than the benchmark. <img class="aligncenter size-full wp-image-5251" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b021f93b74359ce5429eab_pp-yolo-frameratevsmethod.webp" alt="pp-yolo frame rate vs method" width="512" height="295" /> <h4><strong>Mean Average Precision (mAP)</strong></h4> Let us now see how the mAP is affected by the change in the base configuration. <img class="aligncenter size-full wp-image-5252" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b021faaa4d870bbd4d5360_pp-yolo-map.webp" alt="pp-yolo map vs method" width="512" height="317" /> As you can see there is a 5.7% increase (that is a 14.6% relative improvement!) in the mAP score from the base configuration to the sophisticated configuration (denoted by J). <h4><strong>Inference Time</strong></h4> Building models and optimizing them to run as fast as possible requires a fair bit of engineering and tweaking. As you can see in the plot below, the time it takes for the model to make the predictions is drastically reduced from 17.2 ms with the original darknet architecture to 13.7 ms with ResNet as a backbone architecture and the other configurations applied.  <img class="aligncenter size-full wp-image-5253" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b021fc541bf136cb7cb931_ppyolo-inference.webp" alt="pp-yolo inference time vs method" width="512" height="284" /> <h2 id="summary">Summary: PP-YOLO and YOLO Algorithm</h2> Object detection frameworks have come a long way from two-stage detectors to one-stage detectors. Detecting objects with a higher frame rate without sacrificing too much of the model accuracy remains paramount. Particularly in industrial applications where the models need to run on edge devices with limited computational power. <strong>PP-YOLO does not introduce a new way of designing object detection models, however, it provides a valuable case study of which tricks known from a wider field of deep learning work well in the context of one-stage object detectors.</strong> In some ways, this can be seen as a limitation of the currently proposed implementation of PaddlePaddle, which is not (yet?) in the mainstream of deep learning frameworks. It will be interesting to see how PP-YOLO performs in practice and whether it will draw more attention of practitioners to the PaddlePaddle framework itself. If the improvements do translate to better industrial solutions, it may also lead to the adoption of the proposed tweaks. Specifically to implementations of the YOLO algorithm used in more common frameworks such as Tensorflow or PyTorch. <h2 id="learn-more">Learn More</h2><ul><li><a href="https://appsilon.com/object-detection-yolo-algorithm/" target="_blank" rel="noopener noreferrer">YOLO Algorithm and YOLO Object Detection: An Introduction</a></li><li><a href="https://appsilon.com/weight-poisoning-computer-vision/" target="_blank" rel="noopener noreferrer">Are Computer Vision Models Vulnerable to Weight-Poisoning Attacks?</a></li><li><a href="https://appsilon.com/satellite-image-analysis-with-fast-ai-for-disaster-recovery/" target="_blank" rel="noopener noreferrer">Satellite Image Analysis with fast.ai for Disaster Recovery</a></li><li><a href="https://appsilon.com/ai-for-wildlife-image-classification-appsilon-ai4g-project-receives-google-grant/">ML Wildlife Image Classification to Analyze Camera Trap Data Sets</a></li></ul>