feat: article v2

Author: jasperan

custom_pytorch_yolov5/custom_pytorch.md

@@ -3,6 +3,7 @@
 ## Introduction
 
 In this article, we're going to learn how to load a YOLOv5 model into PyTorch, and then augment the detections with three different techniques:
+
 1. Sorting Detections 
 2. Cropping and saving detections
 3. Counting Detected Objects
@@ -26,11 +27,11 @@ But what is the definition of real time? I want every frame that I see in my com
 
 Finally, I'd like to mention my journey through a *very painful* road of finding *a few* bugs on the Windows Operating System and trying to *virtualize* your webcam feed. There's this great plugin that could "replicate" your camera feed into a virtual version that you could use in any program (you could give your computer any program / input and feed it into the webcam stream, so that it looked like your webcam feed was coming from somewhere else), and it was really great:
 
-![OBS great outdated plugin](./images/obs_1.png)
+![OBS great outdated plugin](./images/obs_1.PNG)
 
 This was an [OBS (Open Broadcaster Software)](https://obsproject.com/) plugin. OBS is the go-to program to use when you're planning to make a livestream. However, this plugin was discontinued in OBS version 28, and all problems came with this update. I prepared this bug-compilation image so you can feel the pain too:
 
-![OBS bug compilation](./images/obs_errors.png)
+![OBS bug compilation](./images/obs_errors.PNG)
 
 So, once we've established that there are several roadblocks that prevent us from happily developing in a stable environment, we finally understand the "why" of this article. Let's begin implementing.
 
@@ -44,7 +45,7 @@ Technical requirements are Python 3.8 or higher, and PyTorch 1.7 or higher. [Her
 
 First, we will use the `argparse` to include additional parameters to our Python script. I added three optional parameters:
 
-![argparse](./images/argparse.png)
+![argparse](./images/argparse.PNG)
 > **Note**: the confidence threshold will only display detected objects if the confidence score of the model's prediction is higher than the given value (0.0-1.0).
 
 These argparse parameters' default values can always be modified. The _frequency_ parameter will determine how many frames to detect (e.g. a frame step). If the user specifies any number N, then only 1 frame every N frames will be used for detection. This can be useful if you're expecting your data to be very similar within sequential frames, as the detection of one object, in one frame, will suffice. Specifying this frame step is also beneficial to avoid cost overages when making these predictions (electricity bill beware, or OCI costs if you're using Oracle Cloud).
@@ -53,40 +54,40 @@ After this initial configuration, we're ready to load our custom model. You can
 
 So, we now load the custom weights file: 
 
-![loading PyTorch model](./images/load_model.png)
+![loading PyTorch model](./images/load_model.PNG)
 > **Note**: we specify the model as a custom YOLO detector, and give it the model's weights file as input.
 
 Now, we're ready to get started. We create our main loop, which constantly gets a new image from the input source (either a webcam feed or a screenshot of what we're seeing in our screen) and displays it with the bounding box detections in place:
 
-![main loop](./images/main_loop.png)
+![main loop](./images/main_loop.PNG)
 
 The most important function in that code is the `infer()` function, which returns an image and the result object, transformed into a pandas object for your convenience.
 
-![infer 1](./images/infer_1.png)
+![infer 1](./images/infer_1.PNG)
 
 In this first part of the function, we obtain a new image from the OpenCV video capture object. You can also find a similar implementation but by taking screenshots, instead of using the webcam feed in [this file](https://github.com/oracle-devrel/devo.publishing.other/custom_pytorch_yolov5/files/lightweight_screen_torch_inference.py) 
 
 Now, we need to pass this to our Torch model, which will return bounding box results. However, there's an important consideration to make: since we want our predictions to be as fast as possible, and we know that the mask detection model has been trained with thousands of images of all shapes and sizes, we can consider a technique which will benefit Frames Per Second (FPS) on our program: **rescaling** images into a lower resolution (since my webcam feed had 1080p resolution, and I use a 2560x1440 monitor which causes screenshots to be too detailed, more than I need). 
 For this, I chose a `SCALE_FACTOR` variable to hold this value (between 0-1). Currently, all images will be downscaled to 640 pixels in width and the respective resolution in height, to maintain the original image's aspect ratio.
 
-![infer 2](./images/infer_2.png)
+![infer 2](./images/infer_2.PNG)
 
 Now that we have our downscaled image, we pass it to the model, and it returns the object we wanted:
 
-![infer 3](./images/infer_3.png)
+![infer 3](./images/infer_3.PNG)
 > **Note**: the `size=640` option tells the model we're going to pass it images with that width, so the model will predict results of those dimensions.
 
 The last thing we do is draw the bounding boxes that we obtained into the image, and return the image to display it later.
 
-![infer return](./images/infer_5.png)
+![infer return](./images/infer_5.PNG)
 
 ## 1. Sorting Detections
 
 This first technique is the simplest, and can be useful to add value to the standard YOLO functionality in an unique way. The idea is to quickly manipulate the PyTorch-pandas object to sort values according to one of the columns. 
 
 For this, I suggest two ideas: sorting by confidence score, or by detection coordinates. To illustrate how any of these techniques are useful, let's look at the following image: 
 
-![speed figure](./images/figure_speed.png)
+![speed figure](./images/figure_speed.PNG)
 > **Note**: this image illustrates how sorting detections can be useful. [(image credits)](https://www.linkedin.com/in/muhammad-moin-7776751a0/)
 
 In the image above, an imaginary line is drawn between both sides of the roadway, in this case **horizontally**. Any object passing from one equator to the other in a specific direction is counted as an "inward" or "downward" vehicle. This can be achieved by specifying (x,y) bounds, and any item in the PyTorch-pandas object that surpasses it in any direction is detected.
@@ -112,16 +113,16 @@ To implement this, we will base everything we do on **bounding boxes**. Our PyTo
 
 An important consideration is that, since we're passing images to our model with a width of 640 pixels, we need to keep our previously-mentioned `SCALE_FACTOR` variable. The problem is that the original image has a higher size than the downscaled image (the one we pass the model), so bounding box detection coordinates will also be downscaled. We need to multiply these detections by the scale factor in order to _draw_ these bounding boxes over the original image; and then display it:
 
-![infer 4](./images/infer_4.png)
+![infer 4](./images/infer_4.PNG)
 
 We use the `save_cropped_images()` function to save images, while also accounting for the frequency parameter we set: we'll only save the cropped detections in case the frame is one we're supposed to save.
 Inside this function, we will **upscale** bounding box detections. Also, we'll only save the image if the detected image is higher than (x,y) width and height:
 
-![save cropped images](./images/save_cropped_images.png)
+![save cropped images](./images/save_cropped_images.PNG)
 
 Last thing we do is save the cropped image with OpenCV:
 
-![save image](./images/save_image.png)
+![save image](./images/save_image.PNG)
 
 And we successfully implemented the functionality.
 
@@ -133,12 +134,12 @@ This last technique we're going to learn about is very straightforward and easy
 
 Depending on the problem, you may want to choose one of these two options. In our case, we'll implement the second option:
 
-![draw 2](./images/draw_1.png)
+![draw 2](./images/draw_1.PNG)
 > **Note**: to implement the first option, you just need to *increment* the variable every time, instead of setting it. However, you might benefit from looking at implementations like [DeepSORT](https://github.com/ZQPei/deep_sort_pytorch) or [Zero-Shot Tracking](https://github.com/roboflow/zero-shot-object-tracking), which is able to recognize the same object/detection from sequential frames, and only count them as one; not separate entities. 
 
 With our newly-created global variable, we'll hold a value of our liking. For example, in the code above, I'm detecting the _`mask`_ class. Then, I just need to draw the number of detected objects with OpenCV, along with the bounding boxes on top of the original image:
 
-![draw 1](./images/draw_2.png)
+![draw 1](./images/draw_2.PNG)
 
 As an example, I created this GIF.