Commulative Counting of different types of vehicles

[hafsaahmed96]

Hi! I have integrated the object Counting API for the model I have trained on my own custom dataset. After integrating the vehicle counting/object counting api, the customs classes (bikes , cars , bus , rickshaws) are being detected but they are not being counted separately. On video, I am getting total count of all vehicles only. Can you please tell me how to get commulative counts of each vehicle type as they pass ROI ? I urgently need help in this regard. Thanks in advance.

[mssudarsan]

Hi. I am also having the same issue that when i integrated the object Counting API for the model I have trained on my own custom dataset with vehicle types like car, motorcycle, truck, and bus are being detected but I'm not getting count of detected vehicles.

And when I use a different video I am not able to get any output.

Can you please tell me how cummulative count technique helps and works to count vehicles when they pass ROI ?

Thank you.

[hafsaahmed96]

for integrating, I did some changes like setting inference graph loc to my location of inference graph, changing the model name to my model. After that, It was able to detect on my videos and write on video. The output would be stored in the form of output video. For the another video, you would get output but it will be overwritten on the first video output.

[hafsaahmed96]

@ahmetozlu Please can you tell us what to do or what changes to do in order to get cumulative count of different objects after passing through ROI? We would be really thankful

[Bhuvan-MR]

Hi @hafsaahmed96 I want to integrate yolo v3 model of video object detection to object counting api.
Can you please help me out doing this. I would be really thankful.

[hafsaahmed96]

Sure. Is the model from Tensorflow? The main thing is to set the path of you frozeninferencegraph and labelmap correctly.
Go to the folder tensorflow_object_counting_api-master folder which you would have downloaded/cloned. Then you will find the "utils" folder there. go to the file name backbone.py. Change line no. 18 and 21 (change the path to your frozeninferencegaph and labelmap respectively.

for example:

path_to_ckpt = "/content/drive/MyDrive/models/research/object_detection/inference_graph/frozen_inference_graph.pb"

# List of the strings that is used to add correct label for each box.
path_to_labels = "/content/drive/My Drive/models/research/object_detection/training/labelmap.pbtxt"

..
Now, the next thing is configuring in the script. (vehicle counting, real time counting or whatever your application is)
For example, if you need to do vehicle counting, you have to configure few things
type the name of you input video in line no.14
in line 17, write the full name of tensorflow model you're using, in my case, It was faster rcnn so I wrote it like this

detection_graph, category_index = backbone.set_model('faster_rcnn_inception_v2_coco_2018_01_28')

------Things to make sure-----
*put the video you're going to process in the same directory as the script you'd run
*the name of utils folder in this object counting master folder is same as the one inside the "object detection" folder so this may cause confusion. I implemented that in google colab so I had to shift some files from here and there.

[hafsaahmed96]

One more thing, I downloaded from this
https://github.com/ahmetozlu/tensorflow_object_counting_api

instead of from this repo

[Bhuvan-MR]

@hafsaahmed96, No the thing is I'm using yolo v3 model for detection which is a ".h5" file which doesn't have any inference graphs. I used the same tensorflow model which you have used but the model is not detecting the vehicles properly in the video I'm usinig, but the yolo v3 model is detecting properly. So I want to integrate the yolo model to this counting api. Please refer this https://github.com/OlafenwaMoses/ImageAI/blob/master/imageai/Detection/VIDEO.md#documentation and go through the documentation. I want to combine this detecting model to this tensorflow counting api. That means the detection graph for my model is the box points provided by output arrays, and category index is the classification that the yolo model identifies. So my backbone.py file should have changes wherever model is concerned.

[hafsaahmed96]

@Bhuvan-MR Since I'm a beginner and it's my first time trying the model, unfortunately I can't guide you much. However, for faster RCNN, I did not use it directly. First, we trained the model on our own dataset and then we integrated the custom-trained model. It does detect the classes I trained it on

[hafsaahmed96]

@amaadmirza No, I haven't solved the problem. I'm still looking for ways to get the class/ label name the moment it is detected so that I can store the count in some variable. I would appreciate if I could get some help

[Bhuvan-MR]

@hafsaahmed96, okay please tell me how to integrate the model to this counting api.
I think that might be helpful in some way for me to combine the model I am using and the counting api.

[ChampionTej05]

I can provide you the small hint, in the function "visualize_boxes_and_labels_on_image_array" in the visualization_utils.py , instead of returning counter,csv_line return class_name, counter, csv_line and then in the main function, just maintain separate counters of every category and update them according to the value of "class_name" parameter received from visualization_utils.py file.

[ahmetozlu]

• To be able to count different types of vehicles, please change the line#505 in visualization_utils.py then store each of the detected object with different variable and print the result by using that separated variables: https://github.com/ahmetozlu/vehicle_counting_tensorflow/blob/master/utils/visualization_utils.py#L505

• To integrate the model in this counting API, please change the line#50 in vehicle_detection_main.py: https://github.com/ahmetozlu/vehicle_counting_tensorflow/blob/master/vehicle_detection_main.py#L50

[Bhuvan-MR]

@ahmetozlu but there is no .pb file or inference graph available for yolov3 model.

[ahmetozlu]

You can call the API functions from your yolov3 model back-end.

[Bhuvan-MR]

@ahmetozlu yolov3 is just a .h5 file. Can you please tell me how do i call those api functions from the back-end. Thanks.

[sidharthskumar]

did u solve it?

[sidharthskumar]

@Bhuvan-MR Since I'm a beginner and it's my first time trying the model, unfortunately I can't guide you much. However, for faster RCNN, I did not use it directly. First, we trained the model on our own dataset and then we integrated the custom-trained model. It does detect the classes I trained it on

can u send the the trained model i can use too?

[hafsaahmed96]

@ahmetozlu

• To be able to count different types of vehicles, please change the line#505 in visualization_utils.py then store each of the detected object with different variable and print the result by using that separated variables: https://github.com/ahmetozlu/vehicle_counting_tensorflow/blob/master/utils/visualization_utils.py#L505
• To integrate the model in this counting API, please change the line#50 in vehicle_detection_main.py: https://github.com/ahmetozlu/vehicle_counting_tensorflow/blob/master/vehicle_detection_main.py#L50

Would that give me the class name for only the vehicle that crossed the line? or is it possible it can give me any detected vehicle class name even when the counter is 1

[hafsaahmed96]

My results are not being accurate. if three bikes pass through the ROI line, It detects one only. My test data is of more dense traffic. I want to get the class name of every vehicle crossing the line. For example, if the count for bike should have been 15, I'm only getting 9 even though it is detecting every bike but not counting them all.

[hafsaahmed96]

For example, these lines of codes are using detected vehicle image and comparing its axes to see if it crosses the line. How can I also send detected vehicle image's label or class_name as an argument in the same function? (object_counter_x_axis.count_objects_x_axis) ?

 '''if(bottom > ROI_POSITION): # if the vehicle get in ROI area, vehicle predicted_speed predicted_color algorithms are called - 200 is an arbitrary value, for my case it looks very well to set position of ROI line at y pixel 200'''
  if(x_axis[0] == 1):
    predicted_direction, is_vehicle_detected, update_csv = object_counter_x_axis.count_objects_x_axis(top, bottom, right, left, detected_vehicle_image, ROI_POSITION[0], ROI_POSITION[0]+DEVIATION[0], ROI_POSITION[0]+(DEVIATION[0]*2), DEVIATION[0])
  elif(mode_number[0] == 2):
    predicted_direction, is_vehicle_detected, update_csv = object_counter.count_objects(top, bottom, right, left, detected_vehicle_image, ROI_POSITION[0], ROI_POSITION[0]+DEVIATION[0], ROI_POSITION[0]+(DEVIATION[0]*2), DEVIATION[0])

[amaadmirza]

@hafsaahmed96
def visualize_boxes_and_labels_on_image_array_x_axis()
temp = ""
if (("motorcycle" in display_str_list[0]) or ("car" in display_str_list[0]) or ("truck" in display_str_list[0]) or ("bus" in display_str_list[0])):
try:
temp = display_str_list[0].split(':')
temp = temp[0]
except IOError:
temp = "None"

you can detect and get the label through this chunk of code..

[hafsaahmed96]

but how would I relate this label to the detected_vehicle_image argument given in the function in the above code. This function is responsible for comparing the position of the detected vehicle image with the roi and if it does crosses the ROI line, it simple returns the true (1) in the variable "is_vehicle_detected". what I want is that it somehow can access the label of the same detected_vehicle_image's label if it has crossed the ROI so that I can increase the count of the bus, cars, bikes, bus etc.
Take a look at the following code of the function I'm talking about. It uses crop_img variable to compare the postion of the detected vehicle. That would have been really great if I had known how to access the crop_img's label too!

is_vehicle_detected = [0]
bottom_position_of_detected_vehicle = [0]

def count_objects_x_axis(top, bottom, right, left, crop_img, roi_position, y_min, y_max, deviation):   
        direction = "n.a." # means not available, it is just initialization
        isInROI = True # is the object that is inside Region Of Interest
        update_csv = False

        if (abs(((right+left)/2)-roi_position) < deviation):
          is_vehicle_detected.insert(0,1)
          update_csv = True
          image_saver.save_image(crop_img) # save detected object image

        if(bottom > bottom_position_of_detected_vehicle[0]):
                direction = "down"
        else:
                direction = "up"

        bottom_position_of_detected_vehicle.insert(0,(bottom))

        return direction, is_vehicle_detected, update_csv

In the attached picture, it shows the bus count to be 3 even though no passed has passed the ROI but it was detected once or twice above the ROI line

[amaadmirza]

you can just pass through "temp" argument in sub function and call in this function like

def count_objects(top, bottom, right, left, temp, crop_img, roi_position, y_min, y_max, deviation):   
        direction = "n.a." # means not available, it is just initialization
        isInROI = True # is the object that is inside Region Of Interest
        update_csv = False
        if(temp == "car"):
          # Roi_position
          if (abs(((bottom+top)/2)-350) < 1):
            is_vehicle_detected.insert(0,1)
            update_csv = True
            image_saver.save_image(crop_img) # save detected object image
        elif(temp == "bus" or temp == "truck"):
          if (abs(((bottom+top)/2)-300) < 5):
            is_vehicle_detected.insert(0,1)
            update_csv = True
            image_saver.save_image(crop_img) # save detected object image
            # 
        if(bottom > bottom_position_of_detected_vehicle[0]):
                direction = "down"
        else:
                direction = "up"
        bottom_position_of_detected_vehicle.insert(0,(bottom))
        return direction, is_vehicle_detected, update_csv

[amaadmirza]

you can use this function

def visualize_boxes_and_labels_on_image_array_x_axis(current_frame_number,
                                              image,
                                              mode,
                                              color_recognition_status,
                                              boxes,
                                              classes,
                                              scores,
                                              category_index,
					      targeted_objects=None,
                                              y_reference=None,
                                              deviation=None,
                                              instance_masks=None,
                                              keypoints=None,
                                              use_normalized_coordinates=False,
                                              max_boxes_to_draw=20,
                                              min_score_thresh=.5,
                                              agnostic_mode=False,
                                              line_thickness=4):

and filter upon your required type like this

if (("motorcycle" in display_str_list[0]) or ("car" in display_str_list[0]) or ("truck" in display_str_list[0]) or ("bus" in display_str_list[0])):

after that you map your label with bounding boxes til this function

def count_objects_x_axis() function

[hafsaahmed96]

would it be to okay to use it in the function visualize_boxes_and_labels_on_image_array_y_axis because of the position of my data set and the script for counting vehicles is calling this function

[amaadmirza]

@hafsaahmed96 yes..

[hafsaahmed96]

I tried it all but it's not giving me the output as it should have. Unfortunately, It simply gives me the label of the vehicle detected at the moment if any other vehicle crosses the line. It also doesn't count every vehicle that cross the line simultaneously. It may count around 1 in 5 vehicles or 2 in 5 vehicles and give me the label of any third vehicle. I really need to solve this problem at the earliest. The function that compares the xmin, ymin of the vehicle uses detected_image and it's respective x_min, y_min. Can't there be any way to also access detected_image.label ( I know it's incorrect but just for example)

[amaadmirza]

@hafsaahmed96 you can count all vehicles simultaneously by dividing the ROI on each lane. In your example you have 4 lane so you can set the logic and count...

[hafsaahmed96]

@amaadmirza As you can see from the picture, the traffic is quite heterogeneous and it does not follow the lane discipline. There is no separate lane based on vehicle type, therefore We can't apply separate ROI here. Here's another shot of the traffic.

[haiqalma]

@hafsaahmed96 can i get your email? i have a some problem and i want ask to you :)) thanks

[amaadmirza]

@hafsaahmed96 you can check it through print statement before return that all vehicle that passes through ROI is in printable or not... secondly change the counting formula. Focus on other bounding box values rather than centroid..

[linchunmian]

hi, @hafsaahmed96 @amaadmirza, i am a beginner that conducts a project related to vehicle detection and counting, and similar problems have been reported in my trial. I have run 'vehicle_counting' and 'tensorflow detection and counting', but i encountered two similar problems:
firstly, the value of vehicle passing through the ROI position cannot match to the counting;
secondly, at the beginning of resulting video, there was some irrelevant bounding box labelled 'smurf'.
I am very confused for these problems, can you please help me or guide me to fix? Thanks in advance.
My email addressis '[email protected]'. If any necessary, we can further contact by e-email.

[ZIKO94ZIKO]

HI @hafsaahmed96
Can you please send me your code source Thanks in advance.

[WerdaFarooq]

Hi, @hafsaahmed96 can you please share with me that how did u performed commulative counting of different vehicles at ROI line position.
TIA.

[hafsaahmed96]

@WerdaFarooq @ChunmianLin I tried all the methods stated here but I wasn't able to perform it correctly. It didn't work. That's because I couldn't get the classname of the bounding box detected at the same moment. It mismatched.

[hafsaahmed96]

@ZIKO94ZIKO I only used tensorflow counting API and tensorflow Object detection API. So there's no source code that I may have written on my own. I only did some configuration. But commulative count for detecting different vehicles didn't work correctly.

[hafsaahmed96]

Commulative count works correctly for total vehicle count but not for commulative count of different vehicles

[ZIKO94ZIKO]

Thanks @hafsaahmed96 for your feedback ,actually i'm working on in project about detect lpr and speed cars in real time that is to say i must using the camera input , you have an idea about this project ,

[WerdaFarooq]

@hafsaahmed96 i just wanted to know that how did you performed cumulative counting of different vehicles at ROI line position?