Im facing this error while converting onnx to rknn format, how to solve this ? Im using trained yolov8 model initially. In order to use it in NPU im doing through this approach.
while exporting from torch yolo model to onnx, which opset you are using?
Try using opset version 17 or lower and that should fix it, perhaps.
hi @Thiru , i would suggest what @Prem asked you to do and also follow up upon this code and the following blog. i hope this helps!
Chat GPT suggest it is not related to my issue for the example code you sent. What is the possible way to solve this error ?
Actually im getting the same reshape error even i changed to opset 16 and opset 17. Chatgpt says to use yolov5 to avoid header issue but i have only yolov8 model. give me possible way to solve this issue.
ChatGPT says itmy error is not related to code, it is related to Graphics. so this code which you shared wont suites in my case
It is the issue in the export of model, so it is not related to some graphics issue.
Are you converting static fixed shape onnx input or dynamic shape input?
If you are converting static fixed shape input, just remove the dfl layer from onnx model (as that is only causing the conversion issue) and do that as post-processing as those nodes doesn’t have any real conv or matmul layer which could be accelerated on NPU (so it is safe to remove those layers).
Script for the removal of post-process layer can be found in @riddhi_raj 's code, or you can use gui based onnx surgeon kind of this to remove those layers.
So far i have created lot of onnx files but still the result is zero. because of yolov5’s accuracy i cannot use it. kindly help me to solve this issue and run the model for detection.
you need to remove the DFL layers - specifically the post processing layers from the model (look at DFL head issue) and then try converting and perform inference. thats what we have basically followed. you can find the script to remove nodes in the links previously provided in the Axon-NPU-Guide or use onnx-modifier to remove those nodes via a GUI
Hi @Thiru what chatgpt’s response is saying is that You must remove the entire YOLOv8 post-processing head, not just one reshape node , which is what exactly done in the code mentioned by @riddhi_raj and he had already tested it and that worked for many custom train yolov8 as well as yolo11 models. So that would work.
Besides, AI models are not fully aware of the environment and they don’t have much data, while training, on what actually works. So, it doesn’t always help while debugging for specific hardware like NPUs.
Let me briefly explain again what’s actually happening and what is the actual fix.
YOLOv8 models uses distributed focal loss (dfl) for predicting bounding boxes’ coordinates. This basically calculates the expected value of height/ width / centres from probabilities of it having different values (total 16 values having different probabilities) and calculating this final co-ordinates requires various reshape, softmax, scale, concat, etc operations (which can be easily done on CPU at better speed). Now, after this model produces a single output of shape (1, 84, 8400) (it may change based on number of your classes and input feature shape). What you have to do it is to remove that last processing layer and so there will be 6 output nodes and later you can perform that processing on CPU. And this will surely work because RKNN toolkit parses the graph topologically and since it has reached that reshape node so all nodes before it is already correct and can be parsed. It is a tested working way and you can follow it to obtain a convertible onnx model and then you can convert that. If anything is unclear, please ask!
2 Likes
I would like to pass video into this rknn model. I tried with GPT but it is not detecting and boxes are not shown. do you have any alternate way to fix it ?specifically for passing the video for detection.
what do you mean by passing video in the model? YoloV8 models take input one frame at a time.
yes do you have any code ? to pass a video into frame to pass into the model ? I tried with gpt code, but it is not working means not getting any detections .
What people do is read video frame by frame using opencv or ffmpeg and run detections frame by frame and then display output frame by frame, which automatically looks like video, as video is basically stream of frames. You can pass your code for image to claude and ask them to write a code to take input video path / url and display output with detections, and it should work.
All the codes I have are multi-threaded pipelined code and you would require to modify a lot in those for your custom pre and post processing, so it might not be helpful. Still a simple example for detecting and couting people in a video is here.
Ok i’ll try with it , Thanks for your support
Sure, we’ll close this thread.