Inference Inside OpenCV or Dlib
Jump to navigation
Jump to search
Reasons Not To Perform Inference With OpenCV's or Dlib's DNN
OpenCV’s DNN will accept TensorFlow models and pytorch models but it seems like there are problems getting it to use CUDA and NVIDIA's GPU’s[1][2] which is one of the reasons why we use an external neural network server
Dlib’s DNN can use Nvidia GPU’s internally but can’t import any models but its own.[3] So no TF, Pytorch or even Caffe.
Other reasons why we don’t want to perform inference in our image processor is, in order of importance:
- Modularity
- The other is that often it’s more feasible for relatively lightweight things like frame logic etc. to be done on one machine (like a Raspberry Pi) but for the heavy lifting like DNN inference to be offloaded to some external and much more powerful machine
- We don’t want to put all our eggs in the one OpenCV or Dlib basket.