No we don't. We know image metadata is not reliable as it is easily changed or removed. Just as humans will not need metadata to determine what an image contains, so do we. You can name your image 'safe.jpg' or 'nsfw.jpg' or whatever, that won't change the result.
The same holds true for the EXIF data, or any header / type information we might get from an image URL. The only data that we use for moderation are the actual pixel values.