I have had countless cats over the years, they are by far my favorite animal, If you are not planing to fix him, can I ask why?
In my experience that is the krux of your problem.
If he is an outdoor cat, then you are most likely contributing to more kittens being born....we don't really need...