You can live peacefully by married life and have sex life for children, good children. That is allowed. But why should you have illicit sex life? What is this? Irresponsible life.

"I shall not take responsibility of family life, and I shall go on as so-called bachelor and have illicit life." But that is not good. That is garhitam. Na sadhu manye: "This is not good." This is the advice of Vedic literature.

Ultimately you'll suffer. You are thinking at the present moment that you are avoiding suffering because family life is very responsible life. So if you think that family life is responsible, yes, it is responsible life. If you don't take, if you cannot take the responsibility, then remain as a brahmacari. Why should you marry?

If you practice brahmacarya, then you become free, seventy-five percent freedom immediately. But you do not want to undergo the austerities of a brahmacari, and still, you want to remain as an unmarried man.

(Srila Prabhupada —  December 12, 1970, Indore)