Hmm there is some nonsense out there!
Firstly, there is nitrogen in the poo, for example from the residues of grasses and hay.
Secondly, this statement you read might have been written by a manufacturer of chemical fertiliser as its hidden implication is that growth does not happen without plenty of nitrogen. Yes veg need it, and compost has enough and in a slow-release form too, whereas larger amounts of available N cause problems of sappy, unbalanced growth which many insects are attracted to.
So you are doing just fine!
