Sue I think you can do this. Maybe a catch crop or interplant of spinach, lettuce or anything different between growing cauliflower, to keep a wider range of soil life interested.
I would definitely never remove and replace soil. Just add the two inches per year of compost as cauls’s are hungry for food of all kinds.
See my video on rotation for more info https://www.youtube.com/watch?v=WXQNnJOBQJc
