Richard you explain it well, the worms create air and drainage channels, and other soil life help out too. I find the channels made by worms create a more enduring structure than when soil is broken with tools. So David, this saves you a job.
Also an important aspect of compost, and old manure which is effectively compost, is that nutrients are held in water-insoluble form. A few will leach out but most just stay there until roots and mycorrhizal fungi seek them out, both in soil and near the surface in the compost. Undisturbed soil life is good at helping to feed plants.