Welcome to the site, it sounds like you are doing well already, except for disturbing your soil with a fork! Stopping that will conserve some extra fertility and help maintain a softer surface.
An inch of compost every year should be enough but if your soil was low in organic matter to start with, some more would be good, and the fact that it disappears over a season shows the hunger of your worms and other soil life. Is there any municipal composting near you?
In the 1940s there were also successful no dig gardeners who used thnner layers of compost, with some sawdust on top of their soil (F.C.King, James Gunston).
