Isn't racing also training? Do it!
Unless there is a reason other than you have give ... race. If you have good legs the rest will come. Stay at the pointy end as long as you can in each race and enjoy ...
Race - I'm in a similar situation, just started biking and getting in shape recently and got dragged into CX by some more serious friends. Did my first two races ever the last two weekends, finished 52nd and 61st. It's humbling to finish so far back, but it's also motivation and a whole lot of fun. There are people of all levels to race with, close competition for "not last" is still close competition!
RACE NOW. You might not be able to in the future and you will look back at this as a missed opportunity.