We've been using campsites (in trailer tents, folding campers and caravans) as a couple. with four children or latterly, with grandchildren, since 1985 and have yet to have a stay that has not been mostly, and usually wholly, enjoyable. Nor, to the best of my knowledge, have we ever done anything to spoil anyone else's pleasure in their break.
As has been said, most of the problems are in the mind - if you go away with the idea that everyone else must conform to your ideal of enjoyment, then I am afraid you are doomed. If you start from the premise that you will do what you enjoy, surrounded by other people who are also having a good time in their own way, then you will have no problems. We have never used 'adult only' sites, nor ever been tempted to. Most of the children we have encountered have been just that - children. Not angels, but certainly not litttle devils. Sometimes adults have been a little noisier than the ideal, but we have always been able to carry on doing what we enjoy (a quiet drink, a good book, a decent meal) with minimal disturbance from others.
I am not saying that horrendous events, such as you may have heard about on here, never occur, but they must be pretty rare, since in twenty years of caravanning ( and for the last few years we have used the 'van pretty regularly from February through to November) we have never had a bad experience.
Just get out there - and enjoy yourselves!