Possibly a bit out of your way, but La Croix du Vieux Pont at Berny Riviere just west of Soissons, could be classed as being between Laon and Reims - and the cross-country route from there to further north than Laon is very nice (through Chauny/Tergnier - where there is a great hypermarket). It has a special area for overnighting, which is very nice indeed, and has indoor and outdoor pools, restaurant and take-away on site, supermarket within the site, and plenty of activities for children. Lots of other Brits there, too.
There are other municipal sites in this area, and some may offer access to a municipal pool, but I don't have my Michelin guide to have a look at the moment.
Facebook
Twitter