You problem is finding a site which will be open next week - it's very early season for French sites and though some will be open, there won't be that many.
La Croix du Vieux Pont at Soissons (not that far south) is open all year, and has an indoor pool, but they are apparently refurbishing it, so you'd need to check whether it would be open. Lac d'Orient near Troyes has an indoor pool and opens on 27th March. Lac de Panthier at Vandenesse (near Dijon) has a nice indoor pool, and is open from 3rd April. Camping Le Ranc Davaine in the Ardeche has a lovely pool complex, including an indoor pool, and opens on 1st April.
You can have a look here, and find historical weather records for a lot of towns throughout France - to check temperatures.