I use Primagaz Twiny propane cylinders with the appropriate regulator, all of which is available from DIY outlets and most petrol stations.
If your Bailey is fairly recent with a bulkhead fitted regulator, then you just need to buy the appropriate "pigtail" to match the cylinder that you buy.
A much more difficult problem is to get the caravan registered in France (they have their own registration number and test).
You will need to obtain a "certificate of conformity" from Bailey but be aware that some French departments do not accept the door on the "wrong" side.
You would probably have less of a headache by selling the Bailey in the UK and buying a French registered caravan when you move.
If you are going to tow the caravan on Brit.plates you will be OK until such time as you have to re-register your car or acquire a French registered one.
You could put your Bailey on a site permanently and use it without towing OK.