Hi Ray,
The rust stain is Iron Oxide, and that's insoluble in water and I don't think polish will help either.
Cillit Bang will dissolve the rust - and probably your caravan too - but something much weaker like Citric Acid (Lemon Juice) or Vinegar might just do the job - brush it on with an old toothbrush, brush for a minute or so, and then wash off with water.
If those don't work, then try Cillit Bang or Limescale remover - but only for a few seconds at a time - and wash off stright away.
Robert