Hello Rob, The plastic taste (?TCP?) is is produced by a chemical in used to manufacture the lining of the water pipe. In some pipes it is not adequately fixed in the material and so begins to leach out in to the water being carried. It should not happen. If your caravan is in warranty, then contact your dealer and insist on having the pipes changed, try this even if it just outside the normal warranty period, they might still replace them.
There is no real excuse for this problem, as plastic pipes are successfully manufactured for domestic water supplies which do not taste. Such pipes are available from most good plumber merchants in 10, 15 and 22mm diameter, with quick fit connections. You could re plumb with this.