If the hose taints the water , then perhaps its not fit for purpose and should be replaced by the manufacturer.
I believe the UK caravan manufacturers are required to use potable water grade products in their caravans, so if your UK built caravans does produce tainted water then it is a matter you should take up with your caravan supplier. However becasue of the intermittent way in which caravans are used and the temperature extremes the caravan can encounter, you do need to undertake maintenance of the water pipes periodically to clean and sterilise them.
With properly cleaned pipework and appropriate water supplies there should be no problem using water from the caravans taps for drinking etc. It becomes a matter of personal preference whether you choose to use water for drinks etc from the caravan system or a sperate bottles.
If you don't clean your pipes then there may be a problem down the line.
Other connected pipework such a direct from tap systems, should also be suitable for potable water.
The issue with non approved hose pipes, is related to the materials used, some plastic colourants are highly toxic, and carcinogenic, which means the problems may not show immediately but several years down the line.