Hello James,
If the bits are hard, then it is most likely to be from a water filter. Usually when filters start to loose their active carbon content it means they need changing.Check your hand book for details of the model fitted to your caravan.
The granules are not dangerous unless consumed in very-very large quantities.
If they are soft, then it is algea growth and the pipes need major cleaning to remove the infestation. worst case the pipework may need replaceing.
For hard particles:
When the filter is changed flush a coupole of barrels of water through the system to remove any loose particles.
On the down side, some of the bits may lodge in non return valves fitetd to the pressure switch (if you have one) or water heater preventing the valves from closing fully. Tell tale signes can the pump turning on and off when all the taps are shut, or hot water comming from the cold taps. The relevant valves need to be carefully opened and cleaned.
For soft particles, you may need to fill your system with a sterilising solution for example Milton or equivalent, Hydrogen peroxide, Chempro-SDP. Which ever you choose always follow the manufacture instructions with regard to the dilution. Warnong - do not leave agents that are agressive towards metals in the sytem for longer than the recommeded times.
After cleaning alwasy flush through with a couple of barrels of feresh water.
If you are in any doubt about what is happening seek advice from your local caravan service agent.