I'm sure the so-called antis don't care whether it's 2 wheel drive, 4 wheel drive, car or trucklike design, their argument is basically an attack on the need for extra energy (fuel) needed to move unnecessary deadweight and the consequent pollution caused by burning the extra fuel. Their criticism is surely not levelled against using such vehicles where necessary for commercial purposes, only when used privately and especially when seldom for towing or offroad.
Objectively, except in exceptional circumstances such as frequent off-road or winter use, there is absolutely no real need for a 4x4, off-road vehicle or whatever you want to call it tow a caravan. Even the largest caravans can be legally towed by a saloon car so long as the maximum permissible towload is adequate. Those who claim they need anything bigger do so because they feel safer to stay below the 85% ratio not because they have to.
Actually, contrary to what Clive says, the safety performance of big 4x4's is not, on the whole, that good. Hit a brick wall at 30mph and you'd probably find that your chances of survival are better in a modern saloon than in most big 4x4's, simply because they don't crumple very easily and don't absorb the crash energy as well. OK, if you hit a small car you'd be better off but that's a rather selfish attitude, "I'm alright, Jack, (and all my passengers), never mind the rest".
Therefore, people who tow with big 4x4's do so simply for emotive reasons, same as those who like convertibles and it's pointless to counter opinions with rational arguments.