We visited Edinburgh last year and stopped at Linwater caravan park at east calder. This is about 5 miles out of the city. You would need to use the car to get to the outskirts but you could park and ride.
The site is level and has first class facilities kept clean at all times. The owner (Jean) is very welcoming and helpfull. We had such a good time there nothing else seems to compare now.