Plenty of old churches Lisa. Try Kilpeck church, built around 1140, and see if you can spot the pornographic gargoyle along the eaves :O) This church is just off the Hereford - Abergavenny road and if you're staying in Brecon a nice ride is from Hay-on-Wye to Pandy via Capel Y Ffin. It's a narrow road with passing places. You will pass a ruined priory at Llanthony. Not too far from here is another abbey at Abbeydore.
There is the cathedral at Hereford itself, if you are interested a friend of mine could take you up to the top of the tower if he's free, just let me know the date and I'll check.