Sahih Bukhari Volume 2, Book 26, Hadith Number 655.

Narated By ‘Aisha : Allah’s Apostle said to me, “Were your people not close to the Pre-lslamic period of ignorance, I would have demolished the Ka’ba and would have rebuilt it on its original foundations laid by Abraham (for Quraish had curtailed its building), and I would have built a back door (too).”