Fiqh-us-Sunnah Volume 5, Hajj and ‘Umrah, Fiqh 5.128.
Section : Going Back Home After the Hajj.
Abu Hurairah reported that the Prophet said. “Travelling is but a sort of torture. One is prevented from (properly) eating and drinking. So when one of you (travels and) accomplishes his work, he must hasten back to his family.” This is reported by Bukhari and Muslim. ‘Aishah reported that the Prophet (peace be upon him) said, “When one of you has performed Hajj, he should hasten back to his family, for that will increase his reward.” This was reported by Ad-Daraqutni.
Muslim has reported from Al-‘Ala bin Al-Hadrami that the Prophet (peace be upon him) said, “A pilgrim from abroad may stay for three (days) after completing his Hajj.”