Sahih Muslim Book 4, Hadith Number 1333.

Chapter : Commencement of the time for the evening prayer is immediately after sunset.

‘Aisha reported: The Apostle of Allah (may peace be upon him) one night delayed (observing the ‘Isha prayer) till a great part of the night was over and the people in the mosque had gone to sleep. He (the Holy Prophet) then came out and observed prayer and said: This is the proper time for it; were it not that I would impose a burden on my people (I would normally pray at this time). In the Hadith transmitters by ‘Abd al-Razzaq (the words are): “Were it not that it would impose burden on my people.”