Fiqh-us-Sunnah Volume 1, Purification and Prayer, Fiqh 1.017B.

One should not respond to a greeting or repeat what the caller to prayer is saying. He may speak if there is some necessity (i.e., to guide a blind man who fears he may be harmed). If he sneezes, he should praise Allah to himself and simply move his lips (without making a sound). Ibn ‘Umar related that a man passed by the Prophet, upon whom be peace, and greeted him while he (the Prophet) was urinating. The Prophet did not return his greeting. (This is related by “the group,” except for al-Bukhari.) Abu Sa’eed reported that he heard the Messenger of Allah, upon whom be peace, say, “Isn’t it true that Allah detests those who converse while they relieve themselves?” This was related by Ahmad, Abu Dawud and Ibn Majah.

This hadith seems to support the position that it is forbidden to talk. Many scholars, however, say that it is only disliked, not forbidden.