Maliks Muwatta Book 5, Hadith Number 11.
Section : Paying Attention when the Imam is Giving the Khutba on the Day of Jumua.
Yahya related to me from Malik that he asked Ibn Shihab about talking in the jumua after the imam had come down from the mimbar but before he had said the takbir. Ibn Shihab said, “There is no harm in that.”