Maliks Muwatta Book 29, Hadith Number 51A.

Yahya related to me from Malik from Nafi that Abdullah ibn Umar said, “If a man gives his slave permission to marry, the divorce is in the hand of the slave, and nobody else has any power over his divorce. Nothing is held against a man who takes the slave-girl of his male slave or the slave-girl of his female-slave.”