Fiqh-us-Sunnah Volume 5, Hajj and ‘Umrah, Fiqh 5.076.

There is no harm in crowding around the Black Stone so long as no harm is caused to other people. Ibn ‘Umar used to struggle hard to get through the crowd in order to reach the Black Stone, so much so that sometimes (he was struck on the face and) his nose would bleed. The Prophet (peace be upon him) said to ‘Umar: “O Abu Hafs! You are a strong man. Do not crowd others around the Black Stone. You may hurt a weak person. But if you find an opportunity to reach it in order to kiss it or touch it, do it. If not, then say ‘Allahu-Akbar’ and continue (your tawaf).” (Reported by Ash-Shafi’l in his Sunan)