I don’t see why it would be harmful. Sometimes, there is only so much stuff in there. When fasting, for example, sometimes you’ll have a BM. Better for that to happen then not.
There are some conditions like a narrowed colon that can cause smaller stools, but you can’t logically go from there to “small stools are bad.”