Have never felt that kind of an effect. Our consumption is pretty small, though, with just a little from time to time.
Swerve comes in granular form, which might leave your chocolates “gritty”. But it also comes in powdered form, more like powdered sugar.
The worst side effect I’ve had from erythritol has been a slight headache when I first started using it and way overused it in something that I was eating a lot of. I haven’t had any issue since.
In general, I don’t find Swerve or erythritol to sweeten things quite as well as I would like it to. I find also find that stevia will sweeten but often has a “whang” to the flavor that can get overwhelming when I use too much. So it often happens that the recipes I like best tend to have both of them… at least so far. I haven’t tried monk fruit type sweeteners yet.
Good luck! I hope you can find something or some combination that works well for you. Sometimes it takes some experimenting as individuals really do taste things differently.