I am small in both, just like I am small in everything else. I’ve a climbers, not sprinters, build. The go a size up advice seems to derive from those who have wide shoulders or wide belly, not runts like me.
Castelli is slightly slimmer than Rapha generally. Rapha depends on the range – Pro range is slim, City range is a touch more relaxed (but still slim)