Anas Sarwar's U-turn on supporting the Gender Recognition Reform Bill may have angered his MSPs who lost their frontbench roles.