Half my social circle has gone vegan at this point and I think a lot of the anti-vegan sentiments is people don't like modifying their behaviour to give up their own comfort even when they know something is distressing to someone else. Since a lot of vegans see a very real cruelty that they are generally powerless to stop and other people do not understand their reactions to seeing other people participate in cruelty is often to feel very sad. Since so much of human culture surrounds shared meals having a vegan takes a lot of options off the table entirely and alters other people's options even when they don't intend to.
Like it's not a matter of "well we'll go to your vegetarian restaurant this time and next time we go to a place I'm excited to go" for those of us who care about our friends being upset we basically rarely pick our first choices and more often sacrifice things we are excited for in the name of someone else's comfort. It can be a love language to find restaurants and eat the things on the menu that don't exactly thrill you but other times you just want to have that selfish Birthday dinner where you don't feel compelled to pick a restaurant for someone else.
I think a lot of people reject veganism more forcefully because they don't want to have to participate in that sort of friction. All it takes is one ethical vegan to completly change a friend groups food culture. Even when they bring their own food and try not to make a big deal and mask it not bothering them when they see meat being consumed people are generally compelled to care for people they know and ignoring someone's distress isn't showing care. When people ratchet up the social cost of veganism they are more often than not trying to engineer a social sphere where they do not feel callous, don't have to give up what they like and don't have to do any additional research work or social calculations .