Non-autistic take, and Ive been on both sides.
If you really aren’t comfortable, bring it up with the person later. I have done this, and had this done to me as well.
“I was thinking about what happened the other day, I said I was Ok at the time but I am really not Ok …”