Ok, so I am still not any closer to figuring out what the issue is, but I had it earlier today on my mobile phone when connecting through a mobile data plan. Upon returning home I switched back to wifi and the issue was gone... strangely even when switching back to the mobile dataplan afterwards. Maybe it is somehow related to switching IPs while connecting from different locations or so?
(One pattern of AI scrapers is that they run a full browser server once to get real cookies and so on, and then transfer those to a cheaper server to do the actual scraping, so maybe a mobile data-plan with changing IPs accidentally triggers this).