networking

3003 readers

12 users here now

Community for discussing enterprise networks and the ensuing chaos that comes after inheriting or building one.

founded 2 years ago

MODERATORS

[email protected]

🏚️📞😱 Something is making HTTP requests as Googlebot from my home network (lemm.ee)

submitted 1 week ago by [email protected] to c/[email protected]

8 comments fedilink hide all child comments

A weird and disturbing thing is happening on my home network. I'd like some advice on how to diagnose it. My mastodon host (chaos.social) keeps blocking my IP address. I reached out to the admins and they told me it's because they are getting HTTP requests with user agent string claiming it's a Google bot. They shared a following log line with me.

[12/Mar/2025:08:55:14 +0100] my.ipv4.add.ress "GET /@lazurski HTTP/2.0" 403 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

It is my IP address indeed, and the path is pointing to my profile, so it's not random. It also happened while I was browsing Mastodon using Firefox on my laptop. The 403 response is strange, as I was logged in and also my profile is public and viewing it doesn't require authentication. Maybe they blocked it because of the bot signature?

I have no idea what can be making these requests. Certainly not anything I run on purpose. My Firefox uses it's standard user agent header. At home I have a few devices. At the time of this request I believe only the following were on:

my laptop running NixOS and Firefox (I was actively using it when I got blocked)
a RaspberryPi home server running NixOS
my Android phone running Tusky (a 3rd party Mastodon client)
a broadband router with stock software

I think I can exclude the phone from the suspects, because while the home IP is blocked I use my mobile network connection to access chaos.social and this IP is never blocked. I don't think it's the home server or the router. My suspicion is on Firefox extensions. I only use a few of them:

How can I troubleshoot it? I tried about:logging with networking preset, collected a ton of logs, but couldn't figure out what to do with it. Or maybe it's something completely different? 🤔

top 8 comments

sorted by: hot top controversial new old

[–] [email protected] 15 points 1 week ago (2 children)

You could fire up wireshark and see if and when the traffic is actually originating from your PC, then it's just a matter of turning off extensions.

If it's not firefox, a tool like lsof will be helpful to see what pid is connecting out.

[–] [email protected] 8 points 1 week ago

Wireshark is nice but TLS is going to make sniffing this difficult. Unless you MITM your traffic.

You could start by setting up a local webserver which you can watch the logs for. That let's you see the request and response without getting mastondon involved. Then start turning shit off. Software or hardware that might be doing it. Try with a different browser first because that rules out a bunch of things.

It the other browser is negative then start checking your extensions.

If its still positive after you've tried everything, move to a coffee shop and try again. That tells you if itsyourr home router or something else in your house.

Finally, if its still happening at the coffee shop it's 100% on your laptop.

Try booting a USB live boot image and see if you still have it.

[–] [email protected] 5 points 1 week ago

Yeah, wireshark is a good start.

Something else to consider but requires a different device would be to add a firewall (something like a Protectcli device running OPNsense) between your modem and router and set it up to block the outgoing request and see if it breaks something, or at the very least if you're concerned about security, you're blocking that specific traffic while you troubleshoot the cause using wireshark or some other method.

[–] [email protected] 15 points 1 week ago (2 children)

Is it possible you have some IoT devices on your network such as smart outlets, light bulbs, I don't know, smart toasters, whatever? IoT devices are notorious for zero day exploits and become zombies frequently.

[–] [email protected] 3 points 1 week ago

Good point, but no. I avoid this crap like a dirty nappy.

[–] [email protected] 2 points 1 week ago

Second this! At this point your cats litter box has a cryptominer malware installed and it is pushing that SoC to its very limit.

[–] [email protected] 6 points 1 week ago (1 children)

if its an addon doing it, wouldn't a string like 'googlebot' be in a readable file in the addons dir or one of its xpi (zip) files?

[–] [email protected] 2 points 1 week ago

Good idea. I'll grep it and see.