I'm noticing a huge surge in 404 errors on my website because a bunch of fediverse crawlers seem to think that beesbuzz.biz is a Mastodon instance. I wonder what gave them that idea.

It's not a huge deal or anything, but it's also a bit baffling why these crawlers have each been retrying every 10 minutes for the past few days. fediverse.space and fediverse.network are the two big ones but there's a handful of others as well.

@fluffy fediverse.network tries all the possible URLs

if it's happening every 10 minutes you've enabled monitoring

(log in and disable monitoring to make it slow down)

@ben How would I have enabled monitoring and how do I log in? I don't have an account there or anything, I've never even heard of this before just now.

@fluffy actually, it looks like fediverse.network doesn't have monitoring enabled for that domain


so the "every 10 minutes" thing is probably fediverse.space (which I've never heard of)

@ben okay, you're right, fediverse.space is doing it every 10 minutes. fediverse.network is doing it every 30 minutes. Which is still a lot more often than it should for something that is not and never will be a Mastodon instance.

@fluffy yeah, @href is aware the crawler needs better detection of non-instances

@ben @href I'd think returning a 404 on /api/v1/instance would be a pretty good indication :)

@ben @href Any idea why it would have started doing this just today?

@fluffy hmm, maybe some instance claims it's a peer? are there any activitystreams documents on there?

@ben I've been using Bridgy Fed to republish my stuff as @beesbuzz.biz for a while but that hasn't changed at all since last October. Unless something changed on Bridgy Fed itself today. @snarfed.org ?

@fluffy @ben @beesbuzz.biz nope, no recent bridgy fed changes. it definitely does make the fediverse think your site is an activitypub/mastodon instance, which will trigger requests like these, but that’s always been true. more details: github.com/snarfed/bridgy-fed/

@snarfed @ben @beesbuzz.biz I wonder what caused things to change such that I suddenly was getting the influx of traffic then.

@snarfed actually it looks like this has been going on for a while, it just only happens every few days and lasts for the whole day and then they stop for a while. Whatever.

· · Web · 0 · 0 · 0
Sign in to participate in the conversation
Queer Party!

A silly instance of Mastodon for queer folk and non-queer folk alike. Let's be friends!