I Do Believe We've Been Invaded

[Galadriel55]

Originally Posted by Mithadan

It is the sheer number of scrapers that bothers me. I used to be amused at the number of “guests” but now recognize that they affect the website’s resources (familiar word)?

Ahhh, so that's the reason.

[Elvellon]

There are currently 3959 visitors on the forum: me and 3958 guests. Those must be bots. I believe there's a way to ban these IPs. It makes it incredibly difficult to use the forum because every other time I click a link I get "Resource Limit Reached".

Edit: In fact, I got a "Resource Limit Reached" message while I was trying to post this!

[Mithadan]

They are bots or AI scrapers. The culprit appears to be the Movies forum for some reason. 80% or more of the "visitors" are congregating there. As I understand it, the 508 error, also known as "Resource Limit Reached", results from excessive traffic. The obvious solution would be to take down the Movies forum, but this would be an undesirable option, and there is no guarantee that it would cure the problem.

[Elvellon]

Originally Posted by Mithadan

They are bots or AI scrapers. The culprit appears to be the Movies forum for some reason. 80% or more of the "visitors" are congregating there. As I understand it, the 508 error, also known as "Resource Limit Reached", results from excessive traffic. The obvious solution would be to take down the Movies forum, but this would be an undesirable option, and there is no guarantee that it would cure the problem.

Look at Full-length LotR Parody Movies. It has 17.9 million views for a thread with 650 replies. Other threads on the board with over a million views have replies in the thousands.

There's something very strange going on there that's attracting the bots like moths to a flame. The admins really need to do something about this, it's making the board harder and harder to use.

[Huinesoron]

Originally Posted by Elvellon

Look at Full-length LotR Parody Movies. It has 17.9 million views for a thread with 650 replies.

Yeah, we've viewed it a lot while working on the Zong, but not *that* much.

I'm worried now that I've gone and linked that thread specifically somewhere that's drawing the locusts... I'll try and take a look round, because that's annoying if so.

hS

[Mithadan]

I did not notice that. Huinesoron, check on your end, and I will make some inquiries as well.

[Galadriel55]

Originally Posted by Huinesoron

Yeah, we've viewed it a lot while working on the Zong, but not *that* much.

I'm worried now that I've gone and linked that thread specifically somewhere that's drawing the locusts... I'll try and take a look round, because that's annoying if so.

I did share a link to that thread specifically with a few people, but I haven't posted it anywhere. I'll ask if any of them put it up somewhere else. Indeed, if all those views were real traffic, that would be quite the audience. Is there anything else about the thread that might attract them? Would the sheer number of hyperlinks somehow serve as a magnet?

[Huinesoron]

Originally Posted by Mithadan

I did not notice that. Huinesoron, check on your end, and I will make some inquiries as well.

Okay, so I didn't link to the Downs on any of the videos (which have no views anyway due to being semi-private), or on the Libretto document. I posted one link on another forum back in 2020; I've gone back and removed that, but it doesn't seem a likely source. That was this link to page 9; I don't know if there's any way to check which posts are specifically being viewed?

Originally Posted by Galadriel55

I did share a link to that thread specifically with a few people, but I haven't posted it anywhere. I'll ask if any of them put it up somewhere else. Indeed, if all those views were real traffic, that would be quite the audience. Is there anything else about the thread that might attract them? Would the sheer number of hyperlinks somehow serve as a magnet?

Or the number of pictures (mostly hosted on Imgur), or the number of YouTube links specifically? But I don't know of any actual mechanism for a bot farm to get hold of that information. AI scrapers (like the Wayback machine) would be a possible answer, except I think they work the other way round - they'd cluster somewhere because lots of incoming links lead them there, they wouldn't lurk there because of how much is going out.

I also don't understand why they would. They've sent 4000 bots to our thread today because, say, it links to lots of YouTube videos, and then.. what? What's the profit? What's the point? They're not even posting!

If the Downs tracks visiting IP addresses it might be possible to check them against lists of known scrapers. And if there's a tool to look at what exactly is getting the views (eg, are they coming in through the front page or straight to a post?) we could try fixing that specific thing. All I know how to see is the viewers per forum, not per thread.

hS