We Need To Talk

Just in case: This was written by me as an independent author, and no part of it should be seen as the opinion of any business I work, or have worked, for.

This is a weird subject. I've pitched this a few places but nobody particularly wants it at their fun SEO conference, so I'm writing it in case it somehow helps somebody one day and can save them a whole heap of bother.

But first, context:

A little while ago I was working with a website that has a huge (and ever increasing) volume of UGC. At the time, (I think it's now changed) by default any UGC that was created on the site was publicly accessible and indexable, and the site also allowed you to upload and attach files as part of the UGC creation process.

This site's SEO team asked me to help them bolster a piece of work they were doing in order to get their C-Suite to take content moderation seriously, if only because spammy or otherwise dodgy UGC is a really good way of getting yourself a penalty, and they needed to understand how bad their problem was before it became an even bigger, multidisciplinary problem that could cost people jobs.

So off I went armed with their GSC and some crawl data and very quickly found... more than I had bargained for.

I expected to find torrents, cracked software, maybe even some NSFW content - but I wasn't expecting to find URLs that suggested links to CSAM and all the other absolute worst stuff on the internet - and I definitely wasn't expecting to discover that some of those URLs were high performing pages in the UGC folder on the site either. Maybe I'm a bit naive.

So, what do you do in that situation, or in a situation where there's a chance you'll run into something that most businesses would rather you didn't?

Well, I'm not an expert, but I can absolutely tell you what I did and would do next time if put in the same situation.

Tell IT What You're Doing

Depending on how bloodthirsty your IT/Infra/Security team are, what you're doing probably technically constitutes a breach of any AUP that you might have signed upon joining. They probably won't mind too much so long as you don't try to run anything or click anything (more on that in a minute), but informing the person/people who looks after the health of the computers in your business is always smart - they might even be able to spin you up a VM or provide some sort of airlocked system in order to do the work in in relative safety from Malware and other oogly-booglies. Best have them know what you're doing before you start, and if they refuse to play ball it's at least a conversation that is taken out of your hands.

Don't Click Anything

This advice ranges from "just a good idea" to "absolutely imperative" depending on what it is you find.

If you come across pirated media, maybe NSFW content, a naive click into that content is probably not the end of the world. I wouldn't make a habit of it, but confirming what that content is can be very useful when it comes to presenting this stuff to clients.

If, however, you believe that the content you've found might be more serious (I'm dancing round saying "if you believe what you have found could be CSAM"), then do not click anything. Don't touch it. Don't click it, don't check it is what you think it is, not out of professional obligation or morbid curiosity or anything of the sort, just take a screenshot of the url, save it somewhere, and MOVE ON.

I really cannot overstate how important that last paragraph is. If you take nothing else from this post, let it be that.

In the UK at least, clicking that link is not worth it from a mental health standpoint, it's not worth it from a professional standpoint, and it's definitely not worth it from a legal standpoint. I feel like I probably shouldn't need to say this, but unfortunately "I didn't mean to" doesn't really wash with the law, and you don't need months if not years of legal bother and all the possible repercussions that can come from it as a result of innocently trying to do your job.

But what happens if you have clicked something accidentally? This is where informing your IT team comes in handy. Nuke the VM. Reformat the HDD. Shred the HDD. Do everything you can to quickly and completely destroy and all trace of the thing you may have seen. This is really, really important, for all the reasons I mentioned before.

For the eradication of doubt, in my situation I never did see anything dodgier than the first few seconds of a few marvel movies, but I just want to lay out every possible circumstance you can find yourself in when doing work like this.

Present it Carefully

Maybe this is obvious, I dunno, but most clients don't want to sit on Zoom and see their URLs on a shared screen, filled with absolutely abhorrent keywords and phrases; so that leads us to the question: How do we show them what we've found and how serious it is?

Again, if you're showing them site.com/marvel-antman-1080p-WEBRiP, then I think a list of example urls in a document, or a screenshot of GSC or whatever, does a good job of hammering home what they're looking at; but if it's something you feel that either you or they would be embarrassed to have on a screen during your call, then what you can do is provide that same content in a zip file in order to share with the client that can be looked at after the call in a scenario your client is more happy with.

Please note, I recommend a zip (or other compression) format because many file sharing services will helpfully create image previews of things, and nobody wants to explain why those screenshots have suddenly shown up in the Google drive. You can also password protect a zip should you feel the need - not essential but also not a bad idea at all.

But how do we demonstrate the size of the issue?

The simplest way is probably with some kind of performance data; in my instance I had GSC but you use whatever is available.

A screenshot (redacted if needs be) of how these urls rank, the amount of traffic they're getting and so forth is a really useful tool when it comes to demonstrating the problem either to a wary client or their C-suite, and once again if the URLs are not something you'd like to say out loud you can provide a redacted version in your doc and the unedited version in your evidence zip (along with a line like "please see file.png in evidence.rar for the original version" for the avoidance of any doubt).

All that will help to demonstrate the size of the issue, but how do you demonstrate the severity?

If the above isn't enough, then start providing links. Links to wikipedia pages that you might recognise from the URLs (A couple of urls I saw contained names and titles that have been discussed in numerous true crime podcasts I listen to), or links to articles demonstrating the PR crises that can come from things like this being discovered (Although do make it clear that you are in no way making threats)

In the UK you can remind them of the Online Safety Act coming into effect this year, and your country probably has something similar, even if the possibility of reputational damage from something like this being discovered and written about isn't enough.

Plus, remind these folks that you found the content with freely available tools; a Screaming Frog license is like £100 a year, and I know my way around dev tools but I'm no hacker, so if I could find the content that I found on that one website, so could anyone with an interest in checking how safe our online platforms really are.

Keep in Touch With Your Boss

If for no reason other to ensure them that you are okay (Or, to tell them that you're not). They have a vested interest in keeping you safe, and they can also help you out with support both logistically and emotionally should you stumble across something you shouldn't.

Follow Up

This is kind of up to you. You may not feel comfortable working on that site once you're aware of that content. Your work may even want to sever ties with business (good legal representation will find a way to spin anything like the above as voiding the contract), Or you may be happy to continue working because the client team assures you that they will use the documentation and evidence you put together to begin eradicating this content. Do whatever makes you feel the most comfortable, and absolutely do not allow yourself to be forced into anything.

Anyway

I appreciate this isn't exactly "Ten REGEX tips to extract ANY meta tag", or a funny post about an AI saying something silly. I hope nobody ever actually needs this post, I hope nobody ever has to search for it, or comes across it in an hour of need. However it happened to me and I'm lucky that I wasn't exposed to anything dreadful, so I just thought I'd write this just in case.

Cheers.