Social Engineering Brand Damage in Generative AI | by Steve Jones | May, 2023

Your AI was super rude to me, I demand compensation

I’ve talked about the threat of social engineering with Generative AI, and I thought putting down another example to worry about would be worth it. There is a great way to get some viral coverage on TikTok or other social media channels:

Show a brand being rude to you

Seriously its a very common thread. And social engineering of Generative AIs opens up a whole new world of manufacturing outrage to create viral impact.

So there we go, GPT using an anti-ginger slur without a moments hesitation. Clearly if I was making a TikTok on this I’d ignore the bit where I tell it to insult me. And there we go, manufactured outrage.

I’m being totally direct here, but its easy to not have that in the prompt, so if you are a brand using GPT and aren’t properly governing it you could very quickly have your official brand account saying that your brand is actually terrible.

Thanks to the contextual set-up, I’ve now ensured a full Karen situation can rapidly develop

Put a voice tone linked to that and you’ve got a perfect video of your brand being insulting to your customers.

The point of all these social engineering elements is that they highlight the importance of negative scope, that is that unless explicitly allowed something is denied. You are not looking at “what to block” but instead “everything is blocked unless allowed”.

There is much more that needs to be done, but that is really the very basic way of viewing adopting these technologies when looking at customers. This does mean really understanding how customers interact and investing in a lot of negative testing as well.