Violence against women as an AI-driven business model

Abstract

With generative AI based on large language models, personalized chatbots swept the globe in 2023. The new technology enables an interaction between users and AI, so suggestive that it can cognitively induce a hallucinatory experience of the machine as human. Thus, AI developers have rapidly brought the technology to market in various applications, including the creation of so-called sexting chatbots. Our initial investigation indicates that these chatbots both reproduce and reinforce core aspects of violence against women. It is not only about what is seen on the screen itself - where AI-girlfriends invite the user to control and victimize them - but also in the very communicative interaction where the sexting chatbot appears to reinforce patterns that draw on coercive control. In reality, domestic violence unfolds  as a pattern over time where the perpetrator gradually entraps the woman. In the interaction with a sexting chatbot, however, the user owns the 'AI-girlfriend' from the first moment. She is shaped entirely according to the man's preferences.

We intend to continue the examination of sexting chatbots and its link to gender-based violence. Our initial investigation raises questions beyond the sexting chatbot industry. What could be traced through this phenomenon is a more general business model for generative AI that appears to be based on the search for pre-existing cognitive patterns in users followed by reinforcement of these via fictitious mass-produced individual relationships. Such model could have a rapid and profound impact on the world, in more ways than we have examined so far.