A New Zealand whānau has brought national attention to the growing dangers of artificial intelligence after discovering family photographs had allegedly been taken from social media and manipulated into explicit content without their knowledge or consent.
The deeply distressing case, highlighted on TVNZ’s Marae, has reignited concerns about the misuse of personal images online and the increasing accessibility of AI tools capable of generating realistic but entirely fabricated content.
As technology evolves at a rapid pace, online safety experts are warning that anyone with publicly available photographs on social media could potentially become a target for digital exploitation.
Sean Lyons, Chief Online Safety Officer at Netsafe NZ, says cases involving image-based abuse and AI-generated content are becoming an increasingly significant challenge for victims, families and authorities.
Artificial intelligence has transformed the online landscape, creating new opportunities for innovation, but also new avenues for abuse.
Experts say the misuse of family photos, personal images and publicly available social media content to create explicit or humiliating material is becoming more sophisticated and harder to detect.
Unlike traditional forms of image manipulation, modern AI systems can generate highly realistic content using only a small number of photographs. In many cases, victims may not even be aware their images have been copied, altered or shared online until someone alerts them.
The emotional toll can be severe, affecting mental health, personal relationships, employment and a person’s sense of safety.
For Māori communities, where whakapapa, identity and whānau connections are deeply valued, the unauthorised misuse of personal images can carry an additional layer of cultural and emotional harm.
Netsafe serves as New Zealand’s primary organisation dedicated to online safety and digital harm prevention.
The organisation provides support to individuals experiencing online abuse, cyberbullying, image-based exploitation and harmful digital communications. It also assists victims in understanding their rights, reporting harmful content and navigating the often-complex process of seeking removal of material from online platforms.
In situations involving AI-generated explicit content, Netsafe can help victims document evidence, identify reporting pathways and connect with social media companies, website operators and relevant authorities.
The organisation also works closely with government agencies, schools, community groups and technology companies to improve online safety awareness across New Zealand.
While image-based abuse is not a new phenomenon, experts say the emergence of AI tools has dramatically increased both the scale and accessibility of the problem.
Previously, creating convincing fake images required advanced technical skills and specialised software. Today, many AI image generators are available online and can produce realistic content in a matter of seconds.
Internationally, reports of AI-generated intimate images and deepfake content have surged over the past two years, affecting celebrities, public figures and everyday social media users alike.
Online safety advocates warn that the technology is advancing faster than public awareness, leaving many people unaware of how easily their personal images can be misused.
Experts recommend that social media users take proactive steps to reduce their exposure to digital exploitation.
Reviewing privacy settings regularly can help limit who can view and download personal photographs. Restricting access to trusted friends and family members may reduce the likelihood of images being copied by strangers.
Users are also encouraged to think carefully before sharing photographs that reveal personal details, locations, schools or other identifying information.
Regularly searching for one’s name and images online can sometimes help identify unauthorised use before it spreads more widely.
Parents are advised to discuss online privacy with their children and consider carefully which images of tamariki are shared publicly on digital platforms.
If someone discovers their images have been misused, experts recommend avoiding direct engagement with offenders and instead documenting evidence through screenshots, URLs and dates before reporting the content.
Victims of image-based abuse in New Zealand may have legal options available.
The Harmful Digital Communications Act provides protections against serious online abuse and can be used in situations where harmful content causes significant emotional distress.
Depending on the circumstances, police involvement may also be appropriate, particularly where criminal offences have occurred or where children are involved.
New Zealand’s Privacy Act may offer additional avenues for complaints where personal information has been misused.
Many social media companies and online platforms have reporting systems specifically designed to address non-consensual intimate imagery and AI-generated explicit content. Successful removal often depends on reporting the material quickly and providing sufficient evidence.
Experts stress that victims should not feel isolated or powerless. Support services exist, and early reporting can improve the chances of having harmful material removed.
The case featured on Marae highlights the darker side of rapidly advancing technology and the need for stronger public awareness around online safety.
As artificial intelligence becomes increasingly integrated into everyday life, experts say communities, governments and technology companies must work together to ensure innovation does not come at the expense of privacy, dignity and personal safety.
For whānau affected by digital exploitation, the experience serves as a stark reminder that a simple family photograph shared online can be vulnerable to misuse in ways that were unimaginable only a few years ago.
The challenge now is ensuring New Zealand’s laws, support services and digital platforms keep pace with technology that continues to evolve faster than ever before.
