AI and Young People: Why Safety Research Can’t Wait

Day of AI Australia Program Director Natasha Banks spoke with Dr Rahat Masood from UNSW Sydney about a new research collaboration, and why understanding how AI interacts with children has never been more urgent.

 

Generative AI is already woven into the daily lives of Australian students. It tutors them, chats with them, helps them write assignments, and keeps them company. Yet for all the rapid deployment of these technologies, the evidence base for how they actually affect young people, notably their privacy, their trust, their decision-making, remains thin.

That is about to change.

UNSW Sydney, together with King’s College London have been awarded a Security and Defence PLuS Seed Grant to investigate exactly these questions. Day of AI Australia is a collaborator for the work. 

The project will examine the privacy and security risks that emerge when children interact with generative AI systems, and how those interactions shape young people’s trust, behaviour and decision-making.

Program Director, Natasha Banks, sat down with Dr Rahat Masood, the project’s lead researcher from UNSW Sydney, to talk about what the research aims to investigate and why it matters.

 

Natasha: Congratulations, Rahat. Before we get into the research, can you tell us a bit about your background and what drew you to this area?

Rahat: Thank you Natasha. I am a Senior Lecturer in Cyber Security at UNSW Sydney, where my research focuses on privacy, online safety, and technology-facilitated harms. My work looks at how emerging technologies shape human behaviour, trust, and security and privacy vulnerabilities in online environments.

I became interested in this area because generative AI is already becoming part of children’s everyday lives, yet we still know very little about how these systems affect their privacy, trust, and decision-making. Young people are often the earliest adopters of new technologies, but research and safety frameworks rarely keep pace.

As both a researcher and educator, I think it is critical that we build evidence-based understanding now, before these technologies become even more deeply integrated into childhood and education. That is why this collaboration between UNSW Sydney, King’s College London, and Day of AI Australia is so important to better understand these risks and help build guidance for safer and more responsible use of AI by children and young people.

 

Natasha: We know students are using AI. Why does it matter so much that we study it now, rather than waiting for the technology to mature?

Rahat: The honest answer is that we are already behind. Young Australians are not waiting for the research to catch up. They are engaging with AI right now, through conversational assistants like ChatGPT, AI companions like Replika, educational tutoring tools, and AI-enabled services embedded in the apps and platforms that are part of their everyday lives.

What makes these systems different from earlier technologies is that they interact through natural language. They remember previous conversations, they adapt their responses based on prior interactions, and they personalise content using behavioural data. That is a very intimate form of engagement, and it creates risks that we do not yet fully understand.

Our own research at UNSW Sydney has already surfaced something that should concern every parent and educator. As many as 82.4% of Australian high school students report having seen deepfakes. 7.3% say they have created them. 18.5% have shared deepfake content. That tells us that potentially harmful AI-generated content is already part of young people’s digital experience, not on the horizon, but right now.

 

Natasha: When people think about AI risks for children, they often think about inappropriate content. But you have suggested the risks go deeper than that.

Rahat: That is right, and this is something I think gets missed in a lot of the public conversation. The content risks are real. Deepfakes are genuinely harmful, and the eSafety Commissioner in Australia issued legal notices to several AI companion providers in 2025 precisely because of exposure to sexually explicit conversations and self-harm content. 

The new Age-Restricted Material Codes address this directly, preventing AI chatbots and companions from engaging in those kinds of conversations with minors.

But there is another layer of risk that is less visible and, in some ways, more insidious. As young people become increasingly comfortable with AI tools, there is a real tendency to see them as trustworthy, even authoritative. Children may treat AI systems as trusted companions or tutors, which increases their vulnerability to misinformation, inappropriate advice, or even manipulation.

These systems are also collecting large volumes of conversational and behavioural data. Every time a student asks an AI tutor for help or confides something to an AI companion, that interaction is being recorded and processed. The privacy implications are significant and not yet well understood.

And at a broader level, which is really what the PLuS Alliance grant is focused on, there are serious national security dimensions to this. Conversational AI that interacts with children at scale could create new pathways for behavioural influence, exposure to misinformation, or even radicalisation. Understanding that at a societal level matters, not just for individual child safety, but for the resilience of our communities.

 

Natasha: So what will this research actually do? What are the questions you are setting out to answer?

Rahat: We have structured the project around four core research questions. First, what privacy and security risks actually emerge when children interact with generative AI services? 

Second, how do conversational AI systems shape children’s trust, behaviour and decision-making?

Third, what behavioural influence or misinformation risks arise from persistent AI interactions? 

And fourth, crucially, what safeguards are needed to reduce those risks?

To answer them, we are working across three streams. In the first, we will systematically evaluate widely used conversational AI services, including AI companions and educational chatbots that students are already using, through prompt-based probing to study how these systems respond to sensitive topics and handle personal information. In the second, we will look at user perception and behavioural impact: how children actually trust and interpret AI-generated responses, and how those interactions influence their awareness and decision-making. The third stream synthesises all of that into a risk taxonomy and, importantly, a safeguard framework with practical recommendations that can inform how these systems are designed and governed.

What makes this work distinctive is the combination of empirical technical analysis with human-centred research. Together, these approaches help us understand the gap between what children, parents, and educators expect these systems to do, and what they actually do in practice.

Much of the existing research has focused on AI systems at the API level rather than the child-facing applications students are directly using. The smaller number of studies that do examine child-facing apps often focus only on privacy or safety, but rarely both together. In addition, most prior work studies single-turn conversations, where users ask one question at a time. But many modern AI companions are designed for persistent, multi-session interactions, where children may build trust and disclose increasingly personal information over time. That creates a very different level of behavioural and privacy risk, and it is something we believe needs much deeper investigation.

 

Natasha: Day of AI Australia is a partner in this research, not just a recipient of the findings. Can you talk about what that collaboration looks like?

Rahat: It is a genuinely important part of how this project is designed. Day of AI Australia has something that is quite hard to come by in research: trusted, established relationships with schools and educators across Australia. That network is essential for the user-focused strand of our work. We need to understand how real students, in real classrooms, perceive and interact with these AI tools. That is not something you can replicate in a lab.

The partnership means the research will reflect actual educational contexts. Teachers will help shape the scenarios we study, and students will be part of understanding the impact. That, in turn, means the findings are far more likely to be useful to educators and school leaders, not just to researchers.

The collaboration also brings together complementary expertise from our international partners. Dr Ruba Abu-Salma from King’s College London brings deep expertise in usable privacy and security. She will help design the questionnaires and data collection methodologies for children, parents, and educators, by appropriately considering psychology and different levels of digital understanding.

CybSafe will contribute during the later stages of the project, particularly around refining the risk taxonomy and identifying practical safeguards and recommendations. Their expertise in behavioural cyber risk and human factors will help us better understand the gaps between expected and actual behaviour, and how those gaps can be addressed through safer system design, policy, and education.

 

Natasha: What do you want educators and school leaders reading this to take away?

Rahat: The most important thing is to recognise that AI is already part of students’ everyday lives. Many young people are using these tools regularly, often without fully understanding how the systems work, what data is being collected, or how AI-generated responses may influence their thinking and decisions over time.

I think educators and school leaders should approach this in the same way we approached social media and online safety years ago, not with panic, but with informed engagement. Schools do not need to ban every new technology, but they do need to help students build critical awareness around privacy, trust, and responsible use.

Schools can encourage students not to share sensitive personal information with AI systems, help them understand that AI responses are not always accurate or unbiased, and create open discussions around how these tools influence emotions, behaviour, and decision-making. Educators also need clearer guidance themselves, because many teachers are navigating these technologies at the same pace as students.

 

Natasha: And what about parents? Particularly parents who are not sure what to make of their child’s use of AI tools?

Rahat: I think the most important thing for parents to know is that they do not need to understand every technical detail to start having meaningful conversations with their children about AI. These tools can be useful and educational, but they also come with risks.

My advice to parents would be not to approach this with fear, but with curiosity and open discussion. Ask children what tools they are using, what they like about them, and whether they understand that AI responses are not always accurate or unbiased. Encouraging critical thinking and healthy scepticism is one of the best protections we currently have.

There is still a lot we do not know, and that is exactly why this research matters. 

Parents and carers can access Day of AI Australia’s free Families Toolkit.

 

Natasha: What does success look like at the end of this research project?

Rahat: The most concrete output will be a policy-oriented white paper, a document designed not to sit on a shelf, but to be genuinely useful to policymakers, regulators, educators, technology developers, and digital safety practitioners. Alongside that, we will produce academic publications and a stakeholder seminar that brings researchers, industry partners and digital safety organisations together to work through the findings and what they mean in practice.

But if I am honest, the measure of success for me is whether the risk taxonomy and the safeguard framework we develop actually change something. Whether a technology company building an AI product for schools thinks differently because of this research. Whether a regulator has better evidence to draw on. Whether a teacher feels more equipped to have an informed conversation with their students about the AI tools they are using.

I also hope this project helps build longer-term collaboration between UNSW Sydney, King’s College London, and Day of AI Australia, as we all work towards a common goal of digital safety and education. Ultimately, I hope this project helps create momentum for larger national and international research efforts, including future collaborations with government agencies and broader funding initiatives in Australia and the UK, all focused on improving outcomes for young people and the communities that support them.

 

Natasha: I want to end by asking you something we think about a lot at Day of AI Australia. We work to make sure young people are not just passive consumers of AI, that they understand how it works and can engage with it critically. How does this research connect to that mission for you?

Rahat: I think that mission is incredibly important. For me, this research is not just about identifying risks, it is about helping young people build the awareness and confidence to engage with AI critically and responsibly. AI is going to shape how this generation learns, communicates, and makes decisions, so digital literacy now has to include understanding how AI systems behave, where their limitations are, and how they may influence us.

What makes this work personally meaningful is that it is socially impactful research that connects directly to real lives. The questions we are asking affect children, families, educators, and communities. As a researcher, it is rewarding to work on problems where the outcomes can genuinely help improve people’s lives and support safer and more informed use of technology.

I also hope this project creates a strong foundation for future research in child-centred AI safety. There is still so much we do not understand about long-term AI interaction and behavioural influence. My hope is that future researchers can build on this work to create safer, more transparent, and more accountable AI systems that truly benefit young people and society.

Dr Rahat Masood is a Senior Lecturer at UNSW Sydney’s School of Computer Science and Engineering, where her research focuses on cybersecurity and privacy. She is the lead researcher on this project, working alongside co-lead Dr Ruba Abu-Salma from King’s College London and Dr Jake Renzella from UNSW Sydney.