Associated Projects of the Thinking Head
The ability of the Thinking Head to display/understand emotion/gesture, combined with the development of innovative dialogue management, speech generation and speech understanding systems, will allow meaningful and productive interaction with intelligent systems. This opens up unique opportunities in the fields of education and medicine for the delivery of education, training and diagnostics to all levels of society.
The Flinders Artificial Intelligence Laboratory is currently pursuing collaborative projects targeting education in Indigenous communities (VALIANT), delivery and treatment in the field of autism (the Social Head), and the teaching of second languages (the Teaching Head).
The development of embodied conversational agents (ECAs)
involves a wide range of cutting-edge technologies extending from
multimodal perception to reasoning to synthesis. While each is important
to a successful outcome, it is the synthesis that has the most immediate
impact on the observer. The specifc appearance and voice of an embodied
conversational agent (ECA) can be decisive factors in meeting
its social objectives.
In light of this, we have developed an extensively
customizable system for synthesizing a virtual talking 3D head. Rather
than requiring explicit integration into a codebase, our software runs as
a service that can be controlled by any external client, which substantially
simplifes its deployment into new applications. We have explored
the benefts of this approach across several internal research projects and
student exercises as part of a university topic on ECAs.
The Head X software package is not yet available for public release. Contact us if you are interested in collaborating on a project.
The Talking Head Memory Appointment and Navigation Assistant, or MANA-Calendar is a computer-generated person (head and shoulders) that can remind people about appointments through a calendar application. Appointments can be entered using the Google Calendar system, by a family member, carer, friend, or perhaps even health provider to remind the user about events or activities for the day. Only approved people will have access to the calendar.
Although it is computer based technology, the Talking Head operates in a very natural way which allows it to ‘respond intelligently’ to a range of questions or comments from the user. The Head is able to recognise when a person is in the room, thus increasing its human-like qualities.
People with early onset dementia often remain at home and are supported by a carer who, naturally, cannot be present at all times but is worried that the cared for person will forget routine activities, such as eating lunch. The Calendar provides some comfort to carers, knowing that they can set reminders such as ‘Your lunch is in the fridge’, and these will be given to the user at the appropriate time in a ‘natural’ way.
Another advantage that MANA-Calendar provides is that it is flexible. Carers can put in reminders from home, at work, or from any place where they have access to the internet. The MANA-Calendar system will automatically update and advise the user of the reminder at the appropriate time.
For example, a carer may be at work and remember that a relative will be visiting the user today. Using the Google Calendar system on the internet, the carer can set a reminder for the user and MANA-Calendar will automatically update and advise the user that the relative will be visiting at the set time.
The MANA project has been supported by the MDPP
Concept Video (~8MB) ,
MANA Self-Description Video (~3MB)
VALIANT (Virtual Autonomous LIteracy And Numeracy Tutor) is an innovative computer system that specifically addresses the needs of Indigenous communities to foster numeracy and literacy skills in young children.
This system will be designed so as to simulate the experience of being taught by a human tutor, providing the teacher of a class of students with a virtual assistant to supplement the limited one-on-one contact time.
The project is aimed at engaging Indigenous students in an immersive environment, created by the combination of the Thinking Head and virtual-world simulation software.
These students often have poor attendance at conventional classes, and characteristically progress through classes without achieving the target learning outcomes in relation to literacy and numeracy. The challenge is thus to engage this specific community, who will in general be older and more mature than the infant levels at which more general/traditional literacy and numeracy education is aimed in the public school systems.
Literacy Concept Video (~9MB) ,
Numeracy Concept Video (~7MB)
The Social Head
The Social Head is a virtual coach for social and communication skills. The project targets the development and evaluation of an interactive, immersive computer system to coach communication and social skills to children with Autistic Spectrum Disorder (ASD) and children with a hearing disability.
The Social Head will augment current therapy and intervention programs and acts in a therapist role by delivering structured coaching material and constructive feedback on progress.
The Social Head capitalises on the enthusiasm that young children with disabilities have for technology (especially children with ASD), and on the capability of a technological solution to deliver coaching without fatigue and in a socially neutral context. A child will independently engage with and control the pace of intervention, and build a much-needed and ongoing companionship. The proposal offers a way to improve and practice social, learning or communication skills in a socially neutral setting, which is free of embarrassment or discomfort because of a disability.
The system will also be able to provide the child with constructive feedback regarding progress, something that is integral to current therapeutic best practice.
Concept Video (~12MB)
The Teaching Head
The Teaching Head is a virtual instructor for second-language teaching via a physically-grounded interface.
The aim is to construct a computer system which will assist in the teaching of German to Australian school students.
Instead of using a computer system via the traditional keyboard and mouse, the child interacts with the system by speaking, and by manipulating physical objects.
Situations are described in German by the Thinking Head, and demonstrated visually in a virtual-world simulation on a second computer monitor.
The ability of the Teaching Head to make human-like, natural conversation, provide non-confrontational feedback about progress will allow it to simulate the experience of having a personal German tutor for every student, allowing it to be a valuable helping hand for teachers.
Progress through the material is self-paced, as the system can dynamically adapt the material to suit the student's level of knowledge.