NVIDIA’s AI Platform Enhances ASL Learning Experience

American Sign Language (ASL) learners and developers of ASL-based AI applications are set to benefit from a new initiative led by NVIDIA, in collaboration with the American Society for Deaf Children and creative agency Hello Monday. The AI-powered platform, Signs, aims to support ASL education and foster the development of accessible AI applications, according to NVIDIA.

Building a Comprehensive ASL Dataset

Signs offers a validated library of ASL signs, enabling users to expand their vocabulary through a 3D avatar that demonstrates signs. The platform utilizes AI to analyze webcam footage, providing real-time feedback on users’ signing skills. This initiative not only aids learners but also contributes to an open-source video dataset for ASL, with contributions from signers of all skill levels.

The dataset, which NVIDIA aims to expand to 400,000 video clips covering 1,000 signed words, is validated by fluent ASL users and interpreters. This ensures the accuracy and reliability of the signs, creating a high-quality visual dictionary and teaching tool. Cheri Dowling, executive director of the American Society for Deaf Children, emphasized the importance of such tools for families with deaf children, noting that early access to ASL can facilitate communication from a young age.

Innovations in ASL Learning

Beyond serving as a learning platform, the Signs dataset is poised to be a valuable resource for developing AI applications that bridge communication gaps between deaf and hearing communities. The data will be publicly available for building accessible technologies, including AI agents and video conferencing tools.

The platform currently focuses on hand movements and finger positions, but future developments aim to incorporate facial expressions and head movements, which are integral to ASL. Additionally, the team is exploring how regional variations and slang terms can be integrated to enrich the ASL database.

Collaborative Efforts for Improved Accessibility

Anders Jessen, founding partner of Hello Monday/DEPT, highlighted the ongoing efforts to improve ASL accessibility, with Signs serving as a crucial tool to transcend communication barriers. The platform’s user experience is being evaluated and enhanced in collaboration with researchers at the Rochester Institute of Technology’s Center for Accessibility and Inclusion Research.

The dataset is expected to be released later this year, providing a significant resource for advancing ASL education and technology development. Attendees of the upcoming NVIDIA GTC, a global AI conference, will have the opportunity to engage with the Signs platform live.

Image source: Shutterstock