AI-Powered Sign Language for Everyone
Developing Kazakh-Russian sign language resources using artificial intelligence — making education, communication, and digital life accessible for people with hearing and speech impairments.
Try SignBridge
Development of Sign Language Resources Using Artificial Intelligence for People with Hearing and Speech Impairments
"Breaking Barriers, Connecting Lives."
- Enhanced Communication Accessibility
- Social Inclusion & Equal Opportunities
- Improved Access to Education & Services
- Advanced AI & Machine Learning Solutions
- Economic & Professional Benefits
- Scalability & Technological Advancement
Aigerim Yerimbetova
Project Manager
Call us anytime
+7 701 583 3302
Features
Three core technical capabilities of the project
AI-Powered Sign Language Recognition and Translation
- Uses deep learning models (Transformer, EfficientNet, ResNet) and computer vision to accurately recognize Kazakh-Russian sign language gestures.
- Converts sign language gestures into text and audio with high precision, enabling communication for individuals with hearing impairments.
- Trained using MediaPipe for keypoint detection and ML classification, ensuring real-time accurate gesture recognition.
Bidirectional Speech-to-Sign and Sign-to-Speech Conversion
- Real-time translation of spoken language into sign language and vice versa.
- Interactive 3D avatar that visually represents sign language gestures for hearing users.
- Speech recognition powered by Conformer and Wave2Vec2 models for high-accuracy speech-to-text conversion.
- Speech synthesis using FastSpeech2 and Tacotron2 to convert text into natural-sounding speech.
Integration with Educational and Digital Platforms
- Integrates with learning management systems (LMS), mobile apps, and web platforms for accessible education.
- Supports real-time transcription and translation of classroom lectures and online courses for deaf students.
- Includes an AI-powered sign language dictionary and interactive tutorials for learning and practicing sign language.
Expected Results
Key outcomes anticipated from the program
AI for Sign Language Recognition
Optimized AI enhances Kazakh-Russian sign recognition, improving accessibility for the hearing and speech impaired.
Bridging Communication
AI translates speech to Kazakh-Russian sign language and gestures to text/audio for seamless understanding.
Adaptive AI Translation
A self-learning system that continuously improves translation speed and accuracy over time.
Tech for Inclusive Education
Technology integrated into learning platforms improves knowledge access, promoting social and professional inclusion.
Experiments
Key experiments and their methodologies
Gesture Classification Network (Sign to Text)
Goal: Develop and train a model for recognizing gestures and converting them into text.
Data: Key hand points collected using MediaPipe.
Models: ResNet, YOLOv8.
Metrics: Precision, recall, accuracy, F1-measure.
Avatar Text to Sign
Goal: Translate text into gestures with 3D avatar visualization.
Data: Kazakh-KSL Gloss parallel corpus.
Models: Transformer, Seq2Seq.
Metrics: Precision, recall, BLEU score.
Speech Synthesis
Goal: Generate natural speech from text.
Data: Kazakh-Russian text corpus.
Models: Tacotron 2, WaveNet.
Metrics: MOS (Mean Opinion Score), MCD.
Audio to Text
Goal: Convert audio speech recordings to text.
Data: Kazakh-Russian speech audio recordings.
Models: Conformer, Wav2Vec2.
Metrics: Word Error Rate (WER), Character Error Rate (CER).