Sign Language Processing

Overview

Our organization develops a wide array of tools to process, understand, and translate sign languages. Below is a categorized list of our repositories and their roles within the ecosystem.

Core Libraries

• datasets
  TFDS loaders and utilities for handling sign language datasets.

• pose
  Tools for viewing, augmenting, and handling .pose files, which represent sign language gestures.

Translation and Synthesis

• spoken-to-signed-translation
  A pipeline for translating spoken language to sign language, including text-to-gloss-to-pose conversion.

• pose-anonymization
  Remove identifying information from sign language poses.

• pose-to-video
  Renders pose sequences into photorealistic videos, bringing sign language movements to life.

SignWriting and Visualization

• signwriting-translation
  Translation from spoken languages to SignWriting, facilitating written sign language documentation.

• signwriting-transcription
  Transcribes sign language videos using SignWriting.

• signwriting-animation
  Converts SignWriting into skeletal pose animations.

• signwriting-illustration
  Automatically generate illustrations from SignWriting, aiding in sign language learning.

• signwriting-description
  Describe how to perform signs written in SignWriting.

Experimental and Research Projects

• sign-gpt
  A multi-task GPT model tailored for sign language tasks.

• synthetic-signwriting
  Generates synthetic SignWriting data for pretraining models.

• lexicon-induction
  Induce a lexicon from a continuous sign language corpus.

Benchmarking and Evaluation

• 3d-hands-benchmark
  Tools for benchmarking 3D hand pose estimation models.

• signwriting-evaluation
  Automatic evaluation of SignWriting outputs from machine learning models.

Other Utility Projects

• segmentation
  Sign language pose segmentation on the sentence and sign level.

• recognition
  Recognizes lexical signs from isolated sign videos.

• (OLD) Sign Language Activity Detection training and application - Detect when a signer is a signing in real time.