My name is Anshul Noori, and I’m a rising senior at Phillips Exeter Academy. For the past year, I have been working with a 501(c)3 non-profit organization that trains guide dogs for the blind, where I have been caring for, raising, and training Remington. Hopefully, he will then graduate to become a fully-fledged service animal and will find his new home with someone with a serious visual impairment.

After signing up to train Remington, I learned more about the challenges that blind people face around the world. Did you know that an estimated 43 million people worldwide live with blindness, and another 295 million have moderate to severe visual distance impairment? A study published in Lancet Global Health projected that the number of blind people will nearly triple to 114.6 million by 2050. As many forms of blindness lack treatments or cures, it is critical for our society to invest in adaptive mechanisms to support the blind population.

Guide dogs are one of the most important and effective resources for the visually impaired. They are not only essential for mobility and independence, but these canines improve confidence, safety, emotional support, and facilitate social interactions. Unfortunately, there is a massive gap in the number of people who could benefit from a service dog and those who can access them; it is estimated that only 2% of partially sighted and blind persons in the U.S. have guide dogs, far fewer in other countries.

There are many reasons for this shortage: it can cost up to $50,000 annually to train and care for a guide dog, and an estimated 70% of service dogs in training don’t graduate due to behavioral issues. Training guide dogs is resource and time intensive and, unfortunately, often ends in failure.

Combining my passion for animals, accessibility, and artificial intelligence, I set out to help solve this problem. 

My plan is to train AI model to recognize behavioral and physiological patterns in puppies that correlate with success as guide dogs. Using movement, heart rate, breathing rate, and vocalization data, tracked from a wearable collar, coupled with early training outcomes and video, the model would output a probability score of whether a puppy will successfully graduate. Such a model would help guide dog programs to select those puppies that have the highest chance of successfully completing their training, allowing them to focus their resources more effectively and train more guide dogs.

I have a five-step plan to achieve this vision:

1.  I will scrape large-scale Internet video databases to create a training dataset of both guide dogs and untrained dogs. I will mine public databases for videos of guide dogs within the Public Commons and contact copyright owners for permission to use their materials in the training of this model. For example, a search for “guide dog” on YouTube returned 185,000 results. Using the same strategy, I will also scrape public databases for images or videos of regular dogs of all breeds. This way, I will create a large-scale dataset of millions of video frames of both guide dogs and untrained dogs.

2. I will segment dogs from background clutter in the dataset. I will use the Meta Segment Anything Model 3 (SAM 3) to extract both guide dogs and control counterparts from the dataset. SAM 3 is a foundation model for promptable visual segmentation. This step is necessary because I need my image classifier to only learn the differences between guide dogs and untrained dogs, and to ignore all the other distractions or visual clutter in the training data. SAM 3 is publicly available on GitHub: https://github.com/facebookresearch/sam3.

3. While I form a curated dataset of guide dogs and untrained dogs, I will prototype and build a wearable collar that, utilizing microphones pressed against the skin, a 6-axis IMU and accelerometer, and a temperature sensor, can track physiological markers such as panting, vocalization (barking, whining, panting, licking lips), and heart rate (derived from chest movement along with audio from microphones) which can be formed into various emotional states (stress, relaxation, excitement).

4. I will combine the two data sources into a three-layer transformer model:

The first paired layer (pair 1) will fuse the movement data from the wearable collar with a segmented video output (via SAM 3) to create a virtual skeleton of the guide dog, providing us with stable movement data over time.

The first paired layer (pair 2) will isolate a vocalization state (barking, whining, panting, licking lips) from the audio data pulled from the wearable collar.

The second layer will infer an emotional state from movement and audio data from the first layer, other hardware data (HR, breathing rate, temperature), along with video data for information such as tail movement (stiff, tucked, wagging slow low, wagging fast high).

The third layer, a Time-Series Transformer, will aggregate all available data, both hardware and video, along with the previous 2 layers, over a period of time during which the guide dog will perform and interact with a series tasks and situations that would be expected in the real world of a guide dog for a somebody with a visual impairment (crossing the road, load noises, grocery stores, etc.). This layer will produce a probability score of how likely it is that the puppy will succeed in training along with a confidence score of its prediction.

To do this, I will use a contrastive loss function like InfoNCE loss, through which the model learns to bring closer the embeddings of dogs who successfully graduate guide dog training and push apart those who do not. Positive pairs would be derived from videos labeled with “guide dog,” while negative pairs would be derived from control “dog” videos.

5. Finally, I will partner with non-profit organizations that I already work with and offer them use of my model for free to improve their operations.

In order to proceed, I need a machine built for video data, image recognition, and ML workflows like CNNs and Vision Transformers: a dedicated GPU, ample RAM, and high-speed storage. Owning the hardware allows me to train, fine-tune, and iterate as many times as needed without waiting days for results or racking up a five-figure cloud bill that would exhaust funding before the project is complete.

With a target funding goal of $6,000, I can build an AI workstation that will bring my idea to life and dramatically accelerate the process of selecting and training guide dogs for the visually impaired.

In the spirit of open source, I also want to give back to the community by sharing my knowledge. I plan to document the entire process, step by step, along with all of the issues that I will eventually run into and make my learnings publicly available on my GitHub so that younger kids can learn how to build their own PCs and develop AI/ML applications.

With your support, we can enable more accurate guide dog selection, reduce training costs, and ultimately get more guide dogs into the hands of visually impaired people who desperately need them.

Thank you so much for your consideration!

https://github.com/anshulnoori