Twitter icon
Facebook icon
LinkedIn icon
Google icon
Reddit icon
StumbleUpon icon
Del.icio.us icon

TinyML Dog Bark Stopper

Added to IoTplaybook or last updated on: 02/02/2022
TinyML Dog Bark Stopper

Story

Since my family has started going to school and work, my dog has started going crazy. During the quarantine, she got really attached and developed a separation anxiety. To solve that, we have to detect Clairette's barks when we're gone and respond to them using recordings of our voices.


Video on the project

Things used in this project

Hardware components

Nano 33 BLE Sense

Arduino Nano 33 BLE Sense

 

×

1

Arduino.CC

Arduino Nano R3

Arduino Nano R3

 

×

1

Newark

CPC

 

Adafruit Music Maker FeatherWing

 

×

1

Adafruit

 

Speaker with AUX input

 

×

1

 

Software apps and online services

Edge Impulse Studio

Edge Impulse Studio

 

 

Edge Impulse

Arduino IDE

Arduino IDE

 

 

Aruino CC

Hand tools and fabrication machines

Soldering iron (generic)

Soldering iron (generic)

How To Build

1. Wire following the schematic.

2. Upload bark_listener.ino to Arduino Nano 33 BLE Sense.

3. Upload bark_audio_responder.ino to Arduino Nano.

4. Upload audio files to SD card.

5. Connect Music Maker Feather to AUX speaker.

6. Plug Arduino Nano 33 BLE Sense in.

7. Watch the barks stop!

Entire Setup

1 of 7 - Entire setup

Boards Wired Up

2 of 7 - Boards wired

Arduino Nano 33 BLE Sense

3 of 7 - Arduino Nano 33 BLE Sense

Arduino Nano

4 of 7 - Arduino Nano

Music Maker Feather

5 of 7 - Music Maker Feather

Music Maker and Speaker

6 of 7 - Music Maker and speaker

Nano 33 Plugged In

7 of 7 - Nano 33 plugged in

Training the Model

Since there was no one big dataset good enough for dog bark detection, I used a couple. Ranked by importance in the model, the datasets I used are:

I then used Shawn Hymel's keyword spotting dataset curation script to preprocess the data into one-second clips and add background noise. Every audio clip was then labeled as either "bark" or "other". I also had to do a lot of cleaning of the bark data as many clips from the Google Audioset did not have barks in them: I did this by listening to the audio samples labeled as other and reassigning them based on if they were actually barks or not).

I uploaded all the audio clips to Edge Impulse under their labels; I then used Edge Impulse's Eon Tuner to find and train the best model.

Eon Tuner Validation Accuracy and Performance

1 of  3 - Eon Tuner Validation Accuracy and Performance

Accuracy on Validation Set

2 of 3 - Accuracy on validation set

Model Architecture

3 of 3 - Model architecture

The raw audio is turned into a Mel-filterbank energy spectrogram, an image representation of audio used by the model. The picture below shows that the raw audio between the types does not look that different; the spectrograms make the similarities between the samples most apparent.

Spectrograms

Spectrograms

Deploying the Model

Edge Impulse Output

Edge Impulse Output

Edge Impulse exports an Arduino library that can be used with the Arduino Nano 33 BLE Sense. The library handles the conversion of the audio into a spectrogram and the model inference.

Responding To the Barks

Audio Track Files

Audio track files

Whenever a bark is detected, a random track is played to respond to the dog. My dog is attached to my mom the most, so we recorded her. The tracks range from telling her to stop or telling her that it is okay.

I had to use an external Arduino Nano to control because the Nano 33 runs MBED OS, which doesn't have fast enough interrupts to control the audio (when using the Nano 33, the audio is very choppy).

Schematics - link

Code - GitHub Repo

NathanielFelleke / The-Bark-Stopper

Read More

Download as zip

Credits

Nathaniel Felleke

Nathaniel Felleke

16 year old developer always trying to make new things.

Thanks to Clairette.

 

Hackster.io

This content is provided by our content partner Hackster.io, an Avnet developer community for learning, programming, and building hardware. Visit them online for more great content like this.

This article was originally published at Hackster.io. It was added to IoTplaybook or last modified on 02/02/2022.