Description:

A skill prototype of a social robot, voice assistant, Alexa Echo, who is able to: Sing a song that you forget the name of but remember some lyrics. Sing along with you given that you sing the first part of the song. Sing a duet with you.

Inspiration:

Have you ever only remembered some lyrics of a song but not the song’s name? Have you ever wished to have a companion that may sing with you and be always available when nobody else is? Voice assistant devices such as Amazon Echo have the ability to solve these problems and assist human beings in a more natural way than having us clicking buttons. Therefore, we would like to build a social robot that may sing a song to you even though you are not able to remember the song's name and may even sing a duet with you.

What it does:

Alexa’s Echo sing-off feature: - Has the user sing a song in your natural voice and recognize the song the user sings - Responds to user’s sing-off request after the user asks Alexa to sing along or sing together. Use following commands to activate the feature: + “Alexa, sing along.” if user wants to sing only the first part of the song + “Alexa, sing together.” if user wants to sing part of them song while Alexa sings the other parts. See following scripts: Song: Yesterday User: Yesterday Alexa: all my troubles seemed so far away Now it looks as though they're here to stay User: Oh, I believe in yesterday Suddenly Alexa: I'm not half the man I used to be There's a shadow hanging over me. Oh, yesterday came suddenly Song: Over the rainbow User: Somewhere over the rainbow Alexa: Way up high And the dreams that you dream of Once in a lullaby User: Somewhere over the rainbow Bluebirds fly Alexa: And the dreams that you dream of Dreams really do come true - Allows you to have Alexa to sing the rest of the song or sing a duet with the user. - Starts singing the song after the user stops at a certain part of the songs whether it is in the middle of a sentence or the end of a sentence. - Handle failed request naturally.

How we built it:

Two python files, one for each alexa’s skill (sing along skill and sing together skill) Two json interfaces for alexa Templates.ymal for alexa answers and response. MP3 audio files for singing voices from real female singers on Youtube, processed and cut to fit the application.

Challenges we ran into:

Program Alexa Echo to be able to sing Stop Alexa Echo’s singing at a certain part of the *.mp3 files instead of having to cut the audio into parts and script the duets. Delay Alexa’s response after a certain time. The speed of response depends on the Wifi’s strength. Alexa’s wait time in the duet.

Accomplishments that we're proud of:

Finished a complete simple and presentable prototype of the skills with three sample songs: Happy Birthday covered by Celine Dion Yesterday by the Beatles covered by Samia Abreu Over the rainbow covered by Foy Alexa’s Echo sing-off feature: Has the user sing a song in your natural voice and recognize the song the user sings Responds to user’s sing-off request after the user asks Alexa to sing along. Allows you to have Alexa to sing the rest of the song or sing a duet with the user. Starts singing the song after the user stops at a certain part of the songs whether it is in the middle of a sentence or the end of a sentence

What we learned:

Due to the need of huge amount of voice data and time to train a well performed machine learning model to recognize speech and filter out noises, a voice assistant device such as Amazon Echo and Google Home is a good speech recognition device to build a quick prototype. Voice generated and sound processing is a complicated process. At the moment, our prototype still uses singing voices from real human beings. Automatically generate the script and cut the audio track will speed up development time.

What's next:

Improve the database collection of songs Alexa may help you with and sings to you. Have Alexa start and stop singing depending on when you decide to start and stop singing your parts in real time instead of scripting.

Built with:

Amazon Echo Amazon Developer Portal for alexa Flask app Python Json SSML (Speech Synthesis Markup Language)

Prizes we're going for:

Hexacopter Drone

Intel® Movidius™ Neural Compute Stick

Google Home Mini

DragonBoard 410c

$100 Amazon Gift Cards

LS20 Gaming Headset

Raspberry Pis & PiHut Essential Kits

Hustle Award

Social Entrepreneurship Award

Oculus Go (32 GB)

Grand Prize

Hacker gear & swag from HERE.com

Blu R2 Plus Smartphones

Fujifilm Instax Mini 26

Team Members

Trung Ba Nguyen, Ly (Harriet) Bui
View on Github