Merge remote-tracking branch 'upstream/master'

README.md

@@ -14,60 +14,83 @@ Version: 1.3.0 (2/19/2018)
 
 ## Alexa support
 
-Snowboy now brings hands-free experience to the [Alexa AVS sample app](https://github.com/alexa/alexa-avs-sample-app) on Raspberry Pi! See more info below regarding the performance and how you can use other hotword models.
+Snowboy now brings hands-free experience to the [Alexa AVS sample app](https://github.com/alexa/avs-device-sdk/wiki/Raspberry-Pi-Quick-Start-Guide-with-Script) on Raspberry Pi! See more info below regarding the performance and how you can use other hotword models. The following instructions currently support AVS sdk Version 1.12.1.
 
 **Performance**
 
 The performance of hotword detection usually depends on the actual environment, e.g., is it used with a quality microphone, is it used on the street, in a kitchen, or is there any background noise, etc. So we feel it is best for the users to evaluate it in their real environment. For the evaluation purpose, we have prepared an Android app which can be installed and run out of box: [SnowboyAlexaDemo.apk](https://github.com/Kitt-AI/snowboy/raw/master/resources/alexa/SnowboyAlexaDemo.apk) (please uninstall any previous versions first if you have installed this app before). 
 
+**Kittai KWD Engine**
+
+* Set up [Alexa AVS sample app](https://github.com/alexa/avs-device-sdk/wiki/Raspberry-Pi-Quick-Start-Guide-with-Script) following the official AVS instructions
+
+* Apply patch to replace the Sensory KWD engine with Kittai engine
+```
+# Copy the patch file to the root directory of Alexa AVS sample app. Please replace $ALEXA_AVS_SAMPLE_APP_PATH with the actual path where you
+# cloned the Alexa AVS sample app repository, and replace $SNOWBOY_ROOT_PATH with the actual path where you clone the Snowboy repository
+cd $ALEXA_AVS_SAMPLE_APP_PATH
+cp $SNOWBOY_PATH/resource/alexa/alexa-avs-sample-app/avs-kittai.patch ./
+
+# Apply the patch, this will modify the scripts setup.sh and pi.sh
+patch < avs-kittai.patch
+```
+
+* Re-compile the avs-device-sdk and sample app
+```
+sudo bash setup.sh config.json
+```
+
+* Run the sample app
+```
+sudo bash startsample.sh
+```
+
+Here is a [demo video](https://www.youtube.com/watch?v=wiLEr6TeE58) for how to use Snowboy hotword engine in Alexa Voice Service.
+
 **Personal model**
 
 * Create your personal hotword model through our [website](https://snowboy.kitt.ai) or [hotword API](https://snowboy.kitt.ai/api/v1/train/)
 
-* Replace the hotword model in [Alexa AVS sample app](https://github.com/alexa/alexa-avs-sample-app) (after installation) with your personal model
 
+* Put your personal model in [snowboy/resources](https://github.com/Kitt-AI/snowboy/tree/master/resources)
 ```
-# Please replace YOUR_PERSONAL_MODEL.pmdl with the personal model you just
-# created, and $ALEXA_AVS_SAMPLE_APP_PATH with the actual path where you
-# cloned the Alexa AVS sample app repository.
-cp YOUR_PERSONAL_MODEL.pmdl $ALEXA_AVS_SAMPLE_APP_PATH/samples/wakeWordAgent/ext/resources/alexa.umdl
-```
+# Please put YOUR_PERSONAL_MODEL.pmdl in $ALEXA_AVS_SAMPLE_APP_PATH/third-party/snowboy/resources,
+# and $ALEXA_AVS_SAMPLE_APP_PATH with the actual path where you put the Alexa AVS sample app repository.
+
+cp YOUR_PERSONAL_MODEL.pmdl $ALEXA_AVS_SAMPLE_APP_PATH/third-party/snowboy/resources/
 
-* Set `APPLY_FRONTEND` to `false` and update `SENSITIVITY` in the [Alexa AVS sample app code](https://github.com/alexa/alexa-avs-sample-app/blob/master/samples/wakeWordAgent/src/KittAiSnowboyWakeWordEngine.cpp) and re-compile
+```
 
+* Replace the model name 'alexa.umdl' with your personal model name, update `KITT_AI_SENSITIVITY`, set `KITT_AI_APPLY_FRONT_END_PROCESSING` to `false` in the [Alexa AVS sample app code](https://github.com/alexa/avs-device-sdk/blob/master/KWD/KWDProvider/src/KeywordDetectorProvider.cpp) and re-compile
 ```
-# Please replace $ALEXA_AVS_SAMPLE_APP_PATH with the actual path where you
-# cloned the Alexa AVS sample app repository.
-cd $ALEXA_AVS_SAMPLE_APP_PATH/samples/wakeWordAgent/src/
-
-# Modify KittAiSnowboyWakeWordEngine.cpp and update SENSITIVITY at line 28.
-# Modify KittAiSnowboyWakeWordEngine.cpp and set APPLY_FRONTEND to false at
-# line 30.
-make
+# Modify $ALEXA_AVS_SAMPLE_APP_PATH/avs-device-sdk/blob/master/KWD/KWDProvider/src/KeywordDetectorProvider.cpp:
+#     Replace the model name 'alexa.umdl' with your personal model name 'YOUR_PERSONAL_MODEL.pmdl' at line 52
+#     Update `KITT_AI_SENSITIVITY` at line 26
+#     Set `KITT_AI_APPLY_FRONT_END_PROCESSING` to `false` at line 32
+sudo bash setup.sh config.json
 ```
 
 * Run the wake word agent with engine set to `kitt_ai`!
 
-**Universal model**
+Here is a [demo video](https://www.youtube.com/watch?v=9Bj8kdfwG7I) for how to use a personal model in Alexa Voice Service.
 
-* Replace the hotword model in [Alexa AVS sample app](https://github.com/alexa/alexa-avs-sample-app) (after installation) with your universal model
+**Universal model**
 
+* Put your personal model in [snowboy/resources](https://github.com/Kitt-AI/snowboy/tree/master/resources)
 ```
-# Please replace YOUR_UNIVERSAL_MODEL.umdl with the personal model you just
-# created, and $ALEXA_AVS_SAMPLE_APP_PATH with the actual path where you
-# cloned the Alexa AVS sample app repository.
-cp YOUR_UNIVERSAL_MODEL.umdl $ALEXA_AVS_SAMPLE_APP_PATH/samples/wakeWordAgent/ext/resources/alexa.umdl
-```
+# Please put YOUR_UNIVERSAL_MODEL.umdl in $ALEXA_AVS_SAMPLE_APP_PATH/third-party/snowboy/resources,
+# and $ALEXA_AVS_SAMPLE_APP_PATH with the actual path where you put the Alexa AVS sample app repository.
 
-* Update `SENSITIVITY` in the [Alexa AVS sample app code](https://github.com/alexa/alexa-avs-sample-app/blob/master/samples/wakeWordAgent/src/KittAiSnowboyWakeWordEngine.cpp) and re-compile
+cp YOUR_UNIVERSAL_MODEL.umdl $ALEXA_AVS_SAMPLE_APP_PATH/third-party/snowboy/resources/
 
 ```
-# Please replace $ALEXA_AVS_SAMPLE_APP_PATH with the actual path where you
-# cloned the Alexa AVS sample app repository.
-cd $ALEXA_AVS_SAMPLE_APP_PATH/samples/wakeWordAgent/src/
 
-# Modify KittAiSnowboyWakeWordEngine.cpp and update SENSITIVITY at line 28.
-make
+* Replace the model name 'alexa.umdl' with your universal model name, update `KITT_AI_SENSITIVITY` in the [Alexa AVS sample app code](https://github.com/alexa/avs-device-sdk/blob/master/KWD/KWDProvider/src/KeywordDetectorProvider.cpp) and re-compile
+```
+# Modify $ALEXA_AVS_SAMPLE_APP_PATH/avs-device-sdk/blob/master/KWD/KWDProvider/src/KeywordDetectorProvider.cpp:
+#     Replace the model name 'alexa.umdl' with your universal model name 'YOUR_UNIVERSAL_MODEL.umdl' at line 52
+#     Update `KITT_AI_SENSITIVITY` at line 26
+sudo bash setup.sh config.json
 ```
 
 * Run the wake word agent with engine set to `kitt_ai`!
@@ -167,6 +190,11 @@ Here is the list of the models, and the parameters that you have to use for them
 * **resources/models/snowboy.umdl**: Universal model for the hotword "Snowboy". Set SetSensitivity to 0.5 and ApplyFrontend to false.
 * **resources/models/jarvis.umdl**: Universal model for the hotword "Jarvis" (https://snowboy.kitt.ai/hotword/29). It has two different models for the hotword Jarvis, so you have to use two sensitivites. Set sensitivities to "0.8,0.80" and ApplyFrontend to true.
 * **resources/models/smart_mirror.umdl**: Universal model for the hotword "Smart Mirror" (https://snowboy.kitt.ai/hotword/47). Set sensitivity to Sensitivity to 0.5, and ApplyFrontend to false.
+* **resources/models/subex.umdl**: Universal model for the hotword "Subex" (https://snowboy.kitt.ai/hotword/22014). Set sensitivity to Sensitivity to 0.5, and ApplyFrontend to true.
+* **resources/models/neoya.umdl**: Universal model for the hotword "Neo ya" (https://snowboy.kitt.ai/hotword/22171). It has two different models for the hotword "Neo ya", so you have to use two sensitivites. Set sensitivities to "0.7,0.7", and ApplyFrontend to true.
+* **resources/models/hey_extreme.umdl**: Universal model for the hotword "Hey Extreme" (https://snowboy.kitt.ai/hotword/15428). Set sensitivity to Sensitivity to 0.6, and ApplyFrontend to true.
+* **resources/models/computer.umdl**: Universal model for the hotword "Computer" (https://snowboy.kitt.ai/hotword/46). Set sensitivity to Sensitivity to 0.6, and ApplyFrontend to true.
+* **resources/models/view_glass.umdl**: Universal model for the hotword "View Glass" (https://snowboy.kitt.ai/hotword/7868). Set Sensitivity to 0.7, and ApplyFrontend to true.
 
 ## Precompiled node module
 

README_ZH_CN.md

@@ -156,8 +156,13 @@ Snowboy底层库由C++写成，通过swig被封装成能在多种操作系统和
 
 * **resources/alexa/alexa-avs-sample-app/alexa.umdl**：这个是为[Alexa AVS sample app](https://github.com/alexa/alexa-avs-sample-app)优化过的唤醒词为“Alexa”的通用模型，将`SetSensitivity`设置为`0.6`，并将`ApplyFrontend`设置为true。当`ApplyFrontend`设置为`true`时，这是迄今为止我们公开发布的最好的“Alexa”的模型。
 * **resources/models/snowboy.umdl**：唤醒词为“snowboy”的通用模型。将`SetSensitivity`设置为`0.5`，`ApplyFrontend`设置为`false`。
-* **resources/models/jarvis.umdl**: 唤醒词为“Jarvis” (https://snowboy.kitt.ai/hotword/29)的通用模型，其中包含了对应于“Jarvis”的两个唤醒词模型，所以需要设置两个`sensitivity`。将`SetSensitivity`设置为`0.8,0.8`，`ApplyFrontend`设置为`true`。
-* **resources/models/smart_mirror.umdl**: 唤醒词为“Smart Mirror” (https://snowboy.kitt.ai/hotword/47)的通用模型。将`SetSensitivity`设置为`0.5`，`ApplyFrontend`设置为`false`。
+* **resources/models/jarvis.umdl**: 唤醒词为“Jarvis” (https://snowboy.kitt.ai/hotword/29) 的通用模型，其中包含了对应于“Jarvis”的两个唤醒词模型，所以需要设置两个`sensitivity`。将`SetSensitivity`设置为`0.8,0.8`，`ApplyFrontend`设置为`true`。
+* **resources/models/smart_mirror.umdl**: 唤醒词为“Smart Mirror” (https://snowboy.kitt.ai/hotword/47) 的通用模型。将`SetSensitivity`设置为`0.5`，`ApplyFrontend`设置为`false`。
+* **resources/models/subex.umdl**: 唤醒词为“Subex”(https://snowboy.kitt.ai/hotword/22014) 的通用模型。将`SetSensitivity`设置为`0.5`，`ApplyFrontend`设置为`true`。
+* **resources/models/neoya.umdl**: 唤醒词为“Neo ya”(https://snowboy.kitt.ai/hotword/22171) 的通用模型。其中包含了对应于“Neo ya”的两个>唤醒词模型，所以需要设置两个`sensitivity`。将`SetSensitivity`设置为`0.7,0.7`，`ApplyFrontend`设置为`true`。
+* **resources/models/hey_extreme.umdl**: 唤醒词为“Hey Extreme” (https://snowboy.kitt.ai/hotword/15428)的通用模型。将`SetSensitivity`设置为`0.6`，`ApplyFrontend`设置为`true`。
+* **resources/models/computer.umdl**: 唤醒词为“Computer” (https://snowboy.kitt.ai/hotword/46) 的通用模型。将`SetSensitivity`设置为`0.6`，`ApplyFrontend`设置为`true`。
+* **resources/models/view_glass.umdl**: 唤醒词为“View Glass” (https://snowboy.kitt.ai/hotword/7868) 的通用模型。将`SetSensitivity`设置为`0.7`，`ApplyFrontend`设置为`true`。
 
 ## 预编译node模块
 

resources/alexa/alexa-avs-sample-app/avs-kittai.patch

@@ -0,0 +1,71 @@
+diff -Naur avs-sensory/pi.sh avs-kitt/pi.sh
+--- pi.sh	2019-05-03 18:09:18.063849909 -0700
++++ pi.sh	2019-05-03 18:09:39.273744305 -0700
+@@ -20,18 +20,19 @@
+
+ SOUND_CONFIG="$HOME/.asoundrc"
+ START_SCRIPT="$INSTALL_BASE/startsample.sh"
+-CMAKE_PLATFORM_SPECIFIC=(-DSENSORY_KEY_WORD_DETECTOR=ON \
++CMAKE_PLATFORM_SPECIFIC=(-DKITTAI_KEY_WORD_DETECTOR=ON \
+     -DGSTREAMER_MEDIA_PLAYER=ON -DPORTAUDIO=ON \
+     -DPORTAUDIO_LIB_PATH="$THIRD_PARTY_PATH/portaudio/lib/.libs/libportaudio.$LIB_SUFFIX" \
+     -DPORTAUDIO_INCLUDE_DIR="$THIRD_PARTY_PATH/portaudio/include" \
+-    -DSENSORY_KEY_WORD_DETECTOR_LIB_PATH=$THIRD_PARTY_PATH/alexa-rpi/lib/libsnsr.a \
+-    -DSENSORY_KEY_WORD_DETECTOR_INCLUDE_DIR=$THIRD_PARTY_PATH/alexa-rpi/include)
++    -DKITTAI_KEY_WORD_DETECTOR_LIB_PATH=$THIRD_PARTY_PATH/snowboy/lib/rpi/libsnowboy-detect.a \
++    -DKITTAI_KEY_WORD_DETECTOR_INCLUDE_DIR=$THIRD_PARTY_PATH/snowboy/include \
++    -DCMAKE_BUILD_TYPE=MINSIZEREL)
+
+ GSTREAMER_AUDIO_SINK="alsasink"
+
+ install_dependencies() {
+   sudo apt-get update
+-  sudo apt-get -y install git gcc cmake build-essential libsqlite3-dev libcurl4-openssl-dev libssl1.0-dev libfaad-dev libsoup2.4-dev libgcrypt20-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-good libasound2-dev sox gedit vim python3-pip
++  sudo apt-get -y install git gcc cmake build-essential libsqlite3-dev libcurl4-openssl-dev libssl1.0-dev libfaad-dev libsoup2.4-dev libgcrypt20-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-good libasound2-dev sox gedit vim python3-pip libatlas-base-dev
+   pip install flask commentjson
+ }
+
+@@ -64,18 +65,18 @@
+ build_kwd_engine() {
+   #get sensory and build
+   echo
+-  echo "==============> CLONING AND BUILDING SENSORY =============="
++  echo "==============> CLONING AND BUILDING KITTAI =============="
+   echo
+
+   cd $THIRD_PARTY_PATH
+-  git clone git://github.com/Sensory/alexa-rpi.git
+-  bash ./alexa-rpi/bin/license.sh
++  git clone https://github.com/Kitt-AI/snowboy.git 
++  cp snowboy/resources/alexa/alexa-avs-sample-app/alexa.umdl snowboy/resources/alexa.umdl
+ }
+
+ generate_start_script() {
+   cat << EOF > "$START_SCRIPT"
+   cd "$BUILD_PATH/SampleApp/src"
+
+-  ./SampleApp "$OUTPUT_CONFIG_FILE" "$THIRD_PARTY_PATH/alexa-rpi/models" DEBUG9
++  ./SampleApp "$OUTPUT_CONFIG_FILE" "$THIRD_PARTY_PATH/snowboy/resources" DEBUG9
+ EOF
+ }
+diff -Naur avs-sensory/setup.sh avs-kitt/setup.sh
+--- setup.sh	2019-05-03 18:09:24.383818365 -0700
++++ setup.sh	2019-05-03 18:09:35.193764563 -0700
+@@ -50,6 +50,8 @@
+ LIB_SUFFIX="a"
+ ANDROID_CONFIG_FILE=""
+
++BUIDTYPE="MINSIZEREL"
++
+ # Default device serial number if nothing is specified
+ DEVICE_SERIAL_NUMBER="123456"
+
+@@ -242,7 +244,7 @@
+   cmake "$SOURCE_PATH/avs-device-sdk" \
+       -DCMAKE_BUILD_TYPE=DEBUG \
+       "${CMAKE_PLATFORM_SPECIFIC[@]}"
+-
++  sed -E -i "s:CXX_PLATFORM_DEPENDENT_FLAGS_"$BUILDTYPE"\s+\"(.*)\":CXX_PLATFORM_DEPENDENT_FLAGS_"$BUILDTYPE" \"\1 -D_GLIBCXX_USE_CXX11_ABI=0 -pg\":" ../avs-device-sdk/build/cmake/BuildOptions.cmake
+   cd $BUILD_PATH
+   make SampleApp -j1
+