Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FLUID-5936: fix bug in fluid.textToSpeech.checkTTSSupport, rewrite tests to use IoC testing #732

Merged
merged 45 commits into from
Sep 2, 2016
Merged
Show file tree
Hide file tree
Changes from 43 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
f3de9d5
Merge pull request #6 from fluid-project/master
waharnum Jul 27, 2016
8a0255f
FLUID-5936: use correct API event handler for end of speech
waharnum Jul 28, 2016
a66947d
FLUID-5936: add a comment about the tests currently being skipped in …
waharnum Jul 28, 2016
e85d86a
FLUID-5936: rewrite tests to use the IoC framework
waharnum Jul 28, 2016
2dba2cf
FLUID-5936: restructure tests to better handle variations in asynchro…
waharnum Aug 4, 2016
03f3dd1
FLUID-5936: linting
waharnum Aug 4, 2016
8be2d97
FLUID-5936: potential fix for test failures on Windows
waharnum Aug 5, 2016
15b6cf6
FLUID-5936: continuing the quest for a sequencing approach that will …
waharnum Aug 5, 2016
a92d6ed
FLUID-5936: issue pause/resume commands in test with slight delay
waharnum Aug 5, 2016
9143c35
FLUID-5936: set volume to 0
waharnum Aug 5, 2016
53099b7
FLUID-5936: linting
waharnum Aug 5, 2016
d6077c5
FLUID-5936: 50ms timeout for control interaction
waharnum Aug 8, 2016
d05ac06
FLUID-5936: move asynchronous command issuance into main component fr…
waharnum Aug 8, 2016
3f162b6
FLUID-5936: reset volume to 0
waharnum Aug 8, 2016
5ca02a8
FLUID-5936: use an asyncTest wrapper around the promise to prevent th…
waharnum Aug 15, 2016
13ba66a
FLUID-5936: add a comment about setTimeout of asyncTest wrapper
waharnum Aug 16, 2016
5b08b2a
FLUID-5936: fix comment line length
waharnum Aug 16, 2016
12d3b15
FLUID-5936: simplify asynctest structure
waharnum Aug 17, 2016
5f6809e
FLUID-5936: extract generalized wrapper for choosing which test to ex…
waharnum Aug 17, 2016
fca22cd
Merge pull request #7 from fluid-project/master
waharnum Aug 17, 2016
49bd622
FLUID-5936: use 'task' naming convention.
waharnum Aug 22, 2016
d4e918e
FLUID-5936: comment further on the promise-based test execution
waharnum Aug 22, 2016
ad41acf
FLUID-5936: use task naming convention properly.
waharnum Aug 22, 2016
6f8f445
FLUID-5936: queueing implementation, async wrapper for all speech com…
waharnum Aug 25, 2016
09caf85
FLUID-5936: refactoring
waharnum Aug 25, 2016
5a73622
FLUID-5936: further refactoring
waharnum Aug 25, 2016
5129f87
FLUID-5936: more refactoring, more aggressive testing
waharnum Aug 25, 2016
e35df8e
FLUID-5936: updated comment.
waharnum Aug 25, 2016
1e5af4d
FLUID-5936: refactoring
waharnum Aug 25, 2016
7db392b
FLUID-5936: store the currentUtterance on a member of the component t…
waharnum Aug 29, 2016
2accf4f
FLUID-5936: reduce volume of test to 0
waharnum Aug 29, 2016
543b95d
Merge pull request #8 from fluid-project/master
waharnum Aug 30, 2016
71f116b
Merge branch 'master' into FLUID-5936
waharnum Aug 30, 2016
df09711
NOJIRA: add keyword-spacing rule to linter, correct two files for tha…
waharnum Aug 30, 2016
98a0330
Merge branch 'keyword-spacing' into FLUID-5936
waharnum Aug 30, 2016
00bedf1
FLUID-5936: lintin
waharnum Aug 30, 2016
e9cd164
FLUID-5936: comment in accidental comment out.
waharnum Aug 30, 2016
71bc4ec
FLUID-5936: place currentUtterance in queue structure with texts.
waharnum Aug 30, 2016
5b2c2da
FLUID-5936: add some tests for the currentUtterance implementation.
waharnum Aug 30, 2016
75f3de3
FLUID-5936: remove onpause/onresume events
waharnum Aug 30, 2016
f4d3152
FLUID-5936: modelizing
waharnum Aug 30, 2016
e9e6069
FLUID-5936: implement throttle-based control
waharnum Sep 1, 2016
265f3fe
FLUID-5936: remove unneeded function.
waharnum Sep 1, 2016
839f735
FLUID-5936: store utterance as part of queue array.
waharnum Sep 1, 2016
fe384eb
FLUID-5936: refactor queue implementation to clear queue item on hand…
waharnum Sep 1, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
"allow-null"
],
"indent": ["error", 4],
"keyword-spacing": "error",
"new-cap": ["error", { "properties": false }],
"no-caller": "error",
"no-cond-assign": [
Expand Down
138 changes: 110 additions & 28 deletions src/components/textToSpeech/js/TextToSpeech.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ Licenses.

You may obtain a copy of the ECL 2.0 License and BSD License at
https://github.com/fluid-project/infusion/raw/master/Infusion-LICENSE.txt

Includes code from Underscore.js 1.8.3
http://underscorejs.org
(c) 2009-2016 Jeremy Ashkenas, DocumentCloud and Investigative Reporters & Editors
Underscore may be freely distributed under the MIT license.

*/

/* global speechSynthesis, SpeechSynthesisUtterance*/
Expand Down Expand Up @@ -45,15 +51,15 @@ var fluid_2_0_0 = fluid_2_0_0 || {};
var toSpeak = new SpeechSynthesisUtterance(" "); // short text to attempt to speak
toSpeak.volume = 0; // mutes the Speech Synthesizer
var timeout = setTimeout(function () {
speechSynthesis.cancel();
fluid.textToSpeech.deferredSpeechSynthesisControl("cancel");
promise.reject();
}, delay || 1000);
toSpeak.onstop = function () {
toSpeak.onend = function () {
clearTimeout(timeout);
speechSynthesis.cancel();
fluid.textToSpeech.deferredSpeechSynthesisControl("cancel");
promise.resolve();
};
speechSynthesis.speak(toSpeak);
fluid.textToSpeech.deferredSpeechSynthesisControl("speak", toSpeak);
} else {
setTimeout(promise.reject, 0);
}
Expand All @@ -66,15 +72,19 @@ var fluid_2_0_0 = fluid_2_0_0 || {};
events: {
onStart: null,
onStop: null,
onPause: null,
onResume: null,
onError: null,
onSpeechQueued: null
},
members: {
queue: []
queue: {
texts: []
// out-of-function-scope storage for the currently spoken utterance
// prevents issues with premature garbage collection in
// some browsers
// currentUtterance:
}
},
// Model paths: speaking, pending, paused, utteranceOpts
// Model paths: speaking, pending, paused, utteranceOpts, pauseRequested, resumeRequested
model: {
// Changes to the utteranceOpts will only text that is queued after the change.
// All of these options can be overriden in the queueSpeech method by passing in
Expand All @@ -83,7 +93,7 @@ var fluid_2_0_0 = fluid_2_0_0 || {};
utteranceOpts: {
// text: "", // text to synthesize. avoid as it will override any other text passed in
// lang: "", // the language of the synthesized text
// voiceURI: "" // a uri pointing at a voice synthesizer to use. If not set, will use the default one provided by the browser
// voice: {} // a WebSpeechSynthesis object; if not set, will use the default one provided by the browser
// volume: 1, // a value between 0 and 1
// rate: 1, // a value from 0.1 to 10 although different synthesizers may have a smaller range
// pitch: 1, // a value from 0 to 2
Expand All @@ -94,9 +104,13 @@ var fluid_2_0_0 = fluid_2_0_0 || {};
listener: "fluid.textToSpeech.speak",
args: ["{that}", "{change}.value"]
},
"paused": {
listener: "fluid.textToSpeech.pause",
args: ["{that}", "{change}.value"]
"pauseRequested": {
listener: "fluid.textToSpeech.requestControl",
args: ["{that}", "pause", "{change}"]
},
"resumeRequested": {
listener: "fluid.textToSpeech.requestControl",
args: ["{that}", "resume", "{change}"]
}
},
invokers: {
Expand All @@ -109,12 +123,12 @@ var fluid_2_0_0 = fluid_2_0_0 || {};
args: ["{that}"]
},
pause: {
"this": "speechSynthesis",
"method": "pause"
changePath: "pauseRequested",
value: true
},
resume: {
"this": "speechSynthesis",
"method": "resume"
changePath: "resumeRequested",
value: true
},
getVoices: {
"this": "speechSynthesis",
Expand Down Expand Up @@ -142,23 +156,81 @@ var fluid_2_0_0 = fluid_2_0_0 || {};
}
});

// Issue commands to the speechSynthesis interface with deferral (1 ms timeout);
// this makes the wrapper behave better when issuing commands, especially
// play and pause
fluid.textToSpeech.deferredSpeechSynthesisControl = function (control, args) {
setTimeout(function () {
speechSynthesis[control](args);
}, 1);
};

// Throttle implementation adapted from underscore.js 1.8.3; see
// file header for license details
// Returns a version of a function that will only be called max once
// every "wait" MS
fluid.textToSpeech.throttle = function (func, wait, options) {
var timeout, context, args, result;
var previous = 0;
if (!options) options = {};

var later = function() {
previous = options.leading === false ? 0 : new Date().getTime();
timeout = null;
result = func.apply(context, args);
if (!timeout) context = args = null;
};

var throttled = function() {
var now = new Date().getTime();
if (!previous && options.leading === false) previous = now;
var remaining = wait - (now - previous);
context = this;
args = arguments;
if (remaining <= 0 || remaining > wait) {
if (timeout) {
clearTimeout(timeout);
timeout = null;
}
previous = now;

result = func.apply(context, args);
if (!timeout) context = args = null;
} else if (!timeout && options.trailing !== false) {
timeout = setTimeout(later, remaining);
}
return result;
};

throttled.cancel = function() {
clearTimeout(timeout);
previous = 0;
timeout = context = args = null;
};
return throttled;
};

// Throttled version of deferred speech synthesis control
fluid.textToSpeech.throttleControl = fluid.textToSpeech.throttle(fluid.textToSpeech.deferredSpeechSynthesisControl, 100, {leading: false});

fluid.textToSpeech.speak = function (that, speaking) {
that.events[speaking ? "onStart" : "onStop"].fire();
};

fluid.textToSpeech.pause = function (that, paused) {
if (paused) {
that.events.onPause.fire();
} else if (that.model.speaking) {
that.events.onResume.fire();
fluid.textToSpeech.requestControl = function (that, control, change) {
// If there's a control request (value change to true), clear and
// execute it
if(change.value) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume still fails linting :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What fails (the sequel) is actually my ability to install my precommit hooks on my laptop as well as my work desktop to make sure I can't commit unlinted code.

that.applier.change(change.path, false);
fluid.textToSpeech.throttleControl(control);
}
};

fluid.textToSpeech.handleStart = function (that) {
that.queue.shift();
that.queue.texts.shift();
that.applier.change("speaking", true);

if (that.queue.length) {
if (that.queue.texts.length) {
that.applier.change("pending", true);
}
};
Expand All @@ -170,7 +242,11 @@ var fluid_2_0_0 = fluid_2_0_0 || {};
paused: false
};

if (!that.queue.length) {
if (that.queue.currentUtterance) {
that.queue.currentUtterance = undefined;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually meant to put it in the queue item, rather than to rebase the entire queue structure.
So, on line 280 we would have that.queue.push({text: text, utterance: toSpeak})
queue can then just go back to being a [] and the rest of the changes reverted

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree this is better; I had to refactor the queue behaviour a little so that.queue.shift happens at handleEnd rather than handleStart or we run into the same premature garbage collection behaviour in Safari that we were seeing previously. I don't think this is a problematic change (it also lets handleStart simply be a changePath record rather than a function), but please let me know if you foresee an issue with that behavioural change.

}

if (!that.queue.texts.length) {
var newModel = $.extend({}, that.model, resetValues);
that.applier.change("", newModel);
}
Expand All @@ -186,6 +262,12 @@ var fluid_2_0_0 = fluid_2_0_0 || {};
};

var toSpeak = new SpeechSynthesisUtterance(text);
// Store toSpeak additionally on the queue.currentUtterance member to help deal with the issue
// with premature garbage collection described at https://bugs.chromium.org/p/chromium/issues/detail?id=509488#c11
// this makes the speech synthesis behave much better in Safari in
// particular
that.queue.currentUtterance = toSpeak;

var eventBinding = {
onstart: that.handleStart,
onend: that.handleEnd,
Expand All @@ -195,14 +277,14 @@ var fluid_2_0_0 = fluid_2_0_0 || {};
};
$.extend(toSpeak, that.model.utteranceOpts, options, eventBinding);

that.queue.push(text);
that.queue.texts.push(text);
that.events.onSpeechQueued.fire(text);
speechSynthesis.speak(toSpeak);
fluid.textToSpeech.deferredSpeechSynthesisControl("speak", toSpeak);
};

fluid.textToSpeech.cancel = function (that) {
that.queue = [];
speechSynthesis.cancel();
that.queue.texts = [];
fluid.textToSpeech.deferredSpeechSynthesisControl("cancel");
};

})(jQuery, fluid_2_0_0);
2 changes: 1 addition & 1 deletion src/framework/preferences/js/Panels.js
Original file line number Diff line number Diff line change
Expand Up @@ -440,7 +440,7 @@ var fluid_2_0_0 = fluid_2_0_0 || {};
*/
fluid.prefs.compositePanel.hideInactive = function (that) {
fluid.each(that.options.components, function (componentOpts, componentName) {
if(fluid.prefs.compositePanel.isPanel(componentOpts.type, componentOpts.options) && !fluid.prefs.compositePanel.isActivePanel(that[componentName])) {
if (fluid.prefs.compositePanel.isPanel(componentOpts.type, componentOpts.options) && !fluid.prefs.compositePanel.isActivePanel(that[componentName])) {
that.locate(componentName).hide();
}
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@
<script type="text/javascript" src="../../../test-core/jqUnit/js/jqUnit-browser.js"></script>
<script type="text/javascript" src="../../../lib/jquery-ui/js/jquery.simulate.js"></script>

<!-- needed for IoC testing framework -->
<script type="text/javascript" src="../../../../src/framework/enhancement/js/ContextAwareness.js"></script>
<script src="../../../test-core/utils/js/IoCTestUtils.js"></script>

<!-- These are tests that have been written using this page as data and test supports -->
<script type="text/javascript" src="../js/TextToSpeechTests.js"></script>

Expand Down
Loading