Skip to content

Commit

Permalink
Add topic detection feature to batch transcribe (#71)
Browse files Browse the repository at this point in the history 
* add topic detection feature to batch transcribe

* change to detect-topics and topics

* update readme

* update version

---------

Co-authored-by: aaron_ng <[email protected]>
Co-authored-by: Tudor Evans <[email protected]>
  • Loading branch information
@aaronng91 @TudorCRL
3 people authored Aug 25, 2023
1 parent f67559d commit fc57b65
Show file tree
Hide file tree
Showing 19 changed files with 508 additions and 99 deletions.
6 changes: 6 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [1.11.0] - 2023-08-25

### Added

- Add support for batch topic detection

## [1.10.0] - 2023-08-02

### Added
Expand Down
6 changes: 6 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,12 @@ A complete list of commands and flags can be found in the SDK docs at https://sp
speechmatics batch transcribe --sentiment-analysis --output-format json-v2 example_audio.wav
```
- Submit a job with topic detection
```bash
speechmatics batch transcribe --detect-topics --output-format json-v2 example_audio.wav
```
### Custom Transcription Config File
- Instead of passing all the transcription options via the command line you can also pass a transcription config file.
The config file is a JSON file that contains the transcription options.
Expand Down
2 changes: 1 addition & 1 deletion VERSION
View file
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.10.0
1.11.0
73 changes: 65 additions & 8 deletions docs/_modules/speechmatics/batch_client.html
View file

Large diffs are not rendered by default.

34 changes: 18 additions & 16 deletions docs/_modules/speechmatics/client.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,28 +42,26 @@ <h1>Source code for speechmatics.client</h1><div class="highlight"><pre>
<span class="kn">import</span> <span class="nn">json</span>
<span class="kn">import</span> <span class="nn">logging</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Union</span>
<span class="kn">from</span> <span class="nn">urllib.parse</span> <span class="kn">import</span> <span class="n">parse_qsl</span><span class="p">,</span> <span class="n">urlencode</span><span class="p">,</span> <span class="n">urlparse</span><span class="p">,</span> <span class="n">urlunparse</span>

<span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Union</span>
<span class="kn">import</span> <span class="nn">httpx</span>
<span class="kn">import</span> <span class="nn">websockets</span>

<span class="kn">from</span> <span class="nn">speechmatics.exceptions</span> <span class="kn">import</span> <span class="p">(</span>
<span class="n">EndOfTranscriptException</span><span class="p">,</span>
<span class="n">ForceEndSession</span><span class="p">,</span>
<span class="n">TranscriptionError</span><span class="p">,</span>
<span class="p">)</span>
<span class="kn">from</span> <span class="nn">speechmatics.helpers</span> <span class="kn">import</span> <span class="n">get_version</span><span class="p">,</span> <span class="n">json_utf8</span><span class="p">,</span> <span class="n">read_in_chunks</span>
<span class="kn">from</span> <span class="nn">speechmatics.models</span> <span class="kn">import</span> <span class="p">(</span>
<span class="n">ClientMessageType</span><span class="p">,</span>
<span class="n">ServerMessageType</span><span class="p">,</span>
<span class="n">AudioSettings</span><span class="p">,</span>
<span class="n">ClientMessageType</span><span class="p">,</span>
<span class="n">ConnectionSettings</span><span class="p">,</span>
<span class="n">ServerMessageType</span><span class="p">,</span>
<span class="n">TranscriptionConfig</span><span class="p">,</span>
<span class="n">UsageMode</span><span class="p">,</span>
<span class="p">)</span>
<span class="kn">from</span> <span class="nn">speechmatics.constants</span> <span class="kn">import</span> <span class="n">RT_SELF_SERVICE_URL</span>
<span class="kn">from</span> <span class="nn">speechmatics.config</span> <span class="kn">import</span> <span class="n">read_config_from_home</span>
<span class="kn">from</span> <span class="nn">speechmatics.helpers</span> <span class="kn">import</span> <span class="n">get_version</span><span class="p">,</span> <span class="n">json_utf8</span><span class="p">,</span> <span class="n">read_in_chunks</span>
<span class="kn">from</span> <span class="nn">speechmatics.models</span> <span class="kn">import</span> <span class="n">ClientMessageType</span><span class="p">,</span> <span class="n">ServerMessageType</span>

<span class="n">LOGGER</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>

Expand Down Expand Up @@ -92,16 +90,19 @@ <h1>Source code for speechmatics.client</h1><div class="highlight"><pre>
<span class="bp">self</span><span class="p">,</span>
<span class="n">connection_settings_or_auth_token</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">ConnectionSettings</span><span class="p">,</span> <span class="kc">None</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
<span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot; </span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Args:</span>
<span class="sd"> connection_settings_or_auth_token (Union[str, ConnectionSettings, None], optional): _description_. Defaults to None.</span>
<span class="sd"> If `str`,, assumes auth_token passed and default URL being used</span>
<span class="sd"> If `None`, attempts using auth_token from config.</span>
<span class="sd"> connection_settings_or_auth_token (Union[str, ConnectionSettings, None], optional): Defaults to None.</span>
<span class="sd"> If `str`,, assumes auth_token passed and default URL being used</span>
<span class="sd"> If `None`, attempts using auth_token from config.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">connection_settings_or_auth_token</span><span class="p">,</span> <span class="n">ConnectionSettings</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connection_settings</span> <span class="o">=</span> <span class="n">ConnectionSettings</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="n">UsageMode</span><span class="o">.</span><span class="n">RealTime</span><span class="p">,</span> <span class="n">connection_settings_or_auth_token</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connection_settings</span> <span class="o">=</span> <span class="n">ConnectionSettings</span><span class="o">.</span><span class="n">create</span><span class="p">(</span>
<span class="n">UsageMode</span><span class="o">.</span><span class="n">RealTime</span><span class="p">,</span> <span class="n">connection_settings_or_auth_token</span>
<span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connection_settings</span> <span class="o">=</span> <span class="n">connection_settings_or_auth_token</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connection_settings</span><span class="o">.</span><span class="n">set_missing_values_from_config</span><span class="p">(</span><span class="n">UsageMode</span><span class="o">.</span><span class="n">RealTime</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">websocket</span> <span class="o">=</span> <span class="kc">None</span>
<span class="bp">self</span><span class="o">.</span><span class="n">transcription_config</span> <span class="o">=</span> <span class="kc">None</span>
<span class="bp">self</span><span class="o">.</span><span class="n">translation_config</span> <span class="o">=</span> <span class="kc">None</span>
Expand Down Expand Up @@ -443,7 +444,8 @@ <h1>Source code for speechmatics.client</h1><div class="highlight"><pre>
<span class="n">stream</span><span class="p">,</span>
<span class="n">transcription_config</span><span class="p">:</span> <span class="n">TranscriptionConfig</span><span class="p">,</span>
<span class="n">audio_settings</span><span class="o">=</span><span class="n">AudioSettings</span><span class="p">(),</span>
<span class="n">from_cli</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
<span class="n">from_cli</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
<span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Begin a new recognition session.</span>
<span class="sd"> This will run asynchronously. Most callers may prefer to use</span>
Expand Down Expand Up @@ -487,11 +489,11 @@ <h1>Source code for speechmatics.client</h1><div class="highlight"><pre>
<span class="n">extra_headers</span><span class="p">[</span><span class="s2">&quot;Authorization&quot;</span><span class="p">]</span> <span class="o">=</span> <span class="n">token</span>

<span class="n">url</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">connection_settings</span><span class="o">.</span><span class="n">url</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">url</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">transcription_config</span><span class="o">.</span><span class="n">language</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">url</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">transcription_config</span><span class="o">.</span><span class="n">language</span><span class="o">.</span><span class="n">strip</span><span class="p">()):</span>
<span class="k">if</span> <span class="n">url</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="s2">&quot;/&quot;</span><span class="p">):</span>
<span class="n">url</span> <span class="o">+=</span> <span class="bp">self</span><span class="o">.</span><span class="n">transcription_config</span><span class="o">.</span><span class="n">language</span>
<span class="n">url</span> <span class="o">+=</span> <span class="bp">self</span><span class="o">.</span><span class="n">transcription_config</span><span class="o">.</span><span class="n">language</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">url</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">&quot;/</span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">transcription_config</span><span class="o">.</span><span class="n">language</span><span class="si">}</span><span class="s2">&quot;</span>
<span class="n">url</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">&quot;/</span><span class="si">{</span><span class="bp">self</span><span class="o">.</span><span class="n">transcription_config</span><span class="o">.</span><span class="n">language</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span><span class="si">}</span><span class="s2">&quot;</span>

<span class="c1"># Extend connection url with sdk version information</span>
<span class="n">cli</span> <span class="o">=</span> <span class="s2">&quot;-cli&quot;</span> <span class="k">if</span> <span class="n">from_cli</span> <span class="ow">is</span> <span class="kc">True</span> <span class="k">else</span> <span class="s2">&quot;&quot;</span>
Expand Down
Loading

0 comments on commit fc57b65

Please sign in to comment.