Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented microphone modes #853

Merged
merged 4 commits into from
Oct 10, 2020
Merged

Conversation

LexiconCode
Copy link
Member

@LexiconCode LexiconCode commented Oct 4, 2020

Description

Implemented microphone modes to toggle microphone on, off, and sleep.

The new commands
caster <mode> on, sleep, off (DNS/DPI only)

These modes use grammar/rule exclusivity to block other grammars from being recognized.

  • Kaldi uses still recognize dictation but no action will be executed. The same is true for DNS/DPI but it's not printed out the window.
  • DNS/DPI is also a bit special if you change the mic mode from the DNS/DPI GUI it will stay in sync with Caster mic modes
  • DNS/DPI built-in commands for waking up and going to sleep are still functional as they rely on exclusivity internally. There is no method that I know of to disable these internal commands.

These modes are implemented through 2 rules. The 1st rule CasterMicRule to execute the function to enable exclusivity. The 2nd rule sleep_rule leverages native dragonfly to become exclusive via ExclusiveManager overriding CasterMicRule and all other rules loaded into the engine. sleep_rule uses dragonfly FuncContext to toggle on and off. Both rules use EngineModesManager to manage the different modes based on the engine is running. There are some significant differences. DNS utilizes Natlink and _sync_mode to synchronize modes then uses ExclusiveManager. The other engines utilize just ExclusiveManager.

Note because native dragonfly was used for sleep_rule simplified transformers cannot be used on CasterMicRule or sleep_rule.

Could this implementation be simplified?
If we could create separate CCR merger that's for exclusive grammars only it would drastically simplify the implementation.

What about additional modes?
These are some thoughts for the future but not in this PR. I did not want to take longer to get the sleep mode out to users as it impacts the usability of Caster.

  • Implement set_engine_mode exclusivity with mode rules. dictation, numbers, and spell for all engines
  • Implement timer for sleep mode for all engines but DNS/DPI.
  • Implement hotkey for microphone on-off

Related Issue

#797

Motivation and Context

Motivation for this is to unify microphone modes command regardless of the speech recognition backend for users.

How Has This Been Tested

I've tested this with DNS and Kaldi switching between microphone modes.

Types of changes

  • Docs change / refactoring / dependency upgrade
  • Bug fix (non-breaking change which fixes an issue or bug)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Renamed existing command phrases (we discourage this without a strong rationale).

Checklist

  • I have read the CONTRIBUTING document.
  • My code follows the code style of this project.
  • I have checked that my code does not duplicate functionality elsewhere in Caster.
  • I have checked for and utilized existing command phrases from within Caster (delete if not applicable).
  • My code implements all the features I wish to merge in this pull request.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests pass.

Maintainer/Reviewer Checklist

  • Basic functionality has been tested and works as claimed.
  • New documentation is clear and complete.
  • Code is clear and readable.

@LexiconCode LexiconCode added New Feature A new feature that is not currently implemented. Caster Core Spec Issues for Caster Specs that are not part of the application or language specific grammar/ccr labels Oct 4, 2020
@LexiconCode LexiconCode self-assigned this Oct 4, 2020
extras = [
Choice(
"mic_mode", {
"off": "off",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get this when saying "caster off"

Caster: 'off' is not valid. set_mic_state modes are: 'off' - DPI Only, 'on', 'sleeping'

Is that deliberate? We should also have "caster wake" as an option.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LexiconCode thanks for doing this!

Copy link
Member Author

@LexiconCode LexiconCode Oct 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is intentional ('off' - DPI Only). For other engines besides DPI/DNS to have an off state they need a method to turn the microphone on without voice activation like a keyboard shortcut/GUI toggle. DPI/DNS has that capability. Why sleep with sleeping?

There's a difference between sleep the command and sleeping which is passed as a parameter. sleeping is what Natlink uses as a string to put DNS to sleep natlink.setMicState(mode).

The mode needs to be checked in EngineModesManager because they can be set in the caster settings for the engine to start up in a certain mode. Because the user is defining a string (a mode) within the settings which could be anything must be checked against valid modes.

"caster wake" would be possible but I don't really know why unless there is recognition issues. It's nice to provide something that's concise and intuitive. "wake" does feel natural but that's because of my experience of DNS. I would think most people without that experience would think "on" "off" "sleep".

One other bit of feedback is that the dictation element needs to be weighted for kaldi in sleep_rule?

@LexiconCode LexiconCode added the Caster Issues pertaining to primarily the Caster project. label Oct 5, 2020
@esc123
Copy link

esc123 commented Oct 6, 2020

Using DPI 15.3 on Windows 10, all three commands (on, off, sleep) work perfectly. Good stuff.

@LexiconCode LexiconCode merged commit 8b0dd35 into dictation-toolbox:master Oct 10, 2020
@LexiconCode LexiconCode deleted the sleep branch October 10, 2020 04:11
@kendonB
Copy link
Collaborator

kendonB commented Oct 11, 2020

@LexiconCode is there an easy way to implement automatic sleeping functionality? I often forget that the system is on after using it for a few minutes and end up making it do a bunch of crazy things because I'm speaking around it or listening to something.

@LexiconCode
Copy link
Member Author

@LexiconCode is there an easy way to implement automatic sleeping functionality? I often forget that the system is on after using it for a few minutes and end up making it do a bunch of crazy things because I'm speaking around it or listening to something.

Yes I think that shouldn't be too bad to do. It would involve creating a timer that is reset every time there's a successful recognition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Caster Core Spec Issues for Caster Specs that are not part of the application or language specific grammar/ccr Caster Issues pertaining to primarily the Caster project. New Feature A new feature that is not currently implemented.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants