Send feedback

Getting Started

To get up and running with Serenade, just follow these three steps!

1. Download

Download and install Serenade. Serenade supports macOS, Windows, and Linux—platform-specific installation instructions can be found on the download page.

2. Setup

After installing Serenade, you'll be prompted to activate the app via an email link. If that link doesn't work, you can use the code provided on the activation page to activate manually.

After activation, Serenade can automatically install plugins for supported editors—Atom and VS Code. Make sure to restart your editor after the plugins are installed (to make sure the Serenade editor plugin is running), and you're ready to go!

You can use Serenade with your laptop microphone, though we recommend using a headset for the best accuracy. You can verify that a headset is being used by clicking , and then Settings.

3. Learn

The first time you open Serenade, an interactive tutorial will walk you through writing your first lines of code with voice. Give it a try! You can also see more interactive guides here.

Concepts

Let's start by going through a few concepts that you'll see throughout the Serenade documentation.

Environment

Serenade floats above all your other windows, so you can keep it side-by-side with your editor. You can toggle Serenade by clicking the Listening switch or pressing Alt+Space. Then, as you speak, you'll see a list of transcripts in the Serenade window.

Sometimes, Serenade isn't sure what you said, so you'll see a few different options. The first one will be used automatically, but to use a different option (and undo the first one), just say the number you want to use instead. For instance, to use the second option, just say two. If none of the options are right, you can just say undo.

Documentation

Voice commands are shown in orange. The code that's produced from that voice command is shown in green:

    add print
    print()

Some examples will also include the original code for additional context:

    add argument name
    print()
    print(name)

pink preformatted text is used to represent the syntax for voice commands.

  • add hello

    Text with no formatting is a literal token for each command.

      add hello
      hello
  • add <expression>

    Text in angle brackets is a type token—see Types Reference for a full reference of what each type means.

      add hello comma world
      hello, world
  • add print [<expression>]

    Text in square brackets are optional, and may be a literal token or a type token.

      add print
      print()
      add print in quotes hello
      print("hello world")
  • add (if | else if | while) <condition>

    Text in parentheses represents a list of different tokens that may be used in a position (i.e., an or).

      add if x equals 1
      if (x == 1)
      add while x equals 1
      while (x == 1)

Adding Code

In this section, we'll take a look at voice commands you can use for writing code. Text in angle brackets is a type token—see Types Reference for a full list about what each means.

type

The type command inserts text at the current cursor position, and it's the simplest way to add text. The type command is useful when you just want to insert plain text. For instance, you could use a type command to append text to the end of an existing line of code.

add

Whereas the type command just inserts text, the add command is a more powerful way to write larger programming constructs, like a function or while loop. The add command will try to find the nearest valid position to the cursor to add the construct, or let you know if it can't.

expand / collapse all
  • add (argument | parent | raise) <identifier>
    add argument name
    print()
    print(name)
    add extends animal
    class Bird:
        pass
    class Bird(Animal):
        pass
    add parent animal
    Interchangeable with extends
    class Bird:
        pass
    class Bird(Animal):
        pass
    add print height
    print(height)
    add raise e
    Interchangeable with throw
    raise e
    add throw e
    Interchangeable with raise
    raise e
  • add [catch | decorator | except | finally | print | return value] <expression>
    add height plus equals two
    height += 2
    add catch e
    Interchangeable with except
    try:
        pass
    except e:
        pass
    add decorator with wings
    def fly():
        pass
    @with_wings
    def fly():
        pass
    add except e
    Interchangeable with catch
    try:
        pass
    except e:
        pass
    add finally
    try:
        pass
    except:
        pass
    finally:
        pass
    add return value height
    def fly():
        return
    def fly():
        return height
  • add (class | enum | interface) <identifier>
    add class bird
    class Bird:
        pass
    add enum colors
    class Colors(enum.Enum):
        pass
  • add [<modifiers>] [<type>] method <identifier>
    add method fly
    class Bird:
        pass
    class Bird:
        def fly(self):
            pass
    add string method chirp
    class Bird:
        pass
    class Bird:
        def chirp(self) -> str:
            pass
  • add [<type>] (function | parameter) <identifier>
    add function average
    def average():
        pass
    add int function factorial
    def factorial() -> int:
        pass
    add parameter speed
    def run():
        pass
    def run(speed):
        pass
    add string parameter name
    def say_hello():
        pass
    def say_hello(name: string):
        pass
  • add comment <text>
    add comment TODO
    # TODO
  • add (else | try) <body>
    add else return none
    else:
        return None
    add try x equals parse of y
    try:
        x = parse(y)
    except e:
        pass
  • add (if | else if | while) <condition>
    add if n is less than two
    if n < 2:
        pass
    add else if n is less than ten
    elif n < 10:
        pass
    add while n is greater than ten
    while n > 10:
        pass
  • add for <expression> in <identifier>
    add for n in numbers
    for n in numbers:
        pass
  • add import <identifier> [as <identifier>]
    add import requests
    import requests
    add import numpy as np
    import numpy as np
  • add type <type>
    add type string
    def name():
        return "Scout"
    def name() -> str:
        return "Scout"

newline

The newline command adds a new line at the cursor. To use it, simply say:

Editing Code

In this section, we'll take a look at voice commands you can use for editing existing code.

go to

The go to command moves your cursor around a file. A selector represents something in the code you want to navigate to, like a word, line, function, or argument. If you just say a selector, then a go to command will be run by default, so you can say line fifty as a shortcut for go to line fifty.

For a complete list of selectors, check out the selectors reference.

go to is optional, so saying a selector on its own will move the cursor to the selector:

  • <selector>

You can also optionally set the position of the cursor to the beginning or the end of a selector:

Finally, Serenade can move your cursor in any direction:

  • up
  • down
  • left
  • right

delete

You can delete code with the delete command.

For a complete list of selectors, check out the selectors reference.

  • delete <selector>
    delete if
    delete lines three through five
    delete next two functions
    delete to end of line

copy, cut, paste

You can copy and cut text with the copy and cut commands. Then, you can paste that selection with the paste command.

For a complete list of selectors, check out the selectors reference.

  • (copy | cut) <selector>
    copy first function
    copy lines fifty to sixty
    cut next class
    cut next two words

select

You can highlight text in your editor with the select command. Then, you can use commands like delete to operate on that selection.

For a complete list of selectors, check out the selectors reference.

  • select <selector>
    select superclass
    select previous three characters
    select to end of block
    select words one to three

change

The change command selects the nearest match to the cursor and replaces it with some text.

For a complete list of selectors, check out the selectors reference.

  • change <selector> to <text>
      change phrase fly to jump
      def fly():
          pass
      def jump():
          pass
      change word fly to jump
      def fly():
          pass
      def jump():
          pass

Symbols and Formatting

You'll often want to specify text formatting and add symbols to your code. In this section, we'll see how to do that with voice.

Text Formatting

By default, text will be lowercase, and words will be separated by spaces. To format text using camel case, underscores, etc., you can prefix any text with a style:

  • [capital | camel case | pascal case | all caps | underscores] <text>
    type sing song
    sing song
    type capital sing song
    Sing song
    type camel case sing song
    singSong
    type pascal case sing song
    SingSong
    type all caps sing song
    SING_SONG
    type underscores sing song
    sing_song

Single Symbols

You can include symbols when speaking text in any type, add, change, etc. command.

To specify a single symbol, you say the symbol's name within any text token:

  • [text] <symbol> [text]
    type dot
    .
    add if x plus y mod 2 is equal to 0
    if x + y % 2 == 0:
        pass

  • Below is a list of all of the single symbols you can use:

Enclosure Symbols

Enclosure symbols can be used to wrap text. You can also dictate opening and closing enclosure symbols separately.

  • [text] in <enclosure symbol>s <text>
    add f of n minus one
    f(n - 1)
    add in parens x plus y
    (x + y)
    type in double underscores main
    __main__
    add foo equals in quotes bar
    foo = 'bar'

  • Below is a list of all of the enclosure symbols you can use:

Escaping Symbols

If you need to use the literal text representation of a symbol (e.g., the word dash rather than the - character, you can escape it in a few ways:

  • escape <single symbol>
    type plus
    +
    type escape plus
    plus

Workflow

In this section, we'll see how to use Serenade to control your editor, window, and system.

Editor Controls

You can control your editor window with Serenade, which includes opening files and navigating tabs.

  • save

    Save the current file.

  • open <text>

    Open a file whose name contains the specified text as a substring.

  • style file

    This will automatically format your entire source code file.

  • new tab

    Create a new editor tab.

  • close tab

    Close the current tab.

  • (next | previous) tab

    Switch to the next or previous tab.

  • tab (one | two | three ...)

    Switch to a specific tab.

  • pause

    Pause Serenade, so it stops listening to you.

  • split (left | right | up | down)

    Create an editor split in the specified direction.

  • (left | right | up | down) window

    Navigate to the editor split in the specified direction.

Window Switching

A common part of many workflows is switching applications. To focus an app with Serenade, you can use the focus command. For instance, to bring Firefox to the foreground, you can say:

focus firefox

Here, firefox is a substring of the name of the application you want to focus. You can use the focus command for any application, not just those with supported plugins.

System Controls

Serenade can send keypresses globally into any focused application. press will send specific keystrokes, while type will transcribe text and convert symbols.

  • press <key>
    press enter
    press command k
  • type <text>
    type g mail dot com
    gmail.com

Serenade can also control applications (such as web browsers) with the following functionality without a plugin:

  • (new | close | next | previous) tab
  • undo
  • redo
  • back
  • forward
  • open <url>

Chaining

You can chain commands together to avoid having to pause between commands. For example, you can combine a end of file command with a paste command by saying end of file paste.

Repetition

Serenade supports several options for repetition, or executing the same command multiple times:

  • (delete | paste | indent | dedent) ... <number> (times | <selector>s)

    Add number times to certain commands to execute it that number of times. Or say a selector to modify multiple selectors.

    indent two times
    y = f(x)
        y = f(x)
    delete five characters
    extraordinary
    ordinary
  • repeat [<text> | <number> times]
    repeat will repeat the previous command. Added text will repeat the last command containing that text. You can also repeat more than once.

Custom Commands

With the Serenade API, you can write your own voice commands. Custom commands are written via JavaScript files in your ~/.serenade/scripts directory. Each script in that directory will have access to a global object called serenade that serves as the entry point for the Serenade API. If you have any syntax errors in your scripts, they'll be displayed in the Serenade app.

Snippets

Snippets are a powerful way to create shortcuts for code you type regularly. To define new snippets, add them to ~/.serenade/scripts/snippets.js (you may need to create this file if it doesn't exist yet).

Here's an example of a snippet that creates a new Python method whose name is prefixed with test_.

serenade.language("python").snippet(
  "test method <%name%>",
  "def test_<%name%>(self):<%newline%><%indent%>pass",
  { "name": ["identifier", "underscores"] },
  "method"
);

Now, if you say test method foo, the following code will be generated:

def test_foo(self):
    pass

As you can see, the snippet method takes four parameters:

  • A string that specifies the trigger for the voice command. Surrounding text in <% %> creates a matching slot that matches any text. You can then reference the matched text in the generated snippets, much like regular expression capture groups.
  • A snippet to generate, specified via a Mustache template. If you defined a matching slot called <%name%> in the trigger, then <%name%> in the snippet will be replaced by whatever text was matched in the transcript.
  • A map of slots to styles. Styles describe how text should be formatted, and a slot can have multiple styles. For instance, if a slot represents an identifier (e.g., a class name) where symbols aren't allowed, and that identifier should be pascal case, then the values ["identifier", "pascal"] could be used. Possible style values are:
    • caps: All capital letters.
    • capital: The first letter of the first word capitalized.
    • camel: Camel case.
    • dashes: Dashes between words.
    • lowercase: Spaces between words.
    • pascal: Pascal case.
    • underscores: Underscores between words.
    • expression: An expression where symbols are supported. For instance, saying dash will produce -, not dash.
    • identifier: The name of a function, class, variable, etc., where symbols are not supported. For instance, saying foo dash will produce foo_dash, since foo- isn't a valid function, class, etc. name.
    • condition: Similar to expression, but specifically for the condition of an if, for, while, etc. For instance, in a condition, saying equals will produce ==, since that's typical for a condition.
  • How to add the snippet to your code. In the above example, we're specifying that this block should be added as a method, so if your cursor is outside of a class, it will move to the nearest class before inserting anything, just as it would if you said "add method". The default value for this argument is statement. Possible values include:
    • argument
    • attribute
    • catch
    • decorator
    • else
    • else_if
    • extends
    • finally
    • method
    • parameter
    • return_value
    • statement
    • tag

As another example, here's a snippet to add a new React class in a JavaScript file:

serenade.language("javascript").snippet(
  "new component <%name%>",
  "class <%name%><%cursor%> extends React.Component {<%newline%>}",
  { "identifier": ["identifier", "pascal"] }
);

Notice that you can use the special slot <%cursor%> to specify where the cursor will be placed after the snippet. The full list of special slots is:

  • <%cursor%>: Where the cursor will be placed after the snippet is added.
  • <%indent%>: One additional level of indentation.
  • <%newline%>: A newline.
  • <%terminator%>: The statement terminator for the current language, often a semicolon.

As one last example, here's a snippet to create a Java class with an extends and implements in one command:

serenade.language("java").snippet(
  "new class <%name%> extends <%extends%> implements <%implements%>",
  "public class <%name%><%cursor%> extends <%extends%> implements <%implements%> {<%newline%>}",
  {
    "name": ["pascal", "identifier"],
    "extends": ["pascal", "identifier"],
    "implements": ["pascal", "identifier"]
  },
  "class"
);

Workflow Automation

You can also use the Serenade API to automate workflows. To define new snippets, use the file~/.serenade/scripts/workflows.js (you may need to create this file if it doesn't exist yet).

For instance, to switch to the terminal and then run the command make clean && make, you can create a custom command that looks like this:

serenade.global().command("make", async api => {
  await api.focus("term");
  await api.typeText("make clean && make");
  await api.pressKey("return");
});

The global method specifies that we'd like this command to be triggerable from any application. The command method takes two arguments: the voice trigger for the command, followed by an asynchronous function to execute. An api instance is passed to your async function, which enables you to automate a workflow. Inside of that function, we're using the await keyword to make sure that each operation completes before moving onto the next one. Don't forget to make your function async via the async keyword!

Here's another example that finds text on the current page inside of Chrome when you say a command like find foo:

serenade.app("chrome").command("find <%text%>", async (api, matches) => {
  await api.pressKey("f", ["command"]);
  await api.typeText(matches.text);
});

This time, rather than specifying global(), we used .app("chrome") to make this command valid only when Google Chrome is focused. As with snippets, you can use slots to capture text. In this example, we used a slot called <%text%> to determine what to type into the Chrome search box.

Here's one last example that opens a terminal and then runs two different yarn commands:

serenade.global().command("yarn <%first%> then <%second%>", async (api, matches) => {
  await api.focus("term");
  await api.typeText("yarn " + matches.first);
  await api.pressKey("return");
  await api.wait(3000);
  await api.typeText("yarn " + matches.second);
  await api.pressKey("return");
});

As you can see, when you have multiple slots, the second argument to your callback will be a map, where the keys are the names of your slots, and the values are the matched text.

API Reference

Within your custom commands, you can use the Serenade API for even more control and flexibility.

class Serenade

Methods to create new Builder objects with either a global scope or scoped to a single application. You can access an instance of this class via the serenade global in any script.

class Builder

Methods to register new voice commands.

class API

Methods for workflow automation. An instance of API is passed as the first argument to the callback passed to the command method on a Builder.

Keys

You can speak any key name in order to reference it in a Serenade command.

Types Reference

<text>

A <text> token is arbitrary text, and will automatically include mapped symbols unless they're escaped. <text> supports formatting and symbols—see Symbols and Formatting for more.

<identifier>

An <identifier> token is the name of a function, class, variable, or other entity.

<modifier>

A list of <modifiers> optionally precedes the defintion of a class, enum, function, or method (depending on language). Possible modifiers are:

[abstract | async | default | export | private | protected | public | static]

<selector>

A <selector> describes a block of code. By default, a selector applies to the nearest matching object. You can optionally apply previous or next to each selector, as in previous line or next line. Selectors can also include an ordinal prefix/postfix, as well as a range of objects.

Below is a list of objects you can select:

<expression>

An <expression> is a subset of <text> that automatically applies formatting to code, like spaces around symbols.

<condition>

A <condition> is a subset of <expression> that uses symbol definitions that are more common in conditions (e.g, for an if or while). For instance, equals resolves to == rather than =

<type>

A <type> is a subset of <expression> that uses symbol definitions that are more common in types. For instance, list of string resolves to List<String> in Java, rather than list(string).

Tips and Tricks

Here are some tips and tricks for using Serenade more effectively:

  • Instead of worrying about spacing while writing code, use style file once your code is in a reasonable state, and Serenade will automatically format the entire file.
  • Long type or add commands can be less accurate than shorter commands. If you're seeing low accuracy, try breaking up longer commands into smaller, separate commands.
  • The type and add commands are similar, but have distinct use cases—add works best for writing new lines of code, including larger constructs like functions or classes, while type works best for inserting or appending text to an existing line.
  • Use copy/cut, paste, and go to liberally in order to more quickly move code around.
  • Make sure your system's microphone volume is set to the right level. We've found that around 80%, to make sure the input isn't too loud or too quiet, works best, but try moving it around if Serenade's results aren't accurate.
  • When using Serenade, try to speak conversationally, as though you were pair programming with someone seated next to you. Over-enunciating words can actually make Serenade less accurate!
  • If Serenade doesn't know a word, like numpy, you can spell it in a command, as with add import n u m p y.

Troubleshooting