Fisher Innovation Ellie (2012)

Ellie is a Fisher Innovation project creating a customized voice recognition system with assistance capabilities for use in the home. Ellie can take in spoken English commands, perform tasks and calculations and respond via synthesized speech.

Project Overview

Ellie can take in spoken English commands, perform tasks and calculations, and respond via synthesized speech. What makes Ellie different from other voice command systems is that Ellie listens for input 24/7, this allows her to be a hands free way to easily get tasks done around the home.

Language Model

Ellie’s language model is a customized set of expectations created by Fisher Innovation. By creating a custom language model for Ellie to learn from, we lower the rate of failure in recognizing specific vocal inputs from the user. Additions to the language model can be made at any time from a web based configuration tool which allows large or small blocks of English text to be parsed used for future reference when attempting to understand vocal input. All data related to Ellie’s corpus are stored in MySQL database tables for quick reference and manipulation. For more information on the text corpus, checkout the article on Wikipedia.

External Plugin System

By default Ellie comes pre-packaged with only the basic set of plug-ins to enable basic usage for post installation tests. To add further functionality to Ellie, the application requires the use of external plug-ins. By creating a dependency on external plug-ins to further the feature set of Ellie, you not only gain the ability to setup your system as you require, but this application flow easily allows for developers to create and append their own Ellie plugins and features.

Over the course of pre alpha development many great plug-ins have been created to enable many different tasks and calculations.

  • Apple Airplay Integration
  • MLB (Baseball) Stats and Game Times
  • Bitcoin Exchange Value Data (Mt. Gox)
  • Music Server Control (mpd)
  • Google Services - Gmail, Search, Places, Maps
  • Movie Information via Rotten Tomatoes
  • Serial Communication - Arduino
  • Weather Information (Weather Network)
  • Wolfram Alpha Search Queries
  • X10 Communication (Home Automation)

Plug-in Syntax


# A X10 Plugin for Ellie
# @author: Matt Fisher (
# "X10 is a protocol (set of commands) for controlling lights, appliances,
# and other electrical devices in your house to make it a "smart home".
# The commands (electrical signals) travel over the existing electrical
# lines in your house so no cabling is required."

from subprocess import call
import X10utils

class X10(object):

    # Initialize.
    def __init__(self):
        print '>> Exec: Init X10'

    def initUnit(self, house, unit, initialState):
        """ Adds a unit to the system.

        @param house: The alphabetical value of the house setting on the unit.
        @param unit: The numerical value of the units unit setting.
        @param initialState: A string value of 'on' or 'off' to set the state of the unit.

        if(house.isalpha() == False):
            return False

        if(unit.isnumeric() == False):
            return False

        if(state != 'on' | state != 'off'):
            return False

    def setUnitState(self, house, unit, state):
        """ Sets the state of a specified unit in a specified house.

        @param house: The alphabetical value of the house setting on the unit.
        @param unit: The numerical value of the units unit setting.
        @param state: A string value of 'on' or 'off' to set the state of the unit.

        print '>> Exec: X10.setUnitState - ' + str(house) + ' ' + str(unit) + ' ' + str(state)

        if(house.isalpha() == False):
            print 'b1'
            return False

        # Call bottlerocket and run the command.
        call(["br", "-x " + str(x10Port) + " " + str(house) + str(unit) + " " + str(state)])

    def destroy(self):

Ellie is ran as a daemon background process on any *NIX system with the dependencies installed. Once instantiated, Ellie will make a audible alert that she has completed her run up configuration and then wait for verbal input from someone in the room. Ellie requires an initial dedicated verbal command to start the calculation and response sequence. For example, saying ”Hey Ellie..” will alert the system to ready itself for a command. No commands are listening to until you have got Ellie’s attention. This feature was added to avoid unknowing and unwanted input from general conversation in the vicinity of the microphones.


Ellie would not exist without the aid of a ton of great open source projects. Although the use of additional plug-ins may require further software to run, the base installation of Ellie requires the following.

  • Python 2.6.5
  • Apache2
  • PHP5
  • MySQL Server
  • festival
  • sphinx
  • libasound2-dev prior (Ubuntu)
  • sphinx_lm_convert
  • pocketsphinx
  • python-pocketsphinx
  • text2wfreq
  • wfreq2vocab
  • idngram2lm

Get in touch

Feel free to contact me anytime by any of the mediums listed on the about page or via the contact form below.

Contact Form

Success! you can expect a response within the next day or two.

Email: fisher.matt at