Lab 4: Live Translator

In this lab, you're going to use functions of several imported modules to build a live translator, an app where you can speak into your computer's microphone and hear a translation in another language.

Step 1: Install the packages you'll need

For this lab, we'll need several modules that are not installed by default in Thonny. To do this, in Thonny, click Tools -> Manage Packages.

In the resulting window, type "translate" in the search box and click the "Search on PyPI" button. This should show the translate package in the results - click it.

This should take you to a window with more information about the translate package. This is a package which will allow you to import a Python module with functions for translating text from one language to another - very cool! Click the Install button and wait for it to complete the installation of the package.

Similarly, search for and install the pyttsx3 package. This package is a voice synthesizer, which will allow you to create text-to-speech programs.

Step 2: Create a translation application

The following program imports the translate module and then creates a variable, translator, for the translator object (note that the target language is given as an argument to_lang="Spanish"). It then gets some text input from the user and calls the translator's translate() function with an argument of the text we want translated. Finally, it displays the result. This module's documentation can be found here: https://translate-python.readthedocs.io/en/latest/

Exercise 1: Create a script for this program. You can call it something like translate_text.py. Then, run the script to test it out. Pay attention to each of the variables in this program, and make a note about the purpose of each one.

import translate

translator = translate.Translator(to_lang="Spanish")

english_text = input("Enter some English text: ")
spanish_text = translator.translate(english_text)
print("Spanish text:",spanish_text)

Step 3: Create a text-to-speech application

The following program imports the pyttsx3 text-to-speech module and creates a variable, engine, using the init() function from the module. It then gets text input from the user, saving it to the text variable. There is then a two-step process to get the synthesizer to actually say the text - it passes text as an argument to the engine's say() function and then calls the engine's runAndWait() function.

You might be wondering how someone would know to call these functions in this order. Again, you have to look up the documentation for this module to see how to use it. A full list of all the available functions (as well as example code that goes through these steps in this order) is here: https://pyttsx3.readthedocs.io/en/latest/engine.html

Exercise 2: Create a script for this program. You can call it something like text_to_speech.py. Then, run the script to test it out. Pay attention to each of the variables in this program, and make a note about the purpose of each one.

import pyttsx3

engine = pyttsx3.init()

text = input("What do you want the computer to say? ")
engine.say(text)
engine.runAndWait()

Step 4: Put it together

Challenge Exercise 3: Now that you've seen how to use these two modules independently, put them together into one program that allows the user to type some input and have a translation spoken back in another language.

Note that when you import more than one module in a single script, you should do all of your import statements together at the very beginning of the file, like this:

import translate
import pyttsx3

When you are finished, submit your .py file to the Lab 4: Translator assignment. This assignment is not auto-graded, so you will not get any immediate feedback. Instead, I will manually look at your code and give any appropriate feedback at that time.

Congratulations, you built a really cool app! You may feel a little like you "cheated" in that all of the really hard stuff was taken care of by modules that you imported. However, let me assure that this is how professional programmers work. They learn to use building blocks that others have made, and they put them together in new and intersting ways. This takes advantage of the big computer science idea of abstraction and leverages the collective work of the software development community.

Optional Step 5: Install more packages

The rest of this lab is optional, but I encourage you to give it a shot so that you can get the experience of building something even cooler than the cool app you already made.

We are going to enhance our app so that the user can speak the input into the microphone (instead of typing at the keyboard) and hear their translation spoken back to them. For this, we will need two more packages: PyAudio and SpeechRecognition.

(Note: This probably won't work right off the bat for everyone. If installing PyAudio gives you an error, you can try some of the troubleshooting steps at the end of the lab. Or, you can just skip the rest of the lab.)

For the SpeechRecognition package, make extra sure to get the one with this specific name.

Optional Step 6: A speech-to-text application

We've seen a text-to-speech application, and now we're going to try it the other way around - allowing us to get spoken input from the user for our live translator.

The following code imports the speech_recognition module and then creates a variable, recognizer, for the recognizer object. We then see a new kind of statement that we haven't seen before: with speech_recognition.Microphone() as source:. This statement is basically the same thing as if we had written source = speech_recognition.Microphone() it creates a variable that represents your computer's microphone.

However, the with statement is a little more high-powered in that it comes with some extra error-handling. Because working with external resources like the microphone is prone to problems (e.g., What if another program is already using the microphone? What if there is no microphone? etc.), it allows us to only run the rest of the code if everything goes ok with accessing the microphone. Any code that is indented under the with statement will run only if accessing the microphone works. And, for this module, they require you to use a with statement.

This module's documentation is here: https://github.com/Uberi/speech_recognition/blob/master/reference/library-reference.rst There is more information and examples here: https://pypi.org/project/SpeechRecognition/

Optional Exercise 4: Create a script for this program. You can call it something like speech_to_text.py. Then, run the script to test it out. Pay attention to each of the variables in this program, and make a note about the purpose of each one.

import speech_recognition

recognizer = speech_recognition.Recognizer()

with speech_recognition.Microphone() as source:
    print("Speak")
    audio = recognizer.listen(source)
    text = recognizer.recognize_google(audio)
    print("Text:",text)

Optional Step 7: Putting it ALL Together

Optional Exercise 5: Now that you have the ability to write speech-to-text, text translation, and text-to-speech code; you have everything you need write a program that allows the user to speak into the microphone and hear back a spoken translation. Try putting it all together. If you get it working, you are welcome to submit this version to codePost instead of the solution to the challenge exercise above.

Troubleshooting

troubleshooting PyAudio install

On a Mac, in Thonny, go to Tools -> Open System Shell. This should give you a prompt where you can run some alternative install commands. Try running these in sequence (may require you to enter your password to install things):

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Then restart Thonny. Go to Tools -> Open System Shell and run the commands.

brew install portaudio

pip3 install --user pyaudio

On a Windows computer, try first installing the Community edition of Visual Studio here: https://visualstudio.microsoft.com/downloads/ . Then try installing PyAudio again.

troubleshooting speech_recognition

On my computer, I had trouble with the speech_recognition module when there was background noise in the room - try testing it somewhere quiet.

I also had trouble running it inside of Thonny, and I'm not sure why. The problem went away when I select the "Run current script in terminal" from the Run menu of the toolbar, which opens up an external terminal window to run the script in.

Ultimately, if you can't get this one working, don't worry about it too much. Some modules are a little more finicky than others, and as you get more experience, you will be in a better place to further troubleshoot.