raspberry Pi virtual assistant
January 7, 2024, 06:38
I’m looking to make a virtual assistant ran through chat gpt using the raspberry pi 4.
I have virtually 0 experience with the software so I want to learn why I’m doing things, not just how. Any help is appreciated.
how granular do you want to go?
If i say "here's a github project to do this, with install instructions" is that what you want?
Or you want to actually make something 'front scratch'?
“From scratch” as much as possible would be preferred. I have very limited experience with what I’m doing
Well I suppose
How much harder is doing it from scratch? Is it going to be like an absurd amount of coding and software that I am in no way going to understand
no not really
you just need 3 main concepts. the ai to detect what your saying and turn it into text, the chatgpt api and then an ai to speak out what chatgpts response is
we can do that very easily within python, i can help you
if you want to start from scratch i would first go reseaching for speach to text apis, just ask if you need help with this
I can do that once I know what the words mean
The only thing I’m not 100% on is what’s an api
an API is a way of interacting with a service from the code
it links that bit of code to your account. usually so they can limit you on how much you use it
Because they have to make money
for example you use open air api to use chatgpt. that way they can know how much you've used it and decide how much to charge you
Okay so to clarify the basic components would be:
-The API
-The actual Chat GPT AI
-The text to speech AI
-And then whatever software will be needed to interface with the mic and speakers
yes I mean the pi kinda has that last part sorted so I think that'd be the easiest part
Would I be able to use a headset for testing purposes until I got the more permanent ones
And also is there a certain version of noobs I’m gonna want to use, I know that some software like omxplayer have problems with newer versions
luckily the chatgpt api is easy af
That’s what I’m hoping for because I have virtually 0 clue what I’m doing, I’m gonna need like a literal step by step
Okay so what is the very very first step
Would it be getting the right software on my sd card
yeah probably
you use terminal or gui?
i think best option here for you as a beginner is to first develop the software on your pc and then use it in your raspberry, i guess you would use some interpreted lang like python, so the change of architecture should not be a problem
I suggest you use gui here
and yea develop the programs on your pc and send them to the pi
Okay so firstly what does any of that mean
Idk what terminal or gui means
Second what program would I use to develop this on my pc
ok terminal is something you can use to control your pi
Like the black text box in the top left when your on the pi desktop
you use it by typing etherything out. and it does it for and retrieves the information. but you can't see anything so it's like your blind and telling your friend what to do
yes exactly that
Ok I’m tracking
some people use the pi only from that. they never see like the browser button or anything they just tell the terminal to click it for them. this saves resources but it's much harder for beginners
So I’m gonna use something similar on my pc to write the code and then basically just send it over to the pi and since the framework is pretty similar it shouldn’t be a huge change for the pi
so a gui is the thing you can see, so you click the browser button yourself
yes python is the same in both
you could use thonny on both
I think that’s what I was doing at first like actually typing the git hub urls and opening folders and doing installs with commands
thonny is a python editor for beginners
yeah
Oh so get thonny on my pc to write the code there?
yeah
and it's already installed on your pi
do you know about python at all?
Very very little
ok I would suggest taking some time learning it a bit more otherwise your gonna have no clue what's happening
So I’m just gonna put the pi to the side for now and just work off my computer
yup
Okay
What type of things should I learn
I can give you some tutorials on python if you want?
I know like basic logic gates like if/and lines and some stuff like that
you need to understand python and apis well come with that
ok good. you still need a little more
but DW python is actually really easy once you get into it
you want me to link a tutorial now?
Stuff like libraries and stuff I understand too
Yeah sure
oh you've basically got it.
There was this website I was using to learn that’s basically like learning python to solve like retro game style probelms and I’m trying to find it to refresh myself with the code but I can’t remember what it’s called
do this first, it should be a breeze for you but you will need it https://www.kaggle.com/learn/intro-to-programming
Okay cool. Give me a few and I’ll get back to you after that
yeah, then once youve done that. https://www.kaggle.com/learn/python you can go through this. but don't worry I don't expect you to do all this in a day this could take you a week if needs be
but it's certainly a great place to start
Okay cool, let me take a look at the first one
so gui is graphical user interface, like all the user interface you got on windows, everything is visual and rendered, and a terminal is cli, command line interface, everything is based off commands and text, all there is are characters
if you want to do something 99% of the time you would have to enter a command
instead of clicking something as you would on a gui
Okay yeah makes total sense I’m tracking
<@1071178789939331253> I still get the premises but I’m going through all the lessons thoroughly just to get a solid refresher
One thing that’s tripping me up in variables. Can I not define 1 variable with another?
what do you mean?
like this
py var_1 = 1 var_2 = var_1?
For example:
Births_per_minute = 250
Births_per_hour = births_per_minute * 60
yes you can
are you getting an error?
The terminal keeps saying the births_per_minute variable isn’t defined but my first line of code is defining it
Yes
you might have misspelled it
I got it figured out
Small syntax error
yup
How would I write a more complex arithmetic question to define a variable
This seems to be throwing up a flag
Wait I think I got that covered by changing it from // to just /
Now how can I make the resulting answer round to the nearest whole nunber
you learning programming rn?
I am
you learn fast ngl
I have a small bit of experience
I should know this stuff
But I do tend to pick things up
dont know python, is too complicated for me
i just use rust
What’s rust?
a compilled lang, it was just a joke
another coding launguage, mostly based of python
i never got to like it, python better anyway
Hey <@1071178789939331253> question
wait you saying it unironically?
How can I make the result of a arithmetic questions round to the nearest whole number
if you got 1.5 what would you expect it to round to?
2 or 1?
Well I have 38.38383838383838 repeating and I just want it to round to the nearest whole number for the sake of not needing a crazy high accuracy for the exercise
no it wasnt a joke or anything
numb = round(numb)
its really far from python, like really really far
not based off it or anything like that
I’ve tried that but what should replace “numb” the variable I’m trying to round?
yes
oh i didnt know, can you show me some code as an example?
Ohhhhhhjh
That makes way more sense
you will see in the near future that a programer is just a expert internet searcher, nobody remembers that stuff
I was thinking it was it’s own line not just a part of the equation
google is your best freind, you will learn that some day
<@1071178789939331253> this website you sent me is confusing
It’s asking me to print a fraction of something so I made it spit out the percentage values and it’s saying that’s incorrect. Is there a better way to do it
a fraction is not percentage values
i can send a less confusing tutorial, that one is slightly confusing on how it works ik
No I mean I’m getting the instruction I just don’t know the premise of what their asking
sure
rust
use dialoguer::Select; //dialoguer:Select | this let us print options that the user chooses like "buttons", comes handy in this type of program
use std::error::Error;
use std::io; //standard::input/output | AFAIK this library type let us capture input from the terminal //standard::error::Error | for error handling
fn main() -> Result<(), Box<dyn Error>> {
// good execution and error handling (Result expect () if everything good and everything else as error)
let selection = Select::new()
.item("Fahrenheit to Celsius")
.item("Celsius to Fahrenheit")
.interact()?; //everything else
match selection {
0 => convertor(false, "Celsius"),
1 => convertor(true, "Fahrenheit"),
_ => return Err("Invalid selection".into()), // everything else returns as error
};
Ok(())
}
fn convertor(choice: bool, unit: &str) {
loop {
println!("Input the number to convert");
let mut number = String::new(); // we do not know the value yet
io::stdin()
.read_line(&mut number) // mutable reference
.expect("Failed to read line");
let mut number: f32 = match number.trim().parse() {
Ok(num) => num,
Err(_) => continue, // this is why we are using a loop, continue skips into the next iteration
}; // i could have used promptly :b | update, i do not know how to use promptly
match choice {
false => number = (number - 32.0) 5.0 / 9.0, //f to c
true => number = number 9.0 / 5.0 + 32.0, // c to f
}
println!("The result is {number:.2}°{unit}"); // .2 in number means 2 decimals after point
break;
}
}
because it runs in an actual enviroment, i prefer it this way because you can do what you want
How do I make this function give a fraction instead of a percentage
can you take a screenshot for me?
oh it kinda looks more like js or c# to me
Of my solution or of the problem
of the problem
pretty far from those too, its more like a c / c++ / haskell
Well there’s both
ive never got into c at all so i wouldnt know
and never heard of haskell
purely functional lang, hard af
ok this will take a sec im rusty
Haha no prob
I know this probably isn’t functional to what I’m doing but I just want to try to understand the basis of every step in the process
i think its not asking for percentages
have you filled this in?
as said, its asking about fractions, numbers between 0 an 1
remove the *100
No it’s not that’s the problem lol, that’s what I need to learn how to do
yes
exactly, remove the * 100
and thats it
did you put the * theyre?
Oh yeah that did it
yup
btw they also give you hints if you need it
Yeah cuz I wanted my function to vice me a whole number percentage value
Yeah Ik I’m just trying to do it without those
but they dont need a percentage, they need a fraction. i fraction by definition is just a number devided by a number so thats all you need to do
dw theyres no shame in using hints if you need it. like dont always use them but if you need it use them because this is for learning, not a streak of not using hints lol
Yeah but in the real world there’s no hints and I tend to start relying on the hints and then being screwed when I need to type something not in a tutorial
lol
its funny how they use stdout to read the values instead of using a function, pretty clever.
yeah
dont overthink it
yeah live life how you want to
you will then get used to not having hints, but hints at the start can be good
i mean dont be lazy but rememeber you only have one life, live it how you want
many people started learning how to ride the bycicle using those help wheels. pilots cannot start learning without helps and hints
Oh hell guys now we’re getting to some more complicated functions
Now the hints are gonna start comin lol
good
yup
What concepts should I make sure I understand before I am good to start on the virtual assistant project
if you finish that course and learn about modules and apis your good
most probably yes
Ok cool
there are videos about it, about making a chatgpt chatbot yourself using the api
you can follow that then if you get lost
once you know whats happening i can help you through all of, ive already made some code that does this so you decide how much i want to help you
Oooh ok on makes sense
I’m making good headway, should be done with those lessons in about 30-45 mins
great
Okay I’m about done
Where should I start
with the second one too?
No just the first, I’ll start the second one now
ok great
I think I figured this out
But
Wait nvm yeah I answer my own questions
What would be way to make 2 variables switch values
Like if a=2 and b=1 how can I switch them
There’s a command for it but I can think of it
I don't actually know
That was markedly simple lol
I just put a, b = b, a
>>> Using tuple unpacking: Python allows you to swap the values of two variables with a single line of code using tuple unpacking. This method is concise, easy to understand, and is especially well-suited for swapping two variables.
py
a = 5
b = 10
a, b = b, a
print("a:", a)
print("b:", b)
oh you already got it
Yeah lol I guess you can also do it with a temp value but that seems so much harder
nah, I mean it's a slight waste of space but that's about it
Hey <@1071178789939331253> and <@1005288119807004822> did I over complicate this lol
And if not I’m confused on what this error is meaning
I figured it out
<@1071178789939331253> finished with that second set of lessons
Where we go from here
ok just do some research on speach to text apis
and if you can get some working code from it
If I wanted to make that myself Ide have to get into machine learning and all that wouldn’t I
Oh yeah hell no I’ll definitely be finding an api that’s super complicated
How’s that one sound?
yeah but even I can't do that yet
this looks great
Awesome
So let’s start from scratch
I need to create a new Python project in Thonny
yes
don't do what the GitHub says yet I want to look at the code
Okay
Is that’s gonna be the first thing at the top of the new Python project tho?
ok go into thonny and create a new file. name it virtual assistant or something
yup but we're doing it differently because we need to control it ourselves
py
import speech_recognition as sr
recognizer = sr.Recognizer()
with sr.Microphone() as source:
print("Adjusting noise ")
recognizer.adjust_for_ambient_noise(source, duration=1)
print("Recording for 10 seconds")
recorded_audio = recognizer.listen(source, timeout=10)
print("Done recording")
try:
print("Recognizing the text")
text = recognizer.recognize_google(
recorded_audio,
language="en-US"
)
print("Decoded Text : {}".format(text))
except Exception as ex:
print(ex)
sr.Microphone.list_microphone_names() paste this into thonny and yk talk and see if it works
Okay let’s see standby
heres the error I got
> %Run -c $EDITOR_CONTENT
> Traceback (most recent call last):
> File "<string>", line 1, in <module>
> ModuleNotFoundError: No module named 'speech_recognition'
> >>>
do I need to import the git hub link first
Do I need to do $ git clone https://github.com/Kalebu/Python-Speech-Recognition-
oh yeah run pip install speech_recognition in the terminal
you can get this by clicking on tools and then terminal
ERROR: Could not find a version that satisfies the requirement speech_recognition (from versions: none)
ERROR: No matching distribution found for speech_recognition
[notice] A new release of pip is available: 23.0.1 -> 23.3.2
[notice] To update, run: python.exe -m pip install --upgrade pip
try cloning the repo then
run that command at the bottom there?
how do I do that
run
git clone https://github.com/Kalebu/Python-Speech-Recognition-
'git' is not recognized as an internal or external command,
operable program or batch file.
then run cd Python-Speech-Recognition-
hmm
thats the error it threw up when I did git clone https://github.com/Kalebu/Python-Speech-Recognition-
you might need to find a new ai for this
let me look for some for you
okay, ill look too
<@1071178789939331253>
I mean thats pretty much the entire thing done ritght there lol
yeah but that what we're trying to avoid right?
yeah, I just didnt know if we wanted to take the speech recogniton part from that cuz idk if thats linked to chat gpt
oh I found the reason the other one didn't work
try running pip install SpeechRecognition in the terminal again and run the script I said to put into thonny again
whatre we lookin at
roger
the original GitHub repo
> %Run -c $EDITOR_CONTENT
> Traceback (most recent call last):
> File "C:\Users\Logan\AppData\Local\Programs\Thonny\lib\site-packages\speech_recognition\__init__.py", line 108, in get_pyaudio
> import pyaudio
> ModuleNotFoundError: No module named 'pyaudio'
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "<string>", line 7, in <module>
> File "C:\Users\Logan\AppData\Local\Programs\Thonny\lib\site-packages\speech_recognition\__init__.py", line 80, in __init__
> self.pyaudio_module = self.get_pyaudio()
> File "C:\Users\Logan\AppData\Local\Programs\Thonny\lib\site-packages\speech_recognition\__init__.py", line 110, in get_pyaudio
> raise AttributeError("Could not find PyAudio; check installation")
> AttributeError: Could not find PyAudio; check installation
> >>>
thats the error we got now
it worked fine in the terminal
but script didnt run though
I need to install a program called PyAudio?
try running pip install PyAudio and adding import PyAudio at the start of the script
after this I really got to get to sleep so I'll have to help you in the morning
where do I put it in the script
at the start
oh wait I made a mistake
Traceback (most recent call last):
File "<string>", line 1
pip install PyAudio
^^^^^^^
SyntaxError: invalid syntax
yeah somethin aint right
change it accordingly
sorry I'm extremely rusty
too busy with Linux stuff to do much python anymore
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'PyAudio'
I just installed the module tho
when you installed it have any errors?
nope
went just fine
interesting
it said successfully installed and everything
can I help you fix it in the morning it's half 1 am for me
yeah of course
have a good night, I appreciate you
aw thanks that made me smile
I appreciate you being one of the easiest people here to help
haha I feel like im being a pain
no your not dw
Here you go, it's all open source so if you want to build from scratch you can.
https://openvoiceos.org/
And yes it can run headless on a raspberry pi. Been doing a lot of work with their team lately
Hey thanks for the link
I’m not sure I understand any of this in the slightest but if your here and willing to explain I’m here to listen
What is it that you're not understanding.
I know next to nothing about any of those terms
Like Ide need a step by step on how to actually implement that into any code or into the pi or anything
I can send you the image I am using, you just burn it to the sd, plug in a monitor and keyboard into the pi and do what the voice says
looks great. thanks you
Oh cool, that makes it easy
Do you want to use that link or continue with the one we where doing before
depends, this is probably better if you want the easy and better sollution
i dont doubt that will be 10x easier and will be a lot better than i could do especially the display, but it is the easy way so it wont be challenging at all and you might not learn as much
Is this one going to have all the functionality I want?
should do
apparently it can even use that display for the animation and stuff you want
And he said it’s open source so does that mean I can go back and change or add code whenever I want
I know Ide need a lot more experience but eventually Ide like to be able to go back and add a function to allow it to interface with my pc and other tech at my setup
yup, though i need to check what lauguage, one sec
great
Oh okay awesome
oh cool they seem to have pluggins too
Oh awesome
How do I find the download link for the img
https://downloads.openvoiceos.org/images/raspbian/ this is a bit confusing but id select "latest"-->"ovos-raspbian.zip"
Ok, let me try burning that to this sd and see if that does much
then download it to the sd card
👍
So I burned it and put it into the pi and it started up and ran a bunch of commands and then just shut off and a red light came on on the side of the pi
wait what do you mean by burned?
Like I copied it onto the sD card with the imager
the raspi imager?
Yes
you dont need to do that
apart from just formatting the card with the imager all you need to do is click on that website, click on what i told you to and download it to the sd card
then plug it into your raspberry pi
Okay let’s try that then
reformat the sd card though by selecting "erase" at the bottom of the imager
Yup did that
But now it’s formatted as fat32 isn’t it
ok good then download and plug it in
Just download the zip file straight onto the sD card?
Or extract it to there
yes extract it to there
Ok
Okay we’re copying now
This is all that shows up and won’t change to anything else
hmm, <@625486818321760256> can you send over your image?
also how is that display connected? via a thin wire that connects to a thing on the pi or through the gpio?
It’s a micro hdmi to hdmi
ok good
im waiting on him to reply to us now then
Yeah cool, I’m just gonna do a little more research till then
PS. The QR code on the boot screen is a feature introduced 3 years ago to make troubleshooting easier
Do you have any experience with the image?
The QR code didn’t yield much
anything at all?
Not much help, it might be more useful to you
i thought it just linked to imager download page when an image isnt detected
what does it come up with?
Give me 1 sec, I have to set everything back up
ok
It just links this
Saying I don’t have an OS
yeah thought so
Do you want to try working on that other bit of code in the meantime
From yesterday
oh yeah
try it, we might have to record a wav file and use it that way
not preferable but its a option too
That’s our current error code
go into the tools section and select manage packages
Yeah I tried that last night
But it says PyAudio is installed
it is a module, atleest pypi says so
on there?
ok very weird
ok highlight the name and paste it in so we know we havent spelt it incorrectly
still same
Does it need an update possibly?
i dont think so
in bit when i can ill find some code so you can use wav files instead
some code i know works
Ok
Do you reckon we could try this then?
As a bae
Base*
Yes, one moment
Bare in Mind I customized my image heavily
how heavily and what have you changed?
this is it
py import sounddevice as sd from scipy.io.wavfile import write import assemblyai as aai from text_to_speech import save from ctransformers import AutoModelForCausalLM # Sample rate fs = 44100 # Sample rate seconds = 1 # Duration of recording # Replace with your API token aai.settings.api_key = f"c9be33bfb60b41fbaebb13bd8168a639" # URL of the file to transcribe FILE_URL = r"C:\file_path\output.wav"
Boot screen, making a bunch of private skills amongst other things
my version is super privatized
can you send us a base image you know works?
Would be very much appreciated to get a good starting point
Of course
oh cool, might think about this myself. did you have to code this or install pluggins for it to work?
like code the mods
Code, I am working in partnership with their team to pull this off
ok <@1182277325463765042> click the one that says gui
great. i might do this myself
Ok cool, give me just a few mins
sure
remember to extract it
Here's a good template skill
BTW i would recommend using the imager to set wifi, username and password
i couldnt find out, is etherything in python or is it a mix?
Python
oh wait is this not a full image by its self?
It's very complex to build a custom skill
thanks for the headsup
it has a way to set up wifi if you listen to the voice
assuming you have a speaker
Does it work with a headset? Like a combo “speaker” and mic?
That hasn't been tested yet to my knowledge
i dont think the pi can do it
Oh ok
Might be a problem then, I’m gonna need to fine a way to get a speaker and a mic
Usb mic and aux speaker
Yeah I just don’t have either, I’ll have to go pick some up
god this wouldve made me trying to make it so much simpler
im still probs gonna develop my own for fun but this is cool
It'll have very little functionality unless you know a lot about programming
hmm
its just gonna let me be able to talk to chatgpt
or actually my own model
You can make it do what ever you want if you know enough python
yup
<@625486818321760256> can you make it interface with your pc? Open and close apps and stuff like like?
You could if you wanted to but let's not try and fly before we can walk.
Yeah for sure just curious
Get it set up yet?
can you rotate the camera 90 degrees when taking the photo ?
For what purpose, that is simply the login shell banner
he likes landscap cropped photos, even better if its simply a screenshot
That question can only be answered with, why photo a landscape with the camera in portrait
I’m trying now
It had an error being burnt to the sd card
What was the error
We need logs or a screenshot or something
I’m out for the night, but I will send a screenshot whenever I’m home
ok...
oh actually. you connect it to the internet either through the actual desktop/commands if you have a monitor or you would put the WiFi details in the imager usually
Just because if the wife provider I use I have to register each device with the MAC address before it’ll connect to the wifi
oh weird
do you have a display?
Type the command ip a and then the MAC is the hex code after link/ether
I thought I told you to do that
I normally use the imager
yeah
me too
I mean you need to use headless which is what I use
Exactly