I need help with a Gemini prompt

[Backspace Studios]

I broke my hand yesterday and I'm trying to make an accessibility, hands free code editor. Basically, it will take speech input (speech to text) and pass that text to Gemini, which will infer what I'm trying to write, add the characters to a list and then, using a for loop, send each key with the Keyboard library. The problem is, it keeps getting the syntax wrong and getting case sensitive stuff wrong. I've been trying for 7 hours to get some good sys instructions, but I just can't seem to. I even asked ChatGPT to make some better instructions, but they were worse than the ones I made. Does anyone have any experience doing prompt engineering or just have a better idea for the instructions? Any ideas would be a massive help! Thanks!

My code:

import keyboard
import ast
import re
from google import genai
from google.genai import types
import time

code = []


def query_ai(query):
client = genai.Client(
api_key="API_KEY",
)

model = "gemini-1.5-flash-8b"
contents = [
types.Content(
role="user",
parts=[types.Part.from_text(text=query)],
)
]
generate_content_config = types.GenerateContentConfig(
temperature=0,
response_mime_type="text/plain",
system_instruction=[
types.Part.from_text(text="""Only output a Python list of characters representing the full code I ask for, formatted like ['d','e','f',' ',...]; never include explanations, code fences, "python" labels, or anything outside the list—just the list and nothing else. If I give a command in natural language, fix typos (e.g. change define to def), but don't correct the code apart from typos. (Infer what I want you to do) Remember to double check that you did get every character right in the list and didn't miss any. Unless I make a typo or code mistake my command, make the code 100% correct (e.g. don't do "def define example_function". You would do "def example_function") Make 100% sure you correctly put special character or new line commands (backslash n) In the list don't put square brackets at the start or end. Don't change names of things (e.g. If it say "type define example open bracket close bracket colon", your response should be "def example():", not "def example_function():") If I tell say "/REFRESH_MEM, answer with "/REFRESH" (nothing else). If I send you any code with "/REMEM" before it, that it the current sate of the code. Automatically fix case sensitive stuff and misspelled syntax and library name (e.g. "If I say import custom T K I N T E R", you would change it to "import customtkinter" If I have accidentally added a space, correct it to the right syntax based on context (e.g. If I say "root dot main loop open bracket close bracket", you should say "root.mainloop(), not "root.main_loop(). Every time you correct something, make sure that the correction would make the syntax correct based on context. Search the web for the correct syntax if needed. Keep in mind, you are always being spoken to through text to speech.""")
]
)

# Accumulate all text chunks from the stream
response_text = ""
for chunk in client.models.generate_content_stream(
model=model,
contents=contents,
config=generate_content_config,
):
response_text += chunk.text

# Remove python and if present
cleaned = re.sub(r"^(?:python)?\s*|\s*$", "", response_text.strip())

# Ensure brackets so it can be parsed as a list
if not cleaned.startswith("["):
cleaned = "[" + cleaned
if not cleaned.endswith("]"):
cleaned += "]"


try:
return ast.literal_eval(cleaned)
except SyntaxError:
return "#invalid query"
if name == "main":
while True:
code = query_ai(input("prompt: "))
if str(code).replace("[", "").replace("]", "").replace("'", "").replace(",", "").replace(" ", "").strip() == "/REFRESH":
print("initiate refresh sequence")
for char in code:
keyboard.write(char)
print(char)
time.sleep(0.2)