It's been a while since I last posted on AI, LLMs, and chat bots. Here's what I've been up to. I use Gemini to help me write business letters, and at work I use ChatGPT as a SQL and AMPScript code assist tool. ChatGPT is decent for helping a non software developer learn the quirks, in's, and out's of programming languages and scripts. It has not been smooth sailing. Each day I use ChatGPT with version history enabled, it seems to forget what it just said in its last response to a question or prompt and automatically starts correcting itself if you provide it with its own SQL query to make updates to.
The hallucinations are by far the worst bug in the LLM. ChatGPT proactively adds field names and table aliases that don't exist in the original query or prompt. It also uses SQL commands that aren't available in SFMC's version of SQL. And, after I have already specified that the query needs to work with SFMC's SQL (based on T-SQL), it gives me SQL that I already know can't run in SFMC. I have to remind it each time it does this with an additional prompt that the SQL can't start with "with" before "select".
Anyhow, even with its flaws, dev time has been greatly reduced from a few days to a few hours in most cases.
The prompt you give it is only as good as your understanding of how a tool or proces works. Using an LLM is a lot like using a mobile device app's GPS. Sure, the GPS gets you to your location with turn by turn directions, but it doesn't teach you how to navigate a map by yourself. How many times have you heard people say, oh I don't know where [location] is, I just ask the maps app to get me there?
It won't help you think critically or write professionally unless you ask it how it analyses text or data before giving you a response. Even then, it seems bound by programmed how to.
