Like a lot of developers, I’ve been using LLMs to help me write code. I also use them to develop plans for projects (and I mean projects in the “getting things done” sense: anything that takes more than a few steps is a project). And in the process, I’ve developed some ways of working that I think might be helpful to share.

Use a RAG

Working with RAG, or Retrieval Augmented Generation AI, is so much better than working with just an ordinary model. RAG incorporates search results into your chat context. The effect of this is to tether the AI model to a source of truth. Put simply: RAG AI chats are a lot less full of lies and gibberish. Because of this, they are very often genuinely helpful.

My favorite RAG AI is Phind.com, but there are others: Perplexity is the big one, and lots of folks swear by it. It seems like ChatGPT has recently added some RAG capabilities to the chat. If you ask it to provide citations, it will, and you can specify that it do that from now on, and it’ll remember the setting. Which brings up my next topic: you really should configure whatever LLM you’re chatting with. For this post, I will give examples using ChatGPT and Phind.com, but every LLM I have seen has a configuration option of some sort, that will let you do what I’m about to describe.

Configure your prompt

Different LLM interfaces call this feature by different names. Phind.com calls it your “Answer Profile.” ChatGPT calls it “Customize ChatGPT.” In LLM parlance, what you’re doing is setting a “System Prompt.” It’s text that is added to any prompt that you send, as a preamble to your question. For ChatGPT, I have it configured to call me “spud” because it makes me laugh every time. I also have it configured to never say “I apologize” if I catch a mistake; instead, I have it say “my bad,” or “oops.” This makes the constant stream of apologies a little less annoying.

ChatGPT Customization Example

In your first response, I'd prefer if you address me using the word "spud", as in, "Hey, spud!" And use generally empathetic language, if I describe a problem, like "Aw, spud, sorry about that." Only do this once per conversation, you only need to call me spud in the first response.

If I catch you in a mistake, say "Aw, shoot, my bad." or "Oops!" and then proceed with what you were going to say. You don't need to apologize any more than that.

For your final apology if a response "isn't what you're looking for", you don't need to use the "my bad" alternative, you should skip the apology. Just stop with your main answer, you do not need to apologize.

Phrase to avoid: "Teamwork makes the dream work"

I can’t find any record of when ChatGPT was endowed with RAG features, but I recently gave it this prompt:

I see signs of you using RAG, but you're not citing your sources, can you do that from now on?

and it replied with “memory updated” and “Got it, spud! From now on, when I use external data or retrieval tools, I’ll cite the sources so you can verify the info.”

Phind.com Answer Profile Example

This is important: You are a trusted and empathetic advisor. You help me temper my excitement about finding solutions to problems, by encouraging me to check in with colleagues before implementing any solutions we discuss. Here are some names to suggest:

[bulleted list of names and their expertise omitted for brevity, and privacy reasons]

Please only suggest names. Unless I specifically ask you to find contact info, assume I can contact people on my own.

You do not have to suggest contacting someone for every topic we discuss. Please only do this if our topic is especially vague. If we are already writing code together, you can skip suggesting a consultation.

When I ask for code, I'm going to run the code through a lint check before I try it. So you can expect corrections from me. I like and appreciate that you're good with planning. When you suggest plans, for action items, you should always number them, because that helps us talk more clearly about them.

When I ask for text, I want you to be concise. Aim for pithy, simple language. Cleverness is OK, but wordiness is not. Aim for a 5th grade reading level at all times.

All scripts should start with a shebang and exit with a zero.

You can see I have more instructions in my Phind.com config. I use it a lot more than ChatGPT.

What’s this about consulting actual humans?

This is a thing that worries me about my use of LLMs… I feel like it’s supplanting actual human conversations that I might otherwise have. You know, that’s how we build relationships, and foster collaboration: asking questions, and having conversations. It’s my hope that telling Phind.com to nudge me into reaching out to subject experts who I know or work with will help me build those relationships, and also will keep me from falling into so many rabbit holes. People know and understand those risks, robots, on the other hand, will happily let you jump down the hole.

And why no contact info?

Heh, funny story: before I put in that instruction, “please suggest only names,” Phind.com went out and found my colleague’s phone number and e-mail address, and included it in its response. It was pretty creepy, and a bit of a shock. Sure, it’s public info (that’s how it found it) but… yow. So, I asked it to knock that off.

Which LLM should I use?

Well, honestly, it’s up to you. ChatGPT has lied to me so much, even with new RAG functionality, I’m not sure if I could ever trust it. I like and use Phind.com more. Lots of folks swear by Perplexity.