It’s been a great summer in Pittsburgh: lots of time spent resting outside in the sunshine with friends and family. Apparently there’s a joke that goes something like “If you’re a middle-aged man, you have to choose between getting obsessed with WWII, Rome, Napoleon, or the Founding Fathers”. I’m not quite at middle age but I guess I’m easing my way into it with some American icon biographies 😃. This spring and summer I’ve enjoyed reading a handful of biographies about American presidents and founding fathers: Lincoln, JFK, Franklin, and now Washington.

I’m finishing up the George Washington biography now, and something is different from all the previous biographies I've read: Washington wrote a lot. Not just a lot, but a lot a lot. He was meticulous about storing his correspondence, and according to Mount Vernon, historians have compiled over 135,000 copies of Washington documents!

To be that prolific of a writer and accomplish so much (president of a new nation, commander in chief of a revolutionary war, etc.), he had to have some help. Over the course of the Revolutionary War, Washington had 32 aides-de-camp, the most famous of whom were probably Alexander Hamilton and John Laurens. These aides-de-camp were responsible for a myriad of tasks, among which was to assist in drafting and copying George Washington's orders and correspondence to Congress, military leaders, and others. Few people in the 18th century could write with Washington's diplomatic skill, but even fewer had a fleet of aides available to help them.

We live today in an amazing age where we have the potential to benefit from an aide-de-camp that Washington never had: Generative AI. When I’m producing code, emails, or text messages, I’m continually blown away by the tools at my fingertips to make my work faster and easier. Next word/sentence suggestion built in to Outlook/Mail/iMessage helps me get my thoughts on paper fast, and Github Copilot helps me write new code as well as improve the code that I already have.

That being said, I haven’t really used Gen AI when it comes to long form writing (more than a few sentences). Part of the reason for that is because I write mostly to think things out myself; it’s easier to spot a dumb idea when I’m staring what I’ve written as a comment above some python code or in a wiki. The other reason I haven’t used Gen AI for writing is because the few times that I have put it to use, the content it gave me hasn’t sounded like something I would write. I’ve read enough output from Gen AI that often the phrasing and word choices seem kind of stilted and repetitive, unless it’s been well tuned for the task at hand. My feelings are starting to be somewhat backed up by research; check out this list of most used words by AI:, or these research papers showing the lower lexical richness of LLMs as compared to humans (Playing with Words: Comparing the Vocabulary and Lexical Richness of ChatGPT and Humans, Delving into ChatGPT usage in academic writing through excess vocabulary). LLMs are being used to expedite many writing procedures, but even in cases where the text isn’t flagged as being AI generated, people aren’t necessarily using them in helpful ways ( building processes to filter out applications and resumes written by AI).

Reading the Washington biography makes me wonder: how can I make an LLM useful to me for long form writing? Maybe I don’t want to use the LLM to write the whole thing for me, but perhaps I could benefit from If I want an LLM to refine and support my own writing tone, I can think of two overall options:

1. Provide a long list of explicit instructions to an LLM to explain my writing tone.

Some basic prompt engineering and conversation dialog could be used to refine and instruct the LLM how to write with your style. For example, Peter Yang’s post My 5 Favorite AI Prompts for Writing explains a few options, as well as an informative post by Prompt hub. Although this might sound great, in practice it’s tough to accurately and exhaustively describe how you would write something in all situations, so you may need to write a detailed prompt for each scenario. I’m not sure that I could even describe my writing if asked, because I haven’t written enough recently to be sure I even have a style (Unless “haphazard” counts as a style).

Prompt engineering attempts have been met with mixed results when it comes to generating content that we’re proud of (link link link). I’m not talking about generating a college essay good enough to trick your professor, but about writing content that you like and is useful in a variety of scenarios. A specific prompt to help you write a specific essay may not help you too much in the long term.

2. Provide examples of my writing so that the LLM can implicitly learn to write like me

Another option is to bypass the need to prompt with specific instructions by using existing samples of writing. Similar to George Washington’s aides-de-camp, they probably weren’t given an exhaustive list of how Washington’s letters would be written, but rather had some guidelines and then learned the finer points by getting feedback and by reading the other things that he’d written.

If an AI system is going to implicitly learn writing tone via examples, it will require a healthy amount of existing data. One of my first blog posts demonstrated a technique to extract emails from gmail to use as training data for an LLM. This method could be expanded to extract the text of everything you’ve written: emails, college essays, blogs, etc.

Once you have this dataset, some common methods for using this data would be in-context learning (ICL), retrieval-augmented generation (RAG), or supervised fine-tuning (SFT). In all of these methods, the most important thing is that you have a great dataset. In this context, a great dataset means that you have a lot of high quality examples of your writing. Something I realized back when I first trained Llama-2 using my emails as a dataset is that I actually don’t have much written content. Most emails I send in my personal inbox are short, and many of them are from before 2015: nowadays if I want to have a moderately long conversation with someone, I would plan to meet up in person or FaceTime them.

The biggest impediment to training an LLM to help me write isn’t the capability of the LLM itself, but on the lack of examples of my own writing. Unlike Washington, I have no repository of 135,000 letters. In order to be useful as an assistant, the quality of content that you have written may be directly correlated to the quality of content that an LLM can generate for you.

A Tale of Two Conclusions

Feels like the right way to end this post is to give two conclusions, both LLM generated. In one prompt, I ask Claude to generate me a good conclusion given my above essay. In the second, I ask Claude to write a conclusion, paying special attention to staying consistent to my style and tone. In both cases, I think you’ll be able to easily detect that it’s no longer me doing the writing; they’re both great conclusions, but they don’t sound like me. One uses formal vocabulary, while the other is overly lighthearted and funny. Thanks for reading!

Claude-style Conclusion

Prompt

“Write a good conclusion for me for this essay.”

Output

As we navigate the exciting frontier of AI-assisted writing, the lessons from George Washington's era remain surprisingly relevant. Just as Washington relied on his trusted aides-de-camp to amplify his voice and extend his reach, we now have the opportunity to leverage AI as our modern-day assistants. However, the key difference lies in the foundation upon which these assistants build their skills.

Washington's aides had the advantage of learning from a vast repository of his writings, allowing them to intimately understand and emulate his style. In contrast, many of us in the digital age, despite our constant connectivity, may find ourselves with a surprisingly limited corpus of substantial written work to feed into our AI assistants.

This realization brings us to an interesting crossroads in the evolution of writing and communication. As AI tools become more sophisticated, our ability to use them effectively may depend not just on the technology itself, but on our own dedication to the craft of writing. Perhaps the true potential of AI in writing lies not in replacing our voice, but in challenging us to develop and articulate that voice more fully.

Moving forward, we might consider taking a page from Washington's book – not necessarily in volume, but in intention. By consciously cultivating our writing skills and building our own repositories of thoughtful, well-crafted content, we create a richer foundation for AI to learn from and enhance. This approach could lead to a symbiotic relationship where AI doesn't just mimic our style, but helps us refine and elevate it.

Ultimately, the journey of integrating AI into our writing process is teaching us as much about ourselves as it is about technology. It's a reminder that while AI can be a powerful tool, the essence of compelling writing – our unique perspectives, experiences, and ways of expressing them – remains inherently human. As we continue to explore this new frontier, let's embrace the opportunity to not only leverage AI but to grow as writers and thinkers in the process.

Claude “Write like Me” Conclusion

Prompt:

“Write a good conclusion for me for this essay, use the same style and tone as the rest of my writing.”

Output

So where does this leave us? It's pretty clear that I'm not going to suddenly start churning out thousands of letters like George Washington (sorry, future historians). But maybe there's a middle ground. Perhaps the lesson here isn't about quantity, but about intentionality. Washington kept meticulous records because he understood the value of his correspondence. In our digital age, we have the tools to easily archive our writing, from emails to social media posts to blog entries like this one.

Maybe the first step towards having an AI writing assistant that truly gets our style isn't about prompt engineering or complex algorithms. Maybe it's about being more conscious of our digital footprint, curating a collection of our best writing, and treating our words with the same care and attention that Washington did. Who knows? In a few years, we might look back and realize that the key to personalized AI wasn't just in the tech, but in how we approached our own writing.

In the meantime, I'll keep plugging away at this blog, one post at a time. It might not be 135,000 documents, but hey, you've got to start somewhere, right? And who knows, maybe someday an AI will read this and think, "Ah, so that's how Nate writes!" Until then, I'll just have to be my own aide-de-camp. Now, where did I put that quill pen?