ChatGPT is an artificial intelligence chatbot from OpenAI that has surprised many of us with its abilities. It can provide information on a wide variety of topics, solve complex math problems, and even write comedy sketches and songs. It does all this quickly, efficiently, and in an adaptable, conversational way.
The capabilities of ChatGPT are impressive and the technology is fascinating. But if one has a job in writing, these capabilities may be more frightening than interesting. This is because writers may fear that their jobs will be replaced by AI given the knowledge and efficiency of ChatGPT and other chatbots. Is this fear likely to become reality?
The short answer to this question is no, at least not yet. After examining the strengths and weaknesses of ChatGPT and testing its abilities in different tasks writers often have, ChatGPT does not seem to pose an immediate threat to writing jobs. In fact, ChatGPT may be a helpful tool for writers instead of serving as their replacement.
ChatGPT Strengths and Weaknesses
ChatGPT’s strengths and weaknesses become apparent when viewing how OpenAI’s chatbots performed on different exams they were given. The exam results from these chatbots can be found in this article from OpenAI.
The article shows results from both GPT-4 and GPT-3.5. GPT-4 shows improvement over GPT-3.5 in many areas. It is worth mentioning that according to this introduction to ChatGPT from OpenAI, ChatGPT is from GPT-3.5. Given this, it is safe to assume that the capabilities of ChatGPT are lesser than those of GPT-4. So, a proper evaluation of ChatGPT’s strengths and weaknesses must keep in mind that ChatGPT does not represent the best model OpenAI has created.
Exam scores shown in the article from OpenAI reveal some impressive scores. GPT-4, the strongest version, scored in the 89th percentile on the math SAT, the 88th percentile on the LSAT, and between the 71st and 88th percentile on an AP chemistry exam. However, exam scores seem to reveal a weakness of GPT-4. It scored between the 14th and 44th percentile on an AP English Language and Composition exam and between the 8th and 22nd percentile on an AP English Literature and Composition exam.
Why did GPT-4 not perform nearly as well on English exams? In an article from Bernard Marr on Forbes, he discusses shortcomings of ChatGPT, some of which explain the low scores on English exams. For example, Marr claims ChatGPT has trouble understanding context and generating long form content. These are among the most important skills for excelling in English exams, because these exams often involve answering questions about metaphor, imagery, intention, and other things that require a great understanding of context and subtlety.
If you ask ChatGPT itself about its strengths and weaknesses in writing, it will tell you something similar. This was ChatGPT’s concluding remarks when I asked it about its strengths and weaknesses in writing:
Overall, while AI models like me can generate high-quality and coherent text, human writers bring a unique blend of creativity, emotional intelligence, and subjective judgment to their work. It’s important to recognize that the strengths of AI lie in its speed, efficiency, and access to vast amounts of information, while human writers excel in their creativity, originality, and nuanced understanding of context.
ChatGPT
Recently, a friend and I were discussing ChatGPT. She compared the output of ChatGPT to the output of an online translation tool. This, I think, is a great metaphor for describing ChatGPT’s writing capabilities. Translation tools are helpful because they know the definition of words and they mostly get things right. But they miss nuances in the language that only a person who commonly speaks it would understand. Similarly, ChatGPT has knowledge of many things, but it lacks a certain nuance that only experience can produce.
ChatGPT in Writing Jobs
Since ChatGPT struggles with nuance, context, and long form content, it is safe to assume that ChatGPT’s success in a writing role depends on how important these things are to the role. If a writing job rarely calls for these things, then ChatGPT could perform the job well.
But for content that needs strength where ChatGPT has weakness, it is better to have a human writer. As it turns out, a lot of content falls into this category.
Trying ChatGPT in Writing Tasks
To see ChatGPT’s strengths and weaknesses in action, I have tested it on a few pieces of content. The different types of content I tested it on were a blog post, a social media caption, and a press release.
I asked ChatGPT to write a 250-word blog post about the benefits of being outdoors. In response, it generated a quality blog post with six strong and mostly familiar reasons for why people should spend time in nature.
The blog post contained quality information, but it did not bring anything new or unique to the conversation. A human writer tasked with writing about the same topic may have included personal stories about spending time in nature with their friends and family or a particular outdoor activity they really enjoy. ChatGPT did not offer content like this. But since it does not have personal experience to draw from, it should not be expected to offer such content.
Testing ChatGPT further, I asked it to write an Instagram caption for a post in which I am promoting an instrumental piece of music I just finished. It created the following caption for my hypothetical instrumental track:
Feeling the rhythm, creating the melody. 🎵 Excited to share this instrumental track I just finished! 🎶✨ Pouring my heart and soul into the music. Can’t wait for you to listen and groove along. 🔥💫 #NewMusic #InstrumentalVibes #MusicIsMyPassion
ChatGPT
This is a decent caption, but it is not really my style. Also, the caption is not personalized, and it does not share anything specific about my track. These are problems I have with the caption, but they are not necessarily criticisms of ChatGPT. It would be unrealistic to expect ChatGPT to know my style or specific details about the music I am releasing without me sharing such information with it. But, in the time it would take to effectively share that information with it, I could generate the caption more efficiently by just writing it myself.
This problem of inefficiency becomes more apparent when considering press releases. Press releases often include information that is not yet published on the internet. This information could be quotes from people within the organization, news about an upcoming event or product, or any information exclusive to those within the organization.
I tried to have ChatGPT write a press release promoting a hypothetical event. What ChatGPT generated in response was a press release full of flattering adjectives about the event, but without much detail of what would happen at it. This was obviously because I did not provide ChatGPT with enough details to write an effective press release. But in the time it would take to provide those details, I could write the press release on my own.
Given this, it does not make much sense to have ChatGPT replace people in writing roles. But this does not mean ChatGPT has no place in the writing process. In cases where ChatGPT does not have necessary information or personal perspective is needed, ChatGPT can still be used to offer suggestions to an already existing piece of writing.
Where ChatGPT Can Help
ChatGPT may not have the knowledge or perspective to write content by itself. However, it does possess a great deal of knowledge about the best practices for writing in general. Meaning, once a person creates content that is unique, ChatGPT can review the content and help improve it based off this general knowledge.
For example, when creating a social media post to promote this blog, I began with this caption in mind:
“Is ChatGPT going to replace writing jobs? In this blog, I explore ChatGPT’s writing strengths and weaknesses before discussing how this impacts writing jobs.”
Then, I asked ChatGPT how I can make this caption more appealing to new readers. It suggested I add more “curiosity, excitement, and personal engagement.” Following that, it generated this caption:
🖋️ Can Chat GPT replace writing jobs? Discover the astonishing truth in this captivating blog! ✨✍️ Dive into Chat GPT’s remarkable writing strengths and weaknesses, and join me as we unravel the fascinating implications for the future of writing careers. Let’s embark on an eye-opening journey together! 🚀💡
ChatGPT
This caption is a little heavy on emojis and strong adjectives for my taste. But I do like the basic direction and enthusiasm ChatGPT is recommending. A caption that is somewhere between the enthusiasm of my original caption and the enthusiasm of ChatGPT’s suggestion should work perfectly.
This is the way I plan on approaching AI in my writing for now. I will create content on my own before turning to ChatGPT for suggestions. Then, I can determine whether I think the suggestions are right for me. ChatGPT may not always give great advice. Still, ChatGPT and other AI chatbots have a lot of information at their disposal, and it is easy to ask for their insight.
Are Writing Jobs Safe?
For now, writing jobs should be safe from total AI replacement because a lot of content requires unique perspective, understanding of context, and knowledge of new information. Fortunately for writers, people outperform AI in these areas as of now.
Also, it is worth briefly addressing the fact that AI makes mistakes. According to ChatGPT’s introductory page, ChatGPT sometimes produces errors and nonsense. So, people may still need to oversee AI generated content to make sure it does not make these mistakes. This oversight would be needed even if the content did not involve skills in which humans outperform AI.
There may come a time when AI understands context at a much higher level, is better at processing new information, and almost never makes mistakes. Still, AI cannot have access to information that has not already been given to it or published on the internet. Luckily for writers, it is often the case that content creation is the process of sharing exactly that. Most importantly, it is hard to imagine AI having the type of unique perspective that humans do any time soon. These areas where people outperform AI should keep writing jobs secure for people in the foreseeable future.