In this test we will be comparing the performance of OpenAI’s ChatGPT to Anthropic’s Claude. Both are LLM based AI models designed to offer virtual assistant type services.
Update 2024-08-09 I find I’m using Claude more and more for complex coding tasks. It seems to have access to more up to date data and I’m not getting issues with it using outdated libraries as much. It also seems more reliable at taking on complex tasks which require reasoning. Until ChatGPT5 is released Claude is the more capable LLM for purely text/coding based tasks at this time.
There are free and paid services which are both around the $20/month range.
At time of writing ChatGPT is using the 4o model and Calude is at v3.5 Sonnet. I’m writing this after having used ChatGPT for some time and only recently started using Claude so there may be some biases.
The first difference you’ll notice is that Claude is somewhat limited in it’s capabilities. It can’t connect out to external sites to collect up to date information, it can’t create images and it can’t execute code it has written.
These differences might not be a deal breaker if you are using Claude for something specific that doesn’t require them but if you are used to ChatGPT you may find it somewhat limiting.
For example I often ask ChatGPT to create an illustration for a piece of text I’ve just written…
So why would you use Claude? In certain benchmark studies Claude does outperform (according to their own blog)
You also get a different personality virtual assistant and it’s the first time I’ve experienced this. You get AI without the ChatGPT feel and familiar phrasing, it feels just slightly different and perhaps more conversational, almost more human.
ChatGPT results tend to be very generalized and sometimes vague and there is a noticable human element to Claude. It uses pronouns when referring to itself like “I was created by Anthropic”, where as ChatGPT might refer to itself in the 3rd person with something more like “ChatGPT is an LLM created by OpenAI”. Not the best example but you get the idea that, in my opinion, Claude has a delivers a more personal humanesque experience. I wonder if this has been added by Anthropic as a feature or if it was deliberately reduced by OpenAI to avoid liability.
The models are both very capable and will handle tasks like editing, proof reading, simple coding etc. I think Claude is at a similar level to ChatGPT currently and there’s not a significant difference in their intelligence levels that you can tell as an everyday user.
It will be interesting to see where OpenAI goes next as GPT4 was released in March 2023 and they must have been working on something during that time. The minor updates like GPT4o seem to be not worthy of the v5 title so I’d expect a major update in the next 12 months which could push the boundaries further.
