Speed dating: LLM edition
It's worth exploring the capabilities and personalities of different LLMs by asking them to answer the same questions.
In the last post I talked about the benefits of a good prompt; in this post, I want to discuss another easy way to get the most out of LLMs - by going on a speed-dating round!
I gave GPT-4o and Gemini 2.5 Pro the same prompt as I gave to Claude Sonnet 4.0 about creating a 4-week nature photography course:
I'm new to photography and I have a Sony a6400 camera. I want to learn about nature photography. Please put together a 4-week course with a mix of theory and practical exercises to help me learn how to use the camera and build my skills.
First, this is week 1 from GPT-4o:
šæ 4-Week Nature Photography Course (Beginner-Friendly)###
šø Week 1: Know Your Camera + Fundamentals of Exposure
Goal: Get comfortable using your a6400 and learn the exposure triangle.
š Theory
- Camera basics: modes (Auto, A, S, M, P), buttons, menus.
- What is exposure? Learn ISO, aperture, and shutter speed.
- Focus modes: AF-S, AF-C, Manual Focus.
- File types: JPEG vs RAW (start with JPEG + RAW).
š§Ŗ Practice- Set to Auto Mode and take photos in your backyard or local park.
- Then switch to Aperture Priority (A) and adjust f-stop (e.g. f/2.8 vs f/11).
- Observe how changing aperture affects depth of field.
- Try different ISO settings (100, 400, 1600) in the same scene.
ā Weekly Challenge
Take 5 photos of the same subject using different aperture and ISO settings. Label them and review differences.
If you're interested, there's the full response:
And this is (part of) week 1 from Gemini 2.5 Pro:
Week 1: Understanding Your Camera & The Power of Light
Goal: Get comfortable with your camera's basic settings and how light impacts your photos.
Theory: The Exposure Triangle
- Aperture (f-stop): Crucial for controlling depth of field. For nature, a large f-number (e.g., f/8, f/11, f/16) is often used for landscapes to keep everything sharp, while a small f-number (e.g., f/2.8, f/4) can isolate a subject like a flower with a blurry background.
- Shutter Speed: Essential for freezing motion (e.g., birds in flight, splashing water) or creating motion blur (e.g., silky waterfalls).
- ISO: Keep it as low as possible (e.g., 100-400) for cleaner images, especially in nature where detail is key. Only increase when necessary in low light.
Practical Exercises
Exercise 1.1: Aperture Priority (Av/A Mode):
- Set your camera to Aperture Priority (A) mode.
- Find a flower, a tree, or a single natural object.
- Take photos at various apertures:
- Smallest f-number (e.g., f/2.8 or f/3.5, depending on your lens) ā observe the blurry background.
- Mid-range f-number (e.g., f/8) ā notice more of the background coming into focus.
- Largest f-number (e.g., f/16 or f/22) ā see how much of the scene is sharp.
Review the photos on your LCD. Pay attention to depth of field.
Finally, there's the full response from Claude Sonnet 4.0.
They all had the same prompt, and on a whole, their responses are not too dissimilar, but you can quickly spot the differences between the answers from the three LLMs.
GPT-4o caters for complete newcomers and the learning is at a very gentle pace.
Claude Sonnet 4.0 covers a lot more ground content-wise.
Gemini 2.5 Pro also covers a wider scope of content each week - but unlike Claude, which mentions the concepts with no explanation, it provides easy to understand explanation with references to nature photography.
Each LLM also has a different āvoiceā. Both GPT and Claude are brief and to the point, while Gemini is more conversational. Claude uniquely provided a handy quick reference guide to the camera, as well as recommended accessories and safety and ethical considerations.
I donāt know which course appeals to you the most, but for me, Gemini wins hands down in this speed-dating round. The content is engaging, and the practical exercises are well structured and link nicely back to the theories. Iām actually tempted to follow the course over the coming weeks to hone my photography skills.
Here are the results of another LLM speed-dating round.
The prompt:
Create a 15-minute speech with graphics for high school students to introduce key concepts of AI and how it works, focus on education and engagement.
The responses:
The differences in the content generated is even greater in this round. Claude actually created a presentation deck (thus meeting the graphics request)!
In terms of quality of the content, there are useful facts from each, but Gemini wins again for me overall - not only is the information relevant, itās also a full speech that I can quickly edit and use.
I hope these two examples gave you a taste of the unique abilities and āpersonalitiesā of each of these LLMs.
If you agreed with my assessment and preferred the responses from Gemini 2.5 Pro, does it mean it is the LLM you should use going forward?
No.
Unlike real life dating - you donāt have to settle with one partner.
Each LLM has their own strengths, and with a bit of experimentation, you may find that one LLM is superior in Task X while you prefer another LLM for Task Y.
Also, the LLMs are regularly fine-tuned by the AI labs, which means the way the LLMs respond to your prompts will change over time. So itās wise to test them out regularly, even if you have a preferred default model.
And ā¦ sometimes you just want a short answer!
Why donāt you ā¦
Come up with a few prompts (e.g. history about a famous person/period of time, plot for a short story, or a recipe suggestion based on what you have in your kitchen) and test out several LLMs.
Read the responses and see what you like from each, and how they differ from each other.
Iām keen to learn what you ask, and which LLM provides the most useful/interesting answers.
Programming note
My goal is to post weekly, and if I have the time, twice weekly. Tentatively, the regular posts will be sent on Friday, and bonus posts (like this one) will be sent on Tuesday.
Until Friday!
Vee