Can AI Count to 4? LLM Benchmark with ChatGPT, Claude, Gemini and Ollama

Published 19/06/2025

Description

Can AI count to 4? In this LLM benchmark, I test local and cloud AI models on a simple task: filtering a list of names to keep only the ones with exactly 4 characters. In this video, I compare Ollama-hosted models and leading online AI systems to see which ones can handle a basic prompt reliably, which ones miscount, and which ones overcomplicate a very simple reasoning task. 0:00 - Introduction 0:10 - Description 2:56 - Ollama - Gemma 3 4:16 - Ollama - Llama 3.2 5:56 - Ollama - Exaone 7:38 - Ollama - Qwen 2.5 VL 8:49 - Ollama - Qwen 3 10:04 - Ollama - Phi-4 Reasoning 11:17 - Ollama - Phi-4 Mini Reasoning 12:33 - Online - Gemini 15:41 - Online - Kimi 17:47 - Online - Claude 19:57 - Online - ChatGPT 21:43 - Online - DeepSeek 23:38 - Conclusion I use a deceptively simple name-length test to compare how different models handle counting, filtering, prompt following, and basic reasoning. The video includes local models running through Ollama, cloud AI platforms, a baseline code approach, and a practical look at how easily LLMs can fail on tasks that seem trivial on paper. If you are interested in AI evaluation, prompt engineering, local LLMs, and real-world model behavior, this video gives you a quick benchmark that is easy to understand and surprisingly revealing. #LLMBenchmark #PromptEngineering #Ollama Follow me on X: https://l.samayas.eu/jo3eaYEf3E?P=YT&V=13 Buy me a Coffee: https://l.samayas.eu/jX6xL7EWhd?P=YT&V=13 Studio Equipment Camera Sony Alpha ZV-E10 II: https://l.samayas.eu/k7y90TREl0?P=YT&V=13 Secondary Camera Obsbot Tail Air: https://l.samayas.eu/iFQu9WfmCw?P=YT&V=13 Elgato Prompter: https://l.samayas.eu/fHVrqqXtSc?P=YT&V=13 Microphone Rode Wireless GO II: https://l.samayas.eu/afBINjiBXN?P=YT&V=13