To evaluate the extent to which LLMs have developed the ability to actually reason through a problem rather than doing fuzzy pattern matching on existing answers, I decided to test some leading chat models with the following modified version of the Monty Hall problem
ChatGPT fails a modified Monty Hall problem
ChatGPT fails a modified Monty Hall problem
ChatGPT fails a modified Monty Hall problem
To evaluate the extent to which LLMs have developed the ability to actually reason through a problem rather than doing fuzzy pattern matching on existing answers, I decided to test some leading chat models with the following modified version of the Monty Hall problem