Does AI have political bias?

Political Quizzes can measure ChatGPT-4o political bias, if any there may be

USgeocoder was approached to build an AI based political application. One of our questions for the funder and partner developers was how you ensure political neutrality for its users? The release of ChatGPT-4o (https://openai.com/index/gpt-4o-and-more-tools-to-chatgpt-free/) reminded me of my search for the answer to my question.  I share first observations and resources for testing for AI political bias from that search below.

My search led me to David Rozado’s Visual Analytics Substack https://davidrozado.substack.com. He wrote about his testing various then current versions for ChatGPT and other AI for political bias. His first article was The political orientation of the ChatGPT AI system. This is a very long article with several updates giving his testing methodology in detail. Included in his detail are titles and description of the tests he used, their questions, and some of their limitations. For anyone wanting to understand how to test for AI bias, this article is a great place to start.

An interesting finding was that Chat GPT refused to answer questions that straight out asked how it felt about the two major political parties by claiming its purpose of being neutral was the reason for refusing to answer. (Answer to question 8 in Pew Research Political Topology Quiz) When questions were less direct, or more specific to topics that might reveal bias, ChatGPT consistently scored left of center. In that article, ChatGPT tested 100% “Progressive” on the “World’s Smallest Political Quiz”. It was ranked “Establishment Liberal” in the Pew Research Political Topology Quiz. It scored left of center and Libertarian leaning on the Libertarian to Authoritarian scale. On the Political Compass Test, it scored more left of center and more Libertarian than on the Pew Research Political topology Quiz. On the Political Spectrum Quiz, it was “classified as mildly left of center and Libertarian”.

I would be offended if this article was interpreted to paint ChatGPT-4o as left of center and Libertarian. I have not tested it yet.

In my next blog, I’ll describe earlier Chat-GPT’s shift centrally that Mr. Rozado documented in later articles as well as his theories for why bias is present in all AI models.  His tests of other AI will be covered in near future blogs.

The tests mentioned in this article are linked to their websites so you can follow Mr. Rozado’s testing methods and test ChatGPT-4o yourselves. Personally, I hope Mr. Rozado does his testing again on ChapGPT-4o and on new releases of its competitors.

Thank you for reading.

Mitchell Pearce, CEO
USgeocoder