Before we dive into this week’s topic, on behalf of Aaron, myself and our CIM family I want to wish everyone a Merry Christmas! As the season gets busy, both at work and at home, my prayer is that you’ll find moments to pause, breathe deeply, and reflect on the wonder of “God with us”, not just in history, but also in the hope of Christ’s return.
Along with all the preparations, December typically sees a spike in not only church attendance, but also in faith-related search queries. Last week I was reading the full report of the “AI Christian Benchmark” by The Gospel Coalition and The Keller Center for Cultural Apologetics.
The key question the report set out to answer was: What happens if AI gives unreliable or incomplete information to the most common questions about the Christian faith? With more people turning to AI for answers, whether through chatbots or AI-powered search summaries, this is an important issue for Christians to consider. Typically, when googling a question, the user makes the decision of what sites to trust and use. However, now AI is making the decision if you don’t scroll down.
To explore this, TGC selected the seven most frequently searched questions about Christianity and asked seven popular AI platforms to answer them. A panel of seven scholars then graded the responses, with a score of 65 or above indicating that the answer broadly reflected Nicene Creed Christianity (a foundational statement of Christian belief).
The results were surprising. Averaging the scores across all questions, China’s DeepSeek R1 and Perplexity tied for the highest marks. On the other end, Meta’s Llama 3.7 received the lowest score, with its answers often steering readers away from the Christian faith. Google Gemini, OpenAI GPT 4o, and Claude 4 Sonnet frequently took an “all sides” approach, presenting perspectives from other religions alongside Christian responses. Here’s a quick look at the overall scores:
If you’re interested in the full details and the nuances of the benchmark, I encourage you to read the complete report, http://www.christianbenchmark.ai/. But here’s the main takeaway: If all these AI platforms are trained on the same data (the Internet), why do their answers differ so much? The differences come down to the teams and companies behind each AI, the sources they prioritize, and the guidelines they set for answering religious questions. The authors go on to mention, that the goal would be to see companies take a more hands-off approach to religious.
As the AI Christian Benchmark continues to develop, with version 1.0 released and updates like 1.5 and 2.0 already in progress, I’m excited to see the new insights that will help Christian ministries navigate the evolving AI landscape. I’m also encouraged to see organizations such as Gloo stepping up to bring faith-based solutions to market, supporting ministries in this new era.
Merry Christmas,
Jonathan Meester, VP & Chief Technologist, Computers in Ministry
Have you missed out on past tips? Do you want to share them with your colleagues? Now they can be found on our website https://cimtech.org/blog.
