In the last few years, some of the limitations in using LLMs to try and reason causally have become apparent. Asking an LLM “Is it better to walk or drive to the carwash 50m away?” often produces amusing answers, but highlights a real risk: using next word prediction as a logical proxy is prone to errors by over matching templates extant in training data.

Neuro-symbolic Reasoning aims to offload the most critical logic and inference steps to a separate, interpretable, auditable model (e.g. Knowledge Graph). The LLM is used to parse the question, retrieve facts (the answer) from the companion model, and then write a response using the new information.

The existence of Causal Parrots hidden inside LLMs is a nice way of thinking about this problem. The authors demonstrate how “meta structural causal models” arise as an effective short circuit in the training of LLMs, which while allowing great next token accuracy, fail to generalise to novel or reversed scenarios reliably as they hold only weak versions of physical or logical concepts. They are essentially based on the correlation between words, rather than causal understanding.

Our Neuro-symbolic reasoning engine Sertum is helping customers answer questions, spot patterns and predict challenges. Using a combination of automated knowledge graph construction, graph data science and LLM integration, Sertum not only answers problems, but provides detailed reasoning traces using the customer business logic and information, while keeping your institutional knowledge clean, accessible and up to date.

Causal Parrots: Large Language Models May Talk Causality But Are Not Causal

Sean Yardley – Head of Hawkrose Lab