What is Neural-Symbolic Integration? by Gustav Šír

And while these concepts are commonly instantiated by the computation of hidden neurons/layers in deep learning, such hierarchical abstractions are generally very common to human thinking and logical reasoning, too. Amongst the main advantages of this logic-based approach towards ML have been the transparency to humans, deductive reasoning, inclusion of expert knowledge, and structured generalization from small data. And while the current success and adoption of deep learning largely overshadowed the preceding techniques, these still have some interesting capabilities to offer. In this article, we will look into some of the original symbolic AI principles and how they can be combined with deep learning to leverage the benefits of both of these, seemingly unrelated (or even contradictory), approaches to learning and AI. Symbolic AI’s origins trace back to early AI pioneers like John McCarthy, Herbert Simon, and Allen Newell.

Two major reasons are usually brought forth to motivate the study of neuro-symbolic integration. The first one comes from the field of cognitive science, a highly interdisciplinary field that studies the human mind. In that context, we can understand artificial neural networks as an abstraction of the physical workings of the brain, while we can understand formal logic as an abstraction of what we perceive, through introspection, when contemplating explicit https://chat.openai.com/ cognitive reasoning. In order to advance the understanding of the human mind, it therefore appears to be a natural question to ask how these two abstractions can be related or even unified, or how symbol manipulation can arise from a neural substrate [1]. NSI has traditionally focused on emulating logic reasoning within neural networks, providing various perspectives into the correspondence between symbolic and sub-symbolic representations and computing.

Neuro-symbolic artificial intelligence: a survey

An early body of work in AI is purely focused on symbolic approaches with Symbolists pegged as the “prime movers of the field”. Symbolic AI, also known as rule-based AI or classical AI, uses a symbolic representation of knowledge, such as logic or ontologies, to perform reasoning tasks. Symbolic AI relies on explicit rules and algorithms to make decisions and solve problems, and humans can easily understand and explain their reasoning.

Moreover, neuro-symbolic AI isn’t confined to large-scale models; it can also be applied effectively with much smaller models. For instance, frameworks like NSIL exemplify this integration, demonstrating its utility in tasks such as reasoning and knowledge base completion. Overall, neuro-symbolic AI holds promise for various applications, from understanding language nuances to facilitating decision-making processes. Neuro-Symbolic AI combines the interpretability and logical reasoning of symbolic

AI with the pattern recognition and learning capabilities of data-driven neural networks, enabling new advancements in various domains [59]. Furthermore, this approach finds practical applications in developing systems that can accurately diagnose diseases, discover drugs, design more efficient NLP networks, and make informed financial decisions.

Ensuring interpretability and explainability in advanced Neuro-Symbolic AI systems for military applications is important for a wide range of reasons, including accountability, trust, validation, collaboration, and legal compliance [150]. Military logistics experts can provide knowledge about efficient resource allocation and supply chain management. By leveraging AI-driven systems and advanced strategies, military organizations Chat GPT can use this expertise to optimize logistics, ensuring that resources are deployed effectively during operations [7, 101]. Hence, the military can achieve a higher degree of precision in logistics and supply chain management through the integration of AI technologies. Neuro-Symbolic AI systems have the potential to revolutionize the financial industry by developing systems that can make better financial decisions [74].

Backward chaining occurs in Prolog, where a more limited logical representation is used, Horn Clauses. One of the most successful neural network architectures have been the Convolutional Neural Networks (CNNs) [3]⁴ (tracing back to 1982’s Neocognitron [5]). The distinguishing features introduced in CNNs were the use of shared weights and the idea of pooling. While MYCIN was never used in practice due to ethical concerns, it laid the foundation for modern medical expert systems and clinical decision support systems. The article aims to provide an in-depth overview of Symbolic AI, its key concepts, differences from other AI techniques, and its continued relevance through applications and the evolution of Neuro-Symbolic AI. Once they are built, symbolic methods tend to be faster and more efficient than neural techniques.

Neuro Symbolic AI: Enhancing Common Sense in AI

Examples of LAWS include autonomous drones [83, 84], cruise missiles [85], sentry guns [86], and automated turrets. In the context of LAWS, Neuro-Symbolic AI involves incorporating neural network components for perception and learning, coupled with symbolic reasoning to handle higher-level cognition and decision-making. Non-symbolic AI systems do not manipulate a symbolic representation to find solutions to problems. Instead, they perform calculations according to some principles that have demonstrated to be able to solve problems. Examples of Non-symbolic AI include genetic algorithms, neural networks and deep learning. The origins of non-symbolic AI come from the attempt to mimic a human brain and its complex network of interconnected neurons.

They believed that human intelligence could be modeled through logic and symbol manipulation. Their goal was to create machines that could perform tasks typically requiring human intelligence, such as problem-solving, decision-making, and language understanding. Concerningly, some of the latest GenAI techniques are incredibly confident and predictive, confusing humans who rely on the results. This problem is not just an issue with GenAI or neural networks, but, more broadly, with all statistical AI techniques. Now, new training techniques in generative AI (GenAI) models have automated much of the human effort required to build better systems for symbolic AI.

Historically, the community targeted mostly analysis of the correspondence and theoretical model expressiveness, rather than practical learning applications (which is probably why they have been marginalized by the mainstream research). While the particular techniques in symbolic AI varied greatly, the field was largely based on mathematical logic, which was seen as the proper (“neat”) representation formalism for most of the underlying concepts of symbol manipulation. With this formalism in mind, people used to design large knowledge bases, expert and production rule systems, and specialized programming languages for AI.

Examples include incorporating symbolic reasoning modules into neural networks, embedding neural representations into symbolic knowledge graphs, and developing hybrid architectures that seamlessly combine neural and symbolic components [41]. This enhanced capacity for knowledge representation, reasoning, and learning has the potential to revolutionize AI across diverse domains, including natural language understanding [42], robotics, knowledge-based systems, and scientific discovery [43]. While our paper focuses on a Neuro-Symbolic AI for military applications, it is important to note that the architecture shown in Figure 4 is just one of many possible architectures of a broader and diverse field with many different approaches. A. Symbolic AI, also known as classical or rule-based AI, is an approach that represents knowledge using explicit symbols and rules. It emphasizes logical reasoning, manipulating symbols, and making inferences based on predefined rules.

For example, the Neuro-Symbolic Language Model (NSLM) is a state-of-the-art model that combines a deep learning model with a database of knowledge to answer questions more accurately [61]. Symbolic AI is a traditional approach to AI that focuses on representing and rule-based reasoning about knowledge using symbols such as words or abstract symbols, rules, and formal logic [16, 15, 17, 18]. Symbolic AI systems rely on explicit, human-defined knowledge bases that contain facts, rules, and heuristics. These systems use formal logic to make deductions and inferences making it suitable for tasks involving explicit knowledge and logical reasoning. Such systems also use rule-based reasoning to manipulate symbols and draw conclusions. Symbolic AI systems are often transparent and interpretable, meaning it is relatively easy to understand why a particular decision or inference was made.

Neuro-Symbolic AI models typically aim to bridge this gap by integrating neural networks and symbolic reasoning, creating more robust, adaptable, and flexible AI systems. In Figure 4, we present one example of a Neuro-Symbolic AI architecture that integrates symbolic reasoning with neural networks to enhance decision-making. This hybrid approach allows the AI to leverage both the reasoning capabilities of symbolic knowledge and the learning capabilities of neural networks. A key component of this system is a knowledge graph, which acts as a structured network of interconnected concepts and entities. This graph enables the AI to represent relationships between different pieces of information in the knowledge base, facilitating more complex reasoning and inference. The combination of these two approaches results in a unified knowledge base, with integration occurring at various levels.

At the height of the AI boom, companies such as Symbolics, LMI, and Texas Instruments were selling LISP machines specifically targeted to accelerate the development of AI applications and research. In addition, several artificial intelligence companies, such as Teknowledge and Inference Corporation, were selling expert system shells, training, and consulting to corporations. Our future work will focus on addressing these challenges while exploring innovative applications such as adaptive robots and resilient autonomous systems. These efforts will advance the role of Neuro-Symbolic AI in enhancing national security. We will also investigate optimal human-AI collaboration methods, focusing on human-AI teaming dynamics and designing AI systems that augment human capabilities. This approach ensures that Neuro-Symbolic AI serves as a powerful tool to support, rather than replace, human decision-making in military contexts.

LISP is the second oldest programming language after FORTRAN and was created in 1958 by John McCarthy. Program tracing, stepping, and breakpoints were also provided, along with the ability to change values or functions and continue from breakpoints or errors. It had the first self-hosting compiler, meaning that the compiler itself was originally written in LISP and then ran interpretively to compile the compiler code. Expert systems can operate in either a forward chaining – from evidence to conclusions – or backward chaining – from goals to needed data and prerequisites – manner.

But neither the original, symbolic AI that dominated machine learning research until the late 1980s nor its younger cousin, deep learning, have been able to fully simulate the intelligence it’s capable of. If one looks at the history of AI, the research field is divided into two camps – Symbolic & Non-symbolic AI that followed different path towards building an intelligent system. Symbolists firmly believed in developing an intelligent system based on rules and knowledge and whose actions were interpretable while the non-symbolic approach strived to build a computational system inspired by the human brain. In summary, symbolic AI excels at human-understandable reasoning, while Neural Networks are better suited for handling large and complex data sets.

Many identified the need for well-founded knowledge representation and reasoning to be integrated with deep learning and for sound explainability. Neurosymbolic computing has been an active area of research for many years seeking to bring together robust learning in neural networks with reasoning and explainability by offering symbolic representations for neural models. In this paper, we relate recent and early research in neurosymbolic AI with the objective of identifying the most important ingredients of neurosymbolic AI systems. We focus on research that integrates in a principled way neural network-based learning with symbolic knowledge representation and logical reasoning. Finally, this review identifies promising directions and challenges for the next decade of AI research from the perspective of neurosymbolic computing, commonsense reasoning and causal explanation.

This encoding approach facilitates the formal expression of knowledge and rules, making it easier to interpret and explain system behavior [49]. The symbolic nature of knowledge representation allows human-understandable explanations of reasoning processes. Furthermore, symbolic representations enhance the model transparency, facilitating an understanding of the reasoning behind model decisions. Symbolic knowledge can also be easily shared and integrated with other systems, promoting knowledge transfer and collaboration.

Furthermore, the advancements in Neuro-Symbolic AI for military applications hold significant potential for broader applications in civilian domains, such as healthcare, finance, and transportation. This approach offers increased adaptability, interpretability, and reasoning under uncertainty, revolutionizing traditional methods and pushing the boundaries of both military and civilian effectiveness. Coupled neuro-symbolic systems are increasingly used to solve complex problems such as game playing or scene, word, sentence interpretation. Coupling may be through different methods, including the calling of deep learning systems within a symbolic algorithm, or the acquisition of symbolic rules during training.

Robust fail-safes and validation mechanisms are crucial for ensuring safety and reliability, especially when NLAWS operates autonomously. By integrating neural networks and symbolic reasoning, neuro-symbolic AI can handle perceptual tasks such as image recognition and natural language processing and perform logical inference, theorem proving, and planning based on a structured knowledge base. This integration enables the creation of AI systems that can provide human-understandable explanations for their predictions and decisions, making them more trustworthy and transparent. Neuro-symbolic AI blends traditional AI with neural networks, making it adept at handling complex scenarios.

Employing Explainable AI (XAI) techniques can help build trust in the system’s adaptation capabilities [150]. Additionally, fostering human-AI collaboration, where human operators can intervene and guide the system in complex scenarios, is a promising approach [151, 152]. Symbolic reasoning techniques in AI involve the use of symbolic representations, such as logic and rules, to model and manipulate knowledge [49]. These techniques aim to enable machines to perform logical reasoning and decision-making in a manner that is understandable and explainable to humans [17]. In symbolic reasoning, information is represented using symbols and their relationships.

Militaries worldwide are investing heavily in AI research and development to gain an advantage in future wars. AI has the potential to enhance intelligence collection and accurate analysis, improve cyberwarfare capabilities, and deploy autonomous weapons systems. These applications offer the potential for increased efficiency, reduced risk, and improved operational effectiveness. However, as discussed in Section 5, they also raise ethical, legal, and security concerns that must be addressed [88].

Note the similarity to the propositional and relational machine learning we discussed in the last article. Interestingly, we note that the simple logical XOR function is actually still challenging to learn properly even in modern-day deep learning, which we will discuss in the follow-up article. However, there have also been some major disadvantages including computational complexity, inability to capture real-world noisy problems, numerical values, and uncertainty. Due to these problems, most of the symbolic AI approaches remained in their elegant theoretical forms, and never really saw any larger practical adoption in applications (as compared to what we see today). Symbolic AI has been crucial in developing AI systems for strategic games like chess, where the rules of the game and the logic behind moves can be explicitly defined.

Similarly, Allen’s temporal interval algebra is a simplification of reasoning about time and Region Connection Calculus is a simplification of reasoning about spatial relationships. Cognitive architectures such as ACT-R may have additional capabilities, such as the ability to compile frequently used knowledge into higher-level chunks. A more flexible kind of problem-solving occurs when reasoning about what to do next occurs, rather than simply choosing one of the available actions. This kind of meta-level reasoning is used in Soar and in the BB1 blackboard architecture. Programs were themselves data structures that other programs could operate on, allowing the easy definition of higher-level languages.

In the next article, we will then explore how the sought-after relational NSI can actually be implemented with such a dynamic neural modeling approach. Particularly, we will show how to make neural networks learn directly with relational logic representations (beyond graphs and GNNs), ultimately benefiting both the symbolic and deep learning approaches to ML and AI. Other ways of handling more open-ended domains included probabilistic reasoning systems and machine learning to learn new concepts and rules.

The development of neuro-symbolic AI is still in its early stages, and much work must be done to realize its potential fully. However, the progress made so far and the promising results of current research make it clear that neuro-symbolic AI has the potential to play a major role in shaping the future of AI. When deep learning reemerged in 2012, it was with a kind of take-no-prisoners attitude that has characterized most of the last decade. He gave a talk at an AI workshop at Stanford comparing symbols to aether, one of science’s greatest mistakes. McCarthy’s approach to fix the frame problem was circumscription, a kind of non-monotonic logic where deductions could be made from actions that need only specify what would change while not having to explicitly specify everything that would not change. Other non-monotonic logics provided truth maintenance systems that revised beliefs leading to contradictions.

But these more statistical approaches tend to hallucinate, struggle with math and are opaque. Symbolic AI’s strength lies in its knowledge representation and reasoning through logic, making it more akin to Kahneman’s “System 2” mode of thinking, symbolic ai vs neural networks which is slow, takes work and demands attention. That is because it is based on relatively simple underlying logic that relies on things being true, and on rules providing a means of inferring new things from things already known to be true.

You can foun additiona information about ai customer service and artificial intelligence and NLP. YAGO incorporates WordNet as part of its ontology, to align facts extracted from Wikipedia with WordNet synsets. Recently, awareness is growing that explanations should not only rely on raw system inputs but should reflect background knowledge. Advanced AI techniques can be used to develop modern autonomous weapons systems that can operate without human intervention. These AI-powered unmanned vehicles, drones, and robotic systems can execute a wide range of complex tasks, such as reconnaissance, surveillance, and logistics, without human intervention [90]. Neither pure neural networks nor pure symbolic AI alone can solve such multifaceted challenges.

Robotic Process Automation (RPA) in Business

By using its symbolic knowledge of the environment, the robot can determine the best route to reach its destination. Additionally, a robot employing symbolic reasoning better understands and responds to human instructions and feedback [78]. It uses its symbolic knowledge of human language and behavior to reason about the intended communication. Neuro-Symbolic AI models use a combination of neural networks and symbolic knowledge to enhance the performance of NLP tasks such as answering questions [33], machine translation [60], and text summarization.

Indeed, neuro-symbolic AI has seen a significant increase in activity and research output in recent years, together with an apparent shift in emphasis, as discussed in Ref. [2]. Below, we identify what we believe are the main general research directions the field is currently pursuing. It is of course impossible to give credit to all nuances or all important recent contributions in such a brief overview, but we believe that our literature pointers provide excellent starting points for a deeper engagement with neuro-symbolic AI topics.

Psychologist Daniel Kahneman suggested that neural networks and symbolic approaches correspond to System 1 and System 2 modes of thinking and reasoning. System 1 thinking, as exemplified in neural AI, is better suited for making quick judgments, such as identifying a cat in an image. System 2 analysis, exemplified in symbolic AI, involves slower reasoning processes, such as reasoning about what a cat might be doing and how it relates to other things in the scene. A paper on Neural-symbolic integration talks about how intelligent systems based on symbolic knowledge processing and on artificial neural networks, differ substantially. By combining symbolic and neural reasoning in a single architecture, LNNs can leverage the strengths of both methods to perform a wider range of tasks than either method alone. For example, an LNN can use its neural component to process perceptual input and its symbolic component to perform logical inference and planning based on a structured knowledge base.

Consequently, also the structure of the logical inference on top of this representation can no longer be represented by a fixed boolean circuit. While the aforementioned correspondence between the propositional logic formulae and neural networks has been very direct, transferring the same principle to the relational setting was a major challenge NSI researchers have been traditionally struggling with. The issue is that in the propositional setting, only the (binary) values of the existing input propositions are changing, with the structure of the logical program being fixed. It wasn’t until the 1980’s, when the chain rule for differentiation of nested functions was introduced as the backpropagation method to calculate gradients in such neural networks which, in turn, could be trained by gradient descent methods.

For instance, a neuro-symbolic system would employ symbolic AI’s logic to grasp a shape better while detecting it and a neural network’s pattern recognition ability to identify items. As explained above, nations possessing advanced Neuro-Symbolic AI capabilities could gain a strategic advantage. This could lead to concerns about security and potential misuse of AI technologies, prompting diplomatic efforts to address these issues. Hence, the security and robustness of autonomous weapons systems are crucial for addressing ethical, legal, and safety concerns [137].

2 Practical Applications of Neuro-Symbolic AI

RAID, a DARPA research program, focuses on developing AI technology to assist tactical commanders in predicting enemy tactical movements and countering their actions [38]. These include understanding enemy intentions, detecting deception, and providing real-time decision support. RAID achieves this by combining AI for planning with cognitive modeling, game theory, control theory, and ML [38]. These capabilities have significant value in military planning, executing operations, and intelligence analysis.

These components work together to form a neuro-symbolic AI system that can perform various tasks, combining the strengths of both neural networks and symbolic reasoning. This amalgamation of science and technology brings us closer to achieving artificial general intelligence, a significant milestone in the field. Moreover, it serves as a general catalyst for advancements across multiple domains, driving innovation and progress.

CNNs are good at processing information in parallel, such as the meaning of pixels in an image. New GenAI techniques often use transformer-based neural networks that automate data prep work in training AI systems such as ChatGPT and Google Gemini. Symbolic AI algorithms have played an important role in AI’s history, but they face challenges in learning on their own. After IBM Watson used symbolic reasoning to beat Brad Rutter and Ken Jennings at Jeopardy in 2011, the technology has been eclipsed by neural networks trained by deep learning.

Each approach—symbolic, connectionist, and behavior-based—has advantages, but has been criticized by the other approaches. Symbolic AI has been criticized as disembodied, liable to the qualification problem, and poor in handling the perceptual problems where deep learning excels. In turn, connectionist AI has been criticized as poorly suited for deliberative step-by-step problem solving, incorporating knowledge, and handling planning. Finally, Nouvelle AI excels in reactive and real-world robotics domains but has been criticized for difficulties in incorporating learning and knowledge. During the first AI summer, many people thought that machine intelligence could be achieved in just a few years.

Integrating NLAWS with Neuro-Symbolic AI presents several challenges, particularly in ensuring the interpretability of decisions for human understanding, accountability, and ethical considerations [93, 94]. Even though the primary purpose of these systems is non-lethal, their deployment in conflict situations raises significant ethical concerns. NLAWS must be able to respond effectively to dynamic and unpredictable scenarios, demanding seamless integration with Neuro-Symbolic AI to facilitate learning and reasoning in complex environments. One emerging approach in this context is reservoir computing, which leverages recurrent neural networks with fixed internal dynamics to process temporal information efficiently. This method enhances the system’s ability to handle dynamic inputs and supports the learning and reasoning capabilities required for complex environments [95].

“Deep learning in its present state cannot learn logical rules, since its strength comes from analyzing correlations in the data,” he said. Despite the difference, they have both evolved to become standard approaches to AI and there is are fervent efforts by research community to combine the robustness of neural networks with the expressivity of symbolic knowledge representation. The traditional symbolic approach, introduced by Newell & Simon in 1976 describes AI as the development of models using symbolic manipulation. In the Symbolic approach, AI applications process strings of characters that represent real-world entities or concepts. Symbols can be arranged in structures such as lists, hierarchies, or networks and these structures show how symbols relate to each other.

Article Contents

G-Retriever employs a novel approach for integrating retrieval-based methods into language models, enhancing their ability to access and utilize domain-specific knowledge [52]. Additionally, process Knowledge-infused Learning incorporates structured process knowledge into learning algorithms to improve decision-making and reasoning in complex tasks [53]. The effective integration of expert knowledge holds significant promise for addressing complex challenges across various domains, such as healthcare, finance, robotics, and NLP [47]. For example, expert knowledge plays a crucial role in military operations, enhancing capabilities in strategic planning, tactical decision-making, cybersecurity [54, 55], logistics, and battlefield medical care [56]. Similarly, in a medical diagnosis system, expert knowledge may be encoded as rules describing symptoms and their relationships to specific diseases [56].

Additionally, there are technical challenges to overcome before autonomous weapons systems can be widely deployed [110], such as reliably distinguishing between combatants and civilians operating in complex environments. Military experts can contribute to the development of realistic training simulations by providing domain-specific knowledge. AI-driven simulations and virtual training environments provide a realistic training experience for military personnel, helping them to develop the skills and knowledge they need to succeed in diverse operational scenarios [8, 9]. This helps in preparing military personnel for various scenarios, improving their decision-making skills, strategic thinking, and ability to handle dynamic and complex situations [106]. Beyond training, AI can simulate various scenarios, empowering military planners to test strategies and evaluate potential outcomes before actual deployment [107]. These dynamic models finally enable to skip the preprocessing step of turning the relational representations, such as interpretations of a relational logic program, into the fixed-size vector (tensor) format.

By automatically learning meaningful representations, neural networks can achieve reasonably higher performance on tasks that demand understanding and extraction of relevant information from complex data [39]. For much of the AI era, symbolic approaches held the upper hand in adding value through apps including expert systems, fraud detection and argument mining. But innovations in deep learning and the infrastructure for training large language models (LLMs) have shifted the focus toward neural networks.

Therefore, it is important to use diverse and representative training data to minimize the risk of discriminatory actions by autonomous systems [127]. Autonomous weapons systems must be able to reliably distinguish between combatants and civilians, even in complex and unpredictable environments. If autonomous weapons systems cannot make this distinction accurately, they could lead to indiscriminate attacks and civilian casualties violating international humanitarian law [79, 87].

Implementing secure communication protocols and robust cybersecurity measures is essential to safeguard against such manipulations [10]. Furthermore, reliable communication is crucial for transmitting data to and from autonomous weapons systems. The use of redundant communication channels and fail-safe mechanisms is necessary to ensure uninterrupted operation, even in the event of a channel failure [145].

The work in [34] describes the use of Neuro-Symbolic AI in developing a system to support operational decision-making in the context of the North Atlantic Treaty Organization (NATO). The Neuro-Symbolic modeling system, as presented in [34], employs a combination of neural networks and symbolic reasoning to generate and evaluate different courses of action within a simulated battlespace to help commanders make better decisions. Combining symbolic medical knowledge with neural networks can improve disease diagnosis, drug discovery, and prediction accuracy [69, 70, 71]. This approach has the potential to ultimately make medical AI systems more interpretable, reliable, and generalizable [72]. For example, the work in [73] proposes a Recursive Neural Knowledge Network (RNKN) that combines medical knowledge based on first-order logic for multi-disease diagnosis.

Such machine intelligence would be far superior to the current machine learning algorithms, typically aimed at specific narrow domains. We believe that our results are the first step to direct learning representations in the neural networks towards symbol-like entities that can be manipulated by high-dimensional computing. Such an approach facilitates fast and lifelong learning and paves the way for high-level reasoning and manipulation of objects.

Ensuring resistance to cyber threats such as hacking, data manipulation, and spoofing is essential to prevent misuse and unintended consequences [90, 138]. A reliable, ethical decision-making process, including accurate target identification, proportionality assessment, and adherence to international law, is essential. To enhance the robustness and resilience of Neuro-Symbolic AI systems against adversarial attacks, training the underlying AI model with both clean and adversarial inputs is effective [139, 140]. Additionally, incorporating formal methods for symbolic verification and validation ensures the correctness of symbolic reasoning components [141].

Advantages of multi-agent systems include the ability to divide work among the agents and to increase fault tolerance when agents are lost. Research problems include how agents reach consensus, distributed problem solving, multi-agent learning, multi-agent planning, and distributed constraint optimization. They can simplify sets of spatiotemporal constraints, such as those for RCC or Temporal Algebra, along with solving other kinds of puzzle problems, such as Wordle, Sudoku, cryptarithmetic problems, and so on. Constraint logic programming can be used to solve scheduling problems, for example with constraint handling rules (CHR). Military decision-making often involves complex tasks that require a combination of human and AI capabilities.

Its history was also influenced by Carl Hewitt’s PLANNER, an assertional database with pattern-directed invocation of methods. Predictive maintenance is an application of AI that leverages data analysis and ML techniques to predict when equipment or machinery is likely to fail or require maintenance [97]. AI enables predictive maintenance by analyzing data to predict equipment maintenance needs [98].

Systems such as Lex Machina use rule-based logic to provide legal analytics, leveraging symbolic AI to analyze case law and predict outcomes based on historical data. Symbolic AI has been widely used in healthcare through expert systems that help diagnose diseases and suggest treatments based on a set of rules. Our researchers are working to usher in a new era of AI where machines can learn more like the way humans do, by connecting words with images and mastering abstract concepts. Natural language processing focuses on treating language as data to perform tasks such as identifying topics without necessarily understanding the intended meaning.

• Particularly, we will show how to make neural networks learn directly with relational logic representations (beyond graphs and GNNs), ultimately benefiting both the symbolic and deep learning approaches to ML and AI.
• Over the next few decades, research dollars flowed into symbolic methods used in expert systems, knowledge representation, game playing and logical reasoning.
• Critiques from outside of the field were primarily from philosophers, on intellectual grounds, but also from funding agencies, especially during the two AI winters.
• Military decision-making often involves complex tasks that require a combination of human and AI capabilities.
• Additionally, it examines the challenges of holding individuals accountable for violations of international humanitarian law involving autonomous weapons systems [122].

These two problems are still pronounced in neuro-symbolic AI, which aims to combine the best of the two paradigms. The efficacy of NVSA is demonstrated by solving Raven’s progressive matrices datasets. Compared with state-of-the-art deep neural network and neuro-symbolic approaches, end-to-end training of NVSA achieves a new record of 87.7% average accuracy in RAVEN, and 88.1% in I-RAVEN datasets. Moreover, compared with the symbolic reasoning within the neuro-symbolic approaches, the probabilistic reasoning of NVSA with less expensive operations on the distributed representations is two orders of magnitude faster.

While Deep Blue is famous for its brute-force search and computational power, it also relied on symbolic AI techniques to evaluate board positions based on rules derived from expert human play. Symbolic techniques were at the heart of the IBM Watson DeepQA system, which beat the best human at answering trivia questions in the game Jeopardy! However, this also required much human effort to organize and link all the facts into a symbolic reasoning system, which did not scale well to new use cases in medicine and other domains. “Our vision is to use neural networks as a bridge to get us to the symbolic domain,” Cox said, referring to work that IBM is exploring with its partners. “We are finding that neural networks can get you to the symbolic domain and then you can use a wealth of ideas from symbolic AI to understand the world,” Cox said.

This learned representation captures the essential characteristics and features of the data, allowing the network the ability to generalize well to previously unseen examples. Deep neural networks have demonstrated remarkable success in representation learning, particularly in capturing hierarchical and abstract features from diverse datasets [21, 39]. This success has translated into significant contributions across a wide range of tasks, including image classification, NLP, and recommender systems.