Towards Generating P-Contrastive Explanations for Goal Selection in Extended-BDI Agents

Abstract

As agent-based systems have been growing, more and more the general public has access to them and is influenced by decisions taken by these systems. This increases the necessity for such systems to be capable of explaining themselves to a user. The Beliefs-Desires-Intentions (BDI) is a commonly used agent model that has an two-phase internal goal selection process (desires and intentions) to decide what goals to pursue. Belief-based goal processing (BBGP) model is an extended-BDI model whose selection process consists of four phases. This more-grained behavior may have relevant consequences for the analysis of what an intention is and better explain how an intention becomes what it is. Contrastive explanations are commonly employed by people and can bring benefits to the explanation exchange process. A Property-contrast (P-contrast) explanation is a type of contrastive explanation that compares the properties of an explanation object. Thus, we can take a goal as the object about which we want an explanation and its status in the selection process (that is, how much it has advanced) as a property. This work tackles the problem of generating P-contrast explanations and proposes a method to construct them. The method consists of two phases, which in turn consist of a set of steps. The first step of the second phase returns a set of beliefs that constitute a future explanation. Thus, this work focuses on the first phase and the first step of the second phase. We use a scenario of the cleaner world in order to illustrate the performance of our proposal.

Supported by organization CAPES/Brazil and CNPq Proc. 409523/2021-6.

1 Introduction

In intelligent agents, practical reasoning is the process of figuring out what to do. According to Wooldridge [18], practical reasoning involves two phases: deliberation and means-ends reasoning. Deliberation is decomposed in two parts: (i) firstly, the agent generates a set of possible goals and (ii) then, the agent chooses which goals he will be committed to bring about. This process is known as goal selection. In order to make this decision, a goal may have to pass from more than one stage depending on the practical reasoning model. BDI model [1] is a very known and used practical model that has two stages: (i) desires represent some end state that the agent deems as desirable and (ii) intentions are desires that the agent is committed to, with their respective plans. BBGP model [2] can be seen an extended model of BDI that consider four stages for goal processing: (i) activation, (ii) evaluation, (iii) deliberation, and (iv) checking. Consequently, four different statuses for a goal are defined: (i)active, (ii)pursuable, (iii)chosen, and (iv)executive. Figure 1 shows a parallel between these models. The schema presents the goal reasoning stages and the status of a goal after passing each stage. In [2], the authors claim that chosen goals can be seen as future-directed intentions and executive ones as present-directed intentions. Thus, chosen and executive goals are intentions from a BDI perspective.

Fig. 1.

Schema of the goal reasoning stages in BDI and BBGP models. This figure was extracted from [10].

Contrastive explanations are commonly employed by people and can bring benefits to the exchange process during the explanation. Contrastive questions provide an insight into the questioner’s mental model, allowing to have a better understanding of what they do not know and contrastive explanations usually are more straightforward, more feasible, and less demanding both for the questioner and explainer [8]. In [16], the authors distinguish three types of contrastive questions:

• (P-contrast) Why does object o have property p, rather than property $p'$?

• (O-contrast) Why does object o have property p, while object $o'$ has property $p'$?

• (T-contrast) Why does object o have property p at time t, but property $p'$ at time $t'$?

In this work, we focus on generating explanations for P-contrast questions. We consider goals as the object and the status of goals as properties.

Interest in explainable artificial intelligence (XAI) grounded on social and cognitive sciences studies has grown recently [9] as well as contrastive explanations [15]. Despite such growth, there are relatively few works directed to the end user with such grounding (e.g., [5, 7, 14]). Since the explanation process is an exchange of information most likely involving a person, it should follow the rules of conversation. Grice [4] presents the Cooperation Principle with four categories: quantity, quality, relation, and manner. Each with a set of maxims that two entities in a cooperative dialogue are expected to follow. The “Quantity” category is of particular interest in this work. Its maxims state that one should be as informative as necessary, but not more than that. Thus, we will take into account this category when generating the conditions for future explanations. Regarding explainable agents focused on goal selection, to the best of our knowledge, no contrastive explanation generation approach was found. However, there are some related approaches for generating ontological and causal explanations (e.g., [10, 11, 17]). Against this background, we can formulate our research questions:

• 1. How to generate P-contrastive explanations for goal selection process?

• 2. Considering social and cognitive sciences works, what information is required for that generation?

In order to address the first question, we propose a method for generating P-contrastive explanations, which are constructed from the agent’s execution log. The method does not present a final user-ready explanation. It provides a set of conditions (beliefs) that are required to attribute a causal relation to the P-contrastive question and that are the base for future explanations. Since a goal can advance or go back during goal selection, we generate explanations for each case and also consider the case when a goal changes its status due to a preference order, which happens when there are conflicts between goals and preference is used to determine the status change. Regarding the second question, the method is based on Grice’s Cooperation Principles as requirements for the quality of explanations, specifically on the quantity category. In this sense, the resultant set of conditions includes a concise set of beliefs that are directly related to the status change of goals.

The remainder of this paper is structured as follows. In Sect. 2, we present method for generating explanations. In Sect. 3, we focus on the generation of conditions for P-contrastive explanations. Some related work is presented in Sect. 5. Finally, Sect. 6 is devoted to conclusions and future work.

2 Explanation Generation Method

This section presents the agent architecture and the method for generating explanations.

2.1 Building Blocks

We begin by presenting the agent architecture. An agent in the context of this work is an artificial entity that has a set of beliefs, a set of goals, and a set of rules. The temporal aspect of the agent execution is captured by the goals reasoning cycles. A cycle is considered to be a single execution of the selection process.

First of all we present the logical language that will be used. Let $\mathcal {L}$ be a first order logical (FOL) language which will be used to represent the goals, beliefs, and rules of an agent. The following symbols $\wedge , \vee , \rightarrow$, and $\lnot$ denote the logical connectives conjunction, disjunction, implication, and negation, and $\vdash$ stands for the inference.

Definition 1

(Agent). An agent $\texttt{agt}$ is a tuple $\texttt{agt}=\langle \mathcal {K}\mathcal {B}, \mathcal {G}, \mathcal {P}\rangle$, where $\mathcal {K}\mathcal {B}= \mathcal {B}\cup \mathcal {R}$ is the knowledge base, which is composed of a finite set of beliefs $\mathcal {B}$ and a set of finite rules $\mathcal {R}$, $\mathcal {G}$ is a finite set of goals, and $\mathcal {P}$ is a set of plans. A goal $g \in \mathcal {G}$ is a tuple $g=\langle gid, st\rangle$ where gid is a unique atom representing goal g and st is the current status of goal g. A rule $r \in \mathcal {R}$ is an expression of the form $r=\bigwedge b \rightarrow h$, where $b \in \mathcal {B}$ and $h\in \mathcal {B}$ or $h\in \mathcal {G}$. Let $p \in \mathcal {P}$ be a plan such that $p=\langle g, Gd, Act \rangle$, where $g \in \mathcal {G}$, Gd is the set of guard clauses that act as a precondition for the plan execution, and Act is an ordered list of actions that allows the agent to achieve the goal g.

$\mathcal {B}$ and $\mathcal {G}$ are subsets of literals¹ from the language $\mathcal {L}$. It also holds that $\mathcal {B}$ and $\mathcal {G}$ are pairwise disjoint. For this method, the nature of the agent’s plan library and the actions list are not relevant. In the case of plans, only the goal and the guard clause are required information. A guard clause is a literal in language $\mathcal {L}$.

An explanation can be described as the assignment of causal responsibility, as it presents possible causes for what is being explained [6]. Thus, by using the agent’s beliefs and the causal relationship between them and their goals, contrastive explanations can be constructed. Since there is a causal relation between the beliefs and goals, the rules and plans of the agent knowledge base can be used to define it. Thus, a cause is a relation between a set of beliefs and a single event, which is either a belief or a goal.

Definition 2

(Cause). Let $\texttt{agt}=\langle \mathcal {K}\mathcal {B}, \mathcal {G}, \mathcal {P}\rangle$ be an agent and $\texttt{E}=\mathcal {B}\cup \mathcal {G}$ be a finite set of events. A cause c is a tuple $c=\langle \texttt{e}, \texttt{COND} \rangle$, where $\texttt{e} \in \texttt{E}$ is an event and $\texttt{COND} \subset \mathcal {B}$ is a set of beliefs called conditions. If $\mathcal {K}\mathcal {B}\vdash \texttt{COND}$ then event $\texttt{e}$ is expected to happen. Let $\texttt{C}$ be the set of causes.

Given a cause $c=\langle \texttt{e}, \texttt{COND} \rangle$, we say that it is activated if $\forall b \in \texttt{COND}$ $\mathcal {K}\mathcal {B}\vdash b$ otherwise, if $\exists b \in \texttt{COND}$ such that $\mathcal {K}\mathcal {B}\nvdash b$ then the cause c is deactivated. When two events share a piece of causal history, they are said to be causally related. Besides, the causal history of an event is called a causal chain. Formally, let $c=\langle \texttt{e}, \texttt{COND} \rangle$, $c'=\langle \texttt{e}', \texttt{COND}' \rangle$, and $c''=\langle \texttt{e}'', \texttt{COND}'' \rangle$ be three causes s.t. $c, c', c'' \in \texttt{C}$:

• c’ and c are in a causal chain if $\texttt{e}' \in \texttt{COND}$ or if $\texttt{e}'' \in \texttt{COND} \wedge \texttt{e}' \in \texttt{COND}''$

• c and c’ are causally related if $\exists c'' \in \texttt{C}$, such that c” is in a causal chain with c and c’. These relations are denoted by $\texttt{related}(c,c')\in \{true, false\}$.

Causes describes a relation of a set of beliefs with another belief or goal. This relation can be extracted from rules and plans. Thus, we convert rules and plans to causes as follows:

• Plan to cause conversion: Given a plan $p=\langle g, Gd, Act \rangle$, a cause c from p is $c=\langle g, Gd \rangle$.

• Rule to cause conversion:. Given a rule $r=\bigwedge b \rightarrow h$, a cause c from r is $c=\langle h, \bigwedge b \rangle$.

2.2 The Proposed Method

In order to generate a contrastive explanations about a past decision during goal selection, it is necessary to reproduce the agent’s knowledge base configuration when the explained events happened. Thus, a memory of the execution needs to be kept.

We propose a method that consists of two phases: Interface and Explanation Generator. Figure 2 shows these phases and the required precedures in each phase. In the Interface, four elements are required: i) the execution history is a log-like ordered list of events, ii) the preference function encodes a preference relationship between two events, iii) the causal function is responsible for mapping events to the set of conditions that are causally related to it, and iv) the conflict function is responsible for identifying if two given goals have any sort of incompatibilities between them. The Explanation Generator has four domain-independent elements: i) the presumptions functions that, given a set of beliefs, return the subset containing only the presumptions (that is, beliefs that need no support); ii) the generate related causes procedure is responsible for receiving the posed question, retrieving the required information using the functions defined in Interface and the presumption functions, and returns the set of related conditions; iii) the generate possible explanations procedure takes as input the set of related conditions and outputs sets of explanations, where each possible explication is a subset of the related conditions; and iv) the select an explanation procedure, which takes the set of possible explanations and returns one considered the “most adequate” according to the context. These last elements are out of the scope of this work. Thus, we focus on generating a set of related conditions.

Fig. 2.

Schema of the explanation generation method.

2.2.1 Interface

Each entry in the execution history needs to encapsulate, for each reasoning cycle, the set of goals that changed their status and the set of beliefs that were added or removed from the agent’s knowledge base. Note that the goals changes happened during the deliberation, but the beliefs changes are updated after the deliberation has finished. Thus, for a cycle i, the reconstructed knowledge base needs to apply the changes only up to cycle $i-1$.

Definition 3

(Execution history entry). Let $\texttt{agt}=\langle \mathcal {K}\mathcal {B}, \mathcal {G}, \mathcal {P}\rangle$ be an agent and $\texttt{STATS}$ be a list of possible statuses of a goal. An entry for the execution history is a tuple $hist=\langle i, \mathcal {G}_H, \mathcal {B}_H\rangle$, where i is the cycle identifier, $\mathcal {G}_H$ is the set of changed goals s.t. a changed goal $g_H$ is a tuple $g_H=\langle g, st\rangle$ where $g \in \mathcal {G}$ and $st \in \texttt{STATS}$ is the new status of goal g, and $\mathcal {B}_H$ is the set of changed beliefs s.t. a changed belief $b_H$ is a tuple $b_H=\langle b,v\rangle$ where $b \in \mathcal {B}$ and $v\in \{\texttt{ADD}, \texttt{REM}\}$ represents the beliefs addition or removal, respectively.

We now define the functions employed in the Interface:

Definition 4

(Causal, Preference, and Conflict Functions). Let $\texttt{agt}=\langle \mathcal {K}\mathcal {B}, \mathcal {G}, \mathcal {P}\rangle$ be an agent, $\texttt{E}=\mathcal {B}\cup \mathcal {G}$ be the set of events, and $\texttt{C}$ the set of causes.

• Causal function: $\texttt{causes}: \texttt{E} \longrightarrow 2^{\texttt{C}}$ maps an event $e \in \texttt{E}$ to the set of every cause related to it.

• Preference function: $\texttt{preferred}: \mathcal {G}^2 \times \mathbb {N} \longrightarrow \{-1, 0, 1\}$. This function takes as input two goals and the cycle index where they are conflicting and returns the preference relation between both goals. Thus, $-1$ represents that the first goal of the pair is less preferred than the second one, 0 represents that both are equally preferred, and 1 denotes that the first goal of the pair is more preferred than the second one.

• Conflict function: $\texttt{conflict}: \mathcal {G}^2 \longrightarrow \{True, False\}$. This function takes as input two goals and returns if they are conflicting or not. We have that True denotes that there is a conflict between both goals of the pair and False denotes that there is no conflict.

2.2.2 Explanation Generator

Different patterns can be observed when analyzing how an agent answers questions about its goals. Such patterns represent how the reasoning mechanism deliberates about the selected goal in each step of the goal selection. The three observed patterns are: a) positive filter beliefs, which are beliefs that lead to the advance of the targeted goal to its next status; b) negative filter beliefs, which are beliefs that impede the advance of a given goal to its next status; and c) preference filters, which express the preference relationship between conflicting goals.

The agent’s goals always have some associated status. The set of possible statuses is dependent on the agent model. For example, the BDI model defines two statuses: desire and intention; in turn, the BBGP model defines four: active, pursuable, chosen, and executive. The possible statuses of a goal are ordered. Considering that order, a status can be regarded as subsequent to another if it comes afterward in the list. Thus, given a finite ordered list of possible statuses $\texttt{STATS}=\{st_1, ..., st_n\}$ and $st_j, st_k \in \texttt{STATS}$ two possible statuses. We say that:

• $st_k$ is directly subsequent to $st_j$ if $k=j+1$.

• $st_k$ is subsequent to $st_j$ if $k>j$.

The expanded knowledge base, denoted by $\mathcal {K}\mathcal {B}^+$, is the knowledge base $\mathcal {K}\mathcal {B}$ of the agent and everything that entails from it. It contains everything that the agent considers as true.

Definition 5

(Expanded Knowledge Base). Let $\mathcal {K}\mathcal {B}$ be the knowledge base of the agent, the expanded knowledge base $\mathcal {K}\mathcal {B}^+=\{\varphi \mid \mathcal {K}\mathcal {B}\vdash \varphi \}$, and $\not \exists \varphi \in \mathcal {K}\mathcal {B}^+ \mid \mathcal {K}\mathcal {B}\vdash \varphi$, where $\varphi \not \in \mathcal {K}\mathcal {B}^+$.

The explanations are built using the most fundamental beliefs of the agent, called presumptions, as they need no support to be considered true. Thus, $\forall r \in \mathcal {R}$, if $r: \emptyset \rightarrow h$ then h is a presumption.

There are two types of presumption extraction functions, one when the desired outcome is an activated causal rule and the other when the desired outcome is that a deactivated causal rule. The function $\mathtt {d\_pre}$ maps a set of causes to the corresponding set of presumptions used in them. The following disjunctive form of the function is required for answers that need to deactivate a cause, as it represents the idea that if a single cause were removed, the event would not have happened.

Definition 6

(Disjunctive Presumptions). Let $\texttt{agt}=\langle \mathcal {K}\mathcal {B}, \mathcal {G}, \mathcal {P}\rangle$ be an agent, $\texttt{E}$ be a set of events, $e \in \texttt{E}$ a given event, and $\texttt{C}=\{c_1, ..., c_n\}$ be a set of causes. $\mathtt {d\_pre}$ is defined as follows:

$$\begin{aligned} \mathtt {d\_pre}(\texttt{C}) = \left\{ \begin{array}{ll} \bigcup _{i=1}^n\mathtt {d\_pre}(\mathtt {c_i}) &{}if \mid \texttt{C} \mid >1\\ h &{} if\; \texttt{C}=\{\emptyset \rightarrow h\} \wedge h\in \mathcal {B}\\ \bigcup _{j=1}^m \mathtt {d\_pre}(\texttt{causes}(b_j)) &{} if\; \texttt{C}= b_1, ..., b_m \rightarrow e \end{array} \right. \end{aligned}$$

(1)

Let us now focus on conjunctive presumptions. Every presumption in a cause of event e forms a single conjunctive formula. There is one formula for each different cause of e. This conjunctive form is used for answers that need to activate a cause, as it represents the idea that if all the beliefs that are missing were to become true, the event would have happened.

Definition 7

(Conjunctive Presumptions). Let $\texttt{agt}=\langle \mathcal {K}\mathcal {B}, \mathcal {G}, \mathcal {P}\rangle$ be an agent, $\texttt{E}$ be a set of events, $e \in \texttt{E}$ a given event, and $\texttt{C}=\{c_1, ..., c_n\}$ be a set of causes. $\mathtt {c\_pre}$ is defined as follows:

$$\begin{aligned} \mathtt {c\_pre}(\texttt{C}) = \left\{ \begin{array}{ll} \bigcup _{i=1}^n\mathtt {c\_pre}(\mathtt {c_i}) &{}if \mid \texttt{C} \mid >1\\ h &{} if\; \texttt{C}=\{\emptyset \rightarrow h\} \wedge h\in \mathcal {B}\\ \bigwedge _{j=1}^m \mathtt {c\_pre}(\texttt{causes}(b_j)) &{} if\; \texttt{C}= b_1, ..., b_m \rightarrow e \end{array} \right. \end{aligned}$$

(2)

Figure 3 shows an example of applying the presumption functions on the causes for $\lnot mop(x,y)$. Rectangles denote the event, ellipses denote causes, dashed lines denote that the causes are directly related to the event, solid lines denote that the causes are in a causal chain. If two solid lines are joined by a line, it denotes that their causes are connected by logic conjunction. In (a) it is possible to see the conjunctive function, where the presumptions for each cause form a single formula. In (b) it is possible to see that the causes are disjoint.

Let us recall that there is a preference relation between goals. This relation is especially important between conflicting goals since it is the reason for selecting one over the other. This preference is denoted by $g > g'$, which means that g is more preferred than $g'$. Sometimes two sets of goals need to be compared, especially when the goal being explained has a low overall preference and other compatible goals help it to become selected. Note that both sets of goals must be internally compatible. This relation is denoted by $\mathcal {G}' > \mathcal {G}''$, which means that $\mathcal {G}'$ is preferred over $\mathcal {G}''$. Sometimes a single goal has a very high priority, that is, it is preferred over a set of other goals. This is important when the high priority of such goal can explain its selection over any other goals. We denote it by $g \gg \mathcal {G}'$, which means that g is maximally preferred over a set of goals $\mathcal {G}' \subset \mathcal {G}$.

The procedure Generate Related Causes is tackled in the next section because the output depends on the type of contrast explanation.

Fig. 3.

Example of presumption functions for goal $\lnot mop(x,y)$. (a) shows a conjunction example and (b) shows a disjunction example.

3 Generating Conditions for P-Contrast Explanations

In this section, we focus on generating a set of conditions for P-contrast explanations.

P-Contrast questions are of the form: “Why is goal g in status st rather than in status st’?”. The answer is “a set of absent conditions $\texttt{RC}$ that would ensure that g achieves status $st'$.

Two sub-cases can be defined considering the sequence of the statuses:

• 1. $st' \succ st$, that is, “Why has not g advanced in the goal reasoning stages?

• 2. $st\succ st'$, that is, “Why has not g gone back in the goal reasoning stages?

(CASE 1). For $st' \succ st$, each deliberative pattern is defined as follows:

• Positive: The conditions for the explanation needs to present the set of presumptions that, if present, would have ensured the goal’s progress. This set can be obtained in the following manner: $\texttt{RC}=\mathtt {c\_pre}(\texttt{causes}(g))-\mathcal {K}\mathcal {B}$.

• Negative: The explanation needs to present a set of presumptions that, if were made absent, would have allowed the goal to progress freely. The set is obtained in the following manner: $\texttt{RC}=\mathtt {d\_pre}(\texttt{causes}(\lnot g))\cap \mathcal {K}\mathcal {B}$.

• Preference: Although no presumption was missing or impeding the goal progression, two cases may occur there was a conflict between the goal that is object of the explanation and another goal and the lack of preference made the goal not to be selected to advance. Thus, the conditions of an explanation are assertions about the preference, identifying why g was not selected: $\texttt{RC}=\texttt{conflict}(g,g')=True, \texttt{preferred}(g',g,i)=True$.

(CASE 2). For $\texttt{st} \succ \texttt{st}'$, each deliberative pattern is defined as follows:

• Positive: The conditions for an explanation need to present the set of presumptions that, if present, would have ensured the goal regression. This set can be obtained in the following manner: $\texttt{RC}=\mathtt {d\_pre}(\texttt{causes}(\lnot g))\cap \mathcal {K}\mathcal {B}$.

• Negative: The conditions for an explanation need to present a set of presumptions that, if are in the $\mathcal {K}\mathcal {B}$, would have impeded the goal to stay with status st. The set is obtained in the following manner: $\texttt{RC}=\mathtt {c\_pre}(\texttt{causes}(g)) -\mathcal {K}\mathcal {B}$.

• Preference: Although no presumptions forced the goal regression, a conflict of goals was expected, but the preference made the goal to be selected to continue with the status st. Thus, the conditions of an explanation are assertions about the preference, identifying why g was not selected: $\texttt{RC}=\texttt{conflict}(g,g')=True, \texttt{preferred}(g,g',i)=True$.

4 Case Study: Cleaner World Scenario

In this section, a case study is presented. We show the mental state of the agent and show how the conditions fora p-contrast explanation are generated.

In the cleaner world scenario, the agent is a robot in charge of cleaning a certain area. In the robot’s representation, the area is divided into cells referred to by a coordinate system (x, y). The robot can move along the cells, clean solid and liquid dirt by sweeping and mopping, respectively, recharging itself when necessary, replacing some broken parts, and disposing of the dirt in its internal storage. Lastly, for the robot not to disrupt the passage when there are no jobs at the moment, there is a command for it to rest. There are two special locations for the robot, the workshop and the dumpster. In the workshop, the robot can find replacement parts, recharge, and is where it is assigned to rest. The dumpster is where the dirt is disposed of. To do the mopping, the robot needs a functional mop part, and to sweep, the robot needs a functional vacuum. The mop is a part that the robot can replace by itself, given that there is a replacement available.

On a given day, the following series of events were observed. At time $t_0$, three cells were dirty, where (2,2) and (3,2) had liquid dirt and (3,4) solid dirt. At $t_1$, the robot cleaned (2,2). Followed by cell (3,4) at $t_2$. The robot proceed to go back to the workshop at $t_3$.

4.1 Mental States

The agent’s knowledge base, goals, and plans for the scenario are described next. Let us begin describing the agent’s starting beliefs, that is, the beliefs it had at $t_0$ before the first deliberation cycle. Thus, we have $\mathcal {B}=\{workshop(1,1)$, dumpster(4, 1), $\lnot available(mop)$, at(1, 1), have(battery, 75), have(cargo, 65), $solid\_dirt(3,4)$, $liquid\_dirt(3,2)$, $liquid\_dirt(2,2)\}$. The goals are: $\mathcal {G}=\{(replace(x), 0)$, (mop(x, y), 0), (sweep(x, y), 0), (recharge, 0), (rest, 0), $(dispose,0)\}$, where 0 means that the goal do not have a status yet.

Table 1 shows the rules for the scenario. In the BBGP model, there are four types of rules, each type for each stage. Only the activation and evaluation rules are depicted because the rules for deliberation and checking stages are fixed since they are domain-independent².

Table 1. Activation and Evaluation Rules.

The preference order of goals is exactly the activation rules order, in decreasing order. That is, mop(x, y) has the highest preference and recharge has the lowest one.

Let us recall that the first phase of the method, that is, Interface, requires four elements to be implemented: the execution history, the causal function, the preference function, and the conflict function. For our scenario, the agent’s execution history is depicted in Table 2, where each row is an entry of the log. The log allows for the reconstruction of the agent knowledge base and tracks the goal status changes.

Table 2. Execution History.

The Causal Function: The activation and evaluation rules can be easily converted into causes by attributing the consequent (the goal) as e and the set of beliefs in the antecedent as $\texttt{COND}$. Table 3 shows the mapping generated. The BBGP model has two more stages: deliberation and checking. The deliberation has no rules to be converted, as it evaluates conflicts and preferences among goals, it will be encoded by the conflict and preference functions.

Table 3. Mapping of the Activation and Evaluation rules to causes.

The Preference Function: In the example, the model defines a preference, which was described with the scenario, as such, using the execution history, the total (decreasing) preference relation for the scenario is as follows: $mop(2,2)> mop(3,2)> sweep(3,4)> recharge$.

The Conflict Function: Given the nature of the scenario and the defined goals, every goal is incompatible with each other. This is the case because the goals are localized in space, and the robot needs to be at the specified place to perform the actions to achieve the goal.

4.2 The Conditions for a P-Contrast Explanation

We can now present how the conditions for generating a P-contrast explanation are constructed. Let us start by posing a question: Why mop(3, 2) is active, instead of executive? The rationale of this question is very straightforward: why a goal is not being executed? First, let us identify the information in the question:

• Goal: mop(3, 2)

• Status: Active

• Foil status: Pursuable (from Executive)

• Cycle id: 2

Notice that in the posed question, the contrasting statuses are Active and Executive. In this case, Executive is converted to Pursuable, a previous case from Executive and directly subsequent to Active because the goal did not even became Pursuable so we have to know why this happened. Since the question requires a single time frame, which is cycle 2, the knowledge base for that cycle can be reconstituted using the execution history by adding and removing the beliefs for each cycle to the initial beliefs, until the desired cycle. We have that the relevant beliefs are:

• workshop(1, 1)

• dumpster(4, 1)

• $\lnot available(mop)$

• at(2, 2)

• have(battery, 35)

• $solid\_dirt(3,4)$

• $liquid\_dirt(3,2)$

• broken(mop)

Related Causes Procedure: First, it is necessary to evaluate the relation of the statuses. Be $st=active$ the factual status and $st'=pursuable$ the foil status such that $st \prec st'$, as such the relevant patterns are for “not advanced its status”.

For an active goal become pursuable, it is assessed in the evaluation stage, that is, the second stage of the BBGP model. In the case of evaluation stage, it uses only negative filters and the way of calculating this filter is: $\mathtt {d\_pre}(\texttt{causes} (\lnot mop(3,2))) \cap \mathcal {K}\mathcal {B}^+$.

For this scenario $\mathcal {K}\mathcal {B}^+=\mathcal {K}\mathcal {B}$ because there are no rules with only beliefs that may infer another beliefs. Thus, the calculation is as follows for the negative pattern:

First, the set of causes for (mop(3, 2) is calculated. For simplicity, we only write the set of beliefs that are part of $\texttt{COND}$:

$$\begin{aligned} \texttt{causes}(mop(3,2))={} & {} \{\{\lnot have(battery, 40), liquid\_dirt(3,2)\},\\ {} & {} \,\,\,\{have(cargo,80), liquid\_dirt(3,2)\}\\ {} & {} \,\,\,\{broken(mop), liquid\_dirt(3,2)\}\} \end{aligned}$$

Then, the disjunctive presumptions are calculated:

$$\begin{aligned}\mathtt {d\_pre}(\texttt{causes}(mop(3,2)))= {} & {} \{\lnot have(battery, 40),liquid\_dirt(3,2), \\ {} & {} \,\,\, have(cargo,80),broken(mop)\} \end{aligned}$$

Finally, we obtain the set of conditions:

$$\begin{aligned}\texttt{RC}= \mathtt {d\_pre}(\texttt{causes}(mop(3,2))) \cap \mathcal {K}\mathcal {B}^+= {} & {} \{\lnot have(battery, 40), liquid\_dirt(3,2),\\ {} & {} \,\,\, have(cargo,80),broken(mop)\} \end{aligned}$$

This concludes with the procedure whose output is the set of conditions $\texttt{RC}$, which constitute future explanations.

5 Related Work

In this section, some articles about contrastive explanations generation and explanations for goal selections are presented. Besides, we compare them with our proposal.

To the best of our knowledge, there are few works where contrastive explanations are used in the context of intelligent agents. In [13], the authors propose an approach for generating contrastive explanations for questions of the for “Why P and not Q?”, where P and Q are stand-ins for the current robot plan and the foil one, respectively. An algorithm for generating minimal explanations for users of different levels of expertise is presented. The result is that agents can explain its ongoing or planned behavior in a way that is both tailored to the user’s background and is designed to reduce cognitive burden on the user’s end. In [3], the authors propose an approach for generating contrastive explanations about agents’ preferences in the context of reinforcement learning. The explanations identify the state feature values that certain agents prefers compared to the other agent. In [12], the work focus on generating contrastive explanations in planning, more specifically, for questions such as “Why action A instead of action B?”. Thus, the contrastive explanation that compares properties of the original plan containing A against the contrastive plan containing B. We can notice that the contrastive questions tackled in these works have a different nature compared with the P-contrast question of our approach. In our approach, we consider an object of the explanation and a property about it, which gives one more dimension to the generated explanation.

Regarding explanation generation for goal selection. To the best of our knowledge, only [11] and [10] present an explanation method for goal selection. Their approach also considers BBGP-based agents and uses the formal argumentation base for generating explanations. However, there is no a social sciences grounding and it does not account for contrastive explanations. We can see our approach as a complement to these works.

6 Conclusions and Future Work

This article presented a method for generating P-contrastive explanations about the BBGP-based goal selection process. A set of beliefs – called conditions – is returned to the questioner and is the base for future explanations. Although, we present the phases and all the steps for generating a final explanation, the final procedure was not developed; however, all the building blocks for generating an explanations were established. We illustrate how our method works by applying it to the scenario of the cleaner world.

Let us recall that our method used Grice’s Cooperation Principles as a guide for how the agent should present its explanations, specifically, we base on the quantity category. This category is concerned with the amount of information provided: “be as informative as required, and no more than that”. It is subjective how informative a piece of information is. Not just in the sense that it depends on who receives the information, but it is also not measurable. In a sense, this is also part of the selection problem: how to select the most adequate answer?. We could say that the “most informative” explanations could be a solution to the problem. The method addresses this requirement by avoiding repetitive information, that is, if a condition $c_1$ can answer all the required events, there is no need to include a second condition.

As mentioned above, an important steps of the method were not tackled in this work. Thus, the immediate future work is to generate the set of possible explanations. Besides, we plan to study the selection problem, which focuses on deciding which single explanation will be returned as final answer.

As presented in Introduction, there are other types of contrastive question and explanations. We aim to generate the set of conditions and explanations for these types of explanations. Finally, another possible improvement is to include special questions that can better explain internal behaviors of the agent that are beyond the standard BBGP model. A good example would be a special question type to explain trust deliberation in an agent.