Situation Calculus

CS 475

March 12, 2001

1 Introduction

Situation calculus is an approach to reasoning about action and change (RAC). A fluent is a property of the world that changes over time. The world (at any particular time), called a situations, can be described by a set of fluents. To say that fluent F holds in the situation S, we write

Holds(F, S)  (1)

We will use a special constant S_{0} to denote the initial situation.

For example, in the wumpus world, we can represent the fact that the agent (we call him Agent) is at the square [1, 1], we can write

Holds(At(Agent, [1, 1]), S_{0})  (2)

Fluents only represent the properties of situations. Actions make the world changes from one situation to another situation. Actions have their effects such as the action of forward of the agent makes him being at the square [1, 2] etc. We write Do(a, s) to denote the situation resulting from the execution of action a in situation s. With this notation, we can represent the fact that moving forward (a = F orward) in a situation s will result in the agent being at the square [1, 2] as follows:

\begin{aligned}∀s[ \\ & Holds(At(Agent, [1, 1]), s) \\ & → Holds(At(Agent, [1, 2]), Do(F orward, s)) \\ ] \end{aligned}  (3)

Similarly, we can represent the effect of the action Turn right is that the agent will face East if he faces North before the action is executed as follows.

\begin{aligned}∀s [ \\ & Holds(F acing(Agent, North), s) \\ & → Holds(F acing(Agent, East), Do(T urn right, s)) \\ ] \end{aligned}  (4)

2 Representing the Wumpus World

In the wumpus world, each situation can be characterized by: • The position of the agent

• The location of the gold, the pit, and the wumpus

• The status of the agent (alive or dead)

• The stench of the wumpus that the agent can perceive (smell)

• The breeze of the pit that the agent can perceive (feel)

• The glitter of gold that the agent can perceive (see) In all, we have the following constants

• Agent – our agent

• Gold, P it1, P it2, P it3, and W umpus – that represents the gold, the pits, and the wumpus, respectively, and

• the locations [1, 1], [1, 2], …, [4, 4]

• Stench, Breeze, Glitter we can use the following fluents (predicates) to represent a situation

• At(x, y) – “object x is at the location y”, where x is an object (Agent, Gold, P it1, P it2, P it3, or W umpus), and y is a location

• Alive(Agent) – “Agent is alive” (¬Alive(Agent) – Agent is dead) – Whoelse could be dead?

• Has(z, y) – “location y has the property z”, where z ∈ {Stench, Breeze, Glitter}

and y is a location Some formulas about the properties of the world

1. In the square containing the wumpus and in the directly (not diagonally) adjacent squares the agent will perceive a stench – this can be represented by:

\begin{aligned}∀y, y′∀s[ \\ & Holds(At(W umpus, y), s) ∧ Adjacent(y, y′) \\ & → Holds(Has(y′, Stench), s) \\ ] \end{aligned}  (5)

where Adjacent(y, y′) is a predicate describing the relation between two squares y, y′ [What should we add? Should we write Holds(Adjacent(y, y′), s) or just Adjacent(y, y′)] 2. In the squares directly adjacent to a pit the agent will perceive a breeze

\begin{aligned}∀y, y′∀x∀s[ \\ & (x = P it1 ∨ x = P it2 ∨ x = P it3)∧ \\ & At(x, y) ∧ Adjacent(y, y′) \\ & → Holds(Has(y′, Breeze), s) \\ ] \end{aligned}  (6)

If we have a predicate called P it(x) or and a set of constants P it1, P it2, P it3, we can write the above equation by ∀y, y′∀x∀s[P it(x)∧At(x, y)∧Adjacent(y, y′) → Holds(Has(y′, Breeze), s)

3. Complete the formulization of the environment (page 154 ..)

So far, we have only represented the properties of the environment (the relationship between fluents). What is missing here is the representation of actions and their effects. Now, let see which actions are necessary for us. From the description of the environment, we see the following:

1. F orward – the agent moves forward

2. T urn right – the agent turns right

3. T urn left – the agent turns left

4. Shoot(y, y′) – the agent is in the square y and he shoots to the square y′

5. ???

Representing the effects of actions:

\begin{matrix}Holds(At(Agent, [n, m + 1]), Do(a, s)) ↔ \\ a = F orward ∧ Holds(At(Agent, [n, m]), s) ∧ Holds(F acing(North), s) \\ ∨ \\ Holds(At(Agent, [n, m + 1]), s) ∧ a ≠ F orward \end{matrix} (7)

But the agent can face ‘South’, ‘East’, and ‘West’ also ...

\begin{matrix}Holds(At(Agent, [n, m − 1]), Do(a, s)) ↔ \\ a = F orward ∧ Holds(At(Agent, [n, m]), s) ∧ Holds(F acing(South), s) \\ ∨ \\ Holds(At(Agent, [n, m − 1]), s) ∧ a ≠ F orward \end{matrix} (8)

\begin{matrix}Holds(At(Agent, [n + 1, m]), Do(a, s)) ↔ \\ a = F orward ∧ Holds(At(Agent, [n, m]), s) ∧ Holds(F acing(East), s) \\ ∨ \\ Holds(At(Agent, [n + 1, m]), s) ∧ a ≠ F orward \end{matrix} (9)

Holds(At(Agent, [n − 1, m]), Do(a, s)) ↔
a = F orward ∧ Holds(At(Agent, [n, m]), s) ∧ Holds(F acing(W est), s)
∨
Holds(At(Agent, [n − 1, m]), s) ∧ a ≠ F orward

(10)

Find out what is missing in the above equations? Can we rewrite (7)-(10) as one single axioms? For now, let continue with other actions:

\begin{matrix}Holds(F acing(North), Do(a, s)) ↔ \\ (a = T urn right ∧ Holds(F acing(W est), s)) \\ ∨ \\ (a = T urn left ∧ Holds(F acing(East), s)) \\ ∨ \\ (Holds(F acing(North), s) ∧ a ≠ T urn right ∧ a ≠ T urn left) \end{matrix}  (11)

Write the axioms for other directions!!!

Holds(Alive(W umpus), Do(a, s)) ↔
Holds(Alive(W umpus), s)∧

¬(a = Shoot(y, y′) ∧ Holds(At(Agent, y), s)) ∧ Holds(At(W umpus, y′), s)

(12)

Question: What do I miss in (12)?

Continue with other actions, effects, ...

What holds at the beginning?

\begin{matrix}Holds(Alive(Agent), S_{0}) \\ Holds(Alive(W umpus), S_{0}) \\ Holds(At(Agent, [1, 1]), S_{0}) \\ Holds(At(W umpus, [1, 3]), S_{0}) \\ Holds(At(P it1, [3, 1]), S_{0}) \\ Holds(At(P it2, [3, 3]), S_{0}) \\ Holds(At(P it3, [4, 4]), S_{0}) \\ Holds(At(Gold, [2, 3]), S_{0}) \\ Holds(F acing(North), S_{0}) \end{matrix}  (13)

HOMEWORK Complete the description of the wumpus world.

3 Situation Calculus

The theory in the previous section is a situation calculus theory. In short, a situation calculus theory consists of

• A set of fluents

• A set of actions

• Situations, with S_{0} is a constant denoting the initial situation

• Function Do that maps a pair of an action and a situation into a situation; Do(a, s) denotes the successor situation to s resulting from executing action a

• Predicate Holds whose first parameter is a fluent and second parameter is a situation

• A set of axioms about the initial situation (what is true/false in the initial situation? e.g. Holds(At(Agent, [1, 1]), S_{0}) etc.)

• A set of axioms that describes the effects of actions

• A set of axioms that describes the precondition of actions; for each action a, the theory consists of one formula of the form P oss(a, s) ↔ Holds(φ, s) where φ is a fluent formula.

Given a situation calculus theory (which is essential a set of first order axioms) – under certain conditions we can prove (or predict) what will be true/false after executing a sequence of actions in a situation. This is called the projection problem. We next discuss the assumptions needed in solving the projection problem. But first, for simplicity, let us leave the complicated ’Wumpus’ world and describe the ’Block’ world which will be used as the example for the discussion subsequently.

3.1 The Block World

Let us consider the block world in the following picture.

A C D  E B TABLE

Figure 1: A Block World

We assume that the world consists of the above blocks and an agent who can pick up block and put it down on the table or on top of other blocks which are clear.

Objects of the block world are the blocks, the table, and the agent. The blocks are described by their names, a,b,c,.... The table will be denoted by t. Since there is only one agent and no actions of the environment (for example, the agent will not die in this world - there is no ’Wumpus’ or ’Pit’!), we do not need to have a constant to denote the agent.

We will use the predicate Block(x) to denote that object x is a block.

The actions of the agents are:

1. P ickup(x) - pickup block x,

2. P ut(x, y) - put block x on top of block y, and

3. P ut(x, T) - put block x on the table.

The fluents of the block world are:

1. On(x, y) - block x is on the block y,

2. On(x, T) - block x is on the table,

3. Holding(x) - (the agent) is holding block x, and

4. Clear(x) - block x is clear.

We note that

Clear(x) ↔ ¬∀y(Block(y) ∧ ¬On(y, x))

which means that some properties of the world can be automatically deduced when we know about other properties.

The initial situation can be described by the following axioms

Holds(On(A, C), S_{0})  (14)
Holds(On(C, D), S_{0})  (15)
Holds(On(D, T), S_{0})  (16)
Holds(On(E, B), S_{0})  (17)
Holds(On(B, T), S_{0})  (18)

Now, let us describe the set of precondition axioms and the set of effect axioms.

Precondition axioms: Before an action is executed, it requires that some conditions are true. This condition is called the action’s precondition. For example, to put down a block x on top of block y, the agent must hold x and y must be clear. Another example is that to pick up a block x, the agent must not hold another block and x must be clear etc... We represent these by

P oss(P ickup(x), s) → ∀y(Holds(¬Holding(y), s)) ∧ Holds(Clear(x), s)(19)

P oss(P ut(x, y), s) → Holds(Holding(x), s) ∧ Holds(Clear(y), s) (20)
P oss(P ut(x, T), s) → Holds(Holding(x), s)  (21)

Problem The above formulas do not allow us to prove when an action, say P ickup(x), is possible. I.e., when P oss(P ickup(x), s) is true. Remember how to prove whether a literal is true?

Can we reverse the axioms (change → with ←) and then use them to prove when P oss(P ickup(x), s) is true? No, this will not work since it is not correct! There might be conditions which we have not considered? For example, the block can be too heavy, the agent might be tired etc... So, we need to specify all of these conditions, called qualifications. But there are infinitely many conditions like this?

The above is called the qualification problem. It calls for the need for nonmonotonic reasoning in AI.

To overcome the qualification problem, we make the assumption that the antecedent of the above formula represents the sufficient and necessary conditions for the action to be executed. The final action precondition axioms for the above three actions are:

P oss(P ickup(x), s) ↔ ∀y(Holds(¬Holding(y), s)) ∧ Holds(Clear(x), s)(22)

P oss(P ut(x, y), s) ↔ Holds(Holding(x), s) ∧ Holds(Clear(y), s) (23)
P oss(P ut(x, T), s) ↔ Holds(Holding(x), s)  (24)

We will use the above axioms instead of the axioms (19)-(21).

Given the above axioms and some axioms about the , we can now deduce whether the action P ickup(A) is possible in the initial situation, i.e., P oss(P ickup(A), S_{0}) is true. How?

Effect axioms: Actions have effects. For example, if the agent pickups a block he will hold it in the successor situation.

P oss(P ickup(x), s) → Holds(Holding(x), Do(P ickup(x), s))  (25)
P oss(P ut(x, y), s) → Holds(On(x, y), Do(P ut(x, y), s))  (26)
P oss(P ut(x, T), s) → Holds(On(x, T), Do(P ut(x, T), s))  (27)
P oss(P ickup(x), s) → [Holds(On(x, y), s)
→ Holds(¬On(x, y), Do(P ickup(x), s))]  (28)

But, the above equations only represent the ’positive effects’ of actions (what becomes true) and ’negative effects’ of actions (what becomes false) as well. In (28), for example, if the agent pickups block x which is on a block y in the situation s then On(x, y) will not hold in Do(P ickup(x), s); or if the agent puts block x on a block y in the situation s then Clear(y) will not hold in Do(P ut(x, y), s), etc. Try to complete the set of effect axioms.

Furthermore, there are fluents whose values do not change after the execution of an action. For example, if x is on y in situation s (Holds(On(x, y), s) is true) and the agent puts block z on top of x, then x is still on y in situation Do(P ut(z, x), s) (Holds(On(x, y), Do(P ut(z, x), s)) is true). Representing the unchange effects of actions require axioms of the form

Holds(On(x, y), s) → Holds(On(x, y), Do(P ickup(z), s))

These axioms are called frame axioms.

If we have n actions, m fluents, we would need 2 × n × m frame axioms. This is too many!!!

Representing frame axioms in a compact way is a challenge for quite sometime. To date, there are many solutions to the frame problems. Under reasonable assumptions, we can write, for each fluent f, one successor state axiom in the following form

Holds(f, Do(a, s)) ↔ γ_{f}^{+}(a, s) ∨ (Holds(f, s) ∧ γ_{f}^{−}(a, s))  (29)

where γ_{f}^{+}(a, s) (resp. γ_{f}^{−}(a, s)) summarizes the conditions for the fluent f to become true (resp. false).

For example, in the block world we have

Holds(On(x, y), Do(a, s)) ↔ γ_{On(x,y)}^{+}(a, s)∨(Holds(On(x, y), s)∧γ_{On(x,y)}^{−}(a, s)) (30) where γ_{On(x,y)}^{+}(a, s) = (a = P ut(x, y)) ∧ P oss(a, s) and γ_{On(x,y)}^{−}(a, s) = (a ≠ P ickup(x)) ∧ P oss(a, s). Complete the description!

3.2 Projection Problem

Let us denote the situation calculus theory for the block world by T . We can prove that

T ⊧ Holds(On(A, C), S_{0})
T ⊧ Holds(¬On(A, C), Do(P ickup(A), S_{0}))

In general, we can prove that

T ⊧ Holds(f, Do(a_{n}, …, Do(a_{1}, S_{0})))  (31)

for a fluent f and a sequence of actions a_{1}, …, a_{n}. We also write

T ⊧ Holds(f, Do(α, S_{0})))  (32)

α = [a_{1}, …, a_{n}].

3.3 Planning in Situation Calculus

The projection shows that we could formulate a planning task by asking for a sequence of actions (α) that makes the goal (say, a fluent f) true in the resulting situation of executing α in S_{0}. That is, for a planning task of achieving the goal φ (a fluent formula), we ask for a sequence of actions α such that

T ⊧ Holds(φ, Do(α, S_{0})))  (33)

Example 3.1 Find a sequence of actions that achieves the goal of having A on D. That is, we need to find α such that T ⊧ Holds(On(A, D), Do(α, S_{0})). It is easy to see that one of the possibility is α = [P ut(A, E), P ut(C, T), P ut(A, D)]. Why? Can you find another sequence of actions?

4 Planning in Situation Calculus

A C D  E B TABLE

Figure 2: A Block World

We present a slight different situation calculus theory for the block world in the above picture. It consists of:

1. Constants: A, B, C, D, E, T, two unary (domain) predicate Block and Location, where Block = {A, B, C, D, E} and Location = {A, B, C, D, E, T };

2. Actions: P ickup(x, y), P ut(x, y), and P ut(x, T) (Instead of P ickup(x) we use P ickup(x, y) – to say that the agent pickup the block x from a location y – which can be a block or the table);

3. Fluents: On(x, y), On(x, T), Holding(x), Clear(x).

4. The set of precondition axioms:

P oss(P ickup(x, y), s) ↔ x ≠ T ∧ Location(y) ∧
∀z(Holds(¬Holding(z), s)) ∧
Holds(Clear(x), s)  (34)
P oss(P ut(x, y), s) ↔ Block(x) ∧ Block(y) ∧ x ≠ y ∧
Holds(Holding(x), s) ∧
Holds(Clear(y), s)  (35)
P oss(P ut(x, T), s) ↔ Block(x) ∧ Holds(Holding(x), s) (36)

5. The set of effect axioms:

P oss(P ickup(x, y), s) → Holds(Holding(x), Do(P ickup(x, y), s))(37)
P oss(P ickup(x, y), s) → Holds(¬On(x, y), Do(P ickup(x, y), s)) (38)
P oss(P ickup(x, y), s) → [y ≠ T →
Holds(Clear(y), Do(P ickup(x, y), s))]  (39)
P oss(P ut(x, y), s) → Holds(On(x, y), Do(P ut(x, y), s))  (40)
P oss(P ut(x, y), s) → Holds(¬Holding(x), Do(P ut(x, y), s)) (41)
P oss(P ut(x, y), s) → Holds(¬Clear(y), Do(P ut(x, y), s)) (42)
P oss(P ut(x, T), s) → Holds(¬Holding(x), Do(P ut(x, T), s)) (43)
P oss(P ut(x, T), s) → Holds(On(x, T), Do(P ut(x, T), s)) (44)

6. The set of successor state axioms:

\begin{matrix}Holds(On(x, y), Do(a, s)) ↔ (a = P ut(x, y)) ∧ P oss(a, s) \\ ∨(Holds(On(x, y), s) ∧ ¬(a = P ickup(x, y) ∧ P oss(a, s))) \end{matrix}  (45)

\begin{matrix}Holds(On(x, T), Do(a, s)) ↔ (a = P ut(x, T)) ∧ P oss(a, s) \\ ∨(Holds(On(x, T), s) ∧ ¬(a = P ickup(x, T) ∧ P oss(a, s))) \end{matrix}  (46)

\begin{matrix}Holds(Holding(x), Do(a, s)) ↔ (a = P ickup(x, y)) ∧ P oss(a, s) \\ ∨(Holds(Holding(x), s)∧ \\ ¬((a = P ut(x, z) ∨ a = P ut(x, T)) ∧ P oss(a, s))) \end{matrix} (47)

\begin{matrix}Holds(Clear(x), Do(a, s)) ↔ (a = P ickup(y, x)) ∧ P oss(a, s) \\ ∨(Holds(Clear(x), s)∧ \\ ¬(a = P ut(y, x) ∧ P oss(a, s))) \end{matrix} (48)

What do I miss in the above formulas (45)-(48)?

7. Initial situation:

Holds(On(A, C), S_{0})  (49)
Holds(On(C, D), S_{0})  (50)
Holds(On(D, T), S_{0})  (51)
Holds(On(E, B), S_{0})  (52)
Holds(On(B, T), S_{0})  (53)

Let us denote the theory by T . Under the assumptions that

1. changes are caused by actions and

2. no thing happens before S_{0}

we can view each situation as a sequence of actions. Thus, we can prove what is true/false in each situation and hence we can formulate the planning task as follows.

Planning problem: Given a situation calculus T and a fluent formula φ. A plan achieving φ is a sequence of actions a_{1}, …, a_{n} such that

T ⊧ Holds(φ, Do(a_{n}, …, Do(a_{1}, S_{0})))

Example 4.1 [Planning in Block World]

A C D  E B Initial  E C D  A B Goal

Figure 3: A Planning Problem in Block World

Solving a planning problem means to finding a plan! How can we find a plan?

Generate and Test: The most simple way to find a plan. We can generate a sequence of actions, say α, and then test for T ⊧ Holds(φ, Do(α, S_{0})). So, we have the following algorithm (pseudo code for implementation):

Algorithm 4.1 (First Idea) Given are T and φ

main

while true do α = generate sequence action if testOK(α) return α endwhile

function

generate sequence action return a sequence of actions

function

testOK(α)

if T ⊧ Holds(φ, Do(a_{n}, …, Do(a_{1}, S_{0}))) return true else return false

Implementation problem: Checking if φ holds in Do(α, S_{0}), i.e., checking whether

(∗) T ⊧ Holds(φ, Do(α, S_{0}))

holds or not.

Another problem is how to represent the goal?

Answer: We can use a theorem prover, that tells us what is true in Do(α, S_{0})? to check (*). We can simplify our life (for now) by representing the goal by a set of fluents. Thus, the goal of the planning problem in Example 4.1 can be represented by the set

{On(A, B), On(C, D), On(D, T), On(E, C), On(B, T)}.

Running the algorithm: The set of actions: {P ut(x, y), P ut(x, T), P ickup(x, y) Iteration No. 1 Possible action in S_{0}: P ickup(A, C), P ickup(E, B) We then have two situation S_{1} = P ickup(A, C) and S_{2} = P ickup(E, B). In S_{1}, the following fluents are true: On(C, D), On(D, T), On(E, B), On(B, T), Holding(A) whereas On(A, C), On(C, D), On(D, T), On(B, T), Holding(E) are true in S_{2}. (Notice that we have not listed ALL the fluents that are true in each situation, i.e., this list is incomplete.) The following picture illustrates the different possible situations

Figure 4: A part of the situation tree of the block world