Termination w.r.t. Q of the given QTRS could be proven:

(0) Obligation:

Q restricted rewrite system:
The TRS R consists of the following rules:

f(a, f(b, x)) → f(a, f(a, f(a, x)))
f(b, f(a, x)) → f(b, f(b, f(b, x)))

Q is empty.

(1) DependencyPairsProof (EQUIVALENT transformation)

Using Dependency Pairs [AG00,LPAR04] we result in the following initial DP problem.

(2) Obligation:

Q DP problem:
The TRS P consists of the following rules:

F(a, f(b, x)) → F(a, f(a, f(a, x)))
F(a, f(b, x)) → F(a, f(a, x))
F(a, f(b, x)) → F(a, x)
F(b, f(a, x)) → F(b, f(b, f(b, x)))
F(b, f(a, x)) → F(b, f(b, x))
F(b, f(a, x)) → F(b, x)

The TRS R consists of the following rules:

f(a, f(b, x)) → f(a, f(a, f(a, x)))
f(b, f(a, x)) → f(b, f(b, f(b, x)))

Q is empty.
We have to consider all minimal (P,Q,R)-chains.

(3) DependencyGraphProof (EQUIVALENT transformation)

The approximation of the Dependency Graph [LPAR04,FROCOS05,EDGSTAR] contains 2 SCCs.

(4) Complex Obligation (AND)

(5) Obligation:

Q DP problem:
The TRS P consists of the following rules:

F(b, f(a, x)) → F(b, f(b, x))
F(b, f(a, x)) → F(b, f(b, f(b, x)))
F(b, f(a, x)) → F(b, x)

The TRS R consists of the following rules:

f(a, f(b, x)) → f(a, f(a, f(a, x)))
f(b, f(a, x)) → f(b, f(b, f(b, x)))

Q is empty.
We have to consider all minimal (P,Q,R)-chains.

(6) QDPOrderProof (EQUIVALENT transformation)

We use the reduction pair processor [LPAR04]. Here, we combined the reduction pair processor with the A-transformation [FROCOS05] which results in the following intermediate Q-DP Problem.
The a-transformed P is

b1(a(x)) → b1(b(x))
b1(a(x)) → b1(b(b(x)))
b1(a(x)) → b1(x)

The a-transformed usable rules are

b(a(x)) → b(b(b(x)))

The following pairs can be oriented strictly and are deleted.

F(b, f(a, x)) → F(b, f(b, x))
F(b, f(a, x)) → F(b, f(b, f(b, x)))
F(b, f(a, x)) → F(b, x)

The remaining pairs can at least be oriented weakly.
Used ordering: Combined order from the following AFS and order.
b1(x1)  =  b1(x1)
a(x1)  =  a(x1)
b(x1)  =  b

Recursive Path Order [RPO].
Precedence:

[b1₁, a₁, b]

The following usable rules [FROCOS05] were oriented:

f(b, f(a, x)) → f(b, f(b, f(b, x)))

(7) Obligation:

Q DP problem:
P is empty.
The TRS R consists of the following rules:

f(a, f(b, x)) → f(a, f(a, f(a, x)))
f(b, f(a, x)) → f(b, f(b, f(b, x)))

Q is empty.
We have to consider all minimal (P,Q,R)-chains.

(8) PisEmptyProof (EQUIVALENT transformation)

The TRS P is empty. Hence, there is no (P,Q,R) chain.

(9) TRUE

(10) Obligation:

Q DP problem:
The TRS P consists of the following rules:

F(a, f(b, x)) → F(a, f(a, x))
F(a, f(b, x)) → F(a, f(a, f(a, x)))
F(a, f(b, x)) → F(a, x)

The TRS R consists of the following rules:

f(a, f(b, x)) → f(a, f(a, f(a, x)))
f(b, f(a, x)) → f(b, f(b, f(b, x)))

Q is empty.
We have to consider all minimal (P,Q,R)-chains.

(11) QDPOrderProof (EQUIVALENT transformation)

We use the reduction pair processor [LPAR04]. Here, we combined the reduction pair processor with the A-transformation [FROCOS05] which results in the following intermediate Q-DP Problem.
The a-transformed P is

a1(b(x)) → a1(a(x))
a1(b(x)) → a1(a(a(x)))
a1(b(x)) → a1(x)

The a-transformed usable rules are

a(b(x)) → a(a(a(x)))

The following pairs can be oriented strictly and are deleted.

F(a, f(b, x)) → F(a, f(a, x))
F(a, f(b, x)) → F(a, f(a, f(a, x)))
F(a, f(b, x)) → F(a, x)

The remaining pairs can at least be oriented weakly.
Used ordering: Combined order from the following AFS and order.
a1(x1)  =  a1(x1)
b(x1)  =  b(x1)
a(x1)  =  a

Recursive Path Order [RPO].
Precedence:

[a1₁, b₁, a]

The following usable rules [FROCOS05] were oriented:

f(a, f(b, x)) → f(a, f(a, f(a, x)))

(12) Obligation:

Q DP problem:
P is empty.
The TRS R consists of the following rules:

f(a, f(b, x)) → f(a, f(a, f(a, x)))
f(b, f(a, x)) → f(b, f(b, f(b, x)))

Q is empty.
We have to consider all minimal (P,Q,R)-chains.

(13) PisEmptyProof (EQUIVALENT transformation)

The TRS P is empty. Hence, there is no (P,Q,R) chain.

(14) TRUE