Towards the trace formula for convex-cocompact groups

Ulrich Bunke^{∗}and Martin Olbrich^{†}

March 9, 2000

Contents

1 Introduction 1

2 The distribution Ψ 3 2.1 Invariant distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 An invariant distribution asociated to Γ . . . . . . . . . . . . . . . . . . . . . . . 4 2.3 The Fourier inversion formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.4 The Fouriertransform of Ψ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.5 The distribution Ψ as a regularized trace . . . . . . . . . . . . . . . . . . . . . . . 5

3 The Plancherel theorem and integral kernels 7 3.1 The Plancherel theorems for L^{2}(G) and L^{2}(Γ\G). Support of Plancherel measures 7 3.2 Extension of R and R_{Γ} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.3 The absolute continuous part of L^{2}(Γ\G). Integral kernels for Ř_{Γ}(h). . . . . . . . 10

4 Poisson transforms and asymptotic computations 12 4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4.2 Poisson transformation, c-functions, and asymptotics . . . . . . . . . . . . . . . . 12 4.3 An estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.4 A computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 5 The Fourier transform of Ψ 24

∗Mathematisches Institut, Universität Göttingen, Bunsenstr. 3-5, 37073 Göttingen, GERMANY, Email:bunke@uni-math.gwdg.de

†Mathematisches Institut, Universität Göttingen, Bunsenstr. 3-5, 37073 Göttingen, GERMANY, Email:olbrich@uni-math.gwdg.de

5.1 The contribution of the scattering matrix . . . . . . . . . . . . . . . . . . . . . . 24 5.2 The Fourier transform of Ψ. A conjecture. . . . . . . . . . . . . . . . . . . . . . . 28

6 Resolvent kernels and Selberg zeta functions 31 6.1 Meromorphic continuation of resovent kernels . . . . . . . . . . . . . . . . . . . . 31 6.2 Finite propagation speed estimates . . . . . . . . . . . . . . . . . . . . . . . . . . 34 6.3 Boundary values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 6.4 The regularized trace of the resolvent . . . . . . . . . . . . . . . . . . . . . . . . . 39 6.5 A functional equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 6.6 Selberg zeta functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 6.7 Integrality of N_{Γ}(π) for discrete series representations . . . . . . . . . . . . . . . 42

1 Introduction

In this paper we develop a part of the harmonic analysis associated with a convex cocompact subgroup Γ of a semisimple Lie group G of real rank one that could play the same role as the trace formula in the case of cocompact groups or groups of finite covolume. In these classical situations a smooth, compactly supported, and K-finite function f on G acts by right convolution R_{Γ}(f) on the Hilbert space L^{2}(Γ\G). The trace formula is an expression of the trace of the restriction of this operator to the discrete subspace in terms of f and its Fourier transform f̂. The part involving f is called the geometric side and usually written as a sum of orbital integrals. The Fourier transform enters the trace formula in the case of non-cocompact subgroups, where it is combined with the scattering matrix. We will call this part the contribution of the scattering matrix.

In the present paper we assume that Γ is a convex cocompact subgroup of G. Let X be the symmetric space of G and ∂X be its geodesic boundary. If Γ ⊂ G is a discrete torsion-free subgroup, then there is a Γ-equivariant decomposition ∂X = Ω ∪ Λ, where Λ is the limit set of Γ. We call Γ convex-cocompact if Γ\X ∪ Ω is a compact manifold with boundary.

Since the discrete spectrum of L^{2}(Γ\G) is rather sparse we take the point of view that the the contribution of the scattering matrix is essentially (up to the contribution of the discrete spectrum) the Fourier transform of the geometric side of the trace formula.

Thus our starting point is the geometric side. It is a distribution Ψ on G given as a sum of suitably normalized orbital integrals associated to the hyperbolic conjugacy classes of Γ (see Subsection 2.2). The objective of the trace formula in the case of convex cocompact Γ is an explicit expression for the Fourier transform of Ψ. We are looking for a ”measure” Φ on the unitary dual Ĝ such that

Ψ(f) = ∫Ĝ θ_{π}(f)Φ(dπ) ,  (1)

where θ_{π}(f) := Tr f̂(π) is the character of π. In the present paper we will formulate a precise Conjecture 5.2 about Φ, but we are not able to prove the formula (1) in the general case.

The unitary dual Ĝ has a natural topology. Now observe that the intersection of the support of f̂and the support of the Plancherel measure of L^{2}(Γ\G) is the spectrum of R_{Γ}(f). The Fourier transform of a compactly supported function f is never compactly supported. In order to do our computations we have to approximate R_{Γ}(f) by operators which have compact spectrum. The missing piece is some estimate which eventually allows for dropping the cut-off. Conjecture 5.2 can easily be verified in the case of a negative critical exponent.

In the present paper we will prove a formula which is similar to (1), but where Ψ has a different interpretation. Let R(f) denote the right-convolution operator on L^{2}(G) induced by f. Then both, R(f) and R_{Γ}(f), have smooth integral kernels K_{R(f)}, K_{R_{Γ}(f)}, and, by Lemma 2.2, the value Ψ(f) is nothing else than the integral

Ψ′(f) := ∫_{Γ\G}(K_{R_{Γ}(f)}(g, g) − K_{R(f)}(g, g))μ_{G}(dg) .

We will show that Ψ′ can be applied to functions with compactly-supported Fourier transform, and our main Theorem 5.1 is a formula

Ψ′(f) = ∫Ĝ θ_{π}(f)Φ(dπ) .  (2)

together with an explicit expression for Φ.

We will describe Φ in terms of multiplicities N_{Γ}(π) and a density L_{Γ}(π). If π is a representation of the complementary series (a non-tempered unitary representation of G), then the integer N_{Γ}(π) := Φ({π}) is the multiplicity of π in L^{2}(Γ\G). If π is a discrete series representation, then by Proposition 6.14 N_{Γ}(π) := Φ({π}) is an integer. It is an interesting problem to study this number in detail.

There are interesting operators with non-compact spectrum to which Ψ′ can be applied. Let K ⊂ G be a maximal compact subgroup and Ω be the Casimir operator of G. We fix a K-type γ and consider (z − Ω)^{−1} on L^{2}(Γ\G)(γ) and L^{2}(G)(γ) if z is not in the spectrum. Let K_{Γ}(z) and K(z) denote the corresponding integral kernels. The difference K_{Γ}(z) − K(z) is smooth on the diagonal and goes into Ψ′ if Re(z) ≪ 0. The consideration of these operators provides the link between the continuous part of Φ and the Selberg zeta functions. In fact, using the analysis of the present paper we can show the meromorphic continuation of the logarithmic derivative and the functional equation (Theorem 6.12) of the Selberg zeta functions (By other methods we apriori know that the Selberg zeta functions are meromorphic, see below.). Our work extends previous results [6] in the two-dimensional case and [11] in the spherical case of G = SO(1, n), n ≥ 2.

Using symbolic dynamics of the geodesic flow and the thermodynamic formalism one can show that the Selberg zeta functions themselves are meromorphic functions of finite order [11]. This gives information about the growth of Φ and on the counting function of resonances. It in particular shows that Φ can be applied to Schwartz functions like the Fourier transform f̂ of a K-finite smooth function f of compact support on G.

2 The distribution Ψ

2.1 Invariant distributions

Let G be a semisimple Lie group. We fix once and for all a Haar measure μ_{G} on G. In this subsection we describe two sorts of conjugation invariant distributions on G, namely orbital integrals and characters of irreducible representations.

Let γ ∈ G be a semisimple element. The orbit O_{γ} := {gγg^{−1} | g ∈ G} of γ under conjugation by G is a submanifold of G which can be identified with G_{γ}\G, where G_{γ} denotes the centralizer of γ. The inclusion i_{γ} : G_{γ}\G ≅ O_{γ} ↪ G is a proper map. Therefore the pull-back by i_{γ} is a continuous map

i_{γ}^{∗} : C_{c}^{∞}(G) → C_{c}^{∞}(G_{γ}\G) .

If we choose a Haar measure μ_{G_{γ}} on G_{γ}, then we obtain an induced measure μ_{G_{γ}\G} on G_{γ}\G such that

∫_{G} f(g)μ_{G}(dg) = ∫_{G_{γ}\G} ∫_{G_{γ}} f(hg)μ_{G_{γ}}(dh)μ_{G_{γ}\G}(dg) .

The orbital integral θ_{γ} associated to γ and the choice of the Haar measure μ_{G_{γ}} is, by definition, the composition of i_{γ}^{∗} and the measure μ_{G_{γ}\G}, i.e.

θ_{γ}(f) := μ_{G_{γ}\G} ∘ i_{γ}^{∗}(f) = ∫_{G_{γ}\G} f(gγg^{−1})μ_{G_{γ}\G}(dg) .

We now introduce the character θ_{π} associated to an irreducible admissible representation π of G on a Hilbert space V_{π}. If f ∈ C_{c}^{∞}(G), then

π(f) := ∫_{G} f(g)π(g)μ_{G}(dg)

is a trace class operator on V_{π}. The character θ_{π} is the distribution on G given by

θ_{π}(f) := Tr π(f) .

2.2 An invariant distribution asociated to Γ

Let G be a semisimple linear connected Lie group of real rank one. We consider a torsion-free discrete convex-cocompact, non-cocompact subgroup Γ ⊂ G (see [3], Sec. 2). Let Õ_{Γ} denote the disjoint union of manifolds G_{γ}\G, where γ runs over a set C̃Γ of representatives of the set CΓ \ {[1]} of non-trivial conjugacy classes of Γ :

Õ_{Γ} = ⋃_{γ∈C̃Γ} G_{γ}\G .

The natural map i_{Γ} : Õ_{Γ} → G is proper, and we obtain a continuous map

i_{Γ}^{∗} : C_{c}^{∞}(G) → C_{c}^{∞}(Õ_{Γ}) .

For each γ ∈ C̃Γ we fix a Haar measure μ_{G_{γ}}. Then we define a measure μ_{Γ} on Õ_{Γ} such that its restriction to G_{γ}\G is vol(Γ_{γ}\G_{γ})μ_{G_{γ}\G}. Note that this measure only depends on the Haar measure μ_{G} and not on the choices of μ_{G_{γ}}.

Definition 2.1 The geometric side of the trace formula is the distribution Ψ on G given by

Ψ := μ_{Γ} ∘ i_{Γ}^{∗} .

In terms of orbital integrals we can write

Ψ(f) = ∑_{γ∈C̃Γ} vol(Γ_{γ}\G_{γ})θ_{γ}(f) .

Note that this distribution is in fact a measure, invariant under conjugation, and it only depends on Γ and the Haar measure μ_{G}.

2.3 The Fourier inversion formula

Let Ĝ denote the unitary dual of G. This is the set of equivalence classes of irreducible unitary representations of G equipped with a natural structure of a measurable space. For π ∈ Ĝ the operator π(f) is, by definition, the value of the Fourier tranform of f at π, which we will also denote by f̂(π).

It is a consequence of the Plancherel theorem for G that there is a measure p on Ĝ such that for any f ∈ C_{c}^{∞}(G) and g ∈ G we have

f(g) = ∫Ĝ Tr π(g)^{−1}f̂(π) p(dπ) .

Note that p(dπ) depends on the choice of the Haar measure μ_{G}. Later in the present paper we will state a more precise version of the Plancherel theorem.

If h is a function on Ĝ such that h(π) is a trace class operator on V_{π} for almost all π (mod. p), then we form the function

ȟ(g) := ∫Ĝ Tr π(g)^{−1}h(π) p(dπ)

if the integral exists for all g ∈ G.

2.4 The Fouriertransform of Ψ

The contents of a trace formula for convex-cocompact groups Γ would be an expression of Ψ(f) in terms of the Fourier transform f̂. In other words, we are interested in the Fourier transform of the distribution Ψ. Since Ψ is invariant this expression should only involve the characters θ_{π}(f) = Trf̂(π). Thus there should exist a certain measure Φ on Ĝ such that the following equality holds true for all f ∈ C_{c}^{∞}(G):

Ψ(f) = ∫Ĝ θ_{π}(f)Φ(dπ) .

Note that there is a Paley-Wiener theorem for G which characterizes the range of the Fourier transform as a certain Paley-Wiener space. Apriori, Φ is a functional on this Paley-Wiener space, and it would be a non-trivial statement that this functional is in fact induced by a measure on Ĝ.

2.5 The distribution Ψ as a regularized trace

In the present paper we will not compute the Fourier transform Φ of Ψ in the sense of Subsection 2.4. Rather we will compute the candidate for Φ using a different interpretation of Ψ.

Let R denote the right-regular representation of G on L^{2}(G). It extends to the convolution algebra L^{1}(G) by the formula

R(f) = ∫_{G} f(g)R(g)μ_{G}(dg) .

If f ∈ C_{c}^{∞}(G), then R(f) is an integral operator with smooth integral kernel K_{R(f)}(g, h) = f(g^{−1}h). In a similar manner we have an unitary right-regular representation R_{Γ} of G on the Hilbert space L^{2}(Γ\G) which can be extended to L^{1}(G) using the formula

R_{Γ}(f) = ∫_{G} f(g)R_{Γ}(g)μ_{G}(dg) .

If f ∈ C_{c}^{∞}(G), then R_{Γ}(f) is an integral operator with smooth kernel

K_{R_{Γ}(f)}(g, h) = ∑_{γ∈G} f(g^{−1}γh) .

Indeed, for ϕ ∈ L^{2}(Γ\G) we have

R(f)ϕ(g) = ∫_{G} ϕ(gh)f(h)μ_{G}(dh)
= ∫_{G} ϕ(h)f(g^{−1}h)μ_{G}(dh)
= ∫_{Γ\G} ∑_{γ∈Γ} ϕ(h)f(g^{−1}γh)μ_{G}(dh) .

Lemma 2.2 For f ∈ C_{c}^{∞}(G) we have

Ψ(f) = ∫_{Γ\G}[K_{R_{Γ}(f)}(Γg, Γg) − K_{R(f)}(g, g)]μ_{G}(dg) .

Proof. We compute

Ψ(f) = ∑_{γ∈C̃Γ} vol(Γ_{γ}\G_{γ})θ_{γ}(f)
= ∑_{γ∈C̃Γ} vol(Γ_{γ}\G_{γ}) ∫_{G_{γ}\G} f(g^{−1}γg)μ_{G_{γ}\G}(dg)
= ∑_{γ∈C̃Γ} ∫_{Γ_{γ}\G} f(g^{−1}γg)μ_{G}(dg)
= ∫_{Γ\G} ∑_{γ∈C̃Γ} ∑_{h∈Γ_{γ}\Γ} f(g^{−1}h^{−1}γhg)μ_{G}(dg)
= ∫_{Γ\G} ∑_{1≠γ∈Γ} f(g^{−1}γg)μ_{G}(dg)
= ∫_{Γ\G}[K_{R_{Γ}(f)}(g, g) − K_{R(f)}(g, g)]μ_{G}(dg) .
□

The expression of Ψ(f) in terms of the integral kernels of R(f) and R_{Γ}(f) can be used to define Ψ on other classes of functions or even on certain distributions.

Using the Plancherel theorems for L^{2}(G) and L^{2}(Γ\G) the right-regular representations R and R_{Γ} can be extended. If f is K-finite and f̂ is smooth and has compact support, then we will see that g ↦ [K_{R_{Γ}(f)}(g, g) − K_{R(f)}(g, g)] belongs to L^{1}(Γ\G), and thus

Ψ′(f) := ∫_{Γ\G}[K_{R_{Γ}(f)}(g, g) − K_{R(f)}(g, g)]μ_{G}(dg)

is well-defined. The main result of the present paper is an expression of Ψ′(f) in terms of f̂ for those functions.

As mentioned in the introduction we are going to apply Ψ′ to the difference of distribution kernels of the resolvents (z − Ω)^{−1} of the Casimir operator restricted to a K-type of L^{2}(Γ\G) and L^{2}(G), respectively. In this example the single kernels are not smooth, but their difference is so on the diagonal of Γ\G. Strictly speaking, the integral defining Ψ′ exists for Re(z) ≪ 0. For other values of z we introduce a truncated version Ψ′_{R}, R > 0, and we define the value of Ψ′ as the constant term of the asymptotic expansion of Ψ′_{R} as R → ∞. It seems to be an interesting problem to characterize the functions of Ω (restricted to a K-type) with the property that Ψ′_{R} (applied to the corresponding distribution kernels) admits such an asymptotic expansion.

Given a discrete series representation π of G we can consider the corresponding isotypic components of L^{2}(G) and L^{2}(Γ\G). If we further consider a K-type of π, then the projections onto these components have smooth integral kernels. As a byproduct of the investigation of the resolvents we can show that Ψ′ can be applied to these integral kernels and that its values are integers.

3 The Plancherel theorem and integral kernels

3.1 The Plancherel theorems for L^{2}(G) and L^{2}(Γ\G). Support of Plancherel measures

We start with describing the rough structure of the unitary dual Ĝ. First there is a countable family of square integrable unitary representations, the discrete series Ĝ_{d}. The discretely decomposable subspace L^{2}(G)_{d} ⊂ L^{2}(G) is composed out of these representations each occuring with infinite multiplicity.

The orthogonal complement L^{2}(G)_{ac} of L^{2}(G)_{d} is given by a countable direct sum of direct integrals of unitary principal series representations. We are going to describe their parametrization. Let G = KAN be an Iwasawa decomposition of G. The abelian group A is isomorphic to the multiplicative group ℝ^{+}. Let 𝔞 and 𝔫 denote the Lie algebras of A and N. Then dim_{ℝ}(𝔞) = 1 and the roots of (𝔞, 𝔫), fix an order on 𝔞. Let M = Z_{K}(A) denote the centralizer of A in K. The unitary principal series representations π^{σ,λ} of G are parametrized by the set (σ, λ) ∈ M̂ × i𝔞^{∗}. Let W denote the Weyl group N_{K}(A)/M, where N_{K}(A) denotes the normalizer of A in K. It is isomorphic to ℤ_{2}, and we can choose a representative of the non-trivial element w ∈ N_{K}(A) such that w^{−1} = w. One knows that π^{σ,λ} is equivalent to π^{σ^{w},−λ}, where σ^{w} denotes the Weyl conjugate representation of σ given by σ^{w}(m) := σ(m^{w}). For λ ≠ 0 the representation π^{σ,λ} is irreducible. If σ is equivalent to σ^{w}, i.e. σ is Weyl invariant, then it may happen that π^{σ,0} is reducible. In this case it decomposes into a sum π^{σ,+} ⊕ π^{σ,−} of limits of discrete series representations.

The set of equivalence classes of unitary representations of G which we have listed above is the set of tempered representations. We refer to Sec. 8 of [3] for a discussion of the notion of temperedness for L^{2}(G) and L^{2}(Γ\G).

The Plancherel theorem for L^{2}(G) is, of course, explicitly known for a long time [7]. The Plancherel measure p is supported on the set of tempered representations (compare [1]). In particular, it is absolutely continuous with respect to the Lebesgue measure on i𝔞. Thus we can neglect the point λ = 0. Then L^{2}(G)_{ac} decomposes as a direct integral of unitary principal series representations over M̂ × i𝔞_{+}^{∗} with infinite multiplicity, and the Plancherel measure has full support. Note that the multiplicity space of the representation π can be realized as V_{π}^{∗}.

By Ĝ_{ac} we denote the set of irreducible unitary principal series representations π^{σ,λ}, λ ≠ 0. The remaining unitary representations Ĝ_{c} = Ĝ \ (Ĝ_{d} ∪ Ĝ_{ac}) can be realized as subspaces of principal series representations π^{σ,λ} with λ ∈ 𝔞_{+}^{∗} ∪ {0}. The case of limits of discrete series Ĝ_{ld} (in this case λ = 0) was mentioned above. The representations with parameter λ > 0 are not tempered and belong to the complementary series Ĝ_{cs}.

In [3] we studied the Plancherel theorem for L^{2}(Γ\G). Let us recall its rough structure. The support of the corresponding Plancherel measure p_{Γ} is the union of Ĝ_{d}, Ĝ_{ac}, and a countable subset of Ĝ_{c}. L^{2}(Γ\G) decomposes into sum of subspaces L^{2}(Γ\G)_{cusp}, L^{2}(Γ\G)_{ac}, and L^{2}(Γ\G)_{c}. Here L^{2}(Γ\G)_{cusp} is discretely decomposable into representations of the discrete series, each occuring with infinite multiplicity, L^{2}(Γ\G)_{c} is discretely decomposable into representations belonging to Ĝ_{c}, each occuring with finite multiplicity, and L^{2}(Γ\G)_{ac} is a direct integral of unitary principal series representations with infinite multiplicity over the parameter set M̂ × i𝔞_{+}^{∗}. On this set the Plancherel measure p_{Γ} is absolutely continuous to the Lebesgue measure and has full support. The multiplicity space M_{π} can be realized as a subspace of the Γ-invariant distribution vectors of Vπ̃, i.e., M_{π} ⊂ ^{Γ}V_{π̃,−∞}, where π̃ denotes the dual representation of π. For π ∈ Ĝ_{ac} we are going to describe M_{π} explicitly in Subsection 3.3.

3.2 Extension of R and R_{Γ}

The Plancherel theorem for G provides a G-equivariant unitary equivalence

U : L^{2}(G) →^{∼} ∫Ĝ V_{π}^{∗} ˆ⊗V_{π} p(dπ) ,  (3)

where G acts on L^{2}(G) by the right-regular representation R, and the action on the direct integral is given by g ↦ {π ↦ id_{V_{π}^{∗}} ⊗ π(g)}. We can identify V_{π}^{∗} ˆ⊗V_{π} with the space of Hilbert-Schmidt operators on V_{π}. For ϕ ∈ C_{c}^{∞}(G) we set

U(ϕ) := {π ↦ ˆ ϕ̃(π)} ,

where ϕ̃(g) := ϕ(g^{−1}). This fixes the normalization of the Plancherel measure p.

3.2 Extension of R and R_{Γ} 9

The inverse transformation maps the family π ↦ h(π) to the function

U^{−1}(h)(g) = ∫Ĝ Tr π(g)h(π)p(dπ) .

If f ∈ L^{1}(G), than R(f) is given by g ↦ {π ↦ id_{V_{π}^{∗}} ⊗ f̂(π)}.

The Plancherel theorem for Γ\G provides a G-equivariant unitary equivalence

U_{Γ} : L^{2}(Γ\G) →^{∼} ∫Ĝ M_{π} ˆ⊗V_{π} p_{Γ}(dπ) ,  (4)

where G acts on L^{2}(Γ\G) by the right-regular representation R_{Γ}, and the representation of G on the direct integral is given by g ↦ {π ↦ id_{M_{π}} ⊗ π(g)}. Again, if f ∈ L^{1}(G), than R_{Γ}(f) is given by g ↦ {π ↦ id_{M_{π}} ⊗ f̂(π)}. In order to write down an explicit formula for U_{Γ} we first identify M_{π}^{∗} with Mπ̃ and embed M_{π} ˆ⊗V_{π} into Hom(Mπ̃, V_{π}). For ϕ ∈ C_{c}^{∞}(Γ\G) we define

U_{Γ}(ϕ)(π) := {Mπ̃ ∋ v ↦ ∫_{Γ\G} ϕ(g)π(g^{−1})v μ_{G}(dg) ∈ V_{π}} .

This fixes the normalization of p_{Γ}.

Let now h be a function on supp(p) such that h(π) is a bounded operator on V_{π}. If h is essentially bounded (and measurable in the appropriate sense), then it acts on the direct integral (3) by π ↦ {id_{V_{π}^{∗}} ⊗ h(π)} and thus defines a bounded operator Ř(h) on L^{2}(G) commuting with the left-regular action of G.

In a similar manner, if h is a function on supp(p_{Γ}) such that h(π) is a bounded operator on V_{π}, and h is essentially bounded, then it acts on the direct integral (4) by π ↦ {id_{M_{π}} ⊗ h(π)} and thus defines a bounded operator Ř_{Γ}(h) on L^{2}(Γ\G).

Let us now assume that the h(π) are of trace-class, and that ∫Ĝ ∥h(π)∥_{1} p(dπ) is finite, where ∥.∥_{1} denotes the trace norm ∥A∥_{1} = Tr |A| for a trace class operator A on V_{π}. Then Ř(h) is an integral operator with integral kernel K_{Ř(h)}(g, k) = ȟ(g^{−1}k). Indeed, for ϕ ∈ C_{c}^{∞}(G) we have

(Ř(h)ϕ)(g) = ∫Ĝ Tr π(g)h(π)ˆϕ̃(π)p(dπ)
= ∫Ĝ Tr ∫_{G} π(g)h(π)π(k^{−1})ϕ(k)μ_{G}(dk)p(dπ)
= ∫_{G} ∫Ĝ Tr π(k^{−1}g)h(π)p(dπ)ϕ(k)μ_{G}(dk)
= ∫_{G} ȟ(g^{−1}k)ϕ(k)μ_{G}(dk) .

We are looking for a similar formula for the integral kernel of Ř_{Γ}(h) in Subsection 3.3.

3.3 The absolute continuous part of L^{2}(Γ\G). Integral kernels for Ř_{Γ}(h).

In this subsection we describe in detail the Plancherel decomposition of L^{2}(Γ\G)_{ac}. The goal is to exhibit a class of functions π ↦ h(π) with the property that Ř_{Γ}(h) is an integral operator.

Let P = MAN be a fixed parabolic subgroup. If (σ, λ) ∈ M̂ × 𝔞_{ℂ}^{∗}, then we define the representation σ_{λ} of P by σ_{λ}(man) := σ(m)a^{ρ−λ}, where ρ ∈ 𝔞^{∗} is given by 2ρ(H) = tr ad(H)_{|𝔫}, H ∈ 𝔞, and for λ ∈ 𝔞_{ℂ}^{∗} and a = exp(H) ∈ A we put a^{λ} = e^{λ(H)}. We realize the principal series representation H^{σ,λ} := V_{π^{σ,λ}} as the subspace of C^{−∞}(G ×_{P} V_{σ_{λ}}) of functions such that f_{|K} ∈ L^{2}(K ×_{M} V_{σ}). Then H_{±∞}^{σ,λ} = C^{±∞}(G×_{P} V_{σ_{λ}}) are the spaces of smooth (resp. distribution) vectors of π^{σ,λ}. By restriction to K we obtain canonical isomorphisms H^{σ,λ} ≅ L^{2}(K ×_{M} V_{σ}). It therefore makes sense to speak of smooth functions f on i𝔞^{∗} such that f(λ) ∈ H^{σ,λ}.

Note that ∂X can be identified with G/P. Let G/P = Ω ∪ Λ be the Γ-equivariant decomposition of the space G/P into the (open) domain of discontinuity Ω and the (closed) limit set Λ. As a convex-cocompct subgroup Γ acts freely and cocompactly on Ω. We put B := Γ\Ω. Furthermore, we define the bundle V_{B}(σ_{λ}) → B by V_{B}(σ_{λ}) := Γ\(G ×_{P} V_{σ_{λ}})_{|Ω}. If λ ∈ i𝔞^{∗}, then we have a natural Hilbert space L^{2}(B, V_{B}(σ_{λ})). Again, fixing a volume form on B we obtain identifications of the spaces L^{2}(B, V_{B}(σ_{λ})) with the fixed space L^{2}(B, V_{B}(σ_{0})) so that it makes sense to speak of smooth functions f on i𝔞^{∗} such that f(λ) ∈ L^{2}(B, V_{B}(σ_{λ})). We refer to [3], Sec.3, for more details.

In [3] we defined a family of extension maps ext : L^{2}(B, V_{B}(σ_{λ})) → ^{Γ}H_{−∞}^{σ,λ}. For λ ∈ i𝔞^{∗}, λ ≠ 0, the extension map provides an explicit identification of the space of multiplicities M_{π^{σ,λ}} ⊂ ΓH_{−∞}^{σ̃,−λ} with L^{2}(B, V_{B}(σ̃_{−λ})).

The Plancherel measures p and p_{Γ} on \{σ\} × i𝔞_{+}^{∗} are given by \frac{dim(V_{σ})}{2πω_{X}}p_{σ}(λ)dλ, where p_{σ} is a smooth symmetric function on i𝔞^{∗} of polynomial growth (see [3], Lemma 5.5. (3)), and ω_{X} := lim_{a→∞} a^{−2ρ}vol_{G/K}(KaK) (see [3], Sec. 11). Note that dλ is the real Lebesgue measure on i𝔞.

We now describe the embedding

U_{Γ}^{−1} : \frac{dim(V_{σ})}{2πω_{X}} ∫_{\{σ\}×i𝔞_{+}^{∗}} L^{2}(B, V_{B}(σ̃_{−λ})) ⊗ H^{σ,λ}p_{σ}(λ)dλ → L^{2}(Γ\G)_{ac} .

If v ⊗ w ∈ L^{2}(B, V_{B}(σ̃_{−λ})) ⊗ H_{∞}^{σ,λ}, then we define < v ⊗ w >:= ⟨ext(v), w⟩. Let ϕ be a smooth function of compact support on i𝔞_{+}^{∗} ∪{0} such that ϕ(λ) ∈ L^{2}(B, V_{B}(σ̃_{−λ}))⊗H_{∞}^{σ,λ}, then we have

U_{Γ}^{−1}(ϕ)(g) = \frac{dim(V_{σ})}{2πω_{X}} ∫_{i𝔞_{+}^{∗}} < (1 ⊗ π^{σ,λ}(g))ϕ(λ) > p_{σ}(λ)dλ .

Note that ext may be singular at λ = 0. In this case it has a first-order pole and p_{σ}(0) = 0 (see [3], Prop. 7.4) such that the integral is still well-defined.

3.3 The absolute continuous part of L^{2}(Γ\G). Integral kernels for Ř_{Γ}(h). 11

We now fix a K-type γ ∈ K̂. Let H^{σ,λ}(γ) denote the γ-isotypic component of H^{σ,λ}. By Frobenius reciprocity we have a canonical identification

H^{σ,λ}(γ) =^{α} V_{γ} ⊗ Hom_{K}(V_{γ}, H^{σ,λ}) =^{1⊗β} V_{γ} ⊗ Hom_{M}(V_{γ}, V_{σ}) .  (5)

Here α^{−1}(v ⊗ U) := U(v) and β(U)(v) := U(v)(1). Any operator A ∈ End(V_{γ} ⊗ Hom_{M}(V_{γ}, V_{σ})) gives rise to a finite-dimensional operator F (A) ∈ End(H^{σ,λ}) which is trivial on the orthogonal complement of H^{σ,λ}(γ).

Let q be a smooth function of compact support on M̂ × i𝔞^{∗} such that q(σ, λ) ∈ End(V_{γ} ⊗ Hom_{M}(V_{γ}, V_{σ})). We call q symmetric if it is compatible with the equivalences J_{σ,λ}^{w} : H^{σ,λ} → H^{σ^{w},−λ}, i.e. if F (q(σ^{w}, −λ)) = J_{σ,λ}^{w} ∘ F (q(σ, λ)) ∘ (J_{σ,λ}^{w})^{−1}. If q is symmetric, then we can define the function h_{q} on Ĝ such that h_{q}(π) ∈ End(V_{π}) by h_{q}(π^{σ,λ}) := F (q(σ, λ)) for (σ, λ) ∈ M̂ × i𝔞^{∗}, λ ≠ 0, and by h_{q}(π) = 0 for all other representations.

Let π_{∗} : H_{∞}^{σ,λ} → L^{2}(B, V_{B}(σ_{λ})) denote the push-down map which can be considered here as the adjoint of the extension ext : L^{2}(B, V_{B}(σ̃_{−λ})) → H_{−∞}^{σ̃,−λ}. The composition

π^{σ,λ}(g)h_{q}(π^{σ,λ})π^{σ,λ}(k^{−1})ext π_{∗}

is a finite-dimensional map from H_{∞}^{σ,λ} to H_{∞}^{σ,λ}. It is therefore nuclear and has a well-defined trace.

Lemma 3.1 The operator Ř_{Γ}(h_{q}) has a smooth integral kernel given by

K_{Ř_{Γ}(h_{q})}(g, k) = ∑_{σ∈M̂} \frac{dim(V_{σ})}{4πω_{X}} ∫_{i𝔞^{∗}} Tr π^{σ,λ}(g)h_{q}(π^{σ,λ})π^{σ,λ}(k^{−1})ext ∘ π_{∗} p_{σ}(λ)dλ .

Proof. First of all note that the integral is well defined at λ = 0. If ext ∘ π_{∗} is singular at this point, then it has a pole of at most second order. But then the Plancherel density vanishes at least of second order, too.

Let ϕ ∈ C_{c}^{∞}(Γ\G). In the Plancherel decomposition it is represented by the function π ↦ U_{Γ}(ϕ)(π) ∈ M_{π} ˆ⊗V_{π}. We fix λ for a moment and choose orthonormal bases {v_{i}} of L^{2}(B, V_{B}(σ̃_{−λ})) and {w_{i}} of H^{σ,λ} consisting of smooth sections. Furthermore, let {v^{i}} and {w^{j}} be dual bases of L^{2}(B, V_{B}(σ_{λ})) and H^{σ̃,−λ}, respectively. Then we have

U_{Γ}(ϕ)(π^{σ,λ}) = ∑_{i,j} ∫_{Γ\G} < v^{i} ⊗ π^{σ̃,−λ}(k)w^{j} > ϕ(k)μ_{G}(dk) v_{i} ⊗ w_{j} .

We have

Ř_{Γ}(h_{q})(ϕ)(g) = \frac{dim(V_{σ})}{4πω_{X}} ∑_{σ∈M̂} ∫_{i𝔞^{∗}} < (1 ⊗ π^{σ,λ}(g)h_{q}(π^{σ,λ}))U_{Γ}(ϕ)(π^{σ,λ}) > p_{σ}(λ)dλ . (6)

We now compute

< (1 ⊗ π^{σ,λ}(g)h_{q}(π^{σ,λ}))U_{Γ}(ϕ)(π^{σ,λ}) >
= ∑_{i,j} ∫_{G} < v^{i} ⊗ π^{σ̃,−λ}(k)w^{j} > ϕ(k)μ_{G}(dk) < (1 ⊗ π^{σ,λ}(g)h_{q}(π^{σ,λ}))v_{i} ⊗ w_{j} >
= ∫_{G} ∑_{i,j} < v^{i} ⊗ w^{j} > ϕ(k) < (1 ⊗ π^{σ,λ}(g)h_{q}(π^{σ,λ})π^{σ,λ}(k^{−1}))v_{i} ⊗ w_{j} > μ_{G}(dk)
= ∫_{G} ∑_{i,j} ϕ(k)⟨v^{i}, π_{∗}w^{j}⟩⟨ext(v_{i}), π^{σ,λ}(g)h_{q}(π^{σ,λ})π^{σ,λ}(k^{−1})w_{j}⟩μ_{G}(dk)
= ∫_{G} ∑_{j} ϕ(k)⟨ext ∘ π_{∗}(w^{j}), π^{σ,λ}(g)h_{q}(π^{σ,λ})π^{σ,λ}(k^{−1})w_{j}⟩μ_{G}(dk)
= ∫_{G} ∑_{j,l} ϕ(k)⟨ext ∘ π_{∗}(w^{j}), w_{l}⟩⟨w^{l}, π^{σ,λ}(g)h_{q}(π^{σ,λ})π^{σ,λ}(k^{−1})w_{j}⟩μ_{G}(dk)
= ∫_{G} ∑_{l} ϕ(k)⟨w^{l}, π^{σ,λ}(g)h_{q}(π^{σ,λ})π^{σ,λ}(k^{−1})ext ∘ π_{∗}w_{l}⟩μ_{G}(dk)
= ∫_{G} ϕ(k)Tr π^{σ,λ}(g)h_{q}(π^{σ,λ})π^{σ,λ}(k^{−1})ext ∘ π_{∗} μ_{G}(dk) .

Inserting this computation into (6), the we obtain the desired formula for the integral kernel of Ř_{Γ}(h_{q}). □

4 Poisson transforms and asymptotic computations

4.1 Motivation

Let q be symmetric and define and h_{q} as in subsection 3.3. We want to show that the function g ↦ [K_{Ř_{Γ}(h_{q})}(g, g) − K_{Ř(h_{q})}(g, g)] belongs to L^{1}(Γ\G). It follows that

Ψ′( ˜ȟ_{q}) = ∫_{Γ\G}[K_{Ř_{Γ}(h_{q})}(Γg, Γg) − K_{Ř(h_{q})}(g, g)]μ_{G}(dg)

is well-defined, and we are asking for an expression of Ψ′( ˜ȟ_{q}) in terms of q, respectively h_{q}.

In the present section we show related results using the language of Poisson transformations. In Subsection 5.1 we will provide the translation of these results and solve the problems above.

4.2 Poisson transformation, c-functions, and asymptotics

We fix a K-type γ and a M-type σ. Let T ∈ Hom_{M}(V_{σ}, V_{γ}) and λ ∈ 𝔞_{ℂ}^{∗}. If w ∈ Vγ̃, then by Frobenius reciprocity we consider w ⊗ T^{∗} as an element of H^{σ̃,−λ}(γ̃) which is given by the 4.2 Poisson transformation, c-functions, and asymptotics 13

function k ↦ T^{∗}(γ̃(k^{−1})w) under the canonical identification ϕ_{−λ} : H_{∞}^{σ̃,−λ} →^{∼} C^{∞}(K ×_{M} Vσ̃). We further put Φ_{λ,μ} := ϕ_{λ}^{−1} ∘ ϕ_{μ}. We will also use the notation Φ_{0,λ} for ϕ_{λ}.

The Poisson transformation

P_{λ}^{T} : H_{−∞}^{σ,λ} → C^{∞}(G ×_{K} V_{γ})

is a G-equivariant map which is defined by the relation

⟨w, P_{λ}^{T} (ψ)(g)⟩ = ⟨w ⊗ T^{∗}, π^{σ,λ}(g^{−1})ψ⟩ ,

for all ψ ∈ H_{−∞}^{σ,λ}, w ∈ Vγ̃.

For the definition of the function c_{σ} we refer to [3], Sec. 5. We have the relation

c_{σ}(λ)cσ̃(−λ) = p_{σ}(λ)^{−1} .

It turns out to be useful to introduce the normalized Poisson transformation^{0}P_{λ}^{T} := c_{σ}(−λ)^{−1}P_{λ}^{T}. We introduce the family of operators

P_{λ,a}^{T} : H_{−∞}^{σ,λ} → C^{∞}(K ×_{M} V_{γ}) , a ∈ A_{+}

by

P_{λ,a}^{T}(f)(k) := ^{0}P_{λ}^{T} (f)(ka) .

In order to discuss the asymptotic behaviour of P_{λ,a}^{T} as a → ∞ we need the normalized Knapp-Stein intertwining operators

J_{σ,λ}^{w} : H_{−∞}^{σ,λ} → H_{−∞}^{σ^{w},−λ} .

Note that J_{σ,λ}^{w} ∘ J_{σ^{w},−λ}^{w} = id. We again refer to [3], Sec. 5, for more details. The following is a consequence of [3], Lemma 6.2. Let α ∈ 𝔞^{∗} denote the short root of (𝔞, 𝔫). For λ ∈ i𝔞^{∗} we have

P_{λ,a}^{T} = a^{λ−ρ} \frac{c_{γ}(λ)}{c_{σ}(−λ)}T ∘ Φ_{0,λ} + a^{−λ−ρ}γ(w)T ∘ Φ_{0,−λ} ∘ J_{σ,λ}^{w} + a^{−ρ−α}R_{−∞}(λ, a) ,  (7)

where R_{−∞}(λ, a) ∘ Φ_{λ,0} remains bounded in C^{∞}(i𝔞^{∗}, Hom(H_{−∞}^{σ,0}, C^{−∞}(K ×_{M} V_{γ}))) as a → ∞. Multiplication by T (resp. γ(w)T) is here considered as a map from H_{−∞}^{σ,0} (resp. H_{−∞}^{σ^{w},0}) to C^{−∞}(K ×_{M} V_{γ}) in the natural way. If χ, χ̃ are smooth functions on K/M with disjoint support, then χR_{−∞}(λ, a)χ̃ ∘ Φ_{λ,0} remains bounded in C^{∞}(i𝔞^{∗}, Hom(H_{−∞}^{σ,0}, C^{∞}(K ×_{M} V_{γ}))) as a → ∞.

4.3 An estimate

In order to formulate the result appropriately we introduce the following space C_{Γ}(G) of functions on G. For each compact V ⊂ Ω and integer N we consider the seminorm

|ϕ|_{V,N} := sup_{kah∈V A_{+}M}(1 + | log(a)|)^{N}a^{2ρ}|ϕ(kah)| , ϕ ∈ C(G) .

Here we consider V as a subset of K using the identification G/P = K/M. We define the Fréchet space C_{Γ}(G) as the space of all continuous functions ϕ on G such that |ϕ|_{V,N} < ∞ for all compact V ⊂ Ω and N ∈ ℕ. If ϕ is Γ-invariant and belongs to C_{Γ}(G), then clearly ϕ ∈ L^{1}(Γ\G).

Now let T ∈ Hom_{M}(V_{σ}, V_{γ}), R ∈ Hom_{M}(Vσ̃, Vγ̃), and q ∈ C_{c}^{∞}(i𝔞^{∗}). Then we can define the operator

A_{q} = A_{q}(T, R) := ∫_{i𝔞^{∗}} ^{0}P_{λ}^{T} ∘(ext ∘ π_{∗} − 1)∘(^{0}P_{−λ}^{R})^{∗}q(λ)dλ ∈ Hom(C_{c}^{−∞}(G×_{K}V_{γ}), C^{∞}(G×_{K}V_{γ})) .

This operator has a smooth integral kernel (g, h) ↦ A_{q}(g, h) ∈ End(V_{γ}). The main result of the present subsection is the following estimate.

Proposition 4.1

|A_{q}(g, g)| ∈ C_{Γ}(G) .

Proof. Note, that we only have to show finiteness of the norms |.|_{V,N}, where V ⊂ Ω is compact and has the additional property that V is contained in the interior of a compact subset V_{1} ⊂ Ω satisfying γV_{1} ∩ V_{1} = ∅ for all 1 ≠ γ ∈ Γ. Indeed, any seminorm of C_{Γ}(G) can be majorized by the maximum of finite number of these special ones.

We choose a smooth cut-off function χ̃ on Ω such that supp(χ̃) ⊂ V_{1} and supp(1−χ̃)∩V = ∅. We further choose a compact V_{2} containing V in its interior and being contained in the interior of V_{1}, and a cut-off function χ on Ω such that supp(χ) ⊂ V_{2} and supp(1 − χ) ∩ V = ∅.

Then we can write for k ∈ V

0P_{λ}^{T} ∘ (ext ∘ π_{∗} − 1) ∘ (^{0}P_{−λ}^{R})^{∗}(kah, kah)
= γ(h)^{−1}χ(k) ∘ [P_{λ,a}^{T} ∘ (ext ∘ π_{∗} − 1) ∘ (P_{−λ,a}^{R})^{∗}](k, k) ∘ χ(k)γ(h) .  (8)

In order to employ the off-diagonal localization of the Poisson transformation we write

χ ∘ P_{λ,a}^{T} ∘ (ext ∘ π_{∗} − 1) ∘ (P_{−λ,a}^{R})^{∗} ∘ χ
= χ ∘ P_{λ,a}^{T} ∘ (1 − χ̃) ∘ (ext ∘ π_{∗} − 1) ∘ χ̃ ∘ (P_{−λ,a}^{R})^{∗} ∘ χ  (9)
+χ ∘ P_{λ,a}^{T} ∘ (ext ∘ π_{∗} − 1) ∘ (1 − χ̃) ∘ (P_{−λ,a}^{R})^{∗} ∘ χ .

In (9) we could insert the factor (1 − χ̃) since χ̃ ∘ (ext ∘ π_{∗} − 1) ∘ χ̃ = 0. Using that supp(χ) ∩ supp(1 − χ̃) = ∅ we have

χ ∘ P_{λ,a}^{T} ∘ (1 − χ̃) = a^{−λ−ρ}χ ∘ γ(w)T ∘ Φ_{0,−λ} ∘ J_{σ,λ}^{w} ∘ (1 − χ̃) + a^{−ρ−α}R_{∞}(λ, a) ,

where R_{∞}(λ, a) ∘ Φ_{λ,0} remains bounded in C^{∞}(i𝔞^{∗}, Hom(H_{−∞}^{σ,0}, C^{∞}(K ×_{M} V_{γ}))) as a → ∞. We obtain

χ ∘ P_{λ,a}^{T} ∘ (ext ∘ π_{∗} − 1) ∘ (P_{−λ,a}^{R})^{∗} ∘ χ =

a^{−2ρ}a^{−2λ}χ ∘ γ(w)T ∘ Φ_{0,−λ} ∘ J_{σ,λ}^{w} ∘ (1 − χ̃) ∘ (ext ∘ π_{∗} − 1) ∘ χ̃ ∘ Φ_{λ,0} ∘ R^{∗} \frac{c_{γ}(−λ)^{∗}}{c_{σ}(λ)} ∘ χ (10)

+a^{−2ρ}χ ∘ γ(w)T ∘ Φ_{0,−λ} ∘ J_{σ,λ}^{w} ∘ (1 − χ̃) ∘ (ext ∘ π_{∗} − 1) ∘ χ̃ ∘ (J_{σ̃,−λ}^{w})^{∗} ∘ Φ_{−λ,0} ∘ R^{∗}γ̃(w)^{∗} ∘ χ (11)

+a^{−2ρ}a^{2λ}χ ∘ \frac{c_{γ}(λ)}{c_{σ}(−λ)}T ∘ Φ_{0,λ} ∘ (ext ∘ π_{∗} − 1) ∘ (1 − χ̃) ∘ (J_{σ̃,−λ}^{w})^{∗} ∘ Φ_{−λ,0} ∘ R^{∗}γ̃(w)^{∗} ∘ χ (12)

+a^{−2ρ}χ ∘ γ(w)T ∘ Φ_{0,−λ} ∘ J_{σ,λ}^{w} ∘ (ext ∘ π_{∗} − 1) ∘ (1 − χ̃) ∘ (J_{σ̃,−λ}^{w})^{∗} ∘ Φ_{−λ,0} ∘ R^{∗}γ̃(w)^{∗} ∘ χ(13)

+Q(λ, a) ,

where a^{2ρ+α}Q(λ, a) remains bounded in C^{∞}(i𝔞^{∗}, Hom(C^{−∞}(K ×_{M} V_{γ}), C^{∞}(K ×_{M} V_{γ}))) as a → ∞.

We further compute using that the intertwining operators commute with ext ∘ π_{∗} (compare the proof of Lemma 4.2 for a similar argument), the functional equation of the intertwining operators, and χ ∘ (ext ∘ π_{∗} − 1) ∘ χ = 0

(11) + (13)

= a^{−2ρ}χ ∘ γ(w)T ∘ Φ_{0,−λ} ∘ J_{σ,λ}^{w} ∘ (ext ∘ π_{∗} − 1) ∘ χ̃ ∘ (J_{σ̃,−λ}^{w})^{∗} ∘ Φ_{−λ,0} ∘ R^{∗}γ̃(w)^{∗} ∘ χ

+a^{−2ρ}χ ∘ γ(w)T ∘ Φ_{0,−λ} ∘ J_{σ,λ}^{w} ∘ (ext ∘ π_{∗} − 1) ∘ (1 − χ̃) ∘ (J_{σ̃,−λ}^{w})^{∗} ∘ Φ_{−λ,0} ∘ R^{∗}γ̃(w)^{∗} ∘ χ

= a^{−2ρ}χ ∘ γ(w)T ∘ Φ_{0,−λ} ∘ (ext ∘ π_{∗} − 1) ∘ J_{σ,λ}^{w} ∘ (J_{σ̃,−λ}^{w})^{∗} ∘ Φ_{−λ,0} ∘ R^{∗}γ̃(w)^{∗} ∘ χ
= a^{−2ρ}χ ∘ γ(w)T ∘ Φ_{0,−λ} ∘ (ext ∘ π_{∗} − 1) ∘ Φ_{−λ,0} ∘ R^{∗}γ̃(w)^{∗} ∘ χ
= a^{−2ρ}γ(w)T ∘ Φ_{0,−λ} ∘ χ ∘ (ext ∘ π_{∗} − 1) ∘ χ ∘ Φ_{−λ,0} ∘ R^{∗}γ̃(w)^{∗}
= 0 .

Note that in (10) and (12) one of the intertwining operators is localized off-diagonally. We conclude that the following families of operators

χ ∘ γ(w)T ∘ Φ_{0,−λ} ∘ J_{σ,λ}^{w} ∘ (1 − χ̃) ∘ (ext ∘ π_{∗} − 1) ∘ χ̃ ∘ Φ_{λ,0} ∘ R^{∗} \frac{c_{γ}(−λ)}{c_{σ}(λ)} ∘ χ

χ ∘ \frac{c_{γ}(λ)}{c_{σ}(−λ)}T ∘ Φ_{0,λ} ∘ (ext ∘ π_{∗} − 1) ∘ (1 − χ̃) ∘ (J_{σ̃,−λ}^{w})^{∗} ∘ Φ_{−λ,0} ∘ R^{∗}γ̃(w)^{∗} ∘ χ

belong to C^{∞}(i𝔞^{∗}, Hom(C^{−∞}(K ×_{M} V_{γ}), C^{∞}(K ×_{M} V_{γ}))). Restricting the smooth distribution kernel to the diagonal, multiplying by the smooth compactly supported function q, and integrating over i𝔞^{∗} we obtain the following estimates using the standard theory of the Euclidean Fourier transform. For any N ∈ ℕ

sup_{k∈K} sup_{a∈A_{+}} | ∫_{i𝔞^{∗}} Q(λ, a)(k, k)q(λ)dλ|a^{2ρ}(1 + | log(a)|)^{N} < ∞

sup_{k∈K} sup_{a∈A_{+}}

| ∫_{i𝔞^{∗}} χ(k)^{2}[γ(w)T ∘ Φ_{0,−λ} ∘ J_{σ,λ}^{w} ∘ (1 − χ̃) ∘ (ext ∘ π_{∗} − 1) ∘ Φ_{λ,0} ∘ R^{∗} \frac{c_{γ}(−λ)}{c_{σ}(λ)}](k, k)q(λ)a^{−2λ}dλ|

(1 + | log(a)|)^{N} < ∞

sup_{k∈K} sup_{a∈A_{+}}

| ∫_{i𝔞^{∗}} χ(k)^{2}[\frac{c_{γ}(λ)}{c_{σ}(−λ)}T ∘ Φ_{0,λ} ∘ (ext ∘ π_{∗} − 1) ∘ (1 − χ̃) ∘ (J_{σ̃,−λ}^{w})^{∗} ∘ Φ_{−λ,0} ∘ R^{∗}γ̃(w)^{∗}](k, k)q(λ)a^{2λ}dλ|

(1 + | log(a)|)^{N} < ∞ .

This implies the proposition. □

\underline{Remark :} We have shown in fact that q ↦ |(A_{q})_{|diag}| is a continuous map from C_{c}^{∞}(i𝔞^{∗}) to L^{1}(Γ\G). It would be desirable to extend this map from C_{c}^{∞}(i𝔞^{∗}) to the Schwartz space S(i𝔞^{∗}). It is this technical problem that prevents us to prove that the Fouriertransform Φ of Ψ restricted to the unitary principal series representations is a tempered distribution. If this would true then it is in fact a measure and given by our computations below.

If we would like to show that the map q ↦ |(A_{q})_{|diag}| extends to a map from the Schwartz space to L^{1}(Γ\G) along the lines above we need estimates on the growth of ext as the parameter λ tends to infinity along the imaginary axis. If the imaginary axis is in the domain of convergence of ext, i.e. the critical exponent δ_{Γ} of Γ is negative, then such an estimate is easy to obtain. In the general case one has to estimate the meromorphic continuation of ext, and this is an open problem.

4.4 A computation

In this subsection we want to express ∫_{Γ\G} tr A_{q}(g, g)μ_{G}(dg) in terms of q.

Recall that the symmetric space X = G/K can be compactified by adjoining the boundary ∂X = G/P. As a convex-cocompact group Γ acts freely and properly on X ∪ Ω with compact quotient. Therefore, we can choose a smooth function χ^{Γ} ∈ C_{c}^{∞}(X∪Ω) such that ∑_{γ∈Γ} γ^{∗}χ^{Γ} ≡ 1 on X ∪ Ω. The restriction of χ^{Γ} to X can be lifted to G as a right-K-invariant function which we still denote by χ^{Γ}. We denote by χ_{∞}^{Γ} the right-M-invariant lift to K of the restriction of χ^{Γ} to ∂X = K/M. We write

∫_{Γ\G} tr A_{q}(g, g)μ_{G}(dg) = ∫_{G} χ^{Γ}(g)tr A_{q}(g, g)μ_{G}(dg) .

Let χ_{U} be the characteristic function of the ball B_{U} in X of radius U centered at the origin [K]. Again, we denote its right-K-invariant lift to G by the same symbol. Then we can write

∫_{G} χ^{Γ}(g)tr A_{q}(g, g)μ_{G}(dg) = lim_{U→∞} ∫_{G} χ^{Γ}(g)χ_{U}(g)tr A_{q}(g, g)μ_{G}(dg) .

Given U we fix a function χ_{1} ∈ C_{c}^{∞}(G/K) such that

χ_{1}χ_{U}χ^{Γ} = χ_{U}χ^{Γ} .  (14)

The operator χ_{U}χ^{Γ}^{0}P_{λ}^{T} ∘ (ext ∘ π_{∗} − 1) ∘ (^{0}P_{−λ}^{R})^{∗}χ_{1} has a an integral kernel of compact support. Since the kernel is smooth in the interior of the support it is of trace class. We can write

∫_{G} χ^{Γ}(g)χ_{U}(g)tr A_{q}(g, g)μ_{G}(dg) = ∫_{i𝔞^{∗}} Tr [χ_{U}χ^{Γ}^{0}P_{λ}^{T} ∘ (ext ∘ π_{∗} − 1) ∘ (^{0}P_{−λ}^{R})^{∗}χ_{1}]q(λ)dλ .

Note that

i𝔞^{∗} ∋ λ ↦ Tr [χ_{U}χ^{Γ}^{0}P_{λ}^{T} ∘ (ext ∘ π_{∗} − 1) ∘ (^{0}P_{−λ}^{R})^{∗}χ_{1}]

is a smooth function. We want to compute its limit in the sense of distributions as U → ∞ using Green’s formula.

Note that V_{B}(1_{ρ+α}) is a complex bundle with a real structure which is trivial together with this structure. Indeed, B is orientable, and V_{B}(1_{ρ+α}) is a real power of Λ^{max}T^{∗}B. We choose any non-vanishing positive section ϕ ∈ C^{∞}(B, V_{B}(1_{ρ+α})). For any z ∈ ℂ we can form ϕ^{z} ∈ C^{∞}(B, V_{B}(1_{ρ+zα})). In particular, if we choose z such that zα = λ−μ, then multiplication by ϕ^{z} gives an isomorphism Φ̄_{λ,μ} : C^{∞}(B, V_{B}(σ_{μ})) → C^{∞}(B, V_{B}(σ_{λ})), and similar isomorphisms of the spaces of L^{2}- and distribution sections. If Re(z) = 0, then ext : C^{∞}(B, V_{B}(1_{ρ+zα})) → ^{Γ}H_{−∞}^{1,ρ+zα} is regular (indeed ρ + zα belongs to the domain of convergence). Multiplication by ext(ϕ^{z}) gives a continuous map

Φ̄_{λ,μ} : H_{∞}^{σ,μ} → H_{−∞}^{σ,λ} .

This map is Γ-equivariant and extends in fact to larger subspaces of H_{−∞}^{σ,μ} of distributions which are smooth on neighbourhoods of the limit set Λ.

The ususal trick to bring in Green’s formula is to write

Tr [χ_{U}χ^{Γ}^{0}P_{λ}^{T} ∘ (ext ∘ π_{∗} − 1) ∘ (^{0}P_{−λ}^{R})^{∗}χ_{1}]
= lim_{μ→λ} Tr [χ_{U}χ^{Γ}^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}χ_{1}] .

Let ∇^{γ} denote the invariant connection of the bundle V (γ) = G ×_{K} V_{γ} over X and Δ_{γ} := −(∇^{γ})^{∗}∇^{γ} be the Laplace operator. Then there exists a constant c ∈ ℝ such that (Δ_{γ} + c + λ^{2}) ∘ P_{λ}^{T} = 0. Let n denote the outer unit-normal vector field at B_{U}.

By Green’s formula we have for λ ≠ ±μ

Tr [χ_{U}χ^{Γ}^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}χ_{1}]

= \frac{1}{λ^{2} − μ^{2}} Tr [Δ_{γ}, χ^{Γ}]χ_{U} ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}χ_{1}  (15)
−\frac{1}{λ^{2} − μ^{2}} Tr χ_{|∂B_{U}}^{Γ} ∘ (^{0}P_{λ}^{T})_{|∂B_{U}} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (∇_{n}^{γ}^{0}P_{−μ}^{R})_{|∂B_{U}}^{∗} (16)
+\frac{1}{λ^{2} − μ^{2}} Tr χ_{|∂B_{U}}^{Γ} ∘ (∇_{n}^{γ}^{0}P_{λ}^{T})_{|∂B_{U}} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})_{|∂B_{U}}^{∗} . (17)

Note that the derivatives of χ_{1} drop out because of (14). Moreover, (^{0}P_{λ}^{T})_{|∂B_{U}} : H_{−∞}^{σ,λ} → C^{∞}(∂B_{U}, V (γ)_{|∂B_{U}}) denotes the composition of the Poisson transform and restriction to the boundary of B_{U}, and this operator can be expressed in terms of P_{λ,a}^{T}.

We introduce the following notation. Let a_{U} ∈ A_{+} be such that dist_{X}(a_{U}K, K) = U. We define ω(U) := a_{U}^{−2ρ}vol(∂B_{U}). Note that ω_{X} := lim_{U→∞} ω(U) exists. Let χ_{a}^{Γ} ∈ C^{∞}(K) denote the function k ↦ χ^{Γ}(ka). Note that lim_{a→∞} χ_{a} = χ_{∞}. Then we can write

(16) + (17)

= −\frac{ω_{U}a_{U}^{2ρ}}{λ^{2} − μ^{2}} Tr χ_{a_{U}}^{Γ} ∘ P_{λ,a_{U}}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (∂P_{−μ,a_{U}}^{R})^{∗}

+\frac{ω_{U}a_{U}^{2ρ}}{λ^{2} − μ^{2}} Tr χ_{a_{U}}^{Γ} ∘ ∂P_{λ,a_{U}}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (P_{−μ,a_{U}}^{R})^{∗} ,

Here ∂P_{λ,a}^{T} stands for the derivative of the function a ↦ P_{λ,a}^{T} with respect to the positive fundamental unit vector field on A.

Let χ be a smooth cut-off function on X ∪ Ω of compact support such that

γsupp(χ) ∩ supp(χ) = ∅ , ∀γ ∈ Γ , γ ≠ 1 .  (18)

Note that χ^{Γ} can be decomposed into a finite sum χ^{Γ} = ∑_{i} χ^{i} such that each χ^{i} satisfies (18). We fix a cut-off function χ̃ on Ω satisfying (18) and χ̃ ≡ 1 on a neighbourhood of supp(χ) ∩ Ω. We further define χ_{a}(k) = χ(kaK), χ_{∞}(k) = χ_{|∂X}(kM) and observe that |χ_{a} − χ_{∞}| = O(a^{−α}) for any seminorm |.| of C^{∞}(K). Using that χ̃ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) χ̃ = 0 we can write

(16) + (17) (χ^{Γ} replaced by χ)
= −\frac{ω_{U}a_{U}^{2ρ}}{λ^{2} − μ^{2}} Tr χ_{a_{U}} ∘ P_{λ,a_{U}}^{T} ∘ (1 − χ̃) ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (∂P_{−μ,a_{U}}^{R})^{∗}

−\frac{ω_{U}a_{U}^{2ρ}}{λ^{2} − μ^{2}} Tr χ_{a_{U}} ∘ P_{λ,a_{U}}^{T} ∘ χ̃ ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (1 − χ̃) ∘ (∂P_{−μ,a_{U}}^{R})^{∗}

+\frac{ω_{U}a_{U}^{2ρ}}{λ^{2} − μ^{2}} Tr χ_{a_{U}} ∘ ∂P_{λ,a_{U}}^{T} ∘ (1 − χ̃) ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (P_{−μ,a_{U}}^{R})^{∗}

+\frac{ω_{U}a_{U}^{2ρ}}{λ^{2} − μ^{2}} Tr χ_{a_{U}} ∘ ∂P_{λ,a_{U}}^{T} ∘ χ̃ ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (1 − χ̃) ∘ (P_{−μ,a_{U}}^{R})^{∗} . (19)

We now insert the asymptotic decomposition (7) of the operators P_{λ,a}^{T} as a → ∞ noting that in each line one of these operators is localized off-diagonally. In order to stay in trace class operators we choose a function χ_{1} ∈ C^{∞}(K) such that supp(1 − χ_{1}) ∩ supp(χ) = ∅ and supp(1 − χ̃) ∩ supp(χ_{1}) = ∅. We obtain

(16) + (17) (χ^{Γ} replaced by χ)

= −\frac{ω_{U}a_{U}^{−μ−λ}(−μ − ρ)}{λ^{2} − μ^{2}}Tr R^{∗} \frac{cγ̃(−μ)^{∗}}{c_{σ}(μ)} γ(w)T χ_{∞}Φ_{0,−λ}J_{σ,λ}^{w}(1 − χ̃) (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) Φ_{μ,0}χ_{1}

−\frac{ω_{U}a_{U}^{μ−λ}(μ − ρ)}{λ^{2} − μ^{2}}Tr R^{∗}T χ_{∞}Φ_{0,−λ}J_{σ,λ}^{w}(1 − χ̃) (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) (J_{σ̃,−μ}^{w})^{∗}Φ_{−μ,0}χ_{1}

−\frac{ω_{U}a_{U}^{μ+λ}(μ − ρ)}{λ^{2} − μ^{2}}Tr R^{∗}γ(w)^{−1} \frac{c_{γ}(λ)}{c_{σ}(−λ)}T χ_{∞}Φ_{0,λ}χ̃ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) (1 − χ̃)(J_{σ̃,−μ}^{w})^{∗}Φ_{−μ,0}χ_{1}

−\frac{ω_{U}a_{U}^{μ−λ}(μ − ρ)}{λ^{2} − μ^{2}}Tr R^{∗}T χ_{∞}Φ_{0,λ}J_{σ,λ}^{w}χ̃ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) (1 − χ̃)(J_{σ̃,−μ}^{w})^{∗}Φ_{−μ,0}χ_{1}

+\frac{ω_{U}a_{U}^{−μ−λ}(−λ − ρ)}{λ^{2} − μ^{2}}Tr R^{∗} \frac{cγ̃(−μ)^{∗}}{c_{σ}(μ)} γ(w)T χ_{∞}Φ_{0,−λ}J_{σ,λ}^{w}(1 − χ̃) (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) Φ_{μ,0}χ_{1}

+\frac{ω_{U}a_{U}^{μ−λ}(−λ − ρ)}{λ^{2} − μ^{2}}Tr R^{∗}T χ_{∞}Φ_{0,−λ}J_{σ,λ}^{w}(1 − χ̃) (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) (J_{σ̃,−μ}^{w})^{∗}Φ_{−μ,0}χ_{1}

+\frac{ω_{U}a_{U}^{μ+λ}(λ − ρ)}{λ^{2} − μ^{2}}Tr R^{∗}γ(w)^{−1} \frac{c_{γ}(λ)}{c_{σ}(−λ)}T χ_{∞}Φ_{0,λ} (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) (1 − χ̃)(J_{σ̃,−μ}^{w})^{∗}Φ_{−μ,0}χ_{1}

+\frac{ω_{U}a_{U}^{μ−λ}(−λ − ρ)}{λ^{2} − μ^{2}}Tr R^{∗}T χ_{∞}Φ_{0,−λ}J_{λ}^{w}χ̃ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) (1 − χ̃)(J_{σ̃,−μ}^{w})^{∗}Φ_{−μ,0}χ_{1}

+a_{U}^{−α} \frac{1}{λ^{2} − μ^{2}} R_{χ}(λ, μ, a_{U})

= −\frac{ω_{U}a_{U}^{−μ−λ}}{λ + μ} Tr R^{∗} \frac{cγ̃(−μ)^{∗}}{c_{σ}(−μ)} γ(w)T χ_{∞}Φ_{0,−λ}J_{σ,λ}^{w}(1 − χ̃) (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) Φ_{μ,0}χ_{1}

−\frac{ω_{U}a_{U}^{μ−λ}}{λ − μ} Tr R^{∗}T χ_{∞}Φ_{0,−λ}J_{σ,λ}^{w}(1 − χ̃) (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) J_{σ,−μ}^{w}Φ_{−μ,0}χ_{1}

+\frac{ω_{U}a_{U}^{μ+λ}}{λ + μ} Tr R^{∗}γ(w)^{−1} \frac{c_{γ}(λ)}{c_{σ}(−λ)}T χ_{∞}Φ_{0,λ} (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) (1 − χ̃)(J_{σ̃,−μ}^{w})^{∗}Φ_{−μ,0}χ_{1}

−\frac{ω_{U}a_{U}^{μ−λ}}{λ − μ} Tr R^{∗}T χ_{∞}Φ_{0,−λ}J_{σ,λ}^{w}χ̃ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) (1 − χ̃)(J_{σ̃,−μ}^{w})^{∗}Φ_{−μ,0}χ_{1}

+a_{U}^{−α} \frac{1}{λ^{2} − μ^{2}} R_{χ}(λ, μ, a_{U}) .

The remainder R_{χ}(λ, μ, a) is holomorphic and remains bounded in C^{∞}(i𝔞^{∗} × i𝔞^{∗}) as a → ∞.

We define ⟨R, T ⟩ ∈ ℂ such that R^{∗} ∘ T = ⟨R, T ⟩id_{V_{σ}}. If σ is not Weyl-invariant, i.e. σ^{w} ≅ σ̸, then the compositions R^{∗}cγ̃(−μ)^{∗}γ(w)T, R^{∗}γ(w)^{−1}c_{γ}(λ)T vanish. If the representation σ is Weyl-invariant, then it can be extended to the normalizer N_{K}(M) of M. In particular, we can define σ(w). In this case we define ⟨R, T ⟩(λ) ∈ ℂ such that

σ(w)R^{∗}γ(w)^{−1} \frac{c_{γ}(λ)}{c_{σ}(−λ)}T = ⟨R, T ⟩(λ)id_{V_{σ}} .

Note that R^{∗} \frac{cγ̃(−μ)^{∗}}{c_{σ}(μ)} γ(w)T σ(w)^{−1} = ⟨R, T ⟩(−μ)id_{V_{σ}}. Further we put J_{σ,λ} = σ(w)J_{σ,λ}^{w} : H_{−∞}^{σ,λ} → H_{−∞}^{σ,−λ}. Then we can write

Tr[χχ_{U} ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}χ_{1}]

= −\frac{ω_{U}a_{U}^{−μ−λ}⟨R, T ⟩(−μ)}{λ + μ}Tr Φ_{μ,−λ} ∘ χ_{∞} ∘ J_{σ,λ} ∘ (1 − χ̃) ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ χ_{1}(20)

−\frac{ω_{U}a_{U}^{μ−λ}⟨R, T ⟩}{λ − μ}Tr χ_{∞} ∘ J_{σ,λ}^{w} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (J_{σ̃,−μ}^{w})^{∗} ∘ Φ_{−μ,−λ} ∘ χ_{1}

+\frac{ω_{U}a_{U}^{μ+λ}⟨R, T ⟩(λ)}{λ + μ}Tr χ_{∞} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (1 − χ̃) ∘ (J_{σ̃,−μ})^{∗} ∘ Φ_{−μ,λ} ∘ χ_{1} (21)

+\frac{1}{λ^{2} − μ^{2}} Tr′ [Δ_{γ}, χ] ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}  (22)

+\frac{a_{U}^{−α}}{λ^{2} − μ^{2}} Q_{χ}(λ, μ, a_{U})

Q_{χ}(λ, μ, a_{U})  (23)
:= R_{χ}(λ, μ, a_{U})
−a_{U}^{α}Tr′ [Δ_{γ}, χ](1 − χ_{U}) ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R^{∗}})^{∗}  (24)

We defer the justification of the terms (22), (24) to Lemma 4.3 below. The functional Tr′ here is applied to operators with distribution kernels which are continuous on the diagonal, and it takes the integral of its local trace over the diagonal. Note that the remainder Q_{χ} is independent of the choice of χ_{1}.

The left-hand side of this formula is holomorphic on 𝔞_{ℂ}^{∗} × 𝔞_{ℂ}^{∗}. The terms on the right-hand side may have poles. To aim of the following discussion is to understand these singularities properly.

Lemma 4.2

\frac{1}{λ − μ}Tr χ_{∞} ∘ J_{σ,λ}^{w} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (J_{σ̃,−μ}^{w})^{∗} ∘ Φ_{−μ,−λ} ∘ χ_{1}

is regular for μ = λ.

Proof. We must show that

Tr χ_{∞} ∘ J_{σ,λ}^{w} ∘ (ext ∘ π_{∗} − 1) ∘ (J_{σ̃,−λ}^{w})^{∗} ∘ χ_{1} = 0 .

Recall the definition of the scattering matrix S_{σ,λ}^{w} from [3], Def. 5.6. We are going to employ the relations

ext ∘ S_{σ,λ}^{w} = J_{σ,λ}^{w} ∘ ext , π_{∗} ∘ (J_{σ̃,−λ}^{w})^{∗} = (S_{σ̃,−λ}^{w})^{∗} ∘ π_{∗} , S_{σ,λ}^{w} ∘ (S_{σ̃,−λ}^{w})^{∗} = id .

We now compute

Tr χ_{∞} ∘ J_{σ,λ}^{w} ∘ (ext ∘ π_{∗} − 1) ∘ (J_{σ̃,−λ}^{w})^{∗} ∘ χ_{1}
= Tr (χ_{∞} ∘ ext ∘ Ŝ_{σ,λ}^{w} ∘ (S_{σ̃,−λ}^{w})^{∗} ∘ π_{∗} ∘ χ_{1} − χ_{∞} ∘ J_{σ,λ}^{w} ∘ (J_{σ̃,−λ}^{w})^{∗} ∘ χ_{1}
= Tr χ_{∞} ∘ (ext ∘ π_{∗} − 1) ∘ χ_{1}
= 0 .
□

In particular we have

lim_{U→∞} lim_{μ→λ} −\frac{ω(U)a_{U}^{μ−λ}⟨R, T ⟩}{λ − μ}Tr χ_{∞} ∘ J_{σ,λ}^{w} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (J_{σ̃,−μ}^{w})^{∗} ∘ Φ_{−μ,−λ} ∘ χ_{1}

= ω_{X}⟨R, T ⟩\frac{d}{dμ}_{|μ=λ}Tr χ ∘ J_{σ,λ}^{w} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (J_{σ̃,−μ}^{w})^{∗} ∘ Φ_{−μ,−λ} ∘ χ_{1} .  (25)

If the distribution kernel of an operator admits a continuous restriction to the diagonal, then let DA denote this restriction.

Lemma 4.3 1. For any compact subset Q ⊂ i𝔞^{∗} there is a constant C such that for all k ∈ K and μ, λ ∈ Q

|D[Δ_{γ}, χ] ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}(ka)| < Ca^{−2ρ−α} .

2. We have

Tr′ [Δ_{γ}, χ^{Γ}] ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R^{∗}})^{∗} = 0

(note that we consider the cut-off function χ^{Γ} here).

Proof. The reason that 1. holds true is that |dχ(ka)| ≤ Ca^{−α} and |Δχ(ka)| ≤ Ca^{−α} uniformly in k ∈ K and a ∈ A. We use the decomposition

D[Δ_{γ}, χ] ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}(ka)
= D[Δ_{γ}, χ] ∘^{0}P_{λ}^{T} ∘ χ̃ ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (1 − χ̃) ∘ (^{0}P_{−μ}^{R})^{∗}(ka)  (26)
+D[Δ_{γ}, χ] ∘^{0}P_{λ}^{T} ∘ (1 − χ̃) ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}(ka) .  (27)

The asymptotic behaviour (7) of the operators P_{λ,a}^{T} is uniform for λ in compact subsets of i𝔞^{∗} and can be differentiated with respect to a. We conclude that for any compact subset Q ⊂ i𝔞^{∗} there is a constant C ∈ ℝ such for all λ, μ ∈ Q we have

sup_{k} |D[Δ_{γ}, χ]_{a} ∘^{0}P_{λ}^{T} ∘ (1 − χ̃) ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}(ka)| ≤ Ca^{−2ρ−α}
sup_{k} |D[Δ_{γ}, χ]_{a} ∘^{0}P_{λ}^{T} ∘ (1 − χ̃) ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) (^{0}P_{−μ}^{R})^{∗}(ka)| ≤ Ca^{−2ρ−α} .

We can write χ^{Γ} as a finite sum χ^{Γ} = ∑_{i} χ^{i}, where the cut-off functions χ^{i} obeying (18). For each index i we choose an appropriate cut-off function χ_{1}^{i} as above. It follows from 1. that tr D[Δ_{γ}, χ^{Γ}] ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗} is integrable over G. We compute

Tr′ [Δ_{γ}, χ^{Γ}] ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}

= ∑_{γ∈Γ} Tr′ π(γ)^{−1}χ^{Γ}π(γ) ∘ [Δ_{γ}, χ^{Γ}] ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}
= ∑_{γ∈Γ} Tr′ χ^{Γ} ∘ [Δ_{γ}, (γ^{−1})^{∗}χ^{Γ}] ∘ π(γ) ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ π^{σ,μ}(γ)^{−1} ∘ (^{0}P_{−μ}^{R})^{∗}
= ∑_{γ∈Γ} Tr′ χ^{Γ} ∘ [Δ_{γ}, (γ^{−1})^{∗}χ^{Γ}] ∘^{0}P_{λ}^{T} ∘ (π^{σ,λ}(γ) ∘ ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − π^{σ,λ}(γ) ∘ Φ̄_{λ,μ}) ∘ π^{σ,μ}(γ)^{−1} ∘ (^{0}P_{−μ}^{R^{∗}})^{∗}
= ∑_{γ∈Γ} Tr′ χ^{Γ} ∘ [Δ_{γ}, (γ^{−1})^{∗}χ^{Γ}] ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ} ∘ π^{σ,μ}(γ)) ∘ π^{σ,μ}(γ)^{−1} ∘ (^{0}P_{−μ}^{R})^{∗}
= ∑_{γ∈Γ} Tr′ χ^{Γ} ∘ [Δ_{γ}, (γ^{−1})^{∗}χ^{Γ}] ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ π^{σ,μ}(γ) ∘ π^{σ,μ}(γ)^{−1} ∘ (^{0}P_{−μ}^{R})^{∗}
= ∑_{γ∈Γ} Tr′ χ^{Γ} ∘ [Δ_{γ}, (γ^{−1})^{∗}χ^{Γ}] ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R})^{∗}
= 0

since ∑_{γ∈Γ}(γ^{−1})^{∗}χ^{Γ} ≡ 1. □ Note that the second assertion of the lemma implies

∑_{i} Tr [Δ_{γ}, χ^{i}] ∘^{0}P_{λ}^{T} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (^{0}P_{−μ}^{R^{∗}})^{∗} ∘ χ_{1}^{i} = 0

We now combine (20) and (21) and write

(20) + (21)

= \frac{ω(U)(a_{U}^{μ+λ} − a_{U}^{−μ−λ})}{λ + μ}⟨R, T ⟩(−μ)
Tr Φ_{μ,−λ} ∘ χ_{∞} ∘ J_{σ,λ} ∘ (1 − χ̃) ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ χ_{1}  (28)

+\frac{ω(U)a_{U}^{μ+λ}}{λ + μ} Tr [⟨R, T ⟩(λ)χ_{∞} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (1 − χ̃) ∘ (J_{σ̃,−μ})^{∗} ∘ Φ_{−μ,λ} ∘ χ_{1}

−⟨R, T ⟩(−μ)Φ_{μ,−λ} ∘ χ_{∞} ∘ J_{σ,λ} ∘ (1 − χ̃) ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ χ_{1}] (29)

Note that (28) is regular at λ = μ = 0. In fact, if ext has a pole at λ = 0, then it is of first order and J_{σ,0} = id (see [3], Prop. 7.4). Hence, the composition χ_{∞} ∘ J_{σ,0} ∘ (1 − χ̃) vanishes.

In order to see that (29) is regular at λ = μ = 0, too, observe in addition that

Tr [χ ∘ (ext ∘ π_{∗} − 1) ∘ (1 − χ̃) ∘ (J_{σ̃,0})^{∗} ∘ χ_{1} − χ ∘ J_{σ,0} ∘ (1 − χ̃) (ext ∘ π_{∗} − 1) ∘ χ_{1}] = 0 . (30)

Combining Lemma 4.2 and 4.3, 2., and (30) we conclude that \frac{1}{λ^{2}−μ^{2}} Q_{χ^{Γ}}(λ, μ, a_{U}) is regular at μ = λ, where we set Q_{χ^{Γ}} := ∑_{i} Q_{χ^{i}}. Furthermore, locally uniformly in λ

lim_{U→∞} \frac{a_{U}^{−α}}{λ^{2} − μ^{2}} Q_{χ^{Γ}}(λ, μ, a_{U}) = 0 .

By the Lemma of Riemann-Lebesgue we have

lim_{U→∞} \frac{ω(U)a_{U}^{2λ}}{2λ} Tr [⟨R, T ⟩(λ)χ_{∞} ∘ (ext ∘ π_{∗} − 1) ∘ (1 − χ̃) ∘ (J_{σ̃,−λ})^{∗} ∘ Φ_{−λ,λ} ∘ χ_{1}
−⟨R, T ⟩(−λ)Φ_{λ,−λ} ∘ χ_{∞} ∘ J_{σ,λ} ∘ (1 − χ̃) ∘ (ext ∘ π_{∗} − 1) ∘ χ_{1}]
= 0

as distributions on i𝔞^{∗}. Moreover,

lim_{U→∞} \frac{ω(U)(a_{U}^{2λ} − a_{U}^{−2λ})}{2λ}⟨R, T ⟩(−λ)

Tr Φ_{λ,−λ} ∘ χ_{∞} ∘ J_{σ,λ} ∘ (1 − χ̃) ∘ (ext ∘ π_{∗} − 1) ∘ χ_{1}  (31)
= πω_{X}⟨R, T ⟩(0)δ_{0}(λ) lim_{λ→0} [Tr Φ_{λ,−λ} ∘ χ_{∞} ∘ J_{σ,λ} ∘ (1 − χ̃) ∘ (ext ∘ π_{∗} − 1) ∘ χ_{1}] (32)

in the sense of distributions on i𝔞^{∗}. Combining (25) and (32) we now have shown the following proposition.

Proposition 4.4

∫_{Γ\G} tr A_{q}(g, g)μ_{G}(dg)  (33)
= ω_{X}⟨R, T ⟩ ∫_{i𝔞^{∗}} ∑_{i} \frac{d}{dμ}_{|μ=λ}Tr [χ_{∞}^{i} ∘ J_{σ,λ}^{w} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (J_{σ̃,−μ}^{w})^{∗} ∘ Φ_{−μ,−λ} ∘ χ_{1}^{i}] q(λ)dλ
+ πω_{X}⟨R, T ⟩(0) ∑_{i} lim_{λ→0} (Tr [Φ_{λ,−λ} ∘ χ_{∞}^{i} ∘ J_{σ,λ} ∘ (ext ∘ π_{∗} − 1) ∘ χ_{1}^{i}]) q(0) .

Observe that we can rewrite this in the more invariant form

ω_{X}⟨R, T ⟩ ∫_{i𝔞^{∗}} \frac{d}{dμ}_{|μ=λ}Tr′ [χ_{∞}^{Γ} ∘ J_{σ,λ}^{w} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (J_{σ̃,−μ}^{w})^{∗} ∘ Φ_{−μ,−λ}] q(λ)dλ

+ πω_{X}⟨R, T ⟩(0) lim_{λ→0} (Tr′ [Φ_{λ,−λ} ∘ χ_{∞}^{Γ} ∘ J_{σ,λ} ∘ (ext ∘ π_{∗} − 1)]) q(0) .

5 The Fourier transform of Ψ

5.1 The contribution of the scattering matrix

We consider a symmetric function q ∈ C_{c}^{∞}(i𝔞^{∗}, End(V_{γ} ⊗ Hom_{M}(V_{γ}, V_{σ}))). For π ∈ Ĝ we form h_{q}(π) ∈ End(V_{π}) as in Subsection 3.3. There we have seen that Ř(h_{q}) and Ř_{Γ}(h_{q}) have smooth integral kernels.

We choose a basis {v_{i}}_{i=1,...,dim(γ)} of V_{γ} and a basis {T_{j}}_{j=1,...,dim(Hom_{M} (V_{γ},V_{σ}))} of Hom_{M}(V_{γ}, V_{σ}). Let {v^{i}} be a dual basis of Vγ̃ and {T^{j}} be a dual basis of Hom_{M}(V_{σ}, V_{γ}) = Hom_{M}(V_{γ}, V_{σ})^{∗} (with respect to the pairing ⟨T′, T ⟩ = tr_{V_{γ}}T′ ∘ T, T ∈ Hom_{M}(V_{γ}, V_{σ}), T′ ∈ Hom_{M}(V_{σ}, V_{γ})). Then {ϕ_{ij} := v_{i} ⊗ T_{j}} can be considered as a basis of H^{σ,λ}(γ). Furthermore, {ϕ^{ij} := v^{i} ⊗ (T^{j})^{∗}} can be considered as a basis of H^{σ̃,−λ}(γ̃) = H^{σ,λ}(γ)^{∗}, and we have

⟨ϕ_{ij}, ϕ^{hl}⟩ = ∫_{K}⟨T_{j} ∘ γ(k)^{−1}(v_{i}), (T^{l})^{∗} ∘ γ̃(k)^{−1}(v^{h})⟩dk
= ∫_{K}⟨γ(k) ∘ T^{l} ∘ T_{j} ∘ γ(k)^{−1}(v_{i}), v^{h}⟩dk
= \frac{⟨T^{l}, T_{j}⟩}{dim(V_{γ})}⟨v_{i}, v^{h}⟩
= \frac{1}{dim(V_{γ})}δ_{j}^{l}δ_{i}^{h} .  (34)

For g ∈ G and w ∈ Vγ̃ we define p_{λ,w}^{T}(g) ∈ H_{∞}^{σ̃,−λ} such that ⟨P_{λ}^{T} (ϕ)(g), w⟩ = ⟨p_{λ,w}^{T}(g), ϕ⟩ for all ϕ ∈ H_{−∞}^{σ,λ}. Using (5) we can write p_{λ,w}^{T}(g) = π^{σ̃,−λ}(g)(w ⊗ T^{∗}). We can write

q = ∑_{i,j,k,l} q_{ijkl}v_{i} ⊗ T_{j} ⊗ v^{k} ⊗ T^{l} ,

where the functions q_{ijjk} := ⟨v^{i} ⊗ T^{j}, q(v_{k} ⊗ T_{l})⟩ belong to C_{c}^{∞}(i𝔞^{∗}). Now we can compute

Tr π^{σ,λ}(g)h_{q}(π^{σ,λ}) = dim(V_{γ}) ∑_{i,j}⟨ϕ^{ij}, π^{σ,λ}(g)h_{q}(π^{σ,λ})ϕ_{ij}⟩
= dim(V_{γ}) ∑_{i,j}⟨P_{λ}^{T^{j}}(h_{q}(π^{σ,λ})ϕ_{ij})(g^{−1}), v^{i}⟩
= dim(V_{γ}) ∑_{i,j}⟨h_{q}(π^{σ,λ})ϕ_{ij}, p_{λ,v^{i}}^{T^{j}}(g^{−1})⟩
= dim(V_{γ}) ∑_{i,j,k,l} q_{klij}(λ)⟨v_{k} ⊗ T_{l}, p_{λ,v^{i}}^{T^{j}}(g^{−1})⟩
= dim(V_{γ}) ∑_{i,j,k,l} q_{klij}(λ)⟨P_{−λ}^{T_{l}^{∗}}(p_{λ,v^{i}}^{T^{j}}(g^{−1}))(1), v_{k}⟩
= dim(V_{γ}) ∑_{i,j,k,l} q_{klij}(λ)⟨v^{i}, P_{λ}^{T^{j}} ∘ (P_{−λ}^{T_{l}^{∗}})^{∗}(g^{−1}, 1)(v_{k})⟩ .

5.1 The contribution of the scattering matrix 25 In the last line of this computation P_{λ}^{T^{j}} ∘ (P_{−λ}^{T_{l}^{∗}})^{∗}(g, g′) is the integral kernel of the G-equivariant operator

P_{λ}^{T^{j}} ∘ (P_{−λ}^{T_{j}^{∗}})^{∗} : C_{c}^{−∞}(G ×_{K} V_{γ}) → C^{∞}(G ×_{K} V_{γ}) .

We can express the integral kernel of Ř(h_{q}) via Poisson transforms as follows:

K_{Ř(h_{q})}(g, g_{1}) = ∫Ĝ Tr π(g)h_{q}(π)π(g_{1}^{−1})p(dπ)
= \frac{dim(V_{σ})}{4πω_{X}} ∫_{i𝔞^{∗}} Tr π(g_{1}^{−1}g)h_{q}(π^{σ,λ})p_{σ}(λ)dλ
= ∑_{i,j,k,l} \frac{dim(V_{σ}) dim(V_{γ})}{4πω_{X}} ∫_{i𝔞^{∗}} q_{klij}(λ)⟨v^{i}, P_{λ}^{T^{j}} ∘ (P_{−λ}^{T_{l}^{∗}})^{∗}(g^{−1}g_{1}, 1)(v_{k})⟩p_{σ}(λ)dλ
= ∑_{i,j,k,l} \frac{dim(V_{σ}) dim(V_{γ})}{4πω_{X}} ∫_{i𝔞^{∗}} q_{klij}(λ)⟨v^{i}, P_{λ}^{T^{j}} ∘ (P_{−λ}^{T_{l}^{∗}})^{∗}(g_{1}, g)(v_{k})⟩p_{σ}(λ)dλ
= ∑_{i,j,k,l} \frac{dim(V_{σ}) dim(V_{γ})}{4πω_{X}} ∫_{i𝔞^{∗}} q_{klij}(λ)⟨v^{i},^{0}P_{λ}^{T^{j}} ∘ (^{0}P_{−λ}^{T_{l}^{∗}})^{∗}(g_{1}, g)(v_{k})⟩dλ  (35)

In a similar manner we obtain

Tr π^{σ,λ}(g)h_{q}(π^{σ,λ})π^{σ,λ}(g_{1}^{−1})ext π_{∗}

= Tr h_{q}(π^{σ,λ})π^{σ,λ}(g_{1}^{−1})ext π_{∗} π^{σ,λ}(g)
= dim(V_{γ}) ∑_{k,l}⟨ϕ^{kl}, h_{q}(π^{σ,λ})π^{σ,λ}(g_{1}^{−1})ext π_{∗} π^{σ,λ}(g)ϕ_{kl}⟩
= dim(V_{γ}) ∑_{k,l}⟨h_{q}(π^{σ,λ})^{∗}ϕ^{kl}, π^{σ,λ}(g_{1}^{−1})ext π_{∗} π^{σ,λ}(g)ϕ_{kl}⟩
= dim(V_{γ}) ∑_{i,j,k,l} q_{klij}(λ)⟨v^{i} ⊗ T^{j}, π^{σ,λ}(g_{1}^{−1})ext π_{∗} π^{σ,λ}(g)ϕ_{kl}⟩
= dim(V_{γ}) ∑_{i,j,k,l} q_{klij}(λ)⟨v^{i}, P_{λ}^{T^{j}}(ext π_{∗} π^{σ,λ}(g)ϕ_{kl})(g_{1})⟩
= dim(V_{γ}) ∑_{i,j,k,l} q_{klij}(λ)⟨ext π_{∗} π^{σ,λ}(g)ϕ_{kl}, p_{λ,v^{i}}^{T^{j}}(g_{1})⟩
= dim(V_{γ}) ∑_{i,j,k,l} q_{klij}(λ)⟨ϕ_{kl}, π^{σ̃,−λ}(g^{−1})ext π_{∗}(p_{λ,v^{i}}^{T^{j}}(g_{1}))⟩
= dim(V_{γ}) ∑_{i,j,k,l} q_{klij}(λ)⟨v_{k}, P_{−λ}^{T_{l}^{∗}}(ext π_{∗}(p_{λ,v^{i}}^{T^{j}}(g_{1})))(g)⟩
= dim(V_{γ}) ∑_{i,j,k,l} q_{klij}(λ)⟨v^{i}, P_{λ}^{T^{j}} ∘ ext ∘ π_{∗} ∘ (P_{−λ}^{T_{l}^{∗}})^{∗}(g_{1}, g)(v_{k})⟩

and thus

K_{Ř_{Γ}(h_{q})}(g, g_{1}) = \frac{dim(V_{σ})}{4πω_{X}} ∫_{i𝔞^{∗}} Tr π^{σ,λ}(g)h_{q}(π^{σ,λ})π^{σ,λ}(g_{1}^{−1})p_{σ}(λ)dλ

= ∑_{i,j,k,l} \frac{dim(V_{σ}) dim(V_{γ})}{4πω_{X}} ∫_{i𝔞^{∗}} q_{klij}(λ)⟨v^{i}, P_{λ}^{T^{j}} ∘ ext ∘ π_{∗} ∘ (P_{−λ}^{T_{l}^{∗}})^{∗}(g_{1}, g)(v_{k})⟩p_{σ}(λ)dλ

= ∑_{i,j,k,l} \frac{dim(V_{σ}) dim(V_{γ})}{4πω_{X}} ∫_{i𝔞^{∗}} q_{klij}(λ)⟨v^{i},^{0}P_{λ}^{T^{j}} ∘ ext ∘ π_{∗} ∘ (^{0}P_{−λ}^{T_{l}^{∗}})^{∗}(g_{1}, g)(v_{k})⟩dλ(36)

We conclude that

K_{Ř_{Γ}(h_{q})}(g, g_{1}) − K_{Ř(h_{q})}(g, g_{1})
= ∑_{i,j,k,l} \frac{dim(V_{σ}) dim(V_{γ})}{4πω_{X}} ∫_{i𝔞^{∗}} q_{klij}(λ)⟨v^{i},^{0}P_{λ}^{T^{j}} ∘ (ext ∘ π_{∗} − 1) ∘ (^{0}P_{−λ}^{T_{l}^{∗}})^{∗}(g_{1}, g)(v_{k})⟩dλ

= \frac{dim(V_{σ}) dim(V_{γ})}{4πω_{X}} ∑_{i,j,k,l}⟨v^{i}, A_{q_{klij}}(T^{j}, T_{l}^{∗})(g_{1}, g)v_{k}⟩ .

By Proposition 4.1 the difference

g ↦ (K_{Ř_{Γ}(h_{q})}(g, g) − K_{Ř(h_{q})}(g, g))

is integrable over Γ\G. Using the fact that γ is irreducible we compute

∑_{i,k} ∫_{Γ\G}⟨v^{i}, A_{q_{klij}}(T^{j}, T_{l}^{∗})(g, g)v_{k}⟩μ_{G}(dg)

= ∑_{i,k} ∫_{Γ\G} ∫_{K}⟨v^{i}, A_{q_{klij}}(T^{j}, T_{l}^{∗})(gh, gh)v_{k}⟩μ_{K}(dh)μ_{G}(dg)

= ∑_{i,k} ∫_{Γ\G} ∫_{K}⟨v^{i}, γ(h)^{−1}A_{q_{klij}}(T^{j}, T_{l}^{∗})(g, g)γ(h)v_{k}⟩μ_{K}(dh)μ_{G}(dg)

= ∑_{k} ∫_{Γ\G} tr A_{q_{klkj}}(T^{j}, T_{l}^{∗})(g, g)μ_{G}(dg)

The following formula is now an immediate consequence of Proposition 4.4.

∫_{Γ\G} (K_{Ř_{Γ}(h_{q})}(g, g) − K_{Ř(h_{q})}(g, g)) μ_{G}(dg)  (37)
= \frac{dim(V_{σ}) dim(V_{γ})}{4π} ∫_{i𝔞^{∗}} ∑_{i} ∑_{k,j,l}⟨T_{l}^{∗}, T^{j}⟩

\frac{d}{dμ}_{|μ=λ}Tr [χ_{∞}^{i} ∘ J_{σ,λ}^{w} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (J_{σ̃,−μ}^{w})^{∗} ∘ Φ_{−μ,−λ} ∘ χ_{1}^{i}] q_{klkj}(λ)dλ

+ \frac{dim(V_{σ}) dim(V_{γ})}{4} ∑_{i} ∑_{k,j,l}⟨T_{l}^{∗}, T^{j}⟩(0) lim_{λ→0} (Tr [Φ_{λ,−λ} ∘ χ_{∞}^{i} ∘ J_{σ,λ} ∘ (ext ∘ π_{∗} − 1) ∘ χ_{1}^{i}]) q_{klkj}(0) .

Now we will rewrite this formula in a more invariant fashion. Using (34) we first compute

∑_{k,j,l}⟨T_{l}^{∗}, T^{j}⟩q_{klkj}(λ)  (38)

= \frac{1}{dim(V_{σ})} ∑_{j,l,k} T_{l}(T^{j})⟨v^{k} ⊗ T^{l}, q(λ)v_{k} ⊗ T_{j}⟩
= \frac{1}{dim(V_{σ})} ∑_{l,k}⟨v^{k} ⊗ T^{l}, q(λ)v_{k} ⊗ T_{l}⟩
= \frac{1}{dim(V_{γ}) dim(V_{σ})}Tr h_{q}(π^{σ,λ})  (39)

We assume for a moment that σ is Weyl-invariant. For T ∈ Hom_{M}(V_{σ}, V_{γ}) let T^{♯} ∈ Hom(H_{−∞}^{σ,λ}, V_{γ}) be given by T^{♯}(f) = P_{λ}^{T} (f)(1). Recall the relation ([3], Lemma 5.5, 1.)

T^{♯} ∘ σ(w) ∘ J_{σ,λ}^{w} = c_{σ}(−λ)[γ(w) ∘ c_{γ}(λ) ∘ T ∘ σ(w)^{−1}]^{♯} .

We compute

⟨R, T ⟩(0) = \frac{1}{dim(V_{σ})c_{σ}(0)}Tr σ(w) ∘ R^{∗} ∘ γ(w)^{−1} ∘ c_{γ}(0) ∘ T
= \frac{1}{dim(V_{σ})c_{σ}(0)}Tr γ(w) ∘ c_{γ}(0) ∘ T ∘ σ(w)^{−1} ∘ R^{∗}
= \frac{1}{dim(V_{σ})c_{σ}(0)} ∫_{K} Tr γ(k)^{−1} ∘ γ(w) ∘ c_{γ}(0) ∘ T ∘ σ(w)^{−1} ∘ R^{∗} ∘ γ(k)μ_{K}(dk)
= \frac{1}{dim(V_{σ})c_{σ}(0)} ∑_{i} ∫_{K}⟨γ(k)^{−1} ∘ γ(w) ∘ c_{γ}(0) ∘ T ∘ σ(w)^{−1} ∘ R^{∗} ∘ γ(k)(v_{i}), v^{i}⟩μ_{K}(dk)
= \frac{1}{dim(V_{σ})c_{σ}(0)} ∑_{i}⟨P_{0}^{γ(w)∘c_{γ}(0)∘T ∘σ(w)^{−1}}(v_{i} ⊗ R^{∗})(1), v^{i}⟩
= \frac{1}{dim(V_{σ})c_{σ}(0)} ∑_{i}⟨[γ(w) ∘ c_{γ}(0) ∘ T ∘ σ(w)^{−1}]^{♯}(v_{i} ⊗ R^{∗}), v^{i}⟩
= \frac{1}{dim(V_{σ})} ∑_{i}⟨T^{♯} ∘ σ(w) ∘ J_{σ,0}^{w}(v_{i} ⊗ R^{∗}), v^{i}⟩
= \frac{1}{dim(V_{σ})} ∑_{i} P_{0}^{T} ∘ J_{σ,0}(v_{i} ⊗ R)(1), v^{i}⟩
= \frac{1}{dim(V_{σ})} ∑_{i}⟨J_{σ,0}(v_{i} ⊗ R^{∗}), v^{i} ⊗ T^{∗}⟩

Since J_{σ,0} is K-equivariant, we can write

J_{σ,0}(v_{i} ⊗ R) = v_{i} ⊗ j_{σ,0}(R)

for some j_{σ,0} ∈ End(Hom_{M}(V_{σ}, V_{γ})). We now have

∑_{k,j,l}⟨T_{l}^{∗}, T^{j}⟩(0)q_{klkj}(λ) = \frac{1}{dim(V_{σ})} ∑_{k,j,l} ∑_{i}⟨v_{i} ⊗ j_{σ,0}(T_{l}^{∗}), v^{i} ⊗ T^{j}⟩⟨v^{k} ⊗ T^{l}, q(λ)v_{k} ⊗ T_{j}⟩
= \frac{1}{dim(V_{σ})} ∑_{k,l}⟨v^{k} ⊗ T^{l}, q(λ)(v_{k} ⊗ j_{σ,0}(T_{l}^{∗}))⟩  (40)

= \frac{1}{dim(V_{σ})} ∑_{k,l}⟨v^{k} ⊗ T^{l}, q(λ) ∘ J_{σ,0}(v_{k} ⊗ T_{l}^{∗})⟩
= \frac{1}{dim(V_{σ}) dim(V_{γ})}Tr h_{q}(π^{σ,0}) ∘ J_{σ,0}  (41)

Inserting (39) and (41) into (37) we obtain the following theorem.

Theorem 5.1 If q is a smooth compactly supported symmetric function on M̂ × i𝔞^{∗} such that q(σ, λ) ∈ End(V_{γ} ⊗ Hom_{M}(V_{γ}, V_{σ})), then the difference

g ↦ K_{Ř_{Γ}(h_{q})}(g, g) − K_{Ř(h_{q})}(g, g)

is integrable over Γ\G, and we have

∫_{Γ\G} (K_{Ř_{Γ}(h_{q})}(g, g) − K_{Ř(h_{q})}(g, g)) μ_{G}(dg)
= ∑_{σ∈M̂} \frac{1}{4π} ∑_{i} ∫_{i𝔞^{∗}}

\frac{d}{dμ}_{|μ=λ}Tr [χ_{∞}^{i} ∘ J_{σ,λ}^{w} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (J_{σ̃,−μ}^{w})^{∗} ∘ Φ_{−μ,−λ} ∘ χ_{1}^{i}] Tr h_{q}(π^{σ,λ})dλ

+ \frac{1}{4} ∑_{i} lim_{λ→0} (Tr [Φ_{λ,−λ} ∘ χ_{∞}^{i} ∘ J_{σ,λ} ∘ (ext ∘ π_{∗} − 1) ∘ χ_{1}^{i}]) Tr h_{q}(π^{σ,0}) ∘ J_{σ,0} .

We can again rewrite this formula as follows

∫_{Γ\G} (K_{Ř_{Γ}(h_{q})}(g, g) − K_{Ř(h_{q})}(g, g)) μ_{G}(dg)
= ∑_{σ∈M̂} \frac{1}{4π} ∫_{i𝔞^{∗}}

\frac{d}{dμ}_{|μ=λ}Tr′ [χ_{∞}^{Γ} ∘ J_{σ,λ}^{w} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (J_{σ̃,−μ}^{w})^{∗} ∘ Φ_{−μ,−λ}] Tr h_{q}(π^{σ,λ})dλ

+ \frac{1}{4} lim_{λ→0} (Tr′ [Φ_{λ,−λ} ∘ χ_{∞}^{Γ} ∘ J_{σ,0} ∘ (ext ∘ π_{∗} − 1)]) Tr h_{q}(π^{σ,0}) ∘ J_{σ,0} .

5.2 The Fourier transform of Ψ. A conjecture.

Observe that Theorem 5.1 does not solve our initial problem of computing the Fourier transform of the distribution Ψ. The point is that there is no function f ∈ C_{c}^{∞}(G) such that its Fourier transform f̂ has compact support. In order to extend Theorem 5.1 to f̂ we must extend Proposition 4.1 to Schwartz functions. As explained in the remark at the end of Subsection 4.3 the main obstacle to do this is an estimate of the growth of the extension map ext = ext_{λ} : C^{−∞}(B, V_{B}(σ, λ)) → H_{−∞}^{σ,λ} as λ tends to infinity along the imaginary axis.

The goal of the present subsection is to rewrite the result of the computation of Ψ′ in terms of characters thus obtaining the candidate of the measure Φ. We will also take the discrete spectrum of L^{2}(Γ\G) into account.

Recall that

Tr f̂(π^{σ,λ}) = θ_{π^{σ,λ}}(f) .

If π^{σ,0} is reducible, then it decomposes into a sum of π^{σ,+} ⊕ π^{σ,−} of limits of discrete series representations which are just the ±1 eigenspaces of J_{σ,0}. In this case ext is regular at λ = 0 ([3], Prop. 7.4.). We can write

Tr f̂(π^{σ,0}) ∘ J_{σ,0} = θ_{π^{σ,+}}(f) − θ_{π^{σ,−}}(f) .

If we replace h_{q} by f̂ then formulas (35) and (36) just give the contributions of the continuous spectrum K_{R^{ac}(f)} and K_{R_{Γ}^{ac}(f)} to the integral kernels K_{R(f)} and K_{R_{Γ}(f)}. We have K_{R(f)} = K_{R^{ac}(f)} + K_{R^{d}(f)}, where

K_{R^{d}(f)}(g, g_{1}) = ∑_{π∈Ĝ_{d}} Tr π(g)f̂(π)π(g_{1}^{−1}) .

Since we assume that f is K-finite, and there are only finitely many discrete series representations containing a given K-type, this sum is finite.

Furthermore, K_{R_{Γ}(f)} = K_{R_{Γ}^{ac}(f)} + K_{R_{Γ}^{d}(f)} + K_{R_{Γ}^{c}(f)}. Here K_{R_{Γ}^{d}(f)} = ∑_{π∈Ĝ_{d}} K_{R_{Γ}^{d,π}(f)} is the contribution of discrete series representations (again a finite sum),

K_{R_{Γ}^{d,π}(f)}(g, g_{1}) = ∑_{i,j} \overline{⟨ψ_{i}, ϕ_{j}⟩}⟨ψ_{i}, π(g)f̂(π)π(g_{1}^{−1})ϕ_{j}⟩ ,

where {ϕ_{j}} and {ψ_{i}} are orthonormal bases of the infinite-dimensional Hilbert spaces V_{π} and M_{π}, respectively. The finite sum K_{R_{Γ}^{c}(f)} = ∑_{π∈Ĝ_{c}} K_{R_{Γ}^{c,π}(f)} is the discrete contribution of representations belonging to Ĝ_{c}. If we choose orthogonal bases {ϕ_{j}} and {ψ_{i}} of the Hilbert spaces V_{π} and M_{π} (note that dim(M_{π}) < ∞), then we can write

K_{R_{Γ}^{c,π}(f)}(g, g_{1}) = ∑_{i,j} \overline{⟨ψ_{i}, ϕ_{j}⟩}⟨ψ_{i}, π(g)f̂(π)π(g_{1}^{−1})ϕ_{j}⟩ .

We define the multiplicity of π by N_{Γ}(π) := dim(M_{π}). It is clear that

∫_{Γ\G} K_{R_{Γ}^{c}(f)}(g, g)μ_{G}(dg) = ∑_{π∈Ĝ_{c}} N_{Γ}(π)θ_{π}(f) .

In Lemma 6.13 we will show that if f is K-finite and invariant under conjugation by K, then for each π ∈ Ĝ_{d} we have

K_{R_{Γ}^{d,π}(f)}(g, g) − K_{R^{d,π}(f)}(g, g) ∈ L^{1}(Γ\G) .

Given any A ∈ V_{π̃,K} ⊗ V_{π,K} we define the function Ĝ ∋ π′ ↦ h_{A}(π′) to be zero for all π′ ≠ π and h_{A}(π) := Ā, where Ā := ∫_{K} π̃(k) ⊗ π(k)Adk. The map

V_{π̃,K} ⊗ V_{π,K} ∋ A ↦ T (A) := ∫_{Γ\G}[K_{Ř_{Γ}^{d}(hĀ)}(g, g) − K_{Ř^{d}(hĀ)}(g, g)]μ_{G}(dg)

is well-defined and a (𝔤, K)-invariant functional on V_{π̃,K} ⊗ V_{π,K}. Since V_{π,K} is irreducible it follows that

T (A) = N_{Γ}(π)θ_{π}(A)

for some number N_{Γ}(π) ∈ ℂ which plays the role of the multiplicity of π. Here we consider A as a finite-dimensional operator on V_{π}.

It follows from Lemma 6.13 that K_{R_{Γ}^{ac}(f)}(g, g) − K_{R^{ac}(f)}(g, g) belongs to L^{1}(Γ\G) not only if f̂ is smooth of compact support, K-finite and K-invariant (Proposition 4.1), but also in the case that f ∈ C_{c}^{∞}(G) is K-finite and K-conjugation invariant.

The following conjecture provides the candidate for the measure Φ. Its discrete part is expressed in terms of multiplicities N_{Γ}(π). If π ∈ Ĝ_{c}, then N_{Γ}(π) is just the dimension of the space of multiplicities M_{π} and thus a non-negative integer. If π ∈ Ĝ_{d}, then N_{Γ}(π) is a sort of regularized dimension of M_{π}. We will show in Proposition 6.14 that N_{Γ}(π) is an integer in this case, too. The continuous part of the spectrum will contribute a point measure supported on the irreducible constitutents of the representations π^{σ,0}, and the corresponding weight will be denoted by Ñ_{Γ}(π). The remaining contribution of the continuous spectrum is absolute continuous to the Lebesgue measure dλ on Ĝ_{ac} = M̂ × i𝔞_{+}^{∗} and will be described by the density L_{Γ}(π^{σ,λ}). By Theorem 6.12 this density appears in the functional equation of the Selberg zeta function. In particular, as it can be already seen from the definition below, L_{Γ}(π^{σ,λ}) admits a meromorphic continuation to all of 𝔞_{ℂ}^{∗} as a function of λ. Its residues are closely related to the multiplicities of resonances.

Conjecture 5.2 If f ∈ C_{c}^{∞}(G) is bi-K-finite, then we have

Ψ(f) = ∑_{σ∈M̂} \frac{1}{4π} ∫_{i𝔞^{∗}} L_{Γ}(π^{σ,λ})θ_{π^{σ,λ}}(f)dλ
+ ∑_{σ∈M̂,σ≅σ^{w},π^{σ,0}red.} ∑_{ϵ∈{+,−}} Ñ_{Γ}(π^{σ,ϵ})θ_{π^{σ,ϵ}}(f)
+ ∑_{σ∈M̂,σ≅σ^{w},π^{σ,0} irred.} Ñ_{Γ}(π^{σ,0})θ_{π^{σ,0}}(f)
+ ∑_{π∈Ĝ_{c}∪Ĝ_{d}} N_{Γ}(π)θ_{π}(f) ,

where

L_{Γ}(π^{σ,λ}) := \frac{d}{dμ}_{|μ=λ}Tr′ [χ_{∞}^{Γ} ∘ J_{σ,λ}^{w} ∘ (ext ∘ Φ̄_{λ,μ} ∘ π_{∗} − Φ̄_{λ,μ}) ∘ (J_{σ̃,−μ}^{w})^{∗} ∘ Φ_{−μ,−λ}]

and

Ñ_{Γ}(π^{σ,±}) := ±\frac{1}{4}Tr′ lim_{λ→0} ([Φ_{λ,−λ} ∘ χ_{∞}^{Γ} ∘ J_{σ,λ} ∘ (ext ∘ π_{∗} − 1)])
Ñ_{Γ}(π^{σ,0}) := \frac{1}{4}Tr′ lim_{λ→0} ([Φ_{λ,−λ} ∘ χ_{∞}^{Γ} ∘ J_{σ,λ} ∘ (ext ∘ π_{∗} − 1)]) .

Note that the discussion above does not prove this conjecture. What it does prove is the following theorem.

Theorem 5.3 If Ĝ ∋ π ↦ h(π) ∈ End_{K}(V_{π}) is smooth, of compact support, and factorizes over finitely many K-types, then we have

Ψ′(˜ȟ) = ∑_{σ∈M̂} \frac{1}{4π} ∫_{i𝔞^{∗}} L_{Γ}(π^{σ,λ})Tr h(π^{σ,λ})dλ
+ ∑_{σ∈M̂,σ≅σ^{w},π^{σ,0}red.} ∑_{ϵ∈{+,−}} Ñ_{Γ}(π^{σ,ϵ})Tr h(π^{σ,ϵ})
+ ∑_{σ∈M̂,σ≅σ^{w},π^{σ,0} irred.} Ñ_{Γ}(π^{σ,0})Tr h(π^{σ,0})
+ ∑_{π∈Ĝ_{c}∪Ĝ_{d}} N_{Γ}(π)Tr h(π) .

6 Resolvent kernels and Selberg zeta functions

6.1 Meromorphic continuation of resovent kernels

We fix any K-type γ. The Casimir operator Ω of G gives rise to an unbounded selfadjoint operator on the Hilbert space of sections L^{2}(X, V (γ)). To be precise it is the unique selfadjoint extension of the restriction of Ω to the space of smooth sections with compact support, where we normalize Ω such that it is bounded from below. For each complex number z which is not contained in the spectrum σ(Ω) we let R(z) be the operator (z − Ω)^{−1}. By r(z) we denote its distribution kernel.

The Casimir operator descends to an operator acting on sections of V_{Y} (γ) and induces an unique unbounded selfadjoint operator Ω_{Y} on L^{2}(Y, V_{Y} (γ)). For z ∉ σ(Ω_{Y}) we define the resolvent R_{Y} (z) of the operator (z − Ω_{Y})^{−1} and denote by r_{Y} (z) its distribution kernel.

We consider both, r(z) and r_{Y} (z), as distribution sections of the bundle V (γ) ⊗ V (γ̃) over X × X. We will in particular be interested in the difference d(z) := r_{Y} (z) − r(z).

Let M̂(γ) ⊂ M̂ denote the set of irreducible representations of M which appear in the restriction of γ to M. For each σ ∈ M̂ and λ ∈ 𝔞_{ℂ}^{∗} let z_{σ}(λ) denote the value of Ω on the principal series representation H^{σ,λ}. Note that z_{σ}(λ) is of the form z_{σ}(λ) = c_{σ} − λ^{2} for some c_{σ} ∈ ℝ. We define ℂ_{γ} to be the branched cover of ℂ to which the inverse functions

λ_{σ}(z) = \sqrt{c_{σ} − z}

extend holomorphically for all σ ∈ M̂(γ). We fix one sheet ℂ^{phys} of ℂ_{γ} over the set ℂ \ [b, ∞), b := min_{σ∈M̂} c_{σ}, which we call physical. We will often consider ℂ^{phys} as a subset of ℂ. It follows from the Plancherel theorem for L^{2}(X, V (γ)) and L^{2}(Y, V_{Y} (γ)) that [b, ∞) is the continuous spectrum of Ω and Ω_{Y}. Thus d(z) is defined on the complement of finitely many points of ℂ^{phys} which belong to the discrete spectrum of Ω and Ω_{Y}.

Let Δ denote the diagonal in X × X and define S :=⋃_{1≠γ∈Γ}(1 × γ)Δ ⊂ X × X. Let Ω_{i}, i = 1, 2, denote the Casimir operators of G acting on the first and the second variable of the product X × X. The distribution d(z) satisfies the elliptic differential equation

(2z − Ω_{1} − Ω_{2})d(z) = 0  (42)

on X × X \ S and is therefore smooth on this set.

Lemma 6.1 d(z) extends to ℂ_{γ} as a meromorphic family of smooth sections of V (γ) ⊗ V (γ̃) on X × X \ S.

Proof. We first show

Lemma 6.2 r(z) and r_{Y} (z) extend to ℂ_{γ} as meromorphic families of distributions.

Proof. We give the argument for r_{Y} (z) since r(z) can be considered as a special case where Γ is trivial. Let V_{i} ⊂ X, i = 1, 2 be open subsets such that the restriction to V_{i} of the projection X → Y is a diffeomorphism. We consider ϕ ∈ C_{c}^{∞}(V_{1}, V (γ̃)), ψ ∈ C_{c}^{∞}(V_{2}, V (γ)) as compactly supported sections over Y.

We now employ the Plancherel theorem for L^{2}(Y, V_{Y} (γ)) in order to show that

r_{ϕ,ψ}(z) := r_{Y} (z)(ϕ ⊗ ψ) = ⟨ϕ, (z − Ω_{Y})^{−1}ψ⟩

extends meromorphically to ℂ_{γ}. We decompose ϕ = ∑_{s∈σ_{p}(Ω_{Y} )} ϕ_{s}+ϕ_{ac}, ψ = ∑_{s∈σ_{p}(Ω_{Y} )} ψ_{s}+ψ_{ac} according to the discrete and continuous spectrum of Ω_{Y}. We have r_{ϕ,ψ}(z) = ∑_{s∈σ_{p}(Ω_{Y} )}(z − s)^{−1}⟨ϕ_{s}, ψ_{s}⟩ + ⟨ϕ_{ac}, (z − Ω)^{−1}ψ_{ac}⟩. We now employ the Eisenstein Fourier transformation in order to rewrite the last term of this equation.

For each σ ∈ M̂(γ), we consider the normalized Eisenstein series as a meromorphic family of maps

C^{−∞}(B, V_{B}(σ_{λ})) ⊗ Hom_{M}(V_{σ}, V_{γ}) ∋ f ⊗ T ↦ ^{0}E_{λ}^{T} (f) := ^{0}P_{λ}^{T} ∘ ext(f) ∈ C^{∞}(Y, V_{Y} (γ)) .

For each σ let {T_{i}(σ)}_{i}, T_{i}(σ) ∈ Hom(V_{σ}, V_{γ}), be a base, and let T^{j}(σ) ∈ Hom(Vσ̃, Vγ̃) be the dual base such that T_{i}(σ)^{∗}T^{j}(σ) = δ_{i}^{j}. If ϕ ∈ C_{c}^{∞}(Y, V_{Y} (γ̃)), then its Eisenstein Fourier transform EF Tσ̃(ϕ)(λ) ∈ C^{∞}(B, V_{B}(σ̃_{λ})) ⊗ Hom_{M}(Vσ̃, Vγ̃) is given by

⟨EF Tσ̃(ϕ)(λ), f ⊗ T_{i}(σ)⟩ := ⟨ϕ,^{0}E_{−λ}^{T_{i}(σ)}(f)⟩ = ⟨(^{0}E_{−λ}^{T_{i}(σ)})^{∗}(ϕ), f⟩ .

As a consequence of the Plancherel theorem we obtain for ϕ ∈ C_{c}^{∞}(V_{1}, V (γ̃)), ψ ∈ C_{c}^{∞}(V_{2}, V (γ)) that

⟨ϕ_{ac}, ψ_{ac}⟩ = ∑_{σ∈M̂(γ)} \frac{1}{4πω_{X}} ∫_{i𝔞^{∗}}⟨EF Tσ̃(ϕ)(−λ), EF T_{σ}(ψ)(λ)⟩dλ
= ∑_{σ∈M̂(γ)} \frac{1}{4πω_{X}} ∑_{j} ∫_{i𝔞^{∗}}⟨(^{0}E_{λ}^{T_{j}(σ)})^{∗}(ϕ), (^{0}E_{−λ}^{T^{j}(σ)})^{∗}(ψ)⟩dλ
= ∑_{σ∈M̂(γ)} \frac{1}{4πω_{X}} ∑_{j} ∫_{i𝔞^{∗}}⟨ϕ,^{0}E_{λ}^{T_{j}(σ)} ∘ (^{0}E_{−λ}^{T^{j}(σ)})^{∗}(ψ)⟩dλ .

In a similar manner we obtain

⟨ϕ_{ac}, (z − Ω)^{−1}ψ_{ac}⟩ = ∑_{σ∈M̂(γ)} \frac{1}{4πω_{X}} ∑_{j} ∫_{i𝔞^{∗}}(z − z_{σ}(λ))^{−1}⟨ϕ,^{0}E_{λ}^{T_{j}(σ)} ∘^{0}(E_{−λ}^{T^{j}(σ)})^{∗}(ψ)⟩dλ. (43)

We further investigate the summands in (43) for each σ seperately. So for z ∈ ℂ^{phys} we put

u(z) := ∫_{i𝔞^{∗}}(z − z_{σ}(λ))^{−1} ∑_{j}⟨ϕ,^{0}E_{λ}^{T_{j}(σ)} ∘^{0}(E_{−λ}^{T^{j}(σ)})^{∗}(ψ)⟩dλ .  (44)

If we define U(λ) := u(z_{σ}(λ)), then it is defined for Re(λ) > 0. We claim that

U(−λ) := U(λ) − \frac{2π}{λ} ∑_{j}⟨ϕ,^{0}E_{λ}^{T_{j}(σ)} ∘^{0}(E_{−λ}^{T^{j}(σ)})^{∗}(ψ)⟩

provides a meromorphic continuation of U to all of 𝔞_{ℂ}^{∗}. Indeed, U(λ) is meromorphic for Re(λ) < 0, too. We let λ = ϵ + iμ and show that the jump U(ϵ + iμ) − U(−ϵ + iμ) vanishes as ϵ → 0. Using the functional equation of the Eisenstein series and unitarity of the scattering matrix we see that the integrand in (44) is a symmetric function. We obtain

lim_{ϵ→0}[u(z_{σ}(ϵ + iμ)) − u(z_{σ}(−ϵ + iμ))]

= lim_{ϵ→0} ∫_{i𝔞^{∗}}[\frac{1}{z_{σ}(ϵ + iμ) − z_{σ}(λ)} − \frac{1}{z_{σ}(−ϵ + iμ) − z_{σ}(λ)}] ∑_{j}⟨ϕ,^{0}E_{λ}^{T_{j}(σ)} ∘^{0}(E_{−λ}^{T^{j}(σ)})^{∗}(ψ)⟩dλ
= lim_{ϵ→0} ∫_{𝔞^{∗}}[\frac{1}{−ϵ^{2} − 2iϵμ + μ^{2} − λ^{2}} − \frac{1}{−ϵ^{2} + 2iϵμ + μ^{2} − λ^{2}} ] ∑_{j}⟨ϕ,^{0}E_{iλ}^{T_{j}(σ)} ∘^{0}(E_{−iλ}^{T^{j}(σ)})^{∗}(ψ)⟩dλ
= lim_{ϵ→0} ∫_{0}^{∞}[\frac{1}{−ϵ^{2} − 2iϵμ + μ^{2} − ν} − \frac{1}{−ϵ^{2} + 2iϵμ + μ^{2} − ν}] ∑_{j}⟨ϕ,^{0}E_{i\sqrt{ν}}^{T_{j}(σ)} ∘^{0}(E_{−i\sqrt{ν}}^{T^{j}(σ)})^{∗}(ψ)⟩\frac{1}{\sqrt{ν}} dν
= lim_{ϵ→0} ∫_{0}^{∞}[\frac{1}{−ϵ − 2iμ + \frac{μ^{2}−ν}{ϵ}} − \frac{1}{−ϵ + 2iμ + \frac{μ^{2}−ν}{ϵ}} ] ∑_{j}⟨ϕ,^{0}E_{i\sqrt{ν}}^{T_{j}(σ)} ∘^{0}(E_{−i\sqrt{ν}}^{T^{j}(σ)})^{∗}(ψ)⟩\frac{1}{ϵ\sqrt{ν}} dν
= lim_{ϵ→0} ∫_{−μ^{2}/ϵ}^{∞}[\frac{1}{−ϵ − 2iμ − t} − \frac{1}{−ϵ + 2iμ − t}] ∑_{j}⟨ϕ,^{0}E_{i\sqrt{tϵ+μ^{2}}}^{T_{j}(σ)} ∘^{0}(E_{−i\sqrt{tϵ+μ^{2}}}^{T^{j}(σ)})^{∗}(ψ)⟩\frac{1}{\sqrt{tϵ + μ^{2}}} dt
= ∫_{−∞}^{∞}[\frac{1}{−2iμ − t} − \frac{1}{2iμ − t}] ∑_{j}⟨ϕ,^{0}E_{i\sqrt{μ^{2}}}^{T_{j}(σ)} ∘^{0}(E_{−i\sqrt{μ^{2}}}^{T^{j}(σ)})^{∗}(ψ)⟩\frac{1}{\sqrt{μ^{2}}} dt
= −\frac{2π}{iμ} ∑_{j}⟨ϕ,^{0}E_{iμ}^{T_{j}(σ)} ∘^{0}(E_{−iμ}^{T^{j}(σ)})^{∗}(ψ)⟩ .

It follows from this formula that U is continuous at the imaginary axis. This finishes the proof of the claim. We conclude that (43) extends meromorphically to ℂ_{γ}.

Thus we have shown that r_{Y} (z) is a meromorphic family of distributions if we put the topology induced by the evaluations against sections of the form ϕ ⊗ ψ. We now argue that it is indeed meromorphic with respect to the strong topology. It is clear that r_{Y} (z) is so on the sheet ℂ^{phys}. If z_{1}, z_{2} ∈ ℂ_{γ} project to z ∈ ℂ^{phys}, then by the construction above the difference r_{Y} (z_{1}) − r_{Y} (z_{2}) is a meromorphic family of of smooth sections. We conclude that r_{Y} (z) is a meromorphic family of distributions on ℂ_{γ}. □

We see that d(z) extends to a meromorphic family of distributions on ℂ_{γ}. Since it fulfills the differential equation (42) on Δ \ S we conclude that its restriction to this set is in fact a meromorphic family of smooth sections. This finishes the proof of Lemma 6.1. □

6.2 Finite propagation speed estimates

On X × X we define the function d_{o}(x, y) := inf_{1≠γ∈Γ} d(x, γy). Note that S = {d_{0} = 0}. Given ϵ > 0 we define the neighbourhood S_{ϵ} := {d_{0} ≤ ϵ} of S. Let b := inf σ(Ω_{Y}).

Lemma 6.3 Given ϵ > 0 and a compact subset W ⊂ {Re(z) < b} there is a constant C > 0 such that |d(z)(x, y)| < Ce^{−(d_{0}(x,y)−ϵ)\sqrt{b−Re(z)}} for all (x, y) ∉ S_{2ϵ} and z ∈ W.

Proof. The proof is based on the finite propagation speed of the wave operators cos(tA), cos(tA_{Y}) where A := \sqrt{Ω − b}, A_{Y} := \sqrt{Ω_{Y} − b}. We write

R(z) = ∫_{0}^{∞} e^{−t\sqrt{b−z}} cos(tA)dt, R_{Y} (z) = ∫_{0}^{∞} e^{−t\sqrt{b−z}} cos(tA_{Y})dt.

Finite propagation speed gives

d(z)(x, y) = ∫_{d_{0}(x,y)−ϵ}^{∞} e^{−t\sqrt{b−z}}[cos(tA_{Y}) − cos(tA)]dt (x, y)

on the level of distribution kernels. By partial integration

(Ω_{1} − b)^{N}(Ω_{2} − b)^{N}d(z)(x, y) = (b − z)^{2N} ∫_{d_{0}(x,y)−ϵ}^{∞} e^{−t\sqrt{b−z}}[cos(tA_{Y}) − cos(tA)]dt(x, y).

We now employ the fact that ∫_{d_{0}(x,y)−ϵ}^{∞} e^{−t\sqrt{b−z}} cos(tA_{Y})dt is a bounded operator on L^{2}(Y, V_{Y} (γ)) with norm bounded by C_{1}e^{−(d_{0}(x,y)−ϵ)Re(\sqrt{b−z})}. A similar estimate holds for the other term. If we choose N > dim(X)/4, then we can conclude that

|d(z)(x, y)| < Ce^{−(d_{0}(x,y)−ϵ)Re(\sqrt{b−z})},

where C depends on W, C_{1}, and a uniform estimate of norms of delta distributions as functionals on the Sobolev spaces W^{2N,2}(X, V (γ)) and W^{2N,2}(Y, V_{Y} (γ)) which hold because X, Y as well as the bundles V (γ), V_{Y} (γ) have bounded geometry. We further have employed the fact (which is again a consequence of bounded geometry) that we can use powers of the operator Ω, Ω_{Y} in order to define the norm of the Sobolev spaces. The assertion of the lemma now follows from Re(\sqrt{b − z}) ≥ \sqrt{b − Re(z)}. □

The distribution r(z) is smooth outside the diagonal Δ because it satisfies a differential equation similar to (42). For ϵ > 0 we define the neighbourhood Δ_{ϵ} := {d ≤ ϵ} of Δ.

Lemma 6.4 For ϵ > 0 and a compact subset W ⊂ {Re(z) < b} there is a constant and C > 0 such that |r(z)(x, y)| < Ce^{−(d(x,y)−ϵ)\sqrt{b−Re(z)}} for all (x, y) ∉ Δ_{2ϵ} and z ∈ W.

Proof. The proof is similar to that of Lemma 6.3. Using finite propagation speed we can write

r(z)(x, y) = ∫_{d(x,y)−ϵ}^{∞} e^{−t\sqrt{b−z}} cos(tA)dt(x, y),

(Ω_{1} − b)^{N}(Ω_{2} − b)^{N}r(z)(x, y) = (b − z)^{2N} ∫_{d(x,y)−ϵ}^{∞} e^{−t\sqrt{b−z}} cos(tA)dt(x, y).

We now argue as in the proof of Lemma 6.3 in order to conclude the estimate. □

Let L_{γ} denote the action of γ ∈ Γ on sections of V (γ).

Lemma 6.5 If \sqrt{b − Re(z)} > δ_{Γ} + ρ, then on X × X \ S we have d(z) = ∑_{1≠γ∈Γ}(1 ⊗ L_{γ})r(z).

Proof. It follows from Lemma 6.4 that |(1⊗L_{γ})r(z)(x, y)| < Ce^{−d(x,γ^{−1}y)\sqrt{b−Re(z)}}. Therefore the sum converges locally uniformly on X×X\S. The distribution u(z) := r_{Y} (z)−∑_{γ∈Γ}(1⊗L_{γ})r(z) satisfies the differential equations

(z − Ω_{1})u(z) = 0, (z − Ω_{2})u(z) = 0,

and is therefore a smooth section depending meromorphically on z ∈ ℂ^{phys}. We further have the estimate ∑_{1≠γ∈Γ} |(1 ⊗ L_{γ})r(z)(x, y)| < Ce^{−rd_{0}(x,y)}, where r < \sqrt{b − Re(z)} − δ_{Γ} − ρ. For Re(z) ≪ 0 we see that u(z) defines a bounded operator on L^{2}(Y, V_{Y} (γ)) and therefore vanishes. Since u is meromorphic in z it vanishes for all z with \sqrt{b − Re(z)} > δ_{Γ} + ρ. This proves the lemma. □

6.3 Boundary values

The meromorphic family of eigenfunctions d(z) on X × X \ S has meromorphic families of hyperfunction boundary values. Since we consider a product of rank one spaces it is easy to determine the leading exponents of a joint eigenfunction of Ω_{1}, Ω_{2} with eigenvalue z. These exponents are pairs of elements of 𝔞_{ℂ}^{∗}.

Lemma 6.6 The set of leading exponents of a joint eigenfunction of Ω_{1}, Ω_{2} in the bundle V (γ)⊗ V (γ̃) with generic eigenvalue z is

{μ_{(σ,ϵ),(σ′,ϵ′)}(z) | σ ∈ M̂(γ), σ′ ∈ M̂(γ̃), ϵ, ϵ′ ∈ {+, −}},

where μ_{(σ,ϵ),(σ′,ϵ′)}(z) = (−ρ + ϵλ_{σ}(z), −ρ + ϵ′λ_{σ′}(z)). The corresponding boundary value is a section of the bundle V (γ(σ)_{ϵλ_{σ}(z)}) ⊗ V (γ̃(σ′)_{ϵ′λ_{σ′}(z)}) → ∂X × ∂X, where γ(σ), γ̃(σ′) denote the isotypic components.

Proof. An eigensection of Ω in V (γ) → X has leading exponents −ρ + ϵλ_{σ}(z), ϵ ∈ {+, −}, and the corresponding boundary value is a section of V (γ(σ)_{ϵλ_{σ}(z)}). This implies the lemma. □

Let ∂Δ ⊂ ∂X × ∂X be the diagonal in the boundary and define ∂S := ⋃_{1≠γΓ}(γ × 1)(∂Δ ∩ Ω × Ω). Note that d(z) is a joint eigenfunction in a neighbourhood in X × X of Ω × Ω \ ∂S. Therefore, for generic z it has boundary values along this set [8]. We denote the boundary value associated to the leading exponent τ := μ_{(σ,ϵ),(σ′,ϵ′)}(z) by β_{τ} (f).

Lemma 6.7 We have β_{τ} (d(z)) = 0 (the meromorphic family of hyperfunctions vanishes) except for τ = μ_{(σ,−),(σ̃^{w},−)}(z), σ ∈ M̂(γ), in which case β_{τ} (d(z)) is a meromorphic family of real analytic sections.

Proof. We employ the fact that β_{τ} (d(z)) depends meromorphically on z. Let U ⊂ Ω be such that the restriction of the projection Ω → B is a diffeomorphism. There is a constant c > 0 such that for all (k_{1}, k_{2}) ∈ U × U we have e^{d_{0}(k_{1}a_{1},k_{2}a_{2})} > c| max(a_{1}, a_{2})| (see [3], Cor. 2.4). Using Lemma 6.3 we see that for Re(z) < b we have |d(z)(k_{1}a_{1}, k_{2}a_{2})| < C| max(a_{1}, a_{2})|^{−\sqrt{b−Re(z)}}, where C depends on z. If one of the signs ϵ, ϵ′ is positive, for z ≪ 0 we have \sqrt{b − z} − 2ρ + ϵλ_{σ}(z) + ϵ′λ_{σ′}(z) > 0. For those z we have lim_{min(a_{1},a_{2})→∞} d(z)(k_{1}a_{1}, k_{2}a_{2})(a_{1}, a_{2})^{−μ_{(σ,ϵ),(σ′,ϵ′)}(z)} = 0 uniformly in (k_{1}, k_{2}), where (a, b)^{(μ,ν)} := a^{μ}b^{ν}. This shows that β_{τ} (d(z)) = 0 if one of ϵ, ϵ′ is positive.

We now consider the kernel r(z) on X × X \ Δ. It is a joint eigenfunction of Ω_{1}, Ω_{2} to the eigenvalue z on a neighbourhood of ∂X × ∂X \ ∂Δ and therefore has hyperfunction boundary values along this set. A similar argument as above but using Lemma 6.4 instead of 6.3 shows that β_{τ} (r(z)) = 0 except for ϵ = ϵ′ = −.

Note that r(z) is G-invariant in the sense that for g ∈ G we have L_{g} ⊗ L_{g}r(z) = r(z). If τ = μ_{(σ,−),(σ′,−)}(z), then β_{τ} (r(z)) is a G-invariant hyperfunction section of V (γ(σ)_{−λ_{σ}(z)}) ⊗ V (γ̃(σ′)_{−λ_{σ′(z)}}) over ∂X × ∂X \ ∂Δ. Since this set is an orbit of G, an invariant hyperfunction on this set is smooth, and the evaluation at the point (w, 1) ∈ ∂X × ∂X provides an injection of the space of invariant sections into V_{γ}(σ) ⊗ Vγ̃(σ′). If b is such a G-invariant section, then we have for ma ∈ MA

b(w, 1) = b(maw, ma)
= b(wm^{w}a^{−1}, ma)
= γ(wm^{−1}w) ⊗ γ̃(m^{−1})a^{λ_{σ′}(z)−λ_{σ}(z)}b(w, 1) .

Thus b(w, 1) ∈ [V_{γ^{w}}(σ^{w}) ⊗ Vγ̃(σ′)]^{M}. We conclude that σ′ ≅ σ̃^{w}, and in this case λ_{σ′}(z) = λ_{σ}(z) holds automatically. Thus β_{τ} (r(z)) = 0 if σ′ ≅ ˜σ̸^{w}.

We write

V (γ(σ)_{−λ}) ⊗ V (γ̃(σ̃^{w})_{−λ}) = V (σ_{−λ}) ⊗ V (σ̃_{−λ}^{w}) ⊗ Hom_{M}(V_{σ}, V_{γ}) ⊗ Hom_{M}(V_{}_{σ̃^{w}}, Vγ̃) .

The space of invariant sections of V_{σ_{−λ}} ⊗V_{}_{σ̃_{−λ}^{w}} over ∂X ×∂X \∂Δ is spanned by the distribution kernel ĵ_{σ_{λ}^{w}}^{w} of the Knapp-Stein intertwining operator Ĵ_{σ_{λ}^{w}}^{w}. We conclude that for each σ ∈ M̂(γ) there is a meromorphic family A_{σ}(z) ∈ Hom_{M}(V_{σ}, V_{γ}) ⊗ Hom_{M}(V_{}_{σ̃^{w}}, Vγ̃) such that for τ = μ_{(σ,−),(σ̃^{w},−)}(z) we have β_{τ} (r(z)) = ĵ_{σ_{λσ(z)}^{w}}^{w} ⊗ A_{σ}(z) under the identifications above.

Let τ = μ_{(σ,−),(σ̃^{w},−)}(z). We now employ Lemma 6.5 which states that for Re(z) ≪ 0 we have d(z) = ∑_{1≠γ∈Γ}(L_{γ} ⊗ 1)r(z). The sum converges locally uniformly and thus in the space of smooth section over X × X \ S. We further see that convergence holds locally uniformly in a neighbourhood of Ω × Ω \ ∂S. Thus by [10] we can consider distribution boundary values, and by continuity of the boundary value map we have on Ω × Ω \ ∂S

β_{τ} (d(z)) = ∑_{1≠γ∈Γ}(π^{σ_{−λσ(z)}}(γ) ⊗ 1)β_{τ} (d(z))
= ∑_{1≠γ∈Γ}(π^{σ_{−λσ(z)}}(γ) ⊗ 1)ĵ_{σ^{w},λ_{σ}(z)}^{w} ⊗ A_{σ}(z)
= (ŝ_{σ^{w},λ_{σ}(z)}^{w} − ĵ_{σ^{w},λ_{σ}(z)}^{w}) ⊗ A_{σ}(z) ,

where ŝ_{σ^{w},λ_{σ}(z)}^{w} is the distribution kernel of the scattering matrix Ŝ_{σ^{w},λ_{σ}(z)}^{w}. Here we use the identity π_{∗} ∘ Ĵ_{σ^{w},λ_{σ}(z)}^{w} = Ŝ_{σ^{w},λ_{σ}(z)}^{w} ∘ π_{∗} which implies that the distribution kernel of the scattering matrix Ŝ_{σ^{w},λ}^{w} can be obtained by averaging the distribution kernel of the Knapp-Stein intertwining operator Ĵ_{σ^{w},λ}^{w} for Re(λ) ≫ 0.

It follows from the results of [3] that ŝ_{σ_{λσ(z)}^{w}}^{w} − ĵ_{σ_{λσ(z)}^{w}}^{w} extends to a meromorphic family of smooth sections on all of 𝔞_{ℂ}^{∗}. It follows from [4], Lemma 2.19, 2.20, that it is indeed a meromorphic family of real analytic sections. Strictly speaking, in [4] we only considered the spherical M-type for G = SO(1, n), but the same arguments can be applied in the general case. We conclude that β_{τ} (d(z)) is real analytic as required.

A similar reasoning shows that β_{τ} (d(z)) = 0 for all τ which are not of the form μ_{(σ,−),(σ̃^{w},−)}(z) for some σ ∈ M̂(γ). □

Lemma 6.8 We have an asymptotic expansion (for generic z)

d(z)(k_{1}a, k_{2}a) ∼^{a→∞} ∑_{σ∈M̂(γ)} ∑_{n=0}^{∞} a^{−2ρ−2λ_{σ}(z)−nα}p_{z,σ,n}(k_{1}, k_{2})  (45)

which holds locally uniformly for k ∈ Ω×Ω\∂S, and where the real analytic sections p_{z,σ,n}(k_{1}, k_{2}) of V (γ(σ)_{−λ}) ⊗ V (γ̃(σ̃^{w})_{−λ}) depend meromorphically on z.

Proof. Since the boundary value of d(z) along Ω × Ω \ ∂S is real analytic we can employ [10], Prop. 2.16, in order to conclude that d(z) has an asymptotic expansion with coefficients which depend meromorphically on z. The formula follows from an inspection of the list of leading exponents Lemma 6.7. □

Lemma 6.8 has the following consequence. For generic z we have

tr d(z)(ka, ka) ∼^{a→∞} ∑_{σ∈M̂(γ)} ∑_{n=0}^{∞} a^{−2ρ−2λ_{σ}(z)−nα}p_{z,σ,n}(k)

which holds locally uniformly for k ∈ Ω, and where the real analytic functions p_{z,σ,n} depend meromorphically on z.

6.4 The regularized trace of the resolvent

Lemma 6.9 The integral Q_{γ}(z) := ∫_{X} χ^{Γ}(x)tr d(z)dx converges for Re(z) ≪ 0 and admits a meromorphic continuation to all of ℂ_{γ}.

Proof. Convergence for Re(z) ≪ 0 follows from Lemma 6.3. Fix R ∈ A. We write Q_{γ}(z) = Q_{1}(z, R) + Q_{2}(z, R), where

Q_{1}(z, R) := ∫_{1}^{R} ∫_{K} χ^{Γ}(ka)tr d(z)(ka)dkv(a)da ,

where v is such that dk v(a)da is the volume measure on X. Note that v(a) ∼ a^{2ρ}(ω_{X} + a^{−α}c_{1} + a^{−2α}c_{2} + ...) as a → ∞.

It is clear that Q_{1}(z, R) admits a meromorphic continuation. We have an asymptotic expansion as a → ∞.

u(z, a) := ∫_{K} χ^{Γ}(ka)tr d(z)(ka)dkv(a) ∼ ∑_{σ∈M̂(γ)} ∑_{n=0}^{∞} a^{−2λ_{σ}(z)−nα}q_{z,σ,n} ,

where q depends meromorphically on z. For m ∈ ℕ let

u_{m}(z, a) := u(z, a) − ∑_{σ∈M̂(γ)} ∑_{n=0}^{m} a^{−2λ_{σ}(z)−nα}q_{z,σ,n} .

Given a compact subset W of ℂ_{γ} we can choose m ∈ ℕ_{0} such that ∫_{R}^{∞} u_{m}(z, a)da converges (for generic z) and depends meromorphically on z for all z ∈ W. We further have

∑_{σ∈M̂(γ)} ∑_{n=0}^{m} ∫_{R}^{∞} a^{−2λ_{σ}(z)−nα}q_{z,σ,n}da = ∑_{σ∈M̂(γ)} ∑_{n=0}^{m} \frac{R^{−2λ_{σ}(z)−nα}q_{z,σ,n}}{2λ_{σ}(z) + nα} ,

and this function extends meromorphically to C_{γ}. Since we can choose W arbitrary large we conclude that Q_{γ}(z) admits a meromorphic continuation to all of ℂ_{γ}. □

6.5 A functional equation

Let L(γ) := {c_{σ} | σ ∈ M̂(γ)} be the set of ramification points of ℂ_{γ}, define ℂ^{♯} := ℂ \ L(γ), and let ℂ_{γ}^{♯} ⊂ ℂ_{γ} be the preimage of ℂ^{♯} under the projection ℂ_{γ} → ℂ. Then ℂ_{γ}^{♯} → ℂ^{♯} is a Galois covering with group of deck transformations Π := ⊕_{L(γ)}ℤ_{2}. The action of Π extends to ℂ_{γ} such ℂ_{γ} \ ℂ_{γ}^{♯} consists of fixed points. For l ∈ L(γ) let q_{l} ∈ Π be the corresponding generator. Then we have λ_{σ}(q_{l}z) = −λ_{σ}(z) for all σ ∈ M̂(γ) with c_{σ} = l and λ_{σ′}(q_{l}z) = λ_{σ′}(z) else.

Lemma 6.10 For l ∈ L(γ) we have

Q_{γ}(q_{l}z) − Q_{γ}(z) = ∑_{σ∈M̂(γ),c_{σ}=l} \frac{−[γ : σ]}{2λ_{σ}(z)}L_{Γ}(π^{σ,λ_{σ}(z)}) .

Proof. In the proof of Lemma 6.1 we have seen that (using the notation introduced there)

⟨r_{Y} (q_{l}z) − r_{Y} (z), ϕ ⊗ ψ⟩ = ∑_{σ∈M̂(γ),c_{σ}=l} \frac{−1}{2ω_{X}λ_{σ}(z)} ∑_{j}⟨ϕ,^{0}E_{λ_{σ}(z)}^{T_{j}(σ)} ∘^{0}(E_{−λ_{σ}(z)}^{T^{j}(σ)})^{∗}(ψ)⟩ .

We conclude that

r_{Y} (q_{l}z) − r_{Y} (z) = ∑_{σ∈M̂(γ),c_{σ}=l} \frac{−1}{2ω_{X}λ_{σ}(z)} ∑_{j}^{0}E_{λ_{σ}(z)}^{T_{j}(σ)} ∘^{0}(E_{−λ_{σ}(z)}^{T^{j}(σ)})^{∗} .

The same reasoning applies to the trivial group Γ, where the Eisenstein series get replaced by the Poisson transformations. Thus we can write

d(q_{l}z) − d(z) = ∑_{σ∈M̂(γ),c_{σ}=l} \frac{−1}{2ω_{X}λ_{σ}(z)} ∑_{j}^{0}P_{λ_{σ}(z)}^{T_{j}(σ)} ∘ (ext ∘ π_{∗} − 1) ∘ (^{0}P_{−λ_{σ}(z)}^{T^{j}(σ)})^{∗} .

The proof of Lemma 6.9 shows that Q_{1}(z, R) has an asymptotic expansion

Q_{1}(z, R) ∼ Q_{γ}(z) + ∑_{σ∈M̂(γ)} ∑_{n=0}^{∞} \frac{R^{−2λ_{σ}(z)−nα}q_{z,σ,n}}{2λ_{σ}(z) + nα} .

In particular, if 2λ_{σ}(z) ∉ −ℕ_{0}α for all σ, then Q_{γ}(z) is the constant term in the asymptotic expansion of Q_{1}(z, R).

We can now apply Proposition 4.4 which can be interpreted as the determination of the constant term (as R → ∞) of

∫_{1}^{R} ∫_{K} χ^{Γ}(ka)tr [^{0}P_{λ}^{T_{j}(σ)} ∘ (ext ∘ π_{∗} − 1) ∘ (^{0}P_{−λ}^{T^{j}(σ)})^{∗}] (ka, ka)dkv(a)da

as a distribution on i𝔞^{∗} \ {0}. This shows the desired equation first on z_{σ}(i𝔞^{∗}) and then everywhere by meromorphic continuation. □

6.6 Selberg zeta functions

For a detailed investigation of Selberg zeta functions associated to bundles (for cocompact Γ) we refer to [2]. We assume that σ is irreducible and Weyl invariant, or that it is of the form σ′ ⊕ (σ′)^{w} for some non-Weyl invariant irreducible M-type σ′. In the latter case we define L_{Γ}(π^{σ,λ}) := L_{Γ}(π^{σ′,λ}) + L_{Γ}(π^{(σ′)^{w},λ}).

Let P = MAN be a parabolic subgroup of M. If γ ∈ Γ, then it can be conjugated in G to an element m_{g}a_{g} ∈ MA with a_{g} > 1. Let 𝔫̄ be the negative root space of (𝔤, 𝔞). For Re(λ) > ρ we can define the Selberg zeta function Z_{S}(λ, σ) by the converging infinite product

Z_{S}(λ, σ) := ∏_{1≠[g]∈CΓ} ∏_{k=0}^{∞} det (1 − σ(m_{g}) ⊗ S^{k}(Ad(m_{g}a_{g})_{|𝔫̄})a_{g}^{−λ−ρ}) .

In the case of cocompact Γ it was shown by [5] that Z_{S}(λ, σ) has a meromorphic continuation to all of 𝔞_{ℂ}^{∗}. In [11] it was explained that the argument of [5] extends to the case of convex cocompact subgroups since it is the compactness of the non-wandering set of the geodesic flow of Y that matters and not the compactness of Y. Strictly speaking, [11] deals with the spherical case of SO(1, 2n), but the argument extends to the general case.

There is a virtual representation γ of K (i.e. an element of the integral representation ring of K) such that γ_{|M} = σ in the integral representation ring of M (see [9], [2]). We call γ a lift of σ. Note that γ is not unique. We can extend the material developed above to virtual K-types by taking the traces with corresponding signs. Because of the factor [γ : σ] Lemma 6.10 has the following corollary.

Corollary 6.11 If γ is a lift of σ, then Q_{γ}(z) extends to a twofold branched cover of ℂ associated with λ_{σ}(z).

Theorem 6.12 The Selberg zeta function satisfies

\frac{Z_{S}(λ, σ)}{Z_{S}(−λ, σ)} = exp ∫_{0}^{λ} L_{Γ}(π^{σ,λ})dλ .

In particular, the residues of L_{Γ}(π^{σ,λ}) are integral.

Proof. By [2] Prop. 3.8. we have

Q_{γ}(z_{σ}(λ)) = \frac{1}{2λ}Z′_{S}(λ, σ)/Z_{S}(λ, σ)  (46)

for Re(λ) ≫ 0. Indeed, Q_{γ}(z_{σ}(λ)) is just what is called in [2] the hyperbolic contribution associated to the resolvent. So Corollary 6.11 and Lemma 6.10 yields the functional equation of the logarithmic derivative of the Selberg zeta function

\frac{Z′_{S}(−λ, σ)}{Z_{S}(−λ, σ)} + \frac{Z′_{S}(λ, σ)}{Z_{S}(λ, σ)} = L_{Γ}(π^{σ,λ}) .

Integrating and employing the apriori information that Z_{S}(λ, σ) is meromorphic we obtain the desired functional equation. □

\underline{Remarks :}

1. As explained in the introduction it is known (from the approach to Z_{S} using symbolic dynamics and Ruelles thermodynamic formalism) that Z_{S}(λ) is a meromorphic function of finite order. It follows that L_{Γ}(π^{σ,λ}), as a function of λ, grows at most polynomially.

2. In order to describe the singularities of Z_{S}(λ, σ) we assume that γ is an admissible lift of σ (see [2]). Let n_{λ,σ} denote the (virtual) dimension of the subspace of the L^{2}-kernel of Ω_{Y} − z_{σ}(λ) on V_{Y} (γ) which is generated by non-discrete series representations of G. It follows from Theorem 6.12 that

ord_{λ=μ}Z_{S}(λ, σ) = ( \begin{matrix}res_{λ=μ}L_{Γ}(π^{σ,λ}) + n_{−μ,σ} & Re(μ) < 0 \\ n_{μ,σ} & 0 < Re(μ) \end{matrix}

3. If μ is non-integral and if ext has a pole at μ of at most first order, then one has

res_{λ=μ}L_{Γ}(π^{σ,λ}) = dim ^{Γ}C^{−∞}(Λ, V (σ, μ)) ,

where ^{Γ}C^{−∞}(Λ, V (σ, μ)) is the space of invariant distributions with support on the limit set Λ. If ext has higher order singularity, then the residue has a similar interpretation (see [4], Prop. 5.6) This provides an independent argument for the integrality of the residues of L_{Γ}(π^{σ,λ}).

6.7 Integrality of N_{Γ}(π) for discrete series representations

Let π be a discrete series representation of G containing the K-type γ̃. There are embeddings M_{π}⊗V_{π}(γ̃) ↪ L^{2}(Γ\G)(γ̃) and Vπ̃⊗V_{π}(γ̃) ↪ L^{2}(G). Let A ∈ End_{K}(V_{π}(γ̃)) be given. We extend A by zero to the orthogonal complement of V_{π}(γ̃) thus obtaining an operator in End_{K}(V_{π}) which we will still denote by A. The operator A induces operators Ř_{Γ}(h_{A}) and Ř_{Γ}(h_{A}) on L^{2}(Γ\G) and L^{2}(G), where h_{A} is supported on {π} ⊂ Ĝ and h_{A}(π) := A.

Lemma 6.13 We have

K_{Ř_{Γ}(h_{A})}(g, g) − K_{Ř(h_{A})}(g, g) ∈ L^{1}(Γ\G) .

Proof. Let D(G, γ) be the algebra of invariant differential operators on V (γ). It is isomorphic to (U(𝔤) ⊗_{U(𝔨)} End(V_{γ}))^{K}. If π′ is an admissible representation of G, then D(G, γ) acts in a 6.7 Integrality of N_{Γ}(π) for discrete series representations 43

natural way on (V_{π′} ⊗V_{γ})^{K}. If π′ is irreducible, then (V_{π′} ⊗V_{γ})^{K} is an irreducible representation of D(G, γ). The correspondence π′ ↦ (V_{π′} ⊗ V_{γ}) provides a bijection between the sets of equivalence classes of irreducible representations of G containing the K-type γ̃ and irreducible representations of D(G, γ).

Note that End_{K}(V_{π}(γ̃)) ≅ End((V_{π} ⊗ V_{γ})^{K}). We conclude that there is D_{A} ∈ D(G, γ) that induces the endomorphism A on V_{π}(γ̃). Let z_{0} be the eigenvalue of the Casimir operator on π and Z be the finite set of irreducible representations of G containing the K-type γ̃ such that Ω acts with eigenvalue z_{0}. Then we can choose D_{A} such that it vanishes on all (V_{π′} ⊗ V_{γ})^{K} for π′ ∈ Z, π′ ≠ π.

For simplicity we assume that z_{0} is not a branching point of R(z). In the latter case the following argument can easily be modified. The operators D_{A}R_{Y} (z) and D_{A}R(z) have poles at z_{0} with residues K_{Ř_{Γ}(h_{A})} and K_{Ř(h_{A})}. The difference (D_{A})_{1}d(z) := (D_{A})_{1}r_{Y} (z) − (D_{A})_{1}r(z) of distribution kernels is still a meromorphic family of joint eigenfunctions with real analytic boundary values along Ω × Ω \ ∂S. We have the asymptotic expansion

(D_{A})_{1}d(z)(k_{1}a, k_{2}a) ∼^{a→∞} ∑_{σ∈M̂(γ)} ∑_{n=0}^{∞} a^{−2ρ−2λ_{σ}(z)−nα}p_{z,σ,n,A}(k_{1}, k_{2}) .  (47)

The residue of (D_{A})_{1}d(z) at z_{0} can be computed by integrating (D_{A})_{1}d(z) along a small circle counter-clockwise surrounding z_{0}. If we insert the asymptotic expansion (47) into this integral, then we obtain an asymptotic expansion

res_{z=z_{0}}(D_{A})_{1}d(z)(k_{1}a, k_{2}a) ∼^{a→∞} ∑_{σ∈M̂(γ)} ∑_{n=0}^{∞} res_{z=z_{0}}a^{−2ρ−2λ_{σ}(z)−nα}p_{z,σ,n,A}(k_{1}, k_{2})
∼ ∑_{σ∈M̂(γ)} ∑_{n=0}^{∞} ∑_{m=0}^{finite} log(a)^{m}a^{−2ρ−2λ_{σ}(z_{0})−nα}p_{z_{0},σ,n,m,A}(k_{1}, k_{2}) ,

where p_{z_{0},σ,n,m,A} is a real analytic section on Ω × Ω \ ∂S. Since K_{Ř_{Γ}(h_{A})} and K_{Ř(h_{A})} project onto eigenspaces of square integrable sections we have for k_{1} ≠ k_{2}

K_{Ř(h_{A})}(k_{1}a_{1}, k_{2}a_{2}) ∼^{a_{i}→∞}

∑_{n_{1},n_{2}=0}^{∞} ∑_{m_{1},m_{2}}^{finite} log(a_{1})^{m_{1}} log(a_{2})^{m_{2}}a_{1}^{−ρ−λ_{σ}(z_{0})−n_{1}α}a_{2}^{−ρ−λ_{σ}(z_{0})−n_{2}α}p_{z_{0},σ,n_{1},n_{2},m_{1},m_{2},A}(k_{1}, k_{2})

with p_{z_{0},σ,n_{1},n_{2},m_{1},m_{2},A}(k_{1}, k_{2}) = 0 as long as −λ_{σ}(z_{0})−n_{1}α ≥ 0, −λ_{σ}(z_{0})−n_{2}α ≥ 0, and similar for K_{Ř_{Γ}(h_{A})}. We conclude that Re(−2λ_{σ}(z) − nα) < 0 if p_{z_{0},σ,n,m,A} ≠ 0. The assertion of the lemma now follows. □

Proposition 6.14 If π be a representation of the discrete series of G, then N_{Γ}(π) ∈ ℤ.

Proof. There exists an invariant generalized Dirac operator D acting on a graded vector bundle E → X, E = E^{+} ⊕ E^{−}, such that Vπ̃ ⊕ {0} is the kernel of D. If γ is the graded K-type associated to E, then

γ_{|M} = 0 .  (48)

Let D_{Y} be the induced operator on Y. The distribution kernels of r(z) := (z − D^{2})^{−1} and r_{Y} (z) := (z − D_{Y}^{2} )^{−1} have meromorphic continuations to a branched covering of ℂ. Their difference goes into the functional Ψ′. The function

Q(z) := ∫_{Γ\G} tr (r_{Y} (z) − r(z))(g, g)dg

has a meromorphic continuation to all of ℂ by (48) and Lemma 6.10. Its residue at z = 0 is given by

res_{z=0}Q(z) = N_{Γ}(π) + ∑_{π′∈Ĝ_{c}} n(γ, π′)N_{Γ}(π′) ,

where the sum reflects the fact that a finite number of representations belonging to Ĝ_{c} may contribute to the kernel of D_{Y}. Note that n(γ, π′) ∈ ℤ and N_{Γ}(π′) ∈ ℕ_{0}. We show that res_{z=0}Q(z) = 0 in order to conclude that N_{Γ}(π) = − ∑_{π′∈G_{c}} n(γ, π′)N_{Γ}(π′) ∈ ℤ.

It suffices to show that Q(z) ≡ 0 for Re(z) ≪ 0. This follows from (46), but we will give an independent argument. We can write

r_{Y} (z) − r(z) = \frac{1}{z}((D_{Y}^{2} )_{1}r_{Y} (z) − (D^{2})_{1}r(z)) .

Integrating the restriction of this difference to the diagonal over Γ\G we obtain

Q(z) = \frac{1}{z} ∫_{Γ\G} tr ((D_{Y}^{2} )_{1}r_{Y} (z) − (D^{2})_{1}r(z))(g, g)dg
= \frac{1}{z} ∫_{G} χ^{Γ}(g)tr ((D_{Y}^{2} )_{1}r_{Y} (z) − (D^{2})_{1}r(z))(g, g)dg
= −\frac{1}{z} ∫_{G} χ^{Γ}(g)tr ((D_{Y})_{1}(D_{Y})_{2}r_{Y} (z) − D_{1}D_{2}r(z))(g, g)dg

−\frac{1}{z} ∫_{G} tr c(dχ^{Γ})_{1}((D_{Y})_{1}r_{Y} (z) − D_{1}r(z))(g, g)dg

= −\frac{1}{z} ∫_{G} χ^{Γ}(g)tr ((D_{Y}^{2} )_{1}r_{Y} (z) − D_{1}^{2}r(z))(g, g)dg

−\frac{1}{z} ∫_{G} tr c(dχ^{Γ})_{1}((D_{Y})_{1}r_{Y} (z) − D_{1}r(z))(g, g)dg ,

where c(dχ^{Γ}) denotes Clifford multiplication. We conclude that

Q(z) = −\frac{1}{2z} ∫_{G} tr c(dχ^{Γ})_{1}((D_{Y})_{1}r_{Y} (z) − D_{1}r(z))(g, g)dg .

The right hand side of this equation vanishes as a consequence of ∑_{γ∈Γ} γ^{∗}χ^{Γ} ≡ 1 and the Γinvariance of ((D_{Y})_{1}r_{Y} (z) − D_{1}r(z))(g, g). This finishes the proof of the proposition. □

References

[1] J. N. Bernstein. On the support of Plancherel measure. J. of Geom. and Phys., 5(1988), 663–710.

[2] U. Bunke and M. Olbrich. Selberg Zeta and Theta Functions. Akademie Verlag, 1995.

[3] U. Bunke and M. Olbrich. The spectrum of Kleinian manifolds. Submitted to J. Funct. Anal., Preprint available at http://www.uni-math.gwdg.de/bunke/spzerl.dvi, 1996.

[4] U. Bunke and M. Olbrich. Group cohomology and the singularities of the Selberg zeta function associated to a Kleinian group. Annals. of. Math., 149 (1999), 627–689.

[5] D. Fried. The zeta functions of Ruelle and Selberg I. Ann. scient. éc. norm. sup. 4^{e} Série, 19 (1986), 491–517.

[6] L. Guillopé. Fonctions zéta de Selberg et surface de géométrie finie. Adv. Stud. Pure Math., 21(1992), 33–70.

[7] Harish-Chandra. Harmonic analysis on real reductive groups III. The Maass-Selberg relations and the Plancherel formula. Ann. of Math., 104 (1976), 117–201.

[8] M. Kashiwara and Oshima. Systems of differential equations with regular singularities and their boundary value problems. Ann. Math., 106(1977), 145–200.

[9] R. J. Miatello and J. A. Vargas. On the distribution of the principal series in L^{2}(Γ\G). Trans. AMS, 279 (1983), 63–75.

[10] T. Oshima and J. Sekiguchi. Eigenspaces of invariant differential operators on an affine symmetric space. Invent. math., 57(1980), 1–81.

[11] S. J. Patterson and P. A. Perry. The Divisor of Selberg’s zeta function for Kleinian groups. Preprint, 1999.