Let Ω be a domain in ℝP^{n}. Suppose that there exist a neighborhood U of x ∈ ∂Ω and a real-valued function ρ defined on U ∩Ω satisfying

• ρ is twice differentiable on U ∩ Ω.

• U ∩ Ω is defined by ρ(x) < 0.

• The gradinet ∇ρ = (\frac{∂ρ}{∂x_{1}}, ⋯, \frac{∂ρ}{∂x_{n}}) is never

zero at any point of U ∩ ∂Ω.

In such a case, ρ is called a defining function of Ω at x. Notice that by the Implicit Function Theorem, the boundary U ∩ ∂Ω is C^{2} smooth.

Let Q be a quadratic form

Q(a_{2} ⋯, a_{n}) = {[x_{0} : ⋯: x_{n}] | x_{0}x_{1} + a_{2}x_{2}^{2} + ⋯ + a_{n}x_{n}^{2} < 0}

where a_{i} ∈ ℝ. We denote the Hessian of ρ(x) at p by Hess_{ρ}(p). First, we consider a projective domains with det Hess_{ρ}(p) ≠ 0 at a boundary accumulation point.

For x ∈ ℝP^{n}, we define n−simplex at x to be a convex n−simplex with a vertex x and we call the end point of the n−simplex at x to be the other vertex of the n−simplex at x and the axis of the n−simplex at x to be the line segment connecting x and the other vertex of the n−simplex at x. We call the simpex with vertexes (1, 0, ⋯, 0), ⋯, (0, ⋯, 0, 1) by the standard unit simplex at 0.

Theorem Let Ω be a projective domain not necessarily hyperbolic with non-compact automorphism group Aut(Ω). Assume that there is a sequence {φ_{ν}} ⊂ Aut(Ω) such that for a subsequential limit φ, R(φ) = p ∈ ∂Ω. If ∂Ω is C^{2} in a neighborhood of p and the Hessian at p is non-singular, then Ω is projectively equivalent to one of Q(a_{2}, ⋯, a_{n}) with a_{ℓ} ≠ 0 for all ℓ ∈ {2, ⋯, n}.

PROOF We’ll proove the Theorem for n = 2. One can assume that a_{2} > 0. Then p is a strictly convex point. One can see that q ∉ K(φ) for a subsequential limit φ of {φ_{ν}}. If one takes an open neighborhood N ⊂ Ω of q such that N ∩ K(φ) = ∅, the limit lim_{ν→∞} φ_{ν}(N) is open in R(φ) containing p. Clearly, R(f) is tangential to ∂Ω at p. It follows that dim R(φ) ∩ ∂Ω = dim R(φ). By strictly convexity, one obtains R(φ) = {p}.

Take a simplex E_{0} at p_{0} ∈ ∂Ω such that E_{0} ∩ K(φ) = ∅ and one axis is tangent to ∂Ω at p_{0}. Then the other axis is transversal to T_{p_{0}}∂Ω.

Take an affine coordinate ϕ : U_{0} ⟶ ℝ^{2} with ϕ(p) = 0 and p_{0} ∈ U_{0}. Denote φ_{ν}(p_{0}) = (c_{1}^{ν}, c_{2}^{ν}) = c^{ν} and let {e_{1}^{∘}, e_{2}^{∘}} be the end points of the axes of E_{0}, where the axis having end point e_{1}^{∘}, is not tangential to ∂Ω. Let d^{ν} = (d_{1}^{ν}, d_{2}^{ν}) = φ_{ν}(e_{1}^{∘}) be the end point of the axis of φ_{ν}(E_{0}) which is transversal to T_{φ_{ν}(p_{0})}∂Ω. We know that φ_{ν}(E_{0}) → φ(E_{0}) = p uniformly. Hence φ_{ν}(p_{0}) → p = 0 and d^{ν} → 0 uniformly.

From the Talyor expansion of second order, there exists a neighborhood U of p such that U ∩ Ω is defined by

ρ(x) = x_{1} + δ_{2}x_{2}^{2} + o(|x_{2}|^{2}) < 0

where ρ is the defining function and δ_{2} ∈ ℝ. Note that one has δ_{2} ≠ 0 since det Hess_{ρ}(p) ≠ (I) For each ν,

Φ_{ν} = \{ \begin{matrix}y_{1} = x_{1} − c_{1}^{ν} − a_{2}^{ν}(x_{2} − c_{ℓ}^{ν}) \\ y_{2} = x_{2} − c_{2}^{ν} \end{matrix}

where a_{2}^{ν} are determined so that, in the coordinate (y_{1}, y_{2}), we have

1. Φ_{ν}(c^{ν}) is the point (0, 0) ∈ ∂Ω

2. the tangent space to ∂Ω at (0, 0) is given by {y | y_{1} = 0}.

If Φ_{ν}(d^{ν}) = (β_{1}^{ν}, β_{2}^{ν}), β_{1}^{ν} = d_{1}^{ν}−c_{1}^{ν}−a_{2}^{ν}(d_{2}^{ν}−c_{2}^{ν}). Moreover, one can take the simplex E_{0} at p_{0} so that β_{1}^{ν} > 0 for all ν. For all sufficiently large ν, \frac{∂ρ}{∂x_{1}}(c^{ν}) ≠ 0 since c^{ν} → 0 as ν → ∞.

Determine a_{2}^{ν} so that

\begin{aligned}0 = \frac{∂ρ_{ν}}{∂y_{2}}(0) & = & ∑_{k=1}^{2} \frac{∂ρ}{∂x_{k}}(c^{ν})\frac{∂Φ_{k}^{−1}}{∂y_{2}} (0) \\ & = & a_{2}^{ν} \frac{∂ρ}{∂x_{1}}(c^{ν}) + \frac{∂ρ}{∂x_{2}}(c^{ν}) \end{aligned}

It follows that

0 ≠ \frac{∂ρ_{ν}}{∂y_{1}}(0) = 2 ; ∑ ; k=1 \frac{∂ρ}{∂x_{k}}(c^{ν})\frac{∂Φ_{k}^{−1}}{∂y_{1}} (0) = \frac{∂ρ}{∂x_{1}}(c^{ν})

Since all c_{ℓ}^{ν}’s, d_{ℓ}^{ν}’s, a_{ℓ}^{ν}’s approach 0 as ν → ∞, we have β_{1}^{ν} → 0 as ν → ∞.

The defining function ρ_{ν}(y) is, in this coordinate,

ρ_{ν}(y) = y_{1} + A_{2}^{ν}y_{2}^{2} + δ_{2}y_{2}^{2} + o(|y_{2}|^{2}) < 0

where A_{2}^{ν} is a function of c_{2}^{ν} satisfying

A_{2}^{ν} → 0 as ν → ∞.

One axis of E_{ν} = (Φ_{ν} ∘ φ_{ν})(E_{0}) lie in the tangent space T_{(Φ_{ν}∘φ_{ν})(p_{0})} ∂Ω and the other axis is transversal to this tangent space by definition of Φ_{ν} and φ_{ν} ∈ Aut(Ω). So, β_{1}^{ν} cannot be 0 for all ν. Define ϵ_{ν} > 0 to be β_{1}^{ν}. Note that ϵ_{ν} → 0 as ν → ∞ since φ_{ν}(E_{0}) → p by R(φ) = p.

(II) For each ν, define

Ψ_{ν} = \{ \begin{matrix}u_{1} = y_{1} \\ u_{2} = −\frac{β_{2}^{ν}}{β_{1}^{ν}}y_{1} + y_{2} \end{matrix}

This linear transformation send the axis of the simplex E_{ν} to the normal direction at 0 i.e. on the x_{1}-axis and the other axis are invariant i.e. belong to the tangent space of the boundary ∂Φ_{ν}(Ω) at 0.

The defining function ρ_{ν}(u) is, in u−coordinate,

\begin{matrix}ρ_{ν}(u) = u_{1} + A_{2}^{ν}(u_{2} + \frac{β_{2}^{ν}}{β_{1}^{ν}}u_{1}) + δ_{2}(u_{2} + \frac{β_{2}^{ν}}{β_{1}^{ν}}u_{1})^{2} \\ +o (|(u_{2} + \frac{β_{2}^{ν}}{β_{1}^{ν}}u_{1})|^{2}) < 0 \end{matrix}

(III) By two affine coordinate changes above, one can see that (Ψ_{ν} ∘ Φ_{ν} ∘ φ_{ν})(e_{2}^{∘}) has the first coordinate 0 and we also have (Ψ_{ν} ∘ Φ_{ν} ∘ φ_{ν})(e_{1}^{∘}) = (β_{1}^{ν}, 0) with β_{1}^{ν} > 0. Denote B_{ℓ}^{ν} = |(Ψ_{ν} ∘Φ_{ν} ∘φ_{ν})(e_{ℓ}^{∘})|. Let (0, r_{2ℓ}^{ν}) = (Ψ_{ν} ∘ Φ_{ν} ∘ φ_{ν})(e_{2}^{∘}).

Define R_{ν} to be the affine transformation

defined by the matrix (\frac{r_{ab}^{ν}}{B_{b}^{ν}})^{−1} on T_{0}(∂D_{ν}),

where D_{ν} = (Ψ_{ν} ∘ Φ_{ν} ∘ φ_{ν})(Ω), so that

R_{ν}((Ψ_{ν} ∘ Φ_{ν} ∘ φ_{ν})(e_{ℓ}^{∘}) ∈ u_{j}-axis

for some j which depends on ℓ.

In this w−coordinate,

ρ_{ν}(w) = w_{1}

+A_{2}^{ν}(\frac{β_{2}^{ν}}{β_{1}^{ν}}w_{1} + \frac{r_{22}^{ν}}{B_{2}^{ν}} w_{2}) ^{2}

+δ_{2} (\frac{β_{2}^{ν}}{β_{1}^{ν}}w_{1} + \frac{r_{22}^{ν}}{B_{2}^{ν}} w_{2}) ^{2}

+o (|(\frac{β_{2}^{ν}}{β_{1}^{ν}}w_{1} + \frac{r_{22}^{ν}}{B_{2}^{ν}} w_{2})|^{2}) < 0

(IV) Define an affine coordinate change Γ_{ν} as follows.

Γ_{ν} = \{ \begin{matrix}x̃_{1} = w_{1} \\ x̃_{2} = \frac{√ϵ_{ν}}{B_{2}^{ν}}w_{2} \end{matrix}

Now, all axes of the simplex (Γ_{ν} ∘ R_{ν} ∘ Ψ_{ν} ∘ Φ_{ν} ∘φ_{ν})(E_{0}) which are tangential to (Γ_{ν} ∘R_{ν} ∘ Ψ_{ν} ∘ Φ_{ν} ∘ φ_{ν})(Ω) at 0 are of length √ϵ_{ν}.

In this x̃− coordinate,

ρ_{ν}(x̃) = x̃_{1}

+A_{2}^{ν} (\frac{β_{2}^{ν}}{β_{1}^{ν}} x̃_{1} + \frac{r_{22}^{ν}}{√ϵ_{ν}} x̃_{2}) ^{2}

+δ_{2} (\frac{β_{2}^{ν}}{β_{1}^{ν}} x̃_{1} + \frac{r_{22}^{ν}}{√ϵ_{ν}} x̃_{2}) ^{2}

+o (|(\frac{β_{2}^{ν}}{β_{1}^{ν}} x̃_{1} + r_{22}^{ν} \frac{x̃_{2}}{√ϵ_{ν}})|^{2} ) < 0

Let D_{ν} be the intersection of a small neighborhood U of p = 0 and the domain defined by ρ_{ν}(x̃) < 0.

L_{ν} = \{ \begin{matrix}x̂_{1} = x̃_{1}/ϵ_{ν} \\ x̂_{2} = x̃_{2}/√ϵ_{ν} \end{matrix}

Recall that ϵ_{ν} = β_{1}^{ν}. This sequence preserves the tangent space and acts as a homothety on the tangent space at 0 and each transformation sends the last simlex to the standard unit simplex at 0.

Then the defining function ρ_{ν}(x̂) of the domain L_{ν}(D_{ν}) can be written as

ρ_{ν}(x̂) = ϵ_{ν}x̂_{1}

+A_{2}^{ν} (β_{2}^{ν}x̂_{1} + r_{22}^{ν} x̂_{2}) ^{2}
+δ_{2} (β_{2}^{ν}x̂_{1} + r_{22}^{ν} x̂_{2}) ^{2}

+o (|(β_{2}^{ν}x̂_{1} + r_{22}^{ν} x̂_{2})|^{2}) < 0

By our construction, we see that all √ϵ_{ν}, β_{ℓ}^{ν}’s and r_{ab}^{ν}’s approach 0 as ν → ∞. Compare the decreasing rates of these values and divide the above equation by {|α_{ν}|^{2}}, where {α_{ν}} ∈ {{√ϵ_{ν}}, {β_{2}^{ν}}, {r_{2}^{ν}}} has the slowest decreasing rate by extracting a subsequence if necessary.

ρ_{ν}(x̂) = \frac{ϵ_{ν}}{|α_{ν}|^{2}} x̂_{1}

+A_{2}^{ν} (\frac{β_{σ}^{ν}}{|α_{ν}|}x̂_{1} + \frac{r_{2}^{ν}}{|α_{ν}|}x̂_{k}) ^{2}

+δ_{ℓ} (\frac{β_{2}^{ν}}{|α_{ν}|}x̂_{1} + \frac{r_{22}^{ν}}{|α_{ν}|}x̂_{2}) ^{2}

+\frac{1}{|α_{ν}|^{2}}o (|(β_{2}^{ν}x̂_{1} + r_{22}^{ν} x̂_{2})|^{2}) < 0

Then all limits satisfy

lim_{ν→∞} |\frac{√ϵ_{ν}}{α_{ν}} | ≤ 1 lim_{ν→∞} |\frac{β_{ℓ}^{ν}}{α_{ν}} | ≤ 1 lim_{ν→∞} |\frac{r_{ab}^{ν}}{α_{ν}}| ≤ 1

for ℓ, a, b = 2. Also, observe that

\begin{matrix}c_{ℓ}^{ν} → 0 \\ A_{22}^{ν} → 0 \end{matrix}

\frac{1}{|α_{ν}|^{2}}o (|(β_{2}^{ν}x̂_{1} + r_{22}^{ν} x̂_{2})|^{2}) → 0

as ν → ∞. Hence the limiting defining function becomes a quadratic since δ_{2} ≠ 0 and by a linear change of coordinate if necessary, it must be

(∗)  ρ̂(x) = x_{1} + δ_{2}x_{2}^{2} < 0  with δ_{2} ≠ 0

The domain L_{ν}(D_{ν}) converges to the domain Ω̂ defined by (∗). Note that Ω̂ = Q(a_{2}).

Combining all the coordinate changes above, we have a sequence of projective transformation

G_{ν} := L_{ν} ∘ Γ_{ν} ∘ R_{ν} ∘ Ψ_{ν} ∘ Φ_{ν} ∘ φ_{ν}

A subsequential limit yields a mapping

G : Ω ⟶ Ω̂ ∈ P 𝔤𝔩(3, ℝ)

Here is the crucial step but it is a very short proof.

Lemma G : Ω ⟶ Ω̂ is a projective transformation i.e. Ω̂ is projectively equivalent to Ω.

PROOF By our construction of Φ_{ν}, Ψ_{ν}, R_{ν}, Γ_{ν}, and L_{ν}, G_{ν}(E_{0}) is the standard unit simplex at 0 for all ν. So, E_{0} ∩ K(G) = ∅ if G were singular. Otherwise, there must exist an affine full line contained in lim_{ν→∞} G_{ν}(E_{0}) =the closed standard unit simplex at 0. Since G_{ν} |_{E_{0}}→ G |_{E_{0}} uniformly, G cannot be singular.