Transformation Groups, Vol. 2, No. 1, 1997, pp. 1–33  ◯^{c} Birkhäuser Boston (1997)

THE FINITE IRREDUCIBLE LINEAR GROUPS WITH POLYNOMIAL RING OF INVARIANTS

GREGOR KEMPER

IWR, Im Neuenheimer Feld 368, D–69120 Heidelberg, Germany

Gregor.Kemper@iwr.uniheidelberg.de

GUNTER MALLE ^{∗}

IWR, Im Neuenheimer Feld 368, D–69120 Heidelberg, Germany

malle@urania.iwr.uniheidelberg.de

Abstract. We prove the following result: Let G be a finite irreducible linear group. Then the ring of invariants of G is a polynomial ring if and only if G is generated by pseudoreflections and the pointwise stabilizer in G of any nontrivial subspace has a polynomial ring of invariants. This is well-known in characteristic zero. For the case of positive characteristic we use the classification of finite irreducible groups generated by pseudoreflections due to Kantor, Wagner, Zalesskiǐ and Serežkin. This allows us to obtain a complete list of all irreducible linear groups with a polynomial ring of invariants.

Introduction

Let V be a finite-dimensional vector space and S(V) its symmetric algebra. In this paper we determine the finite irreducible linear groups G ≤ GL(V) for which the ring of invariants S(V)^{G} is polynomial (see Table 7.3). It turns out that our result can be stated in the form of a simple criterion consisting of global and several local conditions which might be extended to the reducible case:

Main Theorem. Let V be a finite-dimensional vector space, G ≤ GL(V) a finite irreducible linear group. Then S(V)^{G} is a polynomial ring if and only if G is generated by pseudoreflections and the pointwise stabilizer in G of any nontrivial subspace of V has a polynomial ring of invariants.

Here an element σ ∈ GL(V) is called a pseudoreflection if ker(1 − σ) is a hyperplane in V.

It is easily seen that the criterion of the Main Theorem is equivalent to the following: G is generated by pseudoreflections and the stabilizer of any

Typeset by AMS-TEX

∗The second author gratefully acknowledges financial support by the Deutsche Forschungsgemeinschaft

Received September 9, 1996. Accepted November 26, 1996.

nontrivial point in V̄ := V ⊗_{k} K has a polynomial ring of invariants, where K denotes an algebraic closure of the ground field k.

It was sometimes conjectured that it might be sufficient that the pointwise stabilizer of any subspace is generated by pseudoreflections. This is not true; a counterexample is given by Sp_{4}(2) ≅ 𝔖_{6} which does not have polynomial ring of invariants (see Example 2.2).

The question of determining linear groups with a polynomial ring of invariants has a long history. In characteristic zero, the previous result is wellknown and remains true even if the irreducibility assumption is dropped. Indeed, it was shown by Shephard–Todd [18] and Chevalley [5] that a finite linear group in characteristic zero has a polynomial ring of invariants if and only if it is generated by pseudoreflections. Then Steinberg showed [20] that for a pseudoreflection group in characteristic zero the pointwise stabilizer of any subspace is again generated by pseudoreflections. Together these results yield the Main Theorem in characteristic 0.

The result of Steinberg was subsequently generalized by Serre [17] to arbitrary characteristic in the following form: if G has polynomial invariants, then the pointwise stabilizer of any subspace also has polynomial invariants (see Theorem 2.1). This allows us to conclude that the Main Theorem is true provided that the order of G is prime to the characteristic of k. The results in the case of positive characteristic dividing the group order were rather less complete. In [17, p. 8–03] it is stated that the ring of invariants of the 4-dimensional orthogonal group of plus-type over a finite field of odd order is not polynomial. This covers in particular the case of the group GO_{4}^{+}(3) which is the 3-modular reduction of the Weyl group of type F_{4} that was later reconsidered by Toda (see [19, p. 218]). On the other hand Demazure [8] shows that Weyl groups have polynomial rings of invariants modulo all good primes. The paper of Nakajima [15] starts the systematic investigation of irreducible groups generated by pseudoreflections. But he leaves out the case of characteristic 2, small ranks and the exceptional groups, where many examples arise (see Table 7.3). Also, in [15, Thm. 5.4] he claims that the 4-dimensional orthogonal group of minus-type does not have polynomial invariants, but his proof is not correct (in fact, we show that this group has polynomial ring of invariants, see Proposition 4.1). More recently, Xu [24] studied some of the exceptional groups.

As in the work of Nakajima, our proof uses the classification of irreducible pseudoreflection groups in positive characteristic. Moreover, for some of the large exceptional examples, the computer algebra packages INVAR [14] for the computation of rings of invariants written by the first author, SINGULAR [10], and GAP [16] were used.

In the final section we give some additional information for dimension 2 and a discussion of the general reducible case and some indecomposable cases.

It would be very interesting to find an a priori proof of the Main Theorem, which then might also apply to the reducible case.

1. The irreducible pseudoreflection groups

In this section we recall the classification by Kantor, Wagner, Zalesskiǐ and Serežkin of irreducible linear groups generated by pseudoreflections. Let k be a field of characteristic p, V a finite dimensional vector space over k. An element σ ∈ GL(V) is called a pseudoreflection if ker(1 − σ) is a hyperplane in V. A pseudoreflection which is not diagonalizable is called a transvection. A diagonalizable pseudoreflection is called a homology; if its nontrivial eigenvalue is −1 it is also called a reflection. The pointwise stabilizer of a subspace W ≤ V is sometimes called a fixator of W.

We consider finite irreducible linear groups G ≤ GL(V) generated by pseudoreflections.

Imprimitive groups A linear group G on V = k^{n} is called a monomial group if there exists a basis of V with respect to which the matrices of elements of G have exactly one nonzero entry in each row and column. The full monomial group on k^{n} with entries in ⟨ζ_{m}⟩, where ζ_{m} ∈ k^{×} is a primitive m-th root of unity, is denoted by G(m, 1, n), and the subgroup generated by all permutation matrices together with the diagonal matrices with entries in ⟨ζ_{m}⟩ and determinant 1 is denoted by G(m, m, n).

If G is an imprimitive irreducible group generated by pseudoreflections, then G is monomial (see for example [25, 1.9]). Thus imprimitive groups are reductions mod p of imprimitive characteristic zero groups (see loc. cit., (1.8)). For later use we state the following result (see also [22, Lemma 2.2]):

Lemma 1.1. Let G = G(m, m, n) ≤ GL_{n}(k) with n ≥ 2. Then G has a unique system of imprimitivity unless (m, n) = (4, 2), (m, n) = (2, 2), (m, n) = (3, 3) or (m, n) = (2, 4).

Proof. Since G is imprimitive, it stabilizes a decomposition

(1.2)  V = ⨁_{i=1}^{n} k e_{i}

of V = k^{n}. Let V = ⊕_{i=1}^{n}k v_{i} be a second decomposition of V into 1dimensional subspaces fixed by G. Thus G acts on the set U := ∪_{i=1}^{n}kv_{i}. We may assume without loss of generality that the first coordinate of each v_{i} is either 0 or 1, and that the first coordinate of v_{1} equals 1. Then let v_{1} = (1, α_{2}, …, α_{n}) := ∑_{j=1}^{n} α_{j}e_{j}, α_{1} = 1, be the representation of v_{1} in the original basis. Note that via the action of 𝔖_{n} we may (and will) moreover assume that α_{2} ≠ 0.

If n ≥ 3 then under the action of G we obtain that U contains

{(1, ζ^{l}α_{2}, ζ^{−l}α_{3}, α_{4}, …, α_{n}) | 0 ≤ l < m}.

Thus m ≤ n. On the other hand, if n ≥ 5 then via the action of 𝔖_{n−1} we find that α_{2} = … = α_{n} and, by the action of diagonal matrices, that no possibility arises. It remains to deal with the cases where m ≤ n ≤ 4. An easy calculation shows the result.

For n = 2 the argument is similar. □

Since by [18] for any irreducible imprimitive pseudoreflection group G there exists an m such that G(m, m, n)◁ G◁ G(m, 1, n), the previous lemma applies to any such group.

Reflection groups

We now state the classification of irreducible primitive pseudoreflection groups. We first consider the case of groups generated by reflections, i.e., diagonalizable pseudoreflections of order 2, so the characteristic of the ground field is odd. We will moreover assume that dim(V) ≥ 3 since the answer for the 2-dimensional case is well-known (see also Section 7). In characteristic zero these groups were first determined by Shephard and Todd. In positive characteristic, the irreducible subgroups of GL_{n}(k), n ≥ 3, generated by reflections were classified by Wagner [23], Zalesskiǐ and Serežkin [26]:

Theorem 1.3 (Wagner, Zalesskiǐ and Serežkin). Let V be a vector space of dimension n ≥ 3 over a finite field of characteristic p ≠ 2 and G ≤ GL(V) an irreducible primitive linear group generated by reflections. Then one of the following holds:

(a) G = RL_{n}(q) or G = RU_{n}(q) where p|q,

(b) Ω_{n}^{(±)}(q) < G ≤ GO_{n}^{(±)}(q), G ≠ SO_{n}^{(±)}(q), where p|q,

(c) G = 𝔖_{n+1}, p |̸(n + 1),

(d) G = 𝔖_{n+2}, p|(n + 2), n ≥ 5,

(e) G = W_{p}(G_{i}), 23 ≤ i ≤ 37, i ≠ 25, 26, 32, and either p ∈ {3, 5, 7} corresponds to one of the columns marked p or np in Table 6.3, or p |̸|G|,

(f) (n, p) = (3, 5), G = 3 · A_{7} × 2 (denoted EJ_{3}(5)),

(g) (n, p) = (4, 3), G = 4 · L_{3}(4) : 2_{2} (denoted J_{4}(4)).

Here we have written RL_{n}(q) (respectively RU_{n}(q)) for the subgroup of GL_{n}(q) (resp. GU_{n}(q)) generated by the reflections contained in it. This subgroup contains SL_{n}(q) (resp. SU_{n}(q)) as a subgroup of index 2. Furthermore, Ω_{n}^{(±)}(q) denotes the commutator subgroup of the orthogonal group. Finally, G_{i} is the i-th complex reflection group in the classification of Shephard and Todd, and W_{p}(G_{i}) denotes a p-modular reduction of its reflection representation. Note that, since we are only considering irreducible groups, this reduction is well-defined. Explicit generators for the groups under (f) and (g) will be given in the proof of Proposition 6.1.

Groups generated by homologies of order > 2

The irreducible linear groups generated by homologies of order > 2 were determined by Mitchell in characteristic 0 and Wagner [22, Thm. 1.2] in positive characteristic:

Theorem 1.4 (Wagner). Let V be a vector space of dimension n ≥ 3 over a finite field of characteristic p and G ≤ GL(V) an irreducible primitive linear group generated by pseudoreflections and containing homologies of order > 2. Then one of the following holds:

(a) SL_{n}(q) < G ≤ GL_{n}(q) or SU_{n}(q) < G ≤ GU_{n}(q), G ≠ RL_{n}(q), RU_{n}(q), where p|q,

(b) G = W_{p}(G_{25}) and p ≠ 3,

(c) G = W_{p}(G_{26}) and p ≠ 2, 3,

(d) G = W_{p}(G_{32}) and p ≠ 3.

Here, as above, W_{p}(G_{i}) denotes the p-modular reductions of the primitive complex pseudoreflection group with numbers i in the list of Shephard and Todd.

Transvection groups Finally, we consider the case where G is generated by transvections. This does not have an analog for finite groups in characteristic 0. These groups have been classified by several authors. A complete list can be found in [12, Thm. II] (see also [21] and [25]):

Theorem 1.5 (Kantor). Let V be an n-dimensional vector space over a finite field of characteristic p and G ≤ GL(V) an irreducible primitive linear group generated by transvections. Then one of the following holds:

(a) G = SL_{n}(q), Sp_{n}(q), SU_{n}(q) with (n, q) ≠ (3, 2), where p|q,

(b) n ≥ 4 is even, p = 2, G = SO_{n}^{±}(q) with q even,

(c) n ≥ 6 is even, p = 2, G = 𝔖_{n+1} or G = 𝔖_{n+2},

(d) (n, p) = (2, 3), G = SL_{2}(5),

(e) (n, p) = (3, 2), G = 3^{.}A_{6},

(f) (n, p) = (6, 2), G = 3^{.}U_{4}(3).2_{2}.

Explicit generators for the groups in (d)–(f) of the theorem will be given in the proof of Proposition 6.6.

Irreducible primitive pseudoreflection groups Let now G ≤ GL(V) be an arbitrary finite irreducible primitive linear group generated by pseudoreflections, where dim(V) ≥ 3. If C is a conjugacy class of pseudoreflections of G, then the subgroup H = ⟨C⟩ generated by C is normal in G and acts irreducibly ([25, 1.10]). Thus G lies in the normalizer in GL(V) of H and we need to collect information about the normalizers of irreducible pseudoreflection groups.

If H is imprimitive with decomposition (1.2) of V, then the normalizer N of H in GL(V) sends this decomposition into another decomposition stabilized by H. By Lemma 1.1 this implies that N fixes the original decomposition, hence acts imprimitively itself, unless (m, n) = (3, 3) or (m, n) = (2, 4). But in the latter two cases the normalizer is easily determined; it is equal to W_{p}(G_{26})Z(GL(V)), respectively W_{p}(F_{4})Z(GL(V)). The pseudoreflection subgroups G of these are known and already occur in Theorems 1.3–1.5.

Thus we may assume that the normal subgroup H is itself primitive. Moreover if G is not one of the groups in Theorems 1.3–1.5, then by Theorem 1.4 no homologies of order > 2 lie in G. Thus G contains transvections as well as reflections. The normalizers of primitive reflection groups H are enumerated in [23, Table II]. Either all intermediate groups H ≤ G ≤ N (H) are contained in the list, or N (H) is an orthogonal group in odd characteristic. But by Theorem 1.5 the latter do not contain transvections. We have thus proved:

Theorem 1.6. Let V be an n-dimensional vector space over a finite field of characteristic p, n ≥ 3, and G ≤ GL(V) an irreducible primitive subgroup generated by pseudoreflections. Then G occurs in Theorems 1.3–1.5.

2. Preliminary reductions

Having recalled the classification of the irreducible linear pseudoreflection groups G ≤ GL(V) we now study the rings of invariants S(V)^{G}.

We first remark that we can perform all calculations over a minimal field of definition for a linear group G since the question whether the ring of invariants is polynomial is unchanged under extensions of the ground field. This is seen from the fact that any minimal system of generators of A := S(V)^{G} as an algebra over k has cardinality dim_{k}(A^{+}/(A^{+})^{2}), where A^{+} is the ideal in A generated by the invariants of positive degree. But for a field K ≥ k we have

(K ⊗_{k} A)^{+} /((K ⊗_{k} A)^{+})^{2} ≅ K ⊗_{k} (A^{+} /(A^{+})^{2} ) .

Also, if G acts irreducibly, its representation can be realized over a finite extension of the prime field. Thus for groups in positive characteristic we may restrict our attention to irreducible pseudoreflection subgroups of GL_{n}(q) for prime powers q.

The following important result, which gives one direction of the assertion of the Main Theorem, will be a powerful tool in ruling out candidates for groups with polynomial ring of invariants.

Theorem 2.1 (Serre). Let G ≤ GL(V) be a finite group such that S(V)^{G} is polynomial. Then the point-wise stabilizer of any subspace W ≤ V has polynomial ring of invariants (and thus is generated by pseudoreflections).

A proof is indicated in [2, V, §6, Ex. 8]; see also [15, Lemma 1.4].

The following example shows that it does not suffice to require that every fixator is generated by pseudoreflections in order to obtain a polynomial ring of invariants and thus refutes a conjecture of Kac [11, Remark 3]:

Example 2.2. Let G = Sp_{4}(2), the four dimensional symplectic group over the field with two elements. This group is isomorphic to the symmetric group 𝔖_{6} on 6 letters in its 4-dimensional irreducible 2-modular representation obtained as the nontrivial constituent of the permutation module. We claim that every fixator of a subspace in V = 𝔽_{2}^{4} ⊗_{𝔽_{2}} k, with k an algebraic closure of 𝔽_{2}, is generated by pseudoreflections. For this it suffices to show the following. If 1 ≠ σ ∈ G then the fixator of its fixed space V^{σ} contains pseudoreflections whose product is equal to σ. Clearly it suffices to check this for representatives of the 11 conjugacy classes of G, which is a routine calculation. (This computation was carried out in the computer algebra system GAP ([16]). On the other hand we prove in Proposition 3.4 that G does not have a polynomial ring of invariants.

A similar example, but for a reflection group, is given by the group RU_{3}(3). Again, all fixators are generated by pseudoreflections, but by Proposition 3.1 its ring of invariants is not polynomial. On the other hand, no counterexamples arise if G does not contain transvections (see Theorem 7.4). □

We also remark that irreducible p-modular reflection groups in general do not even necessarily have rings of invariants which are complete intersections:

Example 2.3. Let G = 𝔖_{7} be the symmetric group in its 5-dimensional irreducible 7-modular representation obtained as the nontrivial constituent of the permutation module. Then there exists a point whose stabilizer is of order 7, generated by an element with a Jordan block of size 5 (see Proposition 5.1). But by [11, Thm. C] for a group whose ring of invariants is a complete intersection all point-stabilizers are generated by elements σ with rk(σ − 1) ≤ 2. □

In order to verify that a given group has polynomial invariants, we will make use of the following simple criterion of the first author [14, Prop. 16]:

Proposition 2.4. Let V be a vector space of dimension n and G ≤ GL(V) a finite group. Then S(V)^{G} is polynomial if and only if there exist homogeneous invariants f_{1}, …, f_{n} ∈ S(V)^{G} of degrees d_{1}, …, d_{n} such that ∏_{i=1}^{n} d_{i} = |G| and the Jacobian determinant J = det((∂f_{i}/∂x_{j})_{i,j}) is nonzero.

If G acts imprimitively, then by a result of Nakajima [15, Thm. 2.4] the ring of invariants S(V)^{G} of G is a polynomial ring. Thus, to prove the Main Theorem we may and will assume from now on that G is irreducible and primitive.

3. Some classical groups

We first study some families of classical groups over finite fields.

The general linear groups If G is equal to the full general linear group GL_{n}(q) of the vector space V = 𝔽_{q}^{n}, then it was shown by Dickson [9] that the ring of invariants S(V)^{GL_{n}(q)} is polynomial, generated by the so-called Dickson-invariants d_{n,i}, 0 ≤ i ≤ n−1, where the degree of d_{n,i} is equal to q^{n} − q^{i}. The quotient GL_{n}(q)/SL_{n}(q) is cyclic of order q − 1, and for each intermediate group SL_{n}(q) ≤ G ≤ GL_{n}(q) the invariant ring S(V)^{G} of G is again polynomial, generated by

{((−1)^{n}d_{n,0})^{1/m} , d_{n,1}, …, d_{n,n−1}},

where m = (GL_{n}(q) : G) is the index of G in GL_{n}(q) (see for example [1, Thms. 8.1.1 and 8.2.1]). This covers in particular the case of 1-dimensional linear groups.

The unitary groups We next consider the general unitary group GU_{n}(q) ≤ GL_{n}(q^{2}). For odd q and G = SU_{n}(q) the following is also proved in [15, Thm. 5.2]:

Proposition 3.1. Let SU_{n}(q) ≤ G ≤ GU_{n}(q) with n ≥ 4 or SU_{3}(q) ≤ G < GU_{3}(q). Then S(V)^{G} is not a polynomial ring.

Proof. First let G = GU_{4}(q), the unitary group leaving invariant the Hermitian form

⟨∑_{i=1}^{4} α_{i}x_{i}, ∑_{i=1}^{4} β_{i}x_{i}⟩ = ∑_{i=1}^{4} α_{i}^{q}β_{5−i}

on 𝔽_{q^{2}}^{4} × 𝔽_{q^{2}}^{4}. The fixator N of the maximal isotropic subspace W spanned by the first two standard basis vectors has order q^{4} and is given by

\{[ \begin{matrix}E_{2} & A \\ 0 & E_{2} \end{matrix} ] | A = (a_{ij}) ∈ 𝔽_{q^{2}}^{2×2}, a_{12}^{q} + a_{12} = a_{21}^{q} + a_{21} = 0, a_{11} = −a_{22}^{q}\} .

This group has the two invariants f_{1} := x_{1} and f_{2} := x_{2}. If the invariant ring is polynomial there exist two further invariants f_{3}, f_{4} with degree product q^{4} and hence deg(f_{4}) ≥ q^{2}, say. Since N ≤ GU_{4}(q) there also exists the unitary invariant

f_{u} := x_{1}^{q}x_{4} + x_{2}^{q}x_{3} + x_{2}x_{3}^{q} + x_{1}x_{4}^{q}

of degree q+1. Since q+1 does not divide |N|, f_{u} would have to be expressed as f_{u} = g_{1}f_{3} +g_{2} with g_{1}, g_{2} ∈ 𝔽_{q}[x_{1}, x_{2}] and f_{3} of degree at most q. Visibly this is not possible, so N does not have polynomial invariants. Since N is already contained in SU_{4}(q), the fixator of W in any intermediate group SU_{4}(q) ≤ G ≤ GU_{4}(q) is also equal to N. Hence by Theorem 2.1 the ring of invariants S(V)^{G} for any such G is not polynomial.

Now in GU_{n}(q) the stabilizer of an anisotropic vector is conjugate in GL_{n}(q^{2}) to GU_{n−1}(q), and we may apply induction together with the case n = 4 and Theorem 2.1.

In dimension 3 we consider the Hermitian form

(3.2) ⟨α_{1}x_{1} + α_{2}x_{2} + α_{3}x_{3}, β_{1}x_{1} + β_{2}x_{2} + β_{3}x_{3}⟩ = α_{1}^{q}β_{3} + α_{2}^{q}β_{2} + α_{3}^{q}β_{1}

on 𝔽_{q^{2}}^{3} × 𝔽_{q^{2}}^{3}. The fixator in SU_{3}(q) ≤ G ≤ GU_{3}(q) of the isotropic first basis vector is given by

H := \{[ \begin{matrix}1 & a & b \\ 0 & c & −a^{q}c \\ 0 & 0 & 1 \end{matrix} ] | a, b, c ∈ 𝔽_{q^{2}}, c^{m} = 1, a^{q+1} + b + b^{q} = 0\} ,

where m = (G : SU_{3}(q)). This group has the invariants x_{1} and x_{1}x_{3}^{q} + x_{1}^{q}x_{3} + x_{2}^{q+1}. Assume that H has a further invariant v of degree less than q + 1 and not a power of x_{1}. Then v is also an invariant of

H_{1} := \{[ \begin{matrix}1 & a & b \\ 0 & 1 & −a^{q} \\ 0 & 0 & 1 \end{matrix} ] | a, b ∈ 𝔽_{q^{2}}, a^{q+1} + b + b^{q} = 0\} .

This group has the invariant w := ∏_{a∈𝔽_{q^{2}}} (ax_{1} + x_{2}) of degree q^{2}. If v does not involve x_{3}, then it is invariant under

\{[ \begin{matrix}1 & a & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} ] | a ∈ 𝔽_{q^{2}}\}

which has a polynomial ring of invariants 𝔽_{q^{2}}[x_{1}, w, x_{3}]. But then the degree of v would be at least q^{2}, which is not the case. Thus v involves x_{3}, hence {x_{1}, v, w} is an algebraically independent system of invariants for H_{1}, with degree product not larger than the group order |H_{1}| = q^{3}, and hence by Proposition 2.4, the invariants of H_{1} would be polynomial. This is a contradiction since H_{1} is not generated by pseudoreflections. So any fundamental system of invariants of H would have to include x_{1}x_{3}^{q} + x_{1}^{q}x_{3} + x_{2}^{q+1}. But |H| is divisible by q + 1 if and only if G = GU_{3}(q). In particular, S(V)^{H} and S(V)^{G} are not polynomial if G ≠ GU_{3}(q). □

Proposition 3.3. The ring of invariants of GU_{3}(q) is polynomial.

Proof. Let G = GU_{3}(q) acting on V = k^{3}, where k = 𝔽_{q^{2}}. We can assume that the Hermitian form on V is given by (3.2). Using the bijection V → V^{∗}, v ↦ v^{∗}, given by v^{∗}(w) = ⟨v, w⟩, we define a Hermitian form on V^{∗} by setting ⟨v^{∗}, w^{∗}⟩ = ⟨w, v⟩. Then G acts on V^{∗} as GU(V^{∗}), and an easy calculation yields ⟨v^{∗}, v^{∗}⟩ = (x_{1}^{q}x_{3} + x_{2}^{q+1} + x_{1}x_{3}^{q})(v^{∗}) for v^{∗} ∈ V^{∗}, where the basis elements of V are regarded as elements of the bidual V^{∗∗}. We obtain the invariants

\begin{aligned}f_{1} & := x_{1}^{q}x_{3} + x_{2}^{q+1} + x_{1}x_{3}^{q} and \\ f_{2} & := P^{q}(f_{1}) = x_{1}^{q^{3}}x_{3} + x_{2}^{q^{3}+1} + x_{1}x_{3}^{q^{3}}, \end{aligned}

where P^{i} denotes the i-th Steenrod operator (see [19, p. 324]). Let V_{0} = {v ∈ V | v ≠ 0 and ⟨v, v⟩ = 0} be the set of isotropic nonzero vectors, of cardinality |V_{0}| = (q^{2} − 1)(q^{3} + 1). Clearly, with every vector all its nonzero multiples are also isotropic, so the set T = {v^{q^{2}−1} | v ∈ V_{0}} has |T | = q^{3} + 1 elements. Hence we can construct another invariant f_{3} of degree q^{3}(q^{2} − 1) by taking

f_{3} := ∑_{h∈T} ∏_{{h′∈T \atop h′≠h}} h′.

Since the product of the degrees of f_{1}, f_{2}, f_{3} equals the group order, we are done if we can show that the Jacobian determinant J = det((∂f_{i}/∂x_{j})_{i,j}) is nonzero (see Proposition 2.4). Indeed, it is sufficient to prove that its reduction mod x_{1}^{q} does not vanish. We obtain

J = | \begin{matrix}x_{3}^{q} & x_{2}^{q} & x_{1}^{q} \\ x_{3}^{q^{3}} & x_{2}^{q^{3}} & x_{1}^{q^{3}} \\ ∂f_{3}/∂x_{1} & ∂f_{3}/∂x_{2} & ∂f_{3}/∂x_{3} \end{matrix} | ≡ | \begin{matrix}x_{3}^{q} & x_{2}^{q} \\ x_{3}^{q^{3}} & x_{2}^{q^{3}} \end{matrix} | ∂f_{3}/∂x_{3} mod x_{1}^{q}.

Since x_{3}^{q}x_{2}^{q^{3}} − x_{2}^{q}x_{3}^{q^{3}} is not a zero divisor in S(V)/(x_{1}^{q}), it is enough to show that ∂f_{3}/∂x_{3} ≢ 0 mod x_{1}^{q}. But x_{1}^{q^{2}−1} is an element of T, hence only the summand of f_{3} with h = x_{1}^{q^{2}−1} contributes mod x_{1}^{q}. We can also exclude all h′ ∈ T which do not involve x_{3} from the product. Setting

U = {α_{1}x_{1} + α_{2}x_{2} + x_{3} | α_{1}^{q} + α_{2}^{q+1} + α_{1} = 0} and g = ∏_{v∈U} v,

we must show that ∂f_{3}/∂x_{3} ≡ ∂/∂x_{3}(g^{q^{2}−1}) = −g^{q^{2}−2} · ∂g/∂x_{3} ≢ 0 mod x_{1}^{q}, or equivalently ∂g/∂x_{3} ≢ 0 mod x_{1}^{q}. We consider the x_{1}^{q−1}-coeffici-ent of ∂g/∂x_{3}:

coeff(∂g/∂x_{3}, x_{1}^{q−1}) = ∑_{v∈U} coeff( ∏_{{v′∈U \atop v′≠v}} v′, x_{1}^{q−1}) =

= ∑_{v∈U} ∑_{{U′⊂U\\{v\} \atop |U′|=q−1}} ∏_{v′∈U′} α_{1}(v′) · ∏_{{v′′∈U\U′ \atop v′′≠v}} (v′′ |_{x_{1}=0}),

where α_{1}(v′) denotes the x_{1}-coefficient of v′. We now work modulo the ideal (x_{1}, x_{3}). For v = α_{1}x_{1} + α_{2}x_{2} + x_{3} ∈ U, we have v ≡ 0 mod (x_{1}, x_{3}) if and only if α_{2} = 0, so v = α_{1}x_{1} + x_{3} with α_{1}^{q} + α_{1} = 0. There exist exactly q such α_{1} ∈ k: For q even, these are the elements of 𝔽_{q} ⊂ k, and for q odd they are α_{1} = 0 and α_{1} = w^{l} with w an element of order q^{2} − 1 in k^{×} and l ≡ (q + 1)/2 mod (q + 1). So we see that the product ∏_{v′∈U′} α_{1}(v′) · ∏_{v′′∈U\U′,v′′≠v} v′′ is ≢ 0 mod (x_{1}, x_{3}) only for v = x_{3} and U′ = {α_{1}x_{1} + x_{3} | α_{1}^{q−1} + 1 = 0}. Hence the above sum has only one contribution mod (x_{1}, x_{3}), and this contribution is nonzero.

This completes the proof that f_{1}, f_{2} and f_{3} generate the invariant ring S(V)^{G} as a polynomial ring. □

The symplectic groups We finally consider the symplectic group G = Sp_{n}(q) in its natural representation on V = 𝔽_{q}^{n} (for odd q see also [15, Thm. 5.2]).

Proposition 3.4. Let G = Sp_{n}(q) with n ≥ 4 even. Then S(V)^{G} is not a polynomial ring.

Proof. Let first n = 4 and G = Sp_{4}(q) the symplectic group with respect to the skew-symmetric form

⟨∑_{i=1}^{4} α_{i}x_{i}, ∑_{i=1}^{4} β_{i}x_{i}⟩ = α_{1}β_{4} + α_{2}β_{3} − α_{3}β_{2} − α_{4}β_{1}.

The fixator of the maximal isotropic subspace spanned by the first two basis vectors is the group of matrices

H := \{[ \begin{matrix}E_{2} & A \\ 0 & E_{2} \end{matrix} ] | A = (a_{ij}) ∈ 𝔽_{q}^{2×2}, a_{11} = a_{22}\} .

By interchanging the last two basis vectors we obtain the group of matrices such that the 2 × 2-submatrix in the upper right corner is symmetric. But it was shown in [14, Ex. 19] that this group does not have polynomial invariants, hence by Theorem 2.1 the same assertion follows for S(V)^{G}.

Now if G = Sp_{n}(q), n ≥ 6, then the fixator of a hyperbolic plane is conjugate to Sp_{n−2}(q), and we may use induction. □

See also [4] or [1, 8.3] for an explicit description of S(V)^{G}.

4. Orthogonal groups

The structure of orthogonal groups in even characteristic is slightly different from the one in odd characteristic, so we treat these cases separately.

Orthogonal groups in odd characteristic Let q be the power of an odd prime. If n is odd, the orthogonal group GO_{n}(q) is a direct product GO_{n}(q) = Z(GO_{n}(q)) × SO_{n}(q), where the center Z(GO_{n}(q)) has order 2 and SO_{n}(q) is an extension of degree 2 of the simple group O_{n}(q). Apart from SO_{n}(q) there exist two further intermediate groups between O_{n}(q) and GO_{n}(q), both of which are generated by reflections. One of them is the direct product Z(GO_{n}(q)) × O_{n}(q), the other is isomorphic (but not conjugate) to SO_{n}(q). We write R^{+}O_{n}(q) (resp. R^{−}O_{n}(q)) for the group whose reflections have centralizer of orthogonal plus-type (resp. minus-type). It depends on n and q whether or not R^{+}O_{n}(q) has nontrivial center. Finally, GO_{n}(q) = ⟨SO_{n}(q), R^{+}O_{n}(q)⟩ is also generated by reflections.

Now assume that n = 2m ≥ 4 is even. Then the orthogonal group GO_{n}^{ϵ}(q) contains its derived group Ω_{n}^{ϵ}(q) as a normal subgroup of index 4, with the factor group isomorphic to 2^{2}. Apart from SO_{n}^{ϵ}(q) there exist two further groups between Ω_{n}^{ϵ}(q) and GO_{n}^{ϵ}(q), and both are generated by reflections. There will be no need to distinguish between them, so we write RO_{n}^{ϵ}(q) for any such group. Furthermore, GO_{n}^{ϵ}(q) is also generated by reflections.

Proposition 4.1. In the situation of Theorem 1.3 let Ω_{n}^{(±)}(q) < G ≤ GO_{n}^{(±)}(q) with q odd and n ≥ 3. Then S(V)^{G} is polynomial if and only if

(a) G = GO_{3}(q), or

(b) G = R^{+}O_{3}(q), or

(c) G = GO_{4}^{−}(q).

Proof. First assume that n = 3. Let GO_{3}(q) be the orthogonal group with respect to the quadratic form 2x_{1}x_{3} + x_{2}^{2}. Then the first standard basis vector is isotropic, with stabilizer

N := \{[ \begin{matrix}1 & ab & −b^{2}/2 \\ 0 & a & −b \\ 0 & 0 & 1 \end{matrix} ] | a = ±1, b ∈ 𝔽_{q}\}.

The eigenvectors for the nontrivial eigenvalues of the reflections in N are anisotropic, with stabilizer of type O_{2}^{+}(q). Thus the reflections in N all lie in R^{+}O_{3}(q). Hence the stabilizer in R^{−}O_{3}(q) does not contain reflections and R^{−}O_{3}(q) does not have polynomial invariants. This shows one direction of the assertion in dimension 3. For the other direction we cite [13, Thm. 2.4(c)] where it is shown that, for any odd q, the groups GO_{3}(q) and R^{+}O_{3}(q) both have a polynomial ring of invariants.

Now let GO_{4}^{−}(q) be the orthogonal group of the quadratic form 2x_{1}x_{4} + x_{2}^{2} − ϵ^{−1}x_{3}^{2} with a nonsquare ϵ ∈ k := 𝔽_{q}, i.e., the subgroup of elements in GL(V) (where V = k^{4}) which respect the scalar product

⟨α_{1}x_{1} + … + α_{4}x_{4}, β_{1}x_{1} + … + β_{4}x_{4}⟩ = α_{1}β_{4} + α_{2}β_{2} − ϵ^{−1}α_{3}β_{3} + α_{4}β_{1},

Mapping v to v^{∗} with v^{∗}(w) = ⟨v, w⟩, we get an isomorphism V ⟶^{∼} V^{∗} with the dual space, which has a scalar product given by ⟨v^{∗}, w^{∗}⟩ = ⟨v, w⟩. With this, G = GO(V^{∗}), and an easy calculation shows that ⟨v^{∗}, v^{∗}⟩ = (2x_{1}x_{4} + x_{2}^{2} − ϵx_{3}^{2})(v^{∗}) for v^{∗} ∈ V^{∗}. We obtain invariants

f_{1} := 2x_{1}x_{4} + x_{2}^{2} − ϵx_{3}^{2}

and by application of Steenrod operations (see [19, p. 324])

\begin{matrix}f_{2} := & \frac{1}{2}P^{1}(f_{1}) = x_{1}x_{4}^{q} + x_{1}^{q}x_{4} + x_{2}^{q+1} − ϵx_{3}^{q+1}, \\ f_{3} := & P^{q}(f_{2}) = x_{1}x_{4}^{q^{2}} + x_{1}^{q^{2}}x_{4} + x_{2}^{q^{2}+1} − ϵx_{3}^{q^{2}+1}. \end{matrix}

These invariants belong to a series of invariants of degrees q^{i} + 1 discovered by Carlisle and Kropholler [3]. As G has order 2q^{2}(q^{2} − 1)(q^{2} + 1) (see, for example, [7]), we need to find another invariant f_{4} of degree q^{2}(q − 1) in order to show that the invariant ring is polynomial.

We take the set V_{0} = {v ∈ V | v ≠ 0 and ⟨v, v⟩ = 0} of isotropic vectors, which has cardinality (q^{2} + 1)(q − 1). Clearly, if v ∈ V_{0}, then all scalar multiples αv, α ∈ k^{×}, also lie in V_{0}. Hence T = {v^{q−1} | v ∈ V_{0}} has q^{2} + 1 elements, and G acts on T. Then

f_{4} := ∑_{h∈T} ∏_{{h′∈T \atop h′≠h}} h′

is an invariant of the desired degree, and it remains to show that the Jacobian determinant J = det((∂f_{i}/∂x_{j})_{i,j}) does not vanish. But

J = | \begin{matrix}2x_{4} & 2x_{2} & −2ϵx_{3} & 2x_{1} \\ x_{4}^{q} & x_{2}^{q} & −ϵx_{3}^{q} & x_{1}^{q} \\ x_{4}^{q^{2}} & x_{2}^{q^{2}} & −ϵx_{3}^{q^{2}} & x_{1}^{q^{2}} \\ ∂f_{4}/∂x_{1} & ∂f_{4}/∂x_{2} & ∂f_{4}/∂x_{3} & ∂f_{4}/∂x_{4} \end{matrix} |

is congruent to

| \begin{matrix}2x_{4} & 2x_{2} & −2ϵx_{3} \\ x_{4}^{q} & x_{2}^{q} & −ϵx_{3}^{q} \\ x_{4}^{q^{2}} & x_{2}^{q^{2}} & −ϵx_{3}^{q^{2}} \end{matrix} | ∂f_{4}/∂x_{4}

modulo the ideal generated by x_{1}. Since the 3-by-3 determinant is different

from zero, it suffices to show that ∂f_{4}/∂x_{4} = ∑_{h∈T} ∂/∂x_{4} (∏_{{h′∈T \atop h′≠h}} h′) ≢ 0

mod x_{1}. Now x_{1}^{q−1} ∈ T, so only the summand with h = x_{1}^{q−1} contributes to the above sum mod x_{1}. Hence we have to show that for

U = {α_{1}x_{1}+α_{2}x_{2}+α_{3}x_{3}+x_{4} | α_{i} ∈ k, 2α_{1}+α_{2}^{2}−ϵ^{−1}α_{3}^{2} = 0} and g = ∏_{v∈U} v,

we have ∂/∂x_{4}(g^{q−1}) = −g^{q−2} · ∂g/∂x_{4} ≢ 0 mod x_{1}. But now we also

specialize x_{4} = 0 in ∂g/∂x_{4} = ∑_{v∈U} (∏_{{v′∈U \atop v′≠v}} v′). There is exactly one v ∈

U such that v ≡ 0 mod (x_{1}, x_{4}), namely v = x_{4}, hence only the summand with v = x_{4} contributes to the above sum mod (x_{1}, x_{4}), and we get ∂g/∂x_{4} ≢ 0 mod (x_{1}, x_{4}). This shows that the Jacobian determinant of f_{1}, …, f_{4} is nonzero, and since the degree product of the f_{i} equals |G|, we conclude that they generate the invariant ring as a polynomial ring.

The fixator in G = GO_{4}^{−}(q) of an anisotropic point is isomorphic to GO_{3}(q). Hence for at least one of the two reflection subgroups of G of index 2 the stabilizer of an anisotropic vector is a 3-dimensional orthogonal group with nonpolynomial invariant ring. But the two reflection subgroups of G are conjugate inside the conformal orthogonal group CO_{4}^{−}(q); thus neither of them has polynomial invariants.

Now let G = GO_{2m}^{+}(q). Then the pointwise stabilizer of a maximal isotropic subspace is conjugate to

(4.2) \{[ \begin{matrix}E_{m} & A \\ 0 & E_{m} \end{matrix} ] | A=(a_{ij})∈𝔽_{q}^{m×m}, a_{ij} =−a_{m+1−j,m+1−i}, a_{i,m+1−i} =0\} ,

so it has order q^{m(m−1)/2} and does not contain reflections. Thus the same is true for the reflection subgroups of G of index 2. Further, if G = GO_{2m+1}(q), then there exist anisotropic points with stabilizer GO_{2m}^{+}(q), and these in turn by the above for m ≥ 2 contain fixators which are not generated by reflections. The same argument deals with the reflection subgroups of GO_{2m+1}(q) of index 2. Finally, if G = GO_{2m}^{−}(q) then the stabilizer of an anisotropic point is a group G = GO_{2m−1}(q), which brings us back to the previous case for m ≥ 3. □

Orthogonal groups in characteristic 2 In even characteristic we only have to consider even-dimensional orthogonal groups. We obtain the following:

Proposition 4.3. In the situation of Theorem 1.5 let G = SO_{2m}^{±}(q) with q even, m ≥ 2. Then S(V)^{G} is polynomial if and only if G = SO_{4}^{−}(q).

Proof. First let G = SO_{2m}^{+}(q), m ≥ 2, be the orthogonal group with respect to the quadratic form ∑_{i=1}^{m} x_{i}x_{2m+1−i}. Then the stabilizer of the maximal isotropic subspace spanned by the first m basis vectors is given by (4.2), so does not contain transvections. This shows the result for the orthogonal groups with maximal Witt index.

Now let G = SO_{2m}^{−}(q) with m ≥ 3. The stabilizer of an anisotropic point is isomorphic to SO_{2m−1}(q) and thus the stabilizer of a suitable 2dimensional anisotropic subspace is isomorphic to SO_{2m−2}^{+}(q). But by the first part for m ≥ 3 the stabilizer in this group of a maximal isotropic subspace is not generated by transvections.

Finally assume G = SO_{4}^{−}(q), i.e., G is the orthogonal group over V = k^{4}, k = 𝔽_{q}, of the form

Q(α_{1}x_{1} + … + α_{4}x_{4}) = α_{1}α_{4} + α_{2}^{2} + ϵα_{3}^{2} + α_{2}α_{3},

where ϵ ∈ k is such that the polynomial t^{2} + t + ϵ ∈ k[t] is irreducible. We obtain a nondegenerate bilinear form by setting ⟨v, w⟩ := Q(v +w)−Q(v)− Q(w), which as in Proposition 4.1 leads to an isomorphism V ⟶^{∼} V^{∗}. Endowing V^{∗} with a quadratic form via this isomorphism, we get G = GO(V^{∗}). Thus the quadratic form of V^{∗} is an invariant, and expressing it in terms of the basis x_{1}, …, x_{4} yields

f_{1} = x_{1}x_{4} + ϵx_{2}^{2} + x_{3}^{2} + x_{2}x_{3} ∈ S(V)^{G}.

Applying Steenrod operations, we obtain further invariants

\begin{matrix}f_{2} := & P^{1}(f_{1}) = x_{1}x_{4}^{q} + x_{1}^{q}x_{4} + x_{2}x_{3}^{q} + x_{2}^{q}x_{3}, \\ f_{3} := & P^{q}(f_{2}) = x_{1}x_{4}^{q^{2}} + x_{1}^{q^{2}}x_{4} + x_{2}x_{3}^{q^{2}} + x_{2}^{q^{2}}x_{3}. \end{matrix}

To construct an invariant f_{4} of degree q^{2}(q−1), we again take an elementary symmetric function of a suitable orbit. The set T = {v^{q−1} | v ∈ V \ {0}, Q(v) = 0} has q^{2} + 1 elements, and we form

f_{4} = ∑_{h∈T} ∏_{{h′∈T \atop h′≠h}} h′.

The Jacobian determinant of (f_{1}, f_{2}, f_{3}, f_{4}) modulo x_{1} is

| \begin{matrix}x_{4} & x_{3} & x_{2} & x_{1} \\ x_{4}^{q} & x_{3}^{q} & x_{2}^{q} & x_{1}^{q} \\ x_{4}^{q^{2}} & x_{3}^{q^{2}} & x_{2}^{q^{2}} & x_{1}^{q^{2}} \\ ∂f_{4}/∂x_{1} & ∂f_{4}/∂x_{2} & ∂f_{4}/∂x_{3} & ∂f_{4}/∂x_{4} \end{matrix} | ≡ | \begin{matrix}x_{4} & x_{3} & x_{2} \\ x_{4}^{q} & x_{3}^{q} & x_{2}^{q} \\ x_{4}^{q^{2}} & x_{3}^{q^{2}} & x_{2}^{q^{2}} \end{matrix} | ∂f_{4}/∂x_{4}.

Just as in the calculation for GO_{4}^{−}(q) in odd characteristic we see that it is sufficient to show that ∂g/∂x_{4} ≢ 0 mod x_{1}, where g = ∏_{v∈U} v with

U = {α_{1}x_{1} + α_{2}x_{2} + α_{3}x_{3} + x_{4} | α_{i} ∈ k, α_{1} + α_{2}^{2} + ϵα_{3}^{2} + α_{2}α_{3} = 0},

and we show this by specializing x_{4} =0. Again, x_{4} is the only element of U which is zero mod (x_{1}, x_{4}), hence ∂g/∂x_{4} ≡∏_{v∈U\{x_{4}}} v ≢0 mod (x_{1}, x_{4}). □

5. The symmetric groups

The case of symmetric groups in odd characteristic was already settled in [15, Thm. 4.5]. For the convenience of the reader we give a simple inductive proof which is valid in arbitrary characteristic:

Proposition 5.1. In the situation of Theorem 1.3 or Theorem 1.5 let G = 𝔖_{n+1}, p |̸(n+1), or G = 𝔖_{n+2}, p|(n+2), n ≥ 3. Then S(V)^{G} is polynomial if and only if G = 𝔖_{n+1}.

Proof. Consider the natural n + 1-dimensional permutation representation of G = 𝔖_{n+1} on W := k^{n+1}. Then by the fundamental theorem on symmetric functions, the ring of invariants S(W)^{G} is polynomial, generated by the elementary symmetric functions. It contains the 1-dimensional invariant subspace W′ consisting of the vectors where all coordinates are equal. Thus we have an epimorphism W → V := W/W′ of faithful G-modules. Since S(W)^{G} is polynomial, it follows by [15, Prop. 4.1] that the invariant ring S(V)^{G} also is polynomial. In fact, it is generated by the elementary symmetric functions of degree larger than one in the original coordinates.

Now consider the natural permutation representation of G = 𝔖_{pm} on 𝔽_{p}^{pm}, where pm ≥ 5. Let Ṽ be the pm − 1-dimensional subspace generated by the differences e_{i} − e_{j} of basis vectors. This contains the 1-dimensional invariant subspace spanned by v := ∑_{i=1}^{pm} e_{i}; we write V for the quotient of Ṽ by 𝔽_{p}v. If p = 2, m = 3, then G is conjugate to Sp_{4}(2), which has a nonpolynomial ring of invariants by Proposition 3.4. If p = 3, m = 2, then G on V is conjugate in GL_{4}(3) to RO_{4}^{−}(3), hence does not have a polynomial ring of invariants by Proposition 4.1. If p ≥ 5, m = 1, then the stabilizer in G of the vector ∑_{i=1}^{p} ie_{i} ∈ Ṽ is trivial. But the stabilizer of its image in V is generated by the p-cycle (1, 2, …, p). Hence it is not generated by pseudoreflections.

Now let m ≥ 2, respectively m ≥ 3 if p = 3, m ≥ 4 if p = 2. Then the fixator of the subspace W̃ := ⟨e_{i} − e_{j} | p(m − 1) < i < j⟩ ≤ Ṽ is the natural subgroup 𝔖_{p(m−1)}, and the same is true for its image W in V. Hence by induction we obtain that V contains a subspace with fixator not generated by pseudoreflections, respectively with nonpolynomial invariants if p = 2. □

For future use we note the following consequence of the previous result and [15, Prop. 4.1] on reducible representations:

Corollary 5.2. Let G = 𝔖_{n}, p|n, n ≥ 5, in its natural permutation representation. Then G has a polynomial ring of invariants, while its submodule of dimension n − 1 does not have polynomial invariants.

6. The exceptional groups

In Table 6.2 we identify some of the p-modular reductions of exceptional complex reflection groups with classical groups (this information can easily be extracted from the Atlas [7]).

Table 6.3 contains data on the exceptional primitive reflection groups. The first column gives the Shephard–Todd name [18], the second one the Coxeter resp. Cohen name [6]. We then list the dimension and the order. The last three columns indicate whether the ring of invariants in characteristic p is polynomial for p ∈ {3, 5, 7}. An entry “–” signifies that either p does not divide |G|, or the reduction becomes reducible, or the reduced group has smaller order than the complex group.

Proposition 6.1. Let G be an exceptional p-modular reflection group as in Theorem 1.3(e)–(g). Then its ring of invariants is polynomial if and only if either p |̸|G| or the corresponding entry in Table 6.3 is ”p”.

Remark. Several cases of the above proposition were already known from the work of Demazure [8] and Xu [24] or easily follow from Nakajima’s results [15] using Table 6.2. Proof. For each group G occurring in Table 6.3 we do one of two things:

(a) We give a set of fundamental polynomial invariants f_{1}, …, f_{n} such that 6.2. Reductions of exceptional complex reflection groups

G_{23}  G_{24}  G_{25}  G_{26}  G_{27}  G_{28}
W_{2}(G)  O_{3}(4) SL_{3}(2) GU_{3}(2) GU_{3}(2)  3^{.}A_{6} reduc.
W_{3}(G)  reduc. reduc. R^{+}O_{3}(9) GO_{4}^{+}(3)
W_{5}(G) R^{+}O_{3}(5)
W_{7}(G)  R^{−}O_{3}(7)
G_{29}  G_{30}  G_{31}  G_{32}  G_{33}  G_{34}
W_{2}(G) SO_{4}^{−}(2) SO_{4}^{+}(4) Sp_{4}(2) GU_{4}(2)  reduc. 3^{.}U_{4}(3).2_{2}
W_{3}(G) Sp_{4}(3) R^{−}O_{5}(3) RO_{6}^{−}(3)
W_{5}(G)  RO_{4}^{+}(5)
G_{35} G_{36}  G_{37}
W_{2}(G) SO_{6}^{−}(2) O_{7}(2) SO_{8}^{+}(2)
W_{3}(G) reduc.

6.3. Invariant rings for higher-dimensional exceptional reflection groups

[18] name dim  |G|  3 5 7
G_{23} H_{3}  3  120  p p –
G_{24} J_{3}(4) 3  336  p – np
G_{27} J_{3}(5) 3  2160  – p –
EJ_{3}(5) 3  15120  – p –
G_{28}  F_{4}  4  1152  np – –
G_{29} N_{4}  4  7680  p p –
G_{30} H_{4}  4  14400  np np –
G_{31} EN_{4}  4  46080  np p –
J_{4}(4) 4  161280  p – –
G_{33} K_{5}  5  51840  np p –
G_{34} K_{6}  6  39191040  – p p
G_{35} E_{6}  6  51840  – p –
G_{36} E_{7}  7  2903040  np p p
G_{37} E_{8}  8 696729600  np np p

the Jacobian determinant does not vanish and the degree product is equal to the group order |G|. In this case the invariants are polynomial by Proposition 2.4. These calculations were mainly done with INVAR [14] and SIN-GULAR [10]. (b) We exhibit a subspace whose fixator in G does not have polynomial invariants. In this case, we used the GAP system [16].

The group W_{3}(H_{3}) can be generated by the matrices

[ \begin{matrix}1 & 0 & 0 \\ 0 & −1 & 0 \\ 0 & 0 & 1 \end{matrix} ] , [ \begin{matrix}1 & 1 & 1 \\ 0 & −1 & 1 \\ 0 & 0 & 1 \end{matrix} ] , [ \begin{matrix}0 & 0 & ζ^{2} \\ 0 & 1 & 0 \\ ζ^{6} & 0 & 0 \end{matrix} ] ∈ GO_{3}(9),

where ζ is a generator of 𝔽_{9}^{×}. We then have invariants f_{2} := 2x_{1}x_{3} + x_{2}^{2} from the quadratic form,

\begin{matrix}f_{6} := & x_{2}^{6} + (x_{1}^{2} − x_{1}x_{3} − x_{3}^{2})x_{2}^{4} − (x_{1}^{4} − x_{1}^{3}x_{3} + x_{1}^{2}x_{3}^{2} + x_{1}x_{3}^{3} + x_{3}^{4})x_{2}^{2} \\ + x_{1}^{6} − x_{1}^{5}x_{3} + x_{1}^{4}x_{3}^{2} − x_{1}^{2}x_{3}^{4} − x_{1}x_{3}^{5} − x_{3}^{6}, \end{matrix}

and f_{10} := x_{1}x_{3}^{9} + x_{1}^{9}x_{3} + x_{2}^{10} = P^{1}(f_{2}). Since the Jacobian determinant of {f_{2}, f_{6}, f_{10}} does not vanish, these invariants are algebraically independent. Their degree product is equal to the group order, which shows that the ring of invariants is polynomial.

By Table 6.2 the 5-modular reduction W_{5}(H_{3}) of the Coxeter group of type H_{3} is conjugate in GL_{3}(5) to R^{+}O_{3}(5), so it has a polynomial ring of invariants by our study of the orthogonal groups in Proposition 4.1.

The group W_{3}(G_{24}) can be generated by the matrices

[ \begin{matrix}0 & 0 & −1 \\ 0 & 1 & 0 \\ −1 & 0 & 0 \end{matrix} ] , [ \begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & −1 \end{matrix} ] , [ \begin{matrix}0 & ζ^{5} & ζ^{3} \\ ζ^{7} & −1 & ζ^{6} \\ ζ & ζ^{2} & −1 \end{matrix} ] ∈ GU_{3}(3),

where ζ is a generator of 𝔽_{9}^{×}. We then have the unitary invariant f_{4} := x_{1}^{4} + x_{2}^{4} + x_{3}^{4}, an invariant of degree 6

\begin{aligned}f_{6} : & = x_{1}^{6} + x_{2}^{6} + x_{3}^{6} − x_{1}^{4}x_{2}^{2} − x_{1}^{2}x_{2}^{4} − x_{1}^{4}x_{3}^{2} − x_{1}^{2}x_{3}^{4} − x_{2}^{4}x_{3}^{2} − x_{2}^{2}x_{3}^{4} \\ & − ζ (x_{3}^{6} + x_{1}^{6} + x_{2}^{6} − x_{1}^{2}x_{2}^{2}x_{3}^{2}), \end{aligned}

and f_{14} := P^{1}(f_{6}).

On the other hand, W_{7}(G_{24}) = R^{−}O_{3}(7), so this group does not have polynomial invariants.

The group W_{5}(G_{27}) can be generated by the matrices

[ \begin{matrix}−1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix} ] , [ \begin{matrix}0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \end{matrix} ] , [ \begin{matrix}−1 & ζ^{24} & ζ^{20} \\ ζ^{2} & −1 & ζ^{4} \\ ζ^{4} & ζ^{20} & 3 \end{matrix} ] ∈ GU_{3}(5),

where ζ is a generator of 𝔽_{25}^{×}. We have the unitary invariant f_{6} := x_{1}^{6} + x_{2}^{6} + x_{3}^{6}, further

f_{12} : = x_{1}^{12} + 2(x_{2}^{2} + x_{3}^{2})x_{1}^{10} − (x_{2}^{4} − x_{2}^{2}x_{3}^{2} + x_{3}^{4})x_{1}^{8} − (x_{2}^{4}x_{3}^{2} + x_{2}^{2}x_{3}^{4})x_{1}^{6}

− (x_{2}^{8} + x_{2}^{6}x_{3}^{2} + x_{2}^{2}x_{3}^{6} + x_{3}^{8})x_{1}^{4} + x_{2}^{12} + 2x_{2}^{10}x_{3}^{2} − x_{2}^{8}x_{3}^{4} − x_{2}^{4}x_{3}^{8} + 2x_{2}^{2}x_{3}^{10}

+ x_{3}^{12} + (2x_{2}^{10} + x_{2}^{8}x_{3}^{2} − x_{2}^{6}x_{3}^{4} − x_{2}^{4}x_{3}^{6} + x_{2}^{2}x_{3}^{8} + 2x_{3}^{10})x_{1}^{2}

and f_{30} := det((∂^{2}f_{12}/∂x_{i}∂x_{j})_{ij}) as the Hessian of f_{12}.

The group EJ_{3}(5) = 3 · A_{7} × 2 can be generated by the matrices

[ \begin{matrix}0 & ζ^{4} & 0 \\ ζ^{20} & 0 & 0 \\ 0 & 0 & 1 \end{matrix} ] , [ \begin{matrix}0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{matrix} ] , [ \begin{matrix}0 & ζ^{19} & ζ^{11} \\ ζ^{23} & 3 & ζ^{22} \\ ζ^{7} & ζ^{14} & 3 \end{matrix} ] ∈ GU_{3}(5),

where ζ is a generator of 𝔽_{25}^{×}. (In this representation, a subgroup G(6, 3, 3) of EJ_{3}(5) becomes monomial with respect to the standard basis.) This group has the unitary invariant f_{6} := x_{1}^{6} + x_{2}^{6} + x_{3}^{6}, and a further invariant

f_{42} := x_{1}^{42} + 3x_{1}^{38}x_{2}^{2}x_{3}^{2} + 3x_{1}^{34}x_{2}^{4}x_{3}^{4} + (x_{2}^{6} + x_{3}^{6})x_{1}^{32}x_{2}^{2}x_{3}^{2} + 2(x_{2}^{12} + x_{3}^{12})x_{1}^{30}

− (x_{2}^{6} + x_{3}^{6})x_{1}^{28}x_{2}^{4}x_{3}^{4} + 3(x_{2}^{6} + x_{3}^{6})^{2}x_{1}^{26}x_{2}^{2}x_{3}^{2} + (x_{2}^{6} + x_{3}^{6})^{3}x_{1}^{24} − x_{1}^{22}x_{2}^{10}x_{3}^{10}

+ (x_{2}^{18} + x_{3}^{18})x_{1}^{20}x_{2}^{2}x_{3}^{2} + (x_{2}^{6} + x_{3}^{6})^{4}x_{1}^{18} + 3(x_{2}^{6} + x_{3}^{6})x_{1}^{16}x_{2}^{10}x_{3}^{10}

+ 3(x_{2}^{24} + x_{3}^{24})x_{1}^{14}x_{2}^{2}x_{3}^{2} + 2(x_{2}^{24} + 3x_{2}^{18}x_{3}^{6} + 3x_{2}^{6}x_{3}^{18} + x_{3}^{24})(x_{2}^{6} + x_{3}^{6})x_{1}^{12}

− (x_{2}^{24} + x_{2}^{18}x_{3}^{6} + 2x_{2}^{12}x_{3}^{12} + x_{2}^{6}x_{3}^{18} + x_{3}^{24})x_{1}^{10}x_{2}^{4}x_{3}^{4}

+ (x_{2}^{30} + x_{2}^{24}x_{3}^{6} + x_{2}^{6}x_{3}^{24} + x_{3}^{30})x_{1}^{8}x_{2}^{2}x_{3}^{2} + 3(x_{2}^{24}x_{3}^{12} + 3x_{2}^{18}x_{3}^{18} + x_{2}^{12}x_{3}^{24})x_{1}^{6}

+ 3(x_{2}^{30} + 3x_{2}^{24}x_{3}^{6} + 3x_{2}^{6}x_{3}^{24} + x_{3}^{30})x_{1}^{4}x_{2}^{4}x_{3}^{4}

+ 3(x_{2}^{36} + 2x_{2}^{30}x_{3}^{6} + x_{2}^{24}x_{3}^{12} + 2x_{2}^{18}x_{3}^{18} + x_{2}^{12}x_{3}^{24} + 2x_{2}^{6}x_{3}^{30} + x_{3}^{36})x_{1}^{2}x_{2}^{2}x_{3}^{2}

+ x_{2}^{42} + 2x_{2}^{30}x_{3}^{12} + x_{2}^{24}x_{3}^{18} + x_{2}^{18}x_{3}^{24} + 2x_{2}^{12}x_{3}^{30} + x_{3}^{42}

of degree 42. A third invariant of degree 60 is obtained by the Steenrod operation as f_{60} := P^{1}(f_{42})/f_{6}.

By Table 6.2 we have W_{3}(F_{4}) = GO_{4}^{+}(3), so this group does not have polynomial invariants.

The group W_{3}(G_{29}) can be generated by the permutation matrices corresponding to the transpositions (2, 3), (3, 4) together with the reflections

[ \begin{matrix}0 & ζ^{2} & 0 & 0 \\ ζ^{6} & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ] , [ \begin{matrix}−1 & 1 & 1 & −1 \\ 1 & −1 & 1 & −1 \\ 1 & 1 & −1 & −1 \\ −1 & −1 & −1 & −1 \end{matrix} ]

in GU_{4}(3), where ζ is a generator of 𝔽_{9}^{×}. We have the unitary invariant f_{4} := x_{1}^{4} + x_{2}^{4} + x_{3}^{4} + x_{4}^{4}, as well as

\begin{matrix}f_{8} := & x_{1}^{8} + x_{2}^{8} + x_{3}^{8} + x_{4}^{8} + x_{1}x_{2}x_{3}x_{4}(x_{1}^{4} + x_{2}^{4} + x_{3}^{4} + x_{4}^{4} − x_{1}x_{2}x_{3}x_{4}), \\ f_{12} := & (P^{1}(f_{8}) − f_{8}^{2})/f_{4}, f_{20} := P^{1}(f_{12}). \end{matrix}

The group W_{5}(G_{29}) can be generated by the permutation matrices corresponding to the transpositions (2, 3), (3, 4) together with the matrices

[ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & −1 \end{matrix} ] , [ \begin{matrix}−2 & 1 & 1 & 2 \\ −1 & −2 & 2 & −1 \\ −1 & 2 & −2 & −1 \\ 2 & 1 & 1 & −2 \end{matrix} ]

in GL_{4}(5). We find the algebraically independent invariants

\begin{matrix}f_{4} := & x_{1}^{4} + x_{2}^{4} + x_{3}^{4} + x_{4}^{4} − x_{1}^{2}x_{2}^{2} − x_{1}^{2}x_{3}^{2} − x_{1}^{2}x_{4}^{2} − x_{2}^{2}x_{3}^{2} − x_{2}^{2}x_{4}^{2} − x_{3}^{2}x_{4}^{2}, \\ f_{8} := & P^{1}(f_{4}) − f_{4}^{2}, f_{12} := P^{1}(f_{8}), f_{20} := P^{5}(f_{8})/f_{8}. \end{matrix}

The group W_{3}(H_{4}) can be generated by the reflections

[ \begin{matrix}−1 & ζ^{3} & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ] [ \begin{matrix}1 & 0 & 0 & 0 \\ ζ^{3} & −1 & 1 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ] , [ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 1 & −1 & 1 \\ 0 & 0 & 0 & 1 \end{matrix} ] , [ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 1 & −1 \end{matrix} ]

in GL_{4}(9), where ζ denotes a generator of 𝔽_{9}^{×}. The stabilizer of the subspace ⟨e_{1} + ζ^{2}e_{4}, e_{3} − e_{4}⟩ is cyclic of order 3 and contains no pseudoreflection. Also, we have W_{5}(H_{4}) = RO_{4}^{+}(5) so in both cases the invariants are not polynomial.

The group W_{3}(G_{31}) has an orbit of length 960 on the points of the underlying vector space 𝔽_{9}^{4}. The stabilizer of such a point has order 48, while the subgroup generated by its reflections has order 16.

The group W_{5}(G_{31}) can be generated by the matrices for W_{5}(G_{29}) together with

[ \begin{matrix}0 & −2 & 0 & 0 \\ 2 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ] ∈ GL_{4}(5).

Here one finds

f_{8} := ∑_{i=1}^{4} x_{i}^{8} − ∑_{i<j} x_{i}^{4}x_{j}^{4} − 2x_{1}^{2}x_{2}^{2}x_{3}^{2}x_{4}^{2},

and f_{12} := P^{1}(f_{8}), f_{20} := P^{5}(f_{8})/f_{8}, f_{24} := P^{1}(f_{20}).

The group J_{4}(4) = 4 · L_{3}(4) : 2_{2} can be generated by the permutation matrices corresponding to the transpositions (2, 3), (3, 4), together with the reflections

[ \begin{matrix}−1 & −1 & −1 & 1 \\ 1 & 0 & −1 & 1 \\ 1 & −1 & 0 & 1 \\ −1 & 1 & 1 & 0 \end{matrix} ] , [ \begin{matrix}−1 & ζ & ζ^{5} & ζ \\ ζ^{7} & −1 & −1 & 1 \\ ζ^{3} & −1 & −1 & −1 \\ ζ^{7} & 1 & −1 & −1 \end{matrix} ]

in GU_{4}(3), where ζ denotes a generator of 𝔽_{9}^{×}. (In this representation, a subgroup G(4, 4, 4) of J_{4}(4) becomes monomial with respect to the standard basis.) This group has the unitary invariants f_{4} := x_{1}^{4} − x_{2}^{4} − x_{3}^{4} − x_{4}^{4} and f_{28} := P^{3}(f_{4}) = x_{1}^{28} + x_{2}^{28} + x_{3}^{28} + x_{4}^{28}, a further invariant

f_{20} := x_{1}^{20} + (x_{1}^{4} − x_{2}^{4} − x_{3}^{4} − x_{4}^{4})^{2}(x_{1}^{8} + x_{2}^{8} + x_{3}^{8} + x_{4}^{8})x_{1}x_{2}x_{3}x_{4}
− (x_{1}^{4} + x_{2}^{4} + x_{3}^{4} + x_{4}^{4})x_{1}^{10}x_{2}^{2}x_{3}^{2}x_{4}^{2} + (x_{2}^{8} + x_{3}^{8} + x_{4}^{8})x_{1}^{9}x_{2}x_{3}x_{4}

− (x_{2}^{12} + x_{2}^{8}x_{3}^{4} + x_{2}^{8}x_{4}^{4} + x_{2}^{4}x_{3}^{8} − x_{2}^{4}x_{3}^{4}x_{4}^{4} + x_{2}^{4}x_{4}^{8} + x_{3}^{12} + x_{3}^{8}x_{4}^{4} + x_{3}^{4}x_{4}^{8}

+ x_{4}^{12})x_{1}^{8} + (x_{1}^{4} + x_{2}^{4} + x_{3}^{4} + x_{4}^{4})x_{1}^{7}x_{2}^{3}x_{3}^{3}x_{4}^{3} + (x_{2}^{8} + x_{2}^{4}x_{3}^{4} + x_{2}^{4}x_{4}^{4} + x_{3}^{8}

+ x_{3}^{4}x_{4}^{4} + x_{4}^{8})x_{1}^{6}x_{2}^{2}x_{3}^{2}x_{4}^{2} + (x_{2}^{16} + x_{2}^{8}x_{3}^{8} − x_{2}^{8}x_{3}^{4}x_{4}^{4} + x_{2}^{8}x_{4}^{8} − x_{2}^{4}x_{3}^{4}x_{4}^{8}
− x_{2}^{4}x_{3}^{4}x_{4}^{8} + x_{3}^{16} + x_{4}^{8}x_{3}^{8} + x_{4}^{16})x_{1}^{4} + (x_{2}^{4} + x_{3}^{4} + x_{4}^{4})^{2}x_{1}^{3}x_{2}^{3}x_{3}^{3}x_{4}^{3}

+ (x_{2}^{12} − x_{2}^{8}x_{3}^{4} − x_{2}^{8}x_{4}^{4} − x_{2}^{4}x_{3}^{8} − x_{2}^{4}x_{3}^{4}x_{4}^{4} − x_{2}^{4}x_{4}^{8} + x_{3}^{12} − x_{3}^{8}x_{4}^{4} − x_{3}^{4}x_{4}^{8}

+ x_{4}^{12})x_{1}^{2}x_{2}^{2}x_{3}^{2}x_{4}^{2} + (x_{2}^{8}x_{3}^{8} + x_{2}^{8}x_{4}^{8} + x_{3}^{8}x_{4}^{8})x_{1}x_{2}x_{3}x_{4} + x_{2}^{20} − x_{2}^{12}x_{3}^{8}
+ x_{2}^{12}x_{3}^{4}x_{4}^{4} − x_{2}^{12}x_{4}^{8} − x_{2}^{8}x_{3}^{12} − x_{2}^{8}x_{3}^{8}x_{4}^{4} − x_{2}^{8}x_{3}^{4}x_{4}^{8} − x_{2}^{8}x_{4}^{12}
+ x_{2}^{4}x_{3}^{12}x_{4}^{4} − x_{2}^{4}x_{3}^{8}x_{4}^{8} + x_{2}^{4}x_{3}^{4}x_{4}^{12} + x_{3}^{20} − x_{3}^{12}x_{4}^{8} − x_{3}^{8}x_{4}^{12} + x_{4}^{20}

of degree 20 and f_{72} as the Hessian of f_{20}.

By Table 6.2 we have W_{3}(G_{33}) = R^{−}O_{5}(3); thus this group has nonpolynomial invariants by Proposition 4.1.

Let Ĝ be the subgroup of GL_{5}(ℂ) generated by the transpositions (1, 2), (2, 3), (3, 4), together with

[ \begin{matrix}0 & ζ & 0 & 0 & 0 \\ ζ^{2} & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \end{matrix} ] , \frac{1}{3} [ \begin{matrix}2 & −ζ & −ζ & −ζ & 3 \\ ζ + 1 & 2 & −1 & −1 & −3(ζ + 1) \\ ζ + 1 & −1 & 2 & −1 & −3(ζ + 1) \\ ζ + 1 & −1 & −1 & 2 & −3(ζ + 1) \\ 2/3 & 2ζ/3 & 2ζ/3 & 2ζ/3 & 3 \end{matrix} ]

in GL_{5}(ℂ), where ζ now denotes a complex third root of unity. Then W_{5}(G_{33}) can be obtained as the 5-modular reduction of Ĝ. The vector 18e_{1}−18e_{2} is a root of a reflection of Ĝ; thus its orbit under Ĝ has length 270. We denote by v_{1}, …, v_{45} a system of representatives of this orbit modulo scalar multiplication by the sixth roots of unity. Then for all i which are multiples of 6 the power sums

(6.4)  p_{i} := ∑_{j=1}^{45} v_{j}^{i}

are invariants for Ĝ and their 5-modular reductions yield invariants for W_{p}(G_{33}). It can be verified with SINGULAR that the following linear combinations of power sums have integral coefficients

f̂_{6} := p_{6}, f̂_{12} := (432 p_{12} − 61 p_{6}^{2})/5, f̂_{18} := p_{18}.

We denote their 5-modular reductions by f_{i} for i ∈ {6, 12, 18}. We further find the invariant f_{4} := x_{1}^{3}x_{5} +x_{2}^{3}x_{5} +x_{3}^{3}x_{5} +x_{4}^{3}x_{5} +3x_{5}^{4} −x_{1}x_{2}x_{3}x_{4}, and f_{10} as the Hessian determinant of f_{4}. Then the Jacobian of {f_{4}, f_{6}, f_{10}, f_{12}, f_{18}} does not vanish, hence these generate the ring of invariants as a polynomial ring.

The group Ĝ of type G_{34} can be generated by the permutation matrices corresponding to the transpositions (1, 2), (2, 3), (3, 4), (4, 5) together with the reflections

[ \begin{matrix}0 & ζ^{4} & 0 & 0 & 0 & 0 \\ ζ^{8} & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 \end{matrix} ] , \frac{1}{3} [ \begin{matrix}2 & −ζ^{4} & −ζ^{4} & −ζ^{4} & −ζ^{4} & ζ^{11}/3 \\ −ζ^{8} & 2 & −1 & −1 & −1 & ζ^{7}/3 \\ −ζ^{8} & −1 & 2 & −1 & −1 & ζ^{7}/3 \\ −ζ^{8} & −1 & −1 & 2 & −1 & ζ^{7}/3 \\ −ζ^{8} & −1 & −1 & −1 & 2 & ζ^{7}/3 \\ 3ζ & 3ζ^{5} & 3ζ^{5} & 3ζ^{5} & 3ζ^{5} & 2 \end{matrix} ]

in GL_{6}(ℂ), where ζ denotes a twelfth root of unity. These generators are integral at the prime 5, so W_{5}(G_{34}) can be obtained as the 5-modular reduction of the above complex representation of Ĝ. The vector 3e_{1} − 3e_{2} is a root of a reflection of Ĝ, and its orbit has length 756. As above we denote by v_{1}, …, v_{126} a system of representatives of this orbit under the action of the center of Ĝ of order 6. Then for all multiples of 6, the power sums analogous to (6.4) yield invariants for W_{p}(G_{34}) via their 5-modular reductions. We set

\begin{matrix}f̂_{6} := p_{6}, f̂_{12} := (162 p_{12} − 17 p_{6}^{2})/5^{3}, f̂_{18} := (104976 p_{18} − 911 p_{6}^{3})/5, \\ f̂_{24} := p_{24}, f̂_{30} := (p_{30} − p_{6}^{5})/5. \end{matrix}

Then f̂_{1}, …, f̂_{5} have integral coefficients. Next we let f_{i} be the 5-modular reductions of the f̂_{i}, 1 ≤ i ≤ 5, and f_{42} := P^{1}(f_{18}). The six invariants thus obtained have a nonzero Jacobi-determinant, so they generate the ring of invariants as a polynomial ring.

The group W_{7}(G_{34}) was already treated by Xu [24]: We let v be a root of a reflection of the complex group Ĝ of type G_{34}, and v_{1}, …, v_{126} representatives of the orbit of v modulo scalars as above. Then with the power sums p_{i} defined as in (6.4), Xu shows that the 7-modular reductions of p_{6}, p_{12}, p_{18}, p_{24}, p_{30} and (p_{42} − p_{6}^{7})/7 are an algebraically independent system of generators of the invariants.

With GAP one finds an orbit of length 672 of the group W_{3}(E_{7}) on the underlying vector space. The stabilizer N of a point in this orbit has an orbit of length 180. The stabilizer N_{1} in N of a point from this orbit has order 24, and is not generated by pseudoreflections. More precisely, N_{1} is isomorphic to a direct product of the alternating group A_{4} with the cyclic group of order 2. So W_{3}(E_{7}) does not have polynomial invariants (see also the proof of Theorem 8.5).

Similarly W_{3}(E_{8}) has a point-stabilizer of type W_{3}(E_{7}), which in turn was treated before. The Weyl group of type E_{8} contains the Weyl group of type H_{4} as a subgroup. Modulo 5 this subgroup is the stabilizer of a vector.

There exists a second vector in the underlying space such that the pointwise stabilizer of the 2-dimensional subspace generated by the two vectors has order 5, so in particular is not generated by pseudoreflections. This deals with the cases with nonpolynomial invariants.

On the other hand, the groups W_{5}(E_{6}), W_{5}(E_{7}), W_{7}(E_{7}) and W_{7}(E_{8}) have polynomial ring of invariants by a Theorem of Demazure [8, Thm. 1 and Prop. 8]. □

By the isomorphisms listed in Table 6.2 and the restrictions made in Theorem 1.4 we only have to consider the following exceptional group generated by homologies of order > 2:

Proposition 6.5. The group W_{5}(G_{32}) does not have a polynomial ring of invariants.

Proof. The group W_{5}(G_{32}) can be generated by the reflections

[ \begin{matrix}ζ^{8} & −ζ^{8} & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ], [ \begin{matrix}1 & 0 & 0 & 0 \\ 1 & ζ^{8} & −ζ^{8} & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ], [ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 1 & ζ^{8} & −ζ^{8} \\ 0 & 0 & 0 & 1 \end{matrix} ], [ \begin{matrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 1 & ζ^{8} \end{matrix} ],

with ζ a generator of 𝔽_{25}^{×}. The vector (ζ^{15}, ζ^{2}, ζ^{2}, ζ^{2}) ∈ 𝔽_{25}^{4} has stabilizer of order 5; in particular it is not generated by pseudoreflections. □

Proposition 6.6. Among the exceptional groups in (d)–(f) of Theorem 1.5 only SL_{2}(5) and 3^{.}A_{6} have a polynomial ring of invariants.

Proof. The group SL_{2}(5) < SL_{2}(9) can be generated by the elements

[ \begin{matrix}1 & 0 \\ ζ^{2} & 1 \end{matrix} ] , [ \begin{matrix}1 & 1 \\ 0 & 1 \end{matrix} ] ∈ SL_{2}(9),

where ζ denotes a generator of 𝔽_{9}^{×}. With the INVAR package one finds the invariants

f_{10} := x_{1}^{9}x_{2} − x_{1}x_{2}^{9}, f_{12} := x_{1}^{12} + x_{1}^{10}x_{2}^{2} − x_{1}^{6}x_{2}^{6} + x_{1}^{2}x_{2}^{10} − x_{2}^{12},

and their Jacobian determinant is nonzero.

The group 3^{.}A_{6} < SL_{3}(4) can be generated by

[ \begin{matrix}0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \end{matrix} ] , [ \begin{matrix}0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{matrix} ] , [ \begin{matrix}1 & 0 & 0 \\ ζ & 1 & 0 \\ ζ^{2} & 0 & 1 \end{matrix} ] ∈ SL_{3}(4),

where ζ is a generator of 𝔽_{4}^{×}. One finds an invariant

f_{6} := x_{1}^{4}x_{2}x_{3} + x_{1}x_{2}^{4}x_{3} + x_{1}x_{2}x_{3}^{4} + x_{1}^{2}x_{2}^{2}x_{3}^{2}

of degree 6, and then f_{12} := P^{4}(f_{6})/f_{6} and f_{15} := P^{5}(f_{6})/f_{6} yield two further algebraically independent invariants.

The group 3^{.}U_{4}(3).2_{2} = W_{2}(G_{34}) < SU_{6}(2) can be generated by the permutation matrices corresponding to the transpositions (1, 2), (2, 3), (3, 4), (4, 5) together with

[ \begin{matrix}0 & ζ & 0 & 0 & 0 & 0 \\ ζ^{2} & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 \end{matrix} ] , [ \begin{matrix}0 & ζ & ζ & ζ & ζ & ζ \\ ζ^{2} & 0 & 1 & 1 & 1 & 1 \\ ζ^{2} & 1 & 0 & 1 & 1 & 1 \\ ζ^{2} & 1 & 1 & 0 & 1 & 1 \\ ζ^{2} & 1 & 1 & 1 & 0 & 1 \\ ζ^{2} & 1 & 1 & 1 & 1 & 0 \end{matrix} ] ∈ GU_{6}(2),

where ζ now denotes a third root of unity. The fixator of the subspace ⟨e_{1}+e_{2}, e_{3}+ e_{4}, e_{5} + e_{6}⟩ has order 32. With INVAR it can be checked that its ring of invariants is not polynomial. So by Theorem 2.1 the ring of invariants of 3^{.}U_{4}(3).2_{2} is also not polynomial. □

7. Proof of the Main Theorem

Let now G ≤ GL(V) be an arbitrary finite irreducible linear group generated by pseudoreflections. If dim(V) = 1 then S(V)^{G} is polynomial. In dimension 2 there is the following result [15, Thm. 5.1]:

Proposition 7.1 (Nakajima). Let G ≤ GL(V), dim(V) = 2, be generated by pseudoreflections. Then S(V)^{G} is polynomial.

In fact, Nakajima proves this assertion under the assumption that the characteristic of the ground field is not 2, and that G contains no transvections if the ground field has characteristic 3. But by looking at his proof, it becomes clear that his argument is valid in all cases if one knows that all irreducible 2-dimensional transvection groups have polynomial ring of invariants. The only nonobvious case was treated in Proposition 6.6.

So we may assume that dim(V) ≥ 3. Furthermore, if G is imprimitive then by [15, Thm. 2.4] the ring S(V)^{G} is again polynomial.

We can now state the result for irreducible reflection groups (recall that by our convention R^{+}O_{3}(q) is the subgroup of GO_{3}(q) whose reflections have centralizer order 2(q − 1)):

Theorem 7.2. Let V be a vector space of dimension n over a field of characteristic p and G ≤ GL(V) a finite irreducible group generated by pseudoreflections. Then S(V)^{G} is polynomial if and only if one of the following holds:

(a) n = 2,

(b) G acts imprimitively,

(c) SL_{n}(q) ≤ G ≤ GL_{n}(q) where p|q,

(d) n = 3, G = GU_{3}(q) where p|q,

(e) n = 3, G = GO_{3}(q) or G = R^{+}O_{3}(q), where 2 ≠ p|q,

(f) n = 4, G = GO_{4}^{−}(q) where p|q,

(g) n ≥ 3, G = 𝔖_{n+1}, p |̸(n + 1), or

(h) (G, n, p) is one of the exceptional groups listed in the lower part of Table 7.3.

7.3. The irreducible primitive groups in dimension ≥3 with polynomial ring of invariants

G  dim  char  degrees  r(G)
SL_{n}(q) ≤ G ≤^{m} GL_{n}(q) n  p|q \frac{q^{n}−1}{m} , q^{n} − q, …, q^{n} − q^{n−1}
GU_{3}(q)  3  p|q  q + 1, q^{3} + 1, q^{3}(q^{2} − 1) q^{5} + q − 1
GO_{3}(q)  3 2 ≠ p|q  2, q + 1, q(q − 1)  q^{2}
R^{+}O_{3}(q)  3 2 ≠ p|q  2, q + 1, q(q − 1)/2 q(q + 1)/2
GO_{4}^{−}(q)  4  p|q  2, q + 1, q^{2} + 1, q^{2}(q − 1) q(q^{2} + 1)
𝔖_{n+1}  n p |̸n + 1  2, 3, …, n + 1 n(n + 1)/2
G_{23}  3  ≠ 2  2, 6, 10  15
G_{24}  3  ≠ 2, 7  4, 6, 14  21
G_{25}  3  ≠ 2, 3  6, 9, 12  24
G_{26}  3  ≠ 2, 3  6, 12, 18  33
3^{.}A_{6}  3  2  6, 12, 15  45
G_{27}  3  ≠ 2, 3  6, 12, 30  45
EJ_{3}(5)  3  5  6, 42, 60  105
G_{28}  4  ≠ 2, 3  2, 6, 8, 12  24
G_{29}  4  ≠ 2  4, 8, 12, 20  40
G_{30}  4 ≠ 2, 3, 5  2, 12, 20, 30  60
G_{31}  4  ≠ 2, 3  8, 12, 20, 24  60
J_{4}(4)  4  3  4, 20, 28, 72  120
G_{32}  4 ≠ 2, 3, 5  12, 18, 24, 30  80
G_{33}  5  ≠ 2, 3  4, 6, 10, 12, 18  45
G_{34}  6  ≠ 2, 3  6, 12, 18, 24, 30, 42  126
G_{35}  6  ≠ 2, 3  2, 5, 6, 8, 9, 12  36
G_{36}  7  ≠ 2, 3  2, 6, 8, 10, 12, 14, 18  63
G_{37}  8 ≠ 2, 3, 5  2, 8, 12, 14, 18, 20, 24, 30  120

Proof. By Theorem 1.6, Proposition 7.1 and [15, Thm. 2.4] it is sufficient to consider the groups listed in Theorems 1.3–1.5. For SL_{n}(q) ≤ G ≤ GL_{n}(q) the assertion is the theorem of Dickson [9]. The unitary and symplectic groups were dealt with in Propositions 3.1–3.4. The orthogonal groups were discussed in Propositions 4.1 and 4.3. The case of symmetric groups was settled in Proposition 5.1. Finally, the exceptional groups were considered in Propositions 6.1–6.6. □

Proof of the Main Theorem. Inspection of the proofs of the results used above shows that for any group G occurring in Theorems 1.3–1.5 we either verified that S(V)^{G} is polynomial, or that there exists a subspace 0 ≠ W < V such that the fixator in G of W does not have a polynomial ring of invariants. By Theorem 2.1 this achieves the proof of the Main Theorem. □

Our proof also yields the following characterization of irreducible groups with a polynomial ring of invariants and not containing transvections:

Theorem 7.4. Let V be a finite-dimensional vector space, G ≤ GL(V) a finite irreducible linear group containing no transvections. Then S(V)^{G} is polynomial if and only if the pointwise stabilizer in G of any subspace of V is generated by pseudoreflections.

8. Further results

The case of dimension 2 It is not hard to determine the irreducible primitive pseudoreflection groups G in dimension 2, using for example the well-known classification of subgroups of GL_{n}(q). In fact, if p divides the order of G, then G necessarily contains transvections, so the normal subgroup H generated by the transvections is one of the irreducible groups in Theorem 1.5 of Kantor. For imprimitive groups H in dimension 2 generated by transvections, the normalizer is again imprimitive, as can be seen from Lemma 1.1 using the remarks preceding Theorem 1.6. Since the symplectic and the special unitary group are both isomorphic to the special linear group in dimension 2 and the orthogonal 2-dimensional groups are imprimitive, either H = SL_{2}(q) where p|q or (n, p) = (2, 3), H = SL_{2}(5). More precisely we have:

Theorem 8.1. Let G be a 2-dimensional irreducible and primitive pseudoreflection group. Then G occurs in Table 8.2, it has polynomial ring of invariants and the degrees are as stated in the table.

Proof. If |G| is prime to the characteristic p of k, then G is the p-modular reduction of a complex primitive pseudoreflection group, hence of one of the groups G_{4}– G_{22}. For these, the degrees are well-known (see for example [1, Table 7.1]). If on the other hand G normalizes a transvection group H, then, by the above, either H = SL_{2}(q) or H = SL_{2}(5) and p = 3. So G is a subgroup of the normalizer N of H in GL_{2}(\overline{𝔽}_{q}). In the case H = SL_{2}(q) this normalizer is generated by GL_{2}(q) together with Z(GL_{2}(\overline{𝔽}_{q})). To determine the pseudoreflection subgroups inside the normalizer, we note that given a diagonalizable element of N, there exists a scalar matrix over some extension field such that the product is a pseudoreflection. On the other hand, by the structure of N, given a homology in N, there exists a scalar matrix such that the product is contained in GL_{2}(q). It follows that all possible G are subgroups of H̃ := ⟨GL_{2}(q), Z(GL_{2}(q^{2}))⟩. Now H̃ is generated over H by the two elements

σ_{1} := [ \begin{matrix}ζ & 0 \\ 0 & 1 \end{matrix} ] , σ_{2} := [ \begin{matrix}ξ & 0 \\ 0 & ξ^{−q} \end{matrix} ] ,

where ζ, ξ denote primitive (q − 1)-st respectively (q^{2} − 1)-st roots of unity in 𝔽_{q^{2}}: Clearly both elements normalize H and σ_{2}^{q+1} ∈ H. The fundamental invariants of H are given by

\begin{aligned}f_{1} & := x_{1}x_{2}(x_{1}^{q−1} − x_{2}^{q−1}), \\ f_{2} & := x_{1}^{q^{2}−q} + x_{2}^{q^{2}−q} + ∑_{i=1}^{q−1} \binom{q − 1}{i}(−1)^{i}x_{1}^{i(q−1)}x_{2}^{(q−i)(q−1)}. \end{aligned}

8.2. The irreducible primitive groups in dimension 2 with polynomial ring of invariants

char  degrees r(G)
SL_{2}(q) ◁ G  p|q \frac{q^{2}−1}{m_{1}} , \frac{q(q^{2}−1)}{m_{2}}  ^{(1)} m_{1}|(q−1), m_{2}|(q+1)
SL_{2}(5) ◁ G  3  \frac{20}{m_{1}} , \frac{60}{m_{2}}  ^{(2)}  m_{1}|2, m_{2}|5
G_{4}  ≠ 2, 3  4, 6  8
G_{5}  ≠ 2, 3  6, 12  16
G_{6}  ≠ 2, 3  4, 12  14
G_{7}  ≠ 2, 3  12, 12  22
G_{8}  ≠ 2, 3  8, 12  18
G_{9}  ≠ 2, 3  8, 24  30
G_{10}  ≠ 2, 3  12, 24  34
G_{11}  ≠ 2, 3  24, 24  46
G_{12}  ≠ 2, 3  6, 8  12
G_{13}  ≠ 2, 3  8, 12  18
G_{14}  ≠ 2, 3  6, 24  28
G_{15}  ≠ 2, 3  12, 24  34
G_{16}  ≠ 2, 3, 5  20, 30  48
G_{17}  ≠ 2, 3, 5  20, 60  78
G_{18}  ≠ 2, 3, 5  30, 60  88
G_{19}  ≠ 2, 3, 5  60, 60 118
G_{20}  ≠ 2, 3, 5  12, 30  40
G_{21}  ≠ 2, 3, 5  12, 60  70
G_{22}  ≠ 2, 3, 5  12, 20  30

(1) : \frac{q^{3}−q^{2}}{m_{1}} + \frac{q^{3}−q^{2}}{m_{2}} − q^{2} − 1, ^{(2)} : \frac{60}{m_{1}} + \frac{60}{m_{2}} − 22

On these, σ_{1} and σ_{2} act as

σ_{1}f_{1} = ζf_{1}, σ_{1}f_{2} = f_{2}, σ_{2}f_{1} = f_{1}, σ_{2}f_{2} = ξ^{1−q}f_{2}.

If for m_{1}|(q − 1) and m_{2}|(q + 1) we let

G(m_{1}, m_{2}) := ⟨SL_{2}(q), σ_{1}^{m_{1}}, σ_{2}^{m_{2}}⟩ ≤ H̃,

then \{f1^{\frac{q−1}{m_{1}}} , f2^{\frac{q+1}{m_{2}}} \} is a system of fundamental invariants of G(m_{1}, m_{2}) with degrees (q^{2} − 1)/m_{1} and q(q^{2} − 1)/m_{2}. On the other hand, an intermediate group H ≤ G ≤ H̃ with a polynomial ring of invariants has to be generated over H by suitable powers σ_{1}^{m_{1}}, σ_{2}^{m_{2}}.

In characteristic 3, a pseudoreflection group containing H = SL_{2}(5) as a normal subgroup has order at most 10|H|. The ring of invariants of the intermediate groups is again generated by suitable powers of the fundamental invariants f_{10} and f_{12} provided in the proof of Proposition 6.6. □

Some remarks on reducible groups We show how to reduce the case of a general reducible group to the indecomposable case. Note, however, that for groups having a fixed point the assertion of the Main Theorem becomes trivial. In particular this applies to p-groups where p is the characteristic of the underlying field.

Proposition 8.3. Assume that the Main Theorem remains true if “irreducible” is weakened to “indecomposable”. Then it is true for all finite linear groups.

Proof. Indeed, let G ≤ GL(V) be a group generated by pseudoreflections, such that there exists a G-invariant decomposition V = V_{1} ⊕ V_{2}. Clearly any reflection in G either pointwise fixes V_{1} or V_{2}. Let G_{i}, i = 1, 2, be the subgroups of G generated by the pseudoreflections in G trivial on V_{3−i}. Then G_{1}, G_{2} together generate G, the intersection is the trivial group, and G_{1} centralizes G_{2}. Thus by construction G = G_{1} × G_{2} with G_{i} trivial on V_{3−i}. Now by Serre’s Theorem 2.1, if G has polynomial invariants, then the fixator of any subspace W ≤ V has polynomial invariants. Conversely, if the fixator of any subspace 0 ≠ W ≤ V in G has polynomial invariants, this is true in particular for subspaces of the form W_{i} ⊕ V_{3−i} with W_{i} ≤ V_{i}. The fixator of such a space in G is the fixator of W_{i} in G_{i} (direct product with the trivial group). Thus any fixator in G_{i} of a nontrivial subspace of V_{i} has polynomial invariants. By our assumption above and induction on the dimension, this implies that G_{i} has a polynomial ring of invariants. Hence the direct product G_{1} × G_{2} on V_{1} ⊕ V_{2} has a polynomial ring of invariants, as claimed. □

We do not know how to handle the general indecomposable case. Nevertheless, we can give a first step in this direction (Theorem 8.5 below). Namely, the results of Kantor and Wagner cited above are even a bit stronger so as to give some information on reducible groups as well. Indeed, [12, Thm. II] and [23, Thm. 4] show:

Theorem 8.4 (Kantor, Wagner). Let V be a finite dimensional vector space over a finite field of characteristic p and G ≤ GL(V) a linear group generated by a conjugacy class of transvections or reflections. Then there exist a G-invariant decomposition V = V_{1} ⊕V_{2} of V such that G acts trivially on V_{1} and indecomposably on V_{2}.

If moreover O_{p}(G) ≤ G′ ∩ Z(G) then either G is irreducible on V_{2} or one of the following holds (where n = dim(V_{2})):

(a) G = GO_{n}(q) with q even and n ≥ 3 odd, and G stabilizes a line,

(b) G = 𝔖_{n+1}, p|n + 1 and G stabilizes either a line or a hyperplane,

(c) G = 𝔖_{n}, p|n and G stabilizes an incident line and hyperplane,

(d) (n, p) = (5, 2), G = W_{2}(G_{33}) stabilizes either a line or a hyperplane,

(e) (n, p) = (6, 3), G = W_{3}(E_{6}) stabilizes either a line or a hyperplane,

(f) (n, p) = (7, 3), G ≅ R^{+}O_{5}(3) stabilizes an incident line and hyperplane.

Here O_{p}(G) denotes the largest normal p-subgroup, G′ is the commutator subgroup and Z(G) the center of G. The group in case (e) can be realized as a point-stabilizer inside W_{3}(E_{7}). We have the following extension of the Main Theorem:

Theorem 8.5. Let G be an indecomposable group generated by a class of transvections or reflections with O_{p}(G) ≤ G′∩Z(G). Then S(V)^{G} is polynomial if and only if the pointwise stabilizer in G of any nontrivial subspace of V has a polynomial ring of invariants.

For dim(V) ≥ 3 the possible primitive groups appear in Tables 7.3 or 8.6.

8.6. Some indecomposable pseudoreflection groups in dimension ≥3 with polynomial ring of invariants

G  dim  char  degrees  r(G)
𝔖_{n+1}  n ≥ 4 p|n + 1  2, 3, …, n + 1  n(n + 1)/2
𝔖_{n}  n  p|n  1, 2, 3, …, n  n(n − 1)/2
W_{2}(G_{33})  5  2  3, 4, 9, 10, 24  45

Proof. If G is irreducible, this is the Main Theorem. For reducible groups, all possibilities for G are listed in Theorem 8.4. For the 3-dimensional orthogonal groups in even characteristic the invariants can be seen to be polynomial as in the proof of Proposition 4.1. For odd n ≥ 5 the orthogonal group GO_{n}(q) acts on V_{2} modulo the invariant line as the symplectic group Sp_{n−1}(q) for which we exhibited a stabilizer with non-polynomial invariants in Proposition 3.4. For the symmetric groups, the assertion was checked in Corollary 5.2 and in the proof of Proposition 5.1.

If the 2-modular reduction G = W_{2}(G_{33}) stabilizes a line, then the factor module by this subspace is again faithful, since G ≅ SU_{4}(2) is simple, and it is the natural 4-dimensional representation of SU_{4}(2). This does not have polynomial invariants by Proposition 3.1, hence the original representation also does not have polynomial invariants.

On the other hand, the 2-modular reduction G = W_{2}(G_{33}) which stabilizes a hyperplane can be generated by the permutation matrices corresponding to the transpositions (1, 2), (2, 3), (3, 4), together with

[ \begin{matrix}0 & ζ & 0 & 0 & 0 \\ ζ^{2} & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \end{matrix} ] , [ \begin{matrix}0 & ζ & ζ & ζ & ζ \\ ζ^{2} & 0 & 1 & 1 & 1 \\ ζ^{2} & 1 & 0 & 1 & 1 \\ ζ^{2} & 1 & 1 & 0 & 1 \\ 0 & 0 & 0 & 0 & 1 \end{matrix} ] ∈ GU_{5}(2),

where ζ denotes a generator of 𝔽_{4}^{×}. This group has the unitary invariant f_{3} := x_{1}^{3} + x_{2}^{3} + x_{3}^{3} + x_{4}^{3}, an invariant f_{4} := x_{1}^{3}x_{5} + x_{2}^{3}x_{5} + x_{3}^{3}x_{5} + x_{4}^{3}x_{5} + x_{5}^{4} + ζx_{1}x_{2}x_{3}x_{4} of degree 4, further f_{9} := P^{2}(f_{3}), f_{10} := P^{2}(f_{4}) and finally

f_{24} := (f_{33} − f_{3}f_{10}^{3} − f_{3}^{3}f_{4}f_{10}^{2} − f_{3}^{4}f_{4}^{3}f_{9} − f_{3}^{5}f_{4}^{2}f_{10} − f_{3}^{11})/(f_{9} + f_{3}^{3})

with f_{33} := P^{8}(f_{9}) = x_{1}^{33} + x_{2}^{33} + x_{3}^{33} + x_{4}^{33}. Then it can be checked by computer that the Jacobian of {f_{3}, f_{4}, f_{9}, f_{10}, f_{24}} does not vanish. Since the degree product equals the group order, these elements generate the ring of invariants as a polynomial ring.

The 3-modular reduction W_{3}(E_{6}) stabilizing a line does not have polynomial invariants since the group still acts faithfully as R^{+}O_{5}(3) on the factor modulo this line. But this orthogonal group was considered in Proposition 4.1.

The 7-dimensional indecomposable representation of R^{+}O_{5}(3) can be obtained as the stabilizer in W_{3}(E_{7}) of a point in an orbit of length 56. Factoring out the invariant line we obtain a 6-dimensional indecomposable representation of R^{+}O_{5}(3) with an invariant hyperplane. (This can also be constructed as a 3modular reduction W_{3}(E_{6}) of the Weyl group of type E_{6}.) This group has an orbit of length 80 and the stabilizer N of a point in this orbit has an orbit of length 162, and the point stabilizer for a point in this orbit is not generated by pseudoreflections. Hence neither the 6-dimensional nor the 7-dimensional group have polynomial invariants. (This gives a different proof that W_{3}(E_{7}) does not have polynomial invariants.) □

On the number of pseudoreflections Let G ≤ GL(V), V = k^{n}, be a finite group generated by pseudoreflections. Assume moreover that the ring of invariants S(V)^{G} is polynomial, generated by homogeneous invariants of degrees d_{1}, …, d_{n}. This implies in particular that |G| = d_{1} ⋯ d_{n}. For k of characteristic 0 it is well known that the number r(G) of pseudoreflections of G is given by

r(G) = ∑_{i=1}^{n}(d_{i} − 1).

Using [1, 3.12] one can see that this formula remains correct in arbitrary characteristic provided that G does not contain any transvections. If G contains transvections and k = 𝔽_{p} is a prime field, the number of pseudoreflections is described by the conjecture of Carlisle and Kropholler [4], now proved by Benson and Crawley-Boevey (see [1, Thm. 3.13.2]). It is possible to give an extension of this formula to arbitrary finite fields, using the results of Landweber and Stong on the invariants of groups all of whose nonidentity elements are pseudoreflections, and Benson and Crawley-Boevey’s idea of the proof of the Carlisle-Kropholler conjecture. This allows an independent check that the degrees and numbers of pseudoreflections given in the tables do fit together. We omit further details.

Rationality of invariants From the results of the paper it can be observed that for irreducible groups G with S(V)^{G} polynomial, with one family of exceptions, the generators of S(V)^{G} are all defined over the prime fields. The only exceptions are the orthogonal groups GO_{4}^{−}(q) for q an even power of a prime (see Propositions 4.1 and 4.3). (For the groups W_{3}(G_{24}) and W_{2}(G_{33}) the given representations do not have this property, but there exist other representations with invariants defined over 𝔽_{3}, respectively 𝔽_{2}.) This phenomenon finds its explanation in the following:

Proposition 8.7. Let K ≥ k be a finite Galois field extension and G ≤ GL_{n}(K) a matrix group. Then the ring of invariants K[x_{1}, …, x_{n}]^{G} of G is generated by invariants defined over k if σ(G) = G for all σ ∈ Gal(K/k). Here Gal(K/k) acts on GL_{n}(K) by application to matrix entries. The converse is true provided that G is finite.

Proof. Take α ∈ K with K = k(α) and let f ∈ K[x_{1}, …, x_{n}]^{G} be an invariant. Then

(8.8)  f = ∑_{i=0}^{m−1} g_{i}α^{i}  with g_{i} ∈ k[x_{1}, …, x_{n}],

where m = [K : k]. For σ ∈ Gal(K/k) we have

σ(f) = ∑_{i=0}^{m−1} g_{i}σ(α)^{i},

so the g_{i} are K-linear combinations of the σ(f) (Vandermonde matrix). Now if σ(G) = G for all σ ∈ Gal(K/k), then σ(f) ∈ K[x_{1}, …, x_{n}]^{G}, so the g_{i} are also invariants. Thus the first direction of the claim follows from equation (8.8).

Conversely, if K[x_{1}, …, x_{n}]^{G} is generated by invariants defined over k, then it is stable under every σ ∈ Gal(K/k). Hence

\begin{aligned}K[x_{1}, …, x_{n}]^{⟨G,σ(G)⟩} & = K[x_{1}, …, x_{n}]^{G} ∩ σ(K[x_{1}, …, x_{n}]^{G}) \\ & = K[x_{1}, …, x_{n}]^{G}. \end{aligned}

Since G is assumed to be finite, this implies ⟨G, σ(G)⟩ = G , or σ(G) = G. □

Of course, it depends on the choice of basis whether there exists a system of invariants defined over the prime field. For the groups G considered in this paper, in characteristics dividing the group order, the assumption of Proposition 8.7 is easily seen to be satisfied.

Acknowledgement. We thank Jürgen Müller for a careful reading of the preliminary version.

References

[1] D. J. Benson, Polynomial Invariants of Finite Groups, LMS Lecture Notes Series, vol. 190, Cambridge University Press, Cambridge, 1993.

[2] N. Bourbaki, Groupes et Algèbres de Lie, IV, V, VI, Hermann, Paris, 1968.

[3] D. Carlisle and P. H. Kropholler, Rational invariants of certain orthogonal and unitary groups, Bull. London Math. Soc. 24 (1992), 57–60.

[4] D. Carlisle and P.H. Kropholler, Modular invariants of finite symplectic groups, preprint (1992).

[5] C. Chevalley, Invariants of finite groups generated by reflections, Amer. J. Math 77 (1955), 778–782.

[6] A. M. Cohen, Finite complex reflection groups, Ann. Scient. Éc. Norm. Sup. 9 (1976), 379–436.

[7] J. H. Conway, R. T. Curtis, S. P. Norton, R. A. Parker, R. A. Wilson, Atlas of Finite Groups, Clarendon Press, Oxford, 1985.

[8] M. Demazure, Invariants symétriques entiers des groupes de Weyl et torsion, Invent. Math. 21 (1973), 287–301.

[9] L. E. Dickson, A fundamental system of invariants of the general modular linear group with a solution of the form problem, Trans. Amer. Math. Soc. 12 (1911), 75–98.

[10] G.–M. Greuel, G. Pfister, H. Schönemann, SINGULAR, a Computer Algebra System for Singularity Theory, Algebraic Geometry and Commutative Algebra, Universität Kaiserslautern, Germany.

[11] V. Kac, K. Watanabe, Finite linear groups whose ring of invariants is a complete intersection, Bull. Amer. Math. Soc. 6 (1982), 221–223.

[12] W. Kantor, Subgroups of classical groups generated by long root elements, Trans. Amer. Math. Soc. 248 (1979), 347–379.

[13] G. Kemper, A constructive approach to Noether’s problem, Manuscripta Math. 90 (1996), 343–363.

[14] G. Kemper, Calculating invariant rings of finite groups over arbitrary fields, J. Symbolic Computation 21 (1996), 351–366.

[15] H. Nakajima, Invariants of finite groups generated by pseudoreflections in positive characteristic, Tsukuba J. Math. 3 (1979), 109–122.

[16] M. Schönert (editor), GAP — Groups, Algorithms, and Programming, fourth ed., Lehrstuhl D für Mathematik, RWTH Aachen, Germany, 1994.

[17] J.–P. Serre, Groupes finis d’automorphismes d’anneaux locaux réguliers, Colloque d’Algèbre ENSJF, Paris, 1967, pp. 8-01–8-11.

[18] G. C. Shephard and J. A. Todd, Finite unitary reflection groups, Canad. J. Math. 6 (1954), 274–304.

[19] L. Smith, Polynomial Invariants of Finite Groups, AK Peters, Wellesley, 1995.

[20] R. Steinberg, Differential equations invariant under finite reflection groups, Trans. Amer. Math. Soc. 112 (1964), 392–400.

[21] A. Wagner, Groups generated by elations, Abh. Math. Seminar Univ. Hamburg 41 (1974), 190–205.

[22] A. Wagner, Collineation groups generated by homologies of order greater than 2, Geom. Dedicata 7 (1978), 387–398.

[23] A. Wagner, Determination of the finite primitive reflection groups over an arbitrary field of characteristic not 2, I, Geom. Dedicata 9 (1980), 239–253; II, ibid. 10, 183–189; III, ibid. 10, 475–523.

[24] C. Xu, Computing invariant polynomials of p-adic reflection groups, Proc. Symp. Appl. Math., vol. 48, Amer. Math. Soc., Providence, 1994.

[25] A. E. Zalesskiǐ and V. N. Serežkin, Linear groups generated by transvections, Izv. Akad. Nauk SSSR, Ser. Mat. 40 (1976), no. 1, 26–49 (in Russian). English translation: A. E. Zalesskiǐ and V. N. Serežkin, Linear groups generated by transvections, Math. USSR Izv. 10 (1976), 25–46.

[26] A. E. Zalesskiǐ and V. N. Serežkin, Finite linear groups generated by reflections, Izv. Akad. Nauk SSSR, Ser. Mat. 44 (1980), no. 6, 1279–1307 (in Russian). English translation: A. E. Zalesskiǐ and V. N. Serežkin, Finite linear groups generated by reflections, Math. USSR Izv. 17 (1981), 477–503.