First of all, we're talking here about a finite group G and we fix a given representation of G over some field k. In other words, we have a finite dimensional vector space V over k and an action of G on V as linear transformations. If we let R = k[V] denote the symmetric algebra of V, the ring of polynomial functions on V, then there is a natural action of G on R as automorphisms of R and we may say that invariant theory asks for those polynomials which are left fixed by the action of G on R. We commonly suppress mention of the representation and just refer to the group.
There is the beautiful result of Shephard and Todd over the complex numbers, that a ring of invariants of a finite group is again a polynomial ring of invariants if and only if the group is generated by pseudo-reflections. Their proof was later extended and clarified, notably by Chevalley, Serre, and Clark and Ewing and, of course, this question is classical. Perhaps the best known example is that of the symmetric groups acting as permutations of n variables - the ring of polynomials invariant under all permutations of the variables is a polynomial algebra on the n elementary symmetric functions. This representation of the symmetric group is generated by transpositions which are indeed pseudo-reflections. For me, this is what mathematics is all about – we have a beautiful connection between the geometry of the representation and the algebraic structure of the ring of invariants.
The same characterization of a group with a polynomial ring of invariants holds for a non-modular group, that is, a group whose order is not divisible by the characteristic p of the ground field k. For arbitrary groups, we know that a polynomial ring of invariants guarantees the group is generated by pseudo-reflections, but this is not sufficient thanks to examples of Nakajima. Nakajima has also characterized those p-groups with polynomial rings of invariants. The question remains open in general: when does a finite group have a polynomial ring of invariants? This seems like the most important question in modular invariant theory to me.
Most of the time, the non-modular case is just that, a case. Emmy Noether proved in 1916 that the ring of invariants of any representation of any group is generated in degrees less than or equal to the group over fields of characteristic 0. We usually call this Noether’s bound. The problem of Noether's Gap is whether or not this bound also holds for the rings of invariants of non-modular groups. This problem has just recently been solved – that is, the gap has been closed - by Peter Fleischmann and Fogarty working independently. They show that Noether’s bound, the order of the group, is still valid for non-modular groups. It’s a wonderful result.
The situation is much worse for modular groups. David Richman proved in 1990 that certain representations of dimension 2n of the cyclic group of order p required a generator of order n(p-1). In particular, for an arbitrary group, the ring of invariants is not generated in degrees less than or equal to the order of the group. We do know that any ring of invariants of any finite group is finitely generated, thanks to Hilbert, and just recently, Kemper and (independently) Karagueuzian and Symonds have produced universal bounds that apply for all modular groups. However, these bounds are extremely large. We don’t know of any rings of invariants where a minimal generating set requires a generator of degree larger than the dimension of the representation times (the order of the group minus one).
We do know that an n-dimensional representation as a permatation group is generated in degrees less than or equal to the maximum of the order of the group or n(n-1)/2, thanks to Mannfred Goebel.
There are some beautiful structural results in modular invariant theory now. A few years ago, I was able to write “It is possible to criticize this area of mathematics as not having enough theorems. In fairness, though, one ought to add that there aren't yet enough examples.” That is no longer true. For example, it is fairly easy to see that non-modular groups have Cohen-Macaulay (CM) rings of invariants using an argument of Eagon and Hochster dating back to 1964. A recent result of Gregor Kemper shows that if a ring of invariants of a modular group is CM, then the group must be generated by bi-reflections, elements that fix a subspace of co-dimension 2. Again, this is a fantastic result that builds a beautiful link between the geometry of the representation and the corresponding algebraic structure. On the other hand, we seem very far from any understanding a characterization of those modular groups with a CM ring of invariants even for p-groups. We have examples of p-groups whose rings of invariants aren't Cohen-Macaulay, and we have examples of p-groups that have hyper-surfaces as their rings of invariants, and (thanks to Nakajima) we know all p-groups that have polynomial rings of invariants. But where are the examples that have rings of invariants that fall in-between hypersurfaces and CM? There is some experimental evidence for the conjecture that a p-group with a CM ring of invariants, then the ring of invariants is, in fact, a Complete Intersection Algebra.