It is very clear, that the most thing that should change in Unix systems are the graphics. Most Unix systems use Xorg to handle all their graphics. I really think that is a very bad and old solution that should be changed.
- The Past
To make things looking clear, lets compare the architecture of the current implementations :
Even graphics card drivers, they are made for every Xorg release and not for a Linux release.
If we add to this that X releases are not as frequent as the two other layers, we can conclude that the Linux graphics mechanism is the Achilles' heel of all the system.
The future may be a lot better. With the include of the Kernel Mode Setting  in last Linux kernels (from 2.6.29) and specially in the 2.6.31, the kernel start getting his real role of the only thing accessing low level hardware.
The KMS allow less flicker, quicker terminal switching (with Ctrl-Alt-F1 to F7) and more.
Another feature is the ability to run Xserver itself as a normal application without root privileges. And allowing more security.
This start to let the Kernel do his job, is only the start. The Linux kernel 2.6.33 will include more work done to provide more low level handling : a KMS page-flipping ioctl  to optimize more and more time, and simplify management for other systems.
The page-flipping is a necessary thing for other projects like the Wayland Server, which is a lightweight replacement of X (and not a rewrite of it).
Xorg has become very old, new models of graphics and inputs come and its becoming more and more complex to make X supports them all.
Multi-touch input is a main area where X can't provide a good solution for that. Even Multi-pointer-X still doesn't answer all questions.
Wayland can become a solution if we design it having all new today problems in mind. It should be lightweight and easy to develop. After that, libraries like Nokia Qt could be easily ported to it.
The thing that everybody should know is that when a project becomes very old, with code that can't be understood, and huge, then it should be rewritten from scratch.
KDE developers faced many critics when they started rewriting KDE to release KDE4. But in the end, we now after releasing a "clean" desktop environment, we start seeing differences.
The design of Wayland, shouldn't be addressed without proposing a model being a skeleton of all manners of view surfaces and abstraction behind that. A recent paper  collected most of problems and it need to be applied on a real world implementation like Wayland.
- Links :