I am about to finish a new visual effect on Unity 7 desktop so that the shaped windows (those that are not rectangular) can have shadows too.
The trick was to extract the window shape using the shape extension (http://www.x.org/releases/X11R7.7/doc/libXext/shapelib.html) and use it to calculate the shadow. That’s a video of my previous experiments: