To the best of my knowledge (haven't actually tested it in 2.4):
Yes, the 30% damage buff of Mass Confusion works on rift guardians.
As for Hex, you cast the spell and a Shaman appears. It then randomly turns some mobs into chicken (or pigs, if you use the most popular rune "Jinx). As an example (I'm not sure about the exact numbers), in a group of 20 mobs, I'd say the Shaman turns about 5 mobs into pigs for about 4 seconds, and during this time those mobs take 30% more damage. Then the Shaman will turn some other mobs into pigs, and so on. The 12 seconds is only the time the Shaman is around; if you re-cast the spell, the first Shaman is unsummoned and a new one appears, which also means the pigs of the previous Shaman break. Therefore, you should not "spam" Hex but cast it only every 12 seconds. Rift guardians obviously can't be turned into pigs and therefore are not affected by Hex.