magician dd worthless?

[Mizaro Moonfire]

Have a level 14 magician. Outside of using my level 1 damage spell all I do is let my pets kill everything for me. Although this is effective I find it quite boring. So far I've had no use of any other damage in spells as they bring Aggro on me. Is this something that will last throughout the game? Will I pretty much have no use for my damaging spells while leveling?