Attack speed is useless at all for this build...
https://us.battle.net/forums/en/d3/topic/20757706310?page=6#post-119
So stop using attack speed rolls on gear! Go for AD or some other useful stuff
To preface set roll recommendations, it is important to note that pet-based builds in Diablo 3 (such as this one) benefit greatly from reaching certain attack speed breakpoints, where the attack animation of the skill itself executes faster than it would at a lower Attack Speed rate. In simple terms, to reach the desired 24-frame breakpoint of attack speed on the
Skeletal Mages, you will require two attack speed rolls on gear and maxed out Attack Speed in your Paragons, for a minimum of 1.61 Attacks per Second in your sheet. This can easily be achieved through your weapon,
Jesseth Skullscythe (since Increased Attack Speed is a natural roll on the item), and a second Attack Speed roll on another suitable piece of gear (gloves or rings), resulting in a minimal loss of other important stats.
Having quoted that from the icy veins guide I will also say that there is other information out there that claims this is not true and that mages have an AS of 2 without a cubed Takers and 3 with a cubed Taskers. So far, I haven't seen any definitive proof one way or the other. This has me looking for actual testing numbers now.
EDIT: I've done some reading of several different articles, research, and testing and I'm inclined to go along with the theory that AS does not, in fact, increase the damage of skeletal mages.