[Sirken]

book 1 chapter 20 (Eddard), only says this about Varys -

the councilor Ned liked least, the eunuch Varys, accosted him the moment he entered. "Lord Stark, i was grievous sad to hear about your troubles on the Kingsroad. We have all been visiting the sept to light candles for Prince Joffrey. I pray for his recovery." His hand left powder stains on Ned's sleeve, and he smelled as foul and sweet as flowers on a grave.

moreover, Varys is never described as having the blue lips that serves as the trade mark sign for the members of the house of the undying (they get this from drinking the shade of the evening). The only people we're told have been drinking it are the Warlocks, Pyat Pree, Daenerys Targaryen (only once), Euron Greyjoy, Victarion Greyjoy (tries a small swallow and spits it out immediately).

and then, Varys despises magic, but especially dark magic. heres his history

Though nothing that Varys says can be completely trusted, he claims to have been born a slave in Lys that was sold and apprenticed to a traveling folly, a troupe of mummers who worked the Free Cities, Oldtown, and occasionally King's Landing, by means of a fat little cog owned by the troupe's master. During their stay in Myr, a man offered a large sum of money for Varys, an offer his master found too tempting to refuse. The man gave Varys a potion that made him powerless to move or speak, but did nothing to dull his senses as his manhood was cut off at the stem, then burnt in a brazier in a Blood Magic ritual. Varys claims to have held a hatred for all things magical ever since.

With the ritual complete, the man had no further use for Varys and tossed him out on the streets. Varys asked the man what he should do, to which the sorcerer replied he supposed that he should die. Varys to spite him vowed to live.

By that time the mummers had sailed and Varys was left alone. He resorted to begging, prostitution and thievery and soon was as good a thief as any in Myr, eventually becoming recognized as the best thief in the city until a rival forced him out and he fled to Pentos.

In Pentos, Varys befriended a poor sellsword named Illyrio Mopatis and they started a business where Varys would steal objects from lesser thieves and Illyrio would get the objects back for their original owners for a small fee. Soon, every man and woman in Pentos who ever had valuables stolen from them all knew who to ask to get their valuables back. In addition, the thieves in Pentos soon began to seek Illyrio and Varys out, half to try (and failing) to kill them, the other half to sell them the objects they stole in order to prevent Varys from stealing them back. Varys and Illyrio grew very rich.

Varys came to the realization that information was much more valuable than gold and gems and began training his spy network, then known as "little mice", to acquire the information, letters, ledgers and charts of the wealthy and powerful. The "mice" were orphans purchased by Varys; he chose those that were small, quick, and quiet, and taught them to read and write, as well as the skills required for sneak thievery such as climbing walls and slipping down chimneys, so that they might copy the information Varys desired without disturbing or alarming their targets.

These secrets increased Varys and Illyrio's wealth tenfold, and in time Varys became so infamous that word of his talents reached the ears of King Aerys across the Narrow Sea, who in his growing paranoia no longer trusted his son, his wife, or his Hand.

and then we also know Varys likes disguises,

Varys employs the use of various disguises to go about unnoticed. Upon seeing one of the disguises Tyrion remarks, "A different look, a different smell, a different way of walking ... most men would be deceived".
As a stout man in a heavy brown robe in roughspun - in cracked, mud-caked boots smelling of sweat. His face hidden by a cowl, hands drawn up into voluminous sleeves. At first he speaks in a strange low voice. Ned is astonished to realize it is Lord Varys. Ned notices his usual cloying tones are gone and his voice is a thin and sharp as a whip.

As a begging brother – in filthy patched robes, bare feet crusty with dirt, a bowl hung about his neck on a leather thong where a septon would have worn a crystal. The smell of him would have gagged a rat. He has a mouth full of rotted teeth.

As a woman – plump, soft, matronly, with a round pin moon of a face and heavy dark curls. For one horrid moment Tyrion thinks its Lollys.

As a dungeon turnkey named Rugen

Personally i dont think Varys is a warlock, nor do i think hes a member of the house of the undying. he lacks the blue lips / Shade of the evening connection.