From a surname which was derived from a place name meaning either "willow town" or "town on the River Wylye" in Old English. The river name is itself of Celtic origin, possibly meaning "tricky".