You can summon up to 3 friends as a Wraith yourself, but each friend can only be a Wraith once. If a Wraith is defeated, you can revive it as soon as you have a new Soul in your Crypt. Friends that were not summoned as Wraiths can go to your castle and summon Wraiths themselves.
To maintain a Maximum of 50 wraiths, you need to have 50 friends! Each friend that has been summoned as a Wraith has a skull icon on their profile picture at the bottom of the game screen.
You can increase the amount of Wraiths you can summon yourself up to 25 if you purchase Sarcophagi. Go to the Construction menu and choose the Improvements tab.