From the 14th Century on, Swiss pikemen were famous as great soldiers and much in demand as mercenaries. But wait, why would King Bob hire expensive Swiss mercenaries when he could simply pass out pikes to some of the peasants? And if they were so tough, why bother with all those other weapons?
One reason is that a formation of pikemen is defensive. Locked together in close ranks and carrying 20-foot spears, the pikemen could advance only slowly, and retreating was pretty much out of the question. If King Joe hires Genoese crossbowmen, he can probably decimate King Bob’s Swiss pikemen before they even get close.
Pikemen were most useful in places like Switzerland, where they could easily defend narrow mountain passes. In common with the Greek phalanx, they were well protected to the front, but weak against attacks from the side or rear.
Starting around 1500, pikemen were organized in a formation called the Spanish Square. As the name suggests, this was a square formation, with pikes pointing in every direction. What gave the Spanish Square its punch was musketeers who occupied the corners of the square and blew away enemies who got too close.
When King Gustavus Adolfus of Sweden landed in Germany in 1630, as part of the 30 Years War, he took a look at the massive Spanish Squares of the Catholic armies and devised a new strategy: He stretched his pike formation out in long lines so they could outflank the Catholics, who were bunched together in squares.
And he moved up light artillery to the front ranks, where it could make big holes in the enemy formations. This was hugely successful. With the improvement of gunpowder weapons during the 17th Century, pikes became less and less effective and were gradually replaced with muskets.
But wait, about that first question: If pikes were so great, why didn’t everyone use them? One reason was that Swiss men had been trained since childhood in the use of pikes.
If you had ever tried it (as I assume you have not), killing an armored man with a 20-foot spear takes some practice. And there is also the difficult business of getting your men together in a tight formation, training them to move together as a unit and support each other in battle. A bunch of conscripted peasants would have a hard time of it.
And did you really want to give pikes to your peasants? This was the Middle Ages. Peasants were often treated badly, and they resented it.
And they greatly outnumbered the lord and his knights. Arming the peasants was an invitation to revolt (which sometimes happened anyway). It was much safer to hire reliable Swiss mercenaries, and if they practiced democracy in their cantons up in the mountains, who cared?
Most Medieval armies weren’t professional soldiers.
When the move “Waterloo” was filmed in 1970, the extras playing British soldiers had to do a scene where they repel Napoleon’s cavalry charge. Despite the charging cavalry stopping well short of their formation, the extras kept breaking rank and fleeing. Mind that these extras were mostly Soviet Army soldiers.
Now if a bunch of rough modern soldiers, admittedly conscripts for the most part, are terrified enough to break rank by a cavalry charge that they know is not real and will stop well before actually reaching them, imagine how terrified a bunch of poorly-trained medieval peasants or town militiamen would have been when they saw a mass of these guys barreling towards them, hell-bent on murder, a wall of pointy lances couched and ready to impale, their giant destrier horses specially trained to kick and trample infantry.
On top of that, knights, especially knights of military orders like the Templars or Teutonic Knights, had the reputation of being the best of the best, and that reputation was well-earned. Ask any modern grunt how he feels about having to fight enemy special forces.
In short, heavy cavalry charges usually worked because of the sheer terror they instilled. It was essentially a game of chicken – if the infantrymen chickened out first and broke rank, they turned into very dead infantrymen on short notice.
However, every now and then the infantrymen did not chicken out, and that is usually when the knights lost. Horses tend to have more common sense than their masters and aren’t especially excited at the idea of charging headlong into a wall of pointy sticks with lots of angry men behind them.
If that wall held up, the knights were in trouble – their horses would turn away, get rammed into by the horses and men behind them, throw their riders and quite possibly cause a deadly pile-up.
Professionalization of militaries along with the proliferation of firearms was what brought about the end of the dominance of knights on the battlefield. Knights increasingly found themselves facing highly-trained and disciplined armies of professional mercenaries who fought for a living and wouldn’t break and scatter like the earlier peasant levies and town watches.
Moreover, with the spread of firearms, even a peasant with two weeks of training could now kill a knight with a lifetime of training with insulting ease.
So medieval generals who had to deal with enemy knights but had few of their own did exactly what the OP suggested – they invested in training highly-disciplined professional pikemen, or hired some from abroad.
The Swiss were especially famous for their pikemen, it being something of a business for them to train and hire out companies of them. Later they were eclipsed by German landsknecht mercenaries.
