I just checked with the original game. On the highest level, penalties start with the 6th colonist, meaning you can have 5 "Tory" colonists with no penalty. The second highest difficulty allowed you 6 working colonists before problems ie: the 7th brought penalties.
Regarding military, you can also give horses to your soldiers to make them dragoons. As well as making them faster and better units, they also give you the advantage of being able to lose twice before you are unarmed. The first loss destroys the horses, the second destroys the muskets. As horses are much easier to produce (just have lots of excess food and a stable) than muskets, you can base your military off replenishing horses and do well.
Regarding military, you can also give horses to your soldiers to make them dragoons. As well as making them faster and better units, they also give you the advantage of being able to lose twice before you are unarmed. The first loss destroys the horses, the second destroys the muskets. As horses are much easier to produce (just have lots of excess food and a stable) than muskets, you can base your military off replenishing horses and do well.
Comment