First, the main opposition to slavery in the North was economic, not moral. The working man in the North feared the expansion of slavery not because of moral misgivings, but because they feared the depressive effect of slavery on wages. The expansion of slavery, geographically as well as the perceived future demand from industry is what fueled Northern opposition to slavery and even that was not deemed sufficient to start a war (Remember, the North invaded). Second, the vast majority of those that fought and died for the South never owned slaves. The notion that they fought and died for something in which they had no ability to participate is a bit far fetched. While the high level politics and rhetoric supports such, the grass roots did not and, as with most things, quite often one has to look past the people making the most noise to find the real drivers.