One of the common features of libertarianism and conservatism is their equal regard for voluntary associations. Indeed, this is one of the main appeals that libertarians make to conservatives, that all the associations in society that conservatives value are perfectly compatible with libertarian theory, provided they are voluntary - and since there's nothing overtly involuntary about institutions like the family, the church, and the community, a conservative society is said to be fully consistent with libertarian politics.
There's no doubting the logic of this argument, but in conversations I've had with libertarians (a group, I will add, that I formerly identified with and with which I still find much commonality) I've noticed a tendency for libertarians to emphasize the "voluntary" over the "association." There are good and obvious reasons for this tendency, but to me it seems to assume a much more contractual society than actually exists.
In How to Be a Conservative, Roger Scruton writes,
We can envisage a society as founded in contract only if we see its members as capable of the free and responsible choice that a contract requires. But only in certain circumstances will human beings develop into rational choosers, capable of undertaking obligations and honouring promises, and oriented towards one another in a posture of responsibility. In the course of acquiring this posture towards others, people acquire obligations of quite another kind - obligations to parents, to family, to place and community, upon all of which they have depended for the nurture without which the human animal cannot develop into the human person. Those obligations are not obligations of justice, such as arise from the free dealings of human adults. The Romans knew them as obligations of piety (pietas), meaning that they stem from the natural gratitude towards what is given, a gratitude that we spontaneously direct to the gods. Today we are reluctant to provide these obligations with such a theological backing, though it is important to see that, for religious believers, unchosen obligations are not only vital to the building from below of a durable social order, but properly owed to God.
So, the person living in society (which is to say, anyone who is not a hermit) acquires duties to that society. It's important to note two features of these duties: they are not legal duties, and they are not voluntary. That is, they are not duties that are enforced by political society, but neither are they duties that the individual freely chooses. Libertarians who are fine with the first feature are likely to bristle at the suggestion of the second, as evidenced by their repeated reminders to me that all social obligations are fine and dandy as long as they're voluntary.
This is a point of distinction between libertarianism and conservatism (admitting the general lack of usefulness of these labels). Conservatives focus on the association, libertarians the voluntary nature of them. It's not necessarily that libertarians reject piety, but they definitely subordinate it to, in Scruton's word, justice. Conservatives, on the other hand, believe that our proper duties apply regardless of whether they are voluntary or not, and to focus only on what is voluntary is to ignore the duties that the individual has to those around him. To insist that man should only accept the responsibilites that he has voluntarily accepted is, the conservative believes, to undermine the duties that are not legal or contractual, but are also not dispensable and irrelevant. Indeed, it is these involuntary duties that form the stability on which society survives and thrives. "Do you see the necessity," Richard Weaver asked,"of accepting duties before you begin to talk of freedoms?"
Most people in modern society - many conservatives and libertarians included - do not understand this, but a society with all freedom and no duties, or only consciously-accepted duties would be fraught with disorder, a description that seems to apply to our current environment.