Government is "in" marriage because marriage is a contract with respect to property; and secondarily because marriage is seen as a privileged state: married people are considered more mainstream socially acceptable than unmarried people, and governments like to keep track of who's acceptable and who isn't.
When I say "government out of marriage," I mean stuff like:
(a) any adults ought to be able to write a contract with any other adults specifying how they will behave toward each other with respect to property, association, inheritance, and so on; and governments shouldn't be able to withhold such a contract from specific groups of people
(b) while many people will want a standard contract with many of the provisions that marriage currently has, governments should not be able to retroactively and unilaterally change that contract, the way they currently can with marriage
(c) children's and parents' rights should not be influenced by legal marriage or lack thereof; all competent adults who want to should be able to adopt and raise children
(d) rights of association and inheritance should not be influenced by legal marriage or lack thereof. For example: people should be able to visit their associated people in hospitals and make medical decisions for their associated people and inherit from their associated people without marriages or blood relationships trumping their rights. People should be able to assign their government benefits to anyone they wish and not just spouses. People should not lose or gain benefits as a result of marriage or lack thereof (e.g., health insurance, social security benefits). No one should be expected to provide medical services for chronically ill family members without compensation.