Anthropologically, soteriologically, and ecclesiologically, we are (or can be) newly born into households. Only those born into these houses are, properly speaking, discipled and disciplined by the fathers of said houses. Only sons and daughters consistently receive fatherly instruction, training, and correction. The potential blessings should be obvious. But this too means judgment must begin at the house of God - since, a fortiori, it is obvious what judgment befalls those who are not members of the house (1 Peter 4:17-18). Now, we know sons and daughters can rebel against fatherly authority. But such cases, as heartbreaking as they are, does not mean there is no reason for sons and daughters to have been in a position to receive instruction and discipline in the first place. At the very least, it must cause sober reflection upon our own relative births and positions into the various, divinely appointed households. But we must remember that all things work together for good to those who love God and are called according to His purpose, not all without exception.
Or, to put the point another way. In addition to the anthropological, ecclesiological, and soteriological households, we can consider the cosmological household of God of which we are all a part, believer and unbeliever alike. Nature has a way of instructing and disciplining men (Romans 1:18-32), because we are all God's offspring (Acts 17:29), His image, in His macrocosmic house of creation (Genesis 1). This too comes with its own advantages and privileges, for life is good.
However, it might be a bit difficult to understand, from a bird-eye view of the reprobate, how he has been divinely favored by the rain and sunshine God sends and makes (Matthew 5:45). That is, should we view such as grace? Gordon Clark thought we could:
One theologian argues that the gospel is both a savor of life unto life and also a savor of death unto death. To the reprobate the preaching of the gospel is no favor because as it increases their knowledge, it increases their responsibility and condemnation. Better if they had never heard the gospel. One can reply, nonetheless, that in some cases the preaching of the gospel may restrain an evil man from some of his evil ways. Since therefore sins are not all equal, and since some are punished with many stripes, but others with few, the preaching of the gospel results in the lessening of the punishment. Thus preaching would be a small favor, a modicum of grace. (link)I am less sure. We could regard it as an advantage or privilege for anyone who already exists and sins to not be immediately cut off from his sinful life, and to that extent what Clark says above can hold true. It is, likewise, certainly a privilege to have once been enlightened, tasted the heavenly gift, have shared in the Holy Spirit, and have tasted the goodness of the word of God and the powers of the age to come. Perhaps this restrains one from committing as much evil as he would otherwise have done. Yet it seems to me that the privileges of these new births which even reprobates can experience must be balanced against the strong language of, for example, Mark 14:21 - spoken of one who was a disciple of Christ (how much more advantage or privilege towards a sinner can one conceived?) - and that baptism can be unto fearful judgment. To whom much is given, much is expected (Luke 12:47-48). If one tramples underfoot the advantages and privileges of God, it would be better for that man if he had not been born.
Nevertheless, this bird's eye view is not available to us. We can only speak in regards to what we know, and what we know is that one is advantaged or privileged to hear the gospel, for one can only please God and be at peace with Him through justification by faith, and faith can only come by hearing the gospel. We cannot know, in this lifetime, who will hear and reject the gospel, and it is far beyond our purview to presume upon who will or won't heed its call. Much less are we able to determine whether, even if it is rejected, it may yet be beneficial to the extent that those who hear and reject it sin less and are punished less severely as a result. Similarly, we may consider the case of baptism to be a privilege and advantage, if not necessarily a means of grace, even in the case that it leads to negative judgment.
Suppose instead that our infants are only welcome to church meetings to the extent that other unbelievers are. In the first place, we should probably think harder than we do before inviting open unbelievers to what is supposed to be a family communion with and worship of God rather than an evangelistic outreach. What is the goal? Unbelievers should normally find our worship strange and alien because in it, we draw near to God (precisely where they usually don't want to be).
But is this normally how our infants grow to view our assembling to worship? They may, of course, have questions of the worship service (Exodus 12:24-27), but do they normally shrink back as we draw near to God and He to His church? Are not our children participants in corporate worship? If not, I do not see how church can even be considered a foster home to unbaptized children. Rather, it looks as though these children get (have?) to visit a family not their own once a week without participating in the corporate life of it then and there, much less throughout the rest of the week.
On the other hand, if they are participants, fathers discipline sons. Likewise, elders administer discipline to their sons, the visible flock over which they shepherd. If infants are never members of the visible church, then they are never disciplined by elders. You might discipline your son, but just because his friend is around when you do doesn't mean his friend is a "participant" in that discipline. To the extent one wants to speak of someone as a participant in worship and yet without discipline, I think such a person would be an illegitimate child rather than a son (Hebrews 12:7-8).
The same could be said regarding instruction. Fathers are to instruct their children (Ephesians 6:4). The elder instructs his sons, the visible flock, in worship. But if infants are never members of the visible church, then they are never instructed by elders. You might instruct your son, but just because his friend is around when you do doesn't mean his friend is a "participant" in that instruction.
Whether or not your son rebels against your discipline and instruction is a separate question. But it is without question he is a legitimate son of yours because of how you behave towards him. Likewise in regards to God towards His elect sons and the elders towards their sons-flocks.
One final question that bears pastoral reflection: if infants are not treated as sons, how will they come to think about the church? Frankly, as a child, I would have felt awkward and out of place in the presence of my friend being instructed or disciplined by his father. I would be viewing an internal family matter as an outsider. To say in the first place that infants cannot be participants in worship - that they are presumed to be outsiders - then, would naturally seem to lead them to doubt, uncertainty, or anguish. Being cognizant of the manner in which we, as the visible church, treat our children from birth as well as how they grow to perceive themselves in virtue of our treatment is easier said than done, especially if some of us actually view them as looking in from the outside of God's visible household.