John Brown wrote this to a young pastor:

"I know the vanity of your heart, and that you will feel mortified that your congregation is very small in comparison with those of your brethren around you. But assure yourself on the word of an old man that when you come to give an account of them to the Lord Christ at his judgment seat, you will think you have had enough."

I heard this in a sermon (I think this one) on membership by Mark Dever that I listened to on the way over to Todd's installation service Saturday. In the sermon Dever gives five reasons for church membership: 1) assurance of salvation, 2) evangelization of the unregenerate, 3) exposing false gospels, 4) edification of the church, and 5) for the glory of God.