Encouraged by the Bible, Puritans banished a variety of radicals from the Puritan colonies including Baptists. Among those banished was Roger Williams, the advocate of religious liberty who established the first Baptist church in America. Williams posed a serious challenge to a colonial society in which the Bible was the paramount guide in every aspect of life, both public and private.