In language modeling, it is nearly always assumed that documents are generated by sampling from a multinomial distribution. Many formal methods of estimation exist for the multinomial case. In this paper, we reexamine language models based on a multiple-Bernoulli distribution. This assumption has been explored in the past, but has never been formalized. Here, we present how smoothed language models can be estimated using this assumption from a formal, Bayesian standpoint.