*BEAST works as an extension of the popular phylogenetic software package BEAST. Because of this, you should familiarize yourself with the tutorial on divergence time estimation to learn the basic ins and outs of using this suite of software.
To set up a *BEAST analysis, open BEAUTi and import your alignment files just like you would for a divergence time analysis. Once this is done we need to tell BEAUTi to employ the multispecies coalescent by clicking the check box in the top left corner.
Doing this results in a couple of things. First, the program will take you to the traits panel where you need to assign each of the sequences to one of the species in your analysis. BEAUTi has some nice tools to try to guess the species names from the tip names. Select “guess trait values” to use these.
After assigning each tip to a species, head back to the partitions pane. Notice that the program has made some changes to the way that the model is set up. First, the site models are now unlinked. We would do this anyway for a standard partitioned analysis, as this accounts for the different base frequencies and substitution rates that occurs in each of our genes. We have also unlinked the clock model among the different genes. We do this because now we are looking to estimate separate coalescent histories for each of the gene partitions. Our information about the probability of the species tree largely comes from the timing of coalescent events, which of course vary from gene to gene. Finally, we have also unlinked the tree model for each partition.
We can set up our site models and clock models in a similar way to how we did things in the divergence time analysis. However, there is a difference in the tree priors. We need to set a prior on the distribution of species tree branching times using the birth-death or Yule models (like before). We also need to specify a model on population size along each of the branches in the species tree. Finally, we must tell BEAUTi where each of the genes came from (Autosomal, mitochondrial, etc.).
Under the priors pane, most of the priors should look familiar and can be set in similar ways as we did in the divergence time tutorial. The one type of prior that we have not yet seen is the species.popMean hyperprior on the gamma distributed population sizes. We can leave this set at 1/x for now.
Following this, we can set up markov chain in the MCMC pane and export our file. As with all other bayesian analyses, it is exceedingly important to run the chain several times, assess MCMC performance, and examine the effect of our priors. In practice, the multispecies coalescent model often does not mix as well as simpler analyses, so careful attention to these issues is particularly important.