I think the implication is that B5's human population is big and diverse enough that all those people were there or passing through. Also I don't think they were all meant to be clergy, just followers of those faiths.