I think there are windows in the ambassadorial quarters, actually, though it took me forever to figure it out. In Londo's quarters, at least, there's a huge, curtained-off rectangle behind one of his sofas that I assume is meant to look like a window, since it gives offf light, and there's also that huge window that more closely resembles a painting in the commander's office. We don't ever get to see what the view is really like, though.