Just updating my own post. Solved it with this setting in the “Customising Blog Page” section:
I do find it odd that this control only seems to control whether or not the comments are counted and that number is displayed. The comments themselves were already displayed.
Anyway, case closed.