Of their day together, what do you think Forster means by giving them such a lovely day, then splitting them apart directly afterwards?
That it was doomed! If they'd been able to spend more time together right after it, it might have been that either the issue of physical intimacy had come up sooner in a less Clive-controlled manner, or their intellectual incompatibility would have become more evident sooner.

But it seems that what happens is that the memory of that lovely day cements their relationship for a time.