Hey Sam, sorry for the pause on this one, I had to do some checking!
There are two ways to work with the new Direct Message API - realtime, and via REST, which is effectively for data synchronisation. The Account Activity API offers webhooks for Direct Message events in realtime. The REST API complements it to bootstrap recent conversation history and synchronise data, but as you’ve found, it is not as immediately responsive as the Account Activity API.
I realise you’re (very likely ) asking in an end-user client app scenario, and the webhooks option is less attractive than the direct REST access - however, that’s the way these two methods have been designed to operate. Thanks for the excellent question.