Well yes, there is enough time in an ideal world, but you must not forget the overhead of processing the messages inside the chip, the processing overhead after each RF event etc. Add to that the fact that your serial transmission is more or less completely unaligned with the RF event.
The RX buffer of the CC2541 is polled asynchronously every time osal_run_system() runs another round. This may not be fast enough, and the buffer may fill up. Apparently, this causes the chip to stop responding, pehaps because it is stuck in some half-way processed incoming message. Granted, it shouldn't freeze, and I'll notify the proper authorities. In the mean time, waiting for command response ensures that at least that message has been read out of the buffer and the space is free.
40ms seems a lot though. Is that the time it takes to get all four responses?
Best regards,
Aslak