Rendering the original file to an image sequence, I had no problems but my PC has only eight cores and they were all used at 100% capacity.
My PC is old and slow with DDR2 RAM so it may not be useful for these tests.
For the activity of the hidden group filter layers and animated transform masks, I turned off the visibility of the animated transform masks and played the animation and it took three minutes to regenerate the cache and showed activity on the visible filter layers of the hidden groups.
(Previous times were 1 minute for all hidden group layer masks and filter layers having visibility turned off and nine minutes for the original file.)
So, non-rendered masks and filter layers are processed with bad consequences for the total time needed.
Has anybody tried turning off the visibility of the non-rendered masks and filter layers to see if that fixes the rendering problems, to see if their processing is the only cause of the problem?