I see what you mean. But since brush size is attached to the brush preset and not any kind of global setting, I don’t think there’s any way to do that - but I’m certainly not a dev so I’m commenting as a user.
There is a plugin you might want to look at. I don’t use it myself so I can’t comment on how well it works but it might offer a bit of a workaround for you.