I do not know all or necessarily any of the answer to that question, but here is how my recent thinking made me work through it (for now).
A loving God wants us to fully realize the joy that a close personal relationship with Him would bring, fueled by our faith and trust in Jesus and our desire to be obedient to the will of God that comes through that faith and trust. A loving God also allows us the freedom to make our own choices. I cannot fully reconcile God's sovereignty with the concept of personal accountability, but I do know we will stand in judgment one day (Hebrews 9:27), the results of which will depend whether we have trusted Christ or not. Lots more can be said about that. Lots more.
But moving on, I see pain and suffering in a fallen world being used to address two things:
- Pride - as sinful beings in a fallen world, we do need to deal with our pride and how it can drive us away from God, away from His will, away from the joyous union with Him.
- Dependence - as sinful beings in a fallen world, we need to come to terms that only dependence on Him will get us through to eternal salvation and security.
Pride cuts against dependence on God and drives us from it. maybe pain and suffering are actually a loving way to draw us to Him.
It may sound cruel, but the temporary pain of a temporal life (even though it does not feel temporary when it is occurring) may be a loving way to draw us to the eternal joy of union woth God.
We are after all sinful beings in need of saving from ourselves.