Guys, Kaiji is a psychological anime.
Is it so hard to consider that perhaps those people are not there?
Considering the human nature, does it make sense that they are just there, holding his money for him, while they live their lives in huge debts?
And they are so happy as well!
No my friends, those people are not real, they are likely a hallucination, brought forth by Kaiji's mind that has already been effected by the unbelievable pressure and the unbearable pain.