I don't think you realize how the Yakuza fuck with people in real life.
First, the only people that take loans from Yakuza are gamblers, NEET, Freeters, and Salarymen with dead end jobs that make wages barely above part time.
Regular banks refuse to give you loans because you don't have enough collateral or a high enough income and so Yakuza are your only resort, and they take full advantage of this.
They're seemingly nice in that you can borrow practically limitless amounts at first, but then they hit you with absurd interest rates and come to your door and threaten you until you pay up, usually in the realm of a thousand bucks a month, leaving you with barely enough money to pay the rent, utilities, and eat.
If you don't pay then your interest rates go up again and the Yakuza become even more threatening and violent before.
This generally continues every month for 10, 15, even 20 years depending on how extreme your debt is. And don't even think about skipping out on town to escape, even if you do disappear then your family and whoever else was stupid enough to cosign for you will be forced to shoulder your debt.
There is zero opportunity for these to invest, and it's very easy to go into debt when moving into a new apartment can have a security deposit requiring up to 6 months rent in advance.
Kaiji is essentially the same as a blue collar worker with a gambling addiction that's in debt to the local bookies, investing is simply not an option and it's not like a NEET like him could ever get a decent full-time job. Even if he did that doesn't mean he'd be able to quickly and easily erase his debt considering how poorly starting salarymen get paid, especially ones with no education and no real potential for advancement.