It's not about some unnatural talent or luck. The anime talks about this kind of shit a lot, and Akagi's ability could be compared to this to the degree that he exhibits it, but it's revealed plenty of times that it's not something so superstitious as magical hell sand, it's pure ability, that comes from his willingness to risk everything. You might say this is unrealistic, it's ridiculous. But really, it's not. The kind of level Akagi is on is just beyond the comprehension of normal people. It's like trying to understand the experiences of a person with some sort of mental disorder, you can get an idea, but you can never truly experience this disorder. Akagi is like that.
There's something in him, something that just isn't in him that other people have, something that allows him to disregard human concepts like loss and gain, to forego concepts like fear. He is "flawed" as a human being, he could never exist as a normal person. He knows only of taking gamble to new heights, and because he doesn't seem to put any real value in money beyond that value that those around him associate with it, the greatest thing he can recognize as having to risk is his life.