It's one of a few choices of design. I don't know if you've ever owned a Rubik's Brand 4x4, but it's a completely different mechanism. Same with the Dayan+MF8 mechanism. In those cases, all of the pieces grab onto an inner, rigid spherical contraption. No internal pieces are necessary, besides the core.
Most "modern" designs pretty much adopt the same piece structure of a 5x5. The middle layer edges/centers are simply made smaller such that they don't show from the outside, and the outer layer pieces overlap them... hence "internal". It's a system that was shown to be reliable on the 5x5, so why not adopt it for other puzzles? I think the first cube to use this sort of design was the Eastsheen 4x4. Then later on V-Cube used it for their 6x6 and patented it for their 4x4 (and all their other even number ordered puzzles, of course). The rest is history.
Edit: pretty well ninja'd by bobthegiraffemonkey.