This analysis fails to take into account any kind of structural considerations. Normal walls are 8 inches because that is what is necessary to support the structure, leave room for wiring, and provide enough insulation. 4-cube-wide walls would be woefully inadequate for all of these concerns. In addition, there is no obvious way to connect cubes to each other. Even with some kind of strong glue, large flat ceilings would be essentially impossible, so many extra cubes would be needed to not only provide some kind of dome or pyramid structure but also to provide enough support to prevent the walls from collapsing. Cubes do not have a high tolerance for being crushed, so even if it is possible to build a house out of them, you would need to be very careful that the house does not collapse under its own weight. In short, simply replicating the dimensions of a typical house's walls would leave you with a thin shell of cubes that would instantly crumble, despite the type of adhesive used, destroying most of the cubes in the process.
I would also note that a typical world-class global average is closer to 8 seconds than 5.5.