It seems for values with a fixed range, it is recommended in machine learning to min-max scale them between -1 and 1. What data type should be used for such values? It seems like a float[n] or double are the only options, but that also seems like it would be memory inefficient as a large portion of the available bits would never be used. Is this a meaningful concern in practice?

CodePudding user response：

but that also seems like it would be memory inefficient as a large portion of the available bits would never be used

That's not how floating point works. It might be a concern for a fixed-point representation, but not floating point.

About half of all floating-point representable values have magnitude less than 1. Effectively, you're only sacrificing 1 bit worth of representable information per value by scaling values this way. (You can think of it as sacrificing the most significant bit of the exponent in the representation.)