That's the vagaries of floating point representation and arithmetic for you.
The reason for the strangeness is that the internal representation is in binary. The sign, mantissa (bit before the 'E') and exponent (bit after the 'E') are split across the 32 bits of a float or the 64 bits of a double.
With only so many bits to work with there is a limit on the precision (number of significant figures, I suppose) that can be achieved. The more bits you have, the better the precision. Think about it. 1.1 in binary is 1.5 in decimal; 1.11 is 1.75; 1.111 is 1.875 and so on. Say you want to represent the decimal number 0.999 in binary: you can never do it exactly as 1.???1 in binary is always going to be 1.???5 in decimal.
Scary huh?!
It's official that computers can't count, and they're running the systems that take care of your bank balance and keep the planes in the air!
Hope that's informative.
Jules