Well, that's floating-point arithmetic for you. log(x, y) simply computes
log(x)/log(y) behind the scenes; since both log computations and the
floating-point division can introduce errors, the result will frequently
not be correctly rounded.
I don't really see the benefit of special-casing log(x, 10). In what
circumstances does it matter that log(x, 10) != log10(x)? I could
understand people being upset that log(10**n, 10) doesn't return n
exactly, but that's what log10 is there for.
See also the discussion in issue 3724.