0

I tried

printf "%.3f\n" 2.555500000000000104805053
printf "%.3f\n" 2.555500000000000104805054

the result was

2.555
2.556

same with printf "%.3f\n" 3.555500000000000104805053

But 0.555500..., 1., 4., 5., 6., 7., 8., 9. are correct -> x.556

What is the problem with rounding with printf?

  • MacOS Ventura 13.2
  • The shell is zsh 5.8.1 (x86_64-apple-darwin22.0), but it appears also in bash-3.2
  • In ksh (shell) the output is identical (2.556) with both commands
Peregrino69
  • 2,337
  • 1
  • 15
  • 22
R 9000
  • 167
  • 6
  • Please do document your OS/shell flavours and versions. Under my GNU bash, version 5.1.16 / GNU coreutils 9.1 / Linux-5.4, your two printf produce identical output : 2,556 – MC68020 Feb 27 '23 at 22:51
  • Double precision is only between 15 and 16 digits (53 bits). The error is in the conversion of the input string: the last 9 digits of both the inputs are only twitching the last bit of the double. – Paul_Pedant Feb 27 '23 at 23:05
  • Identical output as above also under bash v 5.1.4(1) (Debian Bullseye) – Peregrino69 Feb 28 '23 at 00:08
  • @MC68020 macOS Ventura 13.2.. the shell is zsh 5.8.1 (x86_64-apple-darwin22.0)... but it appears also in bash-3.2 – R 9000 Feb 28 '23 at 00:38
  • in ksh (shell) the output is identical (2.556) with both commands. – R 9000 Feb 28 '23 at 00:44
  • Please [edit](https://unix.stackexchange.com/posts/737137/edit) the question instead of answering in comments. This way everything relevant is readily available without digging into comment section. I added the new info now, my edit will be visible when another member of the community approves it. – Peregrino69 Feb 28 '23 at 05:23
  • In ksh93, that will depend on how the C processor represents long doubles on the system. See also [Why does $((0.1)) expand to 0.10000000000000001 in zsh?](https://unix.stackexchange.com/q/422122) – Stéphane Chazelas Feb 28 '23 at 09:27
  • You can use `printf %.999g` to see what number the input is interpreted as. In bash, ksh93 or with GNU printf at least (not zsh), you can use `printf %la` for a hexadecimal representation (so gives you the binary number without needing too many digits contrary to decimal) – Stéphane Chazelas Feb 28 '23 at 09:30

0 Answers0