precision on op:numeric-divide
"Damien Fisher" <[email protected]>
| Newsgroups | gmane.comp.web.query-languages |
|---|---|
| Message-ID | <[email protected]> |
op:numeric-divide has a special case when the two inputs are of type xs:integer, and in this case returns an xs:decimal. What, then, is the answer to op:numeric-divide(1, 3)? Is it 0.3? 0.33? 0.333? In other words, to what precision should irrational answers be returned? And should rational answers always be returned exactly, even if that requires an enormous amount of precision?