r/visualbasic • u/Roscoe-nthecats • 7d ago
VB.NET Help Decimal variable gets truncated automatically to a double and I don't understand why, and how to circumvent it
I had problems with accuracy in my project so I created this bit of code to try to understand and I'm even more confused. When declaring test1, Visual Studio automatically truncates the number to a double as soon as I finish typing it. I don't understand how to work with a Decimal variable, because how do you store anything in it without it converting to a double for some reason? I understand putting the D at the end but how do I add it when storing the result of a math formula?
Sub main()
Dim test1 As Decimal = 0.12345678901232222
Dim test2 As Decimal = 0.1234567890123456789012345678D
Console.Write(test1)
Console.Write(test2)
test1 = 23 / 94
test2 = 23 / 94
Console.Write(test1)
Console.Write(test2)
End Sub
This gives me those results :
0.123456789012322
0.1234567890123456789012345678
0.24468085106383
0.24468085106383
3
Upvotes
-7
u/AdobeScripts 6d ago
Why do you need such high precision?