Floating point rouding in SKILL

[Aby]

Hi,
I am observing some wierd results in SKILL when I try to perform
division.

A - float
B - float

When I do:
A/B
I get:
2.0
But I do:
int(A/B)
I get:
1

And the even more strange thing is, When I call the function which does
this over and over, few times it gives correct result. Has someone
faced this before ?

 

[Aby]

Thanks for the reply. I have had a course on computer arithmetic before
but I don't recollect seeing such inconsistencies. Here is what I am
printing in a loop using the fprintf( "%f" varName):

Iteration #1
coreWidth = 3.000000
netSpace = 1.500000
ratio (i.e. coreWidth/netSpace) = 2.000000
int(ratio) = 2 ----> Correct

Iteration #1
coreWidth = 3.000000
netSpace = 1.500000
ratio (i.e. coreWidth/netSpace) = 2.000000
int(ratio) = 1 ----> Incorrect

How can I get two different results for the exact same inputs in
different iterations?

Thanks

 

[Frank E. Gennari]

On Sat, 14 May 2005 19:42:36 -0700, Aby wrote:

Thanks for the reply. I have had a course on computer arithmetic before
but I don't recollect seeing such inconsistencies. Here is what I am
printing in a loop using the fprintf( "%f" varName):

Iteration #1
coreWidth = 3.000000
netSpace = 1.500000
ratio (i.e. coreWidth/netSpace) = 2.000000
int(ratio) = 2 ----> Correct

Iteration #1
coreWidth = 3.000000
netSpace = 1.500000
ratio (i.e. coreWidth/netSpace) = 2.000000
int(ratio) = 1 ----> Incorrect

How can I get two different results for the exact same inputs in
different iterations?

Thanks

If all you want is to round the floating point number to an integer, then
you can use round() instead of int(), or you could add 0.5 to the number
before calling int() (subtract 0.5 if the number is negative). It's hard
to say what the solution is because I don't know what you want the math to
work out to in other cases such as 3.0/2.0. Even if you want to truncate
the number you probably want to use floor() instead of int().

Frank