strange float behaviour/sensitivity

[hx]

Hello,

I was trying to compare two parameters (float type) in a when
statement when I discovered strange results.
The parameters are coming, one from numeric value, and one is a
parameter of a PCell:
numVal = 0.95
PCParam = 0.95
Since my when loop was not working (when(numVal==PCParam) warn("OK")),
I have tried to display the value of my parameters and here is what I
had : - println(sprintf(nil "numVal : %.20f \n PCParam : %.20f"
numVal PCParam)) -
numVal = 0.94999999999999995559
PCParam = 0.95000000000000006661

Does any one know a way to tune float sensitivity?

HX

 

[Andrew Beckett]

On Thu, 11 Aug 2005 07:10:17 -0500, hxdelecourt@yahoo-dot-fr.no-spam.invalid
(hx) wrote:

Hello,

I was trying to compare two parameters (float type) in a when
statement when I discovered strange results.
The parameters are coming, one from numeric value, and one is a
parameter of a PCell:
numVal = 0.95
PCParam = 0.95
Since my when loop was not working (when(numVal==PCParam) warn("OK")),
I have tried to display the value of my parameters and here is what I
had : - println(sprintf(nil "numVal : %.20f \n PCParam : %.20f"
numVal PCParam)) -
numVal = 0.94999999999999995559
PCParam = 0.95000000000000006661

Does any one know a way to tune float sensitivity?

HX

These are normal rounding errors as seen in many languages (including C). This
has come up a number of times before on this forum - at least I know I've
answered it a few times (it's definitely an FAQ here in customer support).

A good, thorough, discussion on this can be found in this old posting:

http://groups.google.co.uk/group/comp.cad.cadence/browse_thread/thread/6b71e3866556ae3c/ac04cf8839d75173?q=real+equal&rnum=5&hl=en#ac04cf8839d75173

To summarize, never do == on floating point numbers - do something like:

abs(numVal-PCParam)<0.00001

or something like that.

Andrew.

 

[mxn731]

You can use the command below to make it work:

procedure(snapToGrid(val)
prog( (grid result)
grid = 0.05
result = round(val/grid)*grid
return(result)
)
)

modified your when as below:

when(snapToGrid(numVal)==snapToGrid(PCParam) warn("OK"))

Cheers,
--Michael

 

[fogh]

Andrew Beckett wrote:

On Thu, 11 Aug 2005 07:10:17 -0500, hxdelecourt@yahoo-dot-fr.no-spam.invalid
(hx) wrote:


Hello,

I was trying to compare two parameters (float type) in a when
statement when I discovered strange results.
The parameters are coming, one from numeric value, and one is a
parameter of a PCell:
numVal = 0.95
PCParam = 0.95
Since my when loop was not working (when(numVal==PCParam) warn("OK")),
I have tried to display the value of my parameters and here is what I
had : - println(sprintf(nil "numVal : %.20f \n PCParam : %.20f"
numVal PCParam)) -
numVal = 0.94999999999999995559
PCParam = 0.95000000000000006661

Does any one know a way to tune float sensitivity?

HX


These are normal rounding errors as seen in many languages (including C). This
has come up a number of times before on this forum - at least I know I've
answered it a few times (it's definitely an FAQ here in customer support).

A good, thorough, discussion on this can be found in this old posting:

http://groups.google.co.uk/group/comp.cad.cadence/browse_thread/thread/6b71e3866556ae3c/ac04cf8839d75173?q=real+equal&rnum=5&hl=en#ac04cf8839d75173

To summarize, never do == on floating point numbers - do something like:

abs(numVal-PCParam)<0.00001

or something like that.

Andrew,

is seems you have references to articles in c.c.c predating you
working at cadence ? Do you have memories of a former life ?

 

[Andrew Beckett]

On Sat, 13 Aug 2005 23:11:27 -0400, fogh <adff@xs4all.nl> wrote:

A good, thorough, discussion on this can be found in this old posting:

http://groups.google.co.uk/group/comp.cad.cadence/browse_thread/thread/6b71e3866556ae3c/ac04cf8839d75173?q=real+equal&rnum=5&hl=en#ac04cf8839d75173

To summarize, never do == on floating point numbers - do something like:

abs(numVal-PCParam)<0.00001

or something like that.

Andrew,

is seems you have references to articles in c.c.c predating you
working at cadence ? Do you have memories of a former life ?

Yes, I've not completely forgotten my life as an analogue/mixed-signal designer
(i.e. a user of Cadence and other EDA tools) before joining Cadence 10 years
ago. I was a designer (with a software sideline) with several companies for 8-9
years before I moved to the dark side ;-)

As for the reference above, I couldn't find one I'd written (I'm sure I have,
but perhaps because I've answered this question so many times, I've forgotten
exactly where I've answered it!), but I thought this one seemed to cover it
well.

Andrew.