Datawindow Issue

Why does the computed field in a datawindow with this formula :

Round( 10*52.66*0.5*0.95, 2 )

will give the result = 250.13 ??

The exact result is 250.135 and Round( 250.135, 2) = 250.14 !

Am I doing something wrong ?

I tested with all PB versions from 6.5.1 to 10.0 Build 4500 (60 trial) and 
the result is the same...




0
Stefan
4/8/2005 8:03:36 PM
sybase.powerbuilder.datawindow 28057 articles. 4 followers. Follow

4 Replies
521 Views

Similar Articles

[PageSpeed] 30

This is an extreme FAQ:
read this: Losing Accuracy - Understanding Floating Point
and Decimal Number Representations
http://www.sybase.com/detail?id=1012599

> Why does the computed field in a datawindow with this
> formula :
>
> Round( 10*52.66*0.5*0.95, 2 )
>
> will give the result = 250.13 ??
>
> The exact result is 250.135 and Round( 250.135, 2) =
> 250.14 !
>
> Am I doing something wrong ?
>
> I tested with all PB versions from 6.5.1 to 10.0 Build
> 4500 (60 trial) and  the result is the same...
>
>
>
>
0
Philip
4/8/2005 8:40:49 PM
Hi Stefan,

It's true!

-- 
Regards
BMartins (Brazil)


"Stefan" <ionitastefan@hotmail.com> escreveu na mensagem
news:4256e394@forums-2-dub...
> Why does the computed field in a datawindow with this formula :
>
> Round( 10*52.66*0.5*0.95, 2 )
>
> will give the result = 250.13 ??
>
> The exact result is 250.135 and Round( 250.135, 2) = 250.14 !
>
> Am I doing something wrong ?
>
> I tested with all PB versions from 6.5.1 to 10.0 Build 4500 (60 trial) and
> the result is the same...
>
>
>
>


0
BMartins
4/8/2005 8:44:20 PM
Right, but this also points out an inconsistency in the calculation of
Round() as a PowerScript function and Round() as a DataWindow
function.  For example, try the following in PowerScript (not DWE)

double r, s
r = double(10)*double(52.66)*double(0.5)*double(0.95)
s = double(10 * 52.66 * 0.5 * 0.95)

r will be 250.13 and s will be 250.14, however, in DWE the calcuation
of the formula for s is 250.13.  So it would appear the DWE promotes
each operand to double FIRST.

If you want precision here, use decimal types.  Of course, that's a
problem in the DataWindow engine since decimal is not supported there
and everything goes to double.  That's slated to change though in
PB11.



  Round(On 8 Apr 2005 13:40:49 -0700, Philip Salgannik wrote:

>This is an extreme FAQ:
>read this: Losing Accuracy - Understanding Floating Point
>and Decimal Number Representations
>http://www.sybase.com/detail?id=1012599
>
>> Why does the computed field in a datawindow with this
>> formula :
>>
>> Round( 10*52.66*0.5*0.95, 2 )
>>
>> will give the result = 250.13 ??
>>
>> The exact result is 250.135 and Round( 250.135, 2) =
>> 250.14 !
>>
>> Am I doing something wrong ?
>>
>> I tested with all PB versions from 6.5.1 to 10.0 Build
>> 4500 (60 trial) and  the result is the same...
>>
>>
>>
>>

0
Jim
4/8/2005 9:26:25 PM
Exactly.. I noticed there is a difference between the Powerscript Round 
function and the DW Round. The Powerscript Round gives the correct result... 
I'm waiting for PB 11 :-)


"Jim O'Neil [Sybase]" <joneil_at_sybase_dot_com> wrote in message 
news:gusd51dcm08si4pbids56781h9g3rrbfii@4ax.com...
> Right, but this also points out an inconsistency in the calculation of
> Round() as a PowerScript function and Round() as a DataWindow
> function.  For example, try the following in PowerScript (not DWE)
>
> double r, s
> r = double(10)*double(52.66)*double(0.5)*double(0.95)
> s = double(10 * 52.66 * 0.5 * 0.95)
>
> r will be 250.13 and s will be 250.14, however, in DWE the calcuation
> of the formula for s is 250.13.  So it would appear the DWE promotes
> each operand to double FIRST.
>
> If you want precision here, use decimal types.  Of course, that's a
> problem in the DataWindow engine since decimal is not supported there
> and everything goes to double.  That's slated to change though in
> PB11.
>
>
>
>  Round(On 8 Apr 2005 13:40:49 -0700, Philip Salgannik wrote:
>
>>This is an extreme FAQ:
>>read this: Losing Accuracy - Understanding Floating Point
>>and Decimal Number Representations
>>http://www.sybase.com/detail?id=1012599
>>
>>> Why does the computed field in a datawindow with this
>>> formula :
>>>
>>> Round( 10*52.66*0.5*0.95, 2 )
>>>
>>> will give the result = 250.13 ??
>>>
>>> The exact result is 250.135 and Round( 250.135, 2) =
>>> 250.14 !
>>>
>>> Am I doing something wrong ?
>>>
>>> I tested with all PB versions from 6.5.1 to 10.0 Build
>>> 4500 (60 trial) and  the result is the same...
>>>
>>>
>>>
>>>
> 


0
Stefan
4/8/2005 10:52:46 PM
Reply:

Similar Artilces:

DataWindow This, DataWindow That...
Blah blah blah You've heard it enough that you don't even want to comment on how many times... The DataWindow. ....but it can't be enough anymore...can it? This from one of our colleagues (and yes my apologies for taking it out of context)... "...PB covers all you need to do that in ONE Tool. This includes Windowprogramming ( meaning the interface to the user: Windows, Sheets, diallougeboxes, etc.), Database operability AND Reporting! Well, I see als well as many other PB users that some of the implementations and features PB offers are improvable. But thi...

Drop Down Datawindow issues in Datawindow
We recently moved from version 8.0.3 9704 to 8.0.3 9977. Since this move we have had nothing but grief with DropDownDatawindow style columns. Basically - we have columns that are dropdowns that the user can enter their values into or, scroll down and select from. We then have code that verifies that the value entered in the column exists in the drop down list (i.e. is the data entered by the user valid). When we get the value from the column (using either dot notation or, getitem functions) PB is giving us the display value for the drop down NOT the code value. SOMETIMES whe...

How to share the datawindow to inner datawindow of nested datawindow
How to share the datawindow to inner datawindow of nested datawindow. Ex -------- dw_1 is normal datawindow dw_nes is nested datawindow, both are placed one window dw_nes contain dw_child datawindow Question ---------------- i want to share dw_1 and dw_child. I have tried the below code, getting error dw_1.sharedata( dw_nes.object.dw_child) It's the dot notation. You will need to dw_nes.GetChild("dw_child", ldwc) where ldwc is a _local_ datawindowchild. <kzganesan@gmail.com> wrote in message news:4c64c402-bac5-4ee0-83aa-4ac08ffe43e5@r66g2000hsg.go...

Is there any issue in creating dynamic datawindow in Datawindow.Net?
Hi I have created a datawindow object in PB10 and exported in to SRD file. In my dot.net application, I read the file into string and issued the following command to create datawindow dynamically. But it throws an methodfailureexception error as "incorrect syntax at line 18 and column 14. Dim ls_create As String Dim fs As FileStream fs = New FileStream(curFileName, FileMode.Open, FileAccess.Read) 'declaring a FileStream to open the file named file.doc with access mode of reading Dim d As New StreamReader(fs) 'creating a new StreamReader and passing the filestream...

DataWindow.NET : issue using external datawindow
Hello, I created an external datawindow with 3 columns, but I have error messages when I want to use it in my .NET application : When I want to insert rows, I get the message : "InsertRow was called when no DataWindow object was attached." I set in design mode my pbl library and also the datawindow name I want to use. I also tried to set it dynamically in the code, but it changes nothing. I'm sure SQLCA isn't useful for this kind of DataWindow, but I also tried to set it and I get the message "SetTransaction was called when no DataWindow object was att...

DataWindows and datawindow objects
Difference between Datawindow and datawindowobjects? - datawindow is a control on a userobject or a window - datawindowobject (DWObject) is a control within a datawindow, defined in the dataobject of the datawindow. You refer to it as <datawindow-instance>.Object.<datawindow object> HTH, Ben (bede) <Santhoshi> wrote in message news:3f8bee8b.4631.846930886@sybase.com... > Difference between Datawindow and datawindowobjects? Not exactly; DW Control is a "container" that reserves real-estate on the surface of a visual object. DW Obj...

datawindow
Add direct setting and getting capability for the datawindow where clause. This would have to be a string array in case of unions. Return ONLY the where clause(s) in the main select (in case of sub-selects, ignore the where clause or somehow list it as a subselect in the where or a subselect in the select). It would be great if I could just get and set this as a value and have the datawindow automatically change (or add) the where clause as specified. This is for setting the where clause dynamically at run time. Currently have to parse through the datawindow.table.select looking f...

Datawindows with child datawindows
I want to create a datawindow whose only data is three columns that will themselves be drop-down data windows. These child datawindows will be populated in PB script with dw_1.GetChild(). When I create that parent datawindow (New | Datawindow | Freeform), what do I choose in the next dialog box ("Which datasource would you like to use?") External? Thanks. That depends on how you want to populate and use the columns of the main DW. The DDDW is an edit style that assists the user in selecting a value for the column in the main DW. If you want to retrieve the values ...

datawindows in composite datawindow used in datawindow.NET, are not sliding up
Hi, Problem: datawindows in composite datawindow used in "datawindow.NET", are not sliding up , if one dw is not visible or did not retrieve data. Resulting in unwanted spaces. The height of blank white space is equal to the height of the datawindow art design time. When I looked at the generated JavaScript, I noticed there is "div" for the datawindow with height 0.00in which is supposed to be not visible. Tried different options with html/xhtml/xml same result. But slides up ok when exported to pdf(using saveas) on web and in PB10.5 application. Also s...

modifiyin a datawindow in a datawindow
Hi, When I want to modify a datawindow, then it's just dw1.modify() But what do I have to do when I want to modify a datawindow in a datwindow ? I tried : dw1.dw2.modify() but that doesn't work. Anyone can help me out with this ? thanks. Regards, David get the dw via getchild & apply modify In article <4575976f.4dda.1681692777@sybase.com>, David says... > Hi, > When I want to modify a datawindow, then it's just > dw1.modify() > > But what do I have to do when I want to modify a datawindow > in a datwindow ? > I tried : dw...

DataWindow Within DataWindow
How Can I access/modify object/control from a Datawindow within Datawindow. When i access/modify object/control from a Datawindow i used dw_1.modify("name.text = 'new val'") but within another datawindow or child datawindow it doesnt work. Your description is difficult to understand due to the terminology. First, a datawindow is assigned to a control - either a datawindow control or a datastore. Second, the Modify function can only be called within a script. With that as a basis, can you rephrase the question and describe more precisely what you want to do? ...

DataWindow.NET : issue using external datawindow #2
Hello, I created an external datawindow with 3 columns, but I have error messages when I want to use it in my VB.NET application : When I want to insert rows, I get the message : "InsertRow was called when no DataWindow object was attached." I set in design mode my pbl library and also the datawindow name I want to use. I also tried to set it dynamically in the code, but it changes nothing (I can see the DataWindow in design mode). I'm sure SQLCA isn't useful for this kind of DataWindow, but I also tried to set it and I get the message "SetTransaction ...

I will print two datawindow in one page,and I use child datawindow.One master datawindow (dw_master)with two child datawindow(dc1 and dc2)
dw_master("dw_dc1",dc1) dw_master("dw_dc2",dc2) dc1 has a argument (arg1) dc2 has a argument (arg2) Arg1 and Arg2 are send by one column of dw_master. When I use retrieve() function,the datawindow can display normal,but when I print preview or use Zoom or printsetup() ,the childdatawindow dc2 can't display but dc1 normal! Why?? Thanks for your help! I'm having trouble understanding your question. Is this a composite or nested datawindow? On Mon, 6 May 2002 22:49:23 +0800, in powersoft.public.powerbuilder.datawindow spark <fenfei79@hot...

Inserting datawindow to another datawindow
Hello, what is the best (ok, any) method of inserting all visual objects from first datawindow to second datawindow, while keeping the objects from second datawindow on their place? Thank you for your time :-) Jakub Janda The best method would be to go back to design :-)) pbm_thisusuallydoesnothelp:-)) Philip Salgannik "Jakub Janda" <Jakub.Janda@empire.cz> wrote in message news:mRGyciwUAHA.202@forums.sybase.com... > Hello, > > what is the best (ok, any) method of inserting all visual objects > from first datawindow to second datawi...

CrossTab Datawindow and Datawindow plugin
Does anyone know of any problems using the Datawindow plug-in and a crosstab presentation sytle. I'm using web.pb to call my distributed PB server and save the crosstab as a psr file and then return the html pointing to the psr file. The computed fields are getting overlaid but doing it from PowerBuilder works fine. I'm using PB 6.5 Thanks, Bill Anderson ...

Web resources about - Datawindow Issue - sybase.powerbuilder.datawindow

PowerBuilder - Wikipedia, the free encyclopedia
PowerBuilder is an integrated development environment owned by Sybase , a division of SAP . It has been in use since 1991, peaking around 1998 ...

Intelligent Versus Surrogate Keys
Should I use business columns as primary key fields for tables in thedatabase, or generate artificial primary key values? When business-related ...

chevy volt
ProEXR File Description =Attributes= channels (chlist) compression (compression): Zip dataWindow (box2i): [0, 0, 5119, 2475] displayWindow (box2i): ...

Enable - A comprehensive solution for the localization of PowerBuilder applications Untitled Document ...
Enable is localization/ translation tool used by PowerBuilder developers to make their applications multilingual.Enable is a powerful tool that ...

comp.lang.clipper.visual-objects - Google Groups
comp.lang.clipper.visual-objects 1-20 von 33006 Themen werden angezeigt Integrity of a file marcosm...@gmail.com 09:35 Combobox value Lu 08:12 ...

Sybase PowerBuilder tool readied for Microsoft's .Net
Developers can choose Win32 or newer style .Net development

PowerToTheBuilder - PowerBuilder UI Controls
HOME - COMPONENTS - PB UI DESIGNER - DOWNLOAD\PURCHASE - SUPPORT - FAQ - ABOUT Take Control of your UI with PB Ultimate Suite PowerToTheBuilder.com ...

Beautiful iPad Mini 3 Concept Reimagines Appleā€™s Tablet With iPhone 6 Design, Gold Color [Images]
The iPhone 6 may be grabbing all of the recent headlines due to its impending announcement, but spare a thought for those who prefer to stay ...

Java Networking - Real's HowTo
Real's HowTo Custom Search Java Language String and Number Applet AWT Swing Environment IO Javascript interaction JDBC Thread Networking JSP ...

Business Intelligence Products - Database Management, Data Warehousing & Mobility Software - Sybase Inc ...
Sybase offers a variety of Business Intelligence (BI) software like products for database management, data warehousing and mining, data integration ...

Resources last updated: 12/3/2015 4:58:43 AM