String to decimal

This must have been asked a thousand times but I cannot find the answer.

I have a string that I need as a decimal.  It goes into a bunch of 
decimal place precision that caused rounding errors over the course of 
adding thousands of these numbers together.

sting	ls_number
decimal	ld_number

	ls_number = "42,508.26"
	ld_number = real( ls_number )

	// ld_number is now 42508.2617185

How do I avoid this?
0
Daniel
1/10/2008 8:19:30 PM
sybase.powerbuilder.general 62418 articles. 17 followers. Follow

3 Replies
529 Views

Similar Articles

[PageSpeed] 6

Presuming this value is always 2 fixed decimal points, declare as

decimal{2} ld_number

Or use 

ld_number = Dec(ls_number)

You're jumping between fixed and floating point here, so I'd say this
is pretty much expected behavior.

On 10 Jan 2008 12:19:30 -0800, Daniel Coppersmith
<coppersmith2002_A_T_hotmail_D_O_T_N0SPAM.com> wrote:

>This must have been asked a thousand times but I cannot find the answer.
>
>I have a string that I need as a decimal.  It goes into a bunch of 
>decimal place precision that caused rounding errors over the course of 
>adding thousands of these numbers together.
>
>sting	ls_number
>decimal	ld_number
>
>	ls_number = "42,508.26"
>	ld_number = real( ls_number )
>
>	// ld_number is now 42508.2617185
>
>How do I avoid this?
0
Jim
1/10/2008 8:28:39 PM
For a discussion of accuracy of number representations see 
http://www.sybase.com/detail?id=1012599

"Jim O'Neil [Sybase]" <joneil@sybase.com> wrote in message 
news:9qvco3pk6dovia75vra85q7b3e3ksm1ij7@4ax.com...
> Presuming this value is always 2 fixed decimal points, declare as
>
> decimal{2} ld_number
>
> Or use
>
> ld_number = Dec(ls_number)
>
> You're jumping between fixed and floating point here, so I'd say this
> is pretty much expected behavior.
>
> On 10 Jan 2008 12:19:30 -0800, Daniel Coppersmith
> <coppersmith2002_A_T_hotmail_D_O_T_N0SPAM.com> wrote:
>
>>This must have been asked a thousand times but I cannot find the answer.
>>
>>I have a string that I need as a decimal.  It goes into a bunch of
>>decimal place precision that caused rounding errors over the course of
>>adding thousands of these numbers together.
>>
>>sting ls_number
>>decimal ld_number
>>
>> ls_number = "42,508.26"
>> ld_number = real( ls_number )
>>
>> // ld_number is now 42508.2617185
>>
>>How do I avoid this? 


0
Jerry
1/10/2008 9:03:43 PM
Thanks.   I knew it had to be simple.



Jim O'Neil [Sybase] wrote:
> Presuming this value is always 2 fixed decimal points, declare as
> 
> decimal{2} ld_number
> 
> Or use 
> 
> ld_number = Dec(ls_number)
> 
> You're jumping between fixed and floating point here, so I'd say this
> is pretty much expected behavior.
> 
> On 10 Jan 2008 12:19:30 -0800, Daniel Coppersmith
> <coppersmith2002_A_T_hotmail_D_O_T_N0SPAM.com> wrote:
> 
>> This must have been asked a thousand times but I cannot find the answer.
>>
>> I have a string that I need as a decimal.  It goes into a bunch of 
>> decimal place precision that caused rounding errors over the course of 
>> adding thousands of these numbers together.
>>
>> sting	ls_number
>> decimal	ld_number
>>
>> 	ls_number = "42,508.26"
>> 	ld_number = real( ls_number )
>>
>> 	// ld_number is now 42508.2617185
>>
>> How do I avoid this?
0
Daniel
1/11/2008 11:07:11 PM
Reply:

Similar Artilces:

Decimal in String to Decimal Number Conversion
I think I'm out of luck after visting DejaNews (hints but no definitive answer), but figured you all are worth a try... :) My PB program imports a text file containing decimal numbers in the format ####.## (e.g., 1234.56), and then converts them to a number to put into a numeric datawindow column. Problem is that if regional settings are set to use the comma as decimal separator and period as thousands separator, then the resulting number from Real(ls_number) is 123456 rather than 1234.56. This is a big problem. The only solution I have so far is to remove the decimal form th...

CLR.NET datatype for decimal is System.String or Decimal
CLR.NET datatype for decimal is System.String or System.Decimal the documentation says System.string while it is mapped to DbType.decimal I need some understanding on this. any help thanks vsv ...

string to string[]
Hi, I've got this code : string[] Params; string SQL = "SELECT * FROM T_MANAGEMENT_PAGES"; SqlCommand myCommand = new SqlCommand(SQL, myConnection); myConnection.Open(); SqlDataReader myReader = myCommand.ExecuteReader(); try { while (myReader.Read()) { Params_Type = myReader.GetValue(0).ToString(); } } catch { } finally { } myReader.Close(); My problem is to obtain Params_Type. But each time, it says : "impossible to convert '[object]' in 'string[]' &quo...

PowerBuilder Strings
I know this is an odd question, but I'm curious. Are PowerBuiler's strings immutable? yes On 11 May 2005 07:54:36 -0700, "Mark Hampton" <mhampton.removeme@datavantagecorp.com> wrote: >I know this is an odd question, but I'm curious. Are PowerBuiler's strings >immutable? > Whats immutable? "Jim O'Neil [Sybase]" <joneil_at_sybase_dot_com> wrote in message news:0ga4819a6021mm78rbi4g96lhrqlvkhg2n@4ax.com... > yes > > On 11 May 2005 07:54:36 -0700, "Mark Hampton" > <mhampton.removem...

string() = string()
this code does not work (not instance of object) Dim Items(), Other() As String Items = New String() {"A","B","C"} Other = New String() {} Other = Items how can I get all the values ? angiras Hi, If I understand your requirement correctly, this should do it Dim Items() As String = {"A", "B", "C"} Dim Other(Items.Length - 1) As String Items.CopyTo(Other, 0) To more closely match you code above, but this will be less efficient Dim Items(), Other() As String ...

Decimal , string ....
Dear all, I am reading a file with amounts , fixed format 0001321311145 4564646431231 1113123131311 i have to read the values (no problem) and then convert them to a decimal variable with 2 decimals behind the comma. so 13213111,45 45646464312,31 11131231313,11 I am not sure how to convert the string value (13 long) to a dec(11,2) TIA John Dec(stringvar)/100 -- Simon Caldwell Get Real Systems Ltd Holtby Manor, Stamford Bridge Road, York, YO19 5LL Tel +44 (0)1904 481999 Fax +44 (0)1904 481666 Visit us at www.getrealsystems.com Procurement Control S...

String and string?
Hi guys, I was wondering the difference between small letter string type and a capital letter String type. Does anyone know about this? Highly appreciate for your answers! Thankz! The "proper" version is String, as it references the System.String class of the .NET Framework Class Library. The ability to use string is provided by C#, where it is an alias to String. Similarly, the "proper" version of an integer is Int32, as it references System.Int32.  The ability to use int is provided by C#, and the ability to use Integer is provided by VB.NET, and both int and Integer are aliases...

Decimal to String
I have a decimal column (9,3) that I want to convert to a right-justified string without the decimal. For instance, I would like to convert 1.256 to "000000001256", or 15.5 to "000000015500". I'd like to be able to do this within a Select Statement (for a view). Does anyone have an idea? -Dave David Gerdner wrote: > > I have a decimal column (9,3) that I want to convert to a right-justified > string without the decimal. For instance, I would like to convert 1.256 to > "000000001256", or 15.5 to "000000015500". I'd like...

String and String[]
What will be the advantage of using myString: String[10] over myString: String Man wrote: > What will be the advantage of using > myString: String[10] > over > myString: String "String[10]" is a fixed-length, statically allocated, Ansi-based ShortString. "String" is a dynamic-length, dynamically allocated, reference counted, Ansi-based or Unicode-based (depening on Delphi version) Long string. Unless you absolutely need the static, fixed length nature of ShortString, such as in records that interact with external systems, you should stay ...

Decimal conversion error. Powerbuilder only supports 18 decimal digits
Hi, When connecting to my database and trying to get result from a table, I get this error. What does it mean ? Hi Dave; Newer version of PB support 28 digits for a decimal column. Can you tell us .... 1) PB version you are using 2) DBMS version that you are using. 3) Connectivity mechanism 4) Table column's datatype that you are trying to access. 5) DataWindow's column data type for the error. etc -- Regards ... Chris ISUG - NA RUG Director http://chrispollach.pbdjmagazine.com <Dave> wrote in message news:4acdda9f.79ff.1681692777@sybase.com... &g...

string and string[]
hello, whats the difference here between string and string[] thanks "string" denotes an item of type String "string[]" denotes an array of type String itemsThanks, EdMicrosoft MVP - ASP/ASP.NET Gracia mon ami...

What generals generally do
Reading a McChrystal thread elsewhere, I came across someone quoting a US general http://en.wikipedia.org/wiki/Smedley_Butler > I spent 33 years and four months in active military service and > during that period I spent most of my time as a high class thug for > Big Business, for Wall Street and the bankers. In short, I was a > racketeer, a gangster for capitalism. I helped make Mexico and > especially Tampico safe for American oil interests in 1914. I helped > make Haiti and Cuba a decent place for the National City Bank boys to > collect revenues in. I ...

String To Decimal
Hi, How can I convert from string (Text Box)  to Decimal (addition some entries then get result) Thanks, http://tinyurl.com/aqpgm To Decimal method. Eric RamseurRainbow Portal 2.0 AdminDownload Rainbow 2.0!!Rainbow Code Rainbow Portal CommunityC# 2005 Group Please can u write thz method   sorry for disturbing cavalry wrote: Please can u write thz method Scroll down just a little bit on that link above and therz the method in both VB and C#"If you make it idiot proof, they'll build a better idiot" Hi again, i see the like but Access to the req...

"Decimal Conversion Error: PowerBuilder supports only 18 decimal digits
Hi PB 10.5 brings me this error . WHY ? but comes in IDE and not in executable ... I do not understand what is happening .... Is it a migration issue .. My earlier source was in PB10.0 and there is no error . Where we have DisableUnicode=1 option ?? -- Regards Van Note: PB 10.5 supports 28 digit decimal numbers. Where do you get this error message - Oracle DBMS? "Van" <VanJacobson@hotmail.com> wrote in message news:456bf69a@forums-1-dub... > Hi > PB 10.5 brings me this error . WHY ? but comes in IDE and not in executable > .. I do not u...

Web resources about - String to decimal - sybase.powerbuilder.general

Penny (Irish pre-decimal coin) - Wikipedia, the free encyclopedia
The penny (1d) ( Irish : pingin ) coin was the third smallest denomination of the pre-decimal Irish pound , worth 1/240 of a pound or 1/12 of ...

Dewey Decimal Classification - Wikipedia, the free encyclopedia
This highly organized system categorizes books on library shelves in an efficient, specific and repeatable order that makes it easy to find any ...

4th Grade Splash Math Worksheets to learn decimal numbers, multiplication, division & fractions for kids ...
Get 4th Grade Splash Math Worksheets to learn decimal numbers, multiplication, division & fractions for kids on the App Store. See screenshots ...

Dewey Decimal Tattoo - Flickr - Photo Sharing!
Children's librarians represent.

Math of the Day: Sesame Street’s The Count Reads Pi to 10,000 Decimal Places
... has created a true monster out of your favorite Sesame Street vampire, with audio of The Count reading every digit of Pi out 10,000 decimal ...

Paul Harapin heads Decimal Software after VCE ousting
Former VCE Asia-Pacific boss, Paul Harapin, has taken over as CEO of Decimal Software.

Feeling green: the dangers of decimal currency
The curious case of possibly toxic coins, via Austria and a devastatingly original idea for Canberra.

Jobless Rates With Another Decimal
The BLS reported a jobless rate of 8.3% for both January and February. Adding an extra decimal, the unemployment rates would actually be 8.26% ...

JPMorgan Sued by Trader Over $3 Million Decimal Point
JPMorgan Chase & Co. is being sued by a trader who says he accepted a contract from the investment bank because a typographical error made him ...

.Decimal Archive - Houston Gamer
It was only back in August when newly-formed Houston game studio Orb Interactive launched a Kickstarter campaign for “.Decimal”, a sci-fi take ...

Resources last updated: 1/19/2016 10:08:58 PM