No announcement yet.

Unpack String Instruction

  • Filter
  • Time
  • Show
Clear All
new posts

  • Unpack String Instruction

    I receive a string back from a device using RS232 communication. It is a 21 character string. The first 11 characters received are an echo of the command sent. :SENS:PRESS for example and the last 10 characters is the pressure 13.1234567 for example. I'm using the unpack string instruction in the Productivity 2000 programming suite. I have a tag type float 32 to receive the last ten characters but for some reason it only stores like 4 decimal places instead of the full 10 characters. Any ideas why?

  • #2
    You could store the entire number as another string to preserve all the digits. But that's not what you are doing. You are asking the instruction to interpret the digits as a floating point number. If you are not clear on how a floating point number works Google 'floating point' and start reading. 32 bit floating point number can store about 7 decimal digits (total of those before and after the decimal point). It also has limits on exactness. Just as standard decimal numbers can't exactly store the value 1/3 the floating point numbers have trouble storing fractional numbers which aren't the sum of sub multiples of 2. (e.g 1/2, 1/4, 1/8, 1/16 etc). So besides having trouble with that 1/3 it doesn't have a precise representation of 1/10. Another problem is how the Productivity package is showing its value to you. To save screen space it is probably rounding the value to these number of digits. But your whole number, like your example, would not be stored.



    • #3
      Thank you for your explanation