#4 offset and precision problems

[Brian Fitzgerald]

I'm having problems with offset and scale...

for some reason, although my xyz data is in SQL as a
numeric with only 2 decimals allowed, dbms2sde sees
the data as ex. ( 2000000.25999999999) or something
with a lot of decimals.

any help would be greatly appreciated.

see below for output

C:\>dbms2sde -o point -l surfacetest,shapefield -xname
Easting -yname Northing -
zname Elevation -D sdedata -u sdedata -p ***** -key
UNID -i sde_sql -x 100
0000,-1000000,1000000.00

DBMS2SDE alpha 0.341 Nov 5 2002 - an ArcSDE 8.1
application

Running: dbms2sde -o point -l surfacetest,shapefield -x
1000000.000000,-1000000
.000000,1000000.000000 -xname Easting -yname
Northing -zname Elevation -key UNID
-i sde_sql -D sdedata -s GISDATA -u sdedata -p
***********

For fastest performance, use -stable, -load, and -buffer.
Summary information for rows being spatially enabled:

Column Count Minimum Maximum Average
Std. Dev.
--------- ------ ---------- ---------- ---------- ---------
Easting 250315 2065000.00 2069999.96 2067464.22
1551.31
Northing 250315 785000.00 789999.99 787507.81
1403.23
Elevation 250315 204.47 445.21 291.49 50.07
UNID 250315 1 250315 125158.00
72259.86

Disagreement in row count: 250315 versus 0
Key column(s) is not unique -- 0 duplicate rows may
have incorrect shapes.

Rows Enabled Elapsed Time Estimated Time
------------ ------------ --------------

X-coordinate does not convert to ArcSDE internal
storage format.
x-coordinate is 2066816.9199999999 and converts to
1066816920000, but must be an
integer between 0 and 2147483647.
Try adjusting x offset (1000000.0000000000) and/or
scale (1000000.0000000000)
Y-coordinate does not convert to ArcSDE internal
storage format.
y-coordinate is 789999.9900000000 and converts to
1789999990000, but must be an
integer between 0 and 2147483647.
Try adjusting y offset (-1000000.0000000000) and/or
scale (1000000.0000000000)
Failed attempt to spatially enable where "UNID=1"

SE_shape_generate_point encountered a -139 error:
The specified coordinate exceeds the valid coordinate
range

[Mark Ohrenschall]

Logged In: YES
user_id=586167

Hi Brian,

There may be several problems going on here, but let's begin with the easiest. Do you have coordinate values like
2066816.92? If so, you probably only need an xyscale of 100 -- not 1000000. (xyscale is used to multiply the
real-world coordinates by a sufficient factor to get all the significant digits before the decimal point; to scale into an
integer -- but an integer less than 2147483648)

Similarly with x and y offset, if all your real-world values are positive, then you can use a zero offset. (The x and y
offset are added to the real-world values to translate them into positive numbers.)

Let me know if this is the case, and what happens when you run dbms2sde with smaller offsets and scale.

Thanks,

Mark