linear regression #9
Description
Hi Harish,
trying linear regression,
this works:
*from <select acct_no, mydt, x, y
from mytab>
partition by acct_no
order by acct_no,mydt
with
linearRegSlope(x,y) as slope,
linearRegIntercept(x,y) as intercept
select acct_no, slope,intercept
into path='ma_9_2011_08'; *
but this does not, which tries to do regression on ALL the records without partitioning
**
from <select 1 as blah,acct_no, mydt, x, y
from mytab>
partition by blah
order by blah
with
linearRegSlope(x,y) as slope,
linearRegIntercept(x,y) as intercept
select blah, slope,intercept
into path='ma_9_2011_08';
**
Error:
12/02/03 12:24:55 INFO mapred.JobClient: Task Id : attempt_201201151510_0220_r_000002_3, Status : FAILED
java.lang.reflect.UndeclaredThrowableException: Unknown exception in doAs
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1139)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.security.PrivilegedActionException: com.sap.hadoop.ds.list.ByteBasedList$ListFullException
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
... 1 more
Caused by: com.sap.hadoop.ds.list.ByteBasedList$ListFullException
at com.sap.hadoop.ds.list.ByteBasedList.ensureCapacity(ByteBasedList.java:80)
at com.sap.hadoop.ds.list.ByteBasedList.write(ByteBasedList.java:108)
at com.sap.hadoop.ds.list.ByteBasedList.append(ByteBasedList.java:158)
at com.sap.hadoop.ds.list.ByteBasedList$append.call(Unknown Source)
at com.sap.hadoop.windowing.runtime.Partition.leftShift(Partition.groovy:100)
at com.sap.hadoop.windowing.runtime.Partit
attempt_201201151510_0220_r_000002_3: Query:
attempt_201201151510_0220_r_000002_3: tableInput=(hiveTable=WindowingTempTable_1328289790395)
attempt_201201151510_0220_r_000002_3: partitionColumns=all_in_one
attempt_201201151510_0220_r_000002_3: orderColumns=all_in_one ASC
attempt_201201151510_0220_r_000002_3: funcSpecs=[linearregslope(alias=slope, param=[id=bur_fico_scor_no, id=cv_score], type=null, window=null),
attempt_201201151510_0220_r_000002_3: linearregintercept(alias=intercept, param=[id=bur_fico_scor_no, id=cv_score], type=null, window=null)]
attempt_201201151510_0220_r_000002_3: select=slope, intercept
attempt_201201151510_0220_r_000002_3: whereExpr=null