I am trying to convert a var assignment to a val assignment. Currently my code is
// Numerical vectorizing for normalization
var normNumericalColNameArray: Array[String] = Array()
if (!continousPredictors.sameElements(Array(""))) {
if (paramStandardize) {
println("Apply standardization")
stages = new VectorAssembler().setInputCols(continousPredictors).setOutputCol(NumericalFeaturesCol)
stages = new StandardScaler()
.setWithMean(true)
.setWithStd(true)
.setInputCol(NumericalFeaturesCol)
.setOutputCol(StandardizedNumericalFeaturesCol)
normNumericalColNameArray = Array(StandardizedNumericalFeaturesCol)
} else {
println("Not apply standardization")
stages = new VectorAssembler().setInputCols(continousPredictors).setOutputCol(NumericalFeaturesCol)
normNumericalColNameArray = Array(NumericalFeaturesCol)
}
}
stages = new VectorAssembler().setInputCols(normNumericalColNameArray oneHotCatColNamesArray).setOutputCol(FeaturesCol)
and I want to do something like this
val normNumericalColNameArray =
if (continousPredictors.nonEmpty && paramStandardize) {
println("Apply standardization")
stages = new VectorAssembler().setInputCols(continousPredictors).setOutputCol(NumericalFeaturesCol)
stages = new StandardScaler()
.setWithMean(true)
.setWithStd(true)
.setInputCol(NumericalFeaturesCol)
.setOutputCol(StandardizedNumericalFeaturesCol)
Array(StandardizedNumericalFeaturesCol)
} else if (continousPredictors.nonEmpty){
println("Not apply standardization")
stages = new VectorAssembler().setInputCols(continousPredictors).setOutputCol(NumericalFeaturesCol)
Array(NumericalFeaturesCol)
}
stages = new VectorAssembler().setInputCols(normNumericalColNameArray oneHotCatColNamesArray).setOutputCol(FeaturesCol)
and I run into this error
value is not a member of Any
stages = new VectorAssembler().setInputCols(normNumericalColNameArray oneHotCatColNamesArray).setOutputCol(FeaturesCol)
I am trying to return the Array from the if condition into my Val normNumericalColNameArray. Can somebody please help?
CodePudding user response:
The problem is that there is an if
without an else
block, so the result could be nothing (Unit
). Just add an else
clause:
} else if (continousPredictors.nonEmpty) {
println("Not apply standardization")
stages = new VectorAssembler().setInputCols(continousPredictors).setOutputCol(NumericalFeaturesCol)
Array(NumericalFeaturesCol)
} else {
Array()
}
But Array
is a Java type, so prefer to use Vector
or List
if possible.
CodePudding user response:
The problem is in the if statement. You have:
val normNumericalColNameArray = {
if (condition1){
return Array[String]()
}
else if (condition2){
return Array[String]()
}
}
Now, your variable has type Any because if condition1 is not true and condition2 is not true, then your if statement (function) doesn't return anything: you actually made a partial function. Try adding a final else statement, like:
val normNumericalColNameArray = {
if (condition1){
return Array[String]()
}
else if (condition2){
return Array[String]()
}
else{
return Array[String]()
}
}
It will work then. Also, I would recommend to make your normNumericalColNameArray
a lazy val, so to be evaluated only when it is required, since you are modifying stages
in the if statement.