apache spark - scala type mismatch error in graphX code -
i'm new scala, learning apache-spark. wrote simple function in scala graphx
def foo(edge: edgetriplet[map[long, double], double]): iterator[(vertexid, map[long, double])] = { val m = edge.srcattr for((k, v) <- m){ if (v + edge.attr < edge.dstattr.getorelse(k, 10.0)) iterator(edge.dstid, map(k -> v + edge.attr)) else iterator.empty } } errors
name: compile error message: <console>:37: error: type mismatch; found : double required: string iterator(edge.dstid, map(k -> v + edge.attr)) ^ <console>:35: error: type mismatch; found : unit required: iterator[(org.apache.spark.graphx.vertexid, map[long,double])] (which expands to) iterator[(long, map[long,double])] for((k, v) <- m){ ^ stacktrace: why scala treating v string? , cause of second error?
after editing code suggested @alexey, i'm getting error
name: compile error message: <console>:30: error: type mismatch; found : scala.collection.immutable.map[org.apache.spark.graphx.vertexid,scala.collection.immutable.map[long,double]] (which expands to) scala.collection.immutable.map[long,scala.collection.immutable.map[long,double]] required: iterator[(org.apache.spark.graphx.vertexid, map[long,double])] (which expands to) iterator[(long, map[long,double])] (k, v) <- edge.srcattr ^ stacktrace: if helps, i'm implementing different version of sendmessage function code
the first common problem + in scala, unfortunately. in case have not k -> (v + edge.attr), expected, (k -> v) + edge.attr. , + method on tuple2 accepts string. fix it, add correct parentheses.
the second error because for(...) { ... } returns unit (it's translated foreach call). missing yield. in fact if want use for, should like
for { (k, v) <- m x <- if (v + edge.attr < edge.dstattr.getorelse(k, 10.0)) iterator((edge.dstid, map(k -> (v + edge.attr)))) else iterator.empty[(long, map[long,double])] } yield x i'd prefer write as
m.flatmap { case (k, v) => if (v + edge.attr < edge.dstattr.getorelse(k, 10.0)) iterator((edge.dstid, map(k -> (v + edge.attr)))) else iterator.empty[(long, map[long,double])] }
Comments
Post a Comment