scala - Plug a function with currying into Spark UserDefinedFunction -

i have function works expected

import org.apache.spark.sql.expressions.userdefinedfunction import org.apache.spark.sql.functions.udf import scala.collection.mutable.wrappedarray  def arraycontainsany(s: seq[string]): userdefinedfunction = udf((xs: wrappedarray[string]) => !xs.tolist.intersect(s).isempty) 

i need split function userdefinedfunction definition

i have have tried following

// imports again def _arraycontainsany(s: seq[string])(c: wrappedarray[string]): boolean = !c.tolist.intersect(s).isempty def arraycontainsany: userdefinedfunction = udf[boolean, wrappedarray[string], seq[string]](_arraycontainsany) 

but not compile.

the problem seems define function udf[x, y, z] , expects function (z, y) => x , not (z)(y) => x

does know how this?

-- β

option 1:

using method 2 argument lists, when wrapping in udf should pass first argument , use _ resulting function:

def _arraycontainsany(s: seq[string])(xs: mutable.wrappedarray[string]) = xs.tolist.intersect(s).nonempty  def arraycontainsany(s: seq[string]): userdefinedfunction = {   udf(_arraycontainsany(s) _) } 

option 2:

you can create method takes seq[string] , returns function wrappedarray[string] => boolean, call method when creating udf:

def _arraycontainsany(s: seq[string]) =    (xs: mutable.wrappedarray[string]) => xs.tolist.intersect(s).nonempty  def arraycontainsany(s: seq[string]): userdefinedfunction = {   udf(_arraycontainsany(s)) }