scala - Generic map method for spark RDD doesnt compile -
i can't figure out why doesn't compile:
implicit class debughelper[t](ar: rdd[t]) { def debug_restrainer(implicit sc: sparkcontext): rdd[t] = { if (debug_size.isdefined) sc.parallelize(ar.take(debug_size.get)) else ar } }
it gives error: no classtag available t
does know it's complaining about?
if compiler asks classtag
need. sparkcontext
can retrieved existing rdd.
import scala.reflect.classtag implicit class debughelper[t](ar: rdd[t])(implicit val t: classtag[t]) { def debug_restrainer: rdd[t] = { if (debug_size.isdefined) ar.sparkcontext.parallelize(ar.take(debug_size.get)) else ar } }
Comments
Post a Comment