java - Computer Vision - filtering convex hulls and convexity defects with OpenCV -

i have problem processing digital signals. trying detect fingertips, similar solution presented here: hand , finger detection using javacv.

however, not using javacv opencv android different. have managed steps presented in tutorial, filtering of convex hulls , convexity defects. how image looks like:

here image in resolution:

as can see, there many yellow points (convex hulls) , many red points (convexity deffects). between 2 yellow points there no red point, quite strange (how convex hulls calculated?)

what need create simillar filtering function in link provided before, using data structures of opencv.

convex hulls type of matofint ... convexity defects type of matofint4 ...

i created additional data structures, because stupid opencv uses different types of data containing same data, in different methods...

convexhullmatofint = new matofint(); convexhullpointarraylist = new arraylist<point>(); convexhullmatofpoint = new matofpoint(); convexhullmatofpointarraylist = new arraylist<matofpoint>(); 

here did far not working good. problem converting data in wrong way:

creating convex hulls , convexity defects:

public void calculateconvexhulls() {     convexhullmatofint = new matofint();     convexhullpointarraylist = new arraylist<point>();     convexhullmatofpoint = new matofpoint();     convexhullmatofpointarraylist = new arraylist<matofpoint>();      try {         //calculate convex hulls         if(aproximatedcontours.size() > 0)         {             imgproc.convexhull( aproximatedcontours.get(0), convexhullmatofint, false);              for(int j=0; j < convexhullmatofint.tolist().size(); j++)                 convexhullpointarraylist.add(aproximatedcontours.get(0).tolist().get(convexhullmatofint.tolist().get(j)));             convexhullmatofpoint.fromlist(convexhullpointarraylist);             convexhullmatofpointarraylist.add(convexhullmatofpoint);             }     } catch (exception e) {         // todo auto-generated catch block         log.e("calculate convex hulls failed.", "details below");         e.printstacktrace();     } }  public void calculateconvexitydefects() {     mconvexitydefectsmatofint4 = new matofint4();      try {         imgproc.convexitydefects(aproximatedcontours.get(0), convexhullmatofint, mconvexitydefectsmatofint4);          if(!mconvexitydefectsmatofint4.empty())         {             mconvexitydefectsintarraylist = new int[mconvexitydefectsmatofint4.toarray().length];             mconvexitydefectsintarraylist = mconvexitydefectsmatofint4.toarray();         }     } catch (exception e) {         log.e("calculate convex hulls failed.", "details below");         e.printstacktrace();     } } 

filtering:

public void filtercalculatedpoints()     {         arraylist<point> tippts = new arraylist<point>();         arraylist<point> foldpts = new arraylist<point>();         arraylist<integer> depths = new arraylist<integer>();          fingertips = new arraylist<point>();          (int = 0; < mconvexitydefectsintarraylist.length/4; i++)         {             tippts.add(contours.get(0).tolist().get(mconvexitydefectsintarraylist[4*i]));             tippts.add(contours.get(0).tolist().get(mconvexitydefectsintarraylist[4*i+1]));             foldpts.add(contours.get(0).tolist().get(mconvexitydefectsintarraylist[4*i+2]));             depths.add(mconvexitydefectsintarraylist[4*i+3]);         }          int numpoints = foldpts.size();         (int i=0; < numpoints; i++) {             if ((depths.get(i).intvalue()) < min_finger_depth)                 continue;              // @ fold points on either side of tip             int pdx = (i == 0) ? (numpoints-1) : (i - 1);             int sdx = (i == numpoints-1) ? 0 : (i + 1);              int angle = anglebetween(tippts.get(i), foldpts.get(pdx), foldpts.get(sdx));             if (angle >= max_finger_angle)   // angle between finger , folds wide                 continue;               // point fingertip, add list             fingertips.add(tippts.get(i));         }     } 

results (white points - fingertips after filtering):

could me write proper function filtering?

update 14.08.2013

i use standard opencv function contour approximation. have change approximation value resolution change, , hand-to-camera distance, quite hard do. if resolution smaller, finger consist of less pixel, approximation value should lover. same distance. keeping high result in losing finger. think approximation not approach resolving problem, small value useful speed calculations:

imgproc.approxpolydp(frame, frame, 2 , true);  

if use high values, result on image below, if distance , resolution wouldn't change. also, quite surprised default methods hulls points , defects points doesn't have useful arguments pass (min angle, distance etc)...

image below presents effect achieve always, independently resolution or hand-to-camera distance. don't want see yellow points when close palm...

to sum up, know:

• how filter points
• how can make resolution , distance independent approximation work
• if knows or have materials (graphical representation, explanation) data structures used in opencv, happy read it. (mat, matofint, matofpoint, matofpoint2, matofpoint4 etc.)

the convex hull @ low res can used identify position of hand whole, not useful fingers provide region of interest , appropriate scale.

the higher resolution analysis should applied approximated contour, easy skip points not pass "length , angle" criteria last two, though may wish "average in" instead of "skip entirely".

your code example single pass of calculating convexity defects , removing them .. logic error .. need remove points go .. (a) faster , simpler in one-pass (b) avoids removing points @ first pass , having add them later because removal changes previous calcs.

this basic technique simple , works basic open palm. doesn't intrinsically understand hand or gesture though, tuning scale, angle , length parameters ever going "so far".

references techniques: filter length , angle "convexity defect" simen andresen blog http://simena86.github.io/blog/2013/08/12/hand-tracking-and-recognition-with-opencv/

kinect sdk based c# library added finger direction detection http://candescentnui.codeplex.com/ http://blog.candescent.ch/2011/11/improving-finger-detection.html

"self-growing , organized neural gas" (sgong) prof nikos papamarkos http://www.papamarkos.gr/uploaded-files/hand%20gesture%20recognition%20using%20a%20neural%20network%20shape%20fitting%20technique.pdf

commercial product david holz & michael buckwald founders of "leap motion" http://www.engadget.com/2013/03/11/leap-motion-michael-buckwald-interview/