c++ - How to combine two remap() operations into one? -

i have tight loop, camera image, undistort , transform according transformation (e.g. perspective transform). figured out use cv::remap(...) each operation, more efficient using plain matrix operations.

in understanding should possible combine lookup maps 1 , call remap once in every loop iteration. there canonical way this? prefer not implement interpolation stuff myself.

note: procedure should work differently sized maps. in particular case undistortion preserves image dimensions, while other transformation scales image different size.

code illustration:

// input arguments const cv::mat_<math::flt> intrinsic  = getintrinsic(); const cv::mat_<math::flt> distortion = getdistortion(); const cv::mat mnewcameramatrix = cv::getoptimalnewcameramatrix(intrinsic, distortion, myimagesize, 0);  // output arguments cv::mat undistortmapx; cv::mat undistortmapy;  // computes undistortion maps cv::initundistortrectifymap(intrinsic, distortion, cv::mat(),                             newcameramatrix, myimagesize, cv_16sc2,                             undistortmapx, undistortmapy);  // computes undistortion maps // ...computation of mapx , mapy omitted cv::convertmaps(mapx, mapy, skewmapx, skewmapy, cv_16sc2);  for(;;) {     cv::mat originalimage = getnewimage();      cv::mat undistortedimage;     cv::remap(originalimage, undistortedimage, undistortmapx, undistortmapy, cv::inter_linear);      cv::mat skewedimage;     cv::remap(undistortedimage, skewedimage, skewmapx, skewmapy, cv::inter_linear);      outputimage(skewedimage); } 

in case of 2 general mappings, there no choice use approach suggested @michaelburdinov.

however, in special case of 2 mappings known inverse mappings, alternative approach compute maps manually. manual approach more accurate double remap one, since not involve interpolation of coordinate maps.

in practice, of interesting applications match special case. in case because first map corresponds image undistortion (whose inverse operation image distortion, associated known analytical model) , second map corresponds perspective transform (whose inverse can expressed analytically).

computing maps manually quite easy. stated in documentation (link) these maps contain, each pixel in destination image, (x,y) coordinates find appropriate intensity in source image. following code snippet shows how compute maps manually in case:

int dst_width=...,dst_height=...;           // initialize size of output image cv::mat hinv=h.inv(), kinv=k.inv();         // precompute inverse perspective matrix , inverse camera matrix cv::mat map_undist_warped_x32f(dst_height,dst_width,cv_32f);    // allocate x map correct size (n.b. data type used float) cv::mat map_undist_warped_y32f(dst_height,dst_width,cv_32f);    // allocate y map correct size (n.b. data type used float) // loop on rows of output image for(int y=0; y<dst_height; ++y) {     std::vector<cv::point3f> pts_undist_norm(dst_width);     // each pixel on current row, first use inverse perspective mapping, multiply     // inverse camera matrix (i.e. map pixels normalized coordinates prepare use of projectpoints function)     for(int x=0; x<dst_width; ++x) {         cv::mat_<float> pt(3,1); pt << x,y,1;         pt = kinv*hinv*pt;         pts_undist_norm[x].x = pt(0)/pt(2);         pts_undist_norm[x].y = pt(1)/pt(2);         pts_undist_norm[x].z = 1;     }     // each pixel on current row, compose inverse undistortion mapping (i.e. distortion     // mapping) using projectpoints function     std::vector<cv::point2f> pts_dist;     cv::projectpoints(pts_undist_norm,cv::mat::zeros(3,1,cv_32f),cv::mat::zeros(3,1,cv_32f),intrinsic,distortion,pts_dist);     // store result in appropriate pixel of output maps     for(int x=0; x<dst_width; ++x) {         map_undist_warped_x32f.at<float>(y,x) = pts_dist[x].x;         map_undist_warped_y32f.at<float>(y,x) = pts_dist[x].y;     } } // finally, convert float maps signed-integer maps best efficiency of remap function cv::mat map_undist_warped_x16s,map_undist_warped_y16s; cv::convertmaps(map_undist_warped_x32f,map_undist_warped_y32f,map_undist_warped_x16s,map_undist_warped_y16s,cv_16sc2); 

note: h above perspective transform while kshould camera matrix associated undistorted image, should in code called newcameramatrix (which btw not output argument of initundistortrectifymap). depending on specific data, there might additional cases handle (e.g. division pt(2) when might zero, etc).