Python cv2.remap from mesh creates pixelated distortions-CodePudding

I have a mesh that is 4 times smaller than an image, I want to distort the image with the information from the mesh but, when using cv2.remap it makes the distortion pixelated (See image below) . How could I make a smoother distortion?

Original:

Desired output:

My output:

My code:

img = np.array(Image.open('astronaut.jpg')) # Shape -> (512, 512, 3)
mesh = Mesh('astronaut.msh').get_uvs() # Shape -> (128, 128, 2), 2 channels for x and y 

new_mesh = np.zeros((img.shape[1], img.shape[0], 2))
new_mesh[:,:,0] = np.repeat(np.repeat(mesh[:,:,0], 4, axis=0), 4, axis=1)
new_mesh[:,:,1] = np.repeat(np.repeat(mesh[:,:,1], 4, axis=0), 4, axis=1)

nh, nw = img.shape[:2]
xs, ys = np.meshgrid(np.arange(0, nw), np.arange(0, nh))
xs = xs   new_mesh[:,:,0] * 4 # multiply by constant to modulate distort strength
ys = ys   new_mesh[:,:,1] * 4
xs = np.float32(xs)
ys = np.float32(ys)

dst= cv2.remap(img.astype(np.uint8), xs, ys, cv2.INTER_CUBIC)

CodePudding user response：

OpenCV is not to blame. It does exactly what you tell it to.

Those artefacts come from your use of np.repeat. That merely repeats each index element in your map array. You're not upsampling your mesh properly, you effectively just copy 4x4 patches with that code.

Properly upsample your mesh (np.repeat is wrong), then you get good results. You can easily do that using cv.resize and anything other than nearest-neighbor interpolation. If you need to control boundary behavior exactly, you'll need warpAffine and a custom transformation matrix. You could even use cv.pyrUp (twice).

When you've presented a MRE (some data for the mesh), I'll update my answer with working code.