When I was searching internet for an algorithm to correct luminance I came across this

grain_background.png:

Complete example:

import cv2
import numpy as np
from numpy.ma import divide, mean

f = cv2.imread("grain.png")
b = cv2.imread("grain_background.png")

f = f.astype(np.float32)
b = b.astype(np.float32)

C = mean(f) / divide(f, b).mean()
g = divide(f, b) * C

g = g.astype(np.uint8)
cv2.imwrite("grain_out.png", g)

Your need to use masked divide operation because ordinary operation could lead to division by zero => nan values.

Resulting image (output.png):