I want to get pixel-by-pixel color data for an image so I can adapt it into a "8-bit" sort of look using a grid of colored squares.

From the research I've done so far, it looks like the standard way to get that kind of data is using an HTML5 canvas and context.getImageData (example). So far though I've had no luck getting this to work in a React app.

The closest I've gotten is this. I'm sure there are a million things wrong with it, probably to do with how I'm interacting with the DOM, but it at least returns an imageData object. The problem is that the color value for every pixel is 0.

Updated to use ref instead of getElementById

function App() {
  const imgRef = useRef(null);
  const img = <img ref={imgRef} src={headshot} />;

  // presumably we want to wait until after render?
  useEffect(() => {
    if (imgRef === null) {
      console.log("image ref missing");
      return;
    }
    if (imgRef.current === null) {
      console.log("image ref is null");
      return;
    }
    // couldn't use imgRef.current.offsetHeight/Width for these because typscript 
    // thinks imgRef.current is `never`?
    const height = 514; 
    const width = 514;
    const canvas = document.createElement('canvas');
    canvas.height = height; canvas.width = width;
    const context = canvas.getContext && canvas.getContext('2d');
    if (context === null) {
      console.log(`context or image missing`);
      return
    }
    context.drawImage(imgRef.current, 0, 0);
    const imageData = context.getImageData(0, 0, width, height);
    console.log(`Image Data`, imageData);
  }, []);

  return img; 
}

Related: ultimately I want to get this data without actually displaying the image, so any tips on that would also be appreciated.

Thanks!

CodePudding user response：

You're not waiting for the image to load. (As with regular HTML, that happens asynchronously.)

So, instead of using an effect that's run as soon as the component has mounted, hook it up to your image's onLoad. Aside from that:

• You don't need to check whether imgRef is null; the ref box itself never is.
• When using TypeScript and DOM refs, use useRef<HTML...Element>(null); to have ref.current have the correct type.
• There is no need to make imgRef.current a dependency for the handler, since refs changing do not cause components to update.

export default function App() {
  const imgRef = React.useRef<HTMLImageElement>(null);

  const readImageData = React.useCallback(() => {
    const img = imgRef.current;
    if (!img?.width) {
      return;
    }
    const { width, height } = img;
    const canvas = document.createElement("canvas");
    canvas.height = height;
    canvas.width = width;
    const context = canvas.getContext?.("2d");
    if (context === null) {
      return;
    }
    context.drawImage(img, 0, 0);
    const imageData = context.getImageData(0, 0, width, height);
    console.log(`Image Data`, imageData);
  }, []);

  return <img ref={imgRef} src={kitten} onl oad={readImageData} />;
}

EDIT

Do you know how I can do this now without actually displaying the image?

To read an image without actually showing it in the DOM, you'll need new Image - and then you can use an effect.

/** Promisified image loading. */
function loadImage(src: string): Promise<HTMLImageElement> {
  return new Promise((resolve, reject) => {
    const img = new Image();
    img.addEventListener("load", () => {
      resolve(img);
    });
    img.addEventListener("error", reject);
    img.src = src;
  });
}

function analyzeImage(img: HTMLImageElement) {
  const { width, height } = img;
  const canvas = document.createElement("canvas");
  canvas.height = height;
  canvas.width = width;
  const context = canvas.getContext?.("2d");
  if (context === null) {
    return;
  }
  context.drawImage(img, 0, 0);
  const imageData = context.getImageData(0, 0, width, height);
  console.log(`Image Data`, imageData);
}

export default function App() {
  React.useEffect(() => {
    loadImage(kitten).then(analyzeImage);
  }, []);
  return <>hi</>;
}