How can I pass a base64 image to tensorflow JS?-CodePudding

I'm using tensorflow JS to do image classification.
I store training images in the browser indexDB in data format (aka data data:image/jpeg;base64,/9j/4A...)

When I train, I use this function which is supposed to convert my data IMG to a tensor.

const imageToTensor = (imageData: string) => {
    // convert base64 to Image for the fromPixels
    const img = new Image()
    img.src = imageData
    img.width = 224
    img.height = 224

    const imageFeatures = tf.tidy(function () {
        const imageAsTensor = tf.browser.fromPixels(img)
        imageAsTensor.print()
        return imageAsTensor
    })
    return imageFeatures
}

But my imageAsTensor.print() is showing me just a bunch of 000

Tensor
    [[[0, 0, 0],
      [0, 0, 0],
      [0, 0, 0],
      ...,
      [0, 0, 0],
      [0, 0, 0],
      [0, 0, 0]],

     [[0, 0, 0],
      [0, 0, 0],
      [0, 0, 0],
      ...,
      [0, 0, 0],
      [0, 0, 0],
      [0, 0, 0]],

     [[0, 0, 0],
      [0, 0, 0],
      [0, 0, 0],
      ...,
      [0, 0, 0],
      [0, 0, 0],
      [0, 0, 0]],

     ...
     [[0, 0, 0],
      [0, 0, 0],
      [0, 0, 0],
      ...,
      [0, 0, 0],
      [0, 0, 0],
      [0, 0, 0]],

     [[0, 0, 0],
      [0, 0, 0],
      [0, 0, 0],
      ...,
      [0, 0, 0],
      [0, 0, 0],
      [0, 0, 0]],

     [[0, 0, 0],
      [0, 0, 0],
      [0, 0, 0],
      ...,
      [0, 0, 0],
      [0, 0, 0],
      [0, 0, 0]]]

Can't seem to see what I am doing wrong. Am I missing an await or something?

Thanks for your help.

CodePudding user response：

It is hard to find the problem with this snippet but the fromPixels method expects an HTML canvas element or an image element as the input, but you are passing an image element that is not yet loaded. you can edit like this:

const imageToTensor = (imageData: string) => {
    // convert base64 to Image for the fromPixels
    const img = new Image()
    img.src = imageData
    img.width = 224
    img.height = 224

    return new Promise((resolve) => {
        img.onload = () => {
            const imageFeatures = tf.tidy(function () {
                const imageAsTensor = tf.browser.fromPixels(img)
                imageAsTensor.print()
                return imageAsTensor
            })
            resolve(imageFeatures)
        }
    })
}

and then then use the await keyword to wait for the promise to resolve before using the image tensor in your model.

async function trainModel() {
    const imageData = 'data:image/jpeg;base64,/9j/4A...'
    const imageTensor = await imageToTensor(imageData)
    // Use the image tensor to train the model
}