How to avoid "InsufficientMemory" decoding error using Rust Image crate?-CodePudding

I am trying to read an 8K 32bit OpenEXR HDR file with Rust. Using the Image crate to read the file:

use image::io::Reader as ImageReader;

let img = ImageReader::open(r"C:\Users\Marko\Desktop\HDR_Big.exr")
    .expect("File Error")
    .decode()
    .expect("Decode ERROR");

This results in an Decode ERROR: Limits(LimitError { kind: InsufficientMemory })

Reading a 4K file or smaller works fine.

I thought buffering would help so I tried:

use image::io::Reader as ImageReader;
use std::io::BufReader;
use std::fs::File;

let f = File::open(r"C:\Users\Marko\Desktop\HDR_Big.exr").expect("File Error");
let reader = BufReader::new(f);
let img_reader = ImageReader::new(reader)
    .with_guessed_format()
    .expect("Reader Error");
let img = img_reader.decode().expect("Decode ERROR");

But the same error results. Is this a problem with the image crate itself? Can it be avoided?

If it makes any difference for the solution after decoding the image I use the raw data like this:

let data: Vec<f32> = img.to_rgb32f().into_raw();

Thanks!

CodePudding user response：

But the same error results. Is this a problem with the image crate itself? Can it be avoided?

No because it's not a problem and yes it can be avoided.

When an image library faces the open web it's relatively easy to DOS the entire service or exhaust its library cheaply as it's usually possible to request huge images at a very low cost (for instance a 44KB PNG can decompress to a 1GB full-color buffer, and a megabyte-scale jpeg can reach GB-scale buffer sizes).

As a result modern image libraries tend to set limits by default in order to limit the "default" liability of users.

That is the case of image-rs, by default it does not set any width or height limits but it does request that the allocator limits itself to 512MB.

If you wish for higher or no limitations, you can configure the decoder to match.

All of this is surfaced by simply searching for the error name and the library (both "InsufficientMemory image-rs" and "LimitError image-rs" surfaced the information)

CodePudding user response：

By default, image::io::Reader asks the decoder to fit the decoding process in 512 MiB of memory, according to the documentation. It's possible to disable this limitation, using, e.g., Reader::no_limits.