Home > Software design >  Image too big for processing when converting large (1.3 GB) DNG file to PNG using python
Image too big for processing when converting large (1.3 GB) DNG file to PNG using python

Time:09-01

I need to convert a DNG file to PNG using python. I found a post here how to convert DNG: Opencv Python open dng format

The code I tried:

#open dng and convert
import rawpy
import imageio
import os

os.chdir(r'C:\Path\to\dir')
path = r'path\to\file'

with rawpy.imread(path) as raw:
    rgb = raw.postprocess()

rgb_img.save('image.png')

However, it spits out the following errors:

line 13, in <module>
    rgb = raw.postprocess()
  File "rawpy\_rawpy.pyx", line 887, in rawpy._rawpy.RawPy.postprocess
  File "rawpy\_rawpy.pyx", line 790, in rawpy._rawpy.RawPy.dcraw_process
  File "rawpy\_rawpy.pyx", line 438, in rawpy._rawpy.RawPy.ensure_unpack
  File "rawpy\_rawpy.pyx", line 432, in rawpy._rawpy.RawPy.unpack
  File "rawpy\_rawpy.pyx", line 936, in rawpy._rawpy.RawPy.handle_error
rawpy._rawpy.LibRawTooBigError: b'Image too big for processing'

Is there either an alternative to convert the DNG files, or a way to bypass this error?

Info from exiftool:

ExifTool Version Number         : 11.88

File Name                       : DSCF0001.DNG

Directory                       : .

File Size                       : 1313 MB

File Modification Date/Time     : 2022:08:24 12:06:31 01:00

File Access Date/Time           : 2022:08:25 08:17:15 01:00

File Inode Change Date/Time     : 2022:08:24 15:19:04 01:00

File Permissions                : rwxrwxrwx

File Type                       : DNG

File Type Extension             : dng

MIME Type                       : image/x-adobe-dng

Exif Byte Order                 : Little-endian (Intel, II)

Make                            : FUJIFILM

Camera Model Name               : GFX 100

Preview Image Start             : 115208860

Orientation                     : Horizontal (normal)

Rows Per Strip                  : 3000

Preview Image Length            : 3011337

Software                        : FUJIFILM Pixel Shift Combiner 1.2.0.2 (Real Color   High Resolution mode)

Modify Date                     : 2022:08:24 12:06:29

Artist                          : 

Subfile Type                    : Full-resolution image

Image Width                     : 23296

Image Height                    : 17472

Bits Per Sample                 : 16 16 16

Compression                     : JPEG

Photometric Interpretation      : Linear Raw

Samples Per Pixel               : 3

Planar Configuration            : Chunky

Tile Width                      : 128

Tile Length                     : 96

Tile Offsets                    : (Binary data 341026 bytes, use -b option to extract)

Tile Byte Counts                : (Binary data 198743 bytes, use -b option to extract)

Black Level                     : 256 255 256

White Level                     : 65535 65535 65535

Default Scale                   : 1 1

Default Crop Origin             : 16 12

Default Crop Size               : 23264 17448

Anti Alias Strength             : 1

Best Quality Scale              : 1

Opcode List 3                   : WarpRectilinear, FixVignetteRadial

Rating                          : 0

Copyright                       : 

Exposure Time                   : 1/125

F Number                        : 8.0

Exposure Program                : Manual

ISO                             : 100

Sensitivity Type                : Standard Output Sensitivity

Standard Output Sensitivity     : 100

Exif Version                    : 0230

Date/Time Original              : 2019:03:10 00:44:16

Create Date                     : 2019:03:10 00:44:16

Shutter Speed Value             : 1/125

Aperture Value                  : 8.0

Brightness Value                : 8.57

Exposure Compensation           : 0

Max Aperture Value              : 2.0

Metering Mode                   : Multi-segment

Light Source                    : Unknown

Flash                           : No Flash

Focal Length                    : 110.0 mm

Version                         : 0130

Internal Serial Number          : 

Quality                         : NORMAL

White Balance                   : Auto

Saturation                      : 0 (normal)

White Balance Fine Tune         : Red  0, Blue  0

Noise Reduction                 : 0 (normal)

Fuji Flash Mode                 : Manual

Flash Exposure Comp             : 0

Focus Mode                      : Manual

AF Mode                         : No

Focus Pixel                     : 2001 1501

AF-S Priority                   : Release

AF-C Priority                   : Release

Focus Mode 2                    : AF-M

AF Area Mode                    : Single Point

AF Area Point Size              : n/a

AF Area Zone Size               : n/a

AF-C Setting                    : Set 1 (multi-purpose)

AF-C Tracking Sensitivity       : 2

AF-C Speed Tracking Sensitivity : 0

AF-C Zone Area Switching        : Auto

Slow Sync                       : Off

Picture Mode                    : Manual

Exposure Count                  : 1

Shadow Tone                     : 0 (normal)

Highlight Tone                  : 0 (normal)

Lens Modulation Optimizer       : On

Grain Effect                    : Off

Color Chrome Effect             : Off

Crop Mode                       : n/a

Color Chrome FX Blue            : Off

Shutter Type                    : Electronic

Auto Bracketing                 : Unknown (6)

Sequence Number                 : 1

Drive Mode                      : Single

Drive Speed                     : n/a

Blur Warning                    : None

Focus Warning                   : Good

Exposure Warning                : Good

Dynamic Range                   : Standard

Film Mode                       : F0/Standard (Provia)

Dynamic Range Setting           : Manual

Development Dynamic Range       : 100

Min Focal Length                : 110

Max Focal Length                : 110

Max Aperture At Min Focal       : 2

Max Aperture At Max Focal       : 2

Image Stabilization             : Sensor-shift; Off; 0

Image Generation                : Original Image

Image Count                     : 34

Flicker Reduction               : Off (0x0002)

Faces Detected                  : 0

Num Face Elements               : 0

Color Space                     : Uncalibrated

Focal Plane X Resolution        : 5320

Focal Plane Y Resolution        : 5320

Focal Plane Resolution Unit     : cm

File Source                     : Digital Camera

Scene Type                      : Directly photographed

Custom Rendered                 : Normal

Exposure Mode                   : Auto

Focal Length In 35mm Format     : 87 mm

Scene Capture Type              : Standard

Sharpness                       : Unknown (3)

Subject Distance Range          : Unknown (48)

Serial Number                   : 

Lens Info                       : 110mm f/2

Lens Make                       : FUJIFILM

Lens Model                      : GF110mmF2 R LM WR

Lens Serial Number              : 

DNG Version                     : 1.4.0.0

DNG Backward Version            : 1.1.0.0

Unique Camera Model             : FUJIFILM GFX 100

Color Matrix 1                  : 1.7191 -1.1 0.1278 -0.3574 1.1733 0.2076 -0.0002 0.0497 0.654

Color Matrix 2                  : 1.6212 -0.8423 -0.1583 -0.4336 1.2583 0.1937 -0.0195 0.0726 0.6199

Analog Balance                  : 1 1 1

As Shot Neutral                 : 0.5644 1 0.5153

Baseline Exposure               : -0.01

Baseline Noise                  : 1

Baseline Sharpness              : 1.33

Linear Response Limit           : 1

Camera Serial Number            : 

DNG Lens Info                   : 110mm f/2

Shadow Scale                    : 1

DNG Private Data                : (Binary data 114927728 bytes, use -b option to extract)

Calibration Illuminant 1        : Standard Light A

Calibration Illuminant 2        : D65

Aperture                        : 8.0

Image Size                      : 23296x17472

Megapixels                      : 407.0

Preview Image                   : (Binary data 3011337 bytes, use -b option to extract)

Scale Factor To 35 mm Equivalent: 0.8

Shutter Speed                   : 1/125

Circle Of Confusion             : 0.038 mm

Field Of View                   : 23.4 deg

Focal Length                    : 110.0 mm (35 mm equivalent: 87.0 mm)

Hyperfocal Distance             : 39.81 m

Light Value                     : 13.0

CodePudding user response:

There seems to be a limit of 2GB for the fully expanded in memory image. I don't mean the space your DNG requires on disk, I mean the following number:

ImageHeight * ImageWidth * NumberOfChannels * BytesPerSample

So it would be useful if you used exiftool to tell us those parameters, by clicking edit under your question and adding the output from:

exiftool YOURIMAGE.DNG

The limit is enforced in the variable imgdata.params.max_raw_memory_mb which is set to 2048 in the underlying libraw C code. I am not immediately sure how you could set that through the Python interface.


You may have some success converting it to a PNG or a TIFF with ImageMagick as follows:

magick YOURIMAGE.DNG converted.png

If that works, you could use wand which is a Python binding to Imagemgaick.


You could try with ufraw maybe, along the lines of:

ufraw-batch --out-type=png --out-depth=16 --output=result.png YOURIMAGE.dng
  • Related