Home > Enterprise >  Swift iOS - Vision framework text recognition and rectangles
Swift iOS - Vision framework text recognition and rectangles

Time:08-19

I was trying to draw rectangles on the text area found using the Vision framework but they are always a little bit off. I am doing it like this:

    public func drawOccurrencesOnImage(_ occurrences: [CGRect], _ image: UIImage) -> UIImage? {

    UIGraphicsBeginImageContextWithOptions(image.size, false, 0.0)

    image.draw(at: CGPoint.zero)
    let currentContext = UIGraphicsGetCurrentContext()

    currentContext?.addRects(occurrences)
    currentContext?.setStrokeColor(UIColor.red.cgColor)
    currentContext?.setLineWidth(2.0)
    currentContext?.strokePath()

    guard let drawnImage = UIGraphicsGetImageFromCurrentImageContext() else { return UIImage() }

    UIGraphicsEndImageContext()
    return drawnImage
}

But the image returned always looks almost, but not really correct: first image

second image

third image

This is how I create the boxes, exactly the same way as Apple:

        let boundingRects: [CGRect] = observations.compactMap { observation in

        guard let candidate = observation.topCandidates(1).first else { return .zero }

        let stringRange = candidate.string.startIndex..<candidate.string.endIndex
        let boxObservation = try? candidate.boundingBox(for: stringRange)

        let boundingBox = boxObservation?.boundingBox ?? .zero

        return VNImageRectForNormalizedRect(boundingBox,
                                            Int(UIViewController.chosenImage?.width ?? 0),
                                            Int(UIViewController.chosenImage?.height ?? 0))
    }

(source: enter image description here


E.g.

let request = VNRecognizeTextRequest { [self] request, error in
    guard
        let results = request.results as? [VNRecognizedTextObservation],
        error == nil
    else { … }

    let rects = results.map {
        convert(boundingBox: $0.boundingBox, to: CGRect(origin: .zero, size: size))
    }

    let format = UIGraphicsImageRendererFormat()
    format.scale = 1
    let finalImage = UIGraphicsImageRenderer(bounds: bounds, format: format).image { _ in
        image.draw(in: bounds)
        UIColor.red.setStroke()
        for rect in rects {
            let path = UIBezierPath(rect: rect)
            path.lineWidth = 5
            path.stroke()
        }
    }

    DispatchQueue.main.async { [self] in
        imageView.image = finalImage
    }
}
  • Related