I'm trying to compare predictions from different MLModels in SwiftUI. To do that I have to switch between them, but can't because every ML variable has its own class, so I get the error:
Cannot assign value of type 'ModelOne' to type 'ModelTwo'
Here's an example code:
import Foundation
import CoreML
import SwiftUI
let modelone = { //declaration model 1
do {
let config = MLModelConfiguration()
return try ModelOne(configuration: config)
} catch {
/*...*/
}
}()
let modeltwo = { //declaration model 2
do {
let config = MLModelConfiguration()
return try ModelTwo(configuration: config)
} catch {
/*...*/
}
}()
var imageused : UIImage! //image to classify
var modelstring = "" //string of model user chosen
var modelchosen = modelone
Button(action: { //button user decide to use model two
modelstring = "Model Two"
}) {/*...*/}
/*...*/
func classifyphoto() {
guard let image = imageused as UIImage?,
let imagebuffer = image.convertToBuffer() else {
return
}
if modelstring == "Model Two" { //if the user chosen model two, use ModelTwo
modelchosen = modeltwo // Error: Cannot assign value of type 'ModelOne' to type 'ModelTwo'
} else {
modelchosen = modelone}
let output = try? modelchosen.prediction(image: imagebuffer) //prediction with model chosen
if let output = output {
let results = output.classLabelProbs.sorted { $0.1 > $1.1 }
_ = results.map { /*...*/
}
}
}
Thank you!
CodePudding user response：
The issue is that the two model classes do not have a common class or common inherited class. There are several ways to implement what you want. I think this is the best way based on your example.
class MyModel {
var model: MLModel? = nil
init(modelName: String) {
let bundle = Bundle.main
if let modelURL = bundle.url(forResource: modelName, withExtension:"mlmodelc") {
do {
self.model = try MLModel(contentsOf: modelURL)
}
catch {
print("Unable to open MLModel: \(error)")
}
}
}
}
class TestModel {
class func testModels() {
let modelOne = MyModel(modelName: "ModelOne")
let modelTwo = MyModel(modelName: "ModelTwo")
var selectedModel = modelOne
selectedModel = modelTwo
}
}
CodePudding user response：
Swift is a statically typed language which means that in the general case you cannot assign a variable of one type to a variable of another type:
var int: Int = 42
int = "Hello, world!" // Not allowed: cannot assign String to Int
The problem is that modelchosen is of type ModelOne since it is initialized with modelone, thus, you cannot later assign modeltwo to it as you are trying to do.
To make that working, you have first to identify the common capabilities of ModelOne and ModelTwo. Take a look at their definition. For instance, do their .predict(image:) method return the same type? It looks like you are trying to do image classification, so a common capability could be the capability to return a String describing the image (or a list of potential objects, etc.).
When you'll have identified the common capability, you could define an abstract base class (or a protocol) describing the common interface of your networks. for instance, if the two networks can both be initialized with a MLModelConfiuration and are used for classification, then:
class MLClassifier {
init(from config: MLModelConfig) {
fatalError("not implemented")
}
func classify(image: ImageBuffer) -> String {
fatalError("not implemented")
}
}
You can then derive this base class with your two models:
final class ModelOne: MLClassifier {
init(from config: MLModelConfig) {
// the specific implementation for `ModelOne`...
}
func classify(image: ImageBuffer) -> String {
// the specific implementation for `ModelOne`..
}
}
Then, in your could make the variable modelchosen to be of type MLClassifier to erase the underlying concrete type of the model:
var modelchosen: MLClassifier = ModelOne(from: config1)
As MLClassifier is a common base class for both ModelOne and ModelTwo you can dynamically change the type of modelchosen whenever you need:
// Later...
modelchosen = ModelTwo(from: config2)
The variable modelchosen being of type MLClassifier ensures that you can call the .classify(image:) method whatever the concrete model type is:
func classifyphoto() {
guard let image = imageused as UIImage?,
let imagebuffer = image.convertToBuffer() else {
return
}
let output = modelchosen.classify(image: imageBuffer)
// Update the UI...
}
To sum up, the key is to identify the common capabilities of the different types. For instance, if the two models output at some point a classLabelProbs of the same type, then you could use this as the common abstraction.
I wrote this answer using class inheritance as the abstraction mechanism but there are two other methods: protocols and enums with payloads, protocols being the preferred way:
protocol MLClassifier {
init(from config: MLModelConfig)
func classify(image: ImageBuffer) -> String
}
// Implement the protocol for your models
struct ModelOne: MLClassifier {
init(from config: MLModelConfig) { ... }
func classify(image: ImageBuffer) -> String { ... }
}
// Store an instance of any `MLClassfier` using an existential
var classifier: any MLClassifier = ModelOne(from: config1)
// Later...
classifier = ModelTwo(from: config2)
As a last resort, you could wrap everything in a big if-else statement, event though it is not recommended since it is not very readable, is not a good way to encapsulate common behavior and leads to a lot of code repetition:
func classifyphoto() {
guard let image = imageused as UIImage?,
let imagebuffer = image.convertToBuffer() else {
return
}
if modelstring == "Model Two" {
// Use modeltwo
let output = try? modeltwo.prediction(image: imagebuffer)
if let output = output {
let results = output.classLabelProbs.sorted { $0.1 > $1.1 }
_ = results.map { /*...*/ }
} else {
// Use modelone
let output = try? modelone.prediction(image: imagebuffer)
if let output = output {
let results = output.classLabelProbs.sorted { $0.1 > $1.1 }
_ = results.map { /*...*/ }
}
}