Weird 'Native linking failed, undefined Objective-C class' when Xamarin-iOS nuget is gener-CodePudding

I have Xamarin-iOS binding-project which upon being built generates a nuget. Said nuget works as intended in Xamarin-iOS applications if and only if I built it on my Mac.

However, when I build this nuget via Azure Pipelines using MacOS-12 as the host ( iphone16.2 sdk sharpie 3.5.61 clang-1400.0.29.202 exactly as in my localdev Mac) even though the build succeeds in generating the nuget it is poisoned in the sense that upon trying to build a Xamarin Application with it I get the following errors:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -framework CoreFoundation -framework Security -framework VisionKit -framework UserNotificationsUI -framework UniformTypeIdentifiers -framework ThreadNetwork -framework WatchConnectivity [...] -u _BrotliEncoderHasMoreOutput -u _BrotliEncoderDestroyInstance -u _BrotliEncoderCompress -u _mono_pmip

Undefined symbols for architecture arm64:
  "_OBJC_CLASS_$__TtC17McuMgrBindingsiOS17IOSDeviceResetter", referenced from:
      objc-class-ref in registrar.o
  "_OBJC_CLASS_$__TtC17McuMgrBindingsiOS17IOSFirmwareEraser", referenced from:
      objc-class-ref in registrar.o
  "_OBJC_CLASS_$__TtC17McuMgrBindingsiOS19IOSFirmwareUpgrader", referenced from:
      objc-class-ref in registrar.o

ld: symbol(s) not found for architecture arm64

I have inspected the generated dll that lives inside both nugets and the symbols 'TtC17McuMgrBindingsiOS17IOSDeviceResetter', 'TtC17McuMgrBindingsiOS17IOSFirmwareEraser' and 'TtC17McuMgrBindingsiOS19IOSFirmwareUpgrader' do indeed exist on both the local and the azure nuget.

The azure pipeline hosted on MacOS-12 seems to be employing Mono ver. 16.10.1 for the build which is exactly what my localdev has.

I noticed that 'clang' in Azure targets x86 instead of arm64 - maybe this relates to the error observed somehow?

(localdev)
Apple clang version 14.0.0 (clang-1400.0.29.202)
Target: arm64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

(azure)
Apple clang version 14.0.0 (clang-1400.0.29.102)
Target: x86_64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

The build script I'm using to invoke xcodebuild, sharpie and lipo is this one:

#!/usr/bin/env bash

# Builds a fat library for a given xcode project (framework)
#
# Derived from https://github.com/xamcat/xamarin-binding-swift-framework/blob/master/Swift/Scripts/build.fat.sh#L3-L14

IOS_SDK_VERSION="${IOS_SDK_VERSION:-16.2}" # xcodebuild -showsdks

SWIFT_PROJECT_NAME="McuMgrBindingsiOS"
SWIFT_BUILD_PATH="./$SWIFT_PROJECT_NAME/build"
SWIFT_OUTPUT_PATH="./VendorFrameworks/swift-framework-proxy"
SWIFT_BUILD_SCHEME="McuMgrBindingsiOS"
SWIFT_PROJECT_PATH="./$SWIFT_PROJECT_NAME/$SWIFT_PROJECT_NAME.xcodeproj"
SWIFT_PACKAGES_PATH="./packages"
SWIFT_BUILD_CONFIGURATION="Release"

XAMARIN_BINDING_PATH="Xamarin/SwiftFrameworkProxy.Binding"

function print_macos_sdks() {
  xcodebuild -showsdks
}

function build() {
  echo "** Build iOS framework for simulator and device"

  echo "**** (Build 1/5) Cleanup any possible traces of previous builds"

  rm -Rf "$SWIFT_BUILD_PATH"
  rm -Rf "$SWIFT_PACKAGES_PATH"
  rm -Rf "$XAMARIN_BINDING_PATH"

  echo "**** (Build 2/5) Restore packages for 'iphoneos$IOS_SDK_VERSION'"

  xcodebuild \
    -sdk "iphoneos$IOS_SDK_VERSION" \
    -arch arm64 \
    -scheme "$SWIFT_BUILD_SCHEME" \
    -project "$SWIFT_PROJECT_PATH" \
    -configuration "$SWIFT_BUILD_CONFIGURATION" \
    -clonedSourcePackagesDirPath "$SWIFT_PACKAGES_PATH" \
    -resolvePackageDependencies

  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to download dependencies for 'iphoneos$IOS_SDK_VERSION'"
    exit 1
  fi

  echo "**** (Build 3/5) Build for 'iphoneos$IOS_SDK_VERSION'"

  # https://stackoverflow.com/a/74478244/863651
  xcodebuild \
    -sdk "iphoneos$IOS_SDK_VERSION" \
    -arch arm64 \
    -scheme "$SWIFT_BUILD_SCHEME" \
    -project "$SWIFT_PROJECT_PATH" \
    -configuration "$SWIFT_BUILD_CONFIGURATION" \
    -derivedDataPath "$SWIFT_BUILD_PATH" \
    -clonedSourcePackagesDirPath "$SWIFT_PACKAGES_PATH" \
    CODE_SIGN_IDENTITY="" \
    CODE_SIGNING_ALLOWED=NO \
    CODE_SIGNING_REQUIRED=NO

  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to build 'iphoneos$IOS_SDK_VERSION'"
    exit 1
  fi

  echo "**** (Build 4/5) Restore packages for 'iphonesimulator$IOS_SDK_VERSION'"

  xcodebuild \
    -sdk "iphonesimulator$IOS_SDK_VERSION" \
    -arch arm64 \
    -scheme "$SWIFT_BUILD_SCHEME" \
    -project "$SWIFT_PROJECT_PATH" \
    -configuration "$SWIFT_BUILD_CONFIGURATION" \
    -clonedSourcePackagesDirPath "$SWIFT_PACKAGES_PATH" \
    -resolvePackageDependencies

  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to download dependencies for 'iphonesimulator$IOS_SDK_VERSION'"
    exit 1
  fi

  echo "**** (Build 5/5) Build for 'iphonesimulator$IOS_SDK_VERSION'"

  # https://stackoverflow.com/a/74478244/863651
  # https://stackoverflow.com/a/64026089/863651
  xcodebuild \
    -sdk "iphonesimulator$IOS_SDK_VERSION" \
    -scheme "$SWIFT_BUILD_SCHEME" \
    -project "$SWIFT_PROJECT_PATH" \
    -configuration "$SWIFT_BUILD_CONFIGURATION" \
    -derivedDataPath "$SWIFT_BUILD_PATH" \
    -clonedSourcePackagesDirPath "$SWIFT_PACKAGES_PATH" \
    EXCLUDED_ARCHS="arm64" \
    CODE_SIGN_IDENTITY="" \
    CODE_SIGNING_ALLOWED=NO \
    CODE_SIGNING_REQUIRED=NO

  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to build 'iphonesimulator$IOS_SDK_VERSION'"
    exit 1
  fi
}

function create_fat_binaries() {
  echo "** Create fat binaries for Release-iphoneos and Release-iphonesimulator configuration"

  echo "**** (FatBinaries 1/8) Copy one build as a fat framework"

  cp \
    -R \
    "$SWIFT_BUILD_PATH/Build/Products/Release-iphoneos" \
    "$SWIFT_BUILD_PATH/Release-fat"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to copy"
    exit 1
  fi

  echo "**** (FatBinaries 2/8) Combine modules from another build with the fat framework modules"

  cp \
    -R \
    "$SWIFT_BUILD_PATH/Build/Products/Release-iphonesimulator/$SWIFT_PROJECT_NAME.framework/Modules/$SWIFT_PROJECT_NAME.swiftmodule/" \
    "$SWIFT_BUILD_PATH/Release-fat/$SWIFT_PROJECT_NAME.framework/Modules/$SWIFT_PROJECT_NAME.swiftmodule/"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to copy"
    exit 1
  fi

  echo "**** (FatBinaries 3/8) Combine iphoneos   iphonesimulator configuration as fat libraries"

  lipo \
    -create \
    -output "$SWIFT_BUILD_PATH/Release-fat/$SWIFT_PROJECT_NAME.framework/$SWIFT_PROJECT_NAME" \
    "$SWIFT_BUILD_PATH/Build/Products/Release-iphoneos/$SWIFT_PROJECT_NAME.framework/$SWIFT_PROJECT_NAME" \
    "$SWIFT_BUILD_PATH/Build/Products/Release-iphonesimulator/$SWIFT_PROJECT_NAME.framework/$SWIFT_PROJECT_NAME"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to combine configurations"
    exit 1
  fi

  echo "**** (FatBinaries 4/8) Verify results"
  lipo \
    -info \
    "$SWIFT_BUILD_PATH/Release-fat/$SWIFT_PROJECT_NAME.framework/$SWIFT_PROJECT_NAME"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to verify results"
    exit 1
  fi

  echo "**** (FatBinaries 5/8) Copy fat frameworks to the output folder"

  rm -Rf "$SWIFT_OUTPUT_PATH" &&
    mkdir -p "$SWIFT_OUTPUT_PATH" &&
    cp -Rf \
      "$SWIFT_BUILD_PATH/Release-fat/$SWIFT_PROJECT_NAME.framework" \
      "$SWIFT_OUTPUT_PATH"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to copy fat frameworks"
    exit 1
  fi

  echo "**** (FatBinaries 6/8) Generating binding api definition and structs"
  sharpie \
    bind \
    --sdk="iphoneos$IOS_SDK_VERSION" \
    --scope="$SWIFT_OUTPUT_PATH/$SWIFT_PROJECT_NAME.framework/Headers/" \
    --output="$SWIFT_OUTPUT_PATH/XamarinApiDef" \
    --namespace="$SWIFT_PROJECT_NAME" \
    "$SWIFT_OUTPUT_PATH/$SWIFT_PROJECT_NAME.framework/Headers/$SWIFT_PROJECT_NAME-Swift.h"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to generate binding api definitions and structs"
    exit 1
  fi

  echo "**** (FatBinaries 7/8) Replace existing metadata with the updated"

  mkdir -p "$XAMARIN_BINDING_PATH/" &&
    cp \
      -Rf \
      "$SWIFT_OUTPUT_PATH/XamarinApiDef/." \
      "$XAMARIN_BINDING_PATH/"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to replace existing metadata with the updated"
    exit 1
  fi

  echo "**** (FatBinaries 8/8) Replace NativeHandle -> IntPtr in the generated c# files"

  # replace nativehandle -> intptr
  find \
    "$XAMARIN_BINDING_PATH/" \
    -type f \
    -exec sed -i.bak "s/NativeHandle[ ]/IntPtr /gi" {} \;

  # also need to get rid of stupid autogenerated [verify(...)] attributes which are intentionally placed there
  # by sharpie to force manual verification of the .cs files that have been autogenerated
  #
  # https://learn.microsoft.com/en-us/xamarin/cross-platform/macios/binding/objective-sharpie/platform/verify
  find \
    "$XAMARIN_BINDING_PATH/" \
    -type f \
    -exec sed -i.bak 's/\[Verify\s*\(.*\)\]//gi' {} \;

  # adding [model] to the interfaces seems to be mandatory for the azure pipelines to generate a valid nuget for ios   if we
  # omit adding this attribute then the nuget generated by the azure pipelines gets poisoned and it causes a very cryptic runtime error
  # so I'm not 100% sure why the [model] attribute does away with the observed error but it does the trick of solving the problem somehow
  #
  #  find \
  #    "$XAMARIN_BINDING_PATH/" \
  #    -type f \
  #    -exec sed -i.bak 's/interface IOSDeviceResetter/[Model] interface IOSDeviceResetter/gi' {} \;
  #  find \
  #    "$XAMARIN_BINDING_PATH/" \
  #    -type f \
  #    -exec sed -i.bak 's/interface IOSFirmwareEraser/[Model] interface IOSFirmwareEraser/gi' {} \;
  #  find \
  #    "$XAMARIN_BINDING_PATH/" \
  #    -type f \
  #    -exec sed -i.bak 's/interface IOSFirmwareUpgrader/[Model] interface IOSFirmwareUpgrader/gi' {} \;

  # https://stackoverflow.com/a/49477937/863651   its vital to add [BaseType] to the interface otherwise compilation will fail
  find \
    "$XAMARIN_BINDING_PATH/" \
    -type f \
    -exec sed -i.bak 's/interface IOSListenerForDeviceResetter/[BaseType(typeof(NSObject))] [Model] interface IOSListenerForDeviceResetter/gi' {} \;
  find \
    "$XAMARIN_BINDING_PATH/" \
    -type f \
    -exec sed -i.bak 's/interface IOSListenerForFirmwareEraser/[BaseType(typeof(NSObject))] [Model] interface IOSListenerForFirmwareEraser/gi' {} \;
  find \
    "$XAMARIN_BINDING_PATH/" \
    -type f \
    -exec sed -i.bak 's/interface IOSListenerForFirmwareUpgrader/[BaseType(typeof(NSObject))] [Model] interface IOSListenerForFirmwareUpgrader/gi' {} \;
}

function main() {
  print_macos_sdks
  build
  create_fat_binaries

  echo "** Done!"
}

main "$@"

I can provide you with the working and the non-working nugets for you to compare them yourselves if you want - maybe a more experienced pair of eyes can spot something that I can't.

PS: I tried adding [Protocol] before each generated interface in 'ApiDefinition.cs' but even though this solved the original problem it also caused another problem:

Upon attempting to invoke any of the methods of the instantiated class I now get an exception ala 'Foundation.You_Should_Not_Call_base_In_This_Method'

CodePudding user response：

I found out what was wrong.

It turned out that the swift compiler in the Azure Pipelines is acting like a smartass and it strips any and all public classes that are meant to be exported if they're not being used inside the package (stupid as hell but there you have it folks).

All I did was add a dummy swift-class to 'convince' the compiler that these public classes should not be stripped:

import Foundation

// to future maintainers     keep this dummy class around so as to have it reference the exported classes
// to future maintainers
// to future maintainers     omitting this one causes the build environment of the azure pipelines to go completely smart-assinine and strip the
// to future maintainers     public classes thinking that they are not being used anywhere

public class DummyPlaceholder {
    public func Foobar() {
        let _ = SomeClassHere(nil)
        let _ = SomeOtherClassHere(nil, nil)
        ...
    }
}