Gstreamer Android HW accelerated H.264 encoding-CodePudding

I'm working on a robot that streams two camera streams using Gstreamer from a Jetson Nano over UDP to an Android device.

At this point, I'm taking one of the streams and trying to encode the video to display on an Android device. My Gstreamer pipeline looks like this:

rtspsrc location=rtsp://192.168.1.239:8554/test ! application/x-rtp, payload=96 ! rtph264depay ! amcviddec-omxexynosavcdec ! videoconvert ! autovideosink sync=false"

On the phone I'm using pre-built binaries of Gstreamer and have an implementation very similiar to this.

From MediaCodecList Android API, I've managed to find out, that my device has HW acceleration for OMX.Exynos.avc.dec so thats why I'm using it in that pipeline.

Now, when I run the app, the native lib gives me this error alongside with some more errors of the "same" type:

2022-02-15 12:25:13.570 6377-6377/cz.walle.wallecontroller E/GStreamer ahc: 0:00:00.095861231 0x7589b71200 ../sys/androidmedia/gst-android-hardware-camera.c:1777:_init_classes Failed to initialize android.hardware.Camera classes: Failed to get static field ID EFFECT_EMBOSS (Ljava/lang/String;): java.lang.NoSuchFieldError: no "Ljava/lang/String;" field "EFFECT_EMBOSS" in class "Landroid/hardware/Camera$Parameters;" or its superclasses
    java.lang.NoSuchFieldError: no "Ljava/lang/String;" field "EFFECT_EMBOSS" in class "Landroid/hardware/Camera$Parameters;" or its superclasses
        at org.freedesktop.gstreamer.GStreamer.nativeInit(Native Method)
        at org.freedesktop.gstreamer.GStreamer.init(GStreamer.java:18)
        at cz.walle.wallecontroller.ControlActivity.onCreate(ControlActivity.java:41)
        at android.app.Activity.performCreate(Activity.java:8282)
        at android.app.Activity.performCreate(Activity.java:8262)
        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1329)
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:4005)
        at android.app.ActivityThread.ha
2022-02-15 12:25:13.570 6377-6377/cz.walle.wallecontroller E/GStreamer ahs: 0:00:00.096544154 0x7589b71200 ../sys/androidmedia/gst-android-hardware-sensor.c:596:_init_classes Failed to initialize Android classes: Failed to call Java method: java.lang.ClassNotFoundException: Didn't find class "org/freedesktop/gstreamer/androidmedia/GstAhsCallback" on path: DexPathList[[zip file "/data/app/~~cU9DGdEFi5D9pOGi5u1pLw==/cz.walle.wallecontroller-nq-kqx36s-N7ImViW2Lwrw==/base.apk"],nativeLibraryDirectories=[/data/app/~~cU9DGdEFi5D9pOGi5u1pLw==/cz.walle.wallecontroller-nq-kqx36s-N7ImViW2Lwrw==/lib/arm64, /data/app/~~cU9DGdEFi5D9pOGi5u1pLw==/cz.walle.wallecontroller-nq-kqx36s-N7ImViW2Lwrw==/base.apk!/lib/arm64-v8a, /system/lib64, /system/system_ext/lib64]]
    java.lang.ClassNotFoundException: Didn't find class "org/freedesktop/gstreamer/androidmedia/GstAhsCallback" on path: DexPathList[[zip file "/data/app/~~cU9DGdEFi5D9pOGi5u1pLw==/cz.walle.wallecontroller-nq-kqx36s-N7ImViW2Lwrw==/base.apk"],nativeLibraryDirectories=[/data/app/~~cU9DGdEFi5D9pOGi5u1pLw==/cz.walle.wallecontroller-nq-kqx36s-N7ImVi

The entire stacktrace is here.

Anyone has experience with this? When I try to use different encoder like avdec_h264 the stream works, but one CPU core can't keep up with encoding.

CodePudding user response：

You would use decodebin that would select the decoder according to a rank built into plugins. That should select the most efficient one available for decoding:

rtspsrc location=rtsp://192.168.1.239:8554/test latency=300 ! application/x-rtp,encoding-name=H264 ! decodebin ! autovideosink

Note that the optimal latency may not be acheived with the lowest value. It depends on stream-format and network available bandwidth. Just try various values running for quite long duration and see.