Following

It is possible to avoid the allocation of the two objects by passing the myInteger as an argument, instead of relying on the convenience of captured variables and closures. Here is how:

using System;

class Program
{
    static bool logIsEnabled;

    static void Main()
    {
        logIsEnabled = true;
        int myInteger = 13;
        Log(arg => "foo("   arg   ")", myInteger);
    }

    static void Log<TArg>(Func<TArg, string> msg, TArg arg)
    {
        if (logIsEnabled)
        {
            Console.WriteLine(msg(arg));
        }
    }
}

SharpLab output (sanitized):

using System;
using System.Runtime.CompilerServices;

internal class Program
{
    [Serializable]
    [CompilerGenerated]
    private sealed class C
    {
        public static readonly C singleton = new C();

        public static Func<int, string> lambda;

        internal string M(int arg)
        {
            return string.Concat("foo(", arg.ToString(), ")");
        }
    }

    private static bool logIsEnabled;

    private static void Main()
    {
        logIsEnabled = true;
        int arg = 13;
        Log(C.lambda ?? (C.lambda = new Func<int, string>(C.singleton.M)), arg);
    }

    private static void Log<TArg>(Func<TArg, string> msg, TArg arg)
    {
        if (logIsEnabled)
        {
            Console.WriteLine(msg(arg));
        }
    }
}

Now the compiler generated a singleton (the C class), and the Func<TArg, string> is instantiated only once per TArg type. So if your program uses the Log<TArg> with ints, strings and decimals, only a Func<int, string>, a Func<string, string> and a Func<decimal, string> will be created in total, irrespective of how many times the Log<TArg> will be invoked.

In case you want to pass more than one arguments to the Log method, you'll have to write additional Log<TArg1, TArg2>, Log<TArg1, TArg2, TArg3> etc overloads.