Not able to modify C object passed as parameter to Lua function-CodePudding

I have been working on SDL2 2D Game Engine for several years now. Just ditched inheritance approach to define game entities with composition approach where I have Entity class and it has vector of Component classes and recently I got into lua, because I want to define entities using Lua table with optional callback functions.

Working parts

I am using Lua5.4 and C API to bind some engine methods and Entity class to Lua. I use XML file to load list of scripts for each Entity defined by Lua:

  <script name="player" filename="scripts/player.lua" type="entity"/>

Then Entity gets created in C with ScriptComponent which holds a pointer to Lua state. Lua file gets loaded at this point and state is not closed unless Entity is destroyed. player.lua script might look something like this:

  -- Entity
player = {
    -- Entity components
    transform = {
         X = 100,
         Y = 250
    },
    physics = {
        mass = 1.0,
        friction = 0.2
    },
    sprite = {
        id = "player",
        animation = {},
        width = 48,
        height = 48
    },
    collider = {
        type = "player",
        onCollide = function(this, second)
            print("Lua: onCollide() listener called!")
        end
    },
    HP = 100
}

Using this I managed to create each Component class using Lua C API with no issues. Also while loading this I detect and set "onCollide" function in Lua.

Also I have managed to register some Engine functions so I can call them to lua: playSound("jump") in C :

static int lua_playSound(lua_State *L) {
    std::string soundID = (std::string)lua_tostring(L, 1);
    TheSoundManager::Instance()->playSound(soundID, 0);
    return 0;
}

Also have created meta table for Entity class with __index and __gc metamethods and it works if I call these methods with Entity created in Lua outside of player table, such as:

-- This goes in player.lua script after the main table
testEntity = Entity.create() -- works fine, but entity is created in Lua
testEntity:move(400, 400)
testEntity:scale(2, 2)
testEntity:addSprite("slime", "assets/sprite/slime.png", 32, 32)

Problem

Now whenever collision happens and Entity has ScriptComponent, it correctly calls onCollide method in Lua. Even playSound method inside triggers correctly. Problem is when I try to manipulate Entities which are passed as this and seconds arguments to onCollide

onCollide = function(this, second)
        print(type(this)) -- userdata
        print(type(second)) --userdata
        --Entity.scale(this, 10, 10) --segfault
        --this:scale(10, 10) --segfault
        playSound("jump") -- works fine, does not need any metatables
    end

This is how I am calling onCollide method and passing existing C object to Lua:

// This is found in a method which belongs to ScriptComponent class, it holds lua state
// owner is Entity*, all Components have this
// second is also Entity*
if (lua_isfunction(state, -1)) {
    void* self = (Entity*)lua_newuserdata(state, sizeof(Entity));
    self = owner;

    luaL_getmetatable(state, "EntityMetaTable");
    assert(lua_isuserdata(state, -2));
    assert(lua_istable(state, -1));
    lua_setmetatable(state, -2);
    assert(lua_isuserdata(state, -1));

    void* second = (Entity*)lua_newuserdata(state, sizeof(Entity));
    second = entity;
                            
    luaL_getmetatable(state, "EntityMetaTable");
    lua_setmetatable(state, -2);

    // Code always reaches cout statement below unless I try to manipulate Entity
    // objects passed to Lua in Lua                     
    if (luaOk(state, lua_pcall(state, 2, 0, 0))) {
        std::cout << "onCollide() Called sucessfully!!!" << std::endl;
    }
    script->clean(); // Cleans lua stack
    return;

}

So basically I have managed to load data from table, bind and use some methods from C engine and mapped Entity class using metatable and __index and __gc meta methods which work fine for objects created in Lua but not when I try to pass existing C object and set existing meta table.

I still think I will be alright without using any Lua binders, because all I wanted here is to load data for all Components which works fine and script some behaviour based on events which also almost works except for not being able to correctly pass existing C object to onCollide method. Thank you for your help!

CodePudding user response：

It is impossible to "pass an existing C object to Lua". You would either have to copy it to the storage you get with lua_newuserdata, or use lua_newuserdata from the beginning.

Depending on what you can guarantee about the lifetime of the entity, you have several options:

If the lifetime of every entity aligns with the lifetime of the Lua instance, you can let Lua manage the entity object completely through its GC, i.e. call lua_newuserdata and use the "placement new" operator on the resulting object, initializing the entity. Note however that you have to prevent the object from getting freed by the Lua GC somehow if you only use it outside Lua, by adding it to the registry table for example.
If the lifetime of every entity exceeds the lifetime of the Lua instance, you can use *static_cast<Entity**>(lua_newuserdata(state, sizeof(Entity*))) = self;, i.e. allocate a pointer-sized userdata and store the entity pointer there. No need for anything complex here.

If you manage the entity through a smart pointer, you can allocate it in the Lua instance and take advantage of its GC. In this case, the code is a bit more complex however:

#include <type_traits>

#if __GNUG__ && __GNUC__ < 5
#define is_trivially_constructible(T) __has_trivial_constructor(T)
#define is_trivially_destructible(T) __has_trivial_destructor(T)
#else
#define is_trivially_constructible(T) std::is_trivially_constructible<T>::value
#define is_trivially_destructible(T) std::is_trivially_destructible<T>::value
#endif

namespace lua
{
  template <class Type>
  struct _udata
  {
      typedef Type &type;

      static Type &to(lua_State *L, int idx)
      {
          return *reinterpret_cast<Type*>(lua_touserdata(L, idx));
      }

      static Type &check(lua_State *L, int idx, const char *tname)
      {
          return *reinterpret_cast<Type*>(luaL_checkudata(L, idx, tname));
      }
  };

  template <class Type>
  struct _udata<Type[]>
  {
      typedef Type *type;

      static Type *to(lua_State *L, int idx)
      {
          return reinterpret_cast<Type*>(lua_touserdata(L, idx));
      }

      static Type *check(lua_State *L, int idx, const char *tname)
      {
          return reinterpret_cast<Type*>(luaL_checkudata(L, idx, tname));
      }
  };

  template <class Type>
  typename _udata<Type>::type touserdata(lua_State *L, int idx)
  {
      return _udata<Type>::to(L, idx);
  }

  template <class Type>
  typename _udata<Type>::type checkudata(lua_State *L, int idx, const char *tname)
  {
      return _udata<Type>::check(L, idx, tname);
  }

  template <class Type>
  Type &newuserdata(lua_State *L)
  {
      auto data = reinterpret_cast<Type*>(lua_newuserdata(L, sizeof(Type)));
      if(!is_trivially_constructible(Type))
      {
          new (data) Type();
      }
      if(!is_trivially_destructible(Type))
      {
          lua_createtable(L, 0, 2);
          lua_pushboolean(L, false);
          lua_setfield(L, -2, "__metatable");
          lua_pushcfunction(L, [](lua_State *L) {
              lua::touserdata<Type>(L, -1).~Type();
              return 0;
          });
          lua_setfield(L, -2, "__gc");
          lua_setmetatable(L, -2);
      }
      return *data;
  }

  template <class Type>
  void pushuserdata(lua_State *L, const Type &val)
  {
      newuserdata<Type>(L) = val;
  }

  template <class Type, class=typename std::enable_if<!std::is_lvalue_reference<Type>::value>::type>
  void pushuserdata(lua_State *L, Type &&val)
  {
      newuserdata<typename std::remove_reference<Type>::type>(L) = std::move(val);
  }
}

If self is std::shared_ptr<Entity> you can then do lua::pushuserdata(state, self); and it will take care of everything ‒ properly initialize the userdata, and add a __gc metamethod that frees it. You may also do lua::pushuserdata<std::weak_ptr<Entity>>(state, self); if you don't want to let Lua prolong the lifetime of the entities. In-place construction is also possible too, if you modify lua::newuserdata accordingly.