I have a function that takes two 2d matrices of variable dimensions and returns a matrix of equal dimensions.

double (*sumOfMatrices(size_t rows, size_t columns, double matrixA[][columns], double matrixB[][columns]))[] {
    double (*result)[columns]= malloc(rows * sizeof *result);
    for (size_t i = 0; i < rows; i  )
    {
        for (size_t j = 0; j < columns; j  )
        {
            result[i][j]= matrixA[i][j]   matrixB[i][j];
        }
    }
    return result;
}

Then I declare a variable like this

 double (*sumResult)[n]= sumOfMatrices(m, n, matrixA, matrixB);

Is this undefined behavior? The compiler doesn't say anything, yet I feel that initializing a double(*)[n] with a double(*)[] is implicitly promoting the data type. Is there a way that I can return double(*)[columns] instead?

CodePudding user response：

This code is valid.

The return type of the function is double (*)[], i.e. a pointer to an array of unspecified size of type double. The return statement in this function is converting a double (*)[columns] to that type, and the assignment of the return value in the calling function performs the opposite conversion.

Two pointer types are compatible if they point to compatible types. So now we compare the types double [columns] and double []. Two arrays are compatible if the element types are compatible and, if both specify a size, that the sizes are the same. Since only one specifies a size, the array types are compatible, and therefore so are the pointers to the arrays.

Sections 6.7.6.1p2 and 6.7.6.2p6 of the C standard spell out this compatibility in more detail:

6.7.6.1p2

For two pointer types to be compatible, both shall be identically qualified and both shall be pointers to compatible types.

6.7.6.2p6

For two array types to be compatible, both shall have compatible element types, and if both size specifiers are present, and are integer constant expressions, then both size specifiers shall have the same constant value. If the two array types are used in a context which requires them to be compatible, it is undefined behavior if the two size specifiers evaluate to unequal values.

So let's say n is 5. The function then creates a pointer to an array whose type is double (*)[5]. This type is converted to double (*)[] when it is returned from the function, then converted back to double (*)[5] when assigned to sumResult whose type matches the original type.

So both conversions are valid, and use of sumResult is valid.