Undefined behavior using qsort on floating numbers in c-CodePudding

I tried to use qsort to sort an array of floating point values, however it sometimes produces correct sorted result and sometimes doesn't.

The code shows below:

#include <stdlib.h>
#include <stdio.h>

double floatcomp(const void *elem1, const void *elem2) {
    if (*(const double*)elem1 < *(const double*)elem2)
        return -1;
    return *(const double*)elem1 > *(const double*)elem2;
}

int main() {
    double array[10] = { 10.4f, 5.1f, 144.4f, 145.5f,
                         12.4f, 51.1f, 1.4f, -15.7f };
    int i;

    qsort(array, 10, sizeof(double), floatcomp);

    for (i = 0; i < 10; i  )
        printf("%5.2f ", array[i]);

    return 0;
}

The results sometimes are:

-15.70  0.00  0.00  1.40  5.10 10.40 12.40 51.10 144.40 145.50

and sometimes remains unchanged as:

10.40  5.10 144.40 145.50 12.40 51.10  1.40 -15.70  0.00  0.00

The problem is solved when I change the type of the comparator function floatcomp from double to double *, I wonder what is the reason behind that undefined behavior produced from the original code attached above?

CodePudding user response：

As noted in the other answer, the problem is the return value of your comparator: it should be int.

While this should already be sufficient (undefined behavior explains everything), here is one reason why double might not work. Some calling conventions specify that floating-point values are returned in specific registers (e.g. XMM or floating-point stack), while integer and pointer values are returned in other, unrelated registers (e.g. rax). That's why a pointer type, while its value is nonsensical, works just as well as int.

If returning double, floatcomp may leave rax unchanged, which qsort might interpret as "elem1 > elem2 always, regardless of their actual values".

CodePudding user response：

The comparison function should have a return type of int.

Here is a modified version using a generic approach that works for all scalar types:

int floatcomp(const void *elem1, const void *elem2) {
    const double *p1 = elem1;
    const double *p2 = elem2;
    return (*p1 > *p2) - (*p1 < *p2);
}

CodePudding user response：

Others have called out the problems in comments.

Fixed up and taken all together:

#include <stdio.h>

#define COUNT_OF(x) (sizeof(x)/sizeof(*x))

int floatcomp(const void* elem1, const void* elem2)
{
    if(*(const double*)elem1 < *(const double*)elem2)
        return -1;
    return *(const double*)elem1 > *(const double*)elem2;
}

int main()
{
    double array[] = {  10.4,  5.1, 144.4, 145.5,
                        12.4, 51.1,   1.4, -15.7 };

    qsort(array, COUNT_OF(array), sizeof(double), floatcomp);

    for(int i = 0; i < COUNT_OF(array); i  )
        printf("%5.2lf ", array[i]);

    return 0;
}

Results:

Success #stdin #stdout 0s 5444KB
-15.70  1.40  5.10 10.40 12.40 51.10 144.40 145.50