Home > Software engineering >  Positive input exceeds the limit of integer - dart
Positive input exceeds the limit of integer - dart

Time:07-23

I have this function to convert hex to double. The values are some coordinates so the latitude should be a positive number and longitude a negative one.

Something like this:

41.43757591162915, -8.418569400003188

For the Latitude works fine, but returns the error when trying to convert the Longitude.

Expected value: -8.418569400003188 converting from C020D67F4DBDF8F5

I read about this error and it seems it's because the value goes outside the limits of the range.

 double convert(String hexString) =>
      (ByteData(8)..setUint64(0, int.parse(hexString, radix: 16)))
          .getFloat64(0);

  void _processHexDouble(hexString, _regexVar, String name, String _toSplit) {

    final a = _regexVar.firstMatch(hexString);
    final _matchedRegex = a?.group(0); //A9C020D67F4DBDF8F5
    // var number = 0.0;

    try {
      if (_matchedRegex != null) {
        var _splitRegex = _matchedRegex.split(_toSplit);

        print('$name -> ${convert(_splitRegex[1])}'); //C020D67F4DBDF8F5
        // print('$name -> ${number}');
        _logResponses =
            "${_logResponses} $name -> ${convert(_splitRegex[1])} \n";
      }
    } on Exception catch (e) {
      print(e);
    }
  }

How can I fix this without firing this error? Is there a way to limit the size of the value at the conversion, so it's smaller than the limits?

CodePudding user response:

Check the updated solution added to my answer to your previous question (added after feedback from lrn): https://stackoverflow.com/a/73065950/1953515

It fixes the issue:

import 'dart:typed_data';

void main() {
  print(convert('C020D67F4DBDF8F5')); // -8.418940000000001
}

double convert(String hexString) {
  final String hexStringPadded = hexString.padLeft(16, '0');

  return (ByteData(8)
    ..setInt32(0, int.parse(hexStringPadded.substring(0, 8), radix: 16))
    ..setInt32(4, int.parse(hexStringPadded.substring(8, 16), radix: 16)))
      .getFloat64(0);
}
  • Related