Python C-API int128 support












1















In python one can handle very large integers (for instance uuid.uuid4().int.bit_length() gives 128), but the largest int datastructure the C-API documentation offers is long long, and it is a 64-bit int.



I would love to be able to get a C int128 from a PyLong, but it seems there is no tooling for this. PyLong_AsLongLong for instance cannot handle python integers biggers than 2**64.




  • Is there some documentation I missed, and it is actually possible?

  • Is there currently not possible, but some workaround exist? (I would love to use the tooling available in the python C-API for long long with int128, for instance a PyLong_AsInt128AndOverflow function).

  • Is it a planed feature in an forthcoming python release?










share|improve this question

























  • I think you're confusing python's arbitrary length integers with c's integer types. Assuming that uuid.uuid4().int.bit_length() == 128 (which is not always true), you could do something like print((uuid.uuid4().int * 4).bit_length()) and you should get a number <= 130. That shouldn't suggest that Python is using "int130" -- only that Python supports arbitrary length integers.

    – jedwards
    Jan 20 at 15:03











  • I edited my post so it is more accurate. The only thing I would like to do is to get a C int128 from a python PyLong.

    – azmeuk
    Jan 20 at 16:33
















1















In python one can handle very large integers (for instance uuid.uuid4().int.bit_length() gives 128), but the largest int datastructure the C-API documentation offers is long long, and it is a 64-bit int.



I would love to be able to get a C int128 from a PyLong, but it seems there is no tooling for this. PyLong_AsLongLong for instance cannot handle python integers biggers than 2**64.




  • Is there some documentation I missed, and it is actually possible?

  • Is there currently not possible, but some workaround exist? (I would love to use the tooling available in the python C-API for long long with int128, for instance a PyLong_AsInt128AndOverflow function).

  • Is it a planed feature in an forthcoming python release?










share|improve this question

























  • I think you're confusing python's arbitrary length integers with c's integer types. Assuming that uuid.uuid4().int.bit_length() == 128 (which is not always true), you could do something like print((uuid.uuid4().int * 4).bit_length()) and you should get a number <= 130. That shouldn't suggest that Python is using "int130" -- only that Python supports arbitrary length integers.

    – jedwards
    Jan 20 at 15:03











  • I edited my post so it is more accurate. The only thing I would like to do is to get a C int128 from a python PyLong.

    – azmeuk
    Jan 20 at 16:33














1












1








1








In python one can handle very large integers (for instance uuid.uuid4().int.bit_length() gives 128), but the largest int datastructure the C-API documentation offers is long long, and it is a 64-bit int.



I would love to be able to get a C int128 from a PyLong, but it seems there is no tooling for this. PyLong_AsLongLong for instance cannot handle python integers biggers than 2**64.




  • Is there some documentation I missed, and it is actually possible?

  • Is there currently not possible, but some workaround exist? (I would love to use the tooling available in the python C-API for long long with int128, for instance a PyLong_AsInt128AndOverflow function).

  • Is it a planed feature in an forthcoming python release?










share|improve this question
















In python one can handle very large integers (for instance uuid.uuid4().int.bit_length() gives 128), but the largest int datastructure the C-API documentation offers is long long, and it is a 64-bit int.



I would love to be able to get a C int128 from a PyLong, but it seems there is no tooling for this. PyLong_AsLongLong for instance cannot handle python integers biggers than 2**64.




  • Is there some documentation I missed, and it is actually possible?

  • Is there currently not possible, but some workaround exist? (I would love to use the tooling available in the python C-API for long long with int128, for instance a PyLong_AsInt128AndOverflow function).

  • Is it a planed feature in an forthcoming python release?







python cpython python-c-api int128






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Jan 20 at 18:25









Mad Physicist

37k1671103




37k1671103










asked Jan 20 at 14:54









azmeukazmeuk

1,22421634




1,22421634













  • I think you're confusing python's arbitrary length integers with c's integer types. Assuming that uuid.uuid4().int.bit_length() == 128 (which is not always true), you could do something like print((uuid.uuid4().int * 4).bit_length()) and you should get a number <= 130. That shouldn't suggest that Python is using "int130" -- only that Python supports arbitrary length integers.

    – jedwards
    Jan 20 at 15:03











  • I edited my post so it is more accurate. The only thing I would like to do is to get a C int128 from a python PyLong.

    – azmeuk
    Jan 20 at 16:33



















  • I think you're confusing python's arbitrary length integers with c's integer types. Assuming that uuid.uuid4().int.bit_length() == 128 (which is not always true), you could do something like print((uuid.uuid4().int * 4).bit_length()) and you should get a number <= 130. That shouldn't suggest that Python is using "int130" -- only that Python supports arbitrary length integers.

    – jedwards
    Jan 20 at 15:03











  • I edited my post so it is more accurate. The only thing I would like to do is to get a C int128 from a python PyLong.

    – azmeuk
    Jan 20 at 16:33

















I think you're confusing python's arbitrary length integers with c's integer types. Assuming that uuid.uuid4().int.bit_length() == 128 (which is not always true), you could do something like print((uuid.uuid4().int * 4).bit_length()) and you should get a number <= 130. That shouldn't suggest that Python is using "int130" -- only that Python supports arbitrary length integers.

– jedwards
Jan 20 at 15:03





I think you're confusing python's arbitrary length integers with c's integer types. Assuming that uuid.uuid4().int.bit_length() == 128 (which is not always true), you could do something like print((uuid.uuid4().int * 4).bit_length()) and you should get a number <= 130. That shouldn't suggest that Python is using "int130" -- only that Python supports arbitrary length integers.

– jedwards
Jan 20 at 15:03













I edited my post so it is more accurate. The only thing I would like to do is to get a C int128 from a python PyLong.

– azmeuk
Jan 20 at 16:33





I edited my post so it is more accurate. The only thing I would like to do is to get a C int128 from a python PyLong.

– azmeuk
Jan 20 at 16:33












1 Answer
1






active

oldest

votes


















0














There are a couple of different ways you can access the level of precision you want.



Systems with 64-bit longs often have 128-bit long longs. Notice that the article you link says "at least 64 bits". It's worth checking sizeof(long long) in case there's nothing further to do.



Assuming that is not what you are working with, you'll have to look closer at the raw PyLongObject, which is actually a typedef of the private _longobject structure.



The raw bits are accessible through the ob_digit field, with the length given by ob_size. The data type of the digits, and the actual number of boots they hold is given by the typedef digit and the macro PYLONG_BITS_IN_DIGIT. The latter must be smaller than 8 * sizeof(digit), larger than 8, and a multiple of 5 (so 30 or 15, depending on how your build was done).



Luckily for you, there is an "undocumented" method in the C API that will copy the bytes of the number for you: _PyLong_AsByteArray. The comment in longobject.h reads:




/* _PyLong_AsByteArray: Convert the least-significant 8*n bits of long
v to a base-256 integer, stored in array bytes. Normally return 0,
return -1 on error.
If little_endian is 1/true, store the MSB at bytes[n-1] and the LSB at
bytes[0]; else (little_endian is 0/false) store the MSB at bytes[0] and
the LSB at bytes[n-1].
If is_signed is 0/false, it's an error if v < 0; else (v >= 0) n bytes
are filled and there's nothing special about bit 0x80 of the MSB.
If is_signed is 1/true, bytes is filled with the 2's-complement
representation of v's value. Bit 0x80 of the MSB is the sign bit.
Error returns (-1):
+ is_signed is 0 and v < 0. TypeError is set in this case, and bytes
isn't altered.
+ n isn't big enough to hold the full mathematical value of v. For
example, if is_signed is 0 and there are more digits in the v than
fit in n; or if is_signed is 1, v < 0, and n is just 1 bit shy of
being large enough to hold a sign bit. OverflowError is set in this
case, but bytes holds the least-significant n bytes of the true value.
*/



You should be able to get a UUID with something like



PyLongObject *mylong;
unsigned char myuuid[16];

_PyLong_AsByteArray(mylong, myuuid, sizeof(myuuid), 1, 0);





share|improve this answer

























    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54277671%2fpython-c-api-int128-support%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    There are a couple of different ways you can access the level of precision you want.



    Systems with 64-bit longs often have 128-bit long longs. Notice that the article you link says "at least 64 bits". It's worth checking sizeof(long long) in case there's nothing further to do.



    Assuming that is not what you are working with, you'll have to look closer at the raw PyLongObject, which is actually a typedef of the private _longobject structure.



    The raw bits are accessible through the ob_digit field, with the length given by ob_size. The data type of the digits, and the actual number of boots they hold is given by the typedef digit and the macro PYLONG_BITS_IN_DIGIT. The latter must be smaller than 8 * sizeof(digit), larger than 8, and a multiple of 5 (so 30 or 15, depending on how your build was done).



    Luckily for you, there is an "undocumented" method in the C API that will copy the bytes of the number for you: _PyLong_AsByteArray. The comment in longobject.h reads:




    /* _PyLong_AsByteArray: Convert the least-significant 8*n bits of long
    v to a base-256 integer, stored in array bytes. Normally return 0,
    return -1 on error.
    If little_endian is 1/true, store the MSB at bytes[n-1] and the LSB at
    bytes[0]; else (little_endian is 0/false) store the MSB at bytes[0] and
    the LSB at bytes[n-1].
    If is_signed is 0/false, it's an error if v < 0; else (v >= 0) n bytes
    are filled and there's nothing special about bit 0x80 of the MSB.
    If is_signed is 1/true, bytes is filled with the 2's-complement
    representation of v's value. Bit 0x80 of the MSB is the sign bit.
    Error returns (-1):
    + is_signed is 0 and v < 0. TypeError is set in this case, and bytes
    isn't altered.
    + n isn't big enough to hold the full mathematical value of v. For
    example, if is_signed is 0 and there are more digits in the v than
    fit in n; or if is_signed is 1, v < 0, and n is just 1 bit shy of
    being large enough to hold a sign bit. OverflowError is set in this
    case, but bytes holds the least-significant n bytes of the true value.
    */



    You should be able to get a UUID with something like



    PyLongObject *mylong;
    unsigned char myuuid[16];

    _PyLong_AsByteArray(mylong, myuuid, sizeof(myuuid), 1, 0);





    share|improve this answer






























      0














      There are a couple of different ways you can access the level of precision you want.



      Systems with 64-bit longs often have 128-bit long longs. Notice that the article you link says "at least 64 bits". It's worth checking sizeof(long long) in case there's nothing further to do.



      Assuming that is not what you are working with, you'll have to look closer at the raw PyLongObject, which is actually a typedef of the private _longobject structure.



      The raw bits are accessible through the ob_digit field, with the length given by ob_size. The data type of the digits, and the actual number of boots they hold is given by the typedef digit and the macro PYLONG_BITS_IN_DIGIT. The latter must be smaller than 8 * sizeof(digit), larger than 8, and a multiple of 5 (so 30 or 15, depending on how your build was done).



      Luckily for you, there is an "undocumented" method in the C API that will copy the bytes of the number for you: _PyLong_AsByteArray. The comment in longobject.h reads:




      /* _PyLong_AsByteArray: Convert the least-significant 8*n bits of long
      v to a base-256 integer, stored in array bytes. Normally return 0,
      return -1 on error.
      If little_endian is 1/true, store the MSB at bytes[n-1] and the LSB at
      bytes[0]; else (little_endian is 0/false) store the MSB at bytes[0] and
      the LSB at bytes[n-1].
      If is_signed is 0/false, it's an error if v < 0; else (v >= 0) n bytes
      are filled and there's nothing special about bit 0x80 of the MSB.
      If is_signed is 1/true, bytes is filled with the 2's-complement
      representation of v's value. Bit 0x80 of the MSB is the sign bit.
      Error returns (-1):
      + is_signed is 0 and v < 0. TypeError is set in this case, and bytes
      isn't altered.
      + n isn't big enough to hold the full mathematical value of v. For
      example, if is_signed is 0 and there are more digits in the v than
      fit in n; or if is_signed is 1, v < 0, and n is just 1 bit shy of
      being large enough to hold a sign bit. OverflowError is set in this
      case, but bytes holds the least-significant n bytes of the true value.
      */



      You should be able to get a UUID with something like



      PyLongObject *mylong;
      unsigned char myuuid[16];

      _PyLong_AsByteArray(mylong, myuuid, sizeof(myuuid), 1, 0);





      share|improve this answer




























        0












        0








        0







        There are a couple of different ways you can access the level of precision you want.



        Systems with 64-bit longs often have 128-bit long longs. Notice that the article you link says "at least 64 bits". It's worth checking sizeof(long long) in case there's nothing further to do.



        Assuming that is not what you are working with, you'll have to look closer at the raw PyLongObject, which is actually a typedef of the private _longobject structure.



        The raw bits are accessible through the ob_digit field, with the length given by ob_size. The data type of the digits, and the actual number of boots they hold is given by the typedef digit and the macro PYLONG_BITS_IN_DIGIT. The latter must be smaller than 8 * sizeof(digit), larger than 8, and a multiple of 5 (so 30 or 15, depending on how your build was done).



        Luckily for you, there is an "undocumented" method in the C API that will copy the bytes of the number for you: _PyLong_AsByteArray. The comment in longobject.h reads:




        /* _PyLong_AsByteArray: Convert the least-significant 8*n bits of long
        v to a base-256 integer, stored in array bytes. Normally return 0,
        return -1 on error.
        If little_endian is 1/true, store the MSB at bytes[n-1] and the LSB at
        bytes[0]; else (little_endian is 0/false) store the MSB at bytes[0] and
        the LSB at bytes[n-1].
        If is_signed is 0/false, it's an error if v < 0; else (v >= 0) n bytes
        are filled and there's nothing special about bit 0x80 of the MSB.
        If is_signed is 1/true, bytes is filled with the 2's-complement
        representation of v's value. Bit 0x80 of the MSB is the sign bit.
        Error returns (-1):
        + is_signed is 0 and v < 0. TypeError is set in this case, and bytes
        isn't altered.
        + n isn't big enough to hold the full mathematical value of v. For
        example, if is_signed is 0 and there are more digits in the v than
        fit in n; or if is_signed is 1, v < 0, and n is just 1 bit shy of
        being large enough to hold a sign bit. OverflowError is set in this
        case, but bytes holds the least-significant n bytes of the true value.
        */



        You should be able to get a UUID with something like



        PyLongObject *mylong;
        unsigned char myuuid[16];

        _PyLong_AsByteArray(mylong, myuuid, sizeof(myuuid), 1, 0);





        share|improve this answer















        There are a couple of different ways you can access the level of precision you want.



        Systems with 64-bit longs often have 128-bit long longs. Notice that the article you link says "at least 64 bits". It's worth checking sizeof(long long) in case there's nothing further to do.



        Assuming that is not what you are working with, you'll have to look closer at the raw PyLongObject, which is actually a typedef of the private _longobject structure.



        The raw bits are accessible through the ob_digit field, with the length given by ob_size. The data type of the digits, and the actual number of boots they hold is given by the typedef digit and the macro PYLONG_BITS_IN_DIGIT. The latter must be smaller than 8 * sizeof(digit), larger than 8, and a multiple of 5 (so 30 or 15, depending on how your build was done).



        Luckily for you, there is an "undocumented" method in the C API that will copy the bytes of the number for you: _PyLong_AsByteArray. The comment in longobject.h reads:




        /* _PyLong_AsByteArray: Convert the least-significant 8*n bits of long
        v to a base-256 integer, stored in array bytes. Normally return 0,
        return -1 on error.
        If little_endian is 1/true, store the MSB at bytes[n-1] and the LSB at
        bytes[0]; else (little_endian is 0/false) store the MSB at bytes[0] and
        the LSB at bytes[n-1].
        If is_signed is 0/false, it's an error if v < 0; else (v >= 0) n bytes
        are filled and there's nothing special about bit 0x80 of the MSB.
        If is_signed is 1/true, bytes is filled with the 2's-complement
        representation of v's value. Bit 0x80 of the MSB is the sign bit.
        Error returns (-1):
        + is_signed is 0 and v < 0. TypeError is set in this case, and bytes
        isn't altered.
        + n isn't big enough to hold the full mathematical value of v. For
        example, if is_signed is 0 and there are more digits in the v than
        fit in n; or if is_signed is 1, v < 0, and n is just 1 bit shy of
        being large enough to hold a sign bit. OverflowError is set in this
        case, but bytes holds the least-significant n bytes of the true value.
        */



        You should be able to get a UUID with something like



        PyLongObject *mylong;
        unsigned char myuuid[16];

        _PyLong_AsByteArray(mylong, myuuid, sizeof(myuuid), 1, 0);






        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Jan 20 at 18:30

























        answered Jan 20 at 18:17









        Mad PhysicistMad Physicist

        37k1671103




        37k1671103
































            draft saved

            draft discarded




















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54277671%2fpython-c-api-int128-support%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Liquibase includeAll doesn't find base path

            How to use setInterval in EJS file?

            Petrus Granier-Deferre