Converting between vectors

Discussion in 'C++' started by Andrea Crotti, Apr 21, 2011.

  1. I need in short to serialize vector of different types into a stream of
    chars, which is then sent over the network.
    Boost unfortunately is not the answer since I can't use it in the
    project.

    Until now I used the function below, but I found out that is terribly
    buggy, since it doesn't work in all cases of type conversion and even
    size.

    But I can't find a way to fix it and to make it general, any idea /hint?

    Thanks,
    Andrea


    #include <assert.h>
    #include <string>
    #include <cstdlib>
    #include <iostream>
    #include <vector>

    template <typename INP, typename OUT>
    std::vector<OUT> vectorConversion(const std::vector<INP>& input)
    {
    size_t size_inp = sizeof(INP);
    size_t size_out = sizeof(OUT);
    // make sure the sizeof are multiples between them, otherwise it doesn't work!
    assert(((size_inp % size_out) == 0) || ((size_out % size_inp) == 0));
    std::vector<OUT> res;
    size_t size = (size_inp * input.size()) / size_out;
    res.resize(size);
    // is it memcpy a good idea in this case?
    memcpy(&res[0], &input[0], size_inp * input.size());
    return res;
    }

    int main()
    {
    std::vector<int> vec(5, 1);
    std::vector<long> lvec = vectorConversion<int, long>(vec);
    std::vector<int> vec2 = vectorConversion<long, int>(lvec);

    assert(vec == vec2);
    return 0;
    }
     
    Andrea Crotti, Apr 21, 2011
    #1
    1. Advertising

  2. On 4/21/2011 11:36 AM, Andrea Crotti wrote:
    > I need in short to serialize vector of different types into a stream of
    > chars, which is then sent over the network.
    > Boost unfortunately is not the answer since I can't use it in the
    > project.
    >
    > Until now I used the function below, but I found out that is terribly
    > buggy, since it doesn't work in all cases of type conversion and even
    > size.
    >
    > But I can't find a way to fix it and to make it general, any idea /hint?
    >
    > Thanks,
    > Andrea
    >
    >
    > #include<assert.h>
    > #include<string>
    > #include<cstdlib>
    > #include<iostream>
    > #include<vector>
    >
    > template<typename INP, typename OUT>
    > std::vector<OUT> vectorConversion(const std::vector<INP>& input)
    > {
    > size_t size_inp = sizeof(INP);
    > size_t size_out = sizeof(OUT);
    > // make sure the sizeof are multiples between them, otherwise it doesn't work!
    > assert(((size_inp % size_out) == 0) || ((size_out % size_inp) == 0));
    > std::vector<OUT> res;
    > size_t size = (size_inp * input.size()) / size_out;
    > res.resize(size);
    > // is it memcpy a good idea in this case?
    > memcpy(&res[0],&input[0], size_inp * input.size());
    > return res;
    > }
    >
    > int main()
    > {
    > std::vector<int> vec(5, 1);
    > std::vector<long> lvec = vectorConversion<int, long>(vec);
    > std::vector<int> vec2 = vectorConversion<long, int>(lvec);
    >
    > assert(vec == vec2);
    > return 0;
    > }
    >


    First off, you should allow the compiler to detemine the 'INP' type from
    the argument. Redefine your template as

    template<typename OUT, typename INP> ...

    and use it this way:

    std::vector<long> lvec = vectorConversion<long>(vec);

    then the argument you didn't explicitly provide will be deduced.

    Second, 'memcpy' is not a good idea. A loop with assignment from one to
    the other should do just fine.

    Third, try plain initialization from a pair of iterators:

    std::vector<long> lvec(vec.begin(), vec.end());

    I am not sure it should work if the value_type types are different, but
    it costs next to nothing to try.

    V
    --
    I do not respond to top-posted replies, please don't ask
     
    Victor Bazarov, Apr 21, 2011
    #2
    1. Advertising

  3. Victor Bazarov <> writes:

    >
    > First off, you should allow the compiler to detemine the 'INP' type
    > from the argument. Redefine your template as
    >
    > template<typename OUT, typename INP> ...
    >
    > and use it this way:
    >
    > std::vector<long> lvec = vectorConversion<long>(vec);
    >
    > then the argument you didn't explicitly provide will be deduced.
    >
    > Second, 'memcpy' is not a good idea. A loop with assignment from one
    > to the other should do just fine.
    >
    > Third, try plain initialization from a pair of iterators:
    >
    > std::vector<long> lvec(vec.begin(), vec.end());
    >
    > I am not sure it should work if the value_type types are different,
    > but it costs next to nothing to try.
    >
    > V


    Ok thanks, I tried the plain initialization but it doesn't work...

    But also I don't see how the loop can work, I mean a long doesn't fit in
    a char, and on the other way I don't want to use one long for each char,
    I would waste a lot of space.

    What could be another way to do it?
     
    Andrea Crotti, Apr 21, 2011
    #3
  4. A simpler question while I try to figure out how to do this:
    How do I make 2 ints fit in one long and then extract them again?

    This very rough code doesn't work as expected:

    --8<---------------cut here---------------start------------->8---
    long intsToLong(int x, int y)
    {
    long r = (x << sizeof(int));
    r += y;
    return r;
    }

    void longToInts(long l)
    {
    int second = l >> sizeof(int);
    int first = (l & (2 << sizeof(int)));
    std::cout << " first " << first << " second " << second << std::endl;
    }

    int main()
    {
    long l = intsToLong(10, 20);
    std::cout << l << " which gives " << std::endl;
    longToInts(l);
    return 0;
    }
    --8<---------------cut here---------------end--------------->8---
     
    Andrea Crotti, Apr 21, 2011
    #4
  5. Andrea Crotti

    LR Guest

    Andrea Crotti wrote:
    > I need in short to serialize vector of different types into a stream of
    > chars, which is then sent over the network.
    > Boost unfortunately is not the answer since I can't use it in the
    > project.
    >
    > Until now I used the function below, but I found out that is terribly
    > buggy, since it doesn't work in all cases of type conversion and even
    > size.


    Can you please be more specific?

    >
    > But I can't find a way to fix it and to make it general, any idea /hint?
    >
    > Thanks,
    > Andrea
    >
    >
    > #include <assert.h>
    > #include <string>
    > #include <cstdlib>
    > #include <iostream>
    > #include <vector>
    >
    > template <typename INP, typename OUT>
    > std::vector<OUT> vectorConversion(const std::vector<INP>& input)
    > {
    > size_t size_inp = sizeof(INP);
    > size_t size_out = sizeof(OUT);
    > // make sure the sizeof are multiples between them, otherwise it doesn't work!
    > assert(((size_inp % size_out) == 0) || ((size_out % size_inp) == 0));
    > std::vector<OUT> res;
    > size_t size = (size_inp * input.size()) / size_out;


    It looks to me like you want to pack or unpack the values in input to
    res. If that's the case the above line will not work in all cases.
    Consider if size_inp == 2 and input.size() == 3 and size_out == 4.

    > res.resize(size);
    > // is it memcpy a good idea in this case?
    > memcpy(&res[0], &input[0], size_inp * input.size());
    > return res;
    > }


    LR
     
    LR, Apr 21, 2011
    #5
  6. Andrea Crotti

    LR Guest

    Andrea Crotti wrote:
    > A simpler question while I try to figure out how to do this:
    > How do I make 2 ints fit in one long and then extract them again?
    >
    > This very rough code doesn't work as expected:


    What happens?
    >
    > --8<---------------cut here---------------start------------->8---
    > long intsToLong(int x, int y)
    > {
    > long r = (x << sizeof(int));


    On the platform I use, sizeof(int) == 4, so you're shifting x by 4.
    > r += y;


    I suspect that line will cause some problems for negative numbers.
    > return r;
    > }
    >
    > void longToInts(long l)
    > {
    > int second = l >> sizeof(int);
    > int first = (l & (2 << sizeof(int)));


    I'm not sure what the intent of that line is.

    > std::cout << " first " << first << " second " << second << std::endl;
    > }
    >
    > int main()
    > {
    > long l = intsToLong(10, 20);
    > std::cout << l << " which gives " << std::endl;
    > longToInts(l);
    > return 0;
    > }
    > --8<---------------cut here---------------end--------------->8---


    What are sizeof(int) and sizeof(long) on the platform you're using? Does
    this code have to be portable? You may find CHAR_BIT of use.
     
    LR, Apr 21, 2011
    #6
  7. Andrea Crotti

    LR Guest

    Andrea Crotti wrote:
    > I need in short to serialize vector of different types into a stream of
    > chars, which is then sent over the network.
    > Boost unfortunately is not the answer since I can't use it in the
    > project.
    >
    > Until now I used the function below, but I found out that is terribly
    > buggy, since it doesn't work in all cases of type conversion and even
    > size.
    >
    > But I can't find a way to fix it and to make it general, any idea /hint?
    >
    > Thanks,
    > Andrea
    >
    >
    > #include <assert.h>
    > #include <string>
    > #include <cstdlib>
    > #include <iostream>
    > #include <vector>
    >
    > template <typename INP, typename OUT>
    > std::vector<OUT> vectorConversion(const std::vector<INP>& input)
    > {
    > size_t size_inp = sizeof(INP);
    > size_t size_out = sizeof(OUT);
    > // make sure the sizeof are multiples between them, otherwise it doesn't work!
    > assert(((size_inp % size_out) == 0) || ((size_out % size_inp) == 0));


    I neglected to mention that you might want to check to see if INP and
    OUT are integer types using std::numeric_limits::is_integer.

    > std::vector<OUT> res;
    > size_t size = (size_inp * input.size()) / size_out;
    > res.resize(size);
    > // is it memcpy a good idea in this case?
    > memcpy(&res[0], &input[0], size_inp * input.size());
    > return res;
    > }
     
    LR, Apr 21, 2011
    #7
  8. Andrea Crotti

    Noah Roberts Guest

    On 4/21/2011 8:36 AM, Andrea Crotti wrote:
    > Boost unfortunately is not the answer since I can't use it in the
    > project.


    Still.... Use boost.

    --
    http://crazycpp.wordpress.com
     
    Noah Roberts, Apr 21, 2011
    #8
  9. On 4/21/2011 12:32 PM, Andrea Crotti wrote:
    > A simpler question while I try to figure out how to do this:
    > How do I make 2 ints fit in one long and then extract them again?
    > [..]


    There is no guarantee it can be done. You've a better chance of
    actually achieving that with two shorts and a long, although it is not
    guaranteed either since the ranges of values of types are not limited
    from above.

    V
    --
    I do not respond to top-posted replies, please don't ask
     
    Victor Bazarov, Apr 22, 2011
    #9
  10. Andrea Crotti

    Ian Collins Guest

    On 04/22/11 04:32 AM, Andrea Crotti wrote:
    > A simpler question while I try to figure out how to do this:
    > How do I make 2 ints fit in one long and then extract them again?


    You can't portably. You would have to use fixed width types.

    I'd start with a compile time "assert" like

    const bool checkSizes = 1/(sizeof(long)==2*sizeof(int));

    followed by

    const size_t intBits = sizeof(int)*CHAR_BIT;

    > This very rough code doesn't work as expected:
    >
    > --8<---------------cut here---------------start------------->8---
    > long intsToLong(int x, int y)
    > {
    > long r = (x<< sizeof(int));


    Use intBits and check the type you are shifting.

    > r += y;
    > return r;
    > }
    >
    > void longToInts(long l)
    > {
    > int second = l>> sizeof(int);


    Use intBits.

    > int first = (l& (2<< sizeof(int)));


    (2<< sizeof(int))? What value do you expect this to have? Use a
    temporary variable and check its value.

    --
    Ian Collins
     
    Ian Collins, Apr 22, 2011
    #10
  11. LR <> writes:

    >
    > Can you please be more specific?


    The bugs are exactly what you point out later, I never noticed because
    the sizes where always such that it always worked anyway...

    > It looks to me like you want to pack or unpack the values in input to
    > res. If that's the case the above line will not work in all cases.
    > Consider if size_inp == 2 and input.size() == 3 and size_out == 4.


    Yes I want to do that, any suggestions how to do it more generally and
    in a more correct way?
     
    Andrea Crotti, Apr 22, 2011
    #11
  12. LR <> writes:

    > I neglected to mention that you might want to check to see if INP and
    > OUT are integer types using std::numeric_limits::is_integer.
    >


    Why should I check? In general I should not care to check...
    Actually the whole conversion thing only happens from

    type T -> unsigned char
    and viceversa, no I can always suppose that the input or the output has
    size 1 (not sure it makes a difference).
     
    Andrea Crotti, Apr 22, 2011
    #12
  13. Noah Roberts <> writes:

    > On 4/21/2011 8:36 AM, Andrea Crotti wrote:
    >> Boost unfortunately is not the answer since I can't use it in the
    >> project.

    >
    > Still.... Use boost.


    I think I should add a signature "I can't use Boost".
    Is not that I don't want to, but the strict requirement is that I can
    only use the standard library, and actually considered where it will
    deployed it makes perfectly sense.

    Boost is great, but suggesting it when someone already said that is not
    an option doesn't help much unfortunately...
     
    Andrea Crotti, Apr 22, 2011
    #13
  14. "Alf P. Steinbach /Usenet" <> writes:

    >
    > Could you at least hint about about the kind of environment where it
    > will be employed where it makes perfect sense to not use Boost?
    >


    Network routers in an experimental testbed, asking new software there
    is not easy and surely takes too long for the time I have left.
    And it was a decision taken in the beginning, so discuss about it is
    just a waste of time...

    >
    > huh.
    >


    Well I think I wrote 200 times "I can't use boost", that's why
     
    Andrea Crotti, Apr 22, 2011
    #14
  15. Ian Collins <> writes:

    > On 04/22/11 04:32 AM, Andrea Crotti wrote:
    >> A simpler question while I try to figure out how to do this:
    >> How do I make 2 ints fit in one long and then extract them again?

    >
    > You can't portably. You would have to use fixed width types.
    >
    > I'd start with a compile time "assert" like
    >
    > const bool checkSizes = 1/(sizeof(long)==2*sizeof(int));
    >
    > followed by
    >
    > const size_t intBits = sizeof(int)*CHAR_BIT;


    This is just a toy example but I wanted to understand how to do it
    correctly...

    Like this it seems to work fine, is that correct in theory?
    Are there other ways to achieve the same thing in general?

    --8<---------------cut here---------------start------------->8---
    long intsToLong(int x, int y)
    {
    long tmp = x;
    tmp <<= int_bits;
    tmp += y;
    return tmp;
    }

    void longToInts(long l)
    {
    int second = l >> int_bits;
    long mask = (2 >> (int_bits + 1)) - 1;
    int first = l & mask;
    std::cout << " first " << first << " second " << second << std::endl;
    }
    --8<---------------cut here---------------end--------------->8---
     
    Andrea Crotti, Apr 22, 2011
    #15
  16. * Andrea Crotti, on 22.04.2011 11:27:
    > Noah Roberts<> writes:
    >
    >> On 4/21/2011 8:36 AM, Andrea Crotti wrote:
    >>> Boost unfortunately is not the answer since I can't use it in the
    >>> project.

    >>
    >> Still.... Use boost.

    >
    > I think I should add a signature "I can't use Boost".
    > Is not that I don't want to, but the strict requirement is that I can
    > only use the standard library, and actually considered where it will
    > deployed it makes perfectly sense.


    Could you at least hint about about the kind of environment where it will be
    employed where it makes perfect sense to not use Boost?


    > Boost is great, but suggesting it when someone already said that is not
    > an option doesn't help much unfortunately...


    huh.


    Cheers

    - Alf

    --
    blog at <url: http://alfps.wordpress.com>
     
    Alf P. Steinbach /Usenet, Apr 22, 2011
    #16
  17. Andrea Crotti

    Ian Collins Guest

    On 04/22/11 09:49 PM, Andrea Crotti wrote:
    > Ian Collins<> writes:
    >
    >> On 04/22/11 04:32 AM, Andrea Crotti wrote:
    >>> A simpler question while I try to figure out how to do this:
    >>> How do I make 2 ints fit in one long and then extract them again?

    >>
    >> You can't portably. You would have to use fixed width types.
    >>
    >> I'd start with a compile time "assert" like
    >>
    >> const bool checkSizes = 1/(sizeof(long)==2*sizeof(int));
    >>
    >> followed by
    >>
    >> const size_t intBits = sizeof(int)*CHAR_BIT;

    >
    > This is just a toy example but I wanted to understand how to do it
    > correctly...
    >
    > Like this it seems to work fine, is that correct in theory?
    > Are there other ways to achieve the same thing in general?


    Mask and shift is the idiomatic, portable, solution.

    > --8<---------------cut here---------------start------------->8---
    > long intsToLong(int x, int y)
    > {
    > long tmp = x;
    > tmp<<= int_bits;
    > tmp += y;
    > return tmp;
    > }
    >
    > void longToInts(long l)
    > {
    > int second = l>> int_bits;
    > long mask = (2>> (int_bits + 1)) - 1;


    Have you by any chance looked at the value of mask? Hint - what is 2
    left shifted 31 places?

    --
    Ian Collins
     
    Ian Collins, Apr 22, 2011
    #17
  18. Ian Collins <> writes:

    >
    >> --8<---------------cut here---------------start------------->8---
    >> long intsToLong(int x, int y)
    >> {
    >> long tmp = x;
    >> tmp<<= int_bits;
    >> tmp += y;
    >> return tmp;
    >> }
    >>
    >> void longToInts(long l)
    >> {
    >> int second = l>> int_bits;
    >> long mask = (2>> (int_bits + 1)) - 1;

    >
    > Have you by any chance looked at the value of mask? Hint - what is 2
    > left shifted 31 places?


    Yes sure I was shifting in the wrong direction, thanks :)

    So and what if the size of the types is not multiple of each other?
    For example type X is 6 bytes and type Y is 4 bytes.

    Then what is the best way to pack data from one type to the other?
     
    Andrea Crotti, Apr 22, 2011
    #18
  19. Andrea Crotti

    LR Guest

    Andrea Crotti wrote:
    > LR <> writes:
    >
    >>
    >> Can you please be more specific?

    >
    > The bugs are exactly what you point out later, I never noticed because
    > the sizes where always such that it always worked anyway...
    >
    >> It looks to me like you want to pack or unpack the values in input to
    >> res. If that's the case the above line will not work in all cases.
    >> Consider if size_inp == 2 and input.size() == 3 and size_out == 4.

    >
    > Yes I want to do that, any suggestions how to do it more generally and
    > in a more correct way?


    I think you'll have to fix the math problem and also you'll want to
    maintain the number of characters you're converting.

    Personally, I don't think memcpy is a bad way to do this, but perhaps
    there is a better way.

    LR
     
    LR, Apr 22, 2011
    #19
  20. Andrea Crotti

    LR Guest

    Andrea Crotti wrote:
    > LR <> writes:
    >
    >> I neglected to mention that you might want to check to see if INP and
    >> OUT are integer types using std::numeric_limits::is_integer.
    >>

    >
    > Why should I check? In general I should not care to check...
    > Actually the whole conversion thing only happens from
    >
    > type T -> unsigned char
    > and viceversa, no I can always suppose that the input or the output has
    > size 1 (not sure it makes a difference).


    This makes me feel a little uncertain as to the problem you actually
    want to solve, since what you posted converts packs and unpacks long and
    int.

    LR
     
    LR, Apr 22, 2011
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ! aaa
    Replies:
    1
    Views:
    1,116
    ! aaa
    May 28, 2004
  2. Replies:
    3
    Views:
    375
    Anton Spaans
    Aug 5, 2005
  3. shapper

    Check difference between two vectors

    shapper, Feb 15, 2008, in forum: ASP .Net
    Replies:
    1
    Views:
    316
    shapper
    Feb 15, 2008
  4. Replies:
    3
    Views:
    725
    Shadowman
    Mar 26, 2008
  5. Guest
    Replies:
    0
    Views:
    481
    Guest
    Sep 14, 2005
Loading...

Share This Page