Special Data Type Declaration and Usage

Discussion in 'C++' started by Mike Copeland, Nov 15, 2011.

  1. I need to declare an enormous array (100,000 members) of a small
    scalar object, and I want to populate it with default values and change
    some of them as needed throughout the processing. The range of distinct
    values for each element is 0-15, so I could use a 4 bit "mini-byte" if
    something like that existed.
    My question is: can I declare a bit structure data type and use it
    this way? If that's possible, I don't know how to do so, nor how I'd
    manipulate the data values in it. Please advise. TIA
     
    Mike Copeland, Nov 15, 2011
    #1
    1. Advertising

  2. Mike Copeland

    Ian Collins Guest

    On 11/16/11 10:04 AM, Mike Copeland wrote:
    > I need to declare an enormous array (100,000 members) of a small
    > scalar object, and I want to populate it with default values and change
    > some of them as needed throughout the processing. The range of distinct
    > values for each element is 0-15, so I could use a 4 bit "mini-byte" if
    > something like that existed.
    > My question is: can I declare a bit structure data type and use it
    > this way? If that's possible, I don't know how to do so, nor how I'd
    > manipulate the data values in it. Please advise. TIA


    You can't have a single type smaller than a char. You could put a pair
    of your "mini-byes" in a bit-field.

    --
    Ian Collins
     
    Ian Collins, Nov 15, 2011
    #2
    1. Advertising

  3. On Tue, 15 Nov 2011 22:04:25 +0100, Mike Copeland <> wrote:

    > I need to declare an enormous array (100,000 members) of a small
    > scalar object, and I want to populate it with default values and change
    > some of them as needed throughout the processing. The range of distinct
    > values for each element is 0-15, so I could use a 4 bit "mini-byte" if
    > something like that existed.
    > My question is: can I declare a bit structure data type and use it
    > this way? If that's possible, I don't know how to do so, nor how I'd
    > manipulate the data values in it. Please advise. TIA


    Are you sure that the array is "enormous"? Using one (whole!) byte per
    element it would occupy less than 100KiB.

    --
    Gennaro Prota | name.surname gmail.com
    Breeze C++ (preview): <http://sourceforge.net/projects/breeze/>
    Do you need expertise in C++? I'm available.
     
    Gennaro Prota, Nov 15, 2011
    #3
  4. In article <>,
    says...
    > On Tue, 15 Nov 2011 22:04:25 +0100, Mike Copeland <> wrote:
    >
    > > I need to declare an enormous array (100,000 members) of a small
    > > scalar object, and I want to populate it with default values and change
    > > some of them as needed throughout the processing. The range of distinct
    > > values for each element is 0-15, so I could use a 4 bit "mini-byte" if
    > > something like that existed.
    > > My question is: can I declare a bit structure data type and use it
    > > this way? If that's possible, I don't know how to do so, nor how I'd
    > > manipulate the data values in it. Please advise. TIA

    >
    > Are you sure that the array is "enormous"? Using one (whole!) byte per
    > element it would occupy less than 100KiB.
    >

    It's "enormous" to me, as I have many other large data structures in
    this program, and I have to declare it globally (it's accessed from a
    number of subprograms in several source files). My concern is Heap
    allocation, inasmuch as there must be _some_ limit as to how much
    program data can be declared before the program chokes. I;m using
    VS6.0, and I don't know what that limit is.
    I was hoping that I could cut down the total size of this data
    component and still process it efficiently. It seems not possible...
    8<{{
     
    Mike Copeland, Nov 15, 2011
    #4
  5. Mike Copeland

    Ian Collins Guest

    On 11/16/11 10:30 AM, Mike Copeland wrote:
    > In article<>,
    > says...
    >> On Tue, 15 Nov 2011 22:04:25 +0100, Mike Copeland<> wrote:
    >>
    >>> I need to declare an enormous array (100,000 members) of a small
    >>> scalar object, and I want to populate it with default values and change
    >>> some of them as needed throughout the processing. The range of distinct
    >>> values for each element is 0-15, so I could use a 4 bit "mini-byte" if
    >>> something like that existed.
    >>> My question is: can I declare a bit structure data type and use it
    >>> this way? If that's possible, I don't know how to do so, nor how I'd
    >>> manipulate the data values in it. Please advise. TIA

    >>
    >> Are you sure that the array is "enormous"? Using one (whole!) byte per
    >> element it would occupy less than 100KiB.
    >>

    > It's "enormous" to me, as I have many other large data structures in
    > this program, and I have to declare it globally (it's accessed from a
    > number of subprograms in several source files). My concern is Heap
    > allocation, inasmuch as there must be _some_ limit as to how much
    > program data can be declared before the program chokes.


    That depends on the configuration (physical and OS) of the target system.

    > I;m using VS6.0, and I don't know what that limit is.


    Stop right there! An obsolete, non-standard and buggy compiler? I'm
    sure you can do better than that.

    > I was hoping that I could cut down the total size of this data
    > component and still process it efficiently. It seems not possible...
    > 8<{{


    Yes you could cut down the size, but that is often orthogonal to
    efficient processing.

    --
    Ian Collins
     
    Ian Collins, Nov 15, 2011
    #5
  6. In article <>,
    says...
    > On 11/16/11 10:30 AM, Mike Copeland wrote:
    > > In article<>,
    > > says...
    > >> On Tue, 15 Nov 2011 22:04:25 +0100, Mike Copeland<> wrote:
    > >>

    > > It's "enormous" to me, as I have many other large data structures in
    > > this program, and I have to declare it globally (it's accessed from a
    > > number of subprograms in several source files). My concern is Heap
    > > allocation, inasmuch as there must be _some_ limit as to how much
    > > program data can be declared before the program chokes.

    >
    > That depends on the configuration (physical and OS) of the target system.
    >
    > > I'm using VS6.0, and I don't know what that limit is.

    >
    > Stop right there! An obsolete, non-standard and buggy compiler? I'm
    > sure you can do better than that.


    <sigh...> Yes, I hear that all the time I reveal that. However, I'm
    an (old) independent developer with limited resources...and 6.0 is all I
    possess. I know there there are various Express versions available for
    no cost, but the effort to convert to one (with STL, MFC, etc.
    limitations that might be overwhelming) is too daunting to start at my
    age (72).
    VS6.0 works for me, and I have only one client. Thus, for many
    reasons I don't need to be - nor wish to be - competitive. That may
    seem to be heresy to most here, but I don't have the time or energy to
    "keep up" with you guys...8<{{

    > > I was hoping that I could cut down the total size of this data
    > > component and still process it efficiently. It seems not possible...
    > > 8<{{

    >
    > Yes you could cut down the size, but that is often orthogonal to
    > efficient processing.
    >

    Yes, I see, and I appreciate that thought. This was one of my
    concerns.
     
    Mike Copeland, Nov 15, 2011
    #6
  7. On Tue, 15 Nov 2011 14:04:25 -0700, Mike Copeland wrote:

    > I need to declare an enormous array (100,000 members) of a small scalar
    > object, and I want to populate it with default values and change some of
    > them as needed throughout the processing. The range of distinct values
    > for each element is 0-15, so I could use a 4 bit "mini-byte" if
    > something like that existed.
    > My question is: can I declare a bit structure data type and use it
    > this way? If that's possible, I don't know how to do so, nor how I'd
    > manipulate the data values in it. Please advise. TIA


    Dead easy.

    type Element is range 0 .. 15;
    constant Default_Element : Element := 1;
    For Element'size use 4;
    type Element_Array is array (range 1 .. 100000) of Element
    := (others => Default_Element);

    oops, this is comp.lang.c++. Sorry.

    - Brian
     
    Brian Drummond, Nov 15, 2011
    #7
  8. Mike Copeland

    Jorgen Grahn Guest

    On Tue, 2011-11-15, Mike Copeland wrote:
    > I need to declare an enormous array (100,000 members) of a small
    > scalar object, and I want to populate it with default values and change
    > some of them as needed throughout the processing.


    Why does it have to be an array? It would be easier to handle this if
    you could create a class MyBundleOf100000SmallThings and use one of
    those. Internally you may pack them into a std::vector<unsigned char>
    or something. You won't easily be able to say

    enormous[4711] = 2;

    though, although you can easily /read/ from it using that syntax.

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .
     
    Jorgen Grahn, Nov 15, 2011
    #8
  9. Mike Copeland

    Ian Collins Guest

    On 11/16/11 11:19 AM, Jorgen Grahn wrote:
    > On Tue, 2011-11-15, Mike Copeland wrote:
    >> I need to declare an enormous array (100,000 members) of a small
    >> scalar object, and I want to populate it with default values and change
    >> some of them as needed throughout the processing.

    >
    > Why does it have to be an array? It would be easier to handle this if
    > you could create a class MyBundleOf100000SmallThings and use one of
    > those. Internally you may pack them into a std::vector<unsigned char>
    > or something. You won't easily be able to say
    >
    > enormous[4711] = 2;
    >
    > though, although you can easily /read/ from it using that syntax.


    Both operators are possible through the use of a simple proxy object.

    --
    Ian Collins
     
    Ian Collins, Nov 15, 2011
    #9
  10. On Tue, 15 Nov 2011 22:30:03 +0100, Mike Copeland <> wrote:

    > In article <>,
    > says...
    >> On Tue, 15 Nov 2011 22:04:25 +0100, Mike Copeland <>
    >> wrote:
    >>
    >> > I need to declare an enormous array (100,000 members) of a small
    >> > scalar object, and I want to populate it with default values and

    >> change
    >> > some of them as needed throughout the processing. The range of

    >> distinct
    >> > values for each element is 0-15, so I could use a 4 bit "mini-byte" if
    >> > something like that existed.
    >> > My question is: can I declare a bit structure data type and use it
    >> > this way? If that's possible, I don't know how to do so, nor how I'd
    >> > manipulate the data values in it. Please advise. TIA

    >>
    >> Are you sure that the array is "enormous"? Using one (whole!) byte per
    >> element it would occupy less than 100KiB.
    >>

    > It's "enormous" to me, as I have many other large data structures in
    > this program, and I have to declare it globally (it's accessed from a
    > number of subprograms in several source files). My concern is Heap
    > allocation, inasmuch as there must be _some_ limit as to how much
    > program data can be declared before the program chokes. I;m using
    > VS6.0, and I don't know what that limit is.
    > I was hoping that I could cut down the total size of this data
    > component and still process it efficiently. It seems not possible...
    > 8<{{


    It's possible but my advice is "don't do it yet". It seems a premature
    optimization concern. Wrap a simple

    typedef unsigned char byte ; // guaranteed to hold at least [0..255]
    std::vector< byte > v ;

    into a class of your own, which you can change later (if needed) to use
    a packed representation.

    --
    Gennaro Prota | name.surname gmail.com
    Breeze C++ (preview): <http://sourceforge.net/projects/breeze/>
    Do you need expertise in C++? I'm available.
     
    Gennaro Prota, Nov 15, 2011
    #10
  11. On 11/15/2011 5:02 PM, Mike Copeland wrote:
    > In article<>,
    > says...
    >> On 11/16/11 10:30 AM, Mike Copeland wrote:
    >>> I was hoping that I could cut down the total size of this data
    >>> component and still process it efficiently. It seems not possible...
    >>> 8<{{

    >>
    >> Yes you could cut down the size, but that is often orthogonal to
    >> efficient processing.
    >>

    > Yes, I see, and I appreciate that thought. This was one of my
    > concerns.


    Actually, not orthogonal, but often opposed. If you want to save
    memory, you will often sacrifice speed and vice versa. Implement a
    simple vector<char> and see if the speed is sufficient. If not, make
    sure you have the tools to find out where the time is spent in your
    program, *before* you attempt to speed it up.

    V
    --
    I do not respond to top-posted replies, please don't ask
     
    Victor Bazarov, Nov 15, 2011
    #11
  12. Mike Copeland

    Jorgen Grahn Guest

    On Tue, 2011-11-15, Ian Collins wrote:
    > On 11/16/11 11:19 AM, Jorgen Grahn wrote:
    >> On Tue, 2011-11-15, Mike Copeland wrote:
    >>> I need to declare an enormous array (100,000 members) of a small
    >>> scalar object, and I want to populate it with default values and change
    >>> some of them as needed throughout the processing.

    >>
    >> Why does it have to be an array? It would be easier to handle this if
    >> you could create a class MyBundleOf100000SmallThings and use one of
    >> those. Internally you may pack them into a std::vector<unsigned char>
    >> or something. You won't easily be able to say
    >>
    >> enormous[4711] = 2;
    >>
    >> though, although you can easily /read/ from it using that syntax.

    >
    > Both operators are possible through the use of a simple proxy object.


    Thus "won't easily be able to". But yes -- it's worth pointing out
    that if he really wants array-style syntax he can get it. (I think
    I'd normally settle for a small_things.set(index, value) function.)

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .
     
    Jorgen Grahn, Nov 16, 2011
    #12
  13. On Nov 15, 9:30 pm, (Mike Copeland) wrote:
    > In article <>,
    > says...> On Tue, 15 Nov 2011 22:04:25 +0100, Mike Copeland <> wrote:


    > > >    I need to declare an enormous array (100,000 members) of a small
    > > > scalar object, and I want to populate it with default values and change
    > > > some of them as needed throughout the processing.  The range of distinct
    > > > values for each element is 0-15, so I could use a 4 bit "mini-byte"


    the common term for a 4-bit quantity is a "nibble"

    Is your data sparse? That is, are there many more default values than
    other values? If so consider a sparse array. Only store the non-
    default values and return default if the lookup fails.

    > > > if
    > > > something like that existed.
    > > >    My question is: can I declare a bit structure data type and use it
    > > > this way?  If that's possible, I don't know how to do so, nor how I'd
    > > > manipulate the data values in it.  Please advise.  TIA


    there are bit fields but I've never been keen. I normally use explicit
    masks and shifts to extract bit fields. Something like this:-

    <code>

    typedef unsigned char Byte;
    typedef unsigned char Nibble;

    class NibbleContainer
    {
    public:
    NibbleContainer(size_t size, Nibble default_value = 0);

    size_t size() const { return m_size; }

    void set (size_t i, Nibble nib);
    Nibble get (size_t i) const;
    private:
    size_t m_size;
    std::vector<Byte> m_vec;
    };

    inline bool is_odd (size_t n) { return n % 2 == 1; }

    NibbleContainer::NibbleContainer(size_t size, Nibble default_value):
    m_size(size), m_vec ((size + 1) / 2)
    {
    for (size_t i = 0; i < size; i++)
    set (i, default_value);
    }

    void NibbleContainer::set (size_t i, Nibble nib)
    {
    assert (i < m_size);
    assert (nib < 16);

    size_t vec_index = i / 2;

    if (is_odd (i))
    {
    // stick it in the top nibble
    nib <<= 4;
    m_vec [vec_index] &= 0x0f;
    m_vec [vec_index] |= nib;
    }
    else
    {
    // stick it in the bottom nibble
    m_vec [vec_index] &= 0xf0;
    m_vec [vec_index] |= nib;
    }
    }

    Nibble NibbleContainer::get (size_t i) const
    {
    assert (i < m_size);
    size_t vec_index = i / 2;

    if (is_odd (i))
    // get it from the top nibble
    return (m_vec [vec_index] >> 4) & 0x0f;
    else
    // get it from the bottom nibble
    return m_vec [vec_index] & 0x0f;

    }

    </code>

    now that amount of c-like c++ is bound to provoke a reaction...

    > > Are you sure that the array is "enormous"? Using one (whole!) byte per
    > > element it would occupy less than 100KiB.

    >
    >    It's "enormous" to me, as I have many other large data structures in
    > this program, and I have to declare it globally (it's accessed from a
    > number of subprograms in several source files).  My concern is Heap
    > allocation, inasmuch as there must be _some_ limit as to how much
    > program data can be declared before the program chokes.  I;m using
    > VS6.0, and I don't know what that limit is.
    >    I was hoping that I could cut down the total size of this data
    > component and still process it efficiently.  It seems not possible...


    depends what "efficient" means. How many non-defaults do you have? How
    often do they change? How often do you read a value? How often do you
    write a value? etc. etc. You need to measure things and get some hard
    numebrs.
     
    Nick Keighley, Nov 16, 2011
    #13
  14. Mike Copeland

    Jorgen Grahn Guest

    On Wed, 2011-11-16, Nick Keighley wrote:
    > On Nov 15, 9:30 pm, (Mike Copeland) wrote:
    >> In article <>,
    >> says...> On Tue, 15 Nov 2011 22:04:25 +0100, Mike Copeland <> wrote:

    >
    >> > >    I need to declare an enormous array (100,000 members) of a small
    >> > > scalar object, and I want to populate it with default values and change
    >> > > some of them as needed throughout the processing.  The range of distinct
    >> > > values for each element is 0-15, so I could use a 4 bit "mini-byte"

    >
    > the common term for a 4-bit quantity is a "nibble"

    ....

    > there are bit fields but I've never been keen. I normally use explicit
    > masks and shifts to extract bit fields. Something like this:-
    >
    > <code>
    >
    > typedef unsigned char Byte;
    > typedef unsigned char Nibble;
    >
    > class NibbleContainer
    > {
    > public:
    > NibbleContainer(size_t size, Nibble default_value = 0);
    >
    > size_t size() const { return m_size; }
    >
    > void set (size_t i, Nibble nib);
    > Nibble get (size_t i) const;
    > private:
    > size_t m_size;
    > std::vector<Byte> m_vec;
    > };

    ....
    > </code>
    >
    > now that amount of c-like c++ is bound to provoke a reaction...


    If all C-like C++ was /that/ C++-like, I wouldn't dislike it so much!

    Seriously, it looked perfectly fine. I'd do it exactly like that
    (except I probably would use 'unsigned' instead of that 'Nibble'
    typedef).

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .
     
    Jorgen Grahn, Nov 16, 2011
    #14
  15. On Nov 16, 5:16 pm, Jorgen Grahn <> wrote:
    > On Wed, 2011-11-16, Nick Keighley wrote:
    > > On Nov 15, 9:30 pm, (Mike Copeland) wrote:
    > >> In article <>,
    > >> says...> On Tue, 15 Nov 2011 22:04:25 +0100, Mike Copeland <> wrote:



    > >> > >    I need to declare an enormous array (100,000 members) of a small
    > >> > > scalar object, and I want to populate it with default values and change
    > >> > > some of them as needed throughout the processing.  The range of distinct
    > >> > > values for each element is 0-15, so I could use a 4 bit "mini-byte"

    >
    > > the common term for a 4-bit quantity is a "nibble"

    >
    > ...
    >
    > > there are bit fields but I've never been keen. I normally use explicit
    > > masks and shifts to extract bit fields. Something like this:-

    >
    > > <code>

    >
    > > typedef unsigned char Byte;
    > > typedef unsigned char Nibble;

    >
    > > class NibbleContainer
    > > {
    > > public:
    > >     NibbleContainer(size_t size, Nibble default_value = 0);

    >
    > >     size_t size() const { return m_size; }

    >
    > >     void set (size_t i, Nibble nib);
    > >     Nibble get (size_t i) const;
    > > private:
    > >     size_t m_size;
    > >     std::vector<Byte> m_vec;
    > > };

    > ...
    > > </code>

    >
    > > now that amount of c-like c++ is bound to provoke a reaction...

    >
    > If all C-like C++ was /that/ C++-like, I wouldn't dislike it so much!
    >
    > Seriously, it looked perfectly fine. I'd do it exactly like that
    > (except I probably would use 'unsigned' instead of that 'Nibble'
    > typedef).


    I was trying to express the semantics that it was intended to hold
    only 4 bits rather than 8. But without the cost of full-blown class.

    Even if my class doesn't serve his purpose once he's encapsulated it
    he can change the representation without it messing up the rest of his
    program. It can be a straight vector (as other posters have suggested)
    or my packed representation or a sparse array (or hash or tree). etc.
    etc.

    A good software engineerign lesson is if you have hard decision (or
    any decsion!) to make then hide it from the rest of the program. In C+
    + a class is often a good way to do this (not to denigrate templates
    etc.).
     
    Nick Keighley, Nov 17, 2011
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Howard
    Replies:
    1
    Views:
    1,212
  2. kelvSYC
    Replies:
    6
    Views:
    7,376
    Richard Herring
    May 17, 2005
  3. Replies:
    2
    Views:
    439
  4. hvt
    Replies:
    0
    Views:
    1,242
  5. Luca Forlizzi
    Replies:
    4
    Views:
    487
    Luca Forlizzi
    Nov 14, 2010
Loading...

Share This Page