A question for Leigh

Discussion in 'C++' started by Paul, Apr 27, 2011.

  1. Paul

    Paul Guest

    Your recently posted tokeniser functionS seem to be a bit buggy.
    Please see the following, which seems to exhibit a bug in your code:

    /**************code*******************/
    #include <iostream>
    #include <vector>
    #include <string>
    #include <algorithm>


    template <typename C, typename FwdIter1, typename FwdIter2>
    inline FwdIter1 do_tokens(FwdIter1 aFirst, FwdIter1 aLast, FwdIter2
    aDelimeterFirst, FwdIter2 aDelimiterLast, C& aTokens, std::size_t
    aMaxTokens = 0, bool aSkipEmptyTokens = true, bool
    aDelimeterIsSubsequence = false)
    {
    if (aFirst >= aLast)
    return aFirst;

    typedef typename C::value_type value_type;

    FwdIter1 b = aFirst;
    FwdIter1 e = aDelimeterIsSubsequence ? std::search(b, aLast,
    aDelimeterFirst, aDelimiterLast) : std::find_first_of(b, aLast,
    aDelimeterFirst, aDelimiterLast);
    std::size_t tokens = 0;
    while(e != aLast && (aMaxTokens == 0 || tokens < aMaxTokens))
    {
    if (b == e && !aSkipEmptyTokens)
    {
    aTokens.push_back(value_type(b, b));
    ++tokens;
    }
    else if (b != e)
    {
    aTokens.push_back(value_type(b, e));
    ++tokens;
    }
    b = e;
    for (std::size_t i = aDelimeterIsSubsequence ?
    std::distance(aDelimeterFirst, aDelimiterLast) : 1; i > 0; --i)
    ++b;
    e = aDelimeterIsSubsequence ? std::search(b, aLast, aDelimeterFirst,
    aDelimiterLast) : std::find_first_of(b, aLast, aDelimeterFirst,
    aDelimiterLast);
    }
    if (b != e && (aMaxTokens == 0 || tokens < aMaxTokens))
    {
    aTokens.push_back(value_type(b, e));
    b = e;
    }
    return b;
    }
    template <typename C, typename FwdIter1, typename FwdIter2>
    inline FwdIter1 tokens(FwdIter1 aFirst, FwdIter1 aLast, FwdIter2
    aDelimeterFirst, FwdIter2 aDelimiterLast, C& aTokens, std::size_t
    aMaxTokens = 0, bool aSkipEmptyTokens = true, bool
    aDelimeterIsSubsequence = false)
    {
    return do_tokens(aFirst, aLast, aDelimeterFirst, aDelimiterLast,
    aTokens, aMaxTokens, aSkipEmptyTokens, aDelimeterIsSubsequence);
    }
    template <typename CharT, typename Traits, typename Alloc, typename C>
    inline void tokens(const std::basic_string<CharT, Traits, Alloc>&
    aLine, const std::basic_string<CharT, Traits, Alloc>& aDelimeter, C&
    aTokens, std::size_t aMaxTokens = 0, bool aSkipEmptyTokens = true, bool
    aDelimeterIsSubsequence = false)
    {
    do_tokens(aLine.begin(), aLine.end(), aDelimeter.begin(),
    aDelimeter.end(), aTokens, aMaxTokens, aSkipEmptyTokens,
    aDelimeterIsSubsequence);
    }


    int main(){
    typedef std::vector<std::string> words_t;
    words_t words;
    char p[] ="abcd\0xyz\0mnop\0vvv";
    std::string source(p, 18);
    std::string delim = "\0";
    tokens(source, delim, words);
    for (words_t::const_iterator i = words.begin(); i != words.end(); ++i)
    std::cout << "[" << *i << "] ";
    }
    /**********end code*******************/


    Maybe my compiler is buggy, because i have not traced your code to bother
    finding the exact problem, I simply compiled on command line but I find
    unexpected output which suggests your tokeniser functions may be
    problematic. Perhaps you could add another function or two to sort out this
    problem. :)

    I suggest you start again from scratch because there seems to be something
    noobish about your use of templates.
    One example is your use template typenames such as "FwdIter1" when a char*
    is excepted.
    Perhaps you wrote this code along time ago , and I have taught you alot
    since then , therefore you will be in a better position now to rewrite this
    code so that it does not contain any bugs, inappropriate template names,
    bloatation or lack of CPU efficiency..

    HTH & GL.
     
    Paul, Apr 27, 2011
    #1
    1. Advertising

  2. Paul

    Paul Guest

    "Paul" <> wrote in message
    news:S41up.44236$2...
    > Your recently posted tokeniser functionS seem to be a bit buggy.
    > Please see the following, which seems to exhibit a bug in your code:
    >

    C & P error . Please replace previous main with the following:

    int main(){
    typedef std::vector<std::string> words_t;
    words_t words;
    wchar_t p[4] = {'a', '\0', '\0', 'b'};
    std::wstring source(p, 4);
    std::string delim = "\0";
    tokens(source.begin(),source.end(), delim.begin(), delim.end(), words);
    for (words_t::const_iterator i = words.begin(); i != words.end(); ++i)
    std::cout << "[" << *i << "] ";
    }
     
    Paul, Apr 28, 2011
    #2
    1. Advertising

  3. Paul

    Paul Guest

    "Paul" <> wrote in message
    news:hh2up.41026$2...
    >
    > "Paul" <> wrote in message
    > news:S41up.44236$2...
    >> Your recently posted tokeniser functionS seem to be a bit buggy.
    >> Please see the following, which seems to exhibit a bug in your code:
    >>

    Ok this one breaks it:

    int main(){
    typedef std::vector<std::string> words_t;
    words_t words;
    char c[4] = {'\0', '\0', '\0', 'b'};
    std::string source(c, 4);
    wchar_t wc = '\0';
    std::wstring delim(wc,1);
    tokens(source.begin(),source.end(), delim.begin(), delim.end(), words);
    for (words_t::const_iterator i = words.begin(); i != words.end(); ++i)
    std::cout << "[" << *i << "] ";
    }


    Your templates aint typesafe. I knew it was breakable as soon as I seen it.
     
    Paul, Apr 28, 2011
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. sean
    Replies:
    1
    Views:
    620
    Cowboy \(Gregory A. Beamer\)
    Oct 20, 2003
  2. =?Utf-8?B?UnlhbiBTbWl0aA==?=

    Quick Question - Newby Question

    =?Utf-8?B?UnlhbiBTbWl0aA==?=, Feb 14, 2005, in forum: ASP .Net
    Replies:
    4
    Views:
    691
    Iain Norman
    Feb 16, 2005
  3. =?Utf-8?B?YW5kcmV3MDA3?=

    question row filter (more of sql query question)

    =?Utf-8?B?YW5kcmV3MDA3?=, Oct 5, 2005, in forum: ASP .Net
    Replies:
    2
    Views:
    970
    Scott Allen
    Oct 6, 2005
  4. Philip Meyer
    Replies:
    0
    Views:
    442
    Philip Meyer
    Nov 30, 2003
  5. 88888 Dihedral

    Re: Leigh's GUI [OT]

    88888 Dihedral, Jan 28, 2012, in forum: C++
    Replies:
    9
    Views:
    290
    Ian Collins
    Jan 31, 2012
Loading...

Share This Page