Interesting utilities

Discussion in 'C Programming' started by jacob navia, Oct 11, 2011.

  1. jacob navia

    jacob navia Guest

    I am developing in the Mac OS X (10.7) system, and it has some nice
    features that I hadn't seen in any windows/unix system.

    One of them is the "mdfind" utility that is incredibly powerful:

    mdfind some-identifier

    will look in the whole machine for "some-identifier" and write you in
    standard output the name of all files that have a match. It is like

    grep -R "some-identifier" /*.*

    but it comes back in seconds instead of hours like grep.

    It is very useful, and I never found something even remotely equivalent
    under linux/sun os

    Another nifty thing is the splint code checker. (This one is probably
    present under linux also but I never saw it under windows)

    The problem with it is that it is incredibly verbose, swamping the
    few interesting diagonstics under a mountain of rubbish. After a long
    session with it I have trimmed it to give me reasonable short output.

    I put all options in a shell script:

    /bin/sh
    splint -formatconst -dependenttrans -globstate -compdef -mayaliasunique
    -paramuse +boolint -immediatetrans -retvalother -noeffect
    -unqualifiedtrans -compdestroy -temptrans -branchstate -usereleased
    -kepttrans -compmempass -nullstate -nullpass -nullret -boolops
    -mustfreeonly -incondefs +charint -retvalint -predboolint -realcompare $1
     
    jacob navia, Oct 11, 2011
    #1
    1. Advertising

  2. jacob navia wrote:
    >I am developing in the Mac OS X (10.7) system, and it has some nice
    >features that I hadn't seen in any windows/unix system.
    >
    >mdfind some-identifier ...
    >but it comes back in seconds instead of hours like grep.


    That probably means that it looks into a keyword database, built by
    indexing the files in the background, over many hours instead of
    seconds.
    And yes, the same functionality is available under Linux and Windows.

    >Another nifty thing is the splint code checker. (This one is probably
    >present under linux also but I never saw it under windows)


    A windows installer is available at the home page:
    http://www.splint.org/

    --
    Roberto Waltman

    [ Please reply to the group,
    return address is invalid ]
     
    Roberto Waltman, Oct 15, 2011
    #2
    1. Advertising

  3. jacob navia

    jacob navia Guest

    Le 15/10/11 06:35, Roberto Waltman a écrit :
    > jacob navia wrote:
    >> I am developing in the Mac OS X (10.7) system, and it has some nice
    >> features that I hadn't seen in any windows/unix system.
    >>
    >> mdfind some-identifier ...
    >> but it comes back in seconds instead of hours like grep.

    >
    > That probably means that it looks into a keyword database, built by
    > indexing the files in the background, over many hours instead of
    > seconds.
    > And yes, the same functionality is available under Linux and Windows.



    Sorry but I have never even heard about something like that under
    windows/linux.
    Maybe you have an example?

    >
    >> Another nifty thing is the splint code checker. (This one is probably
    >> present under linux also but I never saw it under windows)

    >
    > A windows installer is available at the home page:
    > http://www.splint.org/
    >


    Thanks, I overlooked that one.
     
    jacob navia, Oct 15, 2011
    #3
  4. jacob navia

    Geoff Guest

    On Sat, 15 Oct 2011 13:06:46 +0200, jacob navia <>
    wrote:

    >Sorry but I have never even heard about something like that under
    >windows/linux.
    >Maybe you have an example?


    It's called Spotlight on OS/X and Windows Search since the release of
    Windows XP. I can't remember what it was called in Windows 2000.
    Settings are available in the Windows Control panel applet named
    "Indexing Options" on Windows 7.
     
    Geoff, Oct 15, 2011
    #4
  5. [OT] Interesting utilities

    jacob navia wrote:

    [ re: File indexing / search applications ]
    > ... something like that under
    >windows/linux.
    >Maybe you have an example?


    http://en.wikipedia.org/wiki/Windows_Search
    http://www.microsoft.com/download/en/details.aspx?id=23
    http://www.vandenoever.info/software/strigi/akademy2006.pdf
    http://www.wikinfo.org/index.php/Comparison_of_desktop_search_software
    http://en.wikipedia.org/wiki/Beagle_(software)

    The last page says the Beagle project, (only one I used under Linux,)
    is not being developed anymore. I was not aware of that.

    --
    Roberto Waltman

    [ Please reply to the group,
    return address is invalid ]
     
    Roberto Waltman, Oct 15, 2011
    #5
  6. jacob navia

    Kaz Kylheku Guest

    On 2011-10-11, jacob navia <> wrote:
    > I am developing in the Mac OS X (10.7) system, and it has some nice
    > features that I hadn't seen in any windows/unix system.
    >
    > One of them is the "mdfind" utility that is incredibly powerful:
    >
    > mdfind some-identifier
    >
    > will look in the whole machine for "some-identifier" and write you in
    > standard output the name of all files that have a match. It is like
    >
    > grep -R "some-identifier" /*.*
    >
    > but it comes back in seconds instead of hours like grep.


    So it is not more powerful than grep, since it computes the same thing. The
    word you are looking for is "faster".

    Can it find any regular expression in seconds?

    > It is very useful, and I never found something even remotely equivalent
    > under linux/sun os


    How hard do you look?

    I use id-utils and ctags for doing this over programming projects.
    In id-utils there is a "mkid" utility which indexes the identifiers found
    in a directory tree. This is gathered into a big file called ID.
    The ID file is queried with "lid".

    I just set up an Ubuntu system where "apt-get install id-utils"
    got me mkid in seconds.

    > Another nifty thing is the splint code checker. (This one is probably
    > present under linux also but I never saw it under windows)


    There is a direct pointer to a Windows download page on the Splint
    home page! This kind of thing is not going to come preinstalled on a Wintel PC
    from Best Buy.

    > The problem with it is that it is incredibly verbose, swamping the
    > few interesting diagonstics under a mountain of rubbish. After a long
    > session with it I have trimmed it to give me reasonable short output.


    This is by design. To get value out of splint you have to read the
    documentation and start using the annotations.

    Splint can statically check things like that malloced objects are freed
    somewhere, at least for some programs (maybe not in general).
     
    Kaz Kylheku, Oct 16, 2011
    #6
  7. jacob navia

    Kaz Kylheku Guest

    On 2011-10-15, jacob navia <> wrote:
    > Le 15/10/11 06:35, Roberto Waltman a écrit :
    >> jacob navia wrote:
    >>> I am developing in the Mac OS X (10.7) system, and it has some nice
    >>> features that I hadn't seen in any windows/unix system.
    >>>
    >>> mdfind some-identifier ...
    >>> but it comes back in seconds instead of hours like grep.

    >>
    >> That probably means that it looks into a keyword database, built by
    >> indexing the files in the background, over many hours instead of
    >> seconds.
    >> And yes, the same functionality is available under Linux and Windows.

    >
    > Sorry but I have never even heard about something like that under
    > windows/linux.
    > Maybe you have an example?


    http://freshmeat.net/search?q=file indexing&submit=Search

    45 hits, some of them relevant-looking.

    http://freshmeat.net/search?q=file search&submit=Search

    251 hits.

    Man, you really are still a goof, sorry to say.

    Windows has dynamic file indexing since Vista, if not before.
     
    Kaz Kylheku, Oct 16, 2011
    #7
  8. jacob navia

    Ian Collins Guest

    On 10/16/11 02:18 PM, Kaz Kylheku wrote:
    > On 2011-10-11, jacob navia<> wrote:
    >> I am developing in the Mac OS X (10.7) system, and it has some nice
    >> features that I hadn't seen in any windows/unix system.
    >>
    >> One of them is the "mdfind" utility that is incredibly powerful:
    >>
    >> mdfind some-identifier
    >>
    >> will look in the whole machine for "some-identifier" and write you in
    >> standard output the name of all files that have a match. It is like
    >>
    >> grep -R "some-identifier" /*.*
    >>
    >> but it comes back in seconds instead of hours like grep.

    >
    > So it is not more powerful than grep, since it computes the same thing. The
    > word you are looking for is "faster".
    >
    > Can it find any regular expression in seconds?
    >
    >> It is very useful, and I never found something even remotely equivalent
    >> under linux/sun os

    >
    > How hard do you look?
    >
    > I use id-utils and ctags for doing this over programming projects.
    > In id-utils there is a "mkid" utility which indexes the identifiers found
    > in a directory tree. This is gathered into a big file called ID.
    > The ID file is queried with "lid".
    >
    > I just set up an Ubuntu system where "apt-get install id-utils"
    > got me mkid in seconds.


    These utilities are all well and good on a relatively static filesystem,
    but the fun begins when you want to keep the database as up to date as
    possible.

    I know all about the fun involved, I have to maintain an index over a
    couple of multi-TB filesystems with tens of millions of files in each!

    --
    Ian Collins
     
    Ian Collins, Oct 16, 2011
    #8
  9. jacob navia

    Kaz Kylheku Guest

    On 2011-10-16, XeCycle <> wrote:
    > jacob navia <> writes:
    >> It is very useful, and I never found something even remotely
    >> equivalent under linux/sun os

    >
    > What about `locate' from GNU Findutils?


    Locate searches only file names, not their contents.

    That's been a staple of Linux distros as far as I can remember (which could
    potentially takes us back as far as 1993).

    Long before operating systems "for the rest of them" started getting
    any kind of indexing.
     
    Kaz Kylheku, Oct 16, 2011
    #9
  10. jacob navia

    jacob navia Guest

    Le 16/10/11 03:24, Kaz Kylheku a écrit :
    > On 2011-10-15, jacob navia<> wrote:
    >> Le 15/10/11 06:35, Roberto Waltman a écrit :
    >>> jacob navia wrote:
    >>>> I am developing in the Mac OS X (10.7) system, and it has some nice
    >>>> features that I hadn't seen in any windows/unix system.
    >>>>
    >>>> mdfind some-identifier ...
    >>>> but it comes back in seconds instead of hours like grep.
    >>>
    >>> That probably means that it looks into a keyword database, built by
    >>> indexing the files in the background, over many hours instead of
    >>> seconds.
    >>> And yes, the same functionality is available under Linux and Windows.

    >>
    >> Sorry but I have never even heard about something like that under
    >> windows/linux.
    >> Maybe you have an example?

    >
    > http://freshmeat.net/search?q=file indexing&submit=Search
    >
    > 45 hits, some of them relevant-looking.
    >
    > http://freshmeat.net/search?q=file search&submit=Search
    >
    > 251 hits.
    >
    > Man, you really are still a goof, sorry to say.
    >
    > Windows has dynamic file indexing since Vista, if not before.


    Sure, but it doesn't index everything, as I have seen lately.
    Only some selected directories. SUre you can change that but still,
    it doesn't match the easy of use of not doing anything
    :)

    I used the find utility in windows XP, and the terrible interface
    with the dog just convinced me to avoid it completely. In any case it
    wasn't a background process as it is in the mac


    Under linux there *could* be an indexing utility around of course,
    but it is not automatic like mdfind.

    What is interesting in the mac is the number of automatic utilities
    like mdfind, for instance the indredibly useful backup utility, that
    work out of the box without any need for big configurations, etc.
     
    jacob navia, Oct 16, 2011
    #10
  11. jacob navia

    jacob navia Guest

    Le 16/10/11 03:24, Kaz Kylheku a écrit :
    >> Maybe you have an example?

    >
    > http://freshmeat.net/search?q=file indexing&submit=Search
    >
    > 45 hits, some of them relevant-looking.
    >
    > http://freshmeat.net/search?q=file search&submit=Search
    >
    > 251 hits.
    >
    > Man, you really are still a goof, sorry to say.
    >
    > Windows has dynamic file indexing since Vista, if not before.


    Sure sure....

    In Wikipedia the program "tracker" is listed as a search utility for
    linux.

    Downloaded the package from
    http://projects.gnome.org/tracker
    Then:

    bzip2 -d tracker-0.1.12.bz2
    untarred result
    changed to the direzctory and typed ./configure
    ../configure fails with:
    configure: error: glib-compile-schemas not found.

    Searched google for that.

    After much searching I found out that actually I have that package
    installed in /usr/bin.

    Apparently I am not the only one havinng that problem, there are many
    hits about that error message. But there are no solutions. Some of
    the solutions point to upgrading to a newer version of the OS...
    but I am running already the latest Ubuntu (I try to keep current since
    under linux if you aren't upgrading every 2-3 months the OS nothing
    will work any more)

    The problem is that I have the package but the
    configure script doesn't fdind it (it is in /usr/bin... I really do not
    understand why that directory is not searched)

    I edit the configure script and see that there is an environment
    variable that should contain the path to those "compile-schemas"
    I set the variable, rerun, nothing, same error.

    Now I discover that I am not really a shell/ configure / language
    expert. I can't figure out why the thing isn't found I give up.

    Wasted time: Approx 1 hour.

    mdfind came with the mac already, nothing to do for me.

    But of course I am a "goofer"...

    Look Mr Kaz, I am well aware that an indexing utility could
    exist for linux even one that after several hours of work
    could even work better than mdfind.

    But... really, I do not feel like wasting more time in it.
    Yes, it is free, I did not pay for my Ubuntu version. But I
    have learned the hard way that unless you want to specialize in
    linux administration you will be wasting so much time with it that it
    is just not worth trying to do things like this for somebody
    that is not a linux admin...


    I wonder how would I explain the usage of the linux utility to
    my wife...

    "Look Annie, just download that source package"

    Then, there is no install program, you have to type
    ../configure

    etc...
    `
    Good luck with that

    :-(

    jacob
     
    jacob navia, Oct 16, 2011
    #11
  12. jacob navia

    jacob navia Guest

    Well, there was another way, sorry.

    The ubuntu distribution has a package for tracker binaries.
    Excellent!

    I download the package and apparently installs without problems.

    Obviously there isn't any documentation at all. After some searching
    I discover that it installed in "Accessories". I click in the tracker
    line and a window appears with a text field called "Search"

    I type "iList" and press return. Since I downloaded the source code
    of the containers library in a subdirectory under /home/jacob
    I know that this word appears.

    Nothing, nothing changes, everything remains empty. Mmmmmm...
    Maybe I haven't configured something?

    Open a shell "man tracker" gives no answer, no documentation.

    I type

    tracker-<TAB>

    and see a list of tracker programs. I call the configuration
    program and... yes, my home directory is listed under the
    directories to be searched recursively.

    I see another utility: tracker-search. I write

    $ tracker-search iList
    Results: 0

    Ahhh that is why everything is empty. It doesn't find anything.
    Why?

    Mistery. The configuration program says that it has written the
    configuration and I see /home/jacob in the recursive directory
    list.

    I go to /home/jacob/ccl and write:

    $ grep iList *.c
    containers.h: extern ListInterface iList;

    Yes, I am not dreaming...

    Well, I do not know what is going on. There must be some configuration
    option I did not find out. Probably after some hours of work I could
    get this stuff running, who knows?
     
    jacob navia, Oct 16, 2011
    #12
  13. jacob navia

    Kaz Kylheku Guest

    On 2011-10-16, jacob navia <> wrote:
    > Well, there was another way, sorry.
    >
    > The ubuntu distribution has a package for tracker binaries.
    > Excellent!


    Yes, I just found it in three seconds while looking at your
    other post:

    Is this it?

    http://packages.ubuntu.com/lucid/tracker

    I would not try to compile from scratch packages that have any
    complex dependencies. (Not as a Ubuntu user, that is.)

    > I download the package and apparently installs without problems.
    > Obviously there isn't any documentation at all. After some searching


    I searched the web instead to find the home of this thing.

    http://projects.gnome.org/tracker/documentation.html

    "All the documentation is on the live wiki"

    http://live.gnome.org/Tracker/Documentation/GettingStarted

    Much of the documentation is developer-oriented. This looks like an
    experimental new project of some sorts.

    > Well, I do not know what is going on. There must be some configuration
    > option I did not find out. Probably after some hours of work I could
    > get this stuff running, who knows?


    The wiki page says there is is a:

    "tracker-miner-fs | The file system crawler to mine data from local files"

    I doubt that you will find anything if this has not run and done its job.
     
    Kaz Kylheku, Oct 17, 2011
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. _eee_
    Replies:
    3
    Views:
    492
    Rick Strahl [MVP]
    Feb 28, 2004
  2. Ganesh
    Replies:
    4
    Views:
    378
    Thomas Weidenfeller
    Jan 31, 2005
  3. Marcus Leon

    Ant Utilities?

    Marcus Leon, Feb 4, 2005, in forum: Java
    Replies:
    1
    Views:
    686
    Bryce
    Feb 4, 2005
  4. James Yong
    Replies:
    2
    Views:
    3,611
  5. JKV
    Replies:
    0
    Views:
    416
Loading...

Share This Page