[ANN] Metadata 1.0-rc2

Discussion in 'Ruby' started by Ilmari Heikkinen, Sep 19, 2007.

  1. Getting close to the 1.0 release of the metadata library.
    If there are no major problems with this release candidate, that is.

    tarball: http://dark.fhtr.org/repos/metadata/metadata-1.0-rc2.tar.gz
    git: http://dark.fhtr.org/repos/metadata

    Changes
    -------
    * bittorrent .torrent support
    * untested ape/musepack/wavepack support with apetag gem
    (couldn't find any ape samples)
    * 'name' and 'author' parsed from mplayer output
    * more documentation
    * better handling of images that have no exif info
    * tested ra, wma and m4a
    * tested more image formats
    * fixed audio samplerate and channels for videos
    (i.e. convert the string to int.)
    * bin/mdh now prints the metadata as the default action.
    Use -c to create an MDH file.
    * MDH files have a "MDH#{version_byte}" header now,
    the validity of which is checked by mdh.

    Thanks
    ------

    Konrad Meyer for his patient testing and bug reports.
    Darren Kirby for the heads-up on wmainfo's ASF-parsing capabilities
    (along with being the author of wmainfo-rb and flacinfo-rb.)


    Description
    -----------

    This package `Metadata' comes with a library called `metadata' and
    a small program called `mdh'.

    The library probes files for their metadata (e.g. jpeg dimensions
    and camera make, mp3 artist, pdf text and word count) and returns the
    metadata as a Hash. All strings in the metadata are converted to UTF-8.

    The `mdh'-program can print out file metadata as YAML and package the
    metadata with the file.

    The metadata hash follows the shared file metadata spec naming, with some
    additional fields, see list at the end of this file (Appendix A.)

    For details on the MDH file format, see the end of this file (Appendix B.)


    Usage
    -----

    # print out metadata for myfile.jpg
    mdh myfile.jpg

    # create myfile.jpg.mdh, which consists of an MDH metadata header + myfile.jpg
    mdh -c myfile.jpg

    # print out the metadata header from an MDH file
    mdh -e -p myfile.jpg.mdh

    # strip out the metadata header from an MDH file and save it to myfile.jpg
    mdh -e myfile.jpg.mdh

    # print out the list of options
    mdh -h

    irb> require 'metadata'
    irb> Metadata.extract('myfile.jpg')
    irb> Metadata.extract_text('myfile.pdf')
    irb> Pathname.new("myfile.jpg").metadata


    List of supported formats
    -------------------------

    Audio:
    Whatever you manage to make mplayer play.
    Plus special handlers for FLAC, m4a, ape, musepack, wavepack and wma.
    Successfully tested with:
    mp3, flac, ogg, wav, ra, wma, m4a
    Should also work:
    wv, mpc, ape

    Video:
    Whatever you manage to make mplayer play.
    Successfully tested with:
    wmv, mov, divx, xvid, flv, ogm, mpg, mkv

    Images:
    Should handle pretty much anything (apart from ORF.)
    Successfully tested with:
    jpeg, png, gif, nef, dng, crw, pef, psd, tga, tif, xcf, xpm, ppm, bmp

    Documents:
    Successfully tested with:
    pdf, ppt, odp, sxi, ps, ps.gz, html, txt
    Should work:
    - OpenOffice docs work to some degree (personally, I'm using unoconv to
    convert OO docs to temp PDFs for the text & dimensions extraction, so
    those bits of data are missing.)
    - MS Office docs to some degree (ppt at least, doc and xls should work too,
    dimensions missing due to the above temp PDF -thing.)

    Others:
    - BitTorrent .torrent files
    - Archive contents
    - Whatever `extract' outputs and I am handling


    Requirements
    ------------

    * Ruby 1.8

    * Tons of metadata extraction programs and libs.
    This package has many dependencies since there is no single universal
    metadata header format that all files use. Blame resource forks, filename
    extensions, bags of bytes and mimetypes.

    List of gems:
    flacinfo-rb
    wmainfo-rb
    MP4info
    id3lib-ruby
    apetag

    List of Debian packages:
    dcraw
    libimlib2-ruby
    extract
    libimage-exiftool-perl
    poppler-utils
    mplayer
    html2text
    imagemagick
    unhtml
    pstotext
    antiword
    catdoc
    shared-mime-info

    * You do want to install the latest versions of dcraw and
    shared-mime-info to be able to handle camera raw images.
    http://cybercom.net/~dcoffin/dcraw/
    http://freedesktop.org/wiki/Software/shared-mime-info

    * Python + chardet library
    http://chardet.feedparser.org/


    Install
    -------

    De-compress archive and enter its top directory.
    Then type:

    ($ su)
    # ruby setup.rb

    These simple step installs this program under the default
    location of Ruby libraries. You can also install files into
    your favorite directory by supplying setup.rb some options.
    Try "ruby setup.rb --help".


    Appendix A: Additional metadata fields
    --------------------------------------

    This list contains the metadata fields added to the shared
    file metadata spec.
    http://wiki.freedesktop.org/wiki/Specifications/shared-filemetadata-spec

    field name | field type
    ----------------------------------------------------------------------
    Archive.Contents array of pathnames

    Audio.Band string
    Audio.Composer string
    Audio.Conductor string
    Audio.Copyright string (copyright message)
    Audio.Grouping string
    Audio.Image binary string (embedded image data)
    Audio.InterpretedBy string
    Audio.Lyricist string
    Audio.Publisher string
    Audio.RemixedBy string
    Audio.Subtitle string
    Audio.Tempo integer
    Audio.VariableBitrate boolean
    Audio.Writer string
    Audio.Publicationright string
    Audio.File string
    Audio.EAN/UPC string
    Audio.ISBN string
    Audio.Catalog string
    Audio.LC string
    Audio.Media string
    Audio.Index string
    Audio.Related string
    Audio.ISRC string
    Audio.Abstract string
    Audio.Language string
    Audio.Bibliography string
    Audio.Introplay string
    Audio.Dummy string
    Audio.DebutAlbum string
    Audio.RecordDate string
    Audio.RecordLocation string

    Doc.Album string
    Doc.Artist string
    Doc.Charset string
    Doc.Description string
    Doc.Genre string
    Doc.Language string
    Doc.ModifyDate date
    Doc.PageSizeName string (A4, A5, letter, ...)

    File.Software string (software used to create the file)

    Image.DateCreated date
    Image.DateTimeCreated date
    Image.DateTimeOriginal date
    Image.DimensionUnit string (px, mm, pt, ...)
    Image.EXIF string (exiftool output)
    Image.Frames integer
    Image.Modified date
    Image.OriginatingProgram string

    Location.Latitude float
    Location.Longitude float

    Video.Album string
    Video.Artist string
    Video.Bitrate integer
    Video.Codec string
    Video.Comment string
    Video.Duration float
    Video.Framerate float (frames per second)
    Video.Genre string
    Video.ReleaseDate date
    Video.Title string
    Video.TrackNo integer

    BitTorrent.Files array of {'path' => string, 'length' => integer}
    BitTorrent.Length integer (size of single-file torrents)
    BitTorrent.Announce string (announce url)
    BitTorrent.AnnounceList array of arrays of strings
    BitTorrent.Nodes array of [hostname, port] -arrays


    Appendix B: The MDH file format
    -------------------------------

    MDH files are built as follows:

    bytes | content
    ---------------
    3 | "MDH" - MDH file format identifier
    1 | "\x01" - MDH file format version number
    4 | Long, network byte order - the size of the metadata struct in bytes
    var | YAML - The MDH metadata struct
    var | The actual file contents

    All string fields in the metadata are UTF-8.


    License
    -------

    Ruby's


    --
    Ilmari Heikkinen <ilmari.heikkinen gmail com>
    http://fhtr.blogspot.com
     
    Ilmari Heikkinen, Sep 19, 2007
    #1
    1. Advertising

  2. Ilmari Heikkinen

    Konrad Meyer Guest

    --nextPart1561523.5cnpp0Beel
    Content-Type: text/plain;
    charset="utf-8"
    Content-Transfer-Encoding: quoted-printable
    Content-Disposition: inline

    Quoth Ilmari Heikkinen:
    > Getting close to the 1.0 release of the metadata library.
    > If there are no major problems with this release candidate, that is.
    >=20
    > tarball: http://dark.fhtr.org/repos/metadata/metadata-1.0-rc2.tar.gz
    > git: http://dark.fhtr.org/repos/metadata
    >=20
    > Changes
    > -------
    > * bittorrent .torrent support
    > * untested ape/musepack/wavepack support with apetag gem
    > (couldn't find any ape samples)
    > * 'name' and 'author' parsed from mplayer output
    > * more documentation
    > * better handling of images that have no exif info
    > * tested ra, wma and m4a
    > * tested more image formats
    > * fixed audio samplerate and channels for videos
    > (i.e. convert the string to int.)
    > * bin/mdh now prints the metadata as the default action.
    > Use -c to create an MDH file.
    > * MDH files have a "MDH#{version_byte}" header now,
    > the validity of which is checked by mdh.
    >=20
    > Thanks
    > ------
    >=20
    > Konrad Meyer for his patient testing and bug reports.
    > Darren Kirby for the heads-up on wmainfo's ASF-parsing capabilities
    > (along with being the author of wmainfo-rb and flacinfo-rb.)
    >=20
    >=20
    > Description
    > -----------
    >=20
    > This package `Metadata' comes with a library called `metadata' and
    > a small program called `mdh'.
    >=20
    > The library probes files for their metadata (e.g. jpeg dimensions
    > and camera make, mp3 artist, pdf text and word count) and returns the
    > metadata as a Hash. All strings in the metadata are converted to UTF-8.
    >=20
    > The `mdh'-program can print out file metadata as YAML and package the
    > metadata with the file.
    >=20
    > The metadata hash follows the shared file metadata spec naming, with so=

    me
    > additional fields, see list at the end of this file (Appendix A.)
    >=20
    > For details on the MDH file format, see the end of this file (Appendix =

    B.)
    >=20
    >=20
    > Usage
    > -----
    >=20
    > # print out metadata for myfile.jpg
    > mdh myfile.jpg
    >=20
    > # create myfile.jpg.mdh, which consists of an MDH metadata header +=20

    myfile.jpg
    > mdh -c myfile.jpg
    >=20
    > # print out the metadata header from an MDH file
    > mdh -e -p myfile.jpg.mdh
    >=20
    > # strip out the metadata header from an MDH file and save it to myfile.=

    jpg
    > mdh -e myfile.jpg.mdh
    >=20
    > # print out the list of options
    > mdh -h
    >=20
    > irb> require 'metadata'
    > irb> Metadata.extract('myfile.jpg')
    > irb> Metadata.extract_text('myfile.pdf')
    > irb> Pathname.new("myfile.jpg").metadata
    >=20
    >=20
    > List of supported formats
    > -------------------------
    >=20
    > Audio:
    > Whatever you manage to make mplayer play.
    > Plus special handlers for FLAC, m4a, ape, musepack, wavepack and wma.
    > Successfully tested with:
    > mp3, flac, ogg, wav, ra, wma, m4a
    > Should also work:
    > wv, mpc, ape
    >=20
    > Video:
    > Whatever you manage to make mplayer play.
    > Successfully tested with:
    > wmv, mov, divx, xvid, flv, ogm, mpg, mkv
    >=20
    > Images:
    > Should handle pretty much anything (apart from ORF.)
    > Successfully tested with:
    > jpeg, png, gif, nef, dng, crw, pef, psd, tga, tif, xcf, xpm, ppm, b=

    mp
    >=20
    > Documents:
    > Successfully tested with:
    > pdf, ppt, odp, sxi, ps, ps.gz, html, txt
    > Should work:
    > - OpenOffice docs work to some degree (personally, I'm using unoconv =

    to
    > convert OO docs to temp PDFs for the text & dimensions extraction, =

    so
    > those bits of data are missing.)
    > - MS Office docs to some degree (ppt at least, doc and xls should wor=

    k=20
    too,
    > dimensions missing due to the above temp PDF -thing.)
    >=20
    > Others:
    > - BitTorrent .torrent files
    > - Archive contents
    > - Whatever `extract' outputs and I am handling
    >=20
    >=20
    > Requirements
    > ------------
    >=20
    > * Ruby 1.8
    >=20
    > * Tons of metadata extraction programs and libs.
    > This package has many dependencies since there is no single universal
    > metadata header format that all files use. Blame resource forks,=20

    filename
    > extensions, bags of bytes and mimetypes.
    >=20
    > List of gems:
    > flacinfo-rb
    > wmainfo-rb
    > MP4info
    > id3lib-ruby
    > apetag
    >=20
    > List of Debian packages:
    > dcraw
    > libimlib2-ruby
    > extract
    > libimage-exiftool-perl
    > poppler-utils
    > mplayer
    > html2text
    > imagemagick
    > unhtml
    > pstotext
    > antiword
    > catdoc
    > shared-mime-info
    >=20
    > * You do want to install the latest versions of dcraw and
    > shared-mime-info to be able to handle camera raw images.
    > http://cybercom.net/~dcoffin/dcraw/
    > http://freedesktop.org/wiki/Software/shared-mime-info
    >=20
    > * Python + chardet library
    > http://chardet.feedparser.org/
    >=20
    >=20
    > Install
    > -------
    >=20
    > De-compress archive and enter its top directory.
    > Then type:
    >=20
    > ($ su)
    > # ruby setup.rb
    >=20
    > These simple step installs this program under the default
    > location of Ruby libraries. You can also install files into
    > your favorite directory by supplying setup.rb some options.
    > Try "ruby setup.rb --help".
    >=20
    >=20
    > Appendix A: Additional metadata fields
    > --------------------------------------
    >=20
    > This list contains the metadata fields added to the shared
    > file metadata spec.
    > http://wiki.freedesktop.org/wiki/Specifications/shared-filemetadata-spec
    >=20
    > field name | field type
    > ----------------------------------------------------------------------
    > Archive.Contents array of pathnames
    >=20
    > Audio.Band string
    > Audio.Composer string
    > Audio.Conductor string
    > Audio.Copyright string (copyright message)
    > Audio.Grouping string
    > Audio.Image binary string (embedded image data)
    > Audio.InterpretedBy string
    > Audio.Lyricist string
    > Audio.Publisher string
    > Audio.RemixedBy string
    > Audio.Subtitle string
    > Audio.Tempo integer
    > Audio.VariableBitrate boolean
    > Audio.Writer string
    > Audio.Publicationright string
    > Audio.File string
    > Audio.EAN/UPC string
    > Audio.ISBN string
    > Audio.Catalog string
    > Audio.LC string
    > Audio.Media string
    > Audio.Index string
    > Audio.Related string
    > Audio.ISRC string
    > Audio.Abstract string
    > Audio.Language string
    > Audio.Bibliography string
    > Audio.Introplay string
    > Audio.Dummy string
    > Audio.DebutAlbum string
    > Audio.RecordDate string
    > Audio.RecordLocation string
    >=20
    > Doc.Album string
    > Doc.Artist string
    > Doc.Charset string
    > Doc.Description string
    > Doc.Genre string
    > Doc.Language string
    > Doc.ModifyDate date
    > Doc.PageSizeName string (A4, A5, letter, ...)
    >=20
    > File.Software string (software used to create the file)
    >=20
    > Image.DateCreated date
    > Image.DateTimeCreated date
    > Image.DateTimeOriginal date
    > Image.DimensionUnit string (px, mm, pt, ...)
    > Image.EXIF string (exiftool output)
    > Image.Frames integer
    > Image.Modified date
    > Image.OriginatingProgram string
    >=20
    > Location.Latitude float
    > Location.Longitude float
    >=20
    > Video.Album string
    > Video.Artist string
    > Video.Bitrate integer
    > Video.Codec string
    > Video.Comment string
    > Video.Duration float
    > Video.Framerate float (frames per second)
    > Video.Genre string
    > Video.ReleaseDate date
    > Video.Title string
    > Video.TrackNo integer
    >=20
    > BitTorrent.Files array of {'path' =3D> string, 'length' =3D> i=

    nteger}
    > BitTorrent.Length integer (size of single-file torrents)
    > BitTorrent.Announce string (announce url)
    > BitTorrent.AnnounceList array of arrays of strings
    > BitTorrent.Nodes array of [hostname, port] -arrays
    >=20
    >=20
    > Appendix B: The MDH file format
    > -------------------------------
    >=20
    > MDH files are built as follows:
    >=20
    > bytes | content
    > ---------------
    > 3 | "MDH" - MDH file format identifier
    > 1 | "\x01" - MDH file format version number
    > 4 | Long, network byte order - the size of the metadata struct in=20

    bytes
    > var | YAML - The MDH metadata struct
    > var | The actual file contents
    >=20
    > All string fields in the metadata are UTF-8.
    >=20
    >=20
    > License
    > -------
    >=20
    > Ruby's
    >=20
    >=20
    > --
    > Ilmari Heikkinen <ilmari.heikkinen gmail com>
    > http://fhtr.blogspot.com


    I'm getting some issues with .torrents. Ex:
    $ mdh -p Fedora-7-i386.torrent undefined method `to_utf8' for
    ["F-7-i386-DVD.iso"]:Array
    undefined method `to_utf8' for ["F-7-i386-DVD.iso"]:Array
    /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:822:in `enc_utf8'
    /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:452:in
    `application_x_bittorrent'
    /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:451:in `map'
    /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:451:in
    `application_x_bittorrent'
    /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:146:in `__send__'
    /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:146:in `extract'
    /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:38:in `metadata'
    (wrapped lines indented.)

    HTH,
    =2D-=20
    Konrad Meyer <> http://konrad.sobertillnoon.com/

    --nextPart1561523.5cnpp0Beel
    Content-Type: application/pgp-signature; name=signature.asc
    Content-Description: This is a digitally signed message part.

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.7 (GNU/Linux)

    iD8DBQBG8hrcCHB0oCiR2cwRAjstAKCZa/+NfvxLTxsZQK9rQySmI+tWVACgyiQd
    ch24NY0vMHm1zTeJCyI2H60=
    =xrRi
    -----END PGP SIGNATURE-----

    --nextPart1561523.5cnpp0Beel--
     
    Konrad Meyer, Sep 20, 2007
    #2
    1. Advertising

  3. On 9/20/07, Konrad Meyer <> wrote:
    > I'm getting some issues with .torrents. Ex:
    > $ mdh -p Fedora-7-i386.torrent undefined method `to_utf8' for
    > ["F-7-i386-DVD.iso"]:Array
    > undefined method `to_utf8' for ["F-7-i386-DVD.iso"]:Array
    > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:822:in `enc_utf8'
    > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:452:in
    > `application_x_bittorrent'
    > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:451:in `map'
    > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:451:in
    > `application_x_bittorrent'
    > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:146:in `__send__'
    > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:146:in `extract'
    > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:38:in `metadata'
    > (wrapped lines indented.)
    >


    Thanks, fixed. Along with extracting more metadata from .torrents
    and handling flacs with id3 tags better.

    http://dark.fhtr.org/repos/metadata/metadata-1.0-rc3.tar.gz

    > HTH,
    > --
    > Konrad Meyer <> http://konrad.sobertillnoon.com/
    >
    >


    --
    Ilmari Heikkinen
    http://fhtr.blogspot.com
     
    Ilmari Heikkinen, Sep 20, 2007
    #3
  4. On 9/20/07, Ilmari Heikkinen <> wrote:
    > On 9/20/07, Konrad Meyer <> wrote:
    > > I'm getting some issues with .torrents. Ex:
    > > $ mdh -p Fedora-7-i386.torrent undefined method `to_utf8' for
    > > ["F-7-i386-DVD.iso"]:Array
    > > undefined method `to_utf8' for ["F-7-i386-DVD.iso"]:Array
    > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:822:in `enc_utf8'
    > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:452:in
    > > `application_x_bittorrent'
    > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:451:in `map'
    > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:451:in
    > > `application_x_bittorrent'
    > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:146:in `__send__'
    > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:146:in `extract'
    > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:38:in `metadata'
    > > (wrapped lines indented.)
    > >

    >
    > Thanks, fixed. Along with extracting more metadata from .torrents
    > and handling flacs with id3 tags better.
    >
    > http://dark.fhtr.org/repos/metadata/metadata-1.0-rc3.tar.gz


    Oh, and, if there are any other filetypes that could do with more
    metadata, feel free to request. Especially if some other program
    shows it. And still more especially if the other program is a
    cmdline app with a debian package ;-)
     
    Ilmari Heikkinen, Sep 20, 2007
    #4
  5. Ilmari Heikkinen

    Konrad Meyer Guest

    --nextPart1533880.ocKK8PLenr
    Content-Type: text/plain;
    charset="utf-8"
    Content-Transfer-Encoding: quoted-printable
    Content-Disposition: inline

    Quoth Ilmari Heikkinen:
    > On 9/20/07, Ilmari Heikkinen <> wrote:
    > > On 9/20/07, Konrad Meyer <> wrote:
    > > > I'm getting some issues with .torrents. Ex:
    > > > $ mdh -p Fedora-7-i386.torrent undefined method `to_utf8' for
    > > > ["F-7-i386-DVD.iso"]:Array
    > > > undefined method `to_utf8' for ["F-7-i386-DVD.iso"]:Array
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:822:in `enc_utf8'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:452:in
    > > > `application_x_bittorrent'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:451:in `map'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:451:in
    > > > `application_x_bittorrent'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:146:in `__send__'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:146:in `extract'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:38:in `metadata'
    > > > (wrapped lines indented.)
    > > >

    > >
    > > Thanks, fixed. Along with extracting more metadata from .torrents
    > > and handling flacs with id3 tags better.
    > >
    > > http://dark.fhtr.org/repos/metadata/metadata-1.0-rc3.tar.gz

    >=20
    > Oh, and, if there are any other filetypes that could do with more
    > metadata, feel free to request. Especially if some other program
    > shows it. And still more especially if the other program is a
    > cmdline app with a debian package ;-)


    I've actually been doing all this testing under Fedora 7, all the packages
    needed are here IIRC but they may have slightly different names (especially
    debian -dev -> fedora -devel).

    =2D-=20
    Konrad Meyer <> http://konrad.sobertillnoon.com/

    --nextPart1533880.ocKK8PLenr
    Content-Type: application/pgp-signature; name=signature.asc
    Content-Description: This is a digitally signed message part.

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.7 (GNU/Linux)

    iD8DBQBG8nzBCHB0oCiR2cwRAnRFAKDClNwEDpi45HoBfI6pDiIbhFmSaQCg13aQ
    mVYYiZWA0rFZQ8tCwxTeB8E=
    =ZBWV
    -----END PGP SIGNATURE-----

    --nextPart1533880.ocKK8PLenr--
     
    Konrad Meyer, Sep 20, 2007
    #5
  6. Ilmari Heikkinen

    Konrad Meyer Guest

    --nextPart2488336.psT4BNkGbx
    Content-Type: text/plain;
    charset="utf-8"
    Content-Transfer-Encoding: quoted-printable
    Content-Disposition: inline

    Quoth Ilmari Heikkinen:
    > On 9/20/07, Ilmari Heikkinen <> wrote:
    > > On 9/20/07, Konrad Meyer <> wrote:
    > > > I'm getting some issues with .torrents. Ex:
    > > > $ mdh -p Fedora-7-i386.torrent undefined method `to_utf8' for
    > > > ["F-7-i386-DVD.iso"]:Array
    > > > undefined method `to_utf8' for ["F-7-i386-DVD.iso"]:Array
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:822:in `enc_utf8'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:452:in
    > > > `application_x_bittorrent'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:451:in `map'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:451:in
    > > > `application_x_bittorrent'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:146:in `__send__'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:146:in `extract'
    > > > /usr/lib/ruby/site_ruby/1.8/metadata/extract.rb:38:in `metadata'
    > > > (wrapped lines indented.)
    > > >

    > >
    > > Thanks, fixed. Along with extracting more metadata from .torrents
    > > and handling flacs with id3 tags better.
    > >
    > > http://dark.fhtr.org/repos/metadata/metadata-1.0-rc3.tar.gz

    >=20
    > Oh, and, if there are any other filetypes that could do with more
    > metadata, feel free to request. Especially if some other program
    > shows it. And still more especially if the other program is a
    > cmdline app with a debian package ;-)


    Very cool:

    $ mdh a-downloads/Fedora-7-i386.torrent ---=20
    Doc.Created: 2007-05-29T10:58:50-07:00
    BitTorrent.Name: Fedora-7-i386
    BitTorrent.Files:=20
    - length: 2900602880
    path: F-7-i386-DVD.iso
    - length: 101816320
    path: F-7-i386-rescuecd.iso
    - length: 359
    path: SHA1SUM
    BitTorrent.PieceLength: 262144
    Doc.Title: Fedora-7-i386
    BitTorrent.PieceCount: 11454
    File.Size: 229377
    File.Modified: 2007-05-31T06:28:34-07:00
    File.Format: application/x-bittorrent
    BitTorrent.Announce: http://torrent.linux.duke.edu:6969/announce

    Thanks a lot!
    =2D-=20
    Konrad Meyer <> http://konrad.sobertillnoon.com/

    --nextPart2488336.psT4BNkGbx
    Content-Type: application/pgp-signature; name=signature.asc
    Content-Description: This is a digitally signed message part.

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.7 (GNU/Linux)

    iD8DBQBG8n7vCHB0oCiR2cwRAn4OAJkB9vC+oSfwNNyhaFPGAGdwai4wUgCffVma
    +i0Ne0TaSgq2ahZNrRRXBz4=
    =jsN/
    -----END PGP SIGNATURE-----

    --nextPart2488336.psT4BNkGbx--
     
    Konrad Meyer, Sep 20, 2007
    #6
  7. RnJvbTogSWxtYXJpIEhlaWtraW5lbiBbbWFpbHRvOmlsbWFyaS5oZWlra2luZW5AZ21haWwuY29t
    XSANCiMgdGFyYmFsbDogaHR0cDovL2RhcmsuZmh0ci5vcmcvcmVwb3MvbWV0YWRhdGEvbWV0YWRh
    dGEtMS4wLXJjMi50YXIuZ3oNCg0KaXMgaXQgcG9zc2libGUgZm9yIGEgZ2VtPw0Kd2lsbCB0aGlz
    IHJ1biBvbiB3aW5kb3dzPw0KDQpraW5kIHJlZ2FyZHMgLWJvdHANCg==
     
    Peña, Botp, Sep 21, 2007
    #7
  8. Ilmari Heikkinen

    Konrad Meyer Guest

    --nextPart1492403.i4BGHRKt4Q
    Content-Type: text/plain;
    charset="utf-8"
    Content-Transfer-Encoding: quoted-printable
    Content-Disposition: inline

    Quoth Pe=C3=B1a, Botp:
    > From: Ilmari Heikkinen [mailto:]=20
    > # tarball: http://dark.fhtr.org/repos/metadata/metadata-1.0-rc2.tar.gz
    >=20
    > is it possible for a gem?
    > will this run on windows?
    >=20
    > kind regards -botp


    In short, no (to both).
    In earlier messages I asked for a gem too, and it turns out this is hard to=
    do
    because of a python script that's included. And as far as windows, I think a
    lot of this relies on shelling out, which windows... doesn't have. So you
    might look for another solution (or go to *nix).

    HTH,
    =2D-=20
    Konrad Meyer <> http://konrad.sobertillnoon.com/

    --nextPart1492403.i4BGHRKt4Q
    Content-Type: application/pgp-signature; name=signature.asc
    Content-Description: This is a digitally signed message part.

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.7 (GNU/Linux)

    iD8DBQBG81MZCHB0oCiR2cwRAtG1AJ48CNM6Ct1n9r8bjVK/leEDYaCyWwCggxp4
    NBnh0ykWmoot0X4JE2AH298=
    =s0Tp
    -----END PGP SIGNATURE-----

    --nextPart1492403.i4BGHRKt4Q--
     
    Konrad Meyer, Sep 21, 2007
    #8
  9. T24gOS8yMS8wNywgUGXDsWEsIEJvdHAgPGJvdHBAZGVsbW9udGUtcGhpbC5jb20+IHdyb3RlOgo+
    IEZyb206IElsbWFyaSBIZWlra2luZW4gW21haWx0bzppbG1hcmkuaGVpa2tpbmVuQGdtYWlsLmNv
    bV0KPiAjIHRhcmJhbGw6IGh0dHA6Ly9kYXJrLmZodHIub3JnL3JlcG9zL21ldGFkYXRhL21ldGFk
    YXRhLTEuMC1yYzIudGFyLmd6Cj4KPiBpcyBpdCBwb3NzaWJsZSBmb3IgYSBnZW0/CgpJZiB0aGVy
    ZSdzIGEgd2F5IHRvIGluc3RhbGwgYmluL2NoYXJkZXQgYXMtaXMsIG5vdyBnZW0gYWRkcwoiIyEv
    dXNyL2Jpbi9ydWJ5Ii1ib2lsZXJwbGF0ZSAod2hpY2ggZG9lc24ndCBxdWl0ZSB3b3JrIGZvciBh
    CnB5dGhvbiBzY3JpcHQuKSAuLi5UaG91Z2gsIEkgY291bGQgYWx3YXlzIHR1cm4gaXQgaW50byBh
    IHJ1Ynkgc2NyaXB0CnRoYXQgZXhlY3V0ZXMgYSBiaXQgb2YgcHl0aG9uLiBTbywgeWVhaCwgaXQg
    aXMgcG9zc2libGUuIEknbGwgYnVpbGQgb25lLgoKPiB3aWxsIHRoaXMgcnVuIG9uIHdpbmRvd3M/
    CgpJbiB0aGVvcnkuIEF0IGxlYXN0IHRoZSBSdWJ5IHBhcnRzIChpLmUuIHdtYSwgZmxhYywgbXAz
    LCB0b3JyZW50cy4pCk5vIGlkZWEgaWYgeW91IGNhbiBnZXQgdGhlIGV4dGVybmFsIHByb2dyYW1z
    IGFuZCBsaWJzIGluc3RhbGxlZCBhbmQKcnVubmluZywgc291bmRzIHVubGlrZWx5LiBCdXQgaGV5
    LCBmZWVsIGZyZWUgdG8gcHJvdmUgbWUgd3JvbmcgOi0pCgo+IGtpbmQgcmVnYXJkcyAtYm90cAo+
    CgotLQpJbG1hcmkgSGVpa2tpbmVuCmh0dHA6Ly9maHRyLmJsb2dzcG90LmNvbQo=
     
    Ilmari Heikkinen, Sep 21, 2007
    #9
  10. Ilmari Heikkinen

    Konrad Meyer Guest

    --nextPart2113097.Svia1kiPmc
    Content-Type: text/plain;
    charset="utf-8"
    Content-Transfer-Encoding: quoted-printable
    Content-Disposition: inline

    Quoth Ilmari Heikkinen:
    > On 9/21/07, Ilmari Heikkinen <> wrote:
    > > On 9/21/07, Pe=C3=B1a, Botp <> wrote:
    > > > From: Ilmari Heikkinen [mailto:]
    > > > # tarball: http://dark.fhtr.org/repos/metadata/metadata-1.0-rc2.tar.gz
    > > >
    > > > is it possible for a gem?

    > >
    > > If there's a way to install bin/chardet as-is, now gem adds
    > > "#!/usr/bin/ruby"-boilerplate (which doesn't quite work for a
    > > python script.) ...Though, I could always turn it into a ruby script
    > > that executes a bit of python. So, yeah, it is possible. I'll build one.
    > >
    > > > will this run on windows?

    > >
    > > In theory. At least the Ruby parts (i.e. wma, flac, mp3, torrents.)
    > > No idea if you can get the external programs and libs installed and
    > > running, sounds unlikely. But hey, feel free to prove me wrong :)
    > >

    >=20
    > Or, one could make a VMWare image or somesuch and use it as a
    > metadata appliance. Anyone know how that might work best?


    Uglily, at best.

    =2D-=20
    Konrad Meyer <> http://konrad.sobertillnoon.com/

    --nextPart2113097.Svia1kiPmc
    Content-Type: application/pgp-signature; name=signature.asc
    Content-Description: This is a digitally signed message part.

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.7 (GNU/Linux)

    iD8DBQBG82EiCHB0oCiR2cwRAsrbAJ0aW03TI2aP4Di6yA4cP93ZP1LntwCfRqOm
    nv+UF/rJYYvC8mXJtfsL430=
    =02Je
    -----END PGP SIGNATURE-----

    --nextPart2113097.Svia1kiPmc--
     
    Konrad Meyer, Sep 21, 2007
    #10
  11. Ilmari Heikkinen

    Bill Kelly Guest

    From: "Ilmari Heikkinen" <>
    > On 9/21/07, Ilmari Heikkinen <> wrote:
    >> On 9/21/07, Peña, Botp <> wrote:
    >> > From: Ilmari Heikkinen [mailto:]
    >> > # tarball: http://dark.fhtr.org/repos/metadata/metadata-1.0-rc2.tar.gz
    >> >
    >> > is it possible for a gem?

    >>
    >> If there's a way to install bin/chardet as-is, now gem adds
    >> "#!/usr/bin/ruby"-boilerplate (which doesn't quite work for a
    >> python script.) ...Though, I could always turn it into a ruby script
    >> that executes a bit of python. So, yeah, it is possible. I'll build one.
    >>
    >> > will this run on windows?

    >>
    >> In theory. At least the Ruby parts (i.e. wma, flac, mp3, torrents.)
    >> No idea if you can get the external programs and libs installed and
    >> running, sounds unlikely. But hey, feel free to prove me wrong :)
    >>

    >
    > Or, one could make a VMWare image or somesuch and use it as a
    > metadata appliance. Anyone know how that might work best?


    Or perhaps build the external programs with cygwin?

    (I think most cygwin programs only depend on cygwin1.dll which is
    about 1.2 meg on my system.)


    Regards,

    Bill
     
    Bill Kelly, Sep 21, 2007
    #11
  12. RnJvbTogQmlsbCBLZWxseSBbbWFpbHRvOmJpbGxrQGN0cy5jb21dIA0KIyAoSSB0aGluayBtb3N0
    IGN5Z3dpbiBwcm9ncmFtcyBvbmx5IGRlcGVuZCBvbiBjeWd3aW4xLmRsbCANCiMgd2hpY2ggaXMg
    YWJvdXQgMS4yIG1lZyBvbiBteSBzeXN0ZW0uKQ0KDQppbnRlcmVzdGluZywgaSBkaWRuJ3Qga25v
    dyB0aGF0LiBTbyB3aGF0IGlzIHRoZSBtaW5pbXVtIHNldCBvZiBmaWxlcyBuZWVkZWQgdG8gcnVu
    IGN5Z3dpbj8gaSdtIGFza2luZyBzaW5jZSBpIGhhdmVuJ3QgdXNlZCBvbmUgZm9yIGEgbG9uZyBs
    b25nIHRpbWUgbm93Li4NCg0Ka2luZCByZWdhcmRzIC1ib3RwDQo=
     
    Peña, Botp, Sep 21, 2007
    #12
  13. Ilmari Heikkinen

    Bill Kelly Guest

    From: "Peña, Botp" <>
    > From: Bill Kelly [mailto:]
    > # (I think most cygwin programs only depend on cygwin1.dll
    > # which is about 1.2 meg on my system.)
    >
    > interesting, i didn't know that. So what is the minimum set of files needed to run
    > cygwin? i'm asking since i haven't used one for a long long time now..


    It depends. (lol, pun not intended, but my next sentence was going to involve
    depend.exe)

    Er... using depends.exe [1] one can view a tree of dependencies for a .exe or
    dll file (among other object formats.)

    A number of cygwin-compiled binaries only depend on cygwin1.dll. (Not
    counting windows modules like kernel32.dll.)

    To verify this, I was successfully able to copy just cygwin1.dll and
    cygwin/bin/ftp.exe into a subdirectory elsewhere, and change the shell
    execution path to simply ".", and the program worked.

    Other binaries, like cygwin/bin/ls.exe, required a couple extra .dll's
    (in this case for handling character set encodings) like cygintl-3.dll and
    cygiconv-2.dll

    A more complex program like cygwin/bin/wget.exe still had minimal
    and reasonable dependencies: character set handling and crypto
    libraries:

    cygwin1.dll, cygintl-3.dll, cygiconv-2.dll, cygcrypto-0.9.8.dll, cygssl-0.9.8.dll

    So it appears the cygwin folks have done a nice job at avoiding
    dependency bloat.


    [1] http://www.dependencywalker.com/


    Regards,

    Bill
     
    Bill Kelly, Sep 21, 2007
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Brett Selleck

    Schema Metadata not a Metadata Schema

    Brett Selleck, Sep 4, 2003, in forum: XML
    Replies:
    1
    Views:
    415
    Andy Dingley
    Sep 4, 2003
  2. Huw Collingbourne
    Replies:
    0
    Views:
    112
    Huw Collingbourne
    Feb 1, 2007
  3. KUBO Takehiro

    [ANN] ruby-oci8 1.0.0-rc2

    KUBO Takehiro, Apr 22, 2007, in forum: Ruby
    Replies:
    1
    Views:
    105
    KUBO Takehiro
    Apr 23, 2007
  4. Gregory Brown

    [ANN] Ruby Reports 1.0 RC2 (0.11.0)

    Gregory Brown, May 2, 2007, in forum: Ruby
    Replies:
    0
    Views:
    88
    Gregory Brown
    May 2, 2007
  5. Thomas E Enebo

    [ANN] JRuby 1.5.0.RC2 Released

    Thomas E Enebo, Apr 28, 2010, in forum: Ruby
    Replies:
    2
    Views:
    92
    Roger Pack
    Apr 30, 2010
Loading...

Share This Page