Server response validation?

Discussion in 'Perl Misc' started by Tuxedo, Nov 10, 2012.

  1. Tuxedo

    Tuxedo Guest

    This may be a vague question but which perl network tools or procedures are
    good to determine if a hostname or IP exist and that is suitable for a cgi
    request that allows user-input of any URL string?

    An HTTP status code may return any response, such as 200, 404 or something
    else. Whatever the response may be, as long as there is an actual response,
    DNS record or server existing, a subroutine should return true. If however
    the hostname or IP is non-existing, or if the user input has a wrong
    syntax, the perl code should return false.

    Presumably countless modules and procedures exist that do this and more.
    Any ideas for good solutions would be much appreciated.

    Many thanks,
    Tuxedo
    Tuxedo, Nov 10, 2012
    #1
    1. Advertising

  2. Tuxedo

    Tuxedo Guest

    Ben Morrow wrote:
    >
    > Quoth Tuxedo <>:
    > > This may be a vague question but which perl network tools or procedures
    > > are good to determine if a hostname or IP exist and that is suitable for
    > > a cgi request that allows user-input of any URL string?

    >
    > The core gethostbyname function will tell you if a given name has an
    > address in the DNS. The Net::DNS module will let you make more specific
    > queries if you need to.


    Thanks for the advise, most helpful!

    I'm not sure what I need yet but before making DNS requests I think would
    be a good idea to validate the URL input.

    I read that gethostbyname is not IPv6 compatible which may run into trouble:
    http://stackoverflow.com/questions/11325457/perl-gethostbyname-when-given-ip

    The Net::DNS tools look interesting and in combination with Net::LibIDN it
    appears to support IDNs.

    I'm not quite sure how to validate a URI and I guess there is no simple way
    to do so by existing core modules.

    To cover a growing number of IDNs, IDNsNet-IDN-Encode-2.003 seem to be a
    good tool in converting IDNs to their ASCII representations:
    http://search.cpan.org/~cfaerber/Net-IDN-Encode-2.003/lib/Net/IDN/Encode.pm

    I will test Data::Validate::URI first, which require various non-core
    modules, namely Data::Validate::IP, Net::Domain::TLD as well as
    Net::Netmask.

    > Determining if an IP exists is harder, since you can't in general assume
    > any given IP will respond to any particular protocol. Net::ping would be
    > a start, but a lot of sites block external pings (even though they
    > shouldn't).
    >
    > > An HTTP status code may return any response, such as 200, 404 or
    > > something else. Whatever the response may be, as long as there is an
    > > actual response, DNS record or server existing, a subroutine should
    > > return true. If however the hostname or IP is non-existing, or if the
    > > user input has a wrong syntax, the perl code should return false.

    >
    > Oh, you didn't say 'HTTP'. If you want to check an HTTP server exists,
    > you can either just open a connection to port 80 with IO::Socket::INET
    > (and close it again without sending a response), or send a HEAD request
    > with LWP::Simple.
    >
    > You should be aware that some sites may consider this sort of behaviour
    > to be port-scanning, and may take action to prevent you from doing it.
    > In particular, you should be very careful not to probe any particular
    > server too often, as that would almost certainly be considered abuse.


    Thanks for the words of warning.

    Tuxedo

    >
    > Ben
    >
    Tuxedo, Nov 10, 2012
    #2
    1. Advertising

  3. Tuxedo

    Tuxedo Guest

    Ben Morrow wrote:

    [...]

    > Socket6.pm will give you the getaddrinfo function for making v4/v6
    > hostname lookups.


    [...]

    > Well, URI.pm will do basic syntactic validation, which is a start, or
    > you could use Regexp::Common, which has patterns for validating URIs.


    [...]

    > Hah! You seem to be solving this on your own... good. I didn't know
    > about that. Note that it doesn't appear to do any network tests, so a
    > URI like http://example.com/foo/bar will pass.


    Yes, I read that in the docs. My first step will be to validate a URI
    format and not that it actually exists through a network request. The URI
    may be standard domain, a new style IDN representation or even an IP. The
    network request comes after. So thanks for Socket6.pm, URI.pm and
    Regexp::Common. I will test all!

    Tuxedo
    Tuxedo, Nov 10, 2012
    #3
  4. On 2012-11-10 19:48, Ben Morrow <> wrote:
    > Note that it doesn't appear to do any network tests, so a URI like
    > http://example.com/foo/bar will pass.


    % wget http://example.com/foo/bar
    --2012-11-11 13:43:26-- http://example.com/foo/bar
    Resolving example.com (example.com)... 2001:500:88:200::10, 192.0.43.10
    Connecting to example.com (example.com)|2001:500:88:200::10|:80... connected.
    HTTP request sent, awaiting response... 302 Found
    Location: http://www.iana.org/domains/example/ [following]
    --2012-11-11 13:43:27-- http://www.iana.org/domains/example/
    Resolving www.iana.org (www.iana.org)... 2620:0:2d0:200::8, 192.0.32.8
    Connecting to www.iana.org (www.iana.org)|2620:0:2d0:200::8|:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: unspecified [text/html]
    Saving to: `bar'

    [ <=> ] 2,966 --.-K/s in 0.001s

    2012-11-11 13:43:28 (3.39 MB/s) - `bar' saved [2966]

    SCNR,
    hp


    --
    _ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung:
    |_|_) | Sysadmin WSR | Man feilt solange an seinen Text um, bis
    | | | | die Satzbestandteile des Satzes nicht mehr
    __/ | http://www.hjp.at/ | zusammenpaƟt. -- Ralph Babel
    Peter J. Holzer, Nov 11, 2012
    #4
  5. Tuxedo

    Tuxedo Guest

    Shmuel (Seymour J.) Metz wrote:

    > In <k7k6jr$148$>, on 11/10/2012
    > at 01:20 AM, Tuxedo <> said:
    >
    > >This may be a vague question but which perl network tools or
    > >procedures are good to determine if a hostname or IP exist

    >
    > What does it mean for an IP to exist? Does a hostname exist if there
    > is an AAAA but no A? How do you want to treat CNAME?


    I'm not quite sure.

    > You really need to nail down the specifics of the problem before you
    > look for a solution.


    You are right, my questions were posted somewhat prematurely.

    > Have you looked at Regexp-Common or URI
    > <http://search.cpan.org/~gaas/URI-1.60/>?


    Thanks, I'm looking into these. In the meantime, I found that
    Data::Validate::URI appears to be working well in validating http and/or
    https URI's in case they are the only things one need to do.

    Tuxedo
    Tuxedo, Nov 11, 2012
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Charles A. Lackman
    Replies:
    2
    Views:
    7,972
    Charles A. Lackman
    Jan 11, 2006
  2. Colin Mackay
    Replies:
    0
    Views:
    2,678
    Colin Mackay
    Jun 25, 2003
  3. Matt
    Replies:
    14
    Views:
    4,071
    Chad Z. Hower aka Kudzu
    Jan 30, 2004
  4. =?Utf-8?B?QmFyYmFyYSBBbGRlcnRvbg==?=

    Using Summary Validation control with server Custom validation

    =?Utf-8?B?QmFyYmFyYSBBbGRlcnRvbg==?=, Oct 15, 2004, in forum: ASP .Net
    Replies:
    2
    Views:
    871
    =?Utf-8?B?QmFyYmFyYSBBbGRlcnRvbg==?=
    Oct 15, 2004
  5. =?Utf-8?B?dmlkeWE=?=
    Replies:
    1
    Views:
    740
    Kevin Spencer
    Jun 2, 2005
Loading...

Share This Page