Bind NICs to Mechanize

[John Yuan]

Hi,

I may shooting a wrong question, but I saw the same question here that
left unanswered and gone 60 days more, so I want to raise it once again.

I have 3 NIFs :

1.　PPP1 (111.111.111.111)：PPTP to VPN
　2.　eth2 (192.168.0.2)　　：LAN
　3.　ppp3 (333.333.333.333)：PPPoE to ASDL

and want to bind a specific interface to Mechanize agent somehow like :

agent.bind(Socket.sockaddr_in(80, "111.111.111.111"))
　agent.bind(Socket.sockaddr_in(80, "192.168.0.2"))
　agent.bind(Socket.sockaddr_in(80, "333.333.333.333"))

like the way Socket does. But I haven't have any luck yet.

Did anyone try yet or shouldn't expect Mechanize package working in this
way?

any suggestion would be appreciated !!

Thanks

John

 

[Walton Hoops]

Hi,

I may shooting a wrong question, but I saw the same question here that
left unanswered and gone 60 days more, so I want to raise it once again.

I have 3 NIFs :

1.　PPP1 (111.111.111.111)：PPTP to VPN
　2.　eth2 (192.168.0.2)　　：LAN
　3.　ppp3 (333.333.333.333)：PPPoE to ASDL

and want to bind a specific interface to Mechanize agent somehow like :

agent.bind(Socket.sockaddr_in(80, "111.111.111.111"))
　agent.bind(Socket.sockaddr_in(80, "192.168.0.2"))
　agent.bind(Socket.sockaddr_in(80, "333.333.333.333"))

like the way Socket does. But I haven't have any luck yet.

Did anyone try yet or shouldn't expect Mechanize package working in this
way?

any suggestion would be appreciated !!

Thanks

John

It doesn't work that way. Mechanize is a client, not a server. The NIC
used is decided by the destination IP and the routes in your Operating
System. Only server sockets can be bound to one or more IPs.

 

[John Yuan]

It doesn't work that way. Mechanize is a client, not a server. The NIC
used is decided by the destination IP and the routes in your Operating
System. Only server sockets can be bound to one or more IPs.

Thank you Walton,



I am afraid that your comment would be the most the answer after
spending quite sometime searching for solution. You helped me to put the
period to my approach.

It may not be Ruby specific problem, but let me explain the situation.
My client has 3 accounts on B2B site and all 3 accounts need to accessed
with specific IP address (3 different offices, 3 different IPs). I was
asked to write script to show consolidated inventory list over those 3
accounts.

So I sat up a Debian with 3 NICs, VPN/LAN/PPP to 3 different offices
network and “proxying†3 different IPs to gather 3 account inventory
level and consolidate them “simultaneouslyâ€. Right now to change
interface, I manually change routing to NIFs when switching account, but
the process is slow and it’s accessing account one by one, even though
the connections are simultaneously available, causing slight difference
of the inventory level by the time 3rd account was checked out.

My goal is to launch 3 threads of Mechanize agents to access the same
URI (inventory.b2b.com/login) through 3 different NIFs (ppp1/eth2/ppp3)

 

[John W Higgins]

[Note: parts of this message were removed to make it a legal post.]

Good Evening,

My goal is to launch 3 threads of Mechanize agents to access the same
URI (inventory.b2b.com/login) through 3 different NIFs (ppp1/eth2/ppp3)
.

Your best bet might be to set up 3 proxies and have each of the Mechanize
clients hit a different proxy (using set_proxy for each instance) which
would push each request to the proxy's assigned server.

Something nice and simple like this should work (
http://github.com/igrigorik/em-proxy/blob/master/examples/line_interceptor.rb)
- you would need 3 changes to this file

1) Obviously line 3 needs to have a different port for each of the proxies
2) Line 4 would need to point to a different ip corresponding to each of the
three instances of inventory.b2b.com and the port would be 80 obviously
3) You don't want any substitution here so line 12 would simply be resp

In theory that should push the requests to the appropriate server for you
without a huge amount of work. I think you'll find em-proxy extremely
flexible in terms of getting this task done for you. As with anything that
messes with network traffic - there is some overhead - but you seem much
more concerned with concurrency than milliseconds of delay.

John

 

[John Yuan]

Thank you, Mr. John W Higgins.



I am not familiar with Ruby's server/proxy stuff, so I simply follow
the example (presented in the page you recommended below) before
applying my actual case.

Something nice and simple like this should work (

http://github.com/igrigorik/em-proxy/blob/master/examples/line_interceptor.rb)

I am not fully understand em-proxy but when executing the example ;



# ruby examples/appserver.rb 81

# ruby examples/line_interceptor.rb



And I encountered the error when running line_interceptor ;



# ruby line_interceptor.rb

/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:572:in
`start_tcp_server': no acceptor (RuntimeError)

from
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:572:in
`start_server'

from
/usr/lib/ruby/gems/1.8/gems/em-proxy-0.1.2/lib/../lib/em-proxy/proxy.rb:10:in
`start'

from
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
`call'

from
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
`run_machine'

from
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
`run'

from
/usr/lib/ruby/gems/1.8/gems/em-proxy-0.1.2/lib/../lib/em-proxy/proxy.rb:5:in
`start'

from line_interceptor.rb:6





Only modification I made was :



original> require 'lib/emproxy'



modified>require 'rubygems'

modified>require 'em-proxy'



I think the problem is in either "appserver.rb" or eventmachine's port
specification and googled to find :



http://whatdoesthiserrormean.com/errors/93



which currently under maintenance. I am sure it's simple installation
matter, so I would wait for while until the site back and find reported
errors, but if you could suggest my em-proxy setting-up problem, I will
appreciate very much.



John

 

[John W Higgins]

[Note: parts of this message were removed to make it a legal post.]

Evening,

If you wish to run the example as is you need to run all three lines

ruby examples/appserver.rb 81 - this runs a webserver which is referenced in
line 4 of the line_interceptor example as the final destination. Think of it
as a tiny tiny version of the webserver you are trying to connect to.

ruby examples/line_interceptor.rb - this fires up the proxy - the error you
are getting is that it is trying to connect to the server started just above
which you are I'm guessing not running

curl localhost - this uses curl to make a request that hits the proxy that
then sends the request off to the appserver.rb to get the response -
modifies the result and returns it to you.

You will need rack installed to have appserver.rb work.

Hope this helps move you along......

John

 

[Alexey Bovanenko]

Hi. You cannot bind the client socket. You must bind only server
socket which recieves client connections. In your case you can use
proxy solution for specific sites or you can write paths for selected
sites in gateway table on your computer.

 

[John Yuan]

Thank you, Thank you, Mr. John W Higgins for very quick response.

About Curl, I could not proceed there yet, but as far as curl and rack
gem installation, it was smooth and I did not see any problem.

ruby examples/line_interceptor.rb - this fires up the proxy - the error you
are getting is that it is trying to connect to the server started just above
which you are I'm guessing not running

I tested with FireFox ;

request: http://localhost:81/
respose: hello world:

So I assumed the server was working, but also I saw other comments
explaining the cause of error:

eventmachine-0.12.10/lib/eventmachine.rb:572:in `start_tcp_server':
no acceptor (RuntimeError)


might be that the port (81) could be used. It does not make sense to
me, but somehow I might mess my port use or some connection trouble with
em-proxy. Can you think of any other possibility ?

 

[John Yuan]

Thank you Mr.Alexey Bovanenko

I think you are correct as some people suggested me the same.

you can use proxy solution for specific sites or
you can write paths for selected sites in gateway table on your computer.

I guess you are suggesting to deal with iptables (and/or proxy server)
and route commands to fire the particular packets to targeted site.

At this point, I prefer 100% ruby solution, but until I can make
em-proxy working, I will try debian's command line and independent proxy
server and report back here.

John

 

[John W Higgins]

[Note: parts of this message were removed to make it a legal post.]

Morning,

Thank you, Thank you, Mr. John W Higgins for very quick response.

About Curl, I could not proceed there yet, but as far as curl and rack
gem installation, it was smooth and I did not see any problem.


I tested with FireFox ;

request: http://localhost:81/
respose: hello world:

Are you running a web server on this system? Does firefox return anything
for http://localhost:80 ? If so then that is the problem - you will need to
change the port in the example from 80 to something else. I might have
misread your issue last night. So if you changed the proxy's port from 80 to
lets say 82 then look at http://localhost:82 you should be getting the
response from the proxy.

John

 

[Martin DeMello]

Are you running a web server on this system? Does firefox return anything
for http://localhost:80 ? If so then that is the problem - you will need to
change the port in the example from 80 to something else. I might have
misread your issue last night. So if you changed the proxy's port from 80 to
lets say 82 then look at http://localhost:82 you should be getting the
response from the proxy.

Also note that if you do not run the proxy as root, it will not be
able to listen on any port lower than 1024

martin

 

[John Yuan]

Dear Mr.Martin DeMello.

I was happened to be root so I did not have that problem, but I was not
aware of the point you posted. I would be easily mistaken. I will pay
more attention for handling&choosing the port.


Thank you for your remark.

John

 

[John Yuan]

Dear John W Higgins,

Are you running a web server on this system?

Yes, you are right. I was running Apache2 and after stopping Apache2,
em-proxy example works fine.

Now I do not know how to route the packet from this proxy to specified
NIF ( say, ppp1 ). I expected to be able to set em-proxy something like
Squid proxy server;

Squid setting
http_port 0.0.0.0:81
acl localhost81 src 127.0.0.1/255.255.255.255:81
tcp_outgoing_address ppp1_ip localhost81

I’m not sure if the above is correct for Squid or need additional route
setting, but what I need is that em-proxy specify outgoing ip to route
the packet to specified NIF.

Would you tell me if it’s the correct direction and if so, how can I
set these settings on em-proxy ?

Thank you very much.

John