Receiver in Outputstream.read() stops after 2735 bytes

[Bart Friederichs]

This is all so wrong, I don't know where to start. Have you considered
posting that SSCCE, so you can get some advice how to read and write
streams correctly, without doing silly things like breaking your data
into chunks less than 1MB for no obvious reason?

I have considered it. And I might send it later this week, or month.

And I a completely agree with you that this is not a correct way to fix
the problem. But, the program is in alfa stage, it is a hobby and there
are other parts of the program that should be finished as well. If that
means to have some completely wrong code in the program, but it does
what it needs to do, so be it.

Bart

 

[Bart Friederichs]

That's called 'scp'

Too complex for a normal user. Not cross platform (or is there a decent
Windows scp implementation?) and doesn't work behind a firewall without
touching that firewall.

or 'bittorrent'.

Too complex for a one-time sending of a file.

Running that between two
NATted/firewalled boxes is called 'NAT traversal' or 'hole punching'.

Too complex for normal users.

There are techniques and tools for doing these things - google will help
you find them.

No, they don't exist.

Or you could just use GBridge or similar:

http://www.gbridge.com/

Not cross platform.

Have you read my blog post?

Bart

 

[Martin Gregorie]

Too complex for a normal user.

Easy enough to wrap in a script (shell, Python, ....)

Not cross platform (or is there a decent
Windows scp implementation?)

PuTTY is a Windows implementation of both telnet and ssh.

openSSH offers a Java implementation that you can use to build your own
client and/or server.

and doesn't work behind a firewall without
touching that firewall.

That depends entirely on what direction you want to make the connection.
I connect out through an unmodified firewall all the time.

In any case ssh port forwarding to an sshd server is reasonably common
and should be safe enough if it is configured correctly. The same port is
used for both ssh sessions and scp/sftp file transfers
Same comments I made about firewalls apply to NAT routers.

Too complex for normal users.

1) I'd expect that only sysadmins are authorised to kick holes
in firewalls. 'Normal users' should be actively prevented
from doing so.

2) if they can't type "scp filename user@host:destination/path"
or use an equivalent GUI wrapper they shouldn't be let near
a computer.

No, they don't exist.

Wrongo. I've just listed some.

 

[Lew]

PuTTY is a Windows implementation of both telnet and ssh.

openSSH offers a Java implementation that you can use to build your own
client and/or server.

....

The Cygwin project includes an openSSH implementation that just plain works
from a command shell. I've used it for years, often leveraging its
port-forwarding capabilities.

 

[Nigel Wade]

Too complex for a normal user. Not cross platform (or is there a decent
Windows scp implementation?)

WinSCP, if you want all the widgety-clickety bits and pieces.

and doesn't work behind a firewall without
touching that firewall.

Nothing will, or at least nothing should. After all, that's rather the
point of a firewall in the first place.

If the person using the software isn't sufficiently au fait with the
security requirements, and implications, of opening the requisite holes
in their firewall then you have to question whether they should really be
doing that. Using a piece of software which bypasses the firewall,
without understanding what that software does or what the implications of
its actions are, is exceedingly risky. One prime example of this is
Windows - lots of people with no clue of how the Internet works, or how
to secure themselves from attack, relying on a piece of software
(Windows) to handle it all for them. Look how well that's worked out.

 

[Tom Anderson]

Too complex for a normal user. Not cross platform (or is there a decent
Windows scp implementation?) and doesn't work behind a firewall without
touching that firewall.


Too complex for a one-time sending of a file.


Too complex for normal users.


No, they don't exist.


Not cross platform.

Have you read my blog post?

Yes. Good luck with your project, Bart. You'll need it, because i'm afraid
you're an idiot.

tom

 

[Martin Gregorie]

...

The Cygwin project includes an openSSH implementation that just plain
works from a command shell. I've used it for years, often leveraging
its port-forwarding capabilities.

I thought I'd better mention openSSH's Java package because the OP seems
to think his potential users are incapable of using a CLI scp command. He
can, at least, use the package to build them a GUI.

 

[Driss]

Your requirements are not very clear and your blog post
ain't very clear either...

You want something entirely free, that must be cross-platform
and that is very easy for the user.

You also apparently want two lambdas users to be able to bypass
any kind of firewall.

I'm sorry but if scp is "too complicated" what exactly makes
you think installing Java would be easier for a "normal user"?
(hint: Java ain't installed on all Windows machine nor on all
Linux machines).

Your best bet to bypass firewall is to have one end create an
HTTP server and the other connect to it. Amazon S3 manages
daily terabytes of files transfer using HTTP requests.

If temporarily storing 10 GB on Amazon S3 wasn't an issue
(is it really at $0.15 cents / GB per month, especially
if your 10 GB stays there only one or two days?) then I'd
simply have one side push to Amazon S3 and the other pull
from it. Or to any service using Amazon S3 as storage (I
think if you invite enough people you can get up to 5 GB
for free lifetime at DropBox, they use S3 behind the scene).

In any case solving this by making two very dumb users on
the two sides (apparently they're very dumg) using some
self-reinvented broken Java wheel like you plan to doesn't
seem like that great a plan.

If it was a recurrent problem I'd set up a Webapp that'd
serve as a frontend to S3 and make users enter the domain
name of the site where your webapp is hosted in their
browser.

Even dumb users should be able to figure that out.

The problem is that it takes at least some skill to
do and seen the level of confusion shown by your posts
here and on your blog I doubt it's within your reach

 

[Bart Friederichs]

That depends entirely on what direction you want to make the connection.
I connect out through an unmodified firewall all the time.

out is not the problem. in is. I assume both sides are behind a home-use
router/firewall. Isn't it strange, that in the day of 20MBits/s home
lines, it not easily possible to send a file from computer A to computer
B without a third party? People still use email to send photos.

In any case ssh port forwarding to an sshd server is reasonably common
and should be safe enough if it is configured correctly. The same port is
used for both ssh sessions and scp/sftp file transfers

Okay, consider two home computers, both behind a NATed router (plain
broadband setup), one guy is a filmmaker with a MAC, the other guy is a
cameraman with a Windows XP PC. They have only a working knowledge of
the computer, and also don't want to know anything more. The 'command
line' you speak of, reminds them of DOS and they certainly will not want
to use that.

Both sides have to be able to be listener or connector (from the TCP
view), and both sides should be able to send or receive. The file is
huge (no limit, but typical is around 10GB), so being able to continue
the transfer is a must.

Same comments I made about firewalls apply to NAT routers.

1) I'd expect that only sysadmins are authorised to kick holes
in firewalls. 'Normal users' should be actively prevented
from doing so.

UPnP describes a protocol to do just that. Whether or not it is allowed
in the router, is the only bottleneck I have right now.

2) if they can't type "scp filename user@host:destination/path"
or use an equivalent GUI wrapper they shouldn't be let near
a computer.

They will be able to use an equivalent GUI wrapper (my program).

Wrongo. I've just listed some.

Yes, and I listed reasons why they are not suitable for what *I* want to
accomplish.

Bart

 

[Bart Friederichs]

Your requirements are not very clear and your blog post
ain't very clear either...

You want something entirely free, that must be cross-platform
and that is very easy for the user.

Well, pretty clear if you ask me.

You also apparently want two lambdas users to be able to bypass
any kind of firewall.

Aren't bittorrent clients doing that all the time with UPnP?

I'm sorry but if scp is "too complicated" what exactly makes
you think installing Java would be easier for a "normal user"?
(hint: Java ain't installed on all Windows machine nor on all
Linux machines).

I understand, but installing it, is fairly easy.

Your best bet to bypass firewall is to have one end create an
HTTP server and the other connect to it. Amazon S3 manages
daily terabytes of files transfer using HTTP requests.

If temporarily storing 10 GB on Amazon S3 wasn't an issue
(is it really at $0.15 cents / GB per month, especially
if your 10 GB stays there only one or two days?) then I'd
simply have one side push to Amazon S3 and the other pull
from it. Or to any service using Amazon S3 as storage (I
think if you invite enough people you can get up to 5 GB
for free lifetime at DropBox, they use S3 behind the scene).

I don't want a third party. Why is that so hard to understand? It is
insane that we have broadband connections and cannot transfer files from
A to B in a straightforward way.

I have thought of using the third-party way, in fact, that is what he is
using now. And yes, that works, and yes that costs money. I just think
it is weird to pay money for something that is available already
(bandwidth that is).

In any case solving this by making two very dumb users on
the two sides (apparently they're very dumg) using some
self-reinvented broken Java wheel like you plan to doesn't
seem like that great a plan.

The project can still fail miserably if the whole thing doesn't work.
But if it does, it might make life for this guy a little easier.

The problem is that it takes at least some skill to
do and seen the level of confusion shown by your posts
here and on your blog I doubt it's within your reach

I guess I am completely missing the point here. What is so confused
about the blog post? I have stated some requirements, and all you do is
changing these to fix the problem. That's not fixing the problem.

I agree with you on the sending/receiving bug I have. That should be
fixed well. But for now, I rather have a working proof-of-concept with a
known bug, than correct code.

Bart

 

[Martin Gregorie]

out is not the problem. in is. I assume both sides are behind a home-use
router/firewall. Isn't it strange, that in the day of 20MBits/s home
lines, it not easily possible to send a file from computer A to computer
B without a third party? People still use email to send photos.

Its not strange at all in the hostile Internet environment of attacks by
bot herders, etc. What you're complaining about it is nothing to do with
the technology.

Okay, consider two home computers, both behind a NATed router (plain
broadband setup), one guy is a filmmaker with a MAC, the other guy is a
cameraman with a Windows XP PC. They have only a working knowledge of
the computer, and also don't want to know anything more. The 'command
line' you speak of, reminds them of DOS and they certainly will not want
to use that.

You entirely missed my point, which was to use the openSSH connection
library behind a GUI of your own devising.

Both sides have to be able to be listener or connector (from the TCP
view), and both sides should be able to send or receive. The file is
huge (no limit, but typical is around 10GB), so being able to continue
the transfer is a must.

Since ssh is a fairly secure protocol its reasonable to forward ssh
connections from the firewall to an ssh server, but IMO it would be
extremely foolish to do that with a less secure protocol. End-users will
still have to configure their router to forward port 22 to the host
running sshd but thats a one-off task, but if you want the end-user
systems to accept incoming connections you've no choice over that.

Since you have no control over the type of firewall or NAT router the end-
user has installed the only straight forward way to avoid port forwarding
is to run a store-and-forward relay server.