Start a new topic

Issues with Speed on 6.5

zzzp @ Wed Nov 12 09:07:43 EET 2014
Hi there

Over the LAN I can get around 20MB a second, however over the WAN all jobs sit anywhere from 300kb-500kb.. We have a 1000megabit connection for our WAN and frequently use other methods of transfers getting very fast transfers e.g. Aspera Faspec which often gets 100Mbit a second.

Is there anywhere I could look into this and try fix issue with slow traffic over the WAN? We are using Crush 6.50 Build 3

Thanks very much in advance.

spinkb @ Wed Nov 12 10:39:05 EET 2014
A few really important things missing here...

What protocol are you using?

When you say jobs...your using the Job scheduler in CrushFTP to push files outgoing?

What is your latency in milliseconds between you and the destination.

Let me know,
zzzp @ Thu Nov 13 06:25:08 EET 2014
Thanks for your reply.

So inside the LAN to both these servers I can get good throughput, maybe around 20MBs a second.. But for WAN client connecting in it never really goes above 500kbs even tho these clients have 1g WAN connections.

spinkb @ Thu Nov 13 07:35:05 EET 2014
On your HTTPS port, turn off redirect to https...its redundant, its already HTTPS.

20MB/sec is not good performance. I test on my gigabit LAN at 100MB speeds, and localhost at 270MB/sec speeds. So 20 is not very good...might indicate a different CPU starved issue.

Do a localhost test using plain FTP, no encryption. You should be able to get your disk speed max...and definitely faster than 20MB/sec.

HTTPS uses encryption, so you need a fast CPU to get fast speeds there. But it can still get around 1Gbit speeds.

The really important thing here though is latency. There is a magic calculation that will give you a rough estimate of the max speed to expect for FTP/HTTP(s) based on latency.

524 divided by latency milliseconds = max Mbit

So if you have 30 ping, that means 524 / 30 = 17Mbit max. For a 100ms ping, that is 5.2Mbit. So it doesn't matter if they have 1Gbit WAN, the latency kills your performance.

To get around this, companies use products like Aspera, CrushTunnel, etc. These overcome latency and still give you full speed. CrushTunnel can get up to 1Gbit speeds, just like competitors do, but requires the enterprise license to use this feature. (Still 10x to 100x cheaper than competitors.)

So what is your latency?

zzzp @ Thu Nov 13 11:15:47 EET 2014
Thanks for the reply.

Ok as an example we have WAN client computers which IPs ive pinged ranging from 5m/s to 100m/s and the speed is similar for both e.g around 500Kbs

Now locally on the server I have done some local FTP testing and we are getting very average speeds too, around 40 Megabytes per second. So that indication is of concern. Thats moving straight to a SAN via iSCSI but locally using FTP to do the transfer to itself as such. Disk writes locally also are around the same speed so there is a bottle neck there when moving using other protocols.. Although still speeds over WAN should still be abit quicker I would have thought? We dont have any throttling on our Firewall outbound for the FTP rule. Kind of inclined to build another server and do some WAN testing from there to maybe show up any other potential issues?

spinkb @ Thu Nov 13 12:56:31 EET 2014
There are likely some disk issues then or something if your limited to 40MB on OS and CrushFTP. I write using FTP on localhost at over 270MB/sec, SSD drives.

The 100ms ping is definitely a latency issue slowing down the transfer, and you will either need to do OS TCP tuning on both sides, possibly firewalls too, and routers...or use CrushTunnel to overcome this.

The 5ms site however would have to be in your LAN to get 5ms I think...just leaving the LAN would have to be more than 5ms I something sin't right there. And at 5ms you would be able to get 10MB/sec speeds over FTP, or HTTP(S).

Encryption has a penalty for speed because of the CPU usage. i7 CPU, two cores used gets around 80MB/sec roughly on a single transfer. But you would always be able to still et 100Mbit even sign encryption.

If SFTP is involved, all the rules change as its a different type of protocol doing its own multiplexing and has its own limitations in max speeds.

Login to post a comment