Yes,   "old" commodity hardware saving the day.   Our Cisco firewall was unable to fill the 1 gig pipe due to license (they call it hardware) limitations.   Its a joke really,  in 2016.        

This pfsense box is a 6 interface 2630@2.4GHz with 12 cores and 96 gigs of memory.  lol..    I use most of the RAM for SQUID and ClamAV.  

-greg

On Tue, Nov 15, 2016 at 3:25 AM, Peter Smith <peter.smith@framestore.com> wrote:

Ooh, pfSense traffic charts. :-)
--

On 14 November 2016 at 22:32, greg whynott <greg.whynott@gmail.com> wrote:

Make sure you look at GridFTP WIll.   I can saturate +94% of a 1 gig link between Toronto and Seoul Korea consistently at any time of the day (could of likely used it all but normal studio traffic was sharing it.).  It can use UDT instead of tcp by use of libudt,  but it isn't really necessary in most cases.  

Take a look at these images I have included in this message,  using the same data set to transfer,  one is via straight up rsync with multiple streams in parallel (8),  the other is with GridFTP using -p 8 (also 8 streams).   rsync has a hard time maintaining any consistency where GridFTP pegs it for the duration.     

We will be testing it against the commercial offerings soon,  but based on my past experiences it isn't worth the time to do so...The cat is out of the bag -  its not a black art anymore to use long and wide pipes efficiently.  

 Amazon, S3 and friends are supporting GridFTP and other service providers are getting on board.   

RSYNC on its own:

GridFTP:

On Thu, Apr 14, 2016 at 2:57 AM, Michael Oliver <mcoliver@gmail.com> wrote:

As others have pointed out Aspera and FileCatalyst are both great options.

Free stuff if you like science projects: gridftp, tsunami, udt, fdt, unison, bbcp

If you want to get really crazy there is always the web10g project: https://web10g.org

On Mon, Apr 11, 2016 at 12:13 AM, julian firminger <justdigitalfilm@gmail.com> wrote:

If you're looking to cost cut over Aspera, give FileCatalyst a look.  Good product, great support too.

Julian Firminger

Snr. Systems Administrator, 

United Broadcast Facilities

Amsterdam, The Netherlands

On Sun, Apr 10, 2016 at 4:05 PM, DougMielke <content@studiosysadmins.com> wrote:

Hi Will, I know of a great option that let's you sync unlimited data between unlimted points. In the interest of not using this forum for commectial purposes, please contact me at dougmielkemail@gmail.com cheers Doug

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

--

Michael Oliver
mcoliver@gmail.com
858.336.1438

PGP Public Key: https://goo.gl/a2CnHF

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe

To unsubscribe from the list send a blank e-mail to mailto:studiosysadmins-discuss-request@studiosysadmins.com?subject=unsubscribe