gigbit lan xfer slow on freebsd

Discussion in 'Linux & BSD' started by Dark Atheist, Nov 19, 2007.

  1. Dark Atheist

    Dark Atheist Moderator Political User Folding Team

    Messages:
    6,376
    Location:
    In The Void
    i can only get about 15-20meg sec lan xfers on a gigbit connection (2 pc directly connected using cat6 cable) if i use xp i can get about 30-40meg a sec and the same if i use slackware on there, i have googled about and have found a few things to change using the sysctl command but it seems to have had no effect what so ever :(
     
  2. LordOfLA

    LordOfLA Godlike!

    Messages:
    7,027
    Location:
    Maidenhead, Berkshire, UK
    hard disk throughput :)

    Did you enable soft-updates on your target partition(s)? Makes a big difference :)
     
  3. Dark Atheist

    Dark Atheist Moderator Political User Folding Team

    Messages:
    6,376
    Location:
    In The Void
    Last edited: Nov 19, 2007
  4. Dark Atheist

    Dark Atheist Moderator Political User Folding Team

    Messages:
    6,376
    Location:
    In The Void
    should i do as it advises and do it on all mounted folders/fire systems?

    /usr
    /var

    and so on
     
  5. LordOfLA

    LordOfLA Godlike!

    Messages:
    7,027
    Location:
    Maidenhead, Berkshire, UK
    I set soft updates on every file system but swap :)
     
  6. Dark Atheist

    Dark Atheist Moderator Political User Folding Team

    Messages:
    6,376
    Location:
    In The Void
    so all i need to do is

    tunefs -n enable /var /log /boot (etc), just the once, and it will stick? - i dont need to put this in a conf file ?
     
  7. LordOfLA

    LordOfLA Godlike!

    Messages:
    7,027
    Location:
    Maidenhead, Berkshire, UK
    yup it'll stick.
     
  8. X-Istence

    X-Istence * Political User

    Messages:
    6,498
    Location:
    USA
    Soft update is normally turned on by default when you install FreeBSD. The installer sets the flag for you.

    What are you using to test the transfer speed? FTP? SMB? NFS?
     
  9. Dark Atheist

    Dark Atheist Moderator Political User Folding Team

    Messages:
    6,376
    Location:
    In The Void
    ftp, i max out at 20meg
     
  10. Geffy

    Geffy Moderator Folding Team

    Messages:
    7,805
    Location:
    United Kingdom
    Are you using Jumbo frames?
     
  11. Dark Atheist

    Dark Atheist Moderator Political User Folding Team

    Messages:
    6,376
    Location:
    In The Void
    how would i change that under freebsd ? because after doing some reading 1500 is way too low for gig speed

    em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=b<RXCSUM,TXCSUM,VLAN_MTU>
    inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
    ether 00:0e:a6:f0:9e:0c
    media: Ethernet autoselect (100baseTX <full-duplex>)
    status: active
    re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=1b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
    inet 164.168.1.1 netmask 0xffff0000 broadcast 164.168.255.255
    ether 00:12:17:5a:ea:23
    media: Ethernet autoselect (1000baseTX <full-duplex>)
    status: active

    have tried what it says here
    http://www.enterprisenetworkingplanet.com/nethub/article.php/3485486

    and
    tcp_sendspace mine are setup to 131072
    tcp_recvspace also setup to 131072
    rfc1323 my server setup to 1

    edit: soft updates was not set on / or the ftp upload dir - will see if that helps (it didnt)
     
    Last edited: Nov 21, 2007
  12. X-Istence

    X-Istence * Political User

    Messages:
    6,498
    Location:
    USA
    What gigabit network card is this? What manufacturer. What driver does FreeBSD use for it.

    Also, how fast is your CPU, and when you are doing the transfer, type into a terminal:

    top

    let it run for a few seconds, then type q.

    Then paste the line with percentages to us. Repeat it a few times so we can get an idea where the most "time" is spent.

    I am looking to see how much of your CPU time is spent in system, and in interrupts.

    Edit:

    I personally have an Athlon XP 3200+ with 2 GB's of ram serving my file server, with two D-Link DGE-530T's. When I do a local speed test on the server itself using:

    dd if=/dev/null of=/mnt/storage/outfile bs=512

    and then in another terminal open up:

    systat -iostat 1

    I can see I am doing a consistent 50 MB/sec writes to the hard drive. Which is the max speed the IDE/SATA drives can manage on that motherboard.

    Then over FTP transferring movies, I am actually limited by how fast my Mac can transfer movies from the external drive (35 MB/sec) (over Firewire, it does worse over USB). So if I start two transfers, one from my external drive (35 MB/sec) and one from my internal laptop drive (20 MB/sec), I write to the FTP at about 51 MB/sec. (TCP/IP overhead included, actual writing to HD in the server is about 49 MB/sec)

    I have done no tweaking, of FreeBSD or any of the underlying services. I use vsftpd as my FTP server, as it is one of the most secure and fastest FTP servers available.
     
    Last edited: Nov 21, 2007
  13. Dark Atheist

    Dark Atheist Moderator Political User Folding Team

    Messages:
    6,376
    Location:
    In The Void
    in the freebsd pc its a linksys 1gb card - cpu is a p4 3.4, the intel card is using the em driver and the linksys is using the re driver

    i will try what you suggest in the morning
     
    Last edited: Nov 21, 2007
  14. Geffy

    Geffy Moderator Folding Team

    Messages:
    7,805
    Location:
    United Kingdom
    What does FreeBSD name your network interface? Most of the gigabit drivers are capable of jumbo frames but I think one or two might not be.
     
  15. Dark Atheist

    Dark Atheist Moderator Political User Folding Team

    Messages:
    6,376
    Location:
    In The Void
    em0 = intel (internet)
    re0 = linksys (intranet)

    em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=b<RXCSUM,TXCSUM,VLAN_MTU>
    inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
    ether 00:0e:a6:f0:9e:0c
    media: Ethernet autoselect (100baseTX <full-duplex>)
    status: active
    re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=1b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
    inet 164.168.1.1 netmask 0xffff0000 broadcast 164.168.255.255
    ether 00:12:17:5a:ea:23
    media: Ethernet autoselect (1000baseTX <full-duplex>)
    status: active


    i could have sworn i pasted the output x wanted ealier this morn, but now the post dont look like its here :(

    i'll add it again

    $ top
    last pid: 1665; load averages: 0.56, 0.14, 0.05 up 0+04:30:30 12:56:40
    55 processes: 2 running, 53 sleeping
    CPU states: 7.3% user, 0.0% nice, 18.6% system, 16.9% interrupt, 57.2% idle
    Mem: 85M Active, 314M Inact, 154M Wired, 52K Cache, 112M Buf, 1449M Free
    Swap: 4069M Total, 4069M Free

    PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
    1658 root 1 103 0 3556K 3264K RUN 0 0:09 31.05% glftpd
    784 mysql 8 20 0 319M 56024K kserel 0 0:03 0.00% mysqld
    786 root 1 8 0 18484K 11696K nanslp 0 0:01 0.00% httpd
    872 www 1 4 0 19628K 13696K accept 0 0:00 0.00% httpd
    871 www 1 4 0 21600K 15636K accept 0 0:00 0.00% httpd
    875 www 1 4 0 19644K 13692K accept 0 0:00 0.00% httpd
    873 www 1 4 0 19644K 13692K accept 0 0:00 0.00% httpd
    891 www 1 4 0 19644K 13692K accept 0 0:00 0.00% httpd
    889 www 1 4 0 19620K 13576K accept 0 0:00 0.00% httpd
    874 www 1 4 0 19620K 13576K accept 0 0:00 0.00% httpd
    890 www 1 4 0 19620K 13576K accept 0 0:00 0.00% httpd
    806 root 1 96 0 3504K 2872K select 0 0:00 0.00% sendmail
    675 root 1 96 0 5228K 3100K select 0 0:00 0.00% nmbd
    816 root 1 8 0 1404K 1124K nanslp 0 0:00 0.00% cron
    558 root 1 96 0 1376K 1028K select 0 0:00 0.00% syslogd
    1657 darkness 1 96 0 2440K 1692K CPU0 0 0:00 0.00% top

    systat -iostat 1 - doesnt show up the raid drive though - shows all the other drives
     
    Last edited: Nov 22, 2007
  16. Dark Atheist

    Dark Atheist Moderator Political User Folding Team

    Messages:
    6,376
    Location:
    In The Void
    found a solutions of sorts - seems the card under freebsd cant go higher than 7500 mtu, have changed it now and am getting near 30 meg/sec which is better than the 20 meg/sec i was getting, should i change the mtu to 7500 on the windows box instead of the 9000 its on now ?

    edit: cant as it only does 1500 3000 4500 and 9000 :(
     
  17. X-Istence

    X-Istence * Political User

    Messages:
    6,498
    Location:
    USA
    em, that is an Intel gigabit card as well right?

    Try switching that out for the realtek (re) one.
     
  18. Dark Atheist

    Dark Atheist Moderator Political User Folding Team

    Messages:
    6,376
    Location:
    In The Void
    i have tried both, the realtek (re0) will only go up to 7500 and max speed tops out @ 25meg, intel (em0) will go up to 9000 on the jumbo frames, and tops out at about 30meg, and the intel seems to be giving me a few extra meg on the lan transfers

    edit: have found a few more options to tweak on the windows box - changing optimize for, from cpu to throughput so will see if that helps
     
    Last edited: Nov 27, 2007