«

»

Mar 06

The Updated Home Lab 2.0

(An update to my original post on home labs HERE)

You’ll learn that if you create a home lab it will always be evolving. I had a different vision for my lab when I first started and now I’m on my home lab 2.0.

DSM_4.1

First thing I realized is that I wanted both hosts to have equal resources, so I balanced them out with 24GB of RAM each (Hopefully will upgrade to 32GB shortly). The next step was getting rid of the 1TB Raid1 and going with a 4TB Raid5 and a 256gb Samsung 840 Pro SSD. I realized I would need more space than 1TB for what I was doing and storing, and I wanted the Read-speed of a Raid5. The SSD is fantastic as well.

SamsungSSD
The next step I took was purchasing a 3rd ADATA 8GB USB Flash Drive and installing ESXi on it for my home desktop. When I want to try something new out that might stretch the resources of my current lab I can plug it into my desktop (I7-2600, 16GB ram) and have an extra host for a while ๐Ÿ™‚ then if I need my desktop back I simply remove the flash drive and reboot my system.

adata-n005-2

The third step I took was to enable Jumbo frames on everything. I upped the MTU from 1500 to 5000 and saw a great performance increase (20MB/s to roughly 120MB/s) before tweaking anything else.

DS412+

I have a DNS server in my lab but I decided as well to setup a 2nd DNS server on my Synology DS412+ since it is always on regardless of the state of my hosts. I also setup a Syslog and VPN server on my Synology for other purposes as well.

Now my lab currently looks like the following:

2x Hosts:

  • I7-3770 CPU
  • 24GB Ram
  • 8GB USB Flash drive (boot and logs)
  • 1TB internal HD (because I had them laying around)
  • connected to Synology LUNs (SSD iSCSI, SSD-NFS, iSCSI on Raid-5)
  • **Note** The version of ASRock motherboard I am using doesn’t support VT-d, however the next step up does but will run you another $50+/motherboard.

1 optional host:

  • I7-2600 CPU
  • 16GB RAM
  • 8GB USB Flash Driveย (boot and logs)
  • Connected to Synology LUNs like those above

1 Synology DS412+ NAS:

  • 3x Western Digital Red 2TB 7200 RPM 64MB Hard Drives
  • 1x Samsung 840 Pro SSD 256GB Hard Drive
  • Not currently using link aggregation but using Jumbo Frames
  • DNS, Syslog, and VPN Server (also connected to Amazon Glacier,not active)

1 TP-Link Gigabit Unmanaged 5-port Switch

1 TP-Link TL-WDR3600 Gigabit router (running DD-WRT)

Total Cost was roughly $2,400

All in all this is a very sturdy build. I am currently running roughly 10 VM’s 24/7 and have the capacity to be running probably triple the load. I am currently installing and configuring vCloud Director and Nexentastor for de-duplication.

MORE TO COME!

 

ย  ย  ย ย 

10 comments

1 ping

Skip to comment form

  1. GadgetGuy

    I don’t think adding an SSD to that synology model will offer any performance benefit.

    1. The only Synology models that I’m aware of that utilize their SSD caching feature are the XS+ models
    http://download.synology.com/download/ds/userguide/Synology_SSD_Cache_White_Paper.pdf

    2. SSD caching in a NAS, will still be bottlenecked by a single GB (or even dual GB) connection. The high end synology models (like yours) with mechanical drives alone can nearly saturate this connection. Plus to support dual GB connection, you need a switch that does LACP, which I don’t think an un-managed switch will give you)

    I have the Synology DS1512+ on an HP Procurve 1810G-24 with LACP enabled. I do however use an SSD in my ESXi server. I just bought a single 512GB ssd and am using thin provisioning. So all my VM’s boot fast and operate quickly. I still have about 230+ GB free after 8 VM’s and loading some Datestore ISO’s. For any large storage needs, I just connect the VM’s to the synology via iSCSI for large data volumes.

    This setup looks great overall though. Very similar to my own. Best of luck to you!

    1. Brian Graf

      Thanks for the comment. You are right about the SSD. This week alone I ended up transferring all VM data off my SSD and onto the Raid 5, Pulling the SSD from my NAS and adding it to my laptop ๐Ÿ™‚ I am now seeing 490 MB/s write and 498 MB/s Read with it. This is a much better use for an expensive SSD than placing it in a home NAS. When prices come down a bit more on them I will most likely end up placing one in each of my hosts like you, but until the 512GB come down in price (or I get a nice bonus from work or something) I’ll be without them.

  2. Ewald van Geffen

    Hey, came in from tinkertry. Good stuff, I myself went a little more expensive route. One remark, why the odd-jumbo-frame size? afaik it isn’t standardized to a certain setting but I believe 9k is the ‘customary’ norm for jumboframes. Just asking why, incompability in 9k or 5k gave enough speed already? I wonder.

    1. Brian Graf

      Welcome Ewald and thanks for the comment. I chose 5k for two reasons:
      1) I was having issues with my router and DD-WRT (Might have been a bad flash but it wouldn’t keep the MTU setting saved above 5k)
      2) I saw enough improvement that I didn’t take the time to troubleshoot further.

      I’m currently working on a data center project at work that has kept me traveling and haven’t had much time to work on my lab and blog in the past 3 weeks (until today) so I’ll play with it some more, try re-flashing my router as well and let you know what I find out!

      1. Ewald van Geffen

        If this works for you, it’d stick to it until something else requires you too (not that having a recent router flash is bad either, positive collateral).

        I’d rather play with some new toys than fixing not-broken-ones. If you want to do both, you could try vyatta or pfsense, my favorite router toys.

  3. Mike

    What kind of performance did you get out of the NAS solution with ESXI 5.1 and DS412+

    How many VMs could you run?

    What kind of IOPS were you getting?

    What kind of performance did you see?

    1. Brian Graf

      Hey Mike,

      Thanks for the questions. To answer them quickly, the quick and simple answer to all of your questions is “It Depends”
      What kind of performance did you get out of the NAS solution with ESXI 5.1 and DS412+: The performance was satisfactory depending on the setup I had. Performance has a lot of factors playing into it, such as: Number of Drives, Drive Speeds, Raid-type, Network speeds, and Network Protocol (iSCSI/NFS). That being said, I’ve been pleased with it’s performance.

      How many VMs could you run? once again, it depends… It depends on what you have the VM’s doing, if they are mostly just running, then I imagine you can run quite a few. If you are planning on having them all very active and chugging through data or running SQL databases then obviously not as many. Most of my VM’s were utilizing very few resources and in turn, very little was being exchanged on the NAS. The most VM’s I had running at a given time so far was 12 and it ran fine.

      What kind of IOPS were you getting? I am currently on the road and can’t remember off the top of my head. However I remember getting decent IOPS. With my SSD I was getting above 300MBps between the SSD and the RAID5. I will update this when I get home and check again though

      I know it’s hard to give a definitive answer on this. I will say that I have been extremely impressed with the features and functionality of the Synology NAS, above and beyond being just a Storage device. I use the VPN feature daily as well as many other features Synology Offers.

  4. nick

    I was wondering if you had changed you set up in your Synology, if so what do you have set up and how does it run? I’m about to buy a DS412+ and create two RAID 1 Arrays, the first with 2 x 3TB WD Red’s and then one with 2 x 240GB SSD’s.

    Or I may get the DS713+, as with this device you can add the expansion devices.

    This is my first NAS so would appreciate any input.

    I’m going to be using it for VM/MS labbing.

    1. Brian Graf

      Hi Nick,
      I have changed my setup several times since I wrote this post. Primarily I changed because I now work for VMware and have several labs that are more powerful and robust than my homelab that I use for testing purposes So my homelab is used less… Currently I’m running my DS412+ in a Raid 5 with 3 – 3TB WD Red’s and I have a 1.5TB WD Red that is in the 4th bay that is used for other things such as my home security software.

      I took my Samsung 250GB SSD and threw it in my desktop and use it for when I’m testing things out in VMware Workstation (my desktop is an I7 3770 with 32GB ram) so I have a lot of my templates and VM’s on the SSD and use linked clones.

      As far as the Synology goes, I LOVE IT! if I had the money I would go with a larger synology (DS1513+ or DS1813+) for the ability to expand and scale in the future, however, the DS412+ is a great place to start. In my time with my synology I have done several different configurations including using the SHR (Synology Hybrid Raid), as well as 2 Raid 1’s and Raid 0/ Raid 1. You really can’t go wrong, it’ll come down to a personal preference for yourself as far as which Raid you use.

      Another thing to consider is using Block-level storage or File-level storage. Both are great and have somewhat different purposes. You may want to take some time when you build your lab/NAS to create a few different LUNs and try different connection methods as well (NFS/iSCSI) to see where you personally get the best performance. I found oddly enough, and others have verified this as well, that NFS had faster speeds than iSCSI. This shouldn’t be the case and maybe it had something to do with the Synology DSM version at that time, but like I said, give it a shot and see what works best for you. Let me know how it goes!

      1. nick

        Thank you for the reply. Do you find that the Raid 5 Array with them HDD’s is fit for VMWare with reasonable load? I was tempted to go for 2 RAID 1 Arrays, 2 x WD RED and 2 x 240GB SSD’s, but for the size/price ratio it would probably make sense for a lab to just go the same way you did.

        Yeah I would like more bays… but the Synologies are already pricey as they are. I’m sure it will do me for a good year ๐Ÿ˜€

        I’ve read quite a few blogs over the last weeks that have stated the same thing, tested iSCSI v NFS, NFS always comes out on top. But as you said, I will have fun playing around with it.

        I was going to be creating 2 White box’s for my new lab, after some research the cheaper and also easier (not as fun?) way is going to be to buy some Dells, just about to hit buy on 2 x T110 II E3-1220 V2 with 32GB for my lab now.

        Anyway, great blog, keep it up.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">

twenty − two =