If you don't know why you should be interested in an unRAID server, read this

Having setup my own unRAID server following a lot of reading, reading, checking and more reading on various parts of the Lime Tech website and forums, I thought it would be handy to write up a a guide and quick reference to setting up and managing an unRAID server. This will not cover all the capabilities of the server but should hopefully be a good first port of call before going off to do further reading and explore what you can do with your unRAID server. The guide will be split into four sections each covering a different topic:

• Installation/setup
• Maintenance, updates & extras
• Adding additional drives
• Working with parity & resolving errors (aka oh noes my HDD is dying!)

1. Installation


1.1 - Hardware

There are no hard and fast rules for hardware to run an unRAID server, but the basics are outlined here:

Minimum System Requirements?

•     Intel compatible computer
•     Motherboard and components compatible with unRAID software (based on Slackware), and Linux kernel 2.6.x
•     512MB of RAM (recommend 1GB or more depending on usage)
•     USB flash drive - at least 512MB
•     Can operate without video, monitor, or keyboard; does not use mouse

    See Hardware Compatibility for extensive hardware and compatibility information

This guide assumes you are using an HP Microserver, which is a perfect little workhorse for this project as it is small, quiet, feature rich and supports 5/6 HDDs right off the bat, along with the internal (bootable) USB port. XEntity has already compiled an excellent Microserver Setup Guide.

Whatever hardware you consider, make sure you factor in storage expansion (you weren't really going to buy 5x 3TB HDDs right away, were you?) and support for booting from USB. An unRAID Server Pro license will support up to 20 data + 1 parity + 1 cache drives.

Because unRAID runs directly from bootable flash USB pens, make sure that you purchase a quality USB pen that is likely to have a long and happy life! I cannot stress this enough - the server will be running from your flash device and if it is a cheap one you might end up with a dodgy controller or poor quality chips that will bring your server down in time - I purchased a Corsair Voyager 4GB which was the smallest I could find. Also, if you plan on purchasing a licensed version of unRAID - which no doubt you will at some point like I will do - then you will need a USB pen that has a valid serial number (GUID) for the license key file (cheap/unbranded flash devices may not have a serial):

The license is controlled by a key file for the Plus and Pro versions, which is tied to a unique GUID (Globally Unique IDentifier) associated with your unRAID flash drive. The unRAID software is identical with each license, only the features and drives are limited with the Plus and Basic versions. See the next question for a table of the differences.

The free Basic version is both a fully working 3 drive system, and a freely available demonstration version, that can easily be set up by downloading the latest software (downloads), and preparing a bootable USB flash drive (USB Flash Drive Preparation). There is no registration necessary. If you decide later to purchase a Plus or Pro license, just copy the license file to your unRAID flash drive and reboot, and the full capabilities and drives of the purchased license will then be available. Apart from the limit on the maximum number of drives, the array is the same under all licenses. No conversion of data or software is necessary.

You can read more about this on the Registration Keys page.


1.2 - Installation & 'Preclear' (optional)

First off, download the latest version of unRAID and copy the files onto your formatted flash USB pen following the unRAID server installation which I have summarised below:

Step 1 Plug the Flash into your PC and re-format it using Windows:

•     Open My Computer (XP) or Computer (Vista/Win7), right-click your Flash device and select Format...
•     For File system, leave what Windows sets as default.
•     For Volume label, enter "UNRAID" (all-caps and without the quotes).
•     Select Quick Format and click Start.

Step 2 Click on your Flash device (to open it) and drag the entire contents of the unRAID Server zip file to the Flash.
Step 3 For Windows XP, click the file make_bootable.  A DOS window will open and run the syslinux utility on the Flash.  For Windows Vista or Windows 7, right-click the file make_bootable and select Run as administrator.
Step 4 Once again, right-click your Flash device in My Computer or Computer and select Eject.  Your Flash device is now ready to boot into unRAID Server OS.

'Preclear' (optional)

The preclear_disk.sh script is used to test your disks and prepare them for the unRAID environment. The merits of preclearing are discussed in this forum post where you can also download the script once registered on the forums. In summary, preclearing will "burn-in" a new disk before adding it to your array and allow you to skip the the lengthy "clearing" step performed by unRAID when you add a new disk to the array. Be aware though this takes a looong time. Preclearing takes around 1 day for a 2TB drive but will thoroughly analyse the disk and prepare it for a working lifetime in your server.

You can preclear a disk once the script is copied to the flash USB drive (to the same folder that currently has bzroot and bzimage - on linux, that folder would be /boot or from windows file-explorer,the folder would be at \\tower\flash). If you just plug the flash drive into your windows PC, it will be the top level folder on the drive.

After booting when at the linux console prompt, type in root and press enter then type in the following commands:

cd /boot
preclear_disk.sh -l

to list the disks available to be precleared, then:

preclear_disk.sh -A /dev/sda

replacing /dev/sda with your own drive choice from the list. If you do not have a new 'advanced format' drive that works best with a 4k alignment of its data then remove the -A switch and the drive partition will not start at sector 64 (all recent drives will be advanced format). The screen will update you (slowly!) of progress preclearing the drive.

I would advise preclearing new disks before adding them to the array for peace of mind. If you want to skip this step, simply plug your USB pen into your new server and boot up!


1.3 - First boot

1.3.1 - Web Management Utility

If you are watching the system boot it will take you to a server console screen eventually where you can login as root to perform various maintenance actions later on. For now, you will not need this - or if you are running 'headless' without a monitor you can access it over the network via Telnet (more on that later) - so wait a moment until it finishes booting and from any internet browser window on your regular computer/laptop type into the address bar:
//tower

This will take you to the Web Management Utility. If it does not, check your network connections. Assuming that your have your computer and server plugged into a DHCP enabled router, it should connect right away. If not, or if you need to set your network settings manually, follow the instructions here.

This is the main screen for UnRAID. Since we have not yet added any hard drives to setup an array this screen is fairly empty. Let's start by clicking on the Settings link.


1.3.2 - Web Management Utility - Settings

First off go to the Identification section.

Server Name - this is what you want to name the server. It will then be accessible via //servername instead of //tower. This guide assumes you will be leaving it as is.
Comments - this is what would show up as a description of the server in network neighborhood, for example.

Then go to the Disk Settings section (see pic)


This is how mine is currently configured and should be ideal for most setups.

    Enable auto start - if set to yes, this will attempt to auto-start the array whenever the server is booted. It doesn't always work and I would advise leaving it set to no since when rebooting its usually after reconfiguring something.
    Default spin down delay - Set the amount of time without activity to wait before spinning down disks to save power.
        I'd say an hour is fine for most people but tweak as appropriate for your environment. If set too short you will find yourself waiting for disk spinups. If set too long you will waste power.
    Force NCQ disabled - Disable native command queuing.
        Recommend leaving this as yes.
        Disable NCQ on all disk devices that support NCQ. This typically results in much better write throughput. That is, if this setting is 'yes', then we force NCQ off; if setting is 'no', we leave NCQ queue_depth as-is, i.e, whatever linux driver sets it to. 
    Enable spinup groups - Allow disks to be spun up/down in appropriate groups.
        Recommend leaving this as yes, I have it set as no because I don't feel it is necessary.
        Good information available here.
    Default partition format - leave this set to MBR: 4K-aligned for all new advanced format disks.

The last three settings are for advanced tweaking and should not need to be changed.

Next up is Date & Time

    Time zone - Sets the time zone for the server.
        Set as appropriate for your location.
    Use NTP - Use network time protocol to set the server time based on a remote time server.
        It's recommended to set this to yes, but I personally don't and set the time myself.
    NTP Server 1..3 - You can configure UnRAID to use specific time servers. These options are only available if Use NTP is set to Yes.
        The default pool.ntp.org is recommended.
        More information from Wikipedia and NTP Project Pool
    Current date & time - Set the current date and time.
        If Use NTP is set to Yes, this is irrelevant and NTP will set everything.
        IF you are not using NTP you need to set your time and date here.

If you made any changes here, click Apply. The time may not update and display correctly until after rebooting the server so if things don't look quite right, wait until after the next reboot to panic.

Network Settings

    Obtain IP Address Automatically - When set to yes the server will obtain an IP address from a DHCP server on the network. This has the benefit of set it and forget it but the IP address may change between reboots.
        Set to yes unless you know you need to assign a static IP address to your server.
    IP Address - Set a static IP address for the server. Not available unless Obtain IP Address Automatically is set to No.
        Set as appropriate for your network.
    Netmask - Subnet information. Not available unless Obtain IP Address Automatically is set to No.
        Set as appropriate for your network.
    Gateway - Default gateway/router. Not available unless Obtain IP Address Automatically is set to No.
        Set as appropriate for your network.
    Obtain DNS Server Address - Get DNS servers from DHCP or not.
        If you have Obtain IP Address Automatically set to yes, you probably want this set to yes as well. If not you need to manually assign your DNS servers.
    DNS Server 1..3 - You can assign up to 3 DNS servers for name resolution. Not available unless Obtain DNS Server Address is set to No.

If you have made any changes to these settings click apply. If you changed the IP address you will need to reconnect to your server using the new IP address. If you are using Windows and were connected using //tower (or a new name if changed) open a command prompt (Start Menu > Programs > Accessories > Command Prompt) and type:
ipconfig /flushdns
nbtstat -R
to clear the resolver cache and force the computer to re-query the network for the new name.

Network Services - AFP / NFS / SMB

With your basic server config nearly done, be sure to enable /change settings for AFP (mac), NFS (linux) and SMB (Windows) to suit your own networking needs.
I have AFP and NFS disabled as I only connect from my Windows PC.
For SMB:
Security Mode - This will usually be Workgroup but if not greyed out you may also have Active Directory available. If you do, you should know what to set this to.
Workgroup - Set this to the same Workgroup that your Windows computers belong to.
Local master - If set to yes UnRAID will attempt to become the local browse master
    Probably un-necessary on a small, single subnet network. Leave this set to No unless you have problems browsing to your server
    More information from Wikipedia and from Microsoft

If you made any changes here, click Apply. At this point it is recommended that you reboot the server. To do so, click Main then click the reboot button to restart the server.


1.3.3 - Web Management Utility - Users/Security

You can set passwords for "root" and any other user you define. You can limit specific shares to specific users, and restrict specific users from write permission while granting write permission to others. This is all configured per share on the Share page. Basic "root" security is available on all versions of unRAID, but User Security is only available with an unRAID Plus or Pro license. All security settings can be changed on the Security page. To delete a user, delete the user's name and password and press Save.  For more detailed information on setting up Users see here


1.3.4 - Linux Console Prompt

If you cannot access the Web Management Utilty or need to perform other tasks directly on the server (such as preclearing), you will sometimes need to use the linux console. When the system has been booted you will be greeted with a prompt similar to:

Welcome to Linux 2.6.32.9-unRAID (tty1)
Tower login:

Type root and press enter to log into the system. The prompt should change to:

root@Tower:~#

from here you can run various linux commands, run scripts such as preclear and even move files internally on the server.


1.3.5 - Telnet access

You can Telnet into the server, either using the built in Telnet app or PuTTY (recommended - its a free download). Once you Telnet in you can run important Linux commands from the console prompt as if you were on the server directly.

To open a Telnet session from Windows click Start > Run or open a Command Prompt. In the dialog box type:

telnet tower

If you renamed your server from tower, enter that name instead. If you are using PuTTY simpy choose the Telnet connection open and type in tower as the host name. I would not advise preclearing a drive from a Telnet sessions as you need to keep the Telnet session open for the entire preclear duration, if you close the session the preclear will fail!

Working with parity & resolving errors (aka oh noes my HDD is dying!)




What happens when a hard drive fails?

The whole idea behind unRAID is that it can recover from a SINGLE disk failure. It is actually easy to miss a failure unless you notice degraded performance. (Note you will only have degraded performance reading from the one drive that has actually failed, not from reading other disks in the array.) You may not even notice the degraded performance, as it is very likely it will still be sufficient to serve media files fast enough over the LAN that you would not notice. The only way to tell for sure if unRAID has detected a drive failure is to look for a red ball next to one of your drives on the Main page of the Web interface. But it is easy for even unRAID to miss the fact that a drive has failed if it is not accessed for a while. This is another reason to run the monthly parity check - to make sure that unRAID "knows" that a drive has failed.


What if I get an error?

    If your array has been running fine for days/weeks/months/years and suddenly you notice a non-zero value in the error column of the web interface, what does that mean? Should I be worried?
    Occasionally unRAID will encounter a READ error (not a WRITE error) on a disk. When this happens, unRAID will read the corresponding sector contents of all the other disks + parity to compute the data it was unable to read from the source. It will then WRITE that data back to the source drive. Without going into the technical details, this allows the source drive to fix the bad sector so next time, a read of that sector will be fine. Although this will be reported as an "error", the error has actually been corrected already. This is one of the best and least understood features of unRAID!
    There may be OTHER types of errors than this one, so it is certainly worth your while to capture a syslog after an error is detected, but this is likely what has happened. Also, if you notice this happening more than once in a very great while, you might want to consider testing and replacing the disk in question. Remapped sectors have been linked with higher than normal drive failure.
    After getting an error, run a parity check soon after, to make sure that all is well.


What do I do if I get a red ball next to a hard disk?

    If you have moved your drives around (or sometimes even if you haven't), unRAID can get confused about what drive is assigned to what slot. It will NOT START the array, and some drives may have red balls next to them. You will also see italicized drive serial numbers. You need to go to the Devices page and re-assign the right drives to the right slots. (The italicized serial numbers on the main page will guide you to assign the right drive to the right slot.) You can then safely start the array.

But if you see a red ball next to a single drive, and the unRAID array status indicates it is "Started" that disk has been taken out of service because an attempt to write to it has failed.

    First you should know that unRAID does not take a disk out of service casually, but if a disk experiences a write failure, it will do exactly that, it will take the disk out of service. A write failure is serious. A single write failure will take a disk out of service and unRAID will then show a red indicator next to it in the management interface.
    Many things can cause such a failure that have nothing to do with the drive. Cables can (and do) go bad or wiggle loose. SATA cables in particular are notorious for slipping off their connectors, if they aren't the locking type. PSU's (power supplies) can do weird things and induce failures. Motherboards can go bad. At a minimum, it is worth a little time to recheck all of the connections, to make sure something hasn't come loose. Whenever the computer case is opened, especially just before closing it up, cabling can shift and cause a connection to a drive to fail. When checking for loose connections, take care not to disturb connections to other drives, complicating your failure.
    Drives are self-monitoring through their SMART features. There is a very nice utility called smartctl that is included by unRAID (click this LINK if using pre 4.3 final version of unRAID). Here are some instructions on using it from Tom. Also see unRAID Addons and UnRAID Topical Index, SMART for more Smartctl links. If when trying the smartctl commands below, you get an error about a missing library, then see this post for instructions for installing it.

Obtaining a SMART report

At the unRAID console, or from a Telnet or PuTTY session, type:
smartctl  -a  -d  ata  /dev/sda
Notes:

    Some newer drive controllers do not like the "-d ata" command line option. If you do not get results from the form above, try running the command with the "-A" option instead (e.g., smartctl -a -A /dev/sda). This applies to all of the examples below.

    If you get an error like "error while loading shared libraries: libstdc++.so.6", then you are using a version (such as v4.4.2) that is missing a required library. Please see this post.

Look at the Devices page for the device identifier (within the parentheses) for each disk, and substitute that for 'sda' on the command line.

This command will print out the SMART info for the drive. Refer to this article to better understand the SMART report.

To copy the results to a file called smart.txt on your USB stick that you can use to post to the forums, use this command:

smartctl  -a  -d  ata  /dev/sda >/boot/smart.txt
or

smartctl  -a  -d  ata  /dev/sda | todos >/boot/smart.txt
The second form makes it easier to look at the smart.txt file from a Windows workstation.

The smartctl output will provide a bunch of statistics that the drive captures about itself.

    Perhaps the most important attribute to look at is the "Reallocated_Sector_Ct", the RAW_VALUE is a count of sectors that have been reallocated/remapped. If a sector goes bad, the drive has the ability to "remap" a spare sector to the bad sector. This is done at a low level, within the drive itself, so the OS doesn't even know it happened. (unRAID actually uses this feature to maintain the integrity of your array.) Each time this happens, the reallocated sector count is incremented. Seeing a few reallocated sectors is not necessarily a bad thing, but seeing that number start to go up is often a sign that the drive is failing. Anytime you see a value other than 0 you should closely monitor the drive. If the number holds steady and does not increase even after several parity checks, your drive is likely okay. But if it seems to be going up by even 1 or 2 at a time, start to be concerned. This is likely the first hint that the drive is failing. A special note about reallocated sectors, bad cabling CANNOT cause reallocated sectors to occur.

    An equally important attribute is the "Current_Pending_Sector", the RAW_VALUE is a count of suspect sectors pending reallocation. It should ALWAYS be zero. If not, then you will probably (but not always) see the Reallocated Sector Count increase in the future, when this does return to zero. Before remapping a suspect sector, it tests it one last time, and *may* pass it and not remap it. (There are good reasons why it is designed to work this way.)

    Another important stat to look at is the "Temperature_Celsius". It tracks the current and min/max temperatures of the drive. If your drives are running hot (see recommendations in the "Preventative Maintenance" section above), consider adding active cooling to your hard drives.

    One user had the "UDMA_CRC_Error_Count" greater than zero. Research showed that this can be caused by bad cabling ("Possible causes of UDMA CRC errors are bad interface cables or cable routing problems through electrically noisy environments (e.g., cables are too close to the power supply).")

    Near the end of the smartctl report is a list of the last few errors the drive encountered. Errors that indicate that commands are not recognized is a sign of bad cabling, and not necessarily of a bad drive.

    (Add more info about specific smartctl statistics)

Each of the smartctl attributes are provided in "raw" format (RAW_VALUE) as well as in "normalized" format (VALUE). The raw format is sometimes more human readable (like the temperature in Celsius or the reallocated sector count), but not always. They can also vary wildly from vendor to vendor. The normalized format shows the current value as a normalized value between 255 and 0 (higher is better). If the value falls below the "THRESH" value, it means that the drive is failing. The WORST normalized value is also shown.

When reviewing SMART attributes, see this helpful chart of Known S.M.A.R.T. attributes.
Running a SMART test

Smartctl provides drive tests that you can run. Smartctl does not actually conduct the tests, it just tells the drive to initiate a test on itself. You need to run the SMART report command, shown above, to get the results. If the test is still in progress, the SMART report will tell you that as well.

This short test takes 1 to 3 minutes (remember to substitute your drive's identifier for 'sda' as described above)

smartctl -d ata -tshort /dev/sda
This long test takes about 2 to 4 hours

smartctl -d ata -tlong /dev/sda
To see the results, or the progress of the test, use the SMART report command, shown above in the Obtaining a SMART report section.


How do I re-enable the drive?

Okay, the cable was loose, or I think the failure was a fluke - how can I get unRAID to reuse this same disk that it thought had failed?

If you are sure that the drive is fine, and the SMART report confirms it, and you have not written to the drive since it was taken off-line, then use the Trust My Array procedure, to quickly recover your drive and the array to an all green condition. Remember, it was taken out of service when a "write" to it failed so using the "trust" procedure will effectively forget the data written while the drive was disabled. Unless you are certain you have not written to the disk a reconstruction is much better. The safest option is to reconstruct to drive. Only use the Trust procedure if a reconstruction is not possible.

You can re-enable the hard drive and reconstruct it as follows:

    Stop the array.
    Go to the Devices page and un-assign the disk.
    Go to the main page and start the array.
    Stop the array again.
    Go to the Devices page and re-assign the disk.
    Go to the Main page - system should indicate there is a "new" drive to replace the disabled one. Check the confirmation box and click Start to start a parity-reconstruct of the disk.

What to do if a hard drive fails?

If a hard drive fails and you need to replace it, refer to the unRAID documentation or this wiki page to replace the drive. This is a tried and true process that many many users have used. It is the same process used to replace a disk with a larger capacity disk.

unRAID has been perfectly stable for everything I've done with it. Never had any issues.

so what use is this unRaid server then?


I'm guessing you set it up, then create cifs or nfs shares on it for storing your stuff?

do you then map drives to the shares so that various devices in the house can read/write?

one thing i tried was dlna against FreeNAS. i could not get it to show up in the ps3 and this is the way i wanted it to work if im honest.
alternatively, i'd be happy with streaming to the pi from plex, but not sure if i'll ever get that working either.

would i be better off looking at unRaid or do i just need to build a HTPC using ms software?

It uses reisferfs file system but supports SMB, etc so can be read across the network easily. You either map individual drives in the unRAID array as shares (disk1, disk2, etc) or you can do shares by folder, so if you have a movies folder on each drive the contents of each one will be read as one. I do the first option as I have my own map of where everything is, I prefer that so I'm only spinning up drives as necessary.

There's a ton of guides on configuring unRAID to be used for media server purposes on the official forums. I just use it as a fileserver and let ps3mediaserver handle the media aspect.

No, but I will do later on now!

We use Docker at work for deployment, it can be a real bitch to get configured working the way you expect. What were the kind of problems you were having?