In Search of iSCSI with OpenFiler
Submitted by Scott Dowdle on Wed, 2007-04-04 12:03.
iSCSI | OpenFiler | RHEL | XenServer
OpenFiler iSCSII installed OpenFiler on an extra machine the other day. What's OpenFiler? According to the OpenFiler website:
Openfiler is a Storage Management Operating System. It is powered by the Linux 2.6 kernel and Open Source applications such as Apache, Samba, LVM2, ext3, Linux NFS and iSCSI Enterprise Target. Openfiler combines these ubiquitous technologies into a small, easy to manage solution fronted by a powerful web-based management interface. Openfiler allows you to build a Network Attached Storage (NAS) and/or Storage Area Network (SAN) appliance, using industry-standard hardware, in less than 10 minutes of installation time.
I've never worked with iSCSI before... but now I want to. The reason I'm looking into it is because RHEL 5 and others can use iSCSI disks to install to... and hopefully it'll work well for XenVMs too. Care to follow me on this, the initial leg, of my journey?
What is iSCSI?
According to the iSCSI Wikipedia page:
Internet SCSI (iSCSI) is a network protocol standard, officially ratified on 2003-02-11 by the Internet Engineering Task Force, that allows the use of the SCSI protocol over TCP/IP networks. iSCSI is a transport layer protocol in the SCSI-3 specifications framework. Other protocols in the transport layer include SCSI Parallel Interface and Fibre Channel.
If you are wondering how iSCSI performs compared... to say... Fiber Channel... D. John Shakshober wrote a white paper for Red Hat Magazine entitled, How does iSCSI perform relative to Fiber Channel?. Aren't you lucky? My paraphrase... iSCSI is really fast and comparable to (and sometimes faster than) Fiber Channel... but it really depends on the implimentation.
Installing OpenFiler
Following the download link on the OpenFiler website, one sees that it is offered in three flavors:
openfiler-2.2-x86-disc1.isoopenfiler-2.2-x86.img.gzopenfiler-2.2-x86.vmware.zip
The first is a traditional CD image file that you burn to CD. Oddly enough, while disc1 is part of the name, there are no other CDs. The .img file is a hard disk image for use with Xen although XenSource's product line does not support .img files (only LVM partitions). The .vmware one is a disk image for use with VMware. I downloaded the .iso which is approximately 323MBs in size and burned it to disk. A distro that size is realtively quick to install.
I booted the CD and the install process was pretty darn quick... the GUI install interface appeared to be Red Hat based... although OpenFiler is based on rPath. The install only asked the regular questions with the most importing be related to network settings, partitioning the drive, and setting a root password. The total install time was about 5 minutes. For my testing, I was only using the machine's internal 80GB hard disk so I made two small partitions... one for swap space and one for /... and I left the rest as freespace to be managed by OpenFiler. When the install was done, it spit out the CD as expected and asked me to reboot the machine.
Upon startup a login screen is presented that basically says... this is OpenFiler... the IP address of this machine is blah and you access it with your webrowser via https://ipaddress:446.
OpenFiler's Web Interface
I went to my workstation, opened up a browser and surfed to the given address which presented a license page. After accepting the license it presented a login page with a default username and password set. The first thing you do is change the admin password.
The web interface is fairly straight forward and I'm not going to take the time to go into much detail about all of the various options. Suffice it to say that there is a section for authentication - OpenFiler can authenticate users against OpenLDAP, Windows NT or Active Directory, and NIS. There is a section to define the network which lets you state what machines have access to OpenFiler. There is a section for creating disk volumes. There is a section for creating disk shares. Once a volume or share has been created/defined, you can specify what machines have access.
The OpenFiler web interface is a bit clunky but simple enough once you get into the mindset of storage management. Give it enough time and it starts to make sense. They do have a Administration Guide in PDF which is somewhat helpful but a little out dated.
Creating an iSCSI Target
Since my main goal was to work with iSCSI, I created a 20GB iSCSI volume and then I set what machines could access it... which is done in the Network section first and then in the properties for the iSCSI volume. Defining an individual machine is done by inputing a descriptive machine name, its IP address and then a network mask of 255.255.255.255 in the network section followed up with allow/deny in the individual iSCSI volume preferences. Unlike NFS, FTP and others, iSCSI is designed to be a block disk device and should be accessed by only one machine at a time.
Finding an iSCSI Initiator (client)
Ok, the iSCSI volume is created, access to it set... but now what will I use to access it with?
I plan to buy more RHEL subscriptions / RHN entitlements this summer to go with some upcoming hardware purchases, but I'm not ready to buy today just to try out iSCSI... so I'm waiting for CentOS 5 to come out so I can try an iSCSI based install. I believe CentOS 5 will be released in the next few days... but since I don't have it now, how about XenExpress?
I installed the newly released XenExpress 3.2 yesterday and supposedly it supports iSCSI but after installing it, I'm not sure how to access the OpenFiler iSCSI volume I've created. The XenSource Users Guide does talk about using iSCSI but the only example they give is a single model of iSCSI server from a single vendor. I created a discussion thread on the XenSource Storage forum. XenSource employees my forum posting almost immediately and said that they are actually actively investigating OpenFiler for use with XenSources products (XenExpress, XenServer, and XenEnterprise)... and would get back to me. They also stated that they are working with rPath and the OpenFiler maintainer to produce a XenServer importable XVA release of OpenFiler so it can easily be installed as into a XenVM... since XenSource does not support the previously mentioned .img disk type.
Update: The XenSource folks have responded back and say that OpenFiler works fine as an iSCSI target so I'll have to figure out how to use it, eh?
After reading the Wikipedia iSCSI page I discovered that Microsoft has a free iSCSI initiator driver/app for Windows... and since I recently used VMware's Converter to migrate my physical Windows workstation into a VMware virtual machine on my Linux workstation... I decided to download Microsoft's iSCSI initiator to give it a try.
That seems to have gone really well so far... even inside a VMware virtual machine. I told OpenFiler to give access to the 20GB iSCSI volume to said machine and then I ran the Windows iSCSI initiator and told it to query the OpenFiler machine... and bingo bango... it found the volume.
Windows OpenFiler iSCSI access
I had to format it as NTFS so Windows would know what to do with it... which took about 10 minutes or so... and now it seems to be just a regular drive. Haven't really done much with it... but at the very least, I'm glad I have been able to verify the functionality of OpenFiler's iSCSI implimentation in this rudimentary way... a proof of concept if you will. As soon as I have something else that can access it, that iSCSI volume is going to become ext3 or LVM (if possible). :)
Conclusions
OpenFiler was easy to install although it did take some thought and poking around to understand and use the web interface... but so far as turning any old PC into a storage appliance it works very well in my intial testing. I'll definitely be doing more testing this summer to go with some new hardware purchases... and if all goes well... I hope to use iSCSI for production OS installs including storage for XenVMs.
I'm sure OpenFiler would also be useful for anyone wanting to setup a more traditional file server with NFS or Samba... but that wasn't my goal.
» Scott Dowdle's blog | login to post comments
Thanks for the update
Submitted by Scott Dowdle on Sun, 2009-12-06 08:00.
Greg,
Thanks for the update. As you may have noticed my article about OpenFiler is rather dated.
I'm glad to see you have come up with "OpenFiler, the Missing Manual". I recommend it to anyone getting started with OpenFiler.
» login to post comments
one machine at a time
Submitted by gbjbaanb (not verified) on Fri, 2008-08-01 06:45.
"Unlike NFS, FTP and others, iSCSI is designed to be a block disk device and should be accessed by only one machine at a time."
That's partly true. iSCSI is a SAN technology that gives out disc blocks to clients instead of files, as such 2 machines using the same area can interfere with each other... but that doesn't mean 2 clients cannot use the same iSCSI target, only that you need to format it using a filesystem that supports multiple access. Examples are: VMWare's VMFS, and Redhat's GFS (global file system).
Both those are designed to work on block-device SANs, like iSCSI so you can safely share your iSCSI amongst many machines.
» login to post comments
Wow, what a sales pitch.
Submitted by Anonymous (not verified) on Fri, 2007-12-07 20:50.
Wow, what a sales pitch.
» login to post comments
iSCSI - HBA & TOE
Submitted by Mark (not verified) on Wed, 2007-05-23 17:05.
There are iSCSI HBAs (Host Bus Adapters) available that have TOEs (TCP Offload Engines) on board which take load off the CPU for the iSCSI and TCP/IP protocols. I know QLOGIC makes three 64-bit PCI-X 133 RoHS 6 cards anyway. These would rival any software initiator in many ways, but you'll pay for it. The high-end model retails for almost $1100 US. The mid-range card is about $750 US.
It might surprise you, but the right iSCSI configuration can even out-perform Fibre Channel.
» login to post comments
Bonding 2 gigabit NICs
Submitted by Ken Dyke on Tue, 2007-04-10 18:17.
I suspect that many real world installations of iSCSI use only a single NIC. 2 each gigabit NICs are pretty standard on current server class hardware. Thus a shop would have to go the extra mile to add another NIC and to have the in-house knowledge that it was even possible to bond two NICs together in the first place.
As it happens there was a story last week about Fiber Channel over Ethernet (FCoE). The vendors are looking to have this standardized on 10-Gigabit ethernet in four or five years. Current cost of 10-Gigabit is between $3k-$5k per port.
http://www.crn.com/storage/198800658
A much lower cost solution is available to Linux users in the form of ATA over Ethernet (AoE). The only vendor I am aware of is Coraid. Currently they support SATA over Gigabit. For higher bandwidth needs I would guess that some sort of RAID stripping with multiple NICs could be cooked up. (Maybe I could get them to swing me a demo unit to play with?)(Maybe it is something they already support?)
In light of the recent studies regarding duty life of SCSI and ATA drives that reveal that SCSI is no better... I will be looking real hard at the Coraid products as something I to recommend to clients that have more than trivial storage needs.
» login to post comments
Openfiler is now on version 2.3
It works great as an iSCSI storage repository for virtualization, like XenServer or ESX. Here's my load and usage notes for this.
----------
Greg Porter
Have Linux, will travel.