MontanaLinux

The topic of the BozemanLUG meeting tonight is the Let's Encrypt project. Today I got an SSL certificate for this site from Let's Encrypt and I would like to encourage everyone to use the https version of this site. Your browser should love the new certificate and automatically accept it.

Ghosts of SSL Certs Past
I have used a free start.com SSL cert but it expired. Since then I've used a self-signed one as everything I do on the site is over https since I want to avoid getting my admin account credentials sniffed out over plain-text... but asking the rest of the world to accept my self-signed SSL cert? That's just too much.

Now that I have a good cert, I might force https at some point but not yet.

Future Developments
This site is ancient. The software running this site is ancient... and the distro it is running on is ancient. I hope to change that sometime after the new year. The limiting factor was a lack of public IP addresses... as I wanted to create a new OpenVZ container with newer stuff on a different IP address. In January we should have 5 additional IP addresses and I can at least get started on a new site while leaving this one running until the new one is ready. I'm not even going to guess when I'll be done enough with the new site so this old one can go away, but hopefully sometime in 2016. :)

Dirty MotherboardI doubt many people actually noticed but the server that runs this site (and about a dozen more) and my email... died sometime between 4:30 and 6:40 this morning. I texted (using Google Voice since I do NOT own/want a cell phone) Warren (since the server is in his house) about 6:38 and he responding back at 7:10 saying the server was dead. It would power on for about a second and then immediately turn off. It was full of dust bunnies so Warren cleaned it out and re-seated everything that might have had a loose connection... but still power-on failure. Warren opened up the power supply and cleaned it out. Inspecting both the motherboard and the power supply, there were no obviously bad components... no bad caps. Still no results.

Without a quick fix we decided to try to get a temporary system going... perhaps using a desktop computer with 4 SATA ports. 4 SATA ports are needed because the dead server has 4 hard drives that are in a Linux software RAID 5 configuration. Luckily Warren had access to a spare Dell OptiPlex 960 Core 2 Duo system with 8GB of RAM and 4 SATA ports. Many tower systems that have 2 hard drive bays come with 4 SATA ports so they can support 2 HDs and 2 optical drives. Getting the cables to go where they needed to go (both data and power) for loose drives seemed to work and for the time being the desktop power supply seems to be beefy enough to run all of the drives. Seeing as we run CentOS 6.x for OpenVZ... I was concerned that the network chipset would be too new... and for a little while that seemed to be the case... but Warren booted from a CentOS 6.7 LiveDVD and the network worked fine... so we knew it was a configuration issue from the previous hardware configuration. Turns out the NIC was detected as eth2 rather than eth0. While that could probably have been resolved by nuking a udev rule file somewhere, Warren just moved and edited the eth0 config file so it was eth2 and we were up and running again.

Dirty Power SupplyWhile I do have backups (a couple of days old) and have previously researched a few cloud services (6sync and fastmail are on my radar) just in case, it is great to be back up after a few hours rather than having to worry about transferring a few hundred gigabytes of data before we are back up.

For the mid to long-term we haven't thought about where to go... and are just happy to be up and running again. Warren had it back up about 2 PM so he put a good 7 hours in today. Turns out Warren starts the Montanan Marathon early tomorrow morning, which he has been training all year for, so he wanted to get some rest today. I hope he does get some rest after saving our hobby server. Thanks Warren! I hope you do well in the run tomorrow buddy.

Here is a preview video of the upcoming Fedora 22 release (running in a KVM VM). This is my personal remix (with non-Fedora provided rpmfusion-free packages, google-chrome-stable, and flash-plugin added) and I haven't bothered with the branding nor customization at all... and I don't really publicly distribute it.. but I'd be happy to share my kickstart file if anyone wants it.

In the video I show the install process, and then show all of the desktop environments that are pre-installed which include... GNOME 3.16.1, Plasma 5.3.0, LXQT 0.10, MATE 1.10, XFCE 4.12.1, and Cinnamon 2.4.8. Enjoy!

Please Note: The official Fedora live media always automatically logs the liveuser in... but on my personal remix I haven't bothered to set that up so I have to put in "liveuser". Again, the Fedora media is NOT like that.

Fedora 21 pre-beta LightDMI've been following the development of Fedora 21 since a little before the alpha release. Getting my MontanaLinux remix to build was actually quite easy and the fact that rpmfusion has a rawhide repo means all of the multimedia codecs / applications were good to go as well. I've done few dozen installs as KVM virtual machines and thought it was time to try physical hardware.

Hardware Problems?

First I installed it on my Acer netbook that is 32-bit only and about 5 years old now. The battery in it is shot and smartd has been telling me for over a year that the hard drive has been getting more and more bad sectors... which is a fairly good indicator that the hard drive is going bad. Doing the install from a LiveUSB it took a while because the installer was finding some of the bad spots on the drive. For whatever reason during the install the progress bar immediately said 100% and I knew that was wrong... so I kept switching over to a text console to periodically do a df -h to see how much had been written to the hard drive. Oddly whenever I'd switch over to the text console, the green illuminated power button would go amber and the screen would go blank... which to me meant it was suspending to RAM or something. At that point I'd have to hit a few keys on the keyboard and it would wake back up. For whatever reason it did this at least a dozen times during the install. I really wasn't expecting a good install given the flaws in my hardware and how they were manifesting themselves during the install process... but being patient paid off... and it actually was successful... and seems be working just fine post-install.

Installing it on my Optiplex 9010 desktop at work was also more complicated than I was expecting. For whatever reason (maybe a BIOS setting?) I could NOT get my machine too display the bootloader menu from a LiveUSB although other Dell models at work seemed to work fine. So I burned a DVD with the burner in the Optiplex 9010. Oddly the same drive that wrote the DVD seems unable to read it about 19 out of 20 tries. That meant that I couldn't get it to boot from the DVD either. I finally decided to try something different... and I got an external / USB optical drive and plugged it into the USB port and I was able to get it to successfully read the DVD and the bootloader to appear. With a functioning bootloader I was able to boot the DVD and the live system worked great... and the installer went flawlessly.

Fedora 21 pre-beta actually seems quite stable. As you may recall I have all of the desktop environments installed as part of my remix so I can check them all out... but I primarily use KDE. On both of my machines I have /home as a separate partition so my personal data is retained across installs. I also backup /etc and /root to /home/backups/ so any of my previous configurations (stuff like ssh keys) can be retrieved and used if desired.

Some Notes

I picked lightdm as the default login manager. In the past I've mainly used kdm but KDE is in the process of transitioning to sddm which seems a bit buggy still.

One of the main features in Fedora 21 I'm wanting to play with actually is provided by the rpmfusion repos... ffmpeg 2.3.3. I'm wanting to do some testing with the newer ffmpeg that does a reasonable job at webm encoding with vp9 and opus. I'd also like to try out GNOME 3 under the Wayland display server... which is supposedly working fairly well in Fedora 21... but I haven't tried it yet.

One weird glitch I ran into was with the Google-provided google-chrome-stable package. I'm not much of a Google Chrome user but I do occasionally use it for (legacy) sites that require Adobe Flash. I use Firefox the vast majority of the time... but I've decided to no longer install the Adobe provided flash-plugin package (at version 11.x). As you probably know Google has taken over maintenance of newer Flash versions (currently 15.x) on Linux and include it as part of Google Chrome. As a result, whenever there is a Flash update from Adobe, there is a Google Chrome update that soon follows. Anyway, very early in the Fedora 21 development cycle (pre-alpha), the Google Chrome package refused to install because Fedora 21 had a much newer version of some library (I don't recall which one) and it wanted the older version. A few Google Chrome package updates later... and it is happy with regards to dependencies... but installing it with rpm... it gets stuck on the post-install and just sits there. I had to ^c rpm (which you generally don't want to do) because it wasn't going to finish... and just to be safe I did an rpm --rebuilddb and everything seems fine. The google-chrome-stable package verifies just fine (rpm -V google-chrome-stable) and the package works as expected.

Conclusion

Overall everything I've tried works fine. I like to get started with new Fedora releases as early as possible in the development cycle so I can help report any bugs I find (in Fedora provided packages) and be up-to-speed with all of the new features on release day so I can deploy to other machines immediately. I've been doing it that way for several releases now. I do really appreciate all of the work the Fedora developers put into each release.

CentOS 7 Public QA - KDE Desktop with FirefoxIf you didn't hear the news, Red Hat released Red Hat Enterprise Linux 7 on Tuesday, June 10th. I've done three installs so far at work... and have been reading through their wonderful documentation. I'm really digging the newer versions of things and systemd... yes, especially systemd. No, no, really!

As you also probably know, Red Hat sponsors the CentOS Project now... and they are working hard on getting CentOS 7 done. Andrew from the BillingsLUG predicts CentOS 7 will be out within two weeks of RHEL 7... so that would be by June 24th. My guess is 6 weeks... which would be by July 22nd. If they don't make it in 6 weeks, my next guess is August 10th, because that's my 50th birthday.

Anyway. So yeah, the CentOS Project has been hard AND they have been, unlike in the past, doing everything out in the open... transparency it is called. Yesterday they announced they had the packages building. Then someone on the centos-devel mailing list said they had a Docker CentOS 7 container image. I gave that a try. Then the centos-devs said they had the first build attempt completed although they have NOT gone through all of the packages yet and removed Red Hat's branding... so it's a very preliminary build. Then they announced they had a network install CD (~ 341MB). I gave that a try and it worked great.

Then I decided I wanted to work on my own remix if possible. I used reposync to download all of the packages... and wget to get the handful of other dirs/files in the install tree. Then I made a KVM virtual machine via a network install pointed at my own copy of the tree. Then I added the livecd-creator package that one CentOS developer ported from Fedora. Then I installed fedora-kickstarts from Fedora 19... and hacked on their KDE LiveCD kickstart until I had it building CentOS 7. The first build didn't go so well. For whatever reason, all of the GUI stuff was there except for Xorg. I was able to use that first install, get it going in text-only mode to figure out what packages I needed to add to my kickstart's package list to get X going. Bingo... only three additional lines although two of them had an asterisk in them.

It built. It booted. It installed. It booted and worked post-install. Not bad.

What does it contain? Well, I'm a KDE fan. EL7 only offers GNOME 3 and KDE anyway. So, it has KDE... but oddly they don't offer KDM (KDE Display Manager aka GUI login screen) so it uses GDM (GNOME Display Manager). While Red Hat defaults to the XFS filesystem in their install media (they don't have any Live media by the way, just install-only) livecd-creator would not build the .iso if I set the default to xfs... so I had to set it to ext4. So, the system you get from the live installer has ext4 partitions. While it is the KDE desktop I added some stuff that isn't KDE-specific... like Firefox, Libre Office, GIMP, and Inkscape. I didn't refer to the EPEL 7 repository in my kickstart so the first good build only includes stock packages. Later I'll probably add in EPEL and add some additional packages like tmux, x2goserver... and a few other sundry packages. Any suggestions?

I should have included some screenshots with this post but I'm too lazy and tired after spending about 6 hours working on this little project today. If you want to give it a try let me know and I can email you the URL to the .iso file. Oh, btw... the installed system does not include a working centos.repo file so after a fresh install is booted, one has to manually add one by creating a file named /etc/yum.repos.d/qa-nightly.repo. Put in it the following:

[qa-nightly]
name=qa-nightly
baseurl=http://buildlogs.centos.org/centos/7/os/x86_64-latest
enabled=1
gpgcheck=0

Then you can use yum to install anything else you'd like. I recommend you also add EPEL 7 (epel-release-7-0.1.noarch.rpm). Enjoy! MEL (Montana Enterprise Linux), kiss my grits!

Update: CentOS has since released Public QA LiveMedia of their own.

I keep up with Fedora releases. Fedora 19 was due for an "Alpha" release last Tuesday but they had to delay. As most everyone already knows, delays in Fedora are to be expected. Anyway, I thought I'd check out their Alpha test builds. They actually seem to be working quite well. I did several installs from the "Desktop" media which is GNOME 3.8.x-based. The installs I did were inside of KVM-based virtual machines. Then I added on all of the other desktop environments and tried them out. Even in this early stage, it seems to be quite usable and stable. I obviously did NOT run into any of the "blocker" bugs that were the cause of the Alpha release delay. I think most of those were EFI related.

Remixing from Alpha

I thought I'd try doing a MontanaLinux remix from the development repositories. For those not familiar with MontanaLinux, it is basically the vast majority of desktop environments and desktop managers and a lot of desktop software rolled into a 2+ GB live .iso. It includes packages from rpmfusion (codecs, gnome-mplayer, vlc, etc), Google (google-chrome-stable and google-talkplugin) and Adobe (flash-plugin).

So from my Fedora 19 pre-alpha VM, I installed the various kickstart packages, extracted out the KDE related kickstart (.ks) files, and then melded them into a single file, added the packages I wanted to the %packages section, and then did a tiny bit of customization in the %post and %post --nochroot sections of the kickstart. With a MontanaLinux-F19.ks file done, I proceeded with the build.

It built. I discovered (with help from nirik in #fedora-devel on freenode) that Fedora no longer looks at /etc/sysconfig/desktop for the default desktop environment and display manager. That is done with systemd's systemctl.

The Bug Fairy Always Visits

There are a few glitches here and there but that is to be expected. lightdm was messing me up... so I had to add an exclude in the %packages section. I'm still in the process of refining the kickstart but it seems to work well enough.

For some reason, when I boot the .iso in a KVM VM on a Fedora 18 host I can't use the combination of SPICE/QXL. If I change it to VNC/any, SPICE/VGA, or SPICE/VMVGA it works fine.

I made sure to add in the GNOME 3.8.x Classic extensions so the GNOME Classic mode shows up in the display manager Session options.

I've only been working on this for a few hours so I'm sure I've got a bit of learning left to do. Fedora has since released a number of Fedora 19 updates and I haven't tried those yet. More later.

Update: April 22 - There was an additional flood of updates over the weekend. I guess the current build from Fedora went gold for the Alpha release and they had stockpiled some updates until after. I put in a few more hours on my configs and got the 32 and 64-bit versions built. They are working well and the SPICE/QXL combination now works fine. Updates included KDE 4.10.2 and MATE 1.60 among others.

If anyone wants to try it out, feel free to email me (dowdle@montanalinux.org) and I'll reply with a URL.

Update: April 24 - Fedora DID release Fedora 19 Alpha yesterday. I've done yet another rebuild and created a screencast video (no audio) showing the Live DVD iso booting inside of a KVM Virtual Machine connected to with the SPICE remoting protocol. I didn't do any fancy editing of the video so there are long boring parts where you stare at a blank screen as it boots or loads. In the 25 minute video I boot, do an install, reboot and then do a quick survey of all the desktop environments, some apps as well as how I like to personalize KDE. The pre-release Fedora 19 base has a debug kernel and I'm sure the installer is doing a lot of extra logging... so the installer and the boot is a lot slower than the final product will be but that is to be expected. Everything seems to be working nicely except for Cinnamon.

Anyone who would like to watch it can do so with the link below. Right-click to download or play in your webm compatible browser. It is about 66MB in size... which is about 3 times the size of my first hard drive back in 1986. :)

montanalinux-f19-alpha-remix-20130424-install-firstboot.webm

Fedora 18 Spherical CowI've been remixing Fedora 18 pre-release for quite a while now. As you may recall The Fedora Project has delayed the release of Fedora 18 Beta several times now... mainly due to blocker bugs in their new installer and Fedora Updater (fedup). I think the rest of the distribution has benefited from the delays because I've been running it a while and it has been very solid for me... as or more solid than Fedora 17. In fact, Fedora 17 and Fedora 18 share a lot in common... because a Fedora release, during its lifecycle, gets a lot of updates and upgrades.

I started by putting Fedora 18 on my netbook. Then I put it on my home desktop system. I ran it for more than a month... oh, and by the way, I disable the updates-testing repository. Since it has been so solid on my hardware at home I finally decided, perhaps being a little haphazard, to put it on my workstation at work. When did I decide to do that? Well... I picked the day before Thanksgiving about 1 hour before it was time to go home. Care to follow me on my journey?

Update: (8PM, Thanksgiving) - I noticed an email that said that there was a Fedora 18 Beta release Go/No Go meeting today and that the decision had been made to finally release the beta on Tuesday, Nov. 27th... the so called "exploding turkey" release. Yeah!

If you have been paying any attention to the development work going on with Fedora 18, you're probably aware that they have been running into repeated delays because of a complete rewrite of the anaconda installer. I've been working on remixing Fedora 18 and generally it is in fantastic shape with the exception of a few pieces of the installer that I'll not mention now. Below is a video of me booting the latest build, installing it, doing a firstboot, and then showing off some of the new desktops.

I do the install on top of an existing KVM virtual machine so that's why I nuke the partitions that were already there. The desktops shown are Mate, GNOME 3, and Cinnamon. Also included but not shown are KDE, LXDE, XFCE, openbox and a few other window managers. The latest Firefox, Flash plugin-in, and Google Chrome are included along with several of the multimedia apps and codecs provided by rpmfusion.

There is no sound. I guess I could have put some Euro-synth-pop in there but nooooo....

Direct link, right-click save as:
montanalinux-f18-beta-boot-install-run.webm (25.7 MB)

Rackable NM46X Dual Quadcore AMDWarren Sanders put together a newer server to host this domain on. The system we were using was put together about 5 years ago. When we originally set it up, it was running CentOS 4.0. It was upgraded with each CentOS update and worked its way all the way up to CentOS 4.9. Anyone using RHEL 4.x or a clone can tell you that it is rather old in the tooth. For example it uses PHP 4.x.

A few weeks ago, I migrated all of our OpenVZ containers to the new system. The host node is running Scientific Linux 6.x. The containers continued to run CentOS 4.x Today I decided to make a new container and migrate all of the accounts and data to the new system. I basically went from CentOS 4.9 i386 to Scientific Linux 6.1 x86_64. The migration went pretty smoothly. My wife has a couple of Drupal 6-based sites and they just worked. I have a few Drupal 4.7.x based sites (which includes this one) and there were some issues. The main problem is that the Drupal 4 series is no longer supported / updated... and there are a number of known issues with Drupal 4.7.x and PHP 5.3. Going from PHP 4.x to 5.3.x is indeed a big jump. I also tried using the binary files for MySQL but ran into problems and ended up using mysqldump to export everything to an .sql file and then importing it. I don't know if the issues I ran into were caused by the big jump in MySQL versions or simply because I switched from 32-bit to 64-bit.

Any any event, with my testing and a little bit of php source code editing, this site is back in operation. It took me a couple of hours and there are still a few issues. What issues are those? Well, there will be no new account registrations and existing users can't edit their account information. Other than that, everything seems to work well enough.

I eventually plan on creating a new, Drupal 7-based site. I think trying to upgrade from the existing version through 5, then 6 and finally to 7... would be very problematic... unless there is someone out there who has done it. Problems with upgrading are what have held this site back at Drupal 4. Also being on CentOS 4.9, it was impossible to use Drupal 7 because it requires PHP 5.3.x at a minimum. I'm not really sure how I'm going to go about it... run both the old site and the new site... and manually copy and paste content between them? Or maybe I'll just run the old site in read-only type mode... and just use the new site for new content only. We'll see how it goes. I don't currently have a timeframe for when I'll set up the new site... so my guess is that this site will limp along for a while yet. Just wanted to let everyone know about the big change made today.

If you didn't notice, today is Fedora 16 release day. Yeah! I've been using Fedora 16 for a while now preping my MontanaLinux remix. I made a 41 minute screencast that does two things: 1) Showcases the desktop environments available in Fedora as melded together in the MontanaLinux LiveDVD remix, and 2) Shows using KVM and virt-manager some.

Please pardon my voice and occasionally sniffing... an allergy is bugging me.

fedora16-kvm-virt-manager-montanalinux.webm (85 MB)