Thursday, April 4, 2013

Setting up MariaDB Galera Cluster from Scratch

Okay, this one is a little fun for the techie crowd out there. Nobody can say this blog isn't eclectic at least.

MariaDB is the new database program from the folks who created MySQL, arguably the most popular database on the planet. It was created when Oracle gained control of MySQL because, well, Oracle. Galera is a multi-master "live" database that sits on top of MariaDB. The instructions on the web for installing Galera that I've been able to find are woefully inadequate, and presume a certain level of "already installed and configured".

This instruction set is designed to walk through the install procedure from start to end. Text written in courier is meant to be typed in the command line.

You're going to want to set up three servers to get this going (this is where VMware is very handy).

1. Install CentOS. I used this image:

http://mirror.its.sfu.ca/mirror/CentOS/6/isos/x86_64/CentOS-6.4-x86_64-netinstall.iso

But you should go to www.centos.org and pick the proper mirror.
During installation, since this is a net install, use an appropriate location for the url:
http://mirror.its.sfu.ca/mirror/CentOS/6/os/x86_64/

When prompted for the type of installation, select "Basic Server". Do NOT select "Database Server" or anything else that might install the MySQL database...that will cause problems. Trust me on this.

2. Create repo files for MariaDB
(See: https://downloads.mariadb.org/mariadb/repositories/)

As root: nano /etc/yum.repos.d/MariaDB.repo
[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/5.5/centos6-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1

rpm --import https://yum.mariadb.org/RPM-GPG-KEY-MariaDB

3. Create repo files for Percona
nano /etc/yum.repos.d/Percona.repo

[percona]
name = CentOS $releasever - Percona
baseurl=http://repo.percona.com/centos/$releasever/os/$basearch/
enabled = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-percona
gpgcheck = 1


wget -O /etc/pki/rpm-gpg/RPM-GPG-KEY-percona http://www.percona.com/downloads/RPM-GPG-KEY-percona

4. Install Packages
yum install MariaDB-Galera-server galera MariaDB-client nc xtrabackup 

5. Create my.cnf
Below are the defaults that are required to successfully start the cluster.
(See: http://www.codership.com/wiki/doku.php?id=mysql_galera_configuration)
nano /etc/my.cnf

[mysqld]
query_cache_size=0
binlog_format=ROW
default_storage_engine=innodb
innodb_autoinc_lock_mode=2
# this has to be specified for xtrabackup to work
datadir=/var/lib/mysql
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_provider_options="gcache.size=32G; gcache.page_siz=1G"
# Address of first server started in cluster;
# comment this line out in my.cnf of first server
wsrep_cluster_address=gcomm://10.0.1.190
# This is just an informational string, just make sure
# it matches on all clustered machines
wsrep_cluster_name='gamaria1'
# This is the address of the local machine
wsrep_node_address='10.0.1.192'
# This is just an informational string
# to describe the local machine
wsrep_node_name='node3'
wsrep_sst_method=xtrabackup
wsrep_sst_auth=<user>:<password>

Replace <user> and <password> with the appropriate username and password. When configuring the first node, be sure to set up the appropriate user profile and access permissions. After you've set up the first node, the other nodes get the security information automatically. How cool is that? (Be warned: this bit grants access to the database from any machine on the network...replace % with the IP address of an appropriate machine if you don't want that!)
Sign into the local MariaDB:
mysql -u rootCREATE USER 'monty'@'localhost' IDENTIFIED BY 'some_pass';
GRANT ALL PRIVILEGES ON *.* TO 'monty'@'%' WITH GRANT OPTION;

6. Disable the Firewall
Not the best solution, but the cluster seems to use more than just port 4567. Haven't quite worked this one out yet. Suggestions?
service iptables stop chkconfig iptables off
7. Edit script on all nodes to prevent cluster from erroring out on startup

(See http://www.perconaforum.com/index.php?t=msg&goto=10012&)

nano /usr/bin/wsrep_sst_commonReplace
echo "WSREP_SST: $* ($tst)" >>/dev/stderrWith
echo "WSREP_SST: $* ($tst)" >&2
8. Start the Server
(See: https://kb.askmonty.org/en/getting-started-with-mariadb-galera-cluster/)

For all nodes, start the server as the mysql user: su mysql - 
Don't forget the - at the end! That forces the proper environment. 
First server: mysqld --wsrep_cluster_address=gcomm://
As long as my.cnf is correct (wsrep_cluster_address variable),
you can start other machines with: mysqld


A couple of notes:


Set up the first server following these instructions, get it working, then set up each server in turn, following 1 through 8 again, but making the appropriate mods for each.

Start the first server, let it get to "waiting for connections", then start the other servers.
If you shut down the first server for any reason, when you start it up you must use this:
mysqld --wsrep_cluster_address=gcomm://address_of_another_server

Or you're pooched and will have to shut them all down and start the boot sequence from scratch. Yeah, I made that mistake a few times.

That's it! Kind of. I may have glossed over a couple of points. As I discover problems with these instructions I'll update the post. 

Last, but certainly not least, I can't take credit for these instructions beyond typing them here. Credit goes to John Flaskay, my resident expert-in-everything-linux/security/tcp/etc!

Friday, February 22, 2013

Using a Raspberry Pi as Digital Signage

I developed a digital signage application for my employer some time ago, but we ended up using Mac Mini's at the time. With the advent of the Raspberry Pi computer I thought it might make a good candidate for DS, but I had a heck of a time when it first came out getting any kind of movie playlist working.

I tried VLC, some other media players, trying to find a way to make the built in omxplayer play a playlist, nothing. Nada. Nothing worked. I could not figure out how to make an unattended digital signage device.

So I gave up. For a couple months.

When I decided to revisit the issue, a little research showed that nobody had really solved the playlist issue, so I thought I'd try to find another way around it. And I did.

It was always easy enough to just have a script that plays videos in sequence using omxplayer, however this solution wasn't quite perfect because the stream would have a 1 second pause that would show the desktop while it switched videos. The solution turned out to be simply turning on the screen saver. May sound easy, but it took me a while to figure it out!

So other than a slight pause between videos, this solution works perfectly. Here's the steps to get it going:

WARNING! If you do this, the screen saver will kick in every 1 second of idle time, even if you move the mouse. Make sure you have SSH enabled so you can remote into the system to disable this later if you need to!

1. Set up Raspbian on your Pi device according to their instructions.
2. Make sure you're signed on as pi, not root, then from a command line enter:
sudo apt-get install x11-xserver-utils
mkdir /home/pi/digitalsignage
mkdir /home/pi/digitalsignage/movies
nano /home/pi/digitalsignage/piScreensaver.py

Here I'm using a python script to use the X11 xset command to turn the screen saver on:

import os,sys,time

def SetScreensaver():
    time.sleep(15) # wait for the system to finish booting
    for x in range(0,5): 

        # try turning it on once every minute for the next 5 
        # minutes just in case!
        os.system('xset -display :0.0 s 1 60 -dpms')

        # The -dpms turns the power save option off
        time.sleep(60)
    return
               
if __name__ == '__main__':
    SetScreensaver()
    sys.exit()

Press CTRL-O [ENTER] and CTRL-X to save and exit.

Then we create the player module:
nano /home/pi/digitalsignage/piPlayer.py

Enter this text:
import os,sys,time

def RunPlaylist():
    while 1:
        path    =    '/home/pi/digitalsignage/current.m3u' 

        file    =    open(path,'r')
        files   =    file.readlines()
        file.close()
        os.chdir('/home/pi/digitalsignage/movies')
        for file in files:
            filename = file.strip()
            if '.mp4' in filename:
                os.system('/usr/bin/omxplayer /home/pi/digitalsignage/movies/%s>/dev/null' % filename)
    return 

              
if __name__ == '__main__':
    RunPlaylist()
    sys.exit()


Press CTRL-O [Enter] then CTRL-X to save and exit again.

Next you have to copy your MP4 video files to the /home/pi/digitalsignage/movies folder.
Once done, create the playlist. Let's say for this example we have "video1.mp4" and "weather.mp4", and you've uploaded them to the /home/pi/digitalsignage/movies folder. Type:

nano /home/pi/digitalsignage/current.m3u

Enter the following:
video1.mp4
weather.mp4

Press CTRL-O [Enter] then CTRL-X to save and exit.

Finally, modify the startup:

crontab -e

And add these two lines to the bottom:

@reboot /usr/bin/python /home/pi/digitalsignage/piScreensaver.py
@reboot /usr/bin/python /home/pi/digitalsignage/piPlayer.py


Press CTRL-O [Enter] then CTRL-X to save and exit.

Then reboot:

sudo shutdown -r now

That's it! When your Pi reboots, it will come up with a full video playlist player and start automatically.

Now, I've actually wrapped some other functionality around it for my employer, such as having a web-based management program that will load new videos to the Pi then restart the playlist, that sort of thing, but if you're looking to set up the playlist functionality this should hopefully help!