In this project I am going to install and configure system monitoring software Graylog using salt. Beginning the project by setting up master-slave architecture and testing states first with different guest slave. When testing the states, master is guest os ubuntu and slave is guest os ubuntu.
First state is for mongodb.
After mongodb I started installing elasticsearch which was a tricky one. When I tried to edit the conf file of elasticsearch, it kept displaying an error that said file not found. I spent a lot of time trying to debug the problem, but I couldn’t find solution so I moved to install graylog instead.
After all above steps were done, it was time to test if the graylog was functioning. I navigated to webbrowser typed in the slaves ip address and port 9000 which is used by graylog: 192:168.1.33:9000
In this exercise I am going to install LAMP with a help of salt. My virtual ubuntu server at digital ocean will work as a master and give orders to guest os slave which is ubuntu server. Master: Digital ocean droplet / Ubuntu server 18.04 /1GB memory / 25GB Disk Slave: Guest OS Ubuntu server 18.04 on Windows 10 host PC
Beginning by cloning my git repository on guest ubuntu so I can use my shell script to make it a slave. After the script has run I accepted the key on my master. Now I have working master-slave architecture.
I wanted to install LAMP only by applying a highstate so everything would be fully automatized. To correctly do this without any bigger hassle on the ubuntu guest machine, I wanted to test every state before applying them so I manually tested every state first at another virtual machine. Everything worked just fine except mysql.
I left mysql installation for last on purpose, because I remembered it might be a tricky one. I remembered there were something annoying with mysql passwords at the installation, and wondered what would be the easiest way to install it. Fortunately I found article at terokarvinen.com which was very helpful: http://terokarvinen.com/2018/mysql-automatic-install-with-salt-preseed-database-root-password The thing was preseeding, answering questions on installation beforehand.
At first when I applied that state, it froze completely. I got no feedback from salt master whether the state were correctly applied or not. I checked with salt-run jobs.active and saw that the job was still running but nothing happened. I am not sure what was going on there, but I had to quit for that day so I shut everything down. Few days after that, I came back and changed the password on the sls file and tried to apply it again. It didn’t freeze this time, but I still got some errors.
In this exercise I’m making my windows host desktop as a slave, and creating a new master available to public internet. In my previous exercises my master has been a virtual machine which has no public ip address, which has been kinda boring, so I decided to create a virtual server at digital ocean. I made a virtual ubuntu server 18.04 which has a public IP address, 1GB RAM, 25GB SSD and is located at Frankfurt data center.
a) Since my virtual server is publicly available to internet, it was safe to enable firewall. I used commands: sudo ufw enable & sudo ufw allow OpenSSH. Latter command is important because I might want to acces my server with ssh again. The virtual server was very easy to set up, and I installed salt on it right away. I begun by installing both minion and master on the server to test if the master-slave architecture was working correctly, and it was. After that I tried to make my windows host machine a slave, but I couldn’t succeed at first. Master didn’t recognize any keys when typing sudo salt-key. Then I remembered I had enabled firewall, so I had to make some rules for salt. I typed following commands: sudo ufw enable 4505/tcp 4506/tcp. Salt uses these two ports, and after allowing them master got request from windows slave to accept its key.
b) Using salt locally on windows slave
I used some old virtualmachines in oracle virtualbox while collecting more slaves for my new master, but this wasn’t as easy and fast as I thought it would be. At first, it seemed to be an easy task to just change the masters address in /etc/salt/minion files. So I started four of my old virtualmachines and changed the addresses to be my new masters address. After doing this, I was able to accept all of keys on master except one.
I triple checked that I edited the minion file correctly and was certain that there were no typos. Still I was getting error which said: “The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate” I didn’t find any help from google, so I checked the file again and this time scrolled all the way down.
Removing the extra master address from the minion file fixed the error message, but I still couldn’t connect slaves on my master. Fortunately, there is logs!
c) I tried to locate application configuration files in windows, but I could find only few of them, and editing them didn’t seem to work. I am not familiar with windows’ configuration files, and would like to learn how to edit them using salt.
a) In this exercise I’m going to write a simple shell script, that will make my machine a salt slave. I dont have experience in shell scripting but since this script is quite simple I decided to give it a try: #!/bin/bash sudo apt install salt-minion sudo nano /etc/salt/minion: “master: 192.168.190.128” sudo systemctl restart salt-minion This didn’t work. It just opened the file /etc/salt/minion but didn’t type in my masters ip. So I decided to look it up from the internet, and found article in stackoverflow which helped me to get forward.
I thought it would be good idea to put the script on git so I could easily access it on a computer I want to make as a slave. First I cloned my git srv repo to my slave: git clone https://github.com/arttugit/srv.git Then added the script to git: git add testscript.sh And finally making a commit: git add . git commit; git push; git pull
c) Now it is time to install and try vagrant. My few first tries installing vagrant wasn’t so successful. I begun installing vagrant and virtualbox. After both packages were installed I headed to vagrant website to look for boxes. At this time I was trying to install vagrant on my guest machine xubuntu. I picked the most downloaded one, ubuntu server 14.04 LTS and entered following commands:
After investigating this issue and finding information about it on the web, I found one post that suggested enabling the GUI so I could see an error message. So I enabled the GUI by editing the Vagrantfile.
b) Creating a salt state which enables Apaches users’ homepages. At first, installing everything manually on master: sudo salt ‘*’ pkg.install apache2 After that, enabling the userdir module: sudo a2enmod userdir Then I created a directory named public_html, changed permissions (chmod 777 public_html) and created a file index.html. Then I could access the site via webrowser.
After fixing the error above, I applied the state again and this time it seemed to work.
After creating “default-index.html” file under the /srv/salt/apache/ directory, and applying the state again, apaches homepage was replaced by file I had just created.
c) Next step was to enable php to function on users’ home directories as well. I installed php and made some modifications on apaches conf files. I did this manually on salt master, to make sure everything works before making a state. First I had to install php: sudo apt install php libapache2-mod-php after that, I made slight modifications to php7.2.conf file. I found a tutorial on ubuntus site how to enable php on user dir which helped me to get this working: https://wiki.ubuntu.com/UserDirectoryPHP And after restarting apache daemon, php worked on my home directory.
Then I wanted to make above steps to happen automatically. So I made a state.
Problems I encountered was few syntax errors when writing the state and wrong file name in source file. After fixing those errors, I created info.php file on slaves to test if it was working.
d) In this task, I created a state that would set up name-based virtual host on apache. I haven’t done virtual hosts on apache before, so I configured that manually first. I found a handy tutorial for doing that: https://www.maketecheasier.com/name-based-virtualhost-apache/ So basically what I did was create directories for both of virtualhosts, make configuration files for them and tell apache to use them.
e) I couldn’t get example homepages work for new users. I was able to make a state which would create a index.html file for new users, but it didn’t work as I wanted.
Starting with salt stack. I haven’t used Linux that much yet, and I haven’t heard about salt stack before this course started. Controlling hundreds of slave computers does sound interesting for sure! I have read some get started-tutorials at docs.saltstack.com, and there is much to learn. Citing the tutorial: “You can get a general understanding of how Salt works by seeing it in action.” so let’s get started.
c) At first I installed salt by using ubuntus apt packet manager: sudo apt install salt-master salt-minion After the package installation, I configured the slave by editing the minion file: sudoedit /etc/salt/minion. I added two lines of text:
master: 192.168.190.128 id: arttu Then i restarted the daemon: sudo systemctl restart salt-minion.service. In this case, master and slave was the same computer. Now I had to accept the slaves key on the master by typing sudo salt-key –accept-all which would then accept that computer as a slave. I also started another virtual machine to make it a slave for my master computer. I repeated the above steps on the another machine, and changed the id so I could identify my slaves. Then I would type in: sudo salt ‘*’ cmd.run ‘hostname -I’ to test the connection to my slaves. “*” refers to all slaves, so all slaves would return their hostname.
d) I tried some salt states using examples posted here: https://github.com/joonaleppalahti/CCM/tree/master/salt/srv/salt I used the firewall.sls as and example to apply firewall settings on my slaves. I began by creating a directory /srv/salt/. This directory contains instructions of the slave machines. I wanted to make firewall state so I created a file called firewall.sls.
After adding some firewall rules (sudo ufw allow 22,80,443,4505,4506/tcp), I tried out the state. At first, I encountered some errors.
Apparently, there were incorrect syntax in my firewall.sls file. It was fixed by removing the space at the start of fourth line. And after that, it worked!
e) With salt grains I am able to collect information about my slaves and master. Command sudo salt ‘*’ grains.items returns a lot of information about computers under my control, such as processor type, IP address, OS type, memory and other system information.
f) Time to test salt on windows, and make my host computer a slave. I downloaded and installed salt-minion from https://docs.saltstack.com/en/latest/topics/installation/windows.html on my windows 10 host machine. Installation was fast and simple, all I had to do was to type in my masters IP address, and set a hostname for windows slave. Then i accepted the key on my master.
To enable salt windows repositories, I need git. Using command sudo salt ‘*’ pkg.install git, I can do the installation easily to all computers. After git was installed, I updated windows repos.
Installing VLC from master to all slaves worked. Soon after running command shown above, a shortcut of VLC player appeared on my host Windows desktop! Maybe I should install more apps using my master. I recently reinstalled my windows, so I dont have many apps installed at the moment. So why dont I install few useful apps at the same time, from my guest machine with salt.
All exercises below are part of course “Linux palvelimet”. Exercises above this block are part of course “Palvelinten hallinta”. In the above exercises I’m going to learn how to control hundreds, or even thousands of computers using salt stack!
In this excercise I configured SMB protocol (Server Message Block) on my Ubuntu server. Software package used is Samba.
Samba is easy to use, and when using desktop version of ubuntu its even easier. If you want to access file server, simply navigate to nautilus file manager in ubuntu desktop, select “other locations”, and connect to server using following syntax: “smb://<serversname>/
Configuring Samba on Ubuntu server
First I had to get Samba package using command: $sudo apt-get install samba After succesfully installing samba package, I created a directory for samba: $mkdir /home/arttu/sambashare Next step was to edit the configuration file of samba: $sudo nano /etc/samba/smb.conf
When the commands shown above has been typed in and the configuration saved, I restarted the Samba for those commands to take effect: $systemctl restart smbd.service Samba doesn’t use system account login, so I had to make a user for samba and add password to it.
This is basically how to set up Samba. If you dont have firewall enabled on your server, you are good to go and you have a samba server up and running. If you have enabled firewall, then few rules needs to be added. Samba uses ports 139/tcp and 445/tcp, so we need to allow these ports.
After setting the rules in Ubuntu server, we need to map network drive in windows.
1. Installed Apache2 HTTP Server on my Ubuntu server. Command was: sudo apt-get install apache2.
If I wanted to reinstall apache2 i would type in: sudo apt purge apache2 .
Useful commands for using apache2: apache2ctl start apache2ctl restart apache2ctl stop apache2ctl fullstatus or systemctl start apache2 systemctl restart apache2 systemctl stop apache2 systemctl status apache2 a2enmod
2. /etc/apache2/apache2.conf is the main configuration file. mods-available contains all modules and mods-enabled contains modules which are currently in use. Modules can be enabled with a2enmod command. file “envvars” is default environment variables for apache2ctl.
3. I created a www-page named “testing.html” in /var/www/html. I can reach the site by typing my apache servers ip address to browsers search bar and /testing.html after the ip.
12. I could get a free SSL certificate from Let’s Encrypt project, but first I would have to register a domain name, because Let’s Encrypt certificate authority will not issue certificates for a bare IP address.