Low Orbit Flux Logo 2 F

Splunk Universal Forwarder Install and Setup

We’re going to show you how to setup the Splunk Universal Forwarder. This will allow you to send logs and data from a remote host to a centralized indexer. Before actually setting up the forwarder we are going to show you how to enable receiving on the indexer so that it will have something to connect to.

We’re covering the following on this page:

NOTE - You should just swap in your own specific information In any place where we use an exact version number, IP address, or home directory path.

Splunk Enable Receiving

You need to enable receiving before you can actually receive data from your forwarders. This can be done from the GUI with the following steps.

From the Splunk GUI do the following:

Restart Splunk from the CLI on the Splunk indexer host ( where you installed Splunk Enterprise ):


sudo systemctl stop Splunkd
sudo systemctl start Splunkd

Splunk Enable Receiving CLI

You can also do this from the CLI if you want.

Assuming that you run splunk as the dedicated user “splunk” you will want become that user first.


sudo su - splunk


vi /opt/splunk/etc/system/local/inputs.conf

[splunktcp://9997]
disabled = 0

Install The Splunk Universal Forwarder

Copy the Splunk Forwarder package over to the host like this:


scp Downloads/splunkforwarder-8.2.2.1-ae6821b7c64b-Linux-x86_64.tgz 192.168.3.220:/home/user1

From the host that you want to install the forwarder on, unpack it and copy it into place.


tar xvfz splunkforwarder-8.2.2.1-ae6821b7c64b-Linux-x86_64.tgz
sudo mv splunkforwarder /opt

Create a dedicated splunk user and chown the splunk directory:


sudo adduser splunk
sudo chown -R splunk:splunk /opt/splunkforwarder

Add the new splunk user to the sudo group.

sudo vi /etc/group

...
sudo:x:27:user1,splunk
...
...

Become the splunk user. You will want to make sure that you are logged in as this user before starting for the first time and before enabling in systemd.


sudo su - splunk

Setup the SPLUNK_HOME and PATH environment variables for the current shell while also adding it to your bashrc file to make it persistent.


export SPLUNK_HOME=/opt/splunkforwarder
export PATH=$PATH:$SPLUNK_HOME/bin

echo export SPLUNK_HOME=/opt/splunkforwarder >> ~/.bashrc
echo export PATH=$PATH:$SPLUNK_HOME/bin >> ~/.bashrc

For the first time starting, start the forwarder like this to accept the license without reading it. It will also ask you to create a user and password to manage the forwarder.


 splunk start --accept-license

Enable Splunk start on boot with systemd. Splunk needs to be stoppeed first. Run this while still logged in as the dedicated splunk user.


 splunk stop
 sudo /opt/splunkforwarder/bin/splunk enable boot-start -systemd-managed 1

Now you will see it listed with this command:


 systemctl list-unit-files | grep -i splunk

You can now start and stop it with systemd like this:


 sudo systemctl start SplunkForwarder
 sudo systemctl stop SplunkForwarder

Adding Data Sources

If you are using a dedicated user, make sure you are logged in as that user while setting up data sources.


sudo su - splunk

Specify the Splunk index server to coneect to here. Swap in your own IP or hostname. Also swap in which ever password you had setup.


splunk add forward-server splunk1:9997 -auth admin:password1

These are some examples of data sources that you could add.


splunk add monitor /var/log/nginx -sourcetype nginx -index my_nginx

splunk add monitor /var/log -sourcetype linux_logs -index main
splunk add monitor /var/log -sourcetype journald -index my-test-index2

You will want to make sure that the splunk user has read access to the logs.


sudo chmod -R a+r /var/log

Make sure to restart the forwarder after adding data sources.


sudo systemctl restart SplunkForwarder

Two versions of the default config file, don’t use these:

$SPLUNK_HOME/etc/system/default
$SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/default

The config commands above will modify these files. They can be modified with an editor too.

$SPLUNK_HOME/etc/system/local/outputs.conf
$SPLUNK_HOME/etc/apps/search/local/inputs.conf

Deb / RPM Packages

You could also use the deb and RPM packages but I wouldn’t bother with them.


 rpm -i splunkforwarder-<…>-linux-2.6-x86_64.rpm
 dpkg -i splunk_package_name.deb