Systemd Tutorial / Guide
- systemd - manages the system and services
- systemctl - systemd CLI
- journald - systemd logging framework
- logind - Seats and login handling service
- resolved - Manages DNS
- timesyncd - Manages keeping time in sync
- networkd - Manages network connections
Service Management
- We use nginx as an example service in most examples. You can refer to a service by its full name (ex: nginx.service) or short name (ex: nginx ).
| sudo systemctl start nginx | start service |
| sudo systemctl stop nginx | stop service |
| sudo systemctl restart nginx | restart service |
| sudo systemctl reload nginx | reload service config |
| sudo systemctl reload-or-restart nginx | reload if possible or restart |
| sudo systemctl enable nginx | enable |
| sudo systemctl enable –now unit | enable and start |
| sudo systemctl disable nginx | disable service |
| sudo systemctl reenable unit | disable and re-enable |
| sudo systemctl mask nginx | link to dev null |
| sudo systemctl unmask nginx | set link back |
Listing units:
| systemctl | all active |
| systemctl list-units | all active |
| systemctl list-units –all | all in mem active/inactive |
| systemctl list-unit-files | all unit files ( including not in mem ) |
| systemctl list-units –all –state=inactive | |
| systemctl list-units –type=service |
Status and info:
| systemctl status nginx |
| systemctl cat nginx |
| systemctl list-dependencies nginx |
| systemctl list-dependencies –all nginx |
| systemctl show nginx |
Checks:
| systemctl is-active nginx |
| systemctl is-enabled nginx |
| systemctl is-failed nginx |
More:
| systemctl status pid | |
| systemd-analyze blame | how long it took for each component to boot |
Service commands ( for systemd and sysv init ):
| sudo service nginx start |
| sudo service nginx stop |
| sudo service nginx status |
Systemd Files and Directories
/lib/systemd/system
- system’s copy of unit files
- don’t edit these, override them in another location
/run/systemd/system
- medium priority between /etc/systemd/system and /lib/systemd/system
- temporary, wiped on boot
- systemd puts dynamically created unit files here
/etc/systemd/system
- takes priority over other locations
- modify files here to override
- enable/disable usually creates/removes link to /lib/systemd/system
/usr/lib/systemd/system/
- on Arch
Override or add parts of a unit file:
/etc/systemd/system/example.service.dnginx.conf
Edit Services
| sudo systemctl edit nginx.service | |
| sudo systemctl edit –full nginx.service | full, not just snippet |
| sudo systemctl daemon-reload | |
| systemctl revert unit | revert to vendor version |
An example service:
/etc/systemd/system/myservice1.service[Unit] Description=My useful service [Service] Type=simple ExecStart=/bin/bash /usr/bin/myservice1.sh [Install] WantedBy=multi-user.target
Another example service:
/etc/systemd/system/server1.service[Unit] Description=My good server After=network.target [Service] type=Simple User=user1 Restart=on-failure RestartSec=1 StartLimitBurst=5 StartLimitIntervalSec=10 StartLimitAction=reboot ExecStart=/bin/python /opt/server1/scripts/server1.py [Install] WantedBy=multi-user.target </code>
Targets ( Runlevels )
| systemctl list-units –type=target | active targets |
| systemctl list-unit-files –type=target | show all targets |
| systemctl get-default | show default target |
| sudo systemctl set-default multi-user.target | change default target |
| sudo systemctl set-default graphical.target | change default target |
| systemctl list-dependencies multi-user.target | show target units |
| sudo systemctl isolate multi-user.target | switch to different target (temp) |
Reboot / Poweroff
| sudo systemctl poweroff | |
| sudo systemctl halt | |
| sudo systemctl reboot | |
| sudo systemctl rescue | rescue / single user mode |
| systemctl hibernate | |
| systemctl suspend | |
| systemctl hybrid-sleep |
User Level Units
| systemd –user | user instance started by PAM ( on Arch ) |
| /usr/lib/systemd/user/ | from installed packages |
| ~/.local/share/systemd/user/ | for packages installed in home dir |
| /etc/systemd/user/ | system wide, placed by admin |
| ~/.config/systemd/user/ | user controlled |
Command:
| systemctl –user enable unit |
User env vars:
| ~/.config/environment.d/*.conf | user env vars |
See HERE for more info on user env vars.
Logs
Key info:
| /etc/systemd/journald.conf | The config file |
| /var/log/journal | binary logs go here |
Basic log commands:
| journalctl | show all log entries |
| journalctl -b | only since boot |
| journalctl -k | kernel messages, since boot (dmesg) |
| journalctl -k -b -5 | kernel messages, 5 boots ago |
| journalctl -u nginx | logs for this service |
| journalctl –utc | UTC Timestamps |
Time related info which is useful when working with logs:
| timedatectl list-timezones |
| sudo timedatectl set-timezone zone |
| timedatectl status |
To make sure persistent logging is enabled, first make sure this directory exists:
sudo mkdir -p /var/log/journal
And that this setting is in place:
sudo nano /etc/systemd/journald.conf[Journal] Storage=persistent
More time sepecification:
| journalctl –list-boots | show boots that journald knows about |
| journalctl -b -1 | journal from previous boot |
| journalctl -b caf0524a1d394ce0bdbcff75b94444fe | specific boot |
| journalctl –since “2015-01-10 17:15:00” | |
| journalctl –since “2015-01-10” –until “2015-01-11 03:00” | |
| journalctl –since yesterday | |
| journalctl –since 09:00 –until “1 hour ago” |
Even more time specification:
| journalctl -u nginx –since today |
| journalctl -u nginx -u php-fpm –since today |
By PID, UID, GID:
| journalctl _PID=8088 | by PID |
| journalctl _UID=33 | by user ID |
| journalctl -F _GID | by group ID |
Check the docs for fields:
| man systemd.journal-fields |
Priority:
| journalctl -p err | by priority |
Severity levels
| 0 | emerg |
| 1 | alert |
| 2 | crit |
| 3 | err |
| 4 | warning |
| 5 | notice |
| 6 | info |
| 7 | debug |
- NOTE - Use arrows to see long lines.
Output control:
| journalctl –no-full | truncate long lines |
| journalctl -a | all info, including unprintable chars |
| journalctl –no-pager | output directly, good for piping |
Json and pretty json output:
| journalctl -o json |
| journalctl -o json-pretty |
Available formats:
| cat | just show the message field |
| export | binary, good for backups |
| json | Standard - one entry per line |
| json-pretty | JSON but pretty |
| json-sse | server-sent event compatible JSON |
| short | default syslog style |
| short-iso | default format with ISO 8601 wallclock timestamps |
| short-monotonic | default with monotonic timestamps |
| short-precise | default with microsecond precision |
| verbose | all fields including hidden |
Tail equivalent for journald:
| journalctl -n | most recent 10 |
| journalctl -n 20 | most recent 20 |
| journalctl -f | follow |
Disk usage and log clearing:
| journalctl –disk-usage | show disk usage |
| sudo journalctl –vacuum-size=1G | remove old entries until the log is this size |
| sudo journalctl –vacuum-time=1years | remove entries older than a year |
/etc/systemd/journald.conf Settings:
- volatile storage - /run filesystem
| SystemMaxUse | max disk space to use (persistent storage) |
| SystemKeepFree | amount of disk space to keep free (persistent storage) |
| SystemMaxFileSize | max file size before logs are rotated (persistent storage) |
| RuntimeMaxUse | max disk space to use (volatile storage) |
| RuntimeKeepFree | amount of disk space to keep free (volatile storage) |
| RuntimeMaxFileSize | max file size before logs are rotated (volatile storage) |
Network
Boot
Learn more HERE
Other - Extra Info
Systemd init scripts:
/etc/init.d/
KDE GUI tool:
sudo apt install kde-config-systemd
kcmshell5 kcm_systemd