CS 480 M01 Lecture Notes 11 - Sep 28, 2022
Last Time:
Ch 2. Booting and System Management Daemons
Booting / Bootstrapping and its steps / tasks (broadly defined)
1. Boot process overview
2. System Firmware, Legacy BIOS UEFI
3. Boot Loaders
4. GRUB
6. System Management Daemons
Kernel, init / systemd , Traditional init , systemd vs. the world , systemd in detail
Ch 2. Booting and System Management Daemons Continued
systemd in detail continued
systemd aims to standardize configuration and control of system services ... and more ...
Not a single daemon but collection of programs, daemons, libraries, technologies, and kernel components (~70 different binaries).
unit - entity managed by systemd:
service ,
socket ,
device ,
mount point ,
automount point ,
swap file or partition ,
startup target ,
watched filesystem path ,
timer ,
resource management slice ,
group of externally created processes
Behavior of each unit is defined and configured by a unit file
in
/usr/lib/systemd/system/ (not to be modified) and
/etc/systemd/system (local customization)
rsyncd.service example
Managed by systemctl
systemctl list-units| less
systemctl list-units| wc
systemctl list-unit-files| wc
systemctl list-units --type=service
systemctl status -l ntp # older
systemctl status -l chronyd # new
systemctl status -l cups
systemctl status -l postfix
systemctl stop postfix
systemctl disable postfix
systemctl status -l postfix
systemctl enable postfix
systemctl start postfix
systemctl status -l postfix
Targets : multiuser, graphical, poweroff, rescue, ...
systemctl get-default
systemctl set-default graphical.target
systemctl get-default
systemctl set-default multi-user.target
systemctl isolate multi-user.target
systemctl isolate rescue.target
systemctl isolate emergency
(was telinit ...)
Dependencies among units
- Not all dependencies are explicit.
systemd takes over inetd and also D-Bus IPC - knows what ports or IPC conection point will a service host...
- Assumptions about normal behavior based on unit type - a unit can turn it off by
DefaultDependacies=false
- Explicitly declared in the
[Unit] section of a unit file:
Extend by
systemctl add-wants multi-user.target my.local.service
Execution order is not determined by the dependency done by Before and After clauses in the unit files.
Local services
create the unit file in /etc/systemd/system (say custom.service)
Activate by systemctl enable custom.service
General Rule: do not modify unit files (unless you wrote it). Instead create directory /etc/systemd/system/unit-file.d and add configs in to a file there (xxx.conf)
plus assignment #4 and "Local services and customizations" sub-chapter in our textbook
systemd logging
journal - universal logging framework that includes all kernel and service messages from early boot to final shutdown
journald daemon, msg stored under /run, accessed by journalctl
journald configured by
/etc/systemd/journald.conf (Storage=persistent to retain messages from prior boots)
journalctl --list-boots
Reboot and shutdown procedures
- power button
- shutdown + "wall" message
- halt/reboot (called by halt), kills processes and sync FS
- halt -n (after fsck)
- telinit - no grace period or warning
- poweroff <=> halt + power mgmt
All are links to systemctl now ...
Stratagems for a nonbooting system
Basic approaches:
- don't debug ; just restore a known good state
- bring up just enough to run shell and debug interactively
- boot separate image, mount sick file system, and investigate from there
10. Logging
Huge amount of data may get generated by kernel, system daemons, services and other tools - gets logged => will fill up disk(s) eventually ...
... or quickly - out of control process/logging can bring the system(s) down really fast ... => /var/log to its own partition
Limited useful life, different retention needs for different data
Need to be summarized, filtered, searched, analyzed, compressed and archived
Log message - usually a line of text per event with time stamp, type and severity of the event, process that generated it, ...
Admins need to extract critical / important / useful / ... info and act on it - log management
Check soon when debugging problems and errors
Common schemes
| throw away | suspect |
| reset log files periodically |
| rotate log files | script that moves them: logf logf.0 logf.1 ... |
| compress and archive |
Log Management
- Collect log from various sorces
- Provide structured interface to query, analyze, filter, and monitor messages
- Manage retention and expiration - keep as long as required (laws, policies) or potentially needed or useful
syslog traditional UNIX mngmt system tackling #1 above
Configuration differs widely among operating systems plus ...
Many applications, daemons, startup scripts, ... bypass syslog => varies significantly among UNIX flavors and Linux distros
systemd-journal - recent addition - second attempt to bring sanity to the logging madness
- collects messages
- stores them in an indexed and compressed binary format
- provides command-line interface for viewing and filtering logs
Can stand alone or coexist (with varying integration degrees - configurable) with syslog
Log Files and Locations
/var/log
/var/adm
/etc/syslog.conf or (
/etc/rsyslog.conf or
/etc/syslog-ng/syslog-ng.conf
)
owned by root usually + should have 600 mostly
less privileged process (httpd) may need to write into them => ownership and mode changed appropriately
Columns:
- log files (to archive, summarize, truncate)
- creating program
- how is the filename specified
- suggested frequency of cleanup
- systems using the log file
- content description
| 5th Edition | 4th Edition |
|---|
|
|
Special Log Files / Files not to manage (not text files)
/var/log/wtmp in binary format, entries appended
/var/log/lastlog only the time of the last login for each user
DB binary transaction logs
systemd journal files
binary format - need journalctl to view ( -u sshd , -f, ...)
Next : Systemd journal