2

On 22.04 LTS if I use journalctl to view the logs, I can only see entries from 48 hours ago (and while less important, journalctl --list-boots only lists the current boot).

Here's what I've checked (numbered so comments can reference):

Settings

  1. In journald.conf: These are all set to the Ubuntu standard defaults (see below).

  2. Nothing in logrotate.conf or logrotate.d is set to below 4 weeks (other than rsyslog which is 7 days).

  3. No systemd timers or cron jobs (either in /etc/cron* or crontabs) are set to truncate the journal (grep -r journal /etc/cron*)

Disk space

  1. Journald will use up to 4Gb or 10% of disk space by default. On the partition /var is on, I have ~30G available.

  2. journalctl --disk-usage shows ~800-850G in use.

  3. The journald.conf file is set to Storage=auto, but since /var/log/journal exists (and is populated), that means it's saving the journal to disk ("persistent"). I have set it to "persistent" explicitly and reloaded the systemd unit but it makes no difference.

  4. Perhaps there was some transient disk space issue that made journald truncate, but I waited some time and it didn't come back.

Other

  1. journalctl --verify reports no issues.

  2. Performing a sync followed by a flush makes no change.

Comparison

I have another nearly identical machine which logs back several months (and has less disk space available).

How can I configure the journal so that I can see log entries from dates earlier than 48 hours? The machine is a production server and I would rather not schedule down time for what might be a risky operation to delete the journal and re-set it.

~$ systemd-analyze cat-config systemd/journald.conf

[Journal]
#Storage=auto
#Compress=yes
#Seal=yes
#SplitMode=uid
#SyncIntervalSec=5m
#RateLimitIntervalSec=30s
#RateLimitBurst=10000
#SystemMaxUse=
#SystemKeepFree=
#SystemMaxFileSize=
#SystemMaxFiles=100
#RuntimeMaxUse=
#RuntimeKeepFree=
#RuntimeMaxFileSize=
#RuntimeMaxFiles=100
#MaxRetentionSec=
#MaxFileSec=1month
#ForwardToSyslog=yes
#ForwardToKMsg=no
#ForwardToConsole=no
#ForwardToWall=yes
#TTYPath=/dev/console
#MaxLevelStore=debug
#MaxLevelSyslog=debug
#MaxLevelKMsg=notice
#MaxLevelConsole=info
#MaxLevelWall=emerg
#LineMax=48K
#ReadKMsg=yes
#Audit=no
TommyPeanuts
  • 1,147

1 Answers1

0

The setting #Storage=auto is disabled with a # in your /etc/systemd/journald.conf file. Anyway, it will not use more than 10% of the filesystems and is capped at 4GB. Maybe your logs grow too fast because of some service misbehaving and you can't see older entries getting deleted. Have you checked the current total size with journalctl --disk-usage to start with?

EDIT after the various updates in the question:

I have actually one of my 22.04 machines that has the "same" behaviour. Indeed, it would show only the last boot. There can be a number of reasons for this as explained here: How to find previous boot log after Ubuntu 16.04+ restarts? . On the affected machine, I uncommented Storage=persistent in /etc/systemd/journald.conf and let it reste a day doing 2 reboots. I know do have the last 3 boots properly shown in journalctl --list-boots. I'll leave it a few more day to make sure but you might want to try this as well. This behaviour might be the result of an upgraded machine vs. a fresh install and maybe that explains the difference between your machines.

Joris Le Blansch
  • 9