If a buildroot enviornment does not provide libzstd and kexec_file_load a
zstd compressed kernel, it will suffer from coredump.
The following is observed from the coredump
Core was generated by `/sbin/kexec -s -p --command-line=BOOT_IMAGE=\(hd0,gpt2\)/vmlinuz-6.15.0-0.rc5.250509g9c69f8884904.47.fc43.aarch64\ ro\ rootflags=subvol=root\ irqpoll\ nr_cpus=1\ reset_devices\ cgroup_disable=memory\ udev.children-max=2\ panic=10\ swiotlb=noforce\ novmcoredd\ cma=0\ hugetlb_cma=0\ sbsa_gwdt.pretimeout=0 --initrd=/boot/initramfs-6.15.0-0.rc5.250509g9c69f8884904.47.fc43.aarch64kdump.img /boot/vmlinuz-6.15.0-0.rc5.250509g9c69f8884904.47.fc43.aarch64'.
Program terminated with signal SIGSEGV, Segmentation fault.
143 if (!!memcmp(buf, "PE\0\0", 4)) {
(gdb)
Besides providing some C code to fix the segfault issue in upstream.
There should be explicit requirement in the spec to ask the buildroot
enviorment to provide libzstd. So it can work with the zstd compressed
linux kernel.
With this patch, the following can be observed in the configuration
process of kexec-tools
,,,
checking for zstd.h... yes
checking for ZSTD_decompress in -lzstd... yes
,,,
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Lichen Liu <lichliu@redhat.com>
Currently both production and staging deployments of Packit are enabled
and both of them will create pull requests when there is a new upstream
release. So disable packit-stg to avoid creating duplicate PRs.
Signed-off-by: Coiby Xu <coiby.xu@gmail.com>
- stop mkdir'ing kcp as kcp.c has been gone long time ago
- use autosetup to automatically apply patches
- clean up SOURCE file
Signed-off-by: Coiby Xu <coxu@redhat.com>
Resolves: https://github.com/rhkdump/kdump-utils/issues/22
As found by Dave, built kexec-tools wasn't linked to libraries like
libz.so.1,
# ldd /sbin/kexec
linux-vdso.so.1 (0x0000ffff8cfa1000)
libc.so.6 => /lib64/libc.so.6 (0x0000ffff8cd10000)
/lib/ld-linux-aarch64.so.1 (0x0000ffff8cf54000)
As a consequence, kexec fails to load the aarch64 kernel.
Previous version of /sbin/kexec is linked to liblzma.so and
libz.so,
# ldd /sbin/kexec
linux-vdso.so.1 (0x00007f28d392c000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f28d37e7000)
libz.so.1 => /lib64/libz.so.1 (0x00007f28d37c6000)
libc.so.6 => /lib64/libc.so.6 (0x00007f28d35d4000)
/lib64/ld-linux-x86-64.so.2 (0x00007f28d392e000)
So add build dependencies on those needed libraries.
Fixes: 28ced79 ("Split out kdump-utils and makedumpfile")
Reported-by: Jean Baptiste Trystram <jbtrystram@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
As suggested by Richard [1],
> Given the trivial nature of these files I think `LGPL-2.1-or-later` can be omitted from the license tag.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2239566#c7
Suggested-by: Richard Fontana <rfontana@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Lichen Liu <lichliu@redhat.com>
Fix the following error found by rpmlint,
makedumpfile.x86_64: E: explicit-lib-dependency zlib
You must let rpm find the library dependencies by itself. Do not put unneeded
explicit Requires: tags.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Dave Young <dyoung@redhat.com>
use consistent build flags %make_install to have the benefits like
enabling parallel building automatically.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Dave Young <dyoung@redhat.com>
As suggested by Carl [1],
> /usr/lib/dracut/modules.d/99kdumpbase/monitor_dd_progress has some
> inconsistencies with other scripts in that directory. It is missing the
> .sh extension and is not executable. The latter is resulting in an
> rpmlint error.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2239566#c2
Suggested-by: Carl George <carl@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Dave Young <dyoung@redhat.com>
This patch upstreams the to-be-split-out kdump-utils to
https://github.com/rhkdump/kdump-utils. And it also simplify the .spec
file by putting the installation logic into a Makefile.
Cc: Philipp Rudo <prudo@redhat.com>
Cc: Carl George <carl@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Dave Young <dyoung@redhat.com>
Suggested-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Dave Young <dyoung@redhat.com>
Suggested-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Dave Young <dyoung@redhat.com>
Majority of current kexec-tools users have installed kexec-tools out of
the need for the kdump feature. To ensure a smooth transition, add
kdump-utils as weak dependency. If users only want to use kexec-tools,
they can uninstall kdump-utils.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Suggested-by: Carl George <carl@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Dave Young <dyoung@redhat.com>
When an old version of kexec-tools gets replaced by kdump-utils,
"%systemd_post" will be executed in the post scriptlet which has the
purpose to "systemctl preset kdump" for freshly installed kexec-tools.
But in the case of kdump-utils replacing kexec-tools, it is not needed
so skip this case.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Suggested-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Dave Young <dyoung@redhat.com>
man files doesn't preserve line breaks. Construct a list to list the
cases where additional memory will be reserved.
Fixes: c752cbb2 ("Explain the auto_reset_crashkernel option in more details")
Reported-by: Jie Li <jieli@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Currently, the other options like "raw <partition>" become child items
of the auto_reset_crashkernel option,
auto_reset_crashkernel <yes|no>
...
raw <partition>
...
nfs <nfs mount>
...
...
Fix it by ending the auto_reset_crashkernel with ".RE".
Fixes: 73ced7f4 ("introduce the auto_reset_crashkernel option to kdump.conf")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
With patch series [1], PowerPC supports nr_cpus=1,
so use nr_cpus=1 instead of maxcpu=1 in the kdump environment.
Note this changes is dependent on kernel changes [1]
[1] https://lore.kernel.org/all/170800202447.601034.7290612623478478380.b4-ty@ellerman.id.au/#t
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
The supported targets list has two separate hypervisor sections. Merge
them into one.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
In CentOS Stream a list of supported targets is maintained. Where
"supported" means that these targets are tested regularly. Even though
there are some small differences between CentOS Stream and Fedora when
it comes to provided/supported packages and kernel configs, having this
list in Fedora as well makes sense. As it provides a entry point for
users to find out if a given setup is meant to work or not. Thus include
the supported-kdump-targets.txt from CentOS Stream 9 [1] into Fedora.
[1] 0a09d12d89/supported-kdump-targets.txt
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Currently kdump-dep-generator will source kdump-lib.sh. Notice
kdump-dep-generator have #!/bin/sh so it should be POSIX, but
kdump-lib.sh is a non-POSIX bash script. When Bash is configured to run
in POSIX mode for #!/bin/sh scripts, it will fail with:
/usr/lib/kdump/kdump-lib.sh: line 1042: syntax error near unexpected token `<'
/usr/lib/kdump/kdump-lib.sh: line 1042: ` done < <( _crashkernel_parse "$ck")'
This subshell call is easy to convert into a pipe but we should just
source kdump-lib-initramfs.sh here, the only thing kdump-dep-generator
needs is is_ssh_dump_target which is in kdump-lib-initramfs.sh, also
prevents further POSIX violations.
Signed-off-by: Kairui Song <kasong@tencent.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2242185
grep (3.8) warnings when running the unit tests or running
"kdumpctl reset-crashkernel" on >= F39,
# unit tests
Examples:
1) kdumpctl _find_kernel_path_by_release() returns the kernel path for the given release
When call _find_kernel_path_by_release vmlinuz-6.2.11-200.fc37.x86_64
1.1) WARNING: There was output to stderr but not found expectation
stderr: grep: warning: stray \ before /
# spec/kdumpctl_general_spec.sh:169-172
# kdumpctl reset-crashkernel
grep: warning: stray \ before /
kdump: Updated crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M for kernel=/boot/vmlinuz-6.6.8-200.fc39.x86_64. Please reboot the system for the change to take effect.
This warning can be reproduced by
echo 'kernel="/boot/vmlinuz-6.4.6-200.fc38.x86_64"' | grep -E "^kernel=.*$_release(\/\w+)?\"$"
This patch removes unneeded backslash. It also adds a test for
systemd-boot path. And for simplification, Parameters:dynamic is now
used to generate test data dynamically.
Fixes: 8af05dc4 ("kdumpctl: Add support for systemd-boot paths")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
When using multipath devices as the target for kdump, if user_friendly_name
is also specified, devices default to names like "mpath*", e.g., mpatha.
In dracut, we obtain a persistent device name via get_persistent_dev. However,
dracut currently believes using /dev/mapper/mpath* could cause issues, thus
alternatively names are used, here it's /dev/disk/by-uuid/<FS_UUID>.
During the kdump boot progress, the /dev/disk/by-uuid/<FS_UUID> will exist as
soon as one of the path devices exists, but it won't be usable by systemd,
since multipathd will claim that device as a path device. Then multipathd will
get stopped before it can create the multipath device.
Without user_friendly_name, /dev/mapper/<WWID> is considered a persistent
device name, avoiding the issue.
The exit of multipathd is due to two dependencies in the current dracut module
90multipath/multipathd.service, "Before=initrd-cleanup.service" and
"Conflicts=initrd-cleanup.service".
As per man 5 systemd.unit, if A.service has "Conflicts=B.service", starting
B.service will stop A.service.
This is useful during normal boot. However, we will never switch-root after
capturing vmcore in kdump.
We need to ensure that multipathd is not killed due to such dependency issue.
Without modifying multipathd.service, we add ConditionPathExists=!/proc/vmcore
to skip initrd-cleanup.service in kdump. This approach is beneficial as
it avoid the potential termination of other services that conflict with
initrd-cleanup.service. Also skip initrd-parse-etc.service as it will try to
start initrd-cleanup.service. Both of these services are used for switch root,
so they can be safely skipped in kdump.
Suggested-by: Benjamin Marzinski <bmarzins@redhat.com>
Suggested-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Resolves: https://issues.redhat.com/browse/FC-1046
We require that <user> be explicitly specified in 'ssh <user>@<hostname>'. When
forgetting to specify, such as 'ssh 192.168.10.2', a useful error log should be
printed instead of exiting directly.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Since the previous commit reworks znet configuration to be based on the
active system configuration, there is no dependency on any existing
persistent configuration any more. Hence the old code handling systems with
exactly one s390-specific nmconnection as persistent configuration can be
removed.
Migration of the old persistent device configuration mechanism with
nmconnections (or ifcfg) to zdev is handled independently in s390utils.
[https://github.com/steffen-maier/s390utils/pull/1/commits
("znet: migrate to consolidated persistent device config with zdev
(#1937046,#1937048))"]
Signed-off-by: Steffen Maier <maier@linux.ibm.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
This is a preparation for consolidating s390 network device config with
https://github.com/dracutdevs/dracut/pull/2534
("feat(znet): use zdev for consolidated device configuration")
https://github.com/steffen-maier/s390utils/pull/1/commits
("znet: migrate to consolidated persistent device config with zdev
(#1937046,#1937048))"
("znet: clean up old deprecated persistent device config
(#1937046,#1937048)").
With above consolidation, s390-specific low-level configuration information
will no longer be in NetworkManager connections (nor ifcfg files), but in
the persistent configuration database of chzdev from s390-tools.
Since the kdump dracut module here depends on the "znet" dracut module [1]
and "znet" will copy all persistent configuration into initrd as of above
commit, all s390-specific information would already be in the kdump initrd.
[1] 08de712528 ("Move some dracut module dependencies checks to
module-setup.sh"), 7148c0a30d ("add s390x netdev setup")
However, it is more appropriate and also removes the copy dependency from
"znet" to introduce the consolidated zdev mechanism for importing just the
required network device config from the current active system
configuration. It does not depend on any of the pull requests above.
It does not depend on any existing persistent configuration
and can replace the old function code. This is similar to dracut block
device dependency handling in s390-tools zdev/dracut/95zdev-kdump.
The old code only seems to work if there is exactly one s390-specific
nmconnection (or ifcfg file). Related commits:
b5577c163a ("Simplify setup_znet by copying connection profile to initrd"),
7d47251568 ("Iterate /sys/bus/ccwgroup/devices to tell if we should set up rd.znet"),
8b08b4f17b ("Set up s390 znet cmdline by "nmcli --get-values""),
ce0305d4f9 ("Add a new option 'rd.znet_ifname' in order to use it in udev rules"),
7148c0a30d ("add s390x netdev setup").
A bonding or teaming setup would have multiple following network
interfaces, each of which would need a low-level config if they're s390
channel-attached network devices. The new code should be able to handle
that by iterating the involved network interfaces. Chzdev only exports
something if it's a device type it deems itself responsible for.
Additional debugging output can be generated with e.g. dracut option
"--stdlog 5" (or short -L5). It shows the chzdev export result, the output
of chzdev export and import, and an overview of the resulting persistent
config within the initrd. On systems, which default to using dracut option
"--quiet", you might need an additional "--verbose" to counter "--quiet" so
-L5 has effect. Typically combined with "--debug" to get a shell trace from
building an initrd (Note: --debug does not increase the log levels).
Signed-off-by: Steffen Maier <maier@linux.ibm.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Resolves: https://issues.redhat.com/browse/RHEL-11897
Previously fix 0177e248 ("Use the same /etc/resolve.conf in kdump initrd
if it's managed manually") is problematic,
1) it generated .conf file unrecognized by NetowrkManager ;
2) this .conf file was installed to current file system instead of to the kdump initrd;
3) this incorrect .conf file prevented the starting of NetworkManager.
This patch fixes the above issues and also suppresses a harmless warning
when systemd-resolved.service doesn't exist,
# systemctl -q is-enabled systemd-resolved
Failed to get unit file state for systemd-resolved.service: No such file or directory
Fixes: 0177e248 ("Use the same /etc/resolve.conf in kdump initrd if it's managed manually")
Reported-by: Jie Li <jieli@redhat.com>
Cc: Dave Young <dyoung@redhat.com>
Reviewed-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Resolves: https://issues.redhat.com/browse/RHEL-17451
Explain what factors affect the default crashkernel value and ask users
to reset it manually if needed.
Cc: Baoquan He <bhe@redhat.com>
Cc: Jie Li <jieli@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2247940
Currently, CoreOS image fails to be built. This is because since commit
00c37d8c ("spec: Drop special handling for IA64 machines"), the last
command is now servicelog_notify and it fails to run in such
invocation environment. Thus the %post scriptlet returns a non-zero
exit code which breaks package installation,
Running scriptlet: kexec-tools-2.0.27-4.fc40.ppc64le
/proc/ is not mounted. This is not a supported mode of operation. Please fix your invocation environment to mount /proc/ and /sys/ properly. Proceeding anyway. Your mileage may vary.
servicelog_notify: is not supported on the Unknown platform
warning: %post(kexec-tools-2.0.27-4.fc40.ppc64le) scriptlet failed, exit status 1
Error in POSTIN scriptlet in rpm package kexec-tools
Quoting [1],
> Non-zero exit codes from scriptlets can break installs/upgrades/erases
> such that no further actions will be taken for that package in a
> transaction (see Ordering), which may for example prevent an old version
> of a package from being erased on upgrades, ...
>
> All scriptlets MUST exit with the zero exit status. Because RPM in its
> default configuration does not execute shell scriptlets with the -e
> argument to the shell, excluding explicit exit calls (frowned upon with
> a non-zero argument!), the exit status of the last command in a
> scriptlet determines its exit status...
>
> Usually the most important bit is to apply this to the last command
> executed in a scriptlet, or to add a separate command such as plain “:”
> or “exit 0” as the last one in a scriptlet.
Following the above suggestion, add a separate command ":" as the last
one to the %post scriptlet.
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/
Reported-by: Colin Walters <walters@redhat.com>
Cc: Dusty Mabe <dustymabe@redhat.com>
Cc: Philipp Rudo <prudo@redhat.com>
Fixes: 00c37d8c ("spec: Drop special handling for IA64 machines")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Currently is_system_modified will return immediately when check_*_modified
return a non-zero value, and the remaining checks will not be executed.
For example, if there is a fs-related error exists, and someone changes the
kdump.conf, check_files_modified will return 1 and is_system_modified will
return 1 immediately. This will cause kdumpctl to skip check_fs/drivers_modified,
kdump.service will rebuild the initrd and start successfully, however, any
errors should prevent kdump.service from starting.
This patch will cause check_*_modifed to continue running until an error occurs
or all execution ends.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Acked-by: Tao Liu <ltao@redhat.com>
It's reported that kdump kernel failed to boot and can't dump vmcore
when crashkernel=192M and SME/SEV is active.
This is because swiotlb will be enabled and reserves 64M memory by
default on system with SME/SEV enabled. Then kdump kernel will be out of
memory after taking 64M away for swiotlb init.
So here add extra 64M memory to default crashkernel value so that kdump
kernel can function well as before. When doing that, search journalctl
for the "Memory Encryption Features active: AMD" to check if SME or SEV
is active. This line of log is printed out in kernel function as below
and the type SME is mutual exclusive with type SEV.
***:
arch/x86/mm/mem_encrypt.c:print_mem_encrypt_feature_info()
Note:
1) The conditional check is relying on journalctl log because I didn't
find available system interface to check if SEV is active. Even
though we can check if SME is active via /proc/cpuinfo. For
consistency, I take the same check for both SME and SEV by searching
journalctl.
2) The conditional check is relying on journalctl log, means it won't
work for crashkernel setting in anoconda because the installation
kernel doesn't have the SME/SEV setting. So customer need manually
run 'kdumpctl reset-crashkernel' to reset crashkernel to add the
extra 64M after OS installation.
3) We need watch the line of log printing in
print_mem_encrypt_feature_info() in kernel just in case people may
change it in the future.
Signed-off-by: Baoquan He <bhe@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Currently _crashkernel_add can't deal with larger memory ranges like
terabyte. For example, '_crashkernel_add "128G-1T:4G" "0"' actually
returns empty result. This patch allows _crashkernel_add to address
terabyte, petabyte and exabyte memory ranges.
Fixes: 64f2827a ("kdump-lib: Harden _crashkernel_add")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2243068
Currently, when kexec-tools is installed, the kernel will automatically
have the crashkernel parameter set up. In the case where users only want
the kexec reboot feature, this is not what users want as a 1G-RAM system
will lose 192M memory. Considering Fedora's systemd preset policy has
kdump.service disabled and RHEL' has kdump.service enabled, this patch
makes kexec-tools only reset crashkernel when kdump.service is enabled.
Reported-by: Chris Murphy <bugzilla@colorremedies.com>
Cc: Philipp Rudo <prudo@redhat.com>
Cc: Adam Williamson <awilliam@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
On secure boot enabled systems with static keys, kexec with kexec_file_load(-s)
fails as "Permission Denied" when fadump is enabled.
Similar to kdump, load kernel signing key for fadump as well.
Reported-by: Sachin P Bappalige <sachinpb@linux.vnet.ibm.com>
Signed-off-by: Nayna Jain <nayna@linux.ibm.com>
Kernel signing key is deleted once kdump is loaded. This causes confusion in
debugging since key is no longer visible. Unless someone knows how
kdumpctl script works, it is difficult to find out how kdump could be
loaded when there is no key on .ima keyring.
Remove deletion of kernel signing key once loaded. And then to prevent
multiple loading of same key when kdump service is disabled/enabled, update
key description field as well.
Suggested-by: Mimi Zohar <zohar@linux.ibm.com>
Signed-off-by: Nayna Jain <nayna@linux.ibm.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Currently, some tests failed with "The param /boot/boot/vmlinuz-xxx is
incorrect", for example,
[root@fedora kexec-tools]# shellspec spec/kdumpctl_manage_reset_spec.sh
Examples:
1) kdumpctl reset-crashkernel [--kernel] [--fadump] Test the kdump dump mode --kernel=ALL kdumpctl should warn the user that crashkernel has been udpated
When call reset_crashkernel --kernel=ALL
1.1) The error should include Updated crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M for kernel=/boot/vmlinuz-5.15.6-100.fc34.x86_64
expected "The param /boot/boot/vmlinuz-5.15.6-100.fc34.x86_64 is incorrect
The param /boot/boot/vmlinuz-5.15.6-100.fc34.x86_64 is incorrect
kdump: Updated crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M for kernel=/boot/boot/vmlinuz-5.15.6-100.fc34.x86_64. Please reboot the system for the change to take effect.
The param /boot/boot/vmlinuz-5.14.14-200.fc34.x86_64 is incorrect
The param /boot/boot/vmlinuz-5.14.14-200.fc34.x86_64 is incorrect
kdump: Updated crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M for kernel=/boot/boot/vmlinuz-5.14.14-200.fc34.x86_64. Please reboot the system for the change to take effect.
The param /boot/boot/vmlinuz-0-rescue-e986846f63134c7295458cf36300ba5b is incorrect
The param /boot/boot/vmlinuz-0-rescue-e986846f63134c7295458cf36300ba5b is incorrect
kdump: Updated crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M for kernel=/boot/boot/vmlinuz-0-rescue-e986846f63134c7295458cf36300ba5b. Please reboot the system for the change to take effect." to include "Updated crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M for kernel=/boot/vmlinuz-5.15.6-100.fc34.x86_64"
# spec/kdumpctl_reset_crashkernel_spec.sh:69
This happens because when a system has a boot partition, grubby
automatically prefixes a path with "/boot". The current boot loader
entries used for tests already has the prefix "/boot" in the path and
prefixing a path again will cause the above problem.
grubby uses "mountpoint -q /boot" to tell if there is a boot partition.
This patch mocks mountpoint so grubby knows the boot loader entries
are for a system without a boot partition.
Note this patch also avoids another error seen in the setup phase of the
test "The param /boot/vmlinuz-xxx is incorrect". I believe this error is
a bug of "grubby --update-kernel" in testing mode because running the
grubby in normal mode actually works and "grubby --info=/boot/vmlinuz-*"
also works in testing mode,
[root@fedora support]# grubby --no-etc-grub-update --grub2 --bad-image-okay --env=grub_env -b boot_load_entries --args crashkernel=333M --update-kernel=/boot/vmlinuz-5.15.6-100.fc34.x86_64
The param /boot/vmlinuz-5.15.6-100.fc34.x86_64 is incorrect
[root@fedora support]# grubby --no-etc-grub-update --grub2 --bad-image-okay --env=grub_env -b boot_load_entries --info=/boot/vmlinuz-5.15.6-100.fc34.x86_64
index=0
kernel="/boot/boot/vmlinuz-5.15.6-100.fc34.x86_64"
[root@fedora support]]# grubby --args crashkernel=333M --update-kernel=/boot/vmlinuz-6.0.7-301.fc37.x86_64 && echo "succeed"
succeed
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
The test case for parse_config creates a default kdump.conf in the pwd.
This fails when the pwd is read only. Thus move the default kdump.conf
to /tmp just like it is done for the "bad" kdump.conf. This also allows
to reuse the temporary file used for the "bad" case.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
The two systems are IA64 based which is no longer supported by Fedora
and was only supported in RHEL up to RHEL5. So it is safe to simply drop
the special handling. In case it is still wanted nevertheless the
special handling should be added to kdump-lib.sh:prepare_cmdline rather
than editing the sysconfig in the spec file.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
_crashkernel_add currently always assumes the good case, i.e. that the
value of the crashkernel parameter has the correct syntax and that the
delta added is a number. Both doesn't have to be true when the values
are provided by users. Thus add some additional checks.
Furthermore require the delta to have a explicit unit, i.e. no longer
assume that is in megabytes, i.e. 100 -> 100M.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
rpmbuild throws a warning with
line 80: It's not recommended to have unversioned Obsoletes: Obsoletes: diskdumputils netdump kexec-tools-eppic
In that diskdump and netdump were last used in RHEL4 and
kexec-tools-eppic was removed with Fedora 22. There is no supported
update path in which a current package could replace one of these three.
Thus simply drop the Obsoletes.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Currently the dracut modules are first prepared in a temporary directory
before they are moved to modules.d. All the preparation work can be done
by a single call to 'install' per file. Thus get rid off the indirection
and install the dracut modules directly to modules.d.
While at it merge the three macros to remove the prefix into one.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
When override_resettable was introduced in 2013 with 4b850d2 ("Check if
block device as dump target is resettable") it was forgotten to add the
new option to check_config (today the function is called parse_config).
So if a user would have set override_resettable check_config would have
returned an error ("Invalid kdump config option override_resettable")
and starting the kdump service would have failed. As there has been no
bug report in the last ~10 years it is safe to assume that the option
was never used. Thus simply remove the option.
Fixes: 4b850d2 ("Check if block device as dump target is resettable")
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
With multiple kernel variants on the same architecture, e.g. the 4k and
64k kernel on aarch64, we can no longer assume that the crashkernel
value for the currently running kernel will work for all installed
kernels. This also means that we can no longer update the grub config as
we don't know which value to set it to. Thus get the crashkernel value
for each kernel and stop updating the grub config.
While at it merge the _new_fadump and _fadump_val variables and remove
_read_kernel_arg_in_grub_etc_default which has no user.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Prevent the --fadump option to be used on non-PPC systems. This not only
prevents user errors but also guarantees that _dump_mode and _fadump_val are
empty on these systems.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
_update_kernel_cmdline handles two cmdline parameters at once. This does not
only make the function itself but also its callers more complicated than
necessary. For example in _update_crashkernel the fadump gets "updated" to
the value that has been read from grubby. Thus simplify
_update_kernel_cmdline to only update one parameter at once.
While at it shorten some variable named in the callers.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
condrestart is a left over from the time of SysVinit that is no longer
needed since the kexec-tools switched to systemd (10c91a1 ("Removing
sysvinit files") plus the one before). What's especially intriguing is
that from the beginning (0112f36 ("- Add a kdump sysconfig file and init
script - Spec file additions for pre/post install/uninstall")) the
sub-command never did any actual work (other than not returning an
error). Thus simply remove the condrestart sub-command.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
_get_current_running_kernel_path is identical to
_find_kernel_path_by_release $(uname -r) so simply use this instead of
defining a new function.
While at it simplify reset_crashkernel slightly. This changes the
behavior of the function for the case when KDUMP_KERNELVER is defined
but no kernel with this version is installed. Before, the missing
kernel is silently ignored and the currently running kernel is used
instead. Now, kdumpctl will exit with an error.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Only check whether modules for a given kernel version are installed
instead of searching for a kernel image. It's safer to assume that every
kernel uses kernel modules compared to that it follows certain naming
conventions. Furthermore it is much more lightweight and thus allows to
determine the KDUMP_KERNELVER much earlier for every command in
kdumpctl.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
There is value to use the function in other places as well. For example
it can be used to check whether optional dependencies, like grubby, are
installed. Thus make it more generic so it can be reused in later
commits.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
This includes fixes for
SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.
SC2005 (style): Useless echo? Instead of 'echo $(cmd)', just use 'cmd'.
SC2162 (info): read without -r will mangle backslashes.
SC2086 (info): Double quote to prevent globbing and word splitting.
SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
In addition add some source hints to prevent false positive findings.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
The fadump sysfs nodes /sys/kernel/fadump_[enabled|registered], have
been relocated to /sys/kernel/fadump/[enabled|registered] by kernel
commits d418b19f34ed ("powerpc/fadump: Reorganize /sys/kernel/fadump_*
sysfs files").
To ensure compatibility, symbolic links were added for each relocated
sysfs entry. Nonetheless, note that these symbolic links might be
removed later, as they have been deprecated by kernel commit
3f5f1f22ef10 ("Documentation/ABI: Mark /sys/kernel/fadump_* sysfs files
deprecated")
This patch updates the scripts to use the updated fadump sysfs files.
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
The size of the reserved memory in the functions show_reserved_mem,
check_crash_mem_reserved, and do_estimate are fetched from the sysfs
node `/sys/kernel/kexec_crash_size`. However, in the case of fadump,
the reserved area size is instead present in
/sys/kernel/fadump/mem_reserved.
For example:
$ kdumpctl showmem
kdump: Dump mode is fadump
kdump: Reserved 0MB memory for crash kernel
The above command showed 0MB for Reserved memory which is incorrect, the
actual reservation was 2048MB.
To resolve this issue a new helper function is introduced to fetch
reserved memory size based on the dump mode. For "fadump" mode,
it looks in `/sys/kernel/fadump/mem_reserved`, otherwise, it uses
`/sys/kernel/kexec_crash_size`. And all functions that previously
fetching reserved memory directly from `/sys/kernel/kexec_crash_size`
sysfs node are now updated to use this new function to get the reserved
memory size.
With the fix in place, the `kdumpctl showmem` command will now display
correct reserved memory size.
$ kdumpctl showmem
kdump: Dump mode is fadump
kdump: Reserved 2048MB memory for crash kernel
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
Reported-by: Sachin P Bappalige <sachinpb@linux.vnet.ibm.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
There is confusing message in /var/log/anaconda/packaging.log when installing
kexec-tools during the system installation on ppc64le:
Event Notification Registration successful (id: 1)
Make servicelog_notify slient when there are no erros.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
For kernel 64k variant, it terminates with substring 64k-debug, e.g.
vmlinuz-5.14.0-327.el9.aarch64+64k-debug.
Providing an extra matching pattern to filter out it.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The temporary directory is currently created under the current working
directory. That alone isn't ideal but works most of the time. However,
it will fail when the current working directory is not writable. So make
sure the directory is created within TMPDIR.
Fixes: ea00b7d ("kdumpctl: Move temp file in get_kernel_size to global temp dir")
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
And with commit t5b31b099 ("Simplify the management of the kernel
parameter crashkernel"), the default crashkernel value will be
used for the kernel. But the test VM has a RAM of 768M thus this is no
actual reserved memory for kdump. Even With the old crashkernel=224M,
network dumping tests like nfs-kdump will fail out of memory when
running against current Fedora Cloud images (>=F37).
This patch address the above two issues by
1. increasing the RAM of test VM to 1G
2. installing the kernel-modules which contains the squashfs module in
order to use the dracut squash module for kdump initrd.
Thanks to the dracut squash module, now even crashkernel=192M (the
default crashkernel value for RAM between 1G and 4G) works for
network dumping. Another benefit brought by this change is the default
crashkernel value can be tested as well.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
All the tests failed to run on the Fedora 37 host because the boot
partition failed to be mounted and in turn the key kernel cmdline
parameters like selinux=0 couldn't be added.
The root problem is somehow running lsblk on the second partition
returns an empty label unless we wait for enough time. Before figuring
out the root cause, simply skip check that the second partition
needs to have the boot label.
Note the root problem can be produced by building a test image,
cd tests
./scripts/build-image.sh Fedora-Cloud-Base-37-1.7.x86_64.qcow2 output_image scripts/build-scripts/base-image_test.sh
Source image is qcow2, using snapshot...
Formatting 'build/base-image1.building', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=5368709120 backing_file=Fedora-Cloud-Base-37-1.7.x86_64.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
It's a image with multiple partitions, using last partition as main partition
grep: /boot/grub2/grubenv: No such file or directory
grub2-editenv: error: cannot open `/boot/grub2/grubenv.new': No such file or directory.
/dev/nbd0 disconnected
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Tests failed to run against Fedora 37 or newer cloud images because of
the following error,
It's a image with multiple partitions, using last partition as main partition
'xxx/tests/build/x86_64/kexec-tools-2.0.26-5.fc37.src.rpm' not found
/dev/nbd0 disconnected
make: *** [Makefile:73: xxx/tests/output/test-base-image] Error 1
This is because starting with Fedora 37, rpm changes its API,
# Fedora >= 37
$ rpm -q --specfile kexec-tools.spec
kexec-tools-2.0.26-5.fc37.src
# Fedora 36
$ rpm -q --specfile kexec-tools.spec
kexec-tools-2.0.26-5.fc36
The tests depends on rpm to generate correct RPM name. Fix this issue by
removing the trailing .src from the output of "rpm -q --specfile".
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
After introducing 64k variant kernel on aarch64, an example kernel name
looks like "vmlinuz-5.14.0-316.el9.aarch64+64k". To match the plus
symbol, it demands an escape charater.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
On 4k and 64k kernels, the typical consumption values for SMMU are 36MB
and 384MB, respectively. Hence for 64k kernel, the consumption by smmu
should be taken into account carefully.
To do it by adding the extra 384MB value if installing a 64k kernel.
The upper limit value 384MB is calculated according to the formula in
the kernel smmu driver.
As for mlx5 network cards, it is measured by a pratical test, 200M for
64k variant, 150M for 4k variant
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
On aarch64, both 4K and 64K kernel can be installed, while they demand
different size reserved memory for kdump kernel.
'get_conf PAGE_SIZE' can not work if installing a 64K kernel when
running a 4K kernel. Hence resorting to the kernel release naming rules.
At present, the 64K kernel has the keyword '64k' in its suffix.
The base line for 64K is decided based on 4K. The diff 100M is picked up
since on a high end machine without smmu enabled, the diff of MemFree is
82M.
As for the smmu case, a huge difference in the memory consumption lies
between 64k and 4k driver. And it should be calculated separatedly.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
kdump_get_arch_recommend_crashkernel() expects the kernel version info,
while _update_kernel() provides the absolute path, which contains the
kernel version info.
This patch introduce a dedicated function parse_kver_from_path() to
extract the kernel info from the path
Credit to Philipp, who contributes the original code.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
This help function can manipulate the crashkernel cmdline by adding an
number for each item. Also a basic test case for _crashkernel_add() is
provided in this patch.
Credit to Philipp, who contributes the original code.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
The warning messages should not be included in the generated files.
Redirecting the warning for an unknown architecture to stderr.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
A space was added by mistake and unfortunately fips-mode-setup refuses
an extra parameter,
# fips-mode-setup --is-enabled 2 > /dev/null
# echo $?
2
# fips-mode-setup --is-enabled 2
Check, enable, or disable the system FIPS mode.
usage: /usr/bin/fips-mode-setup --enable|--disable [--no-bootcfg]
usage: /usr/bin/fips-mode-setup --check
usage: /usr/bin/fips-mode-setup --is-enabled
So in this case mkdumprd can never detect if FIPS is enabled. Fix this
mistake.
Fixes: 443a43e0 ("mkdumprd: call dracut with --add-device to install the drivers needed by /boot partition automatically for FIPS")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Currently, kexec-tools only updates the crashkernel to a new default
value only when both two conditions are met,
- auto_reset_crashkernel=yes in kdump.conf
- existing kernels or current running kernel should use the old default
value.
To address seen corner cases, the logic to tell if the second condition
is met becomes quite complex. Instead of making the logic more complex
to support aarch64-64k, this patch drops the second condition to
simplify the management of the crashkernel kernel parameter.
Another change brought by this simplification is kexec-tools will also
set up the kernel crashkernel parameter for a fresh install (previously
it's limited to osbuild).
Note
1. This patch also stop trying to update /etc/default/grub because
a) it only affects the static file /boot/grub2/grub.cfg
b) grubby is recommended to change the kernel command-line parameters
for both Fedora [1] and RHEL9 [2][3]
c) For the cases of aarch64 and POWER, different kernels could have
different default crashkernel value.
2. Starting with Fedora 37, posttrans rpm scriplet distinguish between
package install and upgrade.
[1] https://fedoraproject.org/wiki/GRUB_2
[2] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/managing_monitoring_and_updating_the_kernel/configuring-kernel-command-line-parameters_managing-monitoring-and-updating-the-kernel#changing-kernel-command-line-parameters-for-all-boot-entries_configuring-kernel-command-line-parameters
[3] https://access.redhat.com/solutions/1136173
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Currently, for non-s390x systems, the return code is 1 even when
_update_kernel_cmdline is correctly executed. This makes callers like
reset_crashkernel_after_update fail to print a message if a kernel has
its crashkernel updated. Fix it by put the code inside if block for
s390x.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Currently, kdump doesn't work on many FIPS-enabled systems including
Azure, ESXI, Hyper, POWER and etc. When FIPS is enabled, it needs to
access /boot//.vmlinuz-xxx.hmac to verify the integrity of the kernel.
However, on those systems, /boot fails to be mounted due to a lack of
fs and block device drivers and the system just halted after failing to
verify the integrity of the kernel. For example, on Hyper-V, sd_mod, sg,
scsi_transport_fc, hv_storvsc and hv_vmbus need to be installed in order
for /boot to be mounted.
mkdumprd calls dracut with the --no-hostonly-default-device. Following
the documentation (man dracut),
--no-hostonly-default-device
Do not generate implicit host devices like root, swap, fstab, etc.
Use "--mount" or "--add-device" to explicitly add devices as needed
this patch uses "--add-device" to explicitly add the device of /boot.
Note there is already an attempt to fix it in dracut's 01fips module
i.e. via the commit 83651776 ("fips: ensure fs module for /boot is
installed"). Unfortunately it only installs the file system driver e.g.
xfs.
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
On aarch64, a 64k kernel's name looks like:
vmlinuz-5.14.0-300.el9.aarch64+64k and the corresponding debug kernel's
name looks like: vmlinuz-5.14.0-300.el9.aarch64+64k-debug, which ends
with the suffix -debug instead of +debug.
Fix the matching pattern by [+|-]debug
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
The default systemd-boot installed kernels on fedora end up in the form:
/boot/efi/36b54597c46383/6.4.0-0.rc0.20230427git6e98b09da931.5.fc39.aarch64/linux
Where the kernel version is a directory containing the kernel (linux)
and the initrd. Thus _find_kernel_path_by release needs to be a bit less
strict and allow some futher characters on the grubby (really bootctl)
output.
Signed-off-by: Jeremy Linton <jeremy.linton@arm.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The new aggressive strip option was added in dracut 058, which tell
dracut to build the initramfs stripping more sections of the ELF
binaries (basically strip .symtab, .strtab).
These section are only useful for debugging runtime failures, but in
kdump kernel, neccessary tools for debug any runtime failure are
absent, there is no point keeping these sections.
Stripping these section can help save some memory with almost no side
effect. So let enable --aggressive-strip by default.
Comparison of unpacked initramfs before / after enabling aggressive strip:
du -hs image image.aggressive-strip
31M image
29M image.aggressive-strip
Signed-off-by: Kairui Song <kasong@tencent.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
A Unified Kernel Image (UKI) is a single EFI PE executable combining an
EFI stub, a kernel image, an initrd image, and the kernel command line.
They are defined in the Boot Loader Specification [1] as type #2
entries. UKIs have the advantage that all code as well as meta data that
is required to boot the system, not only the kernel image, is combined
in a single PE file and can be signed for EFI SecureBoot. This extends
the coverage of SecureBoot extensively.
For RHEL support for UKI were included into kernel-ark with 16c7e3ee836e
("redhat: Add sub-RPM with a EFI unified kernel image for virtual
machines").
There are two problems with UKIs from the kdump point of view at the
moment. First, they cannot be directly loaded via kexec_file_load and
second, the initrd included isn't suitable for kdump. In order to enable
kdump on systems with UKIs build the kdump initrd as usual and extract
the kernel image before loading the crash kernel.
[1] https://uapi-group.org/specifications/specs/boot_loader_specification/
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Others will need to use a temporary files, too. In order to avoid
potential clashes of multiple trap handlers move the local temp file
into a global temp dir.
While at it make sure that the trap handler returns the correct exit
code.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The function is only used in do_estimate. Move it to kdumpctl to
prevent confusion.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The function is pretty broken right now. To start with the -o/--omit
option allows a quoted, space separated list of modules. But using 'set'
breaks quotation and thus only considers the first element in the list.
Furthermore dracut uses getopt internally. This means that it is also
possible to pass the list via --omit=.
Fix the function by making use of getopt for parsing the dracut_args.
While at it also add a test cases to cover the functions.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The function is only used in kdumpctl. Thus move it there to keep
kdump-lib small and simple.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The function isn't used anywhere. Thus remove it to keep kdump-lib small
and simple.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The function isn't used anywhere. Thus remove it to keep kdump-lib small
and simple.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
With the NetworkManager rewrite this function in no longer used. This
also allows to remove a lot of dead code in kdump-lib.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The function isn't used anywhere. Thus remove it to keep
kdump-lib-initramfs small and simple.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
If the device/mountpoint for findmnt is omitted findmnt will list all
mounted filesystems. In that case it will always return "true". So
explicitly check if an argument was passed to prevent false-positives.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Relates: https://bugzilla.redhat.com/show_bug.cgi?id=2151504
Currently, when the network isn't ready, kdump would repeatedly print
the same info,
[ 29.537230] kdump[671]: Bad kdump network destination: 192.123.1.21
[ 30.559418] kdump[679]: Bad kdump network destination: 192.123.1.21
[ 31.580189] kdump[687]: Bad kdump network destination: 192.123.1.21
This is not user-friendly and users may think kdump has got stuck. So
also show much time has waited for the network to be ready,
[ 29.546258] kdump[673]: Waiting for network to be ready (50s / 10min)
...
[ 32.608967] kdump[697]: Waiting for network to be ready (56s / 10min)
Note kdump_get_ip_route no longer prints an error message and it's up to
the caller to determine the log level and print relevant messages. And
kdump_collect_netif_usage aborts when kdump_get_ip_route fails.
Reported-by: Martin Pitt <mpitt@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2151504
When a NetworManager connection profile contains a colon in the name,
"nmcli --get-values UUID,FILENAME" by default would escape the colon
because a colon is also used for separating the values. In this case,
99kdumpbase fails to get the correct connection profile path,
kdumpctl[5439]: cp: cannot stat '/run/NetworkManager/system-connections/static-52\\\:54\\\:01.nmconnection': No such file or directory
kdumpctl[5440]: sed: can't read /tmp/1977-DRACUT_KDUMP_NM/ifcfg-static-52-54-01: No such file or directory
kdumpctl[5449]: dracut-install: ERROR: installing '/tmp/1977-DRACUT_KDUMP_NM/ifcfg-static-52-54-01' to '/etc/NetworkManager/system-connections/ifcfg-static-52-54-01'
As a result, dumping vmcore to a remote nfs would fail.
In our case of getting connection profile path, there is no need to escape the
colon so pass "-escape no" to nmcli,
[root@localhost ~]# nmcli --get-values UUID,FILENAME c show
659e09c1-a6bd-3549-9be4-a07a1a9a8ffd:/etc/NetworkManager/system-connections/aa\:bb.nmconnection
[root@localhost ~]# nmcli -escape no --get-values UUID,FILENAME c show
659e09c1-a6bd-3549-9be4-a07a1a9a8ffd:/etc/NetworkManager/system-connections/aa:bb.nmconnection
Suggested-by: Beniamino Galvani <bgalvani@redhat.com>
Reported-by: Martin Pitt <mpitt@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Although upgrading the kernel with `rpm -Uvh` is not recommended, the
kexec-tools plugin prints confusing error logs when a customer upgrades the
kernel through it.
```
kdump: kernel 5.14.0-80.el9.x86_64 doesn't exist
kdump: Couldn't find current running kernel
```
Not finding the currently running kernel will only make kdump unable to copy the
grub entry parameters to the newly installed kernel, so lower the log level.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2140721
Currently, if users specify dumping to nfsv4 target via
dracut_args --mount "<NFS-server-ip>:/var/crash /mnt nfs defaults"
it fails with the following errors,
[ 5.159760] mount[446]: mount.nfs: Protocol not supported
[ 5.164502] systemd[1]: mnt.mount: Mount process exited, code=exited, status=32/n/a
[ 5.167616] systemd[1]: mnt.mount: Failed with result 'exit-code'.
[FAILED] Failed to mount /mnt.
This is because nfsv4-releted drivers are not installed to kdump initrd.
mkdumprd calls dracut with "--hostonly-mode strict". If nfsv4-related
drivers aren't loaded before calling dracut, they won't be installed.
When users specify nfs dumping via dracut_args, kexec-tools won't mount
the nfs fs beforehand hence nfsv4-related drivers won't be installed.
Note dracut only installs the nfs driver i.e. nfsv3 driver for "--mount
... nfs". So also install nfsv4-related drivers when users specify nfs
dumping via dracut_args. Since nfs_layout_nfsv41_files depends on nfsv4,
the nfsv4 driver will be installed automatically.
As for the reason why we support nfs dumping via dracut_args instead of
asking user to use the nfs directive, please refer to commit 74c6f464
("Support special mount information via 'dracut_args'").
Fixes: 4eedcae5 ("dracut-module-setup.sh: don't include multipath-hostonly")
Reported-by: rcheerla@redhat.com
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Probing unnecessary I/O devices wastes memory and in extreme cases can
cause the crashkernel to run OOM. That's why the s390-tools maintain
their own module, 95zdev-kdump [1], that disables auto LUN scanning and
only configures zfcp devices that can be used as dump target. So remove
zfcp.allow_lun_scan from the kernel command line to prevent that we
accidentally overwrite the default set by the module.
[1] https://github.com/ibm-s390-linux/s390-tools/blob/master/zdev/dracut/95zdev-kdump/module-setup.sh
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
`uname -m` was used by mistake. As a result, kexec-tools failed to
update crashkernel=auto during in-place upgrade from RHEL8 to RHEL9.
`uname -m` should be used to get architecture instead.
Fixes: 5951b5e2 ("Don't try to update crashkernel when bootloader is not installed")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Lichen Liu <lichliu@redhat.com>
At the beginning of do_estimate it currently checks whether the
TARGET_INITRD exists and if not fails with an error message. This not
only requires the user to manually trigger the build of the initrd but
also ignores all cases where the TARGET_INITRD exists but need to be
rebuild. For example when there were changes to kdump.conf or when the
system switches from kdump to fadump. All these changes will impact the
outcome of do_estimate. Thus properly check whether the initrd needs to
be rebuild and if it does trigger the rebuild automatically.
To do so move the check whether the TARGET_INITRD has fadump enabled to
is_system_modified and call this function. With this force_(no_)rebuild
options in kdump.conf are ignored to avoid unnecessary rebuilds.
While at it cleanup check_system_modified and rename it to
is_system_modified. Furthermore move printing the info that the initrd
gets rebuild to rebuild_initrd to avoid every caller has the same line.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
check_rebuild uses a bunch of local variables to store the result of the
different checks performed. At the end of the function it then evaluates
which check failed to print an appropriate info and trigger a rebuild if
needed. This not only makes the function hard to read but also requires
all checks to be executed even if an earlier one already determined that
the initrd needs to be rebuild. Thus refractor check_rebuild such that
it only checks whether the initrd needs to rebuild and trigger the
rebuild by the caller (if needed). While at it rename the function to
need_initrd_rebuild.
Furthermore also move setup_initrd to the caller so it is more consisted
with the other users of the function.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Like for 'start' move the printing of the error message to the calling
function. This not only makes the code more consistent to 'start' but
also prevents 'kdumpctl restart' to call 'start' in case 'stop' has
failed. This doesn't impact the case when 'kdumpctl restart' is run
without any crash kernel being loaded as kexec will still return success
in that case.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The function has many block of the kind
if ! cmd; then
derror "Starting kdump: [FAILED]"
return 1
fi
This duplicates code and makes the function hard to read. Thus move the
block to the calling function.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
A recently added unit test found that prepare_cmdline has several
problems. For example an empty remove list will remove all spaces or
when the cmdline contains a parameter with quoted values containing
spaces will only remove the beginning up to the first space. Furthermore
the old design requires lots of subshells and pipes.
This patch rewrites prepare_cmdline in a way that makes the unit test
happy and tries to use as many bash built-ins as possible.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Move the workaround for aws graviton cpus from load_kdump to
prepare_cmdline. This (1) makes the workaround available also for other
callers of prepare_cmdline (although not needed at the moment) and (2)
makes it easier to fix the problems found by the unit test included
earlier as all changes to the cmdline are done at one place now.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
prepare_cmdline is totally broken. For example if the remove list ($2)
is empty it removes all white spaces or if a parameter has a quoted
value containing a white space it only removes the first part of the
parameter up to the first space. Thus add a test case that shows what the
function should do in order to fix it in subsequent patches.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
is_squash_available is only used in dracut-module-setup.sh and mkdumprd.
Neither of the two scripts calls prepare_kdump_bootinfo which determines
and sets KDUMP_KERNELVER. Thus KDUMP_KERNELVER is only non-zero if it
explicitly specified by the user in /etc/sysconfig/kdump (and the file
gets sourced, which is not the case for drachu-module-setup.sh).
In theory this can even lead to bugs. For example consider the case when
a debug kernel is running. In that case kdumpctl will try to use the
non-debug version of the kernel while is_squash_available will make its
decision based on the debug version. So in case the debug kernel has
squash available but the non-debug kernel doesn't mkdumprd will try to
add it nevertheless.
Thus factor out the kernel version detection from prepare_kdump_bootinfo
and make use of the new function when checking for the availability of
those kernel modules.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
mkfadumprd doesn't call prepare_kdump_bootinfo from kdump-lib.sh. Thus
both KDUMP_KERNELVER and DEFAULT_INITRD are always empty. Simply remove
them from the debug print.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Both functions are almost identical. The only differences are (1) the
sysfs node the status is read from and (2) the fact the fadump version
doesn't verify if the file it's trying to read actually exists. Thus
merge the two functions and get rid of the check_current_status wrapper.
While at it rename the function to is_kernel_loaded which explains
better what the function does.
Finally, after moving FADUMP_REGISTER_SYS_NODE shellcheck can no longer
access the definition and starts complaining about it not being quoted.
Thus quote all uses of FADUMP_REGISTER_SYS_NODE.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Currently dracut-early-kdump.sh claims to be POSIX compliant but it
sources kdump-lib.sh which uses bash-only syntax. Thus require bash for
dracut-early-kdump.sh as well.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Fix the following issues found by shellcheck. For the second one disable
shellcheck as EARLY_KEXEC_ARGS can contain multiple arguments.
In dracut-early-kdump.sh line 9:
EARLY_KDUMP_KERNELVER=""
^-------------------^ SC2034: EARLY_KDUMP_KERNELVER appears unused. Verify use (or export if used externally).
In dracut-early-kdump.sh line 61:
if $KEXEC $EARLY_KEXEC_ARGS $standard_kexec_args \
^---------------^ SC2086: Double quote to prevent globbing and word splitting.
For more information:
https://www.shellcheck.net/wiki/SC2034https://www.shellcheck.net/wiki/SC2086
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
With the deprecation of the 'default' option in kdump.conf
check_failure_action_config needed to track which option was used
(default or failure_action). This made the function quite complex.Thus
make option 'default' a true alias of 'failure_action' when parsing
kdump.conf and simplify check_failure_action_config.
Do the same simplifications for check_final_action_config as both
functions are basically identical.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Resolves: bz2155754
Upstream: https://github.com/makedumpfile/makedumpfile
Conflict: None
commit 5f17bdd2128998a3eeeb4521d136a192222fadb6
Author: Kazuhito Hagio <k-hagio-ab@nec.com>
Date: Wed Dec 21 11:06:39 2022 +0900
[PATCH] Fix wrong exclusion of slab pages on Linux 6.2-rc1
* Required for kernel 6.2
Kernel commit 130d4df57390 ("mm/sl[au]b: rearrange struct slab fields to
allow larger rcu_head"), which is contained in Linux 6.2-rc1 and later,
made the offset of slab.slabs equal to page.mapping's one. As a result,
"makedumpfile -d 8", which should exclude user data, excludes some slab
pages wrongly because isAnon() returns true when slab.slabs is an odd
number. With such dumpfiles, crash can fail to start session with an
error like this:
# crash vmlinux dumpfile
...
crash: page excluded: kernel virtual address: ffff8fa047ac2fe8 type: "xa_node shift"
Make isAnon() check that the page is not slab to fix this.
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Reported-by: Baoquan He <bhe@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Currently when using anaconda to install the OS, the following errors
occur,
INF packaging: Configuring (running scriptlet for): kernel-core-5.14.0-70.el9.x86_64 ...
INF dnf.rpm: grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
...
INF packaging: Configuring (running scriptlet for): kexec-tools-2.0.23-9.el9.x86_64 ...
INF dnf.rpm: grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
Or for s390, the following errors occur,
INF packaging: Configuring (running scriptlet for): kernel-core-5.14.0-71.el9.s390x ...
03:37:51,232 INF dnf.rpm: grep: /etc/zipl.conf: No such file or directory
grep: /etc/zipl.conf: No such file or directory
grep: /etc/zipl.conf: No such file or directory
INF packaging: Configuring (running scriptlet for): kexec-tools-2.0.23-9_1.el9_0.s390x ...
INF dnf.rpm: grep: /etc/zipl.conf: No such file or directory
This is because when anaconda installs the packages, bootloader hasn't
been installed and /boot/grub2/grubenv or /etc/zipl.conf doesn't exist.
So don't try to update crashkernel when bootloader isn't ready to avoid
the above errors.
Note this is the second attempt to fix this issue. Previously a file
/tmp/kexec_tools_package_install was created to avoid running the
related code thus to avoid the above errors but unfortunately that
approach has two issues a) somehow osbuild doesn't delete it for RHEL b)
this file could still exist if users manually remove kexec-tools.
Fixes: e218128 ("Only try to reset crashkernel for osbuild during package install")
Reported-by: Jan Stodola <jstodola@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Hyper-V VM with accelerated networking
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2151842
Currently, vmcore dumping to remote fs fails on Azure Hyper-V VM with
accelerated networking because it uses a physical NIC for accrelarated
networking [1]. In this case, the driver for this physical NIC should be
installed as well.
[1] https://learn.microsoft.com/en-us/azure/virtual-network/accelerated-networking-overview
Fixes: a65dde2d ("Reduce kdump memory consumption by only installing needed NIC drivers")
Reported-by: Xiaoqiang Xiong <xxiong@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
interface
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2151500
Currently, kdump initrd fails to be built when dumping vmcore to
localhost via ssh or nfs,
kdumpctl[3331]: Cannot get driver information: Operation not supported
kdumpctl[1991]: dracut: Failed to get the driver of lo
dracut[2020]: Failed to get the driver of lo
kdumpctl[1775]: kdump: mkdumprd: failed to make kdump initrd
kdumpctl[1775]: kdump: Starting kdump: [FAILED]
systemd[1]: kdump.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: kdump.service: Failed with result 'exit-code'.
systemd[1]: Failed to start Crash recovery kernel arming.
systemd[1]: kdump.service: Consumed 1.710s CPU time.
This is because the loopback interface is used for transferring vmcore and
ethtool can't get the driver of the loopback interface. In fact, once
COFNIG_NET is enabled, the loopback device is enabled and there is no driver
for the loopback device. So skip installing driver for the loopback device.
The loopback interface is implemented in linux/drivers/net/loopback.c
and always has the name "lo". So we can safely tell if a network
interface is the loopback interface by its name.
Fixes: a65dde2d ("Reduce kdump memory consumption by only installing needed NIC drivers")
Reported-by: Martin Pitt <mpitt@redhat.com>
Reported-by: Rich Megginson <rmeggins@redhat.com>
Reviewed-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2149246
Latest Workstation live x86_64 image has an excess increase of ~300 MB
in size. This is because kdumpbase module's trap handler overwrites
dracut's handler and DRACUT_TMPDIR which has three unpacked initramfs
files fails to be cleaned up. This patch moves kdumpbase module's
temporary folder under DRACUT_TMPDIR and lets dracut's trap handler do
the cleanup instead.
Fixes: d25b1ee3 ("Add functions to copy NetworkManage connection profiles to the initramfs")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
With kernel commit 607451ce0aa9b ("powerpc/fadump: register for fadump
as early as possible"), 'kdumpctl start' prematurely returns with the
below message:
"Kdump already running: [WARNING]"
instead of setting default initrd with dump capture capability as
required for fadump. Skip status check in fadump mode to avoid this
problem.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Kdump service will create fadump initramfs when needed, but it won't
clean up the fadump initramfs on kernel uninstall. So create a kernel
install hook to do the clean up job.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
In case of fadump, default initrd is rebuilt with dump capturing
capability, as the same initrd is used for booting production kernel
as well as capture kernel.
The original initrd file is backed up with a checksum, to restore
it as the default initrd when fadump is disabled. As the checksum
file is not kernel version specific, switching between different
kernel versions and kdump/fadump dump mode breaks the default initrd
backup/restore logic. Fix this by having a kernel version specific
checksum file.
Also, if backing up initrd fails, retaining the checksum file isn't
useful. Remove it.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
If available, use 'zstd' compression method to optimize the size of
the initrd built with fadump support. Also, 'squash+zstd' is not
preferred because more disk space is consumed with 'squash+zstd' due
to the additional binaries needed for fadump with squash case.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Currently _find_kernel_path_by_release uses grubby and grep to
find the kernel path, if both the normal kernel and it's debug
varient exist, the grep will give more than one kernel strings.
```
kernel="/boot/vmlinuz-5.14.0-139.kpq0.el9.s390x+debug"
kernel="/boot/vmlinuz-5.14.0-139.kpq0.el9.s390x"
```
This will cause an error when installing debug kernel.
```
The param "/boot/vmlinuz-5.14.0-139.kpq0.el9.s390x+debug
/boot/vmlinuz-5.14.0-139.kpq0.el9.s390x" is incorrect
```
Fixes: 945cbbd ("add helper functions to get kernel path by kernel release and the path of current running kernel")
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Now, kdump.conf is generated by gen-kdump-conf.sh, hence adapting
check_config to run that script firstly then check the generated file.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
This commit has the same motivation as the commit 677da8a "sysconfig:
use a simple generator script to maintain".
At present, only the kdump.conf generated for s390x has a slight
difference from the other arches, where the core_collector asks the
makedumpfile to use "-c" option to compress dump data by each page using
zlib, which is more efficient than lzo on s390x.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
network interfaces
Currently, dumping to iSCSI target fails because the global array
(unique_netifs) that stores the network interfaces needed by kdump is
empty. The root cause is change of the array made in a subshell (a child
process) is inaccessible to the parent process. So don't run
kdump_check_setup_iscsi in a subshell.
Fixes: 63c3805c ("Set up kdump network by directly copying NM connection profile to initrd")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
/usr/lib/udev/ccw_init [1] shipped by s390utils extracts the values of
SUBCHANNELS, NETTYPE and LAYER2 from /etc/sysconfig/network-scripts/ifcfg-*
or /etc/NetworkManager/system-connections/*.nmconnection to activate znet
network device. If the connection profile is copied to initrd,
there is no need to set up the "rd.znet" dracut cmdline parameter.
There are two cases addressed by this commit,
1. znet network interface is a slave of bonding/teaming/vlan/bridging
network. The connection profile has been copied to initrd by
kdump_copy_nmconnection_file and it contains the info needed by
ccw_init.
2. znet network interface is a slave of bonding/teaming/vlan/bridging
network. The corresponding ifcfg-*/*.nmconnection file may not contain
info like SUBCHANNELS [2]. In this case, copy the ifcfg-*/*.nmconnection
file that has this info to the kdump initrd. Also to prevent the copied
connection profile from being chosen by NM, set
connection.autoconnect=false for this connection profile.
With this implementation, there is also no need to check if znet is
used beforehand.
Note
1. ccw_init doesn't care if SUBCHANNELS, NETTYPE and LAYER2 comes from
an active NM profile or not. If an inactive NM profile contains this
info, it needs to be copied to the kdump initrd as well.
2. "rd.znet_ifname=$_netdev:${SUBCHANNELS}" is no longer needed needed
because now there is no renaming of s390x network interfaces when
reusing NetworkManager profiles. rd.znet_ifname was introduced in
commit ce0305d ("Add a new option 'rd.znet_ifname' in order to use it
in udev rules") to address the special case of non-persistent
MAC address by renaming a network interface by SUBCHANNELS.
[1] https://src.fedoraproject.org/rpms/s390utils/blob/rawhide/f/ccw_init
[2] https://bugzilla.redhat.com/show_bug.cgi?id=2064708
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Thomas Haller <thaller@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
nm-wait-online-initrd.service installed by dracut's 35-networkmanager
module calls nm-online with "-s" which means it returns immediately when
NetworkManager logs "startup complete". Thus it doesn't truly wait for
network connectivity to be established [1]. Wait for the network to be
truly ready before dumping vmcore. There are two benefits brought by
this approach,
- ssh/nfs dumping won't fail because of that the network is not
ready e.g. [2][3]
- users don't need to use workarounds like rd.net.carrier.timeout to
make sure the network is ready
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1485712
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1909014
[3] https://bugzilla.redhat.com/show_bug.cgi?id=2035451
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Thomas Haller <thaller@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
A NIC may get a different name in the kdump kernel from 1st kernel
in cases like,
- kernel assigned network interface names are not persistent e.g. [1]
- there is an udev rule to rename the NIC in the 1st kernel but the
kdump initrd may not have that rule e.g. [2]
If NM tries to match a NIC with a connection profile based on NIC name
i.e. connection.interface-name, it will fail the above bases. A simple
solution is to ask NM to match a connection profile by MAC address.
Note we don't need to do this for user-created NICs like vlan, bridge and
bond.
An remaining issue is passing the name of a NIC via the kdumpnic dracut
command line parameter which requires passing ifname=<interface>:<MAC> to
have fixed NIC name. But we can simply drop this requirement. kdumpnic
is needed because kdump needs to get the IP by NIC name and use the IP
to created a dumping folder named "{IP}-{DATE}". We can simply pass the
IP to the kdump kernel directly via a new dracut command line parameter
kdumpip instead. In addition to the benefit of simplifying the code,
there are other three benefits brought by this approach,
- make use of whatever network to transfer the vmcore. Because as long
as we have the network to we don't care which NIC is active.
- if obtained IP in the kdump kernel is different from the one in the
1st kernel. "{IP}-{DATE}" would better tell where the dumped vmcore
comes from.
- without passing ifname=<interface>:<MAC> to kdump initrd, the
issue of there are two interfaces with the same MAC address for
Azure Hyper-V NIC SR-IOV [3] is resolved automatically.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1121778
[2] https://bugzilla.redhat.com/show_bug.cgi?id=810107
[3] https://bugzilla.redhat.com/show_bug.cgi?id=1962421
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Thomas Haller <thaller@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
By default, NetworkManger will manage all the network interfaces and
try to set interface IFF_UP to get carrier state. Regardless of whether
the network interface is connected to a cable or not, the NIC driver
will allocate memory resources for e.g. ring buffers when setting IFF_UP.
This could be a waste of memory. For example it's found i40e consumes ~15GB
on a power machine. On this machine, i40e manages four interfaces but only
one interface is valid. This patch use "managed=false" to tell
NetworkManager to not manage network interfaces that are not needed by
kdump by putting 10-kdump-netif_allowlist.conf in the initramfs.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Thomas Haller <thaller@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
This patch setup kdump network by directly copying NM connection profile(s)
for different network setup including bond, bridge, vlan, and team. For
vlan network, rename phydev to parent_netif to improve code readability.
With the new approach, the related code to build up dracut cmdline
parameter such rd.route, ip and etc can be cleaned up. And there is no
need to setup dns when copying .nmconnection directly to initrd
either. Note the bootdev dracut command line parameter is only used by
dracut's 35network-legacy and network-manager doesn't use it, remove
related code as well.
Note
1. kdump_setup_vlan/bond/... are no longer called in subshells in order
to modify global variables like unique_netifs
2. The original kdump_install_net is renamed to better reflect its
current function
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Thomas Haller <thaller@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
kexec-tools depends on dracut's 35network-manager module which will
call nm-initrd-generator. We don't want nm-initrd-generator to generate
connection profiles since we will copy them from 1st kernel to
kdump kernel initramfs. NetworkManager >= 1.35.2 won't generate connection
profiles if there's a connection dir with rd.neednet. For Fedora/RHEL,
this connection dir is /etc/NetworkManager/system-connections. For the
details, please refer to the NetworkManager commit 79885656d3
("initrd: don't add a connection if there's a connection dir with
rd.neednet") [1]. Before the release of NetworkManager >= 1.35.2, we
need to mask /usr/libexec/nm-initrd-generator.
[1] https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1010
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Thomas Haller <thaller@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
nm-wait-online-initrd.service installed by dracut's 35-networkmanager
module calls nm-online with "-s" which means it returns immediately when
NetworkManager logs "startup complete" after certain timeouts are
reached. "startup complete" doesn't necessarily network connectivity has
been established. nm-initrd-generator has a set of timeouts that in most
of cases when applied, "startup-complete" means network connectivity has
been established. So apply it when setting up kdump network.
Suggested-by: Thomas Haller <thaller@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Thomas Haller <thaller@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
According to `man nm-online`,
"By default, connections have the ipv4.may-fail and
ipv6.may-fail properties set to yes; this means that
NetworkManager waits for one of the two address families to
complete configuration before considering the connection
activated. If you need a specific address family configured
before network-online.target is reached, set the corresponding
may-fail property to no."
If a NIC has an IPv4 or IPv6 address, set the corresponding may-fail
property to no. Otherwise, dumping vmcore over IPv6 could fail because
only IPv4 network is ready or vice versa.
Also disable IPv6 if only IPv4 is used and vice versa.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Thomas Haller <thaller@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Each network interface is manged by a NM connection. Given a list of
network interface names, copy the NetworkManager (NM) connection
profiles i.e. .nmconnection files to the kdump initramfs.
Before copying a connection file, clone it to automatically convert a
legacy ifcfg-*[1] file to a .nmconnection file and for the convenience of
editing the connection profile.
[1] https://fedoraproject.org/wiki/Changes/NetworkManager_keyfile_instead_of_ifcfg_rh
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Thomas Haller <thaller@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
6f9235887f ("module-setup.sh: enable
vlan on team interface") skips establishing teaming network by mistake.
Although it could use one of slave netifs to establish connection
to transfer vmcore to remote fs, it breaks the implicit assumption of
creating an identical network topology to the 1st kernel.
Fixes: 6f92358 ("module-setup.sh: enable vlan on team interface")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Thomas Haller <thaller@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Currently, kexec-tools tries to reset crashkernel when using anaconda to
install the system. But grubby isn't ready and complains that,
10:34:17,014 INF packaging: Configuring (running scriptlet for): kexec-tools-2.0.23-9.el9.x86_64 1646034766 53ff7158f8808774f4e3c3c87e504aa7a6d677b537754dac86c87925c8f0a397
10:34:17,205 INF dnf.rpm: grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
kexec-tools is supposed to update the kernel crashkernel parameter after
package upgrade. Unfortunately, the posttrans RPM scriptlet doesn't
distinguish between package install and upgrade. This patch skips
reset_crashkernel_after_update as similar to e218128e ("Only try to
reset crashkernel for osbuild during package install").
Reported-by: Jan Stodola <jstodola@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
When the dump target is lvm2 thinp, if we didn't mount
the dump target first, get_fs_type_from_target will get
empty output:
Before mount:
$ get_fs_type_from_target /dev/vg00/thinlv
After mount:
$ mount /dev/vg00/thinlv /mnt
$ get_fs_type_from_target /dev/vg00/thinlv
ext4
As a result, kdumpctl start will fail with:
$ kdumpctl start
kdump: Dump target is invalid
kdump: Starting kdump: [FAILED]
This patch fix the issue by bypassing check_fs_modified
when the dump target is lvm2 thinp.
Signed-off-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <prudo@redhat.com>
F36 has dropped support on ifcfg and as a result current network tests
fails. Use .nmconnection to set up test network instead.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
With commit fa9201b2 ("fadump: isolate fadump initramfs image within
the default one"), initramfs image gets to hold two images, one for
production kernel boot purpose and the other for capture kernel boot.
Most files are common among the two images. Retain file modification
time to replace duplicate files with hardlinks and save space. Also,
avoid unnecessarily compressing fadump image that is decompressed
immediately anyway.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
With commit fa9201b2 ("fadump: isolate fadump initramfs image within
the default one"), initramfs image gets to hold two squash images, one
for production kernel boot purpose and the other for capture kernel
boot. Having separate images improved reliability for both production
kernel and capture kernel boot scenarios, but the size of initramfs
image became considerably larger.
Instead of having squash images, compressing $initdir without using
squash images reduced the size of initramfs image for fadump case by
around 30%. So, avoid using squash for fadump case.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Previously, all files within $TESTCASEDIR/$test_case are regarded
as shell script files for testing. However there might be config
files under the directory. So let's only iterate the .sh files.
Signed-off-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
The 80lvmthinpool-monitor module is needed for monitor and
autoextend the size of thin pool in 2nd kernel. The module was
integrated in dracut version 057.
If lvmthinpool-monitor module is not found, we will print a warning.
Because we don't want to block the kdump process when the thin pool
capacity is enough and no monitor-and-autoextend actually needed.
Signed-off-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
lvm2 relies on /etc/lvm/lvm.conf to determine its behaviour. The
important configs such as thin_pool_autoextend_threshold and
thin_pool_autoextend_percent will be used during kdump in 2nd
kernel. So if the file is modified, the initramfs should be
rebuild to include the latest.
Signed-off-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
We need to check if a directory or a device is lvm2 thinp target.
First, we use get_block_dump_target() to convert dump path into
block device, then we check if the device is lvm2 thinp target by
cmd lvs.
is_lvm2_thinp_device is now located in kdump-lib-initramfs.sh, for it
will be used in 2nd kernel. is_lvm2_thinp_dump_target is located in
kdump-lib.sh, for it is only used in 1st kernel, and it has dependencies
which exist in kdump-lib.sh.
Signed-off-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Currently, updating kexec-tools on s390x gives the warning
sed: can't read /etc/default/grub: No such file or directory
This happens because s390x doesn't use GRUB and /etc/default/grub
doesn't exist. We need to skip both reading and writing to
/etc/default/grub.
Reported-by: Jie Li <jieli@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
memory requirement for LUKS-encrypted target
When there are multiple key slots, "kdumpctl estimate" uses the least
memory-consuming key slot. For example, when there are two memory slots
created with --pbkdf-memory=1048576 (1G) and --pbkdf-memory=524288 (512M),
"kdumpctl estimate" thinks the extra memory requirement is only 512M.
This will of course lead to OOM if the user uses the more
memory-consuming key slot. Fix it by sorting in reverse order.
Fixes: e9e6a2c ("kdumpctl: Add kdumpctl estimate")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Lichen Liu <lichliu@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2060319
Currently, kexec-tools tries to reset crashkernel when using anaconda to
install the system. But grubby isn't ready and complains that,
10:33:17,631 INF packaging: Configuring (running scriptlet for): kernel-core-5.14.0-70.el9.x86_64 1645746534 03dcd32db234b72440ee6764d59b32347c5f0cd98ac3fb55beb47214a76f33b4
10:34:16,696 INF dnf.rpm: grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
We only need to try resetting crashkernel for osbuild. Skip it for other
cases. To tell if it's package install instead of package upgrade, make
use of %pre to write a file /tmp/kexec-tools-install when "$1 == 1" [1].
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/#_syntax
Reported-by: Jan Stodola <jstodola@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Lichen Liu <lichenliu@redhat.com>
Previously kexec-tools will pass "--compress zstd" to dracut. It
will make dracut to decide whether: a) call mksquashfs to make a
zstd format squash-root.img, b) call cmd zstd to make a initramfs.
Since dracut(>= 057) has decoupled the compressor for dracut and
dracut-squash, So in this patch, we will pass the compressor seperately.
Note:
The is_squash_available && !dracut_has_option --squash-compressor
&& !is_zsdt_command_available case is left unprocessed on purpose.
Actually, the situation when we want to call zstd compression is:
1) If squash function OK, we want dracut to invoke mksquashfs to make
a zstd format squash-root.img within initramfs.
2) If squash function is not OK, and cmd zstd presents, we want dracut
to invoke cmd zstd to make a zstd format initramfs.
is_zstd_command_available check can handle case 2 completely.
However, for the is_squash_available check, it cannot handle case 1
completely. It only checks if the kernel supports squashfs, it doesn't
check whether the squash module has been added by dracut when making
initramfs. In fact, in kexec-tools we are unable to do the check,
there are multiple ways to forbit dracut to load a module, such as
"dracut -o module" and "omit_dracutmodules in dracut.conf".
When squash dracut module is omitted, is_squash_available check will
still pass, so "--compress zstd" will be appended to dracut cmdline,
and it will call cmd zstd to do the compression. However cmd zstd may
not exist, so it fails.
The previous "--compress zstd" is ambiguous, after the intro of
"--squash-compressor", "--squash-compressor" only effect for
mksquashfs and "--compress" only effect for specific cmd.
So for the is_squash_available && !dracut_has_option
--squash-compressor && !is_zsdt_command_available case, we just leave
it to be handled the default way.
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Previously the sync for dump_fs is problematic, it always
return success according to man 2 sync. So it cannot detect
the error of the dump target is full and not all of vmcore
data been written back the disk, which will leave the vmcore
imcomplete and report misleading log as "saving vmcore
complete".
In this patch, we will use "sync -f vmcore" instead, which
will return error if syncfs on the dump target fails. In
this way, vmcore sync related failures, such as autoextend
of lvm2 thinpool fails, can be detected and handled properly.
Signed-off-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
This patch add virtiofs support for kexec-tools by introducing a new option
for /etc/kdump.conf:
virtiofs myfs
Where myfs is a variable tag name specified in qemu cmdline
"-device vhost-user-fs-pci,tag=myfs".
The patch covers the following cases:
1) Dumping VM's vmcore to a virtiofs shared directory;
2) When the VM's rootfs is a virtiofs shared directory and dumping the
VM's vmcore to its subdirectory, such as /var/crash;
3) The combination of case 1 & 2: The VM's rootfs is a virtiofs shared
directory and dumping the VM's vmcore to another virtiofs shared
directory.
Case 2 & 3 need dracut >= 057, otherwise VM cannot boot from virtiofs
shared rootfs. But it is not the issue of kexec-tools.
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Since commit c5bdd2d8f1 ("kdump-lib: use non-debug kernels first"),
non-debug kernel is preferred, over the debug variant, as dump capture
kernel to reduce memory consumption. This works alright for kdump as
the capture kernel is loaded using kexec.
In case of fadump, regular boot loader is used to load the capture
kernel. So, the default kernel needs to be used as capture kernel as
well. But with commit c5bdd2d8f1, initrd of a different kernel is
made dump capture capable, breaking fadump's ability to capture dump
properly. Fix this by sticking with the debug variant in case of
fadump.
Fixes: c5bdd2d8f1 ("kdump-lib: use non-debug kernels first")
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Lichen Liu <lichliu@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
When kdump is configured with a NFS location, and the remote directory does
not exist, kdump.service fails with a confusing error message.
kdumpctl[2172]: kdump: Dump path "/tmp/mkdumprd.ftWhOF/target/dumps"
does not exist in dump target "10.111.113.2:/srv/kdump"
We just need to print the remote directory "dumps" in such case, because
"/tmp/mkdumprd.ftWhOF/target" is the local temporary mount point.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Coiby Xu<coxu@redhat.com>
Decrease the risk that of leaking information that could potentially
be used to exploit the crash further (think location of keys).
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
These kdump.sysconfig.* files are almost identical with a bit difference
in several parameters, just use a simple script to generate them upon
packaging. This should make it easier to maintain, updating a comment or
param for a certain arch can be done in one place.
There are only some comment or empty option differences with the generated
version because some arch's sysconfig is not up-to-dated, this actually
fixes the issue, I used the following script to check these differences:
# for arch in aarch64 i386 ppc64 ppc64le s390x x86_64; do
./gen-kdump-sysconfig.sh $arch > kdump.sysconfig.$arch.new
git checkout HEAD^ kdump.sysconfig.$arch &>/dev/null
echo "$arch:"
diff kdump.sysconfig.$arch kdump.sysconfig.$arch.new; echo ""
done; git reset;
Signed-off-by: Kairui Song <kasong@tencent.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
So fedpkg will fetch the sources that matches given Fedora version.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
New version of qemu-img requires specifying the backing format for the
backing file otherwise it will abort.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Fedora 33 and 34 Cloud Base Images have only one partition with the
following directory structure,
.
├── bin -> usr/bin
├── boot
├── dev
├── etc
├── home
├── root
By comparison, Fedora 35, 36 and 37 Cloud Base Images have multiple
partitions. The root partition which is the last partition has the
following directory,
.
├── home
└── root
├── bin -> usr/bin
├── boot
├── dev
├── etc
├── home
├── root
and the 2nd partition is the boot partition.
This patch address the above changes by mounting {LAST_PARTITION}/root as
to TEMP_ROOT and mount SECOND_PARTITION to TEMP_ROOT/boot. So the test
image can be built successfully.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
s390x doesn't use GRUB. To make sure the boot entries are updated, call
zipl after running grubby.
Suggested-by: smitterl@redhat.com
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
"grubby --zipl" only takes effect when setting default kernel. It's
useless to add "--zipl" when updating kernel command line. Also rename
_update_grub to _update_kernel_cmdline since s390x doesn't use GRUB.
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Resolves: bz2104534
When running "kdumpctl reset-crashkernel --kernel=ALL" on s390x,
sed: can't read /etc/default/grub: No such file or directory
sed: can't read /etc/default/grub: No such file or directory
This happens because s390x doesn't use the grub bootloader and
/etc/default/grub doesn't exist.
Reported-by: smitterl@redhat.com
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Resolves: bz2092012
According to the ostree team [1], the existence of /run/ostree-booted
> is the most stable way to signal/check that a system has been
> booted in ostree-style. It is also used by rpm-ostree at
> compose/install time in the sandboxed environment where scriptlets run,
> in order to signal that the package is being installed/composed into
> an ostree commit (i.e. not directly on a live system). See
> 8ddf5f40d9/src/libpriv/rpmostree-scripts.cxx (L350-L353)
> for reference.
By checking the existence of /run/ostree-booted, we could skip trying to
update kernel cmdline during OSTree compose time.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2092012#c3
Reported-by: Luca BRUNO <lucab@redhat.com>
Suggested-by: Luca BRUNO <lucab@redhat.com>
Fixes: 0adb0f4 ("try to reset kernel crashkernel when kexec-tools updates the default crashkernel value")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Acked-by: Timothée Ravier <siosm@fedoraproject.org>
Resolves: bz2089871
Currently, kexec-tools can't be updated using virt-customize because
older version of kdumpctl can't acquire instance lock for the
get-default-crashkernel subcommand. The reason is /var/lock is linked to
/run/lock which however doesn't exist in the case of virt-customize.
This patch fixes this problem by using /tmp/kdump.lock as the lock
file if /run/lock doesn't exist.
Note
1. The lock file is now created in /run/lock instead of /var/run/lock since
Fedora has adopted adopted /run [2] since F15.
2. %pre scriptlet now always return success since package update won't
be blocked
[1] https://fedoraproject.org/wiki/Features/var-run-tmpfs
Fixes: 0adb0f4 ("try to reset kernel crashkernel when kexec-tools updates the default crashkernel value")
Reported-by: Nicolas Hicher <nhicher@redhat.com>
Suggested-by: Laszlo Ersek <lersek@redhat.com>
Suggested-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Currently, kdump may experience failure on some aws aarch64 platform.
The final scenario is:
[ 79.145089] printk: console [ttyS0] disabled
Then the system has no response any more. And after reboot, there is no
vmcore generated under /var/crash/. More detail [1].
In a short word, it is caused by the irqpoll policy and some unknown
acpi issue. The serial device is hot-removed as a pci device.
More detailed, the irqpoll policy demands to iterate over all interrupt
handler, if the interrupt line is shared, then the handler is
dispatched. And acpi handler acpi_irq() is on a shared interrupt line,
so it is called. But for some unknown reason, the acpi hardware regs
hold wrong state, and the acpi driver decides that a hot-removed event
happens on a pci slot, which finally removes the pci serial device.
To tackle this issue by removing the irqpoll parameter on aws aarch64
platform, until the real root cause in acpi is found and resolved.
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=2080468#c0
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
Resolves: bz2106645
The code of commit 163c02970e takes effect in rhel firstly, later
pulled to Fedora. However, Fedora OS doesn't have 40-redhat.rules
in systemd-udev package. With this commit applied, a false positive
warning message can always been seen as below.
So fixing it by checking if 40-redhat.rules exists before handling.
With this change, the false warning is gone.
[root@ ~]# kdumpctl restart
kdump: kexec: unloaded kdump kernel
kdump: Stopping kdump: [OK]
kdump: No kdump initial ramdisk found.
kdump: Rebuilding /boot/initramfs-5.19.0-rc6+kdump.img
sed: can't read /var/tmp/dracut.NnAV2g/initramfs/usr/lib/udev/rules.d/40-redhat.rules: No such file or directory
kdump: kexec: loaded kdump kernel
kdump: Starting kdump: [OK]
Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
The kernel of CoreOS is not in the standard locations, add
/boot/ostree/* to the boot_dirlist to find the vmlinuz.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
Currently $boot_img can get bad data if running on a platform
that doesn't set BOOT_IMAGE in the kernel command line. For
example, currently:
- s390x Fedora CoreOS machine:
```
[root@cosa-devsh ~]# sed "s/^BOOT_IMAGE=\((\S*)\)\?\(\S*\) .*/\2/" /proc/cmdline
mitigations=auto,nosmt ignition.platform.id=qemu ostree=/ostree/boot.0/fedora-coreos/2a72567ac8f7ed678c3ac89408f795e6ccd4e97b41e14af5f471b6a807e858b9/0 root=UUID=2a88436a-3b6b-4706-b33a-b8270bd87cde rw rootflags=prjquota boot=UUID=f4b2eaa5-9317-4798-85cf-308c477fee4c crashkernel=600M
```
where on a platform that uses GRUB we get:
- x86_64 Fedora CoreOS machine:
```
[root@cosa-devsh ~]# sed "s/^BOOT_IMAGE=\((\S*)\)\?\(\S*\) .*/\2/" /proc/cmdline
/ostree/fedora-coreos-af4f6cc7b9ff486cfa647680b180e989c72c8eed03a34a42e7328e49332bd20e/vmlinuz-5.18.5-200.fc36.x86_64
```
We should change the setting of the boot_img variable such that it will
be empty if BOOT_IMAGE doesn't exist.
With this change on the s390x machine:
```
[root@cosa-devsh ~]# grep -P -o '^BOOT_IMAGE=(\S+)' /proc/cmdline | sed "s/^BOOT_IMAGE=\((\S*)\)\?\(\S*\)/\2/"
[root@cosa-devsh ~]#
```
This change mattered much more before the change in c5bdd2d which changed
the following line from [[ -n $boot_img ]] to [[ "$boot_img" == *"$kdump_kernelver" ]].
Still I think this change has merit.
Signed-off-by: Dusty Mabe <dusty@dustymabe.com>
Acked-by: Coiby Xu <coxu@redhat.com>
There are many variants on OSTree based systems these days so
we should probably refer to the class of systems as "OSTree
based systems". Also, Atomic Host is dead.
Signed-off-by: Dusty Mabe <dusty@dustymabe.com>
Acked-by: Coiby Xu <coxu@redhat.com>
On RHEL9 and Fedora, the arm64 platform only supports 4KB page size.
the reserved memory size can be aligned to that on x86_64.
Introducing a new formula for 4KB on arm64, which bases on x86_64 plus
extra 64MB.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
Kdump uses currently running kernel as default, but when currently
running kernel is a debug kernel, it will consume more memory,
which may cause out-of-memory and fail to collect vmcore.
Now we will try to use non-debug kernels first if possible.
Also extract the logic of determine KDUMP_KERNEL from
prepare_kdump_bootinfo into a function. This function will return
KDUMP_KERNEL given a kernel version.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
For CoreOS based systems we use Ignition for provisioning machines
in the initramfs on first boot. We trigger Ignition right now by
the presence of `ignition.firstboot` in the kernel command line. The
kernel argument is only present on first boot so after a reboot it
no longer is in the kernel command line.
If a kernel crash happens before the first reboot of a machine we
want the `ignition.firstboot` kernel argument to be removed and not
passed on to the crash kernel.
This patch rewrites get_recommend_size to get rid of the following
limitations,
1. only supports ranges in crashkernel sorted in increasing order
2. the first entry of crashkernel should have only a single digit and
it's in gigabytes
Suggested-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Recently, it's found 'kdumpctl estimate' returns 512M while the system
reserves 1024M kdump memory in a case. This happens because the ranges
in /proc/iomem are inclusively. For example, "0-1: System RAM" means 2
bytes of system memory other than 1 byte. Fix this error by adding one
more byte.
Note
1. the function has been simplified as well.
2. define PROC_IOMEM as /proc/iomem for the sake of unit tests
Reported-by: Ruowen Qin <ruqin@redhat.com>
Fixes: 1813189 ("kdump-lib.sh: introduce functions to return recommened mem size")
Suggested-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Make log saving the last step of kdump.sh, so it can catch more info,
for example, the output of post.d hooks will be covered by the log now.
Signed-off-by: Kairui Song <kasong@tencent.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
1. yum is deprecated so use dnf instead
2. use the "kdumpctl reset-crashkernel --fadump=on" API
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
1. yum is deprecated so use dnf instead
2. use the "kdumpctl reset-crashkernel" API
3. ask the users to refer to crashkernel-howto.txt for setting custom
crashkernel value
4. fix a typo
Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
1. clean up left crashkernel.default
2. fix a few typos and grammar mistakes
3. ask the users to refer to `man kdumpctl` for reset-crashkernel
Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
This test prevents the mistake of adding an option to kdump.conf
without changing check_config as is the case with commit 73ced7f
("introduce the auto_reset_crashkernel option to kdump.conf").
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
kdump_get_conf_val allows to retrieves config value defined in
kdump.conf and it also supports sed regex like
"ext[234]\|xfs\|btrfs\|minix\|raw\|nfs\|ssh".
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
This commit adds a relatively thorough test suite for
kdumpctl reset-crashkernel [--fadump=[on|off|nocma]] [--kernel=path_to_kernel] [--reboot]
as implemented in commit 140da74 ("rewrite reset_crashkernel to support
fadump and to used by RPM scriptlet").
grubby have a few options to support its own testing,
- --no-etc-grub-update, not update /etc/default/grub
- --bad-image-okay, don't check the validity of the image
- --env, specify custom grub2 environment block file to avoid modifying
the default /boot/grub2/grubenv
- --bls-directory, specify custom BootLoaderSpec config files to avoid
modifying the default /boot/loader/entries
So the grubby called by kdumpctl is mocked as
@grubby --grub2 --no-etc-grub-update --bad-image-okay --env=$SPEC_TEST_DIR/env_temp -b $SPEC_TEST_DIR/boot_load_entries "$@"
in the tests. To be able to call the actual grubby in the mock function [1],
ShellSpec provides the following command
$ shellspec --gen-bin @grubby
to generate spec/support/bins/@grubby which is used to call the actual grubby.
kdumpctl has implemented its own version of updating /etc/default/grub
in _update_kernel_cmdline_in_grub_etc_default. To avoiding writing to
/etc/default/grub, this function is mocked as outputting its name and
received arguments similar to python unitest's assert_called_with.
[1] https://github.com/shellspec/shellspec#execute-the-actual-command-within-a-mock-function
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
AfterAll is an example group hook [1] which would be run after the group
tests are executed. Use this hook to clean up the files created by mktemp.
[1] https://github.com/shellspec/shellspec#beforeall-afterall---example-group-hook
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Currently there are two issues with unit-testing the functions defined
in kdumpctl and other shell scripts after sourcing them,
- kdumpctl would call main which requires root permission and would
create single instance lock (/var/lock/kdump)
- kdumpctl and other shell scripts directly source files under /usr/lib/kdump/
When ShellSpec load a script via "Include", it defines the__SOURCED__
variable. By making use of __SOURCED__, we can
1. let kdumpctl not call main when kdumpctl is "Include"d by ShellSpec
2. instruct kdumpctl and kdump-lib.sh to source the files in the repo
when running ShelSpec tests
Note coverage/ is added to .gitignore because ShellSpec generates code
coverage results in this folder.
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Make use of the new ${OPT[]} array and simplify local_fs_dump_target to
remove one more file operations.
While at it rename the local_fs_dump_target to is_local_target
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
With the introduction of ${OPT[fstype]} this call to kdump_get_conf_val
can be removed now as well.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The variable is only used for ssh dump targets. Furthermore it is
identical to the value stored in ${OPT[_target]}. Thus drop DUMP_TARGET and
use ${OPT[_target]} instead.
In order to be able to distinguish between the different target types
introduce the internal ${OPT[_fstype]}.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The variable is only used for ssh dump targets. Furthermore it is
identical to the value stored in ${OPT[sshkey]}. Thus drop
SSH_KEY_LOCATION and use ${OPT[sshkey]} instead.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The variable is only used for ssh dump targets. Furthermore it is
identical to the value stored in ${OPT[path]}. Thus drop SAVE_PATH and
use ${OPT[path]} instead.
Also make sure that ${OPT[path]} is always set to the default value when
no entry in kdump.conf is found.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Every call to kdump_get_conf_val parses kdump.conf although the file has
already been parsed in check_config. Thus store the values parsed in
check_config in an array and use them later instead of re-parsing the
file over and over again.
While at it rename check_config to parse_config.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
check_config and check_ssh_config both parse /etc/kdump.conf and are
usually used together. The difference between both is that
check_ssh_config does some extra checks on the format of the provided
ssh destination but ignores invalid or deprecated options in the config.
Thus merge check_ssh_config into check_config. Leave the additional
checks on the ssh destination in check_ssh_config but treat it like the
checks done for e.g. the failure_action.
This slightly changes the behavior of 'kdumpctl propagate', which now
fails if kdump.conf contains an invalid value unrelated to ssh. This
change in behavior isn't problematic because 'kdumpctl propagate' always
needs to be followed by a 'kdumpctl start' to have a working kdump
environment. For the situations where 'propagate' fails now the 'start'
would have failed in the past. So the failure only moved one step ahead
in the sequence.
While at it drop check_ssh_target and call check_and_wait_network_ready
directly.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The function has multiple problems:
1) SSH_{USER,SERVER} aren't defined local
2) Weird use of cut and sed to parse the DUMP_TARGET for the user and
host although check_ssh_config guarantees that it has the format
<user>@<host>.
3) Unnecessary use of a variable for the return value
4) Weird behavior to first unpack the DUMP_TARGET to SSH_USER and
SSH_SERVER and then putting it back together again
5) Definition of variable errmsg that is only used once but breaks
grep-ability of error message.
6) Wrong order when redirecting output of ssh-keygen, see SC2069 [1]
Fix them now.
While at it also improve the error messages in the function.
[1] https://www.shellcheck.net/wiki/SC2069
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
For ssh targets kdumpctl only verifies that the config value has the
correct <user>@<host> format itself. For all other tests, e.g. if the
destination can be reached, it relies on ssh. This allows users to
provide a <host> that isn't the proper hostname but an alias defined in
the ssh_config without failing the tests. If this is done
dracut-module-setup.sh:kdump_get_remote_ip will fail to obtain the
targets ip address. This failure is not detected and thus will not fail
the initramfs creation. The resulting initramfs however doesn't have the
necessary information for setting up the network and thus will fail to
boot.
Prevent the use of alias hostnames by verifying that the given hostname
is the same one ssh would use after parsing the ssh_config.
Note: Don't use getent ahosts to verify that the given host can be
resolved as this requires the network to be up which cannot be
guaranteed when the kdump.conf is parsed.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
The time out was increased to 180 seconds in 680c0d3 ("kdumpctl:
distinguish the failed reason of ssh"). Update the comment to reflect
that change.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
There are currently three identical definitions for the default ssh key.
Combine them into one in kdump-lib-initramfs.sh.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
in prepare_kdump_bootinfo s/defaut/default/.
While at it declare it with the other local variables as local.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
Using syslog for StandardOutput in a service file was deprecated in
systemd v246 with commit f3dc6af20f ("core: automatically update
StandardOuput=syslog to =journal (and similar for StandardError=)").
Thus the following warnings are printed in the crash kernel when
creating a dump.
systemd[1]: /usr/lib/systemd/system/kdump-capture.service:23: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
systemd[1]: /usr/lib/systemd/system/kdump-capture.service:24: Standard output type syslog+console is obsolete, automatically updating to journal+console. Please update your unit file, and consider removing the setting altogether.
Fix this by redirecting the stdout and stderr to the journal.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Coiby Xu <coxu@redhat.com>
do_estimate prints the warning that the reserved crashkernel is lower
than the recommended one even then when both values are identical. This
might cause confusion. So omit printing the warning when both values are
equal.
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
There is a system-wide sync call at the end of mkdumprd, move it to
kdumpctl after rebuild initrd and add another one for mkfadumprd.
Sync only the $TARGET_INITRD to avoid a system-wide sync taking too
long on a system with high disk activity.
Also update the sync in kdumpctl:restore_default_initrd which will
mv the $DEFAULT_INITRD_BAK to $DEFAULT_INITRD.
Signed-off-by: Lichen Liu <lichliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
If GRUB_ETC_DEFAULT use crashkernel=auto or
crashkernel=OLD_DEFAULT_CRASHKERNEL, it should be updated as well.
Add a helper function to read kernel cmdline parameter from
GRUB_ETC_DEFAULT. This function is used to read kernel cmdline
parameter like fadump or crashkernel.
Reviewed-by: Philipp Rudo <prudo@redhat.com>
There is the case where there are multiple entries of the same parameter on
the command line, e.g.
GRUB_CMDLINE_LINUX="crashkernel=110M crashkernel=220M fadump=on crashkernel=330M".
In such an situation _update_kernel_cmdline_in_grub_etc_default only
updates/removes the last entry which is usually not what you want as the
kernel (for crashkernel) takes the last entry it can find.
Thus make sure the case with multiple entries of the same parameter is
handled properly by removing all occurrences of given parameter first.
Note
1. sed command group and conditional control has been used to get rid of
grep.
2. Fully supporting kernel cmdline as documented in
Documentation/admin-guide/kernel-parameters.rst is complex and in
foreseeable future a full implementation is not needed. So simply
document the unsupported cases instead.
Fixes: 140da74 ("rewrite reset_crashkernel to support fadump and to used by RPM scriptlet")
Reported-by: Philipp Rudo <prudo@redhat.com>
Suggested-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
When doing in-place upgrading using leapp on x86_64, kdumpcl can't
acquire instance lock when running in %post RPM scriplet on x86_64,
localhost upgrade[1306]: /bin/kdumpctl: line 49: /var/lock/kdump: No such file or directory
localhost upgrade[1306]: kdump: Create file lock failed
and running "touch /var/lock/dkump" also fails with
"No such file or directory". Thus kdumpctl can't be run in %post
scriptlet. But kdumpctl can be run in %posttrans RPM scriplet.
Besides, it's better to update crashkernel after the kernel has been
updated. So let's update kernel crashkernel in the %posttrans
scriptlet which will be run in the end of a transaction i.e. after
the kernel has been updated.
Note for %posttrans scriptlet, "$1 == 1" means both installing a new
package and upgrading a package.
[1] https://github.com/apptainer/singularity/issues/2386#issuecomment-474747054
Reported-by: Jie Li <jieli@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Previously the output of blkid is not checked. If the output
is empty, the eval will report the following error message:
/lib/kdump/kdump-lib.sh: eval: line 925: syntax error near unexpected token `;'
/lib/kdump/kdump-lib.sh: eval: line 925: `; echo $TYPE'
For example, we can observe such a failing when blkid is invoked
against a lvm thinpool block device:
$ blkid -u filesystem,crypto -o export -- "/dev/block/253\:2"
$ echo $?
2
$ udevadm info /dev/block/253\:2|grep S\:
S: mapper/vg00-thinpoll_tmeta
In this patch, we will use sed instead of eval, to output the
fstype of block device if any.
Signed-off-by: Tao Liu <ltao@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
It's found that the kernel cmdline crashkernel=auto doesn't get updated
when upgrading kexec-tools. This happens because _get_all_kernels_from_grubby
is called with no argument by reset_crashkernel_after_update. When retrieving
all kernel paths on the system, "grubby --info ALL" should be used. Fix this
error by passing "ALL" argument.
Fixes: 0adb0f4 ("try to reset kernel crashkernel when kexec-tools updates the default crashkernel value")
Reported-by: Jie Li <jieli@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Tao Liu <ltao@redhat.com>
_is_osbuild fails because it expects the 1st and 2nd function parameter
to be the environment variable and environ file path respectively. Fix
it by swapping the parameters in read_proc_environ_var.
Note the osbuild environ file path is defined in _OSBUILD_ENVIRON_PATH
so _is_osbuild can be unit-tested by overwriting _OSBUILD_ENVIRON_PATH.
Fixes: 6a3ce83 ("fix the error of parsing the container environ variable for osbuild")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
shellcheck finds the following problem,
$ shellcheck kdump-lib.sh
In kdump-lib.sh line 876:
get_recommend_size "$sys_mem" "$ck_cmdline"
^---------^ SC2154: ck_cmdline is referenced but not assigned (did you mean '_ck_cmdline'?).
s/ck_cmdline/_ck_cmdline to fix kdump_get_arch_recommend_size.
Note s/sys_mem/_sys_mem as well to make the changes consistent.
Fixes: 105c016 ("factor out kdump_get_arch_recommend_crashkernel")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Tao Liu <ltao@redhat.com>
This patch makes the default crashkernel value consistent with previous
one.
Fixes: 105c016 ("factor out kdump_get_arch_recommend_crashkernel")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
The environment variable entries in /proc/[pid]/environ are separated by
null bytes instead of by spaces. Update the sed regex to fix this issue.
Note that,
1. this patch also fixes a issue which is kdumpctl would try to reset
crashkernel even osbuild has provided custom crashkernel value.
2. kernel hook 92-crashkernel.install installed by kexec-tools is
guaranteed to be ran by kernel-install. kexec-tools doesn't recommend
kernel so there is no guarantee kernel is installed after kexec-tools.
But dnf invokes kernel-install in the posttrans scriptlet (of kernel-core)
which is always ran after all packages including kexec-tools and kernel
in a dnf transaction.
3. To be able to do unit tests, the logic of reading environment variable
has been extracted as a separate function.
Fixes: ddd428a ("set up kernel crashkernel for osbuild in kernel hook")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Resolves: bz2025860
Upstream: git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git
commit 186e7b0752d8fce1618fa37519671c834c46340e
Author: Alexander Egorenkov <egorenar@linux.ibm.com>
Date: Wed Dec 15 18:48:53 2021 +0100
s390: handle R_390_PLT32DBL reloc entries in machine_apply_elf_rel()
Starting with gcc 11.3, the C compiler will generate PLT-relative function
calls even if they are local and do not require it. Later on during linking,
the linker will replace all PLT-relative calls to local functions with
PC-relative ones. Unfortunately, the purgatory code of kexec/kdump is
not being linked as a regular executable or shared library would have been,
and therefore, all PLT-relative addresses remain in the generated purgatory
object code unresolved. This in turn lets kexec-tools fail with
"Unknown rela relocation: 0x14 0x73c0901c" for such relocation types.
Furthermore, the clang C compiler has always behaved like described above
and this commit should fix the purgatory code built with the latter.
Because the purgatory code is no regular executable or shared library,
contains only calls to local functions and has no PLT, all R_390_PLT32DBL
relocation entries can be resolved just like a R_390_PC32DBL one.
* https://refspecs.linuxfoundation.org/ELF/zSeries/lzsabi0_zSeries/x1633.html#AEN1699
Relocation entries of purgatory code generated with gcc 11.3
------------------------------------------------------------
$ readelf -r purgatory/purgatory.o
Relocation section '.rela.text' at offset 0x6e8 contains 27 entries:
Offset Info Type Sym. Value Sym. Name + Addend
00000000000c 000300000013 R_390_PC32DBL 0000000000000000 .data + 2
00000000001a 001000000014 R_390_PLT32DBL 0000000000000000 sha256_starts + 2
000000000030 001100000014 R_390_PLT32DBL 0000000000000000 sha256_update + 2
000000000046 001200000014 R_390_PLT32DBL 0000000000000000 sha256_finish + 2
000000000050 000300000013 R_390_PC32DBL 0000000000000000 .data + 102
00000000005a 001300000014 R_390_PLT32DBL 0000000000000000 memcmp + 2
...
000000000118 001600000014 R_390_PLT32DBL 0000000000000000 setup_arch + 2
00000000011e 000300000013 R_390_PC32DBL 0000000000000000 .data + 2
00000000012c 000f00000014 R_390_PLT32DBL 0000000000000000 verify_sha256_digest + 2
000000000142 001700000014 R_390_PLT32DBL 0000000000000000
post_verification[...] + 2
Relocation entries of purgatory code generated with gcc 11.2
------------------------------------------------------------
$ readelf -r purgatory/purgatory.o
Relocation section '.rela.text' at offset 0x6e8 contains 27 entries:
Offset Info Type Sym. Value Sym. Name + Addend
00000000000e 000300000013 R_390_PC32DBL 0000000000000000 .data + 2
00000000001c 001000000013 R_390_PC32DBL 0000000000000000 sha256_starts + 2
000000000036 001100000013 R_390_PC32DBL 0000000000000000 sha256_update + 2
000000000048 001200000013 R_390_PC32DBL 0000000000000000 sha256_finish + 2
000000000052 000300000013 R_390_PC32DBL 0000000000000000 .data + 102
00000000005c 001300000013 R_390_PC32DBL 0000000000000000 memcmp + 2
...
00000000011a 001600000013 R_390_PC32DBL 0000000000000000 setup_arch + 2
000000000120 000300000013 R_390_PC32DBL 0000000000000000 .data + 122
000000000130 000f00000013 R_390_PC32DBL 0000000000000000 verify_sha256_digest + 2
000000000146 001700000013 R_390_PC32DBL 0000000000000000 post_verification[...] + 2
Corresponding s390 kernel discussion:
* https://lore.kernel.org/linux-s390/20211208105801.188140-1-egorenar@linux.ibm.com/T/#u
Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Reported-by: Tao Liu <ltao@redhat.com>
Suggested-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
[hca@linux.ibm.com: changed commit message as requested by Philipp Rudo]
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Simon Horman <horms@verge.net.au>
v2:
- Moved patch 601 -> 401
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
There is a mechanism to keep memory consumption minimum, i.e. equal
to trace_buf_size=1, until tracing by ftrace is actually started:
tracing: keep ring buffer to minimum size till used
73c5162aa3
Since ftrace is usually never used in the kdump 2nd kernel, the kdump
2nd kernel behaves in the same way with or without trace_buf_size=1.
So the issue which the patch want to solve never exists. Let's revert
the patch for better maintainance and avoid confusion.
ref link: https://bugzilla.redhat.com/show_bug.cgi?id=2034501#c20
This reverts commit f39000f.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
fadump is a ppc only feature, mkfadumprd is only needed for fadump, drop
it for other arch.
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Kairui Song <kasong@tencent.com>
This patch will introduce early kdump test.
It reuses the code of nfs kdump test, in order to setup 2 seperated VMs,
one(the client) for trigger the early kdump and crash, the other(the server)
for saving vmcore and check if vmcore exists. In order to minimize the
repetted code, a soft link is made to copy the same server side code.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
When run tests with 2 VMs, for example nfs/ssh kdump tests, client VM will do the
crash and dump, server VM will do vmcore saving and if-vmcore-exists
check.
Previously, when client VM finishes running, run-test.sh will kill the lead background
process, and then check if server VM has outputted "TEST PASSED" or "TEST FAILED" string.
However it didn't wait for server VM to finish. As a result, the server VM's final
outputs are not collected and checked, leaving the test result as "TEST RESULT NOT FOUND"
sometimes.
For example, the following is the pstree status of $(jobs -p) before it
gets killed. We can see the server VM is still running:
run-test.sh,172455 /root/kexec-tools/tests/scripts/run-test.sh --console nfs-early-kdump
└─run-test.sh,172457 /root/kexec-tools/tests/scripts/run-test.sh --console...
└─timeout,172480 --foreground 10m /root/kexec-tools/tests/scripts/run-qemu...
└─qemu-system-x86,172481 -enable-kvm -cpu host -nodefaults...
├─{qemu-system-x86},172489
├─{qemu-system-x86},172492
├─{qemu-system-x86},172493
├─{qemu-system-x86},172628
└─{qemu-system-x86},172629
In this patch, we will wait for $(jobs -p) to finish, in order to get
the complete output of test results.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
kexec-tools runs hostname binary in the case of fence_kdump. Since this
is a trival dependency and should not block the kexec-tools installation
if non-existent, using weak-dependency to resolve it.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Tao Liu <ltao@redhat.com>
Since kdump-lib-initramfs.sh is included by kdump-lib.sh, and
FENCE_KDUMP_SEND is used by both 1st and 2nd kernel, moving
FENCE_KDUMP_SEND from kdump-lib.sh to kdump-lib-initramfs.sh.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Tao Liu <ltao@redhat.com>
This patch will make zstd as recommended instead of required for
kexec-tools. If zstd command/package is unavaliable, it can failback to invoke
gzip when making kdump initramfs.
Fixes: 0311f6e ("Set zstd as the default compression method for kdump initrd")
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
kdumpctl only accepts a specified set of options. Add
auto_reset_crashkernel to this set.
Fixes: 73ced7f ("introduce the auto_reset_crashkernel option to kdump.conf")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Tao Liu <ltao@redhat.com>
When a file doesn't exist or isn't readable, grep complains as follows,
grep: /proc/cmdline: No such file or directory
grep: /etc/kernel/cmdline: No such file or directory
/proc/cmdline doesn't exist when installing package for an OS image and
/etc/kernel/cmdline may not exist if osbuild doesn't want set custom
kernel cmdline.
Use "-s" to suppress the error messages.
Fixes: 0adb0f4 ("try to reset kernel crashkernel when kexec-tools updates the default crashkernel value")
Fixes: ddd428a ("set up kernel crashkernel for osbuild in kernel hook")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Tao Liu <ltao@redhat.com>
The '|' in 'failure_action|default' should be replaced with '\|' when
passed to kdump_get_conf_val function. Because '|' needs to be escaped
to mean OR operation in sed regex, otherwise it will consider
'failure_action|default' as a whole string.
Fixes: ab1ef78 ("kdump-lib.sh: use kdump_get_conf_val to read config values")
v1 -> v2:
Rephased the commit message.
Replaced " with '.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
zstd has better compression ratio and time consumption balance.
When no customized compression method specified in kdump.conf,
we will use zstd as the default compression method.
**The test method:
I installed kexec-tools with and without the patch, executing the following
command for 4 times, and calculate the averange time:
$ rm -f /boot/initramfs-*kdump.img && time kdumpctl rebuild && \
ls -ail /boot/initramfs-*kdump.img
**The test result:
Bare metal x86_64 machine:
dracut with squash module
zlib lzo xz lz4 zstd
real 10.6282 11.0398 11.395 8.6424 10.1676
user 9.8932 11.9072 14.2304 2.8286 8.6468
sys 3.523 3.4626 3.6028 3.5 3.4942
size of
kdump.img 30575616 31419392 27102208 36666368 29236224
dracut without squash module
zlib lzo xz lz4 zstd
real 9.509 19.4876 11.6724 9.0338 10.267
user 10.6028 14.516 17.8662 4.0476 9.0936
sys 2.942 2.9184 3.0662 2.9232 3.0662
size of
kdump.img 19247949 19958120 14505056 21112544 17007764
PowerVM hosted ppc64le VM:
dracut with squash module | dracut without sqaush module
zlib zstd | zlib zstd
real 10.6742 10.7572 | 9.7676 10.5722
user 18.754 19.8338 | 20.7932 13.179
sys 1.8358 1.864 | 1.637 1.663
|
size of |
kdump.img 36917248 35467264 | 21441323 19007108
**discussion
zstd has a better compression ratio and time consumption balance.
v1 -> v2:
Use kdump_get_conf_val() to get dracut_args values of kdump.conf
v2 -> v3:
Attached testing benchmark
v3 -> v4:
Re-measured and re-attached the testing benchmark of x86_64 and ppc64le.
Changed regex '.*[[:space:]]' to '(^|[[:space:]])'
v4 -> v5:
Attacked lzo/xz/lz4 testing benchmark.
v5 -> v6:
Add zstd as required in kexec-tools.spec
Hello Coiby, you may use "RELEASE=34 make test-run", for
CONFIG_RD_ZSTD is enabled since fc-cloud-34
Acked-by: Coiby Xu <coxu@redhat.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Update crashkernel-howto since crashkernel.default has been removed. The
documentation is also simplified as a result.
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
osbuild is a tool to build OS images. It uses bwrap to install packages
inside a sandbox/container. Since the kernel package recommends
kexec-tools which in turn recommends grubby, the installation order would
be grubby -> kexec-tools -> kernel. So we can use the kernel hook
92-crashkernel.install provided by kexec-tools to set up kernel
crashkernel for the target OS image. But in osbuild's case, there is no
current running kernel and running `uname -r` in the container/sandbox
actually returns the host kernel release. To set up kernel crashkernel for
the OS image built by osbuild, a different logic is needed.
We will check if kernel hook is running inside the osbuild container
then set up kernel crashkernel only if osbuild hasn't specified a
custome value. osbuild exposes [1] the container=bwrap-osbuild environment
variable. According to [2], the environment variable is not inherited down
the process tree, so we need to check /proc/1/environ to detect this
environment variable to tell if the kernel hook is running inside a
bwrap-osbuild container. After that we need to know if osbuild wants to use
custom crashkernel value. This is done by checking if /etc/kernel/cmdline
has crashkernel set [3]. /etc/kernel/cmdline is written before packages
are installed.
[1] https://github.com/osbuild/osbuild/pull/926
[2] https://systemd.io/CONTAINER_INTERFACE/
[3] https://bugzilla.redhat.com/show_bug.cgi?id=2024976#c5
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
When kexec-tools updates the default crashkernel value, it will try to
reset the existing installed kernels including the currently running
kernel. So the running kernel could have different kernel cmdline
parameters from /proc/cmdline. When installing a kernel after updating
kexec-tools, /usr/lib/kernel/install.d/20-grub.install would be called
by kernel-install [1] which would use /proc/cmdline to set up new kernel's
cmdline. To address this special case, reset the new kernel's crashkernel
and fadump value to the value that would be used by running kernel after
rebooting by the installation hook. One side effect of this commit is it
would reset the installed kernel's crashkernel even currently running kernel
don't use the default crashkernel value after rebooting. But I think this
side effect is a benefit for the user.
The implementation depends on kernel-install which run the scripts in
/usr/lib/kernel/install.d passing the following arguments,
add KERNEL-VERSION $BOOT/MACHINE-ID/KERNEL-VERSION/ KERNEL-IMAGE [INITRD-FILE ...]
An concrete example is given as follows,
add 5.11.12-300.fc34.x86_64 /boot/e986846f63134c7295458cf36300ba5b/5.11.12-300.fc34.x86_64 /lib/modules/5.11.12-300.fc34.x86_64/vmlinuz
kernel-install could be started by the kernel package's RPM scriplet [2].
As mentioned in previous commit "try to reset kernel crashkernel when
kexec-tools updates the default crashkernel value", kdumpctl has difficulty
running in RPM scriptlet fore CoreOS. But rpm-ostree ignores all kernel hooks,
there is no need to disable the kernel hook for CoreOS/Atomic/Silverblue. But a
collaboration between rpm-ostree and kexec-tools is needed [3] to take care
of this special case.
Note the crashkernel.default support is dropped.
[1] https://www.freedesktop.org/software/systemd/man/kernel-install.html
[2] https://src.fedoraproject.org/rpms/kernel/blob/rawhide/f/kernel.spec#_2680
[3] https://github.com/coreos/rpm-ostree/issues/2894
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
kexec-tools could update the default crashkernel value.
When auto_reset_crashkernel=yes, reset kernel to new crashkernel
value in the following two cases,
- crashkernel=auto is found in the kernel cmdline
- the kernel crashkernel was previously set by kexec-tools i.e.
the kernel is using old default crashkernel value
To tell if the user is using a custom value for the kernel crashkernel
or not, we assume the user would never use the default crashkernel value
as custom value. When kexec-tools gets updated,
1. save the default crashkernel value of the older package to
/tmp/crashkernel (for POWER system, /tmp/crashkernel_fadump is saved
as well).
2. If auto_reset_crashkernel=yes, iterate all installed kernels.
For each kernel, compare its crashkernel value with the old
default crashkernel and reset it if yes
The implementation makes use of two RPM scriptlets [2],
- %pre is run before a package is installed so we can use it to save
old default crashkernel value
- %post is run after a package installed so we can use it to try to reset
kernel crashkernel
There are several problems when running kdumpctl in the RPM scripts
for CoreOS/Atomic/Silverblue, for example, the lock can't be acquired by
kdumpctl, "rpm-ostree kargs" can't be run and etc.. So don't enable this
feature for CoreOS/Atomic/Silverblue.
Note latest shellcheck (0.8.0) gives false positives about the
associative array as of this commit. And Fedora's shellcheck is 0.7.2
and can't even correctly parse the shell code because of the associative
array.
[1] https://github.com/koalaman/shellcheck/issues/2399
[2] https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
This option will determine whether to reset kernel crashkernel
to new default value or not when kexec-tools updates the default
crashkernel value and existing kernels using the old default kernel
crashkernel value. Default to yes.
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Rewrite kdumpctl reset-crashkernel KERNEL_PATH as
kdumpctl reset-crashkernel [--fadump=[on|off|nocma]] [--kernel=path_to_kernel] [--reboot]
This interface would reset a specific kernel to the default crashkernel value
given the kernel path. And it also supports grubby's syntax so there are the
following special cases,
- if --kernel not specified,
- use KDUMP_KERNELVER if it's defined in /etc/sysconfig/kdump
- otherwise use current running kernel, i.e. `uname -r`
- if --kernel=DEFAULT, the default boot kernel is chosen
- if --kernel=ALL, all kernels would have its crashkernel reset to the
default value and the /etc/default/grub is updated as well
--fadump=[on|off|nocma] toggles fadump on/off for the kernel provided
in KERNEL_PATH. If --fadump is omitted, the dump mode is determined by
parsing the kernel command line for the kernel(s) to update.
CoreOS/Atomic/Silverblue needs to be treated as a special case because,
- "rpm-ostree kargs" is used to manage kernel command line parameters
so --kernel doesn't make sense and there is no need to find current
running kernel
- "rpm-ostree kargs" itself would prompt the user to reboot the system
after modify the kernel command line parameter
- POWER is not supported so we can assume the dump mode is always kdump
This interface will also be called by kexec-tools RPM scriptlets [1]
to reset crashkernel.
Note the support of crashkenrel.default is dropped.
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
grubby --info=kernel-path or --add-kernel=kernel-path accepts a kernel
path (e.g. /boot/vmlinuz-5.14.14-200.fc34.x86_64) instead of kernel release
(e.g 5.14.14-200.fc34.x86_64). So we need to know the kernel path given
a kernel release. Although for Fedora/RHEL, the kernel path is
"/boot/vmlinuz-<KERNEL_RELEASE>", a path kernel could also be
/boot/<machine-id>/<KERNEL_RELEASE>/vmlinuz. So the most reliable way to
find the kernel path given a kernel release is to use "grubby --info".
For osbuild, a kernel path may not yet exist but it's valid for
"grubby --update-kernel=KERNEL_PATH". For example, "grubby -info" may
output something as follows,
index=0
kernel="/var/cache/osbuild-worker/osbuild-store/tmp/tmp2prywdy5object/tree/boot/vmlinuz-5.15.10-100.fc34.x86_64"
args="ro no_timer_check net.ifnames=0 console=tty1 console=ttyS0,115200n8"
root="UUID=76a22bf4-f153-4541-b6c7-0332c0dfaeac"
initrd="/var/cache/osbuild-worker/osbuild-store/tmp/tmp2prywdy5object/tree/boot/initramfs-5.15.10-100.fc34.x86_64.img"
There is no need to check if path like
/var/cache/osbuild-worker/osbuild-store/tmp/tmp2prywdy5object/tree/boot/vmlinuz-5.15.10-100.fc34.x86_64
physically exists.
Note these helper functions doesn't support CoreOS/Atomic/Silverblue
since grubby isn't used by them.
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Add a helper function to get dump mode. The dump mode would be
- fadump if fadump=on or fadump=nocma
- kdump if fadump=off or empty fadump
Otherwise return 1.
Also add another helper function to return a kernel's dump mode.
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
This helper function will be used to retrieve the value of kernel
cmdline parameters including crashkernel, fadump, swiotlb and etc.
Suggested-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Provide "kdumpctl get-default-crashkernel" for kdump_anaconda_addon
so crashkernel.default isn't needed.
When fadump is on, kdump_anaconda_addon would need to specify the dump
mode, i.e. "kdumpctl get-default-crashkernel fadump".
This interface would also be used by RPM scriptlet [1] to fetch default
crashkernel value.
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Factor out kdump_get_arch_recommend_crashkernel to prepare for
kdump-anaconda-plugin for example to retrieve the default crashkernel
value.
Note the support of crashkenrel.default is dropped.
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
It has been decided to increase default crashkernel value to reduce the
possibility of OOM.
Fixes: 7b7ddab ("kdump-lib.sh: kdump_get_arch_recommend_size uses crashkernel.default")
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
It seems the save_core function and vmcore detection was used a long
time ago when kdump shares same userspace in first and second kernel.
It's now heavily deprecated (only support cp, hardcoded path, dumpoops
no longer exists) and not used.
Now vmcore will never show up in first kernel for both kdump and fadump
case, and kdumpctl is only used in first kernel, so just remove them.
Signed-off-by: Kairui Song <kasong@tencent.com>
Acked-by: Coiby Xu <coxu@redhat.com>
For earlykdump, kdump-lib-initramfs.sh is sourced by kdump-lib.sh,
however it is not installed in dracut-early-kdump-module-setup.sh. Same
as xargs, which is used by kdump-lib.sh. Otherwise earlykdump will report
file not found errors.
Fixes: a5faa052d4
("kdump-lib-initramfs.sh: prepare to be a POSIX compatible lib")
Fixes: 4f01cb1b0a
("kdump-lib.sh: fix variable quoting issue")
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
Onlining secondary cpus breaks kdump completely on KVM on Power hosts
Though we use maxcpus=1 by default but 40-redhat.rules will bring up all
possible cpus by default.
Thus before we get the kernel fix and the systemd rule fix let's remove
the cpu rule in 40-redhat.rules for ppc64/ppc64le kdump initramfs.
This is back ported from RHEL, and original credit goes to Dave Young
<dyoung@redhat.com>
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Tao Liu <ltao@redhat.com>
When kexec-tools is newly installed, kdump migration action hasn't
registered and the following error could occur,
INF dnf.rpm: Could not find a registered notification tool with the specified command ('/usr/lib/kdump/kdump-migrate-action.sh').
"servicelog_notify --list" could list registered notification tools for
a command but it outputs the above error as well. So simply redirect the
error to /dev/null when running "servicelog_notify --remove".
Fixes: commit 146f662622
("kdump/ppc64: migration action registration clean up")
Acked-by: Tao Liu <ltao@redhat.com>
Acked-by: Hari Bathini <hbathini@linux.ibm.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
When secureboot is enabled, kdumpctl needs to use keyctl to add/remove
a key to/from the .ima keyring.
Fixes: commit 596fa0a07f
("kdumpctl: enable secure boot on ppc64le LPARs")
Signed-off-by: Coiby Xu <coxu@redhat.com>
The Zstandard (zstd) compression method is not enabled:
$ makedumpfile -v
makedumpfile: version 1.7.0 (released on 8 Nov 2021)
lzo enabled
snappy enabled
zstd disabled
This patch will enable it when building kexec-tools rpm package.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
When there more than one binaries, quoting "$val" would make
dracut-install treat multiple binaries as one binary. Take
"extra_bins /usr/sbin/ping /usr/sbin/ip" as an example, the
following error would occur when building initrd,
dracut-install: ERROR: installing '/usr/sbin/ping /usr/sbin/ip'
dracut: FAILED: /usr/lib/dracut/dracut-install -D /var/tmp/dracut.ODrioZ/initramfs -a /usr/sbin/ping /usr/sbin/ip
Fix it by not quoting the variable and bypassing SC2086 shellcheck.
Fixes: commit 86538ca6e2
("bash scripts: fix variable quoting issue")
Acked-by: Tao Liu <ltao@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
nfs service will append extra mount options to kernel mount options.
Such as mountaddr/mountproto options. These options only represent
current mounting details of the 1st kernel, but may not appropriate
for the 2nd kernel for the same reason as commit
d4f04afa47 ("mkdumprd: drop some nfs
mount options when reading from kernel"). This patch will remove
these options.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
The return value of set_ck_kernel or set_grub_ck is wrongly being used
as the exit code. This hook should exit with 0 or it may result in
unexpected behavior of kernel-install.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
To make things cleaner and more human readable, add a short comment for
the POSIX scripts.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
All non-POSIX syntax in second kernel are gone, tested on Fedora 34
with latest dracut, dash now works fine.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
This is a batch update done with:
shfmt -s -w kdump-lib.sh
Clean up code style and reduce code base size, no behaviour change.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Fix a few ambiguous syntax issues and remove some unused variables.
Also refactor some code to make it more robust.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Get rid of let, and remove useless '$' on arithmetic variables.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Replace echo "$(cmd)" and "var=$(cmd); echo $var" with just `cmd`.
And remove some useless cat.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
This fixes word splitting issue with nmcli args. Current kexec-tools
scripts won't call nmcli with correct arguments when there are space in
network interface name.
nmcli expects multiple parameters, but get_nmcli_value_by_field only
accepts two params and depends on shell word splitting to split the
_nm_show_cmd into multiple params, which is very fragile.
So switch the param order, simplified this function and now multiple
params can be used properly.
And get_nmcli_connection_show_cmd_by_ifname returns multiple
nmcli params in a single variable, it depend on shell word splitting to
split the words when calling nmcli. But this is very fragile and break
easily when there are any special character in the connection path.
This function is only introduced to get and cache the nmcli command
which contains the "connection name".
Actually only cache the "connection path" is enough. Callers should
just call get_nmcli_connection_apath_by_ifname to cache the path, and
a new helper get_nmcli_field_by_conpath is introduced here to get value
from nmcli. This way "connection path" can contain any character.
Also get rid of another nmcli_cmd usage in
get_nmcli_connection_apath_by_ifname which stores multiple params in a
single bash variable separated by space.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Updated file syntax with following command:
sed -i -e 's/\(\s\)\[\s\([^]]*\)\s\]/\1\[\[\ \2 \]\]/g' kdump-lib.sh
(replace '[ ]' with '[[ ]]')
sed -i -e 's/`\([^`]*\)`/\$(\1)/g' kdump-lib.sh
(replace `...` with $(...))
And manually updated [[ ... -a ... ]] and [[ ... -o ... ]] with && and
||.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
POSIX doesn't support keyword local, so add double underscore and prefix
to variable names, and reduce variable usage, to avoid any variable name
conflict.
Also reformat the code with `shfmt -s -w kdump-lib-initramfs.sh`.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
This is done with `shfmt -w -s dracut-kdump.sh`. There is no behaviour
change.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
POSIX doesn't support keyword `local`, so this commit reduced variable usage.
Heredoc ("<<<") operation is also not supported, so kdump.conf is now pre-parse
into a temp file. Also fixes many POSIX syntax errors.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Set pipefail will cause POSIX shell to exit with failure. So only do
that in bash.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
There is a workaround for `scp` that it expects IPv6 address to be
quoted with [ ... ], only apply the workaround once and store the
updated `scp` address to reuse it.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
`add_dump_code "<op>"` is just `DUMP_INSTRUCTION="<op>"`, no need a
extra wrapper for that.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
monitor_dd_progress is the only extra binary in KDUMP_SCRIPT_DIR, no
need to change PATH environment variable, just call it directly.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
These dump related functions are only used by dracut-kdump.sh.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
kdump-error-handler.sh does nothing except calling three functions,
it can be easily merged into kdump.sh by using a parameter to run the
error handling routine.
kdump-lib-initramfs.sh was created to hold the three shared functions
and related code, so by merging these two files, kdump-lib-initramfs.sh
can be simplified by a lot.
Following up commits will clean up kdump-lib-initramfs.sh.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Move all functions needed in the second kernel from kdump-lib.sh
to kdump-lib-initramfs.sh, and update shebang headers.
Now, kdump-lib-initramfs.sh is an independent lib script, no longer
depend on kdump-lib.sh, and kdump-lib.sh is no longer needed for
the second kernel.
In later commits, functions in kdump-lib-initramfs.sh will be reworked
to be POSIX compatible, kdump-lib.sh will contain bash only functions.
POSIX shell have very limited features, eg. `local` keyword doesn't
exist in POSIX but we rely on that heavily. So kdump-lib.sh will
use bash syntax and contain the most complex helper and codes.
kdump-lib-initramfs.sh will contain the minimum set of helpers,
and be shared by both the first and second kernel.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
This is a batch update done with:
shfmt -s -w mkfadumprd mkdumprd kdumpctl *-module-setup.sh
Clean up code style and reduce code base size, no behaviour change.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Declare and assign separately to avoid masking return values:
https://github.com/koalaman/shellcheck/wiki/SC2155
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
kdumpctl, mkdumprd, *-module-setup.sh only target bash, since they
only run in first kernel and depend on dracut, and dracut depends
on bash. So use '[[ ]]' to replace '[ ]'.
This is a batch update done with following command:
`sed -i -e 's/\(\s\)\[\s\([^]]*\)\s\]/\1\[\[\ \2 \]\]/g' kdumpctl, mkdumprd, *-module-setup.sh`
and replaced [ ... -a ... ] with [[ ... ]] && [[ ... ]] manually.
See https://tldp.org/LDP/abs/html/testconstructs.html for more details
on '[[ ]]', it's more versatile, safer, and slightly faster than '[ ]'.
This will also help shfmt to clean up the code in later commits.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
This is a batch update done with following command:
`sed -i -e 's/`\([^`]*\)`/\$(\1)/g' mkfadumprd mkdumprd \
kdumpctl dracut-module-setup.sh dracut-fadump-module-setup.sh \
dracut-early-kdump-module-setup.sh`
And manually converted some corner cases. This fixes
all related issues detected by shellcheck.
Make it easier to do clean up in later commits.
Check following link for reasons to switch to the new syntax:
https://github.com/koalaman/shellcheck/wiki/SC2006
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Use bash builtin string substitution instead, as suggested by:
https://github.com/koalaman/shellcheck/wiki/SC2001
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Some `cat` calls are useless, remove them to make it cleaner.
See: https://github.com/koalaman/shellcheck/wiki/SC2002
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Some functions are executed in subshell to avoid variable environment
pollution. But the surrounding $() is not needed, and it may lead to
executing output which is unexpected here.
See: https://github.com/koalaman/shellcheck/wiki/SC2091
Signed-off-by: Kairui Song <kasong@redhat.com>
Suggested-by: Coiby Xu <coxu@redhat.com>
Wrap the variable with {...}, else it may get interpreted as array due
to the '[' char next to it.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
As suggested by:
https://github.com/koalaman/shellcheck/wiki/SC2199
The array is not quoted here but implicitly concatenate still happens,
could be harmless but shellcheck complains about it so fix it.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
To make arguments list work as expected, array is preferred.
Check following link for details:
https://github.com/koalaman/shellcheck/wiki/SC2089
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
For storing arguments, plain string is not a good choice. Array is
preferred:
See: https://github.com/koalaman/shellcheck/wiki/SC2089
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Currently get_ssh_size is not working as expected, it should return the
target's available space, but it will include df's header row string as
the result. Fix this issue by only use the last output line.
And the _opt variable will be used as args so it should be an array.
Also remove the awk call, just use `df --output=avail` instead.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
To make arguments list work as expected, array is preferred.
Use xargs only to parse the "dracut_args" config value, and pass the
array directly to dracut.
Check following link for details:
https://github.com/koalaman/shellcheck/wiki/SC2089
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
For loops over find output are fragile, use a while read loop:
https://github.com/koalaman/shellcheck/wiki/SC2044
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com
Also fixed kdumpctl, use `awk` instead of `cut` to read
core_collector's executable name correctly when its arguments
are not seperated by space.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Add a helper kdump_get_conf_val to replace get_option_value.
It can help cover more corner cases in the code, like when there are
multiple spaces in config file, config value separated by a tab,
heading spaces, or trailing comments.
And this uses "sed group command" and "sed hold buffer", make it much
faster than previous `grep <config> | tail -1`.
This helper is supposed to provide a universal way for kexec-tools
scripts to read in config value. Currently, different scripts are
reading the config in many different fragile ways.
For example, following codes are found in kexec-tools script code base:
1. grep ^force_rebuild $KDUMP_CONFIG_FILE
echo $_force_rebuild | cut -d' ' -f2
2. grep ^kdump_post $KDUMP_CONFIG_FILE | cut -d\ -f2
3. awk '/^sshkey/ {print $2}' $conf_file
4. grep ^path $KDUMP_CONFIG_FILE | cut -d' ' -f2-
1, 2, and 4 will fail if the space is replaced by, e.g. a tab
1 and 2 might fail if there are multiple spaces between config name
and config value:
"kdump_post /var/crash/scripts/kdump-post.sh"
A space will be read instead of config value.
1, 2, 3 will fail if there are space in file path, like:
"kdump_post /var/crash/scripts dir/kdump-post.sh"
4 will fail if there are trailing comments:
"path /var/crash # some comment here"
And all will fail if there are heading space,
" path /var/crash"
And all will most likely cause problems if the config file contains
the same option more than once.
And all of them are slower than the new sed call. Old get_option_value
is also very slow and doesn't handle heading space.
Although we never claim to support heading space or tailing comments
before, it's harmless to be more robust on config reading, and many
conf files in /etc support heading spaces. And have a faster and
safer config reading helper makes it easier to clean up the code.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Add a helper `kdump_read_conf` to replace read_strip_comments.
`kdump_read_conf` does a few more things:
- remove trailing spaces.
- format the content, remove duplicated spaces between name and value.
- read from KDUMP_CONFIG_FILE (/etc/kdump.conf) directly, avoid pasting
"/etc/kdump.conf" path everywhere in the code.
- check if config file exists, just in case.
Also unify the environmental variable, now KDUMP_CONFIG_FILE stands for
the default config location.
This helps avoid some shell pitfalls about spaces when reading config.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
EditorConfig file is helpful for tools like `shfmt`, also could be a
hint for code styling.
The code style spec used in this new added .editorconfig file is
generated based on existing code style.
Following commits will make mkfadumprd, mkdumprd, kdumpctl, kdump-lib.sh,
and *-module-setup.sh only be used in first kernel, so use bash syntax
for these scripts. Other scripts will use sh syntax for better POSIX
compatibility.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Philipp Rudo <prudo@redhat.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
At least, this is a plausible suspect for #1993505 - thanks to
@kevin for identifying it - and fixing it should be safe and
correct, so we may as well do it and see if it helps.
While kdump migration action is registered for LPM event, ensure it is
cleared as appropriate to avoid duplicate/stale notification entries.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Kairui Song <kasong@redhat.com>
On s390x KVM machines, the following errors would show when building kdump
initramfs that dumps vmcore to a remote target,
$ kdumpctl rebuild
/usr/lib/dracut/modules.d/99kdumpbase/module-setup.sh: line 475: /sys/bus/ccwgroup/devices/online: No such file or directory
/usr/lib/dracut/modules.d/99kdumpbase/module-setup.sh: line 476: [: -ne: unary operator expected
This happens because s390x KVM machines use virtual network and
/sys/bus/ccwgroup/devices/ exists but is empty. Fix it by check
the existence of file "/sys/bus/ccwgroup/devices/*/online".
Fixes: commit 7d47251568
("Iterate /sys/bus/ccwgroup/devices to tell if we should set up rd.znet")
BugLink: https://bugzilla.redhat.com/show_bug.cgi?id=1982474
Reported-by: Jie Li <jieli@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
t Acked-by: Kairui Song <kasong@redhat.com>
When `failure_action` is set to `dump_to_rootfs`, the message:
"Waiting for rootfs mount, will timeout after 90 seconds"
is actually wrong. Kdump will simply call `systemctl start sysroot.mount`,
but the timeout value of sysroot.mount depends on the unit service and
dracut parameters. And by default, dracut will set
JobRunningTimeoutSec=0 and JobTimeoutSec=0 for the device units,
which means it will wait forever. (see wait_for_dev function in dracut)
For some devices, this can be fixed by setting rd.timeout=90. But when
initqueue is set enabled during initramfs build, dracut will force set
timeout for host devices to `0`. (see 99base/module-setup.sh).
Depending on dracut / systemd can make things unpredictable and break as
parameters or code change. To make things easy to understand and
maintain, just call `systemctl` with `--no-block` params, and implement
a standalone wait loop. Now `dump_to_rootfs` will actually wait for
90s then timeout.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
Dump capture initramfs needs rebuild after partition migration (LPM).
Use servicelog notification mechanism to invoke kdump rebuild after
migration.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
"prot_virt" enables the kernel to run Secure Execution virtual machines
on s390. These virtual machines are isolated from the hypervisor and
thus protected against tampering by a malicious host. Enabling
"prot_virt" requires a minimum of ~2.5GB memory which exceeds what is
typically reserved for the crashkernel. Thus remove "prot_virt" from the
command line for the 2nd kernel to prevent it to run out-of-memory.
For more discussions about this, see:
https://lists.fedoraproject.org/archives/list/kexec@lists.fedoraproject.org/thread/QSRRNV4ALKXUJC2VM3US4Z2NSQRHVMXB/
Signed-off-by: Philipp Rudo <prudo@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
Packaging guidelines have been amended to not require systemd for scriptlets,
see https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/#_scriptlets.
The comment duplicates what the macro contains.
systemd-sysv-convert binary was removed in 2013, trying to call it is
unlikely to succeed.
chkconfig binary is provided by the chkconfig package, which is not in
Requires. (And makes little sense to call nowadays anyway.)
The new `crashkernel.default` file in kernel package can be used as the
ck_cmdline source.
Also keep the legacy code so old kernel packages will still work.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
This reverts commit 073c30973c, i.e.
re-enable the kexec file load by default since this dual signature
issue no longer bothers Fedora 34.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
When kdump failed due to initqueue timeout, the sysroot.mount and other
serivces could be stuck in `start` but `dead` status:
Example output of systemctl:
dev-disk-by\x2duuid-530830d1\x2df2c7\x2d4c9a\x2d9a82\x2d148609097521.device loaded inactive dead start
<... snip ...>
squash-root.mount loaded active mounted /squash/root
squash.mount loaded active mounted /squash
sysroot.mount loaded inactive dead start /sysroot
<... snip ...>
dracut-cmdline.service loaded active exited dracut cmdline hook
dracut-initqueue.service loaded activating start start dracut initqueue hook
dracut-mount.service loaded inactive dead start dracut mount hook
At this point calling `systemctl start sysroot.mount` will just hang as
systemd will just wait for the services that are stuck in `start`
status. So call `systemctl cancel` here to cancel all pending jobs and
have a clean start for mounting sysroot.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
To track and manage kernel's crashkernel usage by kernel version,
each kernel package will include a crashkernel.default containing the
default `crashkernel=` value of that kernel. So we can use a hook to
update the kernel cmdline of new installed kernel accordingly.
Put it after all other grub boot loader setup hooks, so it can simply
call grubby to modify the kernel cmdline.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
In newer kernel, crashkernel.default will contain the default
crashkernel value of a kernel build. So introduce a new sub command
to help user reset kernel crashkernel size to the default value.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
Now we need this helper again, for `reset-crashkernel`
This reverts commit ff46cfb19e.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
When running with squash module enabled for both initramfs, /dev and
/run are also mounted by squash-init, so move them to newroot as well,
else they might leak.
Also pass `-d` to umount so loop devices (if used) will be force freed.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Hari Bathini <hbathini@linux.ibm.com>
After fadump embedded the fadump initramfs in the normal initramfs,
kdumpctl will mistakenly rebuild the initramfs everytime.
kdumpctl checks the hostonly-kernel-modules.txt file in initramfs
to check if required drivers are included, but the normal initramfs
is built in non-hostonly mode, so it doesn't have a
hostonly-kernel-modules.txt file. The check will always fail.
So let mkfadumprd make a copy of the hostonly-kernel-modules.txt in the
fadump initramfs and let kdumpctl check that file instead.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Hari Bathini <hbathini@linux.ibm.com>
In case of fadump, the initramfs image has to be built to boot into
the production environment as well as to offload the active crash dump
to the specified dump target (for boot after crash). As the same image
would be used for both boot scenarios, it could not be built optimally
while accommodating both cases.
Use --include to include the initramfs image built for offloading
active crash dump to the specified dump target. Also, introduce a new
out-of-tree dracut module (99zz-fadumpinit) that installs a customized
init program while moving the default /init to /init.dracut. This
customized init program is leveraged to isolate fadump image within
the default initramfs image by kicking off default boot process
(exec /init.dracut) for regular boot scenario and activating fadump
initramfs image, if the system is booting after a crash.
If squash is available, ensure default initramfs image is also built
with squash module to reduce memory consumption in capture kernel.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Change spaces to tab to fix alignment issue.
Fixes: commit 7d47251568
("Iterate /sys/bus/ccwgroup/devices to tell if we should set up rd.znet")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
/sys/bus/ccwgroup/devices doesn't exist for non-s390x machines which leads to
the warning "find: '/sys/bus/ccwgroup/devices': No such file or directory".
This warning can be eliminated by checking the existence of
"/sys/bus/ccwgroup/devices" beforehand.
Fixes: commit 7d47251568
("Iterate /sys/bus/ccwgroup/devices to tell if we should set up rd.znet")
Reported-by: Ruowen Qin <ruqin@redhat.com>
BugLink: https://bugzilla.redhat.com/show_bug.cgi?id=1974618
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from upstream.
commit 9a6f589d99dcef114c89fde992157f5467028c8f
Author: Tao Liu <ltao@redhat.com>
Date: Fri Jun 18 18:28:04 2021 +0800
[PATCH] check for invalid physical address of /proc/kcore when making ELF dumpfile
Previously when executing makedumpfile with -E option against
/proc/kcore, makedumpfile will fail:
# makedumpfile -E -d 31 /proc/kcore kcore.dump
...
write_elf_load_segment: Can't convert physaddr(ffffffffffffffff) to an offset.
makedumpfile Failed.
It's because /proc/kcore contains PT_LOAD program headers which have
physaddr (0xffffffffffffffff). With -E option, makedumpfile will
try to convert the physaddr to an offset and fails.
Skip the PT_LOAD program headers which have such physaddr.
Signed-off-by: Tao Liu <ltao@redhat.com>
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from upstream.
commit 38d921a2ef50ebd36258097553626443ffe27496
Author: Coiby Xu <coxu@redhat.com>
Date: Tue Jun 15 18:26:31 2021 +0800
[PATCH] check for invalid physical address of /proc/kcore when finding max_paddr
Kernel commit 464920104bf7adac12722035bfefb3d772eb04d8 ("/proc/kcore:
update physical address for kcore ram and text") sets an invalid paddr
(0xffffffffffffffff = -1) for PT_LOAD segments of not direct mapped
regions:
$ readelf -l /proc/kcore
...
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
NOTE 0x0000000000000120 0x0000000000000000 0x0000000000000000
0x0000000000002320 0x0000000000000000 0x0
LOAD 0x1000000000010000 0xd000000000000000 0xffffffffffffffff
^^^^^^^^^^^^^^^^^^
0x0001f80000000000 0x0001f80000000000 RWE 0x10000
makedumpfile uses max_paddr to calculate the number of sections for
sparse memory model thus wrong number is obtained based on max_paddr
(-1). This error could lead to the failure of copying /proc/kcore
for RHEL-8.5 on ppc64le machine [1]:
$ makedumpfile /proc/kcore vmcore1
get_mem_section: Could not validate mem_section.
get_mm_sparsemem: Can't get the address of mem_section.
makedumpfile Failed.
Let's check if the phys_start of the segment is a valid physical
address to fix this problem.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1965267
Reported-by: Xiaoying Yan <yiyan@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from upstream.
commit 646456862df8926ba10dd7330abf3bf0f887e1b6
Author: Kazuhito Hagio <k-hagio-ab@nec.com>
Date: Wed May 26 14:31:26 2021 +0900
[PATCH] Increase SECTION_MAP_LAST_BIT to 5
* Required for kernel 5.12
Kernel commit 1f90a3477df3 ("mm: teach pfn_to_online_page() about
ZONE_DEVICE section collisions") added a section flag
(SECTION_TAINT_ZONE_DEVICE) and causes makedumpfile an error on
some machines like this:
__vtop4_x86_64: Can't get a valid pmd_pte.
readmem: Can't convert a virtual address(ffffe2bdc2000000) to physical address.
readmem: type_addr: 0, addr:ffffe2bdc2000000, size:32768
__exclude_unnecessary_pages: Can't read the buffer of struct page.
create_2nd_bitmap: Can't exclude unnecessary pages.
Increase SECTION_MAP_LAST_BIT to 5 to fix this. The bit had not
been used until the change, so we can just increase the value.
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
For the log entry that has multiple lines, "makedumpfile --dump-dmesg"
would indent the remaining lines while vmcore-dmesg doesn't. For
example, vmcore-dmesg.txt and vmcore-dmesg.txt.2 are the outputs of
vmcore-dmesg and "makedumpfile --dump-dmesg" respectively,
```
diff -u vmcore-dmesg.txt vmcore-dmesg.txt.2
--- vmcore-dmesg.txt 2021-03-28 22:13:09.986000000 -0400
+++ vmcore-dmesg.txt.2 2021-03-28 22:13:39.920106131 -0400
@@ -397,9 +397,9 @@
[ 1.710742] vc vcsa: hash matches
[ 1.711938] RAS: Correctable Errors collector initialized.
[ 1.713736] Unstable clock detected, switching default tracing clock to "global"
-If you want to keep using the local clock, then add:
- "trace_clock=local"
-on the kernel command line
+ If you want to keep using the local clock, then add:
+ "trace_clock=local"
+ on the kernel command line
[ 1.750539] ata1.01: NODEV after polling detection
[ 1.750973] ata1.00: ATA-7: QEMU HARDDISK, 2.5+, max UDMA/100
[ 1.752885] ata1.00: 8388608 sectors, multi 16: LBA48
```
Quite often, all three tests could fail because of the above difference. So
let's ignore all the spaces. This patch could fix bz1952299 [1].
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1952299
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
test_base_image should depend on EXTRA_RPMS so it gets rebuild when
EXTRA_RPMS changes.
Fixes: commit bbc064f958
("selftest: add EXTRA_RPMs so dracut RPMs can be installed onto
the image to run the tests")
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
When kdump service fails, the current errors do not display the
absolute path of dump location(marked it as "^"), for example:
kdump: kexec: unloaded kdump kernel
kdump: Stopping kdump: [OK]
kdump: Detected change(s) in the following file(s): /etc/kdump.conf
kdump: Rebuilding /boot/initramfs-4.18.0-304.el8.x86_64kdump.img
kdump: Dump path "/var1/crash" does not exist in dump target "UUID=c202ef45-3ac3-4adb-85e7-307a916757f0"
^^^^^^^^^^^
kdump: mkdumprd: failed to make kdump initrd
kdump: Starting kdump: [FAILED]
Here, it should output the absolute path of dump location with this
format: "<mount path>/<path>". To fix it, let's extend the relative
pathname to the absolute pathname in check_user_configured_target().
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
This patch fixes bz1941106 and bz1941905 which passed empty rd.znet to the
kernel command line in the following cases,
- The IBM (Z15) KVM guest uses virtio for all devices including network
device, so there is no znet device for IBM KVM guest. So we can't
assume a s390x machine always has a znet device.
- When a bridged network is used, kexec-tools tries to obtain the znet
configuration from the ifcfg script of the bridged network rather than
from the ifcfg script of znet device.
We can iterate /sys/bus/ccwgroup/devices to tell if there if there is
a znet network device. By getting an ifname from znet, we can also avoid
mistaking the slave netdev as a znet network device in a bridged network
or bonded network.
Note: This patch also assumes there is only one znet device as commit
7148c0a30d ("add s390x netdev setup")
which greatly simplifies the code. According to IBM [1], there could be
more than znet devices for a z/VM system and a z/VM system may have a
non-znet network device like ConnectX. Since kdump_setup_znet was
introduced in 2012 and so far there is no known customer complaint that
invalidates this assumption I think it's safe to assume an IBM z/VM
system only has one znet device. Besides, there is no z/VM system found
on beaker to test the alternative scenarios.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1941905#c13
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Use a modified and minimized version of emergency shell.
The differences of this kdump shell and dracut emergency shell are:
- Kdump shell won't generate a rdsosreport automatically
- Customized prompts
- Never ask root password
- Won't tangle with dracut's emergency_action. If emergency_action is
set, dracut emergency shell will perform dracut's emergency_action
instead of kdump final_action on exit.
- If rd.shell=no is set, kdump shell will still work, dracut emergency
shell won't, even if kdump failure_action is set to shell.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
The wrapper is introduced in commit 002337c, according to the commit
message, the only usage of the wrapper is when dracut-initqueue calls
"systemctl start emergency" directly. In that case, emergency
is started, but not in a isolation mode, which means dracut-initqueue
is still running. On the other hand, emergency will call
"systemctl start dracut-initqueue" again when default action is dump_to_rootfs.
systemd would block on the last dracut-initqueue, waiting for the first
instance to exit, which leaves us hang.
In previous commit we added initqueue status detect in dump_to_rootfs,
so now even without the wrapper, it will not hang.
And actually, previously, with the wrapper, emergency might still hang
for like 30s. When dracut called emergency service because initqueue
timed out, dump_to_rootfs will try start initqueue again and timeout
again. Now with the wrapper removed, we can avoid these two kinds of
hangs, bacause without the isolation we can detect initqueue service
status correctly in such case.
Also remove the invalid header comments in service file, the service
is not part of systemd code. And sync the service spec with dracut.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
kdump's dump_to_rootfs will try to start initqueue unconditionally.
dump_to_rootfs will run after systemd isolate to emergency
target, so this is currently accetable.
But there is a problem when initqueue starts the emergency action
because of initqueue timeout. dump_to_rootfs will start initqueue and
lead to timeout again.
So following patch will remove the previous isolation wrapper, and
detect the service status here. Previous isolation makes the detection
impossible. Now this detection will be valid and helpful to prevent
double timeout or hang.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Coiby Xu <coxu@redhat.com>
Fix the warning observed when KDUMP_KERNELVER is specified:
kdumpctl[10926]: /lib/kdump/kdump-lib.sh: line 697: [: missing `]'
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Kairui Song <kasong@redhat.com>
For crashkernel=auto policy, if total RAM size is under a throttle,
there is no memory reserved for kdump.
Also correct a trivial bug by correcting the arch name.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Add a rough esitimation support, currently, following memory usage are
checked by this sub command:
- System RAM
- Kdump Initramfs size
- Kdump Kernel image size
- Kdump Kernel module size
- Kdump userspace user and other runtime allocated memory (currently
simply using a fixed value: 64M)
- LUKS encryption memory usage
The output of kdumpctl estimate looks like this:
# kdumpctl estimate
Reserved crashkernel: 256M
Recommanded crashkernel: 160M
Kernel image size: 47M
Kernel modules size: 12M
Initramfs size: 19M
Runtime reservation: 64M
Large modules:
xfs: 1892352
nouveau: 2318336
And if the kdump target is encrypted:
# kdumpctl estimate
Encrypted kdump target requires extra memory, assuming using the keyslot with minimun memory requirement
Reserved crashkernel: 256M
Recommanded crashkernel: 655M
Kernel image size: 47M
Kernel modules size: 12M
Initramfs size: 19M
Runtime reservation: 64M
LUKS required size: 512M
Large modules:
xfs: 1892352
nouveau: 2318336
WARNING: Current crashkernel size is lower than recommanded size 655M.
The "Recommanded" value is calculated based on memory usages mentioned
above, and will be adjusted accodingly to be no less than the value provided
by kdump_get_arch_recommend_size.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Simplfy the code and also improve the performance. udevadm call is
heavy.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
kexec-tools needs to disable CMA for kdump kernel cmdline,
otherwise kdump kernel may run out of memory.
This patch strips the inherited cma=, hugetlb_cma= cmd
line from 1st kernel, and sets to be 0 for 2nd kernel.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
apath (a D-Bus active connection path) is used for nmcli connection operations, e.g.
$ nmcli connection show $apath
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
nmcli --get-values <field> connection show /org/freedesktop/NetworkManager/ActiveConnection/1
returns the following value for the corresponding field respectively,
Field Value
IP4.DNS "10.19.42.41 | 10.11.5.19 | 10.5.30.160"
802-3-ethernet.s390-subchannels ""
bond.options "mode=balance-rr"
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
This fixes bz1854037 which happens because kexec-tools generates rd.route for
eth0 instead of for kdump-eth0,
1. "rd.route=168.63.129.16:10.0.0.1:eth0 rd.route=169.254.169.254:10.0.0.1:eth0" is passed to the dracut cmdline by kexec-tools
2. In the 2rd kernel, dracut/modules.d/35network-manager/nm-config.sh calls
/usr/libexec/nm-initrd-generator to generate two .nmconnection files
based on the dracut cmdline, i.e. kdump-eth0.nmconnection and eth0.nmconnection,
- /run/NetworkManager/system-connections/kdump-eth0.nmconnection
[connection]
id=kdump-eth0
uuid=3ef53b1b-3908-437e-a15f-cf1f3ea2678b
type=ethernet
autoconnect-retries=1
interface-name=kdump-eth0
multi-connect=1
permissions=
wait-device-timeout=60000
[ethernet]
mac-address-blacklist=
[ipv4]
address1=10.0.0.4/24,10.0.0.1
dhcp-timeout=90
dns=168.63.129.16;
dns-search=
may-fail=false
method=manual
[ipv6]
addr-gen-mode=eui64
dhcp-timeout=90
dns-search=
method=disabled
[proxy]
- /run/NetworkManager/system-connections/eth0.nmconnection
[connection]
id=eth0
uuid=f224dc22-2891-4d7b-8f66-745029df4b53
type=ethernet
autoconnect-retries=1
interface-name=eth0
multi-connect=1
permissions=
[ethernet]
mac-address-blacklist=
[ipv4]
dhcp-timeout=90
dns=168.63.129.16;
dns-search=
method=auto
route1=168.63.129.16/32,10.0.0.1
route2=169.254.169.254/32,10.0.0.1
[ipv6]
addr-gen-mode=eui64
dhcp-timeout=90
dns-search=
method=auto
[proxy]
3. Since there's eth0.nmconnection, NetworkManager will try to get an IP for eth0 regardless of the fact it's a slave NIC and time out
```
$ ip link show
2: kdump-eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 00:0d:3a:11:86:8b brd ff:ff:ff:ff:ff:ff
3: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master kdump-eth0 state UP mode DEFAULT group default qlen 1000
```
Reported-by: Huijing Hei <hhei@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
vmcore-dmesg-incomplete.txt is generated by shell redirection,
which taking the default umask value. When dmesg collector exits
with non-zero, the file will exist and anyone can have access to
it.
This patch fixed the issue by chmod the file, making it accessible
only to its owner.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
This reverts commit 5633e83318.
vm.zone_reclaim_mode may cause trashing on some machines. And after
second thought, vm.zone_reclaim_mode is barely helpful for machines
with high mem stress, so just revert it.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Kdump scirpt already have default values for core_collector, path in
many other place. Empty kdump.conf still works. Fix this corner case and
fix the error message.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Backport from upstream:
commit 0ef2ca6c9fa2f61f217a4bf5d7fd70f24e12b2eb
Author: Kazuhito Hagio <k-hagio-ab@nec.com>
Date: Thu Feb 4 16:29:06 2021 +0900
[PATCH] Show write byte size in report messages
Show write byte size in report messages. This value can be different
from the size of the actual file because of some holes on dumpfile
data structure.
$ makedumpfile --show-stats -l -d 1 vmcore dump.ld1
...
Total pages : 0x0000000000080000
Write bytes : 377686445
...
# ls -l dump.ld1
-rw------- 1 root root 377691573 Feb 4 16:28 dump.ld1
Note that this value should not be used with /proc/kcore to determine
how much disk space is needed for crash dump, because the real memory
usage when a crash occurs can vary widely.
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from upstream:
commit 6f3e75a558ed50d6ff0b42e3f61c099b2005b7bb
Author: Julien Thierry <jthierry@redhat.com>
Date: Tue Nov 24 10:45:25 2020 +0000
[PATCH 2/2] Add shorthand --show-stats option to show report stats
Provide shorthand --show-stats option to enable report messages
without needing to set a particular value for message-level.
Signed-off-by: Julien Thierry <jthierry@redhat.com>
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from upstream.
commit 3422e1d6bc3511c5af9cb05ba74ad97dd93ffd7f
Author: Julien Thierry <jthierry@redhat.com>
Date: Tue Nov 24 10:45:24 2020 +0000
[PATCH 1/2] Add --dry-run option to prevent writing the dumpfile
Add a --dry-run option to run all operations without writing the
dump to the output file.
Signed-off-by: Julien Thierry <jthierry@redhat.com>
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
If the dump target is the root disk, kdump scripts add an entry in
/etc/fstab for root disk with /sysroot as the mount point. The root
disk, passed through root=<> kernel commandline parameter, is mounted
at /sysroot in read-only mode before switching from initial ramdisk.
So, in fadump mode, a remount of /sysroot to read-write mode is needed
to capture dump successfully, because /sysroot is already mounted as
read-only based on root=<> boot parameter.
Commit e8ef4db8ff ("Fix dump_fs mount point detection and fallback
mount") removed initialization of $_op variable, the variable holding
the options the dump target was mounted with, leading to the below
error as remount was skipped:
kdump[586]: saving to /sysroot/var/crash/127.0.0.1-2021-04-22-07:22:08/
kdump.sh[587]: mkdir: cannot create directory '/sysroot/var/crash/127.0.0.1-2021-04-22-07:22:08/': Read-only file system
kdump[589]: saving vmcore failed
Restore $_op variable initialization in dump_fs() function to fix this.
Fixes: e8ef4db8ff ("Fix dump_fs mount point detection and fallback mount")
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Kairui Song <kasong@redhat.com>
This fix bz1854037 which happens because kexec-tools generates rd.route for
eth0 instead of for kdump-eth0,
1. "rd.route=168.63.129.16:10.0.0.1:eth0 rd.route=169.254.169.254:10.0.0.1:eth0" is passed to the dracut cmdline by kexec-tools
2. In the 2rd kernel,
- dracut/modules.d/40network/net-lib.sh will write /tmp/net.route.eth0 based on rd.route
- dracut/modules.d/45ifcfg/write-ifcfg.sh will copy /tmp/net.route.eth0 to /tmp/icfg and then copytree /tmp/ifcfg to /run/initramfs/state/etc/sysconfig/network-scripts
3. NetworkManager will try to get an IP for eth0 regardless of the fact it's a slave NIC and time out
```
$ ip link show
2: kdump-eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 00:0d:3a:11:86:8b brd ff:ff:ff:ff:ff:ff
3: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master kdump-eth0 state UP mode DEFAULT group default qlen 1000
```
Reported-by: Huijing Hei <hhei@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com
The `/boot` directory on some operating systems might be read-only.
If we cannot write to `$KDUMP_BOOTDIR` when generating the kdump
initrd, attempt to place the generated initrd at `/var/lib/kdump`
instead.
Signed-off by: Kelvin Fan <kelvinfan001@gmail.com>
Acked-by: Kairui Song <kasong@redhat.com>
Recently, dracut-network drops depedency on dhcp-client which requires
ipcalc. Thus the dependency chain
"kexec-tools -> dracut-network -> dhcp-client -> ipcalc"
is broken. When NIC is configured to a static IP, kexec-tools depended
on "ipcalc -m" to get netmask. This commit implements the shell
equivalent of "ipcalc -m".
The following test code shows cal_netmask_by_prefix is consistent with
"ipcalc -m",
#!/bin/bash
. dracut-module-setup.sh
for i in {0..128}; do
mask_expected=$(ipcalc -m fe::/$i| cut -d"=" -f2)
mask_actual=$(cal_netmask_by_prefix $i "-6")
if [[ "$mask_expected" != "$mask_actual" ]]; then
echo "prefix="$i, "expected="$mask_expected, "acutal="$mask_actual
exit
fi
done
echo "IPv6 tests passed"
for i in {0..32}; do
mask_expected=$(ipcalc -m 8.8.8.8/$i| cut -d"=" -f2)
mask_actual=$(cal_netmask_by_prefix $i "")
if [[ "$mask_expected" != "$mask_actual" ]]; then
echo "prefix="$i, "expected="$mask_expected, "acutal="$mask_actual
exit
fi
done
echo "IPv4 tests passed"
i=-2
res=$(cal_netmask_by_prefix "$i" "")
if [[ $? -ne 1 ]]; then
echo "cal_netmask_by_prefix should exit when prefix<0"
exit
fi
res=$(cal_netmask_by_prefix "$i" "")
if [[ $? -ne 1 ]]; then
echo "cal_netmask_by_prefix should exit when prefix<0"
exit
fi
i=33
$(cal_netmask_by_prefix $i "")
if [[ $? -ne 1 ]]; then
echo "cal_netmask_by_prefix should exit when prefix>32 for IPv4"
exit
fi
i=129
$(cal_netmask_by_prefix $i "-6")
if [[ $? -ne 1 ]]; then
echo "cal_netmask_by_prefix should exit when prefix>128 for IPv4"
exit
fi
echo "Bad prefixes tests passed"
echo "All tests passed"
Reported-by: Jie Li <jieli@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
die (in dracut-lib.sh) is supposed to be used in the initramfs environment.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
On some large systems, /sys/devices is huge and it's not a wise idea to
iterate it. `find` may cause tremendous contention on the kernfs_mutex
when there are already stress on /sys, and it will perform very very
poorly.
Simply check if drm class presents should be good enough.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Dracut's run-qemu could find which virtualization technology to the
user in the order of kvm, kqemu, userspace. Using run-qemu could allow
running tests where qemu-kvm doesn't exist.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
dracut will build the PRMs which will be installed onto the image to run
the tests.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Currently, TEST_RPMS would be only using the last RPM.
Append each RPM path to TEST_RPMs instead,
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
When nfs-utils is not installed, mounting as NFS fs would fail.
Currently, the error message is not user-friendly,
mount: /tmp/mkdumprd.HyPGpS/target: bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program.
kdump: Failed to mount on xxx for kdump preflight check.
kdump: mkdumprd: failed to make kdump initrd
Prompt the user to install nfs-utilsa in the error message,
kdump: Failed to mount on xxx for kdump preflight check. Please make sure nfs-utils has been installed.
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Also known as CVE-2021-20269. The kdump dmesg log files(kexec-dmesg.log,
vmcore-dmesg.txt) are generated by shell redirection, which take the
default umask value, making the files readable for group and others.
This patch chmod these files, making them only accessible to owner.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Previously when dumping vmcore to a remote machine through ssh,
the files are created remotely and file permissions are taken
from the default umask value, which making the files accessible to
anyone on the remote machine.
This patch fixed the security issue by setting a customized umask value
before the file creation on the remote machine.
Signed-off-by: Tao Liu <ltao@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
As FADump does not require an explicit elfcorehdr update whenever there is CPU
hotplug event so let's stop kdump service reload for FADump when CPU hotplug
event is triggered.
A new label is added to handle CPU and memory hotplug events separately. The
updated CPU hotplug event handler make sure that kdump service should not be
reloaded when FADump is configured.
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
Reviewed-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
While fadump-howto.txt talks about what happens to network interface
name on setting up a remote dump target in FADump mode, it doesn't
explicitly specify the negative consequences of it. Make it explicit
and provide a recommendation to overcome the same.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Kairui Song <kasong@redhat.com>
On ppc64le LPAR, secure-boot is a little different from bare metal,
Where
host secure boot: /ibm,secure-boot/os-secureboot-enforcing DT property exists
while
guest secure boot: /ibm,secure-boot >= 2
Make kexec-tools adapt to LPAR
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
ipcalc is needed for generating 45route-static.conf. However,
on newer Fedora, e.g. 34, dracut-network drops dependency on
dhcp-client which requires ipcalc. Make kexec-tools explicitly
depends on ipcalc.
Reported-by: Jie Li <jieli@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
The kdump kernel uses resources for ftrace because trace_buf_size, which
specifies the ring buffer size for ftrace, and trace_event, which specifies
a valid trace event, are not removed, but the kdump kernel does not require
ftrace.
trace_buf_size is ignored if the specified size is 0, so specify 1.
Signed-off-by: Hisashi Nagaoka <fj1508ic@fujitsu.com>
Acked-by: Kairui Song <kasong@redhat.com>
There is requirement to decide the recommended memory size for the current
system. And the algorithm is based on /proc/iomem, so it can align with the
algorithm used by reserve_crashkernel() in kernel.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
In fadump mode, it is also useful to observe kdump message through
console. Hence enable it.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
At present, there is no ipv6 example for nfs and ssh dump, let's
add an example to the kdump.conf.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Currently, kdump will fail to save vmcore when using the scp and ipv6.
The reason is that the scp requires IPv6 addresses to be enclosed in
square brackets, but ssh doesn’t require this.
Let's enclose the ipv6 address in square brackets for scp dump.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
There is no need to source the file manually, dracut will always
prepare the dracut lib before calling a module-setup.sh
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Sourcing logger file in kdump-lib.sh will leak kdump helper to dracut,
because module-setup.sh will source kdump-lib.sh. This will make kdump's
function override dracut's ones, and lead to unexpected behaviours.
So include kdump-logger.sh individually and only source it where it really
needed. for module-setup.sh, simply use dracut's logger helper is good
enough so just source kdump-logger.sh in kdump only scripts.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Simplify the code and fix mount point detection. The code logic is now
much simpler: if $1 is not a mount point, call "mount --target $1" again
to try mount it. "mount --target" cmd itself can handle all the /etc/fstab
parsing job, so drop the buggy and complex bash code.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
systemctl is-failed will not work after dracut isolated to the emergency
target, so this judgement is invalid. And the restart is basically
harmless, so just revert this commit.
This reverts commit ad6a93b00d.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
That commit is trying to workaround a kernel VFS bug. Now,
the VFS issue should have been fixed in all recent releases, so
remove this workaround.
This reverts commit 539bff4083.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Currently, the kdump sysconfig document is missed, let's add it to
the kexec-kdump-howto.txt as a document in order to help users better
understand these options in the /etc/sysconfig/kdump.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
The current inline comment is a bit misleading, ssh dump target don't
need to use scp as core_collector, and when using scp as core_collector,
the vmcore could be huge.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Backports:
commit 54aec3878b3f91341e6bc735eda158cca5c54ec9
Author: Alexander Egorenkov <egorenar@linux.ibm.com>
Date: Fri Sep 18 13:55:56 2020 +0200
[PATCH] make use of 'uts_namespace.name' offset in VMCOREINFO
* Required for kernel 5.11
The offset of the field 'init_uts_ns.name' has changed since
kernel commit 9a56493f6942 ("uts: Use generic ns_common::count").
Make use of the offset 'uts_namespace.name' if available in
VMCOREINFO.
Signed-off-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
Both $ipaddrs and $node can hold multiple strings, so use "" to brace them.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Currently, if saving vmcore failed, the final failure information won't
be saved to the kexec-dmesg.log, because the action of saving the log
occurs before the final log is printed, it has no chance to save the
log(marked it with the '^^^' below) to the log file(kexec-dmesg.log).
For example:
[1] console log:
[ 3.589967] kdump[453]: saving vmcore-dmesg.txt to /sysroot//var/crash/127.0.0.1-2020-11-26-14:19:17/
[ 3.627261] kdump[458]: saving vmcore-dmesg.txt complete
[ 3.633923] kdump[460]: saving vmcore
[ 3.661020] kdump[465]: saving vmcore failed
^^^^^^^^^^^^^^^^^^^^
[2] kexec-dmesg.log:
Nov 26 14:19:17 kvm-06-guest25.hv2.lab.eng.bos.redhat.com kdump[453]: saving vmcore-dmesg.txt to /sysroot//var/crash/127.0.0.1-2020-11-26-14:19:17/
Nov 26 14:19:17 kvm-06-guest25.hv2.lab.eng.bos.redhat.com kdump[458]: saving vmcore-dmesg.txt complete
Nov 26 14:19:17 kvm-06-guest25.hv2.lab.eng.bos.redhat.com kdump[460]: saving vmcore
Let's improve it in order to avoid the loss of important information.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
commit 44b073b7ec467aee0d7de381d455b8ace1199184
Author: John Ogness <john.ogness@linutronix.de>
Date: Wed Nov 25 10:10:31 2020 +0106
[PATCH 2/2] printk: use committed/finalized state values
* Required for kernel 5.10
The ringbuffer entries use 2 state values (committed and finalized)
rather than a single flag to represent being available for reading.
Copy the definitions and state lookup function directly from the
kernel source and use the new states.
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Signed-off-by: Kairui Song <kasong@redhat.com>
Backports:
commit c617ec63339222f3a44d73e36677a9acc8954ccd
Author: John Ogness <john.ogness@linutronix.de>
Date: Thu Nov 19 02:41:21 2020 +0000
[PATCH 1/2] printk: add support for lockless ringbuffer
* Required for kernel 5.10
Linux 5.10 introduces a new lockless ringbuffer. The new ringbuffer
is structured completely different to the previous iterations.
Add support for retrieving the ringbuffer from debug information
and/or using vmcoreinfo. The new ringbuffer is detected based on
the availability of the "prb" symbol.
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
systemctl -q --root "$initdir" add-wants X.target X.service is the
recommanded way to add service dependency, and it covers more corner
cases.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Currently, when generating a kdump initramfs, mkdumprd will determine
how much disk space is available, if the dump target's available space
is not greater than the total system memory, mkdumprd will print a
warning to remind that there might not be enough space to save a vmcore.
Some users are complaining that mkdumprd overestimates the needed size.
But actually, the warning covers extreme scenarios such as the slab
explodes with non-zero data or a full vmcore, etc. Therefore, need to
prevent users from having minimum disk space for crash dump.
In view of this, add some descriptions to clarify it in mkdumprd man
page.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
If dracut-initqueue failed in kdump kernel and failure action
is set to dump_to_rootfs, there is no point try again to start the
initqueue. It will also slow down the dump process, and the initqueue
will most like still not work if first attemp failed.
So just try to start sysroot.mount, if it failed, there is no luck.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
The parameter either6 is introduced to dracut by
commit 67354eebbcd4c358b8194ba5fd1ab1cf7dbd42aa
Author: Pingfan Liu <piliu@redhat.com>
Date: Tue Apr 24 16:41:21 2018 +0800
40network: introduce ip=either6 option
But it turns out needless.
On a sensible ipv6 network environment, DHCPv6 can not work properly alone,
because DHCPv6 protocol has no info about the gateway.
An reasonalbe process of ipv6 address set up should look like
host send: Router Solicitation
router reply: Router Advertisements
"Router Advertisements" carries many info like gateway, and if it has
other-config flag set, it carries DNS info etc. As for DHCPv6 address
allocation, it will only start if "Router Advertisements" has the 'managed'
flag set, which directs the host to start a stateful address allocation
from DHCPv6 server.
For more info:
rfc4861: Neighbor Discovery for IP version 6 (IPv6)
rfc5175: IPv6 Router Advertisement Flags Option
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Along with 'on' option, 'fadump=' kernel parameter also supports
'nocma' & 'off' options. Update about these missing options in the
fadump-howto.txt document.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Dracut has switch network-legacy to network-manager by default, which makes
vlan on team easy. So it can be enabled.
Testing network topology with two VMs.
VM1
ens2-\ /----> VLAN8 (192.168.120.50)
---> team0
ens3-/ (192.168.122.10)
VM2
ens2-\ /----> VLAN8 (192.168.120.100)
---> team0
ens3-/ (192.168.122.20)
Both of ens2/ens3 in VM1/VM2 are connected to virbr0.
During test, dump target is set as root@192.168.120.100:/var/crash
then crashing in VM1
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Currently get_bind_mount_source will not work on btrfs, that's because
this function relies on findmnt to detect bind mount.
For a bind mount, findmnt will return different value with "-v" option.
For example, we have /dev/sdc mounted on /mnt/source, and then bind
mount /mnt/source/sub/path to /mnt/bind:
$ findmnt /mnt/bind
TARGET SOURCE FSTYPE OPTIONS
/mnt/bind /dev/sdc[/sub/path] ext4 rw,relatime,seclabel
$ findmnt -v /mnt/bind
TARGET SOURCE FSTYPE OPTIONS
/mnt/bind /dev/sdc ext4 rw,relatime,seclabel
But findmnt also return similiar result for btrfs, on a fresh installed
Fedora 33:
$ findmnt /
TARGET SOURCE FSTYPE OPTIONS
/ /dev/sdb7[/root] btrfs rw,relatime,seclabel,ssd,space_cache,subvolid=256,subvol=/root
$ findmnt -v /
TARGET SOURCE FSTYPE OPTIONS
/ /dev/sdb7 btrfs rw,relatime,seclabel,ssd,space_cache,subvolid=256,subvol=/root
The [...] indicator will contain the subvol of btrfs as well. And if
it's bind mounted under btrfs, it will contain a mixup of btrfs subvol
and the actuall fsroot.
And also, if the bind mount source device is not mounted on /,
get_bind_mount_source will also not work.
So rewrite the get_bind_mount_source function, make it work in every
cases.
Tested with:
- Silverblue's bind mount
- Bind mount with source device mounted not under /
- Btrfs
- Bind mount and source device is Btrfs
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Remove the --real when calling findmnt.
The option is only useful in capture kernel, to avoid
`findmnt` returning the pseudo 'rootfs' for non mounted path.
example, when /kdumproot/mnt/ is not mounted:
kdump:/# findmnt --target /kdumproot/mnt
TARGET SOURCE FSTYPE OPTIONS
/ rootfs rootfs rw,size=61368k,nr_inodes=15342
kdump:/# findmnt --target /kdumproot/mnt
<return 1 and empty output>
But this function will make findmnt also return empty value for bind
mount. So remove it and add an extra if statement for second kernel.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Most watchdogs have a parameter pretimeout, if set to non-zero, it means
before the watchdog really reset the system, it will try to panic the
kernel first, so kdump could kick in, or, just print a panic stacktrace
and then kernel should reset it self.
If we are already in kdump kernel, this is not really helpful, only
increase kernel hanging chance. And it also make thing become complex
as some watchdog triggers the kernel panic in NMI context, which
could also hang the kernel in strange ways, and fail the watchdog it
self. So just disable this parameter.
Also for hpwdt, it have another parameter kdumptimeout, which is
just designed for first kernel. The default behaviour is the watchdog
will simply stop working if timeouted, trigger a panic, and leave the
kernel to kdump. Again, if we are already in kdump this is not helpful.
So also disable that.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Currently the watchdog detection code is broken already, it
get the list of active watchdog drivers, then check if they are
set in the /etc/cmdline.d/* as preload module. But after we
switched to use squash module, /etc/cmdline.d/* is not directly visible.
So just detect whether current needed driver is installed.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
In check_fs_modified, is_nfs_dump_target is already called, the dump
target can't be nfs. No need to check here.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
The driver detection have nothing to do with fs detection, and currently
if the dump target is raw, the block driver detection is skipped which
is wrong. Just split it out and run the block driver detection when dump
target is fs or raw.
Also simplfied the code a bit.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
- ssh-copy-id is bugged and not working, use a more robust way to sync
ssh keys
- systemd-resolvd will bind on port 53 so DHCP server won't work,
disable systemd-resolvd's builtin DNS server
Signed-off-by: Kairui Song <kasong@redhat.com>
Let's remove some redundant descriptions in the usage documentation
of the logger, and make it clear.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Some unused log levels have been removed, and kdump has used the
different options to control the log levels for the first kernel
and the second kernel. Therefore, let's update the kdump sysconfig
accordingly.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
In the /etc/sysconfig/kdump, we usually use the uppercase configuration
name for all options. So let's use the same method to handle this.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Let's add the rd.kdumploglvl option to control log level in the second
kernel, which can make us avoid rebuilding the kdump initramfs after we
change the log level in /etc/sysconfig/kdump.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
The kdump-logger will be used by the system service(daemons), so let's
appropriately convert the logger numeric level to syslog level with the
facility(daemon). The number is constructed by multiplying the facility
by 8 and then adding the level.
About The Syslog Protocol, please refer to the RFC5424 for more details.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Previously, the range of log level is from 1 to 6, and the TRACE
level and FATAL level are not used, therefore, let's remove these
unused log levels.
Now it has only four log levels: error(1), warn(2), info(3)
and debug(4). We have to remap the numeric log level to the logger
priority or syslog log level, which is finished in kdump-logger.sh
module, it is invisible for user.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Let's add sanity checks for the log levels in order to avoid
passing illegal log levels to the logger.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
depend() in module-setup.sh is a better place to setup dracut module
dependency, it will do early check, and fail early if needed module is
missing. Also remove a unneeded helper add_dracut_module.
Also remove the unnecessary return in depend() function.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Let's add some code comments to help better understanding, and
no code changes.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
When using ssh dump target, scp is always used, correct the comment.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Dracut only check if a module failed installtion if the module is listed
in --add params. Without this param, if kdumpbase failed to install due
to any reason, dracut will still build the initramfs only print a
warning. Add this param to ensure it fail early.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
The dracut module is opportunistic about using the built-in squashfs
support only when available, but the spec file hard requires it. Demote
it to a weak dep to truly make it optional.
This caters to environments which strive to stay minimal, like FCOS and
RHCOS. See https://github.com/coreos/fedora-coreos-config/pull/708 for
details.
Because otherwise, `kdumpctl start` will fail anyway. This makes it
easier to enable kdump by simply adding the mandatory karg and leaving
the service enabled.
This reverts commit fa8aa52d94.
For the s390x, the vmlinuz image has only single signature according
to the kernel.spec. The dual signature issue doesn't happens on s390x,
therefore, let's restore it in order to enable the file load on s390x
by default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Currently, the makedumpfile option '--message-level' is set to 1 when
dumping the vmcore, it only displays the progress indicator message,
but there are no common message and error message, it is important to
report some additional messages, especially for the error message,
which is very useful for the debugging.
In view of this, let's change the message level to 7 by default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Commit 08276e9 wrongly raise this warning message to error level, fix
this.
Fixes: 08276e9 ('Rework check_config and warn on any duplicated option')
Signed-off-by: Kairui Song <kasong@redhat.com>
Previously journalctl logs are directly dropped to save memory, but this
make journalctl unusable in kdump kernel and diffcult to debug. So
instead just don't let it read kmsg but keep other logs stored as volatile.
Kernel message are already stored in the kernel log ring buffer,
no need to let journalctl make a copy, especially when in kdump
kernel, ususlly there won't be too much kernel log overlapping
the old ring buffer.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Because the logger is introduced to output the kdump logs, need to
add a documentation for this change and describe how to use it.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Let's use the logger in the second kernel and collect the kernel ring
buffer(dmesg) of the second kernel.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Currently, the kexec option '--debug/-d' is not enabled by default, which
means that users need to set it manually and wait for the next failure to
capture the additional information.
Therefore, let's enable the option '-d' for kexec loading by default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
The kdump logger has the default values of the log levels, but
sometimes, need to change the value of log level in order to
get more debugging messages for troubleshooting.
Here, user will have a chance to reconfigure it.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Since the logger was introduced into kdump, let's enable it for kdump
so that we can output kdump messages according the log level and save
these messages for debugging.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Currently, all messages are directly printed to the console, sometimes,
we also need to output these messages to the journal log according to
the log level.
In view of this, introduce the kdump logger from the dracut module.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Instead of read and parse the kdump.conf multiple times, only read once
and use a single loop to handle the error check, which is faster.
Also check for any duplicated config otion, and error out if there are
duplicated ones.
Now it checks for following errors, most are unchanged from before:
- Any duplicated config options. (New added)
- Deprecated/Invalid kdump config option.
- Duplicated kdump target, will have a different error message of
other duplicated config options.
- Duplicated --mount options in dracut_args.
- Empty config values. All kdump configs should be in
"<config_opt> <config_value>" format.
- Check If raw target is used in fadump mode.
And removed detect of lines start with space, it will not break kdump
anyway.
The performance is measurable better than before for the check_config
function.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
On s390, if Secure-IPL is enabled, then "kexec -s -l" is required.
Otherwise kdump kernel can not be loaded.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
If failure action is set to "shell", user will need more debug info
available in kdump kernel. Especially when serial console is not
available, manually retrieve the log from journalctl is very useful
for debugging kdump issue.
Else, we can still drop journalctl content to save memory assuming
nothing will use it.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Make the test script print following line when the test is finished and vmcore is successfully dumped:
You can retrive the verify the vmcore file using following command:
./scripts/copy-from-image.sh \
/home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/0-server.img \
/var/crash/192.168.77.62-2020-09-02-05:16:26/vmcore.flat ./
Kernel package verion is: kernel-core-5.6.6-300.fc32.x86_64
Also add a helper to copy files out of the VM image.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Now, by execute `make test-run` in tests/, kexec-tools sanity tests
will be performed using VMs. There are currently 3 test cases, for local
kdump, nfs kdump and ssh kdump.
For each test VM, the selftest framework will create a snapshot layer,
do setup as required by the test case, this ensure each test runs in a
clean VM.
This framework will install a custom systemd service that starts when
system have finished booting, and the service will do basic routine
(fetch and set boot counter, etc..), then call the test case which is
installed in /kexec-kdump-test/test.sh in VM.
Each VM will have two serial consoles, one for ordinary console usage,
one for the test communication and log. The test script will watch the
second test console to know the test status.
The test cases are located in tests/scripts/testcases, documents about
the test cases structure will be provided in following commits.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
The Makefile In tests/ could help build a VM image using Fedora cloud
image as base image, or, user can specify a base image using
BASE_IMAGE=<path/to/file>. The current repo will be packeged and
installed in the image, so the image could be used as a test image to
test kexec-tools.
The image building is splited into two steps:
The first step, it either convert the base image to qcow2 or create
a snapshot on it, and install basic packages (dracut, grubby, ...)
and do basic setups (setup crashkernel=, disable selinux, ...).
See tests/scripts/build-scripts/base-image.sh for detail.
The second step, it creates a snapshot on top of the image produced by
the previous step, and install the packaged kexec-tools of current
repo. See tests/scripts/build-scripts/test-base-image.sh for detail.
In this way, if repo's content is changes, `make` will detect it and
only rebuild the second snapshot which speed up the rebuild by a lot.
The image will be located as tests/output/test-base-image, and in qcow2
format. And default user/password is set to root/fedora.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Encrypted target have many issues, so let user check
kexec-kdump-howto.txt, which have more details.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Now all atomic special workaround is removed, we can remove the atomic
detection function.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
kernel installation is not always in a fixed location /boot, there are
multiple different style of kernel installation, and initramfs location
changes with kernel. The two files should be detected together and adapt
to different style.
To do so we use a list of known installation destinations, and a list
of possible kernel image and initrd names. Iterate the two list to
detect the installation location of the two files. If GRUB is in use,
the BOOT_IMAGE= cmdline from GRUB will also be considered. And also
prefers user specified config if given.
Previous atomic workaround is no longer needed as the new detection
method can cover that case.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
There is no need to follow user's configuration when installtion the
kernel and initramfs for early kdump, just use a fixed a symbol link is
enough, this help avoid a check_boot_dir call.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Kdump service will create kdump initramfs when needed, but it won't
clean up the kdump initramfs on kernel uninstall. So create a kernel
install hook to do the clean up job.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit e5cbf4d1ad.
Currently the kernel image is being dual signed, but kexec_file_load
can't handle multiple signature right, so it will always fail. Let's
temporaryly revert "using '-s' by default", until upstream have fixed
the real issue.
Signed-off-by: Kairui Song <kasong@redhat.com>
This reverts commit 8884af04b6.
Currently the kernel image is being dual signed, but kexec_file_load
can't handle multiple signature right, so it will always fail. Let's
temporaryly revert "using '-s' by default", until upstream have fixed
the real issue.
Signed-off-by: Kairui Song <kasong@redhat.com>
It is hard to detect the time that /etc/kdump is removed. And this failure
may cause out-of-date kdump.initrd. To keep things simple, just exit if
/etc/kdump/pre.d and post.d does not exist.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
The following scenario is observed:
kdump: kdump_pre script exited with non-zero status!
[ 5.104841] systemd[1]: Shutting down.
[ 5.122162] printk: systemd-shutdow: 27 output lines suppressed due to ratelimiting
kdump: dump target is /dev/mapper/rhel_hpe--dl380pgen8--02--vm--12-root
kdump: saving to /sysroot//var/crash/127.0.0.1-2020-06-27-03:55:01/
kdump: saving vmcore-dmesg.txt
kdump: saving vmcore-dmesg.txt complete
kdump: saving vmcore
Checking for memory holes : [ 0.0 %] / Checking for memory holes : [100.0 %] | [ 5.516573] systemd-shutdown[1]: Syncing filesystems and block devices.
[ 5.519515] systemd-shutdown[1]: Sending SIGTERM to remaining processes...
It is caused by the following script
if [ $? -ne 0 ]; then
echo "kdump: kdump_pre script exited with non-zero status!"
do_final_action
fi
When do_final_action runs, a systemd service is forked for reboot, then the
subshell returns, and parent continues to execute. Place "exit 1" to stop
executing and make kdump service failure.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Checking modification against a file can not detect a removing file in
"/etc/kdump/post.d/ /etc/kdump/pre.d/". Hence it also needs the
modified time of directory to detect such changes.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Even if the directory "/etc/kdump/pre.d/" is empty, the following false
alarm can be observed during building kdump.initrd:
"/etc/kdump/pre.d/* is not executable"
Suppress it.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
If the directory /etc/kdump/pre.d is optional, then it is hard
to tell between the following cases:
-1. no directory "/etc/kdump/pre.d"
-2. "rm -rf /etc/kdump/pre.d", which removes all scripts under pre.d
For the second case, kdump.img should be rebuilt.
To bail out from this corner case, always creating pre.d and post.d
during rpm installation.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
In secure boot mode, kexec_load syscall is disabled. So, if in secure
boot mode, load kdump kernel with kexec_file_load syscall instead.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Let's use the option KEXEC_ARGS="-s" to enable the kexec file load
by default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Let's use the option KEXEC_ARGS="-s" to enable the kexec file load
by default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit 66ff48ed68.
Since the code logic is changed(which related to the option
'KDUMP_FILE_LOAD=on|off'), accordingly, need to remove the
option from the sysconfig file.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit 6a20bd5447.
Let's restore the logic of secureboot status check, and remove the
option 'KDUMP_FILE_LOAD=on|off'. We will use the option KEXEC_ARGS="-s"
to enable the kexec file load later, which can avoid failures when
the secureboot is enabled.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Since commit ca01cbd, mkdumprd can generate the initramfs even if dump
target is not mounted. But it will refuse to work if the dump target is
not marked as noauto, and not mounted.
If a target is not marked as noauto in fstab, it should be mounted
automatically on boot. If it's not mounted, most likely something is
wrong and user should fix that. So mkdumprd will just show an error
message and quit to avoid further failure.
This commit make the error message more clear and less confusing.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
If target is in format of UUID= or LABEL=, current code will fail as it
expects target to be the absoluta path to the block device. Check for
the fstype instead.
Also simplify the code logic.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Dracut get_persistent_dev function don't recognize UUID= or LABEL=
format, so caller should conver it to the path to the block device
before calling it. There is already such a helper
"kdump_get_persistent_dev", just move it to kdump-lib.sh and rename
it to reuse it,
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Currently, kernel has supported the kexec_file_load() syscall on s390x,
so I follow this commit:
6a20bd5447 ("kdump-lib: switch to the kexec_file_load() syscall on
x86_64 by default")
And also enable the kexec_file_load() syscall for kdump on s390x by
default.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Dracut expects --add-drivers with a single parameter, so mkdumprd
should quota the module list.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
commit 61e0169 changed definition of dump_fs function, so
need to do a mount target conversion before calling it.
Signed-off-by: Kairui Song <kasong@redhat.com>
This patch executes the binary and script files in /etc/kdump/{pre.d,post.d}
just like kdump_pre or kdump_post directive written in /etc/kdump.conf.
Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
This patch adds the binary and script files in /etc/kdump/{pre.d,post.d}
to modified checklist in order to update kdump initramfs when one adds
new scripts or binaries or removes the existing ones under
/etc/kdump/{pre.d, post.d}.
Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
This patch installs the binary and script files under /etc/kdump/{pre.d,post.d}
into new initramfs of kdump.
Signed-off-by: Shinichi Onitsuka <onitsuka.shinic@fujitsu.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Switch root is never used for kdump image, and this will be helpful to
reduce the initramfs size.
Also increase dracut dependency version and the function is
dracut_no_switch_root is new introduced.
This commit is applied to RHEL some time ago, but missing in Fedora as
Fedora's Dracut didn't backport this feature at that time. Now apply
this missing commit.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
On journaling filesystems like XFS, bootloader is likely to pick up
older initrd (without vmcore capture scripts) if system crashes right
after initrd update, as the bootloader (read GRUB) may not replay
filesystem log before reading the initrd from disk. Added steps to
workaround that problem.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Usually kdump kernel don't need a swiotlb. There are two cases known
common case a swiotlb is needed:
- SEV/SME is active.
- crashkernel high reservation is used, or there are crashkernel memory
above 4G.
For both case, if swiotlb is set to a large value in first kernel, kdump
kernel will likely to fail with OOM. So don't inherit swiotlb parameter
by default.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Upstream dracut now use network-manager module by default and since
upstream commit 3dcaa97, network-manager expects user to pass "rd.neednet"
to indicate network is required.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit cee618593c.
Upstream dracut have provided a parameter for adding mandantory network
requirement by appending "rd.neednet" parameter, so we should use that
instead.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
The option 'log_buf_len' in the kernel command line will make kernel
dynamically allocate memory (system ram) from memblock and always hold
the memory allocated. If the size of this option is misused such as
'log_buf_len=64M'. This may cause a short of memory for kdump kernel,
because the size of memory is usually limited in kdump kernel.
In order to avoid this problem, need to remove this option from the
kdump kernel command line.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Previous commit ca01cbd ('mkdumprd: generate usable kdump initramfs
even target is not mounted') applied a wrong version of patch, let me
just fix it.
Signed-off-by: Kairui Song <kasong@redhat.com>
When adding doc for the non-mounted dump target support, I found the
document are a bit uneasy to read due to lack of a proper format, this
commit should it make looks better.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Currently kexec-tools always depend on dump target to be mounted, which
caused some inconvenience for setup.
So for user configured target, allow kdump to start and build initramfs
even if target is not mounted.
When a mounted user configured target is used, the behavior is not
changed.
When a unmounted user configured target is used, mkdumprd will look for
corresponding mount info in fstab, and a entry with noauto option is
founded, mkdumprd will try to mount it inplace with optoins specified
in fstab and do basic checks on the device, then umount it.
If there is no fstab entry, mkdumprd will try to mount it in temporary
path with defaults option, do same basic check and umount it.
If there is a fstab entry but "noauto" option is not used, then there
must be some reason that the target device is not mounted, mkdumprd will
error out.
When path based target is used, there is no behavior change.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Use get_mount_info so that fstab is used as a failback when look for
mount info.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Ignore mount check in kdumpctl, mkdumprd will still fail building and
exit if target is not mounted.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Use is_mounted helper instaed of calling findmnt directly or checking if
"mount" value is empty.
If findmnt looks for fstab as well, some non mounted entry will also
return value. Required to support non-mounted target.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
User a helper to get the path to mount dump target in kdump kernel, and
fix duplicated '/' in the mount path problem.
Fixes: bz1785371
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
By this point, there is still an unresolved vfs kernel issue that blocks
systemd from mounting the dump target properly from time to time. To
prevent systemd from failing by mounting the dump target, we can add
nofail option to the kdump mount point.
But adding nofail will wipe out default dependency of the mount point,
see commit 94a7b43, so systemd randomize the order of calling kdump.sh
and mounting the dump target and lead to unexpected behavior.
However we can use x-systemd.before to ensure the mount is done
in right order.
In dracut-kdump-capture.service, we have "After=initrd.target",
and look at dracut.bootup.7, systems start processing of fstab by
initrd-fs.target, so set "After=initrd.target" could ensure the mount
is ready before pre-pivot and kdump-capture service.
Now with both nofail and x-systemd.before=initrd.target, systemd
will try to mount the dump target before calling kdump, and even if the
mount failed, kdump.sh will still be called and try to mount again. See
dump_fs function, which will try to mount if the target is not mounted.
Kdump will only fail if both mount attemp fails.
Else if the kdump target mount failed or unstable, systemd will directly
jump to kdump failure action, and kdump fails.
This should improve the robustness in general with no other risk.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
In second kernel, kdump always prints redundant '/':
kdump: saving to /sysroot//var/crash/127.0.0.1-2020-03-12-21:32:54/
Just trim it.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This partially reverts commit 6dee286467.
There are reports that NFSv3 is failing after this commit, and after
more debug, I found NFSv4 may not work properly if
"nfs4_disable_idmapping" is set to 0.
The cause of the failure is that kdump.sh runs after dracut's pre-pivot
and clean up hook, many dracut module will install hooks to kill some
running services, so if the dump target requires a service to be running
but it's killed, mount will fail.
Dracut ensures the configured mount points are ready before pre-pivot.
After pre-pivot, any further mounting operation may not work as expected.
Although there is no report of other type of dump target failure except
NFSv3, it's better to revert this, to avoid other potential risk, and wait
for a proper fix for that systemd/kernel issue.
Else, this may bring more trouble for further development.
But still keep the change in kdump-lib-initramfs.sh for better
robustness.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Add recommendations on how much memory is required for FADump. Also,
mention the optimizations applied to default initrd when FADump is
used and how to workaround it.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
In most cases, it always provides a persistent MAC address. But for
the s390 Arch, sometimes, kernel could run in the LPAR mode and it
doesn't provide a persistent MAC address, which caused the kdump
failure.
Currently, some rules rely on the persistent MAC address, for the
above case, which won't work in kdump kernel because non-persistent
MAC could not match with udev rules.
To fix this issue, need to add a new option 'rd.znet_ifname' in order
to provide extra parameters such as 'ifname' and 'subchannels' for
some rules, which ensures kdump can also work appropriately without
the persistent MAC. Please refer to the following commit in dracut:
872eb69936bd ("95znet: Add a rd.znet_ifname= option")
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Since commit 6dee286467 ("Don't mount the dump target unless needed"),
dump_fs function unmounts the dump target just after saving vmcore.
This broke the condition that it's mounted when executing "kdump_post",
which had been stable since RHEL5, and a certain tool which uses the
kdump_post hook to save information of 2nd kernel to the dump target
started to fail.
As unmounting it is done by systemd-shutdown before reboot without
the umount command as below, so let's don't unmount it in dump_fs.
systemd-shutdown[1]: Unmounting file systems.
[547]: Remounting '/sysroot' read-only in with options '(null)'.
EXT4-fs (dm-0): re-mounted. Opts: (null)
[548]: Unmounting '/sysroot'.
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Acked-by: Kairui Song <kasong@redhat.com>
pcs cluster and cluster cib-upgrade may throw some information and disturb
the parsing. Mute them
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
There is a race issue between "pcs" and "kdumpctl restart"
-1. set up cluster
# pcs cluster setup --start mycluster node1 node2
# pcs stonith create kdump fence_kdump pcmk_reboot_action="off"
# pcs stonith level add 1 node1 kdump
# pcs stonith level add 1 node2 kdump
-2. Then here comes the command _immediately_ in kdumpctl
# pcs cluster cib
But due to some pcs internal mechanism, "pcs cluster cib" can not
fetch the updated info in time.
Fix these issue by forcing the upgrade of cib.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
If user configured target is used, path should be used as the absolute
path within the dump target direct, and user should be fully aware of
the path structure within the target device. The adjust_bind_mount_path
call here make it very hard to control the behavior.
Especially, if it's a cross device bind mount, this will likely create a
invalid path in the target. And for atomic case, adjust_bind_mount_path call
here assumes user will always pass root device as the explicitly configured
dump target, which is not true.
If user configured target device is used, the path is always be the
absolute path inside of given target. If user don't know about the path
structure in the target device, then user should either use the path
based config, or carefully exam the target device before using it as a
dump target.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
This commit remove almost all special workaround for atomic, and treat
all bind mounts in any environment equally.
Use a helper get_bind_mount_directory_from_path to get the bind mount
source path of given path.
is_atomic function now only used to determine the right /boot path
for atomic/silverblue environment.
And remove get_mntpoint_from_path(), it's the only function that never
ignore bind mount, and it have no caller after this clean up.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
For user specified target, the config value is used as the dump target,
and SAVE_PATH (path in kdump.conf) value is used as the dump path within
the dump target, no need to do anything extra with the path value.
Current code logic is not only complicated, it also wrongly generate
an redundantly long path in atomic/silverblue environment.
The right way is only check two things, and do nothing else:
1. The path exists within the target;
2. The target is large enough to hold to contain the vmcore.
Currently checking the target still requires it to be mounted so it will
error out if it's not mounted. Will implement some auto mount as next
step.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
get_save_path provides default value fail back and error check, no need
to repeat it again.
Also remove a redundant echo and grep in get_save_path
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
It's basically same with is_user_configured_dump_target and only have
one caller. And the name is confusing, the dump target is always
configured, it's either user configured or path based.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
In /etc/hosts, the alias name can come at the 2nd column, regardless of the
recommendation.
E.g. the following format is valid although not recommended
cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.22.21 fastvm-rhel-7-6-21 fastvm-rhel-7-6-21.localdomain
192.168.22.22 fastvm-rhel-7-6-22 fastvm-rhel-7-6-22.localdomain
192.168.22.21 node1_hb
192.168.22.22 node2_hb
So filtering out both 2nd and 3rd column for matching.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Process substitution is not POSIX standard syntax, so if bash is configured
to strictly follow POSIC, this will fail.
Just use a POSIX friendly syntax instead.
Fixes: bz1708321
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
This help remove redundant spaces and tailing comment in installed
kdump.conf, currently installed kdump.conf always contain extra empty
lines.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
Now modinfo will return "(builtin)" instead of empty string for builtin
module. Sync the code logic.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Since Dracut commit a0d9ad6 loaded-kernel-modules is renamed to
hostonly-kernel-modules and contains all hostonly modules. So check
hostonly-kernel-modules instead for module change.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Latest dracut release stopped creating
$systemdsystemunitdir/initrd.target.wants dir for us, so ensure it
exists before creating the symlink.
Signed-off-by: Kairui Song <kasong@redhat.com>
Tested-and-Reviewed-by: Bhupesh Sharma <bhsharma@redhat.com>
When building kdump initramfs for a SSH dump target, mkdumprd would
check whether it has the write permission on the SSH Server's
$DUMP_TARGET.
However $DUMP_TARGET is missing in the actual error message when the
user doesn't not have the write permission. For example:
# kdumpctl restart
kexec: unloaded kdump kernel
Stopping kdump: [OK]
Could not create temporary directory on :/home/bhsharma/test. Make
sure user has write permission on destination
mkdumprd: failed to make kdump initrd
Starting kdump: [FAILED]
This patch using $1 value passed to mkdumprd, to print the
$DUMP_TARGET inside mkdir_save_path_ssh() function to
fix the issue.
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Currently, is_user_configured_dump_target() doesn't work as expected
due to lack of grep -E option.
As a result, kdump service with a ssh dump configuration can unnecessarily
fail to start due to the non-existence of a directory at where the path
option specifies on the local system:
kdumpctl[9760]: Rebuilding /boot/initramfs-5.4.19-200.fc31.x86_64kdump.img
kdumpctl[9760]: Dump path /var/crash/ssh does not exist.
kdumpctl[9760]: mkdumprd: failed to make kdump initrd
kdumpctl[9760]: Starting kdump: [FAILED]
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Acked-by: Kairui Song <kasong@redhat.com>
In order to check whether the specified makedumpfile parameters are
valid or not when generating initramfs, use the --check-params option,
which was recently added.
With the patch, kdumpctl can point out mistakes in core_collector
option and failed. For example, if there is an practical mistake
that dump_level is -1:
# cat /etc/kdump.conf
core_collector makedumpfile -l --message-level 1 -d -1
# kdumpctl start
Detected change(s) in the following file(s):
/etc/kdump.conf
Rebuilding /boot/initramfs-5.4.19-200.fc31.x86_64kdump.img
Dump_level(-1) is invalid.
makedumpfile parameter check failed.
mkdumprd: failed to make kdump initrd
Starting kdump: [FAILED]
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from the upstream makedumpfile devel branch.
commit 989152e113bfcb4fbfbad6f3aed6f43be4455919
Author: Kazuhito Hagio <k-hagio-ab@nec.com>
Date: Tue Feb 25 16:04:55 2020 -0500
[PATCH] Introduce --check-params option
Currently it's difficult to check whether a makedumpfile command-line
is valid or not without an actual panic. This is inefficient and if
a wrong configuration is not tested, you will miss the vmcore when an
actual panic occurs.
In order for kdump facilities like kexec-tools to be able to check
the specified command-line parameters in advance, introduce the
--check-params option that only checks them and exits immediately.
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Acked-by: Kairui Song <kasong@redhat.com>
(1) explain early kdump a little clearer in "Introduction"
(2) move the notes out to a new "Notes" section for readability and
add a note about need of reconfiguration after kernel update
(3) change journalctl -x option to -b option because -x is unnecessary
and -b will make it very faster if persistent journal is available
(4) shorten the example messages for readability
(5) add a note to "Limitation" about the earliness of early kdump
Signed-off-by: Kazuhito Hagio <k-hagio-ab@nec.com>
Acked-by: Lianbo Jiang <lijiang@redhat.com>
For step2 in early-kdump-howto.txt, --force option of dracut
is necessary to rebuild system initramfs. Without --force option,
executing step2 fails because system initramfs already exists.
Signed-off-by: Shigeki Morishima <s.morishima@jp.fujitsu.com>
Acked-by: Kairui Song <kasong@redhat.com>
Currently while trying to save vmcore via vlan eth interface, the Kdump
kernel fails with network unreachable message.
This is because mkdumprd produces a vlan config that does not get
ip address for vlan on eth device.
Fix the same via this patch.
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
With FADump support added on POWERNV paltform, enable the scripts to
capture /proc/vmcore. Also, if CONFIG_OPAL_CORE is enabled, OPAL core
is preserved and exported on POWERNV platform. So, offload OPAL core,
if it is available.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Kairui Song <kasong@redhat.com>
UEFI Secure boot is a signature verification mechanism, designed to
prevent malicious code being loaded and executed at the early boot
stage. This makes sure that code executed is trusted by firmware.
Previously, with kexec_file_load() interface, kernel prevents unsigned
kernel image from being loaded if secure boot is enabled. So kdump will
detect whether secure boot is enabled firstly, then decide which interface
is chosen to execute, kexec_load() or kexec_file_load(). Otherwise unsigned
kernel loading will fail if secure boot enabled, and kexec_file_load() is
entered.
Now, the implementation of kexec_file_load() is adjusted in below commit.
With this change, if CONFIG_KEXEC_SIG_FORCE is not set, unsigned kernel
still has a chance to be allowed to load under some conditions.
commit 99d5cadfde2b ("kexec_file: split KEXEC_VERIFY_SIG into KEXEC_SIG
and KEXEC_SIG_FORCE")
And in the current Fedora, the CONFIG_KEXEC_SIG_FORCE is not set, only the
CONFIG_KEXEC_SIG and CONFIG_BZIMAGE_VERIFY_SIG are set on x86_64 by default.
It's time to spread kexec_file_load() onto all systems of x86_64, including
Secure-boot platforms and legacy platforms. Please refer to the following
form.
.----------------------------------------------------------------------.
| . | signed kernel | unsigned kernel |
| . types |-----------------------|-----------------------|
| . |Secure boot| Legacy |Secure boot| Legacy |
| . |-----------|-----------|-----------|-----------|
| options . | prev| now | prev| now | | | prev| now |
| . |(file|(file|(only|(file| prev| now |(only|(file|
| . |load)|load)|load)|load)| | |load)|load)|
|----------------------|-----|-----|-----|-----|-----|-----|-----|-----|
|KEXEC_SIG=y | | | | | | | | |
|SIG_FORCE is not set |succ |succ |succ |succ | X | X |succ |succ |
|BZIMAGE_VERIFY_SIG=y | | | | | | | | |
|----------------------|-----|-----|-----|-----|-----|-----|-----|-----|
|KEXEC_SIG=y | | | | | | | | |
|SIG_FORCE is not set | | | | | | | | |
|BZIMAGE_VERIFY_SIG is |fail |fail |succ |fail | X | X |succ |fail |
|not set | | | | | | | | |
|----------------------|-----|-----|-----|-----|-----|-----|-----|-----|
|KEXEC_SIG=y | | | | | | | | |
|SIG_FORCE=y |succ |succ |succ |fail | X | X |succ |fail |
|BZIMAGE_VERIFY_SIG=y | | | | | | | | |
|----------------------|-----|-----|-----|-----|-----|-----|-----|-----|
|KEXEC_SIG=y | | | | | | | | |
|SIG_FORCE=y | | | | | | | | |
|BZIMAGE_VERIFY_SIG is |fail |fail |succ |fail | X | X |succ |fail |
|not set | | | | | | | | |
|----------------------|-----|-----|-----|-----|-----|-----|-----|-----|
|KEXEC_SIG is not set | | | | | | | | |
|SIG_FORCE is not set | | | | | | | | |
|BZIMAGE_VERIFY_SIG is |fail |fail |succ |succ | X | X |succ |succ |
|not set | | | | | | | | |
----------------------------------------------------------------------
Note:
[1] The 'X' indicates that the 1st kernel(unsigned) can not boot when the
Secure boot is enabled.
Hence, in this patch, if on x86_64, let's use the kexec_file_load() only.
See if anything wrong happened in this case, in Fedora firstly for the
time being.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
The dracut initqueue may quit immediately and won't trigger any hook if
there is no "finished" hook still pending (finished hook will be deleted
once it return 0).
This issue start to appear with latest dracut, latest dracut use
network-manager to configure the network,
network-manager module only install "settled" hook, and we didn't
install any other hook. So NFS/SSH dump will fail. iSCSI dump works
because dracut iscsi module will install a "finished" hook to detect if
the iscsi target is up.
So for NFS/SSH we keep initqueue running until the host successfully get
a valid IP address, which means the network is ready.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
sed and awk is heavily used everywhere in the code, but it's not
explicitely installed by kdump dracut module. If the module in dracut
stop installing them (which already happened with latest dracut
upstream), kdump will break.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
For ssh/nfs dump, kdump need the 'ip' tool to get the host ip address
for naming the vmcore. But kdump-module-setup.sh never installed this
tool. kdump-module-setup.sh worked so far as dracut network module will
help install it.
After dracut changed to use 35network-manager for network setup, "ip"
command won't be installed in second kernel by default. So need to
ensure "ip" is installed when installing kdump dracut module.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Previously is_nfs_dump_target didn't cover the case that 'path <value>'
where <value> points to a nfs mount point. This function is never used
in first kernel before so so far it worked.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
This help deduplicate the code. Use a single function instead of
repeat the same logic.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Some of echo $(...) code segment is pointless, just call the command
directly, and remove some useless if / return statement.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Previous commit f13eab6 ('mkdumprd: simplify dracut args parsing')
break dracut arguments parsing for some use case, this should fix it
well.
Passed nfs/local/iscsi/ssh dump test, and with extra dracut_argss.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Currently "systemctl --fail --no-block default" will be executed on
kdump-error-handler exit due to the config. This may makes systemd try
to isolate to the default target again.
The execute chain will be like:
initrd.target -> kdump.sh (failed) -> kdump-error-handler.service ->
failure_action -> final_action -> ExecStopPost (go to initrd.target again)
Currently, reboot/shutdown/halt is called by either by executing
failure_action or final_action. So the loop will be stopped. However
nont of the reboot/shutdown/halt call is blocking so it might lead to
race issue.
Just drop the ExecStopPost to fix this potential issue.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Tested-by: HATAYAMA Daisuke <d.hatayama@fujitsu.com>
Previously dracut_args is stored as an array in the shell code,
so we can just pass this array as argument to dracut.
But when trying to append new dracut argument, we have to manually
handle the quotes and spaces to ensure the appended argument is
splitted into array elements in the right way.
This is complex and hard to read or maintain. Instead, just store the
dracut_args as a plain string, and let xargs help parse it and call
dracut.
Simply passing $dracut_args or "$dracut_args" will either let dracut
consider the whole string as a single argument, or loss all the
quotes. xargs can handle it well and cover more corner cases.
Eg. one corner case before, if we have:
dracut_args --mount "/dev/sda1 /mnt/test xfs rw"
Kdump will fail, because the function add_dracut_arg() will wrongly
change the arguments into: (Notice the extra space.)
dracut_args --mount " /dev/sda1 /mnt/test xfs rw"
Instead of fixing it just use xargs instead. Tested with above config
and multiple other dracut_args values.
Resolves: bz1700136
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
By default kernel have vm.zone_reclaim_mode = 0 and large page
allocation might fail as kernel is very conservative on memory
reclaiming. If the page allocation failure is not handled carefully
it could lead to more serious problems.
This issue can be reproduced by change with following steps:
- Fill up page cache use:
# dd if=/dev/urandom of=/test bs=1M count=1300
- Now the memory is filled with write cache:
# free -m
total used free shared buff/cache available
Mem: 1790 184 132 2 1473 1348
Swap: 2119 7 2112
- Insert a module which simply calls "kmalloc(SZ_1M, GFP_KERNEL)" for
512 times: (Notice: vmalloc don't have such problem)
# insmod debug_module.ko
- Got following allocation failure:
insmod: page allocation failure: order:8, mode:0x40cc0(GFP_KERNEL|__GFP_COMP), nodemask=(null),cpuset=/,mems_allowed=0
- Clean up and repeat again with vm.zone_reclaim_mode = 3, OOM is not
observed.
In kdump kernel there is usually only one online CPU and limited memory,
so we set vm.zone_reclaim_mode = 3 to let kernel reclaim memory more
aggresively to avoid such issue.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
When large amount of memory, about 1TB, is removed with DLPAR memory
remove operation, kdump reload could fail due to race condition with
device tree property update. In such scenario, the subsequent kdump
reload requests would also fail as reload() only proceeds if current
load status is active. Since the possibility of this race condition
couldn't be wished away due to the nature of the scenario, workaround
it by proceeding to load even if current load status is not active as
long as kdump service is active, which kdump udev rules already check
for.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Kairui Song <kasong@redhat.com>
If failure_action is shutdown/reboot/halt, final_action is pointless as
the system will be already stopping. And if final_action is different
from failure_action, it will trigger a systemd race problem and cause
unexpected behavior to occur.
So let the error handler stop and exit after performing failure_action
successfully if failure_action is one of shutdown/reboot/halt.
This way, final_action will not be executed.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Merge kdump_setup_netdev into kdump_install_net.
kdump_install_net is a wrapper of calling kdump_setup_netdev, and
it do following three extra things:
1. Sanitize and resolve the hostname
2. Resolve the route to the destination
3. Set the default gateway for once
There is currently only one caller of kdump_setup_netdev, the iscsi
network setup code, and it's doing 1 and 2 by itself. And there should
only be one default gateway in kdump enviroment, so applying 3 here is
fine.
And the comment of kdump_install_net is wrong and obsoleted, update the
comment too.
Just merge kdump_setup_netdev into kdump_install_net and always use
kdump_install_net instead.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
check_size checks if the specified dump path on the ssh target have
enough space. And if the path doesn't exits, it will fail and exit.
mkdir_save_path_ssh should be called first to check if the path
exists, and create the path if it doesn't exits, so the size check
can always work properly.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
In commit a431a7e354 (module-setup: fix 99kdumpbase network dependency),
the statement for OR operation is still wrong.
The OR condition statement should be: if a || b
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
In kdump.conf, if sshkey points to an invalid ssh key, 'kdumpctl restart'
can bail out immediately instead of retry.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
For fadump, this helps to reduce the risk of boot failure, and
may also help speed up the boot by a bit.
For normal kdump, this will delay the dump target mounting, and no
longer depend on systemd to do the mounting job.
And currently there is a failure that caused by some mount handling
bug with kernel and systemd that is failing the system booting:
[FAILED] Failed to mount /kdumproot/home.
See 'systemctl status kdumproot-home.mount' for details.
[DEPEND] Dependency failed for Local File Systems.
[ OK ] Reached target Remote File Systems (Pre).
[ OK ] Reached target Remote File Systems.
Starting udev Coldplug all Devices...
Starting Create Volatile Files and Directories...
Starting Kdump Emergency...
This patch can bypass it. The fix of root cause is still WIP, but this
patch itself is a nice to have optimization so it's reasonable to do so.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
When trying to setup kdump for fedora-coreos, kdumpctl start fails to
find the correct boot directory since BOOT_IMAGE start with the grub
device name
Signed-off-by: Yuval Turgeman <yturgema@redhat.com>
Print some message during the long wait period to reflect the process.
The message will look like:
Network dump target is not usable, waiting for it to be ready
...
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from the makedumpfile devel branch in upstream.
commit 8425342a52b23d462f10bceeeb1c8a3a43d56bf0
Author: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Date: Fri Sep 6 09:50:34 2019 -0400
[PATCH] Fix inconsistent return value from find_vmemmap()
When -e option is given, the find_vmemmap() returns FAILED(1) if
it failed on x86_64, but on architectures other than that, it is
stub_false() and returns FALSE(0).
if (info->flag_excludevm) {
if (find_vmemmap() == FAILED) {
ERRMSG("Can't find vmemmap pages\n");
#define find_vmemmap() stub_false()
As a result, on the architectures other than x86_64, the -e option
does some unnecessary processing with no effect, and marks the dump
DUMP_DH_EXCLUDED_VMEMMAP unexpectedly.
Also, the functions for the -e option return COMPLETED or FAILED,
which are for command return value, not for function return value.
So let's fix the issue by following the common style that returns
TRUE or FALSE, and avoid confusion.
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from the makedumpfile devel branch in upstream.
commit b461971bfac0f193a0c274c3b657d158e07d4995
Author: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Date: Thu Aug 29 14:51:56 2019 -0400
[PATCH] Fix exclusion range in find_vmemmap_pages()
In the function, since pfn ranges are literally start and end, not start
and end+1, if the struct page of endpfn is at the last in a vmemmap page,
the vmemmap page is dropped by the following code, and not excluded.
npfns_offset = endpfn - vmapp->rep_pfn_start;
vmemmap_offset = npfns_offset * size_table.page;
// round down to page boundary
vmemmap_offset -= (vmemmap_offset % pagesize);
We can use (endpfn+1) here to fix.
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from the makedumpfile devel branch in upstream.
commit aa5ab4cf6c7335392094577380d2eaee8a0a8d52
Author: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Date: Thu Aug 29 12:26:34 2019 -0400
[PATCH] x86_64: Fix incorrect exclusion by -e option with KASLR
The -e option uses info->vmemmap_start for creating a table to determine
the positions of page structures that should be excluded, but it is a
hardcoded value even with KASLR-enabled vmcore. As a result, the option
excludes incorrect pages from it.
To fix this, get the vmemmap start address from info->mem_map_data.
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Acked-by: Kairui Song <kasong@redhat.com>
On a host with ipaddr not ready before kdump service, ssh return errno 255.
While if no ssh-key, ssh also return errno 255. For both of cases, the
current kdump code promote user to run 'kdumpctl propagate'. This confuses
user who already installs ssh-key.
In order to tell these two cases from each other, the ssh warning message
should be involved, and parsed.
For the no ssh-key case , warning message is "Permission denied" or "No
such file or directory". For the other, warning message is "Network
Unreachable"
This patch also does a slight change to enlarge the timeout from 60s to
180s. This value can meet test at the time being
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Currently there are two issues with device dump:
- It may use too much memory
- kdump won't automatically include required driver in second kernel
User should manually reserve enough memory, and include the required
driver by using extra_modules. Add some notes about the issues in
kexec-kdump-howto.txt
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Device dump may use a log of memory and cause OOM issue, so append
"novmcoredd" option for second kernel and disable it by default.
To use device dump, user should remove the vmcoredd parameter
manually.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Bond options in ifcfg is space separated, dracut expected it to be comma
separated, so it have to be parsed and converted during initramfs
building.
The currently parsing and convert pattern is flawed, for example:
" downdelay=0 miimon=100 mode=802.3ad updelay=0 "
is converted to :
":,downdelay=0 miimon=100 mode=802.3ad updelay=0 "
should be:
":downdelay=0,miimon=100,mode=802.3ad,updelay=0"
So fix this issue by using more simple but robust method for processing
the options.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
The localhost is filtered out in case of is_pcs_fence_kdump, do it too in
case of is_generic_fence_kdump.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
'hostname -A' can not get the alias, meanwhile 'hostname -a' is deprecated.
So we should do it by ourselves.
The parsing is based on the format of /etc/hosts, i.e.
IP_address canonical_hostname [aliases...]
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
If dump target is ipv6 address, a host should have ipv6 address ready
before starting kdump service. Otherwise, kdump service fails to start due
to the failure "ssh dump_server_ip mkdir -p $SAVE_PATH".
And user can see message like:
"Could not create root@2620:52:0:10da:46a8:42ff:fe23:3272/var/crash"
I observe a long period (about 30s) on some machine before they got ipv6
address dynamiclly, which is never seen on ipv4 host.
Hence kdump service has a dependency on ipv6 address. But there is no good
way to resolve it. One way is asking user to run the cmd "nmcli connection
modify eth0 ipv6.may-fail false". But this will block systemd until ipv6
address is ready. Despite doing so, kdump can try its best (wait 1 minutes
after it starts up) before failure.
How to implement the wait is arguable. It will involve too many technique
details if explicitly waiting on ipv6 address, instead, just lean on 'ssh'
return value to see the availability of network.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from the makedumpfile devel branch in upstream.
commit 7bdb468c2c99dd780c9a5321f93c79cbfdce2527
Author: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Date: Tue Jul 23 12:24:47 2019 -0400
[PATCH] Increase SECTION_MAP_LAST_BIT to 4
kernel commit 326e1b8f83a4 ("mm/sparsemem: introduce a SECTION_IS_EARLY
flag") added the flag to mem_section->section_mem_map value, and it caused
makedumpfile an error like the following:
readmem: Can't convert a virtual address(fffffc97d1000000) to physical address.
readmem: type_addr: 0, addr:fffffc97d1000000, size:32768
__exclude_unnecessary_pages: Can't read the buffer of struct page.
create_2nd_bitmap: Can't exclude unnecessary pages.
To fix this, SECTION_MAP_LAST_BIT needs to be updated. The bit has not
been used until the addition, so we can just increase the value.
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from the makedumpfile devel branch in upstream.
commit c1b834f80311706db2b5070cbccdcba3aacc90e5
Author: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Date: Tue Jul 23 11:50:52 2019 -0400
[PATCH] Do not proceed when get_num_dumpable_cyclic() fails
Currently, when get_num_dumpable_cyclic() fails and returns FALSE in
create_dump_bitmap(), info->num_dumpable is set to 0 and makedumpfile
proceeds to write a broken dumpfile slowly with incorrect progress
indicator due to the value.
It should not proceed when get_num_dumpable_cyclic() fails.
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
fadump will alter the normal boot initramfs and we don't want a normal
boot to foward and drop the journalctl logs.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
When building for i386, an error occured:
kexec/arch/i386/kexec-x86.c:39:22: error: 'multiboot2_x86_probe'
undeclared here (not in a function); did you mean 'multiboot_x86_probe'?
39 | { "multiboot2-x86", multiboot2_x86_probe, multiboot2_x86_load,
| ^~~~~~~~~~~~~~~~~~~~
| multiboot_x86_probe
kexec/arch/i386/kexec-x86.c:39:44: error: 'multiboot2_x86_load'
undeclared here (not in a function); did you mean 'multiboot_x86_load'?
39 | { "multiboot2-x86", multiboot2_x86_probe, multiboot2_x86_load,
| ^~~~~~~~~~~~~~~~~~~
| multiboot_x86_load
kexec/arch/i386/kexec-x86.c:40:4: error: 'multiboot2_x86_usage'
undeclared here (not in a function); did you mean 'multiboot_x86_usage'?
40 | multiboot2_x86_usage },
| ^~~~~~~~~~~~~~~~~~~~
| multiboot_x86_usage
Fix this issue by putting the definition in the right header, also tidy
up Makefile.
Signed-off-by: Kairui Song <kasong@redhat.com>
The current code only exclude the hostname, while localhost can have alias in
/etc/hosts. All of the alias should be excluded from the fence dump node to
avoid deadlock issue.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Squash module is used to save memory. For fadump this is not neccessary
and may slow down the build time, and make it more fragile.
fadump initramfs is used for normal boot as well, although squash module
is capable of being used for generic normal boot, but there are cases
where is doesn't work well. So disable it and make fadump more robust.
Signed-off-by: Kairui Song <kasong@redhat.com>
Tested-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Dave Young <dyoung@redhat.com>
Don't use any log storage and forward to console directly, this make
console output more useful, and also save more memory. On a fresh
installed Fedora 30 it saved ~5M of memory, and the amount of log being
printed to console is still accetable.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Some firmware will provide a ACPI HEST table with massive amount of
entries, and the way how kernel handles these entries will consume a lot
of memory which will lead to OOM issue in kdump kernel. During testing
on certain machine, disable HEST saved ~60M of memory.
Kdump is only for emergency use in case of a kernel panic, so temporarily
disable hardware error report & recovery related feature is acceptable
in general. So disable HEST support in kdump kernel to save memory.
Currently such issue is only observed on x86_64, so limit this change to
x86_64 only.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
systemd has the following message "OnFailureIsolate is deprecated. Please
use OnFailureJobMode= instead"
Changing the file to meet systemd's requirement
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Backport from the makedumpfile devel branch in upstream.
commit d222b01e516bba73ef9fefee4146734a5f260fa1 (HEAD -> devel)
Author: Lianbo Jiang <lijiang@redhat.com>
Date: Wed Jan 30 10:48:53 2019 +0800
[PATCH] x86_64: Add support for AMD Secure Memory Encryption
On AMD machine with Secure Memory Encryption (SME) feature, if SME is
enabled, page tables contain a specific attribute bit (C-bit) in their
entries to indicate whether a page is encrypted or unencrypted.
So get NUMBER(sme_mask) from vmcoreinfo, which stores the value of
the C-bit position, and drop it to obtain the true physical address.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Looking at the difference of the x86_64 and aarch64 kdump.sysconfig
options for Fedora, one can see the following options which are
different:
Present in kdump.sysconfig.x86_64 but not in kdump.sysconfig.aarch64:
---------------------------------------------------------------------
cgroup_disable=memory
mce=off
numa=off
udev.children-max=2
panic=10
acpi_no_memhotplug
transparent_hugepage=never
nokaslr
Present in kdump.sysconfig.aarch64 but not in kdump.sysconfig.x86_64:
---------------------------------------------------------------------
swiotlb=noforce
After going through all the options, it makes sense to add the
following options added to kdump.sysconfig.aarch64:
KDUMP_COMMANDLINE_APPEND="cgroup_disable=memory udev.children-max=2
panic=10 irqpoll nr_cpus=1
swiotlb=noforce reset_devices"
This has helped reduce the memory footprint of crashkernel on several
aarch64 machines available in the beaker lab. For e.g. I was seeing
OOM issues on large aws ec2 instances with the default crashkernel size
of 512M, and I had to use an increased crashkernel size of 786M on the
same to boot the crash dump kernel.
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
'maxcpus' setting normally don't work on several kdump enabled systems
due to a known udev issue.
Currently the fedora kdump configuration is set as the following on the
aarch64 systems:
# cat /etc/sysconfig/kdump
<..snip..>
# This variable lets us append arguments to the current kdump
# commandline after processed by KDUMP_COMMANDLINE_REMOVE
# KDUMP_COMMANDLINE_APPEND="irqpoll maxcpus=1 reset_devices"
<..snip..>
Since the 'maxcpus' setting doesn't limit the number of SMP CPUs,
so the kdump kernel still boots with all CPUs available on the system.
For e.g on the qualcomm amberwing its 46 CPUs:
# lscpu
Architecture: aarch64
Byte Order: Little Endian
CPU(s): 46
On-line CPU(s) list: 0-45
Thread(s) per core: 1
Core(s) per socket: 46
Socket(s): 1
NUMA node(s): 1
Vendor ID: Qualcomm
Model: 1
Model name: Falkor
Stepping: 0x0
CPU max MHz: 2600.0000
CPU min MHz: 600.0000
BogoMIPS: 40.00
L1d cache: 32K
L1i cache: 64K
L2 cache: 512K
L3 cache: 58880K
NUMA node0 CPU(s): 0-45
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid asimdrdm
This causes the memory consumption in the kdump kernel to swell up and
we can end up having OOM issues in the kdump kernel boot.
Whereas if we use 'nr_cpus=1' in the bootargs, the number of SMP CPUs in
the kdump kernel get limited to 1.
The 'swiotlb=noforce' setting in bootargs provide us extra guarding, to
ensure the crash kernel size requirements do not swell on systems
which support swiotlb.
With the above settings, crashkernel boots properly (without OOM) on all
the aarch64 boards I could test on - qualcomm amberwings, hp-moonshots
and hpe-apache (thunderx2) for crash dump saving on local disk.
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Pingfan Liu <piliu@redhat.com>
Currently kdumpctl rebuild will simply rebuild the initramfs, and
only perform basic config syntax check. But it should also check if the
target path is available when using SSH target, else kdump may fail.
is second kernel. kdumpctl rebuild should cover this case, and create
the path if it doesn't exist.
This patch make rebuild and restart behaves the same, rebuild is
now equal to restart, except it won't check config change or reload
kdump resource.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Although "kdumpctl rebuild" is introduced to help user rebuild the
initramfs without modifying the kdump.conf, if the kdump.conf is
modified and "kdumpctl rebuild" is called, a initramfs with a faulty
kdump.conf will be built.
Kdump will refuse to load the initramfs when restarted, but kdumpctl
reload may load the faulty initramfs. So need to make sure the faulty
build won't be generate in the first place.
Check for kdump.conf error before building the initramfs to ensure such
failure won't happen.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
We don't necessarily have to always rebuild the initramfs when
extra_modules is set. Instead, just detect if any module is updated,
and only rebuild initramfs if found any updated kernel module.
Tested with in-tree kernel modules, out-of-tree kernel modules, weak
modules, all worked as expected.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Previously only the symlink's timestamp is used for checking if file are
modified, this will not trigger a rebuild if the symlink target it
modified.
So check both symlink timestamp and symlink target timestamp, rebuild
the initramfs on both symlink changed and target changed.
Also give a proper error message if the file doesn't exist.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
When reading kdump configs, a single parsing should be enough and this
saves a lot of duplicated striping call which speed up the total load
speed.
Speed up about 2 second when building and 0.1 second for reload in my
tests.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Early kdump inherits the settings of normal kdump, so any changes that
caused normal kdump rebuilding also require rebuilding the system initramfs
to make sure that the changes take effect for early kdump.
Therefore, when the early kdump is enabled, provide a prompt message after
the rebuilding of kdump initramfs is completed.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Currently kdump is not working well with encrypted targets, add document
about this issue.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Add some note about the limitation of kdumpctl's auto detect and rebuild
feature, and suggest the user to rebuild the initramfs manually on
major system change, and don't include the initramfs in disk images.
Put the note about system change in front part of the document so user
will less likely to miss it.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Previous we rebuild the initramfs when kenrel load module list changed,
but this is not very stable as some async services may load/unload
kernel modules, and cause unnecessary initramfs rebuild.
Instead, it's better to just check if the module required to dump to
the dump target is loaded or not, and rebuild if not loaded. This
avoids most false-positives, and ensure local target change is always
covered.
Currently only local fs dump target is covered, because this check
requires the dump target to be mounted when building the initramfs,
this guarantee that the module is in the loaded kernel module list,
else we may still get some false positive.
dracut-install could be leveraged to combine the modalias list with
kernel loaded module list as a more stable module list in the initramfs,
but upstream dracut change need to be done first.
Passed test on a KVM VM, changing the storage between SATA/USB/VirtIO
will trigger initramfs rebuild and didn't notice any false-positive.
Also passed test on my laptop with no false-positive.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit 6b479b6572.
Check initramfs rebuild by looking at if there is any change of load
kernel modules list is not very stable after all. Previously we are
counting on udev to settle before kdump is started to ensure all modules
is ready, but actually any service may cause a kernel module load, even
after udev is settled.
The previous commit is trying to workaround an issue that VM created
with disk snapshot may fail in the kdump initramfs. The better fix is to
not include the kdump initramfs in the disk snapshot at all, as the
kdump initramfs is not generated for a generic use. And With new added
"kdumpctl reload" command, admins could rebuild the image easily, and
should rebuild the initramfs on hardware change manually.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
On powerpc, after hot add cpu and trigger crash on the hot-added cpu, the
kdump kernel hangs after "I'm in purgatory".
The current udev rules expects the dtb to be rebuit on cpu add/remove event.
But since powerpc does not follow the standard cpu hot add framework, it
only ejects online/offline event to user space when cpu is hot
added/removed, instead of add/remove event. Pingfan tried fixing that but
it didn't please the maintainer as it breaks some old userspace tools.
Due to the failure of dtb's rebuilding, KDump kernel fails to get the
'boot_cpuid' and eventually fails to boot [see early_init_dt_scan_cpus() in
arch/powerpc/kernel/prom.c file] if system crashes on hot-added CPU.
Work around it by changing udev rules on powerpc to onlne/offline.
As for offline message, it is even useless on powerpc, and can be dropped.
See the explain: On powerpc, /sys/devices/system/cpu/cpuX nodes are present
for all "possible", irrespective of whether a CPU is hot-added/removed.
crash_notes are already built for all /sys/devices/system/cpu/cpuX nodes and
these nodes are present for all "possible" CPUs
(online/offline/could-be-hot-removed/could-be-hot-added)
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
kdumpctl fails to load a crash kernel if KASLR is enabled.
The incorrect configuration of the kptr_restrict parameter causes that
the kdumpctl fails to load a crash kernel. The problem occurs when the
Kernel Address Space Layout Randomization (KASLR) mechanism is enabled.
The value of kptr_restrict decides how to expose the content of /proc
files. Inappropriate setting will cause that the /proc/kallsyms file is
being printed as all zeros. This configuration will break the kdump
loading process since it needs access to the conntent of /proc/kallsyms
if KASLR is enabled.
Set kptr_restrict to 1 to avoid the problem described above.
Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
These two command are not documented, update the man page to let user
know how to use them.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Use "kdumpctl rebuild" to rebuild the image directly. This could help
admins to rebuild kdump image directly.
Also merge fadump related initramfs backup/restore into setup_initrd,
and do permission only when actually trying to rebuild the image.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Seems some dead codes are left here for historical reason, just remove
them, read the config and strip comments only for once.
This improve the speed by a lot (2.6s -> 0.498s for reading a
simple config in my test case, on HDD) and make the code cleaner.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
With kernel commit 0823c68b054b ("powerpc/fadump: re-register firmware-
assisted dump if already registered") support is enabled to re-register
when FADump is alredy registered. Leverage that option in kdump scripts.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Kairui Song <kasong@redhat.com>
The idea behind adding support for dracut '--rebuild' option was to
ensure the initrd built for fadump takes into consideration all the
build parameters passed to original initrd. Pass original initrd
instead of current default initrd for rebuild as current initrd
might already have build parameters from original initrd along
with parameters from previous fadump intird build making the
build parameters look like this after a few iterations:
-H --persistent-policy 'by-uuid' -f --quiet --hostonly --hostonly-
cmdline --hostonly-i18n --hostonly-mode 'strict' -o 'plymouth dash
resume ifcfg' --mount '/dev/mapper/rhel_zzfp219--lp3-home /kdumproot
//home xfs defaults' -f --kver '4.18.0-60.el8.ppc64le' --quiet
--hostonly --hostonly-cmdline --hostonly-i18n --hostonly-mode 'strict'
-o 'plymouth dash resume ifcfg' --mount '/dev/mapper/rhel_zzfp219--lp3-home
/kdumproot//home xfs defaults' -f --kver '4.18.0-60.el8.ppc64le' --quiet
--hostonly --hostonly-cmdline --hostonly-i18n --hostonly-mode 'strict'
-o 'plymouth dash resume ifcfg' --mount '/dev/mapper/rhel_zzfp219--lp3-home
/kdumproot//home xfs defaults' -f --kver '4.18.0-60.el8.ppc64le' --include
'/tmp/fadump.initramfs' '/etc/fadump.initramfs' --include
'/tmp/fadump.initramfs' '/etc/fadump.initramfs' --include
'/tmp/fadump.initramfs' '/etc/fadump.initramfs'
--
Since it is not desirable to build initrd with stale and/or duplicate
build parameters, use original initrd (backed up) to rebuild fadump
initrd, instead of current default initrd.
Signed-off-by: Hari Bathini <hbathini@linux.ibm.com>
Acked-by: Dave Young <dyoung@redhat.com>
eppic project have moved to github, update to latest upstream snapshot,
change source link and tar file naming style to fit github's URL format.
This fix the O0 warning reported by annocheck and passes all distro package
flag checking.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Backport the patches required to make the hardening build flags work with
kexec-tools and makedumpfile, and enabld hardening flags in spec file.
This will make the pacakge pass all warnings for kexec and makedumpfile
reported by annocheck.
Didn't find any issue with basic tests with kexec and makedumpfile.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Currently we use "\b" (word boundary) as the delimiter for ro option,
which is not correct. For mount options like
"defaults,errors=remount-ro" the ro on the tail will also be replaced
and result in an invalid mount option.
So we use a more strict logic on detecting ro mount option. It should
either starts with "," or "^" (begin of line) and ends with "," or "$"
(end of line), and keep the delimiter untouched. This should ensure
only valid mount option got detected and replaced.
This passed following tests:
defaults,ro,noauto,errors=remount-ro,nobootwait,nofail => defaults,rw,errors=remount-ro,
defaults,errors=remount-ro => defaults,errors=remount-ro
defaults,ro,relatime => defaults,rw,relatime
defaults,ro => defaults,rw
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Since early kdump is generally used for capturing vmcore when
boot-time panic occurs, if a system always reboots after capturing
vmcore, it can go into a crash loop.
To avoid this issue, this patch add a note of 'final_action' option
to the early kdump document.
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
If a crash occurs repeatedly after enabling kdump, the system goes
into a crash loop and the dump target may get filled up by vmcores.
This is likely especially with early kdump.
This patch introduces 'final_action' option to kdump.conf, in order
for users to be able to power off the system even after capturing
a vmcore successfully.
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Lianbo Jiang <lijiang@redhat.com>
Cc: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
In preparation for adding 'final_action' option, since it's confusing
to have the 'final_action' and 'default' options at the same time,
this patch introduces 'failure_action' as an alias of the 'default'
option to /etc/kdump.conf, and makes 'default' obsolete to be removed
in the future.
Also, the "default action" term is renamed to "failure action".
Signed-off-by: Kazuhito Hagio <k-hagio@ab.jp.nec.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Lianbo Jiang <lijiang@redhat.com>
Cc: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Kairui Song <kasong@redhat.com>
earlykdump is not suppose to be loaded for a kdump initramfs, and user
may add it into dracut's config file so it will be included by default.
It will also make the image building always fail because earlykdump
actually detect if it's being used for kdump image and raise an error if
so.
In that case, we always force drop this module to avoid such problem.
Signed-off-by: Kairui Song <ryncsn@gmail.com>
Acked-by: Dave Young <dyoung@redhat.com>
Previously we handled the case when the installed kernel version for
early kdump is different from dracut target, it will be better to
print a warning even if installation successed, to let the user know
that an different kernel is used.
No warn message will be given if the user specified a KDUMP_KERNELVER
value, as in such case a different kernel version is used on purpose.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Currently when earlykdump failed to install required kernel image or
initramfs, it will still install the earlykdump hook and other utils.
But it won't work due to the absent of kernel image or kdump initramfs,
so the hook and installed utils is meanless.
We can't simply fail dracut building, as if earlykdump is included by
dracut config file, this may fail kernel update, where kernel image is
installed but initramfs failed to generate, and then it will fail
booting.
So this patch let it skip earlydkump install if anything is missing and
give a clean error message to let the user better ware of the situation.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
There is currently a problem with earlykdump image building, when a user
is upgrading kernel, dracut will generate new initramfs for the new
kernel, and earlykdump will install currently running version of kernel
into the initramfs, and remain the version based kernel image naming
untouched. But after a reboot the new kernel is running, and it
will try to load the image corresponding to the new kernel version by
file naming.
This patch fixes the problem by creating a symlink with unified stable
naming to the installed kernel image and initramfs, and use the symlink
instand so it will always work despite the kernel version number change.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Early kdump always fails to load the vmlinuz-xxx after the 'binutils'
package has been installed, and outputs the following messages:
...
dracut-cmdline[309]: Cannot determine the file type of /boot/vmlinuz-4.18.0-51.el8.x86_64
dracut-cmdline[309]: kexec: failed to load early-kdump kernel
...
The reason is that the vmlinuz-xxx image is mistakenly stripped when
using dracut to generate the kdump initrd. Because dracut always find
all executable binary files to strip only if the 'binutils' package
is installed, otherwise it will skip the stripping.
Therefore, remove the executable permissions of the vmlinuz-xxx in
'${initrd}' in order to let dracut skip the mistakenly stripping.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Without this patch, when there are two or more spaces after 'path'
configuration phrase with ssh or nfs setting, SAVE_PATH is set to
'/var/crash' in mkdumprd, and in most cases kdump service fails to
start by checking the /var/crash directory regardless of the path
value.
ssh kdump(a)192.168.122.1
path /kdump
^^
This behavior would be too sensitive and different from the other
configurations. With this patch, mkdumprd allows such spaces.
Signed-off-by: Kazuhito Hagio <k-hagio(a)ab.jp.nec.com>
Acked-by: Kairui Song <kasong@redhat.com>
When someone is using a minimal kernel without squash module installed,
including squash dracut module will either either fail to build or fail to
boot the initramfs.
As kdump always build the image for one single kernel, we can safely just
use modprobe to check if a modules is already built in, or it exists and
loadable for the kernel we are using for kdump image, and don't include
the squash module if they are missing. Everything will still work just
fine without squash module.
We do the check in kdump dracut modules not in squash dracut module
because kdump dracut module could leverage of the KDUMP_KERNELVER variable
to know which kernel it should check against, squash dracut module may be
used to build for a generic image.
And we only check for the kernel module dependency, other binary
dependencies are either well checked or well declared in dracut.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
If default action is poweroff, we can observe that the machine is
rebooted, instead of poweroff. That is due to the following two race
processes:
systemctl poweroff
systemctl reboot -f
which is launched by kdump-error-handle.sh.
Unfortunately, although both of them are executed in systemd block
mode, but due to poweroff will tear down some internal things in
systemd, there is no guarantee for the block mode. As we can see
the msg "Failed to execute operation: Connection reset by peer",
which is thrown by "systemctl reboot -f".
poweroff and reboot share most of code, if one fails, then the other
should also fails, so it is meaningless to use reboot as the backup of
poweroff. Using "systemctl poweroff -f", the sdbus will teared down
immediately, which prevent the following "systemctl reboot -f" from
executing. Meanwhile, as man systemctl says:
-f, --force
When used with enable, overwrite any existing conflicting symlinks.
When used with halt, poweroff, reboot or kexec, execute the selected
operation without shutting down all units. However, all processes will
be killed forcibly and all file systems are unmounted or remounted read-only.
Hence, replacing the 'poweroff' with 'systemctl poweroff -f'
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
In commit 1c97aee and commit 227c185 udev rules was rewritten to use
systemd-run to run in a non-blocking mode. The problem is that it's a
bit noise, especially on machine bootup, systemd will always generate
extra logs for service start, you might see your journal full of lines
like these if you have many CPUs (each CPU generates a udev event on
boot):
...
Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler.
Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler.
Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler.
Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler.
...
While system is still booting up, kdump service is not started yet, so
systemd-run calls will end up doing nothing, the throttler being called
by systemd-run will just exit if kdump is not loaded.
This patch avoid systemd-run from being called at first place if kdump
service is not running by checking kdump.service status in udev rule,
so there won't be unnecessary logs.
Also remove the kdump service checking logic in kdump-udev-throttler as
udev is the only expected callee of this script, if it's not being
called at first place when kdump service is running, this checking will
be redundant. And even if any user called this script manually, it will
still work well as this script will call 'kdumpctl reload', it reload
the kdump resource only if kdump is loaded already.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Currently we still don't support multipath route, when parsing multipath
route kdumpctl will wrongly consider 'nexthop' as the destination address,
and raise errors in second kernel.
When multipath route is in use, ip route output should be like this:
$ /sbin/ip route show
default via 192.168.122.1 dev ens1 proto dhcp metric 100
192.168.122.0/24 dev ens1 proto kernel scope link src 192.168.122.161 metric 100
192.168.122.8
nexthop via 192.168.122.1 dev ens1 weight 50
nexthop via 192.168.122.2 dev ens1 weight 5
As we don't care about HA/performance, simply use the rule with highest
weight and ignore the rest.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
nfs service will append extra mount options to kernel mount options.
Those extra options represent current mounting details, but they may
not suitable for the second kernel. IP address may change, and we only
enable a single network stack (v4/v6), if nfs prefered another
network stack, inheriting the options will force nfs service to use
previous network stack and disable nfs's fallback mechanic and fail.
As nfs service have the capability to negotiate required protocols
and detect proper IP address, just drop those options and let nfs
automatically adapt the possible change in the second kernel.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Resolves: bz1561837
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1561837
Currently 'kdump.conf' and 'kdump.conf MAN page' entries state that the
local dump path should be:
<fs type> <partition>
- Will mount -t <fs type> <partition> <mnt>, and copy
/proc/vmcore to <mnt>/<path>/%DATE/.
The correct vmcore path instead should be:
<mnt>/<path>/%HOST_IP-%DATE/
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
In commit b34ce3a reload support was added to kdumpctl but the usage
info is not updated. Now add reload to usage output to let user aware
of the new command.
Signed-off-by: Kairui Song <kasong@redhat.com>
Previously, kdump will restart / reload for many times on hotplug
event, especially memory hotplug events. Hotplugged memory may
generate many udev event as memory are managed and hotplugged in
small chunks by the kernel.
This results in unnecessary system workload and an actually longer
delay of kdump reload and the hotplug event, as udev will either
get blocked or kdumpctl will be waiting for other triggered operation.
To fix this, introduce a kdump-udev-throttler as an agent which will
be called by udev and merge concurrent kdump restart requests. Tested
with a Hyper-V VM which is failing due to udev timeout previously,
no new issues found.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
According to udev's man page, PROGRAM is either used to determine
device's name or whether the device matches the rule. So we should
use RUN insteand. Meanwhile, both RUN / PROGRAM only accepts very
short-running foreground tasks, but kdump restart may take a long
time if there are any device changes that will lead to image rebuild,
which may lead to buggy behavior.
On the other hand, memory / CPU hot plug should never trigger a
initramfs rebuild.
To solve this problem, we will use new introduced "kdumpctl reload"
instead, and use systemd-run to create a transient service unit for
the reload and run it in no-block mode, so udev won't be blocked by
anything.
We need to make systemd-run execute in non-blocking mode, and do not
synchronously wait for the operation to finish, because udev expect
the command line in RUN to be finished immediately, however, kdumpctl
reload may take 0.5-1s for an ordinary reload, or even slower on some
machines. So we give systemd-run an explicit --no-block option to run
in non-blocking mode. Without --no-blocking, systemd-run will verify,
enqueue and wait for the operation to finish. By using the --no-block
option, systemd-run will only verify and enqueue the unit then
return. In this way, we make sure the command is executed
asynchronously, and the status will be monitored and logged by
systemd, which is reliable and non-blocking.
Another thing to mention is that --no-block is only needed after
systemd-v220, before v220 systemd-run uses non-blocking mode by
default and --no-block option is not available on earlier systemd
versions.
Also reformat the udev rules to a more maintanceable format.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Add reload support to kdumpctl, reload will simply unload current
loaded kexec crash kernel and initramfs, and load it again.
Changes in /etc/sysconfig/kdump will take effect with kdumpctl
reload, but reloading will not check the content of
/etc/kdump.conf and won't rebuild anything. reload is fast, the only
time-consuming part of kdumpctl reload is loading kernel and initramfs
with kexec which is always necessary.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
In dracut-049, a new squash module is introduced, it can reduce the
memory usage of kdump initramfs in the capture kernel, this helps a lot
on lowering the risk of OOM failure.
Tested with latest rawhide with NFS, SSH and local dump.
Signed-off-by: Kairui Song <kasong@redhat.com>
Currently the kdumpctl script doesn't check if the raw device is
formatted which might destroy existing data at the time of dump
capture.
This patch addresses this issue, by ensuring kdumpctl prints
a warning in case it finds the raw device to be formatted.
Signed-off-by: Kenneth D'souza <kdsouza@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Currently the script does not check if the dump target is read-only and would
always mount to read-write mode. This caused an issue with nfs mount as the
fstab options would be reconsidered while remounting to read-write mode.
The remount would fail with the below error as all options cannot be changed
runtime.
mount.nfs: mount(2): Invalid argument
mount.nfs: an incorrect mount option was specified
Which in result would not save the vmcore on the dump target.
This patch addresses this issue by checking the dump target status for read-only.
If yes, remount to read-write mode without reconsidering the fstab options.
Signed-off-by: Kenneth D'souza <kdsouza@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Resolves: bz1619122
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1619122
This patch fixes the "Unhandled rela relocation: R_X86_64_PLT32" error
that we are seeing with Fedora 29 (and newer kernels > 4.18) which
trying to run kexec/kdump on x86_64 machines.
The patch is being discussed upstream and has been ACK'ed by Baoquan and
myself (see <https://www.spinics.net/lists/kexec/msg21255.html>) and I
have also tested the same on Fedora 29/rawhide x86_64 machine as well:
Before the patch:
----------------
[root@hp-bl480c-01 ~]# kdumpctl restart
kexec: unloaded kdump kernel
Stopping kdump: [OK]
Unhandled rela relocation: R_X86_64_PLT32
kexec: failed to load kdump kernel
Starting kdump: [FAILED]
After the patch:
---------------
[root@hp-bl480c-01 ~]# kdumpctl restart
kexec: unloaded kdump kernel
Stopping kdump: [OK]
kexec: loaded kdump kernel
Starting kdump: [OK]
Suggested Upstream Fix:
In response to a change in binutils, commit b21ebf2fb4c
(x86: Treat R_X86_64_PLT32 as R_X86_64_PC32) was applied to
the linux kernel during the 4.16 development cycle and has
since been backported to earlier stable kernel series. The
change results in the failure message in $SUBJECT when
rebooting via kexec.
Fix this by replicating the change in kexec.
Signed-off-by: Chris Clayton <chris2553@googlemail.com>
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
Currently the kdumpctl script doesn't check if the path option is
set more than once due to which a vmcore is not captured.
This patch addresses this issue by ensuring that only one path
is specified in /etc/kdump.conf file.
Signed-off-by: Kenneth D'souza <kdsouza@redhat.com>
Acked-by: Kairui Song <kasong@redhat.com>
If nofail or nobootwait option is used, systemd's local-fs.target won't
wait for the mounting to complete, and kdump might start before the
required mount point is ready and then fail.
The host might use nofail for reasons like the device may get unpluged,
and if the device is not mounted and it is set as kdump target as the same
time then kdump service won't start, we will never enter the capture
kernel. By the time we have entered the capture kernel, the target device
must exist and ready to use, or else kdump would fail anyway. So force
remove nofail and nobootwait option.
Also drop rootflags=nofail option, as we don't depend on rootfs anymore
if the dump target don't required it. So the nofail option is no longer
needed.
Signed-off-by: Kairui Song <kasong@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
We test if to include the drm module or not by testing if there are any
drm entry in sysfs. But there is an exception for hyper-v, DRM module
take care of hyperv's framebuffer driver as well but hyperv_fb will
not create any drm entry. So currently we got black screen on
hyperv guest.
Fix by detect hyperv's special entry as well.
Signed-off-by: Kairui Song <kasong@redhat.com>
Kdump anaconda has been included as a subpackage for a long time, which
is not a good practice, as the anaconda plugin should be built as
noarch and it does not belong to kexec-tools. We have created a new
package 'kdump-anaconda-addon', so remove it here.
The release version should be bumped later so that kdump-anaconda-addon
could mark previous versions as obsoleted.
Signed-off-by: Kairui Song <kasong@redhat.com>
armv7hl build failed because no makedumpfile* built but the latest commit
tries to install them.
Exclude armv7hl in the code chunk.
Signed-off-by: Dave Young <dyoung@redhat.com>
kexec_test seems to be no longer used upstream, so we had introduced
the 'kexec-tools-2.0.3-disable-kexec-test.patch' earlier to disable the
same from fedora kexec-tools as well.
However an earlier patch "Remove obsolete kdump tool" now explicitly
installs needed files via appropriate logic in .spec file, so we can
drop this patch now to reduce the maintenance burden.
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1441677
Testing: On x86_64 Fedora machine. After this patch kdump utility and related
man page cannot be found on this machine:
[root@tyan-gt24-09 ~]# which kdump
/usr/bin/which: no kdump in
(/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin)
[root@tyan-gt24-09 ~]# man kdump
No manual entry for kdump
Update the fedora 'kexec-tools.spec' to not install the obsolete
kdump tool.
I have submitted an upstream patch to obsolete the kdump tool from
upstream kexec-tools (which has been accepted), but after an internal
discussion we decided not to backport the upstream 'kexec-tools' patch
(which does the same) for fedora, as we would prefer to manage the
changes directly in the .spec file itself.
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
This commit basically reverts commit c755499fad,
and make use of new introduced tri-state hostonly mode.
Following dracut commits merged multipath-hostonly into multipath
module, and introduced a tri-state hostonly mode.
commit 35e86ac117acbfd699f371f163cdda9db0ebc047
Author: Kairui Song <kasong@redhat.com>
Date: Thu Jul 5 16:20:04 2018 +0800
Merge 90-multipath-hostonly and 90-multipath
commit a695250ec7db21359689e50733c6581a8d211215
Author: Kairui Song <kasong@redhat.com>
Date: Wed Jul 4 17:21:37 2018 +0800
Introduce tri-state hostonly mode
multipath-hostonly module was introduced only for kdump, because kdump
need a more strict hostonly policy for multipath device to save memory.
Now multipath module will provide the behave we wanted by setting
hostonly mode to strict.
Currently, we only rebuilt kdump initramfs on config file change,
fs change, or watchdog related change. This will not cover the case
that hardware changed but fs layout and other configurations still
stays the same, and kdump may fail.
To cover such case, we can detect and compare loaded kernel modules,
if a hardware change requires the image to be rebuilt, loaded kernel
modules must have changed.
Starting from commit 7047294 dracut will record loaded kernel modules
when the image is built if hostonly mode is enabled. With this patch,
kdumpctl will compare the recorded value with currently loaded kernel
modules, and rebuild the image on change.
"kdumpctl start" will be a bit slower, as we have to call lsinitrd one
more time to get the loaded kernel modules list. I measure the time
consumption and we have an overall 0.2s increased loading time.
Time consumption of command "kdumpctl restart":
Before:
real 0m0.587s
user 0m0.481s
sys 0m0.102s
After:
real 0m0.731s
user 0m0.591s
sys 0m0.133s
Time comsumption of command "kdumpctl restart" with image rebuild:
Before (force rebuild):
real 0m10.972s
user 0m8.966s
sys 0m1.318s
After (inserted ~100 new modules):
real 0m11.220s
user 0m9.387s
sys 0m1.337s
Signed-off-by: Kairui Song <kasong@redhat.com>
Kdump always use _proto=dhcp for both ipv4 and ipv6. But for ipv6
the dhcp address assignment is not like ipv4, there are different ways
for it, stateless and stateful, see below document:
https://fedoraproject.org/wiki/IPv6Guide
In case stateless, kernel can do the address assignment, dracut use
_proto=auto6; for stateful case, dracut use _proto=dhcp6.
But it is hard to decide whether stateless or stateful takes effect,
hence, dracut introduces ip=either6 option, which can try both of these
method automatically for us. For detail, refer to dracut:
commit 67354ee 40network: introduce ip=either6 option
We do not see bug reports before because for the most auto6 cases
kernel assign ip address before dhclient, kdump just happened to work.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
When using fence_kdump, module-setup will create a kdump.conf with
fence_kdump_nodes. The node name comes from the cluster xml, which may
use the hostname alias. Later in kdump stage, "fence_kdump_send alias_1
alias_2" sends out notification to peers. Hence it requires /etc/hosts
and nsswitch.conf to make alias work.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Kdump service starts too late, so early crashes will have no chance
to get kdump kernel booting, this will cause crash information to be
lost. It is necessary to add a dracut module in order to load crash
kernel and initramfs as early as possible. You can provide "rd.early
kdump" in grub commandline to enable, then the early kdump will load
those files like the normal kdump, which is disabled by default.
For the normal kdump service, it can check whether the early kdump
has loaded the crash kernel and initramfs. It has no conflict with
the early kdump.
If you rebuild the new initramfs for early kdump, the new initramfs
size will become large, because it will put the vmlinuz and kdump
initramfs into the new initramfs.
In addition, early kdump doesn't support fadump.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Reviewed-by: Kazuhito Hagio <khagio@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
we move some common functions from kdumpctl to kdump-lib.sh, the
functions could be used in other modules, such as early kdump.
It has no bad effect.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Reviewed-by: Kazuhito Hagio <khagio@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
port from rhel, original patch is contributed by Minfei Huang:
Using /sys to determines crashkernel actual size is confusing since
there is no unit of measure.
Add a new command "kdumpctl showmem" to show the reserved memory kindly.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Signed-off-by: Minfei Huang <mhuang@redhat.com>
Acked-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit 2f4149f276.
It is not proved to be right to get auto6 or dhcpv6 in 1st kernel,
pingfan is working on a dracut fix to do some fallback in 2nd kernel initramfs.
So revert this commit
For ppc64(le), the default behavior of kexec always copy "root=" param,
but if dump target is ssh, there will no tools installed in kdump
rd, which help to mount root. As a result, kdump service will fail to
start. So explicitly disable the default behavior with --dt_no_old_root
option.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
When core_collector is changed, the kdump initramfs needs to
be rebuilt before it is loaded.
Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Kdump always use _proto=dhcp for both ipv4 and ipv6. But for ipv6
the dhcp address assignment is not like ipv4, there are different ways
for it, stateless and stateful, see below document:
https://fedoraproject.org/wiki/IPv6Guide
In case stateless, kernel can do the address assignment, dracut use
_proto=auto6; for stateful case, dracut use _proto=dhcp6.
We do not see bug reports before because for the most auto6 cases
kernel assign ip address before dhclient, kdump just happened to work.
Here we use auto6 if possible first. And we take the assumption that
host use auto6 if /proc/sys/net/ipv6/conf/$netdev/autoconf is enabled
Signed-off-by: Pingfan Liu <piliu@redhat.com>
When using "dracut_args --mount" to specify dump target, e.g. nfs like:
path /
core_collector makedumpfile -d 31
dracut_args --mount "host:/path /var/crash nfs defaults"
kdump service should neither guarantees the correctness, nor relabels it.
For current code, since dracut_args dump targets are likely not mounted
so kdump service mistakenly relabel the rootfs, which is meanless and
takes very long time.
Signed-off-by: Pingfan Liu <piliu@redhat.com>
This reverts commit 2040103bd7.
Reason is it's based on the environment of 1st kernel where all
present devices could be active and initialized during bootup.
Then all pci devices will request irqs. While kdump only brings
up those devices which are necessary for vmcore dumping. So this
commit is not meaningful and helpless to very large extent. And
it will print out 'Warning' when calculated result is larger than
1 cpu, actually it's a false positive report most of the time.
So revert the commit, and can check the git history for later
reference.
[dyoung]: on some machine this warning message shows up but
later we found the irq numbers with and without nr_cpus=1 is
quite different so this need more investigation since
the formula is not accurate.
Signed-off-by: Baoquan He <bhe@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Due to the following commit in dracut, which splits out hostonly modules
commit 5ce7cc7337a4c769b223152c083914f2052aa348
Author: Harald Hoyer <harald@redhat.com>
Date: Mon Jul 10 13:28:40 2017 +0200
add 90multipath-hostonly module
hardcoding the wwid of the drives in the initramfs causes problems
when the drives are cloned to a system with the same hardware, but
different disk wwid's
https://bugzilla.redhat.com/show_bug.cgi?id=1457311
So kdump should decide whether to include the hostonly module.
The multipath-hostonly can help kdump to include only the needed mpath device,
in order to use less memory by 2nd kernel.
---- The performance -----
before this patch
[root@localhost ~]# time kdumpctl start
Detected change(s) in the following file(s):
/etc/kdump.conf
Rebuilding /boot/initramfs-4.13.9-300.fc27.x86_64kdump.img
kexec: loaded kdump kernel
Starting kdump: [OK]
real 0m12.485s
user 0m10.096s
sys 0m1.887s
after this patch
root@localhost ~]# time kdumpctl start
Detected change(s) in the following file(s):
/etc/kdump.conf
Rebuilding /boot/initramfs-4.13.9-300.fc27.x86_64kdump.img
kexec: loaded kdump kernel
Starting kdump: [OK]
real 0m15.839s
user 0m13.015s
sys 0m1.853s
Signed-off-by: Pingfan Liu <piliu@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
This reverts commit 8e3b6475c9.
After reading the background of bugzilla:
https://bugzilla.redhat.com/show_bug.cgi?id=1510922
It is not a problem actually, sorry for the noise.
Although root set different "PATH" can lead to wrong script, but it is
different with what the bug described "current working dir" and it is
not a problem worth an update
Since we call dracut directly on current working directory "." so it is
possible to trick root to call fake code.
Thus move to use absolute path instead.
Signed-off-by: Dave Young <dyoung@redhat.com>
Acked-by: Bhupesh Sharma <bhsharma@redhat.com>
Resolves: BZ1484945
https://bugzilla.redhat.com/show_bug.cgi?id=1484945
Currently the kdumpctl script doesn't handle
whitespaces (including TABs) which might be there before
an option name in the kdump.conf
This patch addresses this issue, by ensuring that the
kdumpctl errors out in case it finds any stray space(s)
or tab(s) before a option name.
Reported-by: Kenneth D'souza <kdsouza@redhat.com>
Signed-off-by: Bhupesh Sharma <bhsharma@redhat.com>
Acked-by: Pratyush Anand <panand@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
2017-10-11 09:57:31 +08:00
155 changed files with 3124 additions and 13973 deletions
<property name="label" translatable="yes">Kdump is a kernel crash dumping mechanism. In the event of a system crash, kdump will capture information from your system that can be invaluable in determining the cause of the crash. Note that kdump does require reserving a portion of system memory that will be unavailable for other uses.</property>
<property name="label" translatable="yes">Kdump is a kernel crash dumping mechanism. In the event of a system crash, kdump will capture information from your system that can be invaluable in determining the cause of the crash. Note that kdump does require reserving a portion of system memory that will be unavailable for other uses.</property>
<property name="wrap">True</property>
</object>
<packing>
<property name="expand">False</property>
<property name="fill">True</property>
<property name="position">0</property>
</packing>
</child>
<child>
<object class="GtkGrid" id="grid1">
<property name="visible">True</property>
<property name="can_focus">False</property>
<child>
<object class="GtkLabel" id="toBeReservedLabel">
<property name="visible">True</property>
<property name="can_focus">False</property>
<property name="xalign">0</property>
<property name="label" translatable="yes">Memory To Be _Reserved (MB):</property>
