Unified Source + Adapt workflow with build Script #32
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Newer versions of clang only look for $(COMPAT_GCC_TOOLCHAIN_DIR)as [1], rather than $(COMPAT_GCC_TOOLCHAIN_DIR)$(CROSS_COMPILE_COMPAT)as, resulting in the following build error: $ make -skj"$(nproc)" ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \ CROSS_COMPILE_COMPAT=arm-linux-gnueabi- LLVM=1 O=out/aarch64 distclean \ defconfig arch/arm64/kernel/vdso32/ ... /home/nathan/cbl/toolchains/llvm-binutils/bin/as: unrecognized option '-EL' clang-12: error: assembler command failed with exit code 1 (use -v to see invocation) make[3]: *** [arch/arm64/kernel/vdso32/Makefile:181: arch/arm64/kernel/vdso32/note.o] Error 1 ... Adding the value of CROSS_COMPILE_COMPAT (adding notdir to account for a full path for CROSS_COMPILE_COMPAT) fixes this issue, which matches the solution done for the main Makefile [2]. [1]: llvm/llvm-project@3452a0d [2]: https://lore.kernel.org/lkml/[email protected]/ Signed-off-by: Nathan Chancellor <[email protected]> Cc: [email protected] Link: ClangBuiltLinux/linux#1099 Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Will Deacon <[email protected]> [dl: Backported to 4.14, depends on commit 38253a0ed057c49dde77588eef05fdcb4008ce0b ("vdso32: Invoke clang with correct path to GCC toolchain") from the Pixel 4 kernel] Signed-off-by: Danny Lin <[email protected]> Signed-off-by: Forenche <[email protected]> Signed-off-by: John Vincent <[email protected]>
ld.lld distributed in clang-r353983c AOSP LLVM release (the final AOSP LLVM release for Android Q) did not support `-n` linker flag. It was eventually added to clang-r360593. Android OEM's may wish to still use ld.lld to link their kernels for Q. This flag was disabled for Pixel 4 in go/pag/1258086. This patch is equivalent, but rebased on upstream changes that removed cc-ldoption in favor of ld-option. For Android R, the final AOSP LLVM release, clang-r383902 has long supported `-n` for ld.lld. Change-Id: Iab41c9e1039e163113b428fc487a4a0708822faa Bug: 63740206 Bug: 157279372 Link: ClangBuiltLinux/linux#340 Link: https://bugs.llvm.org/show_bug.cgi?id=40542 Signed-off-by: Nick Desaulniers <[email protected]> Signed-off-by: John Vincent <[email protected]>
modified: arch/arm64/kernel/vdso.c
since we are going to use llvm ias also a fix for /usr/bin/as: unrecognized option '-EL' when compiling with clang neutron clang 19 , clang 18 works fine without this fix
Based on Sergey's test patch [1], this fixes zram with lz4 compression on big endian cpus. Note that the 64-bit preprocessor test is not a cleanup, it's part of the fix, since those identifiers are bogus (for example, __ppc64__ isn't defined anywhere else in the kernel, which means we'd fall into the 32-bit definitions on ppc64). Tested on ppc64 with no regression on x86_64. [1] http://marc.info/?l=linux-kernel&m=145994470805853&w=4 Cc: [email protected] Suggested-by: Sergey Senozhatsky <[email protected]> Signed-off-by: Rui Salvaterra <[email protected]> Reviewed-by: Sergey Senozhatsky <[email protected]> Signed-off-by: Greg Kroah-Hartman <[email protected]>
These identifiers are bogus. The interested architectures should define HAVE_EFFICIENT_UNALIGNED_ACCESS whenever relevant to do so. If this isn't true for some arch, it should be fixed in the arch definition. Signed-off-by: Rui Salvaterra <[email protected]> Reviewed-by: Sergey Senozhatsky <[email protected]> Signed-off-by: Greg Kroah-Hartman <[email protected]>
Patch series "Update LZ4 compressor module", v7. This patchset updates the LZ4 compression module to a version based on LZ4 v1.7.3 allowing to use the fast compression algorithm aka LZ4 fast which provides an "acceleration" parameter as a tradeoff between high compression ratio and high compression speed. We want to use LZ4 fast in order to support compression in lustre and (mostly, based on that) investigate data reduction techniques in behalf of storage systems. Also, it will be useful for other users of LZ4 compression, as with LZ4 fast it is possible to enable applications to use fast and/or high compression depending on the usecase. For instance, ZRAM is offering a LZ4 backend and could benefit from an updated LZ4 in the kernel. LZ4 homepage: http://www.lz4.org/ LZ4 source repository: https://github.com/lz4/lz4 Source version: 1.7.3 Benchmark (taken from [1], Core i5-4300U @1.9GHz): ----------------|--------------|----------------|---------- Compressor | Compression | Decompression | Ratio ----------------|--------------|----------------|---------- memcpy | 4200 MB/s | 4200 MB/s | 1.000 LZ4 fast 50 | 1080 MB/s | 2650 MB/s | 1.375 LZ4 fast 17 | 680 MB/s | 2220 MB/s | 1.607 LZ4 fast 5 | 475 MB/s | 1920 MB/s | 1.886 LZ4 default | 385 MB/s | 1850 MB/s | 2.101 [1] http://fastcompression.blogspot.de/2015/04/sampling-or-faster-lz4.html [PATCH 1/5] lib: Update LZ4 compressor module [PATCH 2/5] lib/decompress_unlz4: Change module to work with new LZ4 module version [PATCH 3/5] crypto: Change LZ4 modules to work with new LZ4 module version [PATCH 4/5] fs/pstore: fs/squashfs: Change usage of LZ4 to work with new LZ4 version [PATCH 5/5] lib/lz4: Remove back-compat wrappers This patch (of 5): Update the LZ4 kernel module to LZ4 v1.7.3 by Yann Collet. The kernel module is inspired by the previous work by Chanho Min. The updated LZ4 module will not break existing code since the patchset contains appropriate changes. API changes: New method LZ4_compress_fast which differs from the variant available in kernel by the new acceleration parameter, allowing to trade compression ratio for more compression speed and vice versa. LZ4_decompress_fast is the respective decompression method, featuring a very fast decoder (multiple GB/s per core), able to reach RAM speed in multi-core systems. The decompressor allows to decompress data compressed with LZ4 fast as well as the LZ4 HC (high compression) algorithm. Also the useful functions LZ4_decompress_safe_partial and LZ4_compress_destsize were added. The latter reverses the logic by trying to compress as much data as possible from source to dest while the former aims to decompress partial blocks of data. A bunch of streaming functions were also added which allow compressig/decompressing data in multiple steps (so called "streaming mode"). The methods lz4_compress and lz4_decompress_unknownoutputsize are now known as LZ4_compress_default respectivley LZ4_decompress_safe. The old methods will be removed since there's no callers left in the code. [[email protected]: fix KERNEL_LZ4 support] Link: http://lkml.kernel.org/r/[email protected] [[email protected]: simplify] [[email protected]: fix the simplification] [[email protected]: fix performance regressions] Link: http://lkml.kernel.org/r/1486898178-17125-2-git-send-email-4sschmid@informatik.uni-hamburg.de [[email protected]: v8] Link: http://lkml.kernel.org/r/1487182598-15351-2-git-send-email-4sschmid@informatik.uni-hamburg.de Link: http://lkml.kernel.org/r/1486321748-19085-2-git-send-email-4sschmid@informatik.uni-hamburg.de Signed-off-by: Sven Schmidt <[email protected]> Signed-off-by: Arnd Bergmann <[email protected]> Cc: Bongkyu Kim <[email protected]> Cc: Rui Salvaterra <[email protected]> Cc: Sergey Senozhatsky <[email protected]> Cc: Greg Kroah-Hartman <[email protected]> Cc: Herbert Xu <[email protected]> Cc: David S. Miller <[email protected]> Cc: Anton Vorontsov <[email protected]> Cc: Colin Cross <[email protected]> Cc: Kees Cook <[email protected]> Cc: Tony Luck <[email protected]> Signed-off-by: Andrew Morton <[email protected]> Signed-off-by: Linus Torvalds <[email protected]>
Remove the functions introduced as wrappers for providing backwards compatibility to the prior LZ4 version. They're not needed anymore since there's no callers left. Link: http://lkml.kernel.org/r/1486321748-19085-6-git-send-email-4sschmid@informatik.uni-hamburg.de Signed-off-by: Sven Schmidt <[email protected]> Cc: Bongkyu Kim <[email protected]> Cc: Rui Salvaterra <[email protected]> Cc: Sergey Senozhatsky <[email protected]> Cc: Greg Kroah-Hartman <[email protected]> Cc: Herbert Xu <[email protected]> Cc: David S. Miller <[email protected]> Cc: Anton Vorontsov <[email protected]> Cc: Colin Cross <[email protected]> Cc: Kees Cook <[email protected]> Cc: Tony Luck <[email protected]> Signed-off-by: Andrew Morton <[email protected]> Signed-off-by: Linus Torvalds <[email protected]>
Don't populate the read-only arrays dec32table and dec64table on the
stack, instead make them both static const. Makes the object code
smaller by over 10K bytes:
Before:
text data bss dec hex filename
31500 0 0 31500 7b0c lib/lz4/lz4_decompress.o
After:
text data bss dec hex filename
20237 176 0 20413 4fbd lib/lz4/lz4_decompress.o
(gcc version 7.2.0 x86_64)
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Colin Ian King <[email protected]>
Cc: Christophe JAILLET <[email protected]>
Cc: Sven Schmidt <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: Joe Perches <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
Signed-off-by: topser9 <[email protected]> modified: drivers/block/zram/Kconfig modified: drivers/block/zram/zram_drv.c
Update the LZ4 compression module based on LZ4 v1.8.3 in order for the erofs file system to use the newest LZ4_decompress_safe_partial() which can now decode exactly the nb of bytes requested [1] to take place of the open hacked code in the erofs file system itself. Currently, apart from the erofs file system, no other users use LZ4_decompress_safe_partial, so no worry about the interface. In addition, LZ4 v1.8.x boosts up decompression speed compared to the current code which is based on LZ4 v1.7.3, mainly due to shortcut optimization for the specific common LZ4-sequences [2]. lzbench testdata (tested in kirin710, 8 cores, 4 big cores at 2189Mhz, 2GB DDR RAM at 1622Mhz, with enwik8 testdata [3]): Compressor name Compress. Decompress. Compr. size Ratio Filename memcpy 5004 MB/s 4924 MB/s 100000000 100.00 enwik8 lz4hc 1.7.3 -9 12 MB/s 653 MB/s 42203253 42.20 enwik8 lz4hc 1.8.0 -9 12 MB/s 908 MB/s 42203096 42.20 enwik8 lz4hc 1.8.3 -9 11 MB/s 965 MB/s 42203094 42.20 enwik8 [1] lz4/lz4#566 lz4/lz4@08d347b [2] v1.8.1 perf: slightly faster compression and decompression speed lz4/lz4@a31b705 v1.8.2 perf: slightly faster HC compression and decompression speed lz4/lz4@45f8603 lz4/lz4@1a191b3 [3] http://mattmahoney.net/dc/textdata.html http://mattmahoney.net/dc/enwik8.zip Link: http://lkml.kernel.org/r/[email protected] Signed-off-by: Gao Xiang <[email protected]> Tested-by: Guo Xuenan <[email protected]> Cc: Colin Ian King <[email protected]> Cc: Yann Collet <[email protected]> Cc: Greg Kroah-Hartman <[email protected]> Cc: Fang Wei <[email protected]> Cc: Chao Yu <[email protected]> Cc: Miao Xie <[email protected]> Cc: Sven Schmidt <[email protected]> Cc: Kyungsik Lee <[email protected]> Cc: <[email protected]> Signed-off-by: Andrew Morton <[email protected]> Signed-off-by: Linus Torvalds <[email protected]> Signed-off-by: madeofgreat <[email protected]> Signed-off-by: MadeOfGreat <[email protected]> Signed-off-by: CloudedQuartz <[email protected]>
Kbuild now complains (rightly) about it. Signed-off-by: Linus Torvalds <[email protected]> Signed-off-by: madeofgreat <[email protected]> Signed-off-by: MadeOfGreat <[email protected]> Signed-off-by: CloudedQuartz <[email protected]>
This patch replaces all memcpy() calls with LZ4_memcpy() which calls __builtin_memcpy() so the compiler can inline it. LZ4 relies heavily on memcpy() with a constant size being inlined. In x86 and i386 pre-boot environments memcpy() cannot be inlined because memcpy() doesn't get defined as __builtin_memcpy(). An equivalent patch has been applied upstream so that the next import won't lose this change [1]. I've measured the kernel decompression speed using QEMU before and after this patch for the x86_64 and i386 architectures. The speed-up is about 10x as shown below. Code Arch Kernel Size Time Speed v5.8 x86_64 11504832 B 148 ms 79 MB/s patch x86_64 11503872 B 13 ms 885 MB/s v5.8 i386 9621216 B 91 ms 106 MB/s patch i386 9620224 B 10 ms 962 MB/s I also measured the time to decompress the initramfs on x86_64, i386, and arm. All three show the same decompression speed before and after, as expected. [1] lz4/lz4#890 Signed-off-by: Nick Terrell <[email protected]> Signed-off-by: Andrew Morton <[email protected]> Cc: Yann Collet <[email protected]> Cc: Gao Xiang <[email protected]> Cc: Sven Schmidt <[email protected]> Cc: Greg Kroah-Hartman <[email protected]> Cc: Ingo Molnar <[email protected]> Cc: Arvind Sankar <[email protected]> Link: http://lkml.kernel.org/r/[email protected] Signed-off-by: Linus Torvalds <[email protected]>
LZ4 final literal copy could be overlapped when doing in-place decompression, so it's unsafe to just use memcpy() on an optimized memcpy approach but memmove() instead. Upstream LZ4 has updated this years ago [1] (and the impact is non-sensible [2] plus only a few bytes remain), this commit just synchronizes LZ4 upstream code to the kernel side as well. It can be observed as EROFS in-place decompression failure on specific files when X86_FEATURE_ERMS is unsupported, memcpy() optimization of commit 59daa70 ("x86, mem: Optimize memcpy by avoiding memory false dependece") will be enabled then. Currently most modern x86-CPUs support ERMS, these CPUs just use "rep movsb" approach so no problem at all. However, it can still be verified with forcely disabling ERMS feature... arch/x86/lib/memcpy_64.S: ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \ - "jmp memcpy_erms", X86_FEATURE_ERMS + "jmp memcpy_orig", X86_FEATURE_ERMS We didn't observe any strange on arm64/arm/x86 platform before since most memcpy() would behave in an increasing address order ("copy upwards" [3]) and it's the correct order of in-place decompression but it really needs an update to memmove() for sure considering it's an undefined behavior according to the standard and some unique optimization already exists in the kernel. [1] lz4/lz4@33cb851 [2] lz4/lz4#717 (comment) [3] https://sourceware.org/bugzilla/show_bug.cgi?id=12518 Link: https://lkml.kernel.org/r/[email protected] Signed-off-by: Gao Xiang <[email protected]> Reviewed-by: Nick Terrell <[email protected]> Cc: Yann Collet <[email protected]> Cc: Miao Xie <[email protected]> Cc: Chao Yu <[email protected]> Cc: Li Guifu <[email protected]> Cc: Guo Xuenan <[email protected]> Signed-off-by: Andrew Morton <[email protected]> Signed-off-by: Linus Torvalds <[email protected]>
This operation was intentional, but tools such as smatch will warn that it might not have been. Signed-off-by: Andrew Morton <[email protected]> Cc: Yann Collet <[email protected]> Cc: Vasily Averin <[email protected]> Cc: Gao Xiang <[email protected]> Link: http://lkml.kernel.org/r/[email protected] Signed-off-by: Linus Torvalds <[email protected]>
Declare LZ4_decompress_safe_withPrefix64k as static to fix sparse warning: > warning: symbol 'LZ4_decompress_safe_withPrefix64k' was not declared. > Should it be static? Link: https://lkml.kernel.org/r/[email protected] Signed-off-by: Rajat Asthana <[email protected]> Reviewed-by: Nick Terrell <[email protected]> Cc: Gao Xiang <[email protected]> Cc: Joe Perches <[email protected]> Signed-off-by: Andrew Morton <[email protected]> Signed-off-by: Linus Torvalds <[email protected]>
When partialDecoding, it is EOF if we've either filled the output buffer or can't proceed with reading an offset for following match. In some extreme corner cases when compressed data is suitably corrupted, UAF will occur. As reported by KASAN [1], LZ4_decompress_safe_partial may lead to read out of bound problem during decoding. lz4 upstream has fixed it [2] and this issue has been disscussed here [3] before. current decompression routine was ported from lz4 v1.8.3, bumping lib/lz4 to v1.9.+ is certainly a huge work to be done later, so, we'd better fix it first. [1] https://lore.kernel.org/all/[email protected]/ [2] lz4/lz4@c5d6f8a [3] https://lore.kernel.org/all/[email protected]/ Link: https://lkml.kernel.org/r/[email protected] Reported-by: [email protected] Signed-off-by: Guo Xuenan <[email protected]> Reviewed-by: Nick Terrell <[email protected]> Acked-by: Gao Xiang <[email protected]> Cc: Yann Collet <[email protected]> Cc: Chengyang Fan <[email protected]> Cc: <[email protected]> Signed-off-by: Andrew Morton <[email protected]> Signed-off-by: Linus Torvalds <[email protected]>
LZ4_decompress_safe_forceExtDict() is only used in lib/lz4/lz4_decompress.c, make it static to fix the build warning about "no previous prototype" [1]. [1] https://lore.kernel.org/lkml/[email protected]/ Link: https://lkml.kernel.org/r/[email protected] Signed-off-by: Tiezhu Yang <[email protected]> Reported-by: kernel test robot <[email protected]> Signed-off-by: Andrew Morton <[email protected]>
Update lz4 module using official repository from revision [1]. Keep in mind lz4hc wasn't updated thus it is not used. It may not compile anymore. [1]: https://github.com/lz4/lz4/tree/4ebe313e00aa52c837ee029ede39a0503a8a39c9
Signed-off-by: Andrzej Perczak <[email protected]>
Change-Id: I3c8dd91df090bb692784a6b7a61c8877b1e1dfba
Signed-off-by: ExtremeXT <[email protected]>
Change-Id: I906a03d15c785ab33ec65c6a2678d2c4a422838f Signed-off-by: Samuel Pascua <[email protected]>
Signed-off-by: Tyler Nijmeh <[email protected]> Signed-off-by: ThunderStorms21th nalas <[email protected]>
commit ec527c318036a65a083ef68d8ba95789d2212246 upstream.
As explained in
0cc3cd21657b ("cpu/hotplug: Boot HT siblings at least once")
we always, no matter what, have to bring up x86 HT siblings during boot at
least once in order to avoid first MCE bringing the system to its knees.
That means that whenever 'nosmt' is supplied on the kernel command-line,
all the HT siblings are as a result sitting in mwait or cpudile after
going through the online-offline cycle at least once.
This causes a serious issue though when a kernel, which saw 'nosmt' on its
commandline, is going to perform resume from hibernation: if the resume
from the hibernated image is successful, cr3 is flipped in order to point
to the address space of the kernel that is being resumed, which in turn
means that all the HT siblings are all of a sudden mwaiting on address
which is no longer valid.
That results in triple fault shortly after cr3 is switched, and machine
reboots.
Fix this by always waking up all the SMT siblings before initiating the
'restore from hibernation' process; this guarantees that all the HT
siblings will be properly carried over to the resumed kernel waiting in
resume_play_dead(), and acted upon accordingly afterwards, based on the
target kernel configuration.
Symmetricaly, the resumed kernel has to push the SMT siblings to mwait
again in case it has SMT disabled; this means it has to online all
the siblings when resuming (so that they come out of hlt) and offline
them again to let them reach mwait.
Cc: 4.19+ <[email protected]> # v4.19+
Debugged-by: Thomas Gleixner <[email protected]>
Fixes: 0cc3cd21657b ("cpu/hotplug: Boot HT siblings at least once")
Signed-off-by: Jiri Kosina <[email protected]>
Acked-by: Pavel Machek <[email protected]>
Reviewed-by: Thomas Gleixner <[email protected]>
Reviewed-by: Josh Poimboeuf <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
[ Upstream commit da6043fe85eb5ec621e34a92540735dcebbea134 ] When looking for a bit by number we make use of the cached result from the preceding lookup to speed up operation. Firstly we check if the requested pfn is within the cached zone and if not lookup the new zone. We then check if the offset for that pfn falls within the existing cached node. This happens regardless of whether the node is within the zone we are now scanning. With certain memory layouts it is possible for this to false trigger creating a temporary alias for the pfn to a different bit. This leads the hibernation code to free memory which it was never allocated with the expected fallout. Ensure the zone we are scanning matches the cached zone before considering the cached node. Deep thanks go to Andrea for many, many, many hours of hacking and testing that went into cornering this bug. Reported-by: Andrea Righi <[email protected]> Tested-by: Andrea Righi <[email protected]> Signed-off-by: Andy Whitcroft <[email protected]> Signed-off-by: Rafael J. Wysocki <[email protected]> Signed-off-by: Sasha Levin <[email protected]>
commit 2351f8d295ed63393190e39c2f7c1fee1a80578f upstream. Currently the kernel threads are not frozen in software_resume(), so between dpm_suspend_start(PMSG_QUIESCE) and resume_target_kernel(), system_freezable_power_efficient_wq can still try to submit SCSI commands and this can cause a panic since the low level SCSI driver (e.g. hv_storvsc) has quiesced the SCSI adapter and can not accept any SCSI commands: https://lkml.org/lkml/2020/4/10/47 At first I posted a fix (https://lkml.org/lkml/2020/4/21/1318) trying to resolve the issue from hv_storvsc, but with the help of Bart Van Assche, I realized it's better to fix software_resume(), since this looks like a generic issue, not only pertaining to SCSI. Cc: All applicable <[email protected]> Signed-off-by: Dexuan Cui <[email protected]> Signed-off-by: Rafael J. Wysocki <[email protected]> Signed-off-by: Greg Kroah-Hartman <[email protected]>
Signed-off-by: topser9 <[email protected]> modified: drivers/misc/Kconfig modified: drivers/misc/Makefile new file: drivers/misc/battery_saver.c new file: include/linux/battery_saver.h
…ranciscoFranco's non-generic driver. Sysfs node: /sys/class/misc/boeffla_wakelock_blocker/wakelock_blocker - list of wakelocks to be blocked, separated by semicolons /sys/class/misc/boeffla_wakelock_blocker/debug - write: 0/1 to switch off and on debug logging into dmesg - read: get current driver internals /sys/class/misc/boeffla_wakelock_blocker/version - show driver version Signed-off-by: andip71 <[email protected]> Signed-off-by: topser9 <[email protected]> modified: drivers/base/power/Makefile new file: drivers/base/power/boeffla_wl_blocker.c new file: drivers/base/power/boeffla_wl_blocker.h modified: drivers/base/power/main.c modified: drivers/base/power/wakeup.c modified: kernel/power/Kconfig
This reverts commit 64c0e76.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.