From patchwork Sat Oct 19 01:40:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Siddhesh Poyarekar X-Patchwork-Id: 99227 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8C4C13858C48 for ; Sat, 19 Oct 2024 01:41:43 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 8C4C13858C48 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1729302103; bh=k3fxlDhgNX2QB9nhUvqaKj9bdoZ3UUadN5ufzNWVxnI=; h=From:To:Cc:Subject:Date:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From; b=oZlTgLPUdv2gnp+ThwmnfTs6mUn4NV+ueIyPOIlXECyMfqZ7qtiptnvdi1R46T5aJ htfjEwK/uN9nV//zTdw9oIghKwWpzv8I6/y97bWk84bfWifqe00krvN+lDhzzJAGyH HaPf4kqgfy3MoA2GWVYG6hazEf/XNfom1g3UdGyE= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from giraffe.ash.relay.mailchannels.net (giraffe.ash.relay.mailchannels.net [23.83.222.69]) by sourceware.org (Postfix) with ESMTPS id 3A5CD3858C48 for ; Sat, 19 Oct 2024 01:40:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3A5CD3858C48 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=sourceware.org Authentication-Results: sourceware.org; spf=fail smtp.mailfrom=sourceware.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 3A5CD3858C48 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=23.83.222.69 ARC-Seal: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1729302021; cv=pass; b=JDki3LGKGhvCnBZmsWIT+hJQN6q/SJMU4F7/J50sN0iF0dIj+9shWh4N9k3C8qBiOt1Lp/cVEeeD4u4qn2enNt7BYtgXxnovwkvm5v6cFemj/BGnuR6si61JnfWTST/AShPIHVP+CUSRbs+sgURuWuORzPTGNPDGONZFcGx/GrU= ARC-Message-Signature: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1729302021; c=relaxed/simple; bh=w9JsBdWBsp4cwoeL0RsUWxUkOASq2fFSlKvSqngoCNc=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=WU4lOOh/Ddgncqz0R9eSWP/9fJYrx6Rh9S1/3rkit3pl3EIB10jrE6RPDV8Z/bisPVEnON0d/hFHitlltPg5nZS0hAqNyJ8Fp+yLUyVtBIWMvmQJRgb3LfSw+pWnjkWKSJHRZpee1X4jlEXP7BZzDrArzyuFngUXKeugjWpSOh0= ARC-Authentication-Results: i=2; server2.sourceware.org X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id C7B2F25BF4; Sat, 19 Oct 2024 01:40:12 +0000 (UTC) Received: from pdx1-sub0-mail-a233.dreamhost.com (trex-6.trex.outbound.svc.cluster.local [100.101.175.246]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 66750269CB; Sat, 19 Oct 2024 01:40:12 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1729302012; a=rsa-sha256; cv=none; b=SmfEIuF5Cq4tQbVTboSrGafTkNU9uO0LgdtXHJNPInTDrA8AHn4/b1sWPHd+gs06u0Benk IKSou7nZu/TWF4zQjY2aNjL8ucjse1tHCzvB86aWB1+rWFk56/Ie5HI7dOrmwT6nXbOVuw AFOcfaUIrTkWnmO870wtDp70FZlnJ816jh8peumWPuIkR1znzwdxSHNw0o6WhzrzgErQTs 5y/zsRytKftutexl9GoaZGaMAA40FXANO60B0gWeGzyZwTFpP2lSwXxCx15e6oc/3Oak1M E1x+4zYK6w0dRJd2Id23zz6dg7dOHP1sBGf3dH+HuKAWOVrewMikrAupZbe/mw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1729302012; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=k3fxlDhgNX2QB9nhUvqaKj9bdoZ3UUadN5ufzNWVxnI=; b=2bsIHn7PCA0mlpRUSdqvlTtuZJBli+CgEEhjXjduvyZOF7U/6mRE8nNPOvL+qsxKF0/Kw5 cA5BCMUQOa43rcQJ/oRIEe2BXM6l6ffULSfqPFDbls4tATIpl/PANlme00+iAp/KvESQsl 3o1EOfpORF75C2c7aGtOjOaG0l6nm5z6rNJQ/LHJ9m6JUiqWvAIten2l8QLr6rv61unt61 WACp3lgWVJQVCf9mWnIfXRd6aJ/8XB6MivKyX7RheA5YHP73LxsYepaU1ybZpQAKa8CVIU r4+8hAa9ZlWgNLOaMkelJVexMsMKt1jTdHMkcs7pIxvA3avkCKGmIh8g9up/gA== ARC-Authentication-Results: i=1; rspamd-6bcbdd57f8-tntgg; auth=pass smtp.auth=dreamhost smtp.mailfrom=siddhesh@sourceware.org X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|siddhesh@gotplt.org X-MailChannels-Auth-Id: dreamhost X-Bored-Tank: 11ed64f74aef9fa0_1729302012699_1430353166 X-MC-Loop-Signature: 1729302012699:2026457506 X-MC-Ingress-Time: 1729302012699 Received: from pdx1-sub0-mail-a233.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.101.175.246 (trex/7.0.2); Sat, 19 Oct 2024 01:40:12 +0000 Received: from fedora.redhat.com (bras-base-toroon4859w-grc-89-184-146-156-41.dsl.bell.ca [184.146.156.41]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: siddhesh@gotplt.org) by pdx1-sub0-mail-a233.dreamhost.com (Postfix) with ESMTPSA id 4XVklv3gX5z66; Fri, 18 Oct 2024 18:40:11 -0700 (PDT) From: Siddhesh Poyarekar To: libc-alpha@sourceware.org Cc: dj@redhat.com, carlos@redhat.com, fweimer@redhat.com, josmyers@redhat.com, alx@kernel.org, nabijaczleweli@nabijaczleweli.xyz, douglas.mcilroy@dartmouth.edu, eggert@cs.ucla.edu Subject: [RFC PATCH] realloc: Make REALLOC_ZERO_BYTES_FREES into a tunable Date: Fri, 18 Oct 2024 21:40:02 -0400 Message-ID: <20241019014002.3684656-1-siddhesh@sourceware.org> X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 X-Spam-Status: No, score=-1170.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_SOFTFAIL, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org Allow users to select this behaviour at runtime, which should allow them flexibility to have glibc realloc do the same thing that many other implementations do for realloc (p, 0). The default behaviour remains unchanged for now. Signed-off-by: Siddhesh Poyarekar --- The have-cake-and-eat-it-too solution. Lightly tested, mainly intended to get feedback from folks. NEWS | 4 +++ elf/dl-tunables.list | 6 ++++ malloc/Makefile | 17 ++++++++++-- malloc/arena.c | 3 ++ malloc/malloc-check.c | 2 +- malloc/malloc-debug.c | 5 +++- malloc/malloc.c | 49 +++++++++++++++++---------------- malloc/mcheck-impl.c | 9 +++++- malloc/tst-realloc-nonfreeing.c | 35 +++++++++++++++++++++++ manual/memory.texi | 11 ++++++-- manual/tunables.texi | 9 ++++++ 11 files changed, 119 insertions(+), 31 deletions(-) create mode 100644 malloc/tst-realloc-nonfreeing.c diff --git a/NEWS b/NEWS index 2fe0396b2d..00abfd896e 100644 --- a/NEWS +++ b/NEWS @@ -31,6 +31,10 @@ Major new features: * The iconv program now supports converting files in place. The program automatically uses a temporary file if required. +* A new tunable glibc.malloc.realloc_zero_bytes_frees can be set to 0 to + make the realloc function return a zero sized pointer instead of NULL + when the requested new size is 0 bytes. + Deprecated and removed features, and other changes affecting compatibility: * The big-endian ARC port (arceb-linux-gnu) has been removed. diff --git a/elf/dl-tunables.list b/elf/dl-tunables.list index 40ac5b3776..5dcf534332 100644 --- a/elf/dl-tunables.list +++ b/elf/dl-tunables.list @@ -82,6 +82,12 @@ glibc { type: SIZE_T minval: 0 } + realloc_zero_bytes_frees { + type: INT_32 + minval: 0 + maxval: 1 + default: 1 + } } elision { diff --git a/malloc/Makefile b/malloc/Makefile index 1630aaf6ac..fcf0806fe1 100644 --- a/malloc/Makefile +++ b/malloc/Makefile @@ -61,6 +61,7 @@ tests := \ tst-pvalloc \ tst-pvalloc-fortify \ tst-realloc \ + tst-realloc-nonfreeing \ tst-reallocarray \ tst-safe-linking \ tst-tcfree1 tst-tcfree2 tst-tcfree3 \ @@ -136,12 +137,17 @@ tests-exclude-hugetlb1 = \ tst-mallocfork2 \ tst-mallocfork3 \ tst-mallocstate \ + tst-realloc-nonfreeing \ # tests-exclude-hugetlb1 + # The tst-free-errno relies on the used malloc page size to mmap an # overlapping region. tests-exclude-hugetlb2 = \ - $(tests-exclude-hugetlb1) \ - tst-free-errno + $(tests-exclude-hugetlb1) \ + tst-free-errno \ + tst-realloc-nonfreeing \ +# tests-exclude-hugetlb2 + tests-malloc-hugetlb1 = \ $(filter-out $(tests-exclude-hugetlb1), $(tests)) tests-malloc-hugetlb2 = \ @@ -338,6 +344,13 @@ tst-malloc-usable-ENV = MALLOC_CHECK_=3 \ tst-malloc-usable-tunables-ENV = GLIBC_TUNABLES=glibc.malloc.check=3 \ LD_PRELOAD=$(objpfx)/libc_malloc_debug.so +tst-realloc-nonfreeing-ENV += \ + GLIBC_TUNABLES=glibc.malloc.realloc_zero_bytes_frees=0 +tst-realloc-nonfreeing-malloc-check-ENV += \ + GLIBC_TUNABLES=glibc.malloc.realloc_zero_bytes_frees=0 +tst-realloc-nonfreeing-mcheck-ENV += \ + GLIBC_TUNABLES=glibc.malloc.realloc_zero_bytes_frees=0 + tst-mxfast-ENV = GLIBC_TUNABLES=glibc.malloc.tcache_count=0:glibc.malloc.mxfast=0 CPPFLAGS-malloc-debug.c += -DUSE_TCACHE=0 diff --git a/malloc/arena.c b/malloc/arena.c index cfb1ff8854..99eb748f9b 100644 --- a/malloc/arena.c +++ b/malloc/arena.c @@ -253,6 +253,7 @@ TUNABLE_CALLBACK_FNDECL (set_tcache_unsorted_limit, size_t) #endif TUNABLE_CALLBACK_FNDECL (set_mxfast, size_t) TUNABLE_CALLBACK_FNDECL (set_hugetlb, size_t) +TUNABLE_CALLBACK_FNDECL (set_realloc_zero_bytes_frees, int32_t) #if USE_TCACHE static void tcache_key_initialize (void); @@ -312,6 +313,8 @@ ptmalloc_init (void) # endif TUNABLE_GET (mxfast, size_t, TUNABLE_CALLBACK (set_mxfast)); TUNABLE_GET (hugetlb, size_t, TUNABLE_CALLBACK (set_hugetlb)); + TUNABLE_GET (realloc_zero_bytes_frees, int32_t, + TUNABLE_CALLBACK (set_realloc_zero_bytes_frees)); if (mp_.hp_pagesize > 0) { diff --git a/malloc/malloc-check.c b/malloc/malloc-check.c index da1158b333..3a4941414b 100644 --- a/malloc/malloc-check.c +++ b/malloc/malloc-check.c @@ -257,7 +257,7 @@ realloc_check (void *oldmem, size_t bytes) if (oldmem == 0) return malloc_check (bytes); - if (bytes == 0) + if (mp_.realloc_zero_bytes_frees == 1 && bytes == 0) { free_check (oldmem); return NULL; diff --git a/malloc/malloc-debug.c b/malloc/malloc-debug.c index c9b4166a1e..1fde57722c 100644 --- a/malloc/malloc-debug.c +++ b/malloc/malloc-debug.c @@ -73,9 +73,12 @@ __malloc_debug_disable (enum malloc_debug_hooks flag) __malloc_debugging_hooks &= ~flag; } +/* The ordering matters here because mcheck.c includes elf/dl-tunables.h, which + ends up messing with the same include in malloc-check.c. This probably + needs to be fixed at some point. */ +#include "malloc-check.c" #include "mcheck.c" #include "mtrace.c" -#include "malloc-check.c" #if SHLIB_COMPAT (libc_malloc_debug, GLIBC_2_0, GLIBC_2_24) extern void (*__malloc_initialize_hook) (void); diff --git a/malloc/malloc.c b/malloc/malloc.c index bcb6e5b83c..be37f2b117 100644 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -177,7 +177,6 @@ USE_PUBLIC_MALLOC_WRAPPERS NOT defined USE_MALLOC_LOCK NOT defined MALLOC_DEBUG NOT defined - REALLOC_ZERO_BYTES_FREES 1 TRIM_FASTBINS 0 Options for customizing MORECORE: @@ -330,20 +329,6 @@ ((__typeof (ptr)) ((((size_t) pos) >> 12) ^ ((size_t) ptr))) #define REVEAL_PTR(ptr) PROTECT_PTR (&ptr, ptr) -/* - The REALLOC_ZERO_BYTES_FREES macro controls the behavior of realloc (p, 0) - when p is nonnull. If the macro is nonzero, the realloc call returns NULL; - otherwise, the call returns what malloc (0) would. In either case, - p is freed. Glibc uses a nonzero REALLOC_ZERO_BYTES_FREES, which - implements common historical practice. - - ISO C17 says the realloc call has implementation-defined behavior, - and it might not even free p. -*/ - -#ifndef REALLOC_ZERO_BYTES_FREES -#define REALLOC_ZERO_BYTES_FREES 1 -#endif /* TRIM_FASTBINS controls whether free() of a very small chunk can @@ -628,9 +613,9 @@ void* __libc_calloc(size_t, size_t); ANSI) and p is NOT freed. if n is for fewer bytes than already held by p, the newly unused - space is lopped off and freed if possible. Unless the #define - REALLOC_ZERO_BYTES_FREES is set, realloc with a size argument of - zero (re)allocates a minimum-sized chunk. + space is lopped off and freed if possible. Unless the + REALLOC_ZERO_BYTES_FREES tunable is set, realloc with a size argument + of zero (re)allocates a minimum-sized chunk. Large chunks that were internally obtained via mmap will always be grown using malloc-copy-free sequences unless the system supports @@ -1861,6 +1846,7 @@ struct malloc_par INTERNAL_SIZE_T mmap_threshold; INTERNAL_SIZE_T arena_test; INTERNAL_SIZE_T arena_max; + int realloc_zero_bytes_frees; /* Transparent Large Page support. */ INTERNAL_SIZE_T thp_pagesize; @@ -1919,7 +1905,8 @@ static struct malloc_par mp_ = .mmap_threshold = DEFAULT_MMAP_THRESHOLD, .trim_threshold = DEFAULT_TRIM_THRESHOLD, #define NARENAS_FROM_NCORES(n) ((n) * (sizeof (long) == 4 ? 2 : 8)) - .arena_test = NARENAS_FROM_NCORES (1) + .arena_test = NARENAS_FROM_NCORES (1), + .realloc_zero_bytes_frees = 1 #if USE_TCACHE , .tcache_count = TCACHE_FILL_COUNT, @@ -3413,12 +3400,11 @@ __libc_realloc (void *oldmem, size_t bytes) if (!__malloc_initialized) ptmalloc_init (); -#if REALLOC_ZERO_BYTES_FREES - if (bytes == 0 && oldmem != NULL) + if (mp_.realloc_zero_bytes_frees && bytes == 0 && oldmem != NULL) { - __libc_free (oldmem); return 0; + __libc_free (oldmem); + return 0; } -#endif /* realloc of null is supposed to be same as malloc */ if (oldmem == 0) @@ -5554,6 +5540,23 @@ do_set_hugetlb (size_t value) return 0; } +/* The REALLOC_ZERO_BYTES_FREES tunable controls the behavior of realloc (p, 0) + when p is nonnull. If the macro is nonzero, the realloc call returns NULL; + otherwise, the call returns what malloc (0) would. In either case, + p is freed. Glibc uses a nonzero REALLOC_ZERO_BYTES_FREES by default to + remain compatible with older programs. + + ISO C17 says the realloc call has implementation-defined behavior, + and it might not even free p. */ +static __always_inline int +do_set_realloc_zero_bytes_frees (int32_t value) +{ + if (value == 0) + mp_.realloc_zero_bytes_frees = 0; + + return 0; +} + int __libc_mallopt (int param_number, int value) { diff --git a/malloc/mcheck-impl.c b/malloc/mcheck-impl.c index eff4873f62..6d1cd8876e 100644 --- a/malloc/mcheck-impl.c +++ b/malloc/mcheck-impl.c @@ -16,6 +16,7 @@ License along with the GNU C Library; if not, see . */ +#include #include #include #include @@ -31,6 +32,7 @@ /* Function to call when something awful happens. */ static void (*abortfunc) (enum mcheck_status); +static int realloc_zero_bytes_frees = 1; struct hdr { @@ -285,7 +287,7 @@ realloc_mcheck_before (void **ptrp, size_t *sizep, size_t *oldsize, return true; } - if (size == 0) + if (realloc_zero_bytes_frees == 1 && size == 0) { __debug_free (ptr); *victimp = NULL; @@ -401,6 +403,11 @@ __mcheck_initialize (void (*func) (enum mcheck_status), bool in_pedantic) } pedantic = in_pedantic; + + if (TUNABLE_GET_FULL (glibc, malloc, realloc_zero_bytes_frees, int32_t, NULL) + == 0) + realloc_zero_bytes_frees = 0; + return 0; } diff --git a/malloc/tst-realloc-nonfreeing.c b/malloc/tst-realloc-nonfreeing.c new file mode 100644 index 0000000000..eec9fcb767 --- /dev/null +++ b/malloc/tst-realloc-nonfreeing.c @@ -0,0 +1,35 @@ +/* Smoke test for realloc when glibc.malloc.realloc_zero_bytes_frees tunable + is set to zero. + Copyright The GNU Toolchain Authors. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +static int +do_test (void) +{ + char *foo = malloc (42); + char *p = realloc (foo, 0); + + TEST_VERIFY (p != NULL); + free (p); + + return 0; +} + +#include diff --git a/manual/memory.texi b/manual/memory.texi index 3710d7ec66..5ade11101e 100644 --- a/manual/memory.texi +++ b/manual/memory.texi @@ -834,9 +834,14 @@ contents. If you pass a null pointer for @var{ptr}, @code{realloc} behaves just like @samp{malloc (@var{newsize})}. -Otherwise, if @var{newsize} is zero -@code{realloc} frees the block and returns @code{NULL}. -Otherwise, if @code{realloc} cannot reallocate the requested size + +Otherwise, if @var{newsize} is zero @code{realloc} will by default free +the block and return @code{NULL}. This behavior can be overridden using +the @code{glibc.malloc.realloc_zero_bytes_frees} tunable, where instead +@code{realloc} will then free @var{ptr} and attempt to return a zero +sized block. + +Finally, if @code{realloc} cannot reallocate the requested size it returns @code{NULL} and sets @code{errno}; the original block is left undisturbed. @end deftypefun diff --git a/manual/tunables.texi b/manual/tunables.texi index 0b1b2898c0..b76b643313 100644 --- a/manual/tunables.texi +++ b/manual/tunables.texi @@ -294,6 +294,15 @@ supported ones. If provided value is invalid, @code{MAP_HUGETLB} will not be used. @end deftp +@deftp Tunable glibc.malloc.realloc_zero_bytes_frees + +By default when the @code{realloc} function is called with the new size as +@var{0}, it frees the input pointer and returns @var{NULL}. Setting the value +of this tunable to @var{0} will result in @code{realloc} instead freeing the +input pointer and attempting to allocate and return a zero sized block. Any +values other than @var{0} are ignored. +@end deftp + @node Dynamic Linking Tunables @section Dynamic Linking Tunables @cindex dynamic linking tunables