From patchwork Mon Dec 2 19:42:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cupertino Miranda X-Patchwork-Id: 102269 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C50CD3858282 for ; Mon, 2 Dec 2024 19:47:58 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C50CD3858282 Authentication-Results: sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-11-20 header.b=IcroR8cF; dkim=pass (1024-bit key, unprotected) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.a=rsa-sha256 header.s=selector2-oracle-onmicrosoft-com header.b=uKvkAGLE X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by sourceware.org (Postfix) with ESMTPS id BCD733858C35 for ; Mon, 2 Dec 2024 19:43:35 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org BCD733858C35 Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=oracle.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org BCD733858C35 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1733168615; cv=pass; b=UZnGXYYzdl2r/l7mRB8YSVcJGoV0ykkGOXOCRdkWDhoFSb4kZuUNO3b+U2kP2ggWfS4rokZcTYN7DhGVWwbWOCpIYbLws+xK/MY0pAyuGxG0RZfzA0HheVz4Ywv2CINWZ/7W1p7SfP3HQz0gpw/tUIpT1fAo9N2sCa7sxg7upKI= ARC-Message-Signature: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1733168615; c=relaxed/simple; bh=pPC46md/0eUt9Jxy8Zg3WnM8BqomImRGP8/bARif5Mc=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-Id: MIME-Version; b=FZZRdDTE0qHN6/k672xS3utDGLrBs/KaE934GkIznSFFoSmofMOJ8EEEbA+7byYxygVhBs1t9xnIBaSKXEC4LMVUP128f/kxUfxdY8d+EDfJa7Z8wQGwSX/IsL78/d1YPu1FEtWwQZRu56DsOezIDZWems3K/zbPIvhSJ5OVfrY= ARC-Authentication-Results: i=2; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org BCD733858C35 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2HfcSk006336 for ; Mon, 2 Dec 2024 19:43:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=HZOI1QxffyuZURYhmXx+RbovTTmAl3p0SGf7/zpj9aA=; b= IcroR8cFTZeJ4iHjjRiFWmz71NHLz+J1Sgci8GyaImysk+PmexNrOtrzlmn4AQPm gp6lNd5cBtDo7w8X517YIfZeOZifTKLuni/6cGLnL/yD/EhlDll7/6Rc5ph8NyUV wabuahi46QOQm9qlkIuIjJQTBmAJgNmAuP1AIGQm4f1hDAyZtLQ7sEl93wUUn1z6 c2CYoPu7qGh98/oB0Y4RSl+r+EarNStT2TmMA2bgjmbz/HTA9oKN29pAQqSjRaJO te20XQypklSqVpNCam+vOLzX/q1wgsE2RcwoCXiIH5CVpxTky+kvn0n13fkStVPf AWOKq9Qua8a4W7CNCgi6kQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437tas4fhw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 02 Dec 2024 19:43:34 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2J1JWw038063 for ; Mon, 2 Dec 2024 19:43:34 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2173.outbound.protection.outlook.com [104.47.58.173]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 437s57dpew-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 02 Dec 2024 19:43:34 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=C66qMWnBs9WIYNjGcX1oZy6kiaD3CQ0+xOk5Vv5GINuI/O87zPV2PZVxcJtRs0JKamIKxhYAjycO87V2L2QFirHukOOScCYTvbWD/cFpodqehqVnmF0dSe94WuS8rBYLyCjWdGpidieWjOxae8P10njNM2/Fk6otEwJYGoK5pqkb6iybxx0kCZ/kz5xr3BgYbD6HGhryHrAvYlhJ+s10EUgzKaj3EN1PLs6vj77aijVxk75vxjxDn/vUNsRG2qSWgohT3NbM2Z2Aj2QyVkuwujgdh8m5bRUKqdk4DsTu7rZXQT3lXs30a9UnklLBKP+b7pJXwxkV6RS7LEL88fcOdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HZOI1QxffyuZURYhmXx+RbovTTmAl3p0SGf7/zpj9aA=; b=Lc0VZZok9+/Detk24yLK/XxXkBFlXAtxzBWaMVYv8W6tmZ5OcjHtQagYXAAqmHI+CbJS6/FFc6cfUJCcqUnv2nWK3O723Iwbq8kQa0asitw8nhLb/dUTj6i5TqwYUu1dTzj+6oHsxGdQy4AGHcphNMwyfxoqyMNVKY3V9ZnXGHv89tlVn9HeaOdQ7377ufbqZpvGhz8RUHB45VEBgzgH+jSFxsxLPtJfWhcHVZvxQL5p0V9WIlt9jsRIXeDkWswFV1dkko3JvrpSXJhXMVwPguI2TnFMIuEngwW0jBMC8ZLvBTva5suPg2tGUrv2L5SG0GbX7C/rbOFPNUN96USeeg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HZOI1QxffyuZURYhmXx+RbovTTmAl3p0SGf7/zpj9aA=; b=uKvkAGLEviOEnkvFA9gvxCR6oJ/klufheo8+zvXtXjzVkk0jAdsfbd2AeEcITiUR0JN7YhzQsx3QLmW5cvKh8AkPNsVsGs8+NQi8PZNA7OzfZAqSJsyWXhwc9R5TDYKAMfsQa/dn4y9bLKX2aQA/wzQe2ky6xyR1DZbpA0f+P+E= Received: from MN2PR10MB4382.namprd10.prod.outlook.com (2603:10b6:208:1d7::13) by SJ0PR10MB4813.namprd10.prod.outlook.com (2603:10b6:a03:2d2::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.18; Mon, 2 Dec 2024 19:43:31 +0000 Received: from MN2PR10MB4382.namprd10.prod.outlook.com ([fe80::5033:84a3:f348:fefb]) by MN2PR10MB4382.namprd10.prod.outlook.com ([fe80::5033:84a3:f348:fefb%7]) with mapi id 15.20.8207.017; Mon, 2 Dec 2024 19:43:31 +0000 From: Cupertino Miranda To: libc-alpha@sourceware.org Cc: jose.marchesi@oracle.com, elena.zannoni@oracle.com, Cupertino Miranda Subject: [PATCH 1/2] malloc: add tcache support for large chunk caching Date: Mon, 2 Dec 2024 19:42:48 +0000 Message-Id: <20241202194249.95317-2-cupertino.miranda@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241202194249.95317-1-cupertino.miranda@oracle.com> References: <20241202194249.95317-1-cupertino.miranda@oracle.com> X-ClientProxiedBy: LNXP265CA0074.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:76::14) To MN2PR10MB4382.namprd10.prod.outlook.com (2603:10b6:208:1d7::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR10MB4382:EE_|SJ0PR10MB4813:EE_ X-MS-Office365-Filtering-Correlation-Id: 042d2f77-0126-4c3d-d2dc-08dd13099979 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|366016|1800799024|10070799003|376014; X-Microsoft-Antispam-Message-Info: 2fjGsR9U0PSdfvL7KBWmWQwqHT5I9/LtBquAsTmqHF2kXC16T7pYzIxoRR7reAwExLi52GDg2xsc6JqHF4qrPhIhnNeFyKyEX3tfhfLnHKCATvHdOSeKVX3LnIuEgVZOOg998+ehwRHfnU7xWeaRjDWOCYpee5QSs7IrfaOJu0wGWpDUaD9onHW7y2Z4ocUvYZVEHNTioBhjnjeX/VgbSxqurGFUc59resy6xeltPa29ObH+HaNNs+L2A0CZQKnJ3ciYs7P82Y52TI2lwk/BgWT2DeTW1iHKIBDFXwnHNM/wynm9Xnnh6N8ALlmO4YFoGIj/p3QKlx61hFJ084wfDq8CtNIZKKMQvPCOUQQQMVzB2Ln+WozMcUTicdWIQsQ6OhO+5p7eQEJfyAcqXhO99cf4zaV5cyJK76TDhJ8tl/wjLJhkXpJvQReJ/imxnj1Vi7Bamfq3fcwtJkzskEhNH0MJkXFZmC0X/9FHVklQLJ6px9CMZJy2+mrqC7p5gejgw7P4ESmRP2cuUJ81SfCNrmOenv0itsTcGnYRTGlEEAN92VbXvzCgXr6kB2P8dGGKS617K77qzynhFRkGCeRR9Yn2xg8OYCeZC9m1HyDpuSlY1jlYoeX4bXXgtvu5JC4PxkDk6vsAQFEG6zLn9BItMNehTYJGHJEARtWSQpwIQpbTRN5bx3nJXlwvxbgIuzYBrAV0nF2DO4MJQQTIyEpuFFcpy4JfVU+xcBqgcXeDtd6S3iNfYeCmT+SDEZmx16cI0KTUquywIFwvcfaKqqGNq2iLA9gweab1PIJdif1Pz68Cg8qq4b+MEVpteZyLxtWKiHA5fLklss4UD8d4jMWcW+SSuJjVnwd/Czf/+E/ys87yumWjO/2lGu/seaZiF8clxpDDvYkr2IB2z5w2P6Xxt4XwTsTLpUNzCFtZcUkt52tY6Ace0D4V+sC9YefDKY6jemABK+DZCUmXKRb0bobQJkfwQ6HGw4CihUCpLHQ8QI5CAUDX1aodhn6pfQuth6yNNpZAeY7lmA5H/k1yfxOcFOtuaQl4gJ4wPodcSrWuCti0ZdgFl9PA3YGpXTBLZv8jM3hLfTxmZs8w5LEPdPMGf8dCtodoYOd/Z8xiB32kKPjAzXBXR41RjjhM9VoLcMNxYcNe0IoPf3HXBP2XcQjX9RH8WIMJoSNSECdOs+SeswsKolGV5fPvgbRQIuOPgq5nmObjWqOUn1kjWDizS/BJUxZUsAH0ihuZXK6kMLAxLo1Tnj19RzysUl2R56ieX9EQW+tlcV2xf5WNJuZO3BhirQzAdU0BoS8BRxidws8Zeyv5O1VWDuNPPDXdSRatzjR0TP1zDv9pP3OcUSepHCA7Pg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4382.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(1800799024)(10070799003)(376014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: VEnqf7lqfKxu3KotGo5WonQaUT1fVYsvmVLbYOgvsem8tXedXvaYh37c3z+6v0txHSIl7qXLSsE3BWXhwEY618uW6T+1yTsph3B6XtUfVQVaf6br2InzmfmHEX9cx7ftouD7TQvQNDtNoXLp78aVuPsCsgpK0nc+HrNmd+Uup4sr2/eypGjcm6S/Ej55fKvEEYsmd1ndakaAoV1vHcRYiu94V9Uw3ugPUi/mZMUETJmj3pHTdWGdfGhBnugwQU8+0e081+YOflVG/W+jJO78tovgpMnoaxNbQQ2xG7/ahCSvM6KMofFWxmkR5OVQ3j0TphsI7naFn/uuPoGXc/c0GsJEtYWltOZiLzJ326b6GrONk+MXo6tYeYfEdWETtf04txX/GqdFdp3DqNnSeIn3EH5AB6Uk4RML55nZxMhfGR6MKWzoom8Tw9kHt7q3V13VWdN72HyGuKNMjtQdwA2Hj55UwKk0yx9MFsX+eEl2UOO9H3pZdNaZ6TceOErgyOvZqcTzzG+1b0tCDONMMrenElDWPPwZlfFlrAkPGLflpt8wwgpS06/j1m/x4r7UepEtCHgYf/EbKrfCxbXFY/oSpESbw81d7k1eWIF3us62h4eSScT0JmR+/wHgF9H9z1BqAz4vg/WXD6sXkifpWD5BvUfGCrfeB0Mw2LBQj/b9FVuPgrYpMFODEOO2sqdXZhRVA/5czFbRfhGPhdPFQcEH86JtgvQ6mgZDGdhZw0Ui8C/z4ggjnFmYINsxfubbp2Fp+MUGO9v9AfXXlHWa/HCXn/9ok5KxAFP5RjrEmbiD1kWtaO+fyD6VehPTfq0vEkgQKRBJUB6KFQePk2T7Q8GZtHD+f6ZKYUWXke5avDMzRRZhd0W8Pzru0uJ1u/gu6TyWSJjEdtR81iqzHsyreK8tTpxr7sU7m3M0QTBLLEWhdnCb1faZ0BZ5+0ONMUkhPi0M3AvQXNddVUK+Pw9yZLIUG1tylrGwKa/dG/KE3QkWSqWutxwzkxvXp8wpZnlanNk/regbIRkHLZo5K07c7R25UAuHJOlSGwdD2ei8sINva20F1gBf1HS9IBj6/Ptea/45Md0DrNT+I3cQuvF0AfVeO9gy7B0PSs6QsZ9+FTTx9DRGSk/H1ZZrlRb6YSXlyMLVCgJO7N3Obmx41Rh/iTYGSp+8CwPLiFjKdApnXrHk117Q5SVImAfvZZM7FUluL/KYvy3POe8gLd2RxiD2OsfiY4UDNtMITWWnJz21rlFn5QteViQHiY9LVHSCkbWX/lywh3L3sgOWrdwJA2BuZCEWILhrACQwXnQKyTNT9KkudQP1Ehk/hxiM2dLdKeTpdzIWGAy1n3wNm6UCFQYmyuUBZWGQkeuxAnTWJKjBWT5lpHib98GzevVA184CpT8cY+RDMPF2AmuPTvk8A08wZiIzLX4GwVRd7sqMvB9+QE+UAkaGSz43codm3K6OXy0CJeZZirkkqzUkBvsUGc5HrEdCLGpaB+z8t4y9jK9lyEc7swwkWye54nGL0Y6WVVZ40qee+gMEDlXpKPZ8YRU7pJHyHqFtmk/vQytUYyYeYEylLR0h7O8WizymHCGsU5xgiKq1KrxJO/zV5CDPCKp6ExctPGulUxmyiEMPun3gMIgCv2E= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: fw8Lv6JQx47UfWAQ3dFRpzCR6vCY64NveMujahonLH/7qk4qwXtQnb390W50epY9Xs/6sifd63OQ/5RWC1yYBwHkgFN8X+ieaWoykT2yx5Qusu2xMXOC2WCCPoKkyEEhlPOD8+VZ/7qlM6cA775O+skywWbdXmCaunPn1exv6eQVZXXmNUQxooUQQgPcMdi4Fyaw9MuWvWEIlmB+VmL+mwi0x5B3tDM9+Hw++jJ7pgudgqIPNRd8GKjl9oHoGx/Nf9aNbdOUldUR0IW8lyiwDebgjfwvWczQls6H4RcRRNYUx/DL39PrQCnwDLspbY05xM/XZnnwDkgTxy139Rdf/3QfSZ+i4Pvf2KG3I1nfPpO1DH4/fTmys9H48VsXEsrW1M6HxPVc4mQDQj5SkAJFxHiuWFceVSqoNkJXxuW7H2RzJlOBomc74JG0bUZHM4BV1U4DIEAvh+STx/t8Gq9NRvYpgHti9P8nuVSeDTa+nFvFnA5ddAifWt6fUOMnC0aDZl8lgTMgyMVVft7mrXBH/rumdDAV7BuSivEC5QXfpyhjjcnk1jkB5dvRVLS2VPsxfCq0wQvaOV7OdlBsoFXpwLdldqYDgxJKqBDXFFqqido= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 042d2f77-0126-4c3d-d2dc-08dd13099979 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4382.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Dec 2024 19:43:30.9158 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0h0ZbCwimqpqcqDJEv1uKdKlO6vTygTAICVLnlbcsKwaHxrecF3DKMwMd7BVlS1ea3JC7LW0ng6kIyQItZqwmBpfk/EaNCNxa2nFVx4Khy4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4813 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_14,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 bulkscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020166 X-Proofpoint-GUID: TosSsWRfn3h0Po3H7msU2jJ9Vp0I64mP X-Proofpoint-ORIG-GUID: TosSsWRfn3h0Po3H7msU2jJ9Vp0I64mP X-Spam-Status: No, score=-13.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org Existing tcache implementation in glibc seems to focus in caching smaller data size allocations, limiting the size of the allocation to 1KB. This patch changes tcache implementation to allow to cache any chunk size allocations. The implementation adds extra bins (linked-lists) which store chunks with different ranges of allocation sizes. Bin selection is done in multiples in powers of 2 and chunks are reversely ordered within the bin. The last bin contains all other sizes of allocations. The patch also includes trimming functionality that gives back to the arena any older non used chunk, once the threshold limit for the tcache size is reached. This patch although by default preserves the same implementation, limitting caches to 1KB chunks, it now allows to increase the max size for the cached chunks with the tunable glibc.malloc.tcache_max. Also in order to define the capacity of the tcache, the tunable glibc.mallc.tcache_max_large_capacity was defined, specifying the refered threshold for removing older non-used cached chunks. --- elf/dl-tunables.list | 3 + malloc/arena.c | 2 + malloc/malloc.c | 383 ++++++++++++++++++++++++++++++++++++------- 3 files changed, 330 insertions(+), 58 deletions(-) diff --git a/elf/dl-tunables.list b/elf/dl-tunables.list index 40ac5b3776..e28d4877fd 100644 --- a/elf/dl-tunables.list +++ b/elf/dl-tunables.list @@ -74,6 +74,9 @@ glibc { tcache_unsorted_limit { type: SIZE_T } + tcache_max_large_capacity { + type: SIZE_T + } mxfast { type: SIZE_T minval: 0 diff --git a/malloc/arena.c b/malloc/arena.c index 91a43ee394..cd88071a08 100644 --- a/malloc/arena.c +++ b/malloc/arena.c @@ -250,6 +250,7 @@ TUNABLE_CALLBACK_FNDECL (set_arena_test, size_t) TUNABLE_CALLBACK_FNDECL (set_tcache_max, size_t) TUNABLE_CALLBACK_FNDECL (set_tcache_count, size_t) TUNABLE_CALLBACK_FNDECL (set_tcache_unsorted_limit, size_t) +TUNABLE_CALLBACK_FNDECL (set_tcache_max_large_capacity, size_t) #endif TUNABLE_CALLBACK_FNDECL (set_mxfast, size_t) TUNABLE_CALLBACK_FNDECL (set_hugetlb, size_t) @@ -309,6 +310,7 @@ ptmalloc_init (void) TUNABLE_GET (tcache_count, size_t, TUNABLE_CALLBACK (set_tcache_count)); TUNABLE_GET (tcache_unsorted_limit, size_t, TUNABLE_CALLBACK (set_tcache_unsorted_limit)); + TUNABLE_GET (tcache_max_large_capacity, size_t, TUNABLE_CALLBACK (set_tcache_max_large_capacity)); # endif TUNABLE_GET (mxfast, size_t, TUNABLE_CALLBACK (set_mxfast)); TUNABLE_GET (hugetlb, size_t, TUNABLE_CALLBACK (set_hugetlb)); diff --git a/malloc/malloc.c b/malloc/malloc.c index 287fa0904d..95d5607236 100644 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -291,14 +291,18 @@ #if USE_TCACHE /* We want 64 entries. This is an arbitrary limit, which tunables can reduce. */ + +/* Last large bin is for alocations beyond 1kB. */ +# define TCACHE_LARGE_BINS 10 # define TCACHE_MAX_BINS 64 -# define MAX_TCACHE_SIZE tidx2usize (TCACHE_MAX_BINS-1) +# define MAX_TCACHE_SIZE (tidx2usize (TCACHE_MAX_BINS-1)) /* Only used to pre-fill the tunables. */ # define tidx2usize(idx) (((size_t) idx) * MALLOC_ALIGNMENT + MINSIZE - SIZE_SZ) /* When "x" is from chunksize(). */ # define csize2tidx(x) (((x) - MINSIZE + MALLOC_ALIGNMENT - 1) / MALLOC_ALIGNMENT) + /* When "x" is a user-provided size. */ # define usize2tidx(x) csize2tidx (request2size (x)) @@ -1891,6 +1895,8 @@ struct malloc_par /* Maximum number of buckets to use. */ size_t tcache_bins; size_t tcache_max_bytes; + /* Maximum tcache capacity for unbound size bins. */ + size_t tcache_max_large_capacity; /* Maximum number of chunks in each bucket. */ size_t tcache_count; /* Maximum number of chunks to remove from the unsorted list, which @@ -1926,8 +1932,9 @@ static struct malloc_par mp_ = , .tcache_count = TCACHE_FILL_COUNT, .tcache_bins = TCACHE_MAX_BINS, - .tcache_max_bytes = tidx2usize (TCACHE_MAX_BINS-1), - .tcache_unsorted_limit = 0 /* No limit. */ + .tcache_max_bytes = MAX_TCACHE_SIZE, + .tcache_unsorted_limit = 0, /* No limit. */ + .tcache_max_large_capacity = TCACHE_FILL_COUNT * MAX_TCACHE_SIZE /* Not enabled by default. */ #endif }; @@ -3110,6 +3117,11 @@ typedef struct tcache_entry struct tcache_entry *next; /* This field exists to detect double frees. */ uintptr_t key; + + /* Entries to allow large data to remove chunks as they exceed max data size. + */ + struct tcache_entry *rm_prev; + struct tcache_entry *rm_next; } tcache_entry; /* There is one of these for each thread, which contains the @@ -3121,6 +3133,12 @@ typedef struct tcache_perthread_struct { uint16_t counts[TCACHE_MAX_BINS]; tcache_entry *entries[TCACHE_MAX_BINS]; + tcache_entry *large_entries[TCACHE_LARGE_BINS]; + size_t large_data_size; + /* Used to remove older entries from tcache when size limit is reached. */ + tcache_entry *large_remove_list; + tcache_entry *last_in_large_remove_list; + bool in_remove_cycle; } tcache_perthread_struct; static __thread bool tcache_shutting_down = false; @@ -3208,6 +3226,153 @@ tcache_next (tcache_entry *e) return (tcache_entry *) REVEAL_PTR (e->next); } +/* Compute large bin index for chunk size. */ +static __always_inline char +large_csize2tidx (size_t x) +{ + char idx = __builtin_clz (tidx2usize (mp_.tcache_bins-1)) - __builtin_clz (x); + return idx < TCACHE_LARGE_BINS ? idx : TCACHE_LARGE_BINS-1; +} + +/* TODO: Do the mangling of all linked-list pointers. + * All linked list pointers were left non-mangled for review/debugging. */ + +static __always_inline void +tcache_trim_add_entry (tcache_entry *e) +{ + /* Add entry to tail of remove list. */ + if(__glibc_unlikely (tcache->large_remove_list == NULL)) + { + tcache->large_remove_list = e; + tcache->last_in_large_remove_list = e; + } + else + { + e->rm_prev = tcache->last_in_large_remove_list; + tcache->last_in_large_remove_list->rm_next = e; + tcache->last_in_large_remove_list = e; + } + + tcache->large_data_size += chunksize(mem2chunk(e)); +} + +static __always_inline void +tcache_trim_remove_entry (tcache_perthread_struct *tc, + tcache_entry *e, bool remove_from_sized_lists) +{ + /* Remove from regular uni-directional sized lists. */ + if (remove_from_sized_lists == true) + { + size_t tc_idx = large_csize2tidx (chunksize (mem2chunk (e))); + tcache_entry **entry = &tc->large_entries[tc_idx]; + while(*entry != NULL && *entry != e) + entry = &(*entry)->next; + + *entry = (*entry)->next; + } + + /* Change the last entry in remove list if entry to be removed is the + * last. */ + if (e == tc->last_in_large_remove_list) + tc->last_in_large_remove_list = e->rm_prev; + + /* Change remotion list entry if the node to remove is the head of the list. + */ + if (e == tc->large_remove_list) + tc->large_remove_list = e->rm_next; + + /* Remove chunk from remotion list */ + if (e->rm_prev != NULL) + e->rm_prev->rm_next = e->rm_next; + if (e->rm_next != NULL) + e->rm_next->rm_prev = e->rm_prev; + + tc->large_data_size -= chunksize(mem2chunk(e)); +} + +static __always_inline void * +tcache_large_get (size_t nb, size_t alignment) +{ + size_t tc_idx = large_csize2tidx (nb); + tcache_entry **entry = &tcache->large_entries[tc_idx]; + + /* Skip all chunks that are too large. */ + while(*entry != NULL + && chunksize (mem2chunk (*entry)) > nb) + entry = &(*entry)->next; + + /* Find an entry that respects alignment. */ + if (alignment != 0) + { + while (*entry != NULL + && chunksize (mem2chunk (*entry)) == nb + && !PTR_IS_ALIGNED (*entry, alignment)) + entry = &(*entry)->next; + } + + + /* Make sure we have an compatible chunk. */ + if (*entry == NULL + || chunksize (mem2chunk (*entry)) != nb) + return NULL; + + tcache_entry *e = *entry; + + e->key = 0; + tcache_trim_remove_entry (tcache, e, false); + + /* Remove chunk from list */ + *entry = (*entry)->next; + return (void *) e; +} + +static __always_inline void +tcache_trim (size_t extra_space) +{ + tcache->in_remove_cycle = true; + if (tcache != NULL + && tcache->large_remove_list != NULL) + while (tcache->large_remove_list != NULL + && tcache->large_data_size + extra_space > mp_.tcache_max_large_capacity) + { + tcache_entry *e = tcache->large_remove_list; + tcache_trim_remove_entry (tcache, e, true); + if (e == NULL) + malloc_printerr ("tcache_trim: Failed to removed entry from cache"); + else + __libc_free (e); + } + tcache->in_remove_cycle = false; +} + +static __always_inline bool +tcache_large_put (mchunkptr chunk, size_t tc_idx) +{ + if (tcache == NULL) + return false; + + tcache_trim(chunksize (chunk)); + + tcache_entry *e = (tcache_entry *) chunk2mem (chunk); + + /* Mark this chunk as "in the tcache" so the test in _int_free will + detect a double free. */ + memset (e, 0, sizeof(tcache_entry)); + e->key = tcache_key; + + /* If it is a large data chunk then add it in the list ordered from + * bigger to smaller chunk size */ + tcache_entry **entry = &tcache->large_entries[tc_idx]; + while(*entry != NULL && chunksize(mem2chunk(*entry)) > chunksize(chunk)) + entry = &(*entry)->next; + + e->next = *entry; + *entry = e; + + tcache_trim_add_entry (e); + return true; +} + /* Verify if the suspicious tcache_entry is double free. It's not expected to execute very often, mark it as noinline. */ static __attribute__ ((noinline)) void @@ -3216,18 +3381,46 @@ tcache_double_free_verify (tcache_entry *e, size_t tc_idx) tcache_entry *tmp; size_t cnt = 0; LIBC_PROBE (memory_tcache_double_free, 2, e, tc_idx); - for (tmp = tcache->entries[tc_idx]; - tmp; - tmp = REVEAL_PTR (tmp->next), ++cnt) + if(tc_idx < mp_.tcache_bins) + { + for (tmp = tcache->entries[tc_idx]; + tmp; + tmp = REVEAL_PTR (tmp->next), ++cnt) + { + if (cnt >= mp_.tcache_count) + malloc_printerr ("free(): too many chunks detected in tcache"); + if (__glibc_unlikely (!aligned_OK (tmp))) + malloc_printerr ("free(): unaligned chunk detected in tcache 2"); + if (tmp == e) + malloc_printerr ("free(): double free detected in tcache 2"); + /* If we get here, it was a coincidence. We've wasted a + few cycles, but don't abort. */ + } + } + else { - if (cnt >= mp_.tcache_count) - malloc_printerr ("free(): too many chunks detected in tcache"); - if (__glibc_unlikely (!aligned_OK (tmp))) - malloc_printerr ("free(): unaligned chunk detected in tcache 2"); - if (tmp == e) - malloc_printerr ("free(): double free detected in tcache 2"); - /* If we get here, it was a coincidence. We've wasted a - few cycles, but don't abort. */ + size_t large_tcache_size = 0; + for (int idx = 0; idx < TCACHE_LARGE_BINS; idx++) + { + tcache_entry **entry = &tcache->large_entries[idx]; + while (*entry != NULL) + { + size_t entry_size = chunksize (mem2chunk (*entry)); + size_t entry_tc_idx = large_csize2tidx (entry_size); + large_tcache_size += entry_size; + + if (__glibc_unlikely (entry_tc_idx != idx)) + malloc_printerr ("free(): large cache entry in incorrect bin"); + if (__glibc_unlikely (!aligned_OK (*entry))) + malloc_printerr ("free(): unaligned chunk detected in tcache 2"); + if (*entry == e) + malloc_printerr ("free(): double free detected in tcache 3"); + + entry = &((*entry)->next); + } + } + if (tcache->large_data_size != large_tcache_size) + malloc_printerr ("free(): large tcache size is not correct"); } } @@ -3238,22 +3431,35 @@ tcache_free (mchunkptr p, INTERNAL_SIZE_T size) { bool done = false; size_t tc_idx = csize2tidx (size); - if (tcache != NULL && tc_idx < mp_.tcache_bins) + + if (__glibc_unlikely (tcache_shutting_down == false) + && __glibc_unlikely (tcache->in_remove_cycle == false) + && tcache != NULL + && size <= mp_.tcache_max_bytes) { /* Check to see if it's already in the tcache. */ tcache_entry *e = (tcache_entry *) chunk2mem (p); /* This test succeeds on double free. However, we don't 100% - trust it (it also matches random payload data at a 1 in - 2^ chance), so verify it's not an unlikely - coincidence before aborting. */ + trust it (it also matches random payload data at a 1 in + 2^ chance), so verify it's not an unlikely + coincidence before aborting. */ if (__glibc_unlikely (e->key == tcache_key)) - tcache_double_free_verify (e, tc_idx); + tcache_double_free_verify (e, tc_idx); - if (tcache->counts[tc_idx] < mp_.tcache_count) + if (tcache != NULL && tc_idx < mp_.tcache_bins) + { + if (tcache->counts[tc_idx] < mp_.tcache_count) + { + tcache_put (p, tc_idx); + done = true; + } + } + else { - tcache_put (p, tc_idx); - done = true; + size_t ltc_idx = large_csize2tidx (size); + if (tcache_large_put (p, ltc_idx) == true) + done = true; } } return done; @@ -3264,6 +3470,7 @@ tcache_thread_shutdown (void) { int i; tcache_perthread_struct *tcache_tmp = tcache; + size_t large_tcache_size = 0; tcache_shutting_down = true; @@ -3275,7 +3482,7 @@ tcache_thread_shutdown (void) /* Free all of the entries and the tcache itself back to the arena heap for coalescing. */ - for (i = 0; i < TCACHE_MAX_BINS; ++i) + for (i = 0; i < mp_.tcache_bins; ++i) { while (tcache_tmp->entries[i]) { @@ -3288,6 +3495,32 @@ tcache_thread_shutdown (void) } } + size_t initial_tcache_large_size = tcache_tmp->large_data_size; + /* Free all of the large entries back to arena heap. */ + for (int idx = 0; idx < TCACHE_LARGE_BINS; idx++) + { + while (tcache_tmp->large_entries[idx]) + { + tcache_entry *e = tcache_tmp->large_entries[idx]; + size_t entry_size = chunksize (mem2chunk (e)); + size_t entry_tc_idx = large_csize2tidx (entry_size); + large_tcache_size += entry_size; + + if (__glibc_unlikely (entry_tc_idx != idx)) + malloc_printerr ("tcache_thread_shutdown(): large cache entry in incorrect bin"); + if (__glibc_unlikely (!aligned_OK (e))) + malloc_printerr ("tcache_thread_shutdown(): unaligned chunk detected in tcache"); + + e->key = 0; + tcache_trim_remove_entry (tcache_tmp, e, true); + + __libc_free (e); + } + } + if (tcache_tmp->large_data_size != 0 + || initial_tcache_large_size != large_tcache_size) + malloc_printerr ("tcache_thread_shutdown(): large tcache was incorrect"); + __libc_free (tcache_tmp); } @@ -3356,22 +3589,34 @@ __libc_malloc (size_t bytes) #if USE_TCACHE /* int_free also calls request2size, be careful to not pad twice. */ size_t tbytes = checked_request2size (bytes); - if (tbytes == 0) + + if (tbytes <= mp_.tcache_max_bytes) { - __set_errno (ENOMEM); - return NULL; - } - size_t tc_idx = csize2tidx (tbytes); + if (tbytes == 0) + { + __set_errno (ENOMEM); + return NULL; + } + size_t tc_idx = csize2tidx (tbytes); - MAYBE_INIT_TCACHE (); + MAYBE_INIT_TCACHE (); - DIAG_PUSH_NEEDS_COMMENT; - if (tc_idx < mp_.tcache_bins - && tcache != NULL - && tcache->counts[tc_idx] > 0) - { - victim = tcache_get (tc_idx); - return tag_new_usable (victim); + DIAG_PUSH_NEEDS_COMMENT; + + if (tc_idx < mp_.tcache_bins) + { + if (tcache != NULL && tcache->counts[tc_idx] > 0) + { + victim = tcache_get (tc_idx); + return tag_new_usable (victim); + } + } + else + { + victim = tcache_large_get (tbytes, 0); + if (victim != NULL) + return tag_new_usable (victim); + } } DIAG_POP_NEEDS_COMMENT; #endif @@ -3657,32 +3902,45 @@ _mid_memalign (size_t alignment, size_t bytes, void *address) } #if USE_TCACHE + MAYBE_INIT_TCACHE (); + { size_t tbytes; tbytes = checked_request2size (bytes); if (tbytes == 0) { - __set_errno (ENOMEM); - return NULL; + __set_errno (ENOMEM); + return NULL; } size_t tc_idx = csize2tidx (tbytes); - if (tc_idx < mp_.tcache_bins - && tcache != NULL - && tcache->counts[tc_idx] > 0) + if (tbytes <= mp_.tcache_max_bytes) { - /* The tcache itself isn't encoded, but the chain is. */ - tcache_entry **tep = & tcache->entries[tc_idx]; - tcache_entry *te = *tep; - while (te != NULL && !PTR_IS_ALIGNED (te, alignment)) + if (tc_idx < mp_.tcache_bins) { - tep = & (te->next); - te = tcache_next (te); + if (tcache != NULL + && tcache->counts[tc_idx] > 0) + { + /* The tcache itself isn't encoded, but the chain is. */ + tcache_entry **tep = & tcache->entries[tc_idx]; + tcache_entry *te = *tep; + while (te != NULL && !PTR_IS_ALIGNED (te, alignment)) + { + tep = & (te->next); + te = tcache_next (te); + } + if (te != NULL) + { + void *victim = tcache_get_n (tc_idx, tep); + return tag_new_usable (victim); + } + } } - if (te != NULL) + else { - void *victim = tcache_get_n (tc_idx, tep); - return tag_new_usable (victim); + void *victim = tcache_large_get (tbytes, alignment); + if (victim != NULL) + return tag_new_usable (victim); } } } @@ -5544,14 +5802,23 @@ do_set_arena_max (size_t value) static __always_inline int do_set_tcache_max (size_t value) { - if (value <= MAX_TCACHE_SIZE) - { - LIBC_PROBE (memory_tunable_tcache_max_bytes, 2, value, mp_.tcache_max_bytes); - mp_.tcache_max_bytes = value; - mp_.tcache_bins = csize2tidx (request2size(value)) + 1; - return 1; - } - return 0; + LIBC_PROBE (memory_tunable_tcache_max_bytes, 2, value, mp_.tcache_max_bytes); + mp_.tcache_max_bytes = value; + if (csize2tidx (request2size(value)) + 1 < MAX_TCACHE_SIZE) + mp_.tcache_bins = csize2tidx (request2size(value)) + 1; + else + mp_.tcache_bins = MAX_TCACHE_SIZE; + + return 1; +} + +static __always_inline int +do_set_tcache_max_large_capacity (size_t value) +{ + LIBC_PROBE (memory_tunable_tcache_max_large_capacity, 2, value, mp_.tcache_max_large_capacity); + mp_.tcache_max_large_capacity = value; + + return 1; } static __always_inline int From patchwork Mon Dec 2 19:42:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cupertino Miranda X-Patchwork-Id: 102268 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 41F1E385840A for ; Mon, 2 Dec 2024 19:44:47 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 41F1E385840A Authentication-Results: sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-11-20 header.b=fGHn7t3d; dkim=pass (1024-bit key, unprotected) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.a=rsa-sha256 header.s=selector2-oracle-onmicrosoft-com header.b=hQp4T9VC X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by sourceware.org (Postfix) with ESMTPS id 4D03E3858414 for ; Mon, 2 Dec 2024 19:43:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4D03E3858414 Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=oracle.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 4D03E3858414 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1733168618; cv=pass; b=km2y1Y6+hPKINdc3skzl8No2UpGsbUQQGsEw8ZXACXn952aO9ddznwWgOrNd9s7Sf+cdVZXNkbFdT2NrHMB7NvSf9O/ahFY/gKtligvAaIGy8g8vIxuljZHUmr1UnfYBw/ZCHpJxfXZu89IT9mJqQfqOWhKGHucJMSaBKQ4URak= ARC-Message-Signature: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1733168618; c=relaxed/simple; bh=/azO5w/Ywk9nkuZClhtX/dfkAThsIW4lYBuwCVituUg=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-Id: MIME-Version; b=ZsFlUg7uZRTtfJOIzt9C6dbngU9VZFfS9D5xvoK5kG5NB85lxIU3Zod9Oc+tqnGt24Fc1hVWEHHDlJDimCHEwdh04wjQg8A0s7QMmotJpYZpuO+J9WKNymxu99M5miC9zImIhO/I6nC3LpLV05TI8X+T0gOxFYgR1587L2jeX7Q= ARC-Authentication-Results: i=2; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 4D03E3858414 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2Hfbcv029545 for ; Mon, 2 Dec 2024 19:43:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=33oPJ1niOHtM1QC8zbWQq8TrYhrjsSUAXHhiQH/E7rA=; b= fGHn7t3dQ/b3qlAOuMHV+YM+f3yIwB6zsbXeF0JZanRO1VO3deQ3S5YZTrLZVhf9 hvoXkrI5V3jNP8rL0lAvyxmbfszIvKkceqx1J2XnA3eckVUKz7NPMw0ClAaVLzqF YENN7B2y8v8nayX6siCU4tzLyDH3nBdpweehpfKouaRCvfs4i/Ah2OihN58fcvtn uq16IuneUG4v0DSM1bc6MflU8H93pmT48dTaiqvhqFCADmp78XMG2Kp3J0HcmpLT uCfoPYr92JfZ66INlIFsgvt4pGtW6HiaHwKtIGbGaI5FmgvAxRPHjwWA+aC5rA6j I91Ph1C8mWO6sqxE8mosFg== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437trbmj06-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 02 Dec 2024 19:43:37 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B2IeqJL037321 for ; Mon, 2 Dec 2024 19:43:37 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2173.outbound.protection.outlook.com [104.47.58.173]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 437s57dpgv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 02 Dec 2024 19:43:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CSZ2v+dNmcFnybYVghIcpa7fWquyPihuJwaYKWir+A8SDxmLPgaLcnW+3gD38EcRI+5VVDND1iPkv0SvKL22W9V6GHhoXhD4ryYkKH3x6j8E7Sh90yrjiuqk1D3H0vXVppJoBCW0aUiBAtKecSpgrqVvDJ2IK1Jx+H5OMiRN1JGTA6bxSjkqP1pU1sXitfaVjeMwMy6cA9DPJMlEcgGHlHpPnkbiJsAyiJCvt1ugROS2IK5fhdPw8t9L33SKpMNtY+kVMuDnIO1audvANLJdE5+r31dI2nMTOKeKyYY5mg60mzTW9gVRQKBI/Kzs7ZkTkJH7xpCYY/it7wUzNMdCkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=33oPJ1niOHtM1QC8zbWQq8TrYhrjsSUAXHhiQH/E7rA=; b=b7TYULuzNnDvOnjRiNvB22uxwhlno5OZf+3aMeSWU3PXD+0CczfPn6b7ueVHr/ARSIprotiLj4vskU+o9NzL+BXAKDyaEIjFY0Nn0txnwsK6dp3UyIiDFnq65mFlyasrnH35wZQl087P2A7yATbgstZqtXNDDtkft+Pig5M/lFFtjVWmo5v+C646Ib0cYNflUfe1w1i9J3r06AenRkipEuOKeh7cY1q9/u770dju9lREUQc7ycYuRZbMmSCwqxAnX2JPOwj4oJfrG6Q1JplTgLTMoRmdbMdixPBzrlyPHQu4wHEHSN0gP1WMgxUJhAZdyCUAz/Bo3rF8EeBhbSL9CQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=33oPJ1niOHtM1QC8zbWQq8TrYhrjsSUAXHhiQH/E7rA=; b=hQp4T9VCJP1Jf2XEMrxMigYYxcWo1qThFJ0MEyuAikou4JvAfhnW3ZIOriIX44ySlDr7YfeN3QRXEC96kNbHfGGeEMqJgR6TAvT/PB/TXicfWMn5IzIkMsAop/3xZl4KYPm61CccXr0K2hhwuFDQh2n78A335WsUTlFZC52YMuU= Received: from MN2PR10MB4382.namprd10.prod.outlook.com (2603:10b6:208:1d7::13) by SJ0PR10MB4813.namprd10.prod.outlook.com (2603:10b6:a03:2d2::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.18; Mon, 2 Dec 2024 19:43:34 +0000 Received: from MN2PR10MB4382.namprd10.prod.outlook.com ([fe80::5033:84a3:f348:fefb]) by MN2PR10MB4382.namprd10.prod.outlook.com ([fe80::5033:84a3:f348:fefb%7]) with mapi id 15.20.8207.017; Mon, 2 Dec 2024 19:43:34 +0000 From: Cupertino Miranda To: libc-alpha@sourceware.org Cc: jose.marchesi@oracle.com, elena.zannoni@oracle.com, Cupertino Miranda Subject: [PATCH 2/2] malloc: add testing for large tcache support. Date: Mon, 2 Dec 2024 19:42:49 +0000 Message-Id: <20241202194249.95317-3-cupertino.miranda@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241202194249.95317-1-cupertino.miranda@oracle.com> References: <20241202194249.95317-1-cupertino.miranda@oracle.com> X-ClientProxiedBy: LO2P265CA0473.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a2::29) To MN2PR10MB4382.namprd10.prod.outlook.com (2603:10b6:208:1d7::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR10MB4382:EE_|SJ0PR10MB4813:EE_ X-MS-Office365-Filtering-Correlation-Id: e230fc5a-f9cb-4cf7-1186-08dd13099bc1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|366016|1800799024|10070799003|376014; X-Microsoft-Antispam-Message-Info: 4UmFrc9XDlqpyYMSVIiE4hmRu3uxRiDnJWuM8KRLxUpQWSix2oKlQJl6I35FRxGnKkpaciCJkXKIgUEUZh+wGRc2iB2JjkZfmvG5HzrszPTxElryhNYrBIOrFP3jV+u79oHAkqAXL6vtfMWk/X7sxnFsb0KkvOwlC26Iqqjf+v0MxZKHNFWUeZLPEnB6QiJRCxQ65JWEl//x4+hEihFeqq9aBYbEYsTNbn1u8w57ebFY3sbd88tJfYspEChMd0/Yc5BsZzTJCIQz9IKcLXLdJ9SSG7CBCF9BZ/6kATkOJNLqQiYzAbo+VPtCWbOGvt8ge1jaGAUyKt+ujSW2Bv2vjEXSheUOXb3dLrg2cPz1XSaZLfd48LMJ1Kzx+7OV84cdJwohv6+UI/WQCeaxVhebPxjEWbaN8ZNVNWge2VvJcVmfh0Zdvpg5khmILgw+hPUE+EyxdMhMPLaSRAlzJ5n5gjCuuhRTGh8zoVPPwcB3FMaYiXn1AP1DEXRKBs1yybY+CtP7tf4ROtUdHt7TgY9KR7iAEFH2JASMA/K9NMqkBdRGC9ZdumPjVOCErJojf/+OtbaRCNQwryUtpvoU8dJU2z0O11Q1awd7YHnO8o61+D44Iz6ZLeOpuGad1N+BEj3+3eIb0mC4JzK7fZ+gBlpvbzD+sxopWcvsuk5LpdQkpmLkcVfSXnJNzdTL9TMfu2meZ99LKFyaDWnb8a+omAPalTxsfA+gjjkH85LQj93DOCMAr9ifSEYt3Qdb34XrgcyNbt3yQK2OPD5Y6zEVK0Gnwgxl+e/PVqDw8dcouoWPwwJ7vJ1q0Yu2c10z16U3f5dLrM4PwvYk4WNX3lW6dB9c+Cv19ocuQhVVSzpopEFfT6aTQjFoMg6s4rxrbYnBUp8jV8z0BZvNgmvLsnxKPtMBmGJZCl9l/mPilcxGCC7yVlABskRZzbEGMWBcxcZLgjd70IFZH17ktNeMTmkk9IDm9QHmx9BPEd+paJb7GCeBoTmHwyrs0TctDKu9tMMxvTysePXCHfW4SxlbsaGWRlvWervh6d+e1rQ6p5VkwQdGWyzsF01Sj6VQfDGTDN3JugY6U5p/+h5+YskagYG3Vz6iYl9AauqgFkT3EXAJ+46DLPr5CQDw+WsGy4YbRmHXnt5Sg5Ud/D3dteaK3jBRbnwCQ/BywkmHuHzhXjbnyZyODdC8dZ2OoPntknPuP6JtR7PcpaAtsXg9R92fkeQee8Pzzym4KYR9zw8McaArkesVlj6d3iuSTWWFF3OheGal1qL+wo7mR5kj2owFM4E90Uy/CWlObc67o2NVH3i4xnCKOh93LbakLvQrMj32RceYZP4U96MhCh+1iV3K9vlnn+xYEQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4382.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(1800799024)(10070799003)(376014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: No2pyvW4dy6+aSnrRvcE+dixDvtin7DwqeNXdyLvHH52FDG8zNv88N8pOna2LZRxPKFPXYL+XLcR/u0+BCp3rCwQN6zFW0KpiPgyx+uT+jSJxAJFHbNAclWbJKqX+5uGjtCSzsOcz0lZ4FXpxJKHWzch94cqOX0zjyF6Jlu93x4CFnh7QaFcj1N776YVu0ykEoKSCLYfvr/mBw2Pn9nvc45Iwotd/X0XIEEbvxlfOV4P+GbBAjpMc7+LQw8tZBuCfMfBMNaAifhaauvrFJmpqa82MrgjKvRf7f8eDG4ooh8pAnnGFj3EpuzJFHB70GxAEDAtaVCNznbEKZDh48+OpaKHPapceWTvS3J1nswdwXpNjUgCdtjZicN30XnhmRQM1DnMAQqTtkNOBWnBB74Qf5K/rhpovligNh9LYcO0v6xLzeoigiHpxKhtTqSuM64UeIVlRwu4ojiqsvEBOHBUCBBYwABBZFab9l2gcTWTMZFKidhqhd+JHRVmdPg69BFSFPWZKyu3fcAc6ha0Y3gUCijl0Z0nY/d2AzjTC4DMUZgHguiE5YNzmIJsvSPK32WWfevpOdVFpwxxQD+rHnE+fzJDjthpSnTNze7Wa70E3qZkxsu0ek5Kazk31dqNXaHVG+b5OLdpLUgqfRJmunCR7HnpQtZpmjAnSOPKW3EKBpGdRDa1MHtQ/pFAdseFx3mDv6ZikkkBGpJ/OIukfMAbkuxQbJy1FrF5mQl+fcgNXtl3f1IbbulKvfKPWmJAuY5O3QXYXBlPkaRPwws458VQ9XQfdrGYkQxukytfEYZVdzbAMQhtbDih+u+YOS/1q1Fj09AoqI1DBglRBN+v24jKa1ND10K3rSYFY5CgLLnU0tzwsPAtyll1TdRrmlPa8YZuA7qGBPTofqZgJHKoGocWNwg50lzZABpxAxjecna8x7rQBcsaDdl0N0ghnUJN26VDy0nJ5+RLxPOWxUAFJruvQF17wu4Jxl5OJwiTnHPBq84Af7q3Drejcev0JUR4GbXousQEkn+zSNeg2sL/UDUsar+8JpTv84WlI7mVrnytkd6YxI1gcQ+VaLi9lvFBfXZ8aoaW5+zaSioBge2Wj6mZjiqumbc9Rw6MESF23Z/spSTxO7e9GlaKCtA1emUOnq0jXnyekVs1BXseLq0APCw/8EQSTqmD2ZrNfpCmHLCoRu2p77xv8+/uVzbz0Ohwjabq44uOjhMDxI5Qjqm/Dq31EabWJa9kgFNQiVgO7jQlr1Bnz0F4UrF8QSGGJmluoy0NZmPWOxZklHK4lJdFkt6dla8D4croso6SIL4oGEwNseeWCaOwVGfjL2fWUQ1ofHaRnwH+4585xrxxBa/wm/305qwmhLFuxNayl2hH2vKAN4OXj3JQHC4fymTU7mfr654k6K0TIr0EyVaDdDMK3vXDzyI041NFolb1JrE8covT3GRPrgsbhykLMDiI93OLrf+zpdLrktvqnlzBT0oAPt1Qlnf5fVkSoRzZjM1Dq3YcEBFPSb5TzRU7dK8Ygf+3xmDBgw4/U5AyEEAPpBI8tOCr42W3tG3cui+Qh9hK864X88WnvPRUIlADEQnejJ8eyE8nBqQbiHP01mklzLkt4AQ5vDLHwiWnkXu8Zjk4FXfXLJo= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: /L4xeixrlIg94p94SCjEfOGmaq27Ko2vbpyJ2JePrsuteXrXYxtv4xohg2rVEOq58KyWFsqw9BDzEt1edP8NEsWCUVz2YT9T6OlEyL/zR3oPUXibjcKhtqzoTwggjv5FHrZ1FmfM0P6Ui6dBf3cHcQ69rACOa41ONLKvOapdtjXu6dxgPMsmDafKeTV0mbqCT6mpGHn2Z3fQIERKcRxTY8b2Pl7kyPfLUgZao4WWxeC3ODq+Zd1vGuvH3a0xbuYH6NPwHxVTb4LtPseAzQ+HN9mbdXzTTgvSaXZVsdSUjA39NQDNdIiHPLV/iTVwBrjPZH1IvquKRMauJIc9vpQo/9OACXw7Ii/Y5HNeXQjnR4b9mgju6tzMFdgBdzPX6na0CGa38TG+nDdZDad+JtLYaQRhlIL4iydYr3UT059MCdtWjEdkaCt96TQNLToqexxQ6vqyeeFKRQumzFVMlK9E46Spvnqx3L/cnecDSNLmCcU9l5MHj8gC/FPa0PMCljR3dBpNix7OC4bjqc5rNCHh1VLhrErF+8WWs61XewP+U8chzfFdpVON8AmTOY1HzzoR+i3efsipYRogSLiLWF2WZ7FWN42t9iex64c3Ko+H9og= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e230fc5a-f9cb-4cf7-1186-08dd13099bc1 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4382.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Dec 2024 19:43:34.6976 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 72kRUR4daZN4yHI+5CZL4zWSdthVpL2U7WqxYI1b7gPGAKxxh0UjECNAypQwgv12KSKAX4Y51L8DJZ6/MqsDiLpTArD17M1N/276lWjKfQ8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4813 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-02_14,2024-12-02_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 bulkscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412020166 X-Proofpoint-GUID: Z0t1JY8k0x5E95s_q-HQuF8awoQ1R-Vd X-Proofpoint-ORIG-GUID: Z0t1JY8k0x5E95s_q-HQuF8awoQ1R-Vd X-Spam-Status: No, score=-13.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org This patch adds large tcache support tests by re-executing all of the malloc tests, using the tunables: glibc.malloc.tcache_max=1048576 glibc.mallc.tcache_max_large_capacity=4194304 Test names are prefixed with "largetcache". --- Rules | 19 +++++++++++++++++++ malloc/Makefile | 11 +++++++++++ 2 files changed, 30 insertions(+) diff --git a/Rules b/Rules index 713c225d2e..9cccdd2160 100644 --- a/Rules +++ b/Rules @@ -151,6 +151,7 @@ tests: $(addprefix $(objpfx),$(filter-out $(tests-unsupported), \ $(tests-malloc-check:%=%-malloc-check) \ $(tests-malloc-hugetlb1:%=%-malloc-hugetlb1) \ $(tests-malloc-hugetlb2:%=%-malloc-hugetlb2)) \ + $(tests-malloc-largetcache:%=%-malloc-largetcache)) \ $(test-srcs)) $(tests-special) \ $(tests-printers-programs) xtests: tests $(xtests-special) @@ -163,6 +164,7 @@ tests: $(tests:%=$(objpfx)%.out) $(tests-internal:%=$(objpfx)%.out) \ $(tests-malloc-check:%=$(objpfx)%-malloc-check.out) \ $(tests-malloc-hugetlb1:%=$(objpfx)%-malloc-hugetlb1.out) \ $(tests-malloc-hugetlb2:%=$(objpfx)%-malloc-hugetlb2.out) \ + $(tests-malloc-largetcache:%=$(objpfx)%-malloc-largetcache.out) \ $(tests-special) $(tests-printers-out) xtests: tests $(xtests:%=$(objpfx)%.out) $(xtests-special) endif # $(run-built-tests) != no @@ -177,6 +179,7 @@ tests-expected = $(tests) $(tests-internal) $(tests-printers) \ $(tests-container) $(tests-malloc-check:%=%-malloc-check) \ $(tests-malloc-hugetlb1:%=%-malloc-hugetlb1) \ $(tests-malloc-hugetlb2:%=%-malloc-hugetlb2) \ + $(tests-malloc-largetcache:%=%-malloc-largetcache) \ $(tests-mcheck:%=%-mcheck) xtests-expected = $(xtests) endif # $(run-built-tests) != no @@ -209,6 +212,7 @@ binaries-mcheck-tests = $(tests-mcheck:%=%-mcheck) binaries-malloc-check-tests = $(tests-malloc-check:%=%-malloc-check) binaries-malloc-hugetlb1-tests = $(tests-malloc-hugetlb1:%=%-malloc-hugetlb1) binaries-malloc-hugetlb2-tests = $(tests-malloc-hugetlb2:%=%-malloc-hugetlb2) +binaries-malloc-largetcache-tests = $(tests-malloc-largetcache:%=%-malloc-largetcache) else binaries-all-notests = binaries-all-tests = $(tests) $(tests-internal) $(xtests) $(test-srcs) @@ -222,6 +226,7 @@ binaries-mcheck-tests = binaries-malloc-check-tests = binaries-malloc-hugetlb1-tests = binaries-malloc-hugetlb2-tests = +binaries-malloc-largetcache-tests = endif binaries-pie = $(binaries-pie-tests) $(binaries-pie-notests) @@ -288,6 +293,14 @@ $(addprefix $(objpfx),$(binaries-malloc-hugetlb2-tests)): %-malloc-hugetlb2: %.o $(+link-tests) endif +ifneq "$(strip $(binaries-malloc-largetcache-tests))" "" +$(addprefix $(objpfx),$(binaries-malloc-largetcache-tests)): %-malloc-largetcache: %.o \ + $(link-extra-libs-tests) \ + $(sort $(filter $(common-objpfx)lib%,$(link-libc))) \ + $(addprefix $(csu-objpfx),start.o) $(+preinit) $(+postinit) + $(+link-tests) +endif + ifneq "$(strip $(binaries-pie-tests))" "" $(addprefix $(objpfx),$(binaries-pie-tests)): %: %.o \ $(link-extra-libs-tests) \ @@ -337,6 +350,12 @@ $(1)-malloc-hugetlb2-ENV += GLIBC_TUNABLES=glibc.malloc.hugetlb=2 endef $(foreach t,$(tests-malloc-hugetlb2),$(eval $(call malloc-hugetlb2-ENVS,$(t)))) +# All malloc-largetcache tests will be run with GLIBC_TUNABLE=glibc.malloc.tcache_max=1048576:glibc.mallc.tcache_max_large_capacity=4194304 +define malloc-largetcache-ENVS +$(1)-malloc-largetcache-ENV += GLIBC_TUNABLES=glibc.malloc.tcache_max=1048576:glibc.mallc.tcache_max_large_capacity=4194304 +endef +$(foreach t,$(tests-malloc-largetcache),$(eval $(call malloc-largetcache-ENVS,$(t)))) + # mcheck tests need the debug DSO to support -lmcheck. define mcheck-ENVS $(1)-mcheck-ENV = LD_PRELOAD=$(common-objpfx)/malloc/libc_malloc_debug.so diff --git a/malloc/Makefile b/malloc/Makefile index 1630aaf6ac..2df2b215da 100644 --- a/malloc/Makefile +++ b/malloc/Makefile @@ -147,6 +147,17 @@ tests-malloc-hugetlb1 = \ tests-malloc-hugetlb2 = \ $(filter-out $(tests-exclude-hugetlb2), $(tests)) +tests-exclude-largetcache = \ + tst-compathooks-off \ + tst-compathooks-on \ + tst-interpose-thread \ + tst-mallocstate \ + tst-malloc-backtrace \ + tst-malloc-usable-tunables + +tests-malloc-largetcache = \ + $(filter-out $(tests-exclude-largetcache), $(tests)) + # -lmcheck needs __malloc_initialize_hook, which was deprecated in 2.24. ifeq ($(have-GLIBC_2.23)$(build-shared),yesyes) # Tests that don't play well with mcheck. They are either bugs in mcheck or