cve-2024-56559
Vulnerability from cvelistv5
Published
2024-12-27 14:23
Modified
2025-01-20 06:23
Severity ?
Summary
In the Linux kernel, the following vulnerability has been resolved: mm/vmalloc: combine all TLB flush operations of KASAN shadow virtual address into one operation When compiling kernel source 'make -j $(nproc)' with the up-and-running KASAN-enabled kernel on a 256-core machine, the following soft lockup is shown: watchdog: BUG: soft lockup - CPU#28 stuck for 22s! [kworker/28:1:1760] CPU: 28 PID: 1760 Comm: kworker/28:1 Kdump: loaded Not tainted 6.10.0-rc5 #95 Workqueue: events drain_vmap_area_work RIP: 0010:smp_call_function_many_cond+0x1d8/0xbb0 Code: 38 c8 7c 08 84 c9 0f 85 49 08 00 00 8b 45 08 a8 01 74 2e 48 89 f1 49 89 f7 48 c1 e9 03 41 83 e7 07 4c 01 e9 41 83 c7 03 f3 90 <0f> b6 01 41 38 c7 7c 08 84 c0 0f 85 d4 06 00 00 8b 45 08 a8 01 75 RSP: 0018:ffffc9000cb3fb60 EFLAGS: 00000202 RAX: 0000000000000011 RBX: ffff8883bc4469c0 RCX: ffffed10776e9949 RDX: 0000000000000002 RSI: ffff8883bb74ca48 RDI: ffffffff8434dc50 RBP: ffff8883bb74ca40 R08: ffff888103585dc0 R09: ffff8884533a1800 R10: 0000000000000004 R11: ffffffffffffffff R12: ffffed1077888d39 R13: dffffc0000000000 R14: ffffed1077888d38 R15: 0000000000000003 FS: 0000000000000000(0000) GS:ffff8883bc400000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00005577b5c8d158 CR3: 0000000004850000 CR4: 0000000000350ef0 Call Trace: <IRQ> ? watchdog_timer_fn+0x2cd/0x390 ? __pfx_watchdog_timer_fn+0x10/0x10 ? __hrtimer_run_queues+0x300/0x6d0 ? sched_clock_cpu+0x69/0x4e0 ? __pfx___hrtimer_run_queues+0x10/0x10 ? srso_return_thunk+0x5/0x5f ? ktime_get_update_offsets_now+0x7f/0x2a0 ? srso_return_thunk+0x5/0x5f ? srso_return_thunk+0x5/0x5f ? hrtimer_interrupt+0x2ca/0x760 ? __sysvec_apic_timer_interrupt+0x8c/0x2b0 ? sysvec_apic_timer_interrupt+0x6a/0x90 </IRQ> <TASK> ? asm_sysvec_apic_timer_interrupt+0x16/0x20 ? smp_call_function_many_cond+0x1d8/0xbb0 ? __pfx_do_kernel_range_flush+0x10/0x10 on_each_cpu_cond_mask+0x20/0x40 flush_tlb_kernel_range+0x19b/0x250 ? srso_return_thunk+0x5/0x5f ? kasan_release_vmalloc+0xa7/0xc0 purge_vmap_node+0x357/0x820 ? __pfx_purge_vmap_node+0x10/0x10 __purge_vmap_area_lazy+0x5b8/0xa10 drain_vmap_area_work+0x21/0x30 process_one_work+0x661/0x10b0 worker_thread+0x844/0x10e0 ? srso_return_thunk+0x5/0x5f ? __kthread_parkme+0x82/0x140 ? __pfx_worker_thread+0x10/0x10 kthread+0x2a5/0x370 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x30/0x70 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1a/0x30 </TASK> Debugging Analysis: 1. The following ftrace log shows that the lockup CPU spends too much time iterating vmap_nodes and flushing TLB when purging vm_area structures. (Some info is trimmed). kworker: funcgraph_entry: | drain_vmap_area_work() { kworker: funcgraph_entry: | mutex_lock() { kworker: funcgraph_entry: 1.092 us | __cond_resched(); kworker: funcgraph_exit: 3.306 us | } ... ... kworker: funcgraph_entry: | flush_tlb_kernel_range() { ... ... kworker: funcgraph_exit: # 7533.649 us | } ... ... kworker: funcgraph_entry: 2.344 us | mutex_unlock(); kworker: funcgraph_exit: $ 23871554 us | } The drain_vmap_area_work() spends over 23 seconds. There are 2805 flush_tlb_kernel_range() calls in the ftrace log. * One is called in __purge_vmap_area_lazy(). * Others are called by purge_vmap_node->kasan_release_vmalloc. purge_vmap_node() iteratively releases kasan vmalloc allocations and flushes TLB for each vmap_area. - [Rough calculation] Each flush_tlb_kernel_range() runs about 7.5ms. -- 2804 * 7.5ms = 21.03 seconds. -- That's why a soft lock is triggered. 2. Extending the soft lockup time can work around the issue (For example, # echo ---truncated---
References
416baaa9-dc9f-4396-8d5f-8c081fb06d67https://git.kernel.org/stable/c/9e9e085effe9b7e342138fde3cf8577d22509932
416baaa9-dc9f-4396-8d5f-8c081fb06d67https://git.kernel.org/stable/c/f9a18889aad9b4c19c6c4550c67ad4f9ed2a354f
Impacted products
Vendor Product Version
Linux Linux Version: 282631cb2447318e2a55b41a665dbe8571c46d70
Version: 282631cb2447318e2a55b41a665dbe8571c46d70
 Create a notification for this product.
   Linux Linux Version: 6.9
 Create a notification for this product.
Show details on NVD website

To clipboard 

{
  "containers": {
    "cna": {
      "affected": [
        {
          "defaultStatus": "unaffected",
          "product": "Linux",
          "programFiles": [
            "include/linux/kasan.h",
            "mm/kasan/shadow.c",
            "mm/vmalloc.c"
          ],
          "repo": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git",
          "vendor": "Linux",
          "versions": [
            {
              "lessThan": "f9a18889aad9b4c19c6c4550c67ad4f9ed2a354f",
              "status": "affected",
              "version": "282631cb2447318e2a55b41a665dbe8571c46d70",
              "versionType": "git"
            },
            {
              "lessThan": "9e9e085effe9b7e342138fde3cf8577d22509932",
              "status": "affected",
              "version": "282631cb2447318e2a55b41a665dbe8571c46d70",
              "versionType": "git"
            }
          ]
        },
        {
          "defaultStatus": "affected",
          "product": "Linux",
          "programFiles": [
            "include/linux/kasan.h",
            "mm/kasan/shadow.c",
            "mm/vmalloc.c"
          ],
          "repo": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git",
          "vendor": "Linux",
          "versions": [
            {
              "status": "affected",
              "version": "6.9"
            },
            {
              "lessThan": "6.9",
              "status": "unaffected",
              "version": "0",
              "versionType": "semver"
            },
            {
              "lessThanOrEqual": "6.12.*",
              "status": "unaffected",
              "version": "6.12.4",
              "versionType": "semver"
            },
            {
              "lessThanOrEqual": "*",
              "status": "unaffected",
              "version": "6.13",
              "versionType": "original_commit_for_fix"
            }
          ]
        }
      ],
      "descriptions": [
        {
          "lang": "en",
          "value": "In the Linux kernel, the following vulnerability has been resolved:\n\nmm/vmalloc: combine all TLB flush operations of KASAN shadow virtual address into one operation\n\nWhen compiling kernel source \u0027make -j $(nproc)\u0027 with the up-and-running\nKASAN-enabled kernel on a 256-core machine, the following soft lockup is\nshown:\n\nwatchdog: BUG: soft lockup - CPU#28 stuck for 22s! [kworker/28:1:1760]\nCPU: 28 PID: 1760 Comm: kworker/28:1 Kdump: loaded Not tainted 6.10.0-rc5 #95\nWorkqueue: events drain_vmap_area_work\nRIP: 0010:smp_call_function_many_cond+0x1d8/0xbb0\nCode: 38 c8 7c 08 84 c9 0f 85 49 08 00 00 8b 45 08 a8 01 74 2e 48 89 f1 49 89 f7 48 c1 e9 03 41 83 e7 07 4c 01 e9 41 83 c7 03 f3 90 \u003c0f\u003e b6 01 41 38 c7 7c 08 84 c0 0f 85 d4 06 00 00 8b 45 08 a8 01 75\nRSP: 0018:ffffc9000cb3fb60 EFLAGS: 00000202\nRAX: 0000000000000011 RBX: ffff8883bc4469c0 RCX: ffffed10776e9949\nRDX: 0000000000000002 RSI: ffff8883bb74ca48 RDI: ffffffff8434dc50\nRBP: ffff8883bb74ca40 R08: ffff888103585dc0 R09: ffff8884533a1800\nR10: 0000000000000004 R11: ffffffffffffffff R12: ffffed1077888d39\nR13: dffffc0000000000 R14: ffffed1077888d38 R15: 0000000000000003\nFS:  0000000000000000(0000) GS:ffff8883bc400000(0000) knlGS:0000000000000000\nCS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033\nCR2: 00005577b5c8d158 CR3: 0000000004850000 CR4: 0000000000350ef0\nCall Trace:\n \u003cIRQ\u003e\n ? watchdog_timer_fn+0x2cd/0x390\n ? __pfx_watchdog_timer_fn+0x10/0x10\n ? __hrtimer_run_queues+0x300/0x6d0\n ? sched_clock_cpu+0x69/0x4e0\n ? __pfx___hrtimer_run_queues+0x10/0x10\n ? srso_return_thunk+0x5/0x5f\n ? ktime_get_update_offsets_now+0x7f/0x2a0\n ? srso_return_thunk+0x5/0x5f\n ? srso_return_thunk+0x5/0x5f\n ? hrtimer_interrupt+0x2ca/0x760\n ? __sysvec_apic_timer_interrupt+0x8c/0x2b0\n ? sysvec_apic_timer_interrupt+0x6a/0x90\n \u003c/IRQ\u003e\n \u003cTASK\u003e\n ? asm_sysvec_apic_timer_interrupt+0x16/0x20\n ? smp_call_function_many_cond+0x1d8/0xbb0\n ? __pfx_do_kernel_range_flush+0x10/0x10\n on_each_cpu_cond_mask+0x20/0x40\n flush_tlb_kernel_range+0x19b/0x250\n ? srso_return_thunk+0x5/0x5f\n ? kasan_release_vmalloc+0xa7/0xc0\n purge_vmap_node+0x357/0x820\n ? __pfx_purge_vmap_node+0x10/0x10\n __purge_vmap_area_lazy+0x5b8/0xa10\n drain_vmap_area_work+0x21/0x30\n process_one_work+0x661/0x10b0\n worker_thread+0x844/0x10e0\n ? srso_return_thunk+0x5/0x5f\n ? __kthread_parkme+0x82/0x140\n ? __pfx_worker_thread+0x10/0x10\n kthread+0x2a5/0x370\n ? __pfx_kthread+0x10/0x10\n ret_from_fork+0x30/0x70\n ? __pfx_kthread+0x10/0x10\n ret_from_fork_asm+0x1a/0x30\n \u003c/TASK\u003e\n\nDebugging Analysis:\n\n  1. The following ftrace log shows that the lockup CPU spends too much\n     time iterating vmap_nodes and flushing TLB when purging vm_area\n     structures. (Some info is trimmed).\n\n     kworker: funcgraph_entry:              |  drain_vmap_area_work() {\n     kworker: funcgraph_entry:              |   mutex_lock() {\n     kworker: funcgraph_entry:  1.092 us    |     __cond_resched();\n     kworker: funcgraph_exit:   3.306 us    |   }\n     ...                                        ...\n     kworker: funcgraph_entry:              |    flush_tlb_kernel_range() {\n     ...                                          ...\n     kworker: funcgraph_exit: # 7533.649 us |    }\n     ...                                         ...\n     kworker: funcgraph_entry:  2.344 us    |   mutex_unlock();\n     kworker: funcgraph_exit: $ 23871554 us | }\n\n     The drain_vmap_area_work() spends over 23 seconds.\n\n     There are 2805 flush_tlb_kernel_range() calls in the ftrace log.\n       * One is called in __purge_vmap_area_lazy().\n       * Others are called by purge_vmap_node-\u003ekasan_release_vmalloc.\n         purge_vmap_node() iteratively releases kasan vmalloc\n         allocations and flushes TLB for each vmap_area.\n           - [Rough calculation] Each flush_tlb_kernel_range() runs\n             about 7.5ms.\n               -- 2804 * 7.5ms = 21.03 seconds.\n               -- That\u0027s why a soft lock is triggered.\n\n  2. Extending the soft lockup time can work around the issue (For example,\n     # echo\n---truncated---"
        }
      ],
      "providerMetadata": {
        "dateUpdated": "2025-01-20T06:23:04.424Z",
        "orgId": "416baaa9-dc9f-4396-8d5f-8c081fb06d67",
        "shortName": "Linux"
      },
      "references": [
        {
          "url": "https://git.kernel.org/stable/c/f9a18889aad9b4c19c6c4550c67ad4f9ed2a354f"
        },
        {
          "url": "https://git.kernel.org/stable/c/9e9e085effe9b7e342138fde3cf8577d22509932"
        }
      ],
      "title": "mm/vmalloc: combine all TLB flush operations of KASAN shadow virtual address into one operation",
      "x_generator": {
        "engine": "bippy-5f407fcff5a0"
      }
    }
  },
  "cveMetadata": {
    "assignerOrgId": "416baaa9-dc9f-4396-8d5f-8c081fb06d67",
    "assignerShortName": "Linux",
    "cveId": "CVE-2024-56559",
    "datePublished": "2024-12-27T14:23:04.688Z",
    "dateReserved": "2024-12-27T14:03:05.993Z",
    "dateUpdated": "2025-01-20T06:23:04.424Z",
    "state": "PUBLISHED"
  },
  "dataType": "CVE_RECORD",
  "dataVersion": "5.1",
  "vulnerability-lookup:meta": {
    "nvd": "{\"cve\":{\"id\":\"CVE-2024-56559\",\"sourceIdentifier\":\"416baaa9-dc9f-4396-8d5f-8c081fb06d67\",\"published\":\"2024-12-27T15:15:14.760\",\"lastModified\":\"2024-12-27T15:15:14.760\",\"vulnStatus\":\"Received\",\"cveTags\":[],\"descriptions\":[{\"lang\":\"en\",\"value\":\"In the Linux kernel, the following vulnerability has been resolved:\\n\\nmm/vmalloc: combine all TLB flush operations of KASAN shadow virtual address into one operation\\n\\nWhen compiling kernel source \u0027make -j $(nproc)\u0027 with the up-and-running\\nKASAN-enabled kernel on a 256-core machine, the following soft lockup is\\nshown:\\n\\nwatchdog: BUG: soft lockup - CPU#28 stuck for 22s! [kworker/28:1:1760]\\nCPU: 28 PID: 1760 Comm: kworker/28:1 Kdump: loaded Not tainted 6.10.0-rc5 #95\\nWorkqueue: events drain_vmap_area_work\\nRIP: 0010:smp_call_function_many_cond+0x1d8/0xbb0\\nCode: 38 c8 7c 08 84 c9 0f 85 49 08 00 00 8b 45 08 a8 01 74 2e 48 89 f1 49 89 f7 48 c1 e9 03 41 83 e7 07 4c 01 e9 41 83 c7 03 f3 90 \u003c0f\u003e b6 01 41 38 c7 7c 08 84 c0 0f 85 d4 06 00 00 8b 45 08 a8 01 75\\nRSP: 0018:ffffc9000cb3fb60 EFLAGS: 00000202\\nRAX: 0000000000000011 RBX: ffff8883bc4469c0 RCX: ffffed10776e9949\\nRDX: 0000000000000002 RSI: ffff8883bb74ca48 RDI: ffffffff8434dc50\\nRBP: ffff8883bb74ca40 R08: ffff888103585dc0 R09: ffff8884533a1800\\nR10: 0000000000000004 R11: ffffffffffffffff R12: ffffed1077888d39\\nR13: dffffc0000000000 R14: ffffed1077888d38 R15: 0000000000000003\\nFS:  0000000000000000(0000) GS:ffff8883bc400000(0000) knlGS:0000000000000000\\nCS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033\\nCR2: 00005577b5c8d158 CR3: 0000000004850000 CR4: 0000000000350ef0\\nCall Trace:\\n \u003cIRQ\u003e\\n ? watchdog_timer_fn+0x2cd/0x390\\n ? __pfx_watchdog_timer_fn+0x10/0x10\\n ? __hrtimer_run_queues+0x300/0x6d0\\n ? sched_clock_cpu+0x69/0x4e0\\n ? __pfx___hrtimer_run_queues+0x10/0x10\\n ? srso_return_thunk+0x5/0x5f\\n ? ktime_get_update_offsets_now+0x7f/0x2a0\\n ? srso_return_thunk+0x5/0x5f\\n ? srso_return_thunk+0x5/0x5f\\n ? hrtimer_interrupt+0x2ca/0x760\\n ? __sysvec_apic_timer_interrupt+0x8c/0x2b0\\n ? sysvec_apic_timer_interrupt+0x6a/0x90\\n \u003c/IRQ\u003e\\n \u003cTASK\u003e\\n ? asm_sysvec_apic_timer_interrupt+0x16/0x20\\n ? smp_call_function_many_cond+0x1d8/0xbb0\\n ? __pfx_do_kernel_range_flush+0x10/0x10\\n on_each_cpu_cond_mask+0x20/0x40\\n flush_tlb_kernel_range+0x19b/0x250\\n ? srso_return_thunk+0x5/0x5f\\n ? kasan_release_vmalloc+0xa7/0xc0\\n purge_vmap_node+0x357/0x820\\n ? __pfx_purge_vmap_node+0x10/0x10\\n __purge_vmap_area_lazy+0x5b8/0xa10\\n drain_vmap_area_work+0x21/0x30\\n process_one_work+0x661/0x10b0\\n worker_thread+0x844/0x10e0\\n ? srso_return_thunk+0x5/0x5f\\n ? __kthread_parkme+0x82/0x140\\n ? __pfx_worker_thread+0x10/0x10\\n kthread+0x2a5/0x370\\n ? __pfx_kthread+0x10/0x10\\n ret_from_fork+0x30/0x70\\n ? __pfx_kthread+0x10/0x10\\n ret_from_fork_asm+0x1a/0x30\\n \u003c/TASK\u003e\\n\\nDebugging Analysis:\\n\\n  1. The following ftrace log shows that the lockup CPU spends too much\\n     time iterating vmap_nodes and flushing TLB when purging vm_area\\n     structures. (Some info is trimmed).\\n\\n     kworker: funcgraph_entry:              |  drain_vmap_area_work() {\\n     kworker: funcgraph_entry:              |   mutex_lock() {\\n     kworker: funcgraph_entry:  1.092 us    |     __cond_resched();\\n     kworker: funcgraph_exit:   3.306 us    |   }\\n     ...                                        ...\\n     kworker: funcgraph_entry:              |    flush_tlb_kernel_range() {\\n     ...                                          ...\\n     kworker: funcgraph_exit: # 7533.649 us |    }\\n     ...                                         ...\\n     kworker: funcgraph_entry:  2.344 us    |   mutex_unlock();\\n     kworker: funcgraph_exit: $ 23871554 us | }\\n\\n     The drain_vmap_area_work() spends over 23 seconds.\\n\\n     There are 2805 flush_tlb_kernel_range() calls in the ftrace log.\\n       * One is called in __purge_vmap_area_lazy().\\n       * Others are called by purge_vmap_node-\u003ekasan_release_vmalloc.\\n         purge_vmap_node() iteratively releases kasan vmalloc\\n         allocations and flushes TLB for each vmap_area.\\n           - [Rough calculation] Each flush_tlb_kernel_range() runs\\n             about 7.5ms.\\n               -- 2804 * 7.5ms = 21.03 seconds.\\n               -- That\u0027s why a soft lock is triggered.\\n\\n  2. Extending the soft lockup time can work around the issue (For example,\\n     # echo\\n---truncated---\"}],\"metrics\":{},\"references\":[{\"url\":\"https://git.kernel.org/stable/c/9e9e085effe9b7e342138fde3cf8577d22509932\",\"source\":\"416baaa9-dc9f-4396-8d5f-8c081fb06d67\"},{\"url\":\"https://git.kernel.org/stable/c/f9a18889aad9b4c19c6c4550c67ad4f9ed2a354f\",\"source\":\"416baaa9-dc9f-4396-8d5f-8c081fb06d67\"}]}}"
  }
}


Log in or create an account to share your comment.




Tags
Taxonomy of the tags.




Sightings

Author Source Type Date

Nomenclature

  • Seen: The vulnerability was mentioned, discussed, or seen somewhere by the user.
  • Confirmed: The vulnerability is confirmed from an analyst perspective.
  • Exploited: This vulnerability was exploited and seen by the user reporting the sighting.
  • Patched: This vulnerability was successfully patched by the user reporting the sighting.
  • Not exploited: This vulnerability was not exploited or seen by the user reporting the sighting.
  • Not confirmed: The user expresses doubt about the veracity of the vulnerability.
  • Not patched: This vulnerability was not successfully patched by the user reporting the sighting.