nvidia 580.76.05 kernel-7 patch + docs
This commit is contained in:
alex
commit
883c712c4c
2 changed files with 214 additions and 0 deletions
67
README.md
Normal file
67
README.md
Normal file
|
|
@ -0,0 +1,67 @@
|
|||
# nvidia-kernel-patches
|
||||
|
||||
Source patches that make NVIDIA's driver build against kernels newer than the driver
|
||||
knows about. Currently: **580.76.05 (kernel-open) on 7.0.14-11-pve** (Proxmox VE 9.2.10,
|
||||
host `spicy`). New driver/kernel combos get their own patch + section as they happen.
|
||||
|
||||
Referenced from the house infra repo (trikilli → `hosts/spicy-rtx3090-bringup.md`),
|
||||
which holds the full bring-up narrative (LXC passthrough, DKMS setup, device majors).
|
||||
This repo is just the patch and how to use it.
|
||||
|
||||
## nvidia-580.76.05-kernel7.patch
|
||||
|
||||
Applies to the `kernel-open/` tree extracted from `NVIDIA-Linux-x86_64-580.76.05.run`.
|
||||
Deployed on spicy 2026-08-10; DKMS source lives at `/usr/src/nvidia-580.76.05` on-host.
|
||||
|
||||
### Why the stock driver fails on kernel 7.x
|
||||
|
||||
Three independent causes:
|
||||
|
||||
1. **GCC 14 breaks NVIDIA's conftest.** The "is function X present" probes are
|
||||
*inverted*: they call X with no arguments and treat a successful compile (old
|
||||
compilers: implicit-declaration warning) as "absent". GCC 14 makes implicit
|
||||
declarations hard errors → every probe fails to compile → everything detected
|
||||
"present" → driver calls APIs that don't exist (and builds Tegra-only code on x86).
|
||||
2. **Kernel 7.x headers need `-std=gnu11 -fms-extensions`.** kbuild passes these;
|
||||
conftest didn't. Without `-fms-extensions`, anonymous tagged-struct members
|
||||
(e.g. `struct filename { struct __filename_head; ... }`) silently drop and a
|
||||
`static_assert` on the struct size kills every probe including fs/device/pci headers.
|
||||
3. **Real API changes** — table below.
|
||||
|
||||
### What the patch does
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `conftest.sh` | Add `-std=gnu11 -fms-extensions -Wno-error=implicit-function-declaration -Wno-error=int-conversion -Wno-error=incompatible-pointer-types` to CFLAGS |
|
||||
| `common/inc/nv-time.h`, `nvidia/os-interface.c` | `in_irq()` → `in_hardirq()` (removed) |
|
||||
| `common/inc/nv-mm.h` | `vm_flags` writes → `vm_flags_set/clear()` / `vma_flags_set_word/clear_word()` (vm_flags is a const union member now) |
|
||||
| `nvidia/nv-dma.c` | `dma_map_ops.map_resource` → `map_phys` (phys-addr DMA rework) |
|
||||
| `nvidia/nv-pci.c` | `pci_resize_resource()` gained 4th arg `exclude_bars` → pass `0` |
|
||||
| `nvidia-uvm/uvm_pmm_gpu.c` | `dev_pagemap_ops.page_free(page)` → `.folio_free(folio)` — thin wrappers added |
|
||||
| `nvidia-uvm/uvm_hmm.c` | `zone_device_page_init(page)` → `(page, pgmap, order)` signature |
|
||||
|
||||
Use **kernel-open**, not the closed blob: GA102+ is fully supported, and the open
|
||||
modules are `Dual MIT/GPL` so they can call the kernel's GPL-only VMA-lock API —
|
||||
the closed blob reimplements those internals and chases a moving target.
|
||||
Exclude `nvidia-drm` (DRM API churn, headless boxes don't need it) and `nvidia-peermem`.
|
||||
|
||||
### Apply + install
|
||||
|
||||
```sh
|
||||
sh NVIDIA-Linux-x86_64-580.76.05.run --extract-only --target nv580
|
||||
cd nv580 && patch -p1 < nvidia-580.76.05-kernel7.patch # paths are kernel-open/...
|
||||
cp -r kernel-open /usr/src/nvidia-580.76.05 # + dkms.conf: set version,
|
||||
# exclude nvidia-drm/peermem
|
||||
dkms add nvidia/580.76.05 && dkms build nvidia/580.76.05 && dkms install nvidia/580.76.05
|
||||
sh NVIDIA-Linux-x86_64-580.76.05.run --no-kernel-modules --silent --no-x-check # userspace
|
||||
```
|
||||
|
||||
Container userspace must match the host module version **exactly**; install inside
|
||||
each LXC with `--no-kernel-modules`.
|
||||
|
||||
### Caveats
|
||||
|
||||
- Every kernel update re-runs the DKMS build against new headers — further API churn
|
||||
means further patching. Verify `nvidia-smi` (host, then containers) after any reboot.
|
||||
- `-Wno-error=incompatible-pointer-types` in conftest is scoped to probe compiles only;
|
||||
the real module build keeps default warnings.
|
||||
147
nvidia-580.76.05-kernel7.patch
Normal file
147
nvidia-580.76.05-kernel7.patch
Normal file
|
|
@ -0,0 +1,147 @@
|
|||
diff -ruN '--exclude=*.log' '--exclude=*.orig' '--exclude=conftest*.c' /tmp/nv-pristine/kernel-open/common/inc/nv-mm.h ./kernel-open/common/inc/nv-mm.h
|
||||
--- /tmp/nv-pristine/kernel-open/common/inc/nv-mm.h 2025-08-07 16:13:32.000000000 -0400
|
||||
+++ ./kernel-open/common/inc/nv-mm.h 2026-08-10 18:38:01.233430548 -0400
|
||||
@@ -202,11 +202,11 @@
|
||||
{
|
||||
#if !NV_CAN_CALL_VMA_START_WRITE
|
||||
nv_vma_start_write(vma);
|
||||
- ACCESS_PRIVATE(vma, __vm_flags) |= flags;
|
||||
+ vma_flags_set_word(&vma->flags, flags);
|
||||
#elif defined(NV_VM_AREA_STRUCT_HAS_CONST_VM_FLAGS)
|
||||
vm_flags_set(vma, flags);
|
||||
#else
|
||||
- vma->vm_flags |= flags;
|
||||
+ vm_flags_set(vma, flags);
|
||||
#endif
|
||||
}
|
||||
|
||||
@@ -214,11 +214,11 @@
|
||||
{
|
||||
#if !NV_CAN_CALL_VMA_START_WRITE
|
||||
nv_vma_start_write(vma);
|
||||
- ACCESS_PRIVATE(vma, __vm_flags) &= ~flags;
|
||||
+ vma_flags_clear_word(&vma->flags, flags);
|
||||
#elif defined(NV_VM_AREA_STRUCT_HAS_CONST_VM_FLAGS)
|
||||
vm_flags_clear(vma, flags);
|
||||
#else
|
||||
- vma->vm_flags &= ~flags;
|
||||
+ vm_flags_clear(vma, flags);
|
||||
#endif
|
||||
}
|
||||
|
||||
diff -ruN '--exclude=*.log' '--exclude=*.orig' '--exclude=conftest*.c' /tmp/nv-pristine/kernel-open/common/inc/nv-time.h ./kernel-open/common/inc/nv-time.h
|
||||
--- /tmp/nv-pristine/kernel-open/common/inc/nv-time.h 2025-08-07 16:13:29.000000000 -0400
|
||||
+++ ./kernel-open/common/inc/nv-time.h 2026-08-10 18:36:21.200197921 -0400
|
||||
@@ -82,7 +82,7 @@
|
||||
ktime_get_raw_ts64(&tm1);
|
||||
#endif
|
||||
|
||||
- if (in_irq() && (us > NV_MAX_ISR_DELAY_US))
|
||||
+ if (in_hardirq() && (us > NV_MAX_ISR_DELAY_US))
|
||||
return NV_ERR_GENERIC;
|
||||
|
||||
mdelay_safe_msec = us / 1000;
|
||||
@@ -127,7 +127,7 @@
|
||||
tm_start = tm_aux;
|
||||
#endif
|
||||
|
||||
- if (in_irq() && (ms > NV_MAX_ISR_DELAY_MS))
|
||||
+ if (in_hardirq() && (ms > NV_MAX_ISR_DELAY_MS))
|
||||
{
|
||||
return NV_ERR_GENERIC;
|
||||
}
|
||||
diff -ruN '--exclude=*.log' '--exclude=*.orig' '--exclude=conftest*.c' /tmp/nv-pristine/kernel-open/conftest.sh ./kernel-open/conftest.sh
|
||||
--- /tmp/nv-pristine/kernel-open/conftest.sh 2025-08-07 15:51:13.000000000 -0400
|
||||
+++ ./kernel-open/conftest.sh 2026-08-10 18:38:01.205430483 -0400
|
||||
@@ -164,7 +164,7 @@
|
||||
MACH_CFLAGS="$MACH_CFLAGS -I$SOURCE_ARCH_HEADERS/asm/mach-default"
|
||||
|
||||
CFLAGS="$BASE_CFLAGS $MACH_CFLAGS $OUTPUT_CFLAGS -include $AUTOCONF_FILE"
|
||||
- CFLAGS="$CFLAGS -I$SOURCE_HEADERS"
|
||||
+ CFLAGS="$CFLAGS -I$SOURCE_HEADERS -std=gnu11 -fms-extensions -Wno-error=implicit-function-declaration -Wno-error=int-conversion -Wno-error=incompatible-pointer-types"
|
||||
CFLAGS="$CFLAGS -I$SOURCE_HEADERS/uapi"
|
||||
CFLAGS="$CFLAGS -I$SOURCE_HEADERS/xen"
|
||||
CFLAGS="$CFLAGS -I$OUTPUT_HEADERS/generated/uapi"
|
||||
diff -ruN '--exclude=*.log' '--exclude=*.orig' '--exclude=conftest*.c' /tmp/nv-pristine/kernel-open/nvidia/nv-dma.c ./kernel-open/nvidia/nv-dma.c
|
||||
--- /tmp/nv-pristine/kernel-open/nvidia/nv-dma.c 2025-08-07 16:13:19.000000000 -0400
|
||||
+++ ./kernel-open/nvidia/nv-dma.c 2026-08-10 18:36:21.214197952 -0400
|
||||
@@ -718,7 +718,7 @@
|
||||
#endif
|
||||
}
|
||||
|
||||
- return (ops->map_resource != NULL);
|
||||
+ return (ops->map_phys != NULL);
|
||||
}
|
||||
|
||||
/* DMA-map a peer device's C2C aperture for peer access. */
|
||||
diff -ruN '--exclude=*.log' '--exclude=*.orig' '--exclude=conftest*.c' /tmp/nv-pristine/kernel-open/nvidia/nv-pci.c ./kernel-open/nvidia/nv-pci.c
|
||||
--- /tmp/nv-pristine/kernel-open/nvidia/nv-pci.c 2025-08-07 16:13:18.000000000 -0400
|
||||
+++ ./kernel-open/nvidia/nv-pci.c 2026-08-10 18:37:07.883305841 -0400
|
||||
@@ -229,7 +229,7 @@
|
||||
|
||||
resize:
|
||||
/* Attempt to resize BAR1 to the largest supported size */
|
||||
- r = pci_resize_resource(pci_dev, NV_GPU_BAR1, requested_size);
|
||||
+ r = pci_resize_resource(pci_dev, NV_GPU_BAR1, requested_size, 0);
|
||||
|
||||
if (r) {
|
||||
if (r == -ENOSPC)
|
||||
diff -ruN '--exclude=*.log' '--exclude=*.orig' '--exclude=conftest*.c' /tmp/nv-pristine/kernel-open/nvidia/os-interface.c ./kernel-open/nvidia/os-interface.c
|
||||
--- /tmp/nv-pristine/kernel-open/nvidia/os-interface.c 2025-08-07 16:13:20.000000000 -0400
|
||||
+++ ./kernel-open/nvidia/os-interface.c 2026-08-10 18:36:21.201197924 -0400
|
||||
@@ -371,7 +371,7 @@
|
||||
|
||||
NvBool NV_API_CALL os_is_isr(void)
|
||||
{
|
||||
- return (in_irq());
|
||||
+ return (in_hardirq());
|
||||
}
|
||||
|
||||
// return TRUE if the caller is the super-user
|
||||
diff -ruN '--exclude=*.log' '--exclude=*.orig' '--exclude=conftest*.c' /tmp/nv-pristine/kernel-open/nvidia-uvm/uvm_hmm.c ./kernel-open/nvidia-uvm/uvm_hmm.c
|
||||
--- /tmp/nv-pristine/kernel-open/nvidia-uvm/uvm_hmm.c 2025-08-07 16:13:56.000000000 -0400
|
||||
+++ ./kernel-open/nvidia-uvm/uvm_hmm.c 2026-08-10 18:37:52.543410234 -0400
|
||||
@@ -2004,7 +2004,7 @@
|
||||
|
||||
hmm_mark_gpu_chunk_referenced(va_block, gpu, gpu_chunk);
|
||||
UVM_ASSERT(!page_count(dpage));
|
||||
- zone_device_page_init(dpage);
|
||||
+ zone_device_page_init(dpage, page_pgmap(dpage), 0);
|
||||
dpage->zone_device_data = gpu_chunk;
|
||||
atomic64_inc(&va_block->hmm.va_space->hmm.allocated_page_count);
|
||||
|
||||
diff -ruN '--exclude=*.log' '--exclude=*.orig' '--exclude=conftest*.c' /tmp/nv-pristine/kernel-open/nvidia-uvm/uvm_pmm_gpu.c ./kernel-open/nvidia-uvm/uvm_pmm_gpu.c
|
||||
--- /tmp/nv-pristine/kernel-open/nvidia-uvm/uvm_pmm_gpu.c 2025-08-07 16:13:54.000000000 -0400
|
||||
+++ ./kernel-open/nvidia-uvm/uvm_pmm_gpu.c 2026-08-10 18:37:52.540410227 -0400
|
||||
@@ -3168,9 +3168,14 @@
|
||||
UVM_ENTRY_RET(devmem_fault(vmf));
|
||||
}
|
||||
|
||||
+static void devmem_folio_free(struct folio *folio)
|
||||
+{
|
||||
+ devmem_page_free(&folio->page);
|
||||
+}
|
||||
+
|
||||
static const struct dev_pagemap_ops uvm_pmm_devmem_ops =
|
||||
{
|
||||
- .page_free = devmem_page_free,
|
||||
+ .folio_free = devmem_folio_free,
|
||||
.migrate_to_ram = devmem_fault_entry,
|
||||
};
|
||||
|
||||
@@ -3325,9 +3330,14 @@
|
||||
nv_kref_put(&p2p_mem->refcount, device_p2p_page_free_wake);
|
||||
}
|
||||
|
||||
+static void device_p2p_folio_free(struct folio *folio)
|
||||
+{
|
||||
+ device_p2p_page_free(&folio->page);
|
||||
+}
|
||||
+
|
||||
static const struct dev_pagemap_ops uvm_device_p2p_pgmap_ops =
|
||||
{
|
||||
- .page_free = device_p2p_page_free,
|
||||
+ .folio_free = device_p2p_folio_free,
|
||||
};
|
||||
|
||||
void uvm_pmm_gpu_device_p2p_init(uvm_gpu_t *gpu)
|
||||
Loading…
Add table
Reference in a new issue