diff --git a/.mailmap b/.mailmap index 43cd2995dbc29ac1ac10fba7e0be76e4aec64116..efd9fa867a8eccd5488aac0c16822b0a2c9f2db4 100644 --- a/.mailmap +++ b/.mailmap @@ -337,10 +337,11 @@ Kalyan Thota Karthikeyan Periyasamy Kathiravan T Kay Sievers -Kees Cook -Kees Cook -Kees Cook -Kees Cook +Kees Cook +Kees Cook +Kees Cook +Kees Cook +Kees Cook Keith Busch Keith Busch Kenneth W Chen diff --git a/Documentation/admin-guide/LSM/tomoyo.rst b/Documentation/admin-guide/LSM/tomoyo.rst index 4bc9c2b4da6f3923db9cea93770e5f5a6da605bb..bdb2c2e2a1b26bf8119406b3404dc582b6322fb3 100644 --- a/Documentation/admin-guide/LSM/tomoyo.rst +++ b/Documentation/admin-guide/LSM/tomoyo.rst @@ -9,8 +9,8 @@ TOMOYO is a name-based MAC extension (LSM module) for the Linux kernel. LiveCD-based tutorials are available at -http://tomoyo.sourceforge.jp/1.8/ubuntu12.04-live.html -http://tomoyo.sourceforge.jp/1.8/centos6-live.html +https://tomoyo.sourceforge.net/1.8/ubuntu12.04-live.html +https://tomoyo.sourceforge.net/1.8/centos6-live.html Though these tutorials use non-LSM version of TOMOYO, they are useful for you to know what TOMOYO is. @@ -21,45 +21,32 @@ How to enable TOMOYO? Build the kernel with ``CONFIG_SECURITY_TOMOYO=y`` and pass ``security=tomoyo`` on kernel's command line. -Please see http://tomoyo.osdn.jp/2.5/ for details. +Please see https://tomoyo.sourceforge.net/2.6/ for details. Where is documentation? ======================= User <-> Kernel interface documentation is available at -https://tomoyo.osdn.jp/2.5/policy-specification/index.html . +https://tomoyo.sourceforge.net/2.6/policy-specification/index.html . Materials we prepared for seminars and symposiums are available at -https://osdn.jp/projects/tomoyo/docs/?category_id=532&language_id=1 . +https://sourceforge.net/projects/tomoyo/files/docs/ . Below lists are chosen from three aspects. What is TOMOYO? TOMOYO Linux Overview - https://osdn.jp/projects/tomoyo/docs/lca2009-takeda.pdf + https://sourceforge.net/projects/tomoyo/files/docs/lca2009-takeda.pdf TOMOYO Linux: pragmatic and manageable security for Linux - https://osdn.jp/projects/tomoyo/docs/freedomhectaipei-tomoyo.pdf + https://sourceforge.net/projects/tomoyo/files/docs/freedomhectaipei-tomoyo.pdf TOMOYO Linux: A Practical Method to Understand and Protect Your Own Linux Box - https://osdn.jp/projects/tomoyo/docs/PacSec2007-en-no-demo.pdf + https://sourceforge.net/projects/tomoyo/files/docs/PacSec2007-en-no-demo.pdf What can TOMOYO do? Deep inside TOMOYO Linux - https://osdn.jp/projects/tomoyo/docs/lca2009-kumaneko.pdf + https://sourceforge.net/projects/tomoyo/files/docs/lca2009-kumaneko.pdf The role of "pathname based access control" in security. - https://osdn.jp/projects/tomoyo/docs/lfj2008-bof.pdf + https://sourceforge.net/projects/tomoyo/files/docs/lfj2008-bof.pdf History of TOMOYO? Realities of Mainlining - https://osdn.jp/projects/tomoyo/docs/lfj2008.pdf - -What is future plan? -==================== - -We believe that inode based security and name based security are complementary -and both should be used together. But unfortunately, so far, we cannot enable -multiple LSM modules at the same time. We feel sorry that you have to give up -SELinux/SMACK/AppArmor etc. when you want to use TOMOYO. - -We hope that LSM becomes stackable in future. Meanwhile, you can use non-LSM -version of TOMOYO, available at http://tomoyo.osdn.jp/1.8/ . -LSM version of TOMOYO is a subset of non-LSM version of TOMOYO. We are planning -to port non-LSM version's functionalities to LSM versions. + https://sourceforge.net/projects/tomoyo/files/docs/lfj2008.pdf diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 555e6b53eb41e304e741132829ac1ab8b0172b06..ebb971a57d04f83c0f81340fad07674a21a49013 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -1921,6 +1921,28 @@ Format: , + i2c_touchscreen_props= [HW,ACPI,X86] + Set device-properties for ACPI-enumerated I2C-attached + touchscreen, to e.g. fix coordinates of upside-down + mounted touchscreens. If you need this option please + submit a drivers/platform/x86/touchscreen_dmi.c patch + adding a DMI quirk for this. + + Format: + :=[:prop_name=val][:...] + Where is one of: + Omit "=" entirely Set a boolean device-property + Unsigned number Set a u32 device-property + Anything else Set a string device-property + + Examples (split over multiple lines): + i2c_touchscreen_props=GDIX1001:touchscreen-inverted-x: + touchscreen-inverted-y + + i2c_touchscreen_props=MSSL1680:touchscreen-size-x=1920: + touchscreen-size-y=1080:touchscreen-inverted-y: + firmware-name=gsl1680-vendor-model.fw:silead,home-button + i8042.debug [HW] Toggle i8042 debug mode i8042.unmask_kbd_data [HW] Enable printing of interrupt data from the KBD port diff --git a/Documentation/arch/riscv/uabi.rst b/Documentation/arch/riscv/uabi.rst index 54d199dce78bf50525b0430dcfc5a6a71429bf68..2b420bab0527a75f09d76c0d7fe9904262830e16 100644 --- a/Documentation/arch/riscv/uabi.rst +++ b/Documentation/arch/riscv/uabi.rst @@ -65,4 +65,6 @@ the extension, or may have deliberately removed it from the listing. Misaligned accesses ------------------- -Misaligned accesses are supported in userspace, but they may perform poorly. +Misaligned scalar accesses are supported in userspace, but they may perform +poorly. Misaligned vector accesses are only supported if the Zicclsm extension +is supported. diff --git a/Documentation/bpf/libbpf/libbpf_overview.rst b/Documentation/bpf/libbpf/libbpf_overview.rst index f36a2d4ffea2b3193b65b76363ec37f9cd616885..f4d22f0c62b0e72a9073d2bd118d391e309dea8e 100644 --- a/Documentation/bpf/libbpf/libbpf_overview.rst +++ b/Documentation/bpf/libbpf/libbpf_overview.rst @@ -219,6 +219,14 @@ compilation and skeleton generation. Using Libbpf-rs will make building user space part of the BPF application easier. Note that the BPF program themselves must still be written in plain C. +libbpf logging +============== + +By default, libbpf logs informational and warning messages to stderr. The +verbosity of these messages can be controlled by setting the environment +variable LIBBPF_LOG_LEVEL to either warn, info, or debug. A custom log +callback can be set using ``libbpf_set_print()``. + Additional Documentation ======================== diff --git a/Documentation/bpf/standardization/abi.rst b/Documentation/bpf/standardization/abi.rst index 0c2e10eeb89a05a10553a0cec5e826a110128a5f..41514137cb7b84924caf2e7aaf0bd3598682c0b7 100644 --- a/Documentation/bpf/standardization/abi.rst +++ b/Documentation/bpf/standardization/abi.rst @@ -23,3 +23,6 @@ The BPF calling convention is defined as: R0 - R5 are scratch registers and BPF programs needs to spill/fill them if necessary across calls. + +The BPF program needs to store the return value into register R0 before doing an +``EXIT``. diff --git a/Documentation/bpf/standardization/instruction-set.rst b/Documentation/bpf/standardization/instruction-set.rst index 00c93eb42613ed924a24c41bde2a2a42c04d4782..8d19810504b8dbb4abe52140444a3f2077e83ac5 100644 --- a/Documentation/bpf/standardization/instruction-set.rst +++ b/Documentation/bpf/standardization/instruction-set.rst @@ -14,6 +14,13 @@ set architecture (ISA). Documentation conventions ========================= +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", +"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and +"OPTIONAL" in this document are to be interpreted as described in +BCP 14 ``_ +`RFC8174 `_ +when, and only when, they appear in all capitals, as shown here. + For brevity and consistency, this document refers to families of types using a shorthand syntax and refers to several expository, mnemonic functions when describing the semantics of instructions. @@ -25,7 +32,7 @@ Types This document refers to integer types with the notation `SN` to specify a type's signedness (`S`) and bit width (`N`), respectively. -.. table:: Meaning of signedness notation. +.. table:: Meaning of signedness notation ==== ========= S Meaning @@ -34,7 +41,7 @@ a type's signedness (`S`) and bit width (`N`), respectively. s signed ==== ========= -.. table:: Meaning of bit-width notation. +.. table:: Meaning of bit-width notation ===== ========= N Bit width @@ -106,9 +113,9 @@ Conformance groups An implementation does not need to support all instructions specified in this document (e.g., deprecated instructions). Instead, a number of conformance -groups are specified. An implementation must support the base32 conformance -group and may support additional conformance groups, where supporting a -conformance group means it must support all instructions in that conformance +groups are specified. An implementation MUST support the base32 conformance +group and MAY support additional conformance groups, where supporting a +conformance group means it MUST support all instructions in that conformance group. The use of named conformance groups enables interoperability between a runtime @@ -209,7 +216,7 @@ For example:: 07 1 0 00 00 11 22 33 44 r1 += 0x11223344 // big Note that most instructions do not use all of the fields. -Unused fields shall be cleared to zero. +Unused fields SHALL be cleared to zero. Wide instruction encoding -------------------------- @@ -256,18 +263,20 @@ Instruction classes The three least significant bits of the 'opcode' field store the instruction class: -===== ===== =============================== =================================== -class value description reference -===== ===== =============================== =================================== -LD 0x0 non-standard load operations `Load and store instructions`_ -LDX 0x1 load into register operations `Load and store instructions`_ -ST 0x2 store from immediate operations `Load and store instructions`_ -STX 0x3 store from register operations `Load and store instructions`_ -ALU 0x4 32-bit arithmetic operations `Arithmetic and jump instructions`_ -JMP 0x5 64-bit jump operations `Arithmetic and jump instructions`_ -JMP32 0x6 32-bit jump operations `Arithmetic and jump instructions`_ -ALU64 0x7 64-bit arithmetic operations `Arithmetic and jump instructions`_ -===== ===== =============================== =================================== +.. table:: Instruction class + + ===== ===== =============================== =================================== + class value description reference + ===== ===== =============================== =================================== + LD 0x0 non-standard load operations `Load and store instructions`_ + LDX 0x1 load into register operations `Load and store instructions`_ + ST 0x2 store from immediate operations `Load and store instructions`_ + STX 0x3 store from register operations `Load and store instructions`_ + ALU 0x4 32-bit arithmetic operations `Arithmetic and jump instructions`_ + JMP 0x5 64-bit jump operations `Arithmetic and jump instructions`_ + JMP32 0x6 32-bit jump operations `Arithmetic and jump instructions`_ + ALU64 0x7 64-bit arithmetic operations `Arithmetic and jump instructions`_ + ===== ===== =============================== =================================== Arithmetic and jump instructions ================================ @@ -285,12 +294,14 @@ For arithmetic and jump instructions (``ALU``, ``ALU64``, ``JMP`` and **s (source)** the source operand location, which unless otherwise specified is one of: - ====== ===== ============================================== - source value description - ====== ===== ============================================== - K 0 use 32-bit 'imm' value as source operand - X 1 use 'src_reg' register value as source operand - ====== ===== ============================================== + .. table:: Source operand location + + ====== ===== ============================================== + source value description + ====== ===== ============================================== + K 0 use 32-bit 'imm' value as source operand + X 1 use 'src_reg' register value as source operand + ====== ===== ============================================== **instruction class** the instruction class (see `Instruction classes`_) @@ -305,27 +316,29 @@ The 'code' field encodes the operation as below, where 'src' refers to the the source operand and 'dst' refers to the value of the destination register. -===== ===== ======= ========================================================== -name code offset description -===== ===== ======= ========================================================== -ADD 0x0 0 dst += src -SUB 0x1 0 dst -= src -MUL 0x2 0 dst \*= src -DIV 0x3 0 dst = (src != 0) ? (dst / src) : 0 -SDIV 0x3 1 dst = (src != 0) ? (dst s/ src) : 0 -OR 0x4 0 dst \|= src -AND 0x5 0 dst &= src -LSH 0x6 0 dst <<= (src & mask) -RSH 0x7 0 dst >>= (src & mask) -NEG 0x8 0 dst = -dst -MOD 0x9 0 dst = (src != 0) ? (dst % src) : dst -SMOD 0x9 1 dst = (src != 0) ? (dst s% src) : dst -XOR 0xa 0 dst ^= src -MOV 0xb 0 dst = src -MOVSX 0xb 8/16/32 dst = (s8,s16,s32)src -ARSH 0xc 0 :term:`sign extending` dst >>= (src & mask) -END 0xd 0 byte swap operations (see `Byte swap instructions`_ below) -===== ===== ======= ========================================================== +.. table:: Arithmetic instructions + + ===== ===== ======= ========================================================== + name code offset description + ===== ===== ======= ========================================================== + ADD 0x0 0 dst += src + SUB 0x1 0 dst -= src + MUL 0x2 0 dst \*= src + DIV 0x3 0 dst = (src != 0) ? (dst / src) : 0 + SDIV 0x3 1 dst = (src != 0) ? (dst s/ src) : 0 + OR 0x4 0 dst \|= src + AND 0x5 0 dst &= src + LSH 0x6 0 dst <<= (src & mask) + RSH 0x7 0 dst >>= (src & mask) + NEG 0x8 0 dst = -dst + MOD 0x9 0 dst = (src != 0) ? (dst % src) : dst + SMOD 0x9 1 dst = (src != 0) ? (dst s% src) : dst + XOR 0xa 0 dst ^= src + MOV 0xb 0 dst = src + MOVSX 0xb 8/16/32 dst = (s8,s16,s32)src + ARSH 0xc 0 :term:`sign extending` dst >>= (src & mask) + END 0xd 0 byte swap operations (see `Byte swap instructions`_ below) + ===== ===== ======= ========================================================== Underflow and overflow are allowed during arithmetic operations, meaning the 64-bit or 32-bit value will wrap. If BPF program execution would @@ -374,7 +387,7 @@ interpreted as a 64-bit signed value. Note that there are varying definitions of the signed modulo operation when the dividend or divisor are negative, where implementations often vary by language such that Python, Ruby, etc. differ from C, Go, Java, -etc. This specification requires that signed modulo use truncated division +etc. This specification requires that signed modulo MUST use truncated division (where -13 % 3 == -1) as implemented in C, Go, etc.:: a % n = a - n * trunc(a / n) @@ -386,6 +399,19 @@ The ``MOVSX`` instruction does a move operation with sign extension. operands into 64-bit operands. Unlike other arithmetic instructions, ``MOVSX`` is only defined for register source operands (``X``). +``{MOV, K, ALU64}`` means:: + + dst = (s64)imm + +``{MOV, X, ALU}`` means:: + + dst = (u32)src + +``{MOVSX, X, ALU}`` with 'offset' 8 means:: + + dst = (u32)(s32)(s8)src + + The ``NEG`` instruction is only defined when the source bit is clear (``K``). @@ -404,15 +430,17 @@ only and do not use a separate source register or immediate value. For ``ALU``, the 1-bit source operand field in the opcode is used to select what byte order the operation converts from or to. For ``ALU64``, the 1-bit source operand field in the opcode is reserved -and must be set to 0. +and MUST be set to 0. -===== ======== ===== ================================================= -class source value description -===== ======== ===== ================================================= -ALU TO_LE 0 convert between host byte order and little endian -ALU TO_BE 1 convert between host byte order and big endian -ALU64 Reserved 0 do byte swap unconditionally -===== ======== ===== ================================================= +.. table:: Byte swap instructions + + ===== ======== ===== ================================================= + class source value description + ===== ======== ===== ================================================= + ALU TO_LE 0 convert between host byte order and little endian + ALU TO_BE 1 convert between host byte order and big endian + ALU64 Reserved 0 do byte swap unconditionally + ===== ======== ===== ================================================= The 'imm' field encodes the width of the swap operations. The following widths are supported: 16, 32 and 64. Width 64 operations belong to the base64 @@ -448,27 +476,29 @@ otherwise identical operations, and indicates the base64 conformance group unless otherwise specified. The 'code' field encodes the operation as below: -======== ===== ======= ================================= =================================================== -code value src_reg description notes -======== ===== ======= ================================= =================================================== -JA 0x0 0x0 PC += offset {JA, K, JMP} only -JA 0x0 0x0 PC += imm {JA, K, JMP32} only -JEQ 0x1 any PC += offset if dst == src -JGT 0x2 any PC += offset if dst > src unsigned -JGE 0x3 any PC += offset if dst >= src unsigned -JSET 0x4 any PC += offset if dst & src -JNE 0x5 any PC += offset if dst != src -JSGT 0x6 any PC += offset if dst > src signed -JSGE 0x7 any PC += offset if dst >= src signed -CALL 0x8 0x0 call helper function by static ID {CALL, K, JMP} only, see `Helper functions`_ -CALL 0x8 0x1 call PC += imm {CALL, K, JMP} only, see `Program-local functions`_ -CALL 0x8 0x2 call helper function by BTF ID {CALL, K, JMP} only, see `Helper functions`_ -EXIT 0x9 0x0 return {CALL, K, JMP} only -JLT 0xa any PC += offset if dst < src unsigned -JLE 0xb any PC += offset if dst <= src unsigned -JSLT 0xc any PC += offset if dst < src signed -JSLE 0xd any PC += offset if dst <= src signed -======== ===== ======= ================================= =================================================== +.. table:: Jump instructions + + ======== ===== ======= ================================= =================================================== + code value src_reg description notes + ======== ===== ======= ================================= =================================================== + JA 0x0 0x0 PC += offset {JA, K, JMP} only + JA 0x0 0x0 PC += imm {JA, K, JMP32} only + JEQ 0x1 any PC += offset if dst == src + JGT 0x2 any PC += offset if dst > src unsigned + JGE 0x3 any PC += offset if dst >= src unsigned + JSET 0x4 any PC += offset if dst & src + JNE 0x5 any PC += offset if dst != src + JSGT 0x6 any PC += offset if dst > src signed + JSGE 0x7 any PC += offset if dst >= src signed + CALL 0x8 0x0 call helper function by static ID {CALL, K, JMP} only, see `Helper functions`_ + CALL 0x8 0x1 call PC += imm {CALL, K, JMP} only, see `Program-local functions`_ + CALL 0x8 0x2 call helper function by BTF ID {CALL, K, JMP} only, see `Helper functions`_ + EXIT 0x9 0x0 return {CALL, K, JMP} only + JLT 0xa any PC += offset if dst < src unsigned + JLE 0xb any PC += offset if dst <= src unsigned + JSLT 0xc any PC += offset if dst < src signed + JSLE 0xd any PC += offset if dst <= src signed + ======== ===== ======= ================================= =================================================== where 'PC' denotes the program counter, and the offset to increment by is in units of 64-bit instructions relative to the instruction following @@ -476,9 +506,6 @@ the jump instruction. Thus 'PC += 1' skips execution of the next instruction if it's a basic instruction or results in undefined behavior if the next instruction is a 128-bit wide instruction. -The BPF program needs to store the return value into register R0 before doing an -``EXIT``. - Example: ``{JSGE, X, JMP32}`` means:: @@ -487,6 +514,10 @@ Example: where 's>=' indicates a signed '>=' comparison. +``{JLE, K, JMP}`` means:: + + if dst <= (u64)(s64)imm goto +offset + ``{JA, K, JMP32}`` means:: gotol +imm @@ -515,14 +546,16 @@ for each program type, but static IDs are unique across all program types. Platforms that support the BPF Type Format (BTF) support identifying a helper function by a BTF ID encoded in the 'imm' field, where the BTF ID -identifies the helper name and type. +identifies the helper name and type. Further documentation of BTF +is outside the scope of this document and is left for future work. Program-local functions ~~~~~~~~~~~~~~~~~~~~~~~ Program-local functions are functions exposed by the same BPF program as the -caller, and are referenced by offset from the call instruction, similar to -``JA``. The offset is encoded in the 'imm' field of the call instruction. -An ``EXIT`` within the program-local function will return to the caller. +caller, and are referenced by offset from the instruction following the call +instruction, similar to ``JA``. The offset is encoded in the 'imm' field of +the call instruction. An ``EXIT`` within the program-local function will +return to the caller. Load and store instructions =========================== @@ -537,6 +570,8 @@ For load and store instructions (``LD``, ``LDX``, ``ST``, and ``STX``), the **mode** The mode modifier is one of: + .. table:: Mode modifier + ============= ===== ==================================== ============= mode modifier value description reference ============= ===== ==================================== ============= @@ -551,6 +586,8 @@ For load and store instructions (``LD``, ``LDX``, ``ST``, and ``STX``), the **sz (size)** The size modifier is one of: + .. table:: Size modifier + ==== ===== ===================== size value description ==== ===== ===================== @@ -619,14 +656,16 @@ The 'imm' field is used to encode the actual atomic operation. Simple atomic operation use a subset of the values defined to encode arithmetic operations in the 'imm' field to encode the atomic operation: -======== ===== =========== -imm value description -======== ===== =========== -ADD 0x00 atomic add -OR 0x40 atomic or -AND 0x50 atomic and -XOR 0xa0 atomic xor -======== ===== =========== +.. table:: Simple atomic operations + + ======== ===== =========== + imm value description + ======== ===== =========== + ADD 0x00 atomic add + OR 0x40 atomic or + AND 0x50 atomic and + XOR 0xa0 atomic xor + ======== ===== =========== ``{ATOMIC, W, STX}`` with 'imm' = ADD means:: @@ -640,13 +679,15 @@ XOR 0xa0 atomic xor In addition to the simple atomic operations, there also is a modifier and two complex atomic operations: -=========== ================ =========================== -imm value description -=========== ================ =========================== -FETCH 0x01 modifier: return old value -XCHG 0xe0 | FETCH atomic exchange -CMPXCHG 0xf0 | FETCH atomic compare and exchange -=========== ================ =========================== +.. table:: Complex atomic operations + + =========== ================ =========================== + imm value description + =========== ================ =========================== + FETCH 0x01 modifier: return old value + XCHG 0xe0 | FETCH atomic exchange + CMPXCHG 0xf0 | FETCH atomic compare and exchange + =========== ================ =========================== The ``FETCH`` modifier is optional for simple atomic operations, and always set for the complex atomic operations. If the ``FETCH`` flag @@ -673,17 +714,19 @@ The following table defines a set of ``{IMM, DW, LD}`` instructions with opcode subtypes in the 'src_reg' field, using new terms such as "map" defined further below: -======= ========================================= =========== ============== -src_reg pseudocode imm type dst type -======= ========================================= =========== ============== -0x0 dst = (next_imm << 32) | imm integer integer -0x1 dst = map_by_fd(imm) map fd map -0x2 dst = map_val(map_by_fd(imm)) + next_imm map fd data address -0x3 dst = var_addr(imm) variable id data address -0x4 dst = code_addr(imm) integer code address -0x5 dst = map_by_idx(imm) map index map -0x6 dst = map_val(map_by_idx(imm)) + next_imm map index data address -======= ========================================= =========== ============== +.. table:: 64-bit immediate instructions + + ======= ========================================= =========== ============== + src_reg pseudocode imm type dst type + ======= ========================================= =========== ============== + 0x0 dst = (next_imm << 32) | imm integer integer + 0x1 dst = map_by_fd(imm) map fd map + 0x2 dst = map_val(map_by_fd(imm)) + next_imm map fd data address + 0x3 dst = var_addr(imm) variable id data address + 0x4 dst = code_addr(imm) integer code address + 0x5 dst = map_by_idx(imm) map index map + 0x6 dst = map_val(map_by_idx(imm)) + next_imm map index data address + ======= ========================================= =========== ============== where @@ -725,5 +768,5 @@ carried over from classic BPF. These instructions used an instruction class of ``LD``, a size modifier of ``W``, ``H``, or ``B``, and a mode modifier of ``ABS`` or ``IND``. The 'dst_reg' and 'offset' fields were set to zero, and 'src_reg' was set to zero for ``ABS``. However, these -instructions are deprecated and should no longer be used. All legacy packet +instructions are deprecated and SHOULD no longer be used. All legacy packet access instructions belong to the "packet" conformance group. diff --git a/Documentation/core-api/swiotlb.rst b/Documentation/core-api/swiotlb.rst index 5ad2c9ca85bcbb87549135ee6e93215f06fbeb94..cf06bae44ff85bae217607f381146af7f446b96d 100644 --- a/Documentation/core-api/swiotlb.rst +++ b/Documentation/core-api/swiotlb.rst @@ -192,7 +192,7 @@ alignment larger than PAGE_SIZE. Dynamic swiotlb --------------- -When CONFIG_DYNAMIC_SWIOTLB is enabled, swiotlb can do on-demand expansion of +When CONFIG_SWIOTLB_DYNAMIC is enabled, swiotlb can do on-demand expansion of the amount of memory available for allocation as bounce buffers. If a bounce buffer request fails due to lack of available space, an asynchronous background task is kicked off to allocate memory from general system memory and turn it diff --git a/Documentation/devicetree/bindings/arm/stm32/st,mlahb.yaml b/Documentation/devicetree/bindings/arm/stm32/st,mlahb.yaml index d2dce238ff5d683ea0ddbdf575f53677fe129a51..3e996346b2644806486757b1dd36bc4cb215a51e 100644 --- a/Documentation/devicetree/bindings/arm/stm32/st,mlahb.yaml +++ b/Documentation/devicetree/bindings/arm/stm32/st,mlahb.yaml @@ -54,11 +54,10 @@ unevaluatedProperties: false examples: - | - mlahb: ahb@38000000 { + ahb { compatible = "st,mlahb", "simple-bus"; #address-cells = <1>; #size-cells = <1>; - reg = <0x10000000 0x40000>; ranges; dma-ranges = <0x00000000 0x38000000 0x10000>, <0x10000000 0x10000000 0x60000>, diff --git a/Documentation/devicetree/bindings/arm/sunxi.yaml b/Documentation/devicetree/bindings/arm/sunxi.yaml index c6d0d8d81ed4deb9dd9a79d52a91c669c74e324d..c2a158b75e4979eaf6fc148970cc08767f6de264 100644 --- a/Documentation/devicetree/bindings/arm/sunxi.yaml +++ b/Documentation/devicetree/bindings/arm/sunxi.yaml @@ -57,17 +57,17 @@ properties: - const: allwinner,sun8i-v3s - description: Anbernic RG35XX (2024) - - items: + items: - const: anbernic,rg35xx-2024 - const: allwinner,sun50i-h700 - description: Anbernic RG35XX Plus - - items: + items: - const: anbernic,rg35xx-plus - const: allwinner,sun50i-h700 - description: Anbernic RG35XX H - - items: + items: - const: anbernic,rg35xx-h - const: allwinner,sun50i-h700 diff --git a/Documentation/devicetree/bindings/net/dsa/vitesse,vsc73xx.txt b/Documentation/devicetree/bindings/net/dsa/vitesse,vsc73xx.txt deleted file mode 100644 index 258bef483673f45d3f1c2993bf321b903a086d9c..0000000000000000000000000000000000000000 --- a/Documentation/devicetree/bindings/net/dsa/vitesse,vsc73xx.txt +++ /dev/null @@ -1,129 +0,0 @@ -Vitesse VSC73xx Switches -======================== - -This defines device tree bindings for the Vitesse VSC73xx switch chips. -The Vitesse company has been acquired by Microsemi and Microsemi has -been acquired Microchip but retains this vendor branding. - -The currently supported switch chips are: -Vitesse VSC7385 SparX-G5 5+1-port Integrated Gigabit Ethernet Switch -Vitesse VSC7388 SparX-G8 8-port Integrated Gigabit Ethernet Switch -Vitesse VSC7395 SparX-G5e 5+1-port Integrated Gigabit Ethernet Switch -Vitesse VSC7398 SparX-G8e 8-port Integrated Gigabit Ethernet Switch - -This switch could have two different management interface. - -If SPI interface is used, the device tree node is an SPI device so it must -reside inside a SPI bus device tree node, see spi/spi-bus.txt - -When the chip is connected to a parallel memory bus and work in memory-mapped -I/O mode, a platform device is used to represent the vsc73xx. In this case it -must reside inside a platform bus device tree node. - -Required properties: - -- compatible: must be exactly one of: - "vitesse,vsc7385" - "vitesse,vsc7388" - "vitesse,vsc7395" - "vitesse,vsc7398" -- gpio-controller: indicates that this switch is also a GPIO controller, - see gpio/gpio.txt -- #gpio-cells: this must be set to <2> and indicates that we are a twocell - GPIO controller, see gpio/gpio.txt - -Optional properties: - -- reset-gpios: a handle to a GPIO line that can issue reset of the chip. - It should be tagged as active low. - -Required subnodes: - -See net/dsa/dsa.txt for a list of additional required and optional properties -and subnodes of DSA switches. - -Examples: - -SPI: -switch@0 { - compatible = "vitesse,vsc7395"; - reg = <0>; - /* Specified for 2.5 MHz or below */ - spi-max-frequency = <2500000>; - gpio-controller; - #gpio-cells = <2>; - - ports { - #address-cells = <1>; - #size-cells = <0>; - - port@0 { - reg = <0>; - label = "lan1"; - }; - port@1 { - reg = <1>; - label = "lan2"; - }; - port@2 { - reg = <2>; - label = "lan3"; - }; - port@3 { - reg = <3>; - label = "lan4"; - }; - vsc: port@6 { - reg = <6>; - ethernet = <&gmac1>; - phy-mode = "rgmii"; - fixed-link { - speed = <1000>; - full-duplex; - pause; - }; - }; - }; -}; - -Platform: -switch@2,0 { - #address-cells = <1>; - #size-cells = <1>; - compatible = "vitesse,vsc7385"; - reg = <0x2 0x0 0x20000>; - reset-gpios = <&gpio0 12 GPIO_ACTIVE_LOW>; - - ports { - #address-cells = <1>; - #size-cells = <0>; - - port@0 { - reg = <0>; - label = "lan1"; - }; - port@1 { - reg = <1>; - label = "lan2"; - }; - port@2 { - reg = <2>; - label = "lan3"; - }; - port@3 { - reg = <3>; - label = "lan4"; - }; - vsc: port@6 { - reg = <6>; - ethernet = <&enet0>; - phy-mode = "rgmii"; - fixed-link { - speed = <1000>; - full-duplex; - pause; - }; - }; - }; - -}; diff --git a/Documentation/devicetree/bindings/net/dsa/vitesse,vsc73xx.yaml b/Documentation/devicetree/bindings/net/dsa/vitesse,vsc73xx.yaml new file mode 100644 index 0000000000000000000000000000000000000000..b99d7a694b70ed38b4442c5e998b4cfac262f532 --- /dev/null +++ b/Documentation/devicetree/bindings/net/dsa/vitesse,vsc73xx.yaml @@ -0,0 +1,162 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/net/dsa/vitesse,vsc73xx.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Vitesse VSC73xx DSA Switches + +maintainers: + - Linus Walleij + +description: + The Vitesse DSA Switches were produced in the early-to-mid 2000s. + + The Vitesse company has been acquired by Microsemi and Microsemi has + been acquired Microchip but the new owner retains this vendor branding. + + The currently supported switch chips are + Vitesse VSC7385 SparX-G5 5+1-port Integrated Gigabit Ethernet Switch + Vitesse VSC7388 SparX-G8 8-port Integrated Gigabit Ethernet Switch + Vitesse VSC7395 SparX-G5e 5+1-port Integrated Gigabit Ethernet Switch + Vitesse VSC7398 SparX-G8e 8-port Integrated Gigabit Ethernet Switch + + This switch can use one of two different management interfaces. + + If SPI interface is used, the device tree node is an SPI device so it must + reside inside a SPI bus device tree node, see spi/spi-bus.txt + + When the chip is connected to a parallel memory bus and work in memory-mapped + I/O mode, a platform device is used to represent the vsc73xx. In this case it + must reside inside a platform bus device tree node. + +properties: + compatible: + enum: + - vitesse,vsc7385 + - vitesse,vsc7388 + - vitesse,vsc7395 + - vitesse,vsc7398 + + reg: + maxItems: 1 + + gpio-controller: true + "#gpio-cells": + const: 2 + + reset-gpios: + description: GPIO to be used to reset the whole device + maxItems: 1 + +allOf: + - $ref: dsa.yaml#/$defs/ethernet-ports + +# This checks if reg is a chipselect so the device is on an SPI +# bus, the if-clause will fail if reg is a tuple such as for a +# platform device. +if: + properties: + reg: + minimum: 0 + maximum: 256 +then: + $ref: /schemas/spi/spi-peripheral-props.yaml# + +required: + - compatible + - reg + +unevaluatedProperties: false + +examples: + - | + #include + + spi { + #address-cells = <1>; + #size-cells = <0>; + + ethernet-switch@0 { + compatible = "vitesse,vsc7395"; + reg = <0>; + spi-max-frequency = <2500000>; + gpio-controller; + #gpio-cells = <2>; + + ethernet-ports { + #address-cells = <1>; + #size-cells = <0>; + + ethernet-port@0 { + reg = <0>; + label = "lan1"; + }; + ethernet-port@1 { + reg = <1>; + label = "lan2"; + }; + ethernet-port@2 { + reg = <2>; + label = "lan3"; + }; + ethernet-port@3 { + reg = <3>; + label = "lan4"; + }; + ethernet-port@6 { + reg = <6>; + ethernet = <&gmac1>; + phy-mode = "rgmii"; + fixed-link { + speed = <1000>; + full-duplex; + pause; + }; + }; + }; + }; + }; + + bus { + #address-cells = <1>; + #size-cells = <1>; + + ethernet-switch@10000000 { + compatible = "vitesse,vsc7385"; + reg = <0x10000000 0x20000>; + reset-gpios = <&gpio0 12 GPIO_ACTIVE_LOW>; + + ethernet-ports { + #address-cells = <1>; + #size-cells = <0>; + + ethernet-port@0 { + reg = <0>; + label = "lan1"; + }; + ethernet-port@1 { + reg = <1>; + label = "lan2"; + }; + ethernet-port@2 { + reg = <2>; + label = "lan3"; + }; + ethernet-port@3 { + reg = <3>; + label = "lan4"; + }; + ethernet-port@6 { + reg = <6>; + ethernet = <&enet0>; + phy-mode = "rgmii"; + fixed-link { + speed = <1000>; + full-duplex; + pause; + }; + }; + }; + }; + }; diff --git a/Documentation/devicetree/bindings/net/pse-pd/microchip,pd692x0.yaml b/Documentation/devicetree/bindings/net/pse-pd/microchip,pd692x0.yaml index 828439398fdf98a591d4fd3272fa8edfef31ae07..fd4244fceced9f80d8f6682b09d977439df877e6 100644 --- a/Documentation/devicetree/bindings/net/pse-pd/microchip,pd692x0.yaml +++ b/Documentation/devicetree/bindings/net/pse-pd/microchip,pd692x0.yaml @@ -24,6 +24,7 @@ properties: managers: type: object + additionalProperties: false description: List of the PD69208T4/PD69204T4/PD69208M PSE managers. Each manager have 4 or 8 physical ports according to the chip version. No need to @@ -47,8 +48,9 @@ properties: - "#size-cells" patternProperties: - "^manager@0[0-9a-b]$": + "^manager@[0-9a-b]$": type: object + additionalProperties: false description: PD69208T4/PD69204T4/PD69208M PSE manager exposing 4 or 8 physical ports. @@ -69,9 +71,14 @@ properties: patternProperties: '^port@[0-7]$': type: object + additionalProperties: false + + properties: + reg: + maxItems: 1 + required: - reg - additionalProperties: false required: - reg diff --git a/Documentation/devicetree/bindings/net/pse-pd/ti,tps23881.yaml b/Documentation/devicetree/bindings/net/pse-pd/ti,tps23881.yaml index 4147adb11e10190620860ef421e9f09c7251b08c..6992d56832bf95c793ff6665b3626d228701c1a7 100644 --- a/Documentation/devicetree/bindings/net/pse-pd/ti,tps23881.yaml +++ b/Documentation/devicetree/bindings/net/pse-pd/ti,tps23881.yaml @@ -29,13 +29,31 @@ properties: of the ports conversion matrix that establishes relationship between the logical ports and the physical channels. type: object + additionalProperties: false + + properties: + "#address-cells": + const: 1 + + "#size-cells": + const: 0 patternProperties: '^channel@[0-7]$': type: object + additionalProperties: false + + properties: + reg: + maxItems: 1 + required: - reg + required: + - "#address-cells" + - "#size-cells" + unevaluatedProperties: false required: diff --git a/Documentation/devicetree/bindings/net/snps,dwmac.yaml b/Documentation/devicetree/bindings/net/snps,dwmac.yaml index 21cc27e75f50a4a574d6ac483345731f91060040..3bab4e1f3fbf67260861c53927045e6a7d5d81f9 100644 --- a/Documentation/devicetree/bindings/net/snps,dwmac.yaml +++ b/Documentation/devicetree/bindings/net/snps,dwmac.yaml @@ -76,6 +76,7 @@ properties: - rockchip,rk3128-gmac - rockchip,rk3228-gmac - rockchip,rk3288-gmac + - rockchip,rk3308-gmac - rockchip,rk3328-gmac - rockchip,rk3366-gmac - rockchip,rk3368-gmac diff --git a/Documentation/devicetree/bindings/net/ti,icssg-prueth.yaml b/Documentation/devicetree/bindings/net/ti,icssg-prueth.yaml index e253fa786092220cba4c15be87d5aaa3284ad678..c296e5711848437995016a33e5e580864998a6c4 100644 --- a/Documentation/devicetree/bindings/net/ti,icssg-prueth.yaml +++ b/Documentation/devicetree/bindings/net/ti,icssg-prueth.yaml @@ -55,6 +55,14 @@ properties: description: phandle to MII_RT module's syscon regmap + ti,pa-stats: + $ref: /schemas/types.yaml#/definitions/phandle + description: + phandle to PA_STATS module's syscon regmap. PA_STATS is a set of + registers where different statistics related to ICSSG, are dumped by + ICSSG firmware. PA_STATS module's syscon regmap will help the device to + access/read/write those statistics. + ti,iep: $ref: /schemas/types.yaml#/definitions/phandle-array maxItems: 2 @@ -194,6 +202,7 @@ examples: "tx1-0", "tx1-1", "tx1-2", "tx1-3", "rx0", "rx1"; ti,mii-g-rt = <&icssg2_mii_g_rt>; + ti,pa-stats = <&icssg2_pa_stats>; ti,iep = <&icssg2_iep0>, <&icssg2_iep1>; interrupt-parent = <&icssg2_intc>; interrupts = <24 0 2>, <25 1 3>; diff --git a/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.yaml b/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.yaml index 5c4498b762c89fe6964b7e3d2dba710e925eef51..070c4c9b86437f77f261889aeea98340d8d44b99 100644 --- a/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.yaml +++ b/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.yaml @@ -128,6 +128,11 @@ properties: Whether to skip executing an SCM call that reassigns the memory region ownership. + qcom,no-msa-ready-indicator: + type: boolean + description: + Don't wait for MSA_READY indicator to complete init. + qcom,smem-states: $ref: /schemas/types.yaml#/definitions/phandle-array description: State bits used by the AP to signal the WLAN Q6. diff --git a/Documentation/devicetree/bindings/net/wireless/qcom,ath11k.yaml b/Documentation/devicetree/bindings/net/wireless/qcom,ath11k.yaml index a2d55bf4c7a518a5d81a9a646c70a319a7e9217c..ff5763dc66a88b635b6a358d4dd68c3ff71cea2a 100644 --- a/Documentation/devicetree/bindings/net/wireless/qcom,ath11k.yaml +++ b/Documentation/devicetree/bindings/net/wireless/qcom,ath11k.yaml @@ -265,15 +265,6 @@ allOf: examples: - | - - q6v5_wcss: remoteproc@cd00000 { - compatible = "qcom,ipq8074-wcss-pil"; - reg = <0xcd00000 0x4040>, - <0x4ab000 0x20>; - reg-names = "qdsp6", - "rmb"; - }; - wifi0: wifi@c000000 { compatible = "qcom,ipq8074-wifi"; reg = <0xc000000 0x2000000>; diff --git a/Documentation/devicetree/bindings/net/xlnx,gmii-to-rgmii.yaml b/Documentation/devicetree/bindings/net/xlnx,gmii-to-rgmii.yaml index 0f781dac6717c648ad8de5c9dc576b7965c1e2b1..eb803ddd13e0a9f8e85fff1cf35bd0d099ce350f 100644 --- a/Documentation/devicetree/bindings/net/xlnx,gmii-to-rgmii.yaml +++ b/Documentation/devicetree/bindings/net/xlnx,gmii-to-rgmii.yaml @@ -31,6 +31,10 @@ properties: phy-handle: $ref: ethernet-controller.yaml#/properties/phy-handle + clocks: + items: + - description: 200/375 MHz free-running clock is used as input clock. + required: - compatible - reg @@ -51,5 +55,6 @@ examples: compatible = "xlnx,gmii-to-rgmii-1.0"; reg = <8>; phy-handle = <&phy>; + clocks = <&dummy>; }; }; diff --git a/Documentation/netlink/specs/dpll.yaml b/Documentation/netlink/specs/dpll.yaml index 95b0eb1486bf7edd30eefaef0b4a61e6f2482d7b..94132d30e0e03506bb287918a9a6a2ac3b682a94 100644 --- a/Documentation/netlink/specs/dpll.yaml +++ b/Documentation/netlink/specs/dpll.yaml @@ -479,6 +479,7 @@ operations: name: pin-get doc: | Get list of pins and its attributes. + - dump request without any attributes given - list all the pins in the system - dump request with target dpll - list all the pins registered with diff --git a/Documentation/netlink/specs/netdev.yaml b/Documentation/netlink/specs/netdev.yaml index 11a32373365ab0e2b70999bdd64a351a6e53560d..959755be4d7f9c15abef7e74813cf8837b8a38fc 100644 --- a/Documentation/netlink/specs/netdev.yaml +++ b/Documentation/netlink/specs/netdev.yaml @@ -349,6 +349,10 @@ attribute-sets: Number of packets dropped due to transient lack of resources, such as buffer space, host descriptors etc. type: uint + - + name: rx-csum-complete + doc: Number of packets that were marked as CHECKSUM_COMPLETE. + type: uint - name: rx-csum-unnecessary doc: Number of packets that were marked as CHECKSUM_UNNECESSARY. diff --git a/Documentation/networking/af_xdp.rst b/Documentation/networking/af_xdp.rst index 72da7057e4cf9643c179493dce22f9e243f31d70..dceeb0d763aa232f418bc229a59a3a6d6b7540b2 100644 --- a/Documentation/networking/af_xdp.rst +++ b/Documentation/networking/af_xdp.rst @@ -329,24 +329,23 @@ XDP_SHARED_UMEM option and provide the initial socket's fd in the sxdp_shared_umem_fd field as you registered the UMEM on that socket. These two sockets will now share one and the same UMEM. -In this case, it is possible to use the NIC's packet steering -capabilities to steer the packets to the right queue. This is not -possible in the previous example as there is only one queue shared -among sockets, so the NIC cannot do this steering as it can only steer -between queues. - -In libxdp (or libbpf prior to version 1.0), you need to use the -xsk_socket__create_shared() API as it takes a reference to a FILL ring -and a COMPLETION ring that will be created for you and bound to the -shared UMEM. You can use this function for all the sockets you create, -or you can use it for the second and following ones and use -xsk_socket__create() for the first one. Both methods yield the same -result. +There is no need to supply an XDP program like the one in the previous +case where sockets were bound to the same queue id and +device. Instead, use the NIC's packet steering capabilities to steer +the packets to the right queue. In the previous example, there is only +one queue shared among sockets, so the NIC cannot do this steering. It +can only steer between queues. + +In libbpf, you need to use the xsk_socket__create_shared() API as it +takes a reference to a FILL ring and a COMPLETION ring that will be +created for you and bound to the shared UMEM. You can use this +function for all the sockets you create, or you can use it for the +second and following ones and use xsk_socket__create() for the first +one. Both methods yield the same result. Note that a UMEM can be shared between sockets on the same queue id and device, as well as between queues on the same device and between -devices at the same time. It is also possible to redirect to any -socket as long as it is bound to the same umem with XDP_SHARED_UMEM. +devices at the same time. XDP_USE_NEED_WAKEUP bind flag ----------------------------- @@ -823,10 +822,6 @@ A: The short answer is no, that is not supported at the moment. The switch, or other distribution mechanism, in your NIC to direct traffic to the correct queue id and socket. - Note that if you are using the XDP_SHARED_UMEM option, it is - possible to switch traffic between any socket bound to the same - umem. - Q: My packets are sometimes corrupted. What is wrong? A: Care has to be taken not to feed the same buffer in the UMEM into diff --git a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst index fed821ef9b0940af4387df99e1b3f110afb0afa0..3bd72577af9ae1d51b2b9df46fb6bf663820ec86 100644 --- a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst +++ b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst @@ -189,22 +189,19 @@ the software port. * - `rx[i]_gro_packets` - Number of received packets processed using hardware-accelerated GRO. The - number of hardware GRO offloaded packets received on ring i. + number of hardware GRO offloaded packets received on ring i. Only true GRO + packets are counted: only packets that are in an SKB with a GRO count > 1. - Acceleration * - `rx[i]_gro_bytes` - Number of received bytes processed using hardware-accelerated GRO. The - number of hardware GRO offloaded bytes received on ring i. + number of hardware GRO offloaded bytes received on ring i. Only true GRO + packets are counted: only packets that are in an SKB with a GRO count > 1. - Acceleration * - `rx[i]_gro_skbs` - - The number of receive SKBs constructed while performing - hardware-accelerated GRO. - - Informative - - * - `rx[i]_gro_match_packets` - - Number of received packets processed using hardware-accelerated GRO that - met the flow table match criteria. + - The number of GRO SKBs constructed from hardware-accelerated GRO. Only SKBs + with a GRO count > 1 are counted. - Informative * - `rx[i]_gro_large_hds` @@ -212,6 +209,15 @@ the software port. headers that require additional memory to be allocated. - Informative + * - `rx[i]_hds_nodata_packets` + - Number of header only packets in header/data split mode [#accel]_. + - Informative + + * - `rx[i]_hds_nodata_bytes` + - Number of bytes for header only packets in header/data split mode + [#accel]_. + - Informative + * - `rx[i]_lro_packets` - The number of LRO packets received on ring i [#accel]_. - Acceleration diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst index 7664c0bfe461c49d30e0fab104e7eb0adeefe9da..a6443851a14249a59ded3b58ff5b062dfa15b1d4 100644 --- a/Documentation/networking/index.rst +++ b/Documentation/networking/index.rst @@ -72,6 +72,7 @@ Contents: mac80211-injection mctp mpls-sysctl + mptcp mptcp-sysctl multiqueue multi-pf-netdev diff --git a/Documentation/networking/ip-sysctl.rst b/Documentation/networking/ip-sysctl.rst index bd50df6a5a42e6f38e97e63050b2036eb3ebf897..6e99eccdb837c12044697f113c5f148829413072 100644 --- a/Documentation/networking/ip-sysctl.rst +++ b/Documentation/networking/ip-sysctl.rst @@ -1196,6 +1196,19 @@ tcp_pingpong_thresh - INTEGER Default: 1 +tcp_rto_min_us - INTEGER + Minimal TCP retransmission timeout (in microseconds). Note that the + rto_min route option has the highest precedence for configuring this + setting, followed by the TCP_BPF_RTO_MIN socket option, followed by + this tcp_rto_min_us sysctl. + + The recommended practice is to use a value less or equal to 200000 + microseconds. + + Possible Values: 1 - INT_MAX + + Default: 200000 + UDP variables ============= diff --git a/Documentation/networking/mptcp-sysctl.rst b/Documentation/networking/mptcp-sysctl.rst index 69975ce25a029400e4e55779ee8220a322b930f9..fd514bba8c431ca6489d76fc4d6b0a25ff52f490 100644 --- a/Documentation/networking/mptcp-sysctl.rst +++ b/Documentation/networking/mptcp-sysctl.rst @@ -7,14 +7,6 @@ MPTCP Sysfs variables /proc/sys/net/mptcp/* Variables =============================== -enabled - BOOLEAN - Control whether MPTCP sockets can be created. - - MPTCP sockets can be created if the value is 1. This is a - per-namespace sysctl. - - Default: 1 (enabled) - add_addr_timeout - INTEGER (seconds) Set the timeout after which an ADD_ADDR control message will be resent to an MPTCP peer that has not acknowledged a previous @@ -25,16 +17,22 @@ add_addr_timeout - INTEGER (seconds) Default: 120 -close_timeout - INTEGER (seconds) - Set the make-after-break timeout: in absence of any close or - shutdown syscall, MPTCP sockets will maintain the status - unchanged for such time, after the last subflow removal, before - moving to TCP_CLOSE. +allow_join_initial_addr_port - BOOLEAN + Allow peers to send join requests to the IP address and port number used + by the initial subflow if the value is 1. This controls a flag that is + sent to the peer at connection time, and whether such join requests are + accepted or denied. - The default value matches TCP_TIMEWAIT_LEN. This is a per-namespace - sysctl. + Joins to addresses advertised with ADD_ADDR are not affected by this + value. - Default: 60 + This is a per-namespace sysctl. + + Default: 1 + +available_schedulers - STRING + Shows the available schedulers choices that are registered. More packet + schedulers may be available, but not loaded. checksum_enabled - BOOLEAN Control whether DSS checksum can be enabled. @@ -44,18 +42,24 @@ checksum_enabled - BOOLEAN Default: 0 -allow_join_initial_addr_port - BOOLEAN - Allow peers to send join requests to the IP address and port number used - by the initial subflow if the value is 1. This controls a flag that is - sent to the peer at connection time, and whether such join requests are - accepted or denied. +close_timeout - INTEGER (seconds) + Set the make-after-break timeout: in absence of any close or + shutdown syscall, MPTCP sockets will maintain the status + unchanged for such time, after the last subflow removal, before + moving to TCP_CLOSE. - Joins to addresses advertised with ADD_ADDR are not affected by this - value. + The default value matches TCP_TIMEWAIT_LEN. This is a per-namespace + sysctl. - This is a per-namespace sysctl. + Default: 60 - Default: 1 +enabled - BOOLEAN + Control whether MPTCP sockets can be created. + + MPTCP sockets can be created if the value is 1. This is a + per-namespace sysctl. + + Default: 1 (enabled) pm_type - INTEGER Set the default path manager type to use for each new MPTCP @@ -74,6 +78,14 @@ pm_type - INTEGER Default: 0 +scheduler - STRING + Select the scheduler of your choice. + + Support for selection of different schedulers. This is a per-namespace + sysctl. + + Default: "default" + stale_loss_cnt - INTEGER The number of MPTCP-level retransmission intervals with no traffic and pending outstanding data on a given subflow required to declare it stale. @@ -85,11 +97,3 @@ stale_loss_cnt - INTEGER This is a per-namespace sysctl. Default: 4 - -scheduler - STRING - Select the scheduler of your choice. - - Support for selection of different schedulers. This is a per-namespace - sysctl. - - Default: "default" diff --git a/Documentation/networking/mptcp.rst b/Documentation/networking/mptcp.rst new file mode 100644 index 0000000000000000000000000000000000000000..17f2bab611644727e19c3969fa08fa974c702d92 --- /dev/null +++ b/Documentation/networking/mptcp.rst @@ -0,0 +1,156 @@ +.. SPDX-License-Identifier: GPL-2.0 + +===================== +Multipath TCP (MPTCP) +===================== + +Introduction +============ + +Multipath TCP or MPTCP is an extension to the standard TCP and is described in +`RFC 8684 (MPTCPv1) `_. It allows a +device to make use of multiple interfaces at once to send and receive TCP +packets over a single MPTCP connection. MPTCP can aggregate the bandwidth of +multiple interfaces or prefer the one with the lowest latency. It also allows a +fail-over if one path is down, and the traffic is seamlessly reinjected on other +paths. + +For more details about Multipath TCP in the Linux kernel, please see the +official website: `mptcp.dev `_. + + +Use cases +========= + +Thanks to MPTCP, being able to use multiple paths in parallel or simultaneously +brings new use-cases, compared to TCP: + +- Seamless handovers: switching from one path to another while preserving + established connections, e.g. to be used in mobility use-cases, like on + smartphones. +- Best network selection: using the "best" available path depending on some + conditions, e.g. latency, losses, cost, bandwidth, etc. +- Network aggregation: using multiple paths at the same time to have a higher + throughput, e.g. to combine fixed and mobile networks to send files faster. + + +Concepts +======== + +Technically, when a new socket is created with the ``IPPROTO_MPTCP`` protocol +(Linux-specific), a *subflow* (or *path*) is created. This *subflow* consists of +a regular TCP connection that is used to transmit data through one interface. +Additional *subflows* can be negotiated later between the hosts. For the remote +host to be able to detect the use of MPTCP, a new field is added to the TCP +*option* field of the underlying TCP *subflow*. This field contains, amongst +other things, a ``MP_CAPABLE`` option that tells the other host to use MPTCP if +it is supported. If the remote host or any middlebox in between does not support +it, the returned ``SYN+ACK`` packet will not contain MPTCP options in the TCP +*option* field. In that case, the connection will be "downgraded" to plain TCP, +and it will continue with a single path. + +This behavior is made possible by two internal components: the path manager, and +the packet scheduler. + +Path Manager +------------ + +The Path Manager is in charge of *subflows*, from creation to deletion, and also +address announcements. Typically, it is the client side that initiates subflows, +and the server side that announces additional addresses via the ``ADD_ADDR`` and +``REMOVE_ADDR`` options. + +Path managers are controlled by the ``net.mptcp.pm_type`` sysctl knob -- see +mptcp-sysctl.rst. There are two types: the in-kernel one (type ``0``) where the +same rules are applied for all the connections (see: ``ip mptcp``) ; and the +userspace one (type ``1``), controlled by a userspace daemon (i.e. `mptcpd +`_) where different rules can be applied for each +connection. The path managers can be controlled via a Netlink API; see +netlink_spec/mptcp_pm.rst. + +To be able to use multiple IP addresses on a host to create multiple *subflows* +(paths), the default in-kernel MPTCP path-manager needs to know which IP +addresses can be used. This can be configured with ``ip mptcp endpoint`` for +example. + +Packet Scheduler +---------------- + +The Packet Scheduler is in charge of selecting which available *subflow(s)* to +use to send the next data packet. It can decide to maximize the use of the +available bandwidth, only to pick the path with the lower latency, or any other +policy depending on the configuration. + +Packet schedulers are controlled by the ``net.mptcp.scheduler`` sysctl knob -- +see mptcp-sysctl.rst. + + +Sockets API +=========== + +Creating MPTCP sockets +---------------------- + +On Linux, MPTCP can be used by selecting MPTCP instead of TCP when creating the +``socket``: + +.. code-block:: C + + int sd = socket(AF_INET(6), SOCK_STREAM, IPPROTO_MPTCP); + +Note that ``IPPROTO_MPTCP`` is defined as ``262``. + +If MPTCP is not supported, ``errno`` will be set to: + +- ``EINVAL``: (*Invalid argument*): MPTCP is not available, on kernels < 5.6. +- ``EPROTONOSUPPORT`` (*Protocol not supported*): MPTCP has not been compiled, + on kernels >= v5.6. +- ``ENOPROTOOPT`` (*Protocol not available*): MPTCP has been disabled using + ``net.mptcp.enabled`` sysctl knob; see mptcp-sysctl.rst. + +MPTCP is then opt-in: applications need to explicitly request it. Note that +applications can be forced to use MPTCP with different techniques, e.g. +``LD_PRELOAD`` (see ``mptcpize``), eBPF (see ``mptcpify``), SystemTAP, +``GODEBUG`` (``GODEBUG=multipathtcp=1``), etc. + +Switching to ``IPPROTO_MPTCP`` instead of ``IPPROTO_TCP`` should be as +transparent as possible for the userspace applications. + +Socket options +-------------- + +MPTCP supports most socket options handled by TCP. It is possible some less +common options are not supported, but contributions are welcome. + +Generally, the same value is propagated to all subflows, including the ones +created after the calls to ``setsockopt()``. eBPF can be used to set different +values per subflow. + +There are some MPTCP specific socket options at the ``SOL_MPTCP`` (284) level to +retrieve info. They fill the ``optval`` buffer of the ``getsockopt()`` system +call: + +- ``MPTCP_INFO``: Uses ``struct mptcp_info``. +- ``MPTCP_TCPINFO``: Uses ``struct mptcp_subflow_data``, followed by an array of + ``struct tcp_info``. +- ``MPTCP_SUBFLOW_ADDRS``: Uses ``struct mptcp_subflow_data``, followed by an + array of ``mptcp_subflow_addrs``. +- ``MPTCP_FULL_INFO``: Uses ``struct mptcp_full_info``, with one pointer to an + array of ``struct mptcp_subflow_info`` (including the + ``struct mptcp_subflow_addrs``), and one pointer to an array of + ``struct tcp_info``, followed by the content of ``struct mptcp_info``. + +Note that at the TCP level, ``TCP_IS_MPTCP`` socket option can be used to know +if MPTCP is currently being used: the value will be set to 1 if it is. + + +Design choices +============== + +A new socket type has been added for MPTCP for the userspace-facing socket. The +kernel is in charge of creating subflow sockets: they are TCP sockets where the +behavior is modified using TCP-ULP. + +MPTCP listen sockets will create "plain" *accepted* TCP sockets if the +connection request from the client didn't ask for MPTCP, making the performance +impact minimal when MPTCP is enabled by default. diff --git a/Documentation/process/maintainer-netdev.rst b/Documentation/process/maintainer-netdev.rst index fd96e4a3cef9c09382e34419ec3f8ac1c5514cf4..5e1fcfad1c4c3eef53cab205b35bc9f38be030dc 100644 --- a/Documentation/process/maintainer-netdev.rst +++ b/Documentation/process/maintainer-netdev.rst @@ -227,7 +227,7 @@ preferably including links to previous postings, for example:: The amount of mooing will depend on packet rate so should match the diurnal cycle quite well. - Signed-of-by: Joe Defarmer + Signed-off-by: Joe Defarmer --- v3: - add a note about time-of-day mooing fluctuation to the commit message diff --git a/MAINTAINERS b/MAINTAINERS index d6c90161c7bfe3886e675396cb2d2d7b44acc3b3..cd3277a98cfeddd6aeab62ec08689fc4faeddc10 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1107,7 +1107,6 @@ L: linux-pm@vger.kernel.org S: Supported F: Documentation/admin-guide/pm/amd-pstate.rst F: drivers/cpufreq/amd-pstate* -F: include/linux/amd-pstate.h F: tools/power/x86/amd_pstate_tracer/amd_pstate_trace.py AMD PTDMA DRIVER @@ -3854,6 +3853,7 @@ BPF JIT for ARM64 M: Daniel Borkmann M: Alexei Starovoitov M: Puranjay Mohan +R: Xu Kuohai L: bpf@vger.kernel.org S: Supported F: arch/arm64/net/ @@ -5187,7 +5187,6 @@ F: Documentation/devicetree/bindings/media/i2c/chrontel,ch7322.yaml F: drivers/media/cec/i2c/ch7322.c CIRRUS LOGIC AUDIO CODEC DRIVERS -M: James Schulman M: David Rhodes M: Richard Fitzgerald L: alsa-devel@alsa-project.org (moderated for non-subscribers) @@ -15238,7 +15237,6 @@ F: drivers/staging/most/ F: include/linux/most.h MOTORCOMM PHY DRIVER -M: Peter Geis M: Frank L: netdev@vger.kernel.org S: Maintained @@ -15752,7 +15750,7 @@ B: https://github.com/multipath-tcp/mptcp_net-next/issues T: git https://github.com/multipath-tcp/mptcp_net-next.git export-net T: git https://github.com/multipath-tcp/mptcp_net-next.git export F: Documentation/netlink/specs/mptcp_pm.yaml -F: Documentation/networking/mptcp-sysctl.rst +F: Documentation/networking/mptcp*.rst F: include/net/mptcp.h F: include/trace/events/mptcp.h F: include/uapi/linux/mptcp*.h @@ -21316,7 +21314,7 @@ F: arch/riscv/boot/dts/starfive/ STARFIVE DWMAC GLUE LAYER M: Emil Renner Berthing -M: Samin Guo +M: Minda Chen S: Maintained F: Documentation/devicetree/bindings/net/starfive,jh7110-dwmac.yaml F: drivers/net/ethernet/stmicro/stmmac/dwmac-starfive.c @@ -22679,7 +22677,7 @@ L: tomoyo-users-en@lists.osdn.me (subscribers-only, for users in English) L: tomoyo-dev@lists.osdn.me (subscribers-only, for developers in Japanese) L: tomoyo-users@lists.osdn.me (subscribers-only, for users in Japanese) S: Maintained -W: https://tomoyo.osdn.jp/ +W: https://tomoyo.sourceforge.net/ F: security/tomoyo/ TOPSTAR LAPTOP EXTRAS DRIVER diff --git a/Makefile b/Makefile index f975b639632809e8599b4b131f2c2e56ebf8f186..7f921ae547f11640bf044d9e7ba1b159ef431c87 100644 --- a/Makefile +++ b/Makefile @@ -2,7 +2,7 @@ VERSION = 6 PATCHLEVEL = 10 SUBLEVEL = 0 -EXTRAVERSION = -rc1 +EXTRAVERSION = -rc2 NAME = Baby Opossum Posse # *DOCUMENTATION* diff --git a/arch/arc/net/bpf_jit.h b/arch/arc/net/bpf_jit.h index ec44873c42d15418f129da4519536502a00a65a2..495f3023e4c1821eba1804d37c5b77b7c08fd086 100644 --- a/arch/arc/net/bpf_jit.h +++ b/arch/arc/net/bpf_jit.h @@ -39,7 +39,7 @@ /************** Functions that the back-end must provide **************/ /* Extension for 32-bit operations. */ -inline u8 zext(u8 *buf, u8 rd); +u8 zext(u8 *buf, u8 rd); /***** Moves *****/ u8 mov_r32(u8 *buf, u8 rd, u8 rs, u8 sign_ext); u8 mov_r32_i32(u8 *buf, u8 reg, s32 imm); diff --git a/arch/arc/net/bpf_jit_arcv2.c b/arch/arc/net/bpf_jit_arcv2.c index 31bfb6e9ce00fbf75c124d4f635969d9caa15fa3..4458e409ca0a84dacfaee10423ad54eaa2752fa0 100644 --- a/arch/arc/net/bpf_jit_arcv2.c +++ b/arch/arc/net/bpf_jit_arcv2.c @@ -62,7 +62,7 @@ enum { * If/when we decide to add ARCv2 instructions that do use register pairs, * the mapping, hopefully, doesn't need to be revisited. */ -const u8 bpf2arc[][2] = { +static const u8 bpf2arc[][2] = { /* Return value from in-kernel function, and exit value from eBPF */ [BPF_REG_0] = {ARC_R_8, ARC_R_9}, /* Arguments from eBPF program to in-kernel function */ @@ -1302,7 +1302,7 @@ static u8 arc_b(u8 *buf, s32 offset) /************* Packers (Deal with BPF_REGs) **************/ -inline u8 zext(u8 *buf, u8 rd) +u8 zext(u8 *buf, u8 rd) { if (rd != BPF_REG_FP) return arc_movi_r(buf, REG_HI(rd), 0); @@ -2235,6 +2235,7 @@ u8 gen_swap(u8 *buf, u8 rd, u8 size, u8 endian, bool force, bool do_zext) break; default: /* The caller must have handled this. */ + break; } } else { /* @@ -2253,6 +2254,7 @@ u8 gen_swap(u8 *buf, u8 rd, u8 size, u8 endian, bool force, bool do_zext) break; default: /* The caller must have handled this. */ + break; } } @@ -2517,7 +2519,7 @@ u8 arc_epilogue(u8 *buf, u32 usage, u16 frame_size) #define JCC64_NR_OF_JMPS 3 /* Number of jumps in jcc64 template. */ #define JCC64_INSNS_TO_END 3 /* Number of insn. inclusive the 2nd jmp to end. */ #define JCC64_SKIP_JMP 1 /* Index of the "skip" jump to "end". */ -const struct { +static const struct { /* * "jit_off" is common between all "jmp[]" and is coupled with * "cond" of each "jmp[]" instance. e.g.: @@ -2883,7 +2885,7 @@ u8 gen_jmp_64(u8 *buf, u8 rd, u8 rs, u8 cond, u32 curr_off, u32 targ_off) * The "ARC_CC_SET" becomes "CC_unequal" because of the "tst" * instruction that precedes the conditional branch. */ -const u8 arcv2_32_jmps[ARC_CC_LAST] = { +static const u8 arcv2_32_jmps[ARC_CC_LAST] = { [ARC_CC_UGT] = CC_great_u, [ARC_CC_UGE] = CC_great_eq_u, [ARC_CC_ULT] = CC_less_u, diff --git a/arch/arc/net/bpf_jit_core.c b/arch/arc/net/bpf_jit_core.c index 6f6b4ffccf2c29202c721a4fcf66504723a12413..e3628922c24a0cf5ea9e978bd65fa9334a2f5e2f 100644 --- a/arch/arc/net/bpf_jit_core.c +++ b/arch/arc/net/bpf_jit_core.c @@ -159,7 +159,7 @@ static void jit_dump(const struct jit_context *ctx) /* Initialise the context so there's no garbage. */ static int jit_ctx_init(struct jit_context *ctx, struct bpf_prog *prog) { - memset(ctx, 0, sizeof(ctx)); + memset(ctx, 0, sizeof(*ctx)); ctx->orig_prog = prog; @@ -167,7 +167,7 @@ static int jit_ctx_init(struct jit_context *ctx, struct bpf_prog *prog) ctx->prog = bpf_jit_blind_constants(prog); if (IS_ERR(ctx->prog)) return PTR_ERR(ctx->prog); - ctx->blinded = (ctx->prog == ctx->orig_prog ? false : true); + ctx->blinded = (ctx->prog != ctx->orig_prog); /* If the verifier doesn't zero-extend, then we have to do it. */ ctx->do_zext = !ctx->prog->aux->verifier_zext; @@ -1182,12 +1182,12 @@ static int jit_prepare(struct jit_context *ctx) } /* - * All the "handle_*()" functions have been called before by the - * "jit_prepare()". If there was an error, we would know by now. - * Therefore, no extra error checking at this point, other than - * a sanity check at the end that expects the calculated length - * (jit.len) to be equal to the length of generated instructions - * (jit.index). + * jit_compile() is the real compilation phase. jit_prepare() is + * invoked before jit_compile() as a dry-run to make sure everything + * will go OK and allocate the necessary memory. + * + * In the end, jit_compile() checks if it has produced the same number + * of instructions as jit_prepare() would. */ static int jit_compile(struct jit_context *ctx) { @@ -1407,9 +1407,9 @@ static struct bpf_prog *do_extra_pass(struct bpf_prog *prog) /* * This function may be invoked twice for the same stream of BPF - * instructions. The "extra pass" happens, when there are "call"s - * involved that their addresses are not known during the first - * invocation. + * instructions. The "extra pass" happens, when there are + * (re)locations involved that their addresses are not known + * during the first run. */ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) { diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index e4546b29dd0cbfbb5b902e15caadab69458b1fac..fd87c4b8f984039ceeba3d2da074f66293733a64 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -146,7 +146,7 @@ /* Coprocessor traps */ .macro __init_el2_cptr __check_hvhe .LnVHE_\@, x1 - mov x0, #(CPACR_EL1_FPEN_EL1EN | CPACR_EL1_FPEN_EL0EN) + mov x0, #CPACR_ELx_FPEN msr cpacr_el1, x0 b .Lskip_set_cptr_\@ .LnVHE_\@: @@ -277,7 +277,7 @@ // (h)VHE case mrs x0, cpacr_el1 // Disable SVE traps - orr x0, x0, #(CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN) + orr x0, x0, #CPACR_ELx_ZEN msr cpacr_el1, x0 b .Lskip_set_cptr_\@ @@ -298,7 +298,7 @@ // (h)VHE case mrs x0, cpacr_el1 // Disable SME traps - orr x0, x0, #(CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN) + orr x0, x0, #CPACR_ELx_SMEN msr cpacr_el1, x0 b .Lskip_set_cptr_sme_\@ diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index e01bb5ca13b7cce66ed0fc6d43a5a8edecc81413..b2adc2c6c82a545551064bad4079c485cc0a5084 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -305,6 +305,12 @@ GENMASK(19, 14) | \ BIT(11)) +#define CPTR_VHE_EL2_RES0 (GENMASK(63, 32) | \ + GENMASK(27, 26) | \ + GENMASK(23, 22) | \ + GENMASK(19, 18) | \ + GENMASK(15, 0)) + /* Hyp Debug Configuration Register bits */ #define MDCR_EL2_E2TB_MASK (UL(0x3)) #define MDCR_EL2_E2TB_SHIFT (UL(24)) diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index 501e3e019c930c77488331af26ba7007e9254b70..21650e7924d45846dacc0c99344d98c03f0eb574 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -557,6 +557,68 @@ static __always_inline void kvm_incr_pc(struct kvm_vcpu *vcpu) vcpu_set_flag((v), e); \ } while (0) +#define __build_check_all_or_none(r, bits) \ + BUILD_BUG_ON(((r) & (bits)) && ((r) & (bits)) != (bits)) + +#define __cpacr_to_cptr_clr(clr, set) \ + ({ \ + u64 cptr = 0; \ + \ + if ((set) & CPACR_ELx_FPEN) \ + cptr |= CPTR_EL2_TFP; \ + if ((set) & CPACR_ELx_ZEN) \ + cptr |= CPTR_EL2_TZ; \ + if ((set) & CPACR_ELx_SMEN) \ + cptr |= CPTR_EL2_TSM; \ + if ((clr) & CPACR_ELx_TTA) \ + cptr |= CPTR_EL2_TTA; \ + if ((clr) & CPTR_EL2_TAM) \ + cptr |= CPTR_EL2_TAM; \ + if ((clr) & CPTR_EL2_TCPAC) \ + cptr |= CPTR_EL2_TCPAC; \ + \ + cptr; \ + }) + +#define __cpacr_to_cptr_set(clr, set) \ + ({ \ + u64 cptr = 0; \ + \ + if ((clr) & CPACR_ELx_FPEN) \ + cptr |= CPTR_EL2_TFP; \ + if ((clr) & CPACR_ELx_ZEN) \ + cptr |= CPTR_EL2_TZ; \ + if ((clr) & CPACR_ELx_SMEN) \ + cptr |= CPTR_EL2_TSM; \ + if ((set) & CPACR_ELx_TTA) \ + cptr |= CPTR_EL2_TTA; \ + if ((set) & CPTR_EL2_TAM) \ + cptr |= CPTR_EL2_TAM; \ + if ((set) & CPTR_EL2_TCPAC) \ + cptr |= CPTR_EL2_TCPAC; \ + \ + cptr; \ + }) + +#define cpacr_clear_set(clr, set) \ + do { \ + BUILD_BUG_ON((set) & CPTR_VHE_EL2_RES0); \ + BUILD_BUG_ON((clr) & CPACR_ELx_E0POE); \ + __build_check_all_or_none((clr), CPACR_ELx_FPEN); \ + __build_check_all_or_none((set), CPACR_ELx_FPEN); \ + __build_check_all_or_none((clr), CPACR_ELx_ZEN); \ + __build_check_all_or_none((set), CPACR_ELx_ZEN); \ + __build_check_all_or_none((clr), CPACR_ELx_SMEN); \ + __build_check_all_or_none((set), CPACR_ELx_SMEN); \ + \ + if (has_vhe() || has_hvhe()) \ + sysreg_clear_set(cpacr_el1, clr, set); \ + else \ + sysreg_clear_set(cptr_el2, \ + __cpacr_to_cptr_clr(clr, set), \ + __cpacr_to_cptr_set(clr, set));\ + } while (0) + static __always_inline void kvm_write_cptr_el2(u64 val) { if (has_vhe() || has_hvhe()) @@ -570,17 +632,16 @@ static __always_inline u64 kvm_get_reset_cptr_el2(struct kvm_vcpu *vcpu) u64 val; if (has_vhe()) { - val = (CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN | - CPACR_EL1_ZEN_EL1EN); + val = (CPACR_ELx_FPEN | CPACR_EL1_ZEN_EL1EN); if (cpus_have_final_cap(ARM64_SME)) val |= CPACR_EL1_SMEN_EL1EN; } else if (has_hvhe()) { - val = (CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN); + val = CPACR_ELx_FPEN; if (!vcpu_has_sve(vcpu) || !guest_owns_fp_regs()) - val |= CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN; + val |= CPACR_ELx_ZEN; if (cpus_have_final_cap(ARM64_SME)) - val |= CPACR_EL1_SMEN_EL1EN | CPACR_EL1_SMEN_EL0EN; + val |= CPACR_ELx_SMEN; } else { val = CPTR_NVHE_EL2_RES1; diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 8170c04fde914c8d09f299f82c9492c8a2afea5b..36b8e97bf49ec4e742f58f1f8cccccf2ed015666 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -76,6 +76,7 @@ static inline enum kvm_mode kvm_get_mode(void) { return KVM_MODE_NONE; }; DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); extern unsigned int __ro_after_init kvm_sve_max_vl; +extern unsigned int __ro_after_init kvm_host_sve_max_vl; int __init kvm_arm_init_sve(void); u32 __attribute_const__ kvm_target_cpu(void); @@ -521,6 +522,20 @@ struct kvm_cpu_context { u64 *vncr_array; }; +struct cpu_sve_state { + __u64 zcr_el1; + + /* + * Ordering is important since __sve_save_state/__sve_restore_state + * relies on it. + */ + __u32 fpsr; + __u32 fpcr; + + /* Must be SVE_VQ_BYTES (128 bit) aligned. */ + __u8 sve_regs[]; +}; + /* * This structure is instantiated on a per-CPU basis, and contains * data that is: @@ -534,7 +549,15 @@ struct kvm_cpu_context { */ struct kvm_host_data { struct kvm_cpu_context host_ctxt; - struct user_fpsimd_state *fpsimd_state; /* hyp VA */ + + /* + * All pointers in this union are hyp VA. + * sve_state is only used in pKVM and if system_supports_sve(). + */ + union { + struct user_fpsimd_state *fpsimd_state; + struct cpu_sve_state *sve_state; + }; /* Ownership of the FP regs */ enum { diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 3e80464f8953172703f01c3cc2202ddff75ecede..b05bceca33850d454c450ae84c9c973c872a1ef7 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -111,7 +111,8 @@ void __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu); void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); -void __sve_restore_state(void *sve_pffr, u32 *fpsr); +void __sve_save_state(void *sve_pffr, u32 *fpsr, int save_ffr); +void __sve_restore_state(void *sve_pffr, u32 *fpsr, int restore_ffr); u64 __guest_enter(struct kvm_vcpu *vcpu); @@ -142,5 +143,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val); extern unsigned long kvm_nvhe_sym(__icache_flags); extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits); +extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl); #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h index ad9cfb5c1ff4e6b9e8352331d1be46afc732a81e..cd56acd9a842ccfaaa486cb4fef22e3777b90172 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -128,4 +128,13 @@ static inline unsigned long hyp_ffa_proxy_pages(void) return (2 * KVM_FFA_MBOX_NR_PAGES) + DIV_ROUND_UP(desc_max, PAGE_SIZE); } +static inline size_t pkvm_host_sve_state_size(void) +{ + if (!system_supports_sve()) + return 0; + + return size_add(sizeof(struct cpu_sve_state), + SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_vl))); +} + #endif /* __ARM64_KVM_PKVM_H__ */ diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 9996a989b52e8742566ff6a8fa978981e2a91d49..59716789fe0f38186cd1700f92757b005f48d1cb 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1931,6 +1931,11 @@ static unsigned long nvhe_percpu_order(void) return size ? get_order(size) : 0; } +static size_t pkvm_host_sve_state_order(void) +{ + return get_order(pkvm_host_sve_state_size()); +} + /* A lookup table holding the hypervisor VA for each vector slot */ static void *hyp_spectre_vector_selector[BP_HARDEN_EL2_SLOTS]; @@ -2310,12 +2315,20 @@ static void __init teardown_subsystems(void) static void __init teardown_hyp_mode(void) { + bool free_sve = system_supports_sve() && is_protected_kvm_enabled(); int cpu; free_hyp_pgds(); for_each_possible_cpu(cpu) { free_page(per_cpu(kvm_arm_hyp_stack_page, cpu)); free_pages(kvm_nvhe_sym(kvm_arm_hyp_percpu_base)[cpu], nvhe_percpu_order()); + + if (free_sve) { + struct cpu_sve_state *sve_state; + + sve_state = per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->sve_state; + free_pages((unsigned long) sve_state, pkvm_host_sve_state_order()); + } } } @@ -2398,6 +2411,58 @@ static int __init kvm_hyp_init_protection(u32 hyp_va_bits) return 0; } +static int init_pkvm_host_sve_state(void) +{ + int cpu; + + if (!system_supports_sve()) + return 0; + + /* Allocate pages for host sve state in protected mode. */ + for_each_possible_cpu(cpu) { + struct page *page = alloc_pages(GFP_KERNEL, pkvm_host_sve_state_order()); + + if (!page) + return -ENOMEM; + + per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->sve_state = page_address(page); + } + + /* + * Don't map the pages in hyp since these are only used in protected + * mode, which will (re)create its own mapping when initialized. + */ + + return 0; +} + +/* + * Finalizes the initialization of hyp mode, once everything else is initialized + * and the initialziation process cannot fail. + */ +static void finalize_init_hyp_mode(void) +{ + int cpu; + + if (system_supports_sve() && is_protected_kvm_enabled()) { + for_each_possible_cpu(cpu) { + struct cpu_sve_state *sve_state; + + sve_state = per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->sve_state; + per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->sve_state = + kern_hyp_va(sve_state); + } + } else { + for_each_possible_cpu(cpu) { + struct user_fpsimd_state *fpsimd_state; + + fpsimd_state = &per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->host_ctxt.fp_regs; + per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->fpsimd_state = + kern_hyp_va(fpsimd_state); + } + } +} + static void pkvm_hyp_init_ptrauth(void) { struct kvm_cpu_context *hyp_ctxt; @@ -2566,6 +2631,10 @@ static int __init init_hyp_mode(void) goto out_err; } + err = init_pkvm_host_sve_state(); + if (err) + goto out_err; + err = kvm_hyp_init_protection(hyp_va_bits); if (err) { kvm_err("Failed to init hyp memory protection\n"); @@ -2730,6 +2799,13 @@ static __init int kvm_arm_init(void) if (err) goto out_subs; + /* + * This should be called after initialization is done and failure isn't + * possible anymore. + */ + if (!in_hyp_mode) + finalize_init_hyp_mode(); + kvm_arm_initialised = true; return 0; diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c index 72d733c74a382f264e48394c6cb88f0fe9f25f45..54090967a33567008da56b9fbef6dd6711756a09 100644 --- a/arch/arm64/kvm/emulate-nested.c +++ b/arch/arm64/kvm/emulate-nested.c @@ -2181,16 +2181,23 @@ void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu) if (forward_traps(vcpu, HCR_NV)) return; + spsr = vcpu_read_sys_reg(vcpu, SPSR_EL2); + spsr = kvm_check_illegal_exception_return(vcpu, spsr); + /* Check for an ERETAx */ esr = kvm_vcpu_get_esr(vcpu); if (esr_iss_is_eretax(esr) && !kvm_auth_eretax(vcpu, &elr)) { /* - * Oh no, ERETAx failed to authenticate. If we have - * FPACCOMBINE, deliver an exception right away. If we - * don't, then let the mangled ELR value trickle down the + * Oh no, ERETAx failed to authenticate. + * + * If we have FPACCOMBINE and we don't have a pending + * Illegal Execution State exception (which has priority + * over FPAC), deliver an exception right away. + * + * Otherwise, let the mangled ELR value trickle down the * ERET handling, and the guest will have a little surprise. */ - if (kvm_has_pauth(vcpu->kvm, FPACCOMBINE)) { + if (kvm_has_pauth(vcpu->kvm, FPACCOMBINE) && !(spsr & PSR_IL_BIT)) { esr &= ESR_ELx_ERET_ISS_ERETA; esr |= FIELD_PREP(ESR_ELx_EC_MASK, ESR_ELx_EC_FPAC); kvm_inject_nested_sync(vcpu, esr); @@ -2201,17 +2208,11 @@ void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu) preempt_disable(); kvm_arch_vcpu_put(vcpu); - spsr = __vcpu_sys_reg(vcpu, SPSR_EL2); - spsr = kvm_check_illegal_exception_return(vcpu, spsr); if (!esr_iss_is_eretax(esr)) elr = __vcpu_sys_reg(vcpu, ELR_EL2); trace_kvm_nested_eret(vcpu, elr, spsr); - /* - * Note that the current exception level is always the virtual EL2, - * since we set HCR_EL2.NV bit only when entering the virtual EL2. - */ *vcpu_pc(vcpu) = elr; *vcpu_cpsr(vcpu) = spsr; diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 1807d3a79a8af8dbc22a5c34f4e289d3359b90f2..521b32868d0d26e7ea802b918aea55cdb407f761 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -90,6 +90,13 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu) fpsimd_save_and_flush_cpu_state(); } } + + /* + * If normal guests gain SME support, maintain this behavior for pKVM + * guests, which don't support SME. + */ + WARN_ON(is_protected_kvm_enabled() && system_supports_sme() && + read_sysreg_s(SYS_SVCR)); } /* @@ -161,9 +168,7 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu) if (has_vhe() && system_supports_sme()) { /* Also restore EL0 state seen on entry */ if (vcpu_get_flag(vcpu, HOST_SME_ENABLED)) - sysreg_clear_set(CPACR_EL1, 0, - CPACR_EL1_SMEN_EL0EN | - CPACR_EL1_SMEN_EL1EN); + sysreg_clear_set(CPACR_EL1, 0, CPACR_ELx_SMEN); else sysreg_clear_set(CPACR_EL1, CPACR_EL1_SMEN_EL0EN, diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index e2f762d959bb3325173ae05bd9ae93ad5c308cef..11098eb7eb44af659bda1c9a9b2d047ff143218e 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -251,6 +251,7 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) case PSR_AA32_MODE_SVC: case PSR_AA32_MODE_ABT: case PSR_AA32_MODE_UND: + case PSR_AA32_MODE_SYS: if (!vcpu_el1_is_32bit(vcpu)) return -EINVAL; break; @@ -276,7 +277,7 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) if (*vcpu_cpsr(vcpu) & PSR_MODE32_BIT) { int i, nr_reg; - switch (*vcpu_cpsr(vcpu)) { + switch (*vcpu_cpsr(vcpu) & PSR_AA32_MODE_MASK) { /* * Either we are dealing with user mode, and only the * first 15 registers (+ PC) must be narrowed to 32bit. diff --git a/arch/arm64/kvm/hyp/aarch32.c b/arch/arm64/kvm/hyp/aarch32.c index 8d9670e6615dc8767e3b1524e04fcd96ae6f63c8..449fa58cf3b63b1cc3d990b76e3d07d4180dc06b 100644 --- a/arch/arm64/kvm/hyp/aarch32.c +++ b/arch/arm64/kvm/hyp/aarch32.c @@ -50,9 +50,23 @@ bool kvm_condition_valid32(const struct kvm_vcpu *vcpu) u32 cpsr_cond; int cond; - /* Top two bits non-zero? Unconditional. */ - if (kvm_vcpu_get_esr(vcpu) >> 30) + /* + * These are the exception classes that could fire with a + * conditional instruction. + */ + switch (kvm_vcpu_trap_get_class(vcpu)) { + case ESR_ELx_EC_CP15_32: + case ESR_ELx_EC_CP15_64: + case ESR_ELx_EC_CP14_MR: + case ESR_ELx_EC_CP14_LS: + case ESR_ELx_EC_FP_ASIMD: + case ESR_ELx_EC_CP10_ID: + case ESR_ELx_EC_CP14_64: + case ESR_ELx_EC_SVC32: + break; + default: return true; + } /* Is condition field valid? */ cond = kvm_vcpu_get_condition(vcpu); diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index 61e6f3ba7b7d17b5d62e3eedf834cdfc1494417d..e950875e31cee4df58d041519b7584356463c91b 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -25,3 +25,9 @@ SYM_FUNC_START(__sve_restore_state) sve_load 0, x1, x2, 3 ret SYM_FUNC_END(__sve_restore_state) + +SYM_FUNC_START(__sve_save_state) + mov x2, #1 + sve_save 0, x1, x2, 3 + ret +SYM_FUNC_END(__sve_save_state) diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index a92566f36022e65339bf7bfddd9691a961817a04..0c4de44534b7a8bb183cfffa1a414c9dc6814687 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -316,10 +316,24 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu) { sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); __sve_restore_state(vcpu_sve_pffr(vcpu), - &vcpu->arch.ctxt.fp_regs.fpsr); + &vcpu->arch.ctxt.fp_regs.fpsr, + true); write_sysreg_el1(__vcpu_sys_reg(vcpu, ZCR_EL1), SYS_ZCR); } +static inline void __hyp_sve_save_host(void) +{ + struct cpu_sve_state *sve_state = *host_data_ptr(sve_state); + + sve_state->zcr_el1 = read_sysreg_el1(SYS_ZCR); + write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); + __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + &sve_state->fpsr, + true); +} + +static void kvm_hyp_save_fpsimd_host(struct kvm_vcpu *vcpu); + /* * We trap the first access to the FP/SIMD to save the host context and * restore the guest context lazily. @@ -330,7 +344,6 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code) { bool sve_guest; u8 esr_ec; - u64 reg; if (!system_supports_fpsimd()) return false; @@ -353,24 +366,15 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code) /* Valid trap. Switch the context: */ /* First disable enough traps to allow us to update the registers */ - if (has_vhe() || has_hvhe()) { - reg = CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN; - if (sve_guest) - reg |= CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN; - - sysreg_clear_set(cpacr_el1, 0, reg); - } else { - reg = CPTR_EL2_TFP; - if (sve_guest) - reg |= CPTR_EL2_TZ; - - sysreg_clear_set(cptr_el2, reg, 0); - } + if (sve_guest || (is_protected_kvm_enabled() && system_supports_sve())) + cpacr_clear_set(0, CPACR_ELx_FPEN | CPACR_ELx_ZEN); + else + cpacr_clear_set(0, CPACR_ELx_FPEN); isb(); /* Write out the host state if it's in the registers */ if (host_owns_fp_regs()) - __fpsimd_save_state(*host_data_ptr(fpsimd_state)); + kvm_hyp_save_fpsimd_host(vcpu); /* Restore the guest state */ if (sve_guest) diff --git a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h index 22f374e9f532968937fd179c6732e458cffeceee..24a9a8330d190396e90abc7202e6b7e86ea1de63 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h @@ -59,7 +59,6 @@ static inline bool pkvm_hyp_vcpu_is_protected(struct pkvm_hyp_vcpu *hyp_vcpu) } void pkvm_hyp_vm_table_init(void *tbl); -void pkvm_host_fpsimd_state_init(void); int __pkvm_init_vm(struct kvm *host_kvm, unsigned long vm_hva, unsigned long pgd_hva); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index d5c48dc98f67947b9c1cd1dee7991594dfb74e87..f43d845f3c4ecdc41c1c661f8bbbb8165678897b 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -23,20 +23,80 @@ DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); void __kvm_hyp_host_forward_smc(struct kvm_cpu_context *host_ctxt); +static void __hyp_sve_save_guest(struct kvm_vcpu *vcpu) +{ + __vcpu_sys_reg(vcpu, ZCR_EL1) = read_sysreg_el1(SYS_ZCR); + /* + * On saving/restoring guest sve state, always use the maximum VL for + * the guest. The layout of the data when saving the sve state depends + * on the VL, so use a consistent (i.e., the maximum) guest VL. + */ + sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); + __sve_save_state(vcpu_sve_pffr(vcpu), &vcpu->arch.ctxt.fp_regs.fpsr, true); + write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); +} + +static void __hyp_sve_restore_host(void) +{ + struct cpu_sve_state *sve_state = *host_data_ptr(sve_state); + + /* + * On saving/restoring host sve state, always use the maximum VL for + * the host. The layout of the data when saving the sve state depends + * on the VL, so use a consistent (i.e., the maximum) host VL. + * + * Setting ZCR_EL2 to ZCR_ELx_LEN_MASK sets the effective length + * supported by the system (or limited at EL3). + */ + write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); + __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + &sve_state->fpsr, + true); + write_sysreg_el1(sve_state->zcr_el1, SYS_ZCR); +} + +static void fpsimd_sve_flush(void) +{ + *host_data_ptr(fp_owner) = FP_STATE_HOST_OWNED; +} + +static void fpsimd_sve_sync(struct kvm_vcpu *vcpu) +{ + if (!guest_owns_fp_regs()) + return; + + cpacr_clear_set(0, CPACR_ELx_FPEN | CPACR_ELx_ZEN); + isb(); + + if (vcpu_has_sve(vcpu)) + __hyp_sve_save_guest(vcpu); + else + __fpsimd_save_state(&vcpu->arch.ctxt.fp_regs); + + if (system_supports_sve()) + __hyp_sve_restore_host(); + else + __fpsimd_restore_state(*host_data_ptr(fpsimd_state)); + + *host_data_ptr(fp_owner) = FP_STATE_HOST_OWNED; +} + static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) { struct kvm_vcpu *host_vcpu = hyp_vcpu->host_vcpu; + fpsimd_sve_flush(); + hyp_vcpu->vcpu.arch.ctxt = host_vcpu->arch.ctxt; hyp_vcpu->vcpu.arch.sve_state = kern_hyp_va(host_vcpu->arch.sve_state); - hyp_vcpu->vcpu.arch.sve_max_vl = host_vcpu->arch.sve_max_vl; + /* Limit guest vector length to the maximum supported by the host. */ + hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_vl); hyp_vcpu->vcpu.arch.hw_mmu = host_vcpu->arch.hw_mmu; hyp_vcpu->vcpu.arch.hcr_el2 = host_vcpu->arch.hcr_el2; hyp_vcpu->vcpu.arch.mdcr_el2 = host_vcpu->arch.mdcr_el2; - hyp_vcpu->vcpu.arch.cptr_el2 = host_vcpu->arch.cptr_el2; hyp_vcpu->vcpu.arch.iflags = host_vcpu->arch.iflags; @@ -54,10 +114,11 @@ static void sync_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) struct vgic_v3_cpu_if *host_cpu_if = &host_vcpu->arch.vgic_cpu.vgic_v3; unsigned int i; + fpsimd_sve_sync(&hyp_vcpu->vcpu); + host_vcpu->arch.ctxt = hyp_vcpu->vcpu.arch.ctxt; host_vcpu->arch.hcr_el2 = hyp_vcpu->vcpu.arch.hcr_el2; - host_vcpu->arch.cptr_el2 = hyp_vcpu->vcpu.arch.cptr_el2; host_vcpu->arch.fault = hyp_vcpu->vcpu.arch.fault; @@ -79,6 +140,17 @@ static void handle___kvm_vcpu_run(struct kvm_cpu_context *host_ctxt) struct pkvm_hyp_vcpu *hyp_vcpu; struct kvm *host_kvm; + /* + * KVM (and pKVM) doesn't support SME guests for now, and + * ensures that SME features aren't enabled in pstate when + * loading a vcpu. Therefore, if SME features enabled the host + * is misbehaving. + */ + if (unlikely(system_supports_sme() && read_sysreg_s(SYS_SVCR))) { + ret = -EINVAL; + goto out; + } + host_kvm = kern_hyp_va(host_vcpu->kvm); hyp_vcpu = pkvm_load_hyp_vcpu(host_kvm->arch.pkvm.handle, host_vcpu->vcpu_idx); @@ -405,11 +477,7 @@ void handle_trap(struct kvm_cpu_context *host_ctxt) handle_host_smc(host_ctxt); break; case ESR_ELx_EC_SVE: - if (has_hvhe()) - sysreg_clear_set(cpacr_el1, 0, (CPACR_EL1_ZEN_EL1EN | - CPACR_EL1_ZEN_EL0EN)); - else - sysreg_clear_set(cptr_el2, CPTR_EL2_TZ, 0); + cpacr_clear_set(0, CPACR_ELx_ZEN); isb(); sve_cond_update_zcr_vq(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); break; diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 16aa4875ddb8c05c1c5761f264e1c2826ae6d1bc..95cf1857425175dd1779442c5c76f17c2b62d766 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -18,6 +18,8 @@ unsigned long __icache_flags; /* Used by kvm_get_vttbr(). */ unsigned int kvm_arm_vmid_bits; +unsigned int kvm_host_sve_max_vl; + /* * Set trap register values based on features in ID_AA64PFR0. */ @@ -63,7 +65,7 @@ static void pvm_init_traps_aa64pfr0(struct kvm_vcpu *vcpu) /* Trap SVE */ if (!FIELD_GET(ARM64_FEATURE_MASK(ID_AA64PFR0_EL1_SVE), feature_ids)) { if (has_hvhe()) - cptr_clear |= CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN; + cptr_clear |= CPACR_ELx_ZEN; else cptr_set |= CPTR_EL2_TZ; } @@ -247,17 +249,6 @@ void pkvm_hyp_vm_table_init(void *tbl) vm_table = tbl; } -void pkvm_host_fpsimd_state_init(void) -{ - unsigned long i; - - for (i = 0; i < hyp_nr_cpus; i++) { - struct kvm_host_data *host_data = per_cpu_ptr(&kvm_host_data, i); - - host_data->fpsimd_state = &host_data->host_ctxt.fp_regs; - } -} - /* * Return the hyp vm structure corresponding to the handle. */ @@ -586,6 +577,8 @@ unlock: if (ret) unmap_donated_memory(hyp_vcpu, sizeof(*hyp_vcpu)); + hyp_vcpu->vcpu.arch.cptr_el2 = kvm_get_reset_cptr_el2(&hyp_vcpu->vcpu); + return ret; } diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 859f22f754d373a8d74cbd613f3d2e62e009a211..f4350ba07b0b0c37440af0783d313108e1cc0a92 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -67,6 +67,28 @@ static int divide_memory_pool(void *virt, unsigned long size) return 0; } +static int pkvm_create_host_sve_mappings(void) +{ + void *start, *end; + int ret, i; + + if (!system_supports_sve()) + return 0; + + for (i = 0; i < hyp_nr_cpus; i++) { + struct kvm_host_data *host_data = per_cpu_ptr(&kvm_host_data, i); + struct cpu_sve_state *sve_state = host_data->sve_state; + + start = kern_hyp_va(sve_state); + end = start + PAGE_ALIGN(pkvm_host_sve_state_size()); + ret = pkvm_create_mappings(start, end, PAGE_HYP); + if (ret) + return ret; + } + + return 0; +} + static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, unsigned long *per_cpu_base, u32 hyp_va_bits) @@ -125,6 +147,8 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, return ret; } + pkvm_create_host_sve_mappings(); + /* * Map the host sections RO in the hypervisor, but transfer the * ownership from the host to the hypervisor itself to make sure they @@ -300,7 +324,6 @@ void __noreturn __pkvm_init_finalise(void) goto out; pkvm_hyp_vm_table_init(vm_table_base); - pkvm_host_fpsimd_state_init(); out: /* * We tail-called to here from handle___pkvm_init() and will not return, diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 6758cd90557061810c2d9baf8bf29d04a8a0eb05..6af179c6356d666b68e426429572ce6d559bbd82 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -48,15 +48,14 @@ static void __activate_traps(struct kvm_vcpu *vcpu) val |= has_hvhe() ? CPACR_EL1_TTA : CPTR_EL2_TTA; if (cpus_have_final_cap(ARM64_SME)) { if (has_hvhe()) - val &= ~(CPACR_EL1_SMEN_EL1EN | CPACR_EL1_SMEN_EL0EN); + val &= ~CPACR_ELx_SMEN; else val |= CPTR_EL2_TSM; } if (!guest_owns_fp_regs()) { if (has_hvhe()) - val &= ~(CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN | - CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN); + val &= ~(CPACR_ELx_FPEN | CPACR_ELx_ZEN); else val |= CPTR_EL2_TFP | CPTR_EL2_TZ; @@ -182,6 +181,25 @@ static bool kvm_handle_pvm_sys64(struct kvm_vcpu *vcpu, u64 *exit_code) kvm_handle_pvm_sysreg(vcpu, exit_code)); } +static void kvm_hyp_save_fpsimd_host(struct kvm_vcpu *vcpu) +{ + /* + * Non-protected kvm relies on the host restoring its sve state. + * Protected kvm restores the host's sve state as not to reveal that + * fpsimd was used by a guest nor leak upper sve bits. + */ + if (unlikely(is_protected_kvm_enabled() && system_supports_sve())) { + __hyp_sve_save_host(); + + /* Re-enable SVE traps if not supported for the guest vcpu. */ + if (!vcpu_has_sve(vcpu)) + cpacr_clear_set(CPACR_ELx_ZEN, 0); + + } else { + __fpsimd_save_state(*host_data_ptr(fpsimd_state)); + } +} + static const exit_handler_fn hyp_exit_handlers[] = { [0 ... ESR_ELx_EC_MAX] = NULL, [ESR_ELx_EC_CP15_32] = kvm_hyp_handle_cp15_32, diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index d7af5f46f22a3ec167c868f89181f9d91178e7f7..8fbb6a2e0559d7e7d8d90ce11b618f35a723c46e 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -93,8 +93,7 @@ static void __activate_traps(struct kvm_vcpu *vcpu) val = read_sysreg(cpacr_el1); val |= CPACR_ELx_TTA; - val &= ~(CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN | - CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN); + val &= ~(CPACR_ELx_ZEN | CPACR_ELx_SMEN); /* * With VHE (HCR.E2H == 1), accesses to CPACR_EL1 are routed to @@ -109,9 +108,9 @@ static void __activate_traps(struct kvm_vcpu *vcpu) if (guest_owns_fp_regs()) { if (vcpu_has_sve(vcpu)) - val |= CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN; + val |= CPACR_ELx_ZEN; } else { - val &= ~(CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN); + val &= ~CPACR_ELx_FPEN; __activate_traps_fpsimd32(vcpu); } @@ -262,6 +261,11 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code) return true; } +static void kvm_hyp_save_fpsimd_host(struct kvm_vcpu *vcpu) +{ + __fpsimd_save_state(*host_data_ptr(fpsimd_state)); +} + static const exit_handler_fn hyp_exit_handlers[] = { [0 ... ESR_ELx_EC_MAX] = NULL, [ESR_ELx_EC_CP15_32] = kvm_hyp_handle_cp15_32, diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c index 6813c7c7f00ab2eedf9d04a9a84bfb5f4858ca77..bae8536cbf003e9b949db4df115f0a7815e55554 100644 --- a/arch/arm64/kvm/nested.c +++ b/arch/arm64/kvm/nested.c @@ -58,8 +58,10 @@ static u64 limit_nv_id_reg(u32 id, u64 val) break; case SYS_ID_AA64PFR1_EL1: - /* Only support SSBS */ - val &= NV_FTR(PFR1, SSBS); + /* Only support BTI, SSBS, CSV2_frac */ + val &= (NV_FTR(PFR1, BT) | + NV_FTR(PFR1, SSBS) | + NV_FTR(PFR1, CSV2_frac)); break; case SYS_ID_AA64MMFR0_EL1: diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 1b7b58cb121f98c3746da33c2d973ef49a428500..3fc8ca164dbe4e8f3da73068a240164ecc91bd17 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -32,6 +32,7 @@ /* Maximum phys_shift supported for any VM on this host */ static u32 __ro_after_init kvm_ipa_limit; +unsigned int __ro_after_init kvm_host_sve_max_vl; /* * ARMv8 Reset Values @@ -51,6 +52,8 @@ int __init kvm_arm_init_sve(void) { if (system_supports_sve()) { kvm_sve_max_vl = sve_max_virtualisable_vl(); + kvm_host_sve_max_vl = sve_max_vl(); + kvm_nvhe_sym(kvm_host_sve_max_vl) = kvm_host_sve_max_vl; /* * The get_sve_reg()/set_sve_reg() ioctl interface will need diff --git a/arch/loongarch/boot/dts/loongson-2k0500-ref.dts b/arch/loongarch/boot/dts/loongson-2k0500-ref.dts index 8aefb0c126722980a345062cae02a6127c02b52e..a34734a6c3ce802f0b735c2689212a459a6372f2 100644 --- a/arch/loongarch/boot/dts/loongson-2k0500-ref.dts +++ b/arch/loongarch/boot/dts/loongson-2k0500-ref.dts @@ -44,14 +44,14 @@ &gmac0 { status = "okay"; - phy-mode = "rgmii"; + phy-mode = "rgmii-id"; bus_id = <0x0>; }; &gmac1 { status = "okay"; - phy-mode = "rgmii"; + phy-mode = "rgmii-id"; bus_id = <0x1>; }; diff --git a/arch/loongarch/boot/dts/loongson-2k1000-ref.dts b/arch/loongarch/boot/dts/loongson-2k1000-ref.dts index 8463fe035386e40daa112740fd9804af6195fb37..23cf26cc3e5f19703220bd20a8e12e1e61a1e941 100644 --- a/arch/loongarch/boot/dts/loongson-2k1000-ref.dts +++ b/arch/loongarch/boot/dts/loongson-2k1000-ref.dts @@ -43,7 +43,7 @@ &gmac0 { status = "okay"; - phy-mode = "rgmii"; + phy-mode = "rgmii-id"; phy-handle = <&phy0>; mdio { compatible = "snps,dwmac-mdio"; @@ -58,7 +58,7 @@ &gmac1 { status = "okay"; - phy-mode = "rgmii"; + phy-mode = "rgmii-id"; phy-handle = <&phy1>; mdio { compatible = "snps,dwmac-mdio"; diff --git a/arch/loongarch/boot/dts/loongson-2k2000-ref.dts b/arch/loongarch/boot/dts/loongson-2k2000-ref.dts index 74b99bd234cc38df9a087915280e86ddb5bd56d4..ea9e6985d0e9fca980eaad34ca4089b6164ad447 100644 --- a/arch/loongarch/boot/dts/loongson-2k2000-ref.dts +++ b/arch/loongarch/boot/dts/loongson-2k2000-ref.dts @@ -92,7 +92,7 @@ &gmac2 { status = "okay"; - phy-mode = "rgmii"; + phy-mode = "rgmii-id"; phy-handle = <&phy2>; mdio { compatible = "snps,dwmac-mdio"; diff --git a/arch/loongarch/include/asm/numa.h b/arch/loongarch/include/asm/numa.h index 27f319b498625718d2349618d39008327b2f9935..b5f9de9f102e444bcb4e563bddc704f52f3a940e 100644 --- a/arch/loongarch/include/asm/numa.h +++ b/arch/loongarch/include/asm/numa.h @@ -56,6 +56,7 @@ extern int early_cpu_to_node(int cpu); static inline void early_numa_add_cpu(int cpuid, s16 node) { } static inline void numa_add_cpu(unsigned int cpu) { } static inline void numa_remove_cpu(unsigned int cpu) { } +static inline void set_cpuid_to_node(int cpuid, s16 node) { } static inline int early_cpu_to_node(int cpu) { diff --git a/arch/loongarch/include/asm/stackframe.h b/arch/loongarch/include/asm/stackframe.h index 45b507a7b06fca8af6e16b36a2a1c3f538368eb2..d9eafd3ee3d1e848675f54f033163ecc66e07d72 100644 --- a/arch/loongarch/include/asm/stackframe.h +++ b/arch/loongarch/include/asm/stackframe.h @@ -42,7 +42,7 @@ .macro JUMP_VIRT_ADDR temp1 temp2 li.d \temp1, CACHE_BASE pcaddi \temp2, 0 - or \temp1, \temp1, \temp2 + bstrins.d \temp1, \temp2, (DMW_PABITS - 1), 0 jirl zero, \temp1, 0xc .endm diff --git a/arch/loongarch/kernel/head.S b/arch/loongarch/kernel/head.S index c4f7de2e28054ceb6458c964c26e86596cea8602..4677ea8fa8e98cad8b11b5ea89d7811500be738e 100644 --- a/arch/loongarch/kernel/head.S +++ b/arch/loongarch/kernel/head.S @@ -22,7 +22,7 @@ _head: .word MZ_MAGIC /* "MZ", MS-DOS header */ .org 0x8 - .dword kernel_entry /* Kernel entry point */ + .dword _kernel_entry /* Kernel entry point (physical address) */ .dword _kernel_asize /* Kernel image effective size */ .quad PHYS_LINK_KADDR /* Kernel image load offset from start of RAM */ .org 0x38 /* 0x20 ~ 0x37 reserved */ diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c index 60e0fe97f61a31604b095044395c40ba6fd5efc8..3d048f1be1438863831e19817711292558d75149 100644 --- a/arch/loongarch/kernel/setup.c +++ b/arch/loongarch/kernel/setup.c @@ -282,7 +282,7 @@ static void __init fdt_setup(void) return; /* Prefer to use built-in dtb, checking its legality first. */ - if (!fdt_check_header(__dtb_start)) + if (IS_ENABLED(CONFIG_BUILTIN_DTB) && !fdt_check_header(__dtb_start)) fdt_pointer = __dtb_start; else fdt_pointer = efi_fdt_pointer(); /* Fallback to firmware dtb */ @@ -351,10 +351,8 @@ void __init platform_init(void) arch_reserve_vmcore(); arch_reserve_crashkernel(); -#ifdef CONFIG_ACPI_TABLE_UPGRADE - acpi_table_upgrade(); -#endif #ifdef CONFIG_ACPI + acpi_table_upgrade(); acpi_gbl_use_default_register_widths = false; acpi_boot_table_init(); #endif diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c index 0dfe2388ef413b673185ab88951b06c532559fff..1436d2465939b2d79744833c8c5a4abe3ead8916 100644 --- a/arch/loongarch/kernel/smp.c +++ b/arch/loongarch/kernel/smp.c @@ -273,7 +273,6 @@ static void __init fdt_smp_setup(void) if (cpuid == loongson_sysconf.boot_cpu_id) { cpu = 0; - numa_add_cpu(cpu); } else { cpu = cpumask_next_zero(-1, cpu_present_mask); } @@ -283,6 +282,9 @@ static void __init fdt_smp_setup(void) set_cpu_present(cpu, true); __cpu_number_map[cpuid] = cpu; __cpu_logical_map[cpu] = cpuid; + + early_numa_add_cpu(cpu, 0); + set_cpuid_to_node(cpuid, 0); } loongson_sysconf.nr_cpus = num_processors; @@ -468,6 +470,7 @@ void smp_prepare_boot_cpu(void) set_cpu_possible(0, true); set_cpu_online(0, true); set_my_cpu_offset(per_cpu_offset(0)); + numa_add_cpu(0); rr_node = first_node(node_online_map); for_each_possible_cpu(cpu) { diff --git a/arch/loongarch/kernel/vmlinux.lds.S b/arch/loongarch/kernel/vmlinux.lds.S index e8e97dbf9ca40ff243e33f1d2cfa9fdbb59d54b4..3c7595342730ed2c1e76c343e16553180fbd167d 100644 --- a/arch/loongarch/kernel/vmlinux.lds.S +++ b/arch/loongarch/kernel/vmlinux.lds.S @@ -6,6 +6,7 @@ #define PAGE_SIZE _PAGE_SIZE #define RO_EXCEPTION_TABLE_ALIGN 4 +#define PHYSADDR_MASK 0xffffffffffff /* 48-bit */ /* * Put .bss..swapper_pg_dir as the first thing in .bss. This will @@ -142,10 +143,11 @@ SECTIONS #ifdef CONFIG_EFI_STUB /* header symbols */ - _kernel_asize = _end - _text; - _kernel_fsize = _edata - _text; - _kernel_vsize = _end - __initdata_begin; - _kernel_rsize = _edata - __initdata_begin; + _kernel_entry = ABSOLUTE(kernel_entry & PHYSADDR_MASK); + _kernel_asize = ABSOLUTE(_end - _text); + _kernel_fsize = ABSOLUTE(_edata - _text); + _kernel_vsize = ABSOLUTE(_end - __initdata_begin); + _kernel_rsize = ABSOLUTE(_edata - __initdata_begin); #endif .gptab.sdata : { diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 3c968f2f4ac44774398e76cf132698e0f18677e4..c88c6d46a5bc0127e29fa2179b58d8cb0409f149 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -137,7 +137,7 @@ config PPC select ARCH_HAS_GCOV_PROFILE_ALL select ARCH_HAS_HUGEPD if HUGETLB_PAGE select ARCH_HAS_KCOV - select ARCH_HAS_KERNEL_FPU_SUPPORT if PPC_FPU + select ARCH_HAS_KERNEL_FPU_SUPPORT if PPC64 && PPC_FPU select ARCH_HAS_MEMBARRIER_CALLBACKS select ARCH_HAS_MEMBARRIER_SYNC_CORE select ARCH_HAS_MEMREMAP_COMPAT_ALIGN if PPC_64S_HASH_MMU diff --git a/arch/powerpc/include/asm/uaccess.h b/arch/powerpc/include/asm/uaccess.h index de10437fd20652ee63a6d214638bded13cdbc6c3..fd594bf6c6a9c5e4a80c9dd5933cd5d1797a326e 100644 --- a/arch/powerpc/include/asm/uaccess.h +++ b/arch/powerpc/include/asm/uaccess.h @@ -92,9 +92,25 @@ __pu_failed: \ : label) #endif +#ifdef CONFIG_CC_IS_CLANG +#define DS_FORM_CONSTRAINT "Z<>" +#else +#define DS_FORM_CONSTRAINT "YZ<>" +#endif + #ifdef __powerpc64__ +#ifdef CONFIG_PPC_KERNEL_PREFIXED #define __put_user_asm2_goto(x, ptr, label) \ __put_user_asm_goto(x, ptr, label, "std") +#else +#define __put_user_asm2_goto(x, addr, label) \ + asm goto ("1: std%U1%X1 %0,%1 # put_user\n" \ + EX_TABLE(1b, %l2) \ + : \ + : "r" (x), DS_FORM_CONSTRAINT (*addr) \ + : \ + : label) +#endif // CONFIG_PPC_KERNEL_PREFIXED #else /* __powerpc64__ */ #define __put_user_asm2_goto(x, addr, label) \ asm goto( \ @@ -165,8 +181,19 @@ do { \ #endif #ifdef __powerpc64__ +#ifdef CONFIG_PPC_KERNEL_PREFIXED #define __get_user_asm2_goto(x, addr, label) \ __get_user_asm_goto(x, addr, label, "ld") +#else +#define __get_user_asm2_goto(x, addr, label) \ + asm_goto_output( \ + "1: ld%U1%X1 %0, %1 # get_user\n" \ + EX_TABLE(1b, %l2) \ + : "=r" (x) \ + : DS_FORM_CONSTRAINT (*addr) \ + : \ + : label) +#endif // CONFIG_PPC_KERNEL_PREFIXED #else /* __powerpc64__ */ #define __get_user_asm2_goto(x, addr, label) \ asm_goto_output( \ diff --git a/arch/powerpc/net/bpf_jit_comp32.c b/arch/powerpc/net/bpf_jit_comp32.c index 43b97032a91c00a5b9ed8cc56c603c3c71dbc999..a0c4f1bde83e86da7524d56aa8a3408a6cacc647 100644 --- a/arch/powerpc/net/bpf_jit_comp32.c +++ b/arch/powerpc/net/bpf_jit_comp32.c @@ -900,6 +900,15 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code /* Get offset into TMP_REG */ EMIT(PPC_RAW_LI(tmp_reg, off)); + /* + * Enforce full ordering for operations with BPF_FETCH by emitting a 'sync' + * before and after the operation. + * + * This is a requirement in the Linux Kernel Memory Model. + * See __cmpxchg_u32() in asm/cmpxchg.h as an example. + */ + if ((imm & BPF_FETCH) && IS_ENABLED(CONFIG_SMP)) + EMIT(PPC_RAW_SYNC()); tmp_idx = ctx->idx * 4; /* load value from memory into r0 */ EMIT(PPC_RAW_LWARX(_R0, tmp_reg, dst_reg, 0)); @@ -953,6 +962,9 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code /* For the BPF_FETCH variant, get old data into src_reg */ if (imm & BPF_FETCH) { + /* Emit 'sync' to enforce full ordering */ + if (IS_ENABLED(CONFIG_SMP)) + EMIT(PPC_RAW_SYNC()); EMIT(PPC_RAW_MR(ret_reg, ax_reg)); if (!fp->aux->verifier_zext) EMIT(PPC_RAW_LI(ret_reg - 1, 0)); /* higher 32-bit */ diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c index 8afc14a4a1258ebab01c802efdaa1c55cb73fd1b..7703dcf48be86bfd82f08f264a1735509b12cbff 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -846,6 +846,15 @@ emit_clear: /* Get offset into TMP_REG_1 */ EMIT(PPC_RAW_LI(tmp1_reg, off)); + /* + * Enforce full ordering for operations with BPF_FETCH by emitting a 'sync' + * before and after the operation. + * + * This is a requirement in the Linux Kernel Memory Model. + * See __cmpxchg_u64() in asm/cmpxchg.h as an example. + */ + if ((imm & BPF_FETCH) && IS_ENABLED(CONFIG_SMP)) + EMIT(PPC_RAW_SYNC()); tmp_idx = ctx->idx * 4; /* load value from memory into TMP_REG_2 */ if (size == BPF_DW) @@ -908,6 +917,9 @@ emit_clear: PPC_BCC_SHORT(COND_NE, tmp_idx); if (imm & BPF_FETCH) { + /* Emit 'sync' to enforce full ordering */ + if (IS_ENABLED(CONFIG_SMP)) + EMIT(PPC_RAW_SYNC()); EMIT(PPC_RAW_MR(ret_reg, _R0)); /* * Skip unnecessary zero-extension for 32-bit cmpxchg. diff --git a/arch/powerpc/platforms/pseries/lparcfg.c b/arch/powerpc/platforms/pseries/lparcfg.c index 6e7029640c0ca2f1f9aad9ec95104e45680cf93a..62da20f9700a974200c05a9d5de908746f2f151d 100644 --- a/arch/powerpc/platforms/pseries/lparcfg.c +++ b/arch/powerpc/platforms/pseries/lparcfg.c @@ -371,8 +371,8 @@ static int read_dt_lpar_name(struct seq_file *m) static void read_lpar_name(struct seq_file *m) { - if (read_rtas_lpar_name(m) && read_dt_lpar_name(m)) - pr_err_once("Error can't get the LPAR name"); + if (read_rtas_lpar_name(m)) + read_dt_lpar_name(m); } #define SPLPAR_MAXLENGTH 1026*(sizeof(char)) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index b94176e25be1810e749cb94686b9c536fd9a2bb0..9f38a5ecbee366d6428aa692159cf5c63ad09225 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -106,7 +106,7 @@ config RISCV select HAS_IOPORT if MMU select HAVE_ARCH_AUDITSYSCALL select HAVE_ARCH_HUGE_VMALLOC if HAVE_ARCH_HUGE_VMAP - select HAVE_ARCH_HUGE_VMAP if MMU && 64BIT && !XIP_KERNEL + select HAVE_ARCH_HUGE_VMAP if MMU && 64BIT select HAVE_ARCH_JUMP_LABEL if !XIP_KERNEL select HAVE_ARCH_JUMP_LABEL_RELATIVE if !XIP_KERNEL select HAVE_ARCH_KASAN if MMU && 64BIT @@ -610,6 +610,18 @@ config TOOLCHAIN_HAS_VECTOR_CRYPTO def_bool $(as-instr, .option arch$(comma) +v$(comma) +zvkb) depends on AS_HAS_OPTION_ARCH +config RISCV_ISA_ZBA + bool "Zba extension support for bit manipulation instructions" + default y + help + Add support for enabling optimisations in the kernel when the Zba + extension is detected at boot. + + The Zba extension provides instructions to accelerate the generation + of addresses that index into arrays of basic data types. + + If you don't know what to do here, say Y. + config RISCV_ISA_ZBB bool "Zbb extension support for bit manipulation instructions" depends on TOOLCHAIN_HAS_ZBB diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index ddb002ed89dea0dbf4956a5161a0f426bcba557f..808b4c78462e5aa363fc55d7871497efa23d57b4 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -10,7 +10,7 @@ #include -#define __arch_xchg_masked(prepend, append, r, p, n) \ +#define __arch_xchg_masked(sc_sfx, prepend, append, r, p, n) \ ({ \ u32 *__ptr32b = (u32 *)((ulong)(p) & ~0x3); \ ulong __s = ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE; \ @@ -25,7 +25,7 @@ "0: lr.w %0, %2\n" \ " and %1, %0, %z4\n" \ " or %1, %1, %z3\n" \ - " sc.w %1, %1, %2\n" \ + " sc.w" sc_sfx " %1, %1, %2\n" \ " bnez %1, 0b\n" \ append \ : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \ @@ -46,7 +46,8 @@ : "memory"); \ }) -#define _arch_xchg(ptr, new, sfx, prepend, append) \ +#define _arch_xchg(ptr, new, sc_sfx, swap_sfx, prepend, \ + sc_append, swap_append) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ __typeof__(*(__ptr)) __new = (new); \ @@ -55,15 +56,15 @@ switch (sizeof(*__ptr)) { \ case 1: \ case 2: \ - __arch_xchg_masked(prepend, append, \ + __arch_xchg_masked(sc_sfx, prepend, sc_append, \ __ret, __ptr, __new); \ break; \ case 4: \ - __arch_xchg(".w" sfx, prepend, append, \ + __arch_xchg(".w" swap_sfx, prepend, swap_append, \ __ret, __ptr, __new); \ break; \ case 8: \ - __arch_xchg(".d" sfx, prepend, append, \ + __arch_xchg(".d" swap_sfx, prepend, swap_append, \ __ret, __ptr, __new); \ break; \ default: \ @@ -73,16 +74,17 @@ }) #define arch_xchg_relaxed(ptr, x) \ - _arch_xchg(ptr, x, "", "", "") + _arch_xchg(ptr, x, "", "", "", "", "") #define arch_xchg_acquire(ptr, x) \ - _arch_xchg(ptr, x, "", "", RISCV_ACQUIRE_BARRIER) + _arch_xchg(ptr, x, "", "", "", \ + RISCV_ACQUIRE_BARRIER, RISCV_ACQUIRE_BARRIER) #define arch_xchg_release(ptr, x) \ - _arch_xchg(ptr, x, "", RISCV_RELEASE_BARRIER, "") + _arch_xchg(ptr, x, "", "", RISCV_RELEASE_BARRIER, "", "") #define arch_xchg(ptr, x) \ - _arch_xchg(ptr, x, ".aqrl", "", "") + _arch_xchg(ptr, x, ".rl", ".aqrl", "", RISCV_FULL_BARRIER, "") #define xchg32(ptr, x) \ ({ \ diff --git a/arch/riscv/kernel/cpu_ops_sbi.c b/arch/riscv/kernel/cpu_ops_sbi.c index 1cc7df740eddc9a2977fde8c428e19fd90ec9126..e6fbaaf549562d6e9ca63a66371441fe8b230cb3 100644 --- a/arch/riscv/kernel/cpu_ops_sbi.c +++ b/arch/riscv/kernel/cpu_ops_sbi.c @@ -72,7 +72,7 @@ static int sbi_cpu_start(unsigned int cpuid, struct task_struct *tidle) /* Make sure tidle is updated */ smp_mb(); bdata->task_ptr = tidle; - bdata->stack_ptr = task_stack_page(tidle) + THREAD_SIZE; + bdata->stack_ptr = task_pt_regs(tidle); /* Make sure boot data is updated */ smp_mb(); hsm_data = __pa(bdata); diff --git a/arch/riscv/kernel/cpu_ops_spinwait.c b/arch/riscv/kernel/cpu_ops_spinwait.c index 613872b0a21acb2a708f194d22f186f376f8a748..24869eb889085e4ae40881230707b54024bad975 100644 --- a/arch/riscv/kernel/cpu_ops_spinwait.c +++ b/arch/riscv/kernel/cpu_ops_spinwait.c @@ -34,8 +34,7 @@ static void cpu_update_secondary_bootdata(unsigned int cpuid, /* Make sure tidle is updated */ smp_mb(); - WRITE_ONCE(__cpu_spinwait_stack_pointer[hartid], - task_stack_page(tidle) + THREAD_SIZE); + WRITE_ONCE(__cpu_spinwait_stack_pointer[hartid], task_pt_regs(tidle)); WRITE_ONCE(__cpu_spinwait_task_pointer[hartid], tidle); } diff --git a/arch/riscv/kvm/aia_device.c b/arch/riscv/kvm/aia_device.c index 0eb689351b7d04128ba93122209f71b830b33efe..5cd407c6a8e4f82389ad3d0a63081d6dcfec7688 100644 --- a/arch/riscv/kvm/aia_device.c +++ b/arch/riscv/kvm/aia_device.c @@ -237,10 +237,11 @@ static gpa_t aia_imsic_ppn(struct kvm_aia *aia, gpa_t addr) static u32 aia_imsic_hart_index(struct kvm_aia *aia, gpa_t addr) { - u32 hart, group = 0; + u32 hart = 0, group = 0; - hart = (addr >> (aia->nr_guest_bits + IMSIC_MMIO_PAGE_SHIFT)) & - GENMASK_ULL(aia->nr_hart_bits - 1, 0); + if (aia->nr_hart_bits) + hart = (addr >> (aia->nr_guest_bits + IMSIC_MMIO_PAGE_SHIFT)) & + GENMASK_ULL(aia->nr_hart_bits - 1, 0); if (aia->nr_group_bits) group = (addr >> aia->nr_group_shift) & GENMASK_ULL(aia->nr_group_bits - 1, 0); diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c index c676275ea0a017617f3f6d766c66302479e4dc45..62874fbca29ff5c71720cb718201d2f9e8c994ba 100644 --- a/arch/riscv/kvm/vcpu_onereg.c +++ b/arch/riscv/kvm/vcpu_onereg.c @@ -724,9 +724,9 @@ static int kvm_riscv_vcpu_set_reg_isa_ext(struct kvm_vcpu *vcpu, switch (reg_subtype) { case KVM_REG_RISCV_ISA_SINGLE: return riscv_vcpu_set_isa_ext_single(vcpu, reg_num, reg_val); - case KVM_REG_RISCV_SBI_MULTI_EN: + case KVM_REG_RISCV_ISA_MULTI_EN: return riscv_vcpu_set_isa_ext_multi(vcpu, reg_num, reg_val, true); - case KVM_REG_RISCV_SBI_MULTI_DIS: + case KVM_REG_RISCV_ISA_MULTI_DIS: return riscv_vcpu_set_isa_ext_multi(vcpu, reg_num, reg_val, false); default: return -ENOENT; diff --git a/arch/riscv/net/bpf_jit.h b/arch/riscv/net/bpf_jit.h index fdbf88ca8b700d021b21df2985651bf58fec8574..1d1c78d4cff1eee85eb244b5da7e3d933274857a 100644 --- a/arch/riscv/net/bpf_jit.h +++ b/arch/riscv/net/bpf_jit.h @@ -18,6 +18,11 @@ static inline bool rvc_enabled(void) return IS_ENABLED(CONFIG_RISCV_ISA_C); } +static inline bool rvzba_enabled(void) +{ + return IS_ENABLED(CONFIG_RISCV_ISA_ZBA) && riscv_has_extension_likely(RISCV_ISA_EXT_ZBA); +} + static inline bool rvzbb_enabled(void) { return IS_ENABLED(CONFIG_RISCV_ISA_ZBB) && riscv_has_extension_likely(RISCV_ISA_EXT_ZBB); @@ -737,6 +742,17 @@ static inline u16 rvc_swsp(u32 imm8, u8 rs2) return rv_css_insn(0x6, imm, rs2, 0x2); } +/* RVZBA instructions. */ +static inline u32 rvzba_sh2add(u8 rd, u8 rs1, u8 rs2) +{ + return rv_r_insn(0x10, rs2, rs1, 0x4, rd, 0x33); +} + +static inline u32 rvzba_sh3add(u8 rd, u8 rs1, u8 rs2) +{ + return rv_r_insn(0x10, rs2, rs1, 0x6, rd, 0x33); +} + /* RVZBB instructions. */ static inline u32 rvzbb_sextb(u8 rd, u8 rs1) { @@ -939,6 +955,14 @@ static inline u16 rvc_sdsp(u32 imm9, u8 rs2) return rv_css_insn(0x7, imm, rs2, 0x2); } +/* RV64-only ZBA instructions. */ + +static inline u32 rvzba_zextw(u8 rd, u8 rs1) +{ + /* add.uw rd, rs1, ZERO */ + return rv_r_insn(0x04, RV_REG_ZERO, rs1, 0, rd, 0x3b); +} + #endif /* __riscv_xlen == 64 */ /* Helper functions that emit RVC instructions when possible. */ @@ -1082,6 +1106,28 @@ static inline void emit_sw(u8 rs1, s32 off, u8 rs2, struct rv_jit_context *ctx) emit(rv_sw(rs1, off, rs2), ctx); } +static inline void emit_sh2add(u8 rd, u8 rs1, u8 rs2, struct rv_jit_context *ctx) +{ + if (rvzba_enabled()) { + emit(rvzba_sh2add(rd, rs1, rs2), ctx); + return; + } + + emit_slli(rd, rs1, 2, ctx); + emit_add(rd, rd, rs2, ctx); +} + +static inline void emit_sh3add(u8 rd, u8 rs1, u8 rs2, struct rv_jit_context *ctx) +{ + if (rvzba_enabled()) { + emit(rvzba_sh3add(rd, rs1, rs2), ctx); + return; + } + + emit_slli(rd, rs1, 3, ctx); + emit_add(rd, rd, rs2, ctx); +} + /* RV64-only helper functions. */ #if __riscv_xlen == 64 @@ -1161,6 +1207,11 @@ static inline void emit_zexth(u8 rd, u8 rs, struct rv_jit_context *ctx) static inline void emit_zextw(u8 rd, u8 rs, struct rv_jit_context *ctx) { + if (rvzba_enabled()) { + emit(rvzba_zextw(rd, rs), ctx); + return; + } + emit_slli(rd, rs, 32, ctx); emit_srli(rd, rd, 32, ctx); } diff --git a/arch/riscv/net/bpf_jit_comp32.c b/arch/riscv/net/bpf_jit_comp32.c index f5ba73bb153d7ea0117eae54b379836bc0a0d8fc..592dd86fbf811afbe7d662a0e2b24d0541f02848 100644 --- a/arch/riscv/net/bpf_jit_comp32.c +++ b/arch/riscv/net/bpf_jit_comp32.c @@ -811,8 +811,7 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx) * if (!prog) * goto out; */ - emit(rv_slli(RV_REG_T0, lo(idx_reg), 2), ctx); - emit(rv_add(RV_REG_T0, RV_REG_T0, lo(arr_reg)), ctx); + emit_sh2add(RV_REG_T0, lo(idx_reg), lo(arr_reg), ctx); off = offsetof(struct bpf_array, ptrs); if (is_12b_check(off, insn)) return -1; diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c index 79a001d5533ea6b728661884c3da0aecb9139eb3..d5cebb0b0afeee7aa014b5a62c1c307521b296c5 100644 --- a/arch/riscv/net/bpf_jit_comp64.c +++ b/arch/riscv/net/bpf_jit_comp64.c @@ -380,8 +380,7 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx) * if (!prog) * goto out; */ - emit_slli(RV_REG_T2, RV_REG_A2, 3, ctx); - emit_add(RV_REG_T2, RV_REG_T2, RV_REG_A1, ctx); + emit_sh3add(RV_REG_T2, RV_REG_A2, RV_REG_A1, ctx); off = offsetof(struct bpf_array, ptrs); if (is_12b_check(off, insn)) return -1; @@ -537,8 +536,10 @@ static void emit_atomic(u8 rd, u8 rs, s16 off, s32 imm, bool is64, /* r0 = atomic_cmpxchg(dst_reg + off16, r0, src_reg); */ case BPF_CMPXCHG: r0 = bpf_to_rv_reg(BPF_REG_0, ctx); - emit(is64 ? rv_addi(RV_REG_T2, r0, 0) : - rv_addiw(RV_REG_T2, r0, 0), ctx); + if (is64) + emit_mv(RV_REG_T2, r0, ctx); + else + emit_addiw(RV_REG_T2, r0, 0, ctx); emit(is64 ? rv_lr_d(r0, 0, rd, 0, 0) : rv_lr_w(r0, 0, rd, 0, 0), ctx); jmp_offset = ninsns_rvoff(8); @@ -868,7 +869,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, stack_size += 8; sreg_off = stack_size; - stack_size = round_up(stack_size, 16); + stack_size = round_up(stack_size, STACK_ALIGN); if (!is_struct_ops) { /* For the trampoline called from function entry, @@ -1097,12 +1098,10 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, /* Load current CPU number in T1 */ emit_ld(RV_REG_T1, offsetof(struct thread_info, cpu), RV_REG_TP, ctx); - /* << 3 because offsets are 8 bytes */ - emit_slli(RV_REG_T1, RV_REG_T1, 3, ctx); /* Load address of __per_cpu_offset array in T2 */ emit_addr(RV_REG_T2, (u64)&__per_cpu_offset, extra_pass, ctx); - /* Add offset of current CPU to __per_cpu_offset */ - emit_add(RV_REG_T1, RV_REG_T2, RV_REG_T1, ctx); + /* Get address of __per_cpu_offset[cpu] in T1 */ + emit_sh3add(RV_REG_T1, RV_REG_T1, RV_REG_T2, ctx); /* Load __per_cpu_offset[cpu] in T1 */ emit_ld(RV_REG_T1, 0, RV_REG_T1, ctx); /* Add the offset to Rd */ @@ -1960,7 +1959,7 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) { int i, stack_adjust = 0, store_offset, bpf_stack_adjust; - bpf_stack_adjust = round_up(ctx->prog->aux->stack_depth, 16); + bpf_stack_adjust = round_up(ctx->prog->aux->stack_depth, STACK_ALIGN); if (bpf_stack_adjust) mark_fp(ctx); @@ -1982,7 +1981,7 @@ void bpf_jit_build_prologue(struct rv_jit_context *ctx, bool is_subprog) if (ctx->arena_vm_start) stack_adjust += 8; - stack_adjust = round_up(stack_adjust, 16); + stack_adjust = round_up(stack_adjust, STACK_ALIGN); stack_adjust += bpf_stack_adjust; store_offset = stack_adjust - 8; diff --git a/arch/x86/events/intel/cstate.c b/arch/x86/events/intel/cstate.c index e64eaa8dda5a45e7d8938d31afd2a0d9434469f9..9d6e8f13d13a7807c11123b160a0c7df50185b27 100644 --- a/arch/x86/events/intel/cstate.c +++ b/arch/x86/events/intel/cstate.c @@ -114,6 +114,7 @@ #include "../perf_event.h" #include "../probe.h" +MODULE_DESCRIPTION("Support for Intel cstate performance events"); MODULE_LICENSE("GPL"); #define DEFINE_CSTATE_FORMAT_ATTR(_var, _name, _format) \ diff --git a/arch/x86/events/intel/uncore.c b/arch/x86/events/intel/uncore.c index 419c517b8594fbdd057a0f36bc707c8b6cd8b58f..c68f5b39952b0a4cc2fcc2bb61a8ba2a1c6b7bed 100644 --- a/arch/x86/events/intel/uncore.c +++ b/arch/x86/events/intel/uncore.c @@ -34,6 +34,7 @@ static struct event_constraint uncore_constraint_fixed = struct event_constraint uncore_constraint_empty = EVENT_CONSTRAINT(0, 0, 0); +MODULE_DESCRIPTION("Support for Intel uncore performance events"); MODULE_LICENSE("GPL"); int uncore_pcibus_to_dieid(struct pci_bus *bus) diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c index 46e673585560312fbc8f54b910ccc247da6adb38..0c5e7a7c43ac07ea63eda49d6d92de47b1caec0a 100644 --- a/arch/x86/events/rapl.c +++ b/arch/x86/events/rapl.c @@ -64,6 +64,7 @@ #include "perf_event.h" #include "probe.h" +MODULE_DESCRIPTION("Support Intel/AMD RAPL energy consumption counters"); MODULE_LICENSE("GPL"); /* diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index ece45b3f6f2073ea81ad65b139173411c77b3d73..f8ca74e7678f3ab8dc44ea0d2cd355cf04c5e068 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2154,6 +2154,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu); int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code, void *insn, int insn_len); +void kvm_mmu_print_sptes(struct kvm_vcpu *vcpu, gpa_t gpa, const char *msg); void kvm_mmu_invlpg(struct kvm_vcpu *vcpu, gva_t gva); void kvm_mmu_invalidate_addr(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, u64 addr, unsigned long roots); diff --git a/arch/x86/include/asm/vmxfeatures.h b/arch/x86/include/asm/vmxfeatures.h index 266daf5b5b842d0b9921d5950285c648077712fc..695f3666488996da42eba757617f47597475dd5a 100644 --- a/arch/x86/include/asm/vmxfeatures.h +++ b/arch/x86/include/asm/vmxfeatures.h @@ -77,7 +77,7 @@ #define VMX_FEATURE_ENCLS_EXITING ( 2*32+ 15) /* "" VM-Exit on ENCLS (leaf dependent) */ #define VMX_FEATURE_RDSEED_EXITING ( 2*32+ 16) /* "" VM-Exit on RDSEED */ #define VMX_FEATURE_PAGE_MOD_LOGGING ( 2*32+ 17) /* "pml" Log dirty pages into buffer */ -#define VMX_FEATURE_EPT_VIOLATION_VE ( 2*32+ 18) /* "" Conditionally reflect EPT violations as #VE exceptions */ +#define VMX_FEATURE_EPT_VIOLATION_VE ( 2*32+ 18) /* Conditionally reflect EPT violations as #VE exceptions */ #define VMX_FEATURE_PT_CONCEAL_VMX ( 2*32+ 19) /* "" Suppress VMX indicators in Processor Trace */ #define VMX_FEATURE_XSAVES ( 2*32+ 20) /* "" Enable XSAVES and XRSTORS in guest */ #define VMX_FEATURE_MODE_BASED_EPT_EXEC ( 2*32+ 22) /* "ept_mode_based_exec" Enable separate EPT EXEC bits for supervisor vs. user */ diff --git a/arch/x86/kernel/cpu/aperfmperf.c b/arch/x86/kernel/cpu/aperfmperf.c index f9a8c7b7943fb2c6de232ae56b02ccefcc1504ee..b3fa61d45352e7120d7be7fb5cdb9d6f071c9729 100644 --- a/arch/x86/kernel/cpu/aperfmperf.c +++ b/arch/x86/kernel/cpu/aperfmperf.c @@ -345,6 +345,7 @@ static DECLARE_WORK(disable_freq_invariance_work, disable_freq_invariance_workfn); DEFINE_PER_CPU(unsigned long, arch_freq_scale) = SCHED_CAPACITY_SCALE; +EXPORT_PER_CPU_SYMBOL_GPL(arch_freq_scale); static void scale_freq_tick(u64 acnt, u64 mcnt) { diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 2b170da84f97fec6a1c49a4793d91f21ceb48195..d4e539d4e158cc561e156c028336080a07478ac8 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1075,6 +1075,10 @@ void get_cpu_address_sizes(struct cpuinfo_x86 *c) c->x86_virt_bits = (eax >> 8) & 0xff; c->x86_phys_bits = eax & 0xff; + + /* Provide a sane default if not enumerated: */ + if (!c->x86_clflush_size) + c->x86_clflush_size = 32; } c->x86_cache_bits = c->x86_phys_bits; @@ -1585,6 +1589,7 @@ static void __init early_identify_cpu(struct cpuinfo_x86 *c) if (have_cpuid_p()) { cpu_detect(c); get_cpu_vendor(c); + intel_unlock_cpuid_leafs(c); get_cpu_cap(c); setup_force_cpu_cap(X86_FEATURE_CPUID); get_cpu_address_sizes(c); @@ -1744,7 +1749,7 @@ static void generic_identify(struct cpuinfo_x86 *c) cpu_detect(c); get_cpu_vendor(c); - + intel_unlock_cpuid_leafs(c); get_cpu_cap(c); get_cpu_address_sizes(c); diff --git a/arch/x86/kernel/cpu/cpu.h b/arch/x86/kernel/cpu/cpu.h index ea9e07d57c8dd2d694d78769ea8729d0dda63dbd..1beccefbaff9a5a50f83c944cd8267d65f484085 100644 --- a/arch/x86/kernel/cpu/cpu.h +++ b/arch/x86/kernel/cpu/cpu.h @@ -61,9 +61,11 @@ extern __ro_after_init enum tsx_ctrl_states tsx_ctrl_state; extern void __init tsx_init(void); void tsx_ap_init(void); +void intel_unlock_cpuid_leafs(struct cpuinfo_x86 *c); #else static inline void tsx_init(void) { } static inline void tsx_ap_init(void) { } +static inline void intel_unlock_cpuid_leafs(struct cpuinfo_x86 *c) { } #endif /* CONFIG_CPU_SUP_INTEL */ extern void init_spectral_chicken(struct cpuinfo_x86 *c); diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c index 3c3e7e5695ba419434e5b64668b83deae507f8e3..fdf3489d92a4980245481389f327527637be3bdd 100644 --- a/arch/x86/kernel/cpu/intel.c +++ b/arch/x86/kernel/cpu/intel.c @@ -269,19 +269,26 @@ detect_keyid_bits: c->x86_phys_bits -= keyid_bits; } +void intel_unlock_cpuid_leafs(struct cpuinfo_x86 *c) +{ + if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) + return; + + if (c->x86 < 6 || (c->x86 == 6 && c->x86_model < 0xd)) + return; + + /* + * The BIOS can have limited CPUID to leaf 2, which breaks feature + * enumeration. Unlock it and update the maximum leaf info. + */ + if (msr_clear_bit(MSR_IA32_MISC_ENABLE, MSR_IA32_MISC_ENABLE_LIMIT_CPUID_BIT) > 0) + c->cpuid_level = cpuid_eax(0); +} + static void early_init_intel(struct cpuinfo_x86 *c) { u64 misc_enable; - /* Unmask CPUID levels if masked: */ - if (c->x86 > 6 || (c->x86 == 6 && c->x86_model >= 0xd)) { - if (msr_clear_bit(MSR_IA32_MISC_ENABLE, - MSR_IA32_MISC_ENABLE_LIMIT_CPUID_BIT) > 0) { - c->cpuid_level = cpuid_eax(0); - get_cpu_cap(c); - } - } - if ((c->x86 == 0xf && c->x86_model >= 0x03) || (c->x86 == 0x6 && c->x86_model >= 0x0e)) set_cpu_cap(c, X86_FEATURE_CONSTANT_TSC); diff --git a/arch/x86/kernel/cpu/topology_amd.c b/arch/x86/kernel/cpu/topology_amd.c index d419deed6a4884a387f5babc999fd599c5a7314e..7d476fa697ca53b352efe76ec659193968938558 100644 --- a/arch/x86/kernel/cpu/topology_amd.c +++ b/arch/x86/kernel/cpu/topology_amd.c @@ -84,9 +84,9 @@ static bool parse_8000_001e(struct topo_scan *tscan, bool has_topoext) /* * If leaf 0xb is available, then the domain shifts are set - * already and nothing to do here. + * already and nothing to do here. Only valid for family >= 0x17. */ - if (!has_topoext) { + if (!has_topoext && tscan->c->x86 >= 0x17) { /* * Leaf 0x80000008 set the CORE domain shift already. * Update the SMT domain, but do not propagate it. diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index d64fb2b3eb69e364772b1d9e515cb232a325f4b8..fec95a7702703f90bff5fc9c7dc4e2f91207c6f4 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -44,6 +44,7 @@ config KVM select KVM_VFIO select HAVE_KVM_PM_NOTIFIER if PM select KVM_GENERIC_HARDWARE_ENABLING + select KVM_WERROR if WERROR help Support hosting fully virtualized guest machines using hardware virtualization extensions. You will need a fairly recent @@ -66,7 +67,7 @@ config KVM_WERROR # FRAME_WARN, i.e. KVM_WERROR=y with KASAN=y requires special tuning. # Building KVM with -Werror and KASAN is still doable via enabling # the kernel-wide WERROR=y. - depends on KVM && EXPERT && !KASAN + depends on KVM && ((EXPERT && !KASAN) || WERROR) help Add -Werror to the build flags for KVM. @@ -97,15 +98,17 @@ config KVM_INTEL config KVM_INTEL_PROVE_VE bool "Check that guests do not receive #VE exceptions" - default KVM_PROVE_MMU || DEBUG_KERNEL - depends on KVM_INTEL + depends on KVM_INTEL && EXPERT help - Checks that KVM's page table management code will not incorrectly let guests receive a virtualization exception. Virtualization exceptions will be trapped by the hypervisor rather than injected in the guest. + Note: some CPUs appear to generate spurious EPT Violations #VEs + that trigger KVM's WARN, in particular with eptad=0 and/or nested + virtualization. + If unsure, say N. config X86_SGX_KVM diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index ebf41023be38293dbc248d75a7125b63cd46c189..acd7d48100a1d3360202f33b5fa778b34d38afeb 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -59,7 +59,17 @@ #define MAX_APIC_VECTOR 256 #define APIC_VECTORS_PER_REG 32 -static bool lapic_timer_advance_dynamic __read_mostly; +/* + * Enable local APIC timer advancement (tscdeadline mode only) with adaptive + * tuning. When enabled, KVM programs the host timer event to fire early, i.e. + * before the deadline expires, to account for the delay between taking the + * VM-Exit (to inject the guest event) and the subsequent VM-Enter to resume + * the guest, i.e. so that the interrupt arrives in the guest with minimal + * latency relative to the deadline programmed by the guest. + */ +static bool lapic_timer_advance __read_mostly = true; +module_param(lapic_timer_advance, bool, 0444); + #define LAPIC_TIMER_ADVANCE_ADJUST_MIN 100 /* clock cycles */ #define LAPIC_TIMER_ADVANCE_ADJUST_MAX 10000 /* clock cycles */ #define LAPIC_TIMER_ADVANCE_NS_INIT 1000 @@ -1854,16 +1864,14 @@ static void __kvm_wait_lapic_expire(struct kvm_vcpu *vcpu) guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc()); trace_kvm_wait_lapic_expire(vcpu->vcpu_id, guest_tsc - tsc_deadline); - if (lapic_timer_advance_dynamic) { - adjust_lapic_timer_advance(vcpu, guest_tsc - tsc_deadline); - /* - * If the timer fired early, reread the TSC to account for the - * overhead of the above adjustment to avoid waiting longer - * than is necessary. - */ - if (guest_tsc < tsc_deadline) - guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc()); - } + adjust_lapic_timer_advance(vcpu, guest_tsc - tsc_deadline); + + /* + * If the timer fired early, reread the TSC to account for the overhead + * of the above adjustment to avoid waiting longer than is necessary. + */ + if (guest_tsc < tsc_deadline) + guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc()); if (guest_tsc < tsc_deadline) __wait_lapic_expire(vcpu, tsc_deadline - guest_tsc); @@ -2812,7 +2820,7 @@ static enum hrtimer_restart apic_timer_fn(struct hrtimer *data) return HRTIMER_NORESTART; } -int kvm_create_lapic(struct kvm_vcpu *vcpu, int timer_advance_ns) +int kvm_create_lapic(struct kvm_vcpu *vcpu) { struct kvm_lapic *apic; @@ -2845,13 +2853,8 @@ int kvm_create_lapic(struct kvm_vcpu *vcpu, int timer_advance_ns) hrtimer_init(&apic->lapic_timer.timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); apic->lapic_timer.timer.function = apic_timer_fn; - if (timer_advance_ns == -1) { + if (lapic_timer_advance) apic->lapic_timer.timer_advance_ns = LAPIC_TIMER_ADVANCE_NS_INIT; - lapic_timer_advance_dynamic = true; - } else { - apic->lapic_timer.timer_advance_ns = timer_advance_ns; - lapic_timer_advance_dynamic = false; - } /* * Stuff the APIC ENABLE bit in lieu of temporarily incrementing diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 0a0ea4b5dd8ce7239b85f5d054828fd5f0eedb6b..a69e706b9080acf97b2947f01d37d2a760064957 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -85,7 +85,7 @@ struct kvm_lapic { struct dest_map; -int kvm_create_lapic(struct kvm_vcpu *vcpu, int timer_advance_ns); +int kvm_create_lapic(struct kvm_vcpu *vcpu); void kvm_free_lapic(struct kvm_vcpu *vcpu); int kvm_apic_has_interrupt(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 662f62dfb2aa9f0d205ee02a88da0b687c2291d6..8d74bdef68c1d39a2b6037de4277a3228686c709 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -336,16 +336,19 @@ static int is_cpuid_PSE36(void) #ifdef CONFIG_X86_64 static void __set_spte(u64 *sptep, u64 spte) { + KVM_MMU_WARN_ON(is_ept_ve_possible(spte)); WRITE_ONCE(*sptep, spte); } static void __update_clear_spte_fast(u64 *sptep, u64 spte) { + KVM_MMU_WARN_ON(is_ept_ve_possible(spte)); WRITE_ONCE(*sptep, spte); } static u64 __update_clear_spte_slow(u64 *sptep, u64 spte) { + KVM_MMU_WARN_ON(is_ept_ve_possible(spte)); return xchg(sptep, spte); } @@ -4101,23 +4104,31 @@ static int get_walk(struct kvm_vcpu *vcpu, u64 addr, u64 *sptes, int *root_level return leaf; } -/* return true if reserved bit(s) are detected on a valid, non-MMIO SPTE. */ -static bool get_mmio_spte(struct kvm_vcpu *vcpu, u64 addr, u64 *sptep) +static int get_sptes_lockless(struct kvm_vcpu *vcpu, u64 addr, u64 *sptes, + int *root_level) { - u64 sptes[PT64_ROOT_MAX_LEVEL + 1]; - struct rsvd_bits_validate *rsvd_check; - int root, leaf, level; - bool reserved = false; + int leaf; walk_shadow_page_lockless_begin(vcpu); if (is_tdp_mmu_active(vcpu)) - leaf = kvm_tdp_mmu_get_walk(vcpu, addr, sptes, &root); + leaf = kvm_tdp_mmu_get_walk(vcpu, addr, sptes, root_level); else - leaf = get_walk(vcpu, addr, sptes, &root); + leaf = get_walk(vcpu, addr, sptes, root_level); walk_shadow_page_lockless_end(vcpu); + return leaf; +} + +/* return true if reserved bit(s) are detected on a valid, non-MMIO SPTE. */ +static bool get_mmio_spte(struct kvm_vcpu *vcpu, u64 addr, u64 *sptep) +{ + u64 sptes[PT64_ROOT_MAX_LEVEL + 1]; + struct rsvd_bits_validate *rsvd_check; + int root, leaf, level; + bool reserved = false; + leaf = get_sptes_lockless(vcpu, addr, sptes, &root); if (unlikely(leaf < 0)) { *sptep = 0ull; return reserved; @@ -4400,9 +4411,6 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, return RET_PF_EMULATE; } - fault->mmu_seq = vcpu->kvm->mmu_invalidate_seq; - smp_rmb(); - /* * Check for a relevant mmu_notifier invalidation event before getting * the pfn from the primary MMU, and before acquiring mmu_lock. @@ -5921,6 +5929,22 @@ emulate: } EXPORT_SYMBOL_GPL(kvm_mmu_page_fault); +void kvm_mmu_print_sptes(struct kvm_vcpu *vcpu, gpa_t gpa, const char *msg) +{ + u64 sptes[PT64_ROOT_MAX_LEVEL + 1]; + int root_level, leaf, level; + + leaf = get_sptes_lockless(vcpu, gpa, sptes, &root_level); + if (unlikely(leaf < 0)) + return; + + pr_err("%s %llx", msg, gpa); + for (level = root_level; level >= leaf; level--) + pr_cont(", spte[%d] = 0x%llx", level, sptes[level]); + pr_cont("\n"); +} +EXPORT_SYMBOL_GPL(kvm_mmu_print_sptes); + static void __kvm_mmu_invalidate_addr(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, u64 addr, hpa_t root_hpa) { diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 5dd5405fa07ac9a23ec3ffdfa71280bf32b67e2a..52fa004a1fbc9f3dbe77b2bb8ef93d2cb74b341b 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -3,6 +3,8 @@ #ifndef KVM_X86_MMU_SPTE_H #define KVM_X86_MMU_SPTE_H +#include + #include "mmu.h" #include "mmu_internal.h" @@ -276,6 +278,13 @@ static inline bool is_shadow_present_pte(u64 pte) return !!(pte & SPTE_MMU_PRESENT_MASK); } +static inline bool is_ept_ve_possible(u64 spte) +{ + return (shadow_present_mask & VMX_EPT_SUPPRESS_VE_BIT) && + !(spte & VMX_EPT_SUPPRESS_VE_BIT) && + (spte & VMX_EPT_RWX_MASK) != VMX_EPT_MISCONFIG_WX_VALUE; +} + /* * Returns true if A/D bits are supported in hardware and are enabled by KVM. * When enabled, KVM uses A/D bits for all non-nested MMUs. Because L1 can diff --git a/arch/x86/kvm/mmu/tdp_iter.h b/arch/x86/kvm/mmu/tdp_iter.h index fae559559a806a8121760a637430b858ec5bacab..2880fd392e0cbb9921aed5b8b5992c54e484530f 100644 --- a/arch/x86/kvm/mmu/tdp_iter.h +++ b/arch/x86/kvm/mmu/tdp_iter.h @@ -21,11 +21,13 @@ static inline u64 kvm_tdp_mmu_read_spte(tdp_ptep_t sptep) static inline u64 kvm_tdp_mmu_write_spte_atomic(tdp_ptep_t sptep, u64 new_spte) { + KVM_MMU_WARN_ON(is_ept_ve_possible(new_spte)); return xchg(rcu_dereference(sptep), new_spte); } static inline void __kvm_tdp_mmu_write_spte(tdp_ptep_t sptep, u64 new_spte) { + KVM_MMU_WARN_ON(is_ept_ve_possible(new_spte)); WRITE_ONCE(*rcu_dereference(sptep), new_spte); } diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 1259dd63defc8e92ed2eb49e291db5a96d7623c9..36539c1b36cd632f5fb32a263cc3c68632b2f1c9 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -626,7 +626,7 @@ static inline int tdp_mmu_zap_spte_atomic(struct kvm *kvm, * SPTEs. */ handle_changed_spte(kvm, iter->as_id, iter->gfn, iter->old_spte, - 0, iter->level, true); + SHADOW_NONPRESENT_VALUE, iter->level, true); return 0; } diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 0623cfaa7bb0ee9f9ca3ec142e99feb41254533b..95095a233a459a9fafb49ac9d244a6eb7becf94e 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -779,6 +779,14 @@ static int __sev_launch_update_vmsa(struct kvm *kvm, struct kvm_vcpu *vcpu, */ fpstate_set_confidential(&vcpu->arch.guest_fpu); vcpu->arch.guest_state_protected = true; + + /* + * SEV-ES guest mandates LBR Virtualization to be _always_ ON. Enable it + * only after setting guest_state_protected because KVM_SET_MSRS allows + * dynamic toggling of LBRV (for performance reason) on write access to + * MSR_IA32_DEBUGCTLMSR when guest_state_protected is not set. + */ + svm_enable_lbrv(vcpu); return 0; } @@ -2406,6 +2414,12 @@ void __init sev_hardware_setup(void) if (!boot_cpu_has(X86_FEATURE_SEV_ES)) goto out; + if (!lbrv) { + WARN_ONCE(!boot_cpu_has(X86_FEATURE_LBRV), + "LBRV must be present for SEV-ES support"); + goto out; + } + /* Has the system been allocated ASIDs for SEV-ES? */ if (min_sev_asid == 1) goto out; @@ -3216,7 +3230,6 @@ static void sev_es_init_vmcb(struct vcpu_svm *svm) struct kvm_vcpu *vcpu = &svm->vcpu; svm->vmcb->control.nested_ctl |= SVM_NESTED_CTL_SEV_ES_ENABLE; - svm->vmcb->control.virt_ext |= LBR_CTL_ENABLE_MASK; /* * An SEV-ES guest requires a VMSA area that is a separate from the @@ -3268,10 +3281,6 @@ static void sev_es_init_vmcb(struct vcpu_svm *svm) /* Clear intercepts on selected MSRs */ set_msr_interception(vcpu, svm->msrpm, MSR_EFER, 1, 1); set_msr_interception(vcpu, svm->msrpm, MSR_IA32_CR_PAT, 1, 1); - set_msr_interception(vcpu, svm->msrpm, MSR_IA32_LASTBRANCHFROMIP, 1, 1); - set_msr_interception(vcpu, svm->msrpm, MSR_IA32_LASTBRANCHTOIP, 1, 1); - set_msr_interception(vcpu, svm->msrpm, MSR_IA32_LASTINTFROMIP, 1, 1); - set_msr_interception(vcpu, svm->msrpm, MSR_IA32_LASTINTTOIP, 1, 1); } void sev_init_vmcb(struct vcpu_svm *svm) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index c8dc25886c16581c2e89e94b685fa6c95ad4fc7a..296c524988f953b3d80a37d7de628c24c9d39e58 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -99,6 +99,7 @@ static const struct svm_direct_access_msrs { { .index = MSR_IA32_SPEC_CTRL, .always = false }, { .index = MSR_IA32_PRED_CMD, .always = false }, { .index = MSR_IA32_FLUSH_CMD, .always = false }, + { .index = MSR_IA32_DEBUGCTLMSR, .always = false }, { .index = MSR_IA32_LASTBRANCHFROMIP, .always = false }, { .index = MSR_IA32_LASTBRANCHTOIP, .always = false }, { .index = MSR_IA32_LASTINTFROMIP, .always = false }, @@ -215,7 +216,7 @@ int vgif = true; module_param(vgif, int, 0444); /* enable/disable LBR virtualization */ -static int lbrv = true; +int lbrv = true; module_param(lbrv, int, 0444); static int tsc_scaling = true; @@ -990,7 +991,7 @@ void svm_copy_lbrs(struct vmcb *to_vmcb, struct vmcb *from_vmcb) vmcb_mark_dirty(to_vmcb, VMCB_LBR); } -static void svm_enable_lbrv(struct kvm_vcpu *vcpu) +void svm_enable_lbrv(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm = to_svm(vcpu); @@ -1000,6 +1001,9 @@ static void svm_enable_lbrv(struct kvm_vcpu *vcpu) set_msr_interception(vcpu, svm->msrpm, MSR_IA32_LASTINTFROMIP, 1, 1); set_msr_interception(vcpu, svm->msrpm, MSR_IA32_LASTINTTOIP, 1, 1); + if (sev_es_guest(vcpu->kvm)) + set_msr_interception(vcpu, svm->msrpm, MSR_IA32_DEBUGCTLMSR, 1, 1); + /* Move the LBR msrs to the vmcb02 so that the guest can see them. */ if (is_guest_mode(vcpu)) svm_copy_lbrs(svm->vmcb, svm->vmcb01.ptr); @@ -1009,6 +1013,8 @@ static void svm_disable_lbrv(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm = to_svm(vcpu); + KVM_BUG_ON(sev_es_guest(vcpu->kvm), vcpu->kvm); + svm->vmcb->control.virt_ext &= ~LBR_CTL_ENABLE_MASK; set_msr_interception(vcpu, svm->msrpm, MSR_IA32_LASTBRANCHFROMIP, 0, 0); set_msr_interception(vcpu, svm->msrpm, MSR_IA32_LASTBRANCHTOIP, 0, 0); @@ -2822,10 +2828,24 @@ static int svm_get_msr_feature(struct kvm_msr_entry *msr) return 0; } +static bool +sev_es_prevent_msr_access(struct kvm_vcpu *vcpu, struct msr_data *msr_info) +{ + return sev_es_guest(vcpu->kvm) && + vcpu->arch.guest_state_protected && + svm_msrpm_offset(msr_info->index) != MSR_INVALID && + !msr_write_intercepted(vcpu, msr_info->index); +} + static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) { struct vcpu_svm *svm = to_svm(vcpu); + if (sev_es_prevent_msr_access(vcpu, msr_info)) { + msr_info->data = 0; + return -EINVAL; + } + switch (msr_info->index) { case MSR_AMD64_TSC_RATIO: if (!msr_info->host_initiated && @@ -2976,6 +2996,10 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) u32 ecx = msr->index; u64 data = msr->data; + + if (sev_es_prevent_msr_access(vcpu, msr)) + return -EINVAL; + switch (ecx) { case MSR_AMD64_TSC_RATIO: @@ -3846,16 +3870,27 @@ static void svm_enable_nmi_window(struct kvm_vcpu *vcpu) struct vcpu_svm *svm = to_svm(vcpu); /* - * KVM should never request an NMI window when vNMI is enabled, as KVM - * allows at most one to-be-injected NMI and one pending NMI, i.e. if - * two NMIs arrive simultaneously, KVM will inject one and set - * V_NMI_PENDING for the other. WARN, but continue with the standard - * single-step approach to try and salvage the pending NMI. + * If NMIs are outright masked, i.e. the vCPU is already handling an + * NMI, and KVM has not yet intercepted an IRET, then there is nothing + * more to do at this time as KVM has already enabled IRET intercepts. + * If KVM has already intercepted IRET, then single-step over the IRET, + * as NMIs aren't architecturally unmasked until the IRET completes. + * + * If vNMI is enabled, KVM should never request an NMI window if NMIs + * are masked, as KVM allows at most one to-be-injected NMI and one + * pending NMI. If two NMIs arrive simultaneously, KVM will inject one + * NMI and set V_NMI_PENDING for the other, but if and only if NMIs are + * unmasked. KVM _will_ request an NMI window in some situations, e.g. + * if the vCPU is in an STI shadow or if GIF=0, KVM can't immediately + * inject the NMI. In those situations, KVM needs to single-step over + * the STI shadow or intercept STGI. */ - WARN_ON_ONCE(is_vnmi_enabled(svm)); + if (svm_get_nmi_mask(vcpu)) { + WARN_ON_ONCE(is_vnmi_enabled(svm)); - if (svm_get_nmi_mask(vcpu) && !svm->awaiting_iret_completion) - return; /* IRET will cause a vm exit */ + if (!svm->awaiting_iret_completion) + return; /* IRET will cause a vm exit */ + } /* * SEV-ES guests are responsible for signaling when a vCPU is ready to @@ -5265,6 +5300,12 @@ static __init int svm_hardware_setup(void) nrips = nrips && boot_cpu_has(X86_FEATURE_NRIPS); + if (lbrv) { + if (!boot_cpu_has(X86_FEATURE_LBRV)) + lbrv = false; + else + pr_info("LBR virtualization supported\n"); + } /* * Note, SEV setup consumes npt_enabled and enable_mmio_caching (which * may be modified by svm_adjust_mmio_mask()), as well as nrips. @@ -5318,14 +5359,6 @@ static __init int svm_hardware_setup(void) svm_x86_ops.set_vnmi_pending = NULL; } - - if (lbrv) { - if (!boot_cpu_has(X86_FEATURE_LBRV)) - lbrv = false; - else - pr_info("LBR virtualization supported\n"); - } - if (!enable_pmu) pr_info("PMU virtualization is disabled\n"); diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index be57213cd295935f055bf46d3856393944108061..0f1472690b593a0ff1a9fd96c77275905ed4311c 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -30,7 +30,7 @@ #define IOPM_SIZE PAGE_SIZE * 3 #define MSRPM_SIZE PAGE_SIZE * 2 -#define MAX_DIRECT_ACCESS_MSRS 47 +#define MAX_DIRECT_ACCESS_MSRS 48 #define MSRPM_OFFSETS 32 extern u32 msrpm_offsets[MSRPM_OFFSETS] __read_mostly; extern bool npt_enabled; @@ -39,6 +39,7 @@ extern int vgif; extern bool intercept_smi; extern bool x2avic_enabled; extern bool vnmi; +extern int lbrv; /* * Clean bits in VMCB. @@ -552,6 +553,7 @@ u32 *svm_vcpu_alloc_msrpm(void); void svm_vcpu_init_msrpm(struct kvm_vcpu *vcpu, u32 *msrpm); void svm_vcpu_free_msrpm(u32 *msrpm); void svm_copy_lbrs(struct vmcb *to_vmcb, struct vmcb *from_vmcb); +void svm_enable_lbrv(struct kvm_vcpu *vcpu); void svm_update_lbrv(struct kvm_vcpu *vcpu); int svm_set_efer(struct kvm_vcpu *vcpu, u64 efer); diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index d5b832126e34580088a36091b62f0b5a66b011b7..643935a0f70ab77b8de56e1c0a145211af5af2ed 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -2242,6 +2242,9 @@ static void prepare_vmcs02_constant_state(struct vcpu_vmx *vmx) vmcs_write64(EPT_POINTER, construct_eptp(&vmx->vcpu, 0, PT64_ROOT_4LEVEL)); + if (vmx->ve_info) + vmcs_write64(VE_INFORMATION_ADDRESS, __pa(vmx->ve_info)); + /* All VMFUNCs are currently emulated through L0 vmexits. */ if (cpu_has_vmx_vmfunc()) vmcs_write64(VM_FUNCTION_CONTROL, 0); @@ -6230,6 +6233,8 @@ static bool nested_vmx_l0_wants_exit(struct kvm_vcpu *vcpu, else if (is_alignment_check(intr_info) && !vmx_guest_inject_ac(vcpu)) return true; + else if (is_ve_fault(intr_info)) + return true; return false; case EXIT_REASON_EXTERNAL_INTERRUPT: return true; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 6051fad5945fa08f9a348fc380799ead2d415991..b3c83c06f8265b60991bec5f6159b0a977e219c3 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -5218,8 +5218,15 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu) if (is_invalid_opcode(intr_info)) return handle_ud(vcpu); - if (KVM_BUG_ON(is_ve_fault(intr_info), vcpu->kvm)) - return -EIO; + if (WARN_ON_ONCE(is_ve_fault(intr_info))) { + struct vmx_ve_information *ve_info = vmx->ve_info; + + WARN_ONCE(ve_info->exit_reason != EXIT_REASON_EPT_VIOLATION, + "Unexpected #VE on VM-Exit reason 0x%x", ve_info->exit_reason); + dump_vmcs(vcpu); + kvm_mmu_print_sptes(vcpu, ve_info->guest_physical_address, "#VE"); + return 1; + } error_code = 0; if (intr_info & INTR_INFO_DELIVER_CODE_MASK) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 082ac6d95a3a08160d54ec832d0c075c0bd71489..8c9e4281d978d73849d840139daa28b465eb8c25 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -164,15 +164,6 @@ module_param(kvmclock_periodic_sync, bool, 0444); static u32 __read_mostly tsc_tolerance_ppm = 250; module_param(tsc_tolerance_ppm, uint, 0644); -/* - * lapic timer advance (tscdeadline mode only) in nanoseconds. '-1' enables - * adaptive tuning starting from default advancement of 1000ns. '0' disables - * advancement entirely. Any other value is used as-is and disables adaptive - * tuning, i.e. allows privileged userspace to set an exact advancement time. - */ -static int __read_mostly lapic_timer_advance_ns = -1; -module_param(lapic_timer_advance_ns, int, 0644); - static bool __read_mostly vector_hashing = true; module_param(vector_hashing, bool, 0444); @@ -12169,7 +12160,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) if (r < 0) return r; - r = kvm_create_lapic(vcpu, lapic_timer_advance_ns); + r = kvm_create_lapic(vcpu); if (r < 0) goto fail_mmu_destroy; diff --git a/block/blk-settings.c b/block/blk-settings.c index a7fe8e90240a6eea7a841e58de4ae355265feeb2..effeb9a639bb459e36e0ae2e8ab9deeeeedc1103 100644 --- a/block/blk-settings.c +++ b/block/blk-settings.c @@ -104,6 +104,7 @@ static int blk_validate_zoned_limits(struct queue_limits *lim) static int blk_validate_limits(struct queue_limits *lim) { unsigned int max_hw_sectors; + unsigned int logical_block_sectors; /* * Unless otherwise specified, default to 512 byte logical blocks and a @@ -134,8 +135,11 @@ static int blk_validate_limits(struct queue_limits *lim) lim->max_hw_sectors = BLK_SAFE_MAX_SECTORS; if (WARN_ON_ONCE(lim->max_hw_sectors < PAGE_SECTORS)) return -EINVAL; + logical_block_sectors = lim->logical_block_size >> SECTOR_SHIFT; + if (WARN_ON_ONCE(logical_block_sectors > lim->max_hw_sectors)) + return -EINVAL; lim->max_hw_sectors = round_down(lim->max_hw_sectors, - lim->logical_block_size >> SECTOR_SHIFT); + logical_block_sectors); /* * The actual max_sectors value is a complex beast and also takes the @@ -153,7 +157,7 @@ static int blk_validate_limits(struct queue_limits *lim) lim->max_sectors = min(max_hw_sectors, BLK_DEF_MAX_SECTORS_CAP); } lim->max_sectors = round_down(lim->max_sectors, - lim->logical_block_size >> SECTOR_SHIFT); + logical_block_sectors); /* * Random default for the maximum number of segments. Driver should not @@ -611,6 +615,8 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b, unsigned int top, bottom, alignment, ret = 0; t->max_sectors = min_not_zero(t->max_sectors, b->max_sectors); + t->max_user_sectors = min_not_zero(t->max_user_sectors, + b->max_user_sectors); t->max_hw_sectors = min_not_zero(t->max_hw_sectors, b->max_hw_sectors); t->max_dev_sectors = min_not_zero(t->max_dev_sectors, b->max_dev_sectors); t->max_write_zeroes_sectors = min(t->max_write_zeroes_sectors, diff --git a/block/blk-stat.h b/block/blk-stat.h index 17e1eb4ec7e25dba273877006d82d901f5864bad..5d7f18ba436d005c865c33d34c5f2c36f9848f12 100644 --- a/block/blk-stat.h +++ b/block/blk-stat.h @@ -64,7 +64,6 @@ struct blk_stat_callback { struct blk_queue_stats *blk_alloc_queue_stats(void); void blk_free_queue_stats(struct blk_queue_stats *); -bool blk_stats_alloc_enable(struct request_queue *q); void blk_stat_add(struct request *rq, u64 now); diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 0be180f9a789904b00d18bbf9505f24bb93f31e7..c1bf73f8c75d95ea30e71bf79914c2caf2e30b6e 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -1399,32 +1399,32 @@ static u64 tg_prfill_limit(struct seq_file *sf, struct blkg_policy_data *pd, bps_dft = U64_MAX; iops_dft = UINT_MAX; - if (tg->bps_conf[READ] == bps_dft && - tg->bps_conf[WRITE] == bps_dft && - tg->iops_conf[READ] == iops_dft && - tg->iops_conf[WRITE] == iops_dft) + if (tg->bps[READ] == bps_dft && + tg->bps[WRITE] == bps_dft && + tg->iops[READ] == iops_dft && + tg->iops[WRITE] == iops_dft) return 0; seq_printf(sf, "%s", dname); - if (tg->bps_conf[READ] == U64_MAX) + if (tg->bps[READ] == U64_MAX) seq_printf(sf, " rbps=max"); else - seq_printf(sf, " rbps=%llu", tg->bps_conf[READ]); + seq_printf(sf, " rbps=%llu", tg->bps[READ]); - if (tg->bps_conf[WRITE] == U64_MAX) + if (tg->bps[WRITE] == U64_MAX) seq_printf(sf, " wbps=max"); else - seq_printf(sf, " wbps=%llu", tg->bps_conf[WRITE]); + seq_printf(sf, " wbps=%llu", tg->bps[WRITE]); - if (tg->iops_conf[READ] == UINT_MAX) + if (tg->iops[READ] == UINT_MAX) seq_printf(sf, " riops=max"); else - seq_printf(sf, " riops=%u", tg->iops_conf[READ]); + seq_printf(sf, " riops=%u", tg->iops[READ]); - if (tg->iops_conf[WRITE] == UINT_MAX) + if (tg->iops[WRITE] == UINT_MAX) seq_printf(sf, " wiops=max"); else - seq_printf(sf, " wiops=%u", tg->iops_conf[WRITE]); + seq_printf(sf, " wiops=%u", tg->iops[WRITE]); seq_printf(sf, "\n"); return 0; diff --git a/block/blk-throttle.h b/block/blk-throttle.h index 393c3d134b9673356920e4cc9ed7e8ce166e6ac9..4d9ef5abdf21cc05f69022b6f42ce4f2275148a7 100644 --- a/block/blk-throttle.h +++ b/block/blk-throttle.h @@ -95,15 +95,11 @@ struct throtl_grp { bool has_rules_bps[2]; bool has_rules_iops[2]; - /* internally used bytes per second rate limits */ + /* bytes per second rate limits */ uint64_t bps[2]; - /* user configured bps limits */ - uint64_t bps_conf[2]; - /* internally used IOPS limits */ + /* IOPS limits */ unsigned int iops[2]; - /* user configured IOPS limits */ - unsigned int iops_conf[2]; /* Number of bytes dispatched in current slice */ uint64_t bytes_disp[2]; diff --git a/block/blk-zoned.c b/block/blk-zoned.c index 03aa4eead39e7bdbf2b66756382eebf9f7d458d2..52abebf56027a8cc4fcc0eed27047d55b006e3c9 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -450,6 +450,25 @@ static inline bool disk_zone_is_conv(struct gendisk *disk, sector_t sector) return test_bit(disk_zone_no(disk, sector), disk->conv_zones_bitmap); } +static bool disk_zone_is_last(struct gendisk *disk, struct blk_zone *zone) +{ + return zone->start + zone->len >= get_capacity(disk); +} + +static bool disk_zone_is_full(struct gendisk *disk, + unsigned int zno, unsigned int offset_in_zone) +{ + if (zno < disk->nr_zones - 1) + return offset_in_zone >= disk->zone_capacity; + return offset_in_zone >= disk->last_zone_capacity; +} + +static bool disk_zone_wplug_is_full(struct gendisk *disk, + struct blk_zone_wplug *zwplug) +{ + return disk_zone_is_full(disk, zwplug->zone_no, zwplug->wp_offset); +} + static bool disk_insert_zone_wplug(struct gendisk *disk, struct blk_zone_wplug *zwplug) { @@ -543,7 +562,7 @@ static inline bool disk_should_remove_zone_wplug(struct gendisk *disk, return false; /* We can remove zone write plugs for zones that are empty or full. */ - return !zwplug->wp_offset || zwplug->wp_offset >= disk->zone_capacity; + return !zwplug->wp_offset || disk_zone_wplug_is_full(disk, zwplug); } static void disk_remove_zone_wplug(struct gendisk *disk, @@ -664,13 +683,12 @@ static void disk_zone_wplug_abort(struct blk_zone_wplug *zwplug) static void disk_zone_wplug_abort_unaligned(struct gendisk *disk, struct blk_zone_wplug *zwplug) { - unsigned int zone_capacity = disk->zone_capacity; unsigned int wp_offset = zwplug->wp_offset; struct bio_list bl = BIO_EMPTY_LIST; struct bio *bio; while ((bio = bio_list_pop(&zwplug->bio_list))) { - if (wp_offset >= zone_capacity || + if (disk_zone_is_full(disk, zwplug->zone_no, wp_offset) || (bio_op(bio) != REQ_OP_ZONE_APPEND && bio_offset_from_zone_start(bio) != wp_offset)) { blk_zone_wplug_bio_io_error(zwplug, bio); @@ -909,7 +927,6 @@ void blk_zone_write_plug_init_request(struct request *req) sector_t req_back_sector = blk_rq_pos(req) + blk_rq_sectors(req); struct request_queue *q = req->q; struct gendisk *disk = q->disk; - unsigned int zone_capacity = disk->zone_capacity; struct blk_zone_wplug *zwplug = disk_get_zone_wplug(disk, blk_rq_pos(req)); unsigned long flags; @@ -933,7 +950,7 @@ void blk_zone_write_plug_init_request(struct request *req) * into the back of the request. */ spin_lock_irqsave(&zwplug->lock, flags); - while (zwplug->wp_offset < zone_capacity) { + while (!disk_zone_wplug_is_full(disk, zwplug)) { bio = bio_list_peek(&zwplug->bio_list); if (!bio) break; @@ -979,7 +996,7 @@ static bool blk_zone_wplug_prepare_bio(struct blk_zone_wplug *zwplug, * We know such BIO will fail, and that would potentially overflow our * write pointer offset beyond the end of the zone. */ - if (zwplug->wp_offset >= disk->zone_capacity) + if (disk_zone_wplug_is_full(disk, zwplug)) goto err; if (bio_op(bio) == REQ_OP_ZONE_APPEND) { @@ -1556,6 +1573,7 @@ void disk_free_zone_resources(struct gendisk *disk) kfree(disk->conv_zones_bitmap); disk->conv_zones_bitmap = NULL; disk->zone_capacity = 0; + disk->last_zone_capacity = 0; disk->nr_zones = 0; } @@ -1600,6 +1618,7 @@ struct blk_revalidate_zone_args { unsigned long *conv_zones_bitmap; unsigned int nr_zones; unsigned int zone_capacity; + unsigned int last_zone_capacity; sector_t sector; }; @@ -1617,6 +1636,7 @@ static int disk_update_zone_resources(struct gendisk *disk, disk->nr_zones = args->nr_zones; disk->zone_capacity = args->zone_capacity; + disk->last_zone_capacity = args->last_zone_capacity; swap(disk->conv_zones_bitmap, args->conv_zones_bitmap); if (disk->conv_zones_bitmap) nr_conv_zones = bitmap_weight(disk->conv_zones_bitmap, @@ -1668,6 +1688,9 @@ static int blk_revalidate_conv_zone(struct blk_zone *zone, unsigned int idx, return -ENODEV; } + if (disk_zone_is_last(disk, zone)) + args->last_zone_capacity = zone->capacity; + if (!disk_need_zone_resources(disk)) return 0; @@ -1693,11 +1716,14 @@ static int blk_revalidate_seq_zone(struct blk_zone *zone, unsigned int idx, /* * Remember the capacity of the first sequential zone and check - * if it is constant for all zones. + * if it is constant for all zones, ignoring the last zone as it can be + * smaller. */ if (!args->zone_capacity) args->zone_capacity = zone->capacity; - if (zone->capacity != args->zone_capacity) { + if (disk_zone_is_last(disk, zone)) { + args->last_zone_capacity = zone->capacity; + } else if (zone->capacity != args->zone_capacity) { pr_warn("%s: Invalid variable zone capacity\n", disk->disk_name); return -ENODEV; @@ -1732,7 +1758,6 @@ static int blk_revalidate_zone_cb(struct blk_zone *zone, unsigned int idx, { struct blk_revalidate_zone_args *args = data; struct gendisk *disk = args->disk; - sector_t capacity = get_capacity(disk); sector_t zone_sectors = disk->queue->limits.chunk_sectors; int ret; @@ -1743,7 +1768,7 @@ static int blk_revalidate_zone_cb(struct blk_zone *zone, unsigned int idx, return -ENODEV; } - if (zone->start >= capacity || !zone->len) { + if (zone->start >= get_capacity(disk) || !zone->len) { pr_warn("%s: Invalid zone start %llu, length %llu\n", disk->disk_name, zone->start, zone->len); return -ENODEV; @@ -1753,7 +1778,7 @@ static int blk_revalidate_zone_cb(struct blk_zone *zone, unsigned int idx, * All zones must have the same size, with the exception on an eventual * smaller last zone. */ - if (zone->start + zone->len < capacity) { + if (!disk_zone_is_last(disk, zone)) { if (zone->len != zone_sectors) { pr_warn("%s: Invalid zoned device with non constant zone size\n", disk->disk_name); diff --git a/drivers/acpi/ac.c b/drivers/acpi/ac.c index 2d4a35e6dd18d55c1ce1127ee81dbfd080a15653..09a87fa222c78fee2e4c40987d07c725e37aa5ff 100644 --- a/drivers/acpi/ac.c +++ b/drivers/acpi/ac.c @@ -145,7 +145,7 @@ static void acpi_ac_notify(acpi_handle handle, u32 event, void *data) dev_name(&adev->dev), event, (u32) ac->state); acpi_notifier_call_chain(adev, event, (u32) ac->state); - kobject_uevent(&ac->charger->dev.kobj, KOBJ_CHANGE); + power_supply_changed(ac->charger); } } @@ -268,7 +268,7 @@ static int acpi_ac_resume(struct device *dev) if (acpi_ac_get_state(ac)) return 0; if (old_state != ac->state) - kobject_uevent(&ac->charger->dev.kobj, KOBJ_CHANGE); + power_supply_changed(ac->charger); return 0; } diff --git a/drivers/acpi/apei/einj-core.c b/drivers/acpi/apei/einj-core.c index 9515bcfe5e973229139587fc2fbd5077a42cfea6..73903a497d73f7549e913ee00d85507398643464 100644 --- a/drivers/acpi/apei/einj-core.c +++ b/drivers/acpi/apei/einj-core.c @@ -909,7 +909,7 @@ static void __exit einj_exit(void) if (einj_initialized) platform_driver_unregister(&einj_driver); - platform_device_del(einj_dev); + platform_device_unregister(einj_dev); } module_init(einj_init); diff --git a/drivers/acpi/ec.c b/drivers/acpi/ec.c index e7793ee9e6498ea5edb671ef7c6f38ba470c9a44..68dd17f96f636f343f76418b1b870161816e50ba 100644 --- a/drivers/acpi/ec.c +++ b/drivers/acpi/ec.c @@ -1333,10 +1333,13 @@ acpi_ec_space_handler(u32 function, acpi_physical_address address, if (ec->busy_polling || bits > 8) acpi_ec_burst_enable(ec); - for (i = 0; i < bytes; ++i, ++address, ++value) + for (i = 0; i < bytes; ++i, ++address, ++value) { result = (function == ACPI_READ) ? acpi_ec_read(ec, address, value) : acpi_ec_write(ec, address, *value); + if (result < 0) + break; + } if (ec->busy_polling || bits > 8) acpi_ec_burst_disable(ec); @@ -1348,8 +1351,10 @@ acpi_ec_space_handler(u32 function, acpi_physical_address address, return AE_NOT_FOUND; case -ETIME: return AE_TIME; - default: + case 0: return AE_OK; + default: + return AE_ERROR; } } diff --git a/drivers/acpi/sbs.c b/drivers/acpi/sbs.c index 94e3c000df2e164824a95157715a204fd3f74526..dc8164b182dccf3fa12fd59f0430d08468ba63a1 100644 --- a/drivers/acpi/sbs.c +++ b/drivers/acpi/sbs.c @@ -610,7 +610,7 @@ static void acpi_sbs_callback(void *context) if (sbs->charger_exists) { acpi_ac_get_present(sbs); if (sbs->charger_present != saved_charger_state) - kobject_uevent(&sbs->charger->dev.kobj, KOBJ_CHANGE); + power_supply_changed(sbs->charger); } if (sbs->manager_present) { @@ -622,7 +622,7 @@ static void acpi_sbs_callback(void *context) acpi_battery_read(bat); if (saved_battery_state == bat->present) continue; - kobject_uevent(&bat->bat->dev.kobj, KOBJ_CHANGE); + power_supply_changed(bat->bat); } } } diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c index 6548f10e61d9c72ca89180e011f8e495058302a1..07d66d2c5f0dd8ecb9ccd5743dd32586b3e37ae8 100644 --- a/drivers/ata/ahci.c +++ b/drivers/ata/ahci.c @@ -429,7 +429,6 @@ static const struct pci_device_id ahci_pci_tbl[] = { { PCI_VDEVICE(INTEL, 0x02d7), board_ahci_pcs_quirk }, /* Comet Lake PCH RAID */ /* Elkhart Lake IDs 0x4b60 & 0x4b62 https://sata-io.org/product/8803 not tested yet */ { PCI_VDEVICE(INTEL, 0x4b63), board_ahci_pcs_quirk }, /* Elkhart Lake AHCI */ - { PCI_VDEVICE(INTEL, 0x7ae2), board_ahci_pcs_quirk }, /* Alder Lake-P AHCI */ /* JMicron 360/1/3/5/6, match class to avoid IDE function */ { PCI_VENDOR_ID_JMICRON, PCI_ANY_ID, PCI_ANY_ID, PCI_ANY_ID, diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c index 4f35aab81a0a38c7d25ffcafefe0dd9aa4135d22..e1bf8a19b3c89e2cdfc257578bdc9b92cc95cb80 100644 --- a/drivers/ata/libata-core.c +++ b/drivers/ata/libata-core.c @@ -4136,8 +4136,9 @@ static const struct ata_blacklist_entry ata_device_blacklist [] = { { "PIONEER BD-RW BDR-207M", NULL, ATA_HORKAGE_NOLPM }, { "PIONEER BD-RW BDR-205", NULL, ATA_HORKAGE_NOLPM }, - /* Crucial BX100 SSD 500GB has broken LPM support */ + /* Crucial devices with broken LPM support */ { "CT500BX100SSD1", NULL, ATA_HORKAGE_NOLPM }, + { "CT240BX500SSD1", NULL, ATA_HORKAGE_NOLPM }, /* 512GB MX100 with MU01 firmware has both queued TRIM and LPM issues */ { "Crucial_CT512MX100*", "MU01", ATA_HORKAGE_NO_NCQ_TRIM | @@ -4155,6 +4156,12 @@ static const struct ata_blacklist_entry ata_device_blacklist [] = { ATA_HORKAGE_ZERO_AFTER_TRIM | ATA_HORKAGE_NOLPM }, + /* AMD Radeon devices with broken LPM support */ + { "R3SL240G", NULL, ATA_HORKAGE_NOLPM }, + + /* Apacer models with LPM issues */ + { "Apacer AS340*", NULL, ATA_HORKAGE_NOLPM }, + /* These specific Samsung models/firmware-revs do not handle LPM well */ { "SAMSUNG MZMPC128HBFU-000MV", "CXM14M1Q", ATA_HORKAGE_NOLPM }, { "SAMSUNG SSD PM830 mSATA *", "CXM13D1Q", ATA_HORKAGE_NOLPM }, diff --git a/drivers/base/regmap/regmap-i2c.c b/drivers/base/regmap/regmap-i2c.c index 3ec611dc0c09fbe251e8e789e64db49127d2fea6..a905e955bbfc7869cf3eed2605f8e616465b21d6 100644 --- a/drivers/base/regmap/regmap-i2c.c +++ b/drivers/base/regmap/regmap-i2c.c @@ -350,7 +350,8 @@ static const struct regmap_bus *regmap_get_i2c_bus(struct i2c_client *i2c, if (quirks->max_write_len && (bus->max_raw_write == 0 || bus->max_raw_write > quirks->max_write_len)) - max_write = quirks->max_write_len; + max_write = quirks->max_write_len - + (config->reg_bits + config->pad_bits) / BITS_PER_BYTE; if (max_read || max_write) { ret_bus = kmemdup(bus, sizeof(*bus), GFP_KERNEL); diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index eb023d26736934d5c54cee57962145f4ee2eb93e..631dca2e4e84428859ef1e2cabb78cfd13868161 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -494,6 +494,7 @@ static ssize_t nullb_device_power_store(struct config_item *item, set_bit(NULLB_DEV_FL_CONFIGURED, &dev->flags); dev->power = newp; + ret = count; } else if (dev->power && !newp) { if (test_and_clear_bit(NULLB_DEV_FL_UP, &dev->flags)) { dev->power = newp; diff --git a/drivers/block/null_blk/zoned.c b/drivers/block/null_blk/zoned.c index 5b5a63adacc17c8929fd2dd10a29935c0f59576b..f118d304f31080f65288af40dba8c602e002bcc9 100644 --- a/drivers/block/null_blk/zoned.c +++ b/drivers/block/null_blk/zoned.c @@ -74,6 +74,17 @@ int null_init_zoned_dev(struct nullb_device *dev, return -EINVAL; } + /* + * If a smaller zone capacity was requested, do not allow a smaller last + * zone at the same time as such zone configuration does not correspond + * to any real zoned device. + */ + if (dev->zone_capacity != dev->zone_size && + dev->size & (dev->zone_size - 1)) { + pr_err("A smaller last zone is not allowed with zone capacity smaller than zone size.\n"); + return -EINVAL; + } + zone_capacity_sects = mb_to_sects(dev->zone_capacity); dev_capacity_sects = mb_to_sects(dev->size); dev->zone_size_sects = mb_to_sects(dev->zone_size); @@ -108,7 +119,7 @@ int null_init_zoned_dev(struct nullb_device *dev, if (dev->zone_max_active && dev->zone_max_open > dev->zone_max_active) { dev->zone_max_open = dev->zone_max_active; pr_info("changed the maximum number of open zones to %u\n", - dev->nr_zones); + dev->zone_max_open); } else if (dev->zone_max_open >= dev->nr_zones - dev->zone_nr_conv) { dev->zone_max_open = 0; pr_info("zone_max_open limit disabled, limit >= zone count\n"); diff --git a/drivers/char/hw_random/core.c b/drivers/char/hw_random/core.c index f5c71a617a993108049f12de67234a39405c27c2..4084df65c9fa33b6b43278e151576a1823049394 100644 --- a/drivers/char/hw_random/core.c +++ b/drivers/char/hw_random/core.c @@ -64,19 +64,6 @@ static size_t rng_buffer_size(void) return RNG_BUFFER_SIZE; } -static void add_early_randomness(struct hwrng *rng) -{ - int bytes_read; - - mutex_lock(&reading_mutex); - bytes_read = rng_get_data(rng, rng_fillbuf, 32, 0); - mutex_unlock(&reading_mutex); - if (bytes_read > 0) { - size_t entropy = bytes_read * 8 * rng->quality / 1024; - add_hwgenerator_randomness(rng_fillbuf, bytes_read, entropy, false); - } -} - static inline void cleanup_rng(struct kref *kref) { struct hwrng *rng = container_of(kref, struct hwrng, ref); @@ -340,13 +327,12 @@ static ssize_t rng_current_store(struct device *dev, const char *buf, size_t len) { int err; - struct hwrng *rng, *old_rng, *new_rng; + struct hwrng *rng, *new_rng; err = mutex_lock_interruptible(&rng_mutex); if (err) return -ERESTARTSYS; - old_rng = current_rng; if (sysfs_streq(buf, "")) { err = enable_best_rng(); } else { @@ -362,11 +348,8 @@ static ssize_t rng_current_store(struct device *dev, new_rng = get_current_rng_nolock(); mutex_unlock(&rng_mutex); - if (new_rng) { - if (new_rng != old_rng) - add_early_randomness(new_rng); + if (new_rng) put_rng(new_rng); - } return err ? : len; } @@ -544,7 +527,6 @@ int hwrng_register(struct hwrng *rng) { int err = -EINVAL; struct hwrng *tmp; - bool is_new_current = false; if (!rng->name || (!rng->data_read && !rng->read)) goto out; @@ -573,25 +555,8 @@ int hwrng_register(struct hwrng *rng) err = set_current_rng(rng); if (err) goto out_unlock; - /* to use current_rng in add_early_randomness() we need - * to take a ref - */ - is_new_current = true; - kref_get(&rng->ref); } mutex_unlock(&rng_mutex); - if (is_new_current || !rng->init) { - /* - * Use a new device's input to add some randomness to - * the system. If this rng device isn't going to be - * used right away, its init function hasn't been - * called yet by set_current_rng(); so only use the - * randomness from devices that don't need an init callback - */ - add_early_randomness(rng); - } - if (is_new_current) - put_rng(rng); return 0; out_unlock: mutex_unlock(&rng_mutex); @@ -602,12 +567,11 @@ EXPORT_SYMBOL_GPL(hwrng_register); void hwrng_unregister(struct hwrng *rng) { - struct hwrng *old_rng, *new_rng; + struct hwrng *new_rng; int err; mutex_lock(&rng_mutex); - old_rng = current_rng; list_del(&rng->list); complete_all(&rng->dying); if (current_rng == rng) { @@ -626,11 +590,8 @@ void hwrng_unregister(struct hwrng *rng) } else mutex_unlock(&rng_mutex); - if (new_rng) { - if (old_rng != new_rng) - add_early_randomness(new_rng); + if (new_rng) put_rng(new_rng); - } wait_for_completion(&rng->cleanup_done); } diff --git a/drivers/char/tpm/Kconfig b/drivers/char/tpm/Kconfig index e63a6a17793c8560085c1571e862338cf4e1c21b..cf0be8a7939de49e2ec2d752ce307d31c80ca2a9 100644 --- a/drivers/char/tpm/Kconfig +++ b/drivers/char/tpm/Kconfig @@ -29,7 +29,7 @@ if TCG_TPM config TCG_TPM2_HMAC bool "Use HMAC and encrypted transactions on the TPM bus" - default y + default X86_64 select CRYPTO_ECDH select CRYPTO_LIB_AESCFB select CRYPTO_LIB_SHA256 diff --git a/drivers/char/tpm/tpm-buf.c b/drivers/char/tpm/tpm-buf.c index 647c6ca92ac3ca3e545a78cfc39c25ec25cc3121..cad0048bcc3c6852de3633b7def9d9f604c83c29 100644 --- a/drivers/char/tpm/tpm-buf.c +++ b/drivers/char/tpm/tpm-buf.c @@ -223,30 +223,4 @@ u32 tpm_buf_read_u32(struct tpm_buf *buf, off_t *offset) } EXPORT_SYMBOL_GPL(tpm_buf_read_u32); -static u16 tpm_buf_tag(struct tpm_buf *buf) -{ - struct tpm_header *head = (struct tpm_header *)buf->data; - - return be16_to_cpu(head->tag); -} - -/** - * tpm_buf_parameters - return the TPM response parameters area of the tpm_buf - * @buf: tpm_buf to use - * - * Where the parameters are located depends on the tag of a TPM - * command (it's immediately after the header for TPM_ST_NO_SESSIONS - * or 4 bytes after for TPM_ST_SESSIONS). Evaluate this and return a - * pointer to the first byte of the parameters area. - * - * @return: pointer to parameters area - */ -u8 *tpm_buf_parameters(struct tpm_buf *buf) -{ - int offset = TPM_HEADER_SIZE; - - if (tpm_buf_tag(buf) == TPM2_ST_SESSIONS) - offset += 4; - return &buf->data[offset]; -} diff --git a/drivers/char/tpm/tpm.h b/drivers/char/tpm/tpm.h index 6b8b9956ba6944f651419a18b18bd2e4defab8b4..7bb87fa5f7a12663256442a6d1cbc0ad39cd9136 100644 --- a/drivers/char/tpm/tpm.h +++ b/drivers/char/tpm/tpm.h @@ -28,7 +28,7 @@ #include #ifdef CONFIG_X86 -#include +#include #endif #define TPM_MINOR 224 /* officially assigned */ diff --git a/drivers/char/tpm/tpm2-cmd.c b/drivers/char/tpm/tpm2-cmd.c index 0cdf892ec2a7d40929787948dc146d432f74d70f..1e856259219e2e9995b9fe7f799422d6c0bf3d87 100644 --- a/drivers/char/tpm/tpm2-cmd.c +++ b/drivers/char/tpm/tpm2-cmd.c @@ -281,6 +281,7 @@ struct tpm2_get_random_out { int tpm2_get_random(struct tpm_chip *chip, u8 *dest, size_t max) { struct tpm2_get_random_out *out; + struct tpm_header *head; struct tpm_buf buf; u32 recd; u32 num_bytes = max; @@ -288,6 +289,7 @@ int tpm2_get_random(struct tpm_chip *chip, u8 *dest, size_t max) int total = 0; int retries = 5; u8 *dest_ptr = dest; + off_t offset; if (!num_bytes || max > TPM_MAX_RNG_DATA) return -EINVAL; @@ -320,7 +322,13 @@ int tpm2_get_random(struct tpm_chip *chip, u8 *dest, size_t max) goto out; } - out = (struct tpm2_get_random_out *)tpm_buf_parameters(&buf); + head = (struct tpm_header *)buf.data; + offset = TPM_HEADER_SIZE; + /* Skip the parameter size field: */ + if (be16_to_cpu(head->tag) == TPM2_ST_SESSIONS) + offset += 4; + + out = (struct tpm2_get_random_out *)&buf.data[offset]; recd = min_t(u32, be16_to_cpu(out->size), num_bytes); if (tpm_buf_length(&buf) < TPM_HEADER_SIZE + diff --git a/drivers/char/tpm/tpm2-sessions.c b/drivers/char/tpm/tpm2-sessions.c index ea8860661876ed9ea64daed40095a4126687cbd4..907ac9956a78f02fee74aead24236ce664133b87 100644 --- a/drivers/char/tpm/tpm2-sessions.c +++ b/drivers/char/tpm/tpm2-sessions.c @@ -80,6 +80,9 @@ /* maximum number of names the TPM must remember for authorization */ #define AUTH_MAX_NAMES 3 +#define AES_KEY_BYTES AES_KEYSIZE_128 +#define AES_KEY_BITS (AES_KEY_BYTES*8) + static int tpm2_create_primary(struct tpm_chip *chip, u32 hierarchy, u32 *handle, u8 *name); @@ -954,6 +957,20 @@ int tpm2_start_auth_session(struct tpm_chip *chip) } EXPORT_SYMBOL(tpm2_start_auth_session); +/* + * A mask containing the object attributes for the kernel held null primary key + * used in HMAC encryption. For more information on specific attributes look up + * to "8.3 TPMA_OBJECT (Object Attributes)". + */ +#define TPM2_OA_NULL_KEY ( \ + TPM2_OA_NO_DA | \ + TPM2_OA_FIXED_TPM | \ + TPM2_OA_FIXED_PARENT | \ + TPM2_OA_SENSITIVE_DATA_ORIGIN | \ + TPM2_OA_USER_WITH_AUTH | \ + TPM2_OA_DECRYPT | \ + TPM2_OA_RESTRICTED) + /** * tpm2_parse_create_primary() - parse the data returned from TPM_CC_CREATE_PRIMARY * @@ -1018,7 +1035,7 @@ static int tpm2_parse_create_primary(struct tpm_chip *chip, struct tpm_buf *buf, val = tpm_buf_read_u32(buf, &offset_t); /* object properties */ - if (val != TPM2_OA_TMPL) + if (val != TPM2_OA_NULL_KEY) return -EINVAL; /* auth policy (empty) */ @@ -1178,7 +1195,7 @@ static int tpm2_create_primary(struct tpm_chip *chip, u32 hierarchy, tpm_buf_append_u16(&template, TPM_ALG_SHA256); /* object properties */ - tpm_buf_append_u32(&template, TPM2_OA_TMPL); + tpm_buf_append_u32(&template, TPM2_OA_NULL_KEY); /* sauth policy (empty) */ tpm_buf_append_u16(&template, 0); diff --git a/drivers/char/tpm/tpm_tis_core.c b/drivers/char/tpm/tpm_tis_core.c index 176cd8dbf1db2c61d5f2ed193c63822e898ac9c6..fdef214b9f6bffc24017c50271e552c6e9d44564 100644 --- a/drivers/char/tpm/tpm_tis_core.c +++ b/drivers/char/tpm/tpm_tis_core.c @@ -1020,7 +1020,8 @@ void tpm_tis_remove(struct tpm_chip *chip) interrupt = 0; tpm_tis_write32(priv, reg, ~TPM_GLOBAL_INT_ENABLE & interrupt); - flush_work(&priv->free_irq_work); + if (priv->free_irq_work.func) + flush_work(&priv->free_irq_work); tpm_tis_clkrun_enable(chip, false); diff --git a/drivers/char/tpm/tpm_tis_core.h b/drivers/char/tpm/tpm_tis_core.h index 13e99cf65efe4483053c2478aff2d3834c68609b..690ad8e9b731906e92c9d08b6541562f69057693 100644 --- a/drivers/char/tpm/tpm_tis_core.h +++ b/drivers/char/tpm/tpm_tis_core.h @@ -210,7 +210,7 @@ static inline int tpm_tis_verify_crc(struct tpm_tis_data *data, size_t len, static inline bool is_bsw(void) { #ifdef CONFIG_X86 - return ((boot_cpu_data.x86_model == INTEL_FAM6_ATOM_AIRMONT) ? 1 : 0); + return (boot_cpu_data.x86_vfm == INTEL_ATOM_AIRMONT) ? 1 : 0; #else return false; #endif diff --git a/drivers/char/tpm/tpm_tis_spi_main.c b/drivers/char/tpm/tpm_tis_spi_main.c index 3f9eaf27b41b89a8ab8133cfcaec7a1e2c3e86f0..c9eca24bbad478ada329f8c1e1eec3fa6d1ab232 100644 --- a/drivers/char/tpm/tpm_tis_spi_main.c +++ b/drivers/char/tpm/tpm_tis_spi_main.c @@ -37,6 +37,7 @@ #include "tpm_tis_spi.h" #define MAX_SPI_FRAMESIZE 64 +#define SPI_HDRSIZE 4 /* * TCG SPI flow control is documented in section 6.4 of the spec[1]. In short, @@ -247,7 +248,7 @@ static int tpm_tis_spi_write_bytes(struct tpm_tis_data *data, u32 addr, int tpm_tis_spi_init(struct spi_device *spi, struct tpm_tis_spi_phy *phy, int irq, const struct tpm_tis_phy_ops *phy_ops) { - phy->iobuf = devm_kmalloc(&spi->dev, MAX_SPI_FRAMESIZE, GFP_KERNEL); + phy->iobuf = devm_kmalloc(&spi->dev, SPI_HDRSIZE + MAX_SPI_FRAMESIZE, GFP_KERNEL); if (!phy->iobuf) return -ENOMEM; diff --git a/drivers/cpufreq/amd-pstate-ut.c b/drivers/cpufreq/amd-pstate-ut.c index f04ae67dda372d2787bba3bb45d57f53515e2bc0..fc275d41d51e9b51a93ccc26f36e5592c4228d60 100644 --- a/drivers/cpufreq/amd-pstate-ut.c +++ b/drivers/cpufreq/amd-pstate-ut.c @@ -26,10 +26,11 @@ #include #include #include -#include #include +#include "amd-pstate.h" + /* * Abbreviations: * amd_pstate_ut: used as a shortform for AMD P-State unit test. diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c index 1b7e82a0ad2e62f4e0f3e520bfa9408888bc4434..9ad62dbe8bfbff6f01a0d89630bb12edda6e9349 100644 --- a/drivers/cpufreq/amd-pstate.c +++ b/drivers/cpufreq/amd-pstate.c @@ -36,7 +36,6 @@ #include #include #include -#include #include #include @@ -46,6 +45,8 @@ #include #include #include + +#include "amd-pstate.h" #include "amd-pstate-trace.h" #define AMD_PSTATE_TRANSITION_LATENCY 20000 @@ -53,6 +54,37 @@ #define CPPC_HIGHEST_PERF_PERFORMANCE 196 #define CPPC_HIGHEST_PERF_DEFAULT 166 +#define AMD_CPPC_EPP_PERFORMANCE 0x00 +#define AMD_CPPC_EPP_BALANCE_PERFORMANCE 0x80 +#define AMD_CPPC_EPP_BALANCE_POWERSAVE 0xBF +#define AMD_CPPC_EPP_POWERSAVE 0xFF + +/* + * enum amd_pstate_mode - driver working mode of amd pstate + */ +enum amd_pstate_mode { + AMD_PSTATE_UNDEFINED = 0, + AMD_PSTATE_DISABLE, + AMD_PSTATE_PASSIVE, + AMD_PSTATE_ACTIVE, + AMD_PSTATE_GUIDED, + AMD_PSTATE_MAX, +}; + +static const char * const amd_pstate_mode_string[] = { + [AMD_PSTATE_UNDEFINED] = "undefined", + [AMD_PSTATE_DISABLE] = "disable", + [AMD_PSTATE_PASSIVE] = "passive", + [AMD_PSTATE_ACTIVE] = "active", + [AMD_PSTATE_GUIDED] = "guided", + NULL, +}; + +struct quirk_entry { + u32 nominal_freq; + u32 lowest_freq; +}; + /* * TODO: We need more time to fine tune processors with shared memory solution * with community together. @@ -669,7 +701,7 @@ static int amd_pstate_set_boost(struct cpufreq_policy *policy, int state) if (state) policy->cpuinfo.max_freq = cpudata->max_freq; else - policy->cpuinfo.max_freq = cpudata->nominal_freq; + policy->cpuinfo.max_freq = cpudata->nominal_freq * 1000; policy->max = policy->cpuinfo.max_freq; diff --git a/include/linux/amd-pstate.h b/drivers/cpufreq/amd-pstate.h similarity index 82% rename from include/linux/amd-pstate.h rename to drivers/cpufreq/amd-pstate.h index d58fc022ec466e2041c081a14783844daadbb84c..e6a28e7f4dbf12a5ef4a86e2f966c2b454fc420f 100644 --- a/include/linux/amd-pstate.h +++ b/drivers/cpufreq/amd-pstate.h @@ -1,7 +1,5 @@ /* SPDX-License-Identifier: GPL-2.0-only */ /* - * linux/include/linux/amd-pstate.h - * * Copyright (C) 2022 Advanced Micro Devices, Inc. * * Author: Meng Li @@ -12,11 +10,6 @@ #include -#define AMD_CPPC_EPP_PERFORMANCE 0x00 -#define AMD_CPPC_EPP_BALANCE_PERFORMANCE 0x80 -#define AMD_CPPC_EPP_BALANCE_POWERSAVE 0xBF -#define AMD_CPPC_EPP_POWERSAVE 0xFF - /********************************************************************* * AMD P-state INTERFACE * *********************************************************************/ @@ -108,30 +101,4 @@ struct amd_cpudata { bool suspended; }; -/* - * enum amd_pstate_mode - driver working mode of amd pstate - */ -enum amd_pstate_mode { - AMD_PSTATE_UNDEFINED = 0, - AMD_PSTATE_DISABLE, - AMD_PSTATE_PASSIVE, - AMD_PSTATE_ACTIVE, - AMD_PSTATE_GUIDED, - AMD_PSTATE_MAX, -}; - -static const char * const amd_pstate_mode_string[] = { - [AMD_PSTATE_UNDEFINED] = "undefined", - [AMD_PSTATE_DISABLE] = "disable", - [AMD_PSTATE_PASSIVE] = "passive", - [AMD_PSTATE_ACTIVE] = "active", - [AMD_PSTATE_GUIDED] = "guided", - NULL, -}; - -struct quirk_entry { - u32 nominal_freq; - u32 lowest_freq; -}; - #endif /* _LINUX_AMD_PSTATE_H */ diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c index 4b986c044741eda32d5e93f9457c7b606ccfacc0..65d3f79104bd509658b8ed6c7c4e7477844e0a08 100644 --- a/drivers/cpufreq/intel_pstate.c +++ b/drivers/cpufreq/intel_pstate.c @@ -1153,7 +1153,8 @@ static void intel_pstate_update_policies(void) static void __intel_pstate_update_max_freq(struct cpudata *cpudata, struct cpufreq_policy *policy) { - intel_pstate_get_hwp_cap(cpudata); + if (hwp_active) + intel_pstate_get_hwp_cap(cpudata); policy->cpuinfo.max_freq = READ_ONCE(global.no_turbo) ? cpudata->pstate.max_freq : cpudata->pstate.turbo_freq; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 00a9f0eef8dd2c6292b7772038cd7af6835d5dd3..3c2b6144be23cd1a6af9988f1e77b51fd4d421a5 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2352,15 +2352,6 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, struct device *dev; int rc; - switch (mode) { - case CXL_DECODER_RAM: - case CXL_DECODER_PMEM: - break; - default: - dev_err(&cxlrd->cxlsd.cxld.dev, "unsupported mode %d\n", mode); - return ERR_PTR(-EINVAL); - } - cxlr = cxl_region_alloc(cxlrd, id); if (IS_ERR(cxlr)) return cxlr; @@ -2415,6 +2406,15 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, { int rc; + switch (mode) { + case CXL_DECODER_RAM: + case CXL_DECODER_PMEM: + break; + default: + dev_err(&cxlrd->cxlsd.cxld.dev, "unsupported mode %d\n", mode); + return ERR_PTR(-EINVAL); + } + rc = memregion_alloc(GFP_KERNEL); if (rc < 0) return ERR_PTR(rc); diff --git a/drivers/dma-buf/st-dma-fence.c b/drivers/dma-buf/st-dma-fence.c index b7c6f7ea9e0c83910b5a7d5001c25e0b96fa7492..6a1bfcd0cc21081cb42db82b3c61032ad2e401b2 100644 --- a/drivers/dma-buf/st-dma-fence.c +++ b/drivers/dma-buf/st-dma-fence.c @@ -540,6 +540,12 @@ static int race_signal_callback(void *arg) t[i].before = pass; t[i].task = kthread_run(thread_signal_callback, &t[i], "dma-fence:%d", i); + if (IS_ERR(t[i].task)) { + ret = PTR_ERR(t[i].task); + while (--i >= 0) + kthread_stop_put(t[i].task); + return ret; + } get_task_struct(t[i].task); } diff --git a/drivers/dma-buf/sync_debug.c b/drivers/dma-buf/sync_debug.c index 101394f16930f81f1ac98da172a50517e7534cff..237bce21d1e724632f56646c914cc31bb9efe876 100644 --- a/drivers/dma-buf/sync_debug.c +++ b/drivers/dma-buf/sync_debug.c @@ -110,12 +110,12 @@ static void sync_print_obj(struct seq_file *s, struct sync_timeline *obj) seq_printf(s, "%s: %d\n", obj->name, obj->value); - spin_lock_irq(&obj->lock); + spin_lock(&obj->lock); /* Caller already disabled IRQ. */ list_for_each(pos, &obj->pt_list) { struct sync_pt *pt = container_of(pos, struct sync_pt, link); sync_print_fence(s, &pt->base, false); } - spin_unlock_irq(&obj->lock); + spin_unlock(&obj->lock); } static void sync_print_sync_file(struct seq_file *s, diff --git a/drivers/dma/ti/k3-udma-glue.c b/drivers/dma/ti/k3-udma-glue.c index c9b93055dc9d3d7a003474b815317a9587401d47..b96b448a0e69d212547a879037e7247a92da219d 100644 --- a/drivers/dma/ti/k3-udma-glue.c +++ b/drivers/dma/ti/k3-udma-glue.c @@ -1531,6 +1531,9 @@ int k3_udma_glue_rx_get_irq(struct k3_udma_glue_rx_channel *rx_chn, flow->virq = k3_ringacc_get_ring_irq_num(flow->ringrx); } + if (!flow->virq) + return -ENXIO; + return flow->virq; } EXPORT_SYMBOL_GPL(k3_udma_glue_rx_get_irq); diff --git a/drivers/firewire/packet-serdes-test.c b/drivers/firewire/packet-serdes-test.c index f93c966e794d97b914b1d681990d6f8a4109b8ae..e83b1fece780eeae4a50d6dbe943085778649e63 100644 --- a/drivers/firewire/packet-serdes-test.c +++ b/drivers/firewire/packet-serdes-test.c @@ -579,4 +579,5 @@ static struct kunit_suite packet_serdes_test_suite = { }; kunit_test_suite(packet_serdes_test_suite); +MODULE_DESCRIPTION("FireWire packet serialization/deserialization unit test suite"); MODULE_LICENSE("GPL"); diff --git a/drivers/firewire/uapi-test.c b/drivers/firewire/uapi-test.c index 2fcbede4fab1f51a622df0511b9695fc9d400175..bc3f10a2e516e076ee63d762b9d1e538fb44f8c2 100644 --- a/drivers/firewire/uapi-test.c +++ b/drivers/firewire/uapi-test.c @@ -86,4 +86,5 @@ static struct kunit_suite structure_layout_test_suite = { }; kunit_test_suite(structure_layout_test_suite); +MODULE_DESCRIPTION("FireWire UAPI unit test suite"); MODULE_LICENSE("GPL"); diff --git a/drivers/firmware/efi/efi-pstore.c b/drivers/firmware/efi/efi-pstore.c index 5b9dc26e6bcb963289debaee0084ed9494782d71..552c78f5f05902a20f78ad7671bbb36d5542a7f2 100644 --- a/drivers/firmware/efi/efi-pstore.c +++ b/drivers/firmware/efi/efi-pstore.c @@ -136,7 +136,7 @@ static int efi_pstore_read_func(struct pstore_record *record, &size, record->buf); if (status != EFI_SUCCESS) { kfree(record->buf); - return -EIO; + return efi_status_to_err(status); } /* @@ -189,7 +189,7 @@ static ssize_t efi_pstore_read(struct pstore_record *record) return 0; if (status != EFI_SUCCESS) - return -EIO; + return efi_status_to_err(status); /* skip variables that don't concern us */ if (efi_guidcmp(guid, LINUX_EFI_CRASH_GUID)) @@ -227,7 +227,7 @@ static int efi_pstore_write(struct pstore_record *record) record->size, record->psi->buf, true); efivar_unlock(); - return status == EFI_SUCCESS ? 0 : -EIO; + return efi_status_to_err(status); }; static int efi_pstore_erase(struct pstore_record *record) @@ -238,7 +238,7 @@ static int efi_pstore_erase(struct pstore_record *record) PSTORE_EFI_ATTRIBUTES, 0, NULL); if (status != EFI_SUCCESS && status != EFI_NOT_FOUND) - return -EIO; + return efi_status_to_err(status); return 0; } diff --git a/drivers/firmware/efi/libstub/loongarch.c b/drivers/firmware/efi/libstub/loongarch.c index 684c9354637c6541c1d695ab93e80551f6c5cc1e..d0ef93551c44f64affb5a152b277a8b0c4e827fb 100644 --- a/drivers/firmware/efi/libstub/loongarch.c +++ b/drivers/firmware/efi/libstub/loongarch.c @@ -41,7 +41,7 @@ static efi_status_t exit_boot_func(struct efi_boot_memmap *map, void *priv) unsigned long __weak kernel_entry_address(unsigned long kernel_addr, efi_loaded_image_t *image) { - return *(unsigned long *)(kernel_addr + 8) - VMLINUX_LOAD_ADDRESS + kernel_addr; + return *(unsigned long *)(kernel_addr + 8) - PHYSADDR(VMLINUX_LOAD_ADDRESS) + kernel_addr; } efi_status_t efi_boot_kernel(void *handle, efi_loaded_image_t *image, diff --git a/drivers/firmware/efi/libstub/zboot.lds b/drivers/firmware/efi/libstub/zboot.lds index ac8c0ef851581f5d3eb51260ac1d2a6b92955bb6..af2c82f7bd9024094470a443d46dfc3381862ec8 100644 --- a/drivers/firmware/efi/libstub/zboot.lds +++ b/drivers/firmware/efi/libstub/zboot.lds @@ -41,6 +41,7 @@ SECTIONS } /DISCARD/ : { + *(.discard .discard.*) *(.modinfo .init.modinfo) } } diff --git a/drivers/firmware/efi/runtime-wrappers.c b/drivers/firmware/efi/runtime-wrappers.c index 5d56bc40a79d727e1b76fcea552827829f900435..708b777857d3417f2205a7bb851790e4e34ebf90 100644 --- a/drivers/firmware/efi/runtime-wrappers.c +++ b/drivers/firmware/efi/runtime-wrappers.c @@ -213,7 +213,7 @@ extern struct semaphore __efi_uv_runtime_lock __alias(efi_runtime_lock); * Calls the appropriate efi_runtime_service() with the appropriate * arguments. */ -static void efi_call_rts(struct work_struct *work) +static void __nocfi efi_call_rts(struct work_struct *work) { const union efi_rts_args *args = efi_rts_work.args; efi_status_t status = EFI_NOT_FOUND; @@ -435,7 +435,7 @@ static efi_status_t virt_efi_set_variable(efi_char16_t *name, return status; } -static efi_status_t +static efi_status_t __nocfi virt_efi_set_variable_nb(efi_char16_t *name, efi_guid_t *vendor, u32 attr, unsigned long data_size, void *data) { @@ -469,7 +469,7 @@ static efi_status_t virt_efi_query_variable_info(u32 attr, return status; } -static efi_status_t +static efi_status_t __nocfi virt_efi_query_variable_info_nb(u32 attr, u64 *storage_space, u64 *remaining_space, u64 *max_variable_size) { @@ -499,10 +499,9 @@ static efi_status_t virt_efi_get_next_high_mono_count(u32 *count) return status; } -static void virt_efi_reset_system(int reset_type, - efi_status_t status, - unsigned long data_size, - efi_char16_t *data) +static void __nocfi +virt_efi_reset_system(int reset_type, efi_status_t status, + unsigned long data_size, efi_char16_t *data) { if (down_trylock(&efi_runtime_lock)) { pr_warn("failed to invoke the reset_system() runtime service:\n" diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 8975cf41a91ac90672167d5abab8f13275ced0dc..48ad0c04aa72b7ea760687070e6ff2e80a4674c9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -196,7 +196,7 @@ int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev, return -EINVAL; vram_size = KFD_XCP_MEMORY_SIZE(adev, xcp_id); - if (adev->gmc.is_app_apu || adev->flags & AMD_IS_APU) { + if (adev->flags & AMD_IS_APU) { system_mem_needed = size; ttm_mem_needed = size; } @@ -233,7 +233,7 @@ int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev, if (adev && xcp_id >= 0) { adev->kfd.vram_used[xcp_id] += vram_needed; adev->kfd.vram_used_aligned[xcp_id] += - (adev->gmc.is_app_apu || adev->flags & AMD_IS_APU) ? + (adev->flags & AMD_IS_APU) ? vram_needed : ALIGN(vram_needed, VRAM_AVAILABLITY_ALIGN); } @@ -261,7 +261,7 @@ void amdgpu_amdkfd_unreserve_mem_limit(struct amdgpu_device *adev, if (adev) { adev->kfd.vram_used[xcp_id] -= size; - if (adev->gmc.is_app_apu || adev->flags & AMD_IS_APU) { + if (adev->flags & AMD_IS_APU) { adev->kfd.vram_used_aligned[xcp_id] -= size; kfd_mem_limit.system_mem_used -= size; kfd_mem_limit.ttm_mem_used -= size; @@ -890,7 +890,7 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, * if peer device has large BAR. In contrast, access over xGMI is * allowed for both small and large BAR configurations of peer device */ - if ((adev != bo_adev && !(adev->gmc.is_app_apu || adev->flags & AMD_IS_APU)) && + if ((adev != bo_adev && !(adev->flags & AMD_IS_APU)) && ((mem->domain == AMDGPU_GEM_DOMAIN_VRAM) || (mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL) || (mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_MMIO_REMAP))) { @@ -1658,7 +1658,7 @@ size_t amdgpu_amdkfd_get_available_memory(struct amdgpu_device *adev, - atomic64_read(&adev->vram_pin_size) - reserved_for_pt; - if (adev->gmc.is_app_apu || adev->flags & AMD_IS_APU) { + if (adev->flags & AMD_IS_APU) { system_mem_available = no_system_mem_limit ? kfd_mem_limit.max_system_mem_limit : kfd_mem_limit.max_system_mem_limit - @@ -1706,7 +1706,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( if (flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) { domain = alloc_domain = AMDGPU_GEM_DOMAIN_VRAM; - if (adev->gmc.is_app_apu || adev->flags & AMD_IS_APU) { + if (adev->flags & AMD_IS_APU) { domain = AMDGPU_GEM_DOMAIN_GTT; alloc_domain = AMDGPU_GEM_DOMAIN_GTT; alloc_flags = 0; @@ -1953,7 +1953,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( if (size) { if (!is_imported && (mem->bo->preferred_domains == AMDGPU_GEM_DOMAIN_VRAM || - ((adev->gmc.is_app_apu || adev->flags & AMD_IS_APU) && + ((adev->flags & AMD_IS_APU) && mem->bo->preferred_domains == AMDGPU_GEM_DOMAIN_GTT))) *size = bo_size; else @@ -2376,7 +2376,7 @@ static int import_obj_create(struct amdgpu_device *adev, (*mem)->bo = bo; (*mem)->va = va; (*mem)->domain = (bo->preferred_domains & AMDGPU_GEM_DOMAIN_VRAM) && - !(adev->gmc.is_app_apu || adev->flags & AMD_IS_APU) ? + !(adev->flags & AMD_IS_APU) ? AMDGPU_GEM_DOMAIN_VRAM : AMDGPU_GEM_DOMAIN_GTT; (*mem)->mapped_to_gpu_memory = 0; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 861ccff78af954228408efa5832858a0c52f63dd..932dc93b2e631d38b7ccf1e1edf0bbadf00eaf41 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -5944,13 +5944,18 @@ static void amdgpu_device_partner_bandwidth(struct amdgpu_device *adev, *speed = PCI_SPEED_UNKNOWN; *width = PCIE_LNK_WIDTH_UNKNOWN; - while ((parent = pci_upstream_bridge(parent))) { - /* skip upstream/downstream switches internal to dGPU*/ - if (parent->vendor == PCI_VENDOR_ID_ATI) - continue; - *speed = pcie_get_speed_cap(parent); - *width = pcie_get_width_cap(parent); - break; + if (amdgpu_device_pcie_dynamic_switching_supported(adev)) { + while ((parent = pci_upstream_bridge(parent))) { + /* skip upstream/downstream switches internal to dGPU*/ + if (parent->vendor == PCI_VENDOR_ID_ATI) + continue; + *speed = pcie_get_speed_cap(parent); + *width = pcie_get_width_cap(parent); + break; + } + } else { + /* use the current speeds rather than max if switching is not supported */ + pcie_bandwidth_available(adev->pdev, NULL, speed, width); } } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h index c8980d5f6540a6b5b966d34c4143519ed3c3f833..7021c4a66fb5e91a282d92f16dbe540ae205f355 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h @@ -46,7 +46,7 @@ struct amdgpu_iv_entry; #define AMDGPU_RAS_GPU_ERR_HBM_BIST_TEST(x) AMDGPU_GET_REG_FIELD(x, 7, 7) #define AMDGPU_RAS_GPU_ERR_SOCKET_ID(x) AMDGPU_GET_REG_FIELD(x, 10, 8) #define AMDGPU_RAS_GPU_ERR_AID_ID(x) AMDGPU_GET_REG_FIELD(x, 12, 11) -#define AMDGPU_RAS_GPU_ERR_HBM_ID(x) AMDGPU_GET_REG_FIELD(x, 13, 13) +#define AMDGPU_RAS_GPU_ERR_HBM_ID(x) AMDGPU_GET_REG_FIELD(x, 14, 13) #define AMDGPU_RAS_GPU_ERR_BOOT_STATUS(x) AMDGPU_GET_REG_FIELD(x, 31, 31) #define AMDGPU_RAS_BOOT_STATUS_POLLING_LIMIT 1000 diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c index 7fdd306a48a0ecb3729a3632c1c38385802e295c..f07647a9a9d97bb87bf1c6704cf12674d3f89b4e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c @@ -706,11 +706,15 @@ int amdgpu_vm_pde_update(struct amdgpu_vm_update_params *params, struct amdgpu_vm_bo_base *entry) { struct amdgpu_vm_bo_base *parent = amdgpu_vm_pt_parent(entry); - struct amdgpu_bo *bo = parent->bo, *pbo; + struct amdgpu_bo *bo, *pbo; struct amdgpu_vm *vm = params->vm; uint64_t pde, pt, flags; unsigned int level; + if (WARN_ON(!parent)) + return -EINVAL; + + bo = parent->bo; for (level = 0, pbo = bo->parent; pbo; ++level) pbo = pbo->parent; diff --git a/drivers/gpu/drm/amd/amdgpu/aqua_vanjaram.c b/drivers/gpu/drm/amd/amdgpu/aqua_vanjaram.c index 414ea3f560a7a54b72ffce7290c1d41ed126798c..d4e2aed2efa3316cc857786c4744424108ebbfaa 100644 --- a/drivers/gpu/drm/amd/amdgpu/aqua_vanjaram.c +++ b/drivers/gpu/drm/amd/amdgpu/aqua_vanjaram.c @@ -422,7 +422,7 @@ __aqua_vanjaram_get_auto_mode(struct amdgpu_xcp_mgr *xcp_mgr) if (adev->gmc.num_mem_partitions == num_xcc / 2) return (adev->flags & AMD_IS_APU) ? AMDGPU_TPX_PARTITION_MODE : - AMDGPU_QPX_PARTITION_MODE; + AMDGPU_CPX_PARTITION_MODE; if (adev->gmc.num_mem_partitions == 2 && !(adev->flags & AMD_IS_APU)) return AMDGPU_DPX_PARTITION_MODE; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c index 9596bca572129d108e5f3d2f80f815e8e011a68a..afc57df421cd9c4d3bf2fb52b56bdfe0d3227afe 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c @@ -408,15 +408,8 @@ struct kfd_dev *kgd2kfd_probe(struct amdgpu_device *adev, bool vf) f2g = &gfx_v11_kfd2kgd; break; case IP_VERSION(11, 0, 3): - if ((adev->pdev->device == 0x7460 && - adev->pdev->revision == 0x00) || - (adev->pdev->device == 0x7461 && - adev->pdev->revision == 0x00)) - /* Note: Compiler version is 11.0.5 while HW version is 11.0.3 */ - gfx_target_version = 110005; - else - /* Note: Compiler version is 11.0.1 while HW version is 11.0.3 */ - gfx_target_version = 110001; + /* Note: Compiler version is 11.0.1 while HW version is 11.0.3 */ + gfx_target_version = 110001; f2g = &gfx_v11_kfd2kgd; break; case IP_VERSION(11, 5, 0): diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c index 4816fcb9803a9d0b631cd5943341f06201c27cf6..8ee3d07ffbdfa243623a93fa4f7aeb56ba02334f 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c @@ -1023,7 +1023,7 @@ int kgd2kfd_init_zone_device(struct amdgpu_device *adev) if (amdgpu_ip_version(adev, GC_HWIP, 0) < IP_VERSION(9, 0, 1)) return -EINVAL; - if (adev->gmc.is_app_apu || adev->flags & AMD_IS_APU) + if (adev->flags & AMD_IS_APU) return 0; pgmap = &kfddev->pgmap; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c index 069b81eeea03cb7f84f9b9acdd958f2339bae124..31e500859ab012500d6678fe36ce89e8936f0f30 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c @@ -2619,8 +2619,7 @@ svm_range_best_restore_location(struct svm_range *prange, return -1; } - if (node->adev->gmc.is_app_apu || - node->adev->flags & AMD_IS_APU) + if (node->adev->flags & AMD_IS_APU) return 0; if (prange->preferred_loc == gpuid || @@ -3338,8 +3337,7 @@ svm_range_best_prefetch_location(struct svm_range *prange) goto out; } - if (bo_node->adev->gmc.is_app_apu || - bo_node->adev->flags & AMD_IS_APU) { + if (bo_node->adev->flags & AMD_IS_APU) { best_loc = 0; goto out; } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.h b/drivers/gpu/drm/amd/amdkfd/kfd_svm.h index 9c37bd0567efa5be8f5da736f0c96eea5d2a6320..70c1776611c472cb9b213729f0567ac20600ef4c 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.h @@ -201,7 +201,6 @@ void svm_range_list_lock_and_flush_work(struct svm_range_list *svms, struct mm_s * is initialized to not 0 when page migration register device memory. */ #define KFD_IS_SVM_API_SUPPORTED(adev) ((adev)->kfd.pgmap.type != 0 ||\ - (adev)->gmc.is_app_apu ||\ ((adev)->flags & AMD_IS_APU)) void svm_range_bo_unref_async(struct svm_range_bo *svm_bo); diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c index 35733d5c51938cb069fd0ee8b37654342449e19a..a5e1a93ddaea2ab8b11274a6d435861d9d2fc7a4 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c @@ -613,6 +613,9 @@ dm_dp_add_mst_connector(struct drm_dp_mst_topology_mgr *mgr, &connector->base, dev->mode_config.tile_property, 0); + connector->colorspace_property = master->base.colorspace_property; + if (connector->colorspace_property) + drm_connector_attach_colorspace_property(connector); drm_connector_set_path_property(connector, pathprop); diff --git a/drivers/gpu/drm/amd/include/atomfirmware.h b/drivers/gpu/drm/amd/include/atomfirmware.h index 1acb2d2c5597b53d556a8690c7b5b851229bbd0a..5716918372001a394967b3dcf620e623f65e9467 100644 --- a/drivers/gpu/drm/amd/include/atomfirmware.h +++ b/drivers/gpu/drm/amd/include/atomfirmware.h @@ -3583,7 +3583,7 @@ struct atom_gpio_voltage_object_v4 uint8_t phase_delay_us; // phase delay in unit of micro second uint8_t reserved; uint32_t gpio_mask_val; // GPIO Mask value - struct atom_voltage_gpio_map_lut voltage_gpio_lut[1]; + struct atom_voltage_gpio_map_lut voltage_gpio_lut[] __counted_by(gpio_entry_num); }; struct atom_svid2_voltage_object_v4 diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c index 706265220292a974e6e1f8d2da588a267eaaee06..90703f4542aba5bda708278b7797219153e60eed 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c +++ b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c @@ -1562,7 +1562,6 @@ static int smu_v14_0_2_i2c_control_init(struct smu_context *smu) smu_i2c->port = i; mutex_init(&smu_i2c->mutex); control->owner = THIS_MODULE; - control->class = I2C_CLASS_SPD; control->dev.parent = &adev->pdev->dev; control->algo = &smu_v14_0_2_i2c_algo; snprintf(control->name, sizeof(control->name), "AMDGPU SMU %d", i); diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c index 94f8c34fc2932867c70f7816631ccd99fbac4691..6a8e45e9d0ecc19a232d489af91fc2464fff9975 100644 --- a/drivers/gpu/drm/drm_buddy.c +++ b/drivers/gpu/drm/drm_buddy.c @@ -239,7 +239,7 @@ int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size) if (size < chunk_size) return -EINVAL; - if (chunk_size < PAGE_SIZE) + if (chunk_size < SZ_4K) return -EINVAL; if (!is_power_of_2(chunk_size)) diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c index 177773bcdbfd4797b1d36e065a77282e8d7750c5..53c003983ad183a55219fd811da2d9adc06e0b28 100644 --- a/drivers/gpu/drm/drm_gem_shmem_helper.c +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c @@ -233,6 +233,8 @@ int drm_gem_shmem_pin_locked(struct drm_gem_shmem_object *shmem) dma_resv_assert_held(shmem->base.resv); + drm_WARN_ON(shmem->base.dev, shmem->base.import_attach); + ret = drm_gem_shmem_get_pages(shmem); return ret; @@ -611,6 +613,9 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct return ret; } + if (is_cow_mapping(vma->vm_flags)) + return -EINVAL; + dma_resv_lock(shmem->base.resv, NULL); ret = drm_gem_shmem_get_pages(shmem); dma_resv_unlock(shmem->base.resv); diff --git a/drivers/gpu/drm/i915/display/intel_audio.c b/drivers/gpu/drm/i915/display/intel_audio.c index ed81e1466c4b5ae1791dfa0b0462ed3ae5aec95d..40e7d862675eed5580d01e6ea707bde3c9af2750 100644 --- a/drivers/gpu/drm/i915/display/intel_audio.c +++ b/drivers/gpu/drm/i915/display/intel_audio.c @@ -1252,17 +1252,6 @@ static const struct component_ops i915_audio_component_bind_ops = { static void i915_audio_component_init(struct drm_i915_private *i915) { u32 aud_freq, aud_freq_init; - int ret; - - ret = component_add_typed(i915->drm.dev, - &i915_audio_component_bind_ops, - I915_COMPONENT_AUDIO); - if (ret < 0) { - drm_err(&i915->drm, - "failed to add audio component (%d)\n", ret); - /* continue with reduced functionality */ - return; - } if (DISPLAY_VER(i915) >= 9) { aud_freq_init = intel_de_read(i915, AUD_FREQ_CNTRL); @@ -1285,6 +1274,21 @@ static void i915_audio_component_init(struct drm_i915_private *i915) /* init with current cdclk */ intel_audio_cdclk_change_post(i915); +} + +static void i915_audio_component_register(struct drm_i915_private *i915) +{ + int ret; + + ret = component_add_typed(i915->drm.dev, + &i915_audio_component_bind_ops, + I915_COMPONENT_AUDIO); + if (ret < 0) { + drm_err(&i915->drm, + "failed to add audio component (%d)\n", ret); + /* continue with reduced functionality */ + return; + } i915->display.audio.component_registered = true; } @@ -1317,6 +1321,12 @@ void intel_audio_init(struct drm_i915_private *i915) i915_audio_component_init(i915); } +void intel_audio_register(struct drm_i915_private *i915) +{ + if (!i915->display.audio.lpe.platdev) + i915_audio_component_register(i915); +} + /** * intel_audio_deinit() - deinitialize the audio driver * @i915: the i915 drm device private data diff --git a/drivers/gpu/drm/i915/display/intel_audio.h b/drivers/gpu/drm/i915/display/intel_audio.h index 9327954b801e579ec836383bd88c7f5995208ffe..576c061d72a45099e295a8d204a6b3d0b4f1dd7b 100644 --- a/drivers/gpu/drm/i915/display/intel_audio.h +++ b/drivers/gpu/drm/i915/display/intel_audio.h @@ -28,6 +28,7 @@ void intel_audio_codec_get_config(struct intel_encoder *encoder, void intel_audio_cdclk_change_pre(struct drm_i915_private *dev_priv); void intel_audio_cdclk_change_post(struct drm_i915_private *dev_priv); void intel_audio_init(struct drm_i915_private *dev_priv); +void intel_audio_register(struct drm_i915_private *i915); void intel_audio_deinit(struct drm_i915_private *dev_priv); void intel_audio_sdp_split_update(const struct intel_crtc_state *crtc_state); diff --git a/drivers/gpu/drm/i915/display/intel_display_driver.c b/drivers/gpu/drm/i915/display/intel_display_driver.c index 89bd032ed995e73e90bc584a0009ddd311a04f62..794b4af380558d5da9d123c59f12b1deadacbdca 100644 --- a/drivers/gpu/drm/i915/display/intel_display_driver.c +++ b/drivers/gpu/drm/i915/display/intel_display_driver.c @@ -540,6 +540,8 @@ void intel_display_driver_register(struct drm_i915_private *i915) intel_display_driver_enable_user_access(i915); + intel_audio_register(i915); + intel_display_debugfs_register(i915); /* diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c index 42619fc05de484fb1f54e2194f71157ed7c98fa0..090724fa766c9eeac0e9f8bcde0949e0d8888a36 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c @@ -255,6 +255,7 @@ struct i915_execbuffer { struct intel_context *context; /* logical state for the request */ struct i915_gem_context *gem_context; /** caller's context */ intel_wakeref_t wakeref; + intel_wakeref_t wakeref_gt0; /** our requests to build */ struct i915_request *requests[MAX_ENGINE_INSTANCE + 1]; @@ -2685,6 +2686,7 @@ static int eb_select_engine(struct i915_execbuffer *eb) { struct intel_context *ce, *child; + struct intel_gt *gt; unsigned int idx; int err; @@ -2708,10 +2710,17 @@ eb_select_engine(struct i915_execbuffer *eb) } } eb->num_batches = ce->parallel.number_children + 1; + gt = ce->engine->gt; for_each_child(ce, child) intel_context_get(child); eb->wakeref = intel_gt_pm_get(ce->engine->gt); + /* + * Keep GT0 active on MTL so that i915_vma_parked() doesn't + * free VMAs while execbuf ioctl is validating VMAs. + */ + if (gt->info.id) + eb->wakeref_gt0 = intel_gt_pm_get(to_gt(gt->i915)); if (!test_bit(CONTEXT_ALLOC_BIT, &ce->flags)) { err = intel_context_alloc_state(ce); @@ -2750,6 +2759,9 @@ eb_select_engine(struct i915_execbuffer *eb) return err; err: + if (gt->info.id) + intel_gt_pm_put(to_gt(gt->i915), eb->wakeref_gt0); + intel_gt_pm_put(ce->engine->gt, eb->wakeref); for_each_child(ce, child) intel_context_put(child); @@ -2763,6 +2775,12 @@ eb_put_engine(struct i915_execbuffer *eb) struct intel_context *child; i915_vm_put(eb->context->vm); + /* + * This works in conjunction with eb_select_engine() to prevent + * i915_vma_parked() from interfering while execbuf validates vmas. + */ + if (eb->gt->info.id) + intel_gt_pm_put(to_gt(eb->gt->i915), eb->wakeref_gt0); intel_gt_pm_put(eb->context->engine->gt, eb->wakeref); for_each_child(eb->context, child) intel_context_put(child); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h index 3560a062d2872fb5d09edc5cabd78f95a590a3fa..5d7446a48ae790d75c8540d0e28957f78e9ecd30 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h @@ -284,7 +284,9 @@ bool i915_gem_object_has_iomem(const struct drm_i915_gem_object *obj); static inline bool i915_gem_object_is_shrinkable(const struct drm_i915_gem_object *obj) { - return i915_gem_object_type_has(obj, I915_GEM_OBJECT_IS_SHRINKABLE); + /* TODO: make DPT shrinkable when it has no bound vmas */ + return i915_gem_object_type_has(obj, I915_GEM_OBJECT_IS_SHRINKABLE) && + !obj->is_dpt; } static inline bool diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c index 65a931ea80e9b846338b00f35951df2d0723f2e6..3527b8f446fe3b89978391c3de2d366eb252f536 100644 --- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c +++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c @@ -196,7 +196,7 @@ static int verify_access(struct drm_i915_private *i915, if (err) goto out_file; - mode = intel_gt_coherent_map_type(to_gt(i915), native_obj, true); + mode = intel_gt_coherent_map_type(to_gt(i915), native_obj, false); vaddr = i915_gem_object_pin_map_unlocked(native_obj, mode); if (IS_ERR(vaddr)) { err = PTR_ERR(vaddr); diff --git a/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c b/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c index d650beb8ed22f6ac99592ea9fa5d2fc1fa20984d..20b9b04ec1e0bf1e8152b9f546a857632ec66381 100644 --- a/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c +++ b/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c @@ -263,8 +263,13 @@ static void signal_irq_work(struct irq_work *work) i915_request_put(rq); } + /* Lazy irq enabling after HW submission */ if (!READ_ONCE(b->irq_armed) && !list_empty(&b->signalers)) intel_breadcrumbs_arm_irq(b); + + /* And confirm that we still want irqs enabled before we yield */ + if (READ_ONCE(b->irq_armed) && !atomic_read(&b->active)) + intel_breadcrumbs_disarm_irq(b); } struct intel_breadcrumbs * @@ -315,13 +320,7 @@ void __intel_breadcrumbs_park(struct intel_breadcrumbs *b) return; /* Kick the work once more to drain the signalers, and disarm the irq */ - irq_work_sync(&b->irq_work); - while (READ_ONCE(b->irq_armed) && !atomic_read(&b->active)) { - local_irq_disable(); - signal_irq_work(&b->irq_work); - local_irq_enable(); - cond_resched(); - } + irq_work_queue(&b->irq_work); } void intel_breadcrumbs_free(struct kref *kref) @@ -404,7 +403,7 @@ static void insert_breadcrumb(struct i915_request *rq) * the request as it may have completed and raised the interrupt as * we were attaching it into the lists. */ - if (!b->irq_armed || __i915_request_is_complete(rq)) + if (!READ_ONCE(b->irq_armed) || __i915_request_is_complete(rq)) irq_work_queue(&b->irq_work); } diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c index 5c8e9ee3b008373789600ed045e5122e1965711b..3b740ca2500091a36e1e1ccf3fa4f2f93dac5f57 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c @@ -885,6 +885,12 @@ static intel_engine_mask_t init_engine_mask(struct intel_gt *gt) if (IS_DG2(gt->i915)) { u8 first_ccs = __ffs(CCS_MASK(gt)); + /* + * Store the number of active cslices before + * changing the CCS engine configuration + */ + gt->ccs.cslices = CCS_MASK(gt); + /* Mask off all the CCS engine */ info->engine_mask &= ~GENMASK(CCS3, CCS0); /* Put back in the first CCS engine */ diff --git a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c index 99b71bb7da0a6bb8a1f905ab71fcdadf61037e78..3c62a44e9106ceec802dc02645365a5e856fb041 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c +++ b/drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c @@ -19,7 +19,7 @@ unsigned int intel_gt_apply_ccs_mode(struct intel_gt *gt) /* Build the value for the fixed CCS load balancing */ for (cslice = 0; cslice < I915_MAX_CCS; cslice++) { - if (CCS_MASK(gt) & BIT(cslice)) + if (gt->ccs.cslices & BIT(cslice)) /* * If available, assign the cslice * to the first available engine... diff --git a/drivers/gpu/drm/i915/gt/intel_gt_types.h b/drivers/gpu/drm/i915/gt/intel_gt_types.h index def7dd0eb6f196d45be6c3f3cd3f767a5fc1eb48..cfdd2ad5e9549c3b50bfcbba9a02eb67d185a163 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_types.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_types.h @@ -207,6 +207,14 @@ struct intel_gt { [MAX_ENGINE_INSTANCE + 1]; enum intel_submission_method submission_method; + struct { + /* + * Mask of the non fused CCS slices + * to be used for the load balancing + */ + intel_engine_mask_t cslices; + } ccs; + /* * Default address space (either GGTT or ppGTT depending on arch). * diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_klvs_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_klvs_abi.h index bebf28e3c4794b45fe520e5d57e4c0099c931af5..525587cfe1af94ad190deca3365bb400a0f479cb 100644 --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_klvs_abi.h +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_klvs_abi.h @@ -29,9 +29,9 @@ */ #define GUC_KLV_LEN_MIN 1u -#define GUC_KLV_0_KEY (0xffff << 16) -#define GUC_KLV_0_LEN (0xffff << 0) -#define GUC_KLV_n_VALUE (0xffffffff << 0) +#define GUC_KLV_0_KEY (0xffffu << 16) +#define GUC_KLV_0_LEN (0xffffu << 0) +#define GUC_KLV_n_VALUE (0xffffffffu << 0) /** * DOC: GuC Self Config KLVs diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c index 7ea244d876ca63ccd1a91ba934d6f8871a3ac6ad..9bb997dbb4b9010e979f3e41c29c2ac48729d957 100644 --- a/drivers/gpu/drm/lima/lima_gem.c +++ b/drivers/gpu/drm/lima/lima_gem.c @@ -185,7 +185,7 @@ static int lima_gem_pin(struct drm_gem_object *obj) if (bo->heap_size) return -EINVAL; - return drm_gem_shmem_pin(&bo->base); + return drm_gem_shmem_pin_locked(&bo->base); } static int lima_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map) diff --git a/drivers/gpu/drm/msm/registers/gen_header.py b/drivers/gpu/drm/msm/registers/gen_header.py index fc3bfdc991d27aecb2d6ffefae1d8d06a58e0f25..3926485bb197b0992232447cb71bf1c1ebd0968c 100644 --- a/drivers/gpu/drm/msm/registers/gen_header.py +++ b/drivers/gpu/drm/msm/registers/gen_header.py @@ -538,7 +538,7 @@ class Parser(object): self.variants.add(reg.domain) def do_validate(self, schemafile): - if self.validate == False: + if not self.validate: return try: @@ -948,7 +948,8 @@ def main(): parser = argparse.ArgumentParser() parser.add_argument('--rnn', type=str, required=True) parser.add_argument('--xml', type=str, required=True) - parser.add_argument('--validate', action=argparse.BooleanOptionalAction) + parser.add_argument('--validate', default=False, action='store_true') + parser.add_argument('--no-validate', dest='validate', action='store_false') subparsers = parser.add_subparsers() subparsers.required = True diff --git a/drivers/gpu/drm/nouveau/nvif/object.c b/drivers/gpu/drm/nouveau/nvif/object.c index 4d1aaee8fe15fcb1a8e2514095066809b8ebe1e7..1d19c87eaec18e6190adc3aa16e49bca48f0f731 100644 --- a/drivers/gpu/drm/nouveau/nvif/object.c +++ b/drivers/gpu/drm/nouveau/nvif/object.c @@ -142,11 +142,16 @@ nvif_object_mthd(struct nvif_object *object, u32 mthd, void *data, u32 size) struct nvif_ioctl_v0 ioctl; struct nvif_ioctl_mthd_v0 mthd; } *args; + u32 args_size; u8 stack[128]; int ret; - if (sizeof(*args) + size > sizeof(stack)) { - if (!(args = kmalloc(sizeof(*args) + size, GFP_KERNEL))) + if (check_add_overflow(sizeof(*args), size, &args_size)) + return -ENOMEM; + + if (args_size > sizeof(stack)) { + args = kmalloc(args_size, GFP_KERNEL); + if (!args) return -ENOMEM; } else { args = (void *)stack; @@ -157,7 +162,7 @@ nvif_object_mthd(struct nvif_object *object, u32 mthd, void *data, u32 size) args->mthd.method = mthd; memcpy(args->mthd.data, data, size); - ret = nvif_object_ioctl(object, args, sizeof(*args) + size, NULL); + ret = nvif_object_ioctl(object, args, args_size, NULL); memcpy(data, args->mthd.data, size); if (args != (void *)stack) kfree(args); @@ -276,7 +281,15 @@ nvif_object_ctor(struct nvif_object *parent, const char *name, u32 handle, object->map.size = 0; if (parent) { - if (!(args = kmalloc(sizeof(*args) + size, GFP_KERNEL))) { + u32 args_size; + + if (check_add_overflow(sizeof(*args), size, &args_size)) { + nvif_object_dtor(object); + return -ENOMEM; + } + + args = kmalloc(args_size, GFP_KERNEL); + if (!args) { nvif_object_dtor(object); return -ENOMEM; } @@ -293,8 +306,7 @@ nvif_object_ctor(struct nvif_object *parent, const char *name, u32 handle, args->new.oclass = oclass; memcpy(args->new.data, data, size); - ret = nvif_object_ioctl(parent, args, sizeof(*args) + size, - &object->priv); + ret = nvif_object_ioctl(parent, args, args_size, &object->priv); memcpy(data, args->new.data, size); kfree(args); if (ret == 0) diff --git a/drivers/gpu/drm/panel/Kconfig b/drivers/gpu/drm/panel/Kconfig index 982324ef5a41b8f26756f6459716028b718e644a..2ae0eb0638f3252e643f3b6368687de3513ee209 100644 --- a/drivers/gpu/drm/panel/Kconfig +++ b/drivers/gpu/drm/panel/Kconfig @@ -340,6 +340,8 @@ config DRM_PANEL_LG_SW43408 depends on OF depends on DRM_MIPI_DSI depends on BACKLIGHT_CLASS_DEVICE + select DRM_DISPLAY_DP_HELPER + select DRM_DISPLAY_HELPER help Say Y here if you want to enable support for LG sw43408 panel. The panel has a 1080x2160@60Hz resolution and uses 24 bit RGB per diff --git a/drivers/gpu/drm/panel/panel-lg-sw43408.c b/drivers/gpu/drm/panel/panel-lg-sw43408.c index 115f4702d59f769bf49ce888aa91d7db0a83cd0c..2b3a73696dcec7251aefe7ba6e4250ec365664b2 100644 --- a/drivers/gpu/drm/panel/panel-lg-sw43408.c +++ b/drivers/gpu/drm/panel/panel-lg-sw43408.c @@ -182,7 +182,7 @@ static int sw43408_backlight_update_status(struct backlight_device *bl) return mipi_dsi_dcs_set_display_brightness_large(dsi, brightness); } -const struct backlight_ops sw43408_backlight_ops = { +static const struct backlight_ops sw43408_backlight_ops = { .update_status = sw43408_backlight_update_status, }; diff --git a/drivers/gpu/drm/panel/panel-sitronix-st7789v.c b/drivers/gpu/drm/panel/panel-sitronix-st7789v.c index 88e80fe98112daa0019ec43217a94178a5b0b1a7..e8f385b9c6182c5fb038a3804c7742b97e36897c 100644 --- a/drivers/gpu/drm/panel/panel-sitronix-st7789v.c +++ b/drivers/gpu/drm/panel/panel-sitronix-st7789v.c @@ -282,15 +282,15 @@ static const struct drm_display_mode et028013dma_mode = { static const struct drm_display_mode jt240mhqs_hwt_ek_e3_mode = { .clock = 6000, .hdisplay = 240, - .hsync_start = 240 + 28, - .hsync_end = 240 + 28 + 10, - .htotal = 240 + 28 + 10 + 10, + .hsync_start = 240 + 38, + .hsync_end = 240 + 38 + 10, + .htotal = 240 + 38 + 10 + 10, .vdisplay = 280, - .vsync_start = 280 + 8, - .vsync_end = 280 + 8 + 4, - .vtotal = 280 + 8 + 4 + 4, - .width_mm = 43, - .height_mm = 37, + .vsync_start = 280 + 48, + .vsync_end = 280 + 48 + 4, + .vtotal = 280 + 48 + 4 + 4, + .width_mm = 37, + .height_mm = 43, .flags = DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC, }; diff --git a/drivers/gpu/drm/panfrost/panfrost_gem.c b/drivers/gpu/drm/panfrost/panfrost_gem.c index d47b40b82b0bc4d189a2fa6772aaf3a91ee54c18..8e0ff3efede7e3206b75d38328dc843b741ce022 100644 --- a/drivers/gpu/drm/panfrost/panfrost_gem.c +++ b/drivers/gpu/drm/panfrost/panfrost_gem.c @@ -192,7 +192,7 @@ static int panfrost_gem_pin(struct drm_gem_object *obj) if (bo->is_heap) return -EINVAL; - return drm_gem_shmem_pin(&bo->base); + return drm_gem_shmem_pin_locked(&bo->base); } static enum drm_gem_object_status panfrost_gem_status(struct drm_gem_object *obj) diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/drm/tests/drm_buddy_test.c index b3be68b03610cf6b614ad8ada2f48ad5fb5ec5ef..dd8fb9f8341a990f63e2d67fc0a2c6f44fd023df 100644 --- a/drivers/gpu/drm/tests/drm_buddy_test.c +++ b/drivers/gpu/drm/tests/drm_buddy_test.c @@ -505,8 +505,8 @@ static void drm_test_buddy_alloc_pathological(struct kunit *test) * Eventually we will have a fully 50% fragmented mm. */ - mm_size = PAGE_SIZE << max_order; - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, PAGE_SIZE), + mm_size = SZ_4K << max_order; + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K), "buddy_init failed\n"); KUNIT_EXPECT_EQ(test, mm.max_order, max_order); @@ -520,7 +520,7 @@ static void drm_test_buddy_alloc_pathological(struct kunit *test) } for (order = top; order--;) { - size = get_size(order, PAGE_SIZE); + size = get_size(order, mm.chunk_size); KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, size, size, &tmp, flags), @@ -534,7 +534,7 @@ static void drm_test_buddy_alloc_pathological(struct kunit *test) } /* There should be one final page for this sub-allocation */ - size = get_size(0, PAGE_SIZE); + size = get_size(0, mm.chunk_size); KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, size, size, &tmp, flags), "buddy_alloc hit -ENOMEM for hole\n"); @@ -544,7 +544,7 @@ static void drm_test_buddy_alloc_pathological(struct kunit *test) list_move_tail(&block->link, &holes); - size = get_size(top, PAGE_SIZE); + size = get_size(top, mm.chunk_size); KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, size, size, &tmp, flags), "buddy_alloc unexpectedly succeeded at top-order %d/%d, it should be full!", @@ -555,7 +555,7 @@ static void drm_test_buddy_alloc_pathological(struct kunit *test) /* Nothing larger than blocks of chunk_size now available */ for (order = 1; order <= max_order; order++) { - size = get_size(order, PAGE_SIZE); + size = get_size(order, mm.chunk_size); KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, size, size, &tmp, flags), "buddy_alloc unexpectedly succeeded at order %d, it should be full!", @@ -584,14 +584,14 @@ static void drm_test_buddy_alloc_pessimistic(struct kunit *test) * page left. */ - mm_size = PAGE_SIZE << max_order; - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, PAGE_SIZE), + mm_size = SZ_4K << max_order; + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K), "buddy_init failed\n"); KUNIT_EXPECT_EQ(test, mm.max_order, max_order); for (order = 0; order < max_order; order++) { - size = get_size(order, PAGE_SIZE); + size = get_size(order, mm.chunk_size); KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, size, size, &tmp, flags), "buddy_alloc hit -ENOMEM with order=%d\n", @@ -604,7 +604,7 @@ static void drm_test_buddy_alloc_pessimistic(struct kunit *test) } /* And now the last remaining block available */ - size = get_size(0, PAGE_SIZE); + size = get_size(0, mm.chunk_size); KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, size, size, &tmp, flags), "buddy_alloc hit -ENOMEM on final alloc\n"); @@ -616,7 +616,7 @@ static void drm_test_buddy_alloc_pessimistic(struct kunit *test) /* Should be completely full! */ for (order = max_order; order--;) { - size = get_size(order, PAGE_SIZE); + size = get_size(order, mm.chunk_size); KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, size, size, &tmp, flags), "buddy_alloc unexpectedly succeeded, it should be full!"); @@ -632,7 +632,7 @@ static void drm_test_buddy_alloc_pessimistic(struct kunit *test) list_del(&block->link); drm_buddy_free_block(&mm, block); - size = get_size(order, PAGE_SIZE); + size = get_size(order, mm.chunk_size); KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, size, size, &tmp, flags), "buddy_alloc hit -ENOMEM with order=%d\n", @@ -647,7 +647,7 @@ static void drm_test_buddy_alloc_pessimistic(struct kunit *test) } /* To confirm, now the whole mm should be available */ - size = get_size(max_order, PAGE_SIZE); + size = get_size(max_order, mm.chunk_size); KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, size, size, &tmp, flags), "buddy_alloc (realloc) hit -ENOMEM with order=%d\n", @@ -678,15 +678,15 @@ static void drm_test_buddy_alloc_optimistic(struct kunit *test) * try to allocate them all. */ - mm_size = PAGE_SIZE * ((1 << (max_order + 1)) - 1); + mm_size = SZ_4K * ((1 << (max_order + 1)) - 1); - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, PAGE_SIZE), + KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K), "buddy_init failed\n"); KUNIT_EXPECT_EQ(test, mm.max_order, max_order); for (order = 0; order <= max_order; order++) { - size = get_size(order, PAGE_SIZE); + size = get_size(order, mm.chunk_size); KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, size, size, &tmp, flags), "buddy_alloc hit -ENOMEM with order=%d\n", @@ -699,7 +699,7 @@ static void drm_test_buddy_alloc_optimistic(struct kunit *test) } /* Should be completely full! */ - size = get_size(0, PAGE_SIZE); + size = get_size(0, mm.chunk_size); KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size, size, size, &tmp, flags), "buddy_alloc unexpectedly succeeded, it should be full!"); @@ -716,7 +716,7 @@ static void drm_test_buddy_alloc_limit(struct kunit *test) LIST_HEAD(allocated); struct drm_buddy mm; - KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, size, PAGE_SIZE)); + KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, size, SZ_4K)); KUNIT_EXPECT_EQ_MSG(test, mm.max_order, DRM_BUDDY_MAX_ORDER, "mm.max_order(%d) != %d\n", mm.max_order, @@ -724,7 +724,7 @@ static void drm_test_buddy_alloc_limit(struct kunit *test) size = mm.chunk_size << mm.max_order; KUNIT_EXPECT_FALSE(test, drm_buddy_alloc_blocks(&mm, start, size, size, - PAGE_SIZE, &allocated, flags)); + mm.chunk_size, &allocated, flags)); block = list_first_entry_or_null(&allocated, struct drm_buddy_block, link); KUNIT_EXPECT_TRUE(test, block); @@ -734,10 +734,10 @@ static void drm_test_buddy_alloc_limit(struct kunit *test) drm_buddy_block_order(block), mm.max_order); KUNIT_EXPECT_EQ_MSG(test, drm_buddy_block_size(&mm, block), - BIT_ULL(mm.max_order) * PAGE_SIZE, + BIT_ULL(mm.max_order) * mm.chunk_size, "block size(%llu) != %llu\n", drm_buddy_block_size(&mm, block), - BIT_ULL(mm.max_order) * PAGE_SIZE); + BIT_ULL(mm.max_order) * mm.chunk_size); drm_buddy_free_list(&mm, &allocated, 0); drm_buddy_fini(&mm); diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c index c7d38469fb4690cd69a5541b0500998290d44f43..e4e3658e6a1384b9d2936de133dcd322065392e2 100644 --- a/drivers/gpu/drm/xe/xe_guc_submit.c +++ b/drivers/gpu/drm/xe/xe_guc_submit.c @@ -1240,6 +1240,7 @@ static int guc_exec_queue_init(struct xe_exec_queue *q) return 0; err_entity: + mutex_unlock(&guc->submission_state.lock); xe_sched_entity_fini(&ge->entity); err_sched: xe_sched_fini(&ge->sched); diff --git a/drivers/gpu/drm/xe/xe_migrate.c b/drivers/gpu/drm/xe/xe_migrate.c index 9f6e9b7f11c848502ebba3b819f3ff829f22304a..65e5a3f4c340f0f6bb3597f94efc099d9ff23e26 100644 --- a/drivers/gpu/drm/xe/xe_migrate.c +++ b/drivers/gpu/drm/xe/xe_migrate.c @@ -34,7 +34,6 @@ #include "xe_sync.h" #include "xe_trace.h" #include "xe_vm.h" -#include "xe_wa.h" /** * struct xe_migrate - migrate context. @@ -300,10 +299,6 @@ static int xe_migrate_prepare_vm(struct xe_tile *tile, struct xe_migrate *m, } /* - * Due to workaround 16017236439, odd instance hardware copy engines are - * faster than even instance ones. - * This function returns the mask involving all fast copy engines and the - * reserved copy engine to be used as logical mask for migrate engine. * Including the reserved copy engine is required to avoid deadlocks due to * migrate jobs servicing the faults gets stuck behind the job that faulted. */ @@ -317,8 +312,7 @@ static u32 xe_migrate_usm_logical_mask(struct xe_gt *gt) if (hwe->class != XE_ENGINE_CLASS_COPY) continue; - if (!XE_WA(gt, 16017236439) || - xe_gt_is_usm_hwe(gt, hwe) || hwe->instance & 1) + if (xe_gt_is_usm_hwe(gt, hwe)) logical_mask |= BIT(hwe->logical_instance); } @@ -369,6 +363,10 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile) if (!hwe || !logical_mask) return ERR_PTR(-EINVAL); + /* + * XXX: Currently only reserving 1 (likely slow) BCS instance on + * PVC, may want to revisit if performance is needed. + */ m->q = xe_exec_queue_create(xe, vm, logical_mask, 1, hwe, EXEC_QUEUE_FLAG_KERNEL | EXEC_QUEUE_FLAG_PERMANENT | diff --git a/drivers/gpu/drm/xe/xe_pcode.c b/drivers/gpu/drm/xe/xe_pcode.c index c010ef16fbf57451e55ededd46adc43e107153a5..a5e7da8cf94416669458e21dc8b0ade051fefa51 100644 --- a/drivers/gpu/drm/xe/xe_pcode.c +++ b/drivers/gpu/drm/xe/xe_pcode.c @@ -191,7 +191,7 @@ int xe_pcode_request(struct xe_gt *gt, u32 mbox, u32 request, drm_WARN_ON_ONCE(>_to_xe(gt)->drm, timeout_base_ms > 1); preempt_disable(); ret = pcode_try_request(gt, mbox, request, reply_mask, reply, &status, - true, timeout_base_ms * 1000, true); + true, 50 * 1000, true); preempt_enable(); out: diff --git a/drivers/hwmon/dell-smm-hwmon.c b/drivers/hwmon/dell-smm-hwmon.c index 48a81c64f00d2535d84dd15aca6c081e2f275207..942526bd4775fa50e56a2003d2ba92e341cf766a 100644 --- a/drivers/hwmon/dell-smm-hwmon.c +++ b/drivers/hwmon/dell-smm-hwmon.c @@ -1545,6 +1545,14 @@ static const struct dmi_system_id i8k_whitelist_fan_control[] __initconst = { }, .driver_data = (void *)&i8k_fan_control_data[I8K_FAN_30A3_31A3], }, + { + .ident = "Dell G15 5511", + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."), + DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "Dell G15 5511"), + }, + .driver_data = (void *)&i8k_fan_control_data[I8K_FAN_30A3_31A3], + }, { } }; diff --git a/drivers/hwmon/intel-m10-bmc-hwmon.c b/drivers/hwmon/intel-m10-bmc-hwmon.c index 6500ca548f9c73f1b2c9ebdce6ae59a82e63c55a..ca2dff158925153032b4e0f50086b0cb08f3c971 100644 --- a/drivers/hwmon/intel-m10-bmc-hwmon.c +++ b/drivers/hwmon/intel-m10-bmc-hwmon.c @@ -429,7 +429,7 @@ static const struct m10bmc_sdata n6000bmc_curr_tbl[] = { }; static const struct m10bmc_sdata n6000bmc_power_tbl[] = { - { 0x724, 0x0, 0x0, 0x0, 0x0, 1, "Board Power" }, + { 0x724, 0x0, 0x0, 0x0, 0x0, 1000, "Board Power" }, }; static const struct hwmon_channel_info * const n6000bmc_hinfo[] = { diff --git a/drivers/hwmon/ltc2992.c b/drivers/hwmon/ltc2992.c index 229aed15d5caa7d2bf4ee2fa39628d4442b70c38..d4a93223cd3b34716bf50dec258b7d1c04fb0b45 100644 --- a/drivers/hwmon/ltc2992.c +++ b/drivers/hwmon/ltc2992.c @@ -876,9 +876,11 @@ static int ltc2992_parse_dt(struct ltc2992_state *st) ret = fwnode_property_read_u32(child, "shunt-resistor-micro-ohms", &val); if (!ret) { - if (!val) + if (!val) { + fwnode_handle_put(child); return dev_err_probe(&st->client->dev, -EINVAL, "shunt resistor value cannot be zero\n"); + } st->r_sense_uohm[addr] = val; } } diff --git a/drivers/hwmon/shtc1.c b/drivers/hwmon/shtc1.c index 1f96e94967ee8db09371860519d98a65ed022d7a..439dd3dba5fc81fc41c29c332e4023e716089cfa 100644 --- a/drivers/hwmon/shtc1.c +++ b/drivers/hwmon/shtc1.c @@ -238,7 +238,7 @@ static int shtc1_probe(struct i2c_client *client) if (np) { data->setup.blocking_io = of_property_read_bool(np, "sensirion,blocking-io"); - data->setup.high_precision = !of_property_read_bool(np, "sensicon,low-precision"); + data->setup.high_precision = !of_property_read_bool(np, "sensirion,low-precision"); } else { if (client->dev.platform_data) data->setup = *(struct shtc1_platform_data *)dev->platform_data; diff --git a/drivers/i2c/busses/i2c-synquacer.c b/drivers/i2c/busses/i2c-synquacer.c index 31ecb2c7e9783517e7a3a7d2bd3ce415e857cda3..4eccbcd0fbfc00b7f67875e880054bdd06515be4 100644 --- a/drivers/i2c/busses/i2c-synquacer.c +++ b/drivers/i2c/busses/i2c-synquacer.c @@ -138,7 +138,6 @@ struct synquacer_i2c { int irq; struct device *dev; void __iomem *base; - struct clk *pclk; u32 pclkrate; u32 speed_khz; u32 timeout_ms; @@ -535,6 +534,7 @@ static const struct i2c_adapter synquacer_i2c_ops = { static int synquacer_i2c_probe(struct platform_device *pdev) { struct synquacer_i2c *i2c; + struct clk *pclk; u32 bus_speed; int ret; @@ -550,13 +550,12 @@ static int synquacer_i2c_probe(struct platform_device *pdev) device_property_read_u32(&pdev->dev, "socionext,pclk-rate", &i2c->pclkrate); - i2c->pclk = devm_clk_get_enabled(&pdev->dev, "pclk"); - if (IS_ERR(i2c->pclk)) - return dev_err_probe(&pdev->dev, PTR_ERR(i2c->pclk), + pclk = devm_clk_get_enabled(&pdev->dev, "pclk"); + if (IS_ERR(pclk)) + return dev_err_probe(&pdev->dev, PTR_ERR(pclk), "failed to get and enable clock\n"); - dev_dbg(&pdev->dev, "clock source %p\n", i2c->pclk); - i2c->pclkrate = clk_get_rate(i2c->pclk); + i2c->pclkrate = clk_get_rate(pclk); if (i2c->pclkrate < SYNQUACER_I2C_MIN_CLK_RATE || i2c->pclkrate > SYNQUACER_I2C_MAX_CLK_RATE) diff --git a/drivers/isdn/hardware/mISDN/avmfritz.c b/drivers/isdn/hardware/mISDN/avmfritz.c index f68569bfef7a711a53b1d9d48ba83c335c8b3fe7..509b362d6465d3d0ddd61e93e7a46f804bdd35a6 100644 --- a/drivers/isdn/hardware/mISDN/avmfritz.c +++ b/drivers/isdn/hardware/mISDN/avmfritz.c @@ -159,6 +159,7 @@ set_debug(const char *val, const struct kernel_param *kp) } MODULE_AUTHOR("Karsten Keil"); +MODULE_DESCRIPTION("mISDN driver for AVM FRITZ!CARD PCI ISDN cards"); MODULE_LICENSE("GPL v2"); MODULE_VERSION(AVMFRITZ_REV); module_param_call(debug, set_debug, param_get_uint, &debug, S_IRUGO | S_IWUSR); diff --git a/drivers/isdn/hardware/mISDN/hfcmulti.c b/drivers/isdn/hardware/mISDN/hfcmulti.c index 2e5cb9dde3ec501391f860db4f6dc8b46cd26141..0d2928d8aeae8f45d28b4263321d87ee4f09785b 100644 --- a/drivers/isdn/hardware/mISDN/hfcmulti.c +++ b/drivers/isdn/hardware/mISDN/hfcmulti.c @@ -221,6 +221,7 @@ static uint hwid = HWID_NONE; static int HFC_cnt, E1_cnt, bmask_cnt, Port_cnt, PCM_cnt = 99; MODULE_AUTHOR("Andreas Eversberg"); +MODULE_DESCRIPTION("mISDN driver for hfc-4s/hfc-8s/hfc-e1 based cards"); MODULE_LICENSE("GPL"); MODULE_VERSION(HFC_MULTI_VERSION); module_param(debug, uint, S_IRUGO | S_IWUSR); diff --git a/drivers/isdn/hardware/mISDN/hfcpci.c b/drivers/isdn/hardware/mISDN/hfcpci.c index fe391de1aba326242633a6b2e12a13b778acd3f6..ce7bccc9faa39970485fb3982a32a5ba31f379f7 100644 --- a/drivers/isdn/hardware/mISDN/hfcpci.c +++ b/drivers/isdn/hardware/mISDN/hfcpci.c @@ -48,6 +48,7 @@ static struct timer_list hfc_tl; static unsigned long hfc_jiffies; MODULE_AUTHOR("Karsten Keil"); +MODULE_DESCRIPTION("mISDN driver for CCD's hfc-pci based cards"); MODULE_LICENSE("GPL"); module_param(debug, uint, S_IRUGO | S_IWUSR); module_param(poll, uint, S_IRUGO | S_IWUSR); diff --git a/drivers/isdn/hardware/mISDN/hfcsusb.c b/drivers/isdn/hardware/mISDN/hfcsusb.c index b82b89888a5e042aee79f222b2ce2fcfdd36a092..e54419a4e731f41572aed08ab65efe8fdf285f69 100644 --- a/drivers/isdn/hardware/mISDN/hfcsusb.c +++ b/drivers/isdn/hardware/mISDN/hfcsusb.c @@ -31,6 +31,7 @@ static DEFINE_RWLOCK(HFClock); MODULE_AUTHOR("Martin Bachem"); +MODULE_DESCRIPTION("mISDN driver for Colognechip HFC-S USB chip"); MODULE_LICENSE("GPL"); module_param(debug, uint, S_IRUGO | S_IWUSR); module_param(poll, int, 0); diff --git a/drivers/isdn/hardware/mISDN/mISDNinfineon.c b/drivers/isdn/hardware/mISDN/mISDNinfineon.c index 88d592bafdb0279635e2ac9ae57798b715282614..30876a01271188376661411a7856d7175dbdae13 100644 --- a/drivers/isdn/hardware/mISDN/mISDNinfineon.c +++ b/drivers/isdn/hardware/mISDN/mISDNinfineon.c @@ -245,6 +245,7 @@ set_debug(const char *val, const struct kernel_param *kp) } MODULE_AUTHOR("Karsten Keil"); +MODULE_DESCRIPTION("mISDN driver for cards based on Infineon ISDN chipsets"); MODULE_LICENSE("GPL v2"); MODULE_VERSION(INFINEON_REV); module_param_call(debug, set_debug, param_get_uint, &debug, S_IRUGO | S_IWUSR); diff --git a/drivers/isdn/hardware/mISDN/mISDNipac.c b/drivers/isdn/hardware/mISDN/mISDNipac.c index 4f8d85bb3ce18ce18b49fef2cd1c630eaa281e14..d0b7271fbda1a20fbd7347ce533736a01c44383a 100644 --- a/drivers/isdn/hardware/mISDN/mISDNipac.c +++ b/drivers/isdn/hardware/mISDN/mISDNipac.c @@ -21,6 +21,7 @@ MODULE_AUTHOR("Karsten Keil"); MODULE_VERSION(ISAC_REV); +MODULE_DESCRIPTION("mISDN driver for ISAC specific functions"); MODULE_LICENSE("GPL v2"); #define ReadISAC(is, o) (is->read_reg(is->dch.hw, o + is->off)) diff --git a/drivers/isdn/hardware/mISDN/mISDNisar.c b/drivers/isdn/hardware/mISDN/mISDNisar.c index 48b3d43e25022a471fc48176d7f29275d8302a26..b3e03c4105448e200731c301a1f638a6d49e05d3 100644 --- a/drivers/isdn/hardware/mISDN/mISDNisar.c +++ b/drivers/isdn/hardware/mISDN/mISDNisar.c @@ -22,6 +22,7 @@ #define ISAR_REV "2.1" MODULE_AUTHOR("Karsten Keil"); +MODULE_DESCRIPTION("mISDN driver for ISAR (Siemens PSB 7110) specific functions"); MODULE_LICENSE("GPL v2"); MODULE_VERSION(ISAR_REV); diff --git a/drivers/isdn/hardware/mISDN/netjet.c b/drivers/isdn/hardware/mISDN/netjet.c index 566c790a9481df2082c93c11ac2d19a044531550..d163850c295e71506df68f5d21f7723ae2596749 100644 --- a/drivers/isdn/hardware/mISDN/netjet.c +++ b/drivers/isdn/hardware/mISDN/netjet.c @@ -114,6 +114,7 @@ set_debug(const char *val, const struct kernel_param *kp) } MODULE_AUTHOR("Karsten Keil"); +MODULE_DESCRIPTION("mISDN driver for NETJet cards"); MODULE_LICENSE("GPL v2"); MODULE_VERSION(NETJET_REV); module_param_call(debug, set_debug, param_get_uint, &debug, S_IRUGO | S_IWUSR); diff --git a/drivers/isdn/hardware/mISDN/speedfax.c b/drivers/isdn/hardware/mISDN/speedfax.c index b530c78eca8ee6128a0be626a1fb5735cd1af61b..0c405261d94069f177cd4950561bd9e641c7d036 100644 --- a/drivers/isdn/hardware/mISDN/speedfax.c +++ b/drivers/isdn/hardware/mISDN/speedfax.c @@ -97,6 +97,7 @@ set_debug(const char *val, const struct kernel_param *kp) } MODULE_AUTHOR("Karsten Keil"); +MODULE_DESCRIPTION("mISDN driver for Sedlbauer Speedfax+ cards"); MODULE_LICENSE("GPL v2"); MODULE_VERSION(SPEEDFAX_REV); MODULE_FIRMWARE("isdn/ISAR.BIN"); diff --git a/drivers/isdn/hardware/mISDN/w6692.c b/drivers/isdn/hardware/mISDN/w6692.c index f3b8db7b48fe08c86b11f37fb50554b5831694cb..ee69212ac351b37da64d9eccd19afecea52be870 100644 --- a/drivers/isdn/hardware/mISDN/w6692.c +++ b/drivers/isdn/hardware/mISDN/w6692.c @@ -101,6 +101,7 @@ set_debug(const char *val, const struct kernel_param *kp) } MODULE_AUTHOR("Karsten Keil"); +MODULE_DESCRIPTION("mISDN driver for Winbond w6692 based cards"); MODULE_LICENSE("GPL v2"); MODULE_VERSION(W6692_REV); module_param_call(debug, set_debug, param_get_uint, &debug, S_IRUGO | S_IWUSR); diff --git a/drivers/isdn/mISDN/core.c b/drivers/isdn/mISDN/core.c index ab8513a7acd52d1525943fb585df08c5b3a0ba7c..e34a7a46754e805a06284afc20d35566c0f53bdc 100644 --- a/drivers/isdn/mISDN/core.c +++ b/drivers/isdn/mISDN/core.c @@ -14,6 +14,7 @@ static u_int debug; MODULE_AUTHOR("Karsten Keil"); +MODULE_DESCRIPTION("Modular ISDN core driver"); MODULE_LICENSE("GPL"); module_param(debug, uint, S_IRUGO | S_IWUSR); diff --git a/drivers/isdn/mISDN/dsp_blowfish.c b/drivers/isdn/mISDN/dsp_blowfish.c index 0aa572f3858da771e1bea19ebaef5eb39f346d3a..0e77c282c8624ed5d44ca3dc471ae161de3c2444 100644 --- a/drivers/isdn/mISDN/dsp_blowfish.c +++ b/drivers/isdn/mISDN/dsp_blowfish.c @@ -73,11 +73,6 @@ * crypto-api for faster implementation */ -struct bf_ctx { - u32 p[18]; - u32 s[1024]; -}; - static const u32 bf_pbox[16 + 2] = { 0x243f6a88, 0x85a308d3, 0x13198a2e, 0x03707344, 0xa4093822, 0x299f31d0, 0x082efa98, 0xec4e6c89, diff --git a/drivers/isdn/mISDN/dsp_core.c b/drivers/isdn/mISDN/dsp_core.c index fae95f166688336bf60850b0588f6add0f1ca352..753232e9fc36ac42c6f081ba329cb1f1190f473f 100644 --- a/drivers/isdn/mISDN/dsp_core.c +++ b/drivers/isdn/mISDN/dsp_core.c @@ -172,6 +172,7 @@ module_param(debug, uint, S_IRUGO | S_IWUSR); module_param(options, uint, S_IRUGO | S_IWUSR); module_param(poll, uint, S_IRUGO | S_IWUSR); module_param(dtmfthreshold, uint, S_IRUGO | S_IWUSR); +MODULE_DESCRIPTION("mISDN driver for Digital Audio Processing of transparent data"); MODULE_LICENSE("GPL"); /*int spinnest = 0;*/ diff --git a/drivers/isdn/mISDN/l1oip_core.c b/drivers/isdn/mISDN/l1oip_core.c index f010b35a05313b57317d45c3ecc663deaa1e7dcc..a5ad88a960d0ec5c18fe7d0142abe49eababad49 100644 --- a/drivers/isdn/mISDN/l1oip_core.c +++ b/drivers/isdn/mISDN/l1oip_core.c @@ -245,6 +245,7 @@ static int debug; static int ulaw; MODULE_AUTHOR("Andreas Eversberg"); +MODULE_DESCRIPTION("mISDN driver for tunneling layer 1 over IP"); MODULE_LICENSE("GPL"); module_param_array(type, uint, NULL, S_IRUGO | S_IWUSR); module_param_array(codec, uint, NULL, S_IRUGO | S_IWUSR); diff --git a/drivers/mailbox/zynqmp-ipi-mailbox.c b/drivers/mailbox/zynqmp-ipi-mailbox.c index 7c90bac3de216a34af3be01df548d98278dd3c7d..4acf5612487cfd77fd5ee4d3556123bfb051fc46 100644 --- a/drivers/mailbox/zynqmp-ipi-mailbox.c +++ b/drivers/mailbox/zynqmp-ipi-mailbox.c @@ -850,7 +850,6 @@ static int xlnx_mbox_init_sgi(struct platform_device *pdev, return ret; } - irq_to_desc(pdata->virq_sgi); irq_set_status_flags(pdata->virq_sgi, IRQ_PER_CPU); /* Setup function for the CPU hot-plug cases */ diff --git a/drivers/md/bcache/alloc.c b/drivers/md/bcache/alloc.c index ce13c272c3872366eedd8a7d94894a81ea78d98c..48ce750bf70af95c50888ab5b8622891160e59d0 100644 --- a/drivers/md/bcache/alloc.c +++ b/drivers/md/bcache/alloc.c @@ -129,12 +129,9 @@ static inline bool can_inc_bucket_gen(struct bucket *b) bool bch_can_invalidate_bucket(struct cache *ca, struct bucket *b) { - BUG_ON(!ca->set->gc_mark_valid); - - return (!GC_MARK(b) || - GC_MARK(b) == GC_MARK_RECLAIMABLE) && - !atomic_read(&b->pin) && - can_inc_bucket_gen(b); + return (ca->set->gc_mark_valid || b->reclaimable_in_gc) && + ((!GC_MARK(b) || GC_MARK(b) == GC_MARK_RECLAIMABLE) && + !atomic_read(&b->pin) && can_inc_bucket_gen(b)); } void __bch_invalidate_one_bucket(struct cache *ca, struct bucket *b) @@ -148,6 +145,7 @@ void __bch_invalidate_one_bucket(struct cache *ca, struct bucket *b) bch_inc_gen(ca, b); b->prio = INITIAL_PRIO; atomic_inc(&b->pin); + b->reclaimable_in_gc = 0; } static void bch_invalidate_one_bucket(struct cache *ca, struct bucket *b) @@ -352,8 +350,7 @@ static int bch_allocator_thread(void *arg) */ retry_invalidate: - allocator_wait(ca, ca->set->gc_mark_valid && - !ca->invalidate_needs_gc); + allocator_wait(ca, !ca->invalidate_needs_gc); invalidate_buckets(ca); /* @@ -501,8 +498,8 @@ int __bch_bucket_alloc_set(struct cache_set *c, unsigned int reserve, ca = c->cache; b = bch_bucket_alloc(ca, reserve, wait); - if (b == -1) - goto err; + if (b < 0) + return -1; k->ptr[0] = MAKE_PTR(ca->buckets[b].gen, bucket_to_sector(c, b), @@ -511,10 +508,6 @@ int __bch_bucket_alloc_set(struct cache_set *c, unsigned int reserve, SET_KEY_PTRS(k, 1); return 0; -err: - bch_bucket_free(c, k); - bkey_put(c, k); - return -1; } int bch_bucket_alloc_set(struct cache_set *c, unsigned int reserve, diff --git a/drivers/md/bcache/bcache.h b/drivers/md/bcache/bcache.h index 4e6afa89921fe0b79c4a760163d66565a2092d53..1d33e40d26ea51f6e433978315df0213259a38ef 100644 --- a/drivers/md/bcache/bcache.h +++ b/drivers/md/bcache/bcache.h @@ -200,6 +200,7 @@ struct bucket { uint8_t gen; uint8_t last_gc; /* Most out of date gen in the btree */ uint16_t gc_mark; /* Bitfield used by GC. See below for field */ + uint16_t reclaimable_in_gc:1; }; /* diff --git a/drivers/md/bcache/btree.c b/drivers/md/bcache/btree.c index d011a7154d3304d3deeb4cc0d2d48ea9728822fa..4e6ccf2c8a0bf3a048e88cba537801d6d8ec3dfd 100644 --- a/drivers/md/bcache/btree.c +++ b/drivers/md/bcache/btree.c @@ -1741,18 +1741,20 @@ static void btree_gc_start(struct cache_set *c) mutex_lock(&c->bucket_lock); - c->gc_mark_valid = 0; c->gc_done = ZERO_KEY; ca = c->cache; for_each_bucket(b, ca) { b->last_gc = b->gen; + if (bch_can_invalidate_bucket(ca, b)) + b->reclaimable_in_gc = 1; if (!atomic_read(&b->pin)) { SET_GC_MARK(b, 0); SET_GC_SECTORS_USED(b, 0); } } + c->gc_mark_valid = 0; mutex_unlock(&c->bucket_lock); } @@ -1809,6 +1811,9 @@ static void bch_btree_gc_finish(struct cache_set *c) for_each_bucket(b, ca) { c->need_gc = max(c->need_gc, bucket_gc_gen(b)); + if (b->reclaimable_in_gc) + b->reclaimable_in_gc = 0; + if (atomic_read(&b->pin)) continue; diff --git a/drivers/md/bcache/request.c b/drivers/md/bcache/request.c index 83d112bd2b1c0e336d21cfbd8f50eb716a2a4c4b..af345dc6fde14f689917ca58447a372104ebe4e8 100644 --- a/drivers/md/bcache/request.c +++ b/drivers/md/bcache/request.c @@ -369,10 +369,24 @@ static bool check_should_bypass(struct cached_dev *dc, struct bio *bio) struct io *i; if (test_bit(BCACHE_DEV_DETACHING, &dc->disk.flags) || - c->gc_stats.in_use > CUTOFF_CACHE_ADD || (bio_op(bio) == REQ_OP_DISCARD)) goto skip; + if (c->gc_stats.in_use > CUTOFF_CACHE_ADD) { + /* + * If cached buckets are all clean now, 'true' will be + * returned and all requests will bypass the cache device. + * Then c->sectors_to_gc has no chance to be negative, and + * gc thread won't wake up and caching won't work forever. + * Here call force_wake_up_gc() to avoid such aftermath. + */ + if (BDEV_STATE(&dc->sb) == BDEV_STATE_CLEAN && + c->gc_mark_valid) + force_wake_up_gc(c); + + goto skip; + } + if (mode == CACHE_MODE_NONE || (mode == CACHE_MODE_WRITEAROUND && op_is_write(bio_op(bio)))) diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c index cc66a27c363a65f2e034706ff0073b327d71f14b..b2d5246cff2102a51aa356548cf934205791a048 100644 --- a/drivers/md/dm-table.c +++ b/drivers/md/dm-table.c @@ -1981,10 +1981,6 @@ int dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, if (!dm_table_supports_secure_erase(t)) limits->max_secure_erase_sectors = 0; - r = queue_limits_set(q, limits); - if (r) - return r; - if (dm_table_supports_flush(t, (1UL << QUEUE_FLAG_WC))) { wc = true; if (dm_table_supports_flush(t, (1UL << QUEUE_FLAG_FUA))) @@ -2036,15 +2032,16 @@ int dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, * For a zoned target, setup the zones related queue attributes * and resources necessary for zone append emulation if necessary. */ - if (blk_queue_is_zoned(q)) { - r = dm_set_zones_restrictions(t, q); + if (IS_ENABLED(CONFIG_BLK_DEV_ZONED) && limits->zoned) { + r = dm_set_zones_restrictions(t, q, limits); if (r) return r; - if (blk_queue_is_zoned(q) && - !static_key_enabled(&zoned_enabled.key)) - static_branch_enable(&zoned_enabled); } + r = queue_limits_set(q, limits); + if (r) + return r; + dm_update_crypto_profile(q, t); /* diff --git a/drivers/md/dm-zone.c b/drivers/md/dm-zone.c index 8e6bcb0d786a1ae7f1ab20ccee16138dd63be01c..5d66d916730efa79521594e5ad40d190e1658404 100644 --- a/drivers/md/dm-zone.c +++ b/drivers/md/dm-zone.c @@ -160,37 +160,6 @@ static int dm_check_zoned_cb(struct blk_zone *zone, unsigned int idx, return 0; } -static int dm_check_zoned(struct mapped_device *md, struct dm_table *t) -{ - struct gendisk *disk = md->disk; - unsigned int nr_conv_zones = 0; - int ret; - - /* Count conventional zones */ - md->zone_revalidate_map = t; - ret = dm_blk_report_zones(disk, 0, UINT_MAX, - dm_check_zoned_cb, &nr_conv_zones); - md->zone_revalidate_map = NULL; - if (ret < 0) { - DMERR("Check zoned failed %d", ret); - return ret; - } - - /* - * If we only have conventional zones, expose the mapped device as - * a regular device. - */ - if (nr_conv_zones >= ret) { - disk->queue->limits.max_open_zones = 0; - disk->queue->limits.max_active_zones = 0; - disk->queue->limits.zoned = false; - clear_bit(DMF_EMULATE_ZONE_APPEND, &md->flags); - disk->nr_zones = 0; - } - - return 0; -} - /* * Revalidate the zones of a mapped device to initialize resource necessary * for zone append emulation. Note that we cannot simply use the block layer @@ -251,9 +220,12 @@ static bool dm_table_supports_zone_append(struct dm_table *t) return true; } -int dm_set_zones_restrictions(struct dm_table *t, struct request_queue *q) +int dm_set_zones_restrictions(struct dm_table *t, struct request_queue *q, + struct queue_limits *lim) { struct mapped_device *md = t->md; + struct gendisk *disk = md->disk; + unsigned int nr_conv_zones = 0; int ret; /* @@ -265,21 +237,37 @@ int dm_set_zones_restrictions(struct dm_table *t, struct request_queue *q) clear_bit(DMF_EMULATE_ZONE_APPEND, &md->flags); } else { set_bit(DMF_EMULATE_ZONE_APPEND, &md->flags); - blk_queue_max_zone_append_sectors(q, 0); + lim->max_zone_append_sectors = 0; } if (!get_capacity(md->disk)) return 0; /* - * Check that the mapped device will indeed be zoned, that is, that it - * has sequential write required zones. + * Count conventional zones to check that the mapped device will indeed + * have sequential write required zones. */ - ret = dm_check_zoned(md, t); - if (ret) + md->zone_revalidate_map = t; + ret = dm_blk_report_zones(disk, 0, UINT_MAX, + dm_check_zoned_cb, &nr_conv_zones); + md->zone_revalidate_map = NULL; + if (ret < 0) { + DMERR("Check zoned failed %d", ret); return ret; - if (!blk_queue_is_zoned(q)) + } + + /* + * If we only have conventional zones, expose the mapped device as + * a regular device. + */ + if (nr_conv_zones >= ret) { + lim->max_open_zones = 0; + lim->max_active_zones = 0; + lim->zoned = false; + clear_bit(DMF_EMULATE_ZONE_APPEND, &md->flags); + disk->nr_zones = 0; return 0; + } if (!md->disk->nr_zones) { DMINFO("%s using %s zone append", @@ -287,7 +275,13 @@ int dm_set_zones_restrictions(struct dm_table *t, struct request_queue *q) queue_emulates_zone_append(q) ? "emulated" : "native"); } - return dm_revalidate_zones(md, t); + ret = dm_revalidate_zones(md, t); + if (ret < 0) + return ret; + + if (!static_key_enabled(&zoned_enabled.key)) + static_branch_enable(&zoned_enabled); + return 0; } /* diff --git a/drivers/md/dm.h b/drivers/md/dm.h index e0c57f19839b290fad1f2337e25f9690d2040466..53ef8207fe2c158b78e6682776e446975a428a79 100644 --- a/drivers/md/dm.h +++ b/drivers/md/dm.h @@ -101,7 +101,8 @@ int dm_setup_md_queue(struct mapped_device *md, struct dm_table *t); /* * Zoned targets related functions. */ -int dm_set_zones_restrictions(struct dm_table *t, struct request_queue *q); +int dm_set_zones_restrictions(struct dm_table *t, struct request_queue *q, + struct queue_limits *lim); void dm_zone_endio(struct dm_io *io, struct bio *clone); #ifdef CONFIG_BLK_DEV_ZONED int dm_blk_report_zones(struct gendisk *disk, sector_t sector, diff --git a/drivers/net/dsa/Kconfig b/drivers/net/dsa/Kconfig index 3092b391031a83f0b15a3140cd04ec6e83e920c9..8508b5145bc148f7f36cc60ca56cd44ea9758610 100644 --- a/drivers/net/dsa/Kconfig +++ b/drivers/net/dsa/Kconfig @@ -102,6 +102,7 @@ config NET_DSA_SMSC_LAN9303 tristate select NET_DSA_TAG_LAN9303 select REGMAP + imply SMSC_PHY help This enables support for the Microchip LAN9303/LAN9354 3 port ethernet switch chips. diff --git a/drivers/net/dsa/hirschmann/hellcreek.h b/drivers/net/dsa/hirschmann/hellcreek.h index 6874cb9dc3614bb9f86f6949a29906e7c91ea769..9c2ed2ba79daccf54b7b6fea73e756e0870d6452 100644 --- a/drivers/net/dsa/hirschmann/hellcreek.h +++ b/drivers/net/dsa/hirschmann/hellcreek.h @@ -12,14 +12,16 @@ #include #include +#include #include -#include -#include -#include #include +#include #include #include #include +#include +#include + #include #include diff --git a/drivers/net/dsa/microchip/ksz_common.c b/drivers/net/dsa/microchip/ksz_common.c index 1e0085cd9a9ad5fb3dccde133bb0b1251bad237a..2818e24e2a51dd39073ee1bc8d51ea7dfa2dc1e0 100644 --- a/drivers/net/dsa/microchip/ksz_common.c +++ b/drivers/net/dsa/microchip/ksz_common.c @@ -3142,7 +3142,7 @@ phy_interface_t ksz_get_xmii(struct ksz_device *dev, int port, bool gbit) else interface = PHY_INTERFACE_MODE_MII; } else if (val == bitval[P_RMII_SEL]) { - interface = PHY_INTERFACE_MODE_RGMII; + interface = PHY_INTERFACE_MODE_RMII; } else { interface = PHY_INTERFACE_MODE_RGMII; if (data8 & P_RGMII_ID_EG_ENABLE) diff --git a/drivers/net/dsa/ocelot/felix.c b/drivers/net/dsa/ocelot/felix.c index 61e95487732dce95fedb608eedff86413eba340c..d12c4e85baa77cfe6c5edd961bde16ef1d2366c8 100644 --- a/drivers/net/dsa/ocelot/felix.c +++ b/drivers/net/dsa/ocelot/felix.c @@ -1050,24 +1050,32 @@ static void felix_phylink_get_caps(struct dsa_switch *ds, int port, config->supported_interfaces); } -static void felix_phylink_mac_config(struct dsa_switch *ds, int port, +static void felix_phylink_mac_config(struct phylink_config *config, unsigned int mode, const struct phylink_link_state *state) { - struct ocelot *ocelot = ds->priv; - struct felix *felix = ocelot_to_felix(ocelot); + struct dsa_port *dp = dsa_phylink_to_port(config); + struct ocelot *ocelot = dp->ds->priv; + int port = dp->index; + struct felix *felix; + + felix = ocelot_to_felix(ocelot); if (felix->info->phylink_mac_config) felix->info->phylink_mac_config(ocelot, port, mode, state); } -static struct phylink_pcs *felix_phylink_mac_select_pcs(struct dsa_switch *ds, - int port, - phy_interface_t iface) +static struct phylink_pcs * +felix_phylink_mac_select_pcs(struct phylink_config *config, + phy_interface_t iface) { - struct ocelot *ocelot = ds->priv; - struct felix *felix = ocelot_to_felix(ocelot); + struct dsa_port *dp = dsa_phylink_to_port(config); + struct ocelot *ocelot = dp->ds->priv; struct phylink_pcs *pcs = NULL; + int port = dp->index; + struct felix *felix; + + felix = ocelot_to_felix(ocelot); if (felix->pcs && felix->pcs[port]) pcs = felix->pcs[port]; @@ -1075,11 +1083,13 @@ static struct phylink_pcs *felix_phylink_mac_select_pcs(struct dsa_switch *ds, return pcs; } -static void felix_phylink_mac_link_down(struct dsa_switch *ds, int port, +static void felix_phylink_mac_link_down(struct phylink_config *config, unsigned int link_an_mode, phy_interface_t interface) { - struct ocelot *ocelot = ds->priv; + struct dsa_port *dp = dsa_phylink_to_port(config); + struct ocelot *ocelot = dp->ds->priv; + int port = dp->index; struct felix *felix; felix = ocelot_to_felix(ocelot); @@ -1088,15 +1098,19 @@ static void felix_phylink_mac_link_down(struct dsa_switch *ds, int port, felix->info->quirks); } -static void felix_phylink_mac_link_up(struct dsa_switch *ds, int port, +static void felix_phylink_mac_link_up(struct phylink_config *config, + struct phy_device *phydev, unsigned int link_an_mode, phy_interface_t interface, - struct phy_device *phydev, int speed, int duplex, bool tx_pause, bool rx_pause) { - struct ocelot *ocelot = ds->priv; - struct felix *felix = ocelot_to_felix(ocelot); + struct dsa_port *dp = dsa_phylink_to_port(config); + struct ocelot *ocelot = dp->ds->priv; + int port = dp->index; + struct felix *felix; + + felix = ocelot_to_felix(ocelot); ocelot_phylink_mac_link_up(ocelot, port, phydev, link_an_mode, interface, speed, duplex, tx_pause, rx_pause, @@ -1583,6 +1597,15 @@ static int felix_setup(struct dsa_switch *ds) felix_port_qos_map_init(ocelot, dp->index); } + if (felix->info->request_irq) { + err = felix->info->request_irq(ocelot); + if (err) { + dev_err(ocelot->dev, "Failed to request IRQ: %pe\n", + ERR_PTR(err)); + goto out_deinit_ports; + } + } + err = ocelot_devlink_sb_register(ocelot); if (err) goto out_deinit_ports; @@ -2083,7 +2106,14 @@ static void felix_get_mm_stats(struct dsa_switch *ds, int port, ocelot_port_get_mm_stats(ocelot, port, stats); } -const struct dsa_switch_ops felix_switch_ops = { +static const struct phylink_mac_ops felix_phylink_mac_ops = { + .mac_select_pcs = felix_phylink_mac_select_pcs, + .mac_config = felix_phylink_mac_config, + .mac_link_down = felix_phylink_mac_link_down, + .mac_link_up = felix_phylink_mac_link_up, +}; + +static const struct dsa_switch_ops felix_switch_ops = { .get_tag_protocol = felix_get_tag_protocol, .change_tag_protocol = felix_change_tag_protocol, .connect_tag_protocol = felix_connect_tag_protocol, @@ -2104,10 +2134,6 @@ const struct dsa_switch_ops felix_switch_ops = { .get_sset_count = felix_get_sset_count, .get_ts_info = felix_get_ts_info, .phylink_get_caps = felix_phylink_get_caps, - .phylink_mac_config = felix_phylink_mac_config, - .phylink_mac_select_pcs = felix_phylink_mac_select_pcs, - .phylink_mac_link_down = felix_phylink_mac_link_down, - .phylink_mac_link_up = felix_phylink_mac_link_up, .port_enable = felix_port_enable, .port_fast_age = felix_port_fast_age, .port_fdb_dump = felix_fdb_dump, @@ -2166,7 +2192,53 @@ const struct dsa_switch_ops felix_switch_ops = { .port_set_host_flood = felix_port_set_host_flood, .port_change_conduit = felix_port_change_conduit, }; -EXPORT_SYMBOL_GPL(felix_switch_ops); + +int felix_register_switch(struct device *dev, resource_size_t switch_base, + int num_flooding_pgids, bool ptp, + bool mm_supported, + enum dsa_tag_protocol init_tag_proto, + const struct felix_info *info) +{ + struct dsa_switch *ds; + struct ocelot *ocelot; + struct felix *felix; + int err; + + felix = devm_kzalloc(dev, sizeof(*felix), GFP_KERNEL); + if (!felix) + return -ENOMEM; + + ds = devm_kzalloc(dev, sizeof(*ds), GFP_KERNEL); + if (!ds) + return -ENOMEM; + + dev_set_drvdata(dev, felix); + + ocelot = &felix->ocelot; + ocelot->dev = dev; + ocelot->num_flooding_pgids = num_flooding_pgids; + ocelot->ptp = ptp; + ocelot->mm_supported = mm_supported; + + felix->info = info; + felix->switch_base = switch_base; + felix->ds = ds; + felix->tag_proto = init_tag_proto; + + ds->dev = dev; + ds->num_ports = info->num_ports; + ds->num_tx_queues = OCELOT_NUM_TC; + ds->ops = &felix_switch_ops; + ds->phylink_mac_ops = &felix_phylink_mac_ops; + ds->priv = ocelot; + + err = dsa_register_switch(ds); + if (err) + dev_err_probe(dev, err, "Failed to register DSA switch\n"); + + return err; +} +EXPORT_SYMBOL_GPL(felix_register_switch); struct net_device *felix_port_to_netdev(struct ocelot *ocelot, int port) { diff --git a/drivers/net/dsa/ocelot/felix.h b/drivers/net/dsa/ocelot/felix.h index dbf5872fe36735f88d55bd06b3f049e4a700242a..211991f494e352dc523f7a98f557a0ea44e25e5c 100644 --- a/drivers/net/dsa/ocelot/felix.h +++ b/drivers/net/dsa/ocelot/felix.h @@ -32,7 +32,6 @@ struct felix_info { const u32 *port_modes; int num_mact_rows; int num_ports; - int num_tx_queues; struct vcap_props *vcap; u16 vcap_pol_base; u16 vcap_pol_max; @@ -64,6 +63,7 @@ struct felix_info { const struct phylink_link_state *state); int (*configure_serdes)(struct ocelot *ocelot, int port, struct device_node *portnp); + int (*request_irq)(struct ocelot *ocelot); }; /* Methods for initializing the hardware resources specific to a tagging @@ -82,8 +82,6 @@ struct felix_tag_proto_ops { struct netlink_ext_ack *extack); }; -extern const struct dsa_switch_ops felix_switch_ops; - /* DSA glue / front-end for struct ocelot */ struct felix { struct dsa_switch *ds; @@ -99,6 +97,11 @@ struct felix { unsigned long host_flood_mc_mask; }; +int felix_register_switch(struct device *dev, resource_size_t switch_base, + int num_flooding_pgids, bool ptp, + bool mm_supported, + enum dsa_tag_protocol init_tag_proto, + const struct felix_info *info); struct net_device *felix_port_to_netdev(struct ocelot *ocelot, int port); int felix_netdev_to_port(struct net_device *dev); diff --git a/drivers/net/dsa/ocelot/felix_vsc9959.c b/drivers/net/dsa/ocelot/felix_vsc9959.c index 85952d841f285677409ee0ac9a551cb0c5617be6..ba37a566da39a4e4fcfcef6ad61440917338c073 100644 --- a/drivers/net/dsa/ocelot/felix_vsc9959.c +++ b/drivers/net/dsa/ocelot/felix_vsc9959.c @@ -2605,6 +2605,28 @@ set: } } +/* The INTB interrupt is shared between for PTP TX timestamp availability + * notification and MAC Merge status change on each port. + */ +static irqreturn_t vsc9959_irq_handler(int irq, void *data) +{ + struct ocelot *ocelot = data; + + ocelot_get_txtstamp(ocelot); + ocelot_mm_irq(ocelot); + + return IRQ_HANDLED; +} + +static int vsc9959_request_irq(struct ocelot *ocelot) +{ + struct pci_dev *pdev = to_pci_dev(ocelot->dev); + + return devm_request_threaded_irq(ocelot->dev, pdev->irq, NULL, + &vsc9959_irq_handler, IRQF_ONESHOT, + "felix-intb", ocelot); +} + static const struct ocelot_ops vsc9959_ops = { .reset = vsc9959_reset, .wm_enc = vsc9959_wm_enc, @@ -2636,7 +2658,6 @@ static const struct felix_info felix_info_vsc9959 = { .vcap_pol_max2 = 0, .num_mact_rows = 2048, .num_ports = VSC9959_NUM_PORTS, - .num_tx_queues = OCELOT_NUM_TC, .quirks = FELIX_MAC_QUIRKS, .quirk_no_xtr_irq = true, .ptp_caps = &vsc9959_ptp_caps, @@ -2645,98 +2666,36 @@ static const struct felix_info felix_info_vsc9959 = { .port_modes = vsc9959_port_modes, .port_setup_tc = vsc9959_port_setup_tc, .port_sched_speed_set = vsc9959_sched_speed_set, + .request_irq = vsc9959_request_irq, }; -/* The INTB interrupt is shared between for PTP TX timestamp availability - * notification and MAC Merge status change on each port. - */ -static irqreturn_t felix_irq_handler(int irq, void *data) -{ - struct ocelot *ocelot = (struct ocelot *)data; - - ocelot_get_txtstamp(ocelot); - ocelot_mm_irq(ocelot); - - return IRQ_HANDLED; -} - static int felix_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { - struct dsa_switch *ds; - struct ocelot *ocelot; - struct felix *felix; + struct device *dev = &pdev->dev; + resource_size_t switch_base; int err; - if (pdev->dev.of_node && !of_device_is_available(pdev->dev.of_node)) { - dev_info(&pdev->dev, "device is disabled, skipping\n"); - return -ENODEV; - } - err = pci_enable_device(pdev); if (err) { - dev_err(&pdev->dev, "device enable failed\n"); - goto err_pci_enable; - } - - felix = kzalloc(sizeof(struct felix), GFP_KERNEL); - if (!felix) { - err = -ENOMEM; - dev_err(&pdev->dev, "Failed to allocate driver memory\n"); - goto err_alloc_felix; + dev_err(dev, "device enable failed: %pe\n", ERR_PTR(err)); + return err; } - pci_set_drvdata(pdev, felix); - ocelot = &felix->ocelot; - ocelot->dev = &pdev->dev; - ocelot->num_flooding_pgids = OCELOT_NUM_TC; - felix->info = &felix_info_vsc9959; - felix->switch_base = pci_resource_start(pdev, VSC9959_SWITCH_PCI_BAR); - pci_set_master(pdev); - err = devm_request_threaded_irq(&pdev->dev, pdev->irq, NULL, - &felix_irq_handler, IRQF_ONESHOT, - "felix-intb", ocelot); - if (err) { - dev_err(&pdev->dev, "Failed to request irq\n"); - goto err_alloc_irq; - } - - ocelot->ptp = 1; - ocelot->mm_supported = true; - - ds = kzalloc(sizeof(struct dsa_switch), GFP_KERNEL); - if (!ds) { - err = -ENOMEM; - dev_err(&pdev->dev, "Failed to allocate DSA switch\n"); - goto err_alloc_ds; - } + switch_base = pci_resource_start(pdev, VSC9959_SWITCH_PCI_BAR); - ds->dev = &pdev->dev; - ds->num_ports = felix->info->num_ports; - ds->num_tx_queues = felix->info->num_tx_queues; - ds->ops = &felix_switch_ops; - ds->priv = ocelot; - felix->ds = ds; - felix->tag_proto = DSA_TAG_PROTO_OCELOT; - - err = dsa_register_switch(ds); - if (err) { - dev_err_probe(&pdev->dev, err, "Failed to register DSA switch\n"); - goto err_register_ds; - } + err = felix_register_switch(dev, switch_base, OCELOT_NUM_TC, + true, true, DSA_TAG_PROTO_OCELOT, + &felix_info_vsc9959); + if (err) + goto out_disable; return 0; -err_register_ds: - kfree(ds); -err_alloc_ds: -err_alloc_irq: - kfree(felix); -err_alloc_felix: +out_disable: pci_disable_device(pdev); -err_pci_enable: return err; } @@ -2749,9 +2708,6 @@ static void felix_pci_remove(struct pci_dev *pdev) dsa_unregister_switch(felix->ds); - kfree(felix->ds); - kfree(felix); - pci_disable_device(pdev); } diff --git a/drivers/net/dsa/ocelot/ocelot_ext.c b/drivers/net/dsa/ocelot/ocelot_ext.c index 22187d831c4b9f970b556e47d7cb59302797c58a..5632a7248cd41777b52ae04d1799b20214397112 100644 --- a/drivers/net/dsa/ocelot/ocelot_ext.c +++ b/drivers/net/dsa/ocelot/ocelot_ext.c @@ -57,7 +57,6 @@ static const struct felix_info vsc7512_info = { .vcap = vsc7514_vcap_props, .num_mact_rows = 1024, .num_ports = VSC7514_NUM_PORTS, - .num_tx_queues = OCELOT_NUM_TC, .port_modes = vsc7512_port_modes, .phylink_mac_config = ocelot_phylink_mac_config, .configure_serdes = ocelot_port_configure_serdes, @@ -65,54 +64,8 @@ static const struct felix_info vsc7512_info = { static int ocelot_ext_probe(struct platform_device *pdev) { - struct device *dev = &pdev->dev; - struct dsa_switch *ds; - struct ocelot *ocelot; - struct felix *felix; - int err; - - felix = kzalloc(sizeof(*felix), GFP_KERNEL); - if (!felix) - return -ENOMEM; - - dev_set_drvdata(dev, felix); - - ocelot = &felix->ocelot; - ocelot->dev = dev; - - ocelot->num_flooding_pgids = 1; - - felix->info = &vsc7512_info; - - ds = kzalloc(sizeof(*ds), GFP_KERNEL); - if (!ds) { - err = -ENOMEM; - dev_err_probe(dev, err, "Failed to allocate DSA switch\n"); - goto err_free_felix; - } - - ds->dev = dev; - ds->num_ports = felix->info->num_ports; - ds->num_tx_queues = felix->info->num_tx_queues; - - ds->ops = &felix_switch_ops; - ds->priv = ocelot; - felix->ds = ds; - felix->tag_proto = DSA_TAG_PROTO_OCELOT; - - err = dsa_register_switch(ds); - if (err) { - dev_err_probe(dev, err, "Failed to register DSA switch\n"); - goto err_free_ds; - } - - return 0; - -err_free_ds: - kfree(ds); -err_free_felix: - kfree(felix); - return err; + return felix_register_switch(&pdev->dev, 0, 1, false, false, + DSA_TAG_PROTO_OCELOT, &vsc7512_info); } static void ocelot_ext_remove(struct platform_device *pdev) @@ -123,9 +76,6 @@ static void ocelot_ext_remove(struct platform_device *pdev) return; dsa_unregister_switch(felix->ds); - - kfree(felix->ds); - kfree(felix); } static void ocelot_ext_shutdown(struct platform_device *pdev) diff --git a/drivers/net/dsa/ocelot/seville_vsc9953.c b/drivers/net/dsa/ocelot/seville_vsc9953.c index 049930da05218f2a54c8a6a0b2d45a0ecc9bd86a..70782649c3957625e367133b251f5c72eb17bc6d 100644 --- a/drivers/net/dsa/ocelot/seville_vsc9953.c +++ b/drivers/net/dsa/ocelot/seville_vsc9953.c @@ -963,7 +963,6 @@ static const struct felix_info seville_info_vsc9953 = { .quirks = FELIX_MAC_QUIRKS, .num_mact_rows = 2048, .num_ports = VSC9953_NUM_PORTS, - .num_tx_queues = OCELOT_NUM_TC, .mdio_bus_alloc = vsc9953_mdio_bus_alloc, .mdio_bus_free = vsc9953_mdio_bus_free, .port_modes = vsc9953_port_modes, @@ -971,62 +970,18 @@ static const struct felix_info seville_info_vsc9953 = { static int seville_probe(struct platform_device *pdev) { - struct dsa_switch *ds; - struct ocelot *ocelot; + struct device *dev = &pdev->dev; struct resource *res; - struct felix *felix; - int err; - - felix = kzalloc(sizeof(struct felix), GFP_KERNEL); - if (!felix) { - err = -ENOMEM; - dev_err(&pdev->dev, "Failed to allocate driver memory\n"); - goto err_alloc_felix; - } - - platform_set_drvdata(pdev, felix); - - ocelot = &felix->ocelot; - ocelot->dev = &pdev->dev; - ocelot->num_flooding_pgids = 1; - felix->info = &seville_info_vsc9953; res = platform_get_resource(pdev, IORESOURCE_MEM, 0); if (!res) { - err = -EINVAL; - dev_err(&pdev->dev, "Invalid resource\n"); - goto err_alloc_felix; - } - felix->switch_base = res->start; - - ds = kzalloc(sizeof(struct dsa_switch), GFP_KERNEL); - if (!ds) { - err = -ENOMEM; - dev_err(&pdev->dev, "Failed to allocate DSA switch\n"); - goto err_alloc_ds; - } - - ds->dev = &pdev->dev; - ds->num_ports = felix->info->num_ports; - ds->ops = &felix_switch_ops; - ds->priv = ocelot; - felix->ds = ds; - felix->tag_proto = DSA_TAG_PROTO_SEVILLE; - - err = dsa_register_switch(ds); - if (err) { - dev_err(&pdev->dev, "Failed to register DSA switch: %d\n", err); - goto err_register_ds; + dev_err(dev, "Invalid resource\n"); + return -EINVAL; } - return 0; - -err_register_ds: - kfree(ds); -err_alloc_ds: -err_alloc_felix: - kfree(felix); - return err; + return felix_register_switch(dev, res->start, 1, false, false, + DSA_TAG_PROTO_SEVILLE, + &seville_info_vsc9953); } static void seville_remove(struct platform_device *pdev) @@ -1037,9 +992,6 @@ static void seville_remove(struct platform_device *pdev) return; dsa_unregister_switch(felix->ds); - - kfree(felix->ds); - kfree(felix); } static void seville_shutdown(struct platform_device *pdev) diff --git a/drivers/net/ethernet/8390/ne2k-pci.c b/drivers/net/ethernet/8390/ne2k-pci.c index 65f56a98c0a06178a2f7499cdf3fc22c46fdaf60..1a34da07c0db26eb96b68bab42ca87ab04bcf120 100644 --- a/drivers/net/ethernet/8390/ne2k-pci.c +++ b/drivers/net/ethernet/8390/ne2k-pci.c @@ -186,17 +186,6 @@ static void ne2k_pci_block_output(struct net_device *dev, const int count, static const struct ethtool_ops ne2k_pci_ethtool_ops; - -/* There is no room in the standard 8390 structure for extra info we need, - * so we build a meta/outer-wrapper structure.. - */ -struct ne2k_pci_card { - struct net_device *dev; - struct pci_dev *pci_dev; -}; - - - /* NEx000-clone boards have a Station Address (SA) PROM (SAPROM) in the packet * buffer memory space. By-the-spec NE2000 clones have 0x57,0x57 in bytes * 0x0e,0x0f of the SAPROM, while other supposed NE2000 clones must be diff --git a/drivers/net/ethernet/adaptec/starfire.c b/drivers/net/ethernet/adaptec/starfire.c index 857361c74f5dd51bf6d685cadca9732e64c65ca7..e1b8794b14c90114878de7537eca6aaa9ea9c712 100644 --- a/drivers/net/ethernet/adaptec/starfire.c +++ b/drivers/net/ethernet/adaptec/starfire.c @@ -441,14 +441,6 @@ enum rx_desc_bits { }; /* Completion queue entry. */ -struct short_rx_done_desc { - __le32 status; /* Low 16 bits is length. */ -}; -struct basic_rx_done_desc { - __le32 status; /* Low 16 bits is length. */ - __le16 vlanid; - __le16 status2; -}; struct csum_rx_done_desc { __le32 status; /* Low 16 bits is length. */ __le16 csum; /* Partial checksum */ diff --git a/drivers/net/ethernet/amazon/ena/ena_com.c b/drivers/net/ethernet/amazon/ena/ena_com.c index 2d8a66ea82fab7f0a023ab469ccc33321d0b4ba3..713a595370bffa5a615073874bd3a99a4a8f286d 100644 --- a/drivers/net/ethernet/amazon/ena/ena_com.c +++ b/drivers/net/ethernet/amazon/ena/ena_com.c @@ -312,7 +312,6 @@ static int ena_com_init_io_sq(struct ena_com_dev *ena_dev, struct ena_com_io_sq *io_sq) { size_t size; - int dev_node = 0; memset(&io_sq->desc_addr, 0x0, sizeof(io_sq->desc_addr)); @@ -325,12 +324,9 @@ static int ena_com_init_io_sq(struct ena_com_dev *ena_dev, size = io_sq->desc_entry_size * io_sq->q_depth; if (io_sq->mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_HOST) { - dev_node = dev_to_node(ena_dev->dmadev); - set_dev_node(ena_dev->dmadev, ctx->numa_node); io_sq->desc_addr.virt_addr = dma_alloc_coherent(ena_dev->dmadev, size, &io_sq->desc_addr.phys_addr, GFP_KERNEL); - set_dev_node(ena_dev->dmadev, dev_node); if (!io_sq->desc_addr.virt_addr) { io_sq->desc_addr.virt_addr = dma_alloc_coherent(ena_dev->dmadev, size, @@ -354,10 +350,7 @@ static int ena_com_init_io_sq(struct ena_com_dev *ena_dev, size = (size_t)io_sq->bounce_buf_ctrl.buffer_size * io_sq->bounce_buf_ctrl.buffers_num; - dev_node = dev_to_node(ena_dev->dmadev); - set_dev_node(ena_dev->dmadev, ctx->numa_node); io_sq->bounce_buf_ctrl.base_buffer = devm_kzalloc(ena_dev->dmadev, size, GFP_KERNEL); - set_dev_node(ena_dev->dmadev, dev_node); if (!io_sq->bounce_buf_ctrl.base_buffer) io_sq->bounce_buf_ctrl.base_buffer = devm_kzalloc(ena_dev->dmadev, size, GFP_KERNEL); @@ -397,7 +390,6 @@ static int ena_com_init_io_cq(struct ena_com_dev *ena_dev, struct ena_com_io_cq *io_cq) { size_t size; - int prev_node = 0; memset(&io_cq->cdesc_addr, 0x0, sizeof(io_cq->cdesc_addr)); @@ -409,11 +401,8 @@ static int ena_com_init_io_cq(struct ena_com_dev *ena_dev, size = io_cq->cdesc_entry_size_in_bytes * io_cq->q_depth; - prev_node = dev_to_node(ena_dev->dmadev); - set_dev_node(ena_dev->dmadev, ctx->numa_node); io_cq->cdesc_addr.virt_addr = dma_alloc_coherent(ena_dev->dmadev, size, &io_cq->cdesc_addr.phys_addr, GFP_KERNEL); - set_dev_node(ena_dev->dmadev, prev_node); if (!io_cq->cdesc_addr.virt_addr) { io_cq->cdesc_addr.virt_addr = dma_alloc_coherent(ena_dev->dmadev, size, &io_cq->cdesc_addr.phys_addr, diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index c437ca1c0fd3996ad6447de5dd7747b0e16bf978..7dc00c0d89924e44a89933932ecd527f5ab72b05 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -512,8 +512,11 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)) { struct bnxt_ptp_cfg *ptp = bp->ptp_cfg; - if (ptp && ptp->tx_tstamp_en && !skb_is_gso(skb) && - atomic_dec_if_positive(&ptp->tx_avail) >= 0) { + if (ptp && ptp->tx_tstamp_en && !skb_is_gso(skb)) { + if (atomic_dec_if_positive(&ptp->tx_avail) < 0) { + atomic64_inc(&ptp->stats.ts_err); + goto tx_no_ts; + } if (!bnxt_ptp_parse(skb, &ptp->tx_seqid, &ptp->tx_hdr_off)) { if (vlan_tag_flags) @@ -526,6 +529,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) } } +tx_no_ts: if (unlikely(skb->no_fcs)) lflags |= cpu_to_le32(TX_BD_FLAGS_NO_CRC); @@ -732,8 +736,10 @@ tx_done: return NETDEV_TX_OK; tx_dma_error: - if (BNXT_TX_PTP_IS_SET(lflags)) + if (BNXT_TX_PTP_IS_SET(lflags)) { + atomic64_inc(&bp->ptp_cfg->stats.ts_err); atomic_inc(&bp->ptp_cfg->tx_avail); + } last_frag = i; @@ -812,10 +818,12 @@ static void __bnxt_tx_int(struct bnxt *bp, struct bnxt_tx_ring_info *txr, if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS)) { if (BNXT_CHIP_P5(bp)) { /* PTP worker takes ownership of the skb */ - if (!bnxt_get_tx_ts_p5(bp, skb)) + if (!bnxt_get_tx_ts_p5(bp, skb)) { skb = NULL; - else + } else { + atomic64_inc(&bp->ptp_cfg->stats.ts_err); atomic_inc(&bp->ptp_cfg->tx_avail); + } } } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c index 8763f8a01457f64e0cec75a28b218e14de308eca..bf157f6cc0427eaca65f00e65398672450c8a0a1 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c @@ -5233,6 +5233,19 @@ static void bnxt_get_rmon_stats(struct net_device *dev, *ranges = bnxt_rmon_ranges; } +static void bnxt_get_ptp_stats(struct net_device *dev, + struct ethtool_ts_stats *ts_stats) +{ + struct bnxt *bp = netdev_priv(dev); + struct bnxt_ptp_cfg *ptp = bp->ptp_cfg; + + if (ptp) { + ts_stats->pkts = ptp->stats.ts_pkts; + ts_stats->lost = ptp->stats.ts_lost; + ts_stats->err = atomic64_read(&ptp->stats.ts_err); + } +} + static void bnxt_get_link_ext_stats(struct net_device *dev, struct ethtool_link_ext_stats *stats) { @@ -5316,4 +5329,5 @@ const struct ethtool_ops bnxt_ethtool_ops = { .get_eth_mac_stats = bnxt_get_eth_mac_stats, .get_eth_ctrl_stats = bnxt_get_eth_ctrl_stats, .get_rmon_stats = bnxt_get_rmon_stats, + .get_ts_stats = bnxt_get_ptp_stats, }; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c index e661ab154d6b27f6ef741f274799b51792e30219..a14d46b9bfdf0ffe946c2e8dd5345d97a771bff3 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c @@ -696,11 +696,13 @@ static void bnxt_stamp_tx_skb(struct bnxt *bp, struct sk_buff *skb) spin_unlock_bh(&ptp->ptp_lock); timestamp.hwtstamp = ns_to_ktime(ns); skb_tstamp_tx(ptp->tx_skb, ×tamp); + ptp->stats.ts_pkts++; } else { if (!time_after_eq(jiffies, ptp->abs_txts_tmo)) { ptp->txts_pending = true; return; } + ptp->stats.ts_lost++; netdev_warn_once(bp->dev, "TS query for TX timer failed rc = %x\n", rc); } @@ -979,6 +981,11 @@ int bnxt_ptp_init(struct bnxt *bp, bool phc_cfg) rc = err; goto out; } + + ptp->stats.ts_pkts = 0; + ptp->stats.ts_lost = 0; + atomic64_set(&ptp->stats.ts_err, 0); + if (BNXT_CHIP_P5(bp)) { spin_lock_bh(&ptp->ptp_lock); bnxt_refclk_read(bp, NULL, &ptp->current_time); @@ -1013,5 +1020,6 @@ void bnxt_ptp_clear(struct bnxt *bp) dev_kfree_skb_any(ptp->tx_skb); ptp->tx_skb = NULL; } + bnxt_unmap_ptp_regs(bp); } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.h index 2c3415c8fc03436174bd3e116a8aa89edcd85c35..8c30b428a428dd6134107f7433e2304d82cb0111 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.h @@ -79,6 +79,12 @@ struct bnxt_pps { struct pps_pin pins[BNXT_MAX_TSIO_PINS]; }; +struct bnxt_ptp_stats { + u64 ts_pkts; + u64 ts_lost; + atomic64_t ts_err; +}; + struct bnxt_ptp_cfg { struct ptp_clock_info ptp_info; struct ptp_clock *ptp_clock; @@ -125,6 +131,8 @@ struct bnxt_ptp_cfg { u32 refclk_mapped_regs[2]; u32 txts_tmo; unsigned long abs_txts_tmo; + + struct bnxt_ptp_stats stats; }; #if BITS_PER_LONG == 32 diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c index 34f02a8ec2ca3adad9dd64ec48df2cc66c8880ea..1d79f6eaa41f6cbfc64d57ab44252e94f24a8345 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_main.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c @@ -92,12 +92,6 @@ static int octeon_console_debug_enabled(u32 console) /* time to wait for possible in-flight requests in milliseconds */ #define WAIT_INFLIGHT_REQUEST msecs_to_jiffies(1000) -struct oct_link_status_resp { - u64 rh; - struct oct_link_info link_info; - u64 status; -}; - struct oct_timestamp_resp { u64 rh; u64 timestamp; diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_droq.c b/drivers/net/ethernet/cavium/liquidio/octeon_droq.c index 0d6ee30affb9137327208b53f4579c6e0e384820..eef12fdd246d34f884ab7ec354394c509ac6d7cc 100644 --- a/drivers/net/ethernet/cavium/liquidio/octeon_droq.c +++ b/drivers/net/ethernet/cavium/liquidio/octeon_droq.c @@ -30,11 +30,6 @@ #include "cn23xx_pf_device.h" #include "cn23xx_vf_device.h" -struct niclist { - struct list_head list; - void *ptr; -}; - struct __dispatch { struct list_head list; struct octeon_recv_info *rinfo; diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethernet/cisco/enic/enic_main.c index f604119efc8098d511258679e48bbcc633611cba..5f26fc3ad65556d772def1d6c39de0b25989f88d 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -1117,18 +1117,30 @@ static int enic_set_vf_port(struct net_device *netdev, int vf, pp->request = nla_get_u8(port[IFLA_PORT_REQUEST]); if (port[IFLA_PORT_PROFILE]) { + if (nla_len(port[IFLA_PORT_PROFILE]) != PORT_PROFILE_MAX) { + memcpy(pp, &prev_pp, sizeof(*pp)); + return -EINVAL; + } pp->set |= ENIC_SET_NAME; memcpy(pp->name, nla_data(port[IFLA_PORT_PROFILE]), PORT_PROFILE_MAX); } if (port[IFLA_PORT_INSTANCE_UUID]) { + if (nla_len(port[IFLA_PORT_INSTANCE_UUID]) != PORT_UUID_MAX) { + memcpy(pp, &prev_pp, sizeof(*pp)); + return -EINVAL; + } pp->set |= ENIC_SET_INSTANCE; memcpy(pp->instance_uuid, nla_data(port[IFLA_PORT_INSTANCE_UUID]), PORT_UUID_MAX); } if (port[IFLA_PORT_HOST_UUID]) { + if (nla_len(port[IFLA_PORT_HOST_UUID]) != PORT_UUID_MAX) { + memcpy(pp, &prev_pp, sizeof(*pp)); + return -EINVAL; + } pp->set |= ENIC_SET_HOST; memcpy(pp->host_uuid, nla_data(port[IFLA_PORT_HOST_UUID]), PORT_UUID_MAX); diff --git a/drivers/net/ethernet/cortina/gemini.c b/drivers/net/ethernet/cortina/gemini.c index 5f0c9e1771dbfe12a15fc4983462d63cbf6259db..73e1c71c5092e4bcaeddb9e2a6dbfc9771afd5c6 100644 --- a/drivers/net/ethernet/cortina/gemini.c +++ b/drivers/net/ethernet/cortina/gemini.c @@ -79,7 +79,8 @@ MODULE_PARM_DESC(debug, "Debug level (0=none,...,16=all)"); #define GMAC0_IRQ4_8 (GMAC0_MIB_INT_BIT | GMAC0_RX_OVERRUN_INT_BIT) #define GMAC_OFFLOAD_FEATURES (NETIF_F_SG | NETIF_F_IP_CSUM | \ - NETIF_F_IPV6_CSUM | NETIF_F_RXCSUM) + NETIF_F_IPV6_CSUM | NETIF_F_RXCSUM | \ + NETIF_F_TSO | NETIF_F_TSO_ECN | NETIF_F_TSO6) /** * struct gmac_queue_page - page buffer per-page info @@ -287,13 +288,13 @@ static void gmac_set_flow_control(struct net_device *netdev, bool tx, bool rx) spin_unlock_irqrestore(&port->config_lock, flags); } -static void gmac_speed_set(struct net_device *netdev) +static void gmac_adjust_link(struct net_device *netdev) { struct gemini_ethernet_port *port = netdev_priv(netdev); struct phy_device *phydev = netdev->phydev; union gmac_status status, old_status; - int pause_tx = 0; - int pause_rx = 0; + bool pause_tx = false; + bool pause_rx = false; status.bits32 = readl(port->gmac_base + GMAC_STATUS); old_status.bits32 = status.bits32; @@ -328,14 +329,9 @@ static void gmac_speed_set(struct net_device *netdev) } if (phydev->duplex == DUPLEX_FULL) { - u16 lcladv = phy_read(phydev, MII_ADVERTISE); - u16 rmtadv = phy_read(phydev, MII_LPA); - u8 cap = mii_resolve_flowctrl_fdx(lcladv, rmtadv); - - if (cap & FLOW_CTRL_RX) - pause_rx = 1; - if (cap & FLOW_CTRL_TX) - pause_tx = 1; + phy_get_pause(phydev, &pause_tx, &pause_rx); + netdev_dbg(netdev, "set negotiated pause params pause TX = %s, pause RX = %s\n", + pause_tx ? "ON" : "OFF", pause_rx ? "ON" : "OFF"); } gmac_set_flow_control(netdev, pause_tx, pause_rx); @@ -366,7 +362,7 @@ static int gmac_setup_phy(struct net_device *netdev) phy = of_phy_get_and_connect(netdev, dev->of_node, - gmac_speed_set); + gmac_adjust_link); if (!phy) return -ENODEV; netdev->phydev = phy; @@ -1148,13 +1144,25 @@ static int gmac_map_tx_bufs(struct net_device *netdev, struct sk_buff *skb, skb_frag_t *skb_frag; dma_addr_t mapping; void *buffer; + u16 mss; int ret; - /* TODO: implement proper TSO using MTU in word3 */ word1 = skb->len; word3 = SOF_BIT; - if (skb->len >= ETH_FRAME_LEN) { + mss = skb_shinfo(skb)->gso_size; + if (mss) { + /* This means we are dealing with TCP and skb->len is the + * sum total of all the segments. The TSO will deal with + * chopping this up for us. + */ + /* The accelerator needs the full frame size here */ + mss += skb_tcp_all_headers(skb); + netdev_dbg(netdev, "segment offloading mss = %04x len=%04x\n", + mss, skb->len); + word1 |= TSS_MTU_ENABLE_BIT; + word3 |= mss; + } else if (skb->len >= ETH_FRAME_LEN) { /* Hardware offloaded checksumming isn't working on frames * bigger than 1514 bytes. A hypothesis about this is that the * checksum buffer is only 1518 bytes, so when the frames get @@ -1169,7 +1177,9 @@ static int gmac_map_tx_bufs(struct net_device *netdev, struct sk_buff *skb, return ret; } word1 |= TSS_BYPASS_BIT; - } else if (skb->ip_summed == CHECKSUM_PARTIAL) { + } + + if (skb->ip_summed == CHECKSUM_PARTIAL) { int tcp = 0; /* We do not switch off the checksumming on non TCP/UDP @@ -2116,6 +2126,19 @@ static void gmac_get_pauseparam(struct net_device *netdev, pparam->autoneg = true; } +static int gmac_set_pauseparam(struct net_device *netdev, + struct ethtool_pauseparam *pparam) +{ + struct phy_device *phydev = netdev->phydev; + + if (!pparam->autoneg) + return -EOPNOTSUPP; + + phy_set_asym_pause(phydev, pparam->rx_pause, pparam->tx_pause); + + return 0; +} + static void gmac_get_ringparam(struct net_device *netdev, struct ethtool_ringparam *rp, struct kernel_ethtool_ringparam *kernel_rp, @@ -2236,6 +2259,7 @@ static const struct ethtool_ops gmac_351x_ethtool_ops = { .set_link_ksettings = gmac_set_ksettings, .nway_reset = gmac_nway_reset, .get_pauseparam = gmac_get_pauseparam, + .set_pauseparam = gmac_set_pauseparam, .get_ringparam = gmac_get_ringparam, .set_ringparam = gmac_set_ringparam, .get_coalesce = gmac_get_coalesce, diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index a72d8a2eb0b315a505c61f9663844fa133120ccf..881ece735dcf1470fed863fdbb765e231ca2dbbf 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -4130,6 +4130,14 @@ free_queue_mem: return ret; } +static void fec_enet_deinit(struct net_device *ndev) +{ + struct fec_enet_private *fep = netdev_priv(ndev); + + netif_napi_del(&fep->napi); + fec_enet_free_queue(ndev); +} + #ifdef CONFIG_OF static int fec_reset_phy(struct platform_device *pdev) { @@ -4524,6 +4532,7 @@ failed_register: fec_enet_mii_remove(fep); failed_mii_init: failed_irq: + fec_enet_deinit(ndev); failed_init: fec_ptp_stop(pdev); failed_reset: @@ -4587,6 +4596,7 @@ fec_drv_remove(struct platform_device *pdev) pm_runtime_put_noidle(&pdev->dev); pm_runtime_disable(&pdev->dev); + fec_enet_deinit(ndev); free_netdev(ndev); } diff --git a/drivers/net/ethernet/freescale/fman/fman_memac.c b/drivers/net/ethernet/freescale/fman/fman_memac.c index 92b8f4ab26f1d7392b76ebf11080b2331bb65876..796e6f4e583d18be6069f78af15fbedf9557378e 100644 --- a/drivers/net/ethernet/freescale/fman/fman_memac.c +++ b/drivers/net/ethernet/freescale/fman/fman_memac.c @@ -1066,7 +1066,6 @@ int memac_initialization(struct mac_device *mac_dev, struct fman_mac_params *params) { int err; - struct device_node *fixed; struct phylink_pcs *pcs; struct fman_mac *memac; unsigned long capabilities; @@ -1222,18 +1221,15 @@ int memac_initialization(struct mac_device *mac_dev, memac->rgmii_no_half_duplex = true; /* Most boards should use MLO_AN_INBAND, but existing boards don't have - * a managed property. Default to MLO_AN_INBAND if nothing else is - * specified. We need to be careful and not enable this if we have a - * fixed link or if we are using MII or RGMII, since those - * configurations modes don't use in-band autonegotiation. + * a managed property. Default to MLO_AN_INBAND rather than MLO_AN_PHY. + * Phylink will allow this to be overriden by a fixed link. We need to + * be careful and not enable this if we are using MII or RGMII, since + * those configurations modes don't use in-band autonegotiation. */ - fixed = of_get_child_by_name(mac_node, "fixed-link"); - if (!fixed && !of_property_read_bool(mac_node, "fixed-link") && - !of_property_read_bool(mac_node, "managed") && + if (!of_property_read_bool(mac_node, "managed") && mac_dev->phy_if != PHY_INTERFACE_MODE_MII && !phy_interface_mode_is_rgmii(mac_dev->phy_if)) - mac_dev->phylink_config.ovr_an_inband = true; - of_node_put(fixed); + mac_dev->phylink_config.default_an_inband = true; err = memac_init(mac_dev->fman_mac); if (err < 0) diff --git a/drivers/net/ethernet/hisilicon/hns3/Makefile b/drivers/net/ethernet/hisilicon/hns3/Makefile index 8e9293e57bfd58f9c3afc43c23cfa9f973df8582..e8af26da1fc1ed01c0cf40cb564a969eee91ee36 100644 --- a/drivers/net/ethernet/hisilicon/hns3/Makefile +++ b/drivers/net/ethernet/hisilicon/hns3/Makefile @@ -15,15 +15,14 @@ hns3-objs = hns3_enet.o hns3_ethtool.o hns3_debugfs.o hns3-$(CONFIG_HNS3_DCB) += hns3_dcbnl.o -obj-$(CONFIG_HNS3_HCLGEVF) += hclgevf.o +obj-$(CONFIG_HNS3_HCLGEVF) += hclgevf.o hclge-common.o -hclgevf-objs = hns3vf/hclgevf_main.o hns3vf/hclgevf_mbx.o hns3vf/hclgevf_devlink.o hns3vf/hclgevf_regs.o \ - hns3_common/hclge_comm_cmd.o hns3_common/hclge_comm_rss.o hns3_common/hclge_comm_tqp_stats.o +hclge-common-objs += hns3_common/hclge_comm_cmd.o hns3_common/hclge_comm_rss.o hns3_common/hclge_comm_tqp_stats.o -obj-$(CONFIG_HNS3_HCLGE) += hclge.o +hclgevf-objs = hns3vf/hclgevf_main.o hns3vf/hclgevf_mbx.o hns3vf/hclgevf_devlink.o hns3vf/hclgevf_regs.o + +obj-$(CONFIG_HNS3_HCLGE) += hclge.o hclge-common.o hclge-objs = hns3pf/hclge_main.o hns3pf/hclge_mdio.o hns3pf/hclge_tm.o hns3pf/hclge_regs.o \ hns3pf/hclge_mbx.o hns3pf/hclge_err.o hns3pf/hclge_debugfs.o hns3pf/hclge_ptp.o hns3pf/hclge_devlink.o \ - hns3_common/hclge_comm_cmd.o hns3_common/hclge_comm_rss.o hns3_common/hclge_comm_tqp_stats.o - hclge-$(CONFIG_HNS3_DCB) += hns3pf/hclge_dcb.o diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c index ea40b594dbac7b58a369ecf480df60b8d939193f..4ad4e8ab2f1f36326f8d3ab105652f4896585682 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_cmd.c @@ -48,6 +48,7 @@ void hclge_comm_cmd_reuse_desc(struct hclge_desc *desc, bool is_read) else desc->flag &= cpu_to_le16(~HCLGE_COMM_CMD_FLAG_WR); } +EXPORT_SYMBOL_GPL(hclge_comm_cmd_reuse_desc); static void hclge_comm_set_default_capability(struct hnae3_ae_dev *ae_dev, bool is_pf) @@ -72,6 +73,7 @@ void hclge_comm_cmd_setup_basic_desc(struct hclge_desc *desc, if (is_read) desc->flag |= cpu_to_le16(HCLGE_COMM_CMD_FLAG_WR); } +EXPORT_SYMBOL_GPL(hclge_comm_cmd_setup_basic_desc); int hclge_comm_firmware_compat_config(struct hnae3_ae_dev *ae_dev, struct hclge_comm_hw *hw, bool en) @@ -517,6 +519,7 @@ int hclge_comm_cmd_send(struct hclge_comm_hw *hw, struct hclge_desc *desc, return ret; } +EXPORT_SYMBOL_GPL(hclge_comm_cmd_send); static void hclge_comm_cmd_uninit_regs(struct hclge_comm_hw *hw) { @@ -553,6 +556,7 @@ void hclge_comm_cmd_uninit(struct hnae3_ae_dev *ae_dev, hclge_comm_free_cmd_desc(&cmdq->csq); hclge_comm_free_cmd_desc(&cmdq->crq); } +EXPORT_SYMBOL_GPL(hclge_comm_cmd_uninit); int hclge_comm_cmd_queue_init(struct pci_dev *pdev, struct hclge_comm_hw *hw) { @@ -591,6 +595,7 @@ err_csq: hclge_comm_free_cmd_desc(&hw->cmq.csq); return ret; } +EXPORT_SYMBOL_GPL(hclge_comm_cmd_queue_init); void hclge_comm_cmd_init_ops(struct hclge_comm_hw *hw, const struct hclge_comm_cmq_ops *ops) @@ -602,6 +607,7 @@ void hclge_comm_cmd_init_ops(struct hclge_comm_hw *hw, cmdq->ops.trace_cmd_get = ops->trace_cmd_get; } } +EXPORT_SYMBOL_GPL(hclge_comm_cmd_init_ops); int hclge_comm_cmd_init(struct hnae3_ae_dev *ae_dev, struct hclge_comm_hw *hw, u32 *fw_version, bool is_pf, @@ -672,3 +678,8 @@ err_cmd_init: return ret; } +EXPORT_SYMBOL_GPL(hclge_comm_cmd_init); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("HNS3: Hisilicon Ethernet PF/VF Common Library"); +MODULE_AUTHOR("Huawei Tech. Co., Ltd."); diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_rss.c b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_rss.c index b4ae2160aff4f3fb7aef8ee4056e752bc46662ae..4e2bb6556b1cea41566f54128d8d4f4f3505782e 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_rss.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_rss.c @@ -62,6 +62,7 @@ int hclge_comm_rss_init_cfg(struct hnae3_handle *nic, return 0; } +EXPORT_SYMBOL_GPL(hclge_comm_rss_init_cfg); void hclge_comm_get_rss_tc_info(u16 rss_size, u8 hw_tc_map, u16 *tc_offset, u16 *tc_valid, u16 *tc_size) @@ -78,6 +79,7 @@ void hclge_comm_get_rss_tc_info(u16 rss_size, u8 hw_tc_map, u16 *tc_offset, tc_offset[i] = (hw_tc_map & BIT(i)) ? rss_size * i : 0; } } +EXPORT_SYMBOL_GPL(hclge_comm_get_rss_tc_info); int hclge_comm_set_rss_tc_mode(struct hclge_comm_hw *hw, u16 *tc_offset, u16 *tc_valid, u16 *tc_size) @@ -113,6 +115,7 @@ int hclge_comm_set_rss_tc_mode(struct hclge_comm_hw *hw, u16 *tc_offset, return ret; } +EXPORT_SYMBOL_GPL(hclge_comm_set_rss_tc_mode); int hclge_comm_set_rss_hash_key(struct hclge_comm_rss_cfg *rss_cfg, struct hclge_comm_hw *hw, const u8 *key, @@ -143,6 +146,7 @@ int hclge_comm_set_rss_hash_key(struct hclge_comm_rss_cfg *rss_cfg, return 0; } +EXPORT_SYMBOL_GPL(hclge_comm_set_rss_hash_key); int hclge_comm_set_rss_tuple(struct hnae3_ae_dev *ae_dev, struct hclge_comm_hw *hw, @@ -185,11 +189,13 @@ int hclge_comm_set_rss_tuple(struct hnae3_ae_dev *ae_dev, rss_cfg->rss_tuple_sets.ipv6_fragment_en = req->ipv6_fragment_en; return 0; } +EXPORT_SYMBOL_GPL(hclge_comm_set_rss_tuple); u32 hclge_comm_get_rss_key_size(struct hnae3_handle *handle) { return HCLGE_COMM_RSS_KEY_SIZE; } +EXPORT_SYMBOL_GPL(hclge_comm_get_rss_key_size); int hclge_comm_parse_rss_hfunc(struct hclge_comm_rss_cfg *rss_cfg, const u8 hfunc, u8 *hash_algo) @@ -217,6 +223,7 @@ void hclge_comm_rss_indir_init_cfg(struct hnae3_ae_dev *ae_dev, for (i = 0; i < ae_dev->dev_specs.rss_ind_tbl_size; i++) rss_cfg->rss_indirection_tbl[i] = i % rss_cfg->rss_size; } +EXPORT_SYMBOL_GPL(hclge_comm_rss_indir_init_cfg); int hclge_comm_get_rss_tuple(struct hclge_comm_rss_cfg *rss_cfg, int flow_type, u8 *tuple_sets) @@ -250,6 +257,7 @@ int hclge_comm_get_rss_tuple(struct hclge_comm_rss_cfg *rss_cfg, int flow_type, return 0; } +EXPORT_SYMBOL_GPL(hclge_comm_get_rss_tuple); static void hclge_comm_append_rss_msb_info(struct hclge_comm_rss_ind_tbl_cmd *req, @@ -304,6 +312,7 @@ int hclge_comm_set_rss_indir_table(struct hnae3_ae_dev *ae_dev, } return 0; } +EXPORT_SYMBOL_GPL(hclge_comm_set_rss_indir_table); int hclge_comm_set_rss_input_tuple(struct hclge_comm_hw *hw, struct hclge_comm_rss_cfg *rss_cfg) @@ -332,6 +341,7 @@ int hclge_comm_set_rss_input_tuple(struct hclge_comm_hw *hw, "failed to configure rss input, ret = %d.\n", ret); return ret; } +EXPORT_SYMBOL_GPL(hclge_comm_set_rss_input_tuple); void hclge_comm_get_rss_hash_info(struct hclge_comm_rss_cfg *rss_cfg, u8 *key, u8 *hfunc) @@ -355,6 +365,7 @@ void hclge_comm_get_rss_hash_info(struct hclge_comm_rss_cfg *rss_cfg, u8 *key, if (key) memcpy(key, rss_cfg->rss_hash_key, HCLGE_COMM_RSS_KEY_SIZE); } +EXPORT_SYMBOL_GPL(hclge_comm_get_rss_hash_info); void hclge_comm_get_rss_indir_tbl(struct hclge_comm_rss_cfg *rss_cfg, u32 *indir, u16 rss_ind_tbl_size) @@ -367,6 +378,7 @@ void hclge_comm_get_rss_indir_tbl(struct hclge_comm_rss_cfg *rss_cfg, for (i = 0; i < rss_ind_tbl_size; i++) indir[i] = rss_cfg->rss_indirection_tbl[i]; } +EXPORT_SYMBOL_GPL(hclge_comm_get_rss_indir_tbl); int hclge_comm_set_rss_algo_key(struct hclge_comm_hw *hw, const u8 hfunc, const u8 *key) @@ -408,6 +420,7 @@ int hclge_comm_set_rss_algo_key(struct hclge_comm_hw *hw, const u8 hfunc, return 0; } +EXPORT_SYMBOL_GPL(hclge_comm_set_rss_algo_key); static u8 hclge_comm_get_rss_hash_bits(struct ethtool_rxnfc *nfc) { @@ -502,3 +515,4 @@ u64 hclge_comm_convert_rss_tuple(u8 tuple_sets) return tuple_data; } +EXPORT_SYMBOL_GPL(hclge_comm_convert_rss_tuple); diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.c b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.c index 618f66d9586b390a4d4ec45aafa4178ec06e5a91..2b31188ff555861a8909b7afc8b26fb0ec5e7187 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_common/hclge_comm_tqp_stats.c @@ -26,6 +26,7 @@ u64 *hclge_comm_tqps_get_stats(struct hnae3_handle *handle, u64 *data) return buff; } +EXPORT_SYMBOL_GPL(hclge_comm_tqps_get_stats); int hclge_comm_tqps_get_sset_count(struct hnae3_handle *handle) { @@ -33,6 +34,7 @@ int hclge_comm_tqps_get_sset_count(struct hnae3_handle *handle) return kinfo->num_tqps * HCLGE_COMM_QUEUE_PAIR_SIZE; } +EXPORT_SYMBOL_GPL(hclge_comm_tqps_get_sset_count); u8 *hclge_comm_tqps_get_strings(struct hnae3_handle *handle, u8 *data) { @@ -56,6 +58,7 @@ u8 *hclge_comm_tqps_get_strings(struct hnae3_handle *handle, u8 *data) return buff; } +EXPORT_SYMBOL_GPL(hclge_comm_tqps_get_strings); int hclge_comm_tqps_update_stats(struct hnae3_handle *handle, struct hclge_comm_hw *hw) @@ -99,6 +102,7 @@ int hclge_comm_tqps_update_stats(struct hnae3_handle *handle, return 0; } +EXPORT_SYMBOL_GPL(hclge_comm_tqps_update_stats); void hclge_comm_reset_tqp_stats(struct hnae3_handle *handle) { @@ -113,3 +117,4 @@ void hclge_comm_reset_tqp_stats(struct hnae3_handle *handle) memset(&tqp->tqp_stats, 0, sizeof(tqp->tqp_stats)); } } +EXPORT_SYMBOL_GPL(hclge_comm_reset_tqp_stats); diff --git a/drivers/net/ethernet/intel/e1000/Makefile b/drivers/net/ethernet/intel/e1000/Makefile index 314c52d44b7c55dfcfd28a9a7c2fd26a95580d53..79491dec47e193115a484d3cb1512bb32f2a0194 100644 --- a/drivers/net/ethernet/intel/e1000/Makefile +++ b/drivers/net/ethernet/intel/e1000/Makefile @@ -7,4 +7,4 @@ obj-$(CONFIG_E1000) += e1000.o -e1000-objs := e1000_main.o e1000_hw.o e1000_ethtool.o e1000_param.o +e1000-y := e1000_main.o e1000_hw.o e1000_ethtool.o e1000_param.o diff --git a/drivers/net/ethernet/intel/e1000e/Makefile b/drivers/net/ethernet/intel/e1000e/Makefile index 0baa15503c381cec82163a14748fff6491209680..18f22b6374d512a47d7f1481975fe88a9536f7a4 100644 --- a/drivers/net/ethernet/intel/e1000e/Makefile +++ b/drivers/net/ethernet/intel/e1000e/Makefile @@ -10,7 +10,6 @@ subdir-ccflags-y += -I$(src) obj-$(CONFIG_E1000E) += e1000e.o -e1000e-objs := 82571.o ich8lan.o 80003es2lan.o \ - mac.o manage.o nvm.o phy.o \ - param.o ethtool.o netdev.o ptp.o - +e1000e-y := 82571.o ich8lan.o 80003es2lan.o \ + mac.o manage.o nvm.o phy.o \ + param.o ethtool.o netdev.o ptp.o diff --git a/drivers/net/ethernet/intel/e1000e/ich8lan.c b/drivers/net/ethernet/intel/e1000e/ich8lan.c index f9e94be36e97f20b5aeeccf43dfe580bf7c01847..2e98a2a0bead951e714668c91a699d6bdf5fd72b 100644 --- a/drivers/net/ethernet/intel/e1000e/ich8lan.c +++ b/drivers/net/ethernet/intel/e1000e/ich8lan.c @@ -1225,6 +1225,28 @@ s32 e1000_enable_ulp_lpt_lp(struct e1000_hw *hw, bool to_sx) } release: + /* Switching PHY interface always returns MDI error + * so disable retry mechanism to avoid wasting time + */ + e1000e_disable_phy_retry(hw); + + /* Force SMBus mode in PHY */ + ret_val = e1000_read_phy_reg_hv_locked(hw, CV_SMB_CTRL, &phy_reg); + if (ret_val) { + e1000e_enable_phy_retry(hw); + hw->phy.ops.release(hw); + goto out; + } + phy_reg |= CV_SMB_CTRL_FORCE_SMBUS; + e1000_write_phy_reg_hv_locked(hw, CV_SMB_CTRL, phy_reg); + + e1000e_enable_phy_retry(hw); + + /* Force SMBus mode in MAC */ + mac_reg = er32(CTRL_EXT); + mac_reg |= E1000_CTRL_EXT_FORCE_SMBUS; + ew32(CTRL_EXT, mac_reg); + hw->phy.ops.release(hw); out: if (ret_val) diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c index 220d62fca55d1b2ba405003773f79396a9dcdb42..da5c59daf8ba94e48d6dc7cf31a61b37f86de37e 100644 --- a/drivers/net/ethernet/intel/e1000e/netdev.c +++ b/drivers/net/ethernet/intel/e1000e/netdev.c @@ -6623,7 +6623,6 @@ static int __e1000_shutdown(struct pci_dev *pdev, bool runtime) struct e1000_hw *hw = &adapter->hw; u32 ctrl, ctrl_ext, rctl, status, wufc; int retval = 0; - u16 smb_ctrl; /* Runtime suspend should only enable wakeup for link changes */ if (runtime) @@ -6697,23 +6696,6 @@ static int __e1000_shutdown(struct pci_dev *pdev, bool runtime) if (retval) return retval; } - - /* Force SMBUS to allow WOL */ - /* Switching PHY interface always returns MDI error - * so disable retry mechanism to avoid wasting time - */ - e1000e_disable_phy_retry(hw); - - e1e_rphy(hw, CV_SMB_CTRL, &smb_ctrl); - smb_ctrl |= CV_SMB_CTRL_FORCE_SMBUS; - e1e_wphy(hw, CV_SMB_CTRL, smb_ctrl); - - e1000e_enable_phy_retry(hw); - - /* Force SMBus mode in MAC */ - ctrl_ext = er32(CTRL_EXT); - ctrl_ext |= E1000_CTRL_EXT_FORCE_SMBUS; - ew32(CTRL_EXT, ctrl_ext); } /* Ensure that the appropriate bits are set in LPI_CTRL diff --git a/drivers/net/ethernet/intel/i40e/Makefile b/drivers/net/ethernet/intel/i40e/Makefile index cad93f323bd5223d912580c1707b3c2fe65a10e0..9faa4339a76c35a40cbbe3ad6f69b4b89c3aa967 100644 --- a/drivers/net/ethernet/intel/i40e/Makefile +++ b/drivers/net/ethernet/intel/i40e/Makefile @@ -10,7 +10,7 @@ subdir-ccflags-y += -I$(src) obj-$(CONFIG_I40E) += i40e.o -i40e-objs := i40e_main.o \ +i40e-y := i40e_main.o \ i40e_ethtool.o \ i40e_adminq.o \ i40e_common.o \ diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c index 1f188c052828b47b37ebb72059dfba61be96d00a..284c3fad5a6e4f757097ecaf5e6a72816c09f67b 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_main.c +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c @@ -11171,6 +11171,8 @@ static void i40e_reset_and_rebuild(struct i40e_pf *pf, bool reinit, ret = i40e_reset(pf); if (!ret) i40e_rebuild(pf, reinit, lock_acquired); + else + dev_err(&pf->pdev->dev, "%s: i40e_reset() FAILED", __func__); } /** @@ -16334,6 +16336,139 @@ unmap: pci_disable_device(pdev); } +/** + * i40e_enable_mc_magic_wake - enable multicast magic packet wake up + * using the mac_address_write admin q function + * @pf: pointer to i40e_pf struct + **/ +static void i40e_enable_mc_magic_wake(struct i40e_pf *pf) +{ + struct i40e_vsi *main_vsi = i40e_pf_get_main_vsi(pf); + struct i40e_hw *hw = &pf->hw; + u8 mac_addr[6]; + u16 flags = 0; + int ret; + + /* Get current MAC address in case it's an LAA */ + if (main_vsi && main_vsi->netdev) { + ether_addr_copy(mac_addr, main_vsi->netdev->dev_addr); + } else { + dev_err(&pf->pdev->dev, + "Failed to retrieve MAC address; using default\n"); + ether_addr_copy(mac_addr, hw->mac.addr); + } + + /* The FW expects the mac address write cmd to first be called with + * one of these flags before calling it again with the multicast + * enable flags. + */ + flags = I40E_AQC_WRITE_TYPE_LAA_WOL; + + if (hw->func_caps.flex10_enable && hw->partition_id != 1) + flags = I40E_AQC_WRITE_TYPE_LAA_ONLY; + + ret = i40e_aq_mac_address_write(hw, flags, mac_addr, NULL); + if (ret) { + dev_err(&pf->pdev->dev, + "Failed to update MAC address registers; cannot enable Multicast Magic packet wake up"); + return; + } + + flags = I40E_AQC_MC_MAG_EN + | I40E_AQC_WOL_PRESERVE_ON_PFR + | I40E_AQC_WRITE_TYPE_UPDATE_MC_MAG; + ret = i40e_aq_mac_address_write(hw, flags, mac_addr, NULL); + if (ret) + dev_err(&pf->pdev->dev, + "Failed to enable Multicast Magic Packet wake up\n"); +} + +/** + * i40e_io_suspend - suspend all IO operations + * @pf: pointer to i40e_pf struct + * + **/ +static int i40e_io_suspend(struct i40e_pf *pf) +{ + struct i40e_hw *hw = &pf->hw; + + set_bit(__I40E_DOWN, pf->state); + + /* Ensure service task will not be running */ + del_timer_sync(&pf->service_timer); + cancel_work_sync(&pf->service_task); + + /* Client close must be called explicitly here because the timer + * has been stopped. + */ + i40e_notify_client_of_netdev_close(pf, false); + + if (test_bit(I40E_HW_CAP_WOL_MC_MAGIC_PKT_WAKE, pf->hw.caps) && + pf->wol_en) + i40e_enable_mc_magic_wake(pf); + + /* Since we're going to destroy queues during the + * i40e_clear_interrupt_scheme() we should hold the RTNL lock for this + * whole section + */ + rtnl_lock(); + + i40e_prep_for_reset(pf); + + wr32(hw, I40E_PFPM_APM, (pf->wol_en ? I40E_PFPM_APM_APME_MASK : 0)); + wr32(hw, I40E_PFPM_WUFC, (pf->wol_en ? I40E_PFPM_WUFC_MAG_MASK : 0)); + + /* Clear the interrupt scheme and release our IRQs so that the system + * can safely hibernate even when there are a large number of CPUs. + * Otherwise hibernation might fail when mapping all the vectors back + * to CPU0. + */ + i40e_clear_interrupt_scheme(pf); + + rtnl_unlock(); + + return 0; +} + +/** + * i40e_io_resume - resume IO operations + * @pf: pointer to i40e_pf struct + * + **/ +static int i40e_io_resume(struct i40e_pf *pf) +{ + struct device *dev = &pf->pdev->dev; + int err; + + /* We need to hold the RTNL lock prior to restoring interrupt schemes, + * since we're going to be restoring queues + */ + rtnl_lock(); + + /* We cleared the interrupt scheme when we suspended, so we need to + * restore it now to resume device functionality. + */ + err = i40e_restore_interrupt_scheme(pf); + if (err) { + dev_err(dev, "Cannot restore interrupt scheme: %d\n", + err); + } + + clear_bit(__I40E_DOWN, pf->state); + i40e_reset_and_rebuild(pf, false, true); + + rtnl_unlock(); + + /* Clear suspended state last after everything is recovered */ + clear_bit(__I40E_SUSPENDED, pf->state); + + /* Restart the service task */ + mod_timer(&pf->service_timer, + round_jiffies(jiffies + pf->service_timer_period)); + + return 0; +} + /** * i40e_pci_error_detected - warning that something funky happened in PCI land * @pdev: PCI device information struct @@ -16358,7 +16493,7 @@ static pci_ers_result_t i40e_pci_error_detected(struct pci_dev *pdev, /* shutdown all operations */ if (!test_bit(__I40E_SUSPENDED, pf->state)) - i40e_prep_for_reset(pf); + i40e_io_suspend(pf); /* Request a slot reset */ return PCI_ERS_RESULT_NEED_RESET; @@ -16380,7 +16515,8 @@ static pci_ers_result_t i40e_pci_error_slot_reset(struct pci_dev *pdev) u32 reg; dev_dbg(&pdev->dev, "%s\n", __func__); - if (pci_enable_device_mem(pdev)) { + /* enable I/O and memory of the device */ + if (pci_enable_device(pdev)) { dev_info(&pdev->dev, "Cannot re-enable PCI device after reset.\n"); result = PCI_ERS_RESULT_DISCONNECT; @@ -16443,54 +16579,7 @@ static void i40e_pci_error_resume(struct pci_dev *pdev) if (test_bit(__I40E_SUSPENDED, pf->state)) return; - i40e_handle_reset_warning(pf, false); -} - -/** - * i40e_enable_mc_magic_wake - enable multicast magic packet wake up - * using the mac_address_write admin q function - * @pf: pointer to i40e_pf struct - **/ -static void i40e_enable_mc_magic_wake(struct i40e_pf *pf) -{ - struct i40e_vsi *main_vsi = i40e_pf_get_main_vsi(pf); - struct i40e_hw *hw = &pf->hw; - u8 mac_addr[6]; - u16 flags = 0; - int ret; - - /* Get current MAC address in case it's an LAA */ - if (main_vsi && main_vsi->netdev) { - ether_addr_copy(mac_addr, main_vsi->netdev->dev_addr); - } else { - dev_err(&pf->pdev->dev, - "Failed to retrieve MAC address; using default\n"); - ether_addr_copy(mac_addr, hw->mac.addr); - } - - /* The FW expects the mac address write cmd to first be called with - * one of these flags before calling it again with the multicast - * enable flags. - */ - flags = I40E_AQC_WRITE_TYPE_LAA_WOL; - - if (hw->func_caps.flex10_enable && hw->partition_id != 1) - flags = I40E_AQC_WRITE_TYPE_LAA_ONLY; - - ret = i40e_aq_mac_address_write(hw, flags, mac_addr, NULL); - if (ret) { - dev_err(&pf->pdev->dev, - "Failed to update MAC address registers; cannot enable Multicast Magic packet wake up"); - return; - } - - flags = I40E_AQC_MC_MAG_EN - | I40E_AQC_WOL_PRESERVE_ON_PFR - | I40E_AQC_WRITE_TYPE_UPDATE_MC_MAG; - ret = i40e_aq_mac_address_write(hw, flags, mac_addr, NULL); - if (ret) - dev_err(&pf->pdev->dev, - "Failed to enable Multicast Magic Packet wake up\n"); + i40e_io_resume(pf); } /** @@ -16552,48 +16641,11 @@ static void i40e_shutdown(struct pci_dev *pdev) static int i40e_suspend(struct device *dev) { struct i40e_pf *pf = dev_get_drvdata(dev); - struct i40e_hw *hw = &pf->hw; /* If we're already suspended, then there is nothing to do */ if (test_and_set_bit(__I40E_SUSPENDED, pf->state)) return 0; - - set_bit(__I40E_DOWN, pf->state); - - /* Ensure service task will not be running */ - del_timer_sync(&pf->service_timer); - cancel_work_sync(&pf->service_task); - - /* Client close must be called explicitly here because the timer - * has been stopped. - */ - i40e_notify_client_of_netdev_close(pf, false); - - if (test_bit(I40E_HW_CAP_WOL_MC_MAGIC_PKT_WAKE, pf->hw.caps) && - pf->wol_en) - i40e_enable_mc_magic_wake(pf); - - /* Since we're going to destroy queues during the - * i40e_clear_interrupt_scheme() we should hold the RTNL lock for this - * whole section - */ - rtnl_lock(); - - i40e_prep_for_reset(pf); - - wr32(hw, I40E_PFPM_APM, (pf->wol_en ? I40E_PFPM_APM_APME_MASK : 0)); - wr32(hw, I40E_PFPM_WUFC, (pf->wol_en ? I40E_PFPM_WUFC_MAG_MASK : 0)); - - /* Clear the interrupt scheme and release our IRQs so that the system - * can safely hibernate even when there are a large number of CPUs. - * Otherwise hibernation might fail when mapping all the vectors back - * to CPU0. - */ - i40e_clear_interrupt_scheme(pf); - - rtnl_unlock(); - - return 0; + return i40e_io_suspend(pf); } /** @@ -16603,39 +16655,11 @@ static int i40e_suspend(struct device *dev) static int i40e_resume(struct device *dev) { struct i40e_pf *pf = dev_get_drvdata(dev); - int err; /* If we're not suspended, then there is nothing to do */ if (!test_bit(__I40E_SUSPENDED, pf->state)) return 0; - - /* We need to hold the RTNL lock prior to restoring interrupt schemes, - * since we're going to be restoring queues - */ - rtnl_lock(); - - /* We cleared the interrupt scheme when we suspended, so we need to - * restore it now to resume device functionality. - */ - err = i40e_restore_interrupt_scheme(pf); - if (err) { - dev_err(dev, "Cannot restore interrupt scheme: %d\n", - err); - } - - clear_bit(__I40E_DOWN, pf->state); - i40e_reset_and_rebuild(pf, false, true); - - rtnl_unlock(); - - /* Clear suspended state last after everything is recovered */ - clear_bit(__I40E_SUSPENDED, pf->state); - - /* Restart the service task */ - mod_timer(&pf->service_timer, - round_jiffies(jiffies + pf->service_timer_period)); - - return 0; + return i40e_io_resume(pf); } static const struct pci_error_handlers i40e_err_handler = { diff --git a/drivers/net/ethernet/intel/iavf/Makefile b/drivers/net/ethernet/intel/iavf/Makefile index 2d154a4e2fd76938b71d1ba9700f33521a868ce8..356ac9faa5bf9de6c386b4828fdd32fae381ee50 100644 --- a/drivers/net/ethernet/intel/iavf/Makefile +++ b/drivers/net/ethernet/intel/iavf/Makefile @@ -11,6 +11,5 @@ subdir-ccflags-y += -I$(src) obj-$(CONFIG_IAVF) += iavf.o -iavf-objs := iavf_main.o iavf_ethtool.o iavf_virtchnl.o iavf_fdir.o \ - iavf_adv_rss.o \ - iavf_txrx.o iavf_common.o iavf_adminq.o +iavf-y := iavf_main.o iavf_ethtool.o iavf_virtchnl.o iavf_fdir.o \ + iavf_adv_rss.o iavf_txrx.o iavf_common.o iavf_adminq.o diff --git a/drivers/net/ethernet/intel/ice/devlink/devlink.c b/drivers/net/ethernet/intel/ice/devlink/devlink.c index c4b69655cdf57db4a5b2613755fe68e147e23605..704e9ad5144e8c2844b6343c63126d1724ad0f08 100644 --- a/drivers/net/ethernet/intel/ice/devlink/devlink.c +++ b/drivers/net/ethernet/intel/ice/devlink/devlink.c @@ -1388,7 +1388,7 @@ enum ice_param_id { ICE_DEVLINK_PARAM_ID_TX_SCHED_LAYERS, }; -static const struct devlink_param ice_devlink_params[] = { +static const struct devlink_param ice_dvl_rdma_params[] = { DEVLINK_PARAM_GENERIC(ENABLE_ROCE, BIT(DEVLINK_PARAM_CMODE_RUNTIME), ice_devlink_enable_roce_get, ice_devlink_enable_roce_set, @@ -1397,6 +1397,9 @@ static const struct devlink_param ice_devlink_params[] = { ice_devlink_enable_iw_get, ice_devlink_enable_iw_set, ice_devlink_enable_iw_validate), +}; + +static const struct devlink_param ice_dvl_sched_params[] = { DEVLINK_PARAM_DRIVER(ICE_DEVLINK_PARAM_ID_TX_SCHED_LAYERS, "tx_scheduling_layers", DEVLINK_PARAM_TYPE_U8, @@ -1464,21 +1467,31 @@ int ice_devlink_register_params(struct ice_pf *pf) { struct devlink *devlink = priv_to_devlink(pf); struct ice_hw *hw = &pf->hw; - size_t params_size; + int status; - params_size = ARRAY_SIZE(ice_devlink_params); + status = devl_params_register(devlink, ice_dvl_rdma_params, + ARRAY_SIZE(ice_dvl_rdma_params)); + if (status) + return status; - if (!hw->func_caps.common_cap.tx_sched_topo_comp_mode_en) - params_size--; + if (hw->func_caps.common_cap.tx_sched_topo_comp_mode_en) + status = devl_params_register(devlink, ice_dvl_sched_params, + ARRAY_SIZE(ice_dvl_sched_params)); - return devl_params_register(devlink, ice_devlink_params, - params_size); + return status; } void ice_devlink_unregister_params(struct ice_pf *pf) { - devl_params_unregister(priv_to_devlink(pf), ice_devlink_params, - ARRAY_SIZE(ice_devlink_params)); + struct devlink *devlink = priv_to_devlink(pf); + struct ice_hw *hw = &pf->hw; + + devl_params_unregister(devlink, ice_dvl_rdma_params, + ARRAY_SIZE(ice_dvl_rdma_params)); + + if (hw->func_caps.common_cap.tx_sched_topo_comp_mode_en) + devl_params_unregister(devlink, ice_dvl_sched_params, + ARRAY_SIZE(ice_dvl_sched_params)); } #define ICE_DEVLINK_READ_BLK_SIZE (1024 * 1024) diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index 6ad8002b22e14a9a86f2ed994a6798d67627b5f3..99a75a59078ef3e6c71c46696b694d00fd082ea1 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -409,7 +409,6 @@ struct ice_vsi { struct ice_tc_cfg tc_cfg; struct bpf_prog *xdp_prog; struct ice_tx_ring **xdp_rings; /* XDP ring array */ - unsigned long *af_xdp_zc_qps; /* tracks AF_XDP ZC enabled qps */ u16 num_xdp_txq; /* Used XDP queues */ u8 xdp_mapping_mode; /* ICE_MAP_MODE_[CONTIG|SCATTER] */ @@ -746,6 +745,25 @@ static inline void ice_set_ring_xdp(struct ice_tx_ring *ring) ring->flags |= ICE_TX_FLAGS_RING_XDP; } +/** + * ice_get_xp_from_qid - get ZC XSK buffer pool bound to a queue ID + * @vsi: pointer to VSI + * @qid: index of a queue to look at XSK buff pool presence + * + * Return: A pointer to xsk_buff_pool structure if there is a buffer pool + * attached and configured as zero-copy, NULL otherwise. + */ +static inline struct xsk_buff_pool *ice_get_xp_from_qid(struct ice_vsi *vsi, + u16 qid) +{ + struct xsk_buff_pool *pool = xsk_get_pool_from_qid(vsi->netdev, qid); + + if (!ice_is_xdp_ena_vsi(vsi)) + return NULL; + + return (pool && pool->dev) ? pool : NULL; +} + /** * ice_xsk_pool - get XSK buffer pool bound to a ring * @ring: Rx ring to use @@ -758,10 +776,7 @@ static inline struct xsk_buff_pool *ice_xsk_pool(struct ice_rx_ring *ring) struct ice_vsi *vsi = ring->vsi; u16 qid = ring->q_index; - if (!ice_is_xdp_ena_vsi(vsi) || !test_bit(qid, vsi->af_xdp_zc_qps)) - return NULL; - - return xsk_get_pool_from_qid(vsi->netdev, qid); + return ice_get_xp_from_qid(vsi, qid); } /** @@ -786,12 +801,7 @@ static inline void ice_tx_xsk_pool(struct ice_vsi *vsi, u16 qid) if (!ring) return; - if (!ice_is_xdp_ena_vsi(vsi) || !test_bit(qid, vsi->af_xdp_zc_qps)) { - ring->xsk_pool = NULL; - return; - } - - ring->xsk_pool = xsk_get_pool_from_qid(vsi->netdev, qid); + ring->xsk_pool = ice_get_xp_from_qid(vsi, qid); } /** @@ -920,9 +930,17 @@ int ice_down(struct ice_vsi *vsi); int ice_down_up(struct ice_vsi *vsi); int ice_vsi_cfg_lan(struct ice_vsi *vsi); struct ice_vsi *ice_lb_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi); + +enum ice_xdp_cfg { + ICE_XDP_CFG_FULL, /* Fully apply new config in .ndo_bpf() */ + ICE_XDP_CFG_PART, /* Save/use part of config in VSI rebuild */ +}; + int ice_vsi_determine_xdp_res(struct ice_vsi *vsi); -int ice_prepare_xdp_rings(struct ice_vsi *vsi, struct bpf_prog *prog); -int ice_destroy_xdp_rings(struct ice_vsi *vsi); +int ice_prepare_xdp_rings(struct ice_vsi *vsi, struct bpf_prog *prog, + enum ice_xdp_cfg cfg_type); +int ice_destroy_xdp_rings(struct ice_vsi *vsi, enum ice_xdp_cfg cfg_type); +void ice_map_xdp_rings(struct ice_vsi *vsi); int ice_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames, u32 flags); diff --git a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h index e76c388b99055f6586c1b04b2af6cdfa35a3dbca..621a2ca7093ebc994a34e10cfa763c05272ce1ac 100644 --- a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h +++ b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h @@ -122,6 +122,7 @@ struct ice_aqc_list_caps_elem { #define ICE_AQC_CAPS_POST_UPDATE_RESET_RESTRICT 0x0077 #define ICE_AQC_CAPS_NVM_MGMT 0x0080 #define ICE_AQC_CAPS_TX_SCHED_TOPO_COMP_MODE 0x0085 +#define ICE_AQC_CAPS_NAC_TOPOLOGY 0x0087 #define ICE_AQC_CAPS_FW_LAG_SUPPORT 0x0092 #define ICE_AQC_BIT_ROCEV2_LAG 0x01 #define ICE_AQC_BIT_SRIOV_LAG 0x02 diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c index 687f6cb2b917afc55de7020c401c5095c6163825..5d396c1a7731482f725561a8eff709ecd3cc793e 100644 --- a/drivers/net/ethernet/intel/ice/ice_base.c +++ b/drivers/net/ethernet/intel/ice/ice_base.c @@ -842,6 +842,9 @@ void ice_vsi_map_rings_to_vectors(struct ice_vsi *vsi) } rx_rings_rem -= rx_rings_per_v; } + + if (ice_is_xdp_ena_vsi(vsi)) + ice_map_xdp_rings(vsi); } /** diff --git a/drivers/net/ethernet/intel/ice/ice_cgu_regs.h b/drivers/net/ethernet/intel/ice/ice_cgu_regs.h index 57abd52386d0d202431ec0dc55dd63a7e5aa9f93..10d9d74f354555555a61d6f75238bf6703777881 100644 --- a/drivers/net/ethernet/intel/ice/ice_cgu_regs.h +++ b/drivers/net/ethernet/intel/ice/ice_cgu_regs.h @@ -23,7 +23,18 @@ union nac_cgu_dword9 { u32 clk_synce0_amp : 2; u32 one_pps_out_amp : 2; u32 misc24 : 12; - } field; + }; + u32 val; +}; + +#define NAC_CGU_DWORD16_E825C 0x40 +union nac_cgu_dword16_e825c { + struct { + u32 synce_remndr : 6; + u32 synce_phlmt_en : 1; + u32 misc13 : 17; + u32 tspll_ck_refclkfreq : 8; + }; u32 val; }; @@ -39,7 +50,7 @@ union nac_cgu_dword19 { u32 japll_ndivratio : 4; u32 japll_iref_ndivratio : 3; u32 misc27 : 1; - } field; + }; u32 val; }; @@ -63,7 +74,23 @@ union nac_cgu_dword22 { u32 fdpllclk_sel_div2 : 1; u32 time1588clk_sel_div2 : 1; u32 misc3 : 1; - } field; + }; + u32 val; +}; + +#define NAC_CGU_DWORD23_E825C 0x5C +union nac_cgu_dword23_e825c { + struct { + u32 cgupll_fbdiv_intgr : 10; + u32 ux56pll_fbdiv_intgr : 10; + u32 misc20 : 4; + u32 ts_pll_enable : 1; + u32 time_sync_tspll_align_sel : 1; + u32 ext_synce_sel : 1; + u32 ref1588_ck_div : 4; + u32 time_ref_sel : 1; + + }; u32 val; }; @@ -77,7 +104,7 @@ union nac_cgu_dword24 { u32 ext_synce_sel : 1; u32 ref1588_ck_div : 4; u32 time_ref_sel : 1; - } field; + }; u32 val; }; @@ -92,7 +119,7 @@ union tspll_cntr_bist_settings { u32 i_plllock_cnt_6_0 : 7; u32 i_plllock_cnt_10_7 : 4; u32 reserved200 : 4; - } field; + }; u32 val; }; @@ -109,7 +136,45 @@ union tspll_ro_bwm_lf { u32 afcdone_cri : 1; u32 feedfwrdgain_cal_cri_7_0 : 8; u32 m2fbdivmod_cri_7_0 : 8; - } field; + }; + u32 val; +}; + +#define TSPLL_RO_LOCK_E825C 0x3f0 +union tspll_ro_lock_e825c { + struct { + u32 bw_freqov_high_cri_7_0 : 8; + u32 bw_freqov_high_cri_9_8 : 2; + u32 reserved455 : 1; + u32 plllock_gain_tran_cri : 1; + u32 plllock_true_lock_cri : 1; + u32 pllunlock_flag_cri : 1; + u32 afcerr_cri : 1; + u32 afcdone_cri : 1; + u32 feedfwrdgain_cal_cri_7_0 : 8; + u32 reserved462 : 8; + }; + u32 val; +}; + +#define TSPLL_BW_TDC_E825C 0x31c +union tspll_bw_tdc_e825c { + struct { + u32 i_tdc_offset_lock_1_0 : 2; + u32 i_bbthresh1_2_0 : 3; + u32 i_bbthresh2_2_0 : 3; + u32 i_tdcsel_1_0 : 2; + u32 i_tdcovccorr_en_h : 1; + u32 i_divretimeren : 1; + u32 i_bw_ampmeas_window : 1; + u32 i_bw_lowerbound_2_0 : 3; + u32 i_bw_upperbound_2_0 : 3; + u32 i_bw_mode_1_0 : 2; + u32 i_ft_mode_sel_2_0 : 3; + u32 i_bwphase_4_0 : 5; + u32 i_plllock_sel_1_0 : 2; + u32 i_afc_divratio : 1; + }; u32 val; }; diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c index 5649b257e6312e57e0040962f7213a606a2581dc..9ae61cd8923e390d36a21a535d8d955df8733432 100644 --- a/drivers/net/ethernet/intel/ice/ice_common.c +++ b/drivers/net/ethernet/intel/ice/ice_common.c @@ -239,6 +239,30 @@ bool ice_is_e810t(struct ice_hw *hw) return false; } +/** + * ice_is_e822 - Check if a device is E822 family device + * @hw: pointer to the hardware structure + * + * Return: true if the device is E822 based, false if not. + */ +bool ice_is_e822(struct ice_hw *hw) +{ + switch (hw->device_id) { + case ICE_DEV_ID_E822C_BACKPLANE: + case ICE_DEV_ID_E822C_QSFP: + case ICE_DEV_ID_E822C_SFP: + case ICE_DEV_ID_E822C_10G_BASE_T: + case ICE_DEV_ID_E822C_SGMII: + case ICE_DEV_ID_E822L_BACKPLANE: + case ICE_DEV_ID_E822L_SFP: + case ICE_DEV_ID_E822L_10G_BASE_T: + case ICE_DEV_ID_E822L_SGMII: + return true; + default: + return false; + } +} + /** * ice_is_e823 * @hw: pointer to the hardware structure @@ -2290,8 +2314,13 @@ ice_parse_1588_func_caps(struct ice_hw *hw, struct ice_hw_func_caps *func_p, info->tmr_index_owned = ((number & ICE_TS_TMR_IDX_OWND_M) != 0); info->tmr_index_assoc = ((number & ICE_TS_TMR_IDX_ASSOC_M) != 0); - info->clk_freq = FIELD_GET(ICE_TS_CLK_FREQ_M, number); - info->clk_src = ((number & ICE_TS_CLK_SRC_M) != 0); + if (!ice_is_e825c(hw)) { + info->clk_freq = FIELD_GET(ICE_TS_CLK_FREQ_M, number); + info->clk_src = ((number & ICE_TS_CLK_SRC_M) != 0); + } else { + info->clk_freq = ICE_TIME_REF_FREQ_156_250; + info->clk_src = ICE_CLK_SRC_TCXO; + } if (info->clk_freq < NUM_ICE_TIME_REF_FREQ) { info->time_ref = (enum ice_time_ref_freq)info->clk_freq; @@ -2564,6 +2593,34 @@ ice_parse_sensor_reading_cap(struct ice_hw *hw, struct ice_hw_dev_caps *dev_p, dev_p->supported_sensors); } +/** + * ice_parse_nac_topo_dev_caps - Parse ICE_AQC_CAPS_NAC_TOPOLOGY cap + * @hw: pointer to the HW struct + * @dev_p: pointer to device capabilities structure + * @cap: capability element to parse + * + * Parse ICE_AQC_CAPS_NAC_TOPOLOGY for device capabilities. + */ +static void ice_parse_nac_topo_dev_caps(struct ice_hw *hw, + struct ice_hw_dev_caps *dev_p, + struct ice_aqc_list_caps_elem *cap) +{ + dev_p->nac_topo.mode = le32_to_cpu(cap->number); + dev_p->nac_topo.id = le32_to_cpu(cap->phys_id) & ICE_NAC_TOPO_ID_M; + + dev_info(ice_hw_to_dev(hw), + "PF is configured in %s mode with IP instance ID %d\n", + (dev_p->nac_topo.mode & ICE_NAC_TOPO_PRIMARY_M) ? + "primary" : "secondary", dev_p->nac_topo.id); + + ice_debug(hw, ICE_DBG_INIT, "dev caps: nac topology is_primary = %d\n", + !!(dev_p->nac_topo.mode & ICE_NAC_TOPO_PRIMARY_M)); + ice_debug(hw, ICE_DBG_INIT, "dev caps: nac topology is_dual = %d\n", + !!(dev_p->nac_topo.mode & ICE_NAC_TOPO_DUAL_M)); + ice_debug(hw, ICE_DBG_INIT, "dev caps: nac topology id = %d\n", + dev_p->nac_topo.id); +} + /** * ice_parse_dev_caps - Parse device capabilities * @hw: pointer to the HW struct @@ -2615,6 +2672,9 @@ ice_parse_dev_caps(struct ice_hw *hw, struct ice_hw_dev_caps *dev_p, case ICE_AQC_CAPS_SENSOR_READING: ice_parse_sensor_reading_cap(hw, dev_p, &cap_resp[i]); break; + case ICE_AQC_CAPS_NAC_TOPOLOGY: + ice_parse_nac_topo_dev_caps(hw, dev_p, &cap_resp[i]); + break; default: /* Don't list common capabilities as unknown */ if (!found) @@ -3043,11 +3103,13 @@ bool ice_is_100m_speed_supported(struct ice_hw *hw) * Note: In the structure of [phy_type_low, phy_type_high], there should * be one bit set, as this function will convert one PHY type to its * speed. - * If no bit gets set, ICE_AQ_LINK_SPEED_UNKNOWN will be returned - * If more than one bit gets set, ICE_AQ_LINK_SPEED_UNKNOWN will be returned + * + * Return: + * * PHY speed for recognized PHY type + * * If no bit gets set, ICE_AQ_LINK_SPEED_UNKNOWN will be returned + * * If more than one bit gets set, ICE_AQ_LINK_SPEED_UNKNOWN will be returned */ -static u16 -ice_get_link_speed_based_on_phy_type(u64 phy_type_low, u64 phy_type_high) +u16 ice_get_link_speed_based_on_phy_type(u64 phy_type_low, u64 phy_type_high) { u16 speed_phy_type_high = ICE_AQ_LINK_SPEED_UNKNOWN; u16 speed_phy_type_low = ICE_AQ_LINK_SPEED_UNKNOWN; @@ -3148,6 +3210,16 @@ ice_get_link_speed_based_on_phy_type(u64 phy_type_low, u64 phy_type_high) case ICE_PHY_TYPE_HIGH_100G_AUI2: speed_phy_type_high = ICE_AQ_LINK_SPEED_100GB; break; + case ICE_PHY_TYPE_HIGH_200G_CR4_PAM4: + case ICE_PHY_TYPE_HIGH_200G_SR4: + case ICE_PHY_TYPE_HIGH_200G_FR4: + case ICE_PHY_TYPE_HIGH_200G_LR4: + case ICE_PHY_TYPE_HIGH_200G_DR4: + case ICE_PHY_TYPE_HIGH_200G_KR4_PAM4: + case ICE_PHY_TYPE_HIGH_200G_AUI4_AOC_ACC: + case ICE_PHY_TYPE_HIGH_200G_AUI4: + speed_phy_type_high = ICE_AQ_LINK_SPEED_200GB; + break; default: speed_phy_type_high = ICE_AQ_LINK_SPEED_UNKNOWN; break; diff --git a/drivers/net/ethernet/intel/ice/ice_common.h b/drivers/net/ethernet/intel/ice/ice_common.h index ffb22c7ce28b7e035f6b711920d31f60ee3f5987..86cc1df469dd9031f58e8d44f1a5f0f3060b2628 100644 --- a/drivers/net/ethernet/intel/ice/ice_common.h +++ b/drivers/net/ethernet/intel/ice/ice_common.h @@ -249,6 +249,7 @@ void ice_stat_update32(struct ice_hw *hw, u32 reg, bool prev_stat_loaded, u64 *prev_stat, u64 *cur_stat); bool ice_is_e810t(struct ice_hw *hw); +bool ice_is_e822(struct ice_hw *hw); bool ice_is_e823(struct ice_hw *hw); bool ice_is_e825c(struct ice_hw *hw); int @@ -261,6 +262,7 @@ int ice_aq_get_gpio(struct ice_hw *hw, u16 gpio_ctrl_handle, u8 pin_idx, bool *value, struct ice_sq_cd *cd); bool ice_is_100m_speed_supported(struct ice_hw *hw); +u16 ice_get_link_speed_based_on_phy_type(u64 phy_type_low, u64 phy_type_high); int ice_aq_set_lldp_mib(struct ice_hw *hw, u8 mib_type, void *buf, u16 buf_size, struct ice_sq_cd *cd); diff --git a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h index cfac1d432c155069b47cf18c2293cc168d7064c4..91cbae1eec89a0b7267a799a6d2445ff72a31783 100644 --- a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h +++ b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h @@ -157,6 +157,8 @@ #define GLGEN_RTRIG_CORER_M BIT(0) #define GLGEN_RTRIG_GLOBR_M BIT(1) #define GLGEN_STAT 0x000B612C +#define GLGEN_SWITCH_MODE_CONFIG 0x000B81E0 +#define GLGEN_SWITCH_MODE_CONFIG_25X4_QUAD_M BIT(2) #define GLGEN_VFLRSTAT(_i) (0x00093A04 + ((_i) * 4)) #define PFGEN_CTRL 0x00091000 #define PFGEN_CTRL_PFSWR_M BIT(0) @@ -177,6 +179,8 @@ #define GLINT_CTL_ITR_GRAN_50_M ICE_M(0xF, 24) #define GLINT_CTL_ITR_GRAN_25_S 28 #define GLINT_CTL_ITR_GRAN_25_M ICE_M(0xF, 28) +#define GLGEN_MAC_LINK_TOPO 0x000B81DC +#define GLGEN_MAC_LINK_TOPO_LINK_TOPO_M GENMASK(1, 0) #define GLINT_DYN_CTL(_INT) (0x00160000 + ((_INT) * 4)) #define GLINT_DYN_CTL_INTENA_M BIT(0) #define GLINT_DYN_CTL_CLEARPBA_M BIT(1) diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index 5371e91f6bbb4b48f4065421ac9a68d82fda17d1..f559e60992fa5cf9575090198ed9b270a9854cd4 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -114,14 +114,8 @@ static int ice_vsi_alloc_arrays(struct ice_vsi *vsi) if (!vsi->q_vectors) goto err_vectors; - vsi->af_xdp_zc_qps = bitmap_zalloc(max_t(int, vsi->alloc_txq, vsi->alloc_rxq), GFP_KERNEL); - if (!vsi->af_xdp_zc_qps) - goto err_zc_qps; - return 0; -err_zc_qps: - devm_kfree(dev, vsi->q_vectors); err_vectors: devm_kfree(dev, vsi->rxq_map); err_rxq_map: @@ -309,8 +303,6 @@ static void ice_vsi_free_arrays(struct ice_vsi *vsi) dev = ice_pf_to_dev(pf); - bitmap_free(vsi->af_xdp_zc_qps); - vsi->af_xdp_zc_qps = NULL; /* free the ring and vector containers */ devm_kfree(dev, vsi->q_vectors); vsi->q_vectors = NULL; @@ -2282,22 +2274,23 @@ static int ice_vsi_cfg_def(struct ice_vsi *vsi) if (ret) goto unroll_vector_base; - ice_vsi_map_rings_to_vectors(vsi); - - /* Associate q_vector rings to napi */ - ice_vsi_set_napi_queues(vsi); - - vsi->stat_offsets_loaded = false; - if (ice_is_xdp_ena_vsi(vsi)) { ret = ice_vsi_determine_xdp_res(vsi); if (ret) goto unroll_vector_base; - ret = ice_prepare_xdp_rings(vsi, vsi->xdp_prog); + ret = ice_prepare_xdp_rings(vsi, vsi->xdp_prog, + ICE_XDP_CFG_PART); if (ret) goto unroll_vector_base; } + ice_vsi_map_rings_to_vectors(vsi); + + /* Associate q_vector rings to napi */ + ice_vsi_set_napi_queues(vsi); + + vsi->stat_offsets_loaded = false; + /* ICE_VSI_CTRL does not need RSS so skip RSS processing */ if (vsi->type != ICE_VSI_CTRL) /* Do not exit if configuring RSS had an issue, at @@ -2437,7 +2430,7 @@ void ice_vsi_decfg(struct ice_vsi *vsi) /* return value check can be skipped here, it always returns * 0 if reset is in progress */ - ice_destroy_xdp_rings(vsi); + ice_destroy_xdp_rings(vsi, ICE_XDP_CFG_PART); ice_vsi_clear_rings(vsi); ice_vsi_free_q_vectors(vsi); @@ -2587,8 +2580,8 @@ void ice_vsi_free_irq(struct ice_vsi *vsi) if (!IS_ENABLED(CONFIG_RFS_ACCEL)) irq_set_affinity_notifier(irq_num, NULL); - /* clear the affinity_mask in the IRQ descriptor */ - irq_set_affinity_hint(irq_num, NULL); + /* clear the affinity_hint in the IRQ descriptor */ + irq_update_affinity_hint(irq_num, NULL); synchronize_irq(irq_num); devm_free_irq(ice_pf_to_dev(pf), irq_num, vsi->q_vectors[i]); } diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index f60c022f79609695bcad9f8ff581c389de02e35c..7d9a4e856f6196c14964f77014f5d40a53f0f107 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -2607,7 +2607,7 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename) } /* assign the mask for this irq */ - irq_set_affinity_hint(irq_num, &q_vector->affinity_mask); + irq_update_affinity_hint(irq_num, &q_vector->affinity_mask); } err = ice_set_cpu_rx_rmap(vsi); @@ -2625,7 +2625,7 @@ free_q_irqs: irq_num = vsi->q_vectors[vector]->irq.virq; if (!IS_ENABLED(CONFIG_RFS_ACCEL)) irq_set_affinity_notifier(irq_num, NULL); - irq_set_affinity_hint(irq_num, NULL); + irq_update_affinity_hint(irq_num, NULL); devm_free_irq(dev, irq_num, &vsi->q_vectors[vector]); } return err; @@ -2707,17 +2707,72 @@ static void ice_vsi_assign_bpf_prog(struct ice_vsi *vsi, struct bpf_prog *prog) bpf_prog_put(old_prog); } +static struct ice_tx_ring *ice_xdp_ring_from_qid(struct ice_vsi *vsi, int qid) +{ + struct ice_q_vector *q_vector; + struct ice_tx_ring *ring; + + if (static_key_enabled(&ice_xdp_locking_key)) + return vsi->xdp_rings[qid % vsi->num_xdp_txq]; + + q_vector = vsi->rx_rings[qid]->q_vector; + ice_for_each_tx_ring(ring, q_vector->tx) + if (ice_ring_is_xdp(ring)) + return ring; + + return NULL; +} + +/** + * ice_map_xdp_rings - Map XDP rings to interrupt vectors + * @vsi: the VSI with XDP rings being configured + * + * Map XDP rings to interrupt vectors and perform the configuration steps + * dependent on the mapping. + */ +void ice_map_xdp_rings(struct ice_vsi *vsi) +{ + int xdp_rings_rem = vsi->num_xdp_txq; + int v_idx, q_idx; + + /* follow the logic from ice_vsi_map_rings_to_vectors */ + ice_for_each_q_vector(vsi, v_idx) { + struct ice_q_vector *q_vector = vsi->q_vectors[v_idx]; + int xdp_rings_per_v, q_id, q_base; + + xdp_rings_per_v = DIV_ROUND_UP(xdp_rings_rem, + vsi->num_q_vectors - v_idx); + q_base = vsi->num_xdp_txq - xdp_rings_rem; + + for (q_id = q_base; q_id < (q_base + xdp_rings_per_v); q_id++) { + struct ice_tx_ring *xdp_ring = vsi->xdp_rings[q_id]; + + xdp_ring->q_vector = q_vector; + xdp_ring->next = q_vector->tx.tx_ring; + q_vector->tx.tx_ring = xdp_ring; + } + xdp_rings_rem -= xdp_rings_per_v; + } + + ice_for_each_rxq(vsi, q_idx) { + vsi->rx_rings[q_idx]->xdp_ring = ice_xdp_ring_from_qid(vsi, + q_idx); + ice_tx_xsk_pool(vsi, q_idx); + } +} + /** * ice_prepare_xdp_rings - Allocate, configure and setup Tx rings for XDP * @vsi: VSI to bring up Tx rings used by XDP * @prog: bpf program that will be assigned to VSI + * @cfg_type: create from scratch or restore the existing configuration * * Return 0 on success and negative value on error */ -int ice_prepare_xdp_rings(struct ice_vsi *vsi, struct bpf_prog *prog) +int ice_prepare_xdp_rings(struct ice_vsi *vsi, struct bpf_prog *prog, + enum ice_xdp_cfg cfg_type) { u16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 }; - int xdp_rings_rem = vsi->num_xdp_txq; struct ice_pf *pf = vsi->back; struct ice_qs_cfg xdp_qs_cfg = { .qs_mutex = &pf->avail_q_mutex, @@ -2730,8 +2785,7 @@ int ice_prepare_xdp_rings(struct ice_vsi *vsi, struct bpf_prog *prog) .mapping_mode = ICE_VSI_MAP_CONTIG }; struct device *dev; - int i, v_idx; - int status; + int status, i; dev = ice_pf_to_dev(pf); vsi->xdp_rings = devm_kcalloc(dev, vsi->num_xdp_txq, @@ -2750,49 +2804,15 @@ int ice_prepare_xdp_rings(struct ice_vsi *vsi, struct bpf_prog *prog) if (ice_xdp_alloc_setup_rings(vsi)) goto clear_xdp_rings; - /* follow the logic from ice_vsi_map_rings_to_vectors */ - ice_for_each_q_vector(vsi, v_idx) { - struct ice_q_vector *q_vector = vsi->q_vectors[v_idx]; - int xdp_rings_per_v, q_id, q_base; - - xdp_rings_per_v = DIV_ROUND_UP(xdp_rings_rem, - vsi->num_q_vectors - v_idx); - q_base = vsi->num_xdp_txq - xdp_rings_rem; - - for (q_id = q_base; q_id < (q_base + xdp_rings_per_v); q_id++) { - struct ice_tx_ring *xdp_ring = vsi->xdp_rings[q_id]; - - xdp_ring->q_vector = q_vector; - xdp_ring->next = q_vector->tx.tx_ring; - q_vector->tx.tx_ring = xdp_ring; - } - xdp_rings_rem -= xdp_rings_per_v; - } - - ice_for_each_rxq(vsi, i) { - if (static_key_enabled(&ice_xdp_locking_key)) { - vsi->rx_rings[i]->xdp_ring = vsi->xdp_rings[i % vsi->num_xdp_txq]; - } else { - struct ice_q_vector *q_vector = vsi->rx_rings[i]->q_vector; - struct ice_tx_ring *ring; - - ice_for_each_tx_ring(ring, q_vector->tx) { - if (ice_ring_is_xdp(ring)) { - vsi->rx_rings[i]->xdp_ring = ring; - break; - } - } - } - ice_tx_xsk_pool(vsi, i); - } - /* omit the scheduler update if in reset path; XDP queues will be * taken into account at the end of ice_vsi_rebuild, where * ice_cfg_vsi_lan is being called */ - if (ice_is_reset_in_progress(pf->state)) + if (cfg_type == ICE_XDP_CFG_PART) return 0; + ice_map_xdp_rings(vsi); + /* tell the Tx scheduler that right now we have * additional queues */ @@ -2842,22 +2862,21 @@ err_map_xdp: /** * ice_destroy_xdp_rings - undo the configuration made by ice_prepare_xdp_rings * @vsi: VSI to remove XDP rings + * @cfg_type: disable XDP permanently or allow it to be restored later * * Detach XDP rings from irq vectors, clean up the PF bitmap and free * resources */ -int ice_destroy_xdp_rings(struct ice_vsi *vsi) +int ice_destroy_xdp_rings(struct ice_vsi *vsi, enum ice_xdp_cfg cfg_type) { u16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 }; struct ice_pf *pf = vsi->back; int i, v_idx; /* q_vectors are freed in reset path so there's no point in detaching - * rings; in case of rebuild being triggered not from reset bits - * in pf->state won't be set, so additionally check first q_vector - * against NULL + * rings */ - if (ice_is_reset_in_progress(pf->state) || !vsi->q_vectors[0]) + if (cfg_type == ICE_XDP_CFG_PART) goto free_qmap; ice_for_each_q_vector(vsi, v_idx) { @@ -2898,7 +2917,7 @@ free_qmap: if (static_key_enabled(&ice_xdp_locking_key)) static_branch_dec(&ice_xdp_locking_key); - if (ice_is_reset_in_progress(pf->state) || !vsi->q_vectors[0]) + if (cfg_type == ICE_XDP_CFG_PART) return 0; ice_vsi_assign_bpf_prog(vsi, NULL); @@ -3009,7 +3028,8 @@ ice_xdp_setup_prog(struct ice_vsi *vsi, struct bpf_prog *prog, if (xdp_ring_err) { NL_SET_ERR_MSG_MOD(extack, "Not enough Tx resources for XDP"); } else { - xdp_ring_err = ice_prepare_xdp_rings(vsi, prog); + xdp_ring_err = ice_prepare_xdp_rings(vsi, prog, + ICE_XDP_CFG_FULL); if (xdp_ring_err) NL_SET_ERR_MSG_MOD(extack, "Setting up XDP Tx resources failed"); } @@ -3020,7 +3040,7 @@ ice_xdp_setup_prog(struct ice_vsi *vsi, struct bpf_prog *prog, NL_SET_ERR_MSG_MOD(extack, "Setting up XDP Rx resources failed"); } else if (ice_is_xdp_ena_vsi(vsi) && !prog) { xdp_features_clear_redirect_target(vsi->netdev); - xdp_ring_err = ice_destroy_xdp_rings(vsi); + xdp_ring_err = ice_destroy_xdp_rings(vsi, ICE_XDP_CFG_FULL); if (xdp_ring_err) NL_SET_ERR_MSG_MOD(extack, "Freeing XDP Tx resources failed"); /* reallocate Rx queues that were used for zero-copy */ diff --git a/drivers/net/ethernet/intel/ice/ice_nvm.c b/drivers/net/ethernet/intel/ice/ice_nvm.c index 84eab92dc03cfe99f791518f3a21358dd7a91822..59e8879ac0598a8d6e7fac474ba0acbcb9981a02 100644 --- a/drivers/net/ethernet/intel/ice/ice_nvm.c +++ b/drivers/net/ethernet/intel/ice/ice_nvm.c @@ -374,11 +374,25 @@ ice_read_nvm_module(struct ice_hw *hw, enum ice_bank_select bank, u32 offset, u1 * * Read the specified word from the copy of the Shadow RAM found in the * specified NVM module. + * + * Note that the Shadow RAM copy is always located after the CSS header, and + * is aligned to 64-byte (32-word) offsets. */ static int ice_read_nvm_sr_copy(struct ice_hw *hw, enum ice_bank_select bank, u32 offset, u16 *data) { - return ice_read_nvm_module(hw, bank, ICE_NVM_SR_COPY_WORD_OFFSET + offset, data); + u32 sr_copy; + + switch (bank) { + case ICE_ACTIVE_FLASH_BANK: + sr_copy = roundup(hw->flash.banks.active_css_hdr_len, 32); + break; + case ICE_INACTIVE_FLASH_BANK: + sr_copy = roundup(hw->flash.banks.inactive_css_hdr_len, 32); + break; + } + + return ice_read_nvm_module(hw, bank, sr_copy + offset, data); } /** @@ -440,8 +454,7 @@ int ice_get_pfa_module_tlv(struct ice_hw *hw, u16 *module_tlv, u16 *module_tlv_len, u16 module_type) { - u16 pfa_len, pfa_ptr; - u16 next_tlv; + u16 pfa_len, pfa_ptr, next_tlv, max_tlv; int status; status = ice_read_sr_word(hw, ICE_SR_PFA_PTR, &pfa_ptr); @@ -454,11 +467,23 @@ ice_get_pfa_module_tlv(struct ice_hw *hw, u16 *module_tlv, u16 *module_tlv_len, ice_debug(hw, ICE_DBG_INIT, "Failed to read PFA length.\n"); return status; } + + /* The Preserved Fields Area contains a sequence of Type-Length-Value + * structures which define its contents. The PFA length includes all + * of the TLVs, plus the initial length word itself, *and* one final + * word at the end after all of the TLVs. + */ + if (check_add_overflow(pfa_ptr, pfa_len - 1, &max_tlv)) { + dev_warn(ice_hw_to_dev(hw), "PFA starts at offset %u. PFA length of %u caused 16-bit arithmetic overflow.\n", + pfa_ptr, pfa_len); + return -EINVAL; + } + /* Starting with first TLV after PFA length, iterate through the list * of TLVs to find the requested one. */ next_tlv = pfa_ptr + 1; - while (next_tlv < pfa_ptr + pfa_len) { + while (next_tlv < max_tlv) { u16 tlv_sub_module_type; u16 tlv_len; @@ -482,10 +507,13 @@ ice_get_pfa_module_tlv(struct ice_hw *hw, u16 *module_tlv, u16 *module_tlv_len, } return -EINVAL; } - /* Check next TLV, i.e. current TLV pointer + length + 2 words - * (for current TLV's type and length) - */ - next_tlv = next_tlv + tlv_len + 2; + + if (check_add_overflow(next_tlv, 2, &next_tlv) || + check_add_overflow(next_tlv, tlv_len, &next_tlv)) { + dev_warn(ice_hw_to_dev(hw), "TLV of type %u and length 0x%04x caused 16-bit arithmetic overflow. The PFA starts at 0x%04x and has length of 0x%04x\n", + tlv_sub_module_type, tlv_len, pfa_ptr, pfa_len); + return -EINVAL; + } } /* Module does not exist */ return -ENOENT; @@ -1009,6 +1037,72 @@ static int ice_determine_active_flash_banks(struct ice_hw *hw) return 0; } +/** + * ice_get_nvm_css_hdr_len - Read the CSS header length from the NVM CSS header + * @hw: pointer to the HW struct + * @bank: whether to read from the active or inactive flash bank + * @hdr_len: storage for header length in words + * + * Read the CSS header length from the NVM CSS header and add the Authentication + * header size, and then convert to words. + * + * Return: zero on success, or a negative error code on failure. + */ +static int +ice_get_nvm_css_hdr_len(struct ice_hw *hw, enum ice_bank_select bank, + u32 *hdr_len) +{ + u16 hdr_len_l, hdr_len_h; + u32 hdr_len_dword; + int status; + + status = ice_read_nvm_module(hw, bank, ICE_NVM_CSS_HDR_LEN_L, + &hdr_len_l); + if (status) + return status; + + status = ice_read_nvm_module(hw, bank, ICE_NVM_CSS_HDR_LEN_H, + &hdr_len_h); + if (status) + return status; + + /* CSS header length is in DWORD, so convert to words and add + * authentication header size + */ + hdr_len_dword = hdr_len_h << 16 | hdr_len_l; + *hdr_len = (hdr_len_dword * 2) + ICE_NVM_AUTH_HEADER_LEN; + + return 0; +} + +/** + * ice_determine_css_hdr_len - Discover CSS header length for the device + * @hw: pointer to the HW struct + * + * Determine the size of the CSS header at the start of the NVM module. This + * is useful for locating the Shadow RAM copy in the NVM, as the Shadow RAM is + * always located just after the CSS header. + * + * Return: zero on success, or a negative error code on failure. + */ +static int ice_determine_css_hdr_len(struct ice_hw *hw) +{ + struct ice_bank_info *banks = &hw->flash.banks; + int status; + + status = ice_get_nvm_css_hdr_len(hw, ICE_ACTIVE_FLASH_BANK, + &banks->active_css_hdr_len); + if (status) + return status; + + status = ice_get_nvm_css_hdr_len(hw, ICE_INACTIVE_FLASH_BANK, + &banks->inactive_css_hdr_len); + if (status) + return status; + + return 0; +} + /** * ice_init_nvm - initializes NVM setting * @hw: pointer to the HW struct @@ -1055,6 +1149,12 @@ int ice_init_nvm(struct ice_hw *hw) return status; } + status = ice_determine_css_hdr_len(hw); + if (status) { + ice_debug(hw, ICE_DBG_NVM, "Failed to determine Shadow RAM copy offsets.\n"); + return status; + } + status = ice_get_nvm_ver_info(hw, ICE_ACTIVE_FLASH_BANK, &flash->nvm); if (status) { ice_debug(hw, ICE_DBG_INIT, "Failed to read NVM info.\n"); diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.c b/drivers/net/ethernet/intel/ice/ice_ptp.c index 0f17fc1181d28375769a671fc593a1f612e749f4..b7ab6fdf710d24ed1f31f0e9473cc46bcd106356 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp.c +++ b/drivers/net/ethernet/intel/ice/ice_ptp.c @@ -7,8 +7,6 @@ #define E810_OUT_PROP_DELAY_NS 1 -#define UNKNOWN_INCVAL_E82X 0x100000000ULL - static const struct ptp_pin_desc ice_pin_desc_e810t[] = { /* name idx func chan */ { "GNSS", GNSS, PTP_PF_EXTTS, 0, { 0, } }, @@ -813,7 +811,7 @@ static enum ice_tx_tstamp_work ice_ptp_tx_tstamp_owner(struct ice_pf *pf) } mutex_unlock(&pf->ptp.ports_owner.lock); - for (i = 0; i < ICE_MAX_QUAD; i++) { + for (i = 0; i < ICE_GET_QUAD_NUM(pf->hw.ptp.num_lports); i++) { u64 tstamp_ready; int err; @@ -1013,6 +1011,28 @@ ice_ptp_release_tx_tracker(struct ice_pf *pf, struct ice_ptp_tx *tx) tx->len = 0; } +/** + * ice_ptp_init_tx_eth56g - Initialize tracking for Tx timestamps + * @pf: Board private structure + * @tx: the Tx tracking structure to initialize + * @port: the port this structure tracks + * + * Initialize the Tx timestamp tracker for this port. ETH56G PHYs + * have independent memory blocks for all ports. + * + * Return: 0 for success, -ENOMEM when failed to allocate Tx tracker + */ +static int ice_ptp_init_tx_eth56g(struct ice_pf *pf, struct ice_ptp_tx *tx, + u8 port) +{ + tx->block = port; + tx->offset = 0; + tx->len = INDEX_PER_PORT_ETH56G; + tx->has_ready_bitmap = 1; + + return ice_ptp_alloc_tx_tracker(tx); +} + /** * ice_ptp_init_tx_e82x - Initialize tracking for Tx timestamps * @pf: Board private structure @@ -1027,7 +1047,7 @@ ice_ptp_release_tx_tracker(struct ice_pf *pf, struct ice_ptp_tx *tx) static int ice_ptp_init_tx_e82x(struct ice_pf *pf, struct ice_ptp_tx *tx, u8 port) { - tx->block = port / ICE_PORTS_PER_QUAD; + tx->block = ICE_GET_QUAD_NUM(port); tx->offset = (port % ICE_PORTS_PER_QUAD) * INDEX_PER_PORT_E82X; tx->len = INDEX_PER_PORT_E82X; tx->has_ready_bitmap = 1; @@ -1210,12 +1230,7 @@ static u64 ice_base_incval(struct ice_pf *pf) struct ice_hw *hw = &pf->hw; u64 incval; - if (ice_is_e810(hw)) - incval = ICE_PTP_NOMINAL_INCVAL_E810; - else if (ice_e82x_time_ref(hw) < NUM_ICE_TIME_REF_FREQ) - incval = ice_e82x_nominal_incval(ice_e82x_time_ref(hw)); - else - incval = UNKNOWN_INCVAL_E82X; + incval = ice_get_base_incval(hw); dev_dbg(ice_pf_to_dev(pf), "PTP: using base increment value of 0x%016llx\n", incval); @@ -1229,8 +1244,8 @@ static u64 ice_base_incval(struct ice_pf *pf) */ static int ice_ptp_check_tx_fifo(struct ice_ptp_port *port) { - int quad = port->port_num / ICE_PORTS_PER_QUAD; int offs = port->port_num % ICE_PORTS_PER_QUAD; + int quad = ICE_GET_QUAD_NUM(port->port_num); struct ice_pf *pf; struct ice_hw *hw; u32 val, phy_sts; @@ -1348,10 +1363,19 @@ ice_ptp_port_phy_stop(struct ice_ptp_port *ptp_port) mutex_lock(&ptp_port->ps_lock); - kthread_cancel_delayed_work_sync(&ptp_port->ov_work); + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + err = ice_stop_phy_timer_eth56g(hw, port, true); + break; + case ICE_PHY_E82X: + kthread_cancel_delayed_work_sync(&ptp_port->ov_work); - err = ice_stop_phy_timer_e82x(hw, port, true); - if (err) + err = ice_stop_phy_timer_e82x(hw, port, true); + break; + default: + err = -ENODEV; + } + if (err && err != -EBUSY) dev_err(ice_pf_to_dev(pf), "PTP failed to set PHY port %d down, err %d\n", port, err); @@ -1385,27 +1409,39 @@ ice_ptp_port_phy_restart(struct ice_ptp_port *ptp_port) mutex_lock(&ptp_port->ps_lock); - kthread_cancel_delayed_work_sync(&ptp_port->ov_work); + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + err = ice_start_phy_timer_eth56g(hw, port); + break; + case ICE_PHY_E82X: + /* Start the PHY timer in Vernier mode */ + kthread_cancel_delayed_work_sync(&ptp_port->ov_work); - /* temporarily disable Tx timestamps while calibrating PHY offset */ - spin_lock_irqsave(&ptp_port->tx.lock, flags); - ptp_port->tx.calibrating = true; - spin_unlock_irqrestore(&ptp_port->tx.lock, flags); - ptp_port->tx_fifo_busy_cnt = 0; + /* temporarily disable Tx timestamps while calibrating + * PHY offset + */ + spin_lock_irqsave(&ptp_port->tx.lock, flags); + ptp_port->tx.calibrating = true; + spin_unlock_irqrestore(&ptp_port->tx.lock, flags); + ptp_port->tx_fifo_busy_cnt = 0; - /* Start the PHY timer in Vernier mode */ - err = ice_start_phy_timer_e82x(hw, port); - if (err) - goto out_unlock; + /* Start the PHY timer in Vernier mode */ + err = ice_start_phy_timer_e82x(hw, port); + if (err) + break; - /* Enable Tx timestamps right away */ - spin_lock_irqsave(&ptp_port->tx.lock, flags); - ptp_port->tx.calibrating = false; - spin_unlock_irqrestore(&ptp_port->tx.lock, flags); + /* Enable Tx timestamps right away */ + spin_lock_irqsave(&ptp_port->tx.lock, flags); + ptp_port->tx.calibrating = false; + spin_unlock_irqrestore(&ptp_port->tx.lock, flags); - kthread_queue_delayed_work(pf->ptp.kworker, &ptp_port->ov_work, 0); + kthread_queue_delayed_work(pf->ptp.kworker, &ptp_port->ov_work, + 0); + break; + default: + err = -ENODEV; + } -out_unlock: if (err) dev_err(ice_pf_to_dev(pf), "PTP failed to set PHY port %d up, err %d\n", port, err); @@ -1429,20 +1465,23 @@ void ice_ptp_link_change(struct ice_pf *pf, u8 port, bool linkup) if (pf->ptp.state != ICE_PTP_READY) return; - if (WARN_ON_ONCE(port >= ICE_NUM_EXTERNAL_PORTS)) + if (WARN_ON_ONCE(port >= hw->ptp.num_lports)) return; ptp_port = &pf->ptp.port; + if (ice_is_e825c(hw) && hw->ptp.is_2x50g_muxed_topo) + port *= 2; if (WARN_ON_ONCE(ptp_port->port_num != port)) return; /* Update cached link status for this port immediately */ ptp_port->link_up = linkup; - switch (hw->phy_model) { + switch (hw->ptp.phy_model) { case ICE_PHY_E810: /* Do not reconfigure E810 PHY */ return; + case ICE_PHY_ETH56G: case ICE_PHY_E82X: ice_ptp_port_phy_restart(ptp_port); return; @@ -1457,42 +1496,62 @@ void ice_ptp_link_change(struct ice_pf *pf, u8 port, bool linkup) * @ena: bool value to enable or disable interrupt * @threshold: Minimum number of packets at which intr is triggered * - * Utility function to enable or disable Tx timestamp interrupt and threshold + * Utility function to configure all the PHY interrupt settings, including + * whether the PHY interrupt is enabled, and what threshold to use. Also + * configures The E82X timestamp owner to react to interrupts from all PHYs. + * + * Return: 0 on success, -EOPNOTSUPP when PHY model incorrect, other error codes + * when failed to configure PHY interrupt for E82X */ static int ice_ptp_cfg_phy_interrupt(struct ice_pf *pf, bool ena, u32 threshold) { + struct device *dev = ice_pf_to_dev(pf); struct ice_hw *hw = &pf->hw; - int err = 0; - int quad; - u32 val; ice_ptp_reset_ts_memory(hw); - for (quad = 0; quad < ICE_MAX_QUAD; quad++) { - err = ice_read_quad_reg_e82x(hw, quad, Q_REG_TX_MEM_GBL_CFG, - &val); - if (err) - break; + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: { + int port; - if (ena) { - val |= Q_REG_TX_MEM_GBL_CFG_INTR_ENA_M; - val &= ~Q_REG_TX_MEM_GBL_CFG_INTR_THR_M; - val |= FIELD_PREP(Q_REG_TX_MEM_GBL_CFG_INTR_THR_M, - threshold); - } else { - val &= ~Q_REG_TX_MEM_GBL_CFG_INTR_ENA_M; + for (port = 0; port < hw->ptp.num_lports; port++) { + int err; + + err = ice_phy_cfg_intr_eth56g(hw, port, ena, threshold); + if (err) { + dev_err(dev, "Failed to configure PHY interrupt for port %d, err %d\n", + port, err); + return err; + } } - err = ice_write_quad_reg_e82x(hw, quad, Q_REG_TX_MEM_GBL_CFG, - val); - if (err) - break; + return 0; } + case ICE_PHY_E82X: { + int quad; - if (err) - dev_err(ice_pf_to_dev(pf), "PTP failed in intr ena, err %d\n", - err); - return err; + for (quad = 0; quad < ICE_GET_QUAD_NUM(hw->ptp.num_lports); + quad++) { + int err; + + err = ice_phy_cfg_intr_e82x(hw, quad, ena, threshold); + if (err) { + dev_err(dev, "Failed to configure PHY interrupt for quad %d, err %d\n", + quad, err); + return err; + } + } + + return 0; + } + case ICE_PHY_E810: + return 0; + case ICE_PHY_UNSUP: + default: + dev_warn(dev, "%s: Unexpected PHY model %d\n", __func__, + hw->ptp.phy_model); + return -EOPNOTSUPP; + } } /** @@ -1714,8 +1773,7 @@ static int ice_ptp_cfg_clkout(struct ice_pf *pf, unsigned int chan, * maintaining phase */ if (start_time < current_time) - start_time = div64_u64(current_time + NSEC_PER_SEC - 1, - NSEC_PER_SEC) * NSEC_PER_SEC + phase; + start_time = roundup_u64(current_time, NSEC_PER_SEC) + phase; if (ice_is_e810(hw)) start_time -= E810_OUT_PROP_DELAY_NS; @@ -1929,11 +1987,14 @@ ice_ptp_settime64(struct ptp_clock_info *info, const struct timespec64 *ts) struct ice_hw *hw = &pf->hw; int err; - /* For Vernier mode, we need to recalibrate after new settime - * Start with disabling timestamp block + /* For Vernier mode on E82X, we need to recalibrate after new settime. + * Start with marking timestamps as invalid. */ - if (pf->ptp.port.link_up) - ice_ptp_port_phy_stop(&pf->ptp.port); + if (hw->ptp.phy_model == ICE_PHY_E82X) { + err = ice_ptp_clear_phy_offset_ready_e82x(hw); + if (err) + dev_warn(ice_pf_to_dev(pf), "Failed to mark timestamps as invalid before settime\n"); + } if (!ice_ptp_lock(hw)) { err = -EBUSY; @@ -1953,7 +2014,7 @@ ice_ptp_settime64(struct ptp_clock_info *info, const struct timespec64 *ts) ice_ptp_enable_all_clkout(pf); /* Recalibrate and re-enable timestamp blocks for E822/E823 */ - if (hw->phy_model == ICE_PHY_E82X) + if (hw->ptp.phy_model == ICE_PHY_E82X) ice_ptp_restart_all_phy(pf); exit: if (err) { @@ -2578,7 +2639,7 @@ static void ice_ptp_maybe_trigger_tx_interrupt(struct ice_pf *pf) if (!ice_pf_src_tmr_owned(pf)) return; - for (i = 0; i < ICE_MAX_QUAD; i++) { + for (i = 0; i < ICE_GET_QUAD_NUM(hw->ptp.num_lports); i++) { u64 tstamp_ready; int err; @@ -3010,12 +3071,10 @@ static int ice_ptp_init_owner(struct ice_pf *pf) /* Release the global hardware lock */ ice_ptp_unlock(hw); - if (!ice_is_e810(hw)) { - /* Enable quad interrupts */ - err = ice_ptp_cfg_phy_interrupt(pf, true, 1); - if (err) - goto err_exit; - } + /* Configure PHY interrupt settings */ + err = ice_ptp_cfg_phy_interrupt(pf, true, 1); + if (err) + goto err_exit; /* Ensure we have a clock device */ err = ice_ptp_create_clock(pf); @@ -3076,7 +3135,10 @@ static int ice_ptp_init_port(struct ice_pf *pf, struct ice_ptp_port *ptp_port) mutex_init(&ptp_port->ps_lock); - switch (hw->phy_model) { + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + return ice_ptp_init_tx_eth56g(pf, &ptp_port->tx, + ptp_port->port_num); case ICE_PHY_E810: return ice_ptp_init_tx_e810(pf, &ptp_port->tx); case ICE_PHY_E82X: @@ -3171,7 +3233,7 @@ static void ice_ptp_remove_auxbus_device(struct ice_pf *pf) */ static void ice_ptp_init_tx_interrupt_mode(struct ice_pf *pf) { - switch (pf->hw.phy_model) { + switch (pf->hw.ptp.phy_model) { case ICE_PHY_E82X: /* E822 based PHY has the clock owner process the interrupt * for all ports. @@ -3207,7 +3269,7 @@ void ice_ptp_init(struct ice_pf *pf) ptp->state = ICE_PTP_INITIALIZING; - ice_ptp_init_phy_model(hw); + ice_ptp_init_hw(hw); ice_ptp_init_tx_interrupt_mode(pf); @@ -3221,6 +3283,9 @@ void ice_ptp_init(struct ice_pf *pf) } ptp->port.port_num = hw->pf_id; + if (ice_is_e825c(hw) && hw->ptp.is_2x50g_muxed_topo) + ptp->port.port_num = hw->pf_id * 2; + err = ice_ptp_init_port(pf, &ptp->port); if (err) goto err; diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.h b/drivers/net/ethernet/intel/ice/ice_ptp.h index 3af20025043a637b5dedf4c075b50ac86e2a981a..e0c23aaedc12520509c8c29f986519ad94ce1189 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp.h +++ b/drivers/net/ethernet/intel/ice/ice_ptp.h @@ -153,6 +153,7 @@ struct ice_ptp_tx { #define INDEX_PER_QUAD 64 #define INDEX_PER_PORT_E82X 16 #define INDEX_PER_PORT_E810 64 +#define INDEX_PER_PORT_ETH56G 64 /** * struct ice_ptp_port - data used to initialize an external port for PTP diff --git a/drivers/net/ethernet/intel/ice/ice_ptp_consts.h b/drivers/net/ethernet/intel/ice/ice_ptp_consts.h index 2c4dab0c48ab594e14c1e7b2e5e306c1e5d5550c..e6980b94a6c1d6c4e4343e86268fc177d0ef0c44 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp_consts.h +++ b/drivers/net/ethernet/intel/ice/ice_ptp_consts.h @@ -9,6 +9,321 @@ */ /* Constants defined for the PTP 1588 clock hardware. */ +const struct ice_phy_reg_info_eth56g eth56g_phy_res[NUM_ETH56G_PHY_RES] = { + /* ETH56G_PHY_REG_PTP */ + { + /* base_addr */ + { + 0x092000, + 0x126000, + 0x1BA000, + 0x24E000, + 0x2E2000, + }, + /* step */ + 0x98, + }, + /* ETH56G_PHY_MEM_PTP */ + { + /* base_addr */ + { + 0x093000, + 0x127000, + 0x1BB000, + 0x24F000, + 0x2E3000, + }, + /* step */ + 0x200, + }, + /* ETH56G_PHY_REG_XPCS */ + { + /* base_addr */ + { + 0x000000, + 0x009400, + 0x128000, + 0x1BC000, + 0x250000, + }, + /* step */ + 0x21000, + }, + /* ETH56G_PHY_REG_MAC */ + { + /* base_addr */ + { + 0x085000, + 0x119000, + 0x1AD000, + 0x241000, + 0x2D5000, + }, + /* step */ + 0x1000, + }, + /* ETH56G_PHY_REG_GPCS */ + { + /* base_addr */ + { + 0x084000, + 0x118000, + 0x1AC000, + 0x240000, + 0x2D4000, + }, + /* step */ + 0x400, + }, +}; + +const +struct ice_eth56g_mac_reg_cfg eth56g_mac_cfg[NUM_ICE_ETH56G_LNK_SPD] = { + [ICE_ETH56G_LNK_SPD_1G] = { + .tx_mode = { .def = 6, }, + .rx_mode = { .def = 6, }, + .blks_per_clk = 1, + .blktime = 0x4000, /* 32 */ + .tx_offset = { + .serdes = 0x6666, /* 51.2 */ + .no_fec = 0xd066, /* 104.2 */ + .sfd = 0x3000, /* 24 */ + .onestep = 0x30000 /* 384 */ + }, + .rx_offset = { + .serdes = 0xffffc59a, /* -29.2 */ + .no_fec = 0xffff0a80, /* -122.75 */ + .sfd = 0x2c00, /* 22 */ + .bs_ds = 0x19a /* 0.8 */ + /* Dynamic bitslip 0 equals to 10 */ + } + }, + [ICE_ETH56G_LNK_SPD_2_5G] = { + .tx_mode = { .def = 6, }, + .rx_mode = { .def = 6, }, + .blks_per_clk = 1, + .blktime = 0x199a, /* 12.8 */ + .tx_offset = { + .serdes = 0x28f6, /* 20.48 */ + .no_fec = 0x53b8, /* 41.86 */ + .sfd = 0x1333, /* 9.6 */ + .onestep = 0x13333 /* 153.6 */ + }, + .rx_offset = { + .serdes = 0xffffe8a4, /* -11.68 */ + .no_fec = 0xffff9a76, /* -50.77 */ + .sfd = 0xf33, /* 7.6 */ + .bs_ds = 0xa4 /* 0.32 */ + } + }, + [ICE_ETH56G_LNK_SPD_10G] = { + .tx_mode = { .def = 1, }, + .rx_mode = { .def = 1, }, + .blks_per_clk = 1, + .blktime = 0x666, /* 3.2 */ + .tx_offset = { + .serdes = 0x234c, /* 17.6484848 */ + .no_fec = 0x8e80, /* 71.25 */ + .fc = 0xb4a4, /* 90.32 */ + .sfd = 0x4a4, /* 2.32 */ + .onestep = 0x4ccd /* 38.4 */ + }, + .rx_offset = { + .serdes = 0xffffeb27, /* -10.42424 */ + .no_fec = 0xffffcccd, /* -25.6 */ + .fc = 0xfffe0014, /* -255.96 */ + .sfd = 0x4a4, /* 2.32 */ + .bs_ds = 0x32 /* 0.0969697 */ + } + }, + [ICE_ETH56G_LNK_SPD_25G] = { + .tx_mode = { + .def = 1, + .rs = 4 + }, + .tx_mk_dly = 4, + .tx_cw_dly = { + .def = 1, + .onestep = 6 + }, + .rx_mode = { + .def = 1, + .rs = 4 + }, + .rx_mk_dly = { + .def = 1, + .rs = 1 + }, + .rx_cw_dly = { + .def = 1, + .rs = 1 + }, + .blks_per_clk = 1, + .blktime = 0x28f, /* 1.28 */ + .mktime = 0x147b, /* 10.24, only if RS-FEC enabled */ + .tx_offset = { + .serdes = 0xe1e, /* 7.0593939 */ + .no_fec = 0x3857, /* 28.17 */ + .fc = 0x48c3, /* 36.38 */ + .rs = 0x8100, /* 64.5 */ + .sfd = 0x1dc, /* 0.93 */ + .onestep = 0x1eb8 /* 15.36 */ + }, + .rx_offset = { + .serdes = 0xfffff7a9, /* -4.1697 */ + .no_fec = 0xffffe71a, /* -12.45 */ + .fc = 0xfffe894d, /* -187.35 */ + .rs = 0xfffff8cd, /* -3.6 */ + .sfd = 0x1dc, /* 0.93 */ + .bs_ds = 0x14 /* 0.0387879, RS-FEC 0 */ + } + }, + [ICE_ETH56G_LNK_SPD_40G] = { + .tx_mode = { .def = 3 }, + .tx_mk_dly = 4, + .tx_cw_dly = { + .def = 1, + .onestep = 6 + }, + .rx_mode = { .def = 4 }, + .rx_mk_dly = { .def = 1 }, + .rx_cw_dly = { .def = 1 }, + .blktime = 0x333, /* 1.6 */ + .mktime = 0xccd, /* 6.4 */ + .tx_offset = { + .serdes = 0x234c, /* 17.6484848 */ + .no_fec = 0x5a8a, /* 45.27 */ + .fc = 0x81b8, /* 64.86 */ + .sfd = 0x4a4, /* 2.32 */ + .onestep = 0x1333 /* 9.6 */ + }, + .rx_offset = { + .serdes = 0xffffeb27, /* -10.42424 */ + .no_fec = 0xfffff594, /* -5.21 */ + .fc = 0xfffe3080, /* -231.75 */ + .sfd = 0x4a4, /* 2.32 */ + .bs_ds = 0xccd /* 6.4 */ + } + }, + [ICE_ETH56G_LNK_SPD_50G] = { + .tx_mode = { .def = 5 }, + .tx_mk_dly = 4, + .tx_cw_dly = { + .def = 1, + .onestep = 6 + }, + .rx_mode = { .def = 5 }, + .rx_mk_dly = { .def = 1 }, + .rx_cw_dly = { .def = 1 }, + .blktime = 0x28f, /* 1.28 */ + .mktime = 0xa3d, /* 5.12 */ + .tx_offset = { + .serdes = 0x13ba, /* 9.86353 */ + .rs = 0x5400, /* 42 */ + .sfd = 0xe6, /* 0.45 */ + .onestep = 0xf5c /* 7.68 */ + }, + .rx_offset = { + .serdes = 0xfffff7e8, /* -4.04706 */ + .rs = 0xfffff994, /* -3.21 */ + .sfd = 0xe6 /* 0.45 */ + } + }, + [ICE_ETH56G_LNK_SPD_50G2] = { + .tx_mode = { + .def = 3, + .rs = 2 + }, + .tx_mk_dly = 4, + .tx_cw_dly = { + .def = 1, + .onestep = 6 + }, + .rx_mode = { + .def = 4, + .rs = 1 + }, + .rx_mk_dly = { .def = 1 }, + .rx_cw_dly = { .def = 1 }, + .blktime = 0x28f, /* 1.28 */ + .mktime = 0xa3d, /* 5.12 */ + .tx_offset = { + .serdes = 0xe1e, /* 7.0593939 */ + .no_fec = 0x3d33, /* 30.6 */ + .rs = 0x5057, /* 40.17 */ + .sfd = 0x1dc, /* 0.93 */ + .onestep = 0xf5c /* 7.68 */ + }, + .rx_offset = { + .serdes = 0xfffff7a9, /* -4.1697 */ + .no_fec = 0xfffff8cd, /* -3.6 */ + .rs = 0xfffff21a, /* -6.95 */ + .sfd = 0x1dc, /* 0.93 */ + .bs_ds = 0xa3d /* 5.12, RS-FEC 0x633 (3.1) */ + } + }, + [ICE_ETH56G_LNK_SPD_100G] = { + .tx_mode = { + .def = 3, + .rs = 2 + }, + .tx_mk_dly = 10, + .tx_cw_dly = { + .def = 3, + .onestep = 6 + }, + .rx_mode = { + .def = 4, + .rs = 1 + }, + .rx_mk_dly = { .def = 5 }, + .rx_cw_dly = { .def = 5 }, + .blks_per_clk = 1, + .blktime = 0x148, /* 0.64 */ + .mktime = 0x199a, /* 12.8 */ + .tx_offset = { + .serdes = 0xe1e, /* 7.0593939 */ + .no_fec = 0x67ec, /* 51.96 */ + .rs = 0x44fb, /* 34.49 */ + .sfd = 0x1dc, /* 0.93 */ + .onestep = 0xf5c /* 7.68 */ + }, + .rx_offset = { + .serdes = 0xfffff7a9, /* -4.1697 */ + .no_fec = 0xfffff5a9, /* -5.17 */ + .rs = 0xfffff6e6, /* -4.55 */ + .sfd = 0x1dc, /* 0.93 */ + .bs_ds = 0x199a /* 12.8, RS-FEC 0x31b (1.552) */ + } + }, + [ICE_ETH56G_LNK_SPD_100G2] = { + .tx_mode = { .def = 5 }, + .tx_mk_dly = 10, + .tx_cw_dly = { + .def = 3, + .onestep = 6 + }, + .rx_mode = { .def = 5 }, + .rx_mk_dly = { .def = 5 }, + .rx_cw_dly = { .def = 5 }, + .blks_per_clk = 1, + .blktime = 0x148, /* 0.64 */ + .mktime = 0x199a, /* 12.8 */ + .tx_offset = { + .serdes = 0x13ba, /* 9.86353 */ + .rs = 0x460a, /* 35.02 */ + .sfd = 0xe6, /* 0.45 */ + .onestep = 0xf5c /* 7.68 */ + }, + .rx_offset = { + .serdes = 0xfffff7e8, /* -4.04706 */ + .rs = 0xfffff548, /* -5.36 */ + .sfd = 0xe6, /* 0.45 */ + .bs_ds = 0x303 /* 1.506 */ + } + } +}; + /* struct ice_time_ref_info_e82x * * E822 hardware can use different sources as the reference for the PTP @@ -155,6 +470,93 @@ const struct ice_cgu_pll_params_e82x e822_cgu_params[NUM_ICE_TIME_REF_FREQ] = { }, }; +const +struct ice_cgu_pll_params_e825c e825c_cgu_params[NUM_ICE_TIME_REF_FREQ] = { + /* ICE_TIME_REF_FREQ_25_000 -> 25 MHz */ + { + /* tspll_ck_refclkfreq */ + 0x19, + /* tspll_ndivratio */ + 1, + /* tspll_fbdiv_intgr */ + 320, + /* tspll_fbdiv_frac */ + 0, + /* ref1588_ck_div */ + 0, + }, + + /* ICE_TIME_REF_FREQ_122_880 -> 122.88 MHz */ + { + /* tspll_ck_refclkfreq */ + 0x29, + /* tspll_ndivratio */ + 3, + /* tspll_fbdiv_intgr */ + 195, + /* tspll_fbdiv_frac */ + 1342177280UL, + /* ref1588_ck_div */ + 0, + }, + + /* ICE_TIME_REF_FREQ_125_000 -> 125 MHz */ + { + /* tspll_ck_refclkfreq */ + 0x3E, + /* tspll_ndivratio */ + 2, + /* tspll_fbdiv_intgr */ + 128, + /* tspll_fbdiv_frac */ + 0, + /* ref1588_ck_div */ + 0, + }, + + /* ICE_TIME_REF_FREQ_153_600 -> 153.6 MHz */ + { + /* tspll_ck_refclkfreq */ + 0x33, + /* tspll_ndivratio */ + 3, + /* tspll_fbdiv_intgr */ + 156, + /* tspll_fbdiv_frac */ + 1073741824UL, + /* ref1588_ck_div */ + 0, + }, + + /* ICE_TIME_REF_FREQ_156_250 -> 156.25 MHz */ + { + /* tspll_ck_refclkfreq */ + 0x1F, + /* tspll_ndivratio */ + 5, + /* tspll_fbdiv_intgr */ + 256, + /* tspll_fbdiv_frac */ + 0, + /* ref1588_ck_div */ + 0, + }, + + /* ICE_TIME_REF_FREQ_245_760 -> 245.76 MHz */ + { + /* tspll_ck_refclkfreq */ + 0x52, + /* tspll_ndivratio */ + 3, + /* tspll_fbdiv_intgr */ + 97, + /* tspll_fbdiv_frac */ + 2818572288UL, + /* ref1588_ck_div */ + 0, + }, +}; + /* struct ice_vernier_info_e82x * * E822 hardware calibrates the delay of the timestamp indication from the diff --git a/drivers/net/ethernet/intel/ice/ice_ptp_hw.c b/drivers/net/ethernet/intel/ice/ice_ptp_hw.c index 2b9423a173bba6a6d31d239d632e84437fb14b02..1e9a4ccd0ea2783f0c96c5c5cfd120bd89296982 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp_hw.c +++ b/drivers/net/ethernet/intel/ice/ice_ptp_hw.c @@ -2,6 +2,7 @@ /* Copyright (C) 2021, Intel Corporation. */ #include +#include #include "ice_common.h" #include "ice_ptp_hw.h" #include "ice_ptp_consts.h" @@ -226,59 +227,2477 @@ static u64 ice_ptp_read_src_incval(struct ice_hw *hw) return ((u64)(hi & INCVAL_HIGH_M) << 32) | lo; } +/** + * ice_read_cgu_reg_e82x - Read a CGU register + * @hw: pointer to the HW struct + * @addr: Register address to read + * @val: storage for register value read + * + * Read the contents of a register of the Clock Generation Unit. Only + * applicable to E822 devices. + * + * Return: 0 on success, other error codes when failed to read from CGU + */ +static int ice_read_cgu_reg_e82x(struct ice_hw *hw, u32 addr, u32 *val) +{ + struct ice_sbq_msg_input cgu_msg = { + .opcode = ice_sbq_msg_rd, + .dest_dev = cgu, + .msg_addr_low = addr + }; + int err; + + err = ice_sbq_rw_reg(hw, &cgu_msg); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to read CGU register 0x%04x, err %d\n", + addr, err); + return err; + } + + *val = cgu_msg.data; + + return 0; +} + +/** + * ice_write_cgu_reg_e82x - Write a CGU register + * @hw: pointer to the HW struct + * @addr: Register address to write + * @val: value to write into the register + * + * Write the specified value to a register of the Clock Generation Unit. Only + * applicable to E822 devices. + * + * Return: 0 on success, other error codes when failed to write to CGU + */ +static int ice_write_cgu_reg_e82x(struct ice_hw *hw, u32 addr, u32 val) +{ + struct ice_sbq_msg_input cgu_msg = { + .opcode = ice_sbq_msg_wr, + .dest_dev = cgu, + .msg_addr_low = addr, + .data = val + }; + int err; + + err = ice_sbq_rw_reg(hw, &cgu_msg); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write CGU register 0x%04x, err %d\n", + addr, err); + return err; + } + + return err; +} + +/** + * ice_clk_freq_str - Convert time_ref_freq to string + * @clk_freq: Clock frequency + * + * Return: specified TIME_REF clock frequency converted to a string + */ +static const char *ice_clk_freq_str(enum ice_time_ref_freq clk_freq) +{ + switch (clk_freq) { + case ICE_TIME_REF_FREQ_25_000: + return "25 MHz"; + case ICE_TIME_REF_FREQ_122_880: + return "122.88 MHz"; + case ICE_TIME_REF_FREQ_125_000: + return "125 MHz"; + case ICE_TIME_REF_FREQ_153_600: + return "153.6 MHz"; + case ICE_TIME_REF_FREQ_156_250: + return "156.25 MHz"; + case ICE_TIME_REF_FREQ_245_760: + return "245.76 MHz"; + default: + return "Unknown"; + } +} + +/** + * ice_clk_src_str - Convert time_ref_src to string + * @clk_src: Clock source + * + * Return: specified clock source converted to its string name + */ +static const char *ice_clk_src_str(enum ice_clk_src clk_src) +{ + switch (clk_src) { + case ICE_CLK_SRC_TCXO: + return "TCXO"; + case ICE_CLK_SRC_TIME_REF: + return "TIME_REF"; + default: + return "Unknown"; + } +} + +/** + * ice_cfg_cgu_pll_e82x - Configure the Clock Generation Unit + * @hw: pointer to the HW struct + * @clk_freq: Clock frequency to program + * @clk_src: Clock source to select (TIME_REF, or TCXO) + * + * Configure the Clock Generation Unit with the desired clock frequency and + * time reference, enabling the PLL which drives the PTP hardware clock. + * + * Return: + * * %0 - success + * * %-EINVAL - input parameters are incorrect + * * %-EBUSY - failed to lock TS PLL + * * %other - CGU read/write failure + */ +static int ice_cfg_cgu_pll_e82x(struct ice_hw *hw, + enum ice_time_ref_freq clk_freq, + enum ice_clk_src clk_src) +{ + union tspll_ro_bwm_lf bwm_lf; + union nac_cgu_dword19 dw19; + union nac_cgu_dword22 dw22; + union nac_cgu_dword24 dw24; + union nac_cgu_dword9 dw9; + int err; + + if (clk_freq >= NUM_ICE_TIME_REF_FREQ) { + dev_warn(ice_hw_to_dev(hw), "Invalid TIME_REF frequency %u\n", + clk_freq); + return -EINVAL; + } + + if (clk_src >= NUM_ICE_CLK_SRC) { + dev_warn(ice_hw_to_dev(hw), "Invalid clock source %u\n", + clk_src); + return -EINVAL; + } + + if (clk_src == ICE_CLK_SRC_TCXO && + clk_freq != ICE_TIME_REF_FREQ_25_000) { + dev_warn(ice_hw_to_dev(hw), + "TCXO only supports 25 MHz frequency\n"); + return -EINVAL; + } + + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD9, &dw9.val); + if (err) + return err; + + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD24, &dw24.val); + if (err) + return err; + + err = ice_read_cgu_reg_e82x(hw, TSPLL_RO_BWM_LF, &bwm_lf.val); + if (err) + return err; + + /* Log the current clock configuration */ + ice_debug(hw, ICE_DBG_PTP, "Current CGU configuration -- %s, clk_src %s, clk_freq %s, PLL %s\n", + dw24.ts_pll_enable ? "enabled" : "disabled", + ice_clk_src_str(dw24.time_ref_sel), + ice_clk_freq_str(dw9.time_ref_freq_sel), + bwm_lf.plllock_true_lock_cri ? "locked" : "unlocked"); + + /* Disable the PLL before changing the clock source or frequency */ + if (dw24.ts_pll_enable) { + dw24.ts_pll_enable = 0; + + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD24, dw24.val); + if (err) + return err; + } + + /* Set the frequency */ + dw9.time_ref_freq_sel = clk_freq; + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD9, dw9.val); + if (err) + return err; + + /* Configure the TS PLL feedback divisor */ + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD19, &dw19.val); + if (err) + return err; + + dw19.tspll_fbdiv_intgr = e822_cgu_params[clk_freq].feedback_div; + dw19.tspll_ndivratio = 1; + + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD19, dw19.val); + if (err) + return err; + + /* Configure the TS PLL post divisor */ + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD22, &dw22.val); + if (err) + return err; + + dw22.time1588clk_div = e822_cgu_params[clk_freq].post_pll_div; + dw22.time1588clk_sel_div2 = 0; + + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD22, dw22.val); + if (err) + return err; + + /* Configure the TS PLL pre divisor and clock source */ + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD24, &dw24.val); + if (err) + return err; + + dw24.ref1588_ck_div = e822_cgu_params[clk_freq].refclk_pre_div; + dw24.tspll_fbdiv_frac = e822_cgu_params[clk_freq].frac_n_div; + dw24.time_ref_sel = clk_src; + + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD24, dw24.val); + if (err) + return err; + + /* Finally, enable the PLL */ + dw24.ts_pll_enable = 1; + + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD24, dw24.val); + if (err) + return err; + + /* Wait to verify if the PLL locks */ + usleep_range(1000, 5000); + + err = ice_read_cgu_reg_e82x(hw, TSPLL_RO_BWM_LF, &bwm_lf.val); + if (err) + return err; + + if (!bwm_lf.plllock_true_lock_cri) { + dev_warn(ice_hw_to_dev(hw), "CGU PLL failed to lock\n"); + return -EBUSY; + } + + /* Log the current clock configuration */ + ice_debug(hw, ICE_DBG_PTP, "New CGU configuration -- %s, clk_src %s, clk_freq %s, PLL %s\n", + dw24.ts_pll_enable ? "enabled" : "disabled", + ice_clk_src_str(dw24.time_ref_sel), + ice_clk_freq_str(dw9.time_ref_freq_sel), + bwm_lf.plllock_true_lock_cri ? "locked" : "unlocked"); + + return 0; +} + +/** + * ice_cfg_cgu_pll_e825c - Configure the Clock Generation Unit for E825-C + * @hw: pointer to the HW struct + * @clk_freq: Clock frequency to program + * @clk_src: Clock source to select (TIME_REF, or TCXO) + * + * Configure the Clock Generation Unit with the desired clock frequency and + * time reference, enabling the PLL which drives the PTP hardware clock. + * + * Return: + * * %0 - success + * * %-EINVAL - input parameters are incorrect + * * %-EBUSY - failed to lock TS PLL + * * %other - CGU read/write failure + */ +static int ice_cfg_cgu_pll_e825c(struct ice_hw *hw, + enum ice_time_ref_freq clk_freq, + enum ice_clk_src clk_src) +{ + union tspll_ro_lock_e825c ro_lock; + union nac_cgu_dword16_e825c dw16; + union nac_cgu_dword23_e825c dw23; + union nac_cgu_dword19 dw19; + union nac_cgu_dword22 dw22; + union nac_cgu_dword24 dw24; + union nac_cgu_dword9 dw9; + int err; + + if (clk_freq >= NUM_ICE_TIME_REF_FREQ) { + dev_warn(ice_hw_to_dev(hw), "Invalid TIME_REF frequency %u\n", + clk_freq); + return -EINVAL; + } + + if (clk_src >= NUM_ICE_CLK_SRC) { + dev_warn(ice_hw_to_dev(hw), "Invalid clock source %u\n", + clk_src); + return -EINVAL; + } + + if (clk_src == ICE_CLK_SRC_TCXO && + clk_freq != ICE_TIME_REF_FREQ_156_250) { + dev_warn(ice_hw_to_dev(hw), + "TCXO only supports 156.25 MHz frequency\n"); + return -EINVAL; + } + + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD9, &dw9.val); + if (err) + return err; + + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD24, &dw24.val); + if (err) + return err; + + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD16_E825C, &dw16.val); + if (err) + return err; + + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD23_E825C, &dw23.val); + if (err) + return err; + + err = ice_read_cgu_reg_e82x(hw, TSPLL_RO_LOCK_E825C, &ro_lock.val); + if (err) + return err; + + /* Log the current clock configuration */ + ice_debug(hw, ICE_DBG_PTP, "Current CGU configuration -- %s, clk_src %s, clk_freq %s, PLL %s\n", + dw24.ts_pll_enable ? "enabled" : "disabled", + ice_clk_src_str(dw23.time_ref_sel), + ice_clk_freq_str(dw9.time_ref_freq_sel), + ro_lock.plllock_true_lock_cri ? "locked" : "unlocked"); + + /* Disable the PLL before changing the clock source or frequency */ + if (dw23.ts_pll_enable) { + dw23.ts_pll_enable = 0; + + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD23_E825C, + dw23.val); + if (err) + return err; + } + + /* Set the frequency */ + dw9.time_ref_freq_sel = clk_freq; + + /* Enable the correct receiver */ + if (clk_src == ICE_CLK_SRC_TCXO) { + dw9.time_ref_en = 0; + dw9.clk_eref0_en = 1; + } else { + dw9.time_ref_en = 1; + dw9.clk_eref0_en = 0; + } + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD9, dw9.val); + if (err) + return err; + + /* Choose the referenced frequency */ + dw16.tspll_ck_refclkfreq = + e825c_cgu_params[clk_freq].tspll_ck_refclkfreq; + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD16_E825C, dw16.val); + if (err) + return err; + + /* Configure the TS PLL feedback divisor */ + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD19, &dw19.val); + if (err) + return err; + + dw19.tspll_fbdiv_intgr = + e825c_cgu_params[clk_freq].tspll_fbdiv_intgr; + dw19.tspll_ndivratio = + e825c_cgu_params[clk_freq].tspll_ndivratio; + + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD19, dw19.val); + if (err) + return err; + + /* Configure the TS PLL post divisor */ + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD22, &dw22.val); + if (err) + return err; + + /* These two are constant for E825C */ + dw22.time1588clk_div = 5; + dw22.time1588clk_sel_div2 = 0; + + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD22, dw22.val); + if (err) + return err; + + /* Configure the TS PLL pre divisor and clock source */ + err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD23_E825C, &dw23.val); + if (err) + return err; + + dw23.ref1588_ck_div = + e825c_cgu_params[clk_freq].ref1588_ck_div; + dw23.time_ref_sel = clk_src; + + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD23_E825C, dw23.val); + if (err) + return err; + + dw24.tspll_fbdiv_frac = + e825c_cgu_params[clk_freq].tspll_fbdiv_frac; + + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD24, dw24.val); + if (err) + return err; + + /* Finally, enable the PLL */ + dw23.ts_pll_enable = 1; + + err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD23_E825C, dw23.val); + if (err) + return err; + + /* Wait to verify if the PLL locks */ + usleep_range(1000, 5000); + + err = ice_read_cgu_reg_e82x(hw, TSPLL_RO_LOCK_E825C, &ro_lock.val); + if (err) + return err; + + if (!ro_lock.plllock_true_lock_cri) { + dev_warn(ice_hw_to_dev(hw), "CGU PLL failed to lock\n"); + return -EBUSY; + } + + /* Log the current clock configuration */ + ice_debug(hw, ICE_DBG_PTP, "New CGU configuration -- %s, clk_src %s, clk_freq %s, PLL %s\n", + dw24.ts_pll_enable ? "enabled" : "disabled", + ice_clk_src_str(dw23.time_ref_sel), + ice_clk_freq_str(dw9.time_ref_freq_sel), + ro_lock.plllock_true_lock_cri ? "locked" : "unlocked"); + + return 0; +} + +/** + * ice_cfg_cgu_pll_dis_sticky_bits_e82x - disable TS PLL sticky bits + * @hw: pointer to the HW struct + * + * Configure the Clock Generation Unit TS PLL sticky bits so they don't latch on + * losing TS PLL lock, but always show current state. + * + * Return: 0 on success, other error codes when failed to read/write CGU + */ +static int ice_cfg_cgu_pll_dis_sticky_bits_e82x(struct ice_hw *hw) +{ + union tspll_cntr_bist_settings cntr_bist; + int err; + + err = ice_read_cgu_reg_e82x(hw, TSPLL_CNTR_BIST_SETTINGS, + &cntr_bist.val); + if (err) + return err; + + /* Disable sticky lock detection so lock err reported is accurate */ + cntr_bist.i_plllock_sel_0 = 0; + cntr_bist.i_plllock_sel_1 = 0; + + return ice_write_cgu_reg_e82x(hw, TSPLL_CNTR_BIST_SETTINGS, + cntr_bist.val); +} + +/** + * ice_cfg_cgu_pll_dis_sticky_bits_e825c - disable TS PLL sticky bits for E825-C + * @hw: pointer to the HW struct + * + * Configure the Clock Generation Unit TS PLL sticky bits so they don't latch on + * losing TS PLL lock, but always show current state. + * + * Return: 0 on success, other error codes when failed to read/write CGU + */ +static int ice_cfg_cgu_pll_dis_sticky_bits_e825c(struct ice_hw *hw) +{ + union tspll_bw_tdc_e825c bw_tdc; + int err; + + err = ice_read_cgu_reg_e82x(hw, TSPLL_BW_TDC_E825C, &bw_tdc.val); + if (err) + return err; + + bw_tdc.i_plllock_sel_1_0 = 0; + + return ice_write_cgu_reg_e82x(hw, TSPLL_BW_TDC_E825C, bw_tdc.val); +} + +/** + * ice_init_cgu_e82x - Initialize CGU with settings from firmware + * @hw: pointer to the HW structure + * + * Initialize the Clock Generation Unit of the E822 device. + * + * Return: 0 on success, other error codes when failed to read/write/cfg CGU + */ +static int ice_init_cgu_e82x(struct ice_hw *hw) +{ + struct ice_ts_func_info *ts_info = &hw->func_caps.ts_func_info; + int err; + + /* Disable sticky lock detection so lock err reported is accurate */ + if (ice_is_e825c(hw)) + err = ice_cfg_cgu_pll_dis_sticky_bits_e825c(hw); + else + err = ice_cfg_cgu_pll_dis_sticky_bits_e82x(hw); + if (err) + return err; + + /* Configure the CGU PLL using the parameters from the function + * capabilities. + */ + if (ice_is_e825c(hw)) + err = ice_cfg_cgu_pll_e825c(hw, ts_info->time_ref, + (enum ice_clk_src)ts_info->clk_src); + else + err = ice_cfg_cgu_pll_e82x(hw, ts_info->time_ref, + (enum ice_clk_src)ts_info->clk_src); + + return err; +} + +/** + * ice_ptp_tmr_cmd_to_src_reg - Convert to source timer command value + * @hw: pointer to HW struct + * @cmd: Timer command + * + * Return: the source timer command register value for the given PTP timer + * command. + */ +static u32 ice_ptp_tmr_cmd_to_src_reg(struct ice_hw *hw, + enum ice_ptp_tmr_cmd cmd) +{ + u32 cmd_val, tmr_idx; + + switch (cmd) { + case ICE_PTP_INIT_TIME: + cmd_val = GLTSYN_CMD_INIT_TIME; + break; + case ICE_PTP_INIT_INCVAL: + cmd_val = GLTSYN_CMD_INIT_INCVAL; + break; + case ICE_PTP_ADJ_TIME: + cmd_val = GLTSYN_CMD_ADJ_TIME; + break; + case ICE_PTP_ADJ_TIME_AT_TIME: + cmd_val = GLTSYN_CMD_ADJ_INIT_TIME; + break; + case ICE_PTP_NOP: + case ICE_PTP_READ_TIME: + cmd_val = GLTSYN_CMD_READ_TIME; + break; + default: + dev_warn(ice_hw_to_dev(hw), + "Ignoring unrecognized timer command %u\n", cmd); + cmd_val = 0; + } + + tmr_idx = ice_get_ptp_src_clock_index(hw); + + return tmr_idx << SEL_CPK_SRC | cmd_val; +} + +/** + * ice_ptp_tmr_cmd_to_port_reg- Convert to port timer command value + * @hw: pointer to HW struct + * @cmd: Timer command + * + * Note that some hardware families use a different command register value for + * the PHY ports, while other hardware families use the same register values + * as the source timer. + * + * Return: the PHY port timer command register value for the given PTP timer + * command. + */ +static u32 ice_ptp_tmr_cmd_to_port_reg(struct ice_hw *hw, + enum ice_ptp_tmr_cmd cmd) +{ + u32 cmd_val, tmr_idx; + + /* Certain hardware families share the same register values for the + * port register and source timer register. + */ + switch (hw->ptp.phy_model) { + case ICE_PHY_E810: + return ice_ptp_tmr_cmd_to_src_reg(hw, cmd) & TS_CMD_MASK_E810; + default: + break; + } + + switch (cmd) { + case ICE_PTP_INIT_TIME: + cmd_val = PHY_CMD_INIT_TIME; + break; + case ICE_PTP_INIT_INCVAL: + cmd_val = PHY_CMD_INIT_INCVAL; + break; + case ICE_PTP_ADJ_TIME: + cmd_val = PHY_CMD_ADJ_TIME; + break; + case ICE_PTP_ADJ_TIME_AT_TIME: + cmd_val = PHY_CMD_ADJ_TIME_AT_TIME; + break; + case ICE_PTP_READ_TIME: + cmd_val = PHY_CMD_READ_TIME; + break; + case ICE_PTP_NOP: + cmd_val = 0; + break; + default: + dev_warn(ice_hw_to_dev(hw), + "Ignoring unrecognized timer command %u\n", cmd); + cmd_val = 0; + } + + tmr_idx = ice_get_ptp_src_clock_index(hw); + + return tmr_idx << SEL_PHY_SRC | cmd_val; +} + /** * ice_ptp_src_cmd - Prepare source timer for a timer command * @hw: pointer to HW structure * @cmd: Timer command * - * Prepare the source timer for an upcoming timer sync command. + * Prepare the source timer for an upcoming timer sync command. + */ +void ice_ptp_src_cmd(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd) +{ + u32 cmd_val = ice_ptp_tmr_cmd_to_src_reg(hw, cmd); + + wr32(hw, GLTSYN_CMD, cmd_val); +} + +/** + * ice_ptp_exec_tmr_cmd - Execute all prepared timer commands + * @hw: pointer to HW struct + * + * Write the SYNC_EXEC_CMD bit to the GLTSYN_CMD_SYNC register, and flush the + * write immediately. This triggers the hardware to begin executing all of the + * source and PHY timer commands synchronously. + */ +static void ice_ptp_exec_tmr_cmd(struct ice_hw *hw) +{ + struct ice_pf *pf = container_of(hw, struct ice_pf, hw); + + guard(spinlock)(&pf->adapter->ptp_gltsyn_time_lock); + wr32(hw, GLTSYN_CMD_SYNC, SYNC_EXEC_CMD); + ice_flush(hw); +} + +/* 56G PHY device functions + * + * The following functions operate on devices with the ETH 56G PHY. + */ + +/** + * ice_write_phy_eth56g - Write a PHY port register + * @hw: pointer to the HW struct + * @phy_idx: PHY index + * @addr: PHY register address + * @val: Value to write + * + * Return: 0 on success, other error codes when failed to write to PHY + */ +static int ice_write_phy_eth56g(struct ice_hw *hw, u8 phy_idx, u32 addr, + u32 val) +{ + struct ice_sbq_msg_input phy_msg; + int err; + + phy_msg.opcode = ice_sbq_msg_wr; + + phy_msg.msg_addr_low = lower_16_bits(addr); + phy_msg.msg_addr_high = upper_16_bits(addr); + + phy_msg.data = val; + phy_msg.dest_dev = hw->ptp.phy.eth56g.phy_addr[phy_idx]; + + err = ice_sbq_rw_reg(hw, &phy_msg); + + if (err) + ice_debug(hw, ICE_DBG_PTP, "PTP failed to send msg to phy %d\n", + err); + + return err; +} + +/** + * ice_read_phy_eth56g - Read a PHY port register + * @hw: pointer to the HW struct + * @phy_idx: PHY index + * @addr: PHY register address + * @val: Value to write + * + * Return: 0 on success, other error codes when failed to read from PHY + */ +static int ice_read_phy_eth56g(struct ice_hw *hw, u8 phy_idx, u32 addr, + u32 *val) +{ + struct ice_sbq_msg_input phy_msg; + int err; + + phy_msg.opcode = ice_sbq_msg_rd; + + phy_msg.msg_addr_low = lower_16_bits(addr); + phy_msg.msg_addr_high = upper_16_bits(addr); + + phy_msg.data = 0; + phy_msg.dest_dev = hw->ptp.phy.eth56g.phy_addr[phy_idx]; + + err = ice_sbq_rw_reg(hw, &phy_msg); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "PTP failed to send msg to phy %d\n", + err); + return err; + } + + *val = phy_msg.data; + + return 0; +} + +/** + * ice_phy_res_address_eth56g - Calculate a PHY port register address + * @port: Port number to be written + * @res_type: resource type (register/memory) + * @offset: Offset from PHY port register base + * @addr: The result address + * + * Return: + * * %0 - success + * * %EINVAL - invalid port number or resource type + */ +static int ice_phy_res_address_eth56g(u8 port, enum eth56g_res_type res_type, + u32 offset, u32 *addr) +{ + u8 lane = port % ICE_PORTS_PER_QUAD; + u8 phy = ICE_GET_QUAD_NUM(port); + + if (res_type >= NUM_ETH56G_PHY_RES) + return -EINVAL; + + *addr = eth56g_phy_res[res_type].base[phy] + + lane * eth56g_phy_res[res_type].step + offset; + return 0; +} + +/** + * ice_write_port_eth56g - Write a PHY port register + * @hw: pointer to the HW struct + * @offset: PHY register offset + * @port: Port number + * @val: Value to write + * @res_type: resource type (register/memory) + * + * Return: + * * %0 - success + * * %EINVAL - invalid port number or resource type + * * %other - failed to write to PHY + */ +static int ice_write_port_eth56g(struct ice_hw *hw, u8 port, u32 offset, + u32 val, enum eth56g_res_type res_type) +{ + u8 phy_port = port % hw->ptp.ports_per_phy; + u8 phy_idx = port / hw->ptp.ports_per_phy; + u32 addr; + int err; + + if (port >= hw->ptp.num_lports) + return -EINVAL; + + err = ice_phy_res_address_eth56g(phy_port, res_type, offset, &addr); + if (err) + return err; + + return ice_write_phy_eth56g(hw, phy_idx, addr, val); +} + +/** + * ice_read_port_eth56g - Read a PHY port register + * @hw: pointer to the HW struct + * @offset: PHY register offset + * @port: Port number + * @val: Value to write + * @res_type: resource type (register/memory) + * + * Return: + * * %0 - success + * * %EINVAL - invalid port number or resource type + * * %other - failed to read from PHY + */ +static int ice_read_port_eth56g(struct ice_hw *hw, u8 port, u32 offset, + u32 *val, enum eth56g_res_type res_type) +{ + u8 phy_port = port % hw->ptp.ports_per_phy; + u8 phy_idx = port / hw->ptp.ports_per_phy; + u32 addr; + int err; + + if (port >= hw->ptp.num_lports) + return -EINVAL; + + err = ice_phy_res_address_eth56g(phy_port, res_type, offset, &addr); + if (err) + return err; + + return ice_read_phy_eth56g(hw, phy_idx, addr, val); +} + +/** + * ice_write_ptp_reg_eth56g - Write a PHY port register + * @hw: pointer to the HW struct + * @port: Port number to be written + * @offset: Offset from PHY port register base + * @val: Value to write + * + * Return: + * * %0 - success + * * %EINVAL - invalid port number or resource type + * * %other - failed to write to PHY + */ +static int ice_write_ptp_reg_eth56g(struct ice_hw *hw, u8 port, u16 offset, + u32 val) +{ + return ice_write_port_eth56g(hw, port, offset, val, ETH56G_PHY_REG_PTP); +} + +/** + * ice_write_mac_reg_eth56g - Write a MAC PHY port register + * parameter + * @hw: pointer to the HW struct + * @port: Port number to be written + * @offset: Offset from PHY port register base + * @val: Value to write + * + * Return: + * * %0 - success + * * %EINVAL - invalid port number or resource type + * * %other - failed to write to PHY + */ +static int ice_write_mac_reg_eth56g(struct ice_hw *hw, u8 port, u32 offset, + u32 val) +{ + return ice_write_port_eth56g(hw, port, offset, val, ETH56G_PHY_REG_MAC); +} + +/** + * ice_write_xpcs_reg_eth56g - Write a PHY port register + * @hw: pointer to the HW struct + * @port: Port number to be written + * @offset: Offset from PHY port register base + * @val: Value to write + * + * Return: + * * %0 - success + * * %EINVAL - invalid port number or resource type + * * %other - failed to write to PHY + */ +static int ice_write_xpcs_reg_eth56g(struct ice_hw *hw, u8 port, u32 offset, + u32 val) +{ + return ice_write_port_eth56g(hw, port, offset, val, + ETH56G_PHY_REG_XPCS); +} + +/** + * ice_read_ptp_reg_eth56g - Read a PHY port register + * @hw: pointer to the HW struct + * @port: Port number to be read + * @offset: Offset from PHY port register base + * @val: Pointer to the value to read (out param) + * + * Return: + * * %0 - success + * * %EINVAL - invalid port number or resource type + * * %other - failed to read from PHY + */ +static int ice_read_ptp_reg_eth56g(struct ice_hw *hw, u8 port, u16 offset, + u32 *val) +{ + return ice_read_port_eth56g(hw, port, offset, val, ETH56G_PHY_REG_PTP); +} + +/** + * ice_read_mac_reg_eth56g - Read a PHY port register + * @hw: pointer to the HW struct + * @port: Port number to be read + * @offset: Offset from PHY port register base + * @val: Pointer to the value to read (out param) + * + * Return: + * * %0 - success + * * %EINVAL - invalid port number or resource type + * * %other - failed to read from PHY + */ +static int ice_read_mac_reg_eth56g(struct ice_hw *hw, u8 port, u16 offset, + u32 *val) +{ + return ice_read_port_eth56g(hw, port, offset, val, ETH56G_PHY_REG_MAC); +} + +/** + * ice_read_gpcs_reg_eth56g - Read a PHY port register + * @hw: pointer to the HW struct + * @port: Port number to be read + * @offset: Offset from PHY port register base + * @val: Pointer to the value to read (out param) + * + * Return: + * * %0 - success + * * %EINVAL - invalid port number or resource type + * * %other - failed to read from PHY + */ +static int ice_read_gpcs_reg_eth56g(struct ice_hw *hw, u8 port, u16 offset, + u32 *val) +{ + return ice_read_port_eth56g(hw, port, offset, val, ETH56G_PHY_REG_GPCS); +} + +/** + * ice_read_port_mem_eth56g - Read a PHY port memory location + * @hw: pointer to the HW struct + * @port: Port number to be read + * @offset: Offset from PHY port register base + * @val: Pointer to the value to read (out param) + * + * Return: + * * %0 - success + * * %EINVAL - invalid port number or resource type + * * %other - failed to read from PHY + */ +static int ice_read_port_mem_eth56g(struct ice_hw *hw, u8 port, u16 offset, + u32 *val) +{ + return ice_read_port_eth56g(hw, port, offset, val, ETH56G_PHY_MEM_PTP); +} + +/** + * ice_write_port_mem_eth56g - Write a PHY port memory location + * @hw: pointer to the HW struct + * @port: Port number to be read + * @offset: Offset from PHY port register base + * @val: Pointer to the value to read (out param) + * + * Return: + * * %0 - success + * * %EINVAL - invalid port number or resource type + * * %other - failed to write to PHY + */ +static int ice_write_port_mem_eth56g(struct ice_hw *hw, u8 port, u16 offset, + u32 val) +{ + return ice_write_port_eth56g(hw, port, offset, val, ETH56G_PHY_MEM_PTP); +} + +/** + * ice_is_64b_phy_reg_eth56g - Check if this is a 64bit PHY register + * @low_addr: the low address to check + * @high_addr: on return, contains the high address of the 64bit register + * + * Write the appropriate high register offset to use. + * + * Return: true if the provided low address is one of the known 64bit PHY values + * represented as two 32bit registers, false otherwise. + */ +static bool ice_is_64b_phy_reg_eth56g(u16 low_addr, u16 *high_addr) +{ + switch (low_addr) { + case PHY_REG_TX_TIMER_INC_PRE_L: + *high_addr = PHY_REG_TX_TIMER_INC_PRE_U; + return true; + case PHY_REG_RX_TIMER_INC_PRE_L: + *high_addr = PHY_REG_RX_TIMER_INC_PRE_U; + return true; + case PHY_REG_TX_CAPTURE_L: + *high_addr = PHY_REG_TX_CAPTURE_U; + return true; + case PHY_REG_RX_CAPTURE_L: + *high_addr = PHY_REG_RX_CAPTURE_U; + return true; + case PHY_REG_TOTAL_TX_OFFSET_L: + *high_addr = PHY_REG_TOTAL_TX_OFFSET_U; + return true; + case PHY_REG_TOTAL_RX_OFFSET_L: + *high_addr = PHY_REG_TOTAL_RX_OFFSET_U; + return true; + case PHY_REG_TX_MEMORY_STATUS_L: + *high_addr = PHY_REG_TX_MEMORY_STATUS_U; + return true; + default: + return false; + } +} + +/** + * ice_is_40b_phy_reg_eth56g - Check if this is a 40bit PHY register + * @low_addr: the low address to check + * @high_addr: on return, contains the high address of the 40bit value + * + * Write the appropriate high register offset to use. + * + * Return: true if the provided low address is one of the known 40bit PHY + * values split into two registers with the lower 8 bits in the low register and + * the upper 32 bits in the high register, false otherwise. + */ +static bool ice_is_40b_phy_reg_eth56g(u16 low_addr, u16 *high_addr) +{ + switch (low_addr) { + case PHY_REG_TIMETUS_L: + *high_addr = PHY_REG_TIMETUS_U; + return true; + case PHY_PCS_REF_TUS_L: + *high_addr = PHY_PCS_REF_TUS_U; + return true; + case PHY_PCS_REF_INC_L: + *high_addr = PHY_PCS_REF_INC_U; + return true; + default: + return false; + } +} + +/** + * ice_read_64b_phy_reg_eth56g - Read a 64bit value from PHY registers + * @hw: pointer to the HW struct + * @port: PHY port to read from + * @low_addr: offset of the lower register to read from + * @val: on return, the contents of the 64bit value from the PHY registers + * @res_type: resource type + * + * Check if the caller has specified a known 40 bit register offset and read + * the two registers associated with a 40bit value and return it in the val + * pointer. + * + * Return: + * * %0 - success + * * %EINVAL - not a 64 bit register + * * %other - failed to read from PHY + */ +static int ice_read_64b_phy_reg_eth56g(struct ice_hw *hw, u8 port, u16 low_addr, + u64 *val, enum eth56g_res_type res_type) +{ + u16 high_addr; + u32 lo, hi; + int err; + + if (!ice_is_64b_phy_reg_eth56g(low_addr, &high_addr)) + return -EINVAL; + + err = ice_read_port_eth56g(hw, port, low_addr, &lo, res_type); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to read from low register %#08x\n, err %d", + low_addr, err); + return err; + } + + err = ice_read_port_eth56g(hw, port, high_addr, &hi, res_type); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to read from high register %#08x\n, err %d", + high_addr, err); + return err; + } + + *val = ((u64)hi << 32) | lo; + + return 0; +} + +/** + * ice_read_64b_ptp_reg_eth56g - Read a 64bit value from PHY registers + * @hw: pointer to the HW struct + * @port: PHY port to read from + * @low_addr: offset of the lower register to read from + * @val: on return, the contents of the 64bit value from the PHY registers + * + * Check if the caller has specified a known 40 bit register offset and read + * the two registers associated with a 40bit value and return it in the val + * pointer. + * + * Return: + * * %0 - success + * * %EINVAL - not a 64 bit register + * * %other - failed to read from PHY + */ +static int ice_read_64b_ptp_reg_eth56g(struct ice_hw *hw, u8 port, u16 low_addr, + u64 *val) +{ + return ice_read_64b_phy_reg_eth56g(hw, port, low_addr, val, + ETH56G_PHY_REG_PTP); +} + +/** + * ice_write_40b_phy_reg_eth56g - Write a 40b value to the PHY + * @hw: pointer to the HW struct + * @port: port to write to + * @low_addr: offset of the low register + * @val: 40b value to write + * @res_type: resource type + * + * Check if the caller has specified a known 40 bit register offset and write + * provided 40b value to the two associated registers by splitting it up into + * two chunks, the lower 8 bits and the upper 32 bits. + * + * Return: + * * %0 - success + * * %EINVAL - not a 40 bit register + * * %other - failed to write to PHY + */ +static int ice_write_40b_phy_reg_eth56g(struct ice_hw *hw, u8 port, + u16 low_addr, u64 val, + enum eth56g_res_type res_type) +{ + u16 high_addr; + u32 lo, hi; + int err; + + if (!ice_is_40b_phy_reg_eth56g(low_addr, &high_addr)) + return -EINVAL; + + lo = FIELD_GET(P_REG_40B_LOW_M, val); + hi = (u32)(val >> P_REG_40B_HIGH_S); + + err = ice_write_port_eth56g(hw, port, low_addr, lo, res_type); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write to low register 0x%08x\n, err %d", + low_addr, err); + return err; + } + + err = ice_write_port_eth56g(hw, port, high_addr, hi, res_type); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write to high register 0x%08x\n, err %d", + high_addr, err); + return err; + } + + return 0; +} + +/** + * ice_write_40b_ptp_reg_eth56g - Write a 40b value to the PHY + * @hw: pointer to the HW struct + * @port: port to write to + * @low_addr: offset of the low register + * @val: 40b value to write + * + * Check if the caller has specified a known 40 bit register offset and write + * provided 40b value to the two associated registers by splitting it up into + * two chunks, the lower 8 bits and the upper 32 bits. + * + * Return: + * * %0 - success + * * %EINVAL - not a 40 bit register + * * %other - failed to write to PHY + */ +static int ice_write_40b_ptp_reg_eth56g(struct ice_hw *hw, u8 port, + u16 low_addr, u64 val) +{ + return ice_write_40b_phy_reg_eth56g(hw, port, low_addr, val, + ETH56G_PHY_REG_PTP); +} + +/** + * ice_write_64b_phy_reg_eth56g - Write a 64bit value to PHY registers + * @hw: pointer to the HW struct + * @port: PHY port to read from + * @low_addr: offset of the lower register to read from + * @val: the contents of the 64bit value to write to PHY + * @res_type: resource type + * + * Check if the caller has specified a known 64 bit register offset and write + * the 64bit value to the two associated 32bit PHY registers. + * + * Return: + * * %0 - success + * * %EINVAL - not a 64 bit register + * * %other - failed to write to PHY + */ +static int ice_write_64b_phy_reg_eth56g(struct ice_hw *hw, u8 port, + u16 low_addr, u64 val, + enum eth56g_res_type res_type) +{ + u16 high_addr; + u32 lo, hi; + int err; + + if (!ice_is_64b_phy_reg_eth56g(low_addr, &high_addr)) + return -EINVAL; + + lo = lower_32_bits(val); + hi = upper_32_bits(val); + + err = ice_write_port_eth56g(hw, port, low_addr, lo, res_type); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write to low register 0x%08x\n, err %d", + low_addr, err); + return err; + } + + err = ice_write_port_eth56g(hw, port, high_addr, hi, res_type); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write to high register 0x%08x\n, err %d", + high_addr, err); + return err; + } + + return 0; +} + +/** + * ice_write_64b_ptp_reg_eth56g - Write a 64bit value to PHY registers + * @hw: pointer to the HW struct + * @port: PHY port to read from + * @low_addr: offset of the lower register to read from + * @val: the contents of the 64bit value to write to PHY + * + * Check if the caller has specified a known 64 bit register offset and write + * the 64bit value to the two associated 32bit PHY registers. + * + * Return: + * * %0 - success + * * %EINVAL - not a 64 bit register + * * %other - failed to write to PHY + */ +static int ice_write_64b_ptp_reg_eth56g(struct ice_hw *hw, u8 port, + u16 low_addr, u64 val) +{ + return ice_write_64b_phy_reg_eth56g(hw, port, low_addr, val, + ETH56G_PHY_REG_PTP); +} + +/** + * ice_read_ptp_tstamp_eth56g - Read a PHY timestamp out of the port memory + * @hw: pointer to the HW struct + * @port: the port to read from + * @idx: the timestamp index to read + * @tstamp: on return, the 40bit timestamp value + * + * Read a 40bit timestamp value out of the two associated entries in the + * port memory block of the internal PHYs of the 56G devices. + * + * Return: + * * %0 - success + * * %other - failed to read from PHY + */ +static int ice_read_ptp_tstamp_eth56g(struct ice_hw *hw, u8 port, u8 idx, + u64 *tstamp) +{ + u16 lo_addr, hi_addr; + u32 lo, hi; + int err; + + lo_addr = (u16)PHY_TSTAMP_L(idx); + hi_addr = (u16)PHY_TSTAMP_U(idx); + + err = ice_read_port_mem_eth56g(hw, port, lo_addr, &lo); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to read low PTP timestamp register, err %d\n", + err); + return err; + } + + err = ice_read_port_mem_eth56g(hw, port, hi_addr, &hi); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to read high PTP timestamp register, err %d\n", + err); + return err; + } + + /* For 56G based internal PHYs, the timestamp is reported with the + * lower 8 bits in the low register, and the upper 32 bits in the high + * register. + */ + *tstamp = ((u64)hi) << TS_PHY_HIGH_S | ((u64)lo & TS_PHY_LOW_M); + + return 0; +} + +/** + * ice_clear_ptp_tstamp_eth56g - Clear a timestamp from the quad block + * @hw: pointer to the HW struct + * @port: the quad to read from + * @idx: the timestamp index to reset + * + * Read and then forcibly clear the timestamp index to ensure the valid bit is + * cleared and the timestamp status bit is reset in the PHY port memory of + * internal PHYs of the 56G devices. + * + * To directly clear the contents of the timestamp block entirely, discarding + * all timestamp data at once, software should instead use + * ice_ptp_reset_ts_memory_quad_eth56g(). + * + * This function should only be called on an idx whose bit is set according to + * ice_get_phy_tx_tstamp_ready(). + * + * Return: + * * %0 - success + * * %other - failed to write to PHY + */ +static int ice_clear_ptp_tstamp_eth56g(struct ice_hw *hw, u8 port, u8 idx) +{ + u64 unused_tstamp; + u16 lo_addr; + int err; + + /* Read the timestamp register to ensure the timestamp status bit is + * cleared. + */ + err = ice_read_ptp_tstamp_eth56g(hw, port, idx, &unused_tstamp); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to read the PHY timestamp register for port %u, idx %u, err %d\n", + port, idx, err); + } + + lo_addr = (u16)PHY_TSTAMP_L(idx); + + err = ice_write_port_mem_eth56g(hw, port, lo_addr, 0); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to clear low PTP timestamp register for port %u, idx %u, err %d\n", + port, idx, err); + return err; + } + + return 0; +} + +/** + * ice_ptp_reset_ts_memory_eth56g - Clear all timestamps from the port block + * @hw: pointer to the HW struct + */ +static void ice_ptp_reset_ts_memory_eth56g(struct ice_hw *hw) +{ + unsigned int port; + + for (port = 0; port < hw->ptp.num_lports; port++) { + ice_write_ptp_reg_eth56g(hw, port, PHY_REG_TX_MEMORY_STATUS_L, + 0); + ice_write_ptp_reg_eth56g(hw, port, PHY_REG_TX_MEMORY_STATUS_U, + 0); + } +} + +/** + * ice_ptp_prep_port_time_eth56g - Prepare one PHY port with initial time + * @hw: pointer to the HW struct + * @port: port number + * @time: time to initialize the PHY port clocks to + * + * Write a new initial time value into registers of a specific PHY port. + * + * Return: + * * %0 - success + * * %other - failed to write to PHY + */ +static int ice_ptp_prep_port_time_eth56g(struct ice_hw *hw, u8 port, + u64 time) +{ + int err; + + /* Tx case */ + err = ice_write_64b_ptp_reg_eth56g(hw, port, PHY_REG_TX_TIMER_INC_PRE_L, + time); + if (err) + return err; + + /* Rx case */ + return ice_write_64b_ptp_reg_eth56g(hw, port, + PHY_REG_RX_TIMER_INC_PRE_L, time); +} + +/** + * ice_ptp_prep_phy_time_eth56g - Prepare PHY port with initial time + * @hw: pointer to the HW struct + * @time: Time to initialize the PHY port clocks to + * + * Program the PHY port registers with a new initial time value. The port + * clock will be initialized once the driver issues an ICE_PTP_INIT_TIME sync + * command. The time value is the upper 32 bits of the PHY timer, usually in + * units of nominal nanoseconds. + * + * Return: + * * %0 - success + * * %other - failed to write to PHY + */ +static int ice_ptp_prep_phy_time_eth56g(struct ice_hw *hw, u32 time) +{ + u64 phy_time; + u8 port; + + /* The time represents the upper 32 bits of the PHY timer, so we need + * to shift to account for this when programming. + */ + phy_time = (u64)time << 32; + + for (port = 0; port < hw->ptp.num_lports; port++) { + int err; + + err = ice_ptp_prep_port_time_eth56g(hw, port, phy_time); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write init time for port %u, err %d\n", + port, err); + return err; + } + } + + return 0; +} + +/** + * ice_ptp_prep_port_adj_eth56g - Prepare a single port for time adjust + * @hw: pointer to HW struct + * @port: Port number to be programmed + * @time: time in cycles to adjust the port clocks + * + * Program the port for an atomic adjustment by writing the Tx and Rx timer + * registers. The atomic adjustment won't be completed until the driver issues + * an ICE_PTP_ADJ_TIME command. + * + * Note that time is not in units of nanoseconds. It is in clock time + * including the lower sub-nanosecond portion of the port timer. + * + * Negative adjustments are supported using 2s complement arithmetic. + * + * Return: + * * %0 - success + * * %other - failed to write to PHY + */ +static int ice_ptp_prep_port_adj_eth56g(struct ice_hw *hw, u8 port, s64 time) +{ + u32 l_time, u_time; + int err; + + l_time = lower_32_bits(time); + u_time = upper_32_bits(time); + + /* Tx case */ + err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_TX_TIMER_INC_PRE_L, + l_time); + if (err) + goto exit_err; + + err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_TX_TIMER_INC_PRE_U, + u_time); + if (err) + goto exit_err; + + /* Rx case */ + err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_RX_TIMER_INC_PRE_L, + l_time); + if (err) + goto exit_err; + + err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_RX_TIMER_INC_PRE_U, + u_time); + if (err) + goto exit_err; + + return 0; + +exit_err: + ice_debug(hw, ICE_DBG_PTP, "Failed to write time adjust for port %u, err %d\n", + port, err); + return err; +} + +/** + * ice_ptp_prep_phy_adj_eth56g - Prep PHY ports for a time adjustment + * @hw: pointer to HW struct + * @adj: adjustment in nanoseconds + * + * Prepare the PHY ports for an atomic time adjustment by programming the PHY + * Tx and Rx port registers. The actual adjustment is completed by issuing an + * ICE_PTP_ADJ_TIME or ICE_PTP_ADJ_TIME_AT_TIME sync command. + * + * Return: + * * %0 - success + * * %other - failed to write to PHY + */ +static int ice_ptp_prep_phy_adj_eth56g(struct ice_hw *hw, s32 adj) +{ + s64 cycles; + u8 port; + + /* The port clock supports adjustment of the sub-nanosecond portion of + * the clock (lowest 32 bits). We shift the provided adjustment in + * nanoseconds by 32 to calculate the appropriate adjustment to program + * into the PHY ports. + */ + cycles = (s64)adj << 32; + + for (port = 0; port < hw->ptp.num_lports; port++) { + int err; + + err = ice_ptp_prep_port_adj_eth56g(hw, port, cycles); + if (err) + return err; + } + + return 0; +} + +/** + * ice_ptp_prep_phy_incval_eth56g - Prepare PHY ports for time adjustment + * @hw: pointer to HW struct + * @incval: new increment value to prepare + * + * Prepare each of the PHY ports for a new increment value by programming the + * port's TIMETUS registers. The new increment value will be updated after + * issuing an ICE_PTP_INIT_INCVAL command. + * + * Return: + * * %0 - success + * * %other - failed to write to PHY + */ +static int ice_ptp_prep_phy_incval_eth56g(struct ice_hw *hw, u64 incval) +{ + u8 port; + + for (port = 0; port < hw->ptp.num_lports; port++) { + int err; + + err = ice_write_40b_ptp_reg_eth56g(hw, port, PHY_REG_TIMETUS_L, + incval); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write incval for port %u, err %d\n", + port, err); + return err; + } + } + + return 0; +} + +/** + * ice_ptp_read_port_capture_eth56g - Read a port's local time capture + * @hw: pointer to HW struct + * @port: Port number to read + * @tx_ts: on return, the Tx port time capture + * @rx_ts: on return, the Rx port time capture + * + * Read the port's Tx and Rx local time capture values. + * + * Return: + * * %0 - success + * * %other - failed to read from PHY + */ +static int ice_ptp_read_port_capture_eth56g(struct ice_hw *hw, u8 port, + u64 *tx_ts, u64 *rx_ts) +{ + int err; + + /* Tx case */ + err = ice_read_64b_ptp_reg_eth56g(hw, port, PHY_REG_TX_CAPTURE_L, + tx_ts); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to read REG_TX_CAPTURE, err %d\n", + err); + return err; + } + + ice_debug(hw, ICE_DBG_PTP, "tx_init = %#016llx\n", *tx_ts); + + /* Rx case */ + err = ice_read_64b_ptp_reg_eth56g(hw, port, PHY_REG_RX_CAPTURE_L, + rx_ts); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to read RX_CAPTURE, err %d\n", + err); + return err; + } + + ice_debug(hw, ICE_DBG_PTP, "rx_init = %#016llx\n", *rx_ts); + + return 0; +} + +/** + * ice_ptp_write_port_cmd_eth56g - Prepare a single PHY port for a timer command + * @hw: pointer to HW struct + * @port: Port to which cmd has to be sent + * @cmd: Command to be sent to the port + * + * Prepare the requested port for an upcoming timer sync command. + * + * Return: + * * %0 - success + * * %other - failed to write to PHY + */ +static int ice_ptp_write_port_cmd_eth56g(struct ice_hw *hw, u8 port, + enum ice_ptp_tmr_cmd cmd) +{ + u32 val = ice_ptp_tmr_cmd_to_port_reg(hw, cmd); + int err; + + /* Tx case */ + err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_TX_TMR_CMD, val); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write back TX_TMR_CMD, err %d\n", + err); + return err; + } + + /* Rx case */ + err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_RX_TMR_CMD, val); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write back RX_TMR_CMD, err %d\n", + err); + return err; + } + + return 0; +} + +/** + * ice_phy_get_speed_eth56g - Get link speed based on PHY link type + * @li: pointer to link information struct + * + * Return: simplified ETH56G PHY speed + */ +static enum ice_eth56g_link_spd +ice_phy_get_speed_eth56g(struct ice_link_status *li) +{ + u16 speed = ice_get_link_speed_based_on_phy_type(li->phy_type_low, + li->phy_type_high); + + switch (speed) { + case ICE_AQ_LINK_SPEED_1000MB: + return ICE_ETH56G_LNK_SPD_1G; + case ICE_AQ_LINK_SPEED_2500MB: + return ICE_ETH56G_LNK_SPD_2_5G; + case ICE_AQ_LINK_SPEED_10GB: + return ICE_ETH56G_LNK_SPD_10G; + case ICE_AQ_LINK_SPEED_25GB: + return ICE_ETH56G_LNK_SPD_25G; + case ICE_AQ_LINK_SPEED_40GB: + return ICE_ETH56G_LNK_SPD_40G; + case ICE_AQ_LINK_SPEED_50GB: + switch (li->phy_type_low) { + case ICE_PHY_TYPE_LOW_50GBASE_SR: + case ICE_PHY_TYPE_LOW_50GBASE_FR: + case ICE_PHY_TYPE_LOW_50GBASE_LR: + case ICE_PHY_TYPE_LOW_50GBASE_KR_PAM4: + case ICE_PHY_TYPE_LOW_50G_AUI1_AOC_ACC: + case ICE_PHY_TYPE_LOW_50G_AUI1: + return ICE_ETH56G_LNK_SPD_50G; + default: + return ICE_ETH56G_LNK_SPD_50G2; + } + case ICE_AQ_LINK_SPEED_100GB: + if (li->phy_type_high || + li->phy_type_low == ICE_PHY_TYPE_LOW_100GBASE_SR2) + return ICE_ETH56G_LNK_SPD_100G2; + else + return ICE_ETH56G_LNK_SPD_100G; + default: + return ICE_ETH56G_LNK_SPD_1G; + } +} + +/** + * ice_phy_cfg_parpcs_eth56g - Configure TUs per PAR/PCS clock cycle + * @hw: pointer to the HW struct + * @port: port to configure + * + * Configure the number of TUs for the PAR and PCS clocks used as part of the + * timestamp calibration process. + * + * Return: + * * %0 - success + * * %other - PHY read/write failed + */ +static int ice_phy_cfg_parpcs_eth56g(struct ice_hw *hw, u8 port) +{ + u8 port_blk = port & ~(ICE_PORTS_PER_QUAD - 1); + u32 val; + int err; + + err = ice_write_xpcs_reg_eth56g(hw, port, PHY_VENDOR_TXLANE_THRESH, + ICE_ETH56G_NOMINAL_THRESH4); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to read VENDOR_TXLANE_THRESH, status: %d", + err); + return err; + } + + switch (ice_phy_get_speed_eth56g(&hw->port_info->phy.link_info)) { + case ICE_ETH56G_LNK_SPD_1G: + case ICE_ETH56G_LNK_SPD_2_5G: + err = ice_read_ptp_reg_eth56g(hw, port_blk, + PHY_GPCS_CONFIG_REG0, &val); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to read PHY_GPCS_CONFIG_REG0, status: %d", + err); + return err; + } + + val &= ~PHY_GPCS_CONFIG_REG0_TX_THR_M; + val |= FIELD_PREP(PHY_GPCS_CONFIG_REG0_TX_THR_M, + ICE_ETH56G_NOMINAL_TX_THRESH); + + err = ice_write_ptp_reg_eth56g(hw, port_blk, + PHY_GPCS_CONFIG_REG0, val); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write PHY_GPCS_CONFIG_REG0, status: %d", + err); + return err; + } + break; + default: + break; + } + + err = ice_write_40b_ptp_reg_eth56g(hw, port, PHY_PCS_REF_TUS_L, + ICE_ETH56G_NOMINAL_PCS_REF_TUS); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write PHY_PCS_REF_TUS, status: %d", + err); + return err; + } + + err = ice_write_40b_ptp_reg_eth56g(hw, port, PHY_PCS_REF_INC_L, + ICE_ETH56G_NOMINAL_PCS_REF_INC); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to write PHY_PCS_REF_INC, status: %d", + err); + return err; + } + + return 0; +} + +/** + * ice_phy_cfg_ptp_1step_eth56g - Configure 1-step PTP settings + * @hw: Pointer to the HW struct + * @port: Port to configure + * + * Return: + * * %0 - success + * * %other - PHY read/write failed + */ +int ice_phy_cfg_ptp_1step_eth56g(struct ice_hw *hw, u8 port) +{ + u8 port_blk = port & ~(ICE_PORTS_PER_QUAD - 1); + u8 blk_port = port & (ICE_PORTS_PER_QUAD - 1); + bool enable, sfd_ena; + u32 val, peer_delay; + int err; + + enable = hw->ptp.phy.eth56g.onestep_ena; + peer_delay = hw->ptp.phy.eth56g.peer_delay; + sfd_ena = hw->ptp.phy.eth56g.sfd_ena; + + /* PHY_PTP_1STEP_CONFIG */ + err = ice_read_ptp_reg_eth56g(hw, port_blk, PHY_PTP_1STEP_CONFIG, &val); + if (err) + return err; + + if (enable) + val |= blk_port; + else + val &= ~blk_port; + + val &= ~(PHY_PTP_1STEP_T1S_UP64_M | PHY_PTP_1STEP_T1S_DELTA_M); + + err = ice_write_ptp_reg_eth56g(hw, port_blk, PHY_PTP_1STEP_CONFIG, val); + if (err) + return err; + + /* PHY_PTP_1STEP_PEER_DELAY */ + val = FIELD_PREP(PHY_PTP_1STEP_PD_DELAY_M, peer_delay); + if (peer_delay) + val |= PHY_PTP_1STEP_PD_ADD_PD_M; + val |= PHY_PTP_1STEP_PD_DLY_V_M; + err = ice_write_ptp_reg_eth56g(hw, port_blk, + PHY_PTP_1STEP_PEER_DELAY(blk_port), val); + if (err) + return err; + + val &= ~PHY_PTP_1STEP_PD_DLY_V_M; + err = ice_write_ptp_reg_eth56g(hw, port_blk, + PHY_PTP_1STEP_PEER_DELAY(blk_port), val); + if (err) + return err; + + /* PHY_MAC_XIF_MODE */ + err = ice_read_mac_reg_eth56g(hw, port, PHY_MAC_XIF_MODE, &val); + if (err) + return err; + + val &= ~(PHY_MAC_XIF_1STEP_ENA_M | PHY_MAC_XIF_TS_BIN_MODE_M | + PHY_MAC_XIF_TS_SFD_ENA_M | PHY_MAC_XIF_GMII_TS_SEL_M); + + switch (ice_phy_get_speed_eth56g(&hw->port_info->phy.link_info)) { + case ICE_ETH56G_LNK_SPD_1G: + case ICE_ETH56G_LNK_SPD_2_5G: + val |= PHY_MAC_XIF_GMII_TS_SEL_M; + break; + default: + break; + } + + val |= FIELD_PREP(PHY_MAC_XIF_1STEP_ENA_M, enable) | + FIELD_PREP(PHY_MAC_XIF_TS_BIN_MODE_M, enable) | + FIELD_PREP(PHY_MAC_XIF_TS_SFD_ENA_M, sfd_ena); + + return ice_write_mac_reg_eth56g(hw, port, PHY_MAC_XIF_MODE, val); +} + +/** + * mul_u32_u32_fx_q9 - Multiply two u32 fixed point Q9 values + * @a: multiplier value + * @b: multiplicand value + * + * Return: result of multiplication + */ +static u32 mul_u32_u32_fx_q9(u32 a, u32 b) +{ + return (u32)(((u64)a * b) >> ICE_ETH56G_MAC_CFG_FRAC_W); +} + +/** + * add_u32_u32_fx - Add two u32 fixed point values and discard overflow + * @a: first value + * @b: second value + * + * Return: result of addition + */ +static u32 add_u32_u32_fx(u32 a, u32 b) +{ + return lower_32_bits(((u64)a + b)); +} + +/** + * ice_ptp_calc_bitslip_eth56g - Calculate bitslip value + * @hw: pointer to the HW struct + * @port: port to configure + * @bs: bitslip multiplier + * @fc: FC-FEC enabled + * @rs: RS-FEC enabled + * @spd: link speed + * + * Return: calculated bitslip value + */ +static u32 ice_ptp_calc_bitslip_eth56g(struct ice_hw *hw, u8 port, u32 bs, + bool fc, bool rs, + enum ice_eth56g_link_spd spd) +{ + u8 port_offset = port & (ICE_PORTS_PER_QUAD - 1); + u8 port_blk = port & ~(ICE_PORTS_PER_QUAD - 1); + u32 bitslip; + int err; + + if (!bs || rs) + return 0; + + if (spd == ICE_ETH56G_LNK_SPD_1G || spd == ICE_ETH56G_LNK_SPD_2_5G) + err = ice_read_gpcs_reg_eth56g(hw, port, PHY_GPCS_BITSLIP, + &bitslip); + else + err = ice_read_ptp_reg_eth56g(hw, port_blk, + PHY_REG_SD_BIT_SLIP(port_offset), + &bitslip); + if (err) + return 0; + + if (spd == ICE_ETH56G_LNK_SPD_1G && !bitslip) { + /* Bitslip register value of 0 corresponds to 10 so substitute + * it for calculations + */ + bitslip = 10; + } else if (spd == ICE_ETH56G_LNK_SPD_10G || + spd == ICE_ETH56G_LNK_SPD_25G) { + if (fc) + bitslip = bitslip * 2 + 32; + else + bitslip = (u32)((s32)bitslip * -1 + 20); + } + + bitslip <<= ICE_ETH56G_MAC_CFG_FRAC_W; + return mul_u32_u32_fx_q9(bitslip, bs); +} + +/** + * ice_ptp_calc_deskew_eth56g - Calculate deskew value + * @hw: pointer to the HW struct + * @port: port to configure + * @ds: deskew multiplier + * @rs: RS-FEC enabled + * @spd: link speed + * + * Return: calculated deskew value + */ +static u32 ice_ptp_calc_deskew_eth56g(struct ice_hw *hw, u8 port, u32 ds, + bool rs, enum ice_eth56g_link_spd spd) +{ + u32 deskew_i, deskew_f; + int err; + + if (!ds) + return 0; + + read_poll_timeout(ice_read_ptp_reg_eth56g, err, + FIELD_GET(PHY_REG_DESKEW_0_VALID, deskew_i), 500, + 50 * USEC_PER_MSEC, false, hw, port, PHY_REG_DESKEW_0, + &deskew_i); + if (err) + return err; + + deskew_f = FIELD_GET(PHY_REG_DESKEW_0_RLEVEL_FRAC, deskew_i); + deskew_i = FIELD_GET(PHY_REG_DESKEW_0_RLEVEL, deskew_i); + + if (rs && spd == ICE_ETH56G_LNK_SPD_50G2) + ds = 0x633; /* 3.1 */ + else if (rs && spd == ICE_ETH56G_LNK_SPD_100G) + ds = 0x31b; /* 1.552 */ + + deskew_i = FIELD_PREP(ICE_ETH56G_MAC_CFG_RX_OFFSET_INT, deskew_i); + /* Shift 3 fractional bits to the end of the integer part */ + deskew_f <<= ICE_ETH56G_MAC_CFG_FRAC_W - PHY_REG_DESKEW_0_RLEVEL_FRAC_W; + return mul_u32_u32_fx_q9(deskew_i | deskew_f, ds); +} + +/** + * ice_phy_set_offsets_eth56g - Set Tx/Rx offset values + * @hw: pointer to the HW struct + * @port: port to configure + * @spd: link speed + * @cfg: structure to store output values + * @fc: FC-FEC enabled + * @rs: RS-FEC enabled + * + * Return: + * * %0 - success + * * %other - failed to write to PHY + */ +static int ice_phy_set_offsets_eth56g(struct ice_hw *hw, u8 port, + enum ice_eth56g_link_spd spd, + const struct ice_eth56g_mac_reg_cfg *cfg, + bool fc, bool rs) +{ + u32 rx_offset, tx_offset, bs_ds; + bool onestep, sfd; + + onestep = hw->ptp.phy.eth56g.onestep_ena; + sfd = hw->ptp.phy.eth56g.sfd_ena; + bs_ds = cfg->rx_offset.bs_ds; + + if (fc) + rx_offset = cfg->rx_offset.fc; + else if (rs) + rx_offset = cfg->rx_offset.rs; + else + rx_offset = cfg->rx_offset.no_fec; + + rx_offset = add_u32_u32_fx(rx_offset, cfg->rx_offset.serdes); + if (sfd) + rx_offset = add_u32_u32_fx(rx_offset, cfg->rx_offset.sfd); + + if (spd < ICE_ETH56G_LNK_SPD_40G) + bs_ds = ice_ptp_calc_bitslip_eth56g(hw, port, bs_ds, fc, rs, + spd); + else + bs_ds = ice_ptp_calc_deskew_eth56g(hw, port, bs_ds, rs, spd); + rx_offset = add_u32_u32_fx(rx_offset, bs_ds); + rx_offset &= ICE_ETH56G_MAC_CFG_RX_OFFSET_INT | + ICE_ETH56G_MAC_CFG_RX_OFFSET_FRAC; + + if (fc) + tx_offset = cfg->tx_offset.fc; + else if (rs) + tx_offset = cfg->tx_offset.rs; + else + tx_offset = cfg->tx_offset.no_fec; + tx_offset += cfg->tx_offset.serdes + cfg->tx_offset.sfd * sfd + + cfg->tx_offset.onestep * onestep; + + ice_write_mac_reg_eth56g(hw, port, PHY_MAC_RX_OFFSET, rx_offset); + return ice_write_mac_reg_eth56g(hw, port, PHY_MAC_TX_OFFSET, tx_offset); +} + +/** + * ice_phy_cfg_mac_eth56g - Configure MAC for PTP + * @hw: Pointer to the HW struct + * @port: Port to configure + * + * Return: + * * %0 - success + * * %other - failed to write to PHY + */ +static int ice_phy_cfg_mac_eth56g(struct ice_hw *hw, u8 port) +{ + const struct ice_eth56g_mac_reg_cfg *cfg; + enum ice_eth56g_link_spd spd; + struct ice_link_status *li; + bool fc = false; + bool rs = false; + bool onestep; + u32 val; + int err; + + onestep = hw->ptp.phy.eth56g.onestep_ena; + li = &hw->port_info->phy.link_info; + spd = ice_phy_get_speed_eth56g(li); + if (!!(li->an_info & ICE_AQ_FEC_EN)) { + if (spd == ICE_ETH56G_LNK_SPD_10G) { + fc = true; + } else { + fc = !!(li->fec_info & ICE_AQ_LINK_25G_KR_FEC_EN); + rs = !!(li->fec_info & ~ICE_AQ_LINK_25G_KR_FEC_EN); + } + } + cfg = ð56g_mac_cfg[spd]; + + err = ice_write_mac_reg_eth56g(hw, port, PHY_MAC_RX_MODULO, 0); + if (err) + return err; + + err = ice_write_mac_reg_eth56g(hw, port, PHY_MAC_TX_MODULO, 0); + if (err) + return err; + + val = FIELD_PREP(PHY_MAC_TSU_CFG_TX_MODE_M, + cfg->tx_mode.def + rs * cfg->tx_mode.rs) | + FIELD_PREP(PHY_MAC_TSU_CFG_TX_MII_MK_DLY_M, cfg->tx_mk_dly) | + FIELD_PREP(PHY_MAC_TSU_CFG_TX_MII_CW_DLY_M, + cfg->tx_cw_dly.def + + onestep * cfg->tx_cw_dly.onestep) | + FIELD_PREP(PHY_MAC_TSU_CFG_RX_MODE_M, + cfg->rx_mode.def + rs * cfg->rx_mode.rs) | + FIELD_PREP(PHY_MAC_TSU_CFG_RX_MII_MK_DLY_M, + cfg->rx_mk_dly.def + rs * cfg->rx_mk_dly.rs) | + FIELD_PREP(PHY_MAC_TSU_CFG_RX_MII_CW_DLY_M, + cfg->rx_cw_dly.def + rs * cfg->rx_cw_dly.rs) | + FIELD_PREP(PHY_MAC_TSU_CFG_BLKS_PER_CLK_M, cfg->blks_per_clk); + err = ice_write_mac_reg_eth56g(hw, port, PHY_MAC_TSU_CONFIG, val); + if (err) + return err; + + err = ice_write_mac_reg_eth56g(hw, port, PHY_MAC_BLOCKTIME, + cfg->blktime); + if (err) + return err; + + err = ice_phy_set_offsets_eth56g(hw, port, spd, cfg, fc, rs); + if (err) + return err; + + if (spd == ICE_ETH56G_LNK_SPD_25G && !rs) + val = 0; + else + val = cfg->mktime; + + return ice_write_mac_reg_eth56g(hw, port, PHY_MAC_MARKERTIME, val); +} + +/** + * ice_phy_cfg_intr_eth56g - Configure TX timestamp interrupt + * @hw: pointer to the HW struct + * @port: the timestamp port + * @ena: enable or disable interrupt + * @threshold: interrupt threshold + * + * Configure TX timestamp interrupt for the specified port + * + * Return: + * * %0 - success + * * %other - PHY read/write failed + */ +int ice_phy_cfg_intr_eth56g(struct ice_hw *hw, u8 port, bool ena, u8 threshold) +{ + int err; + u32 val; + + err = ice_read_ptp_reg_eth56g(hw, port, PHY_REG_TS_INT_CONFIG, &val); + if (err) + return err; + + if (ena) { + val |= PHY_TS_INT_CONFIG_ENA_M; + val &= ~PHY_TS_INT_CONFIG_THRESHOLD_M; + val |= FIELD_PREP(PHY_TS_INT_CONFIG_THRESHOLD_M, threshold); + } else { + val &= ~PHY_TS_INT_CONFIG_ENA_M; + } + + return ice_write_ptp_reg_eth56g(hw, port, PHY_REG_TS_INT_CONFIG, val); +} + +/** + * ice_read_phy_and_phc_time_eth56g - Simultaneously capture PHC and PHY time + * @hw: pointer to the HW struct + * @port: the PHY port to read + * @phy_time: on return, the 64bit PHY timer value + * @phc_time: on return, the lower 64bits of PHC time + * + * Issue a ICE_PTP_READ_TIME timer command to simultaneously capture the PHY + * and PHC timer values. + * + * Return: + * * %0 - success + * * %other - PHY read/write failed + */ +static int ice_read_phy_and_phc_time_eth56g(struct ice_hw *hw, u8 port, + u64 *phy_time, u64 *phc_time) +{ + u64 tx_time, rx_time; + u32 zo, lo; + u8 tmr_idx; + int err; + + tmr_idx = ice_get_ptp_src_clock_index(hw); + + /* Prepare the PHC timer for a ICE_PTP_READ_TIME capture command */ + ice_ptp_src_cmd(hw, ICE_PTP_READ_TIME); + + /* Prepare the PHY timer for a ICE_PTP_READ_TIME capture command */ + err = ice_ptp_one_port_cmd(hw, port, ICE_PTP_READ_TIME); + if (err) + return err; + + /* Issue the sync to start the ICE_PTP_READ_TIME capture */ + ice_ptp_exec_tmr_cmd(hw); + + /* Read the captured PHC time from the shadow time registers */ + zo = rd32(hw, GLTSYN_SHTIME_0(tmr_idx)); + lo = rd32(hw, GLTSYN_SHTIME_L(tmr_idx)); + *phc_time = (u64)lo << 32 | zo; + + /* Read the captured PHY time from the PHY shadow registers */ + err = ice_ptp_read_port_capture_eth56g(hw, port, &tx_time, &rx_time); + if (err) + return err; + + /* If the PHY Tx and Rx timers don't match, log a warning message. + * Note that this should not happen in normal circumstances since the + * driver always programs them together. + */ + if (tx_time != rx_time) + dev_warn(ice_hw_to_dev(hw), "PHY port %u Tx and Rx timers do not match, tx_time 0x%016llX, rx_time 0x%016llX\n", + port, tx_time, rx_time); + + *phy_time = tx_time; + + return 0; +} + +/** + * ice_sync_phy_timer_eth56g - Synchronize the PHY timer with PHC timer + * @hw: pointer to the HW struct + * @port: the PHY port to synchronize + * + * Perform an adjustment to ensure that the PHY and PHC timers are in sync. + * This is done by issuing a ICE_PTP_READ_TIME command which triggers a + * simultaneous read of the PHY timer and PHC timer. Then we use the + * difference to calculate an appropriate 2s complement addition to add + * to the PHY timer in order to ensure it reads the same value as the + * primary PHC timer. + * + * Return: + * * %0 - success + * * %-EBUSY- failed to acquire PTP semaphore + * * %other - PHY read/write failed */ -void ice_ptp_src_cmd(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd) +static int ice_sync_phy_timer_eth56g(struct ice_hw *hw, u8 port) +{ + u64 phc_time, phy_time, difference; + int err; + + if (!ice_ptp_lock(hw)) { + ice_debug(hw, ICE_DBG_PTP, "Failed to acquire PTP semaphore\n"); + return -EBUSY; + } + + err = ice_read_phy_and_phc_time_eth56g(hw, port, &phy_time, &phc_time); + if (err) + goto err_unlock; + + /* Calculate the amount required to add to the port time in order for + * it to match the PHC time. + * + * Note that the port adjustment is done using 2s complement + * arithmetic. This is convenient since it means that we can simply + * calculate the difference between the PHC time and the port time, + * and it will be interpreted correctly. + */ + + ice_ptp_src_cmd(hw, ICE_PTP_NOP); + difference = phc_time - phy_time; + + err = ice_ptp_prep_port_adj_eth56g(hw, port, (s64)difference); + if (err) + goto err_unlock; + + err = ice_ptp_one_port_cmd(hw, port, ICE_PTP_ADJ_TIME); + if (err) + goto err_unlock; + + /* Issue the sync to activate the time adjustment */ + ice_ptp_exec_tmr_cmd(hw); + + /* Re-capture the timer values to flush the command registers and + * verify that the time was properly adjusted. + */ + err = ice_read_phy_and_phc_time_eth56g(hw, port, &phy_time, &phc_time); + if (err) + goto err_unlock; + + dev_info(ice_hw_to_dev(hw), + "Port %u PHY time synced to PHC: 0x%016llX, 0x%016llX\n", + port, phy_time, phc_time); + +err_unlock: + ice_ptp_unlock(hw); + return err; +} + +/** + * ice_stop_phy_timer_eth56g - Stop the PHY clock timer + * @hw: pointer to the HW struct + * @port: the PHY port to stop + * @soft_reset: if true, hold the SOFT_RESET bit of PHY_REG_PS + * + * Stop the clock of a PHY port. This must be done as part of the flow to + * re-calibrate Tx and Rx timestamping offsets whenever the clock time is + * initialized or when link speed changes. + * + * Return: + * * %0 - success + * * %other - failed to write to PHY + */ +int ice_stop_phy_timer_eth56g(struct ice_hw *hw, u8 port, bool soft_reset) +{ + int err; + + err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_TX_OFFSET_READY, 0); + if (err) + return err; + + err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_RX_OFFSET_READY, 0); + if (err) + return err; + + ice_debug(hw, ICE_DBG_PTP, "Disabled clock on PHY port %u\n", port); + + return 0; +} + +/** + * ice_start_phy_timer_eth56g - Start the PHY clock timer + * @hw: pointer to the HW struct + * @port: the PHY port to start + * + * Start the clock of a PHY port. This must be done as part of the flow to + * re-calibrate Tx and Rx timestamping offsets whenever the clock time is + * initialized or when link speed changes. + * + * Return: + * * %0 - success + * * %other - PHY read/write failed + */ +int ice_start_phy_timer_eth56g(struct ice_hw *hw, u8 port) { - u32 cmd_val; + u32 lo, hi; + u64 incval; u8 tmr_idx; + int err; tmr_idx = ice_get_ptp_src_clock_index(hw); - cmd_val = tmr_idx << SEL_CPK_SRC; - switch (cmd) { - case ICE_PTP_INIT_TIME: - cmd_val |= GLTSYN_CMD_INIT_TIME; - break; - case ICE_PTP_INIT_INCVAL: - cmd_val |= GLTSYN_CMD_INIT_INCVAL; - break; - case ICE_PTP_ADJ_TIME: - cmd_val |= GLTSYN_CMD_ADJ_TIME; - break; - case ICE_PTP_ADJ_TIME_AT_TIME: - cmd_val |= GLTSYN_CMD_ADJ_INIT_TIME; - break; - case ICE_PTP_READ_TIME: - cmd_val |= GLTSYN_CMD_READ_TIME; - break; - case ICE_PTP_NOP: - break; - } + err = ice_stop_phy_timer_eth56g(hw, port, false); + if (err) + return err; - wr32(hw, GLTSYN_CMD, cmd_val); + ice_ptp_src_cmd(hw, ICE_PTP_NOP); + + err = ice_phy_cfg_parpcs_eth56g(hw, port); + if (err) + return err; + + err = ice_phy_cfg_ptp_1step_eth56g(hw, port); + if (err) + return err; + + err = ice_phy_cfg_mac_eth56g(hw, port); + if (err) + return err; + + lo = rd32(hw, GLTSYN_INCVAL_L(tmr_idx)); + hi = rd32(hw, GLTSYN_INCVAL_H(tmr_idx)); + incval = (u64)hi << 32 | lo; + + err = ice_write_40b_ptp_reg_eth56g(hw, port, PHY_REG_TIMETUS_L, incval); + if (err) + return err; + + err = ice_ptp_one_port_cmd(hw, port, ICE_PTP_INIT_INCVAL); + if (err) + return err; + + ice_ptp_exec_tmr_cmd(hw); + + err = ice_sync_phy_timer_eth56g(hw, port); + if (err) + return err; + + err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_TX_OFFSET_READY, 1); + if (err) + return err; + + err = ice_write_ptp_reg_eth56g(hw, port, PHY_REG_RX_OFFSET_READY, 1); + if (err) + return err; + + ice_debug(hw, ICE_DBG_PTP, "Enabled clock on PHY port %u\n", port); + + return 0; } /** - * ice_ptp_exec_tmr_cmd - Execute all prepared timer commands + * ice_sb_access_ena_eth56g - Enable SB devices (PHY and others) access * @hw: pointer to HW struct + * @enable: Enable or disable access * - * Write the SYNC_EXEC_CMD bit to the GLTSYN_CMD_SYNC register, and flush the - * write immediately. This triggers the hardware to begin executing all of the - * source and PHY timer commands synchronously. + * Enable sideband devices (PHY and others) access. + */ +static void ice_sb_access_ena_eth56g(struct ice_hw *hw, bool enable) +{ + u32 val = rd32(hw, PF_SB_REM_DEV_CTL); + + if (enable) + val |= BIT(eth56g_phy_0) | BIT(cgu) | BIT(eth56g_phy_1); + else + val &= ~(BIT(eth56g_phy_0) | BIT(cgu) | BIT(eth56g_phy_1)); + + wr32(hw, PF_SB_REM_DEV_CTL, val); +} + +/** + * ice_ptp_init_phc_eth56g - Perform E82X specific PHC initialization + * @hw: pointer to HW struct + * + * Perform PHC initialization steps specific to E82X devices. + * + * Return: + * * %0 - success + * * %other - failed to initialize CGU + */ +static int ice_ptp_init_phc_eth56g(struct ice_hw *hw) +{ + ice_sb_access_ena_eth56g(hw, true); + /* Initialize the Clock Generation Unit */ + return ice_init_cgu_e82x(hw); +} + +/** + * ice_ptp_read_tx_hwtstamp_status_eth56g - Get TX timestamp status + * @hw: pointer to the HW struct + * @ts_status: the timestamp mask pointer + * + * Read the PHY Tx timestamp status mask indicating which ports have Tx + * timestamps available. + * + * Return: + * * %0 - success + * * %other - failed to read from PHY + */ +int ice_ptp_read_tx_hwtstamp_status_eth56g(struct ice_hw *hw, u32 *ts_status) +{ + const struct ice_eth56g_params *params = &hw->ptp.phy.eth56g; + u8 phy, mask; + u32 status; + + mask = (1 << hw->ptp.ports_per_phy) - 1; + *ts_status = 0; + + for (phy = 0; phy < params->num_phys; phy++) { + int err; + + err = ice_read_phy_eth56g(hw, phy, PHY_PTP_INT_STATUS, &status); + if (err) + return err; + + *ts_status |= (status & mask) << (phy * hw->ptp.ports_per_phy); + } + + ice_debug(hw, ICE_DBG_PTP, "PHY interrupt err: %x\n", *ts_status); + + return 0; +} + +/** + * ice_get_phy_tx_tstamp_ready_eth56g - Read the Tx memory status register + * @hw: pointer to the HW struct + * @port: the PHY port to read from + * @tstamp_ready: contents of the Tx memory status register + * + * Read the PHY_REG_TX_MEMORY_STATUS register indicating which timestamps in + * the PHY are ready. A set bit means the corresponding timestamp is valid and + * ready to be captured from the PHY timestamp block. + * + * Return: + * * %0 - success + * * %other - failed to read from PHY + */ +static int ice_get_phy_tx_tstamp_ready_eth56g(struct ice_hw *hw, u8 port, + u64 *tstamp_ready) +{ + int err; + + err = ice_read_64b_ptp_reg_eth56g(hw, port, PHY_REG_TX_MEMORY_STATUS_L, + tstamp_ready); + if (err) { + ice_debug(hw, ICE_DBG_PTP, "Failed to read TX_MEMORY_STATUS for port %u, err %d\n", + port, err); + return err; + } + + return 0; +} + +/** + * ice_is_muxed_topo - detect breakout 2x50G topology for E825C + * @hw: pointer to the HW struct + * + * Return: true if it's 2x50 breakout topology, false otherwise + */ +static bool ice_is_muxed_topo(struct ice_hw *hw) +{ + u8 link_topo; + bool mux; + u32 val; + + val = rd32(hw, GLGEN_SWITCH_MODE_CONFIG); + mux = FIELD_GET(GLGEN_SWITCH_MODE_CONFIG_25X4_QUAD_M, val); + val = rd32(hw, GLGEN_MAC_LINK_TOPO); + link_topo = FIELD_GET(GLGEN_MAC_LINK_TOPO_LINK_TOPO_M, val); + + return (mux && link_topo == ICE_LINK_TOPO_UP_TO_2_LINKS); +} + +/** + * ice_ptp_init_phy_e825c - initialize PHY parameters + * @hw: pointer to the HW struct */ -static void ice_ptp_exec_tmr_cmd(struct ice_hw *hw) +static void ice_ptp_init_phy_e825c(struct ice_hw *hw) { - struct ice_pf *pf = container_of(hw, struct ice_pf, hw); + struct ice_ptp_hw *ptp = &hw->ptp; + struct ice_eth56g_params *params; + u8 phy; + + ptp->phy_model = ICE_PHY_ETH56G; + params = &ptp->phy.eth56g; + params->onestep_ena = false; + params->peer_delay = 0; + params->sfd_ena = false; + params->phy_addr[0] = eth56g_phy_0; + params->phy_addr[1] = eth56g_phy_1; + params->num_phys = 2; + ptp->ports_per_phy = 4; + ptp->num_lports = params->num_phys * ptp->ports_per_phy; + + ice_sb_access_ena_eth56g(hw, true); + for (phy = 0; phy < params->num_phys; phy++) { + u32 phy_rev; + int err; - guard(spinlock)(&pf->adapter->ptp_gltsyn_time_lock); - wr32(hw, GLTSYN_CMD_SYNC, SYNC_EXEC_CMD); - ice_flush(hw); + err = ice_read_phy_eth56g(hw, phy, PHY_REG_REVISION, &phy_rev); + if (err || phy_rev != PHY_REVISION_ETH56G) { + ptp->phy_model = ICE_PHY_UNSUP; + return; + } + } + + ptp->is_2x50g_muxed_topo = ice_is_muxed_topo(hw); } /* E822 family functions @@ -288,18 +2707,21 @@ static void ice_ptp_exec_tmr_cmd(struct ice_hw *hw) /** * ice_fill_phy_msg_e82x - Fill message data for a PHY register access + * @hw: pointer to the HW struct * @msg: the PHY message buffer to fill in * @port: the port to access * @offset: the register offset */ -static void -ice_fill_phy_msg_e82x(struct ice_sbq_msg_input *msg, u8 port, u16 offset) +static void ice_fill_phy_msg_e82x(struct ice_hw *hw, + struct ice_sbq_msg_input *msg, u8 port, + u16 offset) { int phy_port, phy, quadtype; - phy_port = port % ICE_PORTS_PER_PHY_E82X; - phy = port / ICE_PORTS_PER_PHY_E82X; - quadtype = (port / ICE_PORTS_PER_QUAD) % ICE_QUADS_PER_PHY_E82X; + phy_port = port % hw->ptp.ports_per_phy; + phy = port / hw->ptp.ports_per_phy; + quadtype = ICE_GET_QUAD_NUM(port) % + ICE_GET_QUAD_NUM(hw->ptp.ports_per_phy); if (quadtype == 0) { msg->msg_addr_low = P_Q0_L(P_0_BASE + offset, phy_port); @@ -430,7 +2852,7 @@ ice_read_phy_reg_e82x(struct ice_hw *hw, u8 port, u16 offset, u32 *val) struct ice_sbq_msg_input msg = {0}; int err; - ice_fill_phy_msg_e82x(&msg, port, offset); + ice_fill_phy_msg_e82x(hw, &msg, port, offset); msg.opcode = ice_sbq_msg_rd; err = ice_sbq_rw_reg(hw, &msg); @@ -507,7 +2929,7 @@ ice_write_phy_reg_e82x(struct ice_hw *hw, u8 port, u16 offset, u32 val) struct ice_sbq_msg_input msg = {0}; int err; - ice_fill_phy_msg_e82x(&msg, port, offset); + ice_fill_phy_msg_e82x(hw, &msg, port, offset); msg.opcode = ice_sbq_msg_wr; msg.data = val; @@ -546,8 +2968,7 @@ ice_write_40b_phy_reg_e82x(struct ice_hw *hw, u8 port, u16 low_addr, u64 val) low_addr); return -EINVAL; } - - low = (u32)(val & P_REG_40B_LOW_M); + low = FIELD_GET(P_REG_40B_LOW_M, val); high = (u32)(val >> P_REG_40B_HIGH_S); err = ice_write_phy_reg_e82x(hw, port, low_addr, low); @@ -617,24 +3038,30 @@ ice_write_64b_phy_reg_e82x(struct ice_hw *hw, u8 port, u16 low_addr, u64 val) /** * ice_fill_quad_msg_e82x - Fill message data for quad register access + * @hw: pointer to the HW struct * @msg: the PHY message buffer to fill in * @quad: the quad to access * @offset: the register offset * * Fill a message buffer for accessing a register in a quad shared between * multiple PHYs. + * + * Return: + * * %0 - OK + * * %-EINVAL - invalid quad number */ -static int -ice_fill_quad_msg_e82x(struct ice_sbq_msg_input *msg, u8 quad, u16 offset) +static int ice_fill_quad_msg_e82x(struct ice_hw *hw, + struct ice_sbq_msg_input *msg, u8 quad, + u16 offset) { u32 addr; - if (quad >= ICE_MAX_QUAD) + if (quad >= ICE_GET_QUAD_NUM(hw->ptp.num_lports)) return -EINVAL; msg->dest_dev = rmn_0; - if ((quad % ICE_QUADS_PER_PHY_E82X) == 0) + if (!(quad % ICE_GET_QUAD_NUM(hw->ptp.ports_per_phy))) addr = Q_0_BASE + offset; else addr = Q_1_BASE + offset; @@ -661,7 +3088,7 @@ ice_read_quad_reg_e82x(struct ice_hw *hw, u8 quad, u16 offset, u32 *val) struct ice_sbq_msg_input msg = {0}; int err; - err = ice_fill_quad_msg_e82x(&msg, quad, offset); + err = ice_fill_quad_msg_e82x(hw, &msg, quad, offset); if (err) return err; @@ -695,7 +3122,7 @@ ice_write_quad_reg_e82x(struct ice_hw *hw, u8 quad, u16 offset, u32 val) struct ice_sbq_msg_input msg = {0}; int err; - err = ice_fill_quad_msg_e82x(&msg, quad, offset); + err = ice_fill_quad_msg_e82x(hw, &msg, quad, offset); if (err) return err; @@ -751,7 +3178,7 @@ ice_read_phy_tstamp_e82x(struct ice_hw *hw, u8 quad, u8 idx, u64 *tstamp) * lower 8 bits in the low register, and the upper 32 bits in the high * register. */ - *tstamp = ((u64)hi) << TS_PHY_HIGH_S | ((u64)lo & TS_PHY_LOW_M); + *tstamp = FIELD_PREP(TS_PHY_HIGH_M, hi) | FIELD_PREP(TS_PHY_LOW_M, lo); return 0; } @@ -805,302 +3232,19 @@ ice_clear_phy_tstamp_e82x(struct ice_hw *hw, u8 quad, u8 idx) void ice_ptp_reset_ts_memory_quad_e82x(struct ice_hw *hw, u8 quad) { ice_write_quad_reg_e82x(hw, quad, Q_REG_TS_CTRL, Q_REG_TS_CTRL_M); - ice_write_quad_reg_e82x(hw, quad, Q_REG_TS_CTRL, ~(u32)Q_REG_TS_CTRL_M); -} - -/** - * ice_ptp_reset_ts_memory_e82x - Clear all timestamps from all quad blocks - * @hw: pointer to the HW struct - */ -static void ice_ptp_reset_ts_memory_e82x(struct ice_hw *hw) -{ - unsigned int quad; - - for (quad = 0; quad < ICE_MAX_QUAD; quad++) - ice_ptp_reset_ts_memory_quad_e82x(hw, quad); -} - -/** - * ice_read_cgu_reg_e82x - Read a CGU register - * @hw: pointer to the HW struct - * @addr: Register address to read - * @val: storage for register value read - * - * Read the contents of a register of the Clock Generation Unit. Only - * applicable to E822 devices. - */ -static int -ice_read_cgu_reg_e82x(struct ice_hw *hw, u32 addr, u32 *val) -{ - struct ice_sbq_msg_input cgu_msg; - int err; - - cgu_msg.opcode = ice_sbq_msg_rd; - cgu_msg.dest_dev = cgu; - cgu_msg.msg_addr_low = addr; - cgu_msg.msg_addr_high = 0x0; - - err = ice_sbq_rw_reg(hw, &cgu_msg); - if (err) { - ice_debug(hw, ICE_DBG_PTP, "Failed to read CGU register 0x%04x, err %d\n", - addr, err); - return err; - } - - *val = cgu_msg.data; - - return err; -} - -/** - * ice_write_cgu_reg_e82x - Write a CGU register - * @hw: pointer to the HW struct - * @addr: Register address to write - * @val: value to write into the register - * - * Write the specified value to a register of the Clock Generation Unit. Only - * applicable to E822 devices. - */ -static int -ice_write_cgu_reg_e82x(struct ice_hw *hw, u32 addr, u32 val) -{ - struct ice_sbq_msg_input cgu_msg; - int err; - - cgu_msg.opcode = ice_sbq_msg_wr; - cgu_msg.dest_dev = cgu; - cgu_msg.msg_addr_low = addr; - cgu_msg.msg_addr_high = 0x0; - cgu_msg.data = val; - - err = ice_sbq_rw_reg(hw, &cgu_msg); - if (err) { - ice_debug(hw, ICE_DBG_PTP, "Failed to write CGU register 0x%04x, err %d\n", - addr, err); - return err; - } - - return err; -} - -/** - * ice_clk_freq_str - Convert time_ref_freq to string - * @clk_freq: Clock frequency - * - * Convert the specified TIME_REF clock frequency to a string. - */ -static const char *ice_clk_freq_str(u8 clk_freq) -{ - switch ((enum ice_time_ref_freq)clk_freq) { - case ICE_TIME_REF_FREQ_25_000: - return "25 MHz"; - case ICE_TIME_REF_FREQ_122_880: - return "122.88 MHz"; - case ICE_TIME_REF_FREQ_125_000: - return "125 MHz"; - case ICE_TIME_REF_FREQ_153_600: - return "153.6 MHz"; - case ICE_TIME_REF_FREQ_156_250: - return "156.25 MHz"; - case ICE_TIME_REF_FREQ_245_760: - return "245.76 MHz"; - default: - return "Unknown"; - } -} - -/** - * ice_clk_src_str - Convert time_ref_src to string - * @clk_src: Clock source - * - * Convert the specified clock source to its string name. - */ -static const char *ice_clk_src_str(u8 clk_src) -{ - switch ((enum ice_clk_src)clk_src) { - case ICE_CLK_SRC_TCX0: - return "TCX0"; - case ICE_CLK_SRC_TIME_REF: - return "TIME_REF"; - default: - return "Unknown"; - } -} - -/** - * ice_cfg_cgu_pll_e82x - Configure the Clock Generation Unit - * @hw: pointer to the HW struct - * @clk_freq: Clock frequency to program - * @clk_src: Clock source to select (TIME_REF, or TCX0) - * - * Configure the Clock Generation Unit with the desired clock frequency and - * time reference, enabling the PLL which drives the PTP hardware clock. - */ -static int -ice_cfg_cgu_pll_e82x(struct ice_hw *hw, enum ice_time_ref_freq clk_freq, - enum ice_clk_src clk_src) -{ - union tspll_ro_bwm_lf bwm_lf; - union nac_cgu_dword19 dw19; - union nac_cgu_dword22 dw22; - union nac_cgu_dword24 dw24; - union nac_cgu_dword9 dw9; - int err; - - if (clk_freq >= NUM_ICE_TIME_REF_FREQ) { - dev_warn(ice_hw_to_dev(hw), "Invalid TIME_REF frequency %u\n", - clk_freq); - return -EINVAL; - } - - if (clk_src >= NUM_ICE_CLK_SRC) { - dev_warn(ice_hw_to_dev(hw), "Invalid clock source %u\n", - clk_src); - return -EINVAL; - } - - if (clk_src == ICE_CLK_SRC_TCX0 && - clk_freq != ICE_TIME_REF_FREQ_25_000) { - dev_warn(ice_hw_to_dev(hw), - "TCX0 only supports 25 MHz frequency\n"); - return -EINVAL; - } - - err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD9, &dw9.val); - if (err) - return err; - - err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD24, &dw24.val); - if (err) - return err; - - err = ice_read_cgu_reg_e82x(hw, TSPLL_RO_BWM_LF, &bwm_lf.val); - if (err) - return err; - - /* Log the current clock configuration */ - ice_debug(hw, ICE_DBG_PTP, "Current CGU configuration -- %s, clk_src %s, clk_freq %s, PLL %s\n", - dw24.field.ts_pll_enable ? "enabled" : "disabled", - ice_clk_src_str(dw24.field.time_ref_sel), - ice_clk_freq_str(dw9.field.time_ref_freq_sel), - bwm_lf.field.plllock_true_lock_cri ? "locked" : "unlocked"); - - /* Disable the PLL before changing the clock source or frequency */ - if (dw24.field.ts_pll_enable) { - dw24.field.ts_pll_enable = 0; - - err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD24, dw24.val); - if (err) - return err; - } - - /* Set the frequency */ - dw9.field.time_ref_freq_sel = clk_freq; - err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD9, dw9.val); - if (err) - return err; - - /* Configure the TS PLL feedback divisor */ - err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD19, &dw19.val); - if (err) - return err; - - dw19.field.tspll_fbdiv_intgr = e822_cgu_params[clk_freq].feedback_div; - dw19.field.tspll_ndivratio = 1; - - err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD19, dw19.val); - if (err) - return err; - - /* Configure the TS PLL post divisor */ - err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD22, &dw22.val); - if (err) - return err; - - dw22.field.time1588clk_div = e822_cgu_params[clk_freq].post_pll_div; - dw22.field.time1588clk_sel_div2 = 0; - - err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD22, dw22.val); - if (err) - return err; - - /* Configure the TS PLL pre divisor and clock source */ - err = ice_read_cgu_reg_e82x(hw, NAC_CGU_DWORD24, &dw24.val); - if (err) - return err; - - dw24.field.ref1588_ck_div = e822_cgu_params[clk_freq].refclk_pre_div; - dw24.field.tspll_fbdiv_frac = e822_cgu_params[clk_freq].frac_n_div; - dw24.field.time_ref_sel = clk_src; - - err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD24, dw24.val); - if (err) - return err; - - /* Finally, enable the PLL */ - dw24.field.ts_pll_enable = 1; - - err = ice_write_cgu_reg_e82x(hw, NAC_CGU_DWORD24, dw24.val); - if (err) - return err; - - /* Wait to verify if the PLL locks */ - usleep_range(1000, 5000); - - err = ice_read_cgu_reg_e82x(hw, TSPLL_RO_BWM_LF, &bwm_lf.val); - if (err) - return err; - - if (!bwm_lf.field.plllock_true_lock_cri) { - dev_warn(ice_hw_to_dev(hw), "CGU PLL failed to lock\n"); - return -EBUSY; - } - - /* Log the current clock configuration */ - ice_debug(hw, ICE_DBG_PTP, "New CGU configuration -- %s, clk_src %s, clk_freq %s, PLL %s\n", - dw24.field.ts_pll_enable ? "enabled" : "disabled", - ice_clk_src_str(dw24.field.time_ref_sel), - ice_clk_freq_str(dw9.field.time_ref_freq_sel), - bwm_lf.field.plllock_true_lock_cri ? "locked" : "unlocked"); - - return 0; -} - -/** - * ice_init_cgu_e82x - Initialize CGU with settings from firmware - * @hw: pointer to the HW structure - * - * Initialize the Clock Generation Unit of the E822 device. - */ -static int ice_init_cgu_e82x(struct ice_hw *hw) -{ - struct ice_ts_func_info *ts_info = &hw->func_caps.ts_func_info; - union tspll_cntr_bist_settings cntr_bist; - int err; - - err = ice_read_cgu_reg_e82x(hw, TSPLL_CNTR_BIST_SETTINGS, - &cntr_bist.val); - if (err) - return err; - - /* Disable sticky lock detection so lock err reported is accurate */ - cntr_bist.field.i_plllock_sel_0 = 0; - cntr_bist.field.i_plllock_sel_1 = 0; - - err = ice_write_cgu_reg_e82x(hw, TSPLL_CNTR_BIST_SETTINGS, - cntr_bist.val); - if (err) - return err; + ice_write_quad_reg_e82x(hw, quad, Q_REG_TS_CTRL, ~(u32)Q_REG_TS_CTRL_M); +} - /* Configure the CGU PLL using the parameters from the function - * capabilities. - */ - err = ice_cfg_cgu_pll_e82x(hw, ts_info->time_ref, - (enum ice_clk_src)ts_info->clk_src); - if (err) - return err; +/** + * ice_ptp_reset_ts_memory_e82x - Clear all timestamps from all quad blocks + * @hw: pointer to the HW struct + */ +static void ice_ptp_reset_ts_memory_e82x(struct ice_hw *hw) +{ + unsigned int quad; - return 0; + for (quad = 0; quad < ICE_GET_QUAD_NUM(hw->ptp.num_lports); quad++) + ice_ptp_reset_ts_memory_quad_e82x(hw, quad); } /** @@ -1113,7 +3257,7 @@ static int ice_ptp_set_vernier_wl(struct ice_hw *hw) { u8 port; - for (port = 0; port < ICE_NUM_EXTERNAL_PORTS; port++) { + for (port = 0; port < hw->ptp.num_lports; port++) { int err; err = ice_write_phy_reg_e82x(hw, port, P_REG_WL, @@ -1137,15 +3281,14 @@ static int ice_ptp_set_vernier_wl(struct ice_hw *hw) static int ice_ptp_init_phc_e82x(struct ice_hw *hw) { int err; - u32 regval; + u32 val; /* Enable reading switch and PHY registers over the sideband queue */ #define PF_SB_REM_DEV_CTL_SWITCH_READ BIT(1) #define PF_SB_REM_DEV_CTL_PHY0 BIT(2) - regval = rd32(hw, PF_SB_REM_DEV_CTL); - regval |= (PF_SB_REM_DEV_CTL_SWITCH_READ | - PF_SB_REM_DEV_CTL_PHY0); - wr32(hw, PF_SB_REM_DEV_CTL, regval); + val = rd32(hw, PF_SB_REM_DEV_CTL); + val |= (PF_SB_REM_DEV_CTL_SWITCH_READ | PF_SB_REM_DEV_CTL_PHY0); + wr32(hw, PF_SB_REM_DEV_CTL, val); /* Initialize the Clock Generation Unit */ err = ice_init_cgu_e82x(hw); @@ -1178,7 +3321,7 @@ ice_ptp_prep_phy_time_e82x(struct ice_hw *hw, u32 time) */ phy_time = (u64)time << 32; - for (port = 0; port < ICE_NUM_EXTERNAL_PORTS; port++) { + for (port = 0; port < hw->ptp.num_lports; port++) { /* Tx case */ err = ice_write_64b_phy_reg_e82x(hw, port, P_REG_TX_TIMER_INC_PRE_L, @@ -1281,7 +3424,7 @@ ice_ptp_prep_phy_adj_e82x(struct ice_hw *hw, s32 adj) else cycles = -(((s64)-adj) << 32); - for (port = 0; port < ICE_NUM_EXTERNAL_PORTS; port++) { + for (port = 0; port < hw->ptp.num_lports; port++) { int err; err = ice_ptp_prep_port_adj_e82x(hw, port, cycles); @@ -1307,7 +3450,7 @@ ice_ptp_prep_phy_incval_e82x(struct ice_hw *hw, u64 incval) int err; u8 port; - for (port = 0; port < ICE_NUM_EXTERNAL_PORTS; port++) { + for (port = 0; port < hw->ptp.num_lports; port++) { err = ice_write_40b_phy_reg_e82x(hw, port, P_REG_TIMETUS_L, incval); if (err) @@ -1372,51 +3515,20 @@ ice_ptp_read_port_capture(struct ice_hw *hw, u8 port, u64 *tx_ts, u64 *rx_ts) * * Prepare the requested port for an upcoming timer sync command. * - * Do not use this function directly. If you want to configure exactly one - * port, use ice_ptp_one_port_cmd() instead. + * Note there is no equivalent of this operation on E810, as that device + * always handles all external PHYs internally. + * + * Return: + * * %0 - success + * * %other - failed to write to PHY */ static int ice_ptp_write_port_cmd_e82x(struct ice_hw *hw, u8 port, enum ice_ptp_tmr_cmd cmd) { - u32 cmd_val, val; - u8 tmr_idx; + u32 val = ice_ptp_tmr_cmd_to_port_reg(hw, cmd); int err; - tmr_idx = ice_get_ptp_src_clock_index(hw); - cmd_val = tmr_idx << SEL_PHY_SRC; - switch (cmd) { - case ICE_PTP_INIT_TIME: - cmd_val |= PHY_CMD_INIT_TIME; - break; - case ICE_PTP_INIT_INCVAL: - cmd_val |= PHY_CMD_INIT_INCVAL; - break; - case ICE_PTP_ADJ_TIME: - cmd_val |= PHY_CMD_ADJ_TIME; - break; - case ICE_PTP_READ_TIME: - cmd_val |= PHY_CMD_READ_TIME; - break; - case ICE_PTP_ADJ_TIME_AT_TIME: - cmd_val |= PHY_CMD_ADJ_TIME_AT_TIME; - break; - case ICE_PTP_NOP: - break; - } - /* Tx case */ - /* Read, modify, write */ - err = ice_read_phy_reg_e82x(hw, port, P_REG_TX_TMR_CMD, &val); - if (err) { - ice_debug(hw, ICE_DBG_PTP, "Failed to read TX_TMR_CMD, err %d\n", - err); - return err; - } - - /* Modify necessary bits only and perform write */ - val &= ~TS_CMD_MASK; - val |= cmd_val; - err = ice_write_phy_reg_e82x(hw, port, P_REG_TX_TMR_CMD, val); if (err) { ice_debug(hw, ICE_DBG_PTP, "Failed to write back TX_TMR_CMD, err %d\n", @@ -1425,19 +3537,8 @@ static int ice_ptp_write_port_cmd_e82x(struct ice_hw *hw, u8 port, } /* Rx case */ - /* Read, modify, write */ - err = ice_read_phy_reg_e82x(hw, port, P_REG_RX_TMR_CMD, &val); - if (err) { - ice_debug(hw, ICE_DBG_PTP, "Failed to read RX_TMR_CMD, err %d\n", - err); - return err; - } - - /* Modify necessary bits only and perform write */ - val &= ~TS_CMD_MASK; - val |= cmd_val; - - err = ice_write_phy_reg_e82x(hw, port, P_REG_RX_TMR_CMD, val); + err = ice_write_phy_reg_e82x(hw, port, P_REG_RX_TMR_CMD, + val | TS_CMD_RX_TYPE); if (err) { ice_debug(hw, ICE_DBG_PTP, "Failed to write back RX_TMR_CMD, err %d\n", err); @@ -1447,63 +3548,6 @@ static int ice_ptp_write_port_cmd_e82x(struct ice_hw *hw, u8 port, return 0; } -/** - * ice_ptp_one_port_cmd - Prepare one port for a timer command - * @hw: pointer to the HW struct - * @configured_port: the port to configure with configured_cmd - * @configured_cmd: timer command to prepare on the configured_port - * - * Prepare the configured_port for the configured_cmd, and prepare all other - * ports for ICE_PTP_NOP. This causes the configured_port to execute the - * desired command while all other ports perform no operation. - */ -static int -ice_ptp_one_port_cmd(struct ice_hw *hw, u8 configured_port, - enum ice_ptp_tmr_cmd configured_cmd) -{ - u8 port; - - for (port = 0; port < ICE_NUM_EXTERNAL_PORTS; port++) { - enum ice_ptp_tmr_cmd cmd; - int err; - - if (port == configured_port) - cmd = configured_cmd; - else - cmd = ICE_PTP_NOP; - - err = ice_ptp_write_port_cmd_e82x(hw, port, cmd); - if (err) - return err; - } - - return 0; -} - -/** - * ice_ptp_port_cmd_e82x - Prepare all ports for a timer command - * @hw: pointer to the HW struct - * @cmd: timer command to prepare - * - * Prepare all ports connected to this device for an upcoming timer sync - * command. - */ -static int -ice_ptp_port_cmd_e82x(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd) -{ - u8 port; - - for (port = 0; port < ICE_NUM_EXTERNAL_PORTS; port++) { - int err; - - err = ice_ptp_write_port_cmd_e82x(hw, port, cmd); - if (err) - return err; - } - - return 0; -} - /* E822 Vernier calibration functions * * The following functions are used as part of the vernier calibration of @@ -1606,7 +3650,7 @@ static void ice_phy_cfg_lane_e82x(struct ice_hw *hw, u8 port) return; } - quad = port / ICE_PORTS_PER_QUAD; + quad = ICE_GET_QUAD_NUM(port); err = ice_read_quad_reg_e82x(hw, quad, Q_REG_TX_MEM_GBL_CFG, &val); if (err) { @@ -2326,6 +4370,40 @@ int ice_phy_cfg_rx_offset_e82x(struct ice_hw *hw, u8 port) return 0; } +/** + * ice_ptp_clear_phy_offset_ready_e82x - Clear PHY TX_/RX_OFFSET_READY registers + * @hw: pointer to the HW struct + * + * Clear PHY TX_/RX_OFFSET_READY registers, effectively marking all transmitted + * and received timestamps as invalid. + * + * Return: 0 on success, other error codes when failed to write to PHY + */ +int ice_ptp_clear_phy_offset_ready_e82x(struct ice_hw *hw) +{ + u8 port; + + for (port = 0; port < hw->ptp.num_lports; port++) { + int err; + + err = ice_write_phy_reg_e82x(hw, port, P_REG_TX_OR, 0); + if (err) { + dev_warn(ice_hw_to_dev(hw), + "Failed to clear PHY TX_OFFSET_READY register\n"); + return err; + } + + err = ice_write_phy_reg_e82x(hw, port, P_REG_RX_OR, 0); + if (err) { + dev_warn(ice_hw_to_dev(hw), + "Failed to clear PHY RX_OFFSET_READY register\n"); + return err; + } + } + + return 0; +} + /** * ice_read_phy_and_phc_time_e82x - Simultaneously capture PHC and PHY time * @hw: pointer to the HW struct @@ -2636,6 +4714,48 @@ ice_get_phy_tx_tstamp_ready_e82x(struct ice_hw *hw, u8 quad, u64 *tstamp_ready) return 0; } +/** + * ice_phy_cfg_intr_e82x - Configure TX timestamp interrupt + * @hw: pointer to the HW struct + * @quad: the timestamp quad + * @ena: enable or disable interrupt + * @threshold: interrupt threshold + * + * Configure TX timestamp interrupt for the specified quad + * + * Return: 0 on success, other error codes when failed to read/write quad + */ + +int ice_phy_cfg_intr_e82x(struct ice_hw *hw, u8 quad, bool ena, u8 threshold) +{ + int err; + u32 val; + + err = ice_read_quad_reg_e82x(hw, quad, Q_REG_TX_MEM_GBL_CFG, &val); + if (err) + return err; + + val &= ~Q_REG_TX_MEM_GBL_CFG_INTR_ENA_M; + if (ena) { + val |= Q_REG_TX_MEM_GBL_CFG_INTR_ENA_M; + val &= ~Q_REG_TX_MEM_GBL_CFG_INTR_THR_M; + val |= FIELD_PREP(Q_REG_TX_MEM_GBL_CFG_INTR_THR_M, threshold); + } + + return ice_write_quad_reg_e82x(hw, quad, Q_REG_TX_MEM_GBL_CFG, val); +} + +/** + * ice_ptp_init_phy_e82x - initialize PHY parameters + * @ptp: pointer to the PTP HW struct + */ +static void ice_ptp_init_phy_e82x(struct ice_ptp_hw *ptp) +{ + ptp->phy_model = ICE_PHY_E82X; + ptp->num_lports = 8; + ptp->ports_per_phy = 8; +} + /* E810 functions * * The following functions operate on the E810 series devices which use @@ -2863,17 +4983,21 @@ static int ice_clear_phy_tstamp_e810(struct ice_hw *hw, u8 lport, u8 idx) } /** - * ice_ptp_init_phy_e810 - Enable PTP function on the external PHY + * ice_ptp_init_phc_e810 - Perform E810 specific PHC initialization * @hw: pointer to HW struct * - * Enable the timesync PTP functionality for the external PHY connected to - * this function. + * Perform E810-specific PTP hardware clock initialization steps. + * + * Return: 0 on success, other error codes when failed to initialize TimeSync */ -int ice_ptp_init_phy_e810(struct ice_hw *hw) +static int ice_ptp_init_phc_e810(struct ice_hw *hw) { u8 tmr_idx; int err; + /* Ensure synchronization delay is zero */ + wr32(hw, GLTSYN_SYNC_DLAY, 0); + tmr_idx = hw->func_caps.ts_func_info.tmr_index_owned; err = ice_write_phy_reg_e810(hw, ETH_GLTSYN_ENA(tmr_idx), GLTSYN_ENA_TSYN_ENA_M); @@ -2884,21 +5008,6 @@ int ice_ptp_init_phy_e810(struct ice_hw *hw) return err; } -/** - * ice_ptp_init_phc_e810 - Perform E810 specific PHC initialization - * @hw: pointer to HW struct - * - * Perform E810-specific PTP hardware clock initialization steps. - */ -static int ice_ptp_init_phc_e810(struct ice_hw *hw) -{ - /* Ensure synchronization delay is zero */ - wr32(hw, GLTSYN_SYNC_DLAY, 0); - - /* Initialize the PHY */ - return ice_ptp_init_phy_e810(hw); -} - /** * ice_ptp_prep_phy_time_e810 - Prepare PHY port with initial time * @hw: Board private structure @@ -3020,47 +5129,9 @@ static int ice_ptp_prep_phy_incval_e810(struct ice_hw *hw, u64 incval) */ static int ice_ptp_port_cmd_e810(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd) { - u32 cmd_val, val; - int err; - - switch (cmd) { - case ICE_PTP_INIT_TIME: - cmd_val = GLTSYN_CMD_INIT_TIME; - break; - case ICE_PTP_INIT_INCVAL: - cmd_val = GLTSYN_CMD_INIT_INCVAL; - break; - case ICE_PTP_ADJ_TIME: - cmd_val = GLTSYN_CMD_ADJ_TIME; - break; - case ICE_PTP_READ_TIME: - cmd_val = GLTSYN_CMD_READ_TIME; - break; - case ICE_PTP_ADJ_TIME_AT_TIME: - cmd_val = GLTSYN_CMD_ADJ_INIT_TIME; - break; - case ICE_PTP_NOP: - return 0; - } - - /* Read, modify, write */ - err = ice_read_phy_reg_e810(hw, ETH_GLTSYN_CMD, &val); - if (err) { - ice_debug(hw, ICE_DBG_PTP, "Failed to read GLTSYN_CMD, err %d\n", err); - return err; - } + u32 val = ice_ptp_tmr_cmd_to_port_reg(hw, cmd); - /* Modify necessary bits only and perform write */ - val &= ~TS_CMD_MASK_E810; - val |= cmd_val; - - err = ice_write_phy_reg_e810(hw, ETH_GLTSYN_CMD, val); - if (err) { - ice_debug(hw, ICE_DBG_PTP, "Failed to write back GLTSYN_CMD, err %d\n", err); - return err; - } - - return 0; + return ice_write_phy_reg_e810(hw, E810_ETH_GLTSYN_CMD, val); } /** @@ -3242,6 +5313,17 @@ int ice_read_pca9575_reg_e810t(struct ice_hw *hw, u8 offset, u8 *data) return ice_aq_read_i2c(hw, link_topo, 0, addr, 1, data, NULL); } +/** + * ice_ptp_init_phy_e810 - initialize PHY parameters + * @ptp: pointer to the PTP HW struct + */ +static void ice_ptp_init_phy_e810(struct ice_ptp_hw *ptp) +{ + ptp->phy_model = ICE_PHY_E810; + ptp->num_lports = 8; + ptp->ports_per_phy = 4; +} + /* Device agnostic functions * * The following functions implement shared behavior common to both E822 and @@ -3299,18 +5381,126 @@ void ice_ptp_unlock(struct ice_hw *hw) } /** - * ice_ptp_init_phy_model - Initialize hw->phy_model based on device type + * ice_ptp_init_hw - Initialize hw based on device type * @hw: pointer to the HW structure * - * Determine the PHY model for the device, and initialize hw->phy_model + * Determine the PHY model for the device, and initialize hw * for use by other functions. */ -void ice_ptp_init_phy_model(struct ice_hw *hw) +void ice_ptp_init_hw(struct ice_hw *hw) { - if (ice_is_e810(hw)) - hw->phy_model = ICE_PHY_E810; + struct ice_ptp_hw *ptp = &hw->ptp; + + if (ice_is_e822(hw) || ice_is_e823(hw)) + ice_ptp_init_phy_e82x(ptp); + else if (ice_is_e810(hw)) + ice_ptp_init_phy_e810(ptp); + else if (ice_is_e825c(hw)) + ice_ptp_init_phy_e825c(hw); else - hw->phy_model = ICE_PHY_E82X; + ptp->phy_model = ICE_PHY_UNSUP; +} + +/** + * ice_ptp_write_port_cmd - Prepare a single PHY port for a timer command + * @hw: pointer to HW struct + * @port: Port to which cmd has to be sent + * @cmd: Command to be sent to the port + * + * Prepare one port for the upcoming timer sync command. Do not use this for + * programming only a single port, instead use ice_ptp_one_port_cmd() to + * ensure non-modified ports get properly initialized to ICE_PTP_NOP. + * + * Return: + * * %0 - success + * %-EBUSY - PHY type not supported + * * %other - failed to write port command + */ +static int ice_ptp_write_port_cmd(struct ice_hw *hw, u8 port, + enum ice_ptp_tmr_cmd cmd) +{ + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + return ice_ptp_write_port_cmd_eth56g(hw, port, cmd); + case ICE_PHY_E82X: + return ice_ptp_write_port_cmd_e82x(hw, port, cmd); + default: + return -EOPNOTSUPP; + } +} + +/** + * ice_ptp_one_port_cmd - Program one PHY port for a timer command + * @hw: pointer to HW struct + * @configured_port: the port that should execute the command + * @configured_cmd: the command to be executed on the configured port + * + * Prepare one port for executing a timer command, while preparing all other + * ports to ICE_PTP_NOP. This allows executing a command on a single port + * while ensuring all other ports do not execute stale commands. + * + * Return: + * * %0 - success + * * %other - failed to write port command + */ +int ice_ptp_one_port_cmd(struct ice_hw *hw, u8 configured_port, + enum ice_ptp_tmr_cmd configured_cmd) +{ + u32 port; + + for (port = 0; port < hw->ptp.num_lports; port++) { + int err; + + /* Program the configured port with the configured command, + * program all other ports with ICE_PTP_NOP. + */ + if (port == configured_port) + err = ice_ptp_write_port_cmd(hw, port, configured_cmd); + else + err = ice_ptp_write_port_cmd(hw, port, ICE_PTP_NOP); + + if (err) + return err; + } + + return 0; +} + +/** + * ice_ptp_port_cmd - Prepare PHY ports for a timer sync command + * @hw: pointer to HW struct + * @cmd: the timer command to setup + * + * Prepare all PHY ports on this device for the requested timer command. For + * some families this can be done in one shot, but for other families each + * port must be configured individually. + * + * Return: + * * %0 - success + * * %other - failed to write port command + */ +static int ice_ptp_port_cmd(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd) +{ + u32 port; + + /* PHY models which can program all ports simultaneously */ + switch (hw->ptp.phy_model) { + case ICE_PHY_E810: + return ice_ptp_port_cmd_e810(hw, cmd); + default: + break; + } + + /* PHY models which require programming each port separately */ + for (port = 0; port < hw->ptp.num_lports; port++) { + int err; + + err = ice_ptp_write_port_cmd(hw, port, cmd); + if (err) + return err; + } + + return 0; } /** @@ -3331,17 +5521,7 @@ static int ice_ptp_tmr_cmd(struct ice_hw *hw, enum ice_ptp_tmr_cmd cmd) ice_ptp_src_cmd(hw, cmd); /* Next, prepare the ports */ - switch (hw->phy_model) { - case ICE_PHY_E810: - err = ice_ptp_port_cmd_e810(hw, cmd); - break; - case ICE_PHY_E82X: - err = ice_ptp_port_cmd_e82x(hw, cmd); - break; - default: - err = -EOPNOTSUPP; - } - + err = ice_ptp_port_cmd(hw, cmd); if (err) { ice_debug(hw, ICE_DBG_PTP, "Failed to prepare PHY ports for timer command %u, err %d\n", cmd, err); @@ -3383,7 +5563,11 @@ int ice_ptp_init_time(struct ice_hw *hw, u64 time) /* PHY timers */ /* Fill Rx and Tx ports and send msg to PHY */ - switch (hw->phy_model) { + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + err = ice_ptp_prep_phy_time_eth56g(hw, + (u32)(time & 0xFFFFFFFF)); + break; case ICE_PHY_E810: err = ice_ptp_prep_phy_time_e810(hw, time & 0xFFFFFFFF); break; @@ -3425,7 +5609,10 @@ int ice_ptp_write_incval(struct ice_hw *hw, u64 incval) wr32(hw, GLTSYN_SHADJ_L(tmr_idx), lower_32_bits(incval)); wr32(hw, GLTSYN_SHADJ_H(tmr_idx), upper_32_bits(incval)); - switch (hw->phy_model) { + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + err = ice_ptp_prep_phy_incval_eth56g(hw, incval); + break; case ICE_PHY_E810: err = ice_ptp_prep_phy_incval_e810(hw, incval); break; @@ -3491,7 +5678,10 @@ int ice_ptp_adj_clock(struct ice_hw *hw, s32 adj) wr32(hw, GLTSYN_SHADJ_L(tmr_idx), 0); wr32(hw, GLTSYN_SHADJ_H(tmr_idx), adj); - switch (hw->phy_model) { + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + err = ice_ptp_prep_phy_adj_eth56g(hw, adj); + break; case ICE_PHY_E810: err = ice_ptp_prep_phy_adj_e810(hw, adj); break; @@ -3521,7 +5711,9 @@ int ice_ptp_adj_clock(struct ice_hw *hw, s32 adj) */ int ice_read_phy_tstamp(struct ice_hw *hw, u8 block, u8 idx, u64 *tstamp) { - switch (hw->phy_model) { + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + return ice_read_ptp_tstamp_eth56g(hw, block, idx, tstamp); case ICE_PHY_E810: return ice_read_phy_tstamp_e810(hw, block, idx, tstamp); case ICE_PHY_E82X: @@ -3549,7 +5741,9 @@ int ice_read_phy_tstamp(struct ice_hw *hw, u8 block, u8 idx, u64 *tstamp) */ int ice_clear_phy_tstamp(struct ice_hw *hw, u8 block, u8 idx) { - switch (hw->phy_model) { + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + return ice_clear_ptp_tstamp_eth56g(hw, block, idx); case ICE_PHY_E810: return ice_clear_phy_tstamp_e810(hw, block, idx); case ICE_PHY_E82X: @@ -3610,7 +5804,10 @@ static int ice_get_pf_c827_idx(struct ice_hw *hw, u8 *idx) */ void ice_ptp_reset_ts_memory(struct ice_hw *hw) { - switch (hw->phy_model) { + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + ice_ptp_reset_ts_memory_eth56g(hw); + break; case ICE_PHY_E82X: ice_ptp_reset_ts_memory_e82x(hw); break; @@ -3636,7 +5833,9 @@ int ice_ptp_init_phc(struct ice_hw *hw) /* Clear event err indications for auxiliary pins */ (void)rd32(hw, GLTSYN_STAT(src_idx)); - switch (hw->phy_model) { + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + return ice_ptp_init_phc_eth56g(hw); case ICE_PHY_E810: return ice_ptp_init_phc_e810(hw); case ICE_PHY_E82X: @@ -3659,7 +5858,10 @@ int ice_ptp_init_phc(struct ice_hw *hw) */ int ice_get_phy_tx_tstamp_ready(struct ice_hw *hw, u8 block, u64 *tstamp_ready) { - switch (hw->phy_model) { + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + return ice_get_phy_tx_tstamp_ready_eth56g(hw, block, + tstamp_ready); case ICE_PHY_E810: return ice_get_phy_tx_tstamp_ready_e810(hw, block, tstamp_ready); diff --git a/drivers/net/ethernet/intel/ice/ice_ptp_hw.h b/drivers/net/ethernet/intel/ice/ice_ptp_hw.h index 1f3e0312443080177399d3851c97229e1b3b4895..0852a34ade918bbe14548f6660cbe53271db6db2 100644 --- a/drivers/net/ethernet/intel/ice/ice_ptp_hw.h +++ b/drivers/net/ethernet/intel/ice/ice_ptp_hw.h @@ -41,6 +41,41 @@ enum ice_ptp_fec_mode { ICE_PTP_FEC_MODE_RS_FEC }; +enum eth56g_res_type { + ETH56G_PHY_REG_PTP, + ETH56G_PHY_MEM_PTP, + ETH56G_PHY_REG_XPCS, + ETH56G_PHY_REG_MAC, + ETH56G_PHY_REG_GPCS, + NUM_ETH56G_PHY_RES +}; + +enum ice_eth56g_link_spd { + ICE_ETH56G_LNK_SPD_1G, + ICE_ETH56G_LNK_SPD_2_5G, + ICE_ETH56G_LNK_SPD_10G, + ICE_ETH56G_LNK_SPD_25G, + ICE_ETH56G_LNK_SPD_40G, + ICE_ETH56G_LNK_SPD_50G, + ICE_ETH56G_LNK_SPD_50G2, + ICE_ETH56G_LNK_SPD_100G, + ICE_ETH56G_LNK_SPD_100G2, + NUM_ICE_ETH56G_LNK_SPD /* Must be last */ +}; + +/** + * struct ice_phy_reg_info_eth56g - ETH56G PHY register parameters + * @base: base address for each PHY block + * @step: step between PHY lanes + * + * Characteristic information for the various PHY register parameters in the + * ETH56G devices + */ +struct ice_phy_reg_info_eth56g { + u32 base[NUM_ETH56G_PHY_RES]; + u32 step; +}; + /** * struct ice_time_ref_info_e82x * @pll_freq: Frequency of PLL that drives timer ticks in Hz @@ -94,8 +129,75 @@ struct ice_vernier_info_e82x { u32 rx_fixed_delay; }; +#define ICE_ETH56G_MAC_CFG_RX_OFFSET_INT GENMASK(19, 9) +#define ICE_ETH56G_MAC_CFG_RX_OFFSET_FRAC GENMASK(8, 0) +#define ICE_ETH56G_MAC_CFG_FRAC_W 9 /** - * struct ice_cgu_pll_params_e82x + * struct ice_eth56g_mac_reg_cfg - MAC config values for specific PTP registers + * @tx_mode: Tx timestamp compensation mode + * @tx_mk_dly: Tx timestamp marker start strobe delay + * @tx_cw_dly: Tx timestamp codeword start strobe delay + * @rx_mode: Rx timestamp compensation mode + * @rx_mk_dly: Rx timestamp marker start strobe delay + * @rx_cw_dly: Rx timestamp codeword start strobe delay + * @blks_per_clk: number of blocks transferred per clock cycle + * @blktime: block time, fixed point + * @mktime: marker time, fixed point + * @tx_offset: total Tx offset, fixed point + * @rx_offset: total Rx offset, contains value for bitslip/deskew, fixed point + * + * All fixed point registers except Rx offset are 23 bit unsigned ints with + * a 9 bit fractional. + * Rx offset is 11 bit unsigned int with a 9 bit fractional. + */ +struct ice_eth56g_mac_reg_cfg { + struct { + u8 def; + u8 rs; + } tx_mode; + u8 tx_mk_dly; + struct { + u8 def; + u8 onestep; + } tx_cw_dly; + struct { + u8 def; + u8 rs; + } rx_mode; + struct { + u8 def; + u8 rs; + } rx_mk_dly; + struct { + u8 def; + u8 rs; + } rx_cw_dly; + u8 blks_per_clk; + u16 blktime; + u16 mktime; + struct { + u32 serdes; + u32 no_fec; + u32 fc; + u32 rs; + u32 sfd; + u32 onestep; + } tx_offset; + struct { + u32 serdes; + u32 no_fec; + u32 fc; + u32 rs; + u32 sfd; + u32 bs_ds; + } rx_offset; +}; + +extern +const struct ice_eth56g_mac_reg_cfg eth56g_mac_cfg[NUM_ICE_ETH56G_LNK_SPD]; + +/** + * struct ice_cgu_pll_params_e82x - E82X CGU parameters * @refclk_pre_div: Reference clock pre-divisor * @feedback_div: Feedback divisor * @frac_n_div: Fractional divisor @@ -185,9 +287,34 @@ struct ice_cgu_pin_desc { extern const struct ice_cgu_pll_params_e82x e822_cgu_params[NUM_ICE_TIME_REF_FREQ]; +/** + * struct ice_cgu_pll_params_e825c - E825C CGU parameters + * @tspll_ck_refclkfreq: tspll_ck_refclkfreq selection + * @tspll_ndivratio: ndiv ratio that goes directly to the pll + * @tspll_fbdiv_intgr: TS PLL integer feedback divide + * @tspll_fbdiv_frac: TS PLL fractional feedback divide + * @ref1588_ck_div: clock divider for tspll ref + * + * Clock Generation Unit parameters used to program the PLL based on the + * selected TIME_REF/TCXO frequency. + */ +struct ice_cgu_pll_params_e825c { + u32 tspll_ck_refclkfreq; + u32 tspll_ndivratio; + u32 tspll_fbdiv_intgr; + u32 tspll_fbdiv_frac; + u32 ref1588_ck_div; +}; + +extern const struct +ice_cgu_pll_params_e825c e825c_cgu_params[NUM_ICE_TIME_REF_FREQ]; + #define E810C_QSFP_C827_0_HANDLE 2 #define E810C_QSFP_C827_1_HANDLE 3 +/* Table of constants related to possible ETH56G PHY resources */ +extern const struct ice_phy_reg_info_eth56g eth56g_phy_res[NUM_ETH56G_PHY_RES]; + /* Table of constants related to possible TIME_REF sources */ extern const struct ice_time_ref_info_e82x e822_time_ref[NUM_ICE_TIME_REF_FREQ]; @@ -197,7 +324,9 @@ extern const struct ice_vernier_info_e82x e822_vernier[NUM_ICE_PTP_LNK_SPD]; /* Increment value to generate nanoseconds in the GLTSYN_TIME_L register for * the E810 devices. Based off of a PLL with an 812.5 MHz frequency. */ -#define ICE_PTP_NOMINAL_INCVAL_E810 0x13b13b13bULL +#define ICE_E810_PLL_FREQ 812500000 +#define ICE_PTP_NOMINAL_INCVAL_E810 0x13b13b13bULL +#define E810_OUT_PROP_DELAY_NS 1 /* Device agnostic functions */ u8 ice_get_ptp_src_clock_index(struct ice_hw *hw); @@ -208,11 +337,15 @@ int ice_ptp_init_time(struct ice_hw *hw, u64 time); int ice_ptp_write_incval(struct ice_hw *hw, u64 incval); int ice_ptp_write_incval_locked(struct ice_hw *hw, u64 incval); int ice_ptp_adj_clock(struct ice_hw *hw, s32 adj); +int ice_ptp_clear_phy_offset_ready_e82x(struct ice_hw *hw); int ice_read_phy_tstamp(struct ice_hw *hw, u8 block, u8 idx, u64 *tstamp); int ice_clear_phy_tstamp(struct ice_hw *hw, u8 block, u8 idx); void ice_ptp_reset_ts_memory(struct ice_hw *hw); int ice_ptp_init_phc(struct ice_hw *hw); +void ice_ptp_init_hw(struct ice_hw *hw); int ice_get_phy_tx_tstamp_ready(struct ice_hw *hw, u8 block, u64 *tstamp_ready); +int ice_ptp_one_port_cmd(struct ice_hw *hw, u8 configured_port, + enum ice_ptp_tmr_cmd configured_cmd); /* E822 family functions */ int ice_read_quad_reg_e82x(struct ice_hw *hw, u8 quad, u16 offset, u32 *val); @@ -264,9 +397,9 @@ int ice_stop_phy_timer_e82x(struct ice_hw *hw, u8 port, bool soft_reset); int ice_start_phy_timer_e82x(struct ice_hw *hw, u8 port); int ice_phy_cfg_tx_offset_e82x(struct ice_hw *hw, u8 port); int ice_phy_cfg_rx_offset_e82x(struct ice_hw *hw, u8 port); +int ice_phy_cfg_intr_e82x(struct ice_hw *hw, u8 quad, bool ena, u8 threshold); /* E810 family functions */ -int ice_ptp_init_phy_e810(struct ice_hw *hw); int ice_read_sma_ctrl_e810t(struct ice_hw *hw, u8 *data); int ice_write_sma_ctrl_e810t(struct ice_hw *hw, u8 data); int ice_read_pca9575_reg_e810t(struct ice_hw *hw, u8 offset, u8 *data); @@ -280,11 +413,44 @@ int ice_get_cgu_state(struct ice_hw *hw, u8 dpll_idx, u8 *ref_state, u8 *eec_mode, s64 *phase_offset, enum dpll_lock_status *dpll_state); int ice_get_cgu_rclk_pin_info(struct ice_hw *hw, u8 *base_idx, u8 *pin_num); - -void ice_ptp_init_phy_model(struct ice_hw *hw); int ice_cgu_get_output_pin_state_caps(struct ice_hw *hw, u8 pin_id, unsigned long *caps); +/* ETH56G family functions */ +int ice_ptp_read_tx_hwtstamp_status_eth56g(struct ice_hw *hw, u32 *ts_status); +int ice_stop_phy_timer_eth56g(struct ice_hw *hw, u8 port, bool soft_reset); +int ice_start_phy_timer_eth56g(struct ice_hw *hw, u8 port); +int ice_phy_cfg_tx_offset_eth56g(struct ice_hw *hw, u8 port); +int ice_phy_cfg_rx_offset_eth56g(struct ice_hw *hw, u8 port); +int ice_phy_cfg_intr_eth56g(struct ice_hw *hw, u8 port, bool ena, u8 threshold); +int ice_phy_cfg_ptp_1step_eth56g(struct ice_hw *hw, u8 port); + +#define ICE_ETH56G_NOMINAL_INCVAL 0x140000000ULL +#define ICE_ETH56G_NOMINAL_PCS_REF_TUS 0x100000000ULL +#define ICE_ETH56G_NOMINAL_PCS_REF_INC 0x300000000ULL +#define ICE_ETH56G_NOMINAL_THRESH4 0x7777 +#define ICE_ETH56G_NOMINAL_TX_THRESH 0x6 + +/** + * ice_get_base_incval - Get base clock increment value + * @hw: pointer to the HW struct + * + * Return: base clock increment value for supported PHYs, 0 otherwise + */ +static inline u64 ice_get_base_incval(struct ice_hw *hw) +{ + switch (hw->ptp.phy_model) { + case ICE_PHY_ETH56G: + return ICE_ETH56G_NOMINAL_INCVAL; + case ICE_PHY_E810: + return ICE_PTP_NOMINAL_INCVAL_E810; + case ICE_PHY_E82X: + return ice_e82x_nominal_incval(ice_e82x_time_ref(hw)); + default: + return 0; + } +} + #define PFTSYN_SEM_BYTES 4 #define ICE_PTP_CLOCK_INDEX_0 0x00 @@ -312,6 +478,7 @@ int ice_cgu_get_output_pin_state_caps(struct ice_hw *hw, u8 pin_id, #define TS_CMD_MASK_E810 0xFF #define TS_CMD_MASK 0xF #define SYNC_EXEC_CMD 0x3 +#define TS_CMD_RX_TYPE ICE_M(0x18, 0x4) /* Macros to derive port low and high addresses on both quads */ #define P_Q0_L(a, p) ((((a) + (0x2000 * (p)))) & 0xFFFF) @@ -344,11 +511,8 @@ int ice_cgu_get_output_pin_state_caps(struct ice_hw *hw, u8 pin_id, #define Q_REG_TX_MEM_GBL_CFG 0xC08 #define Q_REG_TX_MEM_GBL_CFG_LANE_TYPE_S 0 #define Q_REG_TX_MEM_GBL_CFG_LANE_TYPE_M BIT(0) -#define Q_REG_TX_MEM_GBL_CFG_TX_TYPE_S 1 #define Q_REG_TX_MEM_GBL_CFG_TX_TYPE_M ICE_M(0xFF, 1) -#define Q_REG_TX_MEM_GBL_CFG_INTR_THR_S 9 #define Q_REG_TX_MEM_GBL_CFG_INTR_THR_M ICE_M(0x3F, 9) -#define Q_REG_TX_MEM_GBL_CFG_INTR_ENA_S 15 #define Q_REG_TX_MEM_GBL_CFG_INTR_ENA_M BIT(15) /* Tx Timestamp data registers */ @@ -380,7 +544,7 @@ int ice_cgu_get_output_pin_state_caps(struct ice_hw *hw, u8 pin_id, #define P_REG_TIMETUS_L 0x410 #define P_REG_TIMETUS_U 0x414 -#define P_REG_40B_LOW_M 0xFF +#define P_REG_40B_LOW_M GENMASK(7, 0) #define P_REG_40B_HIGH_S 8 /* PHY window length registers */ @@ -487,7 +651,7 @@ int ice_cgu_get_output_pin_state_caps(struct ice_hw *hw, u8 pin_id, #define ETH_GLTSYN_SHADJ_H(_i) (0x0300037C + ((_i) * 32)) /* E810 timer command register */ -#define ETH_GLTSYN_CMD 0x03000344 +#define E810_ETH_GLTSYN_CMD 0x03000344 /* Source timer incval macros */ #define INCVAL_HIGH_M 0xFF @@ -549,4 +713,115 @@ int ice_cgu_get_output_pin_state_caps(struct ice_hw *hw, u8 pin_id, /* E810T PCA9575 IO controller pin control */ #define ICE_E810T_P0_GNSS_PRSNT_N BIT(4) +/* ETH56G PHY register addresses */ +/* Timestamp PHY incval registers */ +#define PHY_REG_TIMETUS_L 0x8 +#define PHY_REG_TIMETUS_U 0xC + +/* Timestamp PCS registers */ +#define PHY_PCS_REF_TUS_L 0x18 +#define PHY_PCS_REF_TUS_U 0x1C + +/* Timestamp PCS ref incval registers */ +#define PHY_PCS_REF_INC_L 0x20 +#define PHY_PCS_REF_INC_U 0x24 + +/* Timestamp init registers */ +#define PHY_REG_RX_TIMER_INC_PRE_L 0x64 +#define PHY_REG_RX_TIMER_INC_PRE_U 0x68 +#define PHY_REG_TX_TIMER_INC_PRE_L 0x44 +#define PHY_REG_TX_TIMER_INC_PRE_U 0x48 + +/* Timestamp match and adjust target registers */ +#define PHY_REG_RX_TIMER_CNT_ADJ_L 0x6C +#define PHY_REG_RX_TIMER_CNT_ADJ_U 0x70 +#define PHY_REG_TX_TIMER_CNT_ADJ_L 0x4C +#define PHY_REG_TX_TIMER_CNT_ADJ_U 0x50 + +/* Timestamp command registers */ +#define PHY_REG_TX_TMR_CMD 0x40 +#define PHY_REG_RX_TMR_CMD 0x60 + +/* Phy offset ready registers */ +#define PHY_REG_TX_OFFSET_READY 0x54 +#define PHY_REG_RX_OFFSET_READY 0x74 + +/* Phy total offset registers */ +#define PHY_REG_TOTAL_TX_OFFSET_L 0x38 +#define PHY_REG_TOTAL_TX_OFFSET_U 0x3C +#define PHY_REG_TOTAL_RX_OFFSET_L 0x58 +#define PHY_REG_TOTAL_RX_OFFSET_U 0x5C + +/* Timestamp capture registers */ +#define PHY_REG_TX_CAPTURE_L 0x78 +#define PHY_REG_TX_CAPTURE_U 0x7C +#define PHY_REG_RX_CAPTURE_L 0x8C +#define PHY_REG_RX_CAPTURE_U 0x90 + +/* Memory status registers */ +#define PHY_REG_TX_MEMORY_STATUS_L 0x80 +#define PHY_REG_TX_MEMORY_STATUS_U 0x84 + +/* Interrupt config register */ +#define PHY_REG_TS_INT_CONFIG 0x88 + +/* XIF mode config register */ +#define PHY_MAC_XIF_MODE 0x24 +#define PHY_MAC_XIF_1STEP_ENA_M ICE_M(0x1, 5) +#define PHY_MAC_XIF_TS_BIN_MODE_M ICE_M(0x1, 11) +#define PHY_MAC_XIF_TS_SFD_ENA_M ICE_M(0x1, 20) +#define PHY_MAC_XIF_GMII_TS_SEL_M ICE_M(0x1, 21) + +/* GPCS config register */ +#define PHY_GPCS_CONFIG_REG0 0x268 +#define PHY_GPCS_CONFIG_REG0_TX_THR_M ICE_M(0xF, 24) +#define PHY_GPCS_BITSLIP 0x5C + +#define PHY_TS_INT_CONFIG_THRESHOLD_M ICE_M(0x3F, 0) +#define PHY_TS_INT_CONFIG_ENA_M BIT(6) + +/* 1-step PTP config */ +#define PHY_PTP_1STEP_CONFIG 0x270 +#define PHY_PTP_1STEP_T1S_UP64_M ICE_M(0xF, 4) +#define PHY_PTP_1STEP_T1S_DELTA_M ICE_M(0xF, 8) +#define PHY_PTP_1STEP_PEER_DELAY(_port) (0x274 + 4 * (_port)) +#define PHY_PTP_1STEP_PD_ADD_PD_M ICE_M(0x1, 0) +#define PHY_PTP_1STEP_PD_DELAY_M ICE_M(0x3fffffff, 1) +#define PHY_PTP_1STEP_PD_DLY_V_M ICE_M(0x1, 31) + +/* Macros to derive offsets for TimeStampLow and TimeStampHigh */ +#define PHY_TSTAMP_L(x) (((x) * 8) + 0) +#define PHY_TSTAMP_U(x) (((x) * 8) + 4) + +#define PHY_REG_REVISION 0x85000 + +#define PHY_REG_DESKEW_0 0x94 +#define PHY_REG_DESKEW_0_RLEVEL GENMASK(6, 0) +#define PHY_REG_DESKEW_0_RLEVEL_FRAC GENMASK(9, 7) +#define PHY_REG_DESKEW_0_RLEVEL_FRAC_W 3 +#define PHY_REG_DESKEW_0_VALID GENMASK(10, 10) + +#define PHY_REG_GPCS_BITSLIP 0x5C +#define PHY_REG_SD_BIT_SLIP(_port_offset) (0x29C + 4 * (_port_offset)) +#define PHY_REVISION_ETH56G 0x10200 +#define PHY_VENDOR_TXLANE_THRESH 0x2000C + +#define PHY_MAC_TSU_CONFIG 0x40 +#define PHY_MAC_TSU_CFG_RX_MODE_M ICE_M(0x7, 0) +#define PHY_MAC_TSU_CFG_RX_MII_CW_DLY_M ICE_M(0x7, 4) +#define PHY_MAC_TSU_CFG_RX_MII_MK_DLY_M ICE_M(0x7, 8) +#define PHY_MAC_TSU_CFG_TX_MODE_M ICE_M(0x7, 12) +#define PHY_MAC_TSU_CFG_TX_MII_CW_DLY_M ICE_M(0x1F, 16) +#define PHY_MAC_TSU_CFG_TX_MII_MK_DLY_M ICE_M(0x1F, 21) +#define PHY_MAC_TSU_CFG_BLKS_PER_CLK_M ICE_M(0x1, 28) +#define PHY_MAC_RX_MODULO 0x44 +#define PHY_MAC_RX_OFFSET 0x48 +#define PHY_MAC_RX_OFFSET_M ICE_M(0xFFFFFF, 0) +#define PHY_MAC_TX_MODULO 0x4C +#define PHY_MAC_BLOCKTIME 0x50 +#define PHY_MAC_MARKERTIME 0x54 +#define PHY_MAC_TX_OFFSET 0x58 + +#define PHY_PTP_INT_STATUS 0x7FD140 + #endif /* _ICE_PTP_HW_H_ */ diff --git a/drivers/net/ethernet/intel/ice/ice_sbq_cmd.h b/drivers/net/ethernet/intel/ice/ice_sbq_cmd.h index ead75fe2bcdadb34c42ede9815a9c73290a7dfc9..3b0054faf70c771ba9cc71f711c0e10317d1957d 100644 --- a/drivers/net/ethernet/intel/ice/ice_sbq_cmd.h +++ b/drivers/net/ethernet/intel/ice/ice_sbq_cmd.h @@ -47,10 +47,12 @@ struct ice_sbq_evt_desc { }; enum ice_sbq_msg_dev { - rmn_0 = 0x02, - rmn_1 = 0x03, - rmn_2 = 0x04, - cgu = 0x06 + eth56g_phy_0 = 0x02, + rmn_0 = 0x02, + rmn_1 = 0x03, + rmn_2 = 0x04, + cgu = 0x06, + eth56g_phy_1 = 0x0D, }; enum ice_sbq_msg_opcode { diff --git a/drivers/net/ethernet/intel/ice/ice_type.h b/drivers/net/ethernet/intel/ice/ice_type.h index f0796a93f4287e22e490f24a0dc1260cf5d7c777..aac59c85a911e8560968b8a2262b0e746f703ff7 100644 --- a/drivers/net/ethernet/intel/ice/ice_type.h +++ b/drivers/net/ethernet/intel/ice/ice_type.h @@ -322,12 +322,14 @@ enum ice_time_ref_freq { ICE_TIME_REF_FREQ_156_250 = 4, ICE_TIME_REF_FREQ_245_760 = 5, - NUM_ICE_TIME_REF_FREQ + NUM_ICE_TIME_REF_FREQ, + + ICE_TIME_REF_FREQ_INVALID = -1, }; /* Clock source specification */ enum ice_clk_src { - ICE_CLK_SRC_TCX0 = 0, /* Temperature compensated oscillator */ + ICE_CLK_SRC_TCXO = 0, /* Temperature compensated oscillator */ ICE_CLK_SRC_TIME_REF = 1, /* Use TIME_REF reference clock */ NUM_ICE_CLK_SRC @@ -372,6 +374,15 @@ struct ice_ts_dev_info { u8 ts_ll_int_read; }; +#define ICE_NAC_TOPO_PRIMARY_M BIT(0) +#define ICE_NAC_TOPO_DUAL_M BIT(1) +#define ICE_NAC_TOPO_ID_M GENMASK(0xF, 0) + +struct ice_nac_topology { + u32 mode; + u8 id; +}; + /* Function specific capabilities */ struct ice_hw_func_caps { struct ice_hw_common_caps common_cap; @@ -393,6 +404,7 @@ struct ice_hw_dev_caps { u32 num_flow_director_fltr; /* Number of FD filters available */ struct ice_ts_dev_info ts_dev_info; u32 num_funcs; + struct ice_nac_topology nac_topo; /* bitmap of supported sensors * bit 0 - internal temperature sensor * bit 31:1 - Reserved @@ -482,6 +494,8 @@ struct ice_bank_info { u32 orom_size; /* Size of OROM bank */ u32 netlist_ptr; /* Pointer to 1st Netlist bank */ u32 netlist_size; /* Size of Netlist bank */ + u32 active_css_hdr_len; /* Active CSS header length */ + u32 inactive_css_hdr_len; /* Inactive CSS header length */ enum ice_flash_bank nvm_bank; /* Active NVM bank */ enum ice_flash_bank orom_bank; /* Active OROM bank */ enum ice_flash_bank netlist_bank; /* Active Netlist bank */ @@ -818,11 +832,43 @@ struct ice_mbx_data { u16 async_watermark_val; }; +#define ICE_PORTS_PER_QUAD 4 +#define ICE_GET_QUAD_NUM(port) ((port) / ICE_PORTS_PER_QUAD) + +struct ice_eth56g_params { + u8 num_phys; + u8 phy_addr[2]; + bool onestep_ena; + bool sfd_ena; + u32 peer_delay; +}; + +union ice_phy_params { + struct ice_eth56g_params eth56g; +}; + /* PHY model */ enum ice_phy_model { ICE_PHY_UNSUP = -1, - ICE_PHY_E810 = 1, + ICE_PHY_E810 = 1, ICE_PHY_E82X, + ICE_PHY_ETH56G, +}; + +/* Global Link Topology */ +enum ice_global_link_topo { + ICE_LINK_TOPO_UP_TO_2_LINKS, + ICE_LINK_TOPO_UP_TO_4_LINKS, + ICE_LINK_TOPO_UP_TO_8_LINKS, + ICE_LINK_TOPO_RESERVED, +}; + +struct ice_ptp_hw { + enum ice_phy_model phy_model; + union ice_phy_params phy; + u8 num_lports; + u8 ports_per_phy; + bool is_2x50g_muxed_topo; }; /* Port hardware description */ @@ -846,7 +892,6 @@ struct ice_hw { u8 revision_id; u8 pf_id; /* device profile info */ - enum ice_phy_model phy_model; u16 max_burst_size; /* driver sets this value */ @@ -909,12 +954,7 @@ struct ice_hw { /* INTRL granularity in 1 us */ u8 intrl_gran; -#define ICE_MAX_QUAD 2 -#define ICE_QUADS_PER_PHY_E82X 2 -#define ICE_PORTS_PER_PHY_E82X 8 -#define ICE_PORTS_PER_QUAD 4 -#define ICE_PORTS_PER_PHY_E810 4 -#define ICE_NUM_EXTERNAL_PORTS (ICE_MAX_QUAD * ICE_PORTS_PER_QUAD) + struct ice_ptp_hw ptp; /* Active package version (currently active) */ struct ice_pkg_ver active_pkg_ver; @@ -1087,17 +1127,13 @@ struct ice_aq_get_set_rss_lut_params { #define ICE_SR_SECTOR_SIZE_IN_WORDS 0x800 /* CSS Header words */ +#define ICE_NVM_CSS_HDR_LEN_L 0x02 +#define ICE_NVM_CSS_HDR_LEN_H 0x03 #define ICE_NVM_CSS_SREV_L 0x14 #define ICE_NVM_CSS_SREV_H 0x15 -/* Length of CSS header section in words */ -#define ICE_CSS_HEADER_LENGTH 330 - -/* Offset of Shadow RAM copy in the NVM bank area. */ -#define ICE_NVM_SR_COPY_WORD_OFFSET roundup(ICE_CSS_HEADER_LENGTH, 32) - -/* Size in bytes of Option ROM trailer */ -#define ICE_NVM_OROM_TRAILER_LENGTH (2 * ICE_CSS_HEADER_LENGTH) +/* Length of Authentication header section in words */ +#define ICE_NVM_AUTH_HEADER_LEN 0x08 /* The Link Topology Netlist section is stored as a series of words. It is * stored in the NVM as a TLV, with the first two words containing the type diff --git a/drivers/net/ethernet/intel/ice/ice_vsi_vlan_lib.c b/drivers/net/ethernet/intel/ice/ice_vsi_vlan_lib.c index 2e9ad27cb9d13e8f5faa1d28a5d777de8fe0c2d3..6e8f2aab6080153082f1fde3e58effabc5df720a 100644 --- a/drivers/net/ethernet/intel/ice/ice_vsi_vlan_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_vsi_vlan_lib.c @@ -45,14 +45,15 @@ int ice_vsi_add_vlan(struct ice_vsi *vsi, struct ice_vlan *vlan) return -EINVAL; err = ice_fltr_add_vlan(vsi, vlan); - if (err && err != -EEXIST) { + if (!err) + vsi->num_vlan++; + else if (err == -EEXIST) + err = 0; + else dev_err(ice_pf_to_dev(vsi->back), "Failure Adding VLAN %d on VSI %i, status %d\n", vlan->vid, vsi->vsi_num, err); - return err; - } - vsi->num_vlan++; - return 0; + return err; } /** diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c index 7541f223bf4f69cb63f985f553e4e30e8036e9c8..a65955eb23c0bd85adc7f0f8f1a4b39867da8907 100644 --- a/drivers/net/ethernet/intel/ice/ice_xsk.c +++ b/drivers/net/ethernet/intel/ice/ice_xsk.c @@ -269,7 +269,6 @@ static int ice_xsk_pool_disable(struct ice_vsi *vsi, u16 qid) if (!pool) return -EINVAL; - clear_bit(qid, vsi->af_xdp_zc_qps); xsk_pool_dma_unmap(pool, ICE_RX_DMA_ATTR); return 0; @@ -300,8 +299,6 @@ ice_xsk_pool_enable(struct ice_vsi *vsi, struct xsk_buff_pool *pool, u16 qid) if (err) return err; - set_bit(qid, vsi->af_xdp_zc_qps); - return 0; } @@ -349,11 +346,13 @@ ice_realloc_rx_xdp_bufs(struct ice_rx_ring *rx_ring, bool pool_present) int ice_realloc_zc_buf(struct ice_vsi *vsi, bool zc) { struct ice_rx_ring *rx_ring; - unsigned long q; + uint i; + + ice_for_each_rxq(vsi, i) { + rx_ring = vsi->rx_rings[i]; + if (!rx_ring->xsk_pool) + continue; - for_each_set_bit(q, vsi->af_xdp_zc_qps, - max_t(int, vsi->alloc_txq, vsi->alloc_rxq)) { - rx_ring = vsi->rx_rings[q]; if (ice_realloc_rx_xdp_bufs(rx_ring, zc)) return -ENOMEM; } diff --git a/drivers/net/ethernet/intel/idpf/idpf_lib.c b/drivers/net/ethernet/intel/idpf/idpf_lib.c index 52ceda6306a3d0317044234f6798fe0f6c8e5f83..f1ee5584e8fa20556ace18afbc78c00549cf8c24 100644 --- a/drivers/net/ethernet/intel/idpf/idpf_lib.c +++ b/drivers/net/ethernet/intel/idpf/idpf_lib.c @@ -1394,6 +1394,7 @@ static int idpf_vport_open(struct idpf_vport *vport, bool alloc_res) } idpf_rx_init_buf_tail(vport); + idpf_vport_intr_ena(vport); err = idpf_send_config_queues_msg(vport); if (err) { diff --git a/drivers/net/ethernet/intel/idpf/idpf_txrx.c b/drivers/net/ethernet/intel/idpf/idpf_txrx.c index 285da2177ee446e7607b8c0206ea16241e0cf3cf..b023704bbbdab8018bfcbed2a0619da63b1b608c 100644 --- a/drivers/net/ethernet/intel/idpf/idpf_txrx.c +++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c @@ -3746,9 +3746,9 @@ static void idpf_vport_intr_ena_irq_all(struct idpf_vport *vport) */ void idpf_vport_intr_deinit(struct idpf_vport *vport) { + idpf_vport_intr_dis_irq_all(vport); idpf_vport_intr_napi_dis_all(vport); idpf_vport_intr_napi_del_all(vport); - idpf_vport_intr_dis_irq_all(vport); idpf_vport_intr_rel_irq(vport); } @@ -4179,7 +4179,6 @@ int idpf_vport_intr_init(struct idpf_vport *vport) idpf_vport_intr_map_vector_to_qs(vport); idpf_vport_intr_napi_add_all(vport); - idpf_vport_intr_napi_ena_all(vport); err = vport->adapter->dev_ops.reg_ops.intr_reg_init(vport); if (err) @@ -4193,17 +4192,20 @@ int idpf_vport_intr_init(struct idpf_vport *vport) if (err) goto unroll_vectors_alloc; - idpf_vport_intr_ena_irq_all(vport); - return 0; unroll_vectors_alloc: - idpf_vport_intr_napi_dis_all(vport); idpf_vport_intr_napi_del_all(vport); return err; } +void idpf_vport_intr_ena(struct idpf_vport *vport) +{ + idpf_vport_intr_napi_ena_all(vport); + idpf_vport_intr_ena_irq_all(vport); +} + /** * idpf_config_rss - Send virtchnl messages to configure RSS * @vport: virtual port diff --git a/drivers/net/ethernet/intel/idpf/idpf_txrx.h b/drivers/net/ethernet/intel/idpf/idpf_txrx.h index 3d046b81e507a1d29ce038a85ff542a5835dab01..551391e20464709c6cc566d8b02a0b395c04b702 100644 --- a/drivers/net/ethernet/intel/idpf/idpf_txrx.h +++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.h @@ -990,6 +990,7 @@ int idpf_vport_intr_alloc(struct idpf_vport *vport); void idpf_vport_intr_update_itr_ena_irq(struct idpf_q_vector *q_vector); void idpf_vport_intr_deinit(struct idpf_vport *vport); int idpf_vport_intr_init(struct idpf_vport *vport); +void idpf_vport_intr_ena(struct idpf_vport *vport); enum pkt_hash_types idpf_ptype_to_htype(const struct idpf_rx_ptype_decoded *decoded); int idpf_config_rss(struct idpf_vport *vport); int idpf_init_rss(struct idpf_vport *vport); diff --git a/drivers/net/ethernet/intel/igb/Makefile b/drivers/net/ethernet/intel/igb/Makefile index 394c1e0656b9ec2c2183cc50ac82e9b705b5ac7e..463c0d26b9d43cb9ca660f960269a7e659679888 100644 --- a/drivers/net/ethernet/intel/igb/Makefile +++ b/drivers/net/ethernet/intel/igb/Makefile @@ -6,6 +6,6 @@ obj-$(CONFIG_IGB) += igb.o -igb-objs := igb_main.o igb_ethtool.o e1000_82575.o \ - e1000_mac.o e1000_nvm.o e1000_phy.o e1000_mbx.o \ - e1000_i210.o igb_ptp.o igb_hwmon.o +igb-y := igb_main.o igb_ethtool.o e1000_82575.o \ + e1000_mac.o e1000_nvm.o e1000_phy.o e1000_mbx.o \ + e1000_i210.o igb_ptp.o igb_hwmon.o diff --git a/drivers/net/ethernet/intel/igbvf/Makefile b/drivers/net/ethernet/intel/igbvf/Makefile index afd3e36eae75704a2a3b6144db0e4b50c8086bb1..902711d5e6916bc23d9a6915063890ffc2e63f3d 100644 --- a/drivers/net/ethernet/intel/igbvf/Makefile +++ b/drivers/net/ethernet/intel/igbvf/Makefile @@ -6,8 +6,4 @@ obj-$(CONFIG_IGBVF) += igbvf.o -igbvf-objs := vf.o \ - mbx.o \ - ethtool.o \ - netdev.o - +igbvf-y := vf.o mbx.o ethtool.o netdev.o diff --git a/drivers/net/ethernet/intel/igc/Makefile b/drivers/net/ethernet/intel/igc/Makefile index ebffd3054285c20a1395204d0e0beb388a8269e3..efc5e7983dad9551490fba71bdcca15d646989f9 100644 --- a/drivers/net/ethernet/intel/igc/Makefile +++ b/drivers/net/ethernet/intel/igc/Makefile @@ -6,7 +6,7 @@ # obj-$(CONFIG_IGC) += igc.o -igc-$(CONFIG_IGC_LEDS) += igc_leds.o -igc-objs := igc_main.o igc_mac.o igc_i225.o igc_base.o igc_nvm.o igc_phy.o \ -igc_diag.o igc_ethtool.o igc_ptp.o igc_dump.o igc_tsn.o igc_xdp.o +igc-y := igc_main.o igc_mac.o igc_i225.o igc_base.o igc_nvm.o igc_phy.o \ + igc_diag.o igc_ethtool.o igc_ptp.o igc_dump.o igc_tsn.o igc_xdp.o +igc-$(CONFIG_IGC_LEDS) += igc_leds.o diff --git a/drivers/net/ethernet/intel/igc/igc_ethtool.c b/drivers/net/ethernet/intel/igc/igc_ethtool.c index f2c4f1966bb041629411c252ca7c7f06b5fdf51e..0cd2bd695db1dfddd64d2c82027f4206b6609268 100644 --- a/drivers/net/ethernet/intel/igc/igc_ethtool.c +++ b/drivers/net/ethernet/intel/igc/igc_ethtool.c @@ -1629,12 +1629,17 @@ static int igc_ethtool_get_eee(struct net_device *netdev, struct igc_hw *hw = &adapter->hw; u32 eeer; + linkmode_set_bit(ETHTOOL_LINK_MODE_2500baseT_Full_BIT, + edata->supported); + linkmode_set_bit(ETHTOOL_LINK_MODE_1000baseT_Full_BIT, + edata->supported); + linkmode_set_bit(ETHTOOL_LINK_MODE_100baseT_Full_BIT, + edata->supported); + if (hw->dev_spec._base.eee_enable) mii_eee_cap1_mod_linkmode_t(edata->advertised, adapter->eee_advert); - *edata = adapter->eee; - eeer = rd32(IGC_EEER); /* EEE status on negotiated link */ diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 12f004f46082cdfdd68493f43ef2e1653b2e815e..305e05294a26595fdc361851e2c75dd9355b9e20 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -12,6 +12,7 @@ #include #include #include +#include #include @@ -4975,6 +4976,9 @@ void igc_up(struct igc_adapter *adapter) /* start the watchdog. */ hw->mac.get_link_status = true; schedule_work(&adapter->watchdog_task); + + adapter->eee_advert = MDIO_EEE_100TX | MDIO_EEE_1000T | + MDIO_EEE_2_5GT; } /** diff --git a/drivers/net/ethernet/intel/ixgbe/Makefile b/drivers/net/ethernet/intel/ixgbe/Makefile index 4fb0d9e3f2da21db5acf17dbf5b9c091502db442..965e5ce1b3267f289271a451acfe0d3353275385 100644 --- a/drivers/net/ethernet/intel/ixgbe/Makefile +++ b/drivers/net/ethernet/intel/ixgbe/Makefile @@ -6,10 +6,10 @@ obj-$(CONFIG_IXGBE) += ixgbe.o -ixgbe-objs := ixgbe_main.o ixgbe_common.o ixgbe_ethtool.o \ - ixgbe_82599.o ixgbe_82598.o ixgbe_phy.o ixgbe_sriov.o \ - ixgbe_mbx.o ixgbe_x540.o ixgbe_x550.o ixgbe_lib.o ixgbe_ptp.o \ - ixgbe_xsk.o +ixgbe-y := ixgbe_main.o ixgbe_common.o ixgbe_ethtool.o \ + ixgbe_82599.o ixgbe_82598.o ixgbe_phy.o ixgbe_sriov.o \ + ixgbe_mbx.o ixgbe_x540.o ixgbe_x550.o ixgbe_lib.o ixgbe_ptp.o \ + ixgbe_xsk.o ixgbe-$(CONFIG_IXGBE_DCB) += ixgbe_dcb.o ixgbe_dcb_82598.o \ ixgbe_dcb_82599.o ixgbe_dcb_nl.o diff --git a/drivers/net/ethernet/intel/ixgbevf/Makefile b/drivers/net/ethernet/intel/ixgbevf/Makefile index 186a4bb24fdea58a483f161858333313c34373ae..01d3e892f3fa7b6f2ee68174ac978dc7f2b51c3e 100644 --- a/drivers/net/ethernet/intel/ixgbevf/Makefile +++ b/drivers/net/ethernet/intel/ixgbevf/Makefile @@ -6,9 +6,5 @@ obj-$(CONFIG_IXGBEVF) += ixgbevf.o -ixgbevf-objs := vf.o \ - mbx.o \ - ethtool.o \ - ixgbevf_main.o +ixgbevf-y := vf.o mbx.o ethtool.o ixgbevf_main.o ixgbevf-$(CONFIG_IXGBEVF_IPSEC) += ipsec.o - diff --git a/drivers/net/ethernet/intel/libeth/Makefile b/drivers/net/ethernet/intel/libeth/Makefile index cb99203d1dd209fd78b7ccfdebde6d2e88b18722..52492b08113261dd1121a4427077f2ca8e797005 100644 --- a/drivers/net/ethernet/intel/libeth/Makefile +++ b/drivers/net/ethernet/intel/libeth/Makefile @@ -3,4 +3,4 @@ obj-$(CONFIG_LIBETH) += libeth.o -libeth-objs += rx.o +libeth-y := rx.o diff --git a/drivers/net/ethernet/intel/libie/Makefile b/drivers/net/ethernet/intel/libie/Makefile index bf42c5aeeeddabf23a5bb67f6b558b588969bb64..ffd27fab916a6d6b07dd2eb60de8c7d3ac0c84d8 100644 --- a/drivers/net/ethernet/intel/libie/Makefile +++ b/drivers/net/ethernet/intel/libie/Makefile @@ -3,4 +3,4 @@ obj-$(CONFIG_LIBIE) += libie.o -libie-objs += rx.o +libie-y := rx.o diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h index 35834687e40fe9877f5f575ee17f0ead42ee3726..3063a84a45ef62fb1e082b4c22d52784363b7e39 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h @@ -76,6 +76,7 @@ struct rvu_debugfs { struct dump_ctx nix_cq_ctx; struct dump_ctx nix_rq_ctx; struct dump_ctx nix_sq_ctx; + struct dump_ctx nix_tm_ctx; struct cpt_ctx cpt_ctx[MAX_CPT_BLKS]; int npa_qsize_id; int nix_qsize_id; diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c index 881d704644fbee77cf5c9f7137d539463c08c2dd..4a4ef5bd9e0bd5be70028e4f8e2834e37bf3d1b3 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c @@ -1603,6 +1603,367 @@ static void print_nix_cn10k_sq_ctx(struct seq_file *m, (u64)sq_ctx->dropped_pkts); } +static void print_tm_tree(struct seq_file *m, + struct nix_aq_enq_rsp *rsp, u64 sq) +{ + struct nix_sq_ctx_s *sq_ctx = &rsp->sq; + struct nix_hw *nix_hw = m->private; + struct rvu *rvu = nix_hw->rvu; + u16 p1, p2, p3, p4, schq; + int blkaddr; + u64 cfg; + + blkaddr = nix_hw->blkaddr; + schq = sq_ctx->smq; + + cfg = rvu_read64(rvu, blkaddr, NIX_AF_MDQX_PARENT(schq)); + p1 = FIELD_GET(NIX_AF_MDQ_PARENT_MASK, cfg); + + cfg = rvu_read64(rvu, blkaddr, NIX_AF_TL4X_PARENT(p1)); + p2 = FIELD_GET(NIX_AF_TL4_PARENT_MASK, cfg); + + cfg = rvu_read64(rvu, blkaddr, NIX_AF_TL3X_PARENT(p2)); + p3 = FIELD_GET(NIX_AF_TL3_PARENT_MASK, cfg); + + cfg = rvu_read64(rvu, blkaddr, NIX_AF_TL2X_PARENT(p3)); + p4 = FIELD_GET(NIX_AF_TL2_PARENT_MASK, cfg); + seq_printf(m, + "SQ(%llu) -> SMQ(%u) -> TL4(%u) -> TL3(%u) -> TL2(%u) -> TL1(%u)\n", + sq, schq, p1, p2, p3, p4); +} + +/*dumps given tm_tree registers*/ +static int rvu_dbg_nix_tm_tree_display(struct seq_file *m, void *unused) +{ + int qidx, nixlf, rc, id, max_id = 0; + struct nix_hw *nix_hw = m->private; + struct rvu *rvu = nix_hw->rvu; + struct nix_aq_enq_req aq_req; + struct nix_aq_enq_rsp rsp; + struct rvu_pfvf *pfvf; + u16 pcifunc; + + nixlf = rvu->rvu_dbg.nix_tm_ctx.lf; + id = rvu->rvu_dbg.nix_tm_ctx.id; + + if (!rvu_dbg_is_valid_lf(rvu, nix_hw->blkaddr, nixlf, &pcifunc)) + return -EINVAL; + + pfvf = rvu_get_pfvf(rvu, pcifunc); + max_id = pfvf->sq_ctx->qsize; + + memset(&aq_req, 0, sizeof(struct nix_aq_enq_req)); + aq_req.hdr.pcifunc = pcifunc; + aq_req.ctype = NIX_AQ_CTYPE_SQ; + aq_req.op = NIX_AQ_INSTOP_READ; + seq_printf(m, "pcifunc is 0x%x\n", pcifunc); + for (qidx = id; qidx < max_id; qidx++) { + aq_req.qidx = qidx; + + /* Skip SQ's if not initialized */ + if (!test_bit(qidx, pfvf->sq_bmap)) + continue; + + rc = rvu_mbox_handler_nix_aq_enq(rvu, &aq_req, &rsp); + + if (rc) { + seq_printf(m, "Failed to read SQ(%d) context\n", + aq_req.qidx); + continue; + } + print_tm_tree(m, &rsp, aq_req.qidx); + } + return 0; +} + +static ssize_t rvu_dbg_nix_tm_tree_write(struct file *filp, + const char __user *buffer, + size_t count, loff_t *ppos) +{ + struct seq_file *m = filp->private_data; + struct nix_hw *nix_hw = m->private; + struct rvu *rvu = nix_hw->rvu; + struct rvu_pfvf *pfvf; + u16 pcifunc; + u64 nixlf; + int ret; + + ret = kstrtoull_from_user(buffer, count, 10, &nixlf); + if (ret) + return ret; + + if (!rvu_dbg_is_valid_lf(rvu, nix_hw->blkaddr, nixlf, &pcifunc)) + return -EINVAL; + + pfvf = rvu_get_pfvf(rvu, pcifunc); + if (!pfvf->sq_ctx) { + dev_warn(rvu->dev, "SQ context is not initialized\n"); + return -EINVAL; + } + + rvu->rvu_dbg.nix_tm_ctx.lf = nixlf; + return count; +} + +RVU_DEBUG_SEQ_FOPS(nix_tm_tree, nix_tm_tree_display, nix_tm_tree_write); + +static void print_tm_topo(struct seq_file *m, u64 schq, u32 lvl) +{ + struct nix_hw *nix_hw = m->private; + struct rvu *rvu = nix_hw->rvu; + int blkaddr, link, link_level; + struct rvu_hwinfo *hw; + + hw = rvu->hw; + blkaddr = nix_hw->blkaddr; + if (lvl == NIX_TXSCH_LVL_MDQ) { + seq_printf(m, "NIX_AF_SMQ[%llu]_CFG =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_SMQX_CFG(schq))); + seq_printf(m, "NIX_AF_SMQ[%llu]_STATUS =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_SMQX_STATUS(schq))); + seq_printf(m, "NIX_AF_MDQ[%llu]_OUT_MD_COUNT =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_MDQX_OUT_MD_COUNT(schq))); + seq_printf(m, "NIX_AF_MDQ[%llu]_SCHEDULE =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_MDQX_SCHEDULE(schq))); + seq_printf(m, "NIX_AF_MDQ[%llu]_SHAPE =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_MDQX_SHAPE(schq))); + seq_printf(m, "NIX_AF_MDQ[%llu]_CIR =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_MDQX_CIR(schq))); + seq_printf(m, "NIX_AF_MDQ[%llu]_PIR =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_MDQX_PIR(schq))); + seq_printf(m, "NIX_AF_MDQ[%llu]_SW_XOFF =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_MDQX_SW_XOFF(schq))); + seq_printf(m, "NIX_AF_MDQ[%llu]_PARENT =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_MDQX_PARENT(schq))); + seq_puts(m, "\n"); + } + + if (lvl == NIX_TXSCH_LVL_TL4) { + seq_printf(m, "NIX_AF_TL4[%llu]_SDP_LINK_CFG =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL4X_SDP_LINK_CFG(schq))); + seq_printf(m, "NIX_AF_TL4[%llu]_SCHEDULE =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL4X_SCHEDULE(schq))); + seq_printf(m, "NIX_AF_TL4[%llu]_SHAPE =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL4X_SHAPE(schq))); + seq_printf(m, "NIX_AF_TL4[%llu]_CIR =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL4X_CIR(schq))); + seq_printf(m, "NIX_AF_TL4[%llu]_PIR =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL4X_PIR(schq))); + seq_printf(m, "NIX_AF_TL4[%llu]_SW_XOFF =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL4X_SW_XOFF(schq))); + seq_printf(m, "NIX_AF_TL4[%llu]_TOPOLOGY =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL4X_TOPOLOGY(schq))); + seq_printf(m, "NIX_AF_TL4[%llu]_PARENT =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL4X_PARENT(schq))); + seq_printf(m, "NIX_AF_TL4[%llu]_MD_DEBUG0 =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL4X_MD_DEBUG0(schq))); + seq_printf(m, "NIX_AF_TL4[%llu]_MD_DEBUG1 =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL4X_MD_DEBUG1(schq))); + seq_puts(m, "\n"); + } + + if (lvl == NIX_TXSCH_LVL_TL3) { + seq_printf(m, "NIX_AF_TL3[%llu]_SCHEDULE =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL3X_SCHEDULE(schq))); + seq_printf(m, "NIX_AF_TL3[%llu]_SHAPE =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL3X_SHAPE(schq))); + seq_printf(m, "NIX_AF_TL3[%llu]_CIR =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL3X_CIR(schq))); + seq_printf(m, "NIX_AF_TL3[%llu]_PIR =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL3X_PIR(schq))); + seq_printf(m, "NIX_AF_TL3[%llu]_SW_XOFF =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL3X_SW_XOFF(schq))); + seq_printf(m, "NIX_AF_TL3[%llu]_TOPOLOGY =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL3X_TOPOLOGY(schq))); + seq_printf(m, "NIX_AF_TL3[%llu]_PARENT =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL3X_PARENT(schq))); + seq_printf(m, "NIX_AF_TL3[%llu]_MD_DEBUG0 =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL3X_MD_DEBUG0(schq))); + seq_printf(m, "NIX_AF_TL3[%llu]_MD_DEBUG1 =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL3X_MD_DEBUG1(schq))); + + link_level = rvu_read64(rvu, blkaddr, NIX_AF_PSE_CHANNEL_LEVEL) + & 0x01 ? NIX_TXSCH_LVL_TL3 : NIX_TXSCH_LVL_TL2; + if (lvl == link_level) { + seq_printf(m, + "NIX_AF_TL3_TL2[%llu]_BP_STATUS =0x%llx\n", + schq, rvu_read64(rvu, blkaddr, + NIX_AF_TL3_TL2X_BP_STATUS(schq))); + for (link = 0; link < hw->cgx_links; link++) + seq_printf(m, + "NIX_AF_TL3_TL2[%llu]_LINK[%d]_CFG =0x%llx\n", + schq, link, + rvu_read64(rvu, blkaddr, + NIX_AF_TL3_TL2X_LINKX_CFG(schq, link))); + } + seq_puts(m, "\n"); + } + + if (lvl == NIX_TXSCH_LVL_TL2) { + seq_printf(m, "NIX_AF_TL2[%llu]_SHAPE =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL2X_SHAPE(schq))); + seq_printf(m, "NIX_AF_TL2[%llu]_CIR =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL2X_CIR(schq))); + seq_printf(m, "NIX_AF_TL2[%llu]_PIR =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL2X_PIR(schq))); + seq_printf(m, "NIX_AF_TL2[%llu]_SW_XOFF =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL2X_SW_XOFF(schq))); + seq_printf(m, "NIX_AF_TL2[%llu]_TOPOLOGY =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL2X_TOPOLOGY(schq))); + seq_printf(m, "NIX_AF_TL2[%llu]_PARENT =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL2X_PARENT(schq))); + seq_printf(m, "NIX_AF_TL2[%llu]_MD_DEBUG0 =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL2X_MD_DEBUG0(schq))); + seq_printf(m, "NIX_AF_TL2[%llu]_MD_DEBUG1 =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL2X_MD_DEBUG1(schq))); + + link_level = rvu_read64(rvu, blkaddr, NIX_AF_PSE_CHANNEL_LEVEL) + & 0x01 ? NIX_TXSCH_LVL_TL3 : NIX_TXSCH_LVL_TL2; + if (lvl == link_level) { + seq_printf(m, + "NIX_AF_TL3_TL2[%llu]_BP_STATUS =0x%llx\n", + schq, rvu_read64(rvu, blkaddr, + NIX_AF_TL3_TL2X_BP_STATUS(schq))); + for (link = 0; link < hw->cgx_links; link++) + seq_printf(m, + "NIX_AF_TL3_TL2[%llu]_LINK[%d]_CFG =0x%llx\n", + schq, link, rvu_read64(rvu, blkaddr, + NIX_AF_TL3_TL2X_LINKX_CFG(schq, link))); + } + seq_puts(m, "\n"); + } + + if (lvl == NIX_TXSCH_LVL_TL1) { + seq_printf(m, "NIX_AF_TX_LINK[%llu]_NORM_CREDIT =0x%llx\n", + schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TX_LINKX_NORM_CREDIT(schq))); + seq_printf(m, "NIX_AF_TX_LINK[%llu]_HW_XOFF =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TX_LINKX_HW_XOFF(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_SCHEDULE =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_SCHEDULE(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_SHAPE =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL1X_SHAPE(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_CIR =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL1X_CIR(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_SW_XOFF =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, NIX_AF_TL1X_SW_XOFF(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_TOPOLOGY =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_TOPOLOGY(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_MD_DEBUG0 =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_MD_DEBUG0(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_MD_DEBUG1 =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_MD_DEBUG1(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_DROPPED_PACKETS =0x%llx\n", + schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_DROPPED_PACKETS(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_DROPPED_BYTES =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_DROPPED_BYTES(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_RED_PACKETS =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_RED_PACKETS(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_RED_BYTES =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_RED_BYTES(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_YELLOW_PACKETS =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_YELLOW_PACKETS(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_YELLOW_BYTES =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_YELLOW_BYTES(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_GREEN_PACKETS =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_GREEN_PACKETS(schq))); + seq_printf(m, "NIX_AF_TL1[%llu]_GREEN_BYTES =0x%llx\n", schq, + rvu_read64(rvu, blkaddr, + NIX_AF_TL1X_GREEN_BYTES(schq))); + seq_puts(m, "\n"); + } +} + +/*dumps given tm_topo registers*/ +static int rvu_dbg_nix_tm_topo_display(struct seq_file *m, void *unused) +{ + struct nix_hw *nix_hw = m->private; + struct rvu *rvu = nix_hw->rvu; + struct nix_aq_enq_req aq_req; + struct nix_txsch *txsch; + int nixlf, lvl, schq; + u16 pcifunc; + + nixlf = rvu->rvu_dbg.nix_tm_ctx.lf; + + if (!rvu_dbg_is_valid_lf(rvu, nix_hw->blkaddr, nixlf, &pcifunc)) + return -EINVAL; + + memset(&aq_req, 0, sizeof(struct nix_aq_enq_req)); + aq_req.hdr.pcifunc = pcifunc; + aq_req.ctype = NIX_AQ_CTYPE_SQ; + aq_req.op = NIX_AQ_INSTOP_READ; + seq_printf(m, "pcifunc is 0x%x\n", pcifunc); + + for (lvl = 0; lvl < NIX_TXSCH_LVL_CNT; lvl++) { + txsch = &nix_hw->txsch[lvl]; + for (schq = 0; schq < txsch->schq.max; schq++) { + if (TXSCH_MAP_FUNC(txsch->pfvf_map[schq]) == pcifunc) + print_tm_topo(m, schq, lvl); + } + } + return 0; +} + +static ssize_t rvu_dbg_nix_tm_topo_write(struct file *filp, + const char __user *buffer, + size_t count, loff_t *ppos) +{ + struct seq_file *m = filp->private_data; + struct nix_hw *nix_hw = m->private; + struct rvu *rvu = nix_hw->rvu; + struct rvu_pfvf *pfvf; + u16 pcifunc; + u64 nixlf; + int ret; + + ret = kstrtoull_from_user(buffer, count, 10, &nixlf); + if (ret) + return ret; + + if (!rvu_dbg_is_valid_lf(rvu, nix_hw->blkaddr, nixlf, &pcifunc)) + return -EINVAL; + + pfvf = rvu_get_pfvf(rvu, pcifunc); + if (!pfvf->sq_ctx) { + dev_warn(rvu->dev, "SQ context is not initialized\n"); + return -EINVAL; + } + + rvu->rvu_dbg.nix_tm_ctx.lf = nixlf; + return count; +} + +RVU_DEBUG_SEQ_FOPS(nix_tm_topo, nix_tm_topo_display, nix_tm_topo_write); + /* Dumps given nix_sq's context */ static void print_nix_sq_ctx(struct seq_file *m, struct nix_aq_enq_rsp *rsp) { @@ -2349,6 +2710,10 @@ static void rvu_dbg_nix_init(struct rvu *rvu, int blkaddr) nix_hw = &rvu->hw->nix[1]; } + debugfs_create_file("tm_tree", 0600, rvu->rvu_dbg.nix, nix_hw, + &rvu_dbg_nix_tm_tree_fops); + debugfs_create_file("tm_topo", 0600, rvu->rvu_dbg.nix, nix_hw, + &rvu_dbg_nix_tm_topo_fops); debugfs_create_file("sq_ctx", 0600, rvu->rvu_dbg.nix, nix_hw, &rvu_dbg_nix_sq_ctx_fops); debugfs_create_file("rq_ctx", 0600, rvu->rvu_dbg.nix, nix_hw, diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c index e8b73b9d75e3118f56ee42a322d05491b0c325f0..97722ce8c4cb34e2fac06adfb08c189d18989a5b 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c @@ -2519,7 +2519,17 @@ static int npc_mcam_alloc_entries(struct npc_mcam *mcam, u16 pcifunc, * - when available free entries are less. * Lower priority ones out of avaialble free entries are always * chosen when 'high vs low' question arises. + * + * For a VF base MCAM match rule is set by its PF. And all the + * further MCAM rules installed by VF on its own are + * concatenated with the base rule set by its PF. Hence PF entries + * should be at lower priority compared to VF entries. Otherwise + * base rule is hit always and rules installed by VF will be of + * no use. Hence if the request is from PF then allocate low + * priority entries. */ + if (!(pcifunc & RVU_PFVF_FUNC_MASK)) + goto lprio_alloc; /* Get the search range for priority allocation request */ if (req->priority) { @@ -2528,17 +2538,6 @@ static int npc_mcam_alloc_entries(struct npc_mcam *mcam, u16 pcifunc, goto alloc; } - /* For a VF base MCAM match rule is set by its PF. And all the - * further MCAM rules installed by VF on its own are - * concatenated with the base rule set by its PF. Hence PF entries - * should be at lower priority compared to VF entries. Otherwise - * base rule is hit always and rules installed by VF will be of - * no use. Hence if the request is from PF and NOT a priority - * allocation request then allocate low priority entries. - */ - if (!(pcifunc & RVU_PFVF_FUNC_MASK)) - goto lprio_alloc; - /* Find out the search range for non-priority allocation request * * Get MCAM free entry count in middle zone. @@ -2568,6 +2567,18 @@ lprio_alloc: reverse = true; start = 0; end = mcam->bmap_entries; + /* Ensure PF requests are always at bottom and if PF requests + * for higher/lower priority entry wrt reference entry then + * honour that criteria and start search for entries from bottom + * and not in mid zone. + */ + if (!(pcifunc & RVU_PFVF_FUNC_MASK) && + req->priority == NPC_MCAM_HIGHER_PRIO) + end = req->ref_entry; + + if (!(pcifunc & RVU_PFVF_FUNC_MASK) && + req->priority == NPC_MCAM_LOWER_PRIO) + start = req->ref_entry; } alloc: diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h b/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h index 086f05c0376f5e957ed316fc59a838bc0a6bf648..5ec92654e7ad711efdeb6c6912cfe9c2b352ea12 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h @@ -429,6 +429,8 @@ #define NIX_AF_RX_ACTIVE_CYCLES_PCX(a) (0x4800 | (a) << 16) #define NIX_AF_LINKX_CFG(a) (0x4010 | (a) << 17) #define NIX_AF_MDQX_IN_MD_COUNT(a) (0x14e0 | (a) << 16) +#define NIX_AF_SMQX_STATUS(a) (0x730 | (a) << 16) +#define NIX_AF_MDQX_OUT_MD_COUNT(a) (0xdb0 | (a) << 16) #define NIX_PRIV_AF_INT_CFG (0x8000000) #define NIX_PRIV_LFX_CFG (0x8000010) @@ -442,6 +444,11 @@ #define NIX_CONST_MAX_BPIDS GENMASK_ULL(23, 12) #define NIX_CONST_SDP_CHANS GENMASK_ULL(11, 0) +#define NIX_AF_MDQ_PARENT_MASK GENMASK_ULL(24, 16) +#define NIX_AF_TL4_PARENT_MASK GENMASK_ULL(23, 16) +#define NIX_AF_TL3_PARENT_MASK GENMASK_ULL(23, 16) +#define NIX_AF_TL2_PARENT_MASK GENMASK_ULL(20, 16) + /* SSO */ #define SSO_AF_CONST (0x1000) #define SSO_AF_CONST1 (0x1008) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/qos.c b/drivers/net/ethernet/marvell/octeontx2/nic/qos.c index 070711df612ece7978b7e6cd0310203b635e0439..edac008099c0c41def858e1bca01d7d22e8ffe10 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/qos.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/qos.c @@ -1422,7 +1422,10 @@ static int otx2_qos_leaf_to_inner(struct otx2_nic *pfvf, u16 classid, otx2_qos_read_txschq_cfg(pfvf, node, old_cfg); /* delete the txschq nodes allocated for this node */ + otx2_qos_disable_sq(pfvf, qid); + otx2_qos_free_hw_node_schq(pfvf, node); otx2_qos_free_sw_node_schq(pfvf, node); + pfvf->qos.qid_to_sqmap[qid] = OTX2_QOS_INVALID_SQ; /* mark this node as htb inner node */ WRITE_ONCE(node->qid, OTX2_QOS_QID_INNER); @@ -1632,6 +1635,7 @@ static int otx2_qos_leaf_del_last(struct otx2_nic *pfvf, u16 classid, bool force dwrr_del_node = true; /* destroy the leaf node */ + otx2_qos_disable_sq(pfvf, qid); otx2_qos_destroy_node(pfvf, node); pfvf->qos.qid_to_sqmap[qid] = OTX2_QOS_INVALID_SQ; diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.c b/drivers/net/ethernet/mediatek/mtk_eth_soc.c index cae46290a7aee28a2d4feb4ab65ec5340dc91bf8..c84ce54a84a00e88fd463723fd66dbdd6a86c44a 100644 --- a/drivers/net/ethernet/mediatek/mtk_eth_soc.c +++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.c @@ -1131,9 +1131,9 @@ static int mtk_init_fq_dma(struct mtk_eth *eth) { const struct mtk_soc_data *soc = eth->soc; dma_addr_t phy_ring_tail; - int cnt = MTK_QDMA_RING_SIZE; + int cnt = soc->tx.fq_dma_size; dma_addr_t dma_addr; - int i; + int i, j, len; if (MTK_HAS_CAPS(eth->soc->caps, MTK_SRAM)) eth->scratch_ring = eth->sram_base; @@ -1142,40 +1142,46 @@ static int mtk_init_fq_dma(struct mtk_eth *eth) cnt * soc->tx.desc_size, ð->phy_scratch_ring, GFP_KERNEL); + if (unlikely(!eth->scratch_ring)) return -ENOMEM; - eth->scratch_head = kcalloc(cnt, MTK_QDMA_PAGE_SIZE, GFP_KERNEL); - if (unlikely(!eth->scratch_head)) - return -ENOMEM; + phy_ring_tail = eth->phy_scratch_ring + soc->tx.desc_size * (cnt - 1); - dma_addr = dma_map_single(eth->dma_dev, - eth->scratch_head, cnt * MTK_QDMA_PAGE_SIZE, - DMA_FROM_DEVICE); - if (unlikely(dma_mapping_error(eth->dma_dev, dma_addr))) - return -ENOMEM; + for (j = 0; j < DIV_ROUND_UP(soc->tx.fq_dma_size, MTK_FQ_DMA_LENGTH); j++) { + len = min_t(int, cnt - j * MTK_FQ_DMA_LENGTH, MTK_FQ_DMA_LENGTH); + eth->scratch_head[j] = kcalloc(len, MTK_QDMA_PAGE_SIZE, GFP_KERNEL); - phy_ring_tail = eth->phy_scratch_ring + soc->tx.desc_size * (cnt - 1); + if (unlikely(!eth->scratch_head[j])) + return -ENOMEM; - for (i = 0; i < cnt; i++) { - dma_addr_t addr = dma_addr + i * MTK_QDMA_PAGE_SIZE; - struct mtk_tx_dma_v2 *txd; + dma_addr = dma_map_single(eth->dma_dev, + eth->scratch_head[j], len * MTK_QDMA_PAGE_SIZE, + DMA_FROM_DEVICE); - txd = eth->scratch_ring + i * soc->tx.desc_size; - txd->txd1 = addr; - if (i < cnt - 1) - txd->txd2 = eth->phy_scratch_ring + - (i + 1) * soc->tx.desc_size; + if (unlikely(dma_mapping_error(eth->dma_dev, dma_addr))) + return -ENOMEM; - txd->txd3 = TX_DMA_PLEN0(MTK_QDMA_PAGE_SIZE); - if (MTK_HAS_CAPS(soc->caps, MTK_36BIT_DMA)) - txd->txd3 |= TX_DMA_PREP_ADDR64(addr); - txd->txd4 = 0; - if (mtk_is_netsys_v2_or_greater(eth)) { - txd->txd5 = 0; - txd->txd6 = 0; - txd->txd7 = 0; - txd->txd8 = 0; + for (i = 0; i < cnt; i++) { + struct mtk_tx_dma_v2 *txd; + + txd = eth->scratch_ring + (j * MTK_FQ_DMA_LENGTH + i) * soc->tx.desc_size; + txd->txd1 = dma_addr + i * MTK_QDMA_PAGE_SIZE; + if (j * MTK_FQ_DMA_LENGTH + i < cnt) + txd->txd2 = eth->phy_scratch_ring + + (j * MTK_FQ_DMA_LENGTH + i + 1) * soc->tx.desc_size; + + txd->txd3 = TX_DMA_PLEN0(MTK_QDMA_PAGE_SIZE); + if (MTK_HAS_CAPS(soc->caps, MTK_36BIT_DMA)) + txd->txd3 |= TX_DMA_PREP_ADDR64(dma_addr + i * MTK_QDMA_PAGE_SIZE); + + txd->txd4 = 0; + if (mtk_is_netsys_v2_or_greater(eth)) { + txd->txd5 = 0; + txd->txd6 = 0; + txd->txd7 = 0; + txd->txd8 = 0; + } } } @@ -2457,7 +2463,7 @@ static int mtk_tx_alloc(struct mtk_eth *eth) if (MTK_HAS_CAPS(soc->caps, MTK_QDMA)) ring_size = MTK_QDMA_RING_SIZE; else - ring_size = MTK_DMA_SIZE; + ring_size = soc->tx.dma_size; ring->buf = kcalloc(ring_size, sizeof(*ring->buf), GFP_KERNEL); @@ -2465,8 +2471,8 @@ static int mtk_tx_alloc(struct mtk_eth *eth) goto no_tx_mem; if (MTK_HAS_CAPS(soc->caps, MTK_SRAM)) { - ring->dma = eth->sram_base + ring_size * sz; - ring->phys = eth->phy_scratch_ring + ring_size * (dma_addr_t)sz; + ring->dma = eth->sram_base + soc->tx.fq_dma_size * sz; + ring->phys = eth->phy_scratch_ring + soc->tx.fq_dma_size * (dma_addr_t)sz; } else { ring->dma = dma_alloc_coherent(eth->dma_dev, ring_size * sz, &ring->phys, GFP_KERNEL); @@ -2588,6 +2594,7 @@ static void mtk_tx_clean(struct mtk_eth *eth) static int mtk_rx_alloc(struct mtk_eth *eth, int ring_no, int rx_flag) { const struct mtk_reg_map *reg_map = eth->soc->reg_map; + const struct mtk_soc_data *soc = eth->soc; struct mtk_rx_ring *ring; int rx_data_len, rx_dma_size, tx_ring_size; int i; @@ -2595,7 +2602,7 @@ static int mtk_rx_alloc(struct mtk_eth *eth, int ring_no, int rx_flag) if (MTK_HAS_CAPS(eth->soc->caps, MTK_QDMA)) tx_ring_size = MTK_QDMA_RING_SIZE; else - tx_ring_size = MTK_DMA_SIZE; + tx_ring_size = soc->tx.dma_size; if (rx_flag == MTK_RX_FLAGS_QDMA) { if (ring_no) @@ -2610,7 +2617,7 @@ static int mtk_rx_alloc(struct mtk_eth *eth, int ring_no, int rx_flag) rx_dma_size = MTK_HW_LRO_DMA_SIZE; } else { rx_data_len = ETH_DATA_LEN; - rx_dma_size = MTK_DMA_SIZE; + rx_dma_size = soc->rx.dma_size; } ring->frag_size = mtk_max_frag_size(rx_data_len); @@ -3139,7 +3146,10 @@ static void mtk_dma_free(struct mtk_eth *eth) mtk_rx_clean(eth, ð->rx_ring[i], false); } - kfree(eth->scratch_head); + for (i = 0; i < DIV_ROUND_UP(soc->tx.fq_dma_size, MTK_FQ_DMA_LENGTH); i++) { + kfree(eth->scratch_head[i]); + eth->scratch_head[i] = NULL; + } } static bool mtk_hw_reset_check(struct mtk_eth *eth) @@ -5052,11 +5062,14 @@ static const struct mtk_soc_data mt2701_data = { .desc_size = sizeof(struct mtk_tx_dma), .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, + .dma_size = MTK_DMA_SIZE(2K), + .fq_dma_size = MTK_DMA_SIZE(2K), }, .rx = { .desc_size = sizeof(struct mtk_rx_dma), .irq_done_mask = MTK_RX_DONE_INT, .dma_l4_valid = RX_DMA_L4_VALID, + .dma_size = MTK_DMA_SIZE(2K), .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, }, @@ -5076,11 +5089,14 @@ static const struct mtk_soc_data mt7621_data = { .desc_size = sizeof(struct mtk_tx_dma), .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, + .dma_size = MTK_DMA_SIZE(2K), + .fq_dma_size = MTK_DMA_SIZE(2K), }, .rx = { .desc_size = sizeof(struct mtk_rx_dma), .irq_done_mask = MTK_RX_DONE_INT, .dma_l4_valid = RX_DMA_L4_VALID, + .dma_size = MTK_DMA_SIZE(2K), .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, }, @@ -5102,11 +5118,14 @@ static const struct mtk_soc_data mt7622_data = { .desc_size = sizeof(struct mtk_tx_dma), .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, + .dma_size = MTK_DMA_SIZE(2K), + .fq_dma_size = MTK_DMA_SIZE(2K), }, .rx = { .desc_size = sizeof(struct mtk_rx_dma), .irq_done_mask = MTK_RX_DONE_INT, .dma_l4_valid = RX_DMA_L4_VALID, + .dma_size = MTK_DMA_SIZE(2K), .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, }, @@ -5127,11 +5146,14 @@ static const struct mtk_soc_data mt7623_data = { .desc_size = sizeof(struct mtk_tx_dma), .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, + .dma_size = MTK_DMA_SIZE(2K), + .fq_dma_size = MTK_DMA_SIZE(2K), }, .rx = { .desc_size = sizeof(struct mtk_rx_dma), .irq_done_mask = MTK_RX_DONE_INT, .dma_l4_valid = RX_DMA_L4_VALID, + .dma_size = MTK_DMA_SIZE(2K), .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, }, @@ -5150,11 +5172,14 @@ static const struct mtk_soc_data mt7629_data = { .desc_size = sizeof(struct mtk_tx_dma), .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, + .dma_size = MTK_DMA_SIZE(2K), + .fq_dma_size = MTK_DMA_SIZE(2K), }, .rx = { .desc_size = sizeof(struct mtk_rx_dma), .irq_done_mask = MTK_RX_DONE_INT, .dma_l4_valid = RX_DMA_L4_VALID, + .dma_size = MTK_DMA_SIZE(2K), .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, }, @@ -5176,6 +5201,8 @@ static const struct mtk_soc_data mt7981_data = { .desc_size = sizeof(struct mtk_tx_dma_v2), .dma_max_len = MTK_TX_DMA_BUF_LEN_V2, .dma_len_offset = 8, + .dma_size = MTK_DMA_SIZE(2K), + .fq_dma_size = MTK_DMA_SIZE(2K), }, .rx = { .desc_size = sizeof(struct mtk_rx_dma), @@ -5183,6 +5210,7 @@ static const struct mtk_soc_data mt7981_data = { .dma_l4_valid = RX_DMA_L4_VALID_V2, .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, + .dma_size = MTK_DMA_SIZE(2K), }, }; @@ -5202,6 +5230,8 @@ static const struct mtk_soc_data mt7986_data = { .desc_size = sizeof(struct mtk_tx_dma_v2), .dma_max_len = MTK_TX_DMA_BUF_LEN_V2, .dma_len_offset = 8, + .dma_size = MTK_DMA_SIZE(2K), + .fq_dma_size = MTK_DMA_SIZE(2K), }, .rx = { .desc_size = sizeof(struct mtk_rx_dma), @@ -5209,6 +5239,7 @@ static const struct mtk_soc_data mt7986_data = { .dma_l4_valid = RX_DMA_L4_VALID_V2, .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, + .dma_size = MTK_DMA_SIZE(2K), }, }; @@ -5228,6 +5259,8 @@ static const struct mtk_soc_data mt7988_data = { .desc_size = sizeof(struct mtk_tx_dma_v2), .dma_max_len = MTK_TX_DMA_BUF_LEN_V2, .dma_len_offset = 8, + .dma_size = MTK_DMA_SIZE(2K), + .fq_dma_size = MTK_DMA_SIZE(4K), }, .rx = { .desc_size = sizeof(struct mtk_rx_dma_v2), @@ -5235,6 +5268,7 @@ static const struct mtk_soc_data mt7988_data = { .dma_l4_valid = RX_DMA_L4_VALID_V2, .dma_max_len = MTK_TX_DMA_BUF_LEN_V2, .dma_len_offset = 8, + .dma_size = MTK_DMA_SIZE(2K), }, }; @@ -5249,6 +5283,7 @@ static const struct mtk_soc_data rt5350_data = { .desc_size = sizeof(struct mtk_tx_dma), .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, + .dma_size = MTK_DMA_SIZE(2K), }, .rx = { .desc_size = sizeof(struct mtk_rx_dma), @@ -5256,6 +5291,7 @@ static const struct mtk_soc_data rt5350_data = { .dma_l4_valid = RX_DMA_L4_VALID_PDMA, .dma_max_len = MTK_TX_DMA_BUF_LEN, .dma_len_offset = 16, + .dma_size = MTK_DMA_SIZE(2K), }, }; diff --git a/drivers/net/ethernet/mediatek/mtk_eth_soc.h b/drivers/net/ethernet/mediatek/mtk_eth_soc.h index 4eab30b44070633a2357f8517a6ad1e5c8340149..f5174f6cb1bbec5bfc525f31e6b7359b8e900634 100644 --- a/drivers/net/ethernet/mediatek/mtk_eth_soc.h +++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.h @@ -32,7 +32,9 @@ #define MTK_TX_DMA_BUF_LEN 0x3fff #define MTK_TX_DMA_BUF_LEN_V2 0xffff #define MTK_QDMA_RING_SIZE 2048 -#define MTK_DMA_SIZE 512 +#define MTK_DMA_SIZE(x) (SZ_##x) +#define MTK_FQ_DMA_HEAD 32 +#define MTK_FQ_DMA_LENGTH 2048 #define MTK_RX_ETH_HLEN (ETH_HLEN + ETH_FCS_LEN) #define MTK_RX_HLEN (NET_SKB_PAD + MTK_RX_ETH_HLEN + NET_IP_ALIGN) #define MTK_DMA_DUMMY_DESC 0xffffffff @@ -1176,6 +1178,8 @@ struct mtk_soc_data { u32 desc_size; u32 dma_max_len; u32 dma_len_offset; + u32 dma_size; + u32 fq_dma_size; } tx; struct { u32 desc_size; @@ -1183,6 +1187,7 @@ struct mtk_soc_data { u32 dma_l4_valid; u32 dma_max_len; u32 dma_len_offset; + u32 dma_size; } rx; }; @@ -1264,7 +1269,7 @@ struct mtk_eth { struct napi_struct rx_napi; void *scratch_ring; dma_addr_t phy_scratch_ring; - void *scratch_head; + void *scratch_head[MTK_FQ_DMA_HEAD]; struct clk *clks[MTK_CLK_MAX]; struct mii_bus *mii_bus; diff --git a/drivers/net/ethernet/mellanox/mlx4/en_cq.c b/drivers/net/ethernet/mellanox/mlx4/en_cq.c index 1184ac5751e1becb8c50ab9cbe9358aea50ba5c0..461cc2c79c71f540c114ac496bc3e2b9d98b1b41 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_cq.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_cq.c @@ -126,6 +126,7 @@ int mlx4_en_activate_cq(struct mlx4_en_priv *priv, struct mlx4_en_cq *cq, cq_idx = cq_idx % priv->rx_ring_num; rx_cq = priv->rx_cq[cq_idx]; cq->vector = rx_cq->vector; + irq = mlx4_eq_get_irq(mdev->dev, cq->vector); } if (cq->type == RX) @@ -142,18 +143,23 @@ int mlx4_en_activate_cq(struct mlx4_en_priv *priv, struct mlx4_en_cq *cq, if (err) goto free_eq; + cq->cq_idx = cq_idx; cq->mcq.event = mlx4_en_cq_event; switch (cq->type) { case TX: cq->mcq.comp = mlx4_en_tx_irq; netif_napi_add_tx(cq->dev, &cq->napi, mlx4_en_poll_tx_cq); + netif_napi_set_irq(&cq->napi, irq); napi_enable(&cq->napi); + netif_queue_set_napi(cq->dev, cq_idx, NETDEV_QUEUE_TYPE_TX, &cq->napi); break; case RX: cq->mcq.comp = mlx4_en_rx_irq; netif_napi_add(cq->dev, &cq->napi, mlx4_en_poll_rx_cq); + netif_napi_set_irq(&cq->napi, irq); napi_enable(&cq->napi); + netif_queue_set_napi(cq->dev, cq_idx, NETDEV_QUEUE_TYPE_RX, &cq->napi); break; case TX_XDP: /* nothing regarding napi, it's shared with rx ring */ @@ -189,6 +195,14 @@ void mlx4_en_destroy_cq(struct mlx4_en_priv *priv, struct mlx4_en_cq **pcq) void mlx4_en_deactivate_cq(struct mlx4_en_priv *priv, struct mlx4_en_cq *cq) { if (cq->type != TX_XDP) { + enum netdev_queue_type qtype; + + if (cq->type == RX) + qtype = NETDEV_QUEUE_TYPE_RX; + else + qtype = NETDEV_QUEUE_TYPE_TX; + + netif_queue_set_napi(cq->dev, cq->cq_idx, qtype, NULL); napi_disable(&cq->napi); netif_napi_del(&cq->napi); } diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c index 4c089cfa027af969e5511f94bafa4e5553bd93ff..281b34af0bb48f6e9d386544736533f8872f6f97 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include @@ -2073,6 +2074,7 @@ static void mlx4_en_clear_stats(struct net_device *dev) priv->rx_ring[i]->csum_ok = 0; priv->rx_ring[i]->csum_none = 0; priv->rx_ring[i]->csum_complete = 0; + priv->rx_ring[i]->alloc_fail = 0; } } @@ -3099,6 +3101,77 @@ void mlx4_en_set_stats_bitmap(struct mlx4_dev *dev, last_i += NUM_PHY_STATS; } +static void mlx4_get_queue_stats_rx(struct net_device *dev, int i, + struct netdev_queue_stats_rx *stats) +{ + struct mlx4_en_priv *priv = netdev_priv(dev); + const struct mlx4_en_rx_ring *ring; + + spin_lock_bh(&priv->stats_lock); + + if (!priv->port_up || mlx4_is_master(priv->mdev->dev)) + goto out_unlock; + + ring = priv->rx_ring[i]; + stats->packets = READ_ONCE(ring->packets); + stats->bytes = READ_ONCE(ring->bytes); + stats->alloc_fail = READ_ONCE(ring->alloc_fail); + +out_unlock: + spin_unlock_bh(&priv->stats_lock); +} + +static void mlx4_get_queue_stats_tx(struct net_device *dev, int i, + struct netdev_queue_stats_tx *stats) +{ + struct mlx4_en_priv *priv = netdev_priv(dev); + const struct mlx4_en_tx_ring *ring; + + spin_lock_bh(&priv->stats_lock); + + if (!priv->port_up || mlx4_is_master(priv->mdev->dev)) + goto out_unlock; + + ring = priv->tx_ring[TX][i]; + stats->packets = READ_ONCE(ring->packets); + stats->bytes = READ_ONCE(ring->bytes); + +out_unlock: + spin_unlock_bh(&priv->stats_lock); +} + +static void mlx4_get_base_stats(struct net_device *dev, + struct netdev_queue_stats_rx *rx, + struct netdev_queue_stats_tx *tx) +{ + struct mlx4_en_priv *priv = netdev_priv(dev); + + spin_lock_bh(&priv->stats_lock); + + if (!priv->port_up || mlx4_is_master(priv->mdev->dev)) + goto out_unlock; + + if (priv->rx_ring_num) { + rx->packets = 0; + rx->bytes = 0; + rx->alloc_fail = 0; + } + + if (priv->tx_ring_num[TX]) { + tx->packets = 0; + tx->bytes = 0; + } + +out_unlock: + spin_unlock_bh(&priv->stats_lock); +} + +static const struct netdev_stat_ops mlx4_stat_ops = { + .get_queue_stats_rx = mlx4_get_queue_stats_rx, + .get_queue_stats_tx = mlx4_get_queue_stats_tx, + .get_base_stats = mlx4_get_base_stats, +}; + int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port, struct mlx4_en_port_profile *prof) { @@ -3262,6 +3335,7 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port, netif_set_real_num_tx_queues(dev, priv->tx_ring_num[TX]); netif_set_real_num_rx_queues(dev, priv->rx_ring_num); + dev->stat_ops = &mlx4_stat_ops; dev->ethtool_ops = &mlx4_en_ethtool_ops; /* diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index 8328df8645d504ae70008440f0ca948a1d502ab4..15c57e9517e9a118a2b6a96dbc1365431406f947 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -82,8 +82,10 @@ static int mlx4_en_alloc_frags(struct mlx4_en_priv *priv, for (i = 0; i < priv->num_frags; i++, frags++) { if (!frags->page) { - if (mlx4_alloc_page(priv, frags, gfp)) + if (mlx4_alloc_page(priv, frags, gfp)) { + ring->alloc_fail++; return -ENOMEM; + } ring->rx_alloc_pages++; } rx_desc->data[i].addr = cpu_to_be64(frags->dma + diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c index 98688e4dbec5e35de698797d21fcdc0a64124e58..febeadfdd5a584943ea1fd5a0b047bf42872506a 100644 --- a/drivers/net/ethernet/mellanox/mlx4/main.c +++ b/drivers/net/ethernet/mellanox/mlx4/main.c @@ -169,12 +169,6 @@ module_param_array(port_type_array, int, &arr_argc, 0444); MODULE_PARM_DESC(port_type_array, "Array of port types: HW_DEFAULT (0) is default " "1 for IB, 2 for Ethernet"); -struct mlx4_port_config { - struct list_head list; - enum mlx4_port_type port_type[MLX4_MAX_PORTS + 1]; - struct pci_dev *pdev; -}; - static atomic_t pf_loading = ATOMIC_INIT(0); static int mlx4_devlink_ierr_reset_get(struct devlink *devlink, u32 id, diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h index efe3f97b874fec780e40f64b52fbe27d73ff78f0..28b70dcc652e6970094427714685857b6361c207 100644 --- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h +++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h @@ -355,6 +355,7 @@ struct mlx4_en_rx_ring { unsigned long xdp_tx; unsigned long xdp_tx_full; unsigned long dropped; + unsigned long alloc_fail; int hwtstamp_rx_filter; cpumask_var_t affinity_mask; struct xdp_rxq_info xdp_rxq; @@ -379,6 +380,7 @@ struct mlx4_en_cq { #define MLX4_EN_OPCODE_ERROR 0x1e const struct cpumask *aff_mask; + int cq_idx; }; struct mlx4_en_port_profile { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index e85fb71bf0b46239c07a968eab443cef4650b4a1..bec784d25d7bea0453b5a681f5ea712383d97bc8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -80,6 +80,7 @@ struct page_pool; SKB_DATA_ALIGN(sizeof(struct skb_shared_info))) #define MLX5E_RX_MAX_HEAD (256) +#define MLX5E_SHAMPO_LOG_HEADER_ENTRY_SIZE (8) #define MLX5E_SHAMPO_LOG_MAX_HEADER_ENTRY_SIZE (9) #define MLX5E_SHAMPO_WQ_HEADER_PER_PAGE (PAGE_SIZE >> MLX5E_SHAMPO_LOG_MAX_HEADER_ENTRY_SIZE) #define MLX5E_SHAMPO_WQ_BASE_HEAD_ENTRY_SIZE (64) @@ -146,25 +147,6 @@ struct page_pool; #define MLX5E_TX_XSK_POLL_BUDGET 64 #define MLX5E_SQ_RECOVER_MIN_INTERVAL 500 /* msecs */ -#define MLX5E_KLM_UMR_WQE_SZ(sgl_len)\ - (sizeof(struct mlx5e_umr_wqe) +\ - (sizeof(struct mlx5_klm) * (sgl_len))) - -#define MLX5E_KLM_UMR_WQEBBS(klm_entries) \ - (DIV_ROUND_UP(MLX5E_KLM_UMR_WQE_SZ(klm_entries), MLX5_SEND_WQE_BB)) - -#define MLX5E_KLM_UMR_DS_CNT(klm_entries)\ - (DIV_ROUND_UP(MLX5E_KLM_UMR_WQE_SZ(klm_entries), MLX5_SEND_WQE_DS)) - -#define MLX5E_KLM_MAX_ENTRIES_PER_WQE(wqe_size)\ - (((wqe_size) - sizeof(struct mlx5e_umr_wqe)) / sizeof(struct mlx5_klm)) - -#define MLX5E_KLM_ENTRIES_PER_WQE(wqe_size)\ - ALIGN_DOWN(MLX5E_KLM_MAX_ENTRIES_PER_WQE(wqe_size), MLX5_UMR_KLM_NUM_ENTRIES_ALIGNMENT) - -#define MLX5E_MAX_KLM_PER_WQE(mdev) \ - MLX5E_KLM_ENTRIES_PER_WQE(MLX5_SEND_WQE_BB * mlx5e_get_max_sq_aligned_wqebbs(mdev)) - #define mlx5e_state_dereference(priv, p) \ rcu_dereference_protected((p), lockdep_is_held(&(priv)->state_lock)) @@ -1014,7 +996,7 @@ void mlx5e_build_ptys2ethtool_map(void); bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev, u8 page_shift, enum mlx5e_mpwrq_umr_mode umr_mode); -void mlx5e_shampo_dealloc_hd(struct mlx5e_rq *rq, u16 len, u16 start, bool close); +void mlx5e_shampo_dealloc_hd(struct mlx5e_rq *rq); void mlx5e_get_stats(struct net_device *dev, struct rtnl_link_stats64 *stats); void mlx5e_fold_sw_stats64(struct mlx5e_priv *priv, struct rtnl_link_stats64 *s); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c index ec819dfc98be211b0feecaf04132c46c5604f92d..6c9ccccca81e27dc51829ed05d1ca3259a9baa92 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c @@ -1071,18 +1071,18 @@ static u32 mlx5e_shampo_icosq_sz(struct mlx5_core_dev *mdev, struct mlx5e_params *params, struct mlx5e_rq_param *rq_param) { - int max_num_of_umr_per_wqe, max_hd_per_wqe, max_klm_per_umr, rest; + int max_num_of_umr_per_wqe, max_hd_per_wqe, max_ksm_per_umr, rest; void *wqc = MLX5_ADDR_OF(rqc, rq_param->rqc, wq); int wq_size = BIT(MLX5_GET(wq, wqc, log_wq_sz)); u32 wqebbs; - max_klm_per_umr = MLX5E_MAX_KLM_PER_WQE(mdev); + max_ksm_per_umr = MLX5E_MAX_KSM_PER_WQE(mdev); max_hd_per_wqe = mlx5e_shampo_hd_per_wqe(mdev, params, rq_param); - max_num_of_umr_per_wqe = max_hd_per_wqe / max_klm_per_umr; - rest = max_hd_per_wqe % max_klm_per_umr; - wqebbs = MLX5E_KLM_UMR_WQEBBS(max_klm_per_umr) * max_num_of_umr_per_wqe; + max_num_of_umr_per_wqe = max_hd_per_wqe / max_ksm_per_umr; + rest = max_hd_per_wqe % max_ksm_per_umr; + wqebbs = MLX5E_KSM_UMR_WQEBBS(max_ksm_per_umr) * max_num_of_umr_per_wqe; if (rest) - wqebbs += MLX5E_KLM_UMR_WQEBBS(rest); + wqebbs += MLX5E_KSM_UMR_WQEBBS(rest); wqebbs *= wq_size; return wqebbs; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h index 879d698b6119303836915d5dc19f6f910d506890..d1f0f868d494e7bbc201087febbf7760d77588b6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h @@ -34,6 +34,25 @@ #define MLX5E_RX_ERR_CQE(cqe) (get_cqe_opcode(cqe) != MLX5_CQE_RESP_SEND) +#define MLX5E_KSM_UMR_WQE_SZ(sgl_len)\ + (sizeof(struct mlx5e_umr_wqe) +\ + (sizeof(struct mlx5_ksm) * (sgl_len))) + +#define MLX5E_KSM_UMR_WQEBBS(ksm_entries) \ + (DIV_ROUND_UP(MLX5E_KSM_UMR_WQE_SZ(ksm_entries), MLX5_SEND_WQE_BB)) + +#define MLX5E_KSM_UMR_DS_CNT(ksm_entries)\ + (DIV_ROUND_UP(MLX5E_KSM_UMR_WQE_SZ(ksm_entries), MLX5_SEND_WQE_DS)) + +#define MLX5E_KSM_MAX_ENTRIES_PER_WQE(wqe_size)\ + (((wqe_size) - sizeof(struct mlx5e_umr_wqe)) / sizeof(struct mlx5_ksm)) + +#define MLX5E_KSM_ENTRIES_PER_WQE(wqe_size)\ + ALIGN_DOWN(MLX5E_KSM_MAX_ENTRIES_PER_WQE(wqe_size), MLX5_UMR_KSM_NUM_ENTRIES_ALIGNMENT) + +#define MLX5E_MAX_KSM_PER_WQE(mdev) \ + MLX5E_KSM_ENTRIES_PER_WQE(MLX5_SEND_WQE_BB * mlx5e_get_max_sq_aligned_wqebbs(mdev)) + static inline ktime_t mlx5e_cqe_ts_to_ns(cqe_ts_to_ns func, struct mlx5_clock *clock, u64 cqe_ts) { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h index caa34b9c161e5188a9ac1607a01e8c390d95d568..33e32584b07f57e631a5ee801caacb0b4298af1d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h @@ -102,8 +102,14 @@ static inline void mlx5e_udp_gso_handle_tx_skb(struct sk_buff *skb) { int payload_len = skb_shinfo(skb)->gso_size + sizeof(struct udphdr); + struct udphdr *udphdr; - udp_hdr(skb)->len = htons(payload_len); + if (skb->encapsulation) + udphdr = (struct udphdr *)skb_inner_transport_header(skb); + else + udphdr = udp_hdr(skb); + + udphdr->len = htons(payload_len); } struct mlx5e_accel_tx_state { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c index 41a2543a52cda094dc15a1f863edb4ffa8b6d492..e51b03d4c717f1a740d3536fda4d492b3f57f198 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c @@ -750,8 +750,7 @@ err_fs: err_fs_ft: if (rx->allow_tunnel_mode) mlx5_eswitch_unblock_encap(mdev); - mlx5_del_flow_rules(rx->status.rule); - mlx5_modify_header_dealloc(mdev, rx->status.modify_hdr); + mlx5_ipsec_rx_status_destroy(ipsec, rx); err_add: mlx5_destroy_flow_table(rx->ft.status); err_fs_ft_status: diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h index 82064614846f5f88646b12a887bf548ccbe47639..359050f0b54ddc25980bfe7b2ebeb6a9f43c9fde 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h @@ -97,18 +97,11 @@ mlx5e_ipsec_feature_check(struct sk_buff *skb, netdev_features_t features) if (!x || !x->xso.offload_handle) goto out_disable; - if (xo->inner_ipproto) { - /* Cannot support tunnel packet over IPsec tunnel mode - * because we cannot offload three IP header csum - */ - if (x->props.mode == XFRM_MODE_TUNNEL) - goto out_disable; - - /* Only support UDP or TCP L4 checksum */ - if (xo->inner_ipproto != IPPROTO_UDP && - xo->inner_ipproto != IPPROTO_TCP) - goto out_disable; - } + /* Only support UDP or TCP L4 checksum */ + if (xo->inner_ipproto && + xo->inner_ipproto != IPPROTO_UDP && + xo->inner_ipproto != IPPROTO_TCP) + goto out_disable; return features; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index b758bc72ac36bb61b40da488659368548c825005..44a64d062e4225fc2ab8dc40e6cbafd95c4555eb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -74,6 +74,27 @@ #include "lib/devcom.h" #include "lib/sd.h" +static bool mlx5e_hw_gro_supported(struct mlx5_core_dev *mdev) +{ + if (!MLX5_CAP_GEN(mdev, shampo)) + return false; + + /* Our HW-GRO implementation relies on "KSM Mkey" for + * SHAMPO headers buffer mapping + */ + if (!MLX5_CAP_GEN(mdev, fixed_buffer_size)) + return false; + + if (!MLX5_CAP_GEN_2(mdev, min_mkey_log_entity_size_fixed_buffer_valid)) + return false; + + if (MLX5_CAP_GEN_2(mdev, min_mkey_log_entity_size_fixed_buffer) > + MLX5E_SHAMPO_LOG_HEADER_ENTRY_SIZE) + return false; + + return true; +} + bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev, u8 page_shift, enum mlx5e_mpwrq_umr_mode umr_mode) { @@ -504,8 +525,8 @@ static int mlx5e_create_umr_mkey(struct mlx5_core_dev *mdev, return err; } -static int mlx5e_create_umr_klm_mkey(struct mlx5_core_dev *mdev, - u64 nentries, +static int mlx5e_create_umr_ksm_mkey(struct mlx5_core_dev *mdev, + u64 nentries, u8 log_entry_size, u32 *umr_mkey) { int inlen; @@ -525,12 +546,13 @@ static int mlx5e_create_umr_klm_mkey(struct mlx5_core_dev *mdev, MLX5_SET(mkc, mkc, umr_en, 1); MLX5_SET(mkc, mkc, lw, 1); MLX5_SET(mkc, mkc, lr, 1); - MLX5_SET(mkc, mkc, access_mode_1_0, MLX5_MKC_ACCESS_MODE_KLMS); + MLX5_SET(mkc, mkc, access_mode_1_0, MLX5_MKC_ACCESS_MODE_KSM); mlx5e_mkey_set_relaxed_ordering(mdev, mkc); MLX5_SET(mkc, mkc, qpn, 0xffffff); MLX5_SET(mkc, mkc, pd, mdev->mlx5e_res.hw_objs.pdn); MLX5_SET(mkc, mkc, translations_octword_size, nentries); - MLX5_SET(mkc, mkc, length64, 1); + MLX5_SET(mkc, mkc, log_page_size, log_entry_size); + MLX5_SET64(mkc, mkc, len, nentries << log_entry_size); err = mlx5_core_create_mkey(mdev, umr_mkey, in, inlen); kvfree(in); @@ -565,14 +587,16 @@ static int mlx5e_create_rq_umr_mkey(struct mlx5_core_dev *mdev, struct mlx5e_rq static int mlx5e_create_rq_hd_umr_mkey(struct mlx5_core_dev *mdev, struct mlx5e_rq *rq) { - u32 max_klm_size = BIT(MLX5_CAP_GEN(mdev, log_max_klm_list_size)); + u32 max_ksm_size = BIT(MLX5_CAP_GEN(mdev, log_max_klm_list_size)); - if (max_klm_size < rq->mpwqe.shampo->hd_per_wq) { - mlx5_core_err(mdev, "max klm list size 0x%x is smaller than shampo header buffer list size 0x%x\n", - max_klm_size, rq->mpwqe.shampo->hd_per_wq); + if (max_ksm_size < rq->mpwqe.shampo->hd_per_wq) { + mlx5_core_err(mdev, "max ksm list size 0x%x is smaller than shampo header buffer list size 0x%x\n", + max_ksm_size, rq->mpwqe.shampo->hd_per_wq); return -EINVAL; } - return mlx5e_create_umr_klm_mkey(mdev, rq->mpwqe.shampo->hd_per_wq, + + return mlx5e_create_umr_ksm_mkey(mdev, rq->mpwqe.shampo->hd_per_wq, + MLX5E_SHAMPO_LOG_HEADER_ENTRY_SIZE, &rq->mpwqe.shampo->mkey); } @@ -1208,15 +1232,6 @@ void mlx5e_free_rx_missing_descs(struct mlx5e_rq *rq) head = mlx5_wq_ll_get_wqe_next_ix(wq, head); } - if (test_bit(MLX5E_RQ_STATE_SHAMPO, &rq->state)) { - u16 len; - - len = (rq->mpwqe.shampo->pi - rq->mpwqe.shampo->ci) & - (rq->mpwqe.shampo->hd_per_wq - 1); - mlx5e_shampo_dealloc_hd(rq, len, rq->mpwqe.shampo->ci, false); - rq->mpwqe.shampo->pi = rq->mpwqe.shampo->ci; - } - rq->mpwqe.actual_wq_head = wq->head; rq->mpwqe.umr_in_progress = 0; rq->mpwqe.umr_completed = 0; @@ -1244,8 +1259,7 @@ void mlx5e_free_rx_descs(struct mlx5e_rq *rq) } if (test_bit(MLX5E_RQ_STATE_SHAMPO, &rq->state)) - mlx5e_shampo_dealloc_hd(rq, rq->mpwqe.shampo->hd_per_wq, - 0, true); + mlx5e_shampo_dealloc_hd(rq); } else { struct mlx5_wq_cyc *wq = &rq->wqe.wq; u16 missing = mlx5_wq_cyc_missing(wq); @@ -3886,7 +3900,7 @@ mlx5e_get_stats(struct net_device *dev, struct rtnl_link_stats64 *stats) mlx5e_fold_sw_stats64(priv, stats); } - stats->rx_dropped = priv->stats.qcnt.rx_out_of_buffer; + stats->rx_missed_errors = priv->stats.qcnt.rx_out_of_buffer; stats->rx_length_errors = PPORT_802_3_GET(pstats, a_in_range_length_errors) + @@ -4259,13 +4273,19 @@ int mlx5e_set_features(struct net_device *netdev, netdev_features_t features) #define MLX5E_HANDLE_FEATURE(feature, handler) \ mlx5e_handle_feature(netdev, &oper_features, feature, handler) - err |= MLX5E_HANDLE_FEATURE(NETIF_F_LRO, set_feature_lro); - err |= MLX5E_HANDLE_FEATURE(NETIF_F_GRO_HW, set_feature_hw_gro); + if (features & (NETIF_F_GRO_HW | NETIF_F_LRO)) { + err |= MLX5E_HANDLE_FEATURE(NETIF_F_RXFCS, set_feature_rx_fcs); + err |= MLX5E_HANDLE_FEATURE(NETIF_F_LRO, set_feature_lro); + err |= MLX5E_HANDLE_FEATURE(NETIF_F_GRO_HW, set_feature_hw_gro); + } else { + err |= MLX5E_HANDLE_FEATURE(NETIF_F_LRO, set_feature_lro); + err |= MLX5E_HANDLE_FEATURE(NETIF_F_GRO_HW, set_feature_hw_gro); + err |= MLX5E_HANDLE_FEATURE(NETIF_F_RXFCS, set_feature_rx_fcs); + } err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_VLAN_CTAG_FILTER, set_feature_cvlan_filter); err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_TC, set_feature_hw_tc); err |= MLX5E_HANDLE_FEATURE(NETIF_F_RXALL, set_feature_rx_all); - err |= MLX5E_HANDLE_FEATURE(NETIF_F_RXFCS, set_feature_rx_fcs); err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_VLAN_CTAG_RX, set_feature_rx_vlan); #ifdef CONFIG_MLX5_EN_ARFS err |= MLX5E_HANDLE_FEATURE(NETIF_F_NTUPLE, set_feature_arfs); @@ -5332,6 +5352,11 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev) netdev->hw_features |= NETIF_F_HW_VLAN_CTAG_FILTER; netdev->hw_features |= NETIF_F_HW_VLAN_STAG_TX; + if (mlx5e_hw_gro_supported(mdev) && + mlx5e_check_fragmented_striding_rq_cap(mdev, PAGE_SHIFT, + MLX5E_MPWRQ_UMR_MODE_ALIGNED)) + netdev->hw_features |= NETIF_F_GRO_HW; + if (mlx5e_tunnel_any_tx_proto_supported(mdev)) { netdev->hw_enc_features |= NETIF_F_HW_CSUM; netdev->hw_enc_features |= NETIF_F_TSO; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index b5333da20e8a7c8c36a86e53ab3b36edd6540098..43f018567fafc047424f5c44d98292b99dc7c40f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -523,15 +523,23 @@ mlx5e_add_skb_shared_info_frag(struct mlx5e_rq *rq, struct skb_shared_info *sinf static inline void mlx5e_add_skb_frag(struct mlx5e_rq *rq, struct sk_buff *skb, - struct page *page, u32 frag_offset, u32 len, + struct mlx5e_frag_page *frag_page, + u32 frag_offset, u32 len, unsigned int truesize) { - dma_addr_t addr = page_pool_get_dma_addr(page); + dma_addr_t addr = page_pool_get_dma_addr(frag_page->page); + u8 next_frag = skb_shinfo(skb)->nr_frags; dma_sync_single_for_cpu(rq->pdev, addr + frag_offset, len, rq->buff.map_dir); - skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, - page, frag_offset, len, truesize); + + if (skb_can_coalesce(skb, next_frag, frag_page->page, frag_offset)) { + skb_coalesce_rx_frag(skb, next_frag - 1, len, truesize); + } else { + frag_page->frags++; + skb_add_rx_frag(skb, next_frag, frag_page->page, + frag_offset, len, truesize); + } } static inline void @@ -619,25 +627,25 @@ static int bitmap_find_window(unsigned long *bitmap, int len, return min(len, count); } -static void build_klm_umr(struct mlx5e_icosq *sq, struct mlx5e_umr_wqe *umr_wqe, - __be32 key, u16 offset, u16 klm_len, u16 wqe_bbs) +static void build_ksm_umr(struct mlx5e_icosq *sq, struct mlx5e_umr_wqe *umr_wqe, + __be32 key, u16 offset, u16 ksm_len) { - memset(umr_wqe, 0, offsetof(struct mlx5e_umr_wqe, inline_klms)); + memset(umr_wqe, 0, offsetof(struct mlx5e_umr_wqe, inline_ksms)); umr_wqe->ctrl.opmod_idx_opcode = cpu_to_be32((sq->pc << MLX5_WQE_CTRL_WQE_INDEX_SHIFT) | MLX5_OPCODE_UMR); umr_wqe->ctrl.umr_mkey = key; umr_wqe->ctrl.qpn_ds = cpu_to_be32((sq->sqn << MLX5_WQE_CTRL_QPN_SHIFT) - | MLX5E_KLM_UMR_DS_CNT(klm_len)); + | MLX5E_KSM_UMR_DS_CNT(ksm_len)); umr_wqe->uctrl.flags = MLX5_UMR_TRANSLATION_OFFSET_EN | MLX5_UMR_INLINE; umr_wqe->uctrl.xlt_offset = cpu_to_be16(offset); - umr_wqe->uctrl.xlt_octowords = cpu_to_be16(klm_len); + umr_wqe->uctrl.xlt_octowords = cpu_to_be16(ksm_len); umr_wqe->uctrl.mkey_mask = cpu_to_be64(MLX5_MKEY_MASK_FREE); } static int mlx5e_build_shampo_hd_umr(struct mlx5e_rq *rq, struct mlx5e_icosq *sq, - u16 klm_entries, u16 index) + u16 ksm_entries, u16 index) { struct mlx5e_shampo_hd *shampo = rq->mpwqe.shampo; u16 entries, pi, header_offset, err, wqe_bbs, new_entries; @@ -650,20 +658,20 @@ static int mlx5e_build_shampo_hd_umr(struct mlx5e_rq *rq, int headroom, i; headroom = rq->buff.headroom; - new_entries = klm_entries - (shampo->pi & (MLX5_UMR_KLM_NUM_ENTRIES_ALIGNMENT - 1)); - entries = ALIGN(klm_entries, MLX5_UMR_KLM_NUM_ENTRIES_ALIGNMENT); - wqe_bbs = MLX5E_KLM_UMR_WQEBBS(entries); + new_entries = ksm_entries - (shampo->pi & (MLX5_UMR_KSM_NUM_ENTRIES_ALIGNMENT - 1)); + entries = ALIGN(ksm_entries, MLX5_UMR_KSM_NUM_ENTRIES_ALIGNMENT); + wqe_bbs = MLX5E_KSM_UMR_WQEBBS(entries); pi = mlx5e_icosq_get_next_pi(sq, wqe_bbs); umr_wqe = mlx5_wq_cyc_get_wqe(&sq->wq, pi); - build_klm_umr(sq, umr_wqe, shampo->key, index, entries, wqe_bbs); + build_ksm_umr(sq, umr_wqe, shampo->key, index, entries); frag_page = &shampo->pages[page_index]; for (i = 0; i < entries; i++, index++) { dma_info = &shampo->info[index]; - if (i >= klm_entries || (index < shampo->pi && shampo->pi - index < - MLX5_UMR_KLM_NUM_ENTRIES_ALIGNMENT)) - goto update_klm; + if (i >= ksm_entries || (index < shampo->pi && shampo->pi - index < + MLX5_UMR_KSM_NUM_ENTRIES_ALIGNMENT)) + goto update_ksm; header_offset = (index & (MLX5E_SHAMPO_WQ_HEADER_PER_PAGE - 1)) << MLX5E_SHAMPO_LOG_MAX_HEADER_ENTRY_SIZE; if (!(header_offset & (PAGE_SIZE - 1))) { @@ -683,12 +691,11 @@ static int mlx5e_build_shampo_hd_umr(struct mlx5e_rq *rq, dma_info->frag_page = frag_page; } -update_klm: - umr_wqe->inline_klms[i].bcount = - cpu_to_be32(MLX5E_RX_MAX_HEAD); - umr_wqe->inline_klms[i].key = cpu_to_be32(lkey); - umr_wqe->inline_klms[i].va = - cpu_to_be64(dma_info->addr + headroom); +update_ksm: + umr_wqe->inline_ksms[i] = (struct mlx5_ksm) { + .key = cpu_to_be32(lkey), + .va = cpu_to_be64(dma_info->addr + headroom), + }; } sq->db.wqe_info[pi] = (struct mlx5e_icosq_wqe_info) { @@ -720,37 +727,37 @@ err_unmap: static int mlx5e_alloc_rx_hd_mpwqe(struct mlx5e_rq *rq) { struct mlx5e_shampo_hd *shampo = rq->mpwqe.shampo; - u16 klm_entries, num_wqe, index, entries_before; + u16 ksm_entries, num_wqe, index, entries_before; struct mlx5e_icosq *sq = rq->icosq; - int i, err, max_klm_entries, len; + int i, err, max_ksm_entries, len; - max_klm_entries = MLX5E_MAX_KLM_PER_WQE(rq->mdev); - klm_entries = bitmap_find_window(shampo->bitmap, + max_ksm_entries = MLX5E_MAX_KSM_PER_WQE(rq->mdev); + ksm_entries = bitmap_find_window(shampo->bitmap, shampo->hd_per_wqe, shampo->hd_per_wq, shampo->pi); - if (!klm_entries) + if (!ksm_entries) return 0; - klm_entries += (shampo->pi & (MLX5_UMR_KLM_NUM_ENTRIES_ALIGNMENT - 1)); - index = ALIGN_DOWN(shampo->pi, MLX5_UMR_KLM_NUM_ENTRIES_ALIGNMENT); + ksm_entries += (shampo->pi & (MLX5_UMR_KSM_NUM_ENTRIES_ALIGNMENT - 1)); + index = ALIGN_DOWN(shampo->pi, MLX5_UMR_KSM_NUM_ENTRIES_ALIGNMENT); entries_before = shampo->hd_per_wq - index; - if (unlikely(entries_before < klm_entries)) - num_wqe = DIV_ROUND_UP(entries_before, max_klm_entries) + - DIV_ROUND_UP(klm_entries - entries_before, max_klm_entries); + if (unlikely(entries_before < ksm_entries)) + num_wqe = DIV_ROUND_UP(entries_before, max_ksm_entries) + + DIV_ROUND_UP(ksm_entries - entries_before, max_ksm_entries); else - num_wqe = DIV_ROUND_UP(klm_entries, max_klm_entries); + num_wqe = DIV_ROUND_UP(ksm_entries, max_ksm_entries); for (i = 0; i < num_wqe; i++) { - len = (klm_entries > max_klm_entries) ? max_klm_entries : - klm_entries; + len = (ksm_entries > max_ksm_entries) ? max_ksm_entries : + ksm_entries; if (unlikely(index + len > shampo->hd_per_wq)) len = shampo->hd_per_wq - index; err = mlx5e_build_shampo_hd_umr(rq, sq, len, index); if (unlikely(err)) return err; index = (index + len) & (rq->mpwqe.shampo->hd_per_wq - 1); - klm_entries -= len; + ksm_entries -= len; } return 0; @@ -839,44 +846,28 @@ err: return err; } -/* This function is responsible to dealloc SHAMPO header buffer. - * close == true specifies that we are in the middle of closing RQ operation so - * we go over all the entries and if they are not in use we free them, - * otherwise we only go over a specific range inside the header buffer that are - * not in use. - */ -void mlx5e_shampo_dealloc_hd(struct mlx5e_rq *rq, u16 len, u16 start, bool close) +static void +mlx5e_free_rx_shampo_hd_entry(struct mlx5e_rq *rq, u16 header_index) { struct mlx5e_shampo_hd *shampo = rq->mpwqe.shampo; - struct mlx5e_frag_page *deleted_page = NULL; - int hd_per_wq = shampo->hd_per_wq; - struct mlx5e_dma_info *hd_info; - int i, index = start; - - for (i = 0; i < len; i++, index++) { - if (index == hd_per_wq) - index = 0; - - if (close && !test_bit(index, shampo->bitmap)) - continue; + u64 addr = shampo->info[header_index].addr; - hd_info = &shampo->info[index]; - hd_info->addr = ALIGN_DOWN(hd_info->addr, PAGE_SIZE); - if (hd_info->frag_page && hd_info->frag_page != deleted_page) { - deleted_page = hd_info->frag_page; - mlx5e_page_release_fragmented(rq, hd_info->frag_page); - } + if (((header_index + 1) & (MLX5E_SHAMPO_WQ_HEADER_PER_PAGE - 1)) == 0) { + struct mlx5e_dma_info *dma_info = &shampo->info[header_index]; - hd_info->frag_page = NULL; + dma_info->addr = ALIGN_DOWN(addr, PAGE_SIZE); + mlx5e_page_release_fragmented(rq, dma_info->frag_page); } + clear_bit(header_index, shampo->bitmap); +} - if (start + len > hd_per_wq) { - len -= hd_per_wq - start; - bitmap_clear(shampo->bitmap, start, hd_per_wq - start); - start = 0; - } +void mlx5e_shampo_dealloc_hd(struct mlx5e_rq *rq) +{ + struct mlx5e_shampo_hd *shampo = rq->mpwqe.shampo; + int i; - bitmap_clear(shampo->bitmap, start, len); + for_each_set_bit(i, shampo->bitmap, rq->mpwqe.shampo->hd_per_wq) + mlx5e_free_rx_shampo_hd_entry(rq, i); } static void mlx5e_dealloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix) @@ -1612,9 +1603,7 @@ static void mlx5e_shampo_complete_rx_cqe(struct mlx5e_rq *rq, struct mlx5e_rq_stats *stats = rq->stats; stats->packets++; - stats->gro_packets++; stats->bytes += cqe_bcnt; - stats->gro_bytes += cqe_bcnt; if (NAPI_GRO_CB(skb)->count != 1) return; mlx5e_build_rx_skb(cqe, cqe_bcnt, rq, skb); @@ -1964,30 +1953,24 @@ const struct mlx5e_rx_handlers mlx5e_rx_handlers_rep = { #endif static void -mlx5e_fill_skb_data(struct sk_buff *skb, struct mlx5e_rq *rq, - struct mlx5e_frag_page *frag_page, - u32 data_bcnt, u32 data_offset) +mlx5e_shampo_fill_skb_data(struct sk_buff *skb, struct mlx5e_rq *rq, + struct mlx5e_frag_page *frag_page, + u32 data_bcnt, u32 data_offset) { net_prefetchw(skb->data); - while (data_bcnt) { + do { /* Non-linear mode, hence non-XSK, which always uses PAGE_SIZE. */ u32 pg_consumed_bytes = min_t(u32, PAGE_SIZE - data_offset, data_bcnt); - unsigned int truesize; - - if (test_bit(MLX5E_RQ_STATE_SHAMPO, &rq->state)) - truesize = pg_consumed_bytes; - else - truesize = ALIGN(pg_consumed_bytes, BIT(rq->mpwqe.log_stride_sz)); + unsigned int truesize = pg_consumed_bytes; - frag_page->frags++; - mlx5e_add_skb_frag(rq, skb, frag_page->page, data_offset, + mlx5e_add_skb_frag(rq, skb, frag_page, data_offset, pg_consumed_bytes, truesize); data_bcnt -= pg_consumed_bytes; data_offset = 0; frag_page++; - } + } while (data_bcnt); } static struct sk_buff * @@ -2212,8 +2195,8 @@ mlx5e_skb_from_cqe_shampo(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, if (likely(frag_size <= BIT(MLX5E_SHAMPO_LOG_MAX_HEADER_ENTRY_SIZE))) { /* build SKB around header */ dma_sync_single_range_for_cpu(rq->pdev, head->addr, 0, frag_size, rq->buff.map_dir); - prefetchw(hdr); - prefetch(data); + net_prefetchw(hdr); + net_prefetch(data); skb = mlx5e_build_linear_skb(rq, hdr, frag_size, rx_headroom, head_size, 0); if (unlikely(!skb)) @@ -2230,7 +2213,7 @@ mlx5e_skb_from_cqe_shampo(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi, return NULL; } - prefetchw(skb->data); + net_prefetchw(skb->data); mlx5e_copy_skb_header(rq, skb, head->frag_page->page, head->addr, head_offset + rx_headroom, rx_headroom, head_size); @@ -2261,12 +2244,19 @@ mlx5e_shampo_flush_skb(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe, bool match) { struct sk_buff *skb = rq->hw_gro_data->skb; struct mlx5e_rq_stats *stats = rq->stats; + u16 gro_count = NAPI_GRO_CB(skb)->count; - stats->gro_skbs++; if (likely(skb_shinfo(skb)->nr_frags)) mlx5e_shampo_align_fragment(skb, rq->mpwqe.log_stride_sz); - if (NAPI_GRO_CB(skb)->count > 1) + if (gro_count > 1) { + stats->gro_skbs++; + stats->gro_packets += gro_count; + stats->gro_bytes += skb->data_len + skb_headlen(skb) * gro_count; + mlx5e_shampo_update_hdr(rq, cqe, match); + } else { + skb_shinfo(skb)->gso_size = 0; + } napi_gro_receive(rq->cq.napi, skb); rq->hw_gro_data->skb = NULL; } @@ -2279,21 +2269,6 @@ mlx5e_hw_gro_skb_has_enough_space(struct sk_buff *skb, u16 data_bcnt) return PAGE_SIZE * nr_frags + data_bcnt <= GRO_LEGACY_MAX_SIZE; } -static void -mlx5e_free_rx_shampo_hd_entry(struct mlx5e_rq *rq, u16 header_index) -{ - struct mlx5e_shampo_hd *shampo = rq->mpwqe.shampo; - u64 addr = shampo->info[header_index].addr; - - if (((header_index + 1) & (MLX5E_SHAMPO_WQ_HEADER_PER_PAGE - 1)) == 0) { - struct mlx5e_dma_info *dma_info = &shampo->info[header_index]; - - dma_info->addr = ALIGN_DOWN(addr, PAGE_SIZE); - mlx5e_page_release_fragmented(rq, dma_info->frag_page); - } - bitmap_clear(shampo->bitmap, header_index, 1); -} - static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe) { u16 data_bcnt = mpwrq_get_cqe_byte_cnt(cqe) - cqe->shampo.header_size; @@ -2327,8 +2302,6 @@ static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cq goto mpwrq_cqe_out; } - stats->gro_match_packets += match; - if (*skb && (!match || !(mlx5e_hw_gro_skb_has_enough_space(*skb, data_bcnt)))) { match = false; mlx5e_shampo_flush_skb(rq, cqe, match); @@ -2359,21 +2332,30 @@ static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cq } if (likely(head_size)) { - struct mlx5e_frag_page *frag_page; + if (data_bcnt) { + struct mlx5e_frag_page *frag_page; - frag_page = &wi->alloc_units.frag_pages[page_idx]; - mlx5e_fill_skb_data(*skb, rq, frag_page, data_bcnt, data_offset); + frag_page = &wi->alloc_units.frag_pages[page_idx]; + mlx5e_shampo_fill_skb_data(*skb, rq, frag_page, data_bcnt, data_offset); + } else { + stats->hds_nodata_packets++; + stats->hds_nodata_bytes += head_size; + } } mlx5e_shampo_complete_rx_cqe(rq, cqe, cqe_bcnt, *skb); - if (flush) + if (flush && rq->hw_gro_data->skb) mlx5e_shampo_flush_skb(rq, cqe, match); free_hd_entry: - mlx5e_free_rx_shampo_hd_entry(rq, header_index); + if (likely(head_size)) + mlx5e_free_rx_shampo_hd_entry(rq, header_index); mpwrq_cqe_out: if (likely(wi->consumed_strides < rq->mpwqe.num_strides)) return; + if (unlikely(!cstrides)) + return; + wq = &rq->mpwqe.wq; wqe = mlx5_wq_ll_get_wqe(wq, wqe_id); mlx5_wq_ll_pop(wq, cqe->wqe_id, &wqe->next.next_wqe_index); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c index e211c41cec06a8bd4bc776af80dba5b342f7e3f4..db1cac68292f39bbe14d01271a0fa07468baef9c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c @@ -141,7 +141,6 @@ static const struct counter_desc sw_stats_desc[] = { { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_gro_packets) }, { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_gro_bytes) }, { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_gro_skbs) }, - { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_gro_match_packets) }, { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_gro_large_hds) }, { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_ecn_mark) }, { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_removed_vlan_packets) }, @@ -343,8 +342,9 @@ static void mlx5e_stats_grp_sw_update_stats_rq_stats(struct mlx5e_sw_stats *s, s->rx_gro_packets += rq_stats->gro_packets; s->rx_gro_bytes += rq_stats->gro_bytes; s->rx_gro_skbs += rq_stats->gro_skbs; - s->rx_gro_match_packets += rq_stats->gro_match_packets; s->rx_gro_large_hds += rq_stats->gro_large_hds; + s->rx_hds_nodata_packets += rq_stats->hds_nodata_packets; + s->rx_hds_nodata_bytes += rq_stats->hds_nodata_bytes; s->rx_ecn_mark += rq_stats->ecn_mark; s->rx_removed_vlan_packets += rq_stats->removed_vlan_packets; s->rx_csum_none += rq_stats->csum_none; @@ -1186,6 +1186,9 @@ void mlx5e_stats_ts_get(struct mlx5e_priv *priv, ts_stats->err = 0; ts_stats->lost = 0; + if (!ptp) + goto out; + /* Aggregate stats across all TCs */ for (i = 0; i < ptp->num_tc; i++) { struct mlx5e_ptp_cq_stats *stats = @@ -1214,6 +1217,7 @@ void mlx5e_stats_ts_get(struct mlx5e_priv *priv, } } +out: mutex_unlock(&priv->state_lock); } @@ -2053,8 +2057,9 @@ static const struct counter_desc rq_stats_desc[] = { { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, gro_packets) }, { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, gro_bytes) }, { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, gro_skbs) }, - { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, gro_match_packets) }, { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, gro_large_hds) }, + { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, hds_nodata_packets) }, + { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, hds_nodata_bytes) }, { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, ecn_mark) }, { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, removed_vlan_packets) }, { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, wqe_err) }, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h index 650732288616dbb986fb12354371b4738ebf4407..4c5858c1dd829336080763120328a1c4ce40efda 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h @@ -153,8 +153,9 @@ struct mlx5e_sw_stats { u64 rx_gro_packets; u64 rx_gro_bytes; u64 rx_gro_skbs; - u64 rx_gro_match_packets; u64 rx_gro_large_hds; + u64 rx_hds_nodata_packets; + u64 rx_hds_nodata_bytes; u64 rx_mcast_packets; u64 rx_ecn_mark; u64 rx_removed_vlan_packets; @@ -352,8 +353,9 @@ struct mlx5e_rq_stats { u64 gro_packets; u64 gro_bytes; u64 gro_skbs; - u64 gro_match_packets; u64 gro_large_hds; + u64 hds_nodata_packets; + u64 hds_nodata_bytes; u64 mcast_packets; u64 ecn_mark; u64 removed_vlan_packets; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c index 099bf10788899018ab0fa0236c0cbeff39a43941..b09e9abd39f37ffa309bd71fb018c6fadaaf48dd 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c @@ -153,7 +153,11 @@ mlx5e_tx_get_gso_ihs(struct mlx5e_txqsq *sq, struct sk_buff *skb, int *hopbyhop) *hopbyhop = 0; if (skb->encapsulation) { - ihs = skb_inner_tcp_all_headers(skb); + if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4) + ihs = skb_inner_transport_offset(skb) + + sizeof(struct udphdr); + else + ihs = skb_inner_tcp_all_headers(skb); stats->tso_inner_packets++; stats->tso_inner_bytes += skb->len - ihs; } else { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw.c b/drivers/net/ethernet/mellanox/mlx5/core/fw.c index 2d95a9b7b44e197f67a0a6c2e6c3a1c15379a7bb..b61b7d96611413deda2c987282326d84a019531c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw.c @@ -373,6 +373,10 @@ int mlx5_cmd_fast_teardown_hca(struct mlx5_core_dev *dev) do { if (mlx5_get_nic_state(dev) == MLX5_INITIAL_SEG_NIC_INTERFACE_DISABLED) break; + if (pci_channel_offline(dev->pdev)) { + mlx5_core_err(dev, "PCI channel offline, stop waiting for NIC IFC\n"); + return -EACCES; + } cond_resched(); } while (!time_after(jiffies, end)); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c index ad38e31822df10bbf72ab9a3416bc7143efb6b63..a6329ca2d9bffbda0b5a69b4973f061144a9fa67 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/health.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c @@ -248,6 +248,10 @@ recover_from_sw_reset: do { if (mlx5_get_nic_state(dev) == MLX5_INITIAL_SEG_NIC_INTERFACE_DISABLED) break; + if (pci_channel_offline(dev->pdev)) { + mlx5_core_err(dev, "PCI channel offline, stop waiting for NIC IFC\n"); + goto unlock; + } msleep(20); } while (!time_after(jiffies, end)); @@ -317,6 +321,10 @@ int mlx5_health_wait_pci_up(struct mlx5_core_dev *dev) mlx5_core_warn(dev, "device is being removed, stop waiting for PCI\n"); return -ENODEV; } + if (pci_channel_offline(dev->pdev)) { + mlx5_core_err(dev, "PCI channel offline, stop waiting for PCI\n"); + return -EACCES; + } msleep(100); } return 0; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c index f7f0476a4a58d350b52b9966a0efaa0d6964a5a8..d0871c46b8c54376154cdf36e90c13636b7582ef 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c @@ -719,6 +719,7 @@ bool mlx5_lag_check_prereq(struct mlx5_lag *ldev) struct mlx5_core_dev *dev; u8 mode; #endif + bool roce_support; int i; for (i = 0; i < ldev->ports; i++) @@ -743,6 +744,11 @@ bool mlx5_lag_check_prereq(struct mlx5_lag *ldev) if (mlx5_sriov_is_enabled(ldev->pf[i].dev)) return false; #endif + roce_support = mlx5_get_roce_state(ldev->pf[MLX5_LAG_P1].dev); + for (i = 1; i < ldev->ports; i++) + if (mlx5_get_roce_state(ldev->pf[i].dev) != roce_support) + return false; + return true; } @@ -910,8 +916,10 @@ static void mlx5_do_bond(struct mlx5_lag *ldev) } else if (roce_lag) { dev0->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV; mlx5_rescan_drivers_locked(dev0); - for (i = 1; i < ldev->ports; i++) - mlx5_nic_vport_enable_roce(ldev->pf[i].dev); + for (i = 1; i < ldev->ports; i++) { + if (mlx5_get_roce_state(ldev->pf[i].dev)) + mlx5_nic_vport_enable_roce(ldev->pf[i].dev); + } } else if (shared_fdb) { int i; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/port_sel.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/port_sel.c index c16b462ddedf7e6914f0c0b796f68ee2ae2bf189..ab2717012b79b5bd115b73a5cd71e245c6045979 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/port_sel.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/port_sel.c @@ -88,9 +88,13 @@ static int mlx5_lag_create_port_sel_table(struct mlx5_lag *ldev, &dest, 1); if (IS_ERR(lag_definer->rules[idx])) { err = PTR_ERR(lag_definer->rules[idx]); - while (i--) - while (j--) + do { + while (j--) { + idx = i * ldev->buckets + j; mlx5_del_flow_rules(lag_definer->rules[idx]); + } + j = ldev->buckets; + } while (i--); goto destroy_fg; } } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/pci_vsc.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/pci_vsc.c index 6b774e0c2766594250271a2931b77b4540e7ba7c..d0b595ba611014bbfe16712506daf035a012fd7e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/pci_vsc.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/pci_vsc.c @@ -74,6 +74,10 @@ int mlx5_vsc_gw_lock(struct mlx5_core_dev *dev) ret = -EBUSY; goto pci_unlock; } + if (pci_channel_offline(dev->pdev)) { + ret = -EACCES; + goto pci_unlock; + } /* Check if semaphore is already locked */ ret = vsc_read(dev, VSC_SEMAPHORE_OFFSET, &lock_val); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/sd.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/sd.c index dd5d186dc6148f065b986ee5d2363940314816db..f6deb5a3f82024e528436af8fd66d238518a6588 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lib/sd.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/sd.c @@ -100,10 +100,6 @@ static bool ft_create_alias_supported(struct mlx5_core_dev *dev) static bool mlx5_sd_is_supported(struct mlx5_core_dev *dev, u8 host_buses) { - /* Feature is currently implemented for PFs only */ - if (!mlx5_core_is_pf(dev)) - return false; - /* Honor the SW implementation limit */ if (host_buses > MLX5_SD_MAX_GROUP_SZ) return false; @@ -162,6 +158,14 @@ static int sd_init(struct mlx5_core_dev *dev) bool sdm; int err; + /* Feature is currently implemented for PFs only */ + if (!mlx5_core_is_pf(dev)) + return 0; + + /* Block on embedded CPU PFs */ + if (mlx5_core_is_ecpf(dev)) + return 0; + if (!MLX5_CAP_MCAM_REG(dev, mpir)) return 0; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index 6574c145dc1e2da27d2d3a6071028dc010d0a825..459a836a5d9c15321409dc75ebc6a84e66c3fac7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -1298,6 +1298,9 @@ static int mlx5_function_teardown(struct mlx5_core_dev *dev, bool boot) if (!err) mlx5_function_disable(dev, boot); + else + mlx5_stop_health_poll(dev, boot); + return err; } diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_atcam.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_atcam.c index 4b713832fdd5591f627bab82128923796ad57222..07cb1e26ca3eb6c392bef5affd080dffb862110b 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_atcam.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_atcam.c @@ -391,7 +391,8 @@ mlxsw_sp_acl_atcam_region_entry_insert(struct mlxsw_sp *mlxsw_sp, if (err) return err; - lkey_id = aregion->ops->lkey_id_get(aregion, aentry->enc_key, erp_id); + lkey_id = aregion->ops->lkey_id_get(aregion, aentry->ht_key.enc_key, + erp_id); if (IS_ERR(lkey_id)) return PTR_ERR(lkey_id); aentry->lkey_id = lkey_id; @@ -399,7 +400,7 @@ mlxsw_sp_acl_atcam_region_entry_insert(struct mlxsw_sp *mlxsw_sp, kvdl_index = mlxsw_afa_block_first_kvdl_index(rulei->act_block); mlxsw_reg_ptce3_pack(ptce3_pl, true, MLXSW_REG_PTCE3_OP_WRITE_WRITE, priority, region->tcam_region_info, - aentry->enc_key, erp_id, + aentry->ht_key.enc_key, erp_id, aentry->delta_info.start, aentry->delta_info.mask, aentry->delta_info.value, @@ -428,7 +429,7 @@ mlxsw_sp_acl_atcam_region_entry_remove(struct mlxsw_sp *mlxsw_sp, mlxsw_reg_ptce3_pack(ptce3_pl, false, MLXSW_REG_PTCE3_OP_WRITE_WRITE, 0, region->tcam_region_info, - aentry->enc_key, erp_id, + aentry->ht_key.enc_key, erp_id, aentry->delta_info.start, aentry->delta_info.mask, aentry->delta_info.value, @@ -457,7 +458,7 @@ mlxsw_sp_acl_atcam_region_entry_action_replace(struct mlxsw_sp *mlxsw_sp, kvdl_index = mlxsw_afa_block_first_kvdl_index(rulei->act_block); mlxsw_reg_ptce3_pack(ptce3_pl, true, MLXSW_REG_PTCE3_OP_WRITE_UPDATE, priority, region->tcam_region_info, - aentry->enc_key, erp_id, + aentry->ht_key.enc_key, erp_id, aentry->delta_info.start, aentry->delta_info.mask, aentry->delta_info.value, @@ -480,26 +481,23 @@ __mlxsw_sp_acl_atcam_entry_add(struct mlxsw_sp *mlxsw_sp, int err; mlxsw_afk_encode(afk, region->key_info, &rulei->values, - aentry->ht_key.full_enc_key, mask); + aentry->ht_key.enc_key, mask); erp_mask = mlxsw_sp_acl_erp_mask_get(aregion, mask, false); if (IS_ERR(erp_mask)) return PTR_ERR(erp_mask); aentry->erp_mask = erp_mask; aentry->ht_key.erp_id = mlxsw_sp_acl_erp_mask_erp_id(erp_mask); - memcpy(aentry->enc_key, aentry->ht_key.full_enc_key, - sizeof(aentry->enc_key)); /* Compute all needed delta information and clear the delta bits - * from the encrypted key. + * from the encoded key. */ delta = mlxsw_sp_acl_erp_delta(aentry->erp_mask); aentry->delta_info.start = mlxsw_sp_acl_erp_delta_start(delta); aentry->delta_info.mask = mlxsw_sp_acl_erp_delta_mask(delta); aentry->delta_info.value = - mlxsw_sp_acl_erp_delta_value(delta, - aentry->ht_key.full_enc_key); - mlxsw_sp_acl_erp_delta_clear(delta, aentry->enc_key); + mlxsw_sp_acl_erp_delta_value(delta, aentry->ht_key.enc_key); + mlxsw_sp_acl_erp_delta_clear(delta, aentry->ht_key.enc_key); /* Add rule to the list of A-TCAM rules, assuming this * rule is intended to A-TCAM. In case this rule does diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_bloom_filter.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_bloom_filter.c index 95f63fcf4ba1fa6ca22ab6e6f008cdaa72de4516..a54eedb69a3f5be356b47a4eb215b8da223ccff7 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_bloom_filter.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_bloom_filter.c @@ -249,7 +249,7 @@ __mlxsw_sp_acl_bf_key_encode(struct mlxsw_sp_acl_atcam_region *aregion, memcpy(chunk + pad_bytes, &erp_region_id, sizeof(erp_region_id)); memcpy(chunk + key_offset, - &aentry->enc_key[chunk_key_offsets[chunk_index]], + &aentry->ht_key.enc_key[chunk_key_offsets[chunk_index]], chunk_key_len); chunk += chunk_len; } diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_erp.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_erp.c index d231f4d2888beefe838fadf6933e05ade826fdeb..9eee229303cced20f4911dbcc2e0c4ccfe8504fd 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_erp.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_erp.c @@ -1217,18 +1217,6 @@ static bool mlxsw_sp_acl_erp_delta_check(void *priv, const void *parent_obj, return err ? false : true; } -static int mlxsw_sp_acl_erp_hints_obj_cmp(const void *obj1, const void *obj2) -{ - const struct mlxsw_sp_acl_erp_key *key1 = obj1; - const struct mlxsw_sp_acl_erp_key *key2 = obj2; - - /* For hints purposes, two objects are considered equal - * in case the masks are the same. Does not matter what - * the "ctcam" value is. - */ - return memcmp(key1->mask, key2->mask, sizeof(key1->mask)); -} - static void *mlxsw_sp_acl_erp_delta_create(void *priv, void *parent_obj, void *obj) { @@ -1308,7 +1296,6 @@ static void mlxsw_sp_acl_erp_root_destroy(void *priv, void *root_priv) static const struct objagg_ops mlxsw_sp_acl_erp_objagg_ops = { .obj_size = sizeof(struct mlxsw_sp_acl_erp_key), .delta_check = mlxsw_sp_acl_erp_delta_check, - .hints_obj_cmp = mlxsw_sp_acl_erp_hints_obj_cmp, .delta_create = mlxsw_sp_acl_erp_delta_create, .delta_destroy = mlxsw_sp_acl_erp_delta_destroy, .root_create = mlxsw_sp_acl_erp_root_create, diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.h index 79a1d8606512531730c2da490e01f47ac0fc6399..010204f73ea46b7f3c3ca1fb44111b7a65113787 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.h +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.h @@ -167,9 +167,9 @@ struct mlxsw_sp_acl_atcam_region { }; struct mlxsw_sp_acl_atcam_entry_ht_key { - char full_enc_key[MLXSW_REG_PTCEX_FLEX_KEY_BLOCKS_LEN]; /* Encoded - * key. - */ + char enc_key[MLXSW_REG_PTCEX_FLEX_KEY_BLOCKS_LEN]; /* Encoded key, minus + * delta bits. + */ u8 erp_id; }; @@ -181,9 +181,6 @@ struct mlxsw_sp_acl_atcam_entry { struct rhash_head ht_node; struct list_head list; /* Member in entries_list */ struct mlxsw_sp_acl_atcam_entry_ht_key ht_key; - char enc_key[MLXSW_REG_PTCEX_FLEX_KEY_BLOCKS_LEN]; /* Encoded key, - * minus delta bits. - */ struct { u16 start; u8 mask; diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_dpipe.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_dpipe.c index ca80af06465f31af4c2f319755392f97e7d54065..fa6eddd27ecf54c635af64601f964e8ae131f42e 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_dpipe.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_dpipe.c @@ -283,7 +283,7 @@ static u64 mlxsw_sp_dpipe_table_erif_size_get(void *priv) return MLXSW_CORE_RES_GET(mlxsw_sp->core, MAX_RIFS); } -static struct devlink_dpipe_table_ops mlxsw_sp_erif_ops = { +static const struct devlink_dpipe_table_ops mlxsw_sp_erif_ops = { .matches_dump = mlxsw_sp_dpipe_table_erif_matches_dump, .actions_dump = mlxsw_sp_dpipe_table_erif_actions_dump, .entries_dump = mlxsw_sp_dpipe_table_erif_entries_dump, @@ -734,7 +734,7 @@ static u64 mlxsw_sp_dpipe_table_host4_size_get(void *priv) return mlxsw_sp_dpipe_table_host_size_get(mlxsw_sp, AF_INET); } -static struct devlink_dpipe_table_ops mlxsw_sp_host4_ops = { +static const struct devlink_dpipe_table_ops mlxsw_sp_host4_ops = { .matches_dump = mlxsw_sp_dpipe_table_host4_matches_dump, .actions_dump = mlxsw_sp_dpipe_table_host_actions_dump, .entries_dump = mlxsw_sp_dpipe_table_host4_entries_dump, @@ -811,7 +811,7 @@ static u64 mlxsw_sp_dpipe_table_host6_size_get(void *priv) return mlxsw_sp_dpipe_table_host_size_get(mlxsw_sp, AF_INET6); } -static struct devlink_dpipe_table_ops mlxsw_sp_host6_ops = { +static const struct devlink_dpipe_table_ops mlxsw_sp_host6_ops = { .matches_dump = mlxsw_sp_dpipe_table_host6_matches_dump, .actions_dump = mlxsw_sp_dpipe_table_host_actions_dump, .entries_dump = mlxsw_sp_dpipe_table_host6_entries_dump, @@ -1230,7 +1230,7 @@ mlxsw_sp_dpipe_table_adj_size_get(void *priv) return size; } -static struct devlink_dpipe_table_ops mlxsw_sp_dpipe_table_adj_ops = { +static const struct devlink_dpipe_table_ops mlxsw_sp_dpipe_table_adj_ops = { .matches_dump = mlxsw_sp_dpipe_table_adj_matches_dump, .actions_dump = mlxsw_sp_dpipe_table_adj_actions_dump, .entries_dump = mlxsw_sp_dpipe_table_adj_entries_dump, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c b/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c index 6ba8d4aca0a038b88e7f3ae3a8299ea0a55bd7e0..a7146d50f81428bcea7cc716f188328ed5555726 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c @@ -326,6 +326,11 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err_out; } +#ifdef CONFIG_PPC64 + /* Ensure MSI/MSI-X interrupts lie within addressable physical memory */ + pdev->no_64bit_msi = 1; +#endif + err = ionic_setup_one(ionic); if (err) goto err_out; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c b/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c index c3ae11a480242cd5c845b912a906ca6a4061251d..59e5a9f21105202ea7ce2ab9174ea88091fbf256 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c @@ -220,7 +220,7 @@ static int netdev_show(struct seq_file *seq, void *v) { struct net_device *netdev = seq->private; - seq_printf(seq, "%s\n", netdev->name); + seq_printf(seq, "%s\n", netdev_name(netdev)); return 0; } diff --git a/drivers/net/ethernet/pensando/ionic/ionic_if.h b/drivers/net/ethernet/pensando/ionic/ionic_if.h index 9a1825edf0d04242f726baaefa2a1b6769ff208e..9c85c0706c6e1e1e0bcf33e730757eb788961140 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_if.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_if.h @@ -71,7 +71,7 @@ enum ionic_cmd_opcode { IONIC_CMD_FW_CONTROL_V1 = 255, }; -/** +/* * enum ionic_status_code - Device command return codes */ enum ionic_status_code { @@ -112,6 +112,7 @@ enum ionic_notifyq_opcode { /** * struct ionic_admin_cmd - General admin command format * @opcode: Opcode for the command + * @rsvd: reserved byte(s) * @lif_index: LIF index * @cmd_data: Opcode-specific command bytes */ @@ -125,6 +126,7 @@ struct ionic_admin_cmd { /** * struct ionic_admin_comp - General admin command completion format * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @comp_index: Index in the descriptor ring for which this is the completion * @cmd_data: Command-specific bytes * @color: Color bit (Always 0 for commands issued to the @@ -147,6 +149,7 @@ static inline u8 color_match(u8 color, u8 done_color) /** * struct ionic_nop_cmd - NOP command * @opcode: opcode + * @rsvd: reserved byte(s) */ struct ionic_nop_cmd { u8 opcode; @@ -156,6 +159,7 @@ struct ionic_nop_cmd { /** * struct ionic_nop_comp - NOP command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) */ struct ionic_nop_comp { u8 status; @@ -166,6 +170,7 @@ struct ionic_nop_comp { * struct ionic_dev_init_cmd - Device init command * @opcode: opcode * @type: Device type + * @rsvd: reserved byte(s) */ struct ionic_dev_init_cmd { u8 opcode; @@ -176,6 +181,7 @@ struct ionic_dev_init_cmd { /** * struct ionic_dev_init_comp - Device init command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) */ struct ionic_dev_init_comp { u8 status; @@ -185,6 +191,7 @@ struct ionic_dev_init_comp { /** * struct ionic_dev_reset_cmd - Device reset command * @opcode: opcode + * @rsvd: reserved byte(s) */ struct ionic_dev_reset_cmd { u8 opcode; @@ -194,6 +201,7 @@ struct ionic_dev_reset_cmd { /** * struct ionic_dev_reset_comp - Reset command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) */ struct ionic_dev_reset_comp { u8 status; @@ -207,6 +215,7 @@ struct ionic_dev_reset_comp { * struct ionic_dev_identify_cmd - Driver/device identify command * @opcode: opcode * @ver: Highest version of identify supported by driver + * @rsvd: reserved byte(s) */ struct ionic_dev_identify_cmd { u8 opcode; @@ -218,6 +227,7 @@ struct ionic_dev_identify_cmd { * struct ionic_dev_identify_comp - Driver/device identify command completion * @status: Status of the command (enum ionic_status_code) * @ver: Version of identify returned by device + * @rsvd: reserved byte(s) */ struct ionic_dev_identify_comp { u8 status; @@ -242,6 +252,7 @@ enum ionic_os_type { * @kernel_ver: Kernel version, numeric format * @kernel_ver_str: Kernel version, string format * @driver_ver_str: Driver version, string format + * @words: word access to struct contents */ union ionic_drv_identity { struct { @@ -267,7 +278,9 @@ enum ionic_dev_capability { * union ionic_dev_identity - device identity information * @version: Version of device identify * @type: Identify type (0 for now) + * @rsvd: reserved byte(s) * @nports: Number of ports provisioned + * @rsvd2: reserved byte(s) * @nlifs: Number of LIFs provisioned * @nintrs: Number of interrupts provisioned * @ndbpgs_per_lif: Number of doorbell pages per LIF @@ -284,6 +297,7 @@ enum ionic_dev_capability { * @hwstamp_mult: Hardware tick to nanosecond multiplier. * @hwstamp_shift: Hardware tick to nanosecond divisor (power of two). * @capabilities: Device capabilities + * @words: word access to struct contents */ union ionic_dev_identity { struct { @@ -317,6 +331,7 @@ enum ionic_lif_type { * @opcode: opcode * @type: LIF type (enum ionic_lif_type) * @ver: Version of identify returned by device + * @rsvd: reserved byte(s) */ struct ionic_lif_identify_cmd { u8 opcode; @@ -329,6 +344,7 @@ struct ionic_lif_identify_cmd { * struct ionic_lif_identify_comp - LIF identify command completion * @status: Status of the command (enum ionic_status_code) * @ver: Version of identify returned by device + * @rsvd2: reserved byte(s) */ struct ionic_lif_identify_comp { u8 status; @@ -416,7 +432,7 @@ enum ionic_txq_feature { }; /** - * struct ionic_hwstamp_bits - Hardware timestamp decoding bits + * enum ionic_hwstamp_bits - Hardware timestamp decoding bits * @IONIC_HWSTAMP_INVALID: Invalid hardware timestamp value * @IONIC_HWSTAMP_CQ_NEGOFFSET: Timestamp field negative offset * from the base cq descriptor. @@ -429,6 +445,7 @@ enum ionic_hwstamp_bits { /** * struct ionic_lif_logical_qtype - Descriptor of logical to HW queue type * @qtype: Hardware Queue Type + * @rsvd: reserved byte(s) * @qid_count: Number of Queue IDs of the logical type * @qid_base: Minimum Queue ID of the logical type */ @@ -454,12 +471,14 @@ enum ionic_lif_state { /** * union ionic_lif_config - LIF configuration * @state: LIF state (enum ionic_lif_state) + * @rsvd: reserved byte(s) * @name: LIF name * @mtu: MTU * @mac: Station MAC address * @vlan: Default Vlan ID * @features: Features (enum ionic_eth_hw_features) * @queue_count: Queue counts per queue-type + * @words: word access to struct contents */ union ionic_lif_config { struct { @@ -481,33 +500,39 @@ union ionic_lif_config { * @capabilities: LIF capabilities * * @eth: Ethernet identify structure - * @version: Ethernet identify structure version - * @max_ucast_filters: Number of perfect unicast addresses supported - * @max_mcast_filters: Number of perfect multicast addresses supported - * @min_frame_size: Minimum size of frames to be sent - * @max_frame_size: Maximum size of frames to be sent - * @hwstamp_tx_modes: Bitmask of BIT_ULL(enum ionic_txstamp_mode) - * @hwstamp_rx_filters: Bitmask of enum ionic_pkt_class - * @config: LIF config struct with features, mtu, mac, q counts + * @eth.version: Ethernet identify structure version + * @eth.rsvd: reserved byte(s) + * @eth.max_ucast_filters: Number of perfect unicast addresses supported + * @eth.max_mcast_filters: Number of perfect multicast addresses supported + * @eth.min_frame_size: Minimum size of frames to be sent + * @eth.max_frame_size: Maximum size of frames to be sent + * @eth.rsvd2: reserved byte(s) + * @eth.hwstamp_tx_modes: Bitmask of BIT_ULL(enum ionic_txstamp_mode) + * @eth.hwstamp_rx_filters: Bitmask of enum ionic_pkt_class + * @eth.rsvd3: reserved byte(s) + * @eth.config: LIF config struct with features, mtu, mac, q counts * * @rdma: RDMA identify structure - * @version: RDMA version of opcodes and queue descriptors - * @qp_opcodes: Number of RDMA queue pair opcodes supported - * @admin_opcodes: Number of RDMA admin opcodes supported - * @npts_per_lif: Page table size per LIF - * @nmrs_per_lif: Number of memory regions per LIF - * @nahs_per_lif: Number of address handles per LIF - * @max_stride: Max work request stride - * @cl_stride: Cache line stride - * @pte_stride: Page table entry stride - * @rrq_stride: Remote RQ work request stride - * @rsq_stride: Remote SQ work request stride - * @dcqcn_profiles: Number of DCQCN profiles - * @aq_qtype: RDMA Admin Qtype - * @sq_qtype: RDMA Send Qtype - * @rq_qtype: RDMA Receive Qtype - * @cq_qtype: RDMA Completion Qtype - * @eq_qtype: RDMA Event Qtype + * @rdma.version: RDMA version of opcodes and queue descriptors + * @rdma.qp_opcodes: Number of RDMA queue pair opcodes supported + * @rdma.admin_opcodes: Number of RDMA admin opcodes supported + * @rdma.rsvd: reserved byte(s) + * @rdma.npts_per_lif: Page table size per LIF + * @rdma.nmrs_per_lif: Number of memory regions per LIF + * @rdma.nahs_per_lif: Number of address handles per LIF + * @rdma.max_stride: Max work request stride + * @rdma.cl_stride: Cache line stride + * @rdma.pte_stride: Page table entry stride + * @rdma.rrq_stride: Remote RQ work request stride + * @rdma.rsq_stride: Remote SQ work request stride + * @rdma.dcqcn_profiles: Number of DCQCN profiles + * @rdma.rsvd_dimensions: reserved byte(s) + * @rdma.aq_qtype: RDMA Admin Qtype + * @rdma.sq_qtype: RDMA Send Qtype + * @rdma.rq_qtype: RDMA Receive Qtype + * @rdma.cq_qtype: RDMA Completion Qtype + * @rdma.eq_qtype: RDMA Event Qtype + * @words: word access to struct contents */ union ionic_lif_identity { struct { @@ -558,7 +583,9 @@ union ionic_lif_identity { * @opcode: Opcode * @type: LIF type (enum ionic_lif_type) * @index: LIF index + * @rsvd: reserved byte(s) * @info_pa: Destination address for LIF info (struct ionic_lif_info) + * @rsvd2: reserved byte(s) */ struct ionic_lif_init_cmd { u8 opcode; @@ -572,7 +599,9 @@ struct ionic_lif_init_cmd { /** * struct ionic_lif_init_comp - LIF init command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @hw_index: Hardware index of the initialized LIF + * @rsvd2: reserved byte(s) */ struct ionic_lif_init_comp { u8 status; @@ -584,9 +613,11 @@ struct ionic_lif_init_comp { /** * struct ionic_q_identify_cmd - queue identify command * @opcode: opcode + * @rsvd: reserved byte(s) * @lif_type: LIF type (enum ionic_lif_type) * @type: Logical queue type (enum ionic_logical_qtype) * @ver: Highest queue type version that the driver supports + * @rsvd2: reserved byte(s) */ struct ionic_q_identify_cmd { u8 opcode; @@ -600,8 +631,10 @@ struct ionic_q_identify_cmd { /** * struct ionic_q_identify_comp - queue identify command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @comp_index: Index in the descriptor ring for which this is the completion * @ver: Queue type version that can be used with FW + * @rsvd2: reserved byte(s) */ struct ionic_q_identify_comp { u8 status; @@ -615,12 +648,14 @@ struct ionic_q_identify_comp { * union ionic_q_identity - queue identity information * @version: Queue type version that can be used with FW * @supported: Bitfield of queue versions, first bit = ver 0 + * @rsvd: reserved byte(s) * @features: Queue features (enum ionic_q_feature, etc) * @desc_sz: Descriptor size * @comp_sz: Completion descriptor size * @sg_desc_sz: Scatter/Gather descriptor size * @max_sg_elems: Maximum number of Scatter/Gather elements * @sg_desc_stride: Number of Scatter/Gather elements per descriptor + * @words: word access to struct contents */ union ionic_q_identity { struct { @@ -640,8 +675,10 @@ union ionic_q_identity { /** * struct ionic_q_init_cmd - Queue init command * @opcode: opcode + * @rsvd: reserved byte(s) * @type: Logical queue type * @ver: Queue type version + * @rsvd1: reserved byte(s) * @lif_index: LIF index * @index: (LIF, qtype) relative admin queue index * @intr_index: Interrupt control register index, or Event queue index @@ -667,6 +704,7 @@ union ionic_q_identity { * @ring_base: Queue ring base address * @cq_ring_base: Completion queue ring base address * @sg_ring_base: Scatter/Gather ring base address + * @rsvd2: reserved byte(s) * @features: Mask of queue features to enable, if not in the flags above. */ struct ionic_q_init_cmd { @@ -698,9 +736,11 @@ struct ionic_q_init_cmd { /** * struct ionic_q_init_comp - Queue init command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @comp_index: Index in the descriptor ring for which this is the completion * @hw_index: Hardware Queue ID * @hw_type: Hardware Queue type + * @rsvd2: reserved byte(s) * @color: Color */ struct ionic_q_init_comp { @@ -800,7 +840,7 @@ enum ionic_txq_desc_opcode { * will set CWR flag in the first segment if * CWR is set in the template header, and * clear CWR in remaining segments. - * @flags: + * flags: * vlan: * Insert an L2 VLAN header using @vlan_tci * encap: @@ -813,13 +853,14 @@ enum ionic_txq_desc_opcode { * TSO start * tso_eot: * TSO end - * @num_sg_elems: Number of scatter-gather elements in SG + * num_sg_elems: Number of scatter-gather elements in SG * descriptor - * @addr: First data buffer's DMA address + * addr: First data buffer's DMA address * (Subsequent data buffers are on txq_sg_desc) * @len: First data buffer's length, in bytes * @vlan_tci: VLAN tag to insert in the packet (if requested * by @V-bit). Includes .1p and .1q tags + * @hword0: half word padding * @hdr_len: Length of packet headers, including * encapsulating outer header, if applicable * Valid for opcodes IONIC_TXQ_DESC_OPCODE_CALC_CSUM and @@ -830,10 +871,12 @@ enum ionic_txq_desc_opcode { * IONIC_TXQ_DESC_OPCODE_TSO, @hdr_len is up to * inner-most L4 payload, so inclusive of * inner-most L4 header. + * @hword1: half word padding * @mss: Desired MSS value for TSO; only applicable for * IONIC_TXQ_DESC_OPCODE_TSO * @csum_start: Offset from packet to first byte checked in L4 checksum * @csum_offset: Offset from csum_start to L4 checksum field + * @hword2: half word padding */ struct ionic_txq_desc { __le64 cmd; @@ -901,6 +944,7 @@ static inline void decode_txq_desc_cmd(u64 cmd, u8 *opcode, u8 *flags, * struct ionic_txq_sg_elem - Transmit scatter-gather (SG) descriptor element * @addr: DMA address of SG element data buffer * @len: Length of SG element data buffer, in bytes + * @rsvd: reserved byte(s) */ struct ionic_txq_sg_elem { __le64 addr; @@ -927,7 +971,9 @@ struct ionic_txq_sg_desc_v1 { /** * struct ionic_txq_comp - Ethernet transmit queue completion descriptor * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @comp_index: Index in the descriptor ring for which this is the completion + * @rsvd2: reserved byte(s) * @color: Color bit */ struct ionic_txq_comp { @@ -953,6 +999,7 @@ enum ionic_rxq_desc_opcode { * receive, including actual bytes received, * are recorded in Rx completion descriptor. * + * @rsvd: reserved byte(s) * @len: Data buffer's length, in bytes * @addr: Data buffer's DMA address */ @@ -967,6 +1014,7 @@ struct ionic_rxq_desc { * struct ionic_rxq_sg_elem - Receive scatter-gather (SG) descriptor element * @addr: DMA address of SG element data buffer * @len: Length of SG element data buffer, in bytes + * @rsvd: reserved byte(s) */ struct ionic_rxq_sg_elem { __le64 addr; @@ -1170,6 +1218,7 @@ enum ionic_pkt_class { * @lif_index: LIF index * @index: Queue index * @oper: Operation (enum ionic_q_control_oper) + * @rsvd: reserved byte(s) */ struct ionic_q_control_cmd { u8 opcode; @@ -1182,7 +1231,7 @@ struct ionic_q_control_cmd { typedef struct ionic_admin_comp ionic_q_control_comp; -enum q_control_oper { +enum ionic_q_control_oper { IONIC_Q_DISABLE = 0, IONIC_Q_ENABLE = 1, IONIC_Q_HANG_RESET = 2, @@ -1216,7 +1265,7 @@ enum ionic_xcvr_state { IONIC_XCVR_STATE_SPROM_READ_ERR = 4, }; -/** +/* * enum ionic_xcvr_pid - Supported link modes */ enum ionic_xcvr_pid { @@ -1351,6 +1400,7 @@ struct ionic_xcvr_status { * @fec_type: fec type (enum ionic_port_fec_type) * @pause_type: pause type (enum ionic_port_pause_type) * @loopback_mode: loopback mode (enum ionic_port_loopback_mode) + * @words: word access to struct contents */ union ionic_port_config { struct { @@ -1382,6 +1432,7 @@ union ionic_port_config { * @speed: link speed (in Mbps) * @link_down_count: number of times link went from up to down * @fec_type: fec type (enum ionic_port_fec_type) + * @rsvd: reserved byte(s) * @xcvr: transceiver status */ struct ionic_port_status { @@ -1399,6 +1450,7 @@ struct ionic_port_status { * @opcode: opcode * @index: port index * @ver: Highest version of identify supported by driver + * @rsvd: reserved byte(s) */ struct ionic_port_identify_cmd { u8 opcode; @@ -1411,6 +1463,7 @@ struct ionic_port_identify_cmd { * struct ionic_port_identify_comp - Port identify command completion * @status: Status of the command (enum ionic_status_code) * @ver: Version of identify returned by device + * @rsvd: reserved byte(s) */ struct ionic_port_identify_comp { u8 status; @@ -1422,7 +1475,9 @@ struct ionic_port_identify_comp { * struct ionic_port_init_cmd - Port initialization command * @opcode: opcode * @index: port index + * @rsvd: reserved byte(s) * @info_pa: destination address for port info (struct ionic_port_info) + * @rsvd2: reserved byte(s) */ struct ionic_port_init_cmd { u8 opcode; @@ -1435,6 +1490,7 @@ struct ionic_port_init_cmd { /** * struct ionic_port_init_comp - Port initialization command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) */ struct ionic_port_init_comp { u8 status; @@ -1445,6 +1501,7 @@ struct ionic_port_init_comp { * struct ionic_port_reset_cmd - Port reset command * @opcode: opcode * @index: port index + * @rsvd: reserved byte(s) */ struct ionic_port_reset_cmd { u8 opcode; @@ -1455,6 +1512,7 @@ struct ionic_port_reset_cmd { /** * struct ionic_port_reset_comp - Port reset command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) */ struct ionic_port_reset_comp { u8 status; @@ -1510,6 +1568,7 @@ enum ionic_port_attr { * @opcode: Opcode * @index: Port index * @attr: Attribute type (enum ionic_port_attr) + * @rsvd: reserved byte(s) * @state: Port state * @speed: Port speed * @mtu: Port MTU @@ -1518,6 +1577,7 @@ enum ionic_port_attr { * @pause_type: Port pause type setting * @loopback_mode: Port loopback mode * @stats_ctl: Port stats setting + * @rsvd2: reserved byte(s) */ struct ionic_port_setattr_cmd { u8 opcode; @@ -1540,6 +1600,7 @@ struct ionic_port_setattr_cmd { /** * struct ionic_port_setattr_comp - Port set attr command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @color: Color bit */ struct ionic_port_setattr_comp { @@ -1553,6 +1614,7 @@ struct ionic_port_setattr_comp { * @opcode: Opcode * @index: port index * @attr: Attribute type (enum ionic_port_attr) + * @rsvd: reserved byte(s) */ struct ionic_port_getattr_cmd { u8 opcode; @@ -1564,6 +1626,7 @@ struct ionic_port_getattr_cmd { /** * struct ionic_port_getattr_comp - Port get attr command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @state: Port state * @speed: Port speed * @mtu: Port MTU @@ -1571,6 +1634,7 @@ struct ionic_port_getattr_cmd { * @fec_type: Port FEC type setting * @pause_type: Port pause type setting * @loopback_mode: Port loopback mode + * @rsvd2: reserved byte(s) * @color: Color bit */ struct ionic_port_getattr_comp { @@ -1593,9 +1657,11 @@ struct ionic_port_getattr_comp { * struct ionic_lif_status - LIF status register * @eid: most recent NotifyQ event id * @port_num: port the LIF is connected to + * @rsvd: reserved byte(s) * @link_status: port status (enum ionic_port_oper_status) * @link_speed: speed of link in Mbps * @link_down_count: number of times link went from up to down + * @rsvd2: reserved byte(s) */ struct ionic_lif_status { __le64 eid; @@ -1610,7 +1676,9 @@ struct ionic_lif_status { /** * struct ionic_lif_reset_cmd - LIF reset command * @opcode: opcode + * @rsvd: reserved byte(s) * @index: LIF index + * @rsvd2: reserved byte(s) */ struct ionic_lif_reset_cmd { u8 opcode; @@ -1643,9 +1711,11 @@ enum ionic_dev_attr { * struct ionic_dev_setattr_cmd - Set Device attributes on the NIC * @opcode: Opcode * @attr: Attribute type (enum ionic_dev_attr) + * @rsvd: reserved byte(s) * @state: Device state (enum ionic_dev_state) * @name: The bus info, e.g. PCI slot-device-function, 0 terminated * @features: Device features + * @rsvd2: reserved byte(s) */ struct ionic_dev_setattr_cmd { u8 opcode; @@ -1662,7 +1732,9 @@ struct ionic_dev_setattr_cmd { /** * struct ionic_dev_setattr_comp - Device set attr command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @features: Device features + * @rsvd2: reserved byte(s) * @color: Color bit */ struct ionic_dev_setattr_comp { @@ -1679,6 +1751,7 @@ struct ionic_dev_setattr_comp { * struct ionic_dev_getattr_cmd - Get Device attributes from the NIC * @opcode: opcode * @attr: Attribute type (enum ionic_dev_attr) + * @rsvd: reserved byte(s) */ struct ionic_dev_getattr_cmd { u8 opcode; @@ -1687,9 +1760,11 @@ struct ionic_dev_getattr_cmd { }; /** - * struct ionic_dev_setattr_comp - Device set attr command completion + * struct ionic_dev_getattr_comp - Device set attr command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @features: Device features + * @rsvd2: reserved byte(s) * @color: Color bit */ struct ionic_dev_getattr_comp { @@ -1702,7 +1777,7 @@ struct ionic_dev_getattr_comp { u8 color; }; -/** +/* * RSS parameters */ #define IONIC_RSS_HASH_KEY_SIZE 40 @@ -1726,6 +1801,7 @@ enum ionic_rss_hash_types { * @IONIC_LIF_ATTR_RSS: LIF RSS attribute * @IONIC_LIF_ATTR_STATS_CTRL: LIF statistics control attribute * @IONIC_LIF_ATTR_TXSTAMP: LIF TX timestamping mode + * @IONIC_LIF_ATTR_MAX: maximum attribute value */ enum ionic_lif_attr { IONIC_LIF_ATTR_STATE = 0, @@ -1736,6 +1812,7 @@ enum ionic_lif_attr { IONIC_LIF_ATTR_RSS = 5, IONIC_LIF_ATTR_STATS_CTRL = 6, IONIC_LIF_ATTR_TXSTAMP = 7, + IONIC_LIF_ATTR_MAX = 255, }; /** @@ -1749,11 +1826,13 @@ enum ionic_lif_attr { * @mac: Station mac * @features: Features (enum ionic_eth_hw_features) * @rss: RSS properties - * @types: The hash types to enable (see rss_hash_types) - * @key: The hash secret key - * @addr: Address for the indirection table shared memory + * @rss.types: The hash types to enable (see rss_hash_types) + * @rss.key: The hash secret key + * @rss.rsvd: reserved byte(s) + * @rss.addr: Address for the indirection table shared memory * @stats_ctl: stats control commands (enum ionic_stats_ctl_cmd) - * @txstamp: TX Timestamping Mode (enum ionic_txstamp_mode) + * @txstamp_mode: TX Timestamping Mode (enum ionic_txstamp_mode) + * @rsvd: reserved byte(s) */ struct ionic_lif_setattr_cmd { u8 opcode; @@ -1772,7 +1851,7 @@ struct ionic_lif_setattr_cmd { __le64 addr; } rss; u8 stats_ctl; - __le16 txstamp_mode; + __le16 txstamp_mode; u8 rsvd[60]; } __packed; }; @@ -1780,8 +1859,10 @@ struct ionic_lif_setattr_cmd { /** * struct ionic_lif_setattr_comp - LIF set attr command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @comp_index: Index in the descriptor ring for which this is the completion * @features: features (enum ionic_eth_hw_features) + * @rsvd2: reserved byte(s) * @color: Color bit */ struct ionic_lif_setattr_comp { @@ -1800,6 +1881,7 @@ struct ionic_lif_setattr_comp { * @opcode: Opcode * @attr: Attribute type (enum ionic_lif_attr) * @index: LIF index + * @rsvd: reserved byte(s) */ struct ionic_lif_getattr_cmd { u8 opcode; @@ -1811,13 +1893,14 @@ struct ionic_lif_getattr_cmd { /** * struct ionic_lif_getattr_comp - LIF get attr command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @comp_index: Index in the descriptor ring for which this is the completion * @state: LIF state (enum ionic_lif_state) - * @name: The netdev name string, 0 terminated * @mtu: Mtu * @mac: Station mac * @features: Features (enum ionic_eth_hw_features) - * @txstamp: TX Timestamping Mode (enum ionic_txstamp_mode) + * @txstamp_mode: TX Timestamping Mode (enum ionic_txstamp_mode) + * @rsvd2: reserved byte(s) * @color: Color bit */ struct ionic_lif_getattr_comp { @@ -1838,12 +1921,15 @@ struct ionic_lif_getattr_comp { /** * struct ionic_lif_setphc_cmd - Set LIF PTP Hardware Clock * @opcode: Opcode + * @rsvd1: reserved byte(s) * @lif_index: LIF index + * @rsvd2: reserved byte(s) * @tick: Hardware stamp tick of an instant in time. * @nsec: Nanosecond stamp of the same instant. * @frac: Fractional nanoseconds at the same instant. * @mult: Cycle to nanosecond multiplier. * @shift: Cycle to nanosecond divisor (power of two). + * @rsvd3: reserved byte(s) */ struct ionic_lif_setphc_cmd { u8 opcode; @@ -1870,6 +1956,7 @@ enum ionic_rx_mode { /** * struct ionic_rx_mode_set_cmd - Set LIF's Rx mode command * @opcode: opcode + * @rsvd: reserved byte(s) * @lif_index: LIF index * @rx_mode: Rx mode flags: * IONIC_RX_MODE_F_UNICAST: Accept known unicast packets @@ -1878,6 +1965,7 @@ enum ionic_rx_mode { * IONIC_RX_MODE_F_PROMISC: Accept any packets * IONIC_RX_MODE_F_ALLMULTI: Accept any multicast packets * IONIC_RX_MODE_F_RDMA_SNIFFER: Sniff RDMA packets + * @rsvd2: reserved byte(s) */ struct ionic_rx_mode_set_cmd { u8 opcode; @@ -1904,13 +1992,14 @@ enum ionic_rx_filter_match_type { * @qid: Queue ID * @match: Rx filter match type (see IONIC_RX_FILTER_MATCH_xxx) * @vlan: VLAN filter - * @vlan: VLAN ID + * @vlan.vlan: VLAN ID * @mac: MAC filter - * @addr: MAC address (network-byte order) + * @mac.addr: MAC address (network-byte order) * @mac_vlan: MACVLAN filter - * @vlan: VLAN ID - * @addr: MAC address (network-byte order) + * @mac_vlan.vlan: VLAN ID + * @mac_vlan.addr: MAC address (network-byte order) * @pkt_class: Packet classification filter + * @rsvd: reserved byte(s) */ struct ionic_rx_filter_add_cmd { u8 opcode; @@ -1937,8 +2026,10 @@ struct ionic_rx_filter_add_cmd { /** * struct ionic_rx_filter_add_comp - Add LIF Rx filter command completion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @comp_index: Index in the descriptor ring for which this is the completion * @filter_id: Filter ID + * @rsvd2: reserved byte(s) * @color: Color bit */ struct ionic_rx_filter_add_comp { @@ -1953,8 +2044,10 @@ struct ionic_rx_filter_add_comp { /** * struct ionic_rx_filter_del_cmd - Delete LIF Rx filter command * @opcode: opcode + * @rsvd: reserved byte(s) * @lif_index: LIF index * @filter_id: Filter ID + * @rsvd2: reserved byte(s) */ struct ionic_rx_filter_del_cmd { u8 opcode; @@ -2000,6 +2093,7 @@ enum ionic_vf_link_status { * @trust: enable VF trust * @linkstate: set link up or down * @stats_pa: set DMA address for VF stats + * @pad: reserved byte(s) */ struct ionic_vf_setattr_cmd { u8 opcode; @@ -2031,6 +2125,7 @@ struct ionic_vf_setattr_comp { * @opcode: Opcode * @attr: Attribute type (enum ionic_vf_attr) * @vf_index: VF index + * @rsvd: reserved byte(s) */ struct ionic_vf_getattr_cmd { u8 opcode; @@ -2064,8 +2159,8 @@ enum ionic_vf_ctrl_opcode { /** * struct ionic_vf_ctrl_cmd - VF control command * @opcode: Opcode for the command - * @vf_index: VF Index. It is unused if op START_ALL is used. * @ctrl_opcode: VF control operation type + * @vf_index: VF Index. It is unused if op START_ALL is used. */ struct ionic_vf_ctrl_cmd { u8 opcode; @@ -2089,7 +2184,7 @@ struct ionic_vf_ctrl_comp { * struct ionic_qos_identify_cmd - QoS identify command * @opcode: opcode * @ver: Highest version of identify supported by driver - * + * @rsvd: reserved byte(s) */ struct ionic_qos_identify_cmd { u8 opcode; @@ -2101,6 +2196,7 @@ struct ionic_qos_identify_cmd { * struct ionic_qos_identify_comp - QoS identify command completion * @status: Status of the command (enum ionic_status_code) * @ver: Version of identify returned by device + * @rsvd: reserved byte(s) */ struct ionic_qos_identify_comp { u8 status; @@ -2118,7 +2214,7 @@ struct ionic_qos_identify_comp { #define IONIC_QOS_ALL_PCP 0xFF #define IONIC_DSCP_BLOCK_SIZE 8 -/** +/* * enum ionic_qos_class */ enum ionic_qos_class { @@ -2174,6 +2270,7 @@ enum ionic_qos_sched_type { * @dot1q_pcp: Dot1q pcp value * @ndscp: Number of valid dscp values in the ip_dscp field * @ip_dscp: IP dscp values + * @words: word access to struct contents */ union ionic_qos_config { struct { @@ -2219,8 +2316,9 @@ union ionic_qos_config { * union ionic_qos_identity - QoS identity structure * @version: Version of the identify structure * @type: QoS system type - * @nclasses: Number of usable QoS classes + * @rsvd: reserved byte(s) * @config: Current configuration of classes + * @words: word access to struct contents */ union ionic_qos_identity { struct { @@ -2236,7 +2334,9 @@ union ionic_qos_identity { * struct ionic_qos_init_cmd - QoS config init command * @opcode: Opcode * @group: QoS class id + * @rsvd: reserved byte(s) * @info_pa: destination address for qos info + * @rsvd1: reserved byte(s) */ struct ionic_qos_init_cmd { u8 opcode; @@ -2252,6 +2352,7 @@ typedef struct ionic_admin_comp ionic_qos_init_comp; * struct ionic_qos_reset_cmd - QoS config reset command * @opcode: Opcode * @group: QoS class id + * @rsvd: reserved byte(s) */ struct ionic_qos_reset_cmd { u8 opcode; @@ -2260,8 +2361,10 @@ struct ionic_qos_reset_cmd { }; /** - * struct ionic_qos_clear_port_stats_cmd - Qos config reset command + * struct ionic_qos_clear_stats_cmd - Qos config reset command * @opcode: Opcode + * @group_bitmap: bitmap of groups to be cleared + * @rsvd: reserved byte(s) */ struct ionic_qos_clear_stats_cmd { u8 opcode; @@ -2274,6 +2377,7 @@ typedef struct ionic_admin_comp ionic_qos_reset_comp; /** * struct ionic_fw_download_cmd - Firmware download command * @opcode: opcode + * @rsvd: reserved byte(s) * @addr: dma address of the firmware buffer * @offset: offset of the firmware buffer within the full image * @length: number of valid bytes in the firmware buffer @@ -2297,6 +2401,7 @@ typedef struct ionic_admin_comp ionic_fw_download_comp; * @IONIC_FW_INSTALL_STATUS: Firmware installation status * @IONIC_FW_ACTIVATE_ASYNC: Activate firmware asynchronously * @IONIC_FW_ACTIVATE_STATUS: Firmware activate status + * @IONIC_FW_UPDATE_CLEANUP: Clean up after an interrupted fw update */ enum ionic_fw_control_oper { IONIC_FW_RESET = 0, @@ -2312,8 +2417,10 @@ enum ionic_fw_control_oper { /** * struct ionic_fw_control_cmd - Firmware control command * @opcode: opcode + * @rsvd: reserved byte(s) * @oper: firmware control operation (enum ionic_fw_control_oper) * @slot: slot to activate + * @rsvd1: reserved byte(s) */ struct ionic_fw_control_cmd { u8 opcode; @@ -2326,8 +2433,10 @@ struct ionic_fw_control_cmd { /** * struct ionic_fw_control_comp - Firmware control copletion * @status: Status of the command (enum ionic_status_code) + * @rsvd: reserved byte(s) * @comp_index: Index in the descriptor ring for which this is the completion * @slot: Slot where the firmware was installed + * @rsvd1: reserved byte(s) * @color: Color bit */ struct ionic_fw_control_comp { @@ -2346,7 +2455,9 @@ struct ionic_fw_control_comp { /** * struct ionic_rdma_reset_cmd - Reset RDMA LIF cmd * @opcode: opcode + * @rsvd: reserved byte(s) * @lif_index: LIF index + * @rsvd2: reserved byte(s) * * There is no RDMA specific dev command completion struct. Completion uses * the common struct ionic_admin_comp. Only the status is indicated. @@ -2362,6 +2473,7 @@ struct ionic_rdma_reset_cmd { /** * struct ionic_rdma_queue_cmd - Create RDMA Queue command * @opcode: opcode, 52, 53 + * @rsvd: reserved byte(s) * @lif_index: LIF index * @qid_ver: (qid | (RDMA version << 24)) * @cid: intr, eq_id, or cq_id @@ -2369,6 +2481,7 @@ struct ionic_rdma_reset_cmd { * @depth_log2: log base two of queue depth * @stride_log2: log base two of queue stride * @dma_addr: address of the queue memory + * @rsvd2: reserved byte(s) * * The same command struct is used to create an RDMA event queue, completion * queue, or RDMA admin queue. The cid is an interrupt number for an event @@ -2425,6 +2538,7 @@ struct ionic_notifyq_event { * @ecode: event code = IONIC_EVENT_LINK_CHANGE * @link_status: link up/down, with error bits (enum ionic_port_status) * @link_speed: speed of the network link + * @rsvd: reserved byte(s) * * Sent when the network link state changes between UP and DOWN */ @@ -2442,6 +2556,7 @@ struct ionic_link_change_event { * @ecode: event code = IONIC_EVENT_RESET * @reset_code: reset type * @state: 0=pending, 1=complete, 2=error + * @rsvd: reserved byte(s) * * Sent when the NIC or some subsystem is going to be or * has been reset. @@ -2458,6 +2573,7 @@ struct ionic_reset_event { * struct ionic_heartbeat_event - Sent periodically by NIC to indicate health * @eid: event number * @ecode: event code = IONIC_EVENT_HEARTBEAT + * @rsvd: reserved byte(s) */ struct ionic_heartbeat_event { __le64 eid; @@ -2481,6 +2597,7 @@ struct ionic_log_event { * struct ionic_xcvr_event - Transceiver change event * @eid: event number * @ecode: event code = IONIC_EVENT_XCVR + * @rsvd: reserved byte(s) */ struct ionic_xcvr_event { __le64 eid; @@ -2488,7 +2605,7 @@ struct ionic_xcvr_event { u8 rsvd[54]; }; -/** +/* * struct ionic_port_stats - Port statistics structure */ struct ionic_port_stats { @@ -2646,8 +2763,7 @@ enum ionic_oflow_drop_stats { IONIC_OFLOW_DROP_MAX, }; -/** - * struct port_pb_stats - packet buffers system stats +/* struct ionic_port_pb_stats - packet buffers system stats * uses ionic_pb_buffer_drop_stats for drop_counts[] */ struct ionic_port_pb_stats { @@ -2681,7 +2797,9 @@ struct ionic_port_pb_stats { * @pause_type: supported pause types * @loopback_mode: supported loopback mode * @speeds: supported speeds + * @rsvd2: reserved byte(s) * @config: current port configuration + * @words: word access to struct contents */ union ionic_port_identity { struct { @@ -2707,7 +2825,8 @@ union ionic_port_identity { * @status: Port status data * @stats: Port statistics data * @mgmt_stats: Port management statistics data - * @port_pb_drop_stats: uplink pb drop stats + * @rsvd: reserved byte(s) + * @pb_stats: uplink pb drop stats */ struct ionic_port_info { union ionic_port_config config; @@ -2721,7 +2840,7 @@ struct ionic_port_info { struct ionic_port_pb_stats pb_stats; }; -/** +/* * struct ionic_lif_stats - LIF statistics structure */ struct ionic_lif_stats { @@ -2983,8 +3102,10 @@ struct ionic_hwstamp_regs { * bit 4-7 - 4 bit generation number, changes on fw restart * @fw_heartbeat: Firmware heartbeat counter * @serial_num: Serial number + * @rsvd_pad1024: reserved byte(s) * @fw_version: Firmware version - * @hwstamp_regs: Hardware current timestamp registers + * @hwstamp: Hardware current timestamp registers + * @words: word access to struct contents */ union ionic_dev_info_regs { #define IONIC_DEVINFO_FWVERS_BUFLEN 32 @@ -3014,7 +3135,9 @@ union ionic_dev_info_regs { * @done: Done indicator, bit 0 == 1 when command is complete * @cmd: Opcode-specific command bytes * @comp: Opcode-specific response bytes + * @rsvd: reserved byte(s) * @data: Opcode-specific side-data + * @words: word access to struct contents */ union ionic_dev_cmd_regs { struct { @@ -3032,6 +3155,7 @@ union ionic_dev_cmd_regs { * union ionic_dev_regs - Device register format for bar 0 page 0 * @info: Device info registers * @devcmd: Device command registers + * @words: word access to struct contents */ union ionic_dev_regs { struct { @@ -3098,6 +3222,7 @@ union ionic_adminq_comp { * interrupts when armed. * @qid_lo: Queue destination for the producer index and flags (low bits) * @qid_hi: Queue destination for the producer index and flags (high bits) + * @rsvd2: reserved byte(s) */ struct ionic_doorbell { __le16 p_index; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index 24870da3f4848857d2d6012ed82c78d2f031f7a1..23e1f6638b38fc97a19e78a390b525b472564d7b 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -237,12 +237,12 @@ static int ionic_request_irq(struct ionic_lif *lif, struct ionic_qcq *qcq) const char *name; if (lif->registered) - name = lif->netdev->name; + name = netdev_name(lif->netdev); else name = dev_name(dev); snprintf(intr->name, sizeof(intr->name), - "%s-%s-%s", IONIC_DRV_NAME, name, q->name); + "%.5s-%.16s-%.8s", IONIC_DRV_NAME, name, q->name); return devm_request_irq(dev, intr->vector, ionic_isr, 0, intr->name, &qcq->napi); @@ -3388,6 +3388,7 @@ static void ionic_lif_handle_fw_up(struct ionic_lif *lif) * just need to reanimate it. */ ionic_init_devinfo(ionic); + ionic_reset(ionic); err = ionic_identify(ionic); if (err) goto err_out; @@ -3731,7 +3732,7 @@ static void ionic_lif_set_netdev_info(struct ionic_lif *lif) }, }; - strscpy(ctx.cmd.lif_setattr.name, lif->netdev->name, + strscpy(ctx.cmd.lif_setattr.name, netdev_name(lif->netdev), sizeof(ctx.cmd.lif_setattr.name)); ionic_adminq_post_wait(lif, &ctx); diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 5dba6d2d633cb6d26487b9e6db87307ab12ddef0..9e6dee2fc1d4358ed968f1fda35dde14d830858a 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -502,7 +502,7 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, XDP_PACKET_HEADROOM, frag_len, false); dma_sync_single_range_for_cpu(rxq->dev, ionic_rx_buf_pa(buf_info), - XDP_PACKET_HEADROOM, len, + XDP_PACKET_HEADROOM, frag_len, DMA_FROM_DEVICE); prefetchw(&xdp_buf.data_hard_start); @@ -582,10 +582,11 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, buf_info->page_offset, true); __netif_tx_unlock(nq); - if (err) { + if (unlikely(err)) { netdev_dbg(netdev, "tx ionic_xdp_post_frame err %d\n", err); goto out_xdp_abort; } + buf_info->page = NULL; stats->xdp_tx++; /* the Tx completion will free the buffers */ @@ -597,7 +598,7 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, IONIC_PAGE_SIZE, DMA_FROM_DEVICE); err = xdp_do_redirect(netdev, &xdp_buf, xdp_prog); - if (err) { + if (unlikely(err)) { netdev_dbg(netdev, "xdp_do_redirect err %d\n", err); goto out_xdp_abort; } @@ -1058,7 +1059,7 @@ static dma_addr_t ionic_tx_map_single(struct ionic_queue *q, dma_addr_t dma_addr; dma_addr = dma_map_single(dev, data, len, DMA_TO_DEVICE); - if (dma_mapping_error(dev, dma_addr)) { + if (unlikely(dma_mapping_error(dev, dma_addr))) { net_warn_ratelimited("%s: DMA single map failed on %s!\n", dev_name(dev), q->name); q_to_tx_stats(q)->dma_map_err++; @@ -1075,7 +1076,7 @@ static dma_addr_t ionic_tx_map_frag(struct ionic_queue *q, dma_addr_t dma_addr; dma_addr = skb_frag_dma_map(dev, frag, offset, len, DMA_TO_DEVICE); - if (dma_mapping_error(dev, dma_addr)) { + if (unlikely(dma_mapping_error(dev, dma_addr))) { net_warn_ratelimited("%s: DMA frag map failed on %s!\n", dev_name(dev), q->name); q_to_tx_stats(q)->dma_map_err++; @@ -1316,7 +1317,7 @@ static int ionic_tx_tcp_inner_pseudo_csum(struct sk_buff *skb) int err; err = skb_cow_head(skb, 0); - if (err) + if (unlikely(err)) return err; if (skb->protocol == cpu_to_be16(ETH_P_IP)) { @@ -1340,7 +1341,7 @@ static int ionic_tx_tcp_pseudo_csum(struct sk_buff *skb) int err; err = skb_cow_head(skb, 0); - if (err) + if (unlikely(err)) return err; if (skb->protocol == cpu_to_be16(ETH_P_IP)) { @@ -1357,7 +1358,7 @@ static int ionic_tx_tcp_pseudo_csum(struct sk_buff *skb) } static void ionic_tx_tso_post(struct net_device *netdev, struct ionic_queue *q, - struct ionic_tx_desc_info *desc_info, + struct ionic_txq_desc *desc, struct sk_buff *skb, dma_addr_t addr, u8 nsge, u16 len, unsigned int hdrlen, unsigned int mss, @@ -1365,7 +1366,6 @@ static void ionic_tx_tso_post(struct net_device *netdev, struct ionic_queue *q, u16 vlan_tci, bool has_vlan, bool start, bool done) { - struct ionic_txq_desc *desc = &q->txq[q->head_idx]; u8 flags = 0; u64 cmd; @@ -1445,7 +1445,7 @@ static int ionic_tx_tso(struct net_device *netdev, struct ionic_queue *q, err = ionic_tx_tcp_inner_pseudo_csum(skb); else err = ionic_tx_tcp_pseudo_csum(skb); - if (err) { + if (unlikely(err)) { /* clean up mapping from ionic_tx_map_skb */ ionic_tx_desc_unmap_bufs(q, desc_info); return err; @@ -1503,10 +1503,9 @@ static int ionic_tx_tso(struct net_device *netdev, struct ionic_queue *q, seg_rem = min(tso_rem, mss); done = (tso_rem == 0); /* post descriptor */ - ionic_tx_tso_post(netdev, q, desc_info, skb, - desc_addr, desc_nsge, desc_len, - hdrlen, mss, outer_csum, vlan_tci, has_vlan, - start, done); + ionic_tx_tso_post(netdev, q, desc, skb, desc_addr, desc_nsge, + desc_len, hdrlen, mss, outer_csum, vlan_tci, + has_vlan, start, done); start = false; /* Buffer information is stored with the first tso descriptor */ desc_info = &q->tx_info[q->head_idx]; @@ -1731,7 +1730,7 @@ static int ionic_tx_descs_needed(struct ionic_queue *q, struct sk_buff *skb) linearize: if (too_many_frags) { err = skb_linearize(skb); - if (err) + if (unlikely(err)) return err; q_to_tx_stats(q)->linearize++; } @@ -1765,7 +1764,7 @@ static netdev_tx_t ionic_start_hwstamp_xmit(struct sk_buff *skb, else err = ionic_tx(netdev, q, skb); - if (err) + if (unlikely(err)) goto err_out_drop; return NETDEV_TX_OK; @@ -1811,7 +1810,7 @@ netdev_tx_t ionic_start_xmit(struct sk_buff *skb, struct net_device *netdev) else err = ionic_tx(netdev, q, skb); - if (err) + if (unlikely(err)) goto err_out_drop; return NETDEV_TX_OK; diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c index 7b9e04884575e06dffa94ea8bae6e4615801e0e5..9246ea2118ff2ca4a59fd4b3ce327a9fbf739f13 100644 --- a/drivers/net/ethernet/realtek/r8169_main.c +++ b/drivers/net/ethernet/realtek/r8169_main.c @@ -2274,7 +2274,9 @@ static enum mac_version rtl8169_get_mac_version(u16 xid, bool gmii) /* 8168B family. */ { 0x7c8, 0x380, RTL_GIGA_MAC_VER_17 }, - { 0x7c8, 0x300, RTL_GIGA_MAC_VER_11 }, + /* This one is very old and rare, let's see if anybody complains. + * { 0x7c8, 0x300, RTL_GIGA_MAC_VER_11 }, + */ /* 8101 family. */ { 0x7c8, 0x448, RTL_GIGA_MAC_VER_39 }, @@ -5086,12 +5088,10 @@ static void rtl_set_irq_mask(struct rtl8169_private *tp) tp->irq_mask = RxOK | RxErr | TxOK | TxErr | LinkChg; if (tp->mac_version <= RTL_GIGA_MAC_VER_06) - tp->irq_mask |= SYSErr | RxOverflow | RxFIFOOver; + tp->irq_mask |= SYSErr | RxFIFOOver; else if (tp->mac_version == RTL_GIGA_MAC_VER_11) /* special workaround needed */ tp->irq_mask |= RxFIFOOver; - else - tp->irq_mask |= RxOverflow; } static int rtl_alloc_irq(struct rtl8169_private *tp) diff --git a/drivers/net/ethernet/renesas/ravb.h b/drivers/net/ethernet/renesas/ravb.h index b48935ec7e28dc1ec157f05d153f9e0f1f764319..6b2444d31fcc30930d1e6b4c6a63b4ed505da592 100644 --- a/drivers/net/ethernet/renesas/ravb.h +++ b/drivers/net/ethernet/renesas/ravb.h @@ -19,6 +19,7 @@ #include #include #include +#include #define BE_TX_RING_SIZE 64 /* TX ring size for Best Effort */ #define BE_RX_RING_SIZE 1024 /* RX ring size for Best Effort */ @@ -1039,7 +1040,7 @@ struct ravb_ptp { }; struct ravb_hw_info { - bool (*receive)(struct net_device *ndev, int *quota, int q); + int (*receive)(struct net_device *ndev, int budget, int q); void (*set_rate)(struct net_device *ndev); int (*set_feature)(struct net_device *ndev, netdev_features_t features); int (*dmac_init)(struct net_device *ndev); @@ -1051,9 +1052,10 @@ struct ravb_hw_info { int stats_len; u32 tccr_mask; u32 rx_max_frame_size; - u32 rx_max_desc_use; + u32 rx_buffer_size; u32 rx_desc_size; unsigned aligned_tx: 1; + unsigned coalesce_irqs:1; /* Needs software IRQ coalescing */ /* hardware features */ unsigned internal_delay:1; /* AVB-DMAC has internal delays */ @@ -1070,6 +1072,11 @@ struct ravb_hw_info { unsigned half_duplex:1; /* E-MAC supports half duplex mode */ }; +struct ravb_rx_buffer { + struct page *page; + unsigned int offset; +}; + struct ravb_private { struct net_device *ndev; struct platform_device *pdev; @@ -1093,7 +1100,8 @@ struct ravb_private { struct ravb_tx_desc *tx_ring[NUM_TX_QUEUE]; void *tx_align[NUM_TX_QUEUE]; struct sk_buff *rx_1st_skb; - struct sk_buff **rx_skb[NUM_RX_QUEUE]; + struct page_pool *rx_pool[NUM_RX_QUEUE]; + struct ravb_rx_buffer *rx_buffers[NUM_RX_QUEUE]; struct sk_buff **tx_skb[NUM_TX_QUEUE]; u32 rx_over_errors; u32 rx_fifo_errors; diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c index 4d100283c30fb4958feeb23f5686bdd70be4cf25..c1546b916e4ef5812670f2cf3947a2f815dacd93 100644 --- a/drivers/net/ethernet/renesas/ravb_main.c +++ b/drivers/net/ethernet/renesas/ravb_main.c @@ -30,6 +30,7 @@ #include #include #include +#include #include "ravb.h" @@ -113,25 +114,6 @@ static void ravb_set_rate_rcar(struct net_device *ndev) } } -static struct sk_buff * -ravb_alloc_skb(struct net_device *ndev, const struct ravb_hw_info *info, - gfp_t gfp_mask) -{ - struct sk_buff *skb; - u32 reserve; - - skb = __netdev_alloc_skb(ndev, info->rx_max_frame_size + RAVB_ALIGN - 1, - gfp_mask); - if (!skb) - return NULL; - - reserve = (unsigned long)skb->data & (RAVB_ALIGN - 1); - if (reserve) - skb_reserve(skb, RAVB_ALIGN - reserve); - - return skb; -} - /* Get MAC address from the MAC address registers * * Ethernet AVB device doesn't have ROM for MAC address. @@ -257,21 +239,10 @@ static void ravb_rx_ring_free(struct net_device *ndev, int q) { struct ravb_private *priv = netdev_priv(ndev); unsigned int ring_size; - unsigned int i; if (!priv->rx_ring[q].raw) return; - for (i = 0; i < priv->num_rx_ring[q]; i++) { - struct ravb_rx_desc *desc = ravb_rx_get_desc(priv, q, i); - - if (!dma_mapping_error(ndev->dev.parent, - le32_to_cpu(desc->dptr))) - dma_unmap_single(ndev->dev.parent, - le32_to_cpu(desc->dptr), - priv->info->rx_max_frame_size, - DMA_FROM_DEVICE); - } ring_size = priv->info->rx_desc_size * (priv->num_rx_ring[q] + 1); dma_free_coherent(ndev->dev.parent, ring_size, priv->rx_ring[q].raw, priv->rx_desc_dma[q]); @@ -298,13 +269,16 @@ static void ravb_ring_free(struct net_device *ndev, int q) priv->tx_ring[q] = NULL; } - /* Free RX skb ringbuffer */ - if (priv->rx_skb[q]) { - for (i = 0; i < priv->num_rx_ring[q]; i++) - dev_kfree_skb(priv->rx_skb[q][i]); + /* Free RX buffers */ + for (i = 0; i < priv->num_rx_ring[q]; i++) { + if (priv->rx_buffers[q][i].page) + page_pool_put_page(priv->rx_pool[q], + priv->rx_buffers[q][i].page, + 0, true); } - kfree(priv->rx_skb[q]); - priv->rx_skb[q] = NULL; + kfree(priv->rx_buffers[q]); + priv->rx_buffers[q] = NULL; + page_pool_destroy(priv->rx_pool[q]); /* Free aligned TX buffers */ kfree(priv->tx_align[q]); @@ -317,35 +291,64 @@ static void ravb_ring_free(struct net_device *ndev, int q) priv->tx_skb[q] = NULL; } -static void ravb_rx_ring_format(struct net_device *ndev, int q) +static int +ravb_alloc_rx_buffer(struct net_device *ndev, int q, u32 entry, gfp_t gfp_mask, + struct ravb_rx_desc *rx_desc) { struct ravb_private *priv = netdev_priv(ndev); - struct ravb_rx_desc *rx_desc; - unsigned int rx_ring_size; + const struct ravb_hw_info *info = priv->info; + struct ravb_rx_buffer *rx_buff; dma_addr_t dma_addr; - unsigned int i; + unsigned int size; - rx_ring_size = priv->info->rx_desc_size * priv->num_rx_ring[q]; - memset(priv->rx_ring[q].raw, 0, rx_ring_size); - /* Build RX ring buffer */ - for (i = 0; i < priv->num_rx_ring[q]; i++) { - /* RX descriptor */ - rx_desc = ravb_rx_get_desc(priv, q, i); - rx_desc->ds_cc = cpu_to_le16(priv->info->rx_max_desc_use); - dma_addr = dma_map_single(ndev->dev.parent, priv->rx_skb[q][i]->data, - priv->info->rx_max_frame_size, - DMA_FROM_DEVICE); + rx_buff = &priv->rx_buffers[q][entry]; + size = info->rx_buffer_size; + rx_buff->page = page_pool_alloc(priv->rx_pool[q], &rx_buff->offset, + &size, gfp_mask); + if (unlikely(!rx_buff->page)) { /* We just set the data size to 0 for a failed mapping which * should prevent DMA from happening... */ - if (dma_mapping_error(ndev->dev.parent, dma_addr)) - rx_desc->ds_cc = cpu_to_le16(0); - rx_desc->dptr = cpu_to_le32(dma_addr); + rx_desc->ds_cc = cpu_to_le16(0); + return -ENOMEM; + } + + dma_addr = page_pool_get_dma_addr(rx_buff->page) + rx_buff->offset; + dma_sync_single_for_device(ndev->dev.parent, dma_addr, + info->rx_buffer_size, DMA_FROM_DEVICE); + rx_desc->dptr = cpu_to_le32(dma_addr); + + /* The end of the RX buffer is used to store skb shared data, so we need + * to ensure that the hardware leaves enough space for this. + */ + rx_desc->ds_cc = cpu_to_le16(info->rx_buffer_size - + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) - + ETH_FCS_LEN + sizeof(__sum16)); + return 0; +} + +static u32 +ravb_rx_ring_refill(struct net_device *ndev, int q, u32 count, gfp_t gfp_mask) +{ + struct ravb_private *priv = netdev_priv(ndev); + struct ravb_rx_desc *rx_desc; + u32 i, entry; + + for (i = 0; i < count; i++) { + entry = (priv->dirty_rx[q] + i) % priv->num_rx_ring[q]; + rx_desc = ravb_rx_get_desc(priv, q, entry); + + if (!priv->rx_buffers[q][entry].page) { + if (unlikely(ravb_alloc_rx_buffer(ndev, q, entry, + gfp_mask, rx_desc))) + break; + } + /* Descriptor type must be set after all the above writes */ + dma_wmb(); rx_desc->die_dt = DT_FEMPTY; } - rx_desc = ravb_rx_get_desc(priv, q, i); - rx_desc->dptr = cpu_to_le32((u32)priv->rx_desc_dma[q]); - rx_desc->die_dt = DT_LINKFIX; /* type */ + + return i; } /* Format skb and descriptor buffer for Ethernet AVB */ @@ -353,6 +356,7 @@ static void ravb_ring_format(struct net_device *ndev, int q) { struct ravb_private *priv = netdev_priv(ndev); unsigned int num_tx_desc = priv->num_tx_desc; + struct ravb_rx_desc *rx_desc; struct ravb_tx_desc *tx_desc; struct ravb_desc *desc; unsigned int tx_ring_size = sizeof(*tx_desc) * priv->num_tx_ring[q] * @@ -364,7 +368,13 @@ static void ravb_ring_format(struct net_device *ndev, int q) priv->dirty_rx[q] = 0; priv->dirty_tx[q] = 0; - ravb_rx_ring_format(ndev, q); + /* Regular RX descriptors have already been initialized by + * ravb_rx_ring_refill(), we just need to initialize the final link + * descriptor. + */ + rx_desc = ravb_rx_get_desc(priv, q, priv->num_rx_ring[q]); + rx_desc->dptr = cpu_to_le32((u32)priv->rx_desc_dma[q]); + rx_desc->die_dt = DT_LINKFIX; /* type */ memset(priv->tx_ring[q], 0, tx_ring_size); /* Build TX ring buffer */ @@ -408,26 +418,47 @@ static void *ravb_alloc_rx_desc(struct net_device *ndev, int q) static int ravb_ring_init(struct net_device *ndev, int q) { struct ravb_private *priv = netdev_priv(ndev); - const struct ravb_hw_info *info = priv->info; unsigned int num_tx_desc = priv->num_tx_desc; + struct page_pool_params params = { + .order = 0, + .flags = PP_FLAG_DMA_MAP, + .pool_size = priv->num_rx_ring[q], + .nid = NUMA_NO_NODE, + .dev = ndev->dev.parent, + .dma_dir = DMA_FROM_DEVICE, + }; unsigned int ring_size; - struct sk_buff *skb; - unsigned int i; + u32 num_filled; + + /* Allocate RX page pool and buffers */ + priv->rx_pool[q] = page_pool_create(¶ms); + if (IS_ERR(priv->rx_pool[q])) + goto error; + + /* Allocate RX buffers */ + priv->rx_buffers[q] = kcalloc(priv->num_rx_ring[q], + sizeof(*priv->rx_buffers[q]), GFP_KERNEL); + if (!priv->rx_buffers[q]) + goto error; - /* Allocate RX and TX skb rings */ - priv->rx_skb[q] = kcalloc(priv->num_rx_ring[q], - sizeof(*priv->rx_skb[q]), GFP_KERNEL); + /* Allocate TX skb rings */ priv->tx_skb[q] = kcalloc(priv->num_tx_ring[q], sizeof(*priv->tx_skb[q]), GFP_KERNEL); - if (!priv->rx_skb[q] || !priv->tx_skb[q]) + if (!priv->tx_skb[q]) goto error; - for (i = 0; i < priv->num_rx_ring[q]; i++) { - skb = ravb_alloc_skb(ndev, info, GFP_KERNEL); - if (!skb) - goto error; - priv->rx_skb[q][i] = skb; - } + /* Allocate all RX descriptors. */ + if (!ravb_alloc_rx_desc(ndev, q)) + goto error; + + /* Populate RX ring buffer. */ + priv->dirty_rx[q] = 0; + ring_size = priv->info->rx_desc_size * priv->num_rx_ring[q]; + memset(priv->rx_ring[q].raw, 0, ring_size); + num_filled = ravb_rx_ring_refill(ndev, q, priv->num_rx_ring[q], + GFP_KERNEL); + if (num_filled != priv->num_rx_ring[q]) + goto error; if (num_tx_desc > 1) { /* Allocate rings for the aligned buffers */ @@ -437,12 +468,6 @@ static int ravb_ring_init(struct net_device *ndev, int q) goto error; } - /* Allocate all RX descriptors. */ - if (!ravb_alloc_rx_desc(ndev, q)) - goto error; - - priv->dirty_rx[q] = 0; - /* Allocate all TX descriptors. */ ring_size = sizeof(struct ravb_tx_desc) * (priv->num_tx_ring[q] * num_tx_desc + 1); @@ -706,7 +731,9 @@ static void ravb_get_tx_tstamp(struct net_device *ndev) static void ravb_rx_csum_gbeth(struct sk_buff *skb) { + struct skb_shared_info *shinfo = skb_shinfo(skb); __wsum csum_ip_hdr, csum_proto; + skb_frag_t *last_frag; u8 *hw_csum; /* The hardware checksum status is contained in sizeof(__sum16) * 2 = 4 @@ -716,12 +743,24 @@ static void ravb_rx_csum_gbeth(struct sk_buff *skb) if (unlikely(skb->len < sizeof(__sum16) * 2)) return; - hw_csum = skb_tail_pointer(skb) - sizeof(__sum16); + if (skb_is_nonlinear(skb)) { + last_frag = &shinfo->frags[shinfo->nr_frags - 1]; + hw_csum = skb_frag_address(last_frag) + + skb_frag_size(last_frag); + } else { + hw_csum = skb_tail_pointer(skb); + } + + hw_csum -= sizeof(__sum16); csum_proto = csum_unfold((__force __sum16)get_unaligned_le16(hw_csum)); hw_csum -= sizeof(__sum16); csum_ip_hdr = csum_unfold((__force __sum16)get_unaligned_le16(hw_csum)); - skb_trim(skb, skb->len - 2 * sizeof(__sum16)); + + if (skb_is_nonlinear(skb)) + skb_frag_size_sub(last_frag, 2 * sizeof(__sum16)); + else + skb_trim(skb, skb->len - 2 * sizeof(__sum16)); /* TODO: IPV6 Rx checksum */ if (skb->protocol == htons(ETH_P_IP) && !csum_ip_hdr && !csum_proto) @@ -743,30 +782,14 @@ static void ravb_rx_csum(struct sk_buff *skb) skb_trim(skb, skb->len - sizeof(__sum16)); } -static struct sk_buff *ravb_get_skb_gbeth(struct net_device *ndev, int entry, - struct ravb_rx_desc *desc) -{ - struct ravb_private *priv = netdev_priv(ndev); - struct sk_buff *skb; - - skb = priv->rx_skb[RAVB_BE][entry]; - priv->rx_skb[RAVB_BE][entry] = NULL; - dma_unmap_single(ndev->dev.parent, le32_to_cpu(desc->dptr), - ALIGN(priv->info->rx_max_frame_size, 16), - DMA_FROM_DEVICE); - - return skb; -} - /* Packet receive function for Gigabit Ethernet */ -static bool ravb_rx_gbeth(struct net_device *ndev, int *quota, int q) +static int ravb_rx_gbeth(struct net_device *ndev, int budget, int q) { struct ravb_private *priv = netdev_priv(ndev); const struct ravb_hw_info *info = priv->info; struct net_device_stats *stats; struct ravb_rx_desc *desc; struct sk_buff *skb; - dma_addr_t dma_addr; int rx_packets = 0; u8 desc_status; u16 desc_len; @@ -781,7 +804,7 @@ static bool ravb_rx_gbeth(struct net_device *ndev, int *quota, int q) for (i = 0; i < limit; i++, priv->cur_rx[q]++) { entry = priv->cur_rx[q] % priv->num_rx_ring[q]; desc = &priv->rx_ring[q].desc[entry]; - if (rx_packets == *quota || desc->die_dt == DT_FEMPTY) + if (rx_packets == budget || desc->die_dt == DT_FEMPTY) break; /* Descriptor type must be checked before all other reads */ @@ -807,87 +830,110 @@ static bool ravb_rx_gbeth(struct net_device *ndev, int *quota, int q) if (desc_status & MSC_CEEF) stats->rx_missed_errors++; } else { + struct ravb_rx_buffer *rx_buff; + void *rx_addr; + + rx_buff = &priv->rx_buffers[q][entry]; + rx_addr = page_address(rx_buff->page) + rx_buff->offset; die_dt = desc->die_dt & 0xF0; + dma_sync_single_for_cpu(ndev->dev.parent, + le32_to_cpu(desc->dptr), + desc_len, DMA_FROM_DEVICE); + switch (die_dt) { case DT_FSINGLE: - skb = ravb_get_skb_gbeth(ndev, entry, desc); - skb_put(skb, desc_len); - skb->protocol = eth_type_trans(skb, ndev); - if (ndev->features & NETIF_F_RXCSUM) - ravb_rx_csum_gbeth(skb); - napi_gro_receive(&priv->napi[q], skb); - rx_packets++; - stats->rx_bytes += desc_len; - break; case DT_FSTART: - priv->rx_1st_skb = ravb_get_skb_gbeth(ndev, entry, desc); - skb_put(priv->rx_1st_skb, desc_len); + /* Start of packet: Set initial data length. */ + skb = napi_build_skb(rx_addr, + info->rx_buffer_size); + if (unlikely(!skb)) { + stats->rx_errors++; + page_pool_put_page(priv->rx_pool[q], + rx_buff->page, 0, + true); + goto refill; + } + skb_mark_for_recycle(skb); + skb_put(skb, desc_len); + + /* Save this skb if the packet spans multiple + * descriptors. + */ + if (die_dt == DT_FSTART) + priv->rx_1st_skb = skb; break; + case DT_FMID: - skb = ravb_get_skb_gbeth(ndev, entry, desc); - skb_copy_to_linear_data_offset(priv->rx_1st_skb, - priv->rx_1st_skb->len, - skb->data, - desc_len); - skb_put(priv->rx_1st_skb, desc_len); - dev_kfree_skb(skb); - break; case DT_FEND: - skb = ravb_get_skb_gbeth(ndev, entry, desc); - skb_copy_to_linear_data_offset(priv->rx_1st_skb, - priv->rx_1st_skb->len, - skb->data, - desc_len); - skb_put(priv->rx_1st_skb, desc_len); - dev_kfree_skb(skb); - priv->rx_1st_skb->protocol = - eth_type_trans(priv->rx_1st_skb, ndev); + /* Continuing a packet: Add this buffer as an RX + * frag. + */ + + /* rx_1st_skb will be NULL if napi_build_skb() + * failed for the first descriptor of a + * multi-descriptor packet. + */ + if (unlikely(!priv->rx_1st_skb)) { + stats->rx_errors++; + page_pool_put_page(priv->rx_pool[q], + rx_buff->page, 0, + true); + + /* We may find a DT_FSINGLE or DT_FSTART + * descriptor in the queue which we can + * process, so don't give up yet. + */ + continue; + } + skb_add_rx_frag(priv->rx_1st_skb, + skb_shinfo(priv->rx_1st_skb)->nr_frags, + rx_buff->page, rx_buff->offset, + desc_len, info->rx_buffer_size); + + /* Set skb to point at the whole packet so that + * we only need one code path for finishing a + * packet. + */ + skb = priv->rx_1st_skb; + } + + switch (die_dt) { + case DT_FSINGLE: + case DT_FEND: + /* Finishing a packet: Determine protocol & + * checksum, hand off to NAPI and update our + * stats. + */ + skb->protocol = eth_type_trans(skb, ndev); if (ndev->features & NETIF_F_RXCSUM) - ravb_rx_csum_gbeth(priv->rx_1st_skb); - stats->rx_bytes += priv->rx_1st_skb->len; - napi_gro_receive(&priv->napi[q], - priv->rx_1st_skb); + ravb_rx_csum_gbeth(skb); + stats->rx_bytes += skb->len; + napi_gro_receive(&priv->napi[q], skb); rx_packets++; - break; + + /* Clear rx_1st_skb so that it will only be + * non-NULL when valid. + */ + priv->rx_1st_skb = NULL; } + + /* Mark this RX buffer as consumed. */ + rx_buff->page = NULL; } } +refill: /* Refill the RX ring buffers. */ - for (; priv->cur_rx[q] - priv->dirty_rx[q] > 0; priv->dirty_rx[q]++) { - entry = priv->dirty_rx[q] % priv->num_rx_ring[q]; - desc = &priv->rx_ring[q].desc[entry]; - desc->ds_cc = cpu_to_le16(priv->info->rx_max_desc_use); - - if (!priv->rx_skb[q][entry]) { - skb = ravb_alloc_skb(ndev, info, GFP_ATOMIC); - if (!skb) - break; - dma_addr = dma_map_single(ndev->dev.parent, - skb->data, - priv->info->rx_max_frame_size, - DMA_FROM_DEVICE); - skb_checksum_none_assert(skb); - /* We just set the data size to 0 for a failed mapping - * which should prevent DMA from happening... - */ - if (dma_mapping_error(ndev->dev.parent, dma_addr)) - desc->ds_cc = cpu_to_le16(0); - desc->dptr = cpu_to_le32(dma_addr); - priv->rx_skb[q][entry] = skb; - } - /* Descriptor type must be set after all the above writes */ - dma_wmb(); - desc->die_dt = DT_FEMPTY; - } + priv->dirty_rx[q] += ravb_rx_ring_refill(ndev, q, + priv->cur_rx[q] - priv->dirty_rx[q], + GFP_ATOMIC); stats->rx_packets += rx_packets; - *quota -= rx_packets; - return *quota == 0; + return rx_packets; } /* Packet receive function for Ethernet AVB */ -static bool ravb_rx_rcar(struct net_device *ndev, int *quota, int q) +static int ravb_rx_rcar(struct net_device *ndev, int budget, int q) { struct ravb_private *priv = netdev_priv(ndev); const struct ravb_hw_info *info = priv->info; @@ -895,7 +941,6 @@ static bool ravb_rx_rcar(struct net_device *ndev, int *quota, int q) struct ravb_ex_rx_desc *desc; unsigned int limit, i; struct sk_buff *skb; - dma_addr_t dma_addr; struct timespec64 ts; int rx_packets = 0; u8 desc_status; @@ -906,7 +951,7 @@ static bool ravb_rx_rcar(struct net_device *ndev, int *quota, int q) for (i = 0; i < limit; i++, priv->cur_rx[q]++) { entry = priv->cur_rx[q] % priv->num_rx_ring[q]; desc = &priv->rx_ring[q].ex_desc[entry]; - if (rx_packets == *quota || desc->die_dt == DT_FEMPTY) + if (rx_packets == budget || desc->die_dt == DT_FEMPTY) break; /* Descriptor type must be checked before all other reads */ @@ -934,12 +979,23 @@ static bool ravb_rx_rcar(struct net_device *ndev, int *quota, int q) stats->rx_missed_errors++; } else { u32 get_ts = priv->tstamp_rx_ctrl & RAVB_RXTSTAMP_TYPE; - - skb = priv->rx_skb[q][entry]; - priv->rx_skb[q][entry] = NULL; - dma_unmap_single(ndev->dev.parent, le32_to_cpu(desc->dptr), - priv->info->rx_max_frame_size, - DMA_FROM_DEVICE); + struct ravb_rx_buffer *rx_buff; + void *rx_addr; + + rx_buff = &priv->rx_buffers[q][entry]; + rx_addr = page_address(rx_buff->page) + rx_buff->offset; + dma_sync_single_for_cpu(ndev->dev.parent, + le32_to_cpu(desc->dptr), + pkt_len, DMA_FROM_DEVICE); + + skb = napi_build_skb(rx_addr, info->rx_buffer_size); + if (unlikely(!skb)) { + stats->rx_errors++; + page_pool_put_page(priv->rx_pool[q], + rx_buff->page, 0, true); + break; + } + skb_mark_for_recycle(skb); get_ts &= (q == RAVB_NC) ? RAVB_RXTSTAMP_TYPE_V2_L2_EVENT : ~RAVB_RXTSTAMP_TYPE_V2_L2_EVENT; @@ -961,48 +1017,28 @@ static bool ravb_rx_rcar(struct net_device *ndev, int *quota, int q) napi_gro_receive(&priv->napi[q], skb); rx_packets++; stats->rx_bytes += pkt_len; + + /* Mark this RX buffer as consumed. */ + rx_buff->page = NULL; } } /* Refill the RX ring buffers. */ - for (; priv->cur_rx[q] - priv->dirty_rx[q] > 0; priv->dirty_rx[q]++) { - entry = priv->dirty_rx[q] % priv->num_rx_ring[q]; - desc = &priv->rx_ring[q].ex_desc[entry]; - desc->ds_cc = cpu_to_le16(priv->info->rx_max_desc_use); - - if (!priv->rx_skb[q][entry]) { - skb = ravb_alloc_skb(ndev, info, GFP_ATOMIC); - if (!skb) - break; /* Better luck next round. */ - dma_addr = dma_map_single(ndev->dev.parent, skb->data, - priv->info->rx_max_frame_size, - DMA_FROM_DEVICE); - skb_checksum_none_assert(skb); - /* We just set the data size to 0 for a failed mapping - * which should prevent DMA from happening... - */ - if (dma_mapping_error(ndev->dev.parent, dma_addr)) - desc->ds_cc = cpu_to_le16(0); - desc->dptr = cpu_to_le32(dma_addr); - priv->rx_skb[q][entry] = skb; - } - /* Descriptor type must be set after all the above writes */ - dma_wmb(); - desc->die_dt = DT_FEMPTY; - } + priv->dirty_rx[q] += ravb_rx_ring_refill(ndev, q, + priv->cur_rx[q] - priv->dirty_rx[q], + GFP_ATOMIC); stats->rx_packets += rx_packets; - *quota -= rx_packets; - return *quota == 0; + return rx_packets; } /* Packet receive function for Ethernet AVB */ -static bool ravb_rx(struct net_device *ndev, int *quota, int q) +static int ravb_rx(struct net_device *ndev, int budget, int q) { struct ravb_private *priv = netdev_priv(ndev); const struct ravb_hw_info *info = priv->info; - return info->receive(ndev, quota, q); + return info->receive(ndev, budget, q); } static void ravb_rcv_snd_disable(struct net_device *ndev) @@ -1319,13 +1355,12 @@ static int ravb_poll(struct napi_struct *napi, int budget) unsigned long flags; int q = napi - priv->napi; int mask = BIT(q); - int quota = budget; - bool unmask; + int work_done; /* Processing RX Descriptor Ring */ /* Clear RX interrupt */ ravb_write(ndev, ~(mask | RIS0_RESERVED), RIS0); - unmask = !ravb_rx(ndev, "a, q); + work_done = ravb_rx(ndev, budget, q); /* Processing TX Descriptor Ring */ spin_lock_irqsave(&priv->lock, flags); @@ -1344,24 +1379,20 @@ static int ravb_poll(struct napi_struct *napi, int budget) if (priv->rx_fifo_errors != ndev->stats.rx_fifo_errors) ndev->stats.rx_fifo_errors = priv->rx_fifo_errors; - if (!unmask) - goto out; - - napi_complete(napi); - - /* Re-enable RX/TX interrupts */ - spin_lock_irqsave(&priv->lock, flags); - if (!info->irq_en_dis) { - ravb_modify(ndev, RIC0, mask, mask); - ravb_modify(ndev, TIC, mask, mask); - } else { - ravb_write(ndev, mask, RIE0); - ravb_write(ndev, mask, TIE); + if (work_done < budget && napi_complete_done(napi, work_done)) { + /* Re-enable RX/TX interrupts */ + spin_lock_irqsave(&priv->lock, flags); + if (!info->irq_en_dis) { + ravb_modify(ndev, RIC0, mask, mask); + ravb_modify(ndev, TIC, mask, mask); + } else { + ravb_write(ndev, mask, RIE0); + ravb_write(ndev, mask, TIE); + } + spin_unlock_irqrestore(&priv->lock, flags); } - spin_unlock_irqrestore(&priv->lock, flags); -out: - return budget - quota; + return work_done; } static void ravb_set_duplex_gbeth(struct net_device *ndev) @@ -2634,7 +2665,8 @@ static const struct ravb_hw_info ravb_gen3_hw_info = { .stats_len = ARRAY_SIZE(ravb_gstrings_stats), .tccr_mask = TCCR_TSRQ0 | TCCR_TSRQ1 | TCCR_TSRQ2 | TCCR_TSRQ3, .rx_max_frame_size = SZ_2K, - .rx_max_desc_use = SZ_2K - ETH_FCS_LEN + sizeof(__sum16), + .rx_buffer_size = SZ_2K + + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)), .rx_desc_size = sizeof(struct ravb_ex_rx_desc), .internal_delay = 1, .tx_counters = 1, @@ -2658,7 +2690,8 @@ static const struct ravb_hw_info ravb_gen2_hw_info = { .stats_len = ARRAY_SIZE(ravb_gstrings_stats), .tccr_mask = TCCR_TSRQ0 | TCCR_TSRQ1 | TCCR_TSRQ2 | TCCR_TSRQ3, .rx_max_frame_size = SZ_2K, - .rx_max_desc_use = SZ_2K - ETH_FCS_LEN + sizeof(__sum16), + .rx_buffer_size = SZ_2K + + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)), .rx_desc_size = sizeof(struct ravb_ex_rx_desc), .aligned_tx = 1, .gptp = 1, @@ -2679,7 +2712,8 @@ static const struct ravb_hw_info ravb_rzv2m_hw_info = { .stats_len = ARRAY_SIZE(ravb_gstrings_stats), .tccr_mask = TCCR_TSRQ0 | TCCR_TSRQ1 | TCCR_TSRQ2 | TCCR_TSRQ3, .rx_max_frame_size = SZ_2K, - .rx_max_desc_use = SZ_2K - ETH_FCS_LEN + sizeof(__sum16), + .rx_buffer_size = SZ_2K + + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)), .rx_desc_size = sizeof(struct ravb_ex_rx_desc), .multi_irqs = 1, .err_mgmt_irqs = 1, @@ -2702,9 +2736,10 @@ static const struct ravb_hw_info gbeth_hw_info = { .stats_len = ARRAY_SIZE(ravb_gstrings_stats_gbeth), .tccr_mask = TCCR_TSRQ0, .rx_max_frame_size = SZ_8K, - .rx_max_desc_use = 4080, + .rx_buffer_size = SZ_2K, .rx_desc_size = sizeof(struct ravb_rx_desc), .aligned_tx = 1, + .coalesce_irqs = 1, .tx_counters = 1, .carrier_counters = 1, .half_duplex = 1, @@ -2981,6 +3016,12 @@ static int ravb_probe(struct platform_device *pdev) if (info->nc_queues) netif_napi_add(ndev, &priv->napi[RAVB_NC], ravb_poll); + if (info->coalesce_irqs) { + netdev_sw_irq_coalesce_default_on(ndev); + if (num_present_cpus() == 1) + dev_set_threaded(ndev, true); + } + /* Network device register */ error = register_netdev(ndev); if (error) diff --git a/drivers/net/ethernet/smsc/smc91x.c b/drivers/net/ethernet/smsc/smc91x.c index 78ff3af7911ad119c187fbb0fbe6464849c8108f..907498848028b45e632703f71882c5fe97b80bce 100644 --- a/drivers/net/ethernet/smsc/smc91x.c +++ b/drivers/net/ethernet/smsc/smc91x.c @@ -1574,12 +1574,8 @@ smc_ethtool_set_link_ksettings(struct net_device *dev, (cmd->base.port != PORT_TP && cmd->base.port != PORT_AUI)) return -EINVAL; -// lp->port = cmd->base.port; lp->ctl_rfduplx = cmd->base.duplex == DUPLEX_FULL; -// if (netif_running(dev)) -// smc_set_port(dev); - ret = 0; } diff --git a/drivers/net/ethernet/smsc/smc91x.h b/drivers/net/ethernet/smsc/smc91x.h index 45ef5ac0788a82a184f320a14d8c1517e56710d0..38aa4374e813ad5f7b5efd968873de70478909ba 100644 --- a/drivers/net/ethernet/smsc/smc91x.h +++ b/drivers/net/ethernet/smsc/smc91x.h @@ -142,14 +142,14 @@ static inline void _SMC_outw_align4(u16 val, void __iomem *ioaddr, int reg, #define SMC_CAN_USE_32BIT 0 #define SMC_NOWAIT 1 -static inline void mcf_insw(void *a, unsigned char *p, int l) +static inline void mcf_insw(void __iomem *a, unsigned char *p, int l) { u16 *wp = (u16 *) p; while (l-- > 0) *wp++ = readw(a); } -static inline void mcf_outsw(void *a, unsigned char *p, int l) +static inline void mcf_outsw(void __iomem *a, unsigned char *p, int l) { u16 *wp = (u16 *) p; while (l-- > 0) diff --git a/drivers/net/ethernet/stmicro/stmmac/common.h b/drivers/net/ethernet/stmicro/stmmac/common.h index 9cd62b2110a14c9f27f4ecf89602a88d00d81b56..cd36ff4da68c6629d8d36eace16b11d78a6e9432 100644 --- a/drivers/net/ethernet/stmicro/stmmac/common.h +++ b/drivers/net/ethernet/stmicro/stmmac/common.h @@ -271,8 +271,6 @@ struct stmmac_safety_stats { /* PCS defines */ #define STMMAC_PCS_RGMII (1 << 0) #define STMMAC_PCS_SGMII (1 << 1) -#define STMMAC_PCS_TBI (1 << 2) -#define STMMAC_PCS_RTBI (1 << 3) #define SF_DMA_MODE 1 /* DMA STORE-AND-FORWARD Operation Mode */ diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c index 60283543ffc8882cc2be257ac8eec8b3d3a9337a..56649edb18cd54c26bb0bd947c80edc10762efbc 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c @@ -248,7 +248,7 @@ static void intel_speed_mode_2500(struct net_device *ndev, void *intel_data) dev_info(priv->device, "Link Speed Mode: 2.5Gbps\n"); priv->plat->max_speed = 2500; priv->plat->phy_interface = PHY_INTERFACE_MODE_2500BASEX; - priv->plat->mdio_bus_data->xpcs_an_inband = false; + priv->plat->mdio_bus_data->default_an_inband = false; } else { priv->plat->max_speed = 1000; } @@ -586,18 +586,7 @@ static int intel_mgbe_common_data(struct pci_dev *pdev, if (plat->phy_interface == PHY_INTERFACE_MODE_SGMII || plat->phy_interface == PHY_INTERFACE_MODE_1000BASEX) { plat->mdio_bus_data->has_xpcs = true; - plat->mdio_bus_data->xpcs_an_inband = true; - } - - /* For fixed-link setup, we clear xpcs_an_inband */ - if (fwnode) { - struct fwnode_handle *fixed_node; - - fixed_node = fwnode_get_named_child_node(fwnode, "fixed-link"); - if (fixed_node) - plat->mdio_bus_data->xpcs_an_inband = false; - - fwnode_handle_put(fixed_node); + plat->mdio_bus_data->default_an_inband = true; } /* Ensure mdio bus scan skips intel serdes and pcs-xpcs */ diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-qcom-ethqos.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-qcom-ethqos.c index e254b21fdb5986c4c7d620f4505f99ed0cb83c50..d9eed415b0b7bf96c0c4fcacdf1e5aa0f7e21548 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac-qcom-ethqos.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-qcom-ethqos.c @@ -605,6 +605,14 @@ static int ethqos_configure_rgmii(struct qcom_ethqos *ethqos) return 0; } +static void ethqos_set_serdes_speed(struct qcom_ethqos *ethqos, int speed) +{ + if (ethqos->serdes_speed != speed) { + phy_set_speed(ethqos->serdes_phy, speed); + ethqos->serdes_speed = speed; + } +} + /* On interface toggle MAC registers gets reset. * Configure MAC block for SGMII on ethernet phy link up */ @@ -622,9 +630,7 @@ static int ethqos_configure_sgmii(struct qcom_ethqos *ethqos) rgmii_updatel(ethqos, RGMII_CONFIG2_RGMII_CLK_SEL_CFG, RGMII_CONFIG2_RGMII_CLK_SEL_CFG, RGMII_IO_MACRO_CONFIG2); - if (ethqos->serdes_speed != SPEED_2500) - phy_set_speed(ethqos->serdes_phy, SPEED_2500); - ethqos->serdes_speed = SPEED_2500; + ethqos_set_serdes_speed(ethqos, SPEED_2500); stmmac_pcs_ctrl_ane(priv, priv->ioaddr, 0, 0, 0); break; case SPEED_1000: @@ -632,16 +638,12 @@ static int ethqos_configure_sgmii(struct qcom_ethqos *ethqos) rgmii_updatel(ethqos, RGMII_CONFIG2_RGMII_CLK_SEL_CFG, RGMII_CONFIG2_RGMII_CLK_SEL_CFG, RGMII_IO_MACRO_CONFIG2); - if (ethqos->serdes_speed != SPEED_1000) - phy_set_speed(ethqos->serdes_phy, SPEED_1000); - ethqos->serdes_speed = SPEED_1000; + ethqos_set_serdes_speed(ethqos, SPEED_1000); stmmac_pcs_ctrl_ane(priv, priv->ioaddr, 1, 0, 0); break; case SPEED_100: val |= ETHQOS_MAC_CTRL_PORT_SEL | ETHQOS_MAC_CTRL_SPEED_MODE; - if (ethqos->serdes_speed != SPEED_1000) - phy_set_speed(ethqos->serdes_phy, SPEED_1000); - ethqos->serdes_speed = SPEED_1000; + ethqos_set_serdes_speed(ethqos, SPEED_1000); stmmac_pcs_ctrl_ane(priv, priv->ioaddr, 1, 0, 0); break; case SPEED_10: @@ -651,9 +653,7 @@ static int ethqos_configure_sgmii(struct qcom_ethqos *ethqos) FIELD_PREP(RGMII_CONFIG_SGMII_CLK_DVDR, SGMII_10M_RX_CLK_DVDR), RGMII_IO_MACRO_CONFIG); - if (ethqos->serdes_speed != SPEED_1000) - phy_set_speed(ethqos->serdes_phy, ethqos->speed); - ethqos->serdes_speed = SPEED_1000; + ethqos_set_serdes_speed(ethqos, SPEED_1000); stmmac_pcs_ctrl_ane(priv, priv->ioaddr, 1, 0, 0); break; } diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c index 8555299443f4edf2475b95c1785544a1c3b73251..d413d76a893680f658c41ff70b0913918aec5e7a 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c @@ -15,7 +15,7 @@ #include #include #include -#include +#include #include "stmmac.h" #include "stmmac_pcs.h" #include "dwmac1000.h" @@ -404,11 +404,6 @@ static void dwmac1000_ctrl_ane(void __iomem *ioaddr, bool ane, bool srgmi_ral, dwmac_ctrl_ane(ioaddr, GMAC_PCS_BASE, ane, srgmi_ral, loopback); } -static void dwmac1000_rane(void __iomem *ioaddr, bool restart) -{ - dwmac_rane(ioaddr, GMAC_PCS_BASE, restart); -} - static void dwmac1000_get_adv_lp(void __iomem *ioaddr, struct rgmii_adv *adv) { dwmac_get_adv_lp(ioaddr, GMAC_PCS_BASE, adv); @@ -519,7 +514,6 @@ const struct stmmac_ops dwmac1000_ops = { .set_eee_pls = dwmac1000_set_eee_pls, .debug = dwmac1000_debug, .pcs_ctrl_ane = dwmac1000_ctrl_ane, - .pcs_rane = dwmac1000_rane, .pcs_get_adv_lp = dwmac1000_get_adv_lp, .set_mac_loopback = dwmac1000_set_mac_loopback, }; diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c index daf79cdbd3ecfe89969cf9edd03172a256007bf5..adccdd816ea913eda3e0b8326ac304ea8016d25f 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c @@ -12,7 +12,7 @@ Author: Giuseppe Cavallaro *******************************************************************************/ -#include +#include #include "dwmac1000.h" #include "dwmac_dma.h" diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c index 7667d103cd0ebd9670a42360a095cfd322c8ebac..14e847c0e1a91dcdd572803439b63ff6be2e1c5a 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac100_core.c @@ -15,7 +15,7 @@ *******************************************************************************/ #include -#include +#include #include "stmmac.h" #include "dwmac100.h" diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac100_dma.c b/drivers/net/ethernet/stmicro/stmmac/dwmac100_dma.c index dea270f60cc3e215c7ff2b36ca013a77575e29cf..b402fb54f6131b60d649093c4d4baaa3b0d9b52d 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac100_dma.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac100_dma.c @@ -14,7 +14,7 @@ Author: Giuseppe Cavallaro *******************************************************************************/ -#include +#include #include "dwmac100.h" #include "dwmac_dma.h" diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c index b25774d691957fe5244b2b7c285c496331b9c585..dbd9f93b2460f00ec62725044e00c3e279e91c63 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c @@ -758,11 +758,6 @@ static void dwmac4_ctrl_ane(void __iomem *ioaddr, bool ane, bool srgmi_ral, dwmac_ctrl_ane(ioaddr, GMAC_PCS_BASE, ane, srgmi_ral, loopback); } -static void dwmac4_rane(void __iomem *ioaddr, bool restart) -{ - dwmac_rane(ioaddr, GMAC_PCS_BASE, restart); -} - static void dwmac4_get_adv_lp(void __iomem *ioaddr, struct rgmii_adv *adv) { dwmac_get_adv_lp(ioaddr, GMAC_PCS_BASE, adv); @@ -1215,7 +1210,6 @@ const struct stmmac_ops dwmac4_ops = { .set_eee_timer = dwmac4_set_eee_timer, .set_eee_pls = dwmac4_set_eee_pls, .pcs_ctrl_ane = dwmac4_ctrl_ane, - .pcs_rane = dwmac4_rane, .pcs_get_adv_lp = dwmac4_get_adv_lp, .debug = dwmac4_debug, .set_filter = dwmac4_set_filter, @@ -1260,7 +1254,6 @@ const struct stmmac_ops dwmac410_ops = { .set_eee_timer = dwmac4_set_eee_timer, .set_eee_pls = dwmac4_set_eee_pls, .pcs_ctrl_ane = dwmac4_ctrl_ane, - .pcs_rane = dwmac4_rane, .pcs_get_adv_lp = dwmac4_get_adv_lp, .debug = dwmac4_debug, .set_filter = dwmac4_set_filter, @@ -1309,7 +1302,6 @@ const struct stmmac_ops dwmac510_ops = { .set_eee_timer = dwmac4_set_eee_timer, .set_eee_pls = dwmac4_set_eee_pls, .pcs_ctrl_ane = dwmac4_ctrl_ane, - .pcs_rane = dwmac4_rane, .pcs_get_adv_lp = dwmac4_get_adv_lp, .debug = dwmac4_debug, .set_filter = dwmac4_set_filter, diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c index f8e7775bb63364c589da99cb4c954a38f4411567..6a987cf598e4fb7fdee9e42781220b15d72c8462 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c @@ -1554,9 +1554,6 @@ const struct stmmac_ops dwxgmac210_ops = { .reset_eee_mode = dwxgmac2_reset_eee_mode, .set_eee_timer = dwxgmac2_set_eee_timer, .set_eee_pls = dwxgmac2_set_eee_pls, - .pcs_ctrl_ane = NULL, - .pcs_rane = NULL, - .pcs_get_adv_lp = NULL, .debug = NULL, .set_filter = dwxgmac2_set_filter, .safety_feat_config = dwxgmac3_safety_feat_config, @@ -1614,9 +1611,6 @@ const struct stmmac_ops dwxlgmac2_ops = { .reset_eee_mode = dwxgmac2_reset_eee_mode, .set_eee_timer = dwxgmac2_set_eee_timer, .set_eee_pls = dwxgmac2_set_eee_pls, - .pcs_ctrl_ane = NULL, - .pcs_rane = NULL, - .pcs_get_adv_lp = NULL, .debug = NULL, .set_filter = dwxgmac2_set_filter, .safety_feat_config = dwxgmac3_safety_feat_config, diff --git a/drivers/net/ethernet/stmicro/stmmac/hwif.h b/drivers/net/ethernet/stmicro/stmmac/hwif.h index 90384db228b5ce5096602bd549bb85bcf86b3b26..97934ccba5b1abc98c591efb18252332ab15cb6e 100644 --- a/drivers/net/ethernet/stmicro/stmmac/hwif.h +++ b/drivers/net/ethernet/stmicro/stmmac/hwif.h @@ -370,7 +370,6 @@ struct stmmac_ops { /* PCS calls */ void (*pcs_ctrl_ane)(void __iomem *ioaddr, bool ane, bool srgmi_ral, bool loopback); - void (*pcs_rane)(void __iomem *ioaddr, bool restart); void (*pcs_get_adv_lp)(void __iomem *ioaddr, struct rgmii_adv *adv); /* Safety Features */ int (*safety_feat_config)(void __iomem *ioaddr, unsigned int asp, @@ -484,8 +483,6 @@ struct stmmac_ops { stmmac_do_void_callback(__priv, mac, debug, __priv, __args) #define stmmac_pcs_ctrl_ane(__priv, __args...) \ stmmac_do_void_callback(__priv, mac, pcs_ctrl_ane, __args) -#define stmmac_pcs_rane(__priv, __args...) \ - stmmac_do_void_callback(__priv, mac, pcs_rane, __priv, __args) #define stmmac_pcs_get_adv_lp(__priv, __args...) \ stmmac_do_void_callback(__priv, mac, pcs_get_adv_lp, __args) #define stmmac_safety_feat_config(__priv, __args...) \ diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c index 542e2633a6f52223bf15ef31fb17d2377fb4583c..18468c0228f0b2730ed33dec08e39f0319fddca1 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c @@ -11,10 +11,10 @@ #include #include #include +#include #include #include #include -#include #include "stmmac.h" #include "dwmac_dma.h" diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index b3afc7cb7d72df690627cc929af2f07e59d93c79..bbedf2a8c60f243e7363c3b9b8f4e2688b9a045f 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -471,13 +471,6 @@ bool stmmac_eee_init(struct stmmac_priv *priv) { int eee_tw_timer = priv->eee_tw_timer; - /* Using PCS we cannot dial with the phy registers at this stage - * so we do not support extra feature like EEE. - */ - if (priv->hw->pcs == STMMAC_PCS_TBI || - priv->hw->pcs == STMMAC_PCS_RTBI) - return false; - /* Check if MAC core supports the EEE feature. */ if (!priv->dma_cap.eee) return false; @@ -1228,8 +1221,8 @@ static int stmmac_phy_setup(struct stmmac_priv *priv) mdio_bus_data = priv->plat->mdio_bus_data; if (mdio_bus_data) - priv->phylink_config.ovr_an_inband = - mdio_bus_data->xpcs_an_inband; + priv->phylink_config.default_an_inband = + mdio_bus_data->default_an_inband; /* Set the platform/firmware specified interface mode. Note, phylink * deals with the PHY interface mode, not the MAC interface mode. @@ -3953,9 +3946,7 @@ static int __stmmac_open(struct net_device *dev, if (ret < 0) return ret; - if (priv->hw->pcs != STMMAC_PCS_TBI && - priv->hw->pcs != STMMAC_PCS_RTBI && - (!priv->hw->xpcs || + if ((!priv->hw->xpcs || xpcs_get_an_mode(priv->hw->xpcs, mode) != DW_AN_C73)) { ret = stmmac_init_phy(dev); if (ret) { @@ -4097,8 +4088,6 @@ static int stmmac_release(struct net_device *dev) if (priv->plat->serdes_powerdown) priv->plat->serdes_powerdown(dev, priv->plat->bsp_priv); - netif_carrier_off(dev); - stmmac_release_ptp(priv); pm_runtime_put(priv->device); @@ -7739,16 +7728,12 @@ int stmmac_dvr_probe(struct device *device, if (!pm_runtime_enabled(device)) pm_runtime_enable(device); - if (priv->hw->pcs != STMMAC_PCS_TBI && - priv->hw->pcs != STMMAC_PCS_RTBI) { - /* MDIO bus Registration */ - ret = stmmac_mdio_register(ndev); - if (ret < 0) { - dev_err_probe(priv->device, ret, - "%s: MDIO bus (id: %d) registration failed\n", - __func__, priv->plat->bus_id); - goto error_mdio_register; - } + ret = stmmac_mdio_register(ndev); + if (ret < 0) { + dev_err_probe(priv->device, ret, + "MDIO bus (id: %d) registration failed\n", + priv->plat->bus_id); + goto error_mdio_register; } if (priv->plat->speed_mode_2500) @@ -7790,9 +7775,7 @@ error_netdev_register: error_phy_setup: stmmac_pcs_clean(ndev); error_pcs_setup: - if (priv->hw->pcs != STMMAC_PCS_TBI && - priv->hw->pcs != STMMAC_PCS_RTBI) - stmmac_mdio_unregister(ndev); + stmmac_mdio_unregister(ndev); error_mdio_register: stmmac_napi_del(ndev); error_hw_init: @@ -7821,7 +7804,6 @@ void stmmac_dvr_remove(struct device *dev) stmmac_stop_all_dma(priv); stmmac_mac_set(priv, priv->ioaddr, false); - netif_carrier_off(ndev); unregister_netdev(ndev); #ifdef CONFIG_DEBUG_FS @@ -7833,10 +7815,8 @@ void stmmac_dvr_remove(struct device *dev) reset_control_assert(priv->plat->stmmac_ahb_rst); stmmac_pcs_clean(ndev); + stmmac_mdio_unregister(ndev); - if (priv->hw->pcs != STMMAC_PCS_TBI && - priv->hw->pcs != STMMAC_PCS_RTBI) - stmmac_mdio_unregister(ndev); destroy_workqueue(priv->wq); mutex_destroy(&priv->lock); bitmap_free(priv->af_xdp_zc_qps); diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_pcs.h b/drivers/net/ethernet/stmicro/stmmac/stmmac_pcs.h index 13a30e6df4c155a23857fc22723a24bf97ca6e34..1bdf87b237c4efa8c7e27847b18b876e9f0d39be 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_pcs.h +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_pcs.h @@ -74,23 +74,6 @@ static inline void dwmac_pcs_isr(void __iomem *ioaddr, u32 reg, } } -/** - * dwmac_rane - To restart ANE - * @ioaddr: IO registers pointer - * @reg: Base address of the AN Control Register. - * @restart: to restart ANE - * Description: this is to just restart the Auto-Negotiation. - */ -static inline void dwmac_rane(void __iomem *ioaddr, u32 reg, bool restart) -{ - u32 value = readl(ioaddr + GMAC_AN_CTRL(reg)); - - if (restart) - value |= GMAC_AN_CTRL_RAN; - - writel(value, ioaddr + GMAC_AN_CTRL(reg)); -} - /** * dwmac_ctrl_ane - To program the AN Control Register. * @ioaddr: IO registers pointer diff --git a/drivers/net/ethernet/ti/Kconfig b/drivers/net/ethernet/ti/Kconfig index 1729eb0e0b410b7420c33e118e1b91f8e0f6a1a8..f160a3b71499a4fcb9473dead91c043e49572d7c 100644 --- a/drivers/net/ethernet/ti/Kconfig +++ b/drivers/net/ethernet/ti/Kconfig @@ -204,6 +204,7 @@ config TI_ICSSG_PRUETH_SR1 select TI_ICSS_IEP select TI_K3_CPPI_DESC_POOL depends on PRU_REMOTEPROC + depends on NET_SWITCHDEV depends on ARCH_K3 && OF && TI_K3_UDMA_GLUE_LAYER help Support dual Gigabit Ethernet ports over the ICSSG PRU Subsystem. diff --git a/drivers/net/ethernet/ti/Makefile b/drivers/net/ethernet/ti/Makefile index 6e086b4c0384aa7a08ac76d821f917c93388ac12..59cd20a38267c2ba0905c5a083a78df05afd40b2 100644 --- a/drivers/net/ethernet/ti/Makefile +++ b/drivers/net/ethernet/ti/Makefile @@ -39,7 +39,8 @@ icssg-prueth-y := icssg/icssg_prueth.o \ icssg/icssg_config.o \ icssg/icssg_mii_cfg.o \ icssg/icssg_stats.o \ - icssg/icssg_ethtool.o + icssg/icssg_ethtool.o \ + icssg/icssg_switchdev.o obj-$(CONFIG_TI_ICSSG_PRUETH_SR1) += icssg-prueth-sr1.o icssg-prueth-sr1-y := icssg/icssg_prueth_sr1.o \ icssg/icssg_common.o \ diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.c b/drivers/net/ethernet/ti/am65-cpsw-nuss.c index 4e50b37928885960d0c35e8bb833f097061ae4ee..81d9f21086ec93288a8211437b4d29d9ca1e350d 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-nuss.c +++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.c @@ -896,7 +896,7 @@ static int am65_cpsw_nuss_ndo_slave_open(struct net_device *ndev) /* mac_sl should be configured via phy-link interface */ am65_cpsw_sl_ctl_reset(port); - ret = phylink_of_phy_connect(port->slave.phylink, port->slave.phy_node, 0); + ret = phylink_of_phy_connect(port->slave.phylink, port->slave.port_np, 0); if (ret) goto error_cleanup; @@ -2424,10 +2424,10 @@ static int am65_cpsw_nuss_init_rx_chns(struct am65_cpsw_common *common) rx_chn->irq = k3_udma_glue_rx_get_irq(rx_chn->rx_chn, i); - if (rx_chn->irq <= 0) { + if (rx_chn->irq < 0) { dev_err(dev, "Failed to get rx dma irq %d\n", rx_chn->irq); - ret = -ENXIO; + ret = rx_chn->irq; goto err; } } @@ -2611,7 +2611,7 @@ static int am65_cpsw_nuss_init_slave_ports(struct am65_cpsw_common *common) of_property_read_bool(port_np, "ti,mac-only"); /* get phy/link info */ - port->slave.phy_node = port_np; + port->slave.port_np = port_np; ret = of_get_phy_mode(port_np, &port->slave.phy_if); if (ret) { dev_err(dev, "%pOF read phy-mode err %d\n", @@ -2703,6 +2703,7 @@ am65_cpsw_nuss_init_port_ndev(struct am65_cpsw_common *common, u32 port_idx) mutex_init(&ndev_priv->mm_lock); port->qos.link_speed = SPEED_UNKNOWN; SET_NETDEV_DEV(port->ndev, dev); + port->ndev->dev.of_node = port->slave.port_np; eth_hw_addr_set(port->ndev, port->slave.mac_addr); @@ -2760,7 +2761,7 @@ am65_cpsw_nuss_init_port_ndev(struct am65_cpsw_common *common, u32 port_idx) } phylink = phylink_create(&port->slave.phylink_config, - of_node_to_fwnode(port->slave.phy_node), + of_node_to_fwnode(port->slave.port_np), port->slave.phy_if, &am65_cpsw_phylink_mac_ops); if (IS_ERR(phylink)) diff --git a/drivers/net/ethernet/ti/am65-cpsw-nuss.h b/drivers/net/ethernet/ti/am65-cpsw-nuss.h index d8ce88dc9c89a4a8f11308cd6cec41747f24369c..e2ce2be320bd6c152864f36af7dda14b83b0819b 100644 --- a/drivers/net/ethernet/ti/am65-cpsw-nuss.h +++ b/drivers/net/ethernet/ti/am65-cpsw-nuss.h @@ -30,7 +30,7 @@ struct am65_cpts; struct am65_cpsw_slave_data { bool mac_only; struct cpsw_sl *mac_sl; - struct device_node *phy_node; + struct device_node *port_np; phy_interface_t phy_if; struct phy *ifphy; struct phy *serdes_phy; diff --git a/drivers/net/ethernet/ti/icssg/icssg_classifier.c b/drivers/net/ethernet/ti/icssg/icssg_classifier.c index 79ba47bb3602ec21c435cceac66bdc291fa270a3..f7d21da1a0fb62b0112019b6600f21971ce98adb 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_classifier.c +++ b/drivers/net/ethernet/ti/icssg/icssg_classifier.c @@ -455,7 +455,7 @@ void icssg_ft1_set_mac_addr(struct regmap *miig_rt, int slice, u8 *mac_addr) { const u8 mask_addr[] = { 0, 0, 0, 0, 0, 0, }; - rx_class_ft1_set_start_len(miig_rt, slice, 0, 6); + rx_class_ft1_set_start_len(miig_rt, slice, ETH_ALEN, ETH_ALEN); rx_class_ft1_set_da(miig_rt, slice, 0, mac_addr); rx_class_ft1_set_da_mask(miig_rt, slice, 0, mask_addr); rx_class_ft1_cfg_set_type(miig_rt, slice, 0, FT1_CFG_TYPE_EQ); diff --git a/drivers/net/ethernet/ti/icssg/icssg_common.c b/drivers/net/ethernet/ti/icssg/icssg_common.c index 088ab8076db47f67622e73843eab9675f65ef73f..2f716c0d70607a565755b7257172ecd544eefb6e 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_common.c +++ b/drivers/net/ethernet/ti/icssg/icssg_common.c @@ -440,9 +440,7 @@ int prueth_init_rx_chns(struct prueth_emac *emac, fdqring_id = k3_udma_glue_rx_flow_get_fdq_id(rx_chn->rx_chn, i); ret = k3_udma_glue_rx_get_irq(rx_chn->rx_chn, i); - if (ret <= 0) { - if (!ret) - ret = -ENXIO; + if (ret < 0) { netdev_err(ndev, "Failed to get rx dma irq"); goto fail; } @@ -581,6 +579,8 @@ static int emac_rx_packet(struct prueth_emac *emac, u32 flow_id) } else { /* send the filled skb up the n/w stack */ skb_put(skb, pkt_len); + if (emac->prueth->is_switch_mode) + skb->offload_fwd_mark = emac->offload_fwd_mark; skb->protocol = eth_type_trans(skb, ndev); napi_gro_receive(&emac->napi_rx, skb); ndev->stats.rx_bytes += pkt_len; diff --git a/drivers/net/ethernet/ti/icssg/icssg_config.c b/drivers/net/ethernet/ti/icssg/icssg_config.c index 15f2235bf90fad6481e70029eef8e9ebd084e4e6..9444e56b76729d84097fae12249a104b774ead0b 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_config.c +++ b/drivers/net/ethernet/ti/icssg/icssg_config.c @@ -107,28 +107,49 @@ static const struct map hwq_map[2][ICSSG_NUM_OTHER_QUEUES] = { }, }; +static void icssg_config_mii_init_switch(struct prueth_emac *emac) +{ + struct prueth *prueth = emac->prueth; + int mii = prueth_emac_slice(emac); + u32 txcfg_reg, pcnt_reg, txcfg; + struct regmap *mii_rt; + + mii_rt = prueth->mii_rt; + + txcfg_reg = (mii == ICSS_MII0) ? PRUSS_MII_RT_TXCFG0 : + PRUSS_MII_RT_TXCFG1; + pcnt_reg = (mii == ICSS_MII0) ? PRUSS_MII_RT_RX_PCNT0 : + PRUSS_MII_RT_RX_PCNT1; + + txcfg = PRUSS_MII_RT_TXCFG_TX_ENABLE | + PRUSS_MII_RT_TXCFG_TX_AUTO_PREAMBLE | + PRUSS_MII_RT_TXCFG_TX_IPG_WIRE_CLK_EN; + + if (emac->phy_if == PHY_INTERFACE_MODE_MII && mii == ICSS_MII1) + txcfg |= PRUSS_MII_RT_TXCFG_TX_MUX_SEL; + else if (emac->phy_if != PHY_INTERFACE_MODE_MII && mii == ICSS_MII0) + txcfg |= PRUSS_MII_RT_TXCFG_TX_MUX_SEL; + + regmap_write(mii_rt, txcfg_reg, txcfg); + regmap_write(mii_rt, pcnt_reg, 0x1); +} + static void icssg_config_mii_init(struct prueth_emac *emac) { - u32 rxcfg, txcfg, rxcfg_reg, txcfg_reg, pcnt_reg; struct prueth *prueth = emac->prueth; int slice = prueth_emac_slice(emac); + u32 txcfg, txcfg_reg, pcnt_reg; struct regmap *mii_rt; mii_rt = prueth->mii_rt; - rxcfg_reg = (slice == ICSS_MII0) ? PRUSS_MII_RT_RXCFG0 : - PRUSS_MII_RT_RXCFG1; txcfg_reg = (slice == ICSS_MII0) ? PRUSS_MII_RT_TXCFG0 : PRUSS_MII_RT_TXCFG1; pcnt_reg = (slice == ICSS_MII0) ? PRUSS_MII_RT_RX_PCNT0 : PRUSS_MII_RT_RX_PCNT1; - rxcfg = MII_RXCFG_DEFAULT; txcfg = MII_TXCFG_DEFAULT; - if (slice == ICSS_MII1) - rxcfg |= PRUSS_MII_RT_RXCFG_RX_MUX_SEL; - /* In MII mode TX lines swapped inside ICSSG, so TX_MUX_SEL cfg need * to be swapped also comparing to RGMII mode. */ @@ -137,7 +158,6 @@ static void icssg_config_mii_init(struct prueth_emac *emac) else if (emac->phy_if != PHY_INTERFACE_MODE_MII && slice == ICSS_MII1) txcfg |= PRUSS_MII_RT_TXCFG_TX_MUX_SEL; - regmap_write(mii_rt, rxcfg_reg, rxcfg); regmap_write(mii_rt, txcfg_reg, txcfg); regmap_write(mii_rt, pcnt_reg, 0x1); } @@ -257,6 +277,66 @@ static int emac_r30_is_done(struct prueth_emac *emac) return 1; } +static int prueth_switch_buffer_setup(struct prueth_emac *emac) +{ + struct icssg_buffer_pool_cfg __iomem *bpool_cfg; + struct icssg_rxq_ctx __iomem *rxq_ctx; + struct prueth *prueth = emac->prueth; + int slice = prueth_emac_slice(emac); + u32 addr; + int i; + + addr = lower_32_bits(prueth->msmcram.pa); + if (slice) + addr += PRUETH_NUM_BUF_POOLS * PRUETH_EMAC_BUF_POOL_SIZE; + + if (addr % SZ_64K) { + dev_warn(prueth->dev, "buffer pool needs to be 64KB aligned\n"); + return -EINVAL; + } + + bpool_cfg = emac->dram.va + BUFFER_POOL_0_ADDR_OFFSET; + /* workaround for f/w bug. bpool 0 needs to be initialized */ + for (i = 0; i < PRUETH_NUM_BUF_POOLS; i++) { + writel(addr, &bpool_cfg[i].addr); + writel(PRUETH_EMAC_BUF_POOL_SIZE, &bpool_cfg[i].len); + addr += PRUETH_EMAC_BUF_POOL_SIZE; + } + + if (!slice) + addr += PRUETH_NUM_BUF_POOLS * PRUETH_EMAC_BUF_POOL_SIZE; + else + addr += PRUETH_SW_NUM_BUF_POOLS_HOST * PRUETH_SW_BUF_POOL_SIZE_HOST; + + for (i = PRUETH_NUM_BUF_POOLS; + i < 2 * PRUETH_SW_NUM_BUF_POOLS_HOST + PRUETH_NUM_BUF_POOLS; + i++) { + /* The driver only uses first 4 queues per PRU so only initialize them */ + if (i % PRUETH_SW_NUM_BUF_POOLS_HOST < PRUETH_SW_NUM_BUF_POOLS_PER_PRU) { + writel(addr, &bpool_cfg[i].addr); + writel(PRUETH_SW_BUF_POOL_SIZE_HOST, &bpool_cfg[i].len); + addr += PRUETH_SW_BUF_POOL_SIZE_HOST; + } else { + writel(0, &bpool_cfg[i].addr); + writel(0, &bpool_cfg[i].len); + } + } + + if (!slice) + addr += PRUETH_SW_NUM_BUF_POOLS_HOST * PRUETH_SW_BUF_POOL_SIZE_HOST; + else + addr += PRUETH_EMAC_RX_CTX_BUF_SIZE; + + rxq_ctx = emac->dram.va + HOST_RX_Q_PRE_CONTEXT_OFFSET; + for (i = 0; i < 3; i++) + writel(addr, &rxq_ctx->start[i]); + + addr += PRUETH_EMAC_RX_CTX_BUF_SIZE; + writel(addr - SZ_2K, &rxq_ctx->end); + + return 0; +} + static int prueth_emac_buffer_setup(struct prueth_emac *emac) { struct icssg_buffer_pool_cfg __iomem *bpool_cfg; @@ -321,25 +401,63 @@ static void icssg_init_emac_mode(struct prueth *prueth) /* When the device is configured as a bridge and it is being brought * back to the emac mode, the host mac address has to be set as 0. */ + u32 addr = prueth->shram.pa + EMAC_ICSSG_SWITCH_DEFAULT_VLAN_TABLE_OFFSET; + int i; u8 mac[ETH_ALEN] = { 0 }; if (prueth->emacs_initialized) return; - regmap_update_bits(prueth->miig_rt, FDB_GEN_CFG1, - SMEM_VLAN_OFFSET_MASK, 0); - regmap_write(prueth->miig_rt, FDB_GEN_CFG2, 0); + /* Set VLAN TABLE address base */ + regmap_update_bits(prueth->miig_rt, FDB_GEN_CFG1, SMEM_VLAN_OFFSET_MASK, + addr << SMEM_VLAN_OFFSET); + /* Set enable VLAN aware mode, and FDBs for all PRUs */ + regmap_write(prueth->miig_rt, FDB_GEN_CFG2, (FDB_PRU0_EN | FDB_PRU1_EN | FDB_HOST_EN)); + prueth->vlan_tbl = (struct prueth_vlan_tbl __force *)(prueth->shram.va + + EMAC_ICSSG_SWITCH_DEFAULT_VLAN_TABLE_OFFSET); + for (i = 0; i < SZ_4K - 1; i++) { + prueth->vlan_tbl[i].fid = i; + prueth->vlan_tbl[i].fid_c1 = 0; + } /* Clear host MAC address */ icssg_class_set_host_mac_addr(prueth->miig_rt, mac); } +static void icssg_init_switch_mode(struct prueth *prueth) +{ + u32 addr = prueth->shram.pa + EMAC_ICSSG_SWITCH_DEFAULT_VLAN_TABLE_OFFSET; + int i; + + if (prueth->emacs_initialized) + return; + + /* Set VLAN TABLE address base */ + regmap_update_bits(prueth->miig_rt, FDB_GEN_CFG1, SMEM_VLAN_OFFSET_MASK, + addr << SMEM_VLAN_OFFSET); + /* Set enable VLAN aware mode, and FDBs for all PRUs */ + regmap_write(prueth->miig_rt, FDB_GEN_CFG2, FDB_EN_ALL); + prueth->vlan_tbl = (struct prueth_vlan_tbl __force *)(prueth->shram.va + + EMAC_ICSSG_SWITCH_DEFAULT_VLAN_TABLE_OFFSET); + for (i = 0; i < SZ_4K - 1; i++) { + prueth->vlan_tbl[i].fid = i; + prueth->vlan_tbl[i].fid_c1 = 0; + } + + if (prueth->hw_bridge_dev) + icssg_class_set_host_mac_addr(prueth->miig_rt, prueth->hw_bridge_dev->dev_addr); + icssg_set_pvid(prueth, prueth->default_vlan, PRUETH_PORT_HOST); +} + int icssg_config(struct prueth *prueth, struct prueth_emac *emac, int slice) { void __iomem *config = emac->dram.va + ICSSG_CONFIG_OFFSET; struct icssg_flow_cfg __iomem *flow_cfg; int ret; - icssg_init_emac_mode(prueth); + if (prueth->is_switch_mode) + icssg_init_switch_mode(prueth); + else + icssg_init_emac_mode(prueth); memset_io(config, 0, TAS_GATE_MASK_LIST0); icssg_miig_queues_init(prueth, slice); @@ -353,7 +471,10 @@ int icssg_config(struct prueth *prueth, struct prueth_emac *emac, int slice) regmap_update_bits(prueth->miig_rt, ICSSG_CFG_OFFSET, ICSSG_CFG_DEFAULT, ICSSG_CFG_DEFAULT); icssg_miig_set_interface_mode(prueth->miig_rt, slice, emac->phy_if); - icssg_config_mii_init(emac); + if (prueth->is_switch_mode) + icssg_config_mii_init_switch(emac); + else + icssg_config_mii_init(emac); icssg_config_ipg(emac); icssg_update_rgmii_cfg(prueth->miig_rt, emac); @@ -376,7 +497,10 @@ int icssg_config(struct prueth *prueth, struct prueth_emac *emac, int slice) writeb(0, config + SPL_PKT_DEFAULT_PRIORITY); writeb(0, config + QUEUE_NUM_UNTAGGED); - ret = prueth_emac_buffer_setup(emac); + if (prueth->is_switch_mode) + ret = prueth_switch_buffer_setup(emac); + else + ret = prueth_emac_buffer_setup(emac); if (ret) return ret; @@ -477,3 +601,173 @@ void icssg_config_set_speed(struct prueth_emac *emac) writeb(fw_speed, emac->dram.va + PORT_LINK_SPEED_OFFSET); } + +int icssg_send_fdb_msg(struct prueth_emac *emac, struct mgmt_cmd *cmd, + struct mgmt_cmd_rsp *rsp) +{ + struct prueth *prueth = emac->prueth; + int slice = prueth_emac_slice(emac); + int addr, ret; + + addr = icssg_queue_pop(prueth, slice == 0 ? + ICSSG_CMD_POP_SLICE0 : ICSSG_CMD_POP_SLICE1); + if (addr < 0) + return addr; + + /* First 4 bytes have FW owned buffer linking info which should + * not be touched + */ + memcpy_toio(prueth->shram.va + addr + 4, cmd, sizeof(*cmd)); + icssg_queue_push(prueth, slice == 0 ? + ICSSG_CMD_PUSH_SLICE0 : ICSSG_CMD_PUSH_SLICE1, addr); + ret = read_poll_timeout(icssg_queue_pop, addr, addr >= 0, + 2000, 20000000, false, prueth, slice == 0 ? + ICSSG_RSP_POP_SLICE0 : ICSSG_RSP_POP_SLICE1); + if (ret) { + netdev_err(emac->ndev, "Timedout sending HWQ message\n"); + return ret; + } + + memcpy_fromio(rsp, prueth->shram.va + addr, sizeof(*rsp)); + /* Return buffer back for to pool */ + icssg_queue_push(prueth, slice == 0 ? + ICSSG_RSP_PUSH_SLICE0 : ICSSG_RSP_PUSH_SLICE1, addr); + + return 0; +} + +static void icssg_fdb_setup(struct prueth_emac *emac, struct mgmt_cmd *fdb_cmd, + const unsigned char *addr, u8 fid, int cmd) +{ + int slice = prueth_emac_slice(emac); + u8 mac_fid[ETH_ALEN + 2]; + u16 fdb_slot; + + ether_addr_copy(mac_fid, addr); + + /* 1-1 VID-FID mapping is already setup */ + mac_fid[ETH_ALEN] = fid; + mac_fid[ETH_ALEN + 1] = 0; + + fdb_slot = bitrev32(crc32_le(0, mac_fid, 8)) & PRUETH_SWITCH_FDB_MASK; + + fdb_cmd->header = ICSSG_FW_MGMT_CMD_HEADER; + fdb_cmd->type = ICSSG_FW_MGMT_FDB_CMD_TYPE; + fdb_cmd->seqnum = ++(emac->prueth->icssg_hwcmdseq); + fdb_cmd->param = cmd; + fdb_cmd->param |= (slice << 4); + + memcpy(&fdb_cmd->cmd_args[0], addr, 4); + memcpy(&fdb_cmd->cmd_args[1], &addr[4], 2); + fdb_cmd->cmd_args[2] = fdb_slot; + + netdev_dbg(emac->ndev, "MAC %pM slot %X FID %X\n", addr, fdb_slot, fid); +} + +int icssg_fdb_add_del(struct prueth_emac *emac, const unsigned char *addr, + u8 vid, u8 fid_c2, bool add) +{ + struct mgmt_cmd_rsp fdb_cmd_rsp = { 0 }; + struct mgmt_cmd fdb_cmd = { 0 }; + u8 fid = vid; + int ret; + + icssg_fdb_setup(emac, &fdb_cmd, addr, fid, add ? ICSS_CMD_ADD_FDB : ICSS_CMD_DEL_FDB); + + fid_c2 |= ICSSG_FDB_ENTRY_VALID; + fdb_cmd.cmd_args[1] |= ((fid << 16) | (fid_c2 << 24)); + + ret = icssg_send_fdb_msg(emac, &fdb_cmd, &fdb_cmd_rsp); + if (ret) + return ret; + + WARN_ON(fdb_cmd.seqnum != fdb_cmd_rsp.seqnum); + if (fdb_cmd_rsp.status == 1) + return 0; + + return -EINVAL; +} + +int icssg_fdb_lookup(struct prueth_emac *emac, const unsigned char *addr, + u8 vid) +{ + struct mgmt_cmd_rsp fdb_cmd_rsp = { 0 }; + struct mgmt_cmd fdb_cmd = { 0 }; + struct prueth_fdb_slot *slot; + u8 fid = vid; + int ret, i; + + icssg_fdb_setup(emac, &fdb_cmd, addr, fid, ICSS_CMD_GET_FDB_SLOT); + + fdb_cmd.cmd_args[1] |= fid << 16; + + ret = icssg_send_fdb_msg(emac, &fdb_cmd, &fdb_cmd_rsp); + if (ret) + return ret; + + WARN_ON(fdb_cmd.seqnum != fdb_cmd_rsp.seqnum); + + slot = (struct prueth_fdb_slot __force *)(emac->dram.va + FDB_CMD_BUFFER); + for (i = 0; i < 4; i++) { + if (ether_addr_equal(addr, slot->mac) && vid == slot->fid) + return (slot->fid_c2 & ~ICSSG_FDB_ENTRY_VALID); + slot++; + } + + return 0; +} + +void icssg_vtbl_modify(struct prueth_emac *emac, u8 vid, u8 port_mask, + u8 untag_mask, bool add) +{ + struct prueth *prueth = emac->prueth; + struct prueth_vlan_tbl *tbl; + u8 fid_c1; + + tbl = prueth->vlan_tbl; + fid_c1 = tbl[vid].fid_c1; + + /* FID_C1: bit0..2 port membership mask, + * bit3..5 tagging mask for each port + * bit6 Stream VID (not handled currently) + * bit7 MC flood (not handled currently) + */ + if (add) { + fid_c1 |= (port_mask | port_mask << 3); + fid_c1 &= ~(untag_mask << 3); + } else { + fid_c1 &= ~(port_mask | port_mask << 3); + } + + tbl[vid].fid_c1 = fid_c1; +} + +u16 icssg_get_pvid(struct prueth_emac *emac) +{ + struct prueth *prueth = emac->prueth; + u32 pvid; + + if (emac->port_id == PRUETH_PORT_MII0) + pvid = readl(prueth->shram.va + EMAC_ICSSG_SWITCH_PORT1_DEFAULT_VLAN_OFFSET); + else + pvid = readl(prueth->shram.va + EMAC_ICSSG_SWITCH_PORT2_DEFAULT_VLAN_OFFSET); + + pvid = pvid >> 24; + + return pvid; +} + +void icssg_set_pvid(struct prueth *prueth, u8 vid, u8 port) +{ + u32 pvid; + + /* only 256 VLANs are supported */ + pvid = (u32 __force)cpu_to_be32((ETH_P_8021Q << 16) | (vid & 0xff)); + + if (port == PRUETH_PORT_MII0) + writel(pvid, prueth->shram.va + EMAC_ICSSG_SWITCH_PORT1_DEFAULT_VLAN_OFFSET); + else if (port == PRUETH_PORT_MII1) + writel(pvid, prueth->shram.va + EMAC_ICSSG_SWITCH_PORT2_DEFAULT_VLAN_OFFSET); + else + writel(pvid, prueth->shram.va + EMAC_ICSSG_SWITCH_PORT0_DEFAULT_VLAN_OFFSET); +} diff --git a/drivers/net/ethernet/ti/icssg/icssg_config.h b/drivers/net/ethernet/ti/icssg/icssg_config.h index cf2ea4bd22a2d5fad805c112aeda45f6432e5a52..1ac60283923bd3db0d459a0e06a8c997e2716078 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_config.h +++ b/drivers/net/ethernet/ti/icssg/icssg_config.h @@ -35,6 +35,15 @@ struct icssg_flow_cfg { (2 * (PRUETH_EMAC_BUF_POOL_SIZE * PRUETH_NUM_BUF_POOLS + \ PRUETH_EMAC_RX_CTX_BUF_SIZE * 2)) +#define PRUETH_SW_BUF_POOL_SIZE_HOST SZ_4K +#define PRUETH_SW_NUM_BUF_POOLS_HOST 8 +#define PRUETH_SW_NUM_BUF_POOLS_PER_PRU 4 +#define MSMC_RAM_SIZE_SWITCH_MODE \ + (MSMC_RAM_SIZE + \ + (2 * PRUETH_SW_BUF_POOL_SIZE_HOST * PRUETH_SW_NUM_BUF_POOLS_HOST)) + +#define PRUETH_SWITCH_FDB_MASK ((SIZE_OF_FDB / NUMBER_OF_FDB_BUCKET_ENTRIES) - 1) + struct icssg_rxq_ctx { __le32 start[3]; __le32 end; @@ -202,6 +211,23 @@ struct icssg_setclock_desc { #define ICSSG_TS_PUSH_SLICE0 40 #define ICSSG_TS_PUSH_SLICE1 41 +struct mgmt_cmd { + u8 param; + u8 seqnum; + u8 type; + u8 header; + u32 cmd_args[3]; +}; + +struct mgmt_cmd_rsp { + u32 reserved; + u8 status; + u8 seqnum; + u8 type; + u8 header; + u32 cmd_args[3]; +}; + /* FDB FID_C2 flag definitions */ /* Indicates host port membership.*/ #define ICSSG_FDB_ENTRY_P0_MEMBERSHIP BIT(0) diff --git a/drivers/net/ethernet/ti/icssg/icssg_prueth.c b/drivers/net/ethernet/ti/icssg/icssg_prueth.c index 1ea3fbd5e954ebb0d216381fb762c989b07f457b..e13835100754f7ac5fef0e8ae62b66585da3dfa6 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_prueth.c +++ b/drivers/net/ethernet/ti/icssg/icssg_prueth.c @@ -27,13 +27,19 @@ #include #include #include +#include #include "icssg_prueth.h" #include "icssg_mii_rt.h" +#include "icssg_switchdev.h" #include "../k3-cppi-desc-pool.h" #define PRUETH_MODULE_DESCRIPTION "PRUSS ICSSG Ethernet driver" +#define DEFAULT_VID 1 +#define DEFAULT_PORT_MASK 1 +#define DEFAULT_UNTAG_MASK 1 + /* CTRLMMR_ICSSG_RGMII_CTRL register bits */ #define ICSSG_CTRL_RGMII_ID_MODE BIT(24) @@ -112,6 +118,19 @@ static irqreturn_t prueth_tx_ts_irq(int irq, void *dev_id) return IRQ_HANDLED; } +static struct icssg_firmwares icssg_switch_firmwares[] = { + { + .pru = "ti-pruss/am65x-sr2-pru0-prusw-fw.elf", + .rtu = "ti-pruss/am65x-sr2-rtu0-prusw-fw.elf", + .txpru = "ti-pruss/am65x-sr2-txpru0-prusw-fw.elf", + }, + { + .pru = "ti-pruss/am65x-sr2-pru1-prusw-fw.elf", + .rtu = "ti-pruss/am65x-sr2-rtu1-prusw-fw.elf", + .txpru = "ti-pruss/am65x-sr2-txpru1-prusw-fw.elf", + } +}; + static struct icssg_firmwares icssg_emac_firmwares[] = { { .pru = "ti-pruss/am65x-sr2-pru0-prueth-fw.elf", @@ -131,7 +150,10 @@ static int prueth_emac_start(struct prueth *prueth, struct prueth_emac *emac) struct device *dev = prueth->dev; int slice, ret; - firmwares = icssg_emac_firmwares; + if (prueth->is_switch_mode) + firmwares = icssg_switch_firmwares; + else + firmwares = icssg_emac_firmwares; slice = prueth_emac_slice(emac); if (slice < 0) { @@ -417,6 +439,37 @@ const struct icss_iep_clockops prueth_iep_clockops = { .perout_enable = prueth_perout_enable, }; +static int icssg_prueth_add_mcast(struct net_device *ndev, const u8 *addr) +{ + struct prueth_emac *emac = netdev_priv(ndev); + int port_mask = BIT(emac->port_id); + + port_mask |= icssg_fdb_lookup(emac, addr, 0); + icssg_fdb_add_del(emac, addr, 0, port_mask, true); + icssg_vtbl_modify(emac, 0, port_mask, port_mask, true); + + return 0; +} + +static int icssg_prueth_del_mcast(struct net_device *ndev, const u8 *addr) +{ + struct prueth_emac *emac = netdev_priv(ndev); + int port_mask = BIT(emac->port_id); + int other_port_mask; + + other_port_mask = port_mask ^ icssg_fdb_lookup(emac, addr, 0); + + icssg_fdb_add_del(emac, addr, 0, port_mask, false); + icssg_vtbl_modify(emac, 0, port_mask, port_mask, false); + + if (other_port_mask) { + icssg_fdb_add_del(emac, addr, 0, other_port_mask, true); + icssg_vtbl_modify(emac, 0, other_port_mask, other_port_mask, true); + } + + return 0; +} + /** * emac_ndo_open - EMAC device open * @ndev: network adapter device @@ -445,9 +498,8 @@ static int emac_ndo_open(struct net_device *ndev) ether_addr_copy(emac->mac_addr, ndev->dev_addr); icssg_class_set_mac_addr(prueth->miig_rt, slice, emac->mac_addr); - icssg_ft1_set_mac_addr(prueth->miig_rt, slice, emac->mac_addr); - icssg_class_default(prueth->miig_rt, slice, 0, false); + icssg_ft1_set_mac_addr(prueth->miig_rt, slice, emac->mac_addr); /* Notify the stack of the actual queue counts. */ ret = netif_set_real_num_tx_queues(ndev, num_data_chn); @@ -578,6 +630,8 @@ static int emac_ndo_stop(struct net_device *ndev) icssg_class_disable(prueth->miig_rt, prueth_emac_slice(emac)); + __dev_mc_unsync(ndev, icssg_prueth_del_mcast); + atomic_set(&emac->tdown_cnt, emac->tx_ch_num); /* ensure new tdown_cnt value is visible */ smp_mb__after_atomic(); @@ -654,10 +708,7 @@ static void emac_ndo_set_rx_mode_work(struct work_struct *work) return; } - if (!netdev_mc_empty(ndev)) { - emac_set_port_state(emac, ICSSG_EMAC_PORT_MC_FLOODING_ENABLE); - return; - } + __dev_mc_sync(ndev, icssg_prueth_add_mcast, icssg_prueth_del_mcast); } /** @@ -833,6 +884,214 @@ free_ndev: return ret; } +bool prueth_dev_check(const struct net_device *ndev) +{ + if (ndev->netdev_ops == &emac_netdev_ops && netif_running(ndev)) { + struct prueth_emac *emac = netdev_priv(ndev); + + return emac->prueth->is_switch_mode; + } + + return false; +} + +static void prueth_offload_fwd_mark_update(struct prueth *prueth) +{ + int set_val = 0; + int i; + + if (prueth->br_members == (BIT(PRUETH_PORT_MII0) | BIT(PRUETH_PORT_MII1))) + set_val = 1; + + dev_dbg(prueth->dev, "set offload_fwd_mark %d\n", set_val); + + for (i = PRUETH_MAC0; i < PRUETH_NUM_MACS; i++) { + struct prueth_emac *emac = prueth->emac[i]; + + if (!emac || !emac->ndev) + continue; + + emac->offload_fwd_mark = set_val; + } +} + +static void prueth_emac_restart(struct prueth *prueth) +{ + struct prueth_emac *emac0 = prueth->emac[PRUETH_MAC0]; + struct prueth_emac *emac1 = prueth->emac[PRUETH_MAC1]; + + /* Detach the net_device for both PRUeth ports*/ + if (netif_running(emac0->ndev)) + netif_device_detach(emac0->ndev); + if (netif_running(emac1->ndev)) + netif_device_detach(emac1->ndev); + + /* Disable both PRUeth ports */ + emac_set_port_state(emac0, ICSSG_EMAC_PORT_DISABLE); + emac_set_port_state(emac1, ICSSG_EMAC_PORT_DISABLE); + + /* Stop both pru cores for both PRUeth ports*/ + prueth_emac_stop(emac0); + prueth->emacs_initialized--; + prueth_emac_stop(emac1); + prueth->emacs_initialized--; + + /* Start both pru cores for both PRUeth ports */ + prueth_emac_start(prueth, emac0); + prueth->emacs_initialized++; + prueth_emac_start(prueth, emac1); + prueth->emacs_initialized++; + + /* Enable forwarding for both PRUeth ports */ + emac_set_port_state(emac0, ICSSG_EMAC_PORT_FORWARD); + emac_set_port_state(emac1, ICSSG_EMAC_PORT_FORWARD); + + /* Attache net_device for both PRUeth ports */ + netif_device_attach(emac0->ndev); + netif_device_attach(emac1->ndev); +} + +static void icssg_enable_switch_mode(struct prueth *prueth) +{ + struct prueth_emac *emac; + int mac; + + prueth_emac_restart(prueth); + + for (mac = PRUETH_MAC0; mac < PRUETH_NUM_MACS; mac++) { + emac = prueth->emac[mac]; + if (netif_running(emac->ndev)) { + icssg_fdb_add_del(emac, eth_stp_addr, prueth->default_vlan, + ICSSG_FDB_ENTRY_P0_MEMBERSHIP | + ICSSG_FDB_ENTRY_P1_MEMBERSHIP | + ICSSG_FDB_ENTRY_P2_MEMBERSHIP | + ICSSG_FDB_ENTRY_BLOCK, + true); + icssg_vtbl_modify(emac, emac->port_vlan | DEFAULT_VID, + BIT(emac->port_id) | DEFAULT_PORT_MASK, + BIT(emac->port_id) | DEFAULT_UNTAG_MASK, + true); + icssg_set_pvid(prueth, emac->port_vlan, emac->port_id); + emac_set_port_state(emac, ICSSG_EMAC_PORT_VLAN_AWARE_ENABLE); + } + } +} + +static int prueth_netdevice_port_link(struct net_device *ndev, + struct net_device *br_ndev, + struct netlink_ext_ack *extack) +{ + struct prueth_emac *emac = netdev_priv(ndev); + struct prueth *prueth = emac->prueth; + int err; + + if (!prueth->br_members) { + prueth->hw_bridge_dev = br_ndev; + } else { + /* This is adding the port to a second bridge, this is + * unsupported + */ + if (prueth->hw_bridge_dev != br_ndev) + return -EOPNOTSUPP; + } + + err = switchdev_bridge_port_offload(br_ndev, ndev, emac, + &prueth->prueth_switchdev_nb, + &prueth->prueth_switchdev_bl_nb, + false, extack); + if (err) + return err; + + prueth->br_members |= BIT(emac->port_id); + + if (!prueth->is_switch_mode) { + if (prueth->br_members & BIT(PRUETH_PORT_MII0) && + prueth->br_members & BIT(PRUETH_PORT_MII1)) { + prueth->is_switch_mode = true; + prueth->default_vlan = 1; + emac->port_vlan = prueth->default_vlan; + icssg_enable_switch_mode(prueth); + } + } + + prueth_offload_fwd_mark_update(prueth); + + return NOTIFY_DONE; +} + +static void prueth_netdevice_port_unlink(struct net_device *ndev) +{ + struct prueth_emac *emac = netdev_priv(ndev); + struct prueth *prueth = emac->prueth; + + prueth->br_members &= ~BIT(emac->port_id); + + if (prueth->is_switch_mode) { + prueth->is_switch_mode = false; + emac->port_vlan = 0; + prueth_emac_restart(prueth); + } + + prueth_offload_fwd_mark_update(prueth); + + if (!prueth->br_members) + prueth->hw_bridge_dev = NULL; +} + +/* netdev notifier */ +static int prueth_netdevice_event(struct notifier_block *unused, + unsigned long event, void *ptr) +{ + struct netlink_ext_ack *extack = netdev_notifier_info_to_extack(ptr); + struct net_device *ndev = netdev_notifier_info_to_dev(ptr); + struct netdev_notifier_changeupper_info *info; + int ret = NOTIFY_DONE; + + if (ndev->netdev_ops != &emac_netdev_ops) + return NOTIFY_DONE; + + switch (event) { + case NETDEV_CHANGEUPPER: + info = ptr; + + if (netif_is_bridge_master(info->upper_dev)) { + if (info->linking) + ret = prueth_netdevice_port_link(ndev, info->upper_dev, extack); + else + prueth_netdevice_port_unlink(ndev); + } + break; + default: + return NOTIFY_DONE; + } + + return notifier_from_errno(ret); +} + +static int prueth_register_notifiers(struct prueth *prueth) +{ + int ret = 0; + + prueth->prueth_netdevice_nb.notifier_call = &prueth_netdevice_event; + ret = register_netdevice_notifier(&prueth->prueth_netdevice_nb); + if (ret) { + dev_err(prueth->dev, "can't register netdevice notifier\n"); + return ret; + } + + ret = prueth_switchdev_register_notifiers(prueth); + if (ret) + unregister_netdevice_notifier(&prueth->prueth_netdevice_nb); + + return ret; +} + +static void prueth_unregister_notifiers(struct prueth *prueth) +{ + prueth_switchdev_unregister_notifiers(prueth); + unregister_netdevice_notifier(&prueth->prueth_netdevice_nb); +} + static int prueth_probe(struct platform_device *pdev) { struct device_node *eth_node, *eth_ports_node; @@ -960,6 +1219,9 @@ static int prueth_probe(struct platform_device *pdev) } msmc_ram_size = MSMC_RAM_SIZE; + prueth->is_switchmode_supported = prueth->pdata.switch_mode; + if (prueth->is_switchmode_supported) + msmc_ram_size = MSMC_RAM_SIZE_SWITCH_MODE; /* NOTE: FW bug needs buffer base to be 64KB aligned */ prueth->msmcram.va = @@ -1065,6 +1327,14 @@ static int prueth_probe(struct platform_device *pdev) phy_attached_info(prueth->emac[PRUETH_MAC1]->ndev->phydev); } + if (prueth->is_switchmode_supported) { + ret = prueth_register_notifiers(prueth); + if (ret) + goto netdev_unregister; + + sprintf(prueth->switch_id, "%s", dev_name(dev)); + } + dev_info(dev, "TI PRU ethernet driver initialized: %s EMAC mode\n", (!eth0_node || !eth1_node) ? "single" : "dual"); @@ -1134,6 +1404,8 @@ static void prueth_remove(struct platform_device *pdev) struct device_node *eth_node; int i; + prueth_unregister_notifiers(prueth); + for (i = 0; i < PRUETH_NUM_MACS; i++) { if (!prueth->registered_netdevs[i]) continue; @@ -1175,10 +1447,12 @@ static void prueth_remove(struct platform_device *pdev) static const struct prueth_pdata am654_icssg_pdata = { .fdqring_mode = K3_RINGACC_RING_MODE_MESSAGE, .quirk_10m_link_issue = 1, + .switch_mode = 1, }; static const struct prueth_pdata am64x_icssg_pdata = { .fdqring_mode = K3_RINGACC_RING_MODE_RING, + .switch_mode = 1, }; static const struct of_device_id prueth_dt_match[] = { diff --git a/drivers/net/ethernet/ti/icssg/icssg_prueth.h b/drivers/net/ethernet/ti/icssg/icssg_prueth.h index a78c5eb75fb8614e06d43523950a7dcf8def0033..5eeeccb73665944b2501275be36ef3110d44c04f 100644 --- a/drivers/net/ethernet/ti/icssg/icssg_prueth.h +++ b/drivers/net/ethernet/ti/icssg/icssg_prueth.h @@ -186,6 +186,9 @@ struct prueth_emac { struct pruss_mem_region dram; + bool offload_fwd_mark; + int port_vlan; + struct delayed_work stats_work; u64 stats[ICSSG_NUM_STATS]; @@ -198,10 +201,12 @@ struct prueth_emac { * struct prueth_pdata - PRUeth platform data * @fdqring_mode: Free desc queue mode * @quirk_10m_link_issue: 10M link detect errata + * @switch_mode: switch firmware support */ struct prueth_pdata { enum k3_ring_mode fdqring_mode; u32 quirk_10m_link_issue:1; + u32 switch_mode:1; }; struct icssg_firmwares { @@ -232,6 +237,16 @@ struct icssg_firmwares { * @emacs_initialized: num of EMACs/ext ports that are up/running * @iep0: pointer to IEP0 device * @iep1: pointer to IEP1 device + * @vlan_tbl: VLAN-FID table pointer + * @hw_bridge_dev: pointer to HW bridge net device + * @br_members: bitmask of bridge member ports + * @prueth_netdevice_nb: netdevice notifier block + * @prueth_switchdev_nb: switchdev notifier block + * @prueth_switchdev_bl_nb: switchdev blocking notifier block + * @is_switch_mode: flag to indicate if device is in Switch mode + * @is_switchmode_supported: indicates platform support for switch mode + * @switch_id: ID for mapping switch ports to bridge + * @default_vlan: Default VLAN for host */ struct prueth { struct device *dev; @@ -256,6 +271,17 @@ struct prueth { int emacs_initialized; struct icss_iep *iep0; struct icss_iep *iep1; + struct prueth_vlan_tbl *vlan_tbl; + + struct net_device *hw_bridge_dev; + u8 br_members; + struct notifier_block prueth_netdevice_nb; + struct notifier_block prueth_switchdev_nb; + struct notifier_block prueth_switchdev_bl_nb; + bool is_switch_mode; + bool is_switchmode_supported; + unsigned char switch_id[MAX_PHYS_ITEM_ID_LEN]; + int default_vlan; }; struct emac_tx_ts_response { @@ -313,6 +339,16 @@ int icssg_queue_pop(struct prueth *prueth, u8 queue); void icssg_queue_push(struct prueth *prueth, int queue, u16 addr); u32 icssg_queue_level(struct prueth *prueth, int queue); +int icssg_send_fdb_msg(struct prueth_emac *emac, struct mgmt_cmd *cmd, + struct mgmt_cmd_rsp *rsp); +int icssg_fdb_add_del(struct prueth_emac *emac, const unsigned char *addr, + u8 vid, u8 fid_c2, bool add); +int icssg_fdb_lookup(struct prueth_emac *emac, const unsigned char *addr, + u8 vid); +void icssg_vtbl_modify(struct prueth_emac *emac, u8 vid, u8 port_mask, + u8 untag_mask, bool add); +u16 icssg_get_pvid(struct prueth_emac *emac); +void icssg_set_pvid(struct prueth *prueth, u8 vid, u8 port); #define prueth_napi_to_tx_chn(pnapi) \ container_of(pnapi, struct prueth_tx_chn, napi_tx) diff --git a/drivers/net/ethernet/ti/icssg/icssg_switchdev.c b/drivers/net/ethernet/ti/icssg/icssg_switchdev.c new file mode 100644 index 0000000000000000000000000000000000000000..fceb8bb7d34ed517678d7cee826cfd30600f128c --- /dev/null +++ b/drivers/net/ethernet/ti/icssg/icssg_switchdev.c @@ -0,0 +1,477 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Texas Instruments K3 ICSSG Ethernet Switchdev Driver + * + * Copyright (C) 2021 Texas Instruments Incorporated - https://www.ti.com/ + * + */ + +#include +#include +#include +#include +#include + +#include "icssg_prueth.h" +#include "icssg_switchdev.h" +#include "icssg_mii_rt.h" + +struct prueth_switchdev_event_work { + struct work_struct work; + struct switchdev_notifier_fdb_info fdb_info; + struct prueth_emac *emac; + unsigned long event; +}; + +static int prueth_switchdev_stp_state_set(struct prueth_emac *emac, + u8 state) +{ + enum icssg_port_state_cmd emac_state; + int ret = 0; + + switch (state) { + case BR_STATE_FORWARDING: + emac_state = ICSSG_EMAC_PORT_FORWARD; + break; + case BR_STATE_DISABLED: + emac_state = ICSSG_EMAC_PORT_DISABLE; + break; + case BR_STATE_LISTENING: + case BR_STATE_BLOCKING: + emac_state = ICSSG_EMAC_PORT_BLOCK; + break; + default: + return -EOPNOTSUPP; + } + + emac_set_port_state(emac, emac_state); + netdev_dbg(emac->ndev, "STP state: %u\n", emac_state); + + return ret; +} + +static int prueth_switchdev_attr_br_flags_set(struct prueth_emac *emac, + struct net_device *orig_dev, + struct switchdev_brport_flags brport_flags) +{ + enum icssg_port_state_cmd emac_state; + + if (brport_flags.mask & BR_MCAST_FLOOD) + emac_state = ICSSG_EMAC_PORT_MC_FLOODING_ENABLE; + else + emac_state = ICSSG_EMAC_PORT_MC_FLOODING_DISABLE; + + netdev_dbg(emac->ndev, "BR_MCAST_FLOOD: %d port %u\n", + emac_state, emac->port_id); + + emac_set_port_state(emac, emac_state); + + return 0; +} + +static int prueth_switchdev_attr_br_flags_pre_set(struct net_device *netdev, + struct switchdev_brport_flags brport_flags) +{ + if (brport_flags.mask & ~(BR_LEARNING | BR_MCAST_FLOOD)) + return -EINVAL; + + return 0; +} + +static int prueth_switchdev_attr_set(struct net_device *ndev, const void *ctx, + const struct switchdev_attr *attr, + struct netlink_ext_ack *extack) +{ + struct prueth_emac *emac = netdev_priv(ndev); + int ret; + + netdev_dbg(ndev, "attr: id %u port: %u\n", attr->id, emac->port_id); + + switch (attr->id) { + case SWITCHDEV_ATTR_ID_PORT_PRE_BRIDGE_FLAGS: + ret = prueth_switchdev_attr_br_flags_pre_set(ndev, + attr->u.brport_flags); + break; + case SWITCHDEV_ATTR_ID_PORT_STP_STATE: + ret = prueth_switchdev_stp_state_set(emac, + attr->u.stp_state); + netdev_dbg(ndev, "stp state: %u\n", attr->u.stp_state); + break; + case SWITCHDEV_ATTR_ID_PORT_BRIDGE_FLAGS: + ret = prueth_switchdev_attr_br_flags_set(emac, attr->orig_dev, + attr->u.brport_flags); + break; + default: + ret = -EOPNOTSUPP; + break; + } + + return ret; +} + +static void prueth_switchdev_fdb_offload_notify(struct net_device *ndev, + struct switchdev_notifier_fdb_info *rcv) +{ + struct switchdev_notifier_fdb_info info; + + memset(&info, 0, sizeof(info)); + info.addr = rcv->addr; + info.vid = rcv->vid; + info.offloaded = true; + call_switchdev_notifiers(SWITCHDEV_FDB_OFFLOADED, + ndev, &info.info, NULL); +} + +static void prueth_switchdev_event_work(struct work_struct *work) +{ + struct prueth_switchdev_event_work *switchdev_work = + container_of(work, struct prueth_switchdev_event_work, work); + struct prueth_emac *emac = switchdev_work->emac; + struct switchdev_notifier_fdb_info *fdb; + int port_id = emac->port_id; + int ret; + + rtnl_lock(); + switch (switchdev_work->event) { + case SWITCHDEV_FDB_ADD_TO_DEVICE: + fdb = &switchdev_work->fdb_info; + + netdev_dbg(emac->ndev, "prueth_fdb_add: MACID = %pM vid = %u flags = %u %u -- port %d\n", + fdb->addr, fdb->vid, fdb->added_by_user, + fdb->offloaded, port_id); + + if (!fdb->added_by_user) + break; + if (!ether_addr_equal(emac->mac_addr, fdb->addr)) + break; + + ret = icssg_fdb_add_del(emac, fdb->addr, fdb->vid, + BIT(port_id), true); + if (!ret) + prueth_switchdev_fdb_offload_notify(emac->ndev, fdb); + break; + case SWITCHDEV_FDB_DEL_TO_DEVICE: + fdb = &switchdev_work->fdb_info; + + netdev_dbg(emac->ndev, "prueth_fdb_del: MACID = %pM vid = %u flags = %u %u -- port %d\n", + fdb->addr, fdb->vid, fdb->added_by_user, + fdb->offloaded, port_id); + + if (!fdb->added_by_user) + break; + if (!ether_addr_equal(emac->mac_addr, fdb->addr)) + break; + icssg_fdb_add_del(emac, fdb->addr, fdb->vid, + BIT(port_id), false); + break; + default: + break; + } + rtnl_unlock(); + + kfree(switchdev_work->fdb_info.addr); + kfree(switchdev_work); + dev_put(emac->ndev); +} + +static int prueth_switchdev_event(struct notifier_block *unused, + unsigned long event, void *ptr) +{ + struct net_device *ndev = switchdev_notifier_info_to_dev(ptr); + struct prueth_switchdev_event_work *switchdev_work; + struct switchdev_notifier_fdb_info *fdb_info = ptr; + struct prueth_emac *emac = netdev_priv(ndev); + int err; + + if (!prueth_dev_check(ndev)) + return NOTIFY_DONE; + + if (event == SWITCHDEV_PORT_ATTR_SET) { + err = switchdev_handle_port_attr_set(ndev, ptr, + prueth_dev_check, + prueth_switchdev_attr_set); + return notifier_from_errno(err); + } + + switchdev_work = kzalloc(sizeof(*switchdev_work), GFP_ATOMIC); + if (WARN_ON(!switchdev_work)) + return NOTIFY_BAD; + + INIT_WORK(&switchdev_work->work, prueth_switchdev_event_work); + switchdev_work->emac = emac; + switchdev_work->event = event; + + switch (event) { + case SWITCHDEV_FDB_ADD_TO_DEVICE: + case SWITCHDEV_FDB_DEL_TO_DEVICE: + memcpy(&switchdev_work->fdb_info, ptr, + sizeof(switchdev_work->fdb_info)); + switchdev_work->fdb_info.addr = kzalloc(ETH_ALEN, GFP_ATOMIC); + if (!switchdev_work->fdb_info.addr) + goto err_addr_alloc; + ether_addr_copy((u8 *)switchdev_work->fdb_info.addr, + fdb_info->addr); + dev_hold(ndev); + break; + default: + kfree(switchdev_work); + return NOTIFY_DONE; + } + + queue_work(system_long_wq, &switchdev_work->work); + + return NOTIFY_DONE; + +err_addr_alloc: + kfree(switchdev_work); + return NOTIFY_BAD; +} + +static int prueth_switchdev_vlan_add(struct prueth_emac *emac, bool untag, bool pvid, + u8 vid, struct net_device *orig_dev) +{ + bool cpu_port = netif_is_bridge_master(orig_dev); + int untag_mask = 0; + int port_mask; + int ret = 0; + + if (cpu_port) + port_mask = BIT(PRUETH_PORT_HOST); + else + port_mask = BIT(emac->port_id); + + if (untag) + untag_mask = port_mask; + + icssg_vtbl_modify(emac, vid, port_mask, untag_mask, true); + + netdev_dbg(emac->ndev, "VID add vid:%u port_mask:%X untag_mask %X PVID %d\n", + vid, port_mask, untag_mask, pvid); + + if (!pvid) + return ret; + + icssg_set_pvid(emac->prueth, vid, emac->port_id); + + return ret; +} + +static int prueth_switchdev_vlan_del(struct prueth_emac *emac, u16 vid, + struct net_device *orig_dev) +{ + bool cpu_port = netif_is_bridge_master(orig_dev); + int port_mask; + int ret = 0; + + if (cpu_port) + port_mask = BIT(PRUETH_PORT_HOST); + else + port_mask = BIT(emac->port_id); + + icssg_vtbl_modify(emac, vid, port_mask, 0, false); + + if (cpu_port) + icssg_fdb_add_del(emac, emac->mac_addr, vid, + BIT(PRUETH_PORT_HOST), false); + + if (vid == icssg_get_pvid(emac)) + icssg_set_pvid(emac->prueth, 0, emac->port_id); + + netdev_dbg(emac->ndev, "VID del vid:%u port_mask:%X\n", + vid, port_mask); + + return ret; +} + +static int prueth_switchdev_vlans_add(struct prueth_emac *emac, + const struct switchdev_obj_port_vlan *vlan) +{ + bool untag = vlan->flags & BRIDGE_VLAN_INFO_UNTAGGED; + struct net_device *orig_dev = vlan->obj.orig_dev; + bool cpu_port = netif_is_bridge_master(orig_dev); + bool pvid = vlan->flags & BRIDGE_VLAN_INFO_PVID; + + netdev_dbg(emac->ndev, "VID add vid:%u flags:%X\n", + vlan->vid, vlan->flags); + + if (cpu_port && !(vlan->flags & BRIDGE_VLAN_INFO_BRENTRY)) + return 0; + + if (vlan->vid > 0xff) + return 0; + + return prueth_switchdev_vlan_add(emac, untag, pvid, vlan->vid, + orig_dev); +} + +static int prueth_switchdev_vlans_del(struct prueth_emac *emac, + const struct switchdev_obj_port_vlan *vlan) +{ + if (vlan->vid > 0xff) + return 0; + + return prueth_switchdev_vlan_del(emac, vlan->vid, + vlan->obj.orig_dev); +} + +static int prueth_switchdev_mdb_add(struct prueth_emac *emac, + struct switchdev_obj_port_mdb *mdb) +{ + struct net_device *orig_dev = mdb->obj.orig_dev; + u8 port_mask, fid_c2; + bool cpu_port; + int err; + + cpu_port = netif_is_bridge_master(orig_dev); + + if (cpu_port) + port_mask = BIT(PRUETH_PORT_HOST); + else + port_mask = BIT(emac->port_id); + + fid_c2 = icssg_fdb_lookup(emac, mdb->addr, mdb->vid); + + err = icssg_fdb_add_del(emac, mdb->addr, mdb->vid, fid_c2 | port_mask, true); + netdev_dbg(emac->ndev, "MDB add vid %u:%pM ports: %X\n", + mdb->vid, mdb->addr, port_mask); + + return err; +} + +static int prueth_switchdev_mdb_del(struct prueth_emac *emac, + struct switchdev_obj_port_mdb *mdb) +{ + struct net_device *orig_dev = mdb->obj.orig_dev; + int del_mask, ret, fid_c2; + bool cpu_port; + + cpu_port = netif_is_bridge_master(orig_dev); + + if (cpu_port) + del_mask = BIT(PRUETH_PORT_HOST); + else + del_mask = BIT(emac->port_id); + + fid_c2 = icssg_fdb_lookup(emac, mdb->addr, mdb->vid); + + if (fid_c2 & ~del_mask) + ret = icssg_fdb_add_del(emac, mdb->addr, mdb->vid, fid_c2 & ~del_mask, true); + else + ret = icssg_fdb_add_del(emac, mdb->addr, mdb->vid, 0, false); + + netdev_dbg(emac->ndev, "MDB del vid %u:%pM ports: %X\n", + mdb->vid, mdb->addr, del_mask); + + return ret; +} + +static int prueth_switchdev_obj_add(struct net_device *ndev, const void *ctx, + const struct switchdev_obj *obj, + struct netlink_ext_ack *extack) +{ + struct switchdev_obj_port_vlan *vlan = SWITCHDEV_OBJ_PORT_VLAN(obj); + struct switchdev_obj_port_mdb *mdb = SWITCHDEV_OBJ_PORT_MDB(obj); + struct prueth_emac *emac = netdev_priv(ndev); + int err = 0; + + netdev_dbg(ndev, "obj_add: id %u port: %u\n", obj->id, emac->port_id); + + switch (obj->id) { + case SWITCHDEV_OBJ_ID_PORT_VLAN: + err = prueth_switchdev_vlans_add(emac, vlan); + break; + case SWITCHDEV_OBJ_ID_PORT_MDB: + case SWITCHDEV_OBJ_ID_HOST_MDB: + err = prueth_switchdev_mdb_add(emac, mdb); + break; + default: + err = -EOPNOTSUPP; + break; + } + + return err; +} + +static int prueth_switchdev_obj_del(struct net_device *ndev, const void *ctx, + const struct switchdev_obj *obj) +{ + struct switchdev_obj_port_vlan *vlan = SWITCHDEV_OBJ_PORT_VLAN(obj); + struct switchdev_obj_port_mdb *mdb = SWITCHDEV_OBJ_PORT_MDB(obj); + struct prueth_emac *emac = netdev_priv(ndev); + int err = 0; + + netdev_dbg(ndev, "obj_del: id %u port: %u\n", obj->id, emac->port_id); + + switch (obj->id) { + case SWITCHDEV_OBJ_ID_PORT_VLAN: + err = prueth_switchdev_vlans_del(emac, vlan); + break; + case SWITCHDEV_OBJ_ID_PORT_MDB: + case SWITCHDEV_OBJ_ID_HOST_MDB: + err = prueth_switchdev_mdb_del(emac, mdb); + break; + default: + err = -EOPNOTSUPP; + break; + } + + return err; +} + +static int prueth_switchdev_blocking_event(struct notifier_block *unused, + unsigned long event, void *ptr) +{ + struct net_device *dev = switchdev_notifier_info_to_dev(ptr); + int err; + + switch (event) { + case SWITCHDEV_PORT_OBJ_ADD: + err = switchdev_handle_port_obj_add(dev, ptr, + prueth_dev_check, + prueth_switchdev_obj_add); + return notifier_from_errno(err); + case SWITCHDEV_PORT_OBJ_DEL: + err = switchdev_handle_port_obj_del(dev, ptr, + prueth_dev_check, + prueth_switchdev_obj_del); + return notifier_from_errno(err); + case SWITCHDEV_PORT_ATTR_SET: + err = switchdev_handle_port_attr_set(dev, ptr, + prueth_dev_check, + prueth_switchdev_attr_set); + return notifier_from_errno(err); + default: + break; + } + + return NOTIFY_DONE; +} + +int prueth_switchdev_register_notifiers(struct prueth *prueth) +{ + int ret = 0; + + prueth->prueth_switchdev_nb.notifier_call = &prueth_switchdev_event; + ret = register_switchdev_notifier(&prueth->prueth_switchdev_nb); + if (ret) { + dev_err(prueth->dev, "register switchdev notifier fail ret:%d\n", + ret); + return ret; + } + + prueth->prueth_switchdev_bl_nb.notifier_call = &prueth_switchdev_blocking_event; + ret = register_switchdev_blocking_notifier(&prueth->prueth_switchdev_bl_nb); + if (ret) { + dev_err(prueth->dev, "register switchdev blocking notifier ret:%d\n", + ret); + unregister_switchdev_notifier(&prueth->prueth_switchdev_nb); + } + + return ret; +} + +void prueth_switchdev_unregister_notifiers(struct prueth *prueth) +{ + unregister_switchdev_blocking_notifier(&prueth->prueth_switchdev_bl_nb); + unregister_switchdev_notifier(&prueth->prueth_switchdev_nb); +} diff --git a/drivers/net/ethernet/ti/icssg/icssg_switchdev.h b/drivers/net/ethernet/ti/icssg/icssg_switchdev.h new file mode 100644 index 0000000000000000000000000000000000000000..0e64e7760a00a203d2261766ced48a4e77894b9d --- /dev/null +++ b/drivers/net/ethernet/ti/icssg/icssg_switchdev.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2021 Texas Instruments Incorporated - https://www.ti.com/ + */ +#ifndef __NET_TI_ICSSG_SWITCHDEV_H +#define __NET_TI_ICSSG_SWITCHDEV_H + +#include "icssg_prueth.h" + +int prueth_switchdev_register_notifiers(struct prueth *prueth); +void prueth_switchdev_unregister_notifiers(struct prueth *prueth); +bool prueth_dev_check(const struct net_device *ndev); + +#endif /* __NET_TI_ICSSG_SWITCHDEV_H */ diff --git a/drivers/net/fjes/fjes_trace.h b/drivers/net/fjes/fjes_trace.h index 166ef015262beb29b5af12a98fe161a9b4e71e69..37c6071cb333e819914d93b10f752a7aa766df39 100644 --- a/drivers/net/fjes/fjes_trace.h +++ b/drivers/net/fjes/fjes_trace.h @@ -358,7 +358,7 @@ TRACE_EVENT(fjes_stop_req_irq_post, #undef TRACE_INCLUDE_PATH #undef TRACE_INCLUDE_FILE -#define TRACE_INCLUDE_PATH ../../../drivers/net/fjes +#define TRACE_INCLUDE_PATH ../../drivers/net/fjes #define TRACE_INCLUDE_FILE fjes_trace /* This part must be outside protection */ diff --git a/drivers/net/ipvlan/ipvlan_core.c b/drivers/net/ipvlan/ipvlan_core.c index 2d5b021b4ea6053eeb055a76fa4c7d9380cd2a53..fef4eff7753a7acb1e11d9712abd669de7740df6 100644 --- a/drivers/net/ipvlan/ipvlan_core.c +++ b/drivers/net/ipvlan/ipvlan_core.c @@ -439,7 +439,7 @@ static noinline_for_stack int ipvlan_process_v4_outbound(struct sk_buff *skb) memset(IPCB(skb), 0, sizeof(*IPCB(skb))); - err = ip_local_out(net, skb->sk, skb); + err = ip_local_out(net, NULL, skb); if (unlikely(net_xmit_eval(err))) DEV_STATS_INC(dev, tx_errors); else @@ -494,7 +494,7 @@ static int ipvlan_process_v6_outbound(struct sk_buff *skb) memset(IP6CB(skb), 0, sizeof(*IP6CB(skb))); - err = ip6_local_out(dev_net(dev), skb->sk, skb); + err = ip6_local_out(dev_net(dev), NULL, skb); if (unlikely(net_xmit_eval(err))) DEV_STATS_INC(dev, tx_errors); else diff --git a/drivers/net/netconsole.c b/drivers/net/netconsole.c index d7070dd4fe7367cb3aec7aa88a83d6e93e03537a..ab8a0623b1a17dccdd240e799700cb7fc1a10d4f 100644 --- a/drivers/net/netconsole.c +++ b/drivers/net/netconsole.c @@ -1262,6 +1262,8 @@ static int __init init_netconsole(void) while ((target_config = strsep(&input, ";"))) { nt = alloc_param_target(target_config, count); if (IS_ERR(nt)) { + if (IS_ENABLED(CONFIG_NETCONSOLE_DYNAMIC)) + continue; err = PTR_ERR(nt); goto fail; } diff --git a/drivers/net/netkit.c b/drivers/net/netkit.c index a4d2e76a8d587cc6ce7ad7f98e382a1c81f76e67..16789cd446e9e47fd738f12a9cde4e1eeef55fcf 100644 --- a/drivers/net/netkit.c +++ b/drivers/net/netkit.c @@ -55,6 +55,7 @@ static void netkit_prep_forward(struct sk_buff *skb, bool xnet) skb_scrub_packet(skb, xnet); skb->priority = 0; nf_skip_egress(skb, true); + skb_reset_mac_header(skb); } static struct netkit *netkit_priv(const struct net_device *dev) @@ -78,6 +79,7 @@ static netdev_tx_t netkit_xmit(struct sk_buff *skb, struct net_device *dev) skb_orphan_frags(skb, GFP_ATOMIC))) goto drop; netkit_prep_forward(skb, !net_eq(dev_net(dev), dev_net(peer))); + eth_skb_pkt_type(skb, peer); skb->dev = peer; entry = rcu_dereference(nk->active); if (entry) @@ -85,7 +87,7 @@ static netdev_tx_t netkit_xmit(struct sk_buff *skb, struct net_device *dev) switch (ret) { case NETKIT_NEXT: case NETKIT_PASS: - skb->protocol = eth_type_trans(skb, skb->dev); + eth_skb_pull_mac(skb); skb_postpull_rcsum(skb, eth_hdr(skb), ETH_HLEN); if (likely(__netif_rx(skb) == NET_RX_SUCCESS)) { dev_sw_netstats_tx_add(dev, 1, len); @@ -155,6 +157,16 @@ static void netkit_set_multicast(struct net_device *dev) /* Nothing to do, we receive whatever gets pushed to us! */ } +static int netkit_set_macaddr(struct net_device *dev, void *sa) +{ + struct netkit *nk = netkit_priv(dev); + + if (nk->mode != NETKIT_L2) + return -EOPNOTSUPP; + + return eth_mac_addr(dev, sa); +} + static void netkit_set_headroom(struct net_device *dev, int headroom) { struct netkit *nk = netkit_priv(dev), *nk2; @@ -198,6 +210,7 @@ static const struct net_device_ops netkit_netdev_ops = { .ndo_start_xmit = netkit_xmit, .ndo_set_rx_mode = netkit_set_multicast, .ndo_set_rx_headroom = netkit_set_headroom, + .ndo_set_mac_address = netkit_set_macaddr, .ndo_get_iflink = netkit_get_iflink, .ndo_get_peer_dev = netkit_peer_dev, .ndo_get_stats64 = netkit_get_stats, @@ -300,9 +313,11 @@ static int netkit_validate(struct nlattr *tb[], struct nlattr *data[], if (!attr) return 0; - NL_SET_ERR_MSG_ATTR(extack, attr, - "Setting Ethernet address is not supported"); - return -EOPNOTSUPP; + if (nla_len(attr) != ETH_ALEN) + return -EINVAL; + if (!is_valid_ether_addr(nla_data(attr))) + return -EADDRNOTAVAIL; + return 0; } static struct rtnl_link_ops netkit_link_ops; @@ -365,6 +380,9 @@ static int netkit_new_link(struct net *src_net, struct net_device *dev, strscpy(ifname, "nk%d", IFNAMSIZ); ifname_assign_type = NET_NAME_ENUM; } + if (mode != NETKIT_L2 && + (tb[IFLA_ADDRESS] || tbp[IFLA_ADDRESS])) + return -EOPNOTSUPP; net = rtnl_link_get_net(src_net, tbp); if (IS_ERR(net)) @@ -379,7 +397,7 @@ static int netkit_new_link(struct net *src_net, struct net_device *dev, netif_inherit_tso_max(peer, dev); - if (mode == NETKIT_L2) + if (mode == NETKIT_L2 && !(ifmp && tbp[IFLA_ADDRESS])) eth_hw_addr_random(peer); if (ifmp && dev->ifindex) peer->ifindex = ifmp->ifi_index; @@ -402,7 +420,7 @@ static int netkit_new_link(struct net *src_net, struct net_device *dev, if (err < 0) goto err_configure_peer; - if (mode == NETKIT_L2) + if (mode == NETKIT_L2 && !tb[IFLA_ADDRESS]) eth_hw_addr_random(dev); if (tb[IFLA_IFNAME]) nla_strscpy(dev->name, tb[IFLA_IFNAME], IFNAMSIZ); diff --git a/drivers/net/phy/aquantia/Makefile b/drivers/net/phy/aquantia/Makefile index aa77fb63c8ecf13a5cb9ab091eb1278c381eade2..c6c4d494ee2afabf0f2d562ecb4f1f467fdc096f 100644 --- a/drivers/net/phy/aquantia/Makefile +++ b/drivers/net/phy/aquantia/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -aquantia-objs += aquantia_main.o aquantia_firmware.o +aquantia-objs += aquantia_main.o aquantia_firmware.o aquantia_leds.o ifdef CONFIG_HWMON aquantia-objs += aquantia_hwmon.o endif diff --git a/drivers/net/phy/aquantia/aquantia.h b/drivers/net/phy/aquantia/aquantia.h index 1c19ae74ad2b4517bb24af7536c4ed16ce4bea22..c0e1fd9d71520be2bcbbf51c64bd6a4d2164ac16 100644 --- a/drivers/net/phy/aquantia/aquantia.h +++ b/drivers/net/phy/aquantia/aquantia.h @@ -63,6 +63,28 @@ #define VEND1_GLOBAL_CONTROL2_UP_RUN_STALL_OVD BIT(6) #define VEND1_GLOBAL_CONTROL2_UP_RUN_STALL BIT(0) +#define VEND1_GLOBAL_LED_PROV 0xc430 +#define AQR_LED_PROV(x) (VEND1_GLOBAL_LED_PROV + (x)) +#define VEND1_GLOBAL_LED_PROV_LINK2500 BIT(14) +#define VEND1_GLOBAL_LED_PROV_LINK5000 BIT(15) +#define VEND1_GLOBAL_LED_PROV_FORCE_ON BIT(8) +#define VEND1_GLOBAL_LED_PROV_LINK10000 BIT(7) +#define VEND1_GLOBAL_LED_PROV_LINK1000 BIT(6) +#define VEND1_GLOBAL_LED_PROV_LINK100 BIT(5) +#define VEND1_GLOBAL_LED_PROV_RX_ACT BIT(3) +#define VEND1_GLOBAL_LED_PROV_TX_ACT BIT(2) +#define VEND1_GLOBAL_LED_PROV_ACT_STRETCH GENMASK(0, 1) + +#define VEND1_GLOBAL_LED_PROV_LINK_MASK (VEND1_GLOBAL_LED_PROV_LINK100 | \ + VEND1_GLOBAL_LED_PROV_LINK1000 | \ + VEND1_GLOBAL_LED_PROV_LINK10000 | \ + VEND1_GLOBAL_LED_PROV_LINK5000 | \ + VEND1_GLOBAL_LED_PROV_LINK2500) + +#define VEND1_GLOBAL_LED_DRIVE 0xc438 +#define VEND1_GLOBAL_LED_DRIVE_VDD BIT(1) +#define AQR_LED_DRIVE(x) (VEND1_GLOBAL_LED_DRIVE + (x)) + #define VEND1_THERMAL_PROV_HIGH_TEMP_FAIL 0xc421 #define VEND1_THERMAL_PROV_LOW_TEMP_FAIL 0xc422 #define VEND1_THERMAL_PROV_HIGH_TEMP_WARN 0xc423 @@ -87,6 +109,18 @@ #define VEND1_GLOBAL_RSVD_STAT9_MODE GENMASK(7, 0) #define VEND1_GLOBAL_RSVD_STAT9_1000BT2 0x23 +/* MDIO_MMD_C22EXT */ +#define MDIO_C22EXT_STAT_SGMII_RX_GOOD_FRAMES 0xd292 +#define MDIO_C22EXT_STAT_SGMII_RX_BAD_FRAMES 0xd294 +#define MDIO_C22EXT_STAT_SGMII_RX_FALSE_CARRIER 0xd297 +#define MDIO_C22EXT_STAT_SGMII_TX_GOOD_FRAMES 0xd313 +#define MDIO_C22EXT_STAT_SGMII_TX_BAD_FRAMES 0xd315 +#define MDIO_C22EXT_STAT_SGMII_TX_FALSE_CARRIER 0xd317 +#define MDIO_C22EXT_STAT_SGMII_TX_COLLISIONS 0xd318 +#define MDIO_C22EXT_STAT_SGMII_TX_LINE_COLLISIONS 0xd319 +#define MDIO_C22EXT_STAT_SGMII_TX_FRAME_ALIGN_ERR 0xd31a +#define MDIO_C22EXT_STAT_SGMII_TX_RUNT_FRAMES 0xd31b + #define VEND1_GLOBAL_INT_STD_STATUS 0xfc00 #define VEND1_GLOBAL_INT_VEND_STATUS 0xfc01 @@ -113,6 +147,35 @@ #define VEND1_GLOBAL_INT_VEND_MASK_GLOBAL2 BIT(1) #define VEND1_GLOBAL_INT_VEND_MASK_GLOBAL3 BIT(0) +#define AQR_MAX_LEDS 3 + +struct aqr107_hw_stat { + const char *name; + int reg; + int size; +}; + +#define SGMII_STAT(n, r, s) { n, MDIO_C22EXT_STAT_SGMII_ ## r, s } +static const struct aqr107_hw_stat aqr107_hw_stats[] = { + SGMII_STAT("sgmii_rx_good_frames", RX_GOOD_FRAMES, 26), + SGMII_STAT("sgmii_rx_bad_frames", RX_BAD_FRAMES, 26), + SGMII_STAT("sgmii_rx_false_carrier_events", RX_FALSE_CARRIER, 8), + SGMII_STAT("sgmii_tx_good_frames", TX_GOOD_FRAMES, 26), + SGMII_STAT("sgmii_tx_bad_frames", TX_BAD_FRAMES, 26), + SGMII_STAT("sgmii_tx_false_carrier_events", TX_FALSE_CARRIER, 8), + SGMII_STAT("sgmii_tx_collisions", TX_COLLISIONS, 8), + SGMII_STAT("sgmii_tx_line_collisions", TX_LINE_COLLISIONS, 8), + SGMII_STAT("sgmii_tx_frame_alignment_err", TX_FRAME_ALIGN_ERR, 16), + SGMII_STAT("sgmii_tx_runt_frames", TX_RUNT_FRAMES, 22), +}; + +#define AQR107_SGMII_STAT_SZ ARRAY_SIZE(aqr107_hw_stats) + +struct aqr107_priv { + u64 sgmii_stats[AQR107_SGMII_STAT_SZ]; + unsigned long leds_active_low; +}; + #if IS_REACHABLE(CONFIG_HWMON) int aqr_hwmon_probe(struct phy_device *phydev); #else @@ -120,3 +183,18 @@ static inline int aqr_hwmon_probe(struct phy_device *phydev) { return 0; } #endif int aqr_firmware_load(struct phy_device *phydev); + +int aqr_phy_led_blink_set(struct phy_device *phydev, u8 index, + unsigned long *delay_on, + unsigned long *delay_off); +int aqr_phy_led_brightness_set(struct phy_device *phydev, + u8 index, enum led_brightness value); +int aqr_phy_led_hw_is_supported(struct phy_device *phydev, u8 index, + unsigned long rules); +int aqr_phy_led_hw_control_get(struct phy_device *phydev, u8 index, + unsigned long *rules); +int aqr_phy_led_hw_control_set(struct phy_device *phydev, u8 index, + unsigned long rules); +int aqr_phy_led_active_low_set(struct phy_device *phydev, int index, bool enable); +int aqr_phy_led_polarity_set(struct phy_device *phydev, int index, + unsigned long modes); diff --git a/drivers/net/phy/aquantia/aquantia_leds.c b/drivers/net/phy/aquantia/aquantia_leds.c new file mode 100644 index 0000000000000000000000000000000000000000..0516ac02c3f81e7be3b8cb621d238d3794f11463 --- /dev/null +++ b/drivers/net/phy/aquantia/aquantia_leds.c @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: GPL-2.0 +/* LED driver for Aquantia PHY + * + * Author: Daniel Golle + */ + +#include + +#include "aquantia.h" + +int aqr_phy_led_brightness_set(struct phy_device *phydev, + u8 index, enum led_brightness value) +{ + if (index >= AQR_MAX_LEDS) + return -EINVAL; + + return phy_modify_mmd(phydev, MDIO_MMD_VEND1, AQR_LED_PROV(index), + VEND1_GLOBAL_LED_PROV_LINK_MASK | + VEND1_GLOBAL_LED_PROV_FORCE_ON | + VEND1_GLOBAL_LED_PROV_RX_ACT | + VEND1_GLOBAL_LED_PROV_TX_ACT, + value ? VEND1_GLOBAL_LED_PROV_FORCE_ON : 0); +} + +static const unsigned long supported_triggers = (BIT(TRIGGER_NETDEV_LINK) | + BIT(TRIGGER_NETDEV_LINK_100) | + BIT(TRIGGER_NETDEV_LINK_1000) | + BIT(TRIGGER_NETDEV_LINK_2500) | + BIT(TRIGGER_NETDEV_LINK_5000) | + BIT(TRIGGER_NETDEV_LINK_10000) | + BIT(TRIGGER_NETDEV_RX) | + BIT(TRIGGER_NETDEV_TX)); + +int aqr_phy_led_hw_is_supported(struct phy_device *phydev, u8 index, + unsigned long rules) +{ + if (index >= AQR_MAX_LEDS) + return -EINVAL; + + /* All combinations of the supported triggers are allowed */ + if (rules & ~supported_triggers) + return -EOPNOTSUPP; + + return 0; +} + +int aqr_phy_led_hw_control_get(struct phy_device *phydev, u8 index, + unsigned long *rules) +{ + int val; + + if (index >= AQR_MAX_LEDS) + return -EINVAL; + + val = phy_read_mmd(phydev, MDIO_MMD_VEND1, AQR_LED_PROV(index)); + if (val < 0) + return val; + + *rules = 0; + if (val & VEND1_GLOBAL_LED_PROV_LINK100) + *rules |= BIT(TRIGGER_NETDEV_LINK_100); + + if (val & VEND1_GLOBAL_LED_PROV_LINK1000) + *rules |= BIT(TRIGGER_NETDEV_LINK_1000); + + if (val & VEND1_GLOBAL_LED_PROV_LINK2500) + *rules |= BIT(TRIGGER_NETDEV_LINK_2500); + + if (val & VEND1_GLOBAL_LED_PROV_LINK5000) + *rules |= BIT(TRIGGER_NETDEV_LINK_5000); + + if (val & VEND1_GLOBAL_LED_PROV_LINK10000) + *rules |= BIT(TRIGGER_NETDEV_LINK_10000); + + if (val & VEND1_GLOBAL_LED_PROV_RX_ACT) + *rules |= BIT(TRIGGER_NETDEV_RX); + + if (val & VEND1_GLOBAL_LED_PROV_TX_ACT) + *rules |= BIT(TRIGGER_NETDEV_TX); + + return 0; +} + +int aqr_phy_led_hw_control_set(struct phy_device *phydev, u8 index, + unsigned long rules) +{ + u16 val = 0; + + if (index >= AQR_MAX_LEDS) + return -EINVAL; + + if (rules & (BIT(TRIGGER_NETDEV_LINK_100) | BIT(TRIGGER_NETDEV_LINK))) + val |= VEND1_GLOBAL_LED_PROV_LINK100; + + if (rules & (BIT(TRIGGER_NETDEV_LINK_1000) | BIT(TRIGGER_NETDEV_LINK))) + val |= VEND1_GLOBAL_LED_PROV_LINK1000; + + if (rules & (BIT(TRIGGER_NETDEV_LINK_2500) | BIT(TRIGGER_NETDEV_LINK))) + val |= VEND1_GLOBAL_LED_PROV_LINK2500; + + if (rules & (BIT(TRIGGER_NETDEV_LINK_5000) | BIT(TRIGGER_NETDEV_LINK))) + val |= VEND1_GLOBAL_LED_PROV_LINK5000; + + if (rules & (BIT(TRIGGER_NETDEV_LINK_10000) | BIT(TRIGGER_NETDEV_LINK))) + val |= VEND1_GLOBAL_LED_PROV_LINK10000; + + if (rules & BIT(TRIGGER_NETDEV_RX)) + val |= VEND1_GLOBAL_LED_PROV_RX_ACT; + + if (rules & BIT(TRIGGER_NETDEV_TX)) + val |= VEND1_GLOBAL_LED_PROV_TX_ACT; + + return phy_modify_mmd(phydev, MDIO_MMD_VEND1, AQR_LED_PROV(index), + VEND1_GLOBAL_LED_PROV_LINK_MASK | + VEND1_GLOBAL_LED_PROV_FORCE_ON | + VEND1_GLOBAL_LED_PROV_RX_ACT | + VEND1_GLOBAL_LED_PROV_TX_ACT, val); +} + +int aqr_phy_led_active_low_set(struct phy_device *phydev, int index, bool enable) +{ + return phy_modify_mmd(phydev, MDIO_MMD_VEND1, AQR_LED_DRIVE(index), + VEND1_GLOBAL_LED_DRIVE_VDD, enable); +} + +int aqr_phy_led_polarity_set(struct phy_device *phydev, int index, unsigned long modes) +{ + struct aqr107_priv *priv = phydev->priv; + bool active_low = false; + u32 mode; + + if (index >= AQR_MAX_LEDS) + return -EINVAL; + + for_each_set_bit(mode, &modes, __PHY_LED_MODES_NUM) { + switch (mode) { + case PHY_LED_ACTIVE_LOW: + active_low = true; + break; + default: + return -EINVAL; + } + } + + /* Save LED driver vdd state to restore on SW reset */ + if (active_low) + priv->leds_active_low |= BIT(index); + + return aqr_phy_led_active_low_set(phydev, index, active_low); +} diff --git a/drivers/net/phy/aquantia/aquantia_main.c b/drivers/net/phy/aquantia/aquantia_main.c index d34cdec47636b1f4a17c83769810fbce9d5a7cf6..6c14355744b7e8ff45a3f9597db245fe61b32d0f 100644 --- a/drivers/net/phy/aquantia/aquantia_main.c +++ b/drivers/net/phy/aquantia/aquantia_main.c @@ -84,49 +84,12 @@ #define MDIO_AN_RX_VEND_STAT3 0xe832 #define MDIO_AN_RX_VEND_STAT3_AFR BIT(0) -/* MDIO_MMD_C22EXT */ -#define MDIO_C22EXT_STAT_SGMII_RX_GOOD_FRAMES 0xd292 -#define MDIO_C22EXT_STAT_SGMII_RX_BAD_FRAMES 0xd294 -#define MDIO_C22EXT_STAT_SGMII_RX_FALSE_CARRIER 0xd297 -#define MDIO_C22EXT_STAT_SGMII_TX_GOOD_FRAMES 0xd313 -#define MDIO_C22EXT_STAT_SGMII_TX_BAD_FRAMES 0xd315 -#define MDIO_C22EXT_STAT_SGMII_TX_FALSE_CARRIER 0xd317 -#define MDIO_C22EXT_STAT_SGMII_TX_COLLISIONS 0xd318 -#define MDIO_C22EXT_STAT_SGMII_TX_LINE_COLLISIONS 0xd319 -#define MDIO_C22EXT_STAT_SGMII_TX_FRAME_ALIGN_ERR 0xd31a -#define MDIO_C22EXT_STAT_SGMII_TX_RUNT_FRAMES 0xd31b - /* Sleep and timeout for checking if the Processor-Intensive * MDIO operation is finished */ #define AQR107_OP_IN_PROG_SLEEP 1000 #define AQR107_OP_IN_PROG_TIMEOUT 100000 -struct aqr107_hw_stat { - const char *name; - int reg; - int size; -}; - -#define SGMII_STAT(n, r, s) { n, MDIO_C22EXT_STAT_SGMII_ ## r, s } -static const struct aqr107_hw_stat aqr107_hw_stats[] = { - SGMII_STAT("sgmii_rx_good_frames", RX_GOOD_FRAMES, 26), - SGMII_STAT("sgmii_rx_bad_frames", RX_BAD_FRAMES, 26), - SGMII_STAT("sgmii_rx_false_carrier_events", RX_FALSE_CARRIER, 8), - SGMII_STAT("sgmii_tx_good_frames", TX_GOOD_FRAMES, 26), - SGMII_STAT("sgmii_tx_bad_frames", TX_BAD_FRAMES, 26), - SGMII_STAT("sgmii_tx_false_carrier_events", TX_FALSE_CARRIER, 8), - SGMII_STAT("sgmii_tx_collisions", TX_COLLISIONS, 8), - SGMII_STAT("sgmii_tx_line_collisions", TX_LINE_COLLISIONS, 8), - SGMII_STAT("sgmii_tx_frame_alignment_err", TX_FRAME_ALIGN_ERR, 16), - SGMII_STAT("sgmii_tx_runt_frames", TX_RUNT_FRAMES, 22), -}; -#define AQR107_SGMII_STAT_SZ ARRAY_SIZE(aqr107_hw_stats) - -struct aqr107_priv { - u64 sgmii_stats[AQR107_SGMII_STAT_SZ]; -}; - static int aqr107_get_sset_count(struct phy_device *phydev) { return AQR107_SGMII_STAT_SZ; @@ -512,7 +475,9 @@ static void aqr107_chip_info(struct phy_device *phydev) static int aqr107_config_init(struct phy_device *phydev) { - int ret; + struct aqr107_priv *priv = phydev->priv; + u32 led_active_low; + int ret, index = 0; /* Check that the PHY interface type is compatible */ if (phydev->interface != PHY_INTERFACE_MODE_SGMII && @@ -533,7 +498,19 @@ static int aqr107_config_init(struct phy_device *phydev) if (!ret) aqr107_chip_info(phydev); - return aqr107_set_downshift(phydev, MDIO_AN_VEND_PROV_DOWNSHIFT_DFLT); + ret = aqr107_set_downshift(phydev, MDIO_AN_VEND_PROV_DOWNSHIFT_DFLT); + if (ret) + return ret; + + /* Restore LED polarity state after reset */ + for_each_set_bit(led_active_low, &priv->leds_active_low, AQR_MAX_LEDS) { + ret = aqr_phy_led_active_low_set(phydev, index, led_active_low); + if (ret) + return ret; + index++; + } + + return 0; } static int aqcs109_config_init(struct phy_device *phydev) @@ -823,6 +800,11 @@ static struct phy_driver aqr_driver[] = { .get_strings = aqr107_get_strings, .get_stats = aqr107_get_stats, .link_change_notify = aqr107_link_change_notify, + .led_brightness_set = aqr_phy_led_brightness_set, + .led_hw_is_supported = aqr_phy_led_hw_is_supported, + .led_hw_control_set = aqr_phy_led_hw_control_set, + .led_hw_control_get = aqr_phy_led_hw_control_get, + .led_polarity_set = aqr_phy_led_polarity_set, }, { PHY_ID_MATCH_MODEL(PHY_ID_AQCS109), @@ -842,6 +824,11 @@ static struct phy_driver aqr_driver[] = { .get_strings = aqr107_get_strings, .get_stats = aqr107_get_stats, .link_change_notify = aqr107_link_change_notify, + .led_brightness_set = aqr_phy_led_brightness_set, + .led_hw_is_supported = aqr_phy_led_hw_is_supported, + .led_hw_control_set = aqr_phy_led_hw_control_set, + .led_hw_control_get = aqr_phy_led_hw_control_get, + .led_polarity_set = aqr_phy_led_polarity_set, }, { PHY_ID_MATCH_MODEL(PHY_ID_AQR111), @@ -861,6 +848,11 @@ static struct phy_driver aqr_driver[] = { .get_strings = aqr107_get_strings, .get_stats = aqr107_get_stats, .link_change_notify = aqr107_link_change_notify, + .led_brightness_set = aqr_phy_led_brightness_set, + .led_hw_is_supported = aqr_phy_led_hw_is_supported, + .led_hw_control_set = aqr_phy_led_hw_control_set, + .led_hw_control_get = aqr_phy_led_hw_control_get, + .led_polarity_set = aqr_phy_led_polarity_set, }, { PHY_ID_MATCH_MODEL(PHY_ID_AQR111B0), @@ -880,6 +872,11 @@ static struct phy_driver aqr_driver[] = { .get_strings = aqr107_get_strings, .get_stats = aqr107_get_stats, .link_change_notify = aqr107_link_change_notify, + .led_brightness_set = aqr_phy_led_brightness_set, + .led_hw_is_supported = aqr_phy_led_hw_is_supported, + .led_hw_control_set = aqr_phy_led_hw_control_set, + .led_hw_control_get = aqr_phy_led_hw_control_get, + .led_polarity_set = aqr_phy_led_polarity_set, }, { PHY_ID_MATCH_MODEL(PHY_ID_AQR405), @@ -906,6 +903,11 @@ static struct phy_driver aqr_driver[] = { .get_strings = aqr107_get_strings, .get_stats = aqr107_get_stats, .link_change_notify = aqr107_link_change_notify, + .led_brightness_set = aqr_phy_led_brightness_set, + .led_hw_is_supported = aqr_phy_led_hw_is_supported, + .led_hw_control_set = aqr_phy_led_hw_control_set, + .led_hw_control_get = aqr_phy_led_hw_control_get, + .led_polarity_set = aqr_phy_led_polarity_set, }, { PHY_ID_MATCH_MODEL(PHY_ID_AQR412), @@ -943,6 +945,11 @@ static struct phy_driver aqr_driver[] = { .get_strings = aqr107_get_strings, .get_stats = aqr107_get_stats, .link_change_notify = aqr107_link_change_notify, + .led_brightness_set = aqr_phy_led_brightness_set, + .led_hw_is_supported = aqr_phy_led_hw_is_supported, + .led_hw_control_set = aqr_phy_led_hw_control_set, + .led_hw_control_get = aqr_phy_led_hw_control_get, + .led_polarity_set = aqr_phy_led_polarity_set, }, { PHY_ID_MATCH_MODEL(PHY_ID_AQR113C), @@ -962,6 +969,11 @@ static struct phy_driver aqr_driver[] = { .get_strings = aqr107_get_strings, .get_stats = aqr107_get_stats, .link_change_notify = aqr107_link_change_notify, + .led_brightness_set = aqr_phy_led_brightness_set, + .led_hw_is_supported = aqr_phy_led_hw_is_supported, + .led_hw_control_set = aqr_phy_led_hw_control_set, + .led_hw_control_get = aqr_phy_led_hw_control_get, + .led_polarity_set = aqr_phy_led_polarity_set, }, { PHY_ID_MATCH_MODEL(PHY_ID_AQR114C), @@ -981,6 +993,11 @@ static struct phy_driver aqr_driver[] = { .get_strings = aqr107_get_strings, .get_stats = aqr107_get_stats, .link_change_notify = aqr107_link_change_notify, + .led_brightness_set = aqr_phy_led_brightness_set, + .led_hw_is_supported = aqr_phy_led_hw_is_supported, + .led_hw_control_set = aqr_phy_led_hw_control_set, + .led_hw_control_get = aqr_phy_led_hw_control_get, + .led_polarity_set = aqr_phy_led_polarity_set, }, { PHY_ID_MATCH_MODEL(PHY_ID_AQR813), @@ -1000,6 +1017,11 @@ static struct phy_driver aqr_driver[] = { .get_strings = aqr107_get_strings, .get_stats = aqr107_get_stats, .link_change_notify = aqr107_link_change_notify, + .led_brightness_set = aqr_phy_led_brightness_set, + .led_hw_is_supported = aqr_phy_led_hw_is_supported, + .led_hw_control_set = aqr_phy_led_hw_control_set, + .led_hw_control_get = aqr_phy_led_hw_control_get, + .led_polarity_set = aqr_phy_led_polarity_set, }, }; diff --git a/drivers/net/phy/micrel.c b/drivers/net/phy/micrel.c index 13e30ea7eec5d1e8cbfca6811d09a347660e06dd..5aada7cf3da72632510612ac5863953f9d964808 100644 --- a/drivers/net/phy/micrel.c +++ b/drivers/net/phy/micrel.c @@ -866,6 +866,17 @@ static int ksz8061_config_init(struct phy_device *phydev) { int ret; + /* Chip can be powered down by the bootstrap code. */ + ret = phy_read(phydev, MII_BMCR); + if (ret < 0) + return ret; + if (ret & BMCR_PDOWN) { + ret = phy_write(phydev, MII_BMCR, ret & ~BMCR_PDOWN); + if (ret < 0) + return ret; + usleep_range(1000, 2000); + } + ret = phy_write_mmd(phydev, MDIO_MMD_PMAPMD, MDIO_DEVID1, 0xB61A); if (ret) return ret; @@ -1939,7 +1950,7 @@ static const struct ksz9477_errata_write ksz9477_errata_writes[] = { {0x1c, 0x20, 0xeeee}, }; -static int ksz9477_config_init(struct phy_device *phydev) +static int ksz9477_phy_errata(struct phy_device *phydev) { int err; int i; @@ -1967,16 +1978,30 @@ static int ksz9477_config_init(struct phy_device *phydev) return err; } + err = genphy_restart_aneg(phydev); + if (err) + return err; + + return err; +} + +static int ksz9477_config_init(struct phy_device *phydev) +{ + int err; + + /* Only KSZ9897 family of switches needs this fix. */ + if ((phydev->phy_id & 0xf) == 1) { + err = ksz9477_phy_errata(phydev); + if (err) + return err; + } + /* According to KSZ9477 Errata DS80000754C (Module 4) all EEE modes * in this switch shall be regarded as broken. */ if (phydev->dev_flags & MICREL_NO_EEE) phydev->eee_broken_modes = -1; - err = genphy_restart_aneg(phydev); - if (err) - return err; - return kszphy_config_init(phydev); } @@ -2085,6 +2110,71 @@ static int kszphy_resume(struct phy_device *phydev) return 0; } +static int ksz9477_resume(struct phy_device *phydev) +{ + int ret; + + /* No need to initialize registers if not powered down. */ + ret = phy_read(phydev, MII_BMCR); + if (ret < 0) + return ret; + if (!(ret & BMCR_PDOWN)) + return 0; + + genphy_resume(phydev); + + /* After switching from power-down to normal mode, an internal global + * reset is automatically generated. Wait a minimum of 1 ms before + * read/write access to the PHY registers. + */ + usleep_range(1000, 2000); + + /* Only KSZ9897 family of switches needs this fix. */ + if ((phydev->phy_id & 0xf) == 1) { + ret = ksz9477_phy_errata(phydev); + if (ret) + return ret; + } + + /* Enable PHY Interrupts */ + if (phy_interrupt_is_valid(phydev)) { + phydev->interrupts = PHY_INTERRUPT_ENABLED; + if (phydev->drv->config_intr) + phydev->drv->config_intr(phydev); + } + + return 0; +} + +static int ksz8061_resume(struct phy_device *phydev) +{ + int ret; + + /* This function can be called twice when the Ethernet device is on. */ + ret = phy_read(phydev, MII_BMCR); + if (ret < 0) + return ret; + if (!(ret & BMCR_PDOWN)) + return 0; + + genphy_resume(phydev); + usleep_range(1000, 2000); + + /* Re-program the value after chip is reset. */ + ret = phy_write_mmd(phydev, MDIO_MMD_PMAPMD, MDIO_DEVID1, 0xB61A); + if (ret) + return ret; + + /* Enable PHY Interrupts */ + if (phy_interrupt_is_valid(phydev)) { + phydev->interrupts = PHY_INTERRUPT_ENABLED; + if (phydev->drv->config_intr) + phydev->drv->config_intr(phydev); + } + + return 0; +} + static int kszphy_probe(struct phy_device *phydev) { const struct kszphy_type *type = phydev->drv->driver_data; @@ -4029,7 +4119,7 @@ static int lan8841_config_intr(struct phy_device *phydev) if (phydev->interrupts == PHY_INTERRUPT_ENABLED) { err = phy_read(phydev, LAN8814_INTS); - if (err) + if (err < 0) return err; /* Enable / disable interrupts. It is OK to enable PTP interrupt @@ -4045,6 +4135,14 @@ static int lan8841_config_intr(struct phy_device *phydev) return err; err = phy_read(phydev, LAN8814_INTS); + if (err < 0) + return err; + + /* Getting a positive value doesn't mean that is an error, it + * just indicates what was the status. Therefore make sure to + * clear the value and say that there is no error. + */ + err = 0; } return err; @@ -5327,10 +5425,11 @@ static struct phy_driver ksphy_driver[] = { /* PHY_BASIC_FEATURES */ .probe = kszphy_probe, .config_init = ksz8061_config_init, + .soft_reset = genphy_soft_reset, .config_intr = kszphy_config_intr, .handle_interrupt = kszphy_handle_interrupt, .suspend = kszphy_suspend, - .resume = kszphy_resume, + .resume = ksz8061_resume, }, { .phy_id = PHY_ID_KSZ9021, .phy_id_mask = 0x000ffffe, @@ -5484,7 +5583,7 @@ static struct phy_driver ksphy_driver[] = { .config_intr = kszphy_config_intr, .handle_interrupt = kszphy_handle_interrupt, .suspend = genphy_suspend, - .resume = genphy_resume, + .resume = ksz9477_resume, .get_features = ksz9477_get_features, } }; diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c index 994471fad833f99ae0727b0cde3a5dbf0f2f0f57..02427378acfda1e9b6faeda242a80887da37bee1 100644 --- a/drivers/net/phy/phylink.c +++ b/drivers/net/phy/phylink.c @@ -885,26 +885,31 @@ static int phylink_parse_mode(struct phylink *pl, const char *managed; unsigned long caps; + if (pl->config->default_an_inband) + pl->cfg_link_an_mode = MLO_AN_INBAND; + dn = fwnode_get_named_child_node(fwnode, "fixed-link"); if (dn || fwnode_property_present(fwnode, "fixed-link")) pl->cfg_link_an_mode = MLO_AN_FIXED; fwnode_handle_put(dn); if ((fwnode_property_read_string(fwnode, "managed", &managed) == 0 && - strcmp(managed, "in-band-status") == 0) || - pl->config->ovr_an_inband) { + strcmp(managed, "in-band-status") == 0)) { if (pl->cfg_link_an_mode == MLO_AN_FIXED) { phylink_err(pl, "can't use both fixed-link and in-band-status\n"); return -EINVAL; } + pl->cfg_link_an_mode = MLO_AN_INBAND; + } + + if (pl->cfg_link_an_mode == MLO_AN_INBAND) { linkmode_zero(pl->supported); phylink_set(pl->supported, MII); phylink_set(pl->supported, Autoneg); phylink_set(pl->supported, Asym_Pause); phylink_set(pl->supported, Pause); - pl->cfg_link_an_mode = MLO_AN_INBAND; switch (pl->link_config.interface) { case PHY_INTERFACE_MODE_SGMII: diff --git a/drivers/net/phy/xilinx_gmii2rgmii.c b/drivers/net/phy/xilinx_gmii2rgmii.c index 7b1bc5fcef9b2a923a2803b6149eece41714fbf3..7c51daecf18ee04ed047eae4da252aa8f8437592 100644 --- a/drivers/net/phy/xilinx_gmii2rgmii.c +++ b/drivers/net/phy/xilinx_gmii2rgmii.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #define XILINX_GMII2RGMII_REG 0x10 @@ -85,11 +86,17 @@ static int xgmiitorgmii_probe(struct mdio_device *mdiodev) struct device *dev = &mdiodev->dev; struct device_node *np = dev->of_node, *phy_node; struct gmii2rgmii *priv; + struct clk *clkin; priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL); if (!priv) return -ENOMEM; + clkin = devm_clk_get_optional_enabled(dev, NULL); + if (IS_ERR(clkin)) + return dev_err_probe(dev, PTR_ERR(clkin), + "Failed to get and enable clock from Device Tree\n"); + phy_node = of_parse_phandle(np, "phy-handle", 0); if (!phy_node) { dev_err(dev, "Couldn't parse phy-handle\n"); diff --git a/drivers/net/usb/lan78xx.c b/drivers/net/usb/lan78xx.c index 5a2c38b63012ceb6a1f3be523ff0f643c9eec914..8adf77e3557e7a49ba340de53ab71db027db048c 100644 --- a/drivers/net/usb/lan78xx.c +++ b/drivers/net/usb/lan78xx.c @@ -380,11 +380,6 @@ struct skb_data { /* skb->cb is one of these */ int num_of_packet; }; -struct usb_context { - struct usb_ctrlrequest req; - struct lan78xx_net *dev; -}; - #define EVENT_TX_HALT 0 #define EVENT_RX_HALT 1 #define EVENT_RX_MEMORY 2 @@ -2946,6 +2941,8 @@ static int lan78xx_reset(struct lan78xx_net *dev) return ret; buf |= HW_CFG_MEF_; + buf |= HW_CFG_CLK125_EN_; + buf |= HW_CFG_REFCLK25_EN_; ret = lan78xx_write_reg(dev, HW_CFG, buf); if (ret < 0) @@ -3034,8 +3031,11 @@ static int lan78xx_reset(struct lan78xx_net *dev) return ret; /* LAN7801 only has RGMII mode */ - if (dev->chipid == ID_REV_CHIP_ID_7801_) + if (dev->chipid == ID_REV_CHIP_ID_7801_) { buf &= ~MAC_CR_GMII_EN_; + /* Enable Auto Duplex and Auto speed */ + buf |= MAC_CR_AUTO_DUPLEX_ | MAC_CR_AUTO_SPEED_; + } if (dev->chipid == ID_REV_CHIP_ID_7800_ || dev->chipid == ID_REV_CHIP_ID_7850_) { diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c index 19df1cd9f07243c06ab742fbda00c19c73593d55..15e12f46d0ea2950bc3a4558e27abc7c355d66b5 100644 --- a/drivers/net/usb/r8152.c +++ b/drivers/net/usb/r8152.c @@ -1774,6 +1774,7 @@ static int vendor_mac_passthru_addr_read(struct r8152 *tp, struct sockaddr *sa) goto amacout; } memcpy(sa->sa_data, buf, 6); + tp->netdev->addr_assign_type = NET_ADDR_STOLEN; netif_info(tp, probe, tp->netdev, "Using pass-thru MAC addr %pM\n", sa->sa_data); @@ -8554,6 +8555,19 @@ static int rtl8152_system_resume(struct r8152 *tp) usb_submit_urb(tp->intr_urb, GFP_NOIO); } + /* If the device is RTL8152_INACCESSIBLE here then we should do a + * reset. This is important because the usb_lock_device_for_reset() + * that happens as a result of usb_queue_reset_device() will silently + * fail if the device was suspended or if too much time passed. + * + * NOTE: The device is locked here so we can directly do the reset. + * We don't need usb_lock_device_for_reset() because that's just a + * wrapper over device_lock() and device_resume() (which calls us) + * does that for us. + */ + if (test_bit(RTL8152_INACCESSIBLE, &tp->flags)) + usb_reset_device(tp->udev); + return 0; } @@ -8634,6 +8648,13 @@ static int rtl8152_system_suspend(struct r8152 *tp) tasklet_enable(&tp->tx_tl); } + /* If we're inaccessible here then some of the work that we did to + * get the adapter ready for suspend didn't work. Queue up a wakeup + * event so we can try again. + */ + if (test_bit(RTL8152_INACCESSIBLE, &tp->flags)) + pm_wakeup_event(&tp->udev->dev, 0); + return 0; } diff --git a/drivers/net/usb/smsc75xx.c b/drivers/net/usb/smsc75xx.c index 0726e18bee6fcf3cbe4b3d2006d40cc2f781b64c..78c821349f484c4b21b9ebf08bb46b21d7e0bf77 100644 --- a/drivers/net/usb/smsc75xx.c +++ b/drivers/net/usb/smsc75xx.c @@ -61,11 +61,6 @@ struct smsc75xx_priv { u8 suspend_flags; }; -struct usb_context { - struct usb_ctrlrequest req; - struct usbnet *dev; -}; - static bool turbo_mode = true; module_param(turbo_mode, bool, 0644); MODULE_PARM_DESC(turbo_mode, "Enable multiple frames per Rx transaction"); diff --git a/drivers/net/usb/smsc95xx.c b/drivers/net/usb/smsc95xx.c index cbea246664795f27618908838b37384e8f3b67d0..8e82184be5e7d9954b83aae904a37bb7eda175bb 100644 --- a/drivers/net/usb/smsc95xx.c +++ b/drivers/net/usb/smsc95xx.c @@ -879,7 +879,7 @@ static int smsc95xx_start_rx_path(struct usbnet *dev) static int smsc95xx_reset(struct usbnet *dev) { struct smsc95xx_priv *pdata = dev->driver_priv; - u32 read_buf, write_buf, burst_cap; + u32 read_buf, burst_cap; int ret = 0, timeout; netif_dbg(dev, ifup, dev->net, "entering smsc95xx_reset\n"); @@ -1003,10 +1003,13 @@ static int smsc95xx_reset(struct usbnet *dev) return ret; netif_dbg(dev, ifup, dev->net, "ID_REV = 0x%08x\n", read_buf); + ret = smsc95xx_read_reg(dev, LED_GPIO_CFG, &read_buf); + if (ret < 0) + return ret; /* Configure GPIO pins as LED outputs */ - write_buf = LED_GPIO_CFG_SPD_LED | LED_GPIO_CFG_LNK_LED | - LED_GPIO_CFG_FDX_LED; - ret = smsc95xx_write_reg(dev, LED_GPIO_CFG, write_buf); + read_buf |= LED_GPIO_CFG_SPD_LED | LED_GPIO_CFG_LNK_LED | + LED_GPIO_CFG_FDX_LED; + ret = smsc95xx_write_reg(dev, LED_GPIO_CFG, read_buf); if (ret < 0) return ret; diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 4a802c0ea2cbc381ecc8a48c7ca28bc75c4808f8..61a57d134544f958682f1b2caa66e3548fc199db 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -2686,6 +2686,7 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd { struct scatterlist *sgs[5], hdr, stat; u32 out_num = 0, tmp, in_num = 0; + bool ok; int ret; /* Caller should know better */ @@ -2731,8 +2732,9 @@ static bool virtnet_send_command_reply(struct virtnet_info *vi, u8 class, u8 cmd } unlock: + ok = vi->ctrl->status == VIRTIO_NET_OK; mutex_unlock(&vi->cvq_lock); - return vi->ctrl->status == VIRTIO_NET_OK; + return ok; } static bool virtnet_send_command(struct virtnet_info *vi, u8 class, u8 cmd, @@ -4257,7 +4259,6 @@ static int virtnet_send_rx_notf_coal_cmds(struct virtnet_info *vi, struct virtio_net_ctrl_coal_rx *coal_rx __free(kfree) = NULL; bool rx_ctrl_dim_on = !!ec->use_adaptive_rx_coalesce; struct scatterlist sgs_rx; - int ret = 0; int i; if (rx_ctrl_dim_on && !virtio_has_feature(vi->vdev, VIRTIO_NET_F_VQ_NOTF_COAL)) @@ -4267,27 +4268,27 @@ static int virtnet_send_rx_notf_coal_cmds(struct virtnet_info *vi, ec->rx_max_coalesced_frames != vi->intr_coal_rx.max_packets)) return -EINVAL; - /* Acquire all queues dim_locks */ - for (i = 0; i < vi->max_queue_pairs; i++) - mutex_lock(&vi->rq[i].dim_lock); - if (rx_ctrl_dim_on && !vi->rx_dim_enabled) { vi->rx_dim_enabled = true; - for (i = 0; i < vi->max_queue_pairs; i++) + for (i = 0; i < vi->max_queue_pairs; i++) { + mutex_lock(&vi->rq[i].dim_lock); vi->rq[i].dim_enabled = true; - goto unlock; + mutex_unlock(&vi->rq[i].dim_lock); + } + return 0; } coal_rx = kzalloc(sizeof(*coal_rx), GFP_KERNEL); - if (!coal_rx) { - ret = -ENOMEM; - goto unlock; - } + if (!coal_rx) + return -ENOMEM; if (!rx_ctrl_dim_on && vi->rx_dim_enabled) { vi->rx_dim_enabled = false; - for (i = 0; i < vi->max_queue_pairs; i++) + for (i = 0; i < vi->max_queue_pairs; i++) { + mutex_lock(&vi->rq[i].dim_lock); vi->rq[i].dim_enabled = false; + mutex_unlock(&vi->rq[i].dim_lock); + } } /* Since the per-queue coalescing params can be set, @@ -4300,22 +4301,19 @@ static int virtnet_send_rx_notf_coal_cmds(struct virtnet_info *vi, if (!virtnet_send_command(vi, VIRTIO_NET_CTRL_NOTF_COAL, VIRTIO_NET_CTRL_NOTF_COAL_RX_SET, - &sgs_rx)) { - ret = -EINVAL; - goto unlock; - } + &sgs_rx)) + return -EINVAL; vi->intr_coal_rx.max_usecs = ec->rx_coalesce_usecs; vi->intr_coal_rx.max_packets = ec->rx_max_coalesced_frames; for (i = 0; i < vi->max_queue_pairs; i++) { + mutex_lock(&vi->rq[i].dim_lock); vi->rq[i].intr_coal.max_usecs = ec->rx_coalesce_usecs; vi->rq[i].intr_coal.max_packets = ec->rx_max_coalesced_frames; - } -unlock: - for (i = vi->max_queue_pairs - 1; i >= 0; i--) mutex_unlock(&vi->rq[i].dim_lock); + } - return ret; + return 0; } static int virtnet_send_notf_coal_cmds(struct virtnet_info *vi, @@ -4417,9 +4415,9 @@ static void virtnet_rx_dim_work(struct work_struct *work) if (err) pr_debug("%s: Failed to send dim parameters on rxq%d\n", dev->name, qnum); - dim->state = DIM_START_MEASURE; } out: + dim->state = DIM_START_MEASURE; mutex_unlock(&rq->dim_lock); } diff --git a/drivers/net/vmxnet3/Makefile b/drivers/net/vmxnet3/Makefile index f82870c10205ab98c152a165fc008bd5005ee07f..59ef494ce2e05c8d823038c6ddd7c65179541250 100644 --- a/drivers/net/vmxnet3/Makefile +++ b/drivers/net/vmxnet3/Makefile @@ -2,7 +2,7 @@ # # Linux driver for VMware's vmxnet3 ethernet NIC. # -# Copyright (C) 2007-2022, VMware, Inc. All Rights Reserved. +# Copyright (C) 2007-2024, VMware, Inc. All Rights Reserved. # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the diff --git a/drivers/net/vmxnet3/vmxnet3_defs.h b/drivers/net/vmxnet3/vmxnet3_defs.h index 41d6767283a63cc76c522a6f7738d21fb0905da4..5c5148768039c28a286540b49d7d3f8c26bec6ea 100644 --- a/drivers/net/vmxnet3/vmxnet3_defs.h +++ b/drivers/net/vmxnet3/vmxnet3_defs.h @@ -1,7 +1,7 @@ /* * Linux driver for VMware's vmxnet3 ethernet NIC. * - * Copyright (C) 2008-2022, VMware, Inc. All Rights Reserved. + * Copyright (C) 2008-2024, VMware, Inc. All Rights Reserved. * * This program is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License as published by the @@ -80,6 +80,8 @@ enum { #define VMXNET3_IO_TYPE(addr) ((addr) >> 24) #define VMXNET3_IO_REG(addr) ((addr) & 0xFFFFFF) +#define VMXNET3_PMC_PSEUDO_TSC 0x10003 + enum { VMXNET3_CMD_FIRST_SET = 0xCAFE0000, VMXNET3_CMD_ACTIVATE_DEV = VMXNET3_CMD_FIRST_SET, @@ -123,6 +125,8 @@ enum { VMXNET3_CMD_GET_RESERVED4, VMXNET3_CMD_GET_MAX_CAPABILITIES, VMXNET3_CMD_GET_DCR0_REG, + VMXNET3_CMD_GET_TSRING_DESC_SIZE, + VMXNET3_CMD_GET_DISABLED_OFFLOADS, }; /* @@ -254,6 +258,24 @@ struct Vmxnet3_RxDesc { #define VMXNET3_RCD_HDR_INNER_SHIFT 13 +struct Vmxnet3TSInfo { + u64 tsData:56; + u64 tsType:4; + u64 tsi:1; //bit to indicate to set ts + u64 pad:3; + u64 pad2; +}; + +struct Vmxnet3_TxTSDesc { + struct Vmxnet3TSInfo ts; + u64 pad[14]; +}; + +struct Vmxnet3_RxTSDesc { + struct Vmxnet3TSInfo ts; + u64 pad[14]; +}; + struct Vmxnet3_RxCompDesc { #ifdef __BIG_ENDIAN_BITFIELD u32 ext2:1; @@ -427,6 +449,13 @@ union Vmxnet3_GenericDesc { #define VMXNET3_RXDATA_DESC_SIZE_ALIGN 64 #define VMXNET3_RXDATA_DESC_SIZE_MASK (VMXNET3_RXDATA_DESC_SIZE_ALIGN - 1) +/* Rx TS Ring buffer size must be a multiple of 64 bytes */ +#define VMXNET3_RXTS_DESC_SIZE_ALIGN 64 +#define VMXNET3_RXTS_DESC_SIZE_MASK (VMXNET3_RXTS_DESC_SIZE_ALIGN - 1) +/* Tx TS Ring buffer size must be a multiple of 64 bytes */ +#define VMXNET3_TXTS_DESC_SIZE_ALIGN 64 +#define VMXNET3_TXTS_DESC_SIZE_MASK (VMXNET3_TXTS_DESC_SIZE_ALIGN - 1) + /* Max ring size */ #define VMXNET3_TX_RING_MAX_SIZE 4096 #define VMXNET3_TC_RING_MAX_SIZE 4096 @@ -439,6 +468,9 @@ union Vmxnet3_GenericDesc { #define VMXNET3_RXDATA_DESC_MAX_SIZE 2048 +#define VMXNET3_TXTS_DESC_MAX_SIZE 256 +#define VMXNET3_RXTS_DESC_MAX_SIZE 256 + /* a list of reasons for queue stop */ enum { @@ -546,6 +578,24 @@ struct Vmxnet3_RxQueueConf { }; +struct Vmxnet3_LatencyConf { + u16 sampleRate; + u16 pad; +}; + +struct Vmxnet3_TxQueueTSConf { + __le64 txTSRingBasePA; + __le16 txTSRingDescSize; /* size of tx timestamp ring buffer */ + u16 pad; + struct Vmxnet3_LatencyConf latencyConf; +}; + +struct Vmxnet3_RxQueueTSConf { + __le64 rxTSRingBasePA; + __le16 rxTSRingDescSize; /* size of rx timestamp ring buffer */ + u16 pad[3]; +}; + enum vmxnet3_intr_mask_mode { VMXNET3_IMM_AUTO = 0, VMXNET3_IMM_ACTIVE = 1, @@ -679,7 +729,8 @@ struct Vmxnet3_TxQueueDesc { /* Driver read after a GET command */ struct Vmxnet3_QueueStatus status; struct UPT1_TxStats stats; - u8 _pad[88]; /* 128 aligned */ + struct Vmxnet3_TxQueueTSConf tsConf; + u8 _pad[72]; /* 128 aligned */ }; @@ -689,7 +740,8 @@ struct Vmxnet3_RxQueueDesc { /* Driver read after a GET commad */ struct Vmxnet3_QueueStatus status; struct UPT1_RxStats stats; - u8 __pad[88]; /* 128 aligned */ + struct Vmxnet3_RxQueueTSConf tsConf; + u8 __pad[72]; /* 128 aligned */ }; struct Vmxnet3_SetPolling { @@ -861,4 +913,7 @@ struct Vmxnet3_DriverShared { /* when new capability is introduced, update VMXNET3_CAP_MAX */ #define VMXNET3_CAP_MAX VMXNET3_CAP_VERSION_7_MAX +#define VMXNET3_OFFLOAD_TSO BIT(0) +#define VMXNET3_OFFLOAD_LRO BIT(1) + #endif /* _VMXNET3_DEFS_H_ */ diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c index 89ca6e75fcc6b066bad0cb3c2b18734df3a0dec3..b70654c7ad34ea773a838d40bcb7e0131166981e 100644 --- a/drivers/net/vmxnet3/vmxnet3_drv.c +++ b/drivers/net/vmxnet3/vmxnet3_drv.c @@ -1,7 +1,7 @@ /* * Linux driver for VMware's vmxnet3 ethernet NIC. * - * Copyright (C) 2008-2022, VMware, Inc. All Rights Reserved. + * Copyright (C) 2008-2024, VMware, Inc. All Rights Reserved. * * This program is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License as published by the @@ -143,6 +143,32 @@ vmxnet3_tq_stop(struct vmxnet3_tx_queue *tq, struct vmxnet3_adapter *adapter) netif_stop_subqueue(adapter->netdev, (tq - adapter->tx_queue)); } +static u64 +vmxnet3_get_cycles(int pmc) +{ +#ifdef CONFIG_X86 + return native_read_pmc(pmc); +#else + return 0; +#endif +} + +static bool +vmxnet3_apply_timestamp(struct vmxnet3_tx_queue *tq, u16 rate) +{ +#ifdef CONFIG_X86 + if (rate > 0) { + if (tq->tsPktCount == 1) { + if (rate != 1) + tq->tsPktCount = rate; + return true; + } + tq->tsPktCount--; + } +#endif + return false; +} + /* Check if capability is supported by UPT device or * UPT is even requested */ @@ -498,6 +524,12 @@ vmxnet3_tq_destroy(struct vmxnet3_tx_queue *tq, tq->data_ring.base, tq->data_ring.basePA); tq->data_ring.base = NULL; } + if (tq->ts_ring.base) { + dma_free_coherent(&adapter->pdev->dev, + tq->tx_ring.size * tq->tx_ts_desc_size, + tq->ts_ring.base, tq->ts_ring.basePA); + tq->ts_ring.base = NULL; + } if (tq->comp_ring.base) { dma_free_coherent(&adapter->pdev->dev, tq->comp_ring.size * sizeof(struct Vmxnet3_TxCompDesc), @@ -535,6 +567,10 @@ vmxnet3_tq_init(struct vmxnet3_tx_queue *tq, memset(tq->data_ring.base, 0, tq->data_ring.size * tq->txdata_desc_size); + if (tq->ts_ring.base) + memset(tq->ts_ring.base, 0, + tq->tx_ring.size * tq->tx_ts_desc_size); + /* reset the tx comp ring contents to 0 and reset comp ring states */ memset(tq->comp_ring.base, 0, tq->comp_ring.size * sizeof(struct Vmxnet3_TxCompDesc)); @@ -573,6 +609,18 @@ vmxnet3_tq_create(struct vmxnet3_tx_queue *tq, goto err; } + if (tq->tx_ts_desc_size != 0) { + tq->ts_ring.base = dma_alloc_coherent(&adapter->pdev->dev, + tq->tx_ring.size * tq->tx_ts_desc_size, + &tq->ts_ring.basePA, GFP_KERNEL); + if (!tq->ts_ring.base) { + netdev_err(adapter->netdev, "failed to allocate tx ts ring\n"); + tq->tx_ts_desc_size = 0; + } + } else { + tq->ts_ring.base = NULL; + } + tq->comp_ring.base = dma_alloc_coherent(&adapter->pdev->dev, tq->comp_ring.size * sizeof(struct Vmxnet3_TxCompDesc), &tq->comp_ring.basePA, GFP_KERNEL); @@ -861,6 +909,11 @@ vmxnet3_map_pkt(struct sk_buff *skb, struct vmxnet3_tx_ctx *ctx, /* set the last buf_info for the pkt */ tbi->skb = skb; tbi->sop_idx = ctx->sop_txd - tq->tx_ring.base; + if (tq->tx_ts_desc_size != 0) { + ctx->ts_txd = (struct Vmxnet3_TxTSDesc *)((u8 *)tq->ts_ring.base + + tbi->sop_idx * tq->tx_ts_desc_size); + ctx->ts_txd->ts.tsi = 0; + } return 0; } @@ -968,7 +1021,7 @@ vmxnet3_parse_hdr(struct sk_buff *skb, struct vmxnet3_tx_queue *tq, skb_headlen(skb)); } - if (skb->len <= VMXNET3_HDR_COPY_SIZE) + if (skb->len <= tq->txdata_desc_size) ctx->copy_size = skb->len; /* make sure headers are accessible directly */ @@ -1259,6 +1312,14 @@ vmxnet3_tq_xmit(struct sk_buff *skb, struct vmxnet3_tx_queue *tq, gdesc->txd.tci = skb_vlan_tag_get(skb); } + if (tq->tx_ts_desc_size != 0 && + adapter->latencyConf->sampleRate != 0) { + if (vmxnet3_apply_timestamp(tq, adapter->latencyConf->sampleRate)) { + ctx.ts_txd->ts.tsData = vmxnet3_get_cycles(VMXNET3_PMC_PSEUDO_TSC); + ctx.ts_txd->ts.tsi = 1; + } + } + /* Ensure that the write to (&gdesc->txd)->gen will be observed after * all other writes to &gdesc->txd. */ @@ -1608,6 +1669,15 @@ skip_xdp: skip_page_frags = false; ctx->skb = rbi->skb; + if (rq->rx_ts_desc_size != 0 && rcd->ext2) { + struct Vmxnet3_RxTSDesc *ts_rxd; + + ts_rxd = (struct Vmxnet3_RxTSDesc *)((u8 *)rq->ts_ring.base + + idx * rq->rx_ts_desc_size); + ts_rxd->ts.tsData = vmxnet3_get_cycles(VMXNET3_PMC_PSEUDO_TSC); + ts_rxd->ts.tsi = 1; + } + rxDataRingUsed = VMXNET3_RX_DATA_RING(adapter, rcd->rqID); len = rxDataRingUsed ? rcd->len : rbi->len; @@ -2007,6 +2077,13 @@ static void vmxnet3_rq_destroy(struct vmxnet3_rx_queue *rq, rq->data_ring.base = NULL; } + if (rq->ts_ring.base) { + dma_free_coherent(&adapter->pdev->dev, + rq->rx_ring[0].size * rq->rx_ts_desc_size, + rq->ts_ring.base, rq->ts_ring.basePA); + rq->ts_ring.base = NULL; + } + if (rq->comp_ring.base) { dma_free_coherent(&adapter->pdev->dev, rq->comp_ring.size * sizeof(struct Vmxnet3_RxCompDesc), @@ -2034,8 +2111,8 @@ vmxnet3_rq_destroy_all_rxdataring(struct vmxnet3_adapter *adapter) rq->data_ring.base, rq->data_ring.basePA); rq->data_ring.base = NULL; - rq->data_ring.desc_size = 0; } + rq->data_ring.desc_size = 0; } } @@ -2090,6 +2167,10 @@ vmxnet3_rq_init(struct vmxnet3_rx_queue *rq, } vmxnet3_rq_alloc_rx_buf(rq, 1, rq->rx_ring[1].size - 1, adapter); + if (rq->ts_ring.base) + memset(rq->ts_ring.base, 0, + rq->rx_ring[0].size * rq->rx_ts_desc_size); + /* reset the comp ring */ rq->comp_ring.next2proc = 0; memset(rq->comp_ring.base, 0, rq->comp_ring.size * @@ -2160,6 +2241,21 @@ vmxnet3_rq_create(struct vmxnet3_rx_queue *rq, struct vmxnet3_adapter *adapter) rq->data_ring.desc_size = 0; } + if (rq->rx_ts_desc_size != 0) { + sz = rq->rx_ring[0].size * rq->rx_ts_desc_size; + rq->ts_ring.base = + dma_alloc_coherent(&adapter->pdev->dev, sz, + &rq->ts_ring.basePA, + GFP_KERNEL); + if (!rq->ts_ring.base) { + netdev_err(adapter->netdev, + "rx ts ring will be disabled\n"); + rq->rx_ts_desc_size = 0; + } + } else { + rq->ts_ring.base = NULL; + } + sz = rq->comp_ring.size * sizeof(struct Vmxnet3_RxCompDesc); rq->comp_ring.base = dma_alloc_coherent(&adapter->pdev->dev, sz, &rq->comp_ring.basePA, @@ -2759,6 +2855,8 @@ vmxnet3_setup_driver_shared(struct vmxnet3_adapter *adapter) struct Vmxnet3_DSDevReadExt *devReadExt = &shared->devReadExt; struct Vmxnet3_TxQueueConf *tqc; struct Vmxnet3_RxQueueConf *rqc; + struct Vmxnet3_TxQueueTSConf *tqtsc; + struct Vmxnet3_RxQueueTSConf *rqtsc; int i; memset(shared, 0, sizeof(*shared)); @@ -2815,6 +2913,11 @@ vmxnet3_setup_driver_shared(struct vmxnet3_adapter *adapter) tqc->compRingSize = cpu_to_le32(tq->comp_ring.size); tqc->ddLen = cpu_to_le32(0); tqc->intrIdx = tq->comp_ring.intr_idx; + if (VMXNET3_VERSION_GE_9(adapter)) { + tqtsc = &adapter->tqd_start[i].tsConf; + tqtsc->txTSRingBasePA = cpu_to_le64(tq->ts_ring.basePA); + tqtsc->txTSRingDescSize = cpu_to_le16(tq->tx_ts_desc_size); + } } /* rx queue settings */ @@ -2837,6 +2940,11 @@ vmxnet3_setup_driver_shared(struct vmxnet3_adapter *adapter) rqc->rxDataRingDescSize = cpu_to_le16(rq->data_ring.desc_size); } + if (VMXNET3_VERSION_GE_9(adapter)) { + rqtsc = &adapter->rqd_start[i].tsConf; + rqtsc->rxTSRingBasePA = cpu_to_le64(rq->ts_ring.basePA); + rqtsc->rxTSRingDescSize = cpu_to_le16(rq->rx_ts_desc_size); + } } #ifdef VMXNET3_RSS @@ -3299,6 +3407,8 @@ vmxnet3_create_queues(struct vmxnet3_adapter *adapter, u32 tx_ring_size, tq->stopped = true; tq->adapter = adapter; tq->qid = i; + tq->tx_ts_desc_size = adapter->tx_ts_desc_size; + tq->tsPktCount = 1; err = vmxnet3_tq_create(tq, adapter); /* * Too late to change num_tx_queues. We cannot do away with @@ -3320,6 +3430,7 @@ vmxnet3_create_queues(struct vmxnet3_adapter *adapter, u32 tx_ring_size, rq->shared = &adapter->rqd_start[i].ctrl; rq->adapter = adapter; rq->data_ring.desc_size = rxdata_desc_size; + rq->rx_ts_desc_size = adapter->rx_ts_desc_size; err = vmxnet3_rq_create(rq, adapter); if (err) { if (i == 0) { @@ -3361,14 +3472,15 @@ vmxnet3_open(struct net_device *netdev) if (VMXNET3_VERSION_GE_3(adapter)) { unsigned long flags; u16 txdata_desc_size; + u32 ret; spin_lock_irqsave(&adapter->cmd_lock, flags); VMXNET3_WRITE_BAR1_REG(adapter, VMXNET3_REG_CMD, VMXNET3_CMD_GET_TXDATA_DESC_SIZE); - txdata_desc_size = VMXNET3_READ_BAR1_REG(adapter, - VMXNET3_REG_CMD); + ret = VMXNET3_READ_BAR1_REG(adapter, VMXNET3_REG_CMD); spin_unlock_irqrestore(&adapter->cmd_lock, flags); + txdata_desc_size = ret & 0xffff; if ((txdata_desc_size < VMXNET3_TXDATA_DESC_MIN_SIZE) || (txdata_desc_size > VMXNET3_TXDATA_DESC_MAX_SIZE) || (txdata_desc_size & VMXNET3_TXDATA_DESC_SIZE_MASK)) { @@ -3377,10 +3489,40 @@ vmxnet3_open(struct net_device *netdev) } else { adapter->txdata_desc_size = txdata_desc_size; } + if (VMXNET3_VERSION_GE_9(adapter)) + adapter->rxdata_desc_size = (ret >> 16) & 0xffff; } else { adapter->txdata_desc_size = sizeof(struct Vmxnet3_TxDataDesc); } + if (VMXNET3_VERSION_GE_9(adapter)) { + unsigned long flags; + u16 tx_ts_desc_size = 0; + u16 rx_ts_desc_size = 0; + u32 ret; + + spin_lock_irqsave(&adapter->cmd_lock, flags); + VMXNET3_WRITE_BAR1_REG(adapter, VMXNET3_REG_CMD, + VMXNET3_CMD_GET_TSRING_DESC_SIZE); + ret = VMXNET3_READ_BAR1_REG(adapter, VMXNET3_REG_CMD); + spin_unlock_irqrestore(&adapter->cmd_lock, flags); + if (ret > 0) { + tx_ts_desc_size = (ret & 0xff); + rx_ts_desc_size = ((ret >> 16) & 0xff); + } + if (tx_ts_desc_size > VMXNET3_TXTS_DESC_MAX_SIZE || + tx_ts_desc_size & VMXNET3_TXTS_DESC_SIZE_MASK) + tx_ts_desc_size = 0; + if (rx_ts_desc_size > VMXNET3_RXTS_DESC_MAX_SIZE || + rx_ts_desc_size & VMXNET3_RXTS_DESC_SIZE_MASK) + rx_ts_desc_size = 0; + adapter->tx_ts_desc_size = tx_ts_desc_size; + adapter->rx_ts_desc_size = rx_ts_desc_size; + } else { + adapter->tx_ts_desc_size = 0; + adapter->rx_ts_desc_size = 0; + } + err = vmxnet3_create_queues(adapter, adapter->tx_ring_size, adapter->rx_ring_size, @@ -3503,6 +3645,15 @@ static void vmxnet3_declare_features(struct vmxnet3_adapter *adapter) { struct net_device *netdev = adapter->netdev; + unsigned long flags; + + if (VMXNET3_VERSION_GE_9(adapter)) { + spin_lock_irqsave(&adapter->cmd_lock, flags); + VMXNET3_WRITE_BAR1_REG(adapter, VMXNET3_REG_CMD, + VMXNET3_CMD_GET_DISABLED_OFFLOADS); + adapter->disabledOffloads = VMXNET3_READ_BAR1_REG(adapter, VMXNET3_REG_CMD); + spin_unlock_irqrestore(&adapter->cmd_lock, flags); + } netdev->hw_features = NETIF_F_SG | NETIF_F_RXCSUM | NETIF_F_HW_CSUM | NETIF_F_HW_VLAN_CTAG_TX | @@ -3520,6 +3671,16 @@ vmxnet3_declare_features(struct vmxnet3_adapter *adapter) NETIF_F_GSO_UDP_TUNNEL_CSUM; } + if (adapter->disabledOffloads & VMXNET3_OFFLOAD_TSO) { + netdev->hw_features &= ~(NETIF_F_TSO | NETIF_F_TSO6); + netdev->hw_enc_features &= ~(NETIF_F_TSO | NETIF_F_TSO6); + } + + if (adapter->disabledOffloads & VMXNET3_OFFLOAD_LRO) { + netdev->hw_features &= ~(NETIF_F_LRO); + netdev->hw_enc_features &= ~(NETIF_F_LRO); + } + if (VMXNET3_VERSION_GE_7(adapter)) { unsigned long flags; @@ -3790,7 +3951,7 @@ vmxnet3_probe_device(struct pci_dev *pdev, struct net_device *netdev; struct vmxnet3_adapter *adapter; u8 mac[ETH_ALEN]; - int size; + int size, i; int num_tx_queues; int num_rx_queues; int queues; @@ -3857,42 +4018,14 @@ vmxnet3_probe_device(struct pci_dev *pdev, goto err_alloc_pci; ver = VMXNET3_READ_BAR1_REG(adapter, VMXNET3_REG_VRRS); - if (ver & (1 << VMXNET3_REV_7)) { - VMXNET3_WRITE_BAR1_REG(adapter, - VMXNET3_REG_VRRS, - 1 << VMXNET3_REV_7); - adapter->version = VMXNET3_REV_7 + 1; - } else if (ver & (1 << VMXNET3_REV_6)) { - VMXNET3_WRITE_BAR1_REG(adapter, - VMXNET3_REG_VRRS, - 1 << VMXNET3_REV_6); - adapter->version = VMXNET3_REV_6 + 1; - } else if (ver & (1 << VMXNET3_REV_5)) { - VMXNET3_WRITE_BAR1_REG(adapter, - VMXNET3_REG_VRRS, - 1 << VMXNET3_REV_5); - adapter->version = VMXNET3_REV_5 + 1; - } else if (ver & (1 << VMXNET3_REV_4)) { - VMXNET3_WRITE_BAR1_REG(adapter, - VMXNET3_REG_VRRS, - 1 << VMXNET3_REV_4); - adapter->version = VMXNET3_REV_4 + 1; - } else if (ver & (1 << VMXNET3_REV_3)) { - VMXNET3_WRITE_BAR1_REG(adapter, - VMXNET3_REG_VRRS, - 1 << VMXNET3_REV_3); - adapter->version = VMXNET3_REV_3 + 1; - } else if (ver & (1 << VMXNET3_REV_2)) { - VMXNET3_WRITE_BAR1_REG(adapter, - VMXNET3_REG_VRRS, - 1 << VMXNET3_REV_2); - adapter->version = VMXNET3_REV_2 + 1; - } else if (ver & (1 << VMXNET3_REV_1)) { - VMXNET3_WRITE_BAR1_REG(adapter, - VMXNET3_REG_VRRS, - 1 << VMXNET3_REV_1); - adapter->version = VMXNET3_REV_1 + 1; - } else { + for (i = VMXNET3_REV_9; i >= VMXNET3_REV_1; i--) { + if (ver & (1 << i)) { + VMXNET3_WRITE_BAR1_REG(adapter, VMXNET3_REG_VRRS, 1 << i); + adapter->version = i + 1; + break; + } + } + if (i < VMXNET3_REV_1) { dev_err(&pdev->dev, "Incompatible h/w version (0x%x) for adapter\n", ver); err = -EBUSY; @@ -3992,6 +4125,8 @@ vmxnet3_probe_device(struct pci_dev *pdev, } adapter->rqd_start = (struct Vmxnet3_RxQueueDesc *)(adapter->tqd_start + adapter->num_tx_queues); + if (VMXNET3_VERSION_GE_9(adapter)) + adapter->latencyConf = &adapter->tqd_start->tsConf.latencyConf; adapter->pm_conf = dma_alloc_coherent(&adapter->pdev->dev, sizeof(struct Vmxnet3_PMConf), diff --git a/drivers/net/vmxnet3/vmxnet3_ethtool.c b/drivers/net/vmxnet3/vmxnet3_ethtool.c index 7e8008d5378aee43e7ded8e93039b3fbfa2102bc..471f91c4204a1a5883d7147e90cf0f4285518054 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethtool.c +++ b/drivers/net/vmxnet3/vmxnet3_ethtool.c @@ -1,7 +1,7 @@ /* * Linux driver for VMware's vmxnet3 ethernet NIC. * - * Copyright (C) 2008-2022, VMware, Inc. All Rights Reserved. + * Copyright (C) 2008-2024, VMware, Inc. All Rights Reserved. * * This program is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License as published by the diff --git a/drivers/net/vmxnet3/vmxnet3_int.h b/drivers/net/vmxnet3/vmxnet3_int.h index 915aaf18c40943200e86bfe28564a17b4a737a47..9f24d66dbb27971b66bc7ac15083c61f67a0288a 100644 --- a/drivers/net/vmxnet3/vmxnet3_int.h +++ b/drivers/net/vmxnet3/vmxnet3_int.h @@ -1,7 +1,7 @@ /* * Linux driver for VMware's vmxnet3 ethernet NIC. * - * Copyright (C) 2008-2022, VMware, Inc. All Rights Reserved. + * Copyright (C) 2008-2024, VMware, Inc. All Rights Reserved. * * This program is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License as published by the @@ -72,18 +72,20 @@ /* * Version numbers */ -#define VMXNET3_DRIVER_VERSION_STRING "1.7.0.0-k" +#define VMXNET3_DRIVER_VERSION_STRING "1.9.0.0-k" /* Each byte of this 32-bit integer encodes a version number in * VMXNET3_DRIVER_VERSION_STRING. */ -#define VMXNET3_DRIVER_VERSION_NUM 0x01070000 +#define VMXNET3_DRIVER_VERSION_NUM 0x01090000 #if defined(CONFIG_PCI_MSI) /* RSS only makes sense if MSI-X is supported. */ #define VMXNET3_RSS #endif +#define VMXNET3_REV_9 8 /* Vmxnet3 Rev. 9 */ +#define VMXNET3_REV_8 7 /* Vmxnet3 Rev. 8 */ #define VMXNET3_REV_7 6 /* Vmxnet3 Rev. 7 */ #define VMXNET3_REV_6 5 /* Vmxnet3 Rev. 6 */ #define VMXNET3_REV_5 4 /* Vmxnet3 Rev. 5 */ @@ -191,6 +193,11 @@ struct vmxnet3_tx_data_ring { dma_addr_t basePA; }; +struct vmxnet3_tx_ts_ring { + struct Vmxnet3_TxTSDesc *base; + dma_addr_t basePA; +}; + #define VMXNET3_MAP_NONE 0 #define VMXNET3_MAP_SINGLE BIT(0) #define VMXNET3_MAP_PAGE BIT(1) @@ -243,6 +250,7 @@ struct vmxnet3_tx_ctx { u32 copy_size; /* # of bytes copied into the data ring */ union Vmxnet3_GenericDesc *sop_txd; union Vmxnet3_GenericDesc *eop_txd; + struct Vmxnet3_TxTSDesc *ts_txd; }; struct vmxnet3_tx_queue { @@ -252,6 +260,7 @@ struct vmxnet3_tx_queue { struct vmxnet3_cmd_ring tx_ring; struct vmxnet3_tx_buf_info *buf_info; struct vmxnet3_tx_data_ring data_ring; + struct vmxnet3_tx_ts_ring ts_ring; struct vmxnet3_comp_ring comp_ring; struct Vmxnet3_TxQueueCtrl *shared; struct vmxnet3_tq_driver_stats stats; @@ -260,6 +269,8 @@ struct vmxnet3_tx_queue { * stopped */ int qid; u16 txdata_desc_size; + u16 tx_ts_desc_size; + u16 tsPktCount; } ____cacheline_aligned; enum vmxnet3_rx_buf_type { @@ -307,6 +318,11 @@ struct vmxnet3_rx_data_ring { u16 desc_size; }; +struct vmxnet3_rx_ts_ring { + struct Vmxnet3_RxTSDesc *base; + dma_addr_t basePA; +}; + struct vmxnet3_rx_queue { char name[IFNAMSIZ + 8]; /* To identify interrupt */ struct vmxnet3_adapter *adapter; @@ -314,6 +330,7 @@ struct vmxnet3_rx_queue { struct vmxnet3_cmd_ring rx_ring[2]; struct vmxnet3_rx_data_ring data_ring; struct vmxnet3_comp_ring comp_ring; + struct vmxnet3_rx_ts_ring ts_ring; struct vmxnet3_rx_ctx rx_ctx; u32 qid; /* rqID in RCD for buffer from 1st ring */ u32 qid2; /* rqID in RCD for buffer from 2nd ring */ @@ -323,6 +340,7 @@ struct vmxnet3_rx_queue { struct vmxnet3_rq_driver_stats stats; struct page_pool *page_pool; struct xdp_rxq_info xdp_rxq; + u16 rx_ts_desc_size; } ____cacheline_aligned; #define VMXNET3_DEVICE_MAX_TX_QUEUES 32 @@ -432,6 +450,11 @@ struct vmxnet3_adapter { u16 rx_prod_offset; u16 rx_prod2_offset; struct bpf_prog __rcu *xdp_bpf_prog; + struct Vmxnet3_LatencyConf *latencyConf; + /* Size of buffer in the ts ring */ + u16 tx_ts_desc_size; + u16 rx_ts_desc_size; + u32 disabledOffloads; }; #define VMXNET3_WRITE_BAR0_REG(adapter, reg, val) \ @@ -463,6 +486,10 @@ struct vmxnet3_adapter { (adapter->version >= VMXNET3_REV_6 + 1) #define VMXNET3_VERSION_GE_7(adapter) \ (adapter->version >= VMXNET3_REV_7 + 1) +#define VMXNET3_VERSION_GE_8(adapter) \ + (adapter->version >= VMXNET3_REV_8 + 1) +#define VMXNET3_VERSION_GE_9(adapter) \ + (adapter->version >= VMXNET3_REV_9 + 1) /* must be a multiple of VMXNET3_RING_SIZE_ALIGN */ #define VMXNET3_DEF_TX_RING_SIZE 512 diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c index f78dd0438843b151d15d343d6d2e713d57ab1b3a..567cb3faab709c43ae7deff0fd5a155327899e12 100644 --- a/drivers/net/vxlan/vxlan_core.c +++ b/drivers/net/vxlan/vxlan_core.c @@ -1446,6 +1446,10 @@ static bool vxlan_snoop(struct net_device *dev, struct vxlan_fdb *f; u32 ifindex = 0; + /* Ignore packets from invalid src-address */ + if (!is_valid_ether_addr(src_mac)) + return true; + #if IS_ENABLED(CONFIG_IPV6) if (src_ip->sa.sa_family == AF_INET6 && (ipv6_addr_type(&src_ip->sin6.sin6_addr) & IPV6_ADDR_LINKLOCAL)) @@ -1616,10 +1620,6 @@ static bool vxlan_set_mac(struct vxlan_dev *vxlan, if (ether_addr_equal(eth_hdr(skb)->h_source, vxlan->dev->dev_addr)) return false; - /* Ignore packets from invalid src-address */ - if (!is_valid_ether_addr(eth_hdr(skb)->h_source)) - return false; - /* Get address from the outer IP header */ if (vxlan_get_sk_family(vs) == AF_INET) { saddr.sin.sin_addr.s_addr = ip_hdr(skb)->saddr; diff --git a/drivers/net/wireless/ath/ath10k/Kconfig b/drivers/net/wireless/ath/ath10k/Kconfig index e6ea884cafc190ac1bcbbf15d80823896e259572..876aed76583318724e4dab0b537d5c93c77460c6 100644 --- a/drivers/net/wireless/ath/ath10k/Kconfig +++ b/drivers/net/wireless/ath/ath10k/Kconfig @@ -45,6 +45,7 @@ config ATH10K_SNOC depends on ATH10K depends on ARCH_QCOM || COMPILE_TEST depends on QCOM_SMEM + depends on QCOM_RPROC_COMMON || QCOM_RPROC_COMMON=n select QCOM_SCM select QCOM_QMI_HELPERS help @@ -67,6 +68,12 @@ config ATH10K_DEBUGFS If unsure, say Y to make it easier to debug problems. +config ATH10K_LEDS + bool + depends on ATH10K + depends on LEDS_CLASS=y || LEDS_CLASS=MAC80211 + default y + config ATH10K_SPECTRAL bool "Atheros ath10k spectral scan support" depends on ATH10K_DEBUGFS diff --git a/drivers/net/wireless/ath/ath10k/Makefile b/drivers/net/wireless/ath/ath10k/Makefile index 142c777b287f42aff6537b1d7a5938d6c5753495..02bf9b629038d328a632e848c21d25d32d55531f 100644 --- a/drivers/net/wireless/ath/ath10k/Makefile +++ b/drivers/net/wireless/ath/ath10k/Makefile @@ -19,6 +19,7 @@ ath10k_core-$(CONFIG_ATH10K_SPECTRAL) += spectral.o ath10k_core-$(CONFIG_NL80211_TESTMODE) += testmode.o ath10k_core-$(CONFIG_ATH10K_TRACING) += trace.o ath10k_core-$(CONFIG_THERMAL) += thermal.o +ath10k_core-$(CONFIG_ATH10K_LEDS) += leds.o ath10k_core-$(CONFIG_MAC80211_DEBUGFS) += debugfs_sta.o ath10k_core-$(CONFIG_PM) += wow.o ath10k_core-$(CONFIG_DEV_COREDUMP) += coredump.o diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c index bdf0552cd1c3094a92dc2655dc0a832fce5193a9..b3294287bce1f7785231d3bf0f90cb15806937b2 100644 --- a/drivers/net/wireless/ath/ath10k/core.c +++ b/drivers/net/wireless/ath/ath10k/core.c @@ -27,6 +27,7 @@ #include "testmode.h" #include "wmi-ops.h" #include "coredump.h" +#include "leds.h" unsigned int ath10k_debug_mask; EXPORT_SYMBOL(ath10k_debug_mask); @@ -68,6 +69,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca988x hw2.0", .patch_load_addr = QCA988X_HW_2_0_PATCH_LOAD_ADDR, .uart_pin = 7, + .led_pin = 1, .cc_wraparound_type = ATH10K_HW_CC_WRAP_SHIFTED_ALL, .otp_exe_param = 0, .channel_counters_freq_hz = 88000, @@ -108,6 +110,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca988x hw2.0 ubiquiti", .patch_load_addr = QCA988X_HW_2_0_PATCH_LOAD_ADDR, .uart_pin = 7, + .led_pin = 0, .cc_wraparound_type = ATH10K_HW_CC_WRAP_SHIFTED_ALL, .otp_exe_param = 0, .channel_counters_freq_hz = 88000, @@ -149,6 +152,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca9887 hw1.0", .patch_load_addr = QCA9887_HW_1_0_PATCH_LOAD_ADDR, .uart_pin = 7, + .led_pin = 1, .cc_wraparound_type = ATH10K_HW_CC_WRAP_SHIFTED_ALL, .otp_exe_param = 0, .channel_counters_freq_hz = 88000, @@ -190,6 +194,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca6174 hw3.2 sdio", .patch_load_addr = QCA6174_HW_3_0_PATCH_LOAD_ADDR, .uart_pin = 19, + .led_pin = 0, .otp_exe_param = 0, .channel_counters_freq_hz = 88000, .max_probe_resp_desc_thres = 0, @@ -226,6 +231,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca6164 hw2.1", .patch_load_addr = QCA6174_HW_2_1_PATCH_LOAD_ADDR, .uart_pin = 6, + .led_pin = 0, .otp_exe_param = 0, .channel_counters_freq_hz = 88000, .max_probe_resp_desc_thres = 0, @@ -266,6 +272,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca6174 hw2.1", .patch_load_addr = QCA6174_HW_2_1_PATCH_LOAD_ADDR, .uart_pin = 6, + .led_pin = 0, .otp_exe_param = 0, .channel_counters_freq_hz = 88000, .max_probe_resp_desc_thres = 0, @@ -306,6 +313,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca6174 hw3.0", .patch_load_addr = QCA6174_HW_3_0_PATCH_LOAD_ADDR, .uart_pin = 6, + .led_pin = 0, .otp_exe_param = 0, .channel_counters_freq_hz = 88000, .max_probe_resp_desc_thres = 0, @@ -346,6 +354,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca6174 hw3.2", .patch_load_addr = QCA6174_HW_3_0_PATCH_LOAD_ADDR, .uart_pin = 6, + .led_pin = 0, .otp_exe_param = 0, .channel_counters_freq_hz = 88000, .max_probe_resp_desc_thres = 0, @@ -390,6 +399,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca99x0 hw2.0", .patch_load_addr = QCA99X0_HW_2_0_PATCH_LOAD_ADDR, .uart_pin = 7, + .led_pin = 17, .otp_exe_param = 0x00000700, .continuous_frag_desc = true, .cck_rate_map_rev2 = true, @@ -436,6 +446,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca9984/qca9994 hw1.0", .patch_load_addr = QCA9984_HW_1_0_PATCH_LOAD_ADDR, .uart_pin = 7, + .led_pin = 17, .cc_wraparound_type = ATH10K_HW_CC_WRAP_SHIFTED_EACH, .otp_exe_param = 0x00000700, .continuous_frag_desc = true, @@ -488,6 +499,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca9888 hw2.0", .patch_load_addr = QCA9888_HW_2_0_PATCH_LOAD_ADDR, .uart_pin = 7, + .led_pin = 17, .cc_wraparound_type = ATH10K_HW_CC_WRAP_SHIFTED_EACH, .otp_exe_param = 0x00000700, .continuous_frag_desc = true, @@ -538,6 +550,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca9377 hw1.0", .patch_load_addr = QCA9377_HW_1_0_PATCH_LOAD_ADDR, .uart_pin = 6, + .led_pin = 0, .otp_exe_param = 0, .channel_counters_freq_hz = 88000, .max_probe_resp_desc_thres = 0, @@ -578,6 +591,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca9377 hw1.1", .patch_load_addr = QCA9377_HW_1_0_PATCH_LOAD_ADDR, .uart_pin = 6, + .led_pin = 0, .otp_exe_param = 0, .channel_counters_freq_hz = 88000, .max_probe_resp_desc_thres = 0, @@ -620,6 +634,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca9377 hw1.1 sdio", .patch_load_addr = QCA9377_HW_1_0_PATCH_LOAD_ADDR, .uart_pin = 19, + .led_pin = 0, .otp_exe_param = 0, .channel_counters_freq_hz = 88000, .max_probe_resp_desc_thres = 0, @@ -653,6 +668,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .name = "qca4019 hw1.0", .patch_load_addr = QCA4019_HW_1_0_PATCH_LOAD_ADDR, .uart_pin = 7, + .led_pin = 0, .cc_wraparound_type = ATH10K_HW_CC_WRAP_SHIFTED_EACH, .otp_exe_param = 0x0010000, .continuous_frag_desc = true, @@ -698,6 +714,7 @@ static const struct ath10k_hw_params ath10k_hw_params_list[] = { .dev_id = 0, .bus = ATH10K_BUS_SNOC, .name = "wcn3990 hw1.0", + .led_pin = 0, .continuous_frag_desc = true, .tx_chain_mask = 0x7, .rx_chain_mask = 0x7, @@ -3224,6 +3241,10 @@ int ath10k_core_start(struct ath10k *ar, enum ath10k_firmware_mode mode, goto err_hif_stop; } + status = ath10k_leds_start(ar); + if (status) + goto err_hif_stop; + return 0; err_hif_stop: @@ -3482,9 +3503,18 @@ static void ath10k_core_register_work(struct work_struct *work) goto err_spectral_destroy; } + status = ath10k_leds_register(ar); + if (status) { + ath10k_err(ar, "could not register leds: %d\n", + status); + goto err_thermal_unregister; + } + set_bit(ATH10K_FLAG_CORE_REGISTERED, &ar->dev_flags); return; +err_thermal_unregister: + ath10k_thermal_unregister(ar); err_spectral_destroy: ath10k_spectral_destroy(ar); err_debug_destroy: @@ -3520,6 +3550,8 @@ void ath10k_core_unregister(struct ath10k *ar) if (!test_bit(ATH10K_FLAG_CORE_REGISTERED, &ar->dev_flags)) return; + ath10k_leds_unregister(ar); + ath10k_thermal_unregister(ar); /* Stop spectral before unregistering from mac80211 to remove the * relayfs debugfs file cleanly. Otherwise the parent debugfs tree diff --git a/drivers/net/wireless/ath/ath10k/core.h b/drivers/net/wireless/ath/ath10k/core.h index b00099f0b24e74cb2bc584d46ee6c9a7604c7364..446dca74f06a63ebd5db7c21053da3b9530dc030 100644 --- a/drivers/net/wireless/ath/ath10k/core.h +++ b/drivers/net/wireless/ath/ath10k/core.h @@ -15,6 +15,7 @@ #include #include #include +#include #include "htt.h" #include "htc.h" @@ -1258,6 +1259,13 @@ struct ath10k { bool utf_monitor; } testmode; + struct { + struct gpio_led wifi_led; + struct led_classdev cdev; + char label[48]; + u32 gpio_state_pin; + } leds; + struct { /* protected by data_lock */ u32 rx_crc_err_drop; diff --git a/drivers/net/wireless/ath/ath10k/hw.h b/drivers/net/wireless/ath/ath10k/hw.h index 48897e5eca06a51455fd4d4e016caead007b0b2f..442091c6dfd2028510c41ac8bce3e305c4caef2f 100644 --- a/drivers/net/wireless/ath/ath10k/hw.h +++ b/drivers/net/wireless/ath/ath10k/hw.h @@ -512,6 +512,7 @@ struct ath10k_hw_params { const char *name; u32 patch_load_addr; int uart_pin; + int led_pin; u32 otp_exe_param; /* Type of hw cycle counter wraparound logic, for more info diff --git a/drivers/net/wireless/ath/ath10k/leds.c b/drivers/net/wireless/ath/ath10k/leds.c new file mode 100644 index 0000000000000000000000000000000000000000..9b1d04eb4265bd3dd0bc73df2eed57ffbeaeef3c --- /dev/null +++ b/drivers/net/wireless/ath/ath10k/leds.c @@ -0,0 +1,90 @@ +// SPDX-License-Identifier: ISC +/* + * Copyright (c) 2005-2011 Atheros Communications Inc. + * Copyright (c) 2011-2017 Qualcomm Atheros, Inc. + * Copyright (c) 2018 Sebastian Gottschall + * Copyright (c) 2018 The Linux Foundation. All rights reserved. + */ + +#include + +#include "core.h" +#include "wmi.h" +#include "wmi-ops.h" + +#include "leds.h" + +static int ath10k_leds_set_brightness_blocking(struct led_classdev *led_cdev, + enum led_brightness brightness) +{ + struct ath10k *ar = container_of(led_cdev, struct ath10k, + leds.cdev); + struct gpio_led *led = &ar->leds.wifi_led; + + mutex_lock(&ar->conf_mutex); + + if (ar->state != ATH10K_STATE_ON) + goto out; + + ar->leds.gpio_state_pin = (brightness != LED_OFF) ^ led->active_low; + ath10k_wmi_gpio_output(ar, led->gpio, ar->leds.gpio_state_pin); + +out: + mutex_unlock(&ar->conf_mutex); + + return 0; +} + +int ath10k_leds_start(struct ath10k *ar) +{ + if (ar->hw_params.led_pin == 0) + /* leds not supported */ + return 0; + + /* under some circumstances, the gpio pin gets reconfigured + * to default state by the firmware, so we need to + * reconfigure it this behaviour has only ben seen on + * QCA9984 and QCA99XX devices so far + */ + ath10k_wmi_gpio_config(ar, ar->hw_params.led_pin, 0, + WMI_GPIO_PULL_NONE, WMI_GPIO_INTTYPE_DISABLE); + ath10k_wmi_gpio_output(ar, ar->hw_params.led_pin, 1); + + return 0; +} + +int ath10k_leds_register(struct ath10k *ar) +{ + int ret; + + if (ar->hw_params.led_pin == 0) + /* leds not supported */ + return 0; + + snprintf(ar->leds.label, sizeof(ar->leds.label), "ath10k-%s", + wiphy_name(ar->hw->wiphy)); + ar->leds.wifi_led.active_low = 1; + ar->leds.wifi_led.gpio = ar->hw_params.led_pin; + ar->leds.wifi_led.name = ar->leds.label; + ar->leds.wifi_led.default_state = LEDS_GPIO_DEFSTATE_KEEP; + + ar->leds.cdev.name = ar->leds.label; + ar->leds.cdev.brightness_set_blocking = ath10k_leds_set_brightness_blocking; + ar->leds.cdev.default_trigger = ar->leds.wifi_led.default_trigger; + + ret = led_classdev_register(wiphy_dev(ar->hw->wiphy), &ar->leds.cdev); + if (ret) + return ret; + + return 0; +} + +void ath10k_leds_unregister(struct ath10k *ar) +{ + if (ar->hw_params.led_pin == 0) + /* leds not supported */ + return; + + led_classdev_unregister(&ar->leds.cdev); +} + diff --git a/drivers/net/wireless/ath/ath10k/leds.h b/drivers/net/wireless/ath/ath10k/leds.h new file mode 100644 index 0000000000000000000000000000000000000000..56325b0875e5ca4ae51f923b96478a90edd62c29 --- /dev/null +++ b/drivers/net/wireless/ath/ath10k/leds.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: ISC */ +/* + * Copyright (c) 2005-2011 Atheros Communications Inc. + * Copyright (c) 2011-2017 Qualcomm Atheros, Inc. + * Copyright (c) 2018 Sebastian Gottschall + * Copyright (c) 2018 The Linux Foundation. All rights reserved. + */ + +#ifndef _LEDS_H_ +#define _LEDS_H_ + +#include "core.h" + +#ifdef CONFIG_ATH10K_LEDS +void ath10k_leds_unregister(struct ath10k *ar); +int ath10k_leds_start(struct ath10k *ar); +int ath10k_leds_register(struct ath10k *ar); +#else +static inline void ath10k_leds_unregister(struct ath10k *ar) +{ +} + +static inline int ath10k_leds_start(struct ath10k *ar) +{ + return 0; +} + +static inline int ath10k_leds_register(struct ath10k *ar) +{ + return 0; +} + +#endif +#endif /* _LEDS_H_ */ diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c index e322b528baafce39127cf9ea8decc681f5604e26..3bf67b2ecd6dd32c92b7af9029f2bc3dbf1a1788 100644 --- a/drivers/net/wireless/ath/ath10k/mac.c +++ b/drivers/net/wireless/ath/ath10k/mac.c @@ -25,6 +25,7 @@ #include "wmi-tlv.h" #include "wmi-ops.h" #include "wow.h" +#include "leds.h" /*********/ /* Rates */ diff --git a/drivers/net/wireless/ath/ath10k/qmi.c b/drivers/net/wireless/ath/ath10k/qmi.c index 38e939f572a9ed5aeafee612838ba04f25e7c006..f1f33af0170a08744ae658130e29781c0e5bedfb 100644 --- a/drivers/net/wireless/ath/ath10k/qmi.c +++ b/drivers/net/wireless/ath/ath10k/qmi.c @@ -1040,6 +1040,10 @@ static void ath10k_qmi_driver_event_work(struct work_struct *work) switch (event->type) { case ATH10K_QMI_EVENT_SERVER_ARRIVE: ath10k_qmi_event_server_arrive(qmi); + if (qmi->no_msa_ready_indicator) { + ath10k_info(ar, "qmi not waiting for msa_ready indicator"); + ath10k_qmi_event_msa_ready(qmi); + } break; case ATH10K_QMI_EVENT_SERVER_EXIT: ath10k_qmi_event_server_exit(qmi); @@ -1048,6 +1052,10 @@ static void ath10k_qmi_driver_event_work(struct work_struct *work) ath10k_qmi_event_fw_ready_ind(qmi); break; case ATH10K_QMI_EVENT_MSA_READY_IND: + if (qmi->no_msa_ready_indicator) { + ath10k_warn(ar, "qmi unexpected msa_ready indicator"); + break; + } ath10k_qmi_event_msa_ready(qmi); break; default: @@ -1077,6 +1085,9 @@ int ath10k_qmi_init(struct ath10k *ar, u32 msa_size) if (of_property_read_bool(dev->of_node, "qcom,msa-fixed-perm")) qmi->msa_fixed_perm = true; + if (of_property_read_bool(dev->of_node, "qcom,no-msa-ready-indicator")) + qmi->no_msa_ready_indicator = true; + ret = qmi_handle_init(&qmi->qmi_hdl, WLFW_BDF_DOWNLOAD_REQ_MSG_V01_MAX_MSG_LEN, &ath10k_qmi_ops, qmi_msg_handler); diff --git a/drivers/net/wireless/ath/ath10k/qmi.h b/drivers/net/wireless/ath/ath10k/qmi.h index 89464239fe96aa8d53c5434bd9b3e82dd2033ba1..0816eb4e4a18a3f3641eb64d11a4db2a36ad7616 100644 --- a/drivers/net/wireless/ath/ath10k/qmi.h +++ b/drivers/net/wireless/ath/ath10k/qmi.h @@ -107,6 +107,7 @@ struct ath10k_qmi { char fw_build_timestamp[MAX_TIMESTAMP_LEN + 1]; struct ath10k_qmi_cal_data cal_data[MAX_NUM_CAL_V01]; bool msa_fixed_perm; + bool no_msa_ready_indicator; enum ath10k_qmi_state state; }; diff --git a/drivers/net/wireless/ath/ath10k/wmi-ops.h b/drivers/net/wireless/ath/ath10k/wmi-ops.h index aa57d807491ca1932fd663218af8f3645c4fd26c..f3f6b5954b270dbc5c03cb8474981695a6c8f885 100644 --- a/drivers/net/wireless/ath/ath10k/wmi-ops.h +++ b/drivers/net/wireless/ath/ath10k/wmi-ops.h @@ -226,7 +226,10 @@ struct wmi_ops { const struct wmi_bb_timing_cfg_arg *arg); struct sk_buff *(*gen_per_peer_per_tid_cfg)(struct ath10k *ar, const struct wmi_per_peer_per_tid_cfg_arg *arg); + struct sk_buff *(*gen_gpio_config)(struct ath10k *ar, u32 gpio_num, + u32 input, u32 pull_type, u32 intr_mode); + struct sk_buff *(*gen_gpio_output)(struct ath10k *ar, u32 gpio_num, u32 set); }; int ath10k_wmi_cmd_send(struct ath10k *ar, struct sk_buff *skb, u32 cmd_id); @@ -1122,6 +1125,35 @@ ath10k_wmi_force_fw_hang(struct ath10k *ar, return ath10k_wmi_cmd_send(ar, skb, ar->wmi.cmd->force_fw_hang_cmdid); } +static inline int ath10k_wmi_gpio_config(struct ath10k *ar, u32 gpio_num, + u32 input, u32 pull_type, u32 intr_mode) +{ + struct sk_buff *skb; + + if (!ar->wmi.ops->gen_gpio_config) + return -EOPNOTSUPP; + + skb = ar->wmi.ops->gen_gpio_config(ar, gpio_num, input, pull_type, intr_mode); + if (IS_ERR(skb)) + return PTR_ERR(skb); + + return ath10k_wmi_cmd_send(ar, skb, ar->wmi.cmd->gpio_config_cmdid); +} + +static inline int ath10k_wmi_gpio_output(struct ath10k *ar, u32 gpio_num, u32 set) +{ + struct sk_buff *skb; + + if (!ar->wmi.ops->gen_gpio_config) + return -EOPNOTSUPP; + + skb = ar->wmi.ops->gen_gpio_output(ar, gpio_num, set); + if (IS_ERR(skb)) + return PTR_ERR(skb); + + return ath10k_wmi_cmd_send(ar, skb, ar->wmi.cmd->gpio_output_cmdid); +} + static inline int ath10k_wmi_dbglog_cfg(struct ath10k *ar, u64 module_enable, u32 log_level) { diff --git a/drivers/net/wireless/ath/ath10k/wmi-tlv.c b/drivers/net/wireless/ath/ath10k/wmi-tlv.c index aed97fd121babf6813afc0f50707b327310046c0..dbaf26d6a7a61806ac0f7d19ee9d69ead306aecf 100644 --- a/drivers/net/wireless/ath/ath10k/wmi-tlv.c +++ b/drivers/net/wireless/ath/ath10k/wmi-tlv.c @@ -4606,6 +4606,8 @@ static const struct wmi_ops wmi_tlv_ops = { .gen_echo = ath10k_wmi_tlv_op_gen_echo, .gen_vdev_spectral_conf = ath10k_wmi_tlv_op_gen_vdev_spectral_conf, .gen_vdev_spectral_enable = ath10k_wmi_tlv_op_gen_vdev_spectral_enable, + /* .gen_gpio_config not implemented */ + /* .gen_gpio_output not implemented */ }; static const struct wmi_peer_flags_map wmi_tlv_peer_flags_map = { diff --git a/drivers/net/wireless/ath/ath10k/wmi.c b/drivers/net/wireless/ath/ath10k/wmi.c index 80d255aaff1be7595da4c48eb60dbc212479c48a..fe2344598364973eae9916a5368744b70ec6325c 100644 --- a/drivers/net/wireless/ath/ath10k/wmi.c +++ b/drivers/net/wireless/ath/ath10k/wmi.c @@ -7493,6 +7493,49 @@ ath10k_wmi_op_gen_peer_set_param(struct ath10k *ar, u32 vdev_id, return skb; } +static struct sk_buff *ath10k_wmi_op_gen_gpio_config(struct ath10k *ar, + u32 gpio_num, u32 input, + u32 pull_type, u32 intr_mode) +{ + struct wmi_gpio_config_cmd *cmd; + struct sk_buff *skb; + + skb = ath10k_wmi_alloc_skb(ar, sizeof(*cmd)); + if (!skb) + return ERR_PTR(-ENOMEM); + + cmd = (struct wmi_gpio_config_cmd *)skb->data; + cmd->pull_type = __cpu_to_le32(pull_type); + cmd->gpio_num = __cpu_to_le32(gpio_num); + cmd->input = __cpu_to_le32(input); + cmd->intr_mode = __cpu_to_le32(intr_mode); + + ath10k_dbg(ar, ATH10K_DBG_WMI, "wmi gpio_config gpio_num 0x%08x input 0x%08x pull_type 0x%08x intr_mode 0x%08x\n", + gpio_num, input, pull_type, intr_mode); + + return skb; +} + +static struct sk_buff *ath10k_wmi_op_gen_gpio_output(struct ath10k *ar, + u32 gpio_num, u32 set) +{ + struct wmi_gpio_output_cmd *cmd; + struct sk_buff *skb; + + skb = ath10k_wmi_alloc_skb(ar, sizeof(*cmd)); + if (!skb) + return ERR_PTR(-ENOMEM); + + cmd = (struct wmi_gpio_output_cmd *)skb->data; + cmd->gpio_num = __cpu_to_le32(gpio_num); + cmd->set = __cpu_to_le32(set); + + ath10k_dbg(ar, ATH10K_DBG_WMI, "wmi gpio_output gpio_num 0x%08x set 0x%08x\n", + gpio_num, set); + + return skb; +} + static struct sk_buff * ath10k_wmi_op_gen_set_psmode(struct ath10k *ar, u32 vdev_id, enum wmi_sta_ps_mode psmode) @@ -9157,6 +9200,9 @@ static const struct wmi_ops wmi_ops = { .fw_stats_fill = ath10k_wmi_main_op_fw_stats_fill, .get_vdev_subtype = ath10k_wmi_op_get_vdev_subtype, .gen_echo = ath10k_wmi_op_gen_echo, + .gen_gpio_config = ath10k_wmi_op_gen_gpio_config, + .gen_gpio_output = ath10k_wmi_op_gen_gpio_output, + /* .gen_bcn_tmpl not implemented */ /* .gen_prb_tmpl not implemented */ /* .gen_p2p_go_bcn_ie not implemented */ @@ -9227,6 +9273,8 @@ static const struct wmi_ops wmi_10_1_ops = { .fw_stats_fill = ath10k_wmi_10x_op_fw_stats_fill, .get_vdev_subtype = ath10k_wmi_op_get_vdev_subtype, .gen_echo = ath10k_wmi_op_gen_echo, + .gen_gpio_config = ath10k_wmi_op_gen_gpio_config, + .gen_gpio_output = ath10k_wmi_op_gen_gpio_output, /* .gen_bcn_tmpl not implemented */ /* .gen_prb_tmpl not implemented */ /* .gen_p2p_go_bcn_ie not implemented */ @@ -9299,6 +9347,8 @@ static const struct wmi_ops wmi_10_2_ops = { .gen_delba_send = ath10k_wmi_op_gen_delba_send, .fw_stats_fill = ath10k_wmi_10x_op_fw_stats_fill, .get_vdev_subtype = ath10k_wmi_op_get_vdev_subtype, + .gen_gpio_config = ath10k_wmi_op_gen_gpio_config, + .gen_gpio_output = ath10k_wmi_op_gen_gpio_output, /* .gen_pdev_enable_adaptive_cca not implemented */ }; @@ -9370,6 +9420,8 @@ static const struct wmi_ops wmi_10_2_4_ops = { ath10k_wmi_op_gen_pdev_enable_adaptive_cca, .get_vdev_subtype = ath10k_wmi_10_2_4_op_get_vdev_subtype, .gen_bb_timing = ath10k_wmi_10_2_4_op_gen_bb_timing, + .gen_gpio_config = ath10k_wmi_op_gen_gpio_config, + .gen_gpio_output = ath10k_wmi_op_gen_gpio_output, /* .gen_bcn_tmpl not implemented */ /* .gen_prb_tmpl not implemented */ /* .gen_p2p_go_bcn_ie not implemented */ @@ -9451,6 +9503,8 @@ static const struct wmi_ops wmi_10_4_ops = { .gen_pdev_bss_chan_info_req = ath10k_wmi_10_2_op_gen_pdev_bss_chan_info, .gen_echo = ath10k_wmi_op_gen_echo, .gen_pdev_get_tpc_config = ath10k_wmi_10_2_4_op_gen_pdev_get_tpc_config, + .gen_gpio_config = ath10k_wmi_op_gen_gpio_config, + .gen_gpio_output = ath10k_wmi_op_gen_gpio_output, }; int ath10k_wmi_attach(struct ath10k *ar) diff --git a/drivers/net/wireless/ath/ath10k/wmi.h b/drivers/net/wireless/ath/ath10k/wmi.h index 2379501225a4aba43b7629d10eb5c2b18c044d30..0faefc0a9a405978d99b71628bcba8b47ce9e800 100644 --- a/drivers/net/wireless/ath/ath10k/wmi.h +++ b/drivers/net/wireless/ath/ath10k/wmi.h @@ -3034,6 +3034,41 @@ enum wmi_10_4_feature_mask { }; +/* WMI_GPIO_CONFIG_CMDID */ +enum { + WMI_GPIO_PULL_NONE, + WMI_GPIO_PULL_UP, + WMI_GPIO_PULL_DOWN, +}; + +enum { + WMI_GPIO_INTTYPE_DISABLE, + WMI_GPIO_INTTYPE_RISING_EDGE, + WMI_GPIO_INTTYPE_FALLING_EDGE, + WMI_GPIO_INTTYPE_BOTH_EDGE, + WMI_GPIO_INTTYPE_LEVEL_LOW, + WMI_GPIO_INTTYPE_LEVEL_HIGH +}; + +/* WMI_GPIO_CONFIG_CMDID */ +struct wmi_gpio_config_cmd { + __le32 gpio_num; /* GPIO number to be setup */ + __le32 input; /* 0 - Output/ 1 - Input */ + __le32 pull_type; /* Pull type defined above */ + __le32 intr_mode; /* Interrupt mode defined above (Input) */ +} __packed; + +/* WMI_GPIO_OUTPUT_CMDID */ +struct wmi_gpio_output_cmd { + __le32 gpio_num; /* GPIO number to be setup */ + __le32 set; /* Set the GPIO pin*/ +} __packed; + +/* WMI_GPIO_INPUT_EVENTID */ +struct wmi_gpio_input_event { + __le32 gpio_num; /* GPIO number which changed state */ +} __packed; + struct wmi_ext_resource_config_10_4_cmd { /* contains enum wmi_host_platform_type */ __le32 host_platform_config; diff --git a/drivers/net/wireless/ath/ath11k/ahb.c b/drivers/net/wireless/ath/ath11k/ahb.c index ca0f17ddebbaac749a4b04fa11d72f378ab0a027..e3ff4786c714de41c5514576eff189902b4f680d 100644 --- a/drivers/net/wireless/ath/ath11k/ahb.c +++ b/drivers/net/wireless/ath/ath11k/ahb.c @@ -954,6 +954,36 @@ static int ath11k_ahb_setup_msa_resources(struct ath11k_base *ab) return 0; } +static int ath11k_ahb_ce_remap(struct ath11k_base *ab) +{ + const struct ce_remap *ce_remap = ab->hw_params.ce_remap; + struct platform_device *pdev = ab->pdev; + + if (!ce_remap) { + /* no separate CE register space */ + ab->mem_ce = ab->mem; + return 0; + } + + /* ce register space is moved out of wcss unlike ipq8074 or ipq6018 + * and the space is not contiguous, hence remapping the CE registers + * to a new space for accessing them. + */ + ab->mem_ce = ioremap(ce_remap->base, ce_remap->size); + if (!ab->mem_ce) { + dev_err(&pdev->dev, "ce ioremap error\n"); + return -ENOMEM; + } + + return 0; +} + +static void ath11k_ahb_ce_unmap(struct ath11k_base *ab) +{ + if (ab->hw_params.ce_remap) + iounmap(ab->mem_ce); +} + static int ath11k_ahb_fw_resources_init(struct ath11k_base *ab) { struct ath11k_ahb *ab_ahb = ath11k_ahb_priv(ab); @@ -1146,25 +1176,13 @@ static int ath11k_ahb_probe(struct platform_device *pdev) if (ret) goto err_core_free; - ab->mem_ce = ab->mem; - - if (ab->hw_params.ce_remap) { - const struct ce_remap *ce_remap = ab->hw_params.ce_remap; - /* ce register space is moved out of wcss unlike ipq8074 or ipq6018 - * and the space is not contiguous, hence remapping the CE registers - * to a new space for accessing them. - */ - ab->mem_ce = ioremap(ce_remap->base, ce_remap->size); - if (!ab->mem_ce) { - dev_err(&pdev->dev, "ce ioremap error\n"); - ret = -ENOMEM; - goto err_core_free; - } - } + ret = ath11k_ahb_ce_remap(ab); + if (ret) + goto err_core_free; ret = ath11k_ahb_fw_resources_init(ab); if (ret) - goto err_core_free; + goto err_ce_unmap; ret = ath11k_ahb_setup_smp2p_handle(ab); if (ret) @@ -1216,6 +1234,9 @@ err_release_smp2p_handle: err_fw_deinit: ath11k_ahb_fw_resource_deinit(ab); +err_ce_unmap: + ath11k_ahb_ce_unmap(ab); + err_core_free: ath11k_core_free(ab); platform_set_drvdata(pdev, NULL); @@ -1248,9 +1269,7 @@ static void ath11k_ahb_free_resources(struct ath11k_base *ab) ath11k_ahb_release_smp2p_handle(ab); ath11k_ahb_fw_resource_deinit(ab); ath11k_ce_free_pipes(ab); - - if (ab->hw_params.ce_remap) - iounmap(ab->mem_ce); + ath11k_ahb_ce_unmap(ab); ath11k_core_free(ab); platform_set_drvdata(pdev, NULL); diff --git a/drivers/net/wireless/ath/ath11k/ce.h b/drivers/net/wireless/ath/ath11k/ce.h index 69946fc700777be654e5b2f17bd9f74caa8b0b5c..bcde2fcf02cf786c082ecf4bc37bec4f853f33ff 100644 --- a/drivers/net/wireless/ath/ath11k/ce.h +++ b/drivers/net/wireless/ath/ath11k/ce.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. - * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2022, 2024 Qualcomm Innovation Center, Inc. All rights reserved. */ #ifndef ATH11K_CE_H @@ -146,7 +146,7 @@ struct ath11k_ce_ring { /* Host address space */ void *base_addr_owner_space_unaligned; /* CE address space */ - u32 base_addr_ce_space_unaligned; + dma_addr_t base_addr_ce_space_unaligned; /* Actual start of descriptors. * Aligned to descriptor-size boundary. @@ -156,7 +156,7 @@ struct ath11k_ce_ring { void *base_addr_owner_space; /* CE address space */ - u32 base_addr_ce_space; + dma_addr_t base_addr_ce_space; /* HAL ring id */ u32 hal_ring_id; diff --git a/drivers/net/wireless/ath/ath11k/core.c b/drivers/net/wireless/ath/ath11k/core.c index 3cc817a3b4a4047600591804bbc57aa45cba866f..03187df26000c6ef19f79378022cfbf887b5673e 100644 --- a/drivers/net/wireless/ath/ath11k/core.c +++ b/drivers/net/wireless/ath/ath11k/core.c @@ -62,7 +62,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .ce_ie_addr = &ath11k_ce_ie_addr_ipq8074, .single_pdev_only = false, .rxdma1_enable = true, - .num_rxmda_per_pdev = 1, + .num_rxdma_per_pdev = 1, .rx_mac_buf_ring = false, .vdev_start_delay = false, .htt_peer_map_v2 = true, @@ -148,7 +148,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .ce_ie_addr = &ath11k_ce_ie_addr_ipq8074, .single_pdev_only = false, .rxdma1_enable = true, - .num_rxmda_per_pdev = 1, + .num_rxdma_per_pdev = 1, .rx_mac_buf_ring = false, .vdev_start_delay = false, .htt_peer_map_v2 = true, @@ -232,7 +232,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .ce_ie_addr = &ath11k_ce_ie_addr_ipq8074, .single_pdev_only = true, .rxdma1_enable = false, - .num_rxmda_per_pdev = 2, + .num_rxdma_per_pdev = 2, .rx_mac_buf_ring = true, .vdev_start_delay = true, .htt_peer_map_v2 = false, @@ -320,7 +320,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .svc_to_ce_map_len = 18, .ce_ie_addr = &ath11k_ce_ie_addr_ipq8074, .rxdma1_enable = true, - .num_rxmda_per_pdev = 1, + .num_rxdma_per_pdev = 1, .rx_mac_buf_ring = false, .vdev_start_delay = false, .htt_peer_map_v2 = true, @@ -404,7 +404,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .ce_ie_addr = &ath11k_ce_ie_addr_ipq8074, .single_pdev_only = true, .rxdma1_enable = false, - .num_rxmda_per_pdev = 2, + .num_rxdma_per_pdev = 2, .rx_mac_buf_ring = true, .vdev_start_delay = true, .htt_peer_map_v2 = false, @@ -492,7 +492,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .svc_to_ce_map_len = 14, .single_pdev_only = true, .rxdma1_enable = false, - .num_rxmda_per_pdev = 2, + .num_rxdma_per_pdev = 2, .rx_mac_buf_ring = true, .vdev_start_delay = true, .htt_peer_map_v2 = false, @@ -580,7 +580,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .ce_ie_addr = &ath11k_ce_ie_addr_ipq8074, .single_pdev_only = true, .rxdma1_enable = false, - .num_rxmda_per_pdev = 1, + .num_rxdma_per_pdev = 1, .rx_mac_buf_ring = true, .vdev_start_delay = true, .htt_peer_map_v2 = false, @@ -604,7 +604,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .coldboot_cal_ftm = true, .cbcal_restart_fw = false, .fw_mem_mode = 0, - .num_vdevs = 16 + 1, + .num_vdevs = 3, .num_peers = 512, .supports_suspend = false, .hal_desc_sz = sizeof(struct hal_rx_desc_qcn9074), @@ -673,7 +673,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .ce_ie_addr = &ath11k_ce_ie_addr_ipq5018, .ce_remap = &ath11k_ce_remap_ipq5018, .rxdma1_enable = true, - .num_rxmda_per_pdev = RXDMA_PER_PDEV_5018, + .num_rxdma_per_pdev = RXDMA_PER_PDEV_5018, .rx_mac_buf_ring = false, .vdev_start_delay = false, .htt_peer_map_v2 = true, @@ -744,7 +744,7 @@ static const struct ath11k_hw_params ath11k_hw_params[] = { .ce_ie_addr = &ath11k_ce_ie_addr_ipq8074, .single_pdev_only = true, .rxdma1_enable = false, - .num_rxmda_per_pdev = 2, + .num_rxdma_per_pdev = 2, .rx_mac_buf_ring = true, .vdev_start_delay = true, .htt_peer_map_v2 = false, @@ -1009,6 +1009,16 @@ int ath11k_core_resume(struct ath11k_base *ab) return -ETIMEDOUT; } + if (ab->hw_params.current_cc_support && + ar->alpha2[0] != 0 && ar->alpha2[1] != 0) { + ret = ath11k_reg_set_cc(ar); + if (ret) { + ath11k_warn(ab, "failed to set country code during resume: %d\n", + ret); + return ret; + } + } + ret = ath11k_dp_rx_pktlog_start(ab); if (ret) ath11k_warn(ab, "failed to start rx pktlog during resume: %d\n", @@ -1801,7 +1811,7 @@ static int ath11k_core_start(struct ath11k_base *ab) } /* put hardware to DBS mode */ - if (ab->hw_params.single_pdev_only && ab->hw_params.num_rxmda_per_pdev > 1) { + if (ab->hw_params.single_pdev_only && ab->hw_params.num_rxdma_per_pdev > 1) { ret = ath11k_wmi_set_hw_mode(ab, WMI_HOST_HW_MODE_DBS); if (ret) { ath11k_err(ab, "failed to send dbs mode: %d\n", ret); @@ -1978,23 +1988,20 @@ static void ath11k_update_11d(struct work_struct *work) struct ath11k_base *ab = container_of(work, struct ath11k_base, update_11d_work); struct ath11k *ar; struct ath11k_pdev *pdev; - struct wmi_set_current_country_params set_current_param = {}; int ret, i; - spin_lock_bh(&ab->base_lock); - memcpy(&set_current_param.alpha2, &ab->new_alpha2, 2); - spin_unlock_bh(&ab->base_lock); - - ath11k_dbg(ab, ATH11K_DBG_WMI, "update 11d new cc %c%c\n", - set_current_param.alpha2[0], - set_current_param.alpha2[1]); - for (i = 0; i < ab->num_radios; i++) { pdev = &ab->pdevs[i]; ar = pdev->ar; - memcpy(&ar->alpha2, &set_current_param.alpha2, 2); - ret = ath11k_wmi_send_set_current_country_cmd(ar, &set_current_param); + spin_lock_bh(&ab->base_lock); + memcpy(&ar->alpha2, &ab->new_alpha2, 2); + spin_unlock_bh(&ab->base_lock); + + ath11k_dbg(ab, ATH11K_DBG_WMI, "update 11d new cc %c%c for pdev %d\n", + ar->alpha2[0], ar->alpha2[1], i); + + ret = ath11k_reg_set_cc(ar); if (ret) ath11k_warn(ar->ab, "pdev id %d failed set current country code: %d\n", diff --git a/drivers/net/wireless/ath/ath11k/core.h b/drivers/net/wireless/ath/ath11k/core.h index 205f40ee6b666eca73ec351571245150618db4f4..df24f0e409afb1b17345c14a29f5b7c24edd45c8 100644 --- a/drivers/net/wireless/ath/ath11k/core.h +++ b/drivers/net/wireless/ath/ath11k/core.h @@ -330,6 +330,9 @@ struct ath11k_chan_power_info { s8 tx_power; }; +/* ath11k only deals with 160 MHz, so 8 subchannels */ +#define ATH11K_NUM_PWR_LEVELS 8 + /** * struct ath11k_reg_tpc_power_info - regulatory TPC power info * @is_psd_power: is PSD power or not @@ -346,10 +349,10 @@ struct ath11k_reg_tpc_power_info { u8 eirp_power; enum wmi_reg_6ghz_ap_type ap_power_type; u8 num_pwr_levels; - u8 reg_max[IEEE80211_MAX_NUM_PWR_LEVEL]; + u8 reg_max[ATH11K_NUM_PWR_LEVELS]; u8 ap_constraint_power; - s8 tpe[IEEE80211_MAX_NUM_PWR_LEVEL]; - struct ath11k_chan_power_info chan_power_info[IEEE80211_MAX_NUM_PWR_LEVEL]; + s8 tpe[ATH11K_NUM_PWR_LEVELS]; + struct ath11k_chan_power_info chan_power_info[ATH11K_NUM_PWR_LEVELS]; }; struct ath11k_vif { diff --git a/drivers/net/wireless/ath/ath11k/debugfs.c b/drivers/net/wireless/ath/ath11k/debugfs.c index 414a5ce279f785489bf8a79cbeb9855a1d9cacdb..57281a135dd7fa6b8610636f47873c8bba21053c 100644 --- a/drivers/net/wireless/ath/ath11k/debugfs.c +++ b/drivers/net/wireless/ath/ath11k/debugfs.c @@ -668,7 +668,7 @@ static ssize_t ath11k_write_extd_rx_stats(struct file *file, ar->debug.rx_filter = tlv_filter.rx_filter; - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { ring_id = ar->dp.rx_mon_status_refill_ring[i].refill_buf_ring.ring_id; ret = ath11k_dp_tx_htt_rx_filter_setup(ar->ab, ring_id, ar->dp.mac_id, HAL_RXDMA_MONITOR_STATUS, @@ -1112,7 +1112,7 @@ static ssize_t ath11k_write_pktlog_filter(struct file *file, } /* Clear rx filter set for monitor mode and rx status */ - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { ring_id = ar->dp.rx_mon_status_refill_ring[i].refill_buf_ring.ring_id; ret = ath11k_dp_tx_htt_rx_filter_setup(ar->ab, ring_id, ar->dp.mac_id, HAL_RXDMA_MONITOR_STATUS, @@ -1171,7 +1171,7 @@ static ssize_t ath11k_write_pktlog_filter(struct file *file, HTT_RX_FP_DATA_FILTER_FLASG3; } - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { ring_id = ar->dp.rx_mon_status_refill_ring[i].refill_buf_ring.ring_id; ret = ath11k_dp_tx_htt_rx_filter_setup(ab, ring_id, ar->dp.mac_id + i, diff --git a/drivers/net/wireless/ath/ath11k/dp.c b/drivers/net/wireless/ath/ath11k/dp.c index 1a62407e5a9f23ec882a5c28c869904a055f8d21..fbf666d0ecf1dc36677357d149f19b1320a9605d 100644 --- a/drivers/net/wireless/ath/ath11k/dp.c +++ b/drivers/net/wireless/ath/ath11k/dp.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: BSD-3-Clause-Clear /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2023 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ #include @@ -830,8 +830,8 @@ int ath11k_dp_service_srng(struct ath11k_base *ab, if (ab->hw_params.ring_mask->rx_mon_status[grp_id]) { for (i = 0; i < ab->num_radios; i++) { - for (j = 0; j < ab->hw_params.num_rxmda_per_pdev; j++) { - int id = i * ab->hw_params.num_rxmda_per_pdev + j; + for (j = 0; j < ab->hw_params.num_rxdma_per_pdev; j++) { + int id = i * ab->hw_params.num_rxdma_per_pdev + j; if (ab->hw_params.ring_mask->rx_mon_status[grp_id] & BIT(id)) { @@ -853,8 +853,8 @@ int ath11k_dp_service_srng(struct ath11k_base *ab, ath11k_dp_process_reo_status(ab); for (i = 0; i < ab->num_radios; i++) { - for (j = 0; j < ab->hw_params.num_rxmda_per_pdev; j++) { - int id = i * ab->hw_params.num_rxmda_per_pdev + j; + for (j = 0; j < ab->hw_params.num_rxdma_per_pdev; j++) { + int id = i * ab->hw_params.num_rxdma_per_pdev + j; if (ab->hw_params.ring_mask->rxdma2host[grp_id] & BIT(id)) { work_done = ath11k_dp_process_rxdma_err(ab, id, budget); @@ -913,7 +913,7 @@ void ath11k_dp_pdev_pre_alloc(struct ath11k_base *ab) spin_lock_init(&dp->rx_refill_buf_ring.idr_lock); atomic_set(&dp->num_tx_pending, 0); init_waitqueue_head(&dp->tx_empty_waitq); - for (j = 0; j < ab->hw_params.num_rxmda_per_pdev; j++) { + for (j = 0; j < ab->hw_params.num_rxdma_per_pdev; j++) { idr_init(&dp->rx_mon_status_refill_ring[j].bufs_idr); spin_lock_init(&dp->rx_mon_status_refill_ring[j].idr_lock); } diff --git a/drivers/net/wireless/ath/ath11k/dp_rx.c b/drivers/net/wireless/ath/ath11k/dp_rx.c index afd481f5858f031b6cba2d417dfa6c38775ce2e9..198fb359a688c5248768e2db5e6a5a76a387c665 100644 --- a/drivers/net/wireless/ath/ath11k/dp_rx.c +++ b/drivers/net/wireless/ath/ath11k/dp_rx.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: BSD-3-Clause-Clear /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2023 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ #include @@ -311,7 +311,7 @@ static void ath11k_dp_service_mon_ring(struct timer_list *t) struct ath11k_base *ab = from_timer(ab, t, mon_reap_timer); int i; - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) ath11k_dp_rx_process_mon_rings(ab, i, NULL, DP_MON_SERVICE_BUDGET); mod_timer(&ab->mon_reap_timer, jiffies + @@ -324,7 +324,7 @@ static int ath11k_dp_purge_mon_ring(struct ath11k_base *ab) unsigned long timeout = jiffies + msecs_to_jiffies(DP_MON_PURGE_TIMEOUT_MS); do { - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) reaped += ath11k_dp_rx_process_mon_rings(ab, i, NULL, DP_MON_SERVICE_BUDGET); @@ -468,7 +468,7 @@ static int ath11k_dp_rxdma_pdev_buf_free(struct ath11k *ar) rx_ring = &dp->rxdma_mon_buf_ring; ath11k_dp_rxdma_buf_ring_free(ar, rx_ring); - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { rx_ring = &dp->rx_mon_status_refill_ring[i]; ath11k_dp_rxdma_buf_ring_free(ar, rx_ring); } @@ -506,7 +506,7 @@ static int ath11k_dp_rxdma_pdev_buf_setup(struct ath11k *ar) ath11k_dp_rxdma_ring_buf_setup(ar, rx_ring, HAL_RXDMA_MONITOR_BUF); } - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { rx_ring = &dp->rx_mon_status_refill_ring[i]; ath11k_dp_rxdma_ring_buf_setup(ar, rx_ring, HAL_RXDMA_MONITOR_STATUS); } @@ -522,7 +522,7 @@ static void ath11k_dp_rx_pdev_srng_free(struct ath11k *ar) ath11k_dp_srng_cleanup(ab, &dp->rx_refill_buf_ring.refill_buf_ring); - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { if (ab->hw_params.rx_mac_buf_ring) ath11k_dp_srng_cleanup(ab, &dp->rx_mac_buf_ring[i]); @@ -585,7 +585,7 @@ static int ath11k_dp_rx_pdev_srng_alloc(struct ath11k *ar) } if (ar->ab->hw_params.rx_mac_buf_ring) { - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { ret = ath11k_dp_srng_setup(ar->ab, &dp->rx_mac_buf_ring[i], HAL_RXDMA_BUF, 1, @@ -598,7 +598,7 @@ static int ath11k_dp_rx_pdev_srng_alloc(struct ath11k *ar) } } - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { ret = ath11k_dp_srng_setup(ar->ab, &dp->rxdma_err_dst_ring[i], HAL_RXDMA_DST, 0, dp->mac_id + i, DP_RXDMA_ERR_DST_RING_SIZE); @@ -608,7 +608,7 @@ static int ath11k_dp_rx_pdev_srng_alloc(struct ath11k *ar) } } - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { srng = &dp->rx_mon_status_refill_ring[i].refill_buf_ring; ret = ath11k_dp_srng_setup(ar->ab, srng, @@ -2990,11 +2990,52 @@ ath11k_dp_rx_mon_update_status_buf_state(struct ath11k_mon_data *pmon, } } +static enum dp_mon_status_buf_state +ath11k_dp_rx_mon_buf_done(struct ath11k_base *ab, struct hal_srng *srng, + struct dp_rxdma_ring *rx_ring) +{ + struct ath11k_skb_rxcb *rxcb; + struct hal_tlv_hdr *tlv; + struct sk_buff *skb; + void *status_desc; + dma_addr_t paddr; + u32 cookie; + int buf_id; + u8 rbm; + + status_desc = ath11k_hal_srng_src_next_peek(ab, srng); + if (!status_desc) + return DP_MON_STATUS_NO_DMA; + + ath11k_hal_rx_buf_addr_info_get(status_desc, &paddr, &cookie, &rbm); + + buf_id = FIELD_GET(DP_RXDMA_BUF_COOKIE_BUF_ID, cookie); + + spin_lock_bh(&rx_ring->idr_lock); + skb = idr_find(&rx_ring->bufs_idr, buf_id); + spin_unlock_bh(&rx_ring->idr_lock); + + if (!skb) + return DP_MON_STATUS_NO_DMA; + + rxcb = ATH11K_SKB_RXCB(skb); + dma_sync_single_for_cpu(ab->dev, rxcb->paddr, + skb->len + skb_tailroom(skb), + DMA_FROM_DEVICE); + + tlv = (struct hal_tlv_hdr *)skb->data; + if (FIELD_GET(HAL_TLV_HDR_TAG, tlv->tl) != HAL_RX_STATUS_BUFFER_DONE) + return DP_MON_STATUS_NO_DMA; + + return DP_MON_STATUS_REPLINISH; +} + static int ath11k_dp_rx_reap_mon_status_ring(struct ath11k_base *ab, int mac_id, int *budget, struct sk_buff_head *skb_list) { struct ath11k *ar; const struct ath11k_hw_hal_params *hal_params; + enum dp_mon_status_buf_state reap_status; struct ath11k_pdev_dp *dp; struct dp_rxdma_ring *rx_ring; struct ath11k_mon_data *pmon; @@ -3057,15 +3098,38 @@ static int ath11k_dp_rx_reap_mon_status_ring(struct ath11k_base *ab, int mac_id, ath11k_warn(ab, "mon status DONE not set %lx, buf_id %d\n", FIELD_GET(HAL_TLV_HDR_TAG, tlv->tl), buf_id); - /* If done status is missing, hold onto status - * ring until status is done for this status - * ring buffer. - * Keep HP in mon_status_ring unchanged, - * and break from here. - * Check status for same buffer for next time + /* RxDMA status done bit might not be set even + * though tp is moved by HW. */ - pmon->buf_state = DP_MON_STATUS_NO_DMA; - break; + + /* If done status is missing: + * 1. As per MAC team's suggestion, + * when HP + 1 entry is peeked and if DMA + * is not done and if HP + 2 entry's DMA done + * is set. skip HP + 1 entry and + * start processing in next interrupt. + * 2. If HP + 2 entry's DMA done is not set, + * poll onto HP + 1 entry DMA done to be set. + * Check status for same buffer for next time + * dp_rx_mon_status_srng_process + */ + + reap_status = ath11k_dp_rx_mon_buf_done(ab, srng, + rx_ring); + if (reap_status == DP_MON_STATUS_NO_DMA) + continue; + + spin_lock_bh(&rx_ring->idr_lock); + idr_remove(&rx_ring->bufs_idr, buf_id); + spin_unlock_bh(&rx_ring->idr_lock); + + dma_unmap_single(ab->dev, rxcb->paddr, + skb->len + skb_tailroom(skb), + DMA_FROM_DEVICE); + + dev_kfree_skb_any(skb); + pmon->buf_state = DP_MON_STATUS_REPLINISH; + goto move_next; } spin_lock_bh(&rx_ring->idr_lock); @@ -4391,7 +4455,7 @@ int ath11k_dp_rx_pdev_alloc(struct ath11k_base *ab, int mac_id) } if (ab->hw_params.rx_mac_buf_ring) { - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { ring_id = dp->rx_mac_buf_ring[i].ring_id; ret = ath11k_dp_tx_htt_srng_setup(ab, ring_id, mac_id + i, HAL_RXDMA_BUF); @@ -4403,7 +4467,7 @@ int ath11k_dp_rx_pdev_alloc(struct ath11k_base *ab, int mac_id) } } - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { ring_id = dp->rxdma_err_dst_ring[i].ring_id; ret = ath11k_dp_tx_htt_srng_setup(ab, ring_id, mac_id + i, HAL_RXDMA_DST); @@ -4443,7 +4507,7 @@ int ath11k_dp_rx_pdev_alloc(struct ath11k_base *ab, int mac_id) } config_refill_ring: - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { ring_id = dp->rx_mon_status_refill_ring[i].refill_buf_ring.ring_id; ret = ath11k_dp_tx_htt_srng_setup(ab, ring_id, mac_id + i, HAL_RXDMA_MONITOR_STATUS); diff --git a/drivers/net/wireless/ath/ath11k/dp_tx.c b/drivers/net/wireless/ath/ath11k/dp_tx.c index 272b1c35f98df2db0afbc9ff90b90796cfd289a9..21819b74170104aedc8c6b670de3cc83da093529 100644 --- a/drivers/net/wireless/ath/ath11k/dp_tx.c +++ b/drivers/net/wireless/ath/ath11k/dp_tx.c @@ -1035,7 +1035,7 @@ int ath11k_dp_tx_htt_h2t_ppdu_stats_req(struct ath11k *ar, u32 mask) int ret; int i; - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { skb = ath11k_htc_alloc_skb(ab, len); if (!skb) return -ENOMEM; @@ -1218,7 +1218,7 @@ int ath11k_dp_tx_htt_monitor_mode_ring_config(struct ath11k *ar, bool reset) &tlv_filter); } else if (!reset) { /* set in monitor mode only */ - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { ring_id = dp->rx_mac_buf_ring[i].ring_id; ret = ath11k_dp_tx_htt_rx_filter_setup(ar->ab, ring_id, dp->mac_id + i, @@ -1231,7 +1231,7 @@ int ath11k_dp_tx_htt_monitor_mode_ring_config(struct ath11k *ar, bool reset) if (ret) return ret; - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { ring_id = dp->rx_mon_status_refill_ring[i].refill_buf_ring.ring_id; if (!reset) { tlv_filter.rx_filter = diff --git a/drivers/net/wireless/ath/ath11k/hal.c b/drivers/net/wireless/ath/ath11k/hal.c index f3d04568c221b6355b0da2f23bb850c9a01732bf..f02599bd1c36b15b03d2eb488e861e881c2ef49f 100644 --- a/drivers/net/wireless/ath/ath11k/hal.c +++ b/drivers/net/wireless/ath/ath11k/hal.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: BSD-3-Clause-Clear /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2023 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ #include #include "hal_tx.h" @@ -796,6 +796,20 @@ u32 *ath11k_hal_srng_src_get_next_reaped(struct ath11k_base *ab, return desc; } +u32 *ath11k_hal_srng_src_next_peek(struct ath11k_base *ab, struct hal_srng *srng) +{ + u32 next_hp; + + lockdep_assert_held(&srng->lock); + + next_hp = (srng->u.src_ring.hp + srng->entry_size) % srng->ring_size; + + if (next_hp != srng->u.src_ring.cached_tp) + return srng->ring_base_vaddr + next_hp; + + return NULL; +} + u32 *ath11k_hal_srng_src_peek(struct ath11k_base *ab, struct hal_srng *srng) { lockdep_assert_held(&srng->lock); diff --git a/drivers/net/wireless/ath/ath11k/hal.h b/drivers/net/wireless/ath/ath11k/hal.h index e453c137385e5944d7ce88ef77a6d639aa6dbfc7..dc8bbe0730170edfc7542fafa24185de03c22816 100644 --- a/drivers/net/wireless/ath/ath11k/hal.h +++ b/drivers/net/wireless/ath/ath11k/hal.h @@ -947,6 +947,8 @@ u32 *ath11k_hal_srng_dst_peek(struct ath11k_base *ab, struct hal_srng *srng); int ath11k_hal_srng_dst_num_free(struct ath11k_base *ab, struct hal_srng *srng, bool sync_hw_ptr); u32 *ath11k_hal_srng_src_peek(struct ath11k_base *ab, struct hal_srng *srng); +u32 *ath11k_hal_srng_src_next_peek(struct ath11k_base *ab, + struct hal_srng *srng); u32 *ath11k_hal_srng_src_get_next_reaped(struct ath11k_base *ab, struct hal_srng *srng); u32 *ath11k_hal_srng_src_reap_next(struct ath11k_base *ab, diff --git a/drivers/net/wireless/ath/ath11k/hw.h b/drivers/net/wireless/ath/ath11k/hw.h index 14ef4eb48f807dede8da4f43d63547a1795ee75a..300322535766eb722e0fc77c439901a653992108 100644 --- a/drivers/net/wireless/ath/ath11k/hw.h +++ b/drivers/net/wireless/ath/ath11k/hw.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2023 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ #ifndef ATH11K_HW_H @@ -167,7 +167,7 @@ struct ath11k_hw_params { bool single_pdev_only; bool rxdma1_enable; - int num_rxmda_per_pdev; + int num_rxdma_per_pdev; bool rx_mac_buf_ring; bool vdev_start_delay; bool htt_peer_map_v2; diff --git a/drivers/net/wireless/ath/ath11k/mac.c b/drivers/net/wireless/ath/ath11k/mac.c index 4f62e38ba48b3a144ba658ce11f4c8193024d89a..a1800c75d32b18dcf960404a2117e54263d43712 100644 --- a/drivers/net/wireless/ath/ath11k/mac.c +++ b/drivers/net/wireless/ath/ath11k/mac.c @@ -6108,7 +6108,7 @@ static int ath11k_mac_config_mon_status_default(struct ath11k *ar, bool enable) tlv_filter.rx_filter = ath11k_debugfs_rx_filter(ar); } - for (i = 0; i < ab->hw_params.num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params.num_rxdma_per_pdev; i++) { ring_id = ar->dp.rx_mon_status_refill_ring[i].refill_buf_ring.ring_id; ret = ath11k_dp_tx_htt_rx_filter_setup(ar->ab, ring_id, ar->dp.mac_id + i, @@ -7507,32 +7507,6 @@ static int ath11k_mac_stop_vdev_early(struct ieee80211_hw *hw, return 0; } -static u8 ath11k_mac_get_tpe_count(u8 txpwr_intrprt, u8 txpwr_cnt) -{ - switch (txpwr_intrprt) { - /* Refer "Table 9-276-Meaning of Maximum Transmit Power Count subfield - * if the Maximum Transmit Power Interpretation subfield is 0 or 2" of - * "IEEE Std 802.11ax 2021". - */ - case IEEE80211_TPE_LOCAL_EIRP: - case IEEE80211_TPE_REG_CLIENT_EIRP: - txpwr_cnt = txpwr_cnt <= 3 ? txpwr_cnt : 3; - txpwr_cnt = txpwr_cnt + 1; - break; - /* Refer "Table 9-277-Meaning of Maximum Transmit Power Count subfield - * if Maximum Transmit Power Interpretation subfield is 1 or 3" of - * "IEEE Std 802.11ax 2021". - */ - case IEEE80211_TPE_LOCAL_EIRP_PSD: - case IEEE80211_TPE_REG_CLIENT_EIRP_PSD: - txpwr_cnt = txpwr_cnt <= 4 ? txpwr_cnt : 4; - txpwr_cnt = txpwr_cnt ? (BIT(txpwr_cnt - 1)) : 1; - break; - } - - return txpwr_cnt; -} - static u8 ath11k_mac_get_num_pwr_levels(struct cfg80211_chan_def *chan_def) { if (chan_def->chan->flags & IEEE80211_CHAN_PSD) { @@ -7688,7 +7662,7 @@ void ath11k_mac_fill_reg_tpc_info(struct ath11k *ar, struct ieee80211_channel *chan, *temp_chan; u8 pwr_lvl_idx, num_pwr_levels, pwr_reduction; bool is_psd_power = false, is_tpe_present = false; - s8 max_tx_power[IEEE80211_MAX_NUM_PWR_LEVEL], + s8 max_tx_power[ATH11K_NUM_PWR_LEVELS], psd_power, tx_power; s8 eirp_power = 0; u16 start_freq, center_freq; @@ -7701,7 +7675,8 @@ void ath11k_mac_fill_reg_tpc_info(struct ath11k *ar, is_tpe_present = true; num_pwr_levels = arvif->reg_tpc_info.num_pwr_levels; } else { - num_pwr_levels = ath11k_mac_get_num_pwr_levels(&ctx->def); + num_pwr_levels = + ath11k_mac_get_num_pwr_levels(&bss_conf->chanreq.oper); } for (pwr_lvl_idx = 0; pwr_lvl_idx < num_pwr_levels; pwr_lvl_idx++) { @@ -7858,33 +7833,23 @@ static void ath11k_mac_parse_tx_pwr_env(struct ath11k *ar, struct ath11k_base *ab = ar->ab; struct ath11k_vif *arvif = ath11k_vif_to_arvif(vif); struct ieee80211_bss_conf *bss_conf = &vif->bss_conf; - struct ieee80211_tx_pwr_env *single_tpe; + struct ieee80211_parsed_tpe_eirp *non_psd = NULL; + struct ieee80211_parsed_tpe_psd *psd = NULL; enum wmi_reg_6ghz_client_type client_type; struct cur_regulatory_info *reg_info; + u8 local_tpe_count, reg_tpe_count; + bool use_local_tpe; int i; - u8 pwr_count, pwr_interpret, pwr_category; - u8 psd_index = 0, non_psd_index = 0, local_tpe_count = 0, reg_tpe_count = 0; - bool use_local_tpe, non_psd_set = false, psd_set = false; reg_info = &ab->reg_info_store[ar->pdev_idx]; client_type = reg_info->client_type; - for (i = 0; i < bss_conf->tx_pwr_env_num; i++) { - single_tpe = &bss_conf->tx_pwr_env[i]; - pwr_category = u8_get_bits(single_tpe->tx_power_info, - IEEE80211_TX_PWR_ENV_INFO_CATEGORY); - pwr_interpret = u8_get_bits(single_tpe->tx_power_info, - IEEE80211_TX_PWR_ENV_INFO_INTERPRET); - - if (pwr_category == client_type) { - if (pwr_interpret == IEEE80211_TPE_LOCAL_EIRP || - pwr_interpret == IEEE80211_TPE_LOCAL_EIRP_PSD) - local_tpe_count++; - else if (pwr_interpret == IEEE80211_TPE_REG_CLIENT_EIRP || - pwr_interpret == IEEE80211_TPE_REG_CLIENT_EIRP_PSD) - reg_tpe_count++; - } - } + local_tpe_count = + bss_conf->tpe.max_local[client_type].valid + + bss_conf->tpe.psd_local[client_type].valid; + reg_tpe_count = + bss_conf->tpe.max_reg_client[client_type].valid + + bss_conf->tpe.psd_reg_client[client_type].valid; if (!reg_tpe_count && !local_tpe_count) { ath11k_warn(ab, @@ -7897,83 +7862,44 @@ static void ath11k_mac_parse_tx_pwr_env(struct ath11k *ar, use_local_tpe = false; } - for (i = 0; i < bss_conf->tx_pwr_env_num; i++) { - single_tpe = &bss_conf->tx_pwr_env[i]; - pwr_category = u8_get_bits(single_tpe->tx_power_info, - IEEE80211_TX_PWR_ENV_INFO_CATEGORY); - pwr_interpret = u8_get_bits(single_tpe->tx_power_info, - IEEE80211_TX_PWR_ENV_INFO_INTERPRET); - - if (pwr_category != client_type) - continue; - - /* get local transmit power envelope */ - if (use_local_tpe) { - if (pwr_interpret == IEEE80211_TPE_LOCAL_EIRP) { - non_psd_index = i; - non_psd_set = true; - } else if (pwr_interpret == IEEE80211_TPE_LOCAL_EIRP_PSD) { - psd_index = i; - psd_set = true; - } - /* get regulatory transmit power envelope */ - } else { - if (pwr_interpret == IEEE80211_TPE_REG_CLIENT_EIRP) { - non_psd_index = i; - non_psd_set = true; - } else if (pwr_interpret == IEEE80211_TPE_REG_CLIENT_EIRP_PSD) { - psd_index = i; - psd_set = true; - } - } + if (use_local_tpe) { + psd = &bss_conf->tpe.psd_local[client_type]; + if (!psd->valid) + psd = NULL; + non_psd = &bss_conf->tpe.max_local[client_type]; + if (!non_psd->valid) + non_psd = NULL; + } else { + psd = &bss_conf->tpe.psd_reg_client[client_type]; + if (!psd->valid) + psd = NULL; + non_psd = &bss_conf->tpe.max_reg_client[client_type]; + if (!non_psd->valid) + non_psd = NULL; } - if (non_psd_set && !psd_set) { - single_tpe = &bss_conf->tx_pwr_env[non_psd_index]; - pwr_count = u8_get_bits(single_tpe->tx_power_info, - IEEE80211_TX_PWR_ENV_INFO_COUNT); - pwr_interpret = u8_get_bits(single_tpe->tx_power_info, - IEEE80211_TX_PWR_ENV_INFO_INTERPRET); + if (non_psd && !psd) { arvif->reg_tpc_info.is_psd_power = false; arvif->reg_tpc_info.eirp_power = 0; - arvif->reg_tpc_info.num_pwr_levels = - ath11k_mac_get_tpe_count(pwr_interpret, pwr_count); + arvif->reg_tpc_info.num_pwr_levels = non_psd->count; for (i = 0; i < arvif->reg_tpc_info.num_pwr_levels; i++) { ath11k_dbg(ab, ATH11K_DBG_MAC, "non PSD power[%d] : %d\n", - i, single_tpe->tx_power[i]); - arvif->reg_tpc_info.tpe[i] = single_tpe->tx_power[i] / 2; + i, non_psd->power[i]); + arvif->reg_tpc_info.tpe[i] = non_psd->power[i] / 2; } } - if (psd_set) { - single_tpe = &bss_conf->tx_pwr_env[psd_index]; - pwr_count = u8_get_bits(single_tpe->tx_power_info, - IEEE80211_TX_PWR_ENV_INFO_COUNT); - pwr_interpret = u8_get_bits(single_tpe->tx_power_info, - IEEE80211_TX_PWR_ENV_INFO_INTERPRET); - arvif->reg_tpc_info.is_psd_power = true; + if (psd) { + arvif->reg_tpc_info.num_pwr_levels = psd->count; - if (pwr_count == 0) { + for (i = 0; i < arvif->reg_tpc_info.num_pwr_levels; i++) { ath11k_dbg(ab, ATH11K_DBG_MAC, - "TPE PSD power : %d\n", single_tpe->tx_power[0]); - arvif->reg_tpc_info.num_pwr_levels = - ath11k_mac_get_num_pwr_levels(&ctx->def); - - for (i = 0; i < arvif->reg_tpc_info.num_pwr_levels; i++) - arvif->reg_tpc_info.tpe[i] = single_tpe->tx_power[0] / 2; - } else { - arvif->reg_tpc_info.num_pwr_levels = - ath11k_mac_get_tpe_count(pwr_interpret, pwr_count); - - for (i = 0; i < arvif->reg_tpc_info.num_pwr_levels; i++) { - ath11k_dbg(ab, ATH11K_DBG_MAC, - "TPE PSD power[%d] : %d\n", - i, single_tpe->tx_power[i]); - arvif->reg_tpc_info.tpe[i] = single_tpe->tx_power[i] / 2; - } + "TPE PSD power[%d] : %d\n", + i, psd->power[i]); + arvif->reg_tpc_info.tpe[i] = psd->power[i] / 2; } } } @@ -7988,8 +7914,6 @@ ath11k_mac_op_assign_vif_chanctx(struct ieee80211_hw *hw, struct ath11k_base *ab = ar->ab; struct ath11k_vif *arvif = ath11k_vif_to_arvif(vif); int ret; - struct cur_regulatory_info *reg_info; - enum ieee80211_ap_reg_power power_type; mutex_lock(&ar->conf_mutex); @@ -8000,17 +7924,6 @@ ath11k_mac_op_assign_vif_chanctx(struct ieee80211_hw *hw, if (ath11k_wmi_supports_6ghz_cc_ext(ar) && ctx->def.chan->band == NL80211_BAND_6GHZ && arvif->vdev_type == WMI_VDEV_TYPE_STA) { - reg_info = &ab->reg_info_store[ar->pdev_idx]; - power_type = vif->bss_conf.power_type; - - ath11k_dbg(ab, ATH11K_DBG_MAC, "chanctx power type %d\n", power_type); - - if (power_type == IEEE80211_REG_UNSET_AP) { - ret = -EINVAL; - goto out; - } - - ath11k_reg_handle_chan_list(ab, reg_info, power_type); arvif->chanctx = *ctx; ath11k_mac_parse_tx_pwr_env(ar, vif, ctx); } @@ -8864,12 +8777,8 @@ ath11k_mac_op_reconfig_complete(struct ieee80211_hw *hw, ieee80211_wake_queues(ar->hw); if (ar->ab->hw_params.current_cc_support && - ar->alpha2[0] != 0 && ar->alpha2[1] != 0) { - struct wmi_set_current_country_params set_current_param = {}; - - memcpy(&set_current_param.alpha2, ar->alpha2, 2); - ath11k_wmi_send_set_current_country_cmd(ar, &set_current_param); - } + ar->alpha2[0] != 0 && ar->alpha2[1] != 0) + ath11k_reg_set_cc(ar); if (ab->is_reset) { recovery_count = atomic_inc_return(&ab->recovery_count); @@ -9104,7 +9013,6 @@ static void ath11k_mac_op_ipv6_changed(struct ieee80211_hw *hw, struct ath11k_vif *arvif = ath11k_vif_to_arvif(vif); struct inet6_ifaddr *ifa6; struct ifacaddr6 *ifaca6; - struct list_head *p; u32 count, scope; ath11k_dbg(ar->ab, ATH11K_DBG_MAC, "op ipv6 changed\n"); @@ -9120,11 +9028,10 @@ static void ath11k_mac_op_ipv6_changed(struct ieee80211_hw *hw, memcpy(offload->mac_addr, vif->addr, ETH_ALEN); /* get unicast address */ - list_for_each(p, &idev->addr_list) { + list_for_each_entry(ifa6, &idev->addr_list, if_list) { if (count >= ATH11K_IPV6_MAX_COUNT) goto generate; - ifa6 = list_entry(p, struct inet6_ifaddr, if_list); if (ifa6->flags & IFA_F_DADFAILED) continue; scope = ipv6_addr_src_scope(&ifa6->addr); @@ -9626,6 +9533,8 @@ static int ath11k_mac_op_sta_state(struct ieee80211_hw *hw, struct ath11k *ar = hw->priv; struct ath11k_vif *arvif = ath11k_vif_to_arvif(vif); struct ath11k_sta *arsta = ath11k_sta_to_arsta(sta); + enum ieee80211_ap_reg_power power_type; + struct cur_regulatory_info *reg_info; struct ath11k_peer *peer; int ret = 0; @@ -9705,6 +9614,29 @@ static int ath11k_mac_op_sta_state(struct ieee80211_hw *hw, ath11k_warn(ar->ab, "Unable to authorize peer %pM vdev %d: %d\n", sta->addr, arvif->vdev_id, ret); } + + if (!ret && + ath11k_wmi_supports_6ghz_cc_ext(ar) && + arvif->vdev_type == WMI_VDEV_TYPE_STA && + arvif->chanctx.def.chan && + arvif->chanctx.def.chan->band == NL80211_BAND_6GHZ) { + reg_info = &ar->ab->reg_info_store[ar->pdev_idx]; + power_type = vif->bss_conf.power_type; + + if (power_type == IEEE80211_REG_UNSET_AP) { + ath11k_warn(ar->ab, "invalid power type %d\n", + power_type); + ret = -EINVAL; + } else { + ret = ath11k_reg_handle_chan_list(ar->ab, + reg_info, + power_type); + if (ret) + ath11k_warn(ar->ab, + "failed to handle chan list with power type %d\n", + power_type); + } + } } else if (old_state == IEEE80211_STA_AUTHORIZED && new_state == IEEE80211_STA_ASSOC) { spin_lock_bh(&ar->ab->base_lock); @@ -10313,11 +10245,8 @@ static int __ath11k_mac_register(struct ath11k *ar) } if (ab->hw_params.current_cc_support && ab->new_alpha2[0]) { - struct wmi_set_current_country_params set_current_param = {}; - - memcpy(&set_current_param.alpha2, ab->new_alpha2, 2); memcpy(&ar->alpha2, ab->new_alpha2, 2); - ret = ath11k_wmi_send_set_current_country_cmd(ar, &set_current_param); + ret = ath11k_reg_set_cc(ar); if (ret) ath11k_warn(ar->ab, "failed set cc code for mac register: %d\n", ret); diff --git a/drivers/net/wireless/ath/ath11k/pcic.c b/drivers/net/wireless/ath/ath11k/pcic.c index 79eb3f9c902f4b77f3eb4481ed9d07e23038f54a..debe7c5919ef006ee911c94882c82ea3bcf61548 100644 --- a/drivers/net/wireless/ath/ath11k/pcic.c +++ b/drivers/net/wireless/ath/ath11k/pcic.c @@ -561,6 +561,7 @@ static int ath11k_pcic_ext_irq_config(struct ath11k_base *ab) { int i, j, n, ret, num_vectors = 0; u32 user_base_data = 0, base_vector = 0; + struct ath11k_ext_irq_grp *irq_grp; unsigned long irq_flags; ret = ath11k_pcic_get_user_msi_assignment(ab, "DP", &num_vectors, @@ -574,14 +575,16 @@ static int ath11k_pcic_ext_irq_config(struct ath11k_base *ab) irq_flags |= IRQF_NOBALANCING; for (i = 0; i < ATH11K_EXT_IRQ_GRP_NUM_MAX; i++) { - struct ath11k_ext_irq_grp *irq_grp = &ab->ext_irq_grp[i]; + irq_grp = &ab->ext_irq_grp[i]; u32 num_irq = 0; irq_grp->ab = ab; irq_grp->grp_id = i; irq_grp->napi_ndev = alloc_netdev_dummy(0); - if (!irq_grp->napi_ndev) - return -ENOMEM; + if (!irq_grp->napi_ndev) { + ret = -ENOMEM; + goto fail_allocate; + } netif_napi_add(irq_grp->napi_ndev, &irq_grp->napi, ath11k_pcic_ext_grp_napi_poll); @@ -606,11 +609,8 @@ static int ath11k_pcic_ext_irq_config(struct ath11k_base *ab) int irq = ath11k_pcic_get_msi_irq(ab, vector); if (irq < 0) { - for (n = 0; n <= i; n++) { - irq_grp = &ab->ext_irq_grp[n]; - free_netdev(irq_grp->napi_ndev); - } - return irq; + ret = irq; + goto fail_irq; } ab->irq_num[irq_idx] = irq; @@ -635,6 +635,15 @@ static int ath11k_pcic_ext_irq_config(struct ath11k_base *ab) } return 0; +fail_irq: + /* i ->napi_ndev was properly allocated. Free it also */ + i += 1; +fail_allocate: + for (n = 0; n < i; n++) { + irq_grp = &ab->ext_irq_grp[n]; + free_netdev(irq_grp->napi_ndev); + } + return ret; } int ath11k_pcic_config_irq(struct ath11k_base *ab) diff --git a/drivers/net/wireless/ath/ath11k/qmi.c b/drivers/net/wireless/ath/ath11k/qmi.c index d4a243b64f6c3b23d650b39702a631cb1dde35a9..aa160e6fe24f1d545d85e0e406e2779b18dcce47 100644 --- a/drivers/net/wireless/ath/ath11k/qmi.c +++ b/drivers/net/wireless/ath/ath11k/qmi.c @@ -2293,7 +2293,7 @@ static int ath11k_qmi_load_file_target_mem(struct ath11k_base *ab, struct qmi_txn txn; const u8 *temp = data; void __iomem *bdf_addr = NULL; - int ret; + int ret = 0; u32 remaining = len; req = kzalloc(sizeof(*req), GFP_KERNEL); diff --git a/drivers/net/wireless/ath/ath11k/reg.c b/drivers/net/wireless/ath/ath11k/reg.c index 737fcd450d4bdad3c654cd5c38e158f14ef0778a..b0f289784dd3a2e1798c81b37a22d4fb0e69f183 100644 --- a/drivers/net/wireless/ath/ath11k/reg.c +++ b/drivers/net/wireless/ath/ath11k/reg.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: BSD-3-Clause-Clear /* * Copyright (c) 2018-2019 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2023 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ #include @@ -49,7 +49,6 @@ ath11k_reg_notifier(struct wiphy *wiphy, struct regulatory_request *request) { struct ieee80211_hw *hw = wiphy_to_ieee80211_hw(wiphy); struct wmi_init_country_params init_country_param; - struct wmi_set_current_country_params set_current_param = {}; struct ath11k *ar = hw->priv; int ret; @@ -83,9 +82,8 @@ ath11k_reg_notifier(struct wiphy *wiphy, struct regulatory_request *request) * reg info */ if (ar->ab->hw_params.current_cc_support) { - memcpy(&set_current_param.alpha2, request->alpha2, 2); - memcpy(&ar->alpha2, &set_current_param.alpha2, 2); - ret = ath11k_wmi_send_set_current_country_cmd(ar, &set_current_param); + memcpy(&ar->alpha2, request->alpha2, 2); + ret = ath11k_reg_set_cc(ar); if (ret) ath11k_warn(ar->ab, "failed set current country code: %d\n", ret); @@ -878,7 +876,7 @@ int ath11k_reg_handle_chan_list(struct ath11k_base *ab, ath11k_reg_reset_info(reg_info); if (ab->hw_params.single_pdev_only && - pdev_idx < ab->hw_params.num_rxmda_per_pdev) + pdev_idx < ab->hw_params.num_rxdma_per_pdev) return 0; goto fallback; } @@ -1017,3 +1015,11 @@ void ath11k_reg_free(struct ath11k_base *ab) kfree(ab->new_regd[i]); } } + +int ath11k_reg_set_cc(struct ath11k *ar) +{ + struct wmi_set_current_country_params set_current_param = {}; + + memcpy(&set_current_param.alpha2, ar->alpha2, 2); + return ath11k_wmi_send_set_current_country_cmd(ar, &set_current_param); +} diff --git a/drivers/net/wireless/ath/ath11k/reg.h b/drivers/net/wireless/ath/ath11k/reg.h index 64edb794260abcbe2d92eb4098d3b17712554ebe..263ea90619483ec8e1c235dd7ae7b2b1f537d706 100644 --- a/drivers/net/wireless/ath/ath11k/reg.h +++ b/drivers/net/wireless/ath/ath11k/reg.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2019 The Linux Foundation. All rights reserved. - * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ #ifndef ATH11K_REG_H @@ -45,5 +45,5 @@ ath11k_reg_ap_pwr_convert(enum ieee80211_ap_reg_power power_type); int ath11k_reg_handle_chan_list(struct ath11k_base *ab, struct cur_regulatory_info *reg_info, enum ieee80211_ap_reg_power power_type); - +int ath11k_reg_set_cc(struct ath11k *ar); #endif diff --git a/drivers/net/wireless/ath/ath11k/wmi.c b/drivers/net/wireless/ath/ath11k/wmi.c index 6ff01c45f16591cc4a3de0dcf38a9b764b2dae6a..38f175dd15578a312bce3d457aee1cd7b8acad51 100644 --- a/drivers/net/wireless/ath/ath11k/wmi.c +++ b/drivers/net/wireless/ath/ath11k/wmi.c @@ -9082,7 +9082,7 @@ int ath11k_wmi_attach(struct ath11k_base *ab) ab->wmi_ab.preferred_hw_mode = WMI_HOST_HW_MODE_MAX; /* It's overwritten when service_ext_ready is handled */ - if (ab->hw_params.single_pdev_only && ab->hw_params.num_rxmda_per_pdev > 1) + if (ab->hw_params.single_pdev_only && ab->hw_params.num_rxdma_per_pdev > 1) ab->wmi_ab.preferred_hw_mode = WMI_HOST_HW_MODE_SINGLE; /* TODO: Init remaining wmi soc resources required */ diff --git a/drivers/net/wireless/ath/ath12k/ce.h b/drivers/net/wireless/ath/ath12k/ce.h index 79af3b6159f1c772f057af429328ede0483999ed..857bc5f9e946a9ab3f100987a7f7822bcbf66f56 100644 --- a/drivers/net/wireless/ath/ath12k/ce.h +++ b/drivers/net/wireless/ath/ath12k/ce.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2018-2021 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2022 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2022, 2024 Qualcomm Innovation Center, Inc. All rights reserved. */ #ifndef ATH12K_CE_H @@ -119,7 +119,7 @@ struct ath12k_ce_ring { /* Host address space */ void *base_addr_owner_space_unaligned; /* CE address space */ - u32 base_addr_ce_space_unaligned; + dma_addr_t base_addr_ce_space_unaligned; /* Actual start of descriptors. * Aligned to descriptor-size boundary. @@ -129,7 +129,7 @@ struct ath12k_ce_ring { void *base_addr_owner_space; /* CE address space */ - u32 base_addr_ce_space; + dma_addr_t base_addr_ce_space; /* HAL ring id */ u32 hal_ring_id; diff --git a/drivers/net/wireless/ath/ath12k/core.c b/drivers/net/wireless/ath/ath12k/core.c index 6663f4e1792de1f778d7f849a1a1594931e9e505..4c3eab4686c240df874e9ba824c7fdd635a938ff 100644 --- a/drivers/net/wireless/ath/ath12k/core.c +++ b/drivers/net/wireless/ath/ath12k/core.c @@ -50,19 +50,16 @@ int ath12k_core_suspend(struct ath12k_base *ab) if (!ab->hw_params->supports_suspend) return -EOPNOTSUPP; - rcu_read_lock(); for (i = 0; i < ab->num_radios; i++) { - ar = ath12k_mac_get_ar_by_pdev_id(ab, i); + ar = ab->pdevs[i].ar; if (!ar) continue; ret = ath12k_mac_wait_tx_complete(ar); if (ret) { ath12k_warn(ab, "failed to wait tx complete: %d\n", ret); - rcu_read_unlock(); return ret; } } - rcu_read_unlock(); /* PM framework skips suspend_late/resume_early callbacks * if other devices report errors in their suspend callbacks. @@ -994,9 +991,8 @@ void ath12k_core_halt(struct ath12k *ar) static void ath12k_core_pre_reconfigure_recovery(struct ath12k_base *ab) { struct ath12k *ar; - struct ath12k_pdev *pdev; struct ath12k_hw *ah; - int i; + int i, j; spin_lock_bh(&ab->base_lock); ab->stats.fw_crash_counter++; @@ -1006,35 +1002,32 @@ static void ath12k_core_pre_reconfigure_recovery(struct ath12k_base *ab) set_bit(ATH12K_FLAG_CRASH_FLUSH, &ab->dev_flags); for (i = 0; i < ab->num_hw; i++) { - if (!ab->ah[i]) + ah = ab->ah[i]; + if (!ah || ah->state == ATH12K_HW_STATE_OFF) continue; - ah = ab->ah[i]; ieee80211_stop_queues(ah->hw); - } - for (i = 0; i < ab->num_radios; i++) { - pdev = &ab->pdevs[i]; - ar = pdev->ar; - if (!ar || ar->state == ATH12K_STATE_OFF) - continue; + for (j = 0; j < ah->num_radio; j++) { + ar = &ah->radio[j]; - ath12k_mac_drain_tx(ar); - complete(&ar->scan.started); - complete(&ar->scan.completed); - complete(&ar->scan.on_channel); - complete(&ar->peer_assoc_done); - complete(&ar->peer_delete_done); - complete(&ar->install_key_done); - complete(&ar->vdev_setup_done); - complete(&ar->vdev_delete_done); - complete(&ar->bss_survey_done); - - wake_up(&ar->dp.tx_empty_waitq); - idr_for_each(&ar->txmgmt_idr, - ath12k_mac_tx_mgmt_pending_free, ar); - idr_destroy(&ar->txmgmt_idr); - wake_up(&ar->txmgmt_empty_waitq); + ath12k_mac_drain_tx(ar); + complete(&ar->scan.started); + complete(&ar->scan.completed); + complete(&ar->scan.on_channel); + complete(&ar->peer_assoc_done); + complete(&ar->peer_delete_done); + complete(&ar->install_key_done); + complete(&ar->vdev_setup_done); + complete(&ar->vdev_delete_done); + complete(&ar->bss_survey_done); + + wake_up(&ar->dp.tx_empty_waitq); + idr_for_each(&ar->txmgmt_idr, + ath12k_mac_tx_mgmt_pending_free, ar); + idr_destroy(&ar->txmgmt_idr); + wake_up(&ar->txmgmt_empty_waitq); + } } wake_up(&ab->wmi_ab.tx_credits_wq); @@ -1043,41 +1036,51 @@ static void ath12k_core_pre_reconfigure_recovery(struct ath12k_base *ab) static void ath12k_core_post_reconfigure_recovery(struct ath12k_base *ab) { + struct ath12k_hw *ah; struct ath12k *ar; - struct ath12k_pdev *pdev; - int i; + int i, j; - for (i = 0; i < ab->num_radios; i++) { - pdev = &ab->pdevs[i]; - ar = pdev->ar; - if (!ar || ar->state == ATH12K_STATE_OFF) + for (i = 0; i < ab->num_hw; i++) { + ah = ab->ah[i]; + if (!ah || ah->state == ATH12K_HW_STATE_OFF) continue; - mutex_lock(&ar->conf_mutex); + mutex_lock(&ah->hw_mutex); + + switch (ah->state) { + case ATH12K_HW_STATE_ON: + ah->state = ATH12K_HW_STATE_RESTARTING; + + for (j = 0; j < ah->num_radio; j++) { + ar = &ah->radio[j]; - switch (ar->state) { - case ATH12K_STATE_ON: - ar->state = ATH12K_STATE_RESTARTING; - ath12k_core_halt(ar); - ieee80211_restart_hw(ath12k_ar_to_hw(ar)); + mutex_lock(&ar->conf_mutex); + ath12k_core_halt(ar); + mutex_unlock(&ar->conf_mutex); + } + + /* Restart after all the link/radio halt */ + ieee80211_restart_hw(ah->hw); break; - case ATH12K_STATE_OFF: + case ATH12K_HW_STATE_OFF: ath12k_warn(ab, - "cannot restart radio %d that hasn't been started\n", + "cannot restart hw %d that hasn't been started\n", i); break; - case ATH12K_STATE_RESTARTING: + case ATH12K_HW_STATE_RESTARTING: break; - case ATH12K_STATE_RESTARTED: - ar->state = ATH12K_STATE_WEDGED; + case ATH12K_HW_STATE_RESTARTED: + ah->state = ATH12K_HW_STATE_WEDGED; fallthrough; - case ATH12K_STATE_WEDGED: + case ATH12K_HW_STATE_WEDGED: ath12k_warn(ab, - "device is wedged, will not restart radio %d\n", i); + "device is wedged, will not restart hw %d\n", i); break; } - mutex_unlock(&ar->conf_mutex); + + mutex_unlock(&ah->hw_mutex); } + complete(&ab->driver_recovery); } @@ -1185,6 +1188,29 @@ int ath12k_core_pre_init(struct ath12k_base *ab) return 0; } +static int ath12k_core_panic_handler(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct ath12k_base *ab = container_of(nb, struct ath12k_base, + panic_nb); + + return ath12k_hif_panic_handler(ab); +} + +static int ath12k_core_panic_notifier_register(struct ath12k_base *ab) +{ + ab->panic_nb.notifier_call = ath12k_core_panic_handler; + + return atomic_notifier_chain_register(&panic_notifier_list, + &ab->panic_nb); +} + +static void ath12k_core_panic_notifier_unregister(struct ath12k_base *ab) +{ + atomic_notifier_chain_unregister(&panic_notifier_list, + &ab->panic_nb); +} + int ath12k_core_init(struct ath12k_base *ab) { int ret; @@ -1195,11 +1221,17 @@ int ath12k_core_init(struct ath12k_base *ab) return ret; } + ret = ath12k_core_panic_notifier_register(ab); + if (ret) + ath12k_warn(ab, "failed to register panic handler: %d\n", ret); + return 0; } void ath12k_core_deinit(struct ath12k_base *ab) { + ath12k_core_panic_notifier_unregister(ab); + mutex_lock(&ab->core_lock); ath12k_core_pdev_destroy(ab); @@ -1262,6 +1294,16 @@ struct ath12k_base *ath12k_core_alloc(struct device *dev, size_t priv_size, ab->qmi.num_radios = U8_MAX; ab->mlo_capable_flags = ATH12K_INTRA_DEVICE_MLO_SUPPORT; + /* Device index used to identify the devices in a group. + * + * In Intra-device MLO, only one device present in a group, + * so it is always zero. + * + * In Inter-device MLO, Multiple device present in a group, + * expect non-zero value. + */ + ab->device_id = 0; + return ab; err_free_wq: diff --git a/drivers/net/wireless/ath/ath12k/core.h b/drivers/net/wireless/ath/ath12k/core.h index 47dde440121001cbfd54256b49f357631bd8b984..d03326a68a9f53e391b2f459d097bfa1598426cc 100644 --- a/drivers/net/wireless/ath/ath12k/core.h +++ b/drivers/net/wireless/ath/ath12k/core.h @@ -14,6 +14,7 @@ #include #include #include +#include #include "qmi.h" #include "htc.h" #include "wmi.h" @@ -146,7 +147,7 @@ struct ath12k_ext_irq_grp { u32 grp_id; u64 timestamp; struct napi_struct napi; - struct net_device napi_ndev; + struct net_device *napi_ndev; }; struct ath12k_smbios_bdf { @@ -180,8 +181,6 @@ struct ath12k_he { u32 heop_param; }; -#define MAX_RADIOS 3 - enum { WMI_HOST_TP_SCALE_MAX = 0, WMI_HOST_TP_SCALE_50 = 1, @@ -212,10 +211,6 @@ enum ath12k_dev_flags { ATH12K_FLAG_EXT_IRQ_ENABLED, }; -enum ath12k_monitor_flags { - ATH12K_FLAG_MONITOR_ENABLED, -}; - struct ath12k_tx_conf { bool changed; u16 ac; @@ -454,15 +449,15 @@ struct ath12k_sta { #define ATH12K_MIN_5G_FREQ 4150 #define ATH12K_MIN_6G_FREQ 5925 #define ATH12K_MAX_6G_FREQ 7115 -#define ATH12K_NUM_CHANS 100 +#define ATH12K_NUM_CHANS 101 #define ATH12K_MAX_5G_CHAN 173 -enum ath12k_state { - ATH12K_STATE_OFF, - ATH12K_STATE_ON, - ATH12K_STATE_RESTARTING, - ATH12K_STATE_RESTARTED, - ATH12K_STATE_WEDGED, +enum ath12k_hw_state { + ATH12K_HW_STATE_OFF, + ATH12K_HW_STATE_ON, + ATH12K_HW_STATE_RESTARTING, + ATH12K_HW_STATE_RESTARTED, + ATH12K_HW_STATE_WEDGED, /* Add other states as required */ }; @@ -511,7 +506,6 @@ struct ath12k { u32 ht_cap_info; u32 vht_cap_info; struct ath12k_he ar_he; - enum ath12k_state state; bool supports_6ghz; struct { struct completion started; @@ -533,7 +527,6 @@ struct ath12k { unsigned long dev_flags; unsigned int filter_flags; - unsigned long monitor_flags; u32 min_tx_power; u32 max_tx_power; u32 txpower_limit_2g; @@ -636,10 +629,18 @@ struct ath12k { struct ath12k_hw { struct ieee80211_hw *hw; + struct ath12k_base *ab; + + /* Protect the write operation of the hardware state ath12k_hw::state + * between hardware start<=>reconfigure<=>stop transitions. + */ + struct mutex hw_mutex; + enum ath12k_hw_state state; bool regd_updated; bool use_6ghz_regd; - u8 num_radio; + + /* Keep last */ struct ath12k radio[] __aligned(sizeof(void *)); }; @@ -689,6 +690,7 @@ struct mlo_timestamp { struct ath12k_pdev { struct ath12k *ar; u32 pdev_id; + u32 hw_link_id; struct ath12k_pdev_cap cap; u8 mac_addr[ETH_ALEN]; struct mlo_timestamp timestamp; @@ -747,6 +749,7 @@ struct ath12k_base { struct ath12k_qmi qmi; struct ath12k_wmi_base wmi_ab; struct completion fw_ready; + u8 device_id; int num_radios; /* HW channel counters frequency value in hertz common to all MACs */ u32 cc_freq_hz; @@ -923,6 +926,8 @@ struct ath12k_base { #endif /* CONFIG_ACPI */ + struct notifier_block panic_nb; + /* must be last */ u8 drv_priv[] __aligned(sizeof(void *)); }; @@ -1037,6 +1042,11 @@ static inline struct ath12k *ath12k_ah_to_ar(struct ath12k_hw *ah, u8 hw_link_id return &ah->radio[hw_link_id]; } +static inline struct ath12k_hw *ath12k_ar_to_ah(struct ath12k *ar) +{ + return ar->ah; +} + static inline struct ieee80211_hw *ath12k_ar_to_hw(struct ath12k *ar) { return ar->ah->hw; diff --git a/drivers/net/wireless/ath/ath12k/dp.c b/drivers/net/wireless/ath/ath12k/dp.c index 7843c76a82c17f8dac490e33b9809a5a477c6678..61aa78d8bd8c8fef4872e15e63a32a76c927c934 100644 --- a/drivers/net/wireless/ath/ath12k/dp.c +++ b/drivers/net/wireless/ath/ath12k/dp.c @@ -132,7 +132,9 @@ static int ath12k_dp_srng_find_ring_in_mask(int ring_num, const u8 *grp_mask) static int ath12k_dp_srng_calculate_msi_group(struct ath12k_base *ab, enum hal_ring_type type, int ring_num) { + const struct ath12k_hal_tcl_to_wbm_rbm_map *map; const u8 *grp_mask; + int i; switch (type) { case HAL_WBM2SW_RELEASE: @@ -140,6 +142,14 @@ static int ath12k_dp_srng_calculate_msi_group(struct ath12k_base *ab, grp_mask = &ab->hw_params->ring_mask->rx_wbm_rel[0]; ring_num = 0; } else { + map = ab->hw_params->hal_ops->tcl_to_wbm_rbm_map; + for (i = 0; i < ab->hw_params->max_tx_ring; i++) { + if (ring_num == map[i].wbm_ring_num) { + ring_num = i; + break; + } + } + grp_mask = &ab->hw_params->ring_mask->tx[0]; } break; @@ -457,8 +467,6 @@ static void ath12k_dp_srng_common_cleanup(struct ath12k_base *ab) ath12k_dp_srng_cleanup(ab, &dp->tx_ring[i].tcl_comp_ring); ath12k_dp_srng_cleanup(ab, &dp->tx_ring[i].tcl_data_ring); } - ath12k_dp_srng_cleanup(ab, &dp->tcl_status_ring); - ath12k_dp_srng_cleanup(ab, &dp->tcl_cmd_ring); ath12k_dp_srng_cleanup(ab, &dp->wbm_desc_rel_ring); } @@ -479,20 +487,6 @@ static int ath12k_dp_srng_common_setup(struct ath12k_base *ab) goto err; } - ret = ath12k_dp_srng_setup(ab, &dp->tcl_cmd_ring, HAL_TCL_CMD, 0, 0, - DP_TCL_CMD_RING_SIZE); - if (ret) { - ath12k_warn(ab, "failed to set up tcl_cmd ring :%d\n", ret); - goto err; - } - - ret = ath12k_dp_srng_setup(ab, &dp->tcl_status_ring, HAL_TCL_STATUS, - 0, 0, DP_TCL_STATUS_RING_SIZE); - if (ret) { - ath12k_warn(ab, "failed to set up tcl_status ring :%d\n", ret); - goto err; - } - for (i = 0; i < ab->hw_params->max_tx_ring; i++) { map = ab->hw_params->hal_ops->tcl_to_wbm_rbm_map; tx_comp_ring_num = map[i].wbm_ring_num; @@ -616,6 +610,7 @@ static int ath12k_dp_scatter_idle_link_desc_setup(struct ath12k_base *ab, int i; int ret = 0; u32 end_offset, cookie; + enum hal_rx_buf_return_buf_manager rbm = dp->idle_link_rbm; n_entries_per_buf = HAL_WBM_IDLE_SCATTER_BUF_SIZE / ath12k_hal_srng_get_entrysize(ab, HAL_WBM_IDLE_LINK); @@ -646,7 +641,8 @@ static int ath12k_dp_scatter_idle_link_desc_setup(struct ath12k_base *ab, paddr = link_desc_banks[i].paddr; while (n_entries) { cookie = DP_LINK_DESC_COOKIE_SET(n_entries, i); - ath12k_hal_set_link_desc_addr(scatter_buf, cookie, paddr); + ath12k_hal_set_link_desc_addr(scatter_buf, cookie, + paddr, rbm); n_entries--; paddr += HAL_LINK_DESC_SIZE; if (rem_entries) { @@ -790,6 +786,7 @@ int ath12k_dp_link_desc_setup(struct ath12k_base *ab, u32 paddr; int i, ret; u32 cookie; + enum hal_rx_buf_return_buf_manager rbm = ab->dp.idle_link_rbm; tot_mem_sz = n_link_desc * HAL_LINK_DESC_SIZE; tot_mem_sz += HAL_LINK_DESC_ALIGN; @@ -850,8 +847,7 @@ int ath12k_dp_link_desc_setup(struct ath12k_base *ab, while (n_entries && (desc = ath12k_hal_srng_src_get_next_entry(ab, srng))) { cookie = DP_LINK_DESC_COOKIE_SET(n_entries, i); - ath12k_hal_set_link_desc_addr(desc, - cookie, paddr); + ath12k_hal_set_link_desc_addr(desc, cookie, paddr, rbm); n_entries--; paddr += HAL_LINK_DESC_SIZE; } @@ -881,11 +877,9 @@ int ath12k_dp_service_srng(struct ath12k_base *ab, enum dp_monitor_mode monitor_mode; u8 ring_mask; - while (i < ab->hw_params->max_tx_ring) { - if (ab->hw_params->ring_mask->tx[grp_id] & - BIT(ab->hw_params->hal_ops->tcl_to_wbm_rbm_map[i].wbm_ring_num)) - ath12k_dp_tx_completion_handler(ab, i); - i++; + if (ab->hw_params->ring_mask->tx[grp_id]) { + i = fls(ab->hw_params->ring_mask->tx[grp_id]) - 1; + ath12k_dp_tx_completion_handler(ab, i); } if (ab->hw_params->ring_mask->rx_err[grp_id]) { @@ -921,8 +915,8 @@ int ath12k_dp_service_srng(struct ath12k_base *ab, monitor_mode = ATH12K_DP_RX_MONITOR_MODE; ring_mask = ab->hw_params->ring_mask->rx_mon_dest[grp_id]; for (i = 0; i < ab->num_radios; i++) { - for (j = 0; j < ab->hw_params->num_rxmda_per_pdev; j++) { - int id = i * ab->hw_params->num_rxmda_per_pdev + j; + for (j = 0; j < ab->hw_params->num_rxdma_per_pdev; j++) { + int id = i * ab->hw_params->num_rxdma_per_pdev + j; if (ring_mask & BIT(id)) { work_done = @@ -942,8 +936,8 @@ int ath12k_dp_service_srng(struct ath12k_base *ab, monitor_mode = ATH12K_DP_TX_MONITOR_MODE; ring_mask = ab->hw_params->ring_mask->tx_mon_dest[grp_id]; for (i = 0; i < ab->num_radios; i++) { - for (j = 0; j < ab->hw_params->num_rxmda_per_pdev; j++) { - int id = i * ab->hw_params->num_rxmda_per_pdev + j; + for (j = 0; j < ab->hw_params->num_rxdma_per_pdev; j++) { + int id = i * ab->hw_params->num_rxdma_per_pdev + j; if (ring_mask & BIT(id)) { work_done = @@ -1031,7 +1025,7 @@ static void ath12k_dp_service_mon_ring(struct timer_list *t) struct ath12k_base *ab = from_timer(ab, t, mon_reap_timer); int i; - for (i = 0; i < ab->hw_params->num_rxmda_per_pdev; i++) + for (i = 0; i < ab->hw_params->num_rxdma_per_pdev; i++) ath12k_dp_mon_process_ring(ab, i, NULL, DP_MON_SERVICE_BUDGET, ATH12K_DP_RX_MONITOR_MODE); @@ -1355,13 +1349,14 @@ static inline void *ath12k_dp_cc_get_desc_addr_ptr(struct ath12k_base *ab, struct ath12k_rx_desc_info *ath12k_dp_get_rx_desc(struct ath12k_base *ab, u32 cookie) { + struct ath12k_dp *dp = &ab->dp; struct ath12k_rx_desc_info **desc_addr_ptr; u16 start_ppt_idx, end_ppt_idx, ppt_idx, spt_idx; ppt_idx = u32_get_bits(cookie, ATH12K_DP_CC_COOKIE_PPT); spt_idx = u32_get_bits(cookie, ATH12K_DP_CC_COOKIE_SPT); - start_ppt_idx = ATH12K_RX_SPT_PAGE_OFFSET; + start_ppt_idx = dp->rx_ppt_base + ATH12K_RX_SPT_PAGE_OFFSET; end_ppt_idx = start_ppt_idx + ATH12K_NUM_RX_SPT_PAGES; if (ppt_idx < start_ppt_idx || @@ -1369,6 +1364,7 @@ struct ath12k_rx_desc_info *ath12k_dp_get_rx_desc(struct ath12k_base *ab, spt_idx > ATH12K_MAX_SPT_ENTRIES) return NULL; + ppt_idx = ppt_idx - dp->rx_ppt_base; desc_addr_ptr = ath12k_dp_cc_get_desc_addr_ptr(ab, ppt_idx, spt_idx); return *desc_addr_ptr; @@ -1403,7 +1399,7 @@ static int ath12k_dp_cc_desc_init(struct ath12k_base *ab) struct ath12k_rx_desc_info *rx_descs, **rx_desc_addr; struct ath12k_tx_desc_info *tx_descs, **tx_desc_addr; u32 i, j, pool_id, tx_spt_page; - u32 ppt_idx; + u32 ppt_idx, cookie_ppt_idx; spin_lock_bh(&dp->rx_desc_lock); @@ -1418,10 +1414,11 @@ static int ath12k_dp_cc_desc_init(struct ath12k_base *ab) } ppt_idx = ATH12K_RX_SPT_PAGE_OFFSET + i; + cookie_ppt_idx = dp->rx_ppt_base + ppt_idx; dp->spt_info->rxbaddr[i] = &rx_descs[0]; for (j = 0; j < ATH12K_MAX_SPT_ENTRIES; j++) { - rx_descs[j].cookie = ath12k_dp_cc_cookie_gen(ppt_idx, j); + rx_descs[j].cookie = ath12k_dp_cc_cookie_gen(cookie_ppt_idx, j); rx_descs[j].magic = ATH12K_DP_RX_DESC_MAGIC; list_add_tail(&rx_descs[j].list, &dp->rx_desc_free_list); @@ -1482,6 +1479,7 @@ static int ath12k_dp_cmem_init(struct ath12k_base *ab, end = start + ATH12K_NUM_TX_SPT_PAGES; break; case ATH12K_DP_RX_DESC: + cmem_base += ATH12K_PPT_ADDR_OFFSET(dp->rx_ppt_base); start = ATH12K_RX_SPT_PAGE_OFFSET; end = start + ATH12K_NUM_RX_SPT_PAGES; break; @@ -1524,6 +1522,8 @@ static int ath12k_dp_cc_init(struct ath12k_base *ab) return -ENOMEM; } + dp->rx_ppt_base = ab->device_id * ATH12K_NUM_RX_SPT_PAGES; + for (i = 0; i < dp->num_spt_pages; i++) { dp->spt_info[i].vaddr = dma_alloc_coherent(ab->dev, ATH12K_PAGE_SIZE, @@ -1587,6 +1587,24 @@ static int ath12k_dp_reoq_lut_setup(struct ath12k_base *ab) return 0; } +static enum hal_rx_buf_return_buf_manager +ath12k_dp_get_idle_link_rbm(struct ath12k_base *ab) +{ + switch (ab->device_id) { + case 0: + return HAL_RX_BUF_RBM_WBM_DEV0_IDLE_DESC_LIST; + case 1: + return HAL_RX_BUF_RBM_WBM_DEV1_IDLE_DESC_LIST; + case 2: + return HAL_RX_BUF_RBM_WBM_DEV2_IDLE_DESC_LIST; + default: + ath12k_warn(ab, "invalid %d device id, so choose default rbm\n", + ab->device_id); + WARN_ON(1); + return HAL_RX_BUF_RBM_WBM_DEV0_IDLE_DESC_LIST; + } +} + int ath12k_dp_alloc(struct ath12k_base *ab) { struct ath12k_dp *dp = &ab->dp; @@ -1603,6 +1621,7 @@ int ath12k_dp_alloc(struct ath12k_base *ab) spin_lock_init(&dp->reo_cmd_lock); dp->reo_cmd_cache_flush_count = 0; + dp->idle_link_rbm = ath12k_dp_get_idle_link_rbm(ab); ret = ath12k_wbm_idle_ring_setup(ab, &n_link_desc); if (ret) { diff --git a/drivers/net/wireless/ath/ath12k/dp.h b/drivers/net/wireless/ath/ath12k/dp.h index 5cf0d21ef184b6f2ed60105fe8759dd37e98fcc4..742094545089f8602a3731c4b74da4894c987e75 100644 --- a/drivers/net/wireless/ath/ath12k/dp.h +++ b/drivers/net/wireless/ath/ath12k/dp.h @@ -325,10 +325,9 @@ struct ath12k_dp { u8 htt_tgt_ver_major; u8 htt_tgt_ver_minor; struct dp_link_desc_bank link_desc_banks[DP_LINK_DESC_BANKS_MAX]; + enum hal_rx_buf_return_buf_manager idle_link_rbm; struct dp_srng wbm_idle_ring; struct dp_srng wbm_desc_rel_ring; - struct dp_srng tcl_cmd_ring; - struct dp_srng tcl_status_ring; struct dp_srng reo_reinject_ring; struct dp_srng rx_rel_ring; struct dp_srng reo_except_ring; @@ -351,6 +350,7 @@ struct ath12k_dp { struct ath12k_hp_update_timer tx_ring_timer[DP_TCL_NUM_RING_MAX]; struct ath12k_spt_info *spt_info; u32 num_spt_pages; + u32 rx_ppt_base; struct list_head rx_desc_free_list; /* protects the free desc list */ spinlock_t rx_desc_lock; diff --git a/drivers/net/wireless/ath/ath12k/dp_mon.c b/drivers/net/wireless/ath/ath12k/dp_mon.c index 6b0b72477540767080094e8fdb28a5b7f7d9772c..5c6749bc4039d20d439892fb03c4b96e51cea972 100644 --- a/drivers/net/wireless/ath/ath12k/dp_mon.c +++ b/drivers/net/wireless/ath/ath12k/dp_mon.c @@ -1903,43 +1903,6 @@ ath12k_dp_mon_tx_parse_status_tlv(struct ath12k_base *ab, break; } - case HAL_MON_BUF_ADDR: { - struct dp_rxdma_mon_ring *buf_ring = &ab->dp.tx_mon_buf_ring; - struct dp_mon_packet_info *packet_info = - (struct dp_mon_packet_info *)tlv_data; - int buf_id = u32_get_bits(packet_info->cookie, - DP_RXDMA_BUF_COOKIE_BUF_ID); - struct sk_buff *msdu; - struct dp_mon_mpdu *mon_mpdu = tx_ppdu_info->tx_mon_mpdu; - struct ath12k_skb_rxcb *rxcb; - - spin_lock_bh(&buf_ring->idr_lock); - msdu = idr_remove(&buf_ring->bufs_idr, buf_id); - spin_unlock_bh(&buf_ring->idr_lock); - - if (unlikely(!msdu)) { - ath12k_warn(ab, "monitor destination with invalid buf_id %d\n", - buf_id); - return DP_MON_TX_STATUS_PPDU_NOT_DONE; - } - - rxcb = ATH12K_SKB_RXCB(msdu); - dma_unmap_single(ab->dev, rxcb->paddr, - msdu->len + skb_tailroom(msdu), - DMA_FROM_DEVICE); - - if (!mon_mpdu->head) - mon_mpdu->head = msdu; - else if (mon_mpdu->tail) - mon_mpdu->tail->next = msdu; - - mon_mpdu->tail = msdu; - - ath12k_dp_mon_buf_replenish(ab, buf_ring, 1); - status = DP_MON_TX_BUFFER_ADDR; - break; - } - case HAL_TX_MPDU_END: list_add_tail(&tx_ppdu_info->tx_mon_mpdu->list, &tx_ppdu_info->dp_tx_mon_mpdu_list); @@ -2088,8 +2051,7 @@ int ath12k_dp_mon_srng_process(struct ath12k *ar, int mac_id, int *budget, mon_dst_ring = &pdev_dp->rxdma_mon_dst_ring[srng_id]; buf_ring = &dp->rxdma_mon_buf_ring; } else { - mon_dst_ring = &pdev_dp->tx_mon_dst_ring[srng_id]; - buf_ring = &dp->tx_mon_buf_ring; + return 0; } srng = &ab->hal.srng_list[mon_dst_ring->ring_id]; diff --git a/drivers/net/wireless/ath/ath12k/dp_rx.c b/drivers/net/wireless/ath/ath12k/dp_rx.c index 75df622f25d8545fc3ed8277c06606f3adc9d115..cb1f308f096b1d77061fd99b61f90a7da106aa14 100644 --- a/drivers/net/wireless/ath/ath12k/dp_rx.c +++ b/drivers/net/wireless/ath/ath12k/dp_rx.c @@ -422,8 +422,6 @@ static int ath12k_dp_rxdma_buf_free(struct ath12k_base *ab) ath12k_dp_rxdma_mon_buf_ring_free(ab, &dp->rxdma_mon_buf_ring); - ath12k_dp_rxdma_mon_buf_ring_free(ab, &dp->tx_mon_buf_ring); - return 0; } @@ -476,15 +474,6 @@ static int ath12k_dp_rxdma_buf_setup(struct ath12k_base *ab) "failed to setup HAL_RXDMA_MONITOR_BUF\n"); return ret; } - - ret = ath12k_dp_rxdma_mon_ring_buf_setup(ab, - &dp->tx_mon_buf_ring, - HAL_TX_MONITOR_BUF); - if (ret) { - ath12k_warn(ab, - "failed to setup HAL_TX_MONITOR_BUF\n"); - return ret; - } } return 0; @@ -496,10 +485,8 @@ static void ath12k_dp_rx_pdev_srng_free(struct ath12k *ar) struct ath12k_base *ab = ar->ab; int i; - for (i = 0; i < ab->hw_params->num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params->num_rxdma_per_pdev; i++) ath12k_dp_srng_cleanup(ab, &dp->rxdma_mon_dst_ring[i]); - ath12k_dp_srng_cleanup(ab, &dp->tx_mon_dst_ring[i]); - } } void ath12k_dp_rx_pdev_reo_cleanup(struct ath12k_base *ab) @@ -543,7 +530,7 @@ static int ath12k_dp_rx_pdev_srng_alloc(struct ath12k *ar) int ret; u32 mac_id = dp->mac_id; - for (i = 0; i < ab->hw_params->num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params->num_rxdma_per_pdev; i++) { ret = ath12k_dp_srng_setup(ar->ab, &dp->rxdma_mon_dst_ring[i], HAL_RXDMA_MONITOR_DST, @@ -554,17 +541,6 @@ static int ath12k_dp_rx_pdev_srng_alloc(struct ath12k *ar) "failed to setup HAL_RXDMA_MONITOR_DST\n"); return ret; } - - ret = ath12k_dp_srng_setup(ar->ab, - &dp->tx_mon_dst_ring[i], - HAL_TX_MONITOR_DST, - 0, mac_id + i, - DP_TX_MONITOR_DEST_RING_SIZE); - if (ret) { - ath12k_warn(ar->ab, - "failed to setup HAL_TX_MONITOR_DST\n"); - return ret; - } } return 0; @@ -2383,8 +2359,10 @@ void ath12k_dp_rx_h_ppdu(struct ath12k *ar, struct hal_rx_desc *rx_desc, channel_num = meta_data; center_freq = meta_data >> 16; - if (center_freq >= 5935 && center_freq <= 7105) { + if (center_freq >= ATH12K_MIN_6G_FREQ && + center_freq <= ATH12K_MAX_6G_FREQ) { rx_status->band = NL80211_BAND_6GHZ; + rx_status->freq = center_freq; } else if (channel_num >= 1 && channel_num <= 14) { rx_status->band = NL80211_BAND_2GHZ; } else if (channel_num >= 36 && channel_num <= 173) { @@ -2402,8 +2380,9 @@ void ath12k_dp_rx_h_ppdu(struct ath12k *ar, struct hal_rx_desc *rx_desc, rx_desc, sizeof(*rx_desc)); } - rx_status->freq = ieee80211_channel_to_frequency(channel_num, - rx_status->band); + if (rx_status->band != NL80211_BAND_6GHZ) + rx_status->freq = ieee80211_channel_to_frequency(channel_num, + rx_status->band); ath12k_dp_rx_h_rate(ar, rx_desc, rx_status); } @@ -2649,7 +2628,8 @@ try_again: if (!desc_info) { desc_info = ath12k_dp_get_rx_desc(ab, cookie); if (!desc_info) { - ath12k_warn(ab, "Invalid cookie in manual desc retrieval"); + ath12k_warn(ab, "Invalid cookie in manual descriptor retrieval: 0x%x\n", + cookie); continue; } } @@ -2762,6 +2742,7 @@ int ath12k_dp_rx_peer_frag_setup(struct ath12k *ar, const u8 *peer_mac, int vdev peer = ath12k_peer_find(ab, vdev_id, peer_mac); if (!peer) { spin_unlock_bh(&ab->base_lock); + crypto_free_shash(tfm); ath12k_warn(ab, "failed to find the peer to set up fragment info\n"); return -ENOENT; } @@ -2991,9 +2972,10 @@ static int ath12k_dp_rx_h_defrag_reo_reinject(struct ath12k *ar, struct hal_srng *srng; dma_addr_t link_paddr, buf_paddr; u32 desc_bank, msdu_info, msdu_ext_info, mpdu_info; - u32 cookie, hal_rx_desc_sz, dest_ring_info0; + u32 cookie, hal_rx_desc_sz, dest_ring_info0, queue_addr_hi; int ret; struct ath12k_rx_desc_info *desc_info; + enum hal_rx_buf_return_buf_manager idle_link_rbm = dp->idle_link_rbm; u8 dst_ind; hal_rx_desc_sz = ab->hal.hal_desc_sz; @@ -3027,7 +3009,7 @@ static int ath12k_dp_rx_h_defrag_reo_reinject(struct ath12k *ar, buf_paddr = dma_map_single(ab->dev, defrag_skb->data, defrag_skb->len + skb_tailroom(defrag_skb), - DMA_FROM_DEVICE); + DMA_TO_DEVICE); if (dma_mapping_error(ab->dev, buf_paddr)) return -ENOMEM; @@ -3071,7 +3053,7 @@ static int ath12k_dp_rx_h_defrag_reo_reinject(struct ath12k *ar, ath12k_hal_rx_buf_addr_info_set(&reo_ent_ring->buf_addr_info, link_paddr, cookie, - HAL_RX_BUF_RBM_WBM_CHIP0_IDLE_DESC_LIST); + idle_link_rbm); mpdu_info = u32_encode_bits(1, RX_MPDU_DESC_INFO0_MSDU_COUNT) | u32_encode_bits(0, RX_MPDU_DESC_INFO0_FRAG_FLAG) | @@ -3083,13 +3065,11 @@ static int ath12k_dp_rx_h_defrag_reo_reinject(struct ath12k *ar, reo_ent_ring->rx_mpdu_info.peer_meta_data = reo_dest_ring->rx_mpdu_info.peer_meta_data; - /* Firmware expects physical address to be filled in queue_addr_lo in - * the MLO scenario and in case of non MLO peer meta data needs to be - * filled. - * TODO: Need to handle for MLO scenario. - */ - reo_ent_ring->queue_addr_lo = reo_dest_ring->rx_mpdu_info.peer_meta_data; - reo_ent_ring->info0 = le32_encode_bits(dst_ind, + reo_ent_ring->queue_addr_lo = cpu_to_le32(lower_32_bits(rx_tid->paddr)); + queue_addr_hi = upper_32_bits(rx_tid->paddr); + reo_ent_ring->info0 = le32_encode_bits(queue_addr_hi, + HAL_REO_ENTR_RING_INFO0_QUEUE_ADDR_HI) | + le32_encode_bits(dst_ind, HAL_REO_ENTR_RING_INFO0_DEST_IND); reo_ent_ring->info1 = le32_encode_bits(rx_tid->cur_sn, @@ -3113,7 +3093,7 @@ err_free_desc: spin_unlock_bh(&dp->rx_desc_lock); err_unmap_dma: dma_unmap_single(ab->dev, buf_paddr, defrag_skb->len + skb_tailroom(defrag_skb), - DMA_FROM_DEVICE); + DMA_TO_DEVICE); return ret; } @@ -3346,7 +3326,8 @@ ath12k_dp_process_rx_err_buf(struct ath12k *ar, struct hal_reo_dest_ring *desc, if (!desc_info) { desc_info = ath12k_dp_get_rx_desc(ab, cookie); if (!desc_info) { - ath12k_warn(ab, "Invalid cookie in manual desc retrieval"); + ath12k_warn(ab, "Invalid cookie in DP rx error descriptor retrieval: 0x%x\n", + cookie); return -EINVAL; } } @@ -3451,7 +3432,7 @@ int ath12k_dp_rx_process_err(struct ath12k_base *ab, struct napi_struct *napi, (paddr - link_desc_banks[desc_bank].paddr); ath12k_hal_rx_msdu_link_info_get(link_desc_va, &num_msdus, msdu_cookies, &rbm); - if (rbm != HAL_RX_BUF_RBM_WBM_CHIP0_IDLE_DESC_LIST && + if (rbm != dp->idle_link_rbm && rbm != HAL_RX_BUF_RBM_SW3_BM && rbm != ab->hw_params->hal_params->rx_buf_rbm) { ab->soc_stats.invalid_rbm++; @@ -3765,7 +3746,8 @@ int ath12k_dp_rx_process_wbm_err(struct ath12k_base *ab, if (!desc_info) { desc_info = ath12k_dp_get_rx_desc(ab, err_info.cookie); if (!desc_info) { - ath12k_warn(ab, "Invalid cookie in manual desc retrieval"); + ath12k_warn(ab, "Invalid cookie in DP WBM rx error descriptor retrieval: 0x%x\n", + err_info.cookie); continue; } } @@ -3961,7 +3943,7 @@ void ath12k_dp_rx_free(struct ath12k_base *ab) ath12k_dp_srng_cleanup(ab, &dp->rx_refill_buf_ring.refill_buf_ring); - for (i = 0; i < ab->hw_params->num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params->num_rxdma_per_pdev; i++) { if (ab->hw_params->rx_mac_buf_ring) ath12k_dp_srng_cleanup(ab, &dp->rx_mac_buf_ring[i]); } @@ -3970,7 +3952,6 @@ void ath12k_dp_rx_free(struct ath12k_base *ab) ath12k_dp_srng_cleanup(ab, &dp->rxdma_err_dst_ring[i]); ath12k_dp_srng_cleanup(ab, &dp->rxdma_mon_buf_ring.refill_buf_ring); - ath12k_dp_srng_cleanup(ab, &dp->tx_mon_buf_ring.refill_buf_ring); ath12k_dp_rxdma_buf_free(ab); } @@ -4028,7 +4009,7 @@ int ath12k_dp_rxdma_ring_sel_config_wcn7850(struct ath12k_base *ab) struct ath12k_dp *dp = &ab->dp; struct htt_rx_ring_tlv_filter tlv_filter = {0}; u32 ring_id; - int ret; + int ret = 0; u32 hal_rx_desc_sz = ab->hal.hal_desc_sz; int i; @@ -4054,7 +4035,7 @@ int ath12k_dp_rxdma_ring_sel_config_wcn7850(struct ath12k_base *ab) * and modify the rx_desc struct */ - for (i = 0; i < ab->hw_params->num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params->num_rxdma_per_pdev; i++) { ring_id = dp->rx_mac_buf_ring[i].ring_id; ret = ath12k_dp_tx_htt_rx_filter_setup(ab, ring_id, i, HAL_RXDMA_BUF, @@ -4081,7 +4062,7 @@ int ath12k_dp_rx_htt_setup(struct ath12k_base *ab) } if (ab->hw_params->rx_mac_buf_ring) { - for (i = 0; i < ab->hw_params->num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params->num_rxdma_per_pdev; i++) { ring_id = dp->rx_mac_buf_ring[i].ring_id; ret = ath12k_dp_tx_htt_srng_setup(ab, ring_id, i, HAL_RXDMA_BUF); @@ -4113,15 +4094,6 @@ int ath12k_dp_rx_htt_setup(struct ath12k_base *ab) ret); return ret; } - - ring_id = dp->tx_mon_buf_ring.refill_buf_ring.ring_id; - ret = ath12k_dp_tx_htt_srng_setup(ab, ring_id, - 0, HAL_TX_MONITOR_BUF); - if (ret) { - ath12k_warn(ab, "failed to configure rxdma_mon_buf_ring %d\n", - ret); - return ret; - } } ret = ab->hw_params->hw_ops->rxdma_ring_sel_config(ab); @@ -4141,9 +4113,6 @@ int ath12k_dp_rx_alloc(struct ath12k_base *ab) idr_init(&dp->rxdma_mon_buf_ring.bufs_idr); spin_lock_init(&dp->rxdma_mon_buf_ring.idr_lock); - idr_init(&dp->tx_mon_buf_ring.bufs_idr); - spin_lock_init(&dp->tx_mon_buf_ring.idr_lock); - ret = ath12k_dp_srng_setup(ab, &dp->rx_refill_buf_ring.refill_buf_ring, HAL_RXDMA_BUF, 0, 0, @@ -4154,7 +4123,7 @@ int ath12k_dp_rx_alloc(struct ath12k_base *ab) } if (ab->hw_params->rx_mac_buf_ring) { - for (i = 0; i < ab->hw_params->num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params->num_rxdma_per_pdev; i++) { ret = ath12k_dp_srng_setup(ab, &dp->rx_mac_buf_ring[i], HAL_RXDMA_BUF, 1, @@ -4186,15 +4155,6 @@ int ath12k_dp_rx_alloc(struct ath12k_base *ab) ath12k_warn(ab, "failed to setup HAL_RXDMA_MONITOR_BUF\n"); return ret; } - - ret = ath12k_dp_srng_setup(ab, - &dp->tx_mon_buf_ring.refill_buf_ring, - HAL_TX_MONITOR_BUF, 0, 0, - DP_TX_MONITOR_BUF_RING_SIZE); - if (ret) { - ath12k_warn(ab, "failed to setup DP_TX_MONITOR_BUF_RING_SIZE\n"); - return ret; - } } ret = ath12k_dp_rxdma_buf_setup(ab); @@ -4223,7 +4183,7 @@ int ath12k_dp_rx_pdev_alloc(struct ath12k_base *ab, int mac_id) return ret; } - for (i = 0; i < ab->hw_params->num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params->num_rxdma_per_pdev; i++) { ring_id = dp->rxdma_mon_dst_ring[i].ring_id; ret = ath12k_dp_tx_htt_srng_setup(ab, ring_id, mac_id + i, @@ -4234,17 +4194,6 @@ int ath12k_dp_rx_pdev_alloc(struct ath12k_base *ab, int mac_id) i, ret); return ret; } - - ring_id = dp->tx_mon_dst_ring[i].ring_id; - ret = ath12k_dp_tx_htt_srng_setup(ab, ring_id, - mac_id + i, - HAL_TX_MONITOR_DST); - if (ret) { - ath12k_warn(ab, - "failed to configure tx_mon_dst_ring %d %d\n", - i, ret); - return ret; - } } out: return 0; diff --git a/drivers/net/wireless/ath/ath12k/dp_tx.c b/drivers/net/wireless/ath/ath12k/dp_tx.c index 9b6d7d72f57c4a452a48c3e431dfa630f3456210..c4cfa7cf7cb93aa9cd52a0bfbc16c262a5aa1bfc 100644 --- a/drivers/net/wireless/ath/ath12k/dp_tx.c +++ b/drivers/net/wireless/ath/ath12k/dp_tx.c @@ -124,6 +124,44 @@ static void ath12k_hal_tx_cmd_ext_desc_setup(struct ath12k_base *ab, HAL_TX_MSDU_EXT_INFO1_ENCRYPT_TYPE); } +#define HTT_META_DATA_ALIGNMENT 0x8 + +static void *ath12k_dp_metadata_align_skb(struct sk_buff *skb, u8 tail_len) +{ + struct sk_buff *tail; + void *metadata; + + if (unlikely(skb_cow_data(skb, tail_len, &tail) < 0)) + return NULL; + + metadata = pskb_put(skb, tail, tail_len); + memset(metadata, 0, tail_len); + return metadata; +} + +/* Preparing HTT Metadata when utilized with ext MSDU */ +static int ath12k_dp_prepare_htt_metadata(struct sk_buff *skb) +{ + struct hal_tx_msdu_metadata *desc_ext; + u8 htt_desc_size; + /* Size rounded of multiple of 8 bytes */ + u8 htt_desc_size_aligned; + + htt_desc_size = sizeof(struct hal_tx_msdu_metadata); + htt_desc_size_aligned = ALIGN(htt_desc_size, HTT_META_DATA_ALIGNMENT); + + desc_ext = ath12k_dp_metadata_align_skb(skb, htt_desc_size_aligned); + if (!desc_ext) + return -ENOMEM; + + desc_ext->info0 = le32_encode_bits(1, HAL_TX_MSDU_METADATA_INFO0_ENCRYPT_FLAG) | + le32_encode_bits(0, HAL_TX_MSDU_METADATA_INFO0_ENCRYPT_TYPE) | + le32_encode_bits(1, + HAL_TX_MSDU_METADATA_INFO0_HOST_TX_DESC_POOL); + + return 0; +} + int ath12k_dp_tx(struct ath12k *ar, struct ath12k_vif *arvif, struct sk_buff *skb) { @@ -145,6 +183,7 @@ int ath12k_dp_tx(struct ath12k *ar, struct ath12k_vif *arvif, u8 ring_selector, ring_map = 0; bool tcl_ring_retry; bool msdu_ext_desc = false; + bool add_htt_metadata = false; if (test_bit(ATH12K_FLAG_CRASH_FLUSH, &ar->ab->dev_flags)) return -ESHUTDOWN; @@ -248,6 +287,18 @@ tcl_ring_sel: goto fail_remove_tx_buf; } + if (!test_bit(ATH12K_FLAG_HW_CRYPTO_DISABLED, &ar->ab->dev_flags) && + !(skb_cb->flags & ATH12K_SKB_HW_80211_ENCAP) && + !(skb_cb->flags & ATH12K_SKB_CIPHER_SET) && + ieee80211_has_protected(hdr->frame_control)) { + /* Add metadata for sw encrypted vlan group traffic */ + add_htt_metadata = true; + msdu_ext_desc = true; + ti.flags0 |= u32_encode_bits(1, HAL_TCL_DATA_CMD_INFO2_TO_FW); + ti.encap_type = HAL_TCL_ENCAP_TYPE_RAW; + ti.encrypt_type = HAL_ENCRYPT_TYPE_OPEN; + } + tx_desc->skb = skb; tx_desc->mac_id = ar->pdev_idx; ti.desc_id = tx_desc->desc_id; @@ -269,6 +320,15 @@ tcl_ring_sel: msg = (struct hal_tx_msdu_ext_desc *)skb_ext_desc->data; ath12k_hal_tx_cmd_ext_desc_setup(ab, msg, &ti); + if (add_htt_metadata) { + ret = ath12k_dp_prepare_htt_metadata(skb_ext_desc); + if (ret < 0) { + ath12k_dbg(ab, ATH12K_DBG_DP_TX, + "Failed to add HTT meta data, dropping packet\n"); + goto fail_unmap_dma; + } + } + ti.paddr = dma_map_single(ab->dev, skb_ext_desc->data, skb_ext_desc->len, DMA_TO_DEVICE); ret = dma_mapping_error(ab->dev, ti.paddr); @@ -352,15 +412,15 @@ static void ath12k_dp_tx_free_txbuf(struct ath12k_base *ab, u8 pdev_id = ath12k_hw_mac_id_to_pdev_id(ab->hw_params, mac_id); skb_cb = ATH12K_SKB_CB(msdu); + ar = ab->pdevs[pdev_id].ar; dma_unmap_single(ab->dev, skb_cb->paddr, msdu->len, DMA_TO_DEVICE); if (skb_cb->paddr_ext_desc) dma_unmap_single(ab->dev, skb_cb->paddr_ext_desc, sizeof(struct hal_tx_msdu_ext_desc), DMA_TO_DEVICE); - dev_kfree_skb_any(msdu); + ieee80211_free_txskb(ar->ah->hw, msdu); - ar = ab->pdevs[pdev_id].ar; if (atomic_dec_and_test(&ar->dp.num_tx_pending)) wake_up(&ar->dp.tx_empty_waitq); } @@ -393,8 +453,12 @@ ath12k_dp_tx_htt_tx_complete_buf(struct ath12k_base *ab, if (ts->acked) { if (!(info->flags & IEEE80211_TX_CTL_NO_ACK)) { info->flags |= IEEE80211_TX_STAT_ACK; - info->status.ack_signal = ATH12K_DEFAULT_NOISE_FLOOR + - ts->ack_rssi; + info->status.ack_signal = ts->ack_rssi; + + if (!test_bit(WMI_TLV_SERVICE_HW_DB2DBM_CONVERSION_SUPPORT, + ab->wmi_ab.svc_map)) + info->status.ack_signal += ATH12K_DEFAULT_NOISE_FLOOR; + info->status.flags = IEEE80211_TX_STATUS_ACK_SIGNAL_VALID; } else { info->flags |= IEEE80211_TX_STAT_NOACK_TRANSMITTED; @@ -448,6 +512,7 @@ static void ath12k_dp_tx_complete_msdu(struct ath12k *ar, struct hal_tx_status *ts) { struct ath12k_base *ab = ar->ab; + struct ath12k_hw *ah = ar->ah; struct ieee80211_tx_info *info; struct ath12k_skb_cb *skb_cb; @@ -466,12 +531,12 @@ static void ath12k_dp_tx_complete_msdu(struct ath12k *ar, rcu_read_lock(); if (!rcu_dereference(ab->pdevs_active[ar->pdev_idx])) { - dev_kfree_skb_any(msdu); + ieee80211_free_txskb(ah->hw, msdu); goto exit; } if (!skb_cb->vif) { - dev_kfree_skb_any(msdu); + ieee80211_free_txskb(ah->hw, msdu); goto exit; } @@ -481,17 +546,39 @@ static void ath12k_dp_tx_complete_msdu(struct ath12k *ar, /* skip tx rate update from ieee80211_status*/ info->status.rates[0].idx = -1; - if (ts->status == HAL_WBM_TQM_REL_REASON_FRAME_ACKED && - !(info->flags & IEEE80211_TX_CTL_NO_ACK)) { - info->flags |= IEEE80211_TX_STAT_ACK; - info->status.ack_signal = ATH12K_DEFAULT_NOISE_FLOOR + - ts->ack_rssi; - info->status.flags = IEEE80211_TX_STATUS_ACK_SIGNAL_VALID; - } + switch (ts->status) { + case HAL_WBM_TQM_REL_REASON_FRAME_ACKED: + if (!(info->flags & IEEE80211_TX_CTL_NO_ACK)) { + info->flags |= IEEE80211_TX_STAT_ACK; + info->status.ack_signal = ts->ack_rssi; - if (ts->status == HAL_WBM_TQM_REL_REASON_CMD_REMOVE_TX && - (info->flags & IEEE80211_TX_CTL_NO_ACK)) - info->flags |= IEEE80211_TX_STAT_NOACK_TRANSMITTED; + if (!test_bit(WMI_TLV_SERVICE_HW_DB2DBM_CONVERSION_SUPPORT, + ab->wmi_ab.svc_map)) + info->status.ack_signal += ATH12K_DEFAULT_NOISE_FLOOR; + + info->status.flags = IEEE80211_TX_STATUS_ACK_SIGNAL_VALID; + } + break; + case HAL_WBM_TQM_REL_REASON_CMD_REMOVE_TX: + if (info->flags & IEEE80211_TX_CTL_NO_ACK) { + info->flags |= IEEE80211_TX_STAT_NOACK_TRANSMITTED; + break; + } + fallthrough; + case HAL_WBM_TQM_REL_REASON_CMD_REMOVE_MPDU: + case HAL_WBM_TQM_REL_REASON_DROP_THRESHOLD: + case HAL_WBM_TQM_REL_REASON_CMD_REMOVE_AGED_FRAMES: + /* The failure status is due to internal firmware tx failure + * hence drop the frame; do not update the status of frame to + * the upper layer + */ + ieee80211_free_txskb(ah->hw, msdu); + goto exit; + default: + ath12k_dbg(ab, ATH12K_DBG_DP_TX, "tx frame is not acked status %d\n", + ts->status); + break; + } /* NOTE: Tx rate status reporting. Tx completion status does not have * necessary information (for example nss) to build the tx rate. @@ -669,14 +756,6 @@ ath12k_dp_tx_get_ring_id_type(struct ath12k_base *ab, *htt_ring_id = HTT_RXDMA_MONITOR_DESC_RING; *htt_ring_type = HTT_SW_TO_HW_RING; break; - case HAL_TX_MONITOR_BUF: - *htt_ring_id = HTT_TX_MON_HOST2MON_BUF_RING; - *htt_ring_type = HTT_SW_TO_HW_RING; - break; - case HAL_TX_MONITOR_DST: - *htt_ring_id = HTT_TX_MON_MON2HOST_DEST_RING; - *htt_ring_type = HTT_HW_TO_SW_RING; - break; default: ath12k_warn(ab, "Unsupported ring type in DP :%d\n", ring_type); ret = -EINVAL; @@ -854,7 +933,7 @@ int ath12k_dp_tx_htt_h2t_ppdu_stats_req(struct ath12k *ar, u32 mask) int ret; int i; - for (i = 0; i < ab->hw_params->num_rxmda_per_pdev; i++) { + for (i = 0; i < ab->hw_params->num_rxdma_per_pdev; i++) { skb = ath12k_htc_alloc_skb(ab, len); if (!skb) return -ENOMEM; @@ -1044,13 +1123,7 @@ int ath12k_dp_tx_htt_monitor_mode_ring_config(struct ath12k *ar, bool reset) struct ath12k_base *ab = ar->ab; int ret; - ret = ath12k_dp_tx_htt_tx_monitor_mode_ring_config(ar, reset); - if (ret) { - ath12k_err(ab, "failed to setup tx monitor filter %d\n", ret); - return ret; - } - - ret = ath12k_dp_tx_htt_tx_monitor_mode_ring_config(ar, reset); + ret = ath12k_dp_tx_htt_rx_monitor_mode_ring_config(ar, reset); if (ret) { ath12k_err(ab, "failed to setup rx monitor filter %d\n", ret); return ret; @@ -1209,31 +1282,3 @@ err_free: dev_kfree_skb_any(skb); return ret; } - -int ath12k_dp_tx_htt_tx_monitor_mode_ring_config(struct ath12k *ar, bool reset) -{ - struct ath12k_base *ab = ar->ab; - struct ath12k_dp *dp = &ab->dp; - struct htt_tx_ring_tlv_filter tlv_filter = {0}; - int ret, ring_id; - - ring_id = dp->tx_mon_buf_ring.refill_buf_ring.ring_id; - - /* TODO: Need to set upstream/downstream tlv filters - * here - */ - - if (ab->hw_params->rxdma1_enable) { - ret = ath12k_dp_tx_htt_tx_filter_setup(ar->ab, ring_id, 0, - HAL_TX_MONITOR_BUF, - DP_RXDMA_REFILL_RING_SIZE, - &tlv_filter); - if (ret) { - ath12k_err(ab, - "failed to setup filter for monitor buf %d\n", ret); - return ret; - } - } - - return 0; -} diff --git a/drivers/net/wireless/ath/ath12k/dp_tx.h b/drivers/net/wireless/ath/ath12k/dp_tx.h index 436d77e5e9ee8e221596d72a9efe192cd5f7a01d..55ff8cc721e37f68967142941c11be77c1fc783c 100644 --- a/drivers/net/wireless/ath/ath12k/dp_tx.h +++ b/drivers/net/wireless/ath/ath12k/dp_tx.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2018-2021 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2022 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2022, 2024 Qualcomm Innovation Center, Inc. All rights reserved. */ #ifndef ATH12K_DP_TX_H @@ -12,7 +12,7 @@ struct ath12k_dp_htt_wbm_tx_status { bool acked; - int ack_rssi; + s8 ack_rssi; }; int ath12k_dp_tx_htt_h2t_ver_req_msg(struct ath12k_base *ab); @@ -36,6 +36,5 @@ int ath12k_dp_tx_htt_tx_filter_setup(struct ath12k_base *ab, u32 ring_id, int mac_id, enum hal_ring_type ring_type, int tx_buf_size, struct htt_tx_ring_tlv_filter *htt_tlv_filter); -int ath12k_dp_tx_htt_tx_monitor_mode_ring_config(struct ath12k *ar, bool reset); int ath12k_dp_tx_htt_monitor_mode_ring_config(struct ath12k *ar, bool reset); #endif diff --git a/drivers/net/wireless/ath/ath12k/hal.c b/drivers/net/wireless/ath/ath12k/hal.c index 78310da8cfe8e87d33d5307a66dbeb32cdd88fae..ca04bfae8bdccc578005d7c4c45a0d1de7a5f923 100644 --- a/drivers/net/wireless/ath/ath12k/hal.c +++ b/drivers/net/wireless/ath/ath12k/hal.c @@ -1969,14 +1969,15 @@ u32 ath12k_hal_ce_dst_status_get_length(struct hal_ce_srng_dst_status_desc *desc } void ath12k_hal_set_link_desc_addr(struct hal_wbm_link_desc *desc, u32 cookie, - dma_addr_t paddr) + dma_addr_t paddr, + enum hal_rx_buf_return_buf_manager rbm) { desc->buf_addr_info.info0 = le32_encode_bits((paddr & HAL_ADDR_LSB_REG_MASK), BUFFER_ADDR_INFO0_ADDR); desc->buf_addr_info.info1 = le32_encode_bits(((u64)paddr >> HAL_ADDR_MSB_REG_SHIFT), BUFFER_ADDR_INFO1_ADDR) | - le32_encode_bits(1, BUFFER_ADDR_INFO1_RET_BUF_MGR) | + le32_encode_bits(rbm, BUFFER_ADDR_INFO1_RET_BUF_MGR) | le32_encode_bits(cookie, BUFFER_ADDR_INFO1_SW_COOKIE); } diff --git a/drivers/net/wireless/ath/ath12k/hal.h b/drivers/net/wireless/ath/ath12k/hal.h index dbb9205bfa10dceeaded035b4a999bb09bdea899..8a78bb9a10bc15a3c715c2c2b03ed18e21b5816e 100644 --- a/drivers/net/wireless/ath/ath12k/hal.h +++ b/drivers/net/wireless/ath/ath12k/hal.h @@ -770,12 +770,12 @@ struct hal_srng_config { * enum hal_rx_buf_return_buf_manager - manager for returned rx buffers * * @HAL_RX_BUF_RBM_WBM_IDLE_BUF_LIST: Buffer returned to WBM idle buffer list - * @HAL_RX_BUF_RBM_WBM_CHIP0_IDLE_DESC_LIST: Descriptor returned to WBM idle - * descriptor list, where the chip 0 WBM is chosen in case of a multi-chip config - * @HAL_RX_BUF_RBM_WBM_CHIP1_IDLE_DESC_LIST: Descriptor returned to WBM idle - * descriptor list, where the chip 1 WBM is chosen in case of a multi-chip config - * @HAL_RX_BUF_RBM_WBM_CHIP2_IDLE_DESC_LIST: Descriptor returned to WBM idle - * descriptor list, where the chip 2 WBM is chosen in case of a multi-chip config + * @HAL_RX_BUF_RBM_WBM_DEV0_IDLE_DESC_LIST: Descriptor returned to WBM idle + * descriptor list, where the device 0 WBM is chosen in case of a multi-device config + * @HAL_RX_BUF_RBM_WBM_DEV1_IDLE_DESC_LIST: Descriptor returned to WBM idle + * descriptor list, where the device 1 WBM is chosen in case of a multi-device config + * @HAL_RX_BUF_RBM_WBM_DEV2_IDLE_DESC_LIST: Descriptor returned to WBM idle + * descriptor list, where the device 2 WBM is chosen in case of a multi-device config * @HAL_RX_BUF_RBM_FW_BM: Buffer returned to FW * @HAL_RX_BUF_RBM_SW0_BM: For ring 0 -- returned to host * @HAL_RX_BUF_RBM_SW1_BM: For ring 1 -- returned to host @@ -788,9 +788,9 @@ struct hal_srng_config { enum hal_rx_buf_return_buf_manager { HAL_RX_BUF_RBM_WBM_IDLE_BUF_LIST, - HAL_RX_BUF_RBM_WBM_CHIP0_IDLE_DESC_LIST, - HAL_RX_BUF_RBM_WBM_CHIP1_IDLE_DESC_LIST, - HAL_RX_BUF_RBM_WBM_CHIP2_IDLE_DESC_LIST, + HAL_RX_BUF_RBM_WBM_DEV0_IDLE_DESC_LIST, + HAL_RX_BUF_RBM_WBM_DEV1_IDLE_DESC_LIST, + HAL_RX_BUF_RBM_WBM_DEV2_IDLE_DESC_LIST, HAL_RX_BUF_RBM_FW_BM, HAL_RX_BUF_RBM_SW0_BM, HAL_RX_BUF_RBM_SW1_BM, @@ -1113,7 +1113,8 @@ dma_addr_t ath12k_hal_srng_get_tp_addr(struct ath12k_base *ab, dma_addr_t ath12k_hal_srng_get_hp_addr(struct ath12k_base *ab, struct hal_srng *srng); void ath12k_hal_set_link_desc_addr(struct hal_wbm_link_desc *desc, u32 cookie, - dma_addr_t paddr); + dma_addr_t paddr, + enum hal_rx_buf_return_buf_manager rbm); u32 ath12k_hal_ce_get_desc_size(enum hal_ce_desc type); void ath12k_hal_ce_src_set_desc(struct hal_ce_srng_src_desc *desc, dma_addr_t paddr, u32 len, u32 id, u8 byte_swap_data); diff --git a/drivers/net/wireless/ath/ath12k/hal_desc.h b/drivers/net/wireless/ath/ath12k/hal_desc.h index 63340256d3f64ef20c7759d0d2958477315a9b26..02b7db06b24ee542b62d9b4f8219bb4b2b635de6 100644 --- a/drivers/net/wireless/ath/ath12k/hal_desc.h +++ b/drivers/net/wireless/ath/ath12k/hal_desc.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2018-2021 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2022 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2022, 2024 Qualcomm Innovation Center, Inc. All rights reserved. */ #include "core.h" @@ -2048,6 +2048,19 @@ struct hal_wbm_release_ring { * fw with fw_reason2. * @HAL_WBM_TQM_REL_REASON_CMD_REMOVE_RESEAON3: Remove command initiated by * fw with fw_reason3. + * @HAL_WBM_TQM_REL_REASON_CMD_DISABLE_QUEUE: Remove command initiated by + * fw with disable queue. + * @HAL_WBM_TQM_REL_REASON_CMD_TILL_NONMATCHING: Remove command initiated by + * fw to remove all mpdu until 1st non-match. + * @HAL_WBM_TQM_REL_REASON_DROP_THRESHOLD: Dropped due to drop threshold + * criteria + * @HAL_WBM_TQM_REL_REASON_DROP_LINK_DESC_UNAVAIL: Dropped due to link desc + * not available + * @HAL_WBM_TQM_REL_REASON_DROP_OR_INVALID_MSDU: Dropped due drop bit set or + * null flow + * @HAL_WBM_TQM_REL_REASON_MULTICAST_DROP: Dropped due mcast drop set for VDEV + * @HAL_WBM_TQM_REL_REASON_VDEV_MISMATCH_DROP: Dropped due to being set with + * 'TCL_drop_reason' */ enum hal_wbm_tqm_rel_reason { HAL_WBM_TQM_REL_REASON_FRAME_ACKED, @@ -2058,6 +2071,13 @@ enum hal_wbm_tqm_rel_reason { HAL_WBM_TQM_REL_REASON_CMD_REMOVE_RESEAON1, HAL_WBM_TQM_REL_REASON_CMD_REMOVE_RESEAON2, HAL_WBM_TQM_REL_REASON_CMD_REMOVE_RESEAON3, + HAL_WBM_TQM_REL_REASON_CMD_DISABLE_QUEUE, + HAL_WBM_TQM_REL_REASON_CMD_TILL_NONMATCHING, + HAL_WBM_TQM_REL_REASON_DROP_THRESHOLD, + HAL_WBM_TQM_REL_REASON_DROP_LINK_DESC_UNAVAIL, + HAL_WBM_TQM_REL_REASON_DROP_OR_INVALID_MSDU, + HAL_WBM_TQM_REL_REASON_MULTICAST_DROP, + HAL_WBM_TQM_REL_REASON_VDEV_MISMATCH_DROP, }; struct hal_wbm_buffer_ring { @@ -2964,4 +2984,29 @@ struct hal_mon_dest_desc { * updated by SRNG. */ +#define HAL_TX_MSDU_METADATA_INFO0_ENCRYPT_FLAG BIT(8) +#define HAL_TX_MSDU_METADATA_INFO0_ENCRYPT_TYPE GENMASK(16, 15) +#define HAL_TX_MSDU_METADATA_INFO0_HOST_TX_DESC_POOL BIT(31) + +struct hal_tx_msdu_metadata { + __le32 info0; + __le32 rsvd0[6]; +} __packed; + +/* hal_tx_msdu_metadata + * valid_encrypt_type + * if set, encrypt type is valid + * encrypt_type + * 0 = NO_ENCRYPT, + * 1 = ENCRYPT, + * 2 ~ 3 - Reserved + * host_tx_desc_pool + * If set, Firmware allocates tx_descriptors + * in WAL_BUFFERID_TX_HOST_DATA_EXP,instead + * of WAL_BUFFERID_TX_TCL_DATA_EXP. + * Use cases: + * Any time firmware uses TQM-BYPASS for Data + * TID, firmware expect host to set this bit. + */ + #endif /* ATH12K_HAL_DESC_H */ diff --git a/drivers/net/wireless/ath/ath12k/hal_tx.h b/drivers/net/wireless/ath/ath12k/hal_tx.h index 7c837094a6f798e00fb628482d65961c61e03582..3cf5973771d78d0663fe300a627fdd2cf4399f70 100644 --- a/drivers/net/wireless/ath/ath12k/hal_tx.h +++ b/drivers/net/wireless/ath/ath12k/hal_tx.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause-Clear */ /* * Copyright (c) 2018-2021 The Linux Foundation. All rights reserved. - * Copyright (c) 2021-2022 Qualcomm Innovation Center, Inc. All rights reserved. + * Copyright (c) 2021-2022, 2024 Qualcomm Innovation Center, Inc. All rights reserved. */ #ifndef ATH12K_HAL_TX_H @@ -57,7 +57,7 @@ struct hal_tx_info { struct hal_tx_status { enum hal_wbm_rel_src_module buf_rel_source; enum hal_wbm_tqm_rel_reason status; - u8 ack_rssi; + s8 ack_rssi; u32 flags; /* %HAL_TX_STATUS_FLAGS_ */ u32 ppdu_id; u8 try_cnt; diff --git a/drivers/net/wireless/ath/ath12k/hif.h b/drivers/net/wireless/ath/ath12k/hif.h index 7f0926fe751d20f6c6fb65250bc01b41cbc58a8f..0e53ec269fa4f2a9d1290b09a9bdf23f3ec294a9 100644 --- a/drivers/net/wireless/ath/ath12k/hif.h +++ b/drivers/net/wireless/ath/ath12k/hif.h @@ -30,6 +30,7 @@ struct ath12k_hif_ops { void (*ce_irq_enable)(struct ath12k_base *ab); void (*ce_irq_disable)(struct ath12k_base *ab); void (*get_ce_msi_idx)(struct ath12k_base *ab, u32 ce_id, u32 *msi_idx); + int (*panic_handler)(struct ath12k_base *ab); }; static inline int ath12k_hif_map_service_to_pipe(struct ath12k_base *ab, u16 service_id, @@ -147,4 +148,12 @@ static inline void ath12k_hif_power_down(struct ath12k_base *ab, bool is_suspend ab->hif.ops->power_down(ab, is_suspend); } +static inline int ath12k_hif_panic_handler(struct ath12k_base *ab) +{ + if (!ab->hif.ops->panic_handler) + return NOTIFY_DONE; + + return ab->hif.ops->panic_handler(ab); +} + #endif /* ATH12K_HIF_H */ diff --git a/drivers/net/wireless/ath/ath12k/hw.c b/drivers/net/wireless/ath/ath12k/hw.c index f4c827015821573d53b9c1da39444815ad9aac62..2e11ea7635740ce289ee7d248e71caf3d6d9c78f 100644 --- a/drivers/net/wireless/ath/ath12k/hw.c +++ b/drivers/net/wireless/ath/ath12k/hw.c @@ -544,9 +544,6 @@ static const struct ath12k_hw_ring_mask ath12k_hw_ring_mask_qcn9274 = { }, .rx_mon_dest = { 0, 0, 0, - ATH12K_RX_MON_RING_MASK_0, - ATH12K_RX_MON_RING_MASK_1, - ATH12K_RX_MON_RING_MASK_2, }, .rx = { 0, 0, 0, 0, @@ -572,16 +569,15 @@ static const struct ath12k_hw_ring_mask ath12k_hw_ring_mask_qcn9274 = { ATH12K_HOST2RXDMA_RING_MASK_0, }, .tx_mon_dest = { - ATH12K_TX_MON_RING_MASK_0, - ATH12K_TX_MON_RING_MASK_1, + 0, 0, 0, }, }; static const struct ath12k_hw_ring_mask ath12k_hw_ring_mask_wcn7850 = { .tx = { ATH12K_TX_RING_MASK_0, + ATH12K_TX_RING_MASK_1, ATH12K_TX_RING_MASK_2, - ATH12K_TX_RING_MASK_4, }, .rx_mon_dest = { }, @@ -884,14 +880,15 @@ static const struct ath12k_hw_params ath12k_hw_params[] = { .hal_params = &ath12k_hw_hal_params_qcn9274, .rxdma1_enable = false, - .num_rxmda_per_pdev = 1, + .num_rxdma_per_pdev = 1, .num_rxdma_dst_ring = 0, .rx_mac_buf_ring = false, .vdev_start_delay = false, .interface_modes = BIT(NL80211_IFTYPE_STATION) | BIT(NL80211_IFTYPE_AP) | - BIT(NL80211_IFTYPE_MESH_POINT), + BIT(NL80211_IFTYPE_MESH_POINT) | + BIT(NL80211_IFTYPE_AP_VLAN), .supports_monitor = false, .idle_ps = false, @@ -926,6 +923,7 @@ static const struct ath12k_hw_params ath12k_hw_params[] = { .supports_sta_ps = false, .acpi_guid = NULL, + .supports_dynamic_smps_6ghz = true, }, { .name = "wcn7850 hw2.0", @@ -956,7 +954,7 @@ static const struct ath12k_hw_params ath12k_hw_params[] = { .hal_params = &ath12k_hw_hal_params_wcn7850, .rxdma1_enable = false, - .num_rxmda_per_pdev = 2, + .num_rxdma_per_pdev = 2, .num_rxdma_dst_ring = 1, .rx_mac_buf_ring = true, .vdev_start_delay = true, @@ -1001,6 +999,7 @@ static const struct ath12k_hw_params ath12k_hw_params[] = { .supports_sta_ps = true, .acpi_guid = &wcn7850_uuid, + .supports_dynamic_smps_6ghz = false, }, { .name = "qcn9274 hw2.0", @@ -1029,14 +1028,15 @@ static const struct ath12k_hw_params ath12k_hw_params[] = { .hal_params = &ath12k_hw_hal_params_qcn9274, .rxdma1_enable = false, - .num_rxmda_per_pdev = 1, + .num_rxdma_per_pdev = 1, .num_rxdma_dst_ring = 0, .rx_mac_buf_ring = false, .vdev_start_delay = false, .interface_modes = BIT(NL80211_IFTYPE_STATION) | BIT(NL80211_IFTYPE_AP) | - BIT(NL80211_IFTYPE_MESH_POINT), + BIT(NL80211_IFTYPE_MESH_POINT) | + BIT(NL80211_IFTYPE_AP_VLAN), .supports_monitor = false, .idle_ps = false, @@ -1071,6 +1071,7 @@ static const struct ath12k_hw_params ath12k_hw_params[] = { .supports_sta_ps = false, .acpi_guid = NULL, + .supports_dynamic_smps_6ghz = true, }, }; diff --git a/drivers/net/wireless/ath/ath12k/hw.h b/drivers/net/wireless/ath/ath12k/hw.h index 3f450ee93f34bbb9463619c332a95519beb72747..af33bf11416b7100d36be6dd38e69b315e46f1a5 100644 --- a/drivers/net/wireless/ath/ath12k/hw.h +++ b/drivers/net/wireless/ath/ath12k/hw.h @@ -80,6 +80,7 @@ #define TARGET_RX_BATCHMODE 1 #define TARGET_RX_PEER_METADATA_VER_V1A 2 #define TARGET_RX_PEER_METADATA_VER_V1B 3 +#define TARGET_EMA_MAX_PROFILE_PERIOD 8 #define ATH12K_HW_DEFAULT_QUEUE 0 #define ATH12K_HW_MAX_QUEUES 4 @@ -174,7 +175,7 @@ struct ath12k_hw_params { const struct ath12k_hw_hal_params *hal_params; bool rxdma1_enable:1; - int num_rxmda_per_pdev; + int num_rxdma_per_pdev; int num_rxdma_dst_ring; bool rx_mac_buf_ring:1; bool vdev_start_delay:1; @@ -215,6 +216,7 @@ struct ath12k_hw_params { bool supports_sta_ps; const guid_t *acpi_guid; + bool supports_dynamic_smps_6ghz; }; struct ath12k_hw_ops { diff --git a/drivers/net/wireless/ath/ath12k/mac.c b/drivers/net/wireless/ath/ath12k/mac.c index 805cb084484a4f6acfccc4faab6c54592b504d8d..509c02bffdae87dc413d7069f4677e533e088fd9 100644 --- a/drivers/net/wireless/ath/ath12k/mac.c +++ b/drivers/net/wireless/ath/ath12k/mac.c @@ -91,6 +91,10 @@ static const struct ieee80211_channel ath12k_5ghz_channels[] = { }; static const struct ieee80211_channel ath12k_6ghz_channels[] = { + /* Operating Class 136 */ + CHAN6G(2, 5935, 0), + + /* Operating Classes 131-135 */ CHAN6G(1, 5955, 0), CHAN6G(5, 5975, 0), CHAN6G(9, 5995, 0), @@ -863,9 +867,12 @@ static int ath12k_mac_vdev_setup_sync(struct ath12k *ar) static int ath12k_monitor_vdev_up(struct ath12k *ar, int vdev_id) { + struct ath12k_wmi_vdev_up_params params = {}; int ret; - ret = ath12k_wmi_vdev_up(ar, vdev_id, 0, ar->mac_addr); + params.vdev_id = vdev_id; + params.bssid = ar->mac_addr; + ret = ath12k_wmi_vdev_up(ar, ¶ms); if (ret) { ath12k_warn(ar->ab, "failed to put up monitor vdev %i: %d\n", vdev_id, ret); @@ -882,6 +889,7 @@ static int ath12k_mac_monitor_vdev_start(struct ath12k *ar, int vdev_id, { struct ieee80211_channel *channel; struct wmi_vdev_start_req_arg arg = {}; + struct ath12k_wmi_vdev_up_params params = {}; int ret; lockdep_assert_held(&ar->conf_mutex); @@ -922,7 +930,9 @@ static int ath12k_mac_monitor_vdev_start(struct ath12k *ar, int vdev_id, return ret; } - ret = ath12k_wmi_vdev_up(ar, vdev_id, 0, ar->mac_addr); + params.vdev_id = vdev_id; + params.bssid = ar->mac_addr; + ret = ath12k_wmi_vdev_up(ar, ¶ms); if (ret) { ath12k_warn(ar->ab, "failed to put up monitor vdev %i: %d\n", vdev_id, ret); @@ -1289,37 +1299,188 @@ static int ath12k_mac_remove_vendor_ie(struct sk_buff *skb, unsigned int oui, return 0; } +static void ath12k_mac_set_arvif_ies(struct ath12k_vif *arvif, struct sk_buff *bcn, + u8 bssid_index, bool *nontx_profile_found) +{ + struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)bcn->data; + const struct element *elem, *nontx, *index, *nie; + const u8 *start, *tail; + u16 rem_len; + u8 i; + + start = bcn->data + ieee80211_get_hdrlen_from_skb(bcn) + sizeof(mgmt->u.beacon); + tail = skb_tail_pointer(bcn); + rem_len = tail - start; + + arvif->rsnie_present = false; + arvif->wpaie_present = false; + + if (cfg80211_find_ie(WLAN_EID_RSN, start, rem_len)) + arvif->rsnie_present = true; + if (cfg80211_find_vendor_ie(WLAN_OUI_MICROSOFT, WLAN_OUI_TYPE_MICROSOFT_WPA, + start, rem_len)) + arvif->wpaie_present = true; + + /* Return from here for the transmitted profile */ + if (!bssid_index) + return; + + /* Initial rsnie_present for the nontransmitted profile is set to be same as that + * of the transmitted profile. It will be changed if security configurations are + * different. + */ + *nontx_profile_found = false; + for_each_element_id(elem, WLAN_EID_MULTIPLE_BSSID, start, rem_len) { + /* Fixed minimum MBSSID element length with at least one + * nontransmitted BSSID profile is 12 bytes as given below; + * 1 (max BSSID indicator) + + * 2 (Nontransmitted BSSID profile: Subelement ID + length) + + * 4 (Nontransmitted BSSID Capabilities: tag + length + info) + * 2 (Nontransmitted BSSID SSID: tag + length) + * 3 (Nontransmitted BSSID Index: tag + length + BSSID index + */ + if (elem->datalen < 12 || elem->data[0] < 1) + continue; /* Max BSSID indicator must be >=1 */ + + for_each_element(nontx, elem->data + 1, elem->datalen - 1) { + start = nontx->data; + + if (nontx->id != 0 || nontx->datalen < 4) + continue; /* Invalid nontransmitted profile */ + + if (nontx->data[0] != WLAN_EID_NON_TX_BSSID_CAP || + nontx->data[1] != 2) { + continue; /* Missing nontransmitted BSS capabilities */ + } + + if (nontx->data[4] != WLAN_EID_SSID) + continue; /* Missing SSID for nontransmitted BSS */ + + index = cfg80211_find_elem(WLAN_EID_MULTI_BSSID_IDX, + start, nontx->datalen); + if (!index || index->datalen < 1 || index->data[0] == 0) + continue; /* Invalid MBSSID Index element */ + + if (index->data[0] == bssid_index) { + *nontx_profile_found = true; + if (cfg80211_find_ie(WLAN_EID_RSN, + nontx->data, + nontx->datalen)) { + arvif->rsnie_present = true; + return; + } else if (!arvif->rsnie_present) { + return; /* Both tx and nontx BSS are open */ + } + + nie = cfg80211_find_ext_elem(WLAN_EID_EXT_NON_INHERITANCE, + nontx->data, + nontx->datalen); + if (!nie || nie->datalen < 2) + return; /* Invalid non-inheritance element */ + + for (i = 1; i < nie->datalen - 1; i++) { + if (nie->data[i] == WLAN_EID_RSN) { + arvif->rsnie_present = false; + break; + } + } + + return; + } + } + } +} + +static int ath12k_mac_setup_bcn_tmpl_ema(struct ath12k_vif *arvif) +{ + struct ieee80211_bss_conf *bss_conf = &arvif->vif->bss_conf; + struct ath12k_wmi_bcn_tmpl_ema_arg ema_args; + struct ieee80211_ema_beacons *beacons; + struct ath12k_vif *tx_arvif; + bool nontx_profile_found = false; + int ret = 0; + u8 i; + + tx_arvif = ath12k_vif_to_arvif(arvif->vif->mbssid_tx_vif); + beacons = ieee80211_beacon_get_template_ema_list(ath12k_ar_to_hw(tx_arvif->ar), + tx_arvif->vif, 0); + if (!beacons || !beacons->cnt) { + ath12k_warn(arvif->ar->ab, + "failed to get ema beacon templates from mac80211\n"); + return -EPERM; + } + + if (tx_arvif == arvif) + ath12k_mac_set_arvif_ies(arvif, beacons->bcn[0].skb, 0, NULL); + + for (i = 0; i < beacons->cnt; i++) { + if (tx_arvif != arvif && !nontx_profile_found) + ath12k_mac_set_arvif_ies(arvif, beacons->bcn[i].skb, + bss_conf->bssid_index, + &nontx_profile_found); + + ema_args.bcn_cnt = beacons->cnt; + ema_args.bcn_index = i; + ret = ath12k_wmi_bcn_tmpl(tx_arvif->ar, tx_arvif->vdev_id, + &beacons->bcn[i].offs, + beacons->bcn[i].skb, &ema_args); + if (ret) { + ath12k_warn(tx_arvif->ar->ab, + "failed to set ema beacon template id %i error %d\n", + i, ret); + break; + } + } + + if (tx_arvif != arvif && !nontx_profile_found) + ath12k_warn(arvif->ar->ab, + "nontransmitted bssid index %u not found in beacon template\n", + bss_conf->bssid_index); + + ieee80211_beacon_free_ema_list(beacons); + return ret; +} + static int ath12k_mac_setup_bcn_tmpl(struct ath12k_vif *arvif) { + struct ath12k_vif *tx_arvif = arvif; struct ath12k *ar = arvif->ar; struct ath12k_base *ab = ar->ab; - struct ieee80211_hw *hw = ath12k_ar_to_hw(ar); struct ieee80211_vif *vif = arvif->vif; struct ieee80211_mutable_offsets offs = {}; + bool nontx_profile_found = false; struct sk_buff *bcn; - struct ieee80211_mgmt *mgmt; - u8 *ies; int ret; if (arvif->vdev_type != WMI_VDEV_TYPE_AP) return 0; - bcn = ieee80211_beacon_get_template(hw, vif, &offs, 0); + if (vif->mbssid_tx_vif) { + tx_arvif = ath12k_vif_to_arvif(vif->mbssid_tx_vif); + if (tx_arvif != arvif && arvif->is_up) + return 0; + + if (vif->bss_conf.ema_ap) + return ath12k_mac_setup_bcn_tmpl_ema(arvif); + } + + bcn = ieee80211_beacon_get_template(ath12k_ar_to_hw(tx_arvif->ar), tx_arvif->vif, + &offs, 0); if (!bcn) { ath12k_warn(ab, "failed to get beacon template from mac80211\n"); return -EPERM; } - ies = bcn->data + ieee80211_get_hdrlen_from_skb(bcn); - ies += sizeof(mgmt->u.beacon); - - if (cfg80211_find_ie(WLAN_EID_RSN, ies, (skb_tail_pointer(bcn) - ies))) - arvif->rsnie_present = true; - - if (cfg80211_find_vendor_ie(WLAN_OUI_MICROSOFT, - WLAN_OUI_TYPE_MICROSOFT_WPA, - ies, (skb_tail_pointer(bcn) - ies))) - arvif->wpaie_present = true; + if (tx_arvif == arvif) { + ath12k_mac_set_arvif_ies(arvif, bcn, 0, NULL); + } else { + ath12k_mac_set_arvif_ies(arvif, bcn, + arvif->vif->bss_conf.bssid_index, + &nontx_profile_found); + if (!nontx_profile_found) + ath12k_warn(ab, + "nontransmitted profile not found in beacon template\n"); + } if (arvif->vif->type == NL80211_IFTYPE_AP && arvif->vif->p2p) { ret = ath12k_mac_setup_bcn_p2p_ie(arvif, bcn); @@ -1344,7 +1505,7 @@ static int ath12k_mac_setup_bcn_tmpl(struct ath12k_vif *arvif) } } - ret = ath12k_wmi_bcn_tmpl(ar, arvif->vdev_id, &offs, bcn); + ret = ath12k_wmi_bcn_tmpl(ar, arvif->vdev_id, &offs, bcn, NULL); if (ret) ath12k_warn(ab, "failed to submit beacon template command: %d\n", @@ -1358,6 +1519,7 @@ free_bcn_skb: static void ath12k_control_beaconing(struct ath12k_vif *arvif, struct ieee80211_bss_conf *info) { + struct ath12k_wmi_vdev_up_params params = {}; struct ath12k *ar = arvif->ar; int ret; @@ -1385,8 +1547,15 @@ static void ath12k_control_beaconing(struct ath12k_vif *arvif, ether_addr_copy(arvif->bssid, info->bssid); - ret = ath12k_wmi_vdev_up(arvif->ar, arvif->vdev_id, arvif->aid, - arvif->bssid); + params.vdev_id = arvif->vdev_id; + params.aid = arvif->aid; + params.bssid = arvif->bssid; + if (arvif->vif->mbssid_tx_vif) { + params.tx_bssid = ath12k_vif_to_arvif(arvif->vif->mbssid_tx_vif)->bssid; + params.nontx_profile_idx = info->bssid_index; + params.nontx_profile_cnt = 1 << info->bssid_indicator; + } + ret = ath12k_wmi_vdev_up(arvif->ar, ¶ms); if (ret) { ath12k_warn(ar->ab, "failed to bring up vdev %d: %i\n", arvif->vdev_id, ret); @@ -2028,18 +2197,89 @@ static void ath12k_peer_assoc_h_he(struct ath12k *ar, } } +static void ath12k_peer_assoc_h_he_6ghz(struct ath12k *ar, + struct ieee80211_vif *vif, + struct ieee80211_sta *sta, + struct ath12k_wmi_peer_assoc_arg *arg) +{ + const struct ieee80211_sta_he_cap *he_cap = &sta->deflink.he_cap; + struct cfg80211_chan_def def; + enum nl80211_band band; + u8 ampdu_factor, mpdu_density; + + if (WARN_ON(ath12k_mac_vif_chan(vif, &def))) + return; + + band = def.chan->band; + + if (!arg->he_flag || band != NL80211_BAND_6GHZ || !sta->deflink.he_6ghz_capa.capa) + return; + + if (sta->deflink.bandwidth == IEEE80211_STA_RX_BW_40) + arg->bw_40 = true; + + if (sta->deflink.bandwidth == IEEE80211_STA_RX_BW_80) + arg->bw_80 = true; + + if (sta->deflink.bandwidth == IEEE80211_STA_RX_BW_160) + arg->bw_160 = true; + + if (sta->deflink.bandwidth == IEEE80211_STA_RX_BW_320) + arg->bw_320 = true; + + arg->peer_he_caps_6ghz = le16_to_cpu(sta->deflink.he_6ghz_capa.capa); + + mpdu_density = u32_get_bits(arg->peer_he_caps_6ghz, + IEEE80211_HE_6GHZ_CAP_MIN_MPDU_START); + arg->peer_mpdu_density = ath12k_parse_mpdudensity(mpdu_density); + + /* From IEEE Std 802.11ax-2021 - Section 10.12.2: An HE STA shall be capable of + * receiving A-MPDU where the A-MPDU pre-EOF padding length is up to the value + * indicated by the Maximum A-MPDU Length Exponent Extension field in the HE + * Capabilities element and the Maximum A-MPDU Length Exponent field in HE 6 GHz + * Band Capabilities element in the 6 GHz band. + * + * Here, we are extracting the Max A-MPDU Exponent Extension from HE caps and + * factor is the Maximum A-MPDU Length Exponent from HE 6 GHZ Band capability. + */ + ampdu_factor = u8_get_bits(he_cap->he_cap_elem.mac_cap_info[3], + IEEE80211_HE_MAC_CAP3_MAX_AMPDU_LEN_EXP_MASK) + + u32_get_bits(arg->peer_he_caps_6ghz, + IEEE80211_HE_6GHZ_CAP_MAX_AMPDU_LEN_EXP); + + arg->peer_max_mpdu = (1u << (IEEE80211_HE_6GHZ_MAX_AMPDU_FACTOR + + ampdu_factor)) - 1; +} + +static int ath12k_get_smps_from_capa(const struct ieee80211_sta_ht_cap *ht_cap, + const struct ieee80211_he_6ghz_capa *he_6ghz_capa, + int *smps) +{ + if (!ht_cap->ht_supported && !he_6ghz_capa->capa) + return -EOPNOTSUPP; + + if (ht_cap->ht_supported) + *smps = u16_get_bits(ht_cap->cap, IEEE80211_HT_CAP_SM_PS); + else + *smps = le16_get_bits(he_6ghz_capa->capa, + IEEE80211_HE_6GHZ_CAP_SM_PS); + + if (*smps >= ARRAY_SIZE(ath12k_smps_map)) + return -EINVAL; + + return 0; +} + static void ath12k_peer_assoc_h_smps(struct ieee80211_sta *sta, struct ath12k_wmi_peer_assoc_arg *arg) { + const struct ieee80211_he_6ghz_capa *he_6ghz_capa = &sta->deflink.he_6ghz_capa; const struct ieee80211_sta_ht_cap *ht_cap = &sta->deflink.ht_cap; int smps; - if (!ht_cap->ht_supported) + if (ath12k_get_smps_from_capa(ht_cap, he_6ghz_capa, &smps)) return; - smps = ht_cap->cap & IEEE80211_HT_CAP_SM_PS; - smps >>= IEEE80211_HT_CAP_SM_PS_SHIFT; - switch (smps) { case WLAN_HT_CAP_SM_PS_STATIC: arg->static_mimops_flag = true; @@ -2500,6 +2740,7 @@ static void ath12k_peer_assoc_prepare(struct ath12k *ar, ath12k_peer_assoc_h_ht(ar, vif, sta, arg); ath12k_peer_assoc_h_vht(ar, vif, sta, arg); ath12k_peer_assoc_h_he(ar, vif, sta, arg); + ath12k_peer_assoc_h_he_6ghz(ar, vif, sta, arg); ath12k_peer_assoc_h_eht(ar, vif, sta, arg); ath12k_peer_assoc_h_qos(ar, vif, sta, arg); ath12k_peer_assoc_h_phymode(ar, vif, sta, arg); @@ -2510,18 +2751,14 @@ static void ath12k_peer_assoc_prepare(struct ath12k *ar, static int ath12k_setup_peer_smps(struct ath12k *ar, struct ath12k_vif *arvif, const u8 *addr, - const struct ieee80211_sta_ht_cap *ht_cap) + const struct ieee80211_sta_ht_cap *ht_cap, + const struct ieee80211_he_6ghz_capa *he_6ghz_capa) { - int smps; + int smps, ret = 0; - if (!ht_cap->ht_supported) - return 0; - - smps = ht_cap->cap & IEEE80211_HT_CAP_SM_PS; - smps >>= IEEE80211_HT_CAP_SM_PS_SHIFT; - - if (smps >= ARRAY_SIZE(ath12k_smps_map)) - return -EINVAL; + ret = ath12k_get_smps_from_capa(ht_cap, he_6ghz_capa, &smps); + if (ret < 0) + return ret; return ath12k_wmi_set_peer_param(ar, addr, arvif->vdev_id, WMI_PEER_MIMO_PS_STATE, @@ -2533,6 +2770,7 @@ static void ath12k_bss_assoc(struct ath12k *ar, struct ieee80211_bss_conf *bss_conf) { struct ieee80211_vif *vif = arvif->vif; + struct ath12k_wmi_vdev_up_params params = {}; struct ath12k_wmi_peer_assoc_arg peer_arg; struct ieee80211_sta *ap_sta; struct ath12k_peer *peer; @@ -2572,7 +2810,8 @@ static void ath12k_bss_assoc(struct ath12k *ar, } ret = ath12k_setup_peer_smps(ar, arvif, bss_conf->bssid, - &ap_sta->deflink.ht_cap); + &ap_sta->deflink.ht_cap, + &ap_sta->deflink.he_6ghz_capa); if (ret) { ath12k_warn(ar->ab, "failed to setup peer SMPS for vdev %d: %d\n", arvif->vdev_id, ret); @@ -2584,7 +2823,10 @@ static void ath12k_bss_assoc(struct ath12k *ar, arvif->aid = vif->cfg.aid; ether_addr_copy(arvif->bssid, bss_conf->bssid); - ret = ath12k_wmi_vdev_up(ar, arvif->vdev_id, arvif->aid, arvif->bssid); + params.vdev_id = arvif->vdev_id; + params.aid = arvif->aid; + params.bssid = arvif->bssid; + ret = ath12k_wmi_vdev_up(ar, ¶ms); if (ret) { ath12k_warn(ar->ab, "failed to set vdev %d up: %d\n", arvif->vdev_id, ret); @@ -3879,7 +4121,8 @@ static int ath12k_station_assoc(struct ath12k *ar, return 0; ret = ath12k_setup_peer_smps(ar, arvif, sta->addr, - &sta->deflink.ht_cap); + &sta->deflink.ht_cap, + &sta->deflink.he_6ghz_capa); if (ret) { ath12k_warn(ar->ab, "failed to setup peer SMPS for vdev %d: %d\n", arvif->vdev_id, ret); @@ -5269,6 +5512,7 @@ static void ath12k_mac_setup_sband_iftype_data(struct ath12k *ar, static int __ath12k_set_antenna(struct ath12k *ar, u32 tx_ant, u32 rx_ant) { + struct ath12k_hw *ah = ath12k_ar_to_ah(ar); int ret; lockdep_assert_held(&ar->conf_mutex); @@ -5289,8 +5533,8 @@ static int __ath12k_set_antenna(struct ath12k *ar, u32 tx_ant, u32 rx_ant) ar->cfg_tx_chainmask = tx_ant; ar->cfg_rx_chainmask = rx_ant; - if (ar->state != ATH12K_STATE_ON && - ar->state != ATH12K_STATE_RESTARTED) + if (ah->state != ATH12K_HW_STATE_ON && + ah->state != ATH12K_HW_STATE_RESTARTED) return 0; ret = ath12k_wmi_pdev_set_param(ar, WMI_PDEV_PARAM_TX_CHAIN_MASK, @@ -5614,27 +5858,14 @@ static void ath12k_mac_wait_reconfigure(struct ath12k_base *ab) static int ath12k_mac_start(struct ath12k *ar) { + struct ath12k_hw *ah = ar->ah; struct ath12k_base *ab = ar->ab; struct ath12k_pdev *pdev = ar->pdev; int ret; - mutex_lock(&ar->conf_mutex); + lockdep_assert_held(&ah->hw_mutex); - switch (ar->state) { - case ATH12K_STATE_OFF: - ar->state = ATH12K_STATE_ON; - break; - case ATH12K_STATE_RESTARTING: - ar->state = ATH12K_STATE_RESTARTED; - ath12k_mac_wait_reconfigure(ab); - break; - case ATH12K_STATE_RESTARTED: - case ATH12K_STATE_WEDGED: - case ATH12K_STATE_ON: - WARN_ON(1); - ret = -EINVAL; - goto err; - } + mutex_lock(&ar->conf_mutex); ret = ath12k_wmi_pdev_set_param(ar, WMI_PDEV_PARAM_PMF_QOS, 1, pdev->pdev_id); @@ -5726,7 +5957,6 @@ static int ath12k_mac_start(struct ath12k *ar) return 0; err: - ar->state = ATH12K_STATE_OFF; mutex_unlock(&ar->conf_mutex); return ret; @@ -5749,9 +5979,30 @@ static int ath12k_mac_op_start(struct ieee80211_hw *hw) ath12k_drain_tx(ah); + guard(mutex)(&ah->hw_mutex); + + switch (ah->state) { + case ATH12K_HW_STATE_OFF: + ah->state = ATH12K_HW_STATE_ON; + break; + case ATH12K_HW_STATE_RESTARTING: + ah->state = ATH12K_HW_STATE_RESTARTED; + ath12k_mac_wait_reconfigure(ah->ab); + break; + case ATH12K_HW_STATE_RESTARTED: + case ATH12K_HW_STATE_WEDGED: + case ATH12K_HW_STATE_ON: + ah->state = ATH12K_HW_STATE_OFF; + + WARN_ON(1); + return -EINVAL; + } + for_each_ar(ah, ar, i) { ret = ath12k_mac_start(ar); if (ret) { + ah->state = ATH12K_HW_STATE_OFF; + ath12k_err(ar->ab, "fail to start mac operations in pdev idx %d ret %d\n", ar->pdev_idx, ret); goto fail_start; @@ -5759,11 +6010,13 @@ static int ath12k_mac_op_start(struct ieee80211_hw *hw) } return 0; + fail_start: for (; i > 0; i--) { ar = ath12k_ah_to_ar(ah, i - 1); ath12k_mac_stop(ar); } + return ret; } @@ -5826,9 +6079,12 @@ int ath12k_mac_rfkill_enable_radio(struct ath12k *ar, bool enable) static void ath12k_mac_stop(struct ath12k *ar) { + struct ath12k_hw *ah = ar->ah; struct htt_ppdu_stats_info *ppdu_stats, *tmp; int ret; + lockdep_assert_held(&ah->hw_mutex); + mutex_lock(&ar->conf_mutex); ret = ath12k_mac_config_mon_status_default(ar, false); if (ret && (ret != -EOPNOTSUPP)) @@ -5836,7 +6092,6 @@ static void ath12k_mac_stop(struct ath12k *ar) ret); clear_bit(ATH12K_CAC_RUNNING, &ar->dev_flags); - ar->state = ATH12K_STATE_OFF; mutex_unlock(&ar->conf_mutex); cancel_delayed_work_sync(&ar->scan.timeout); @@ -5865,8 +6120,14 @@ static void ath12k_mac_op_stop(struct ieee80211_hw *hw) ath12k_drain_tx(ah); + mutex_lock(&ah->hw_mutex); + + ah->state = ATH12K_HW_STATE_OFF; + for_each_ar(ah, ar, i) ath12k_mac_stop(ar); + + mutex_unlock(&ah->hw_mutex); } static u8 @@ -5892,17 +6153,59 @@ ath12k_mac_get_vdev_stats_id(struct ath12k_vif *arvif) return vdev_stats_id; } -static void ath12k_mac_setup_vdev_create_arg(struct ath12k_vif *arvif, - struct ath12k_wmi_vdev_create_arg *arg) +static int ath12k_mac_setup_vdev_params_mbssid(struct ath12k_vif *arvif, + u32 *flags, u32 *tx_vdev_id) +{ + struct ieee80211_vif *tx_vif = arvif->vif->mbssid_tx_vif; + struct ath12k *ar = arvif->ar; + struct ath12k_vif *tx_arvif; + + if (!tx_vif) + return 0; + + tx_arvif = ath12k_vif_to_arvif(tx_vif); + + if (arvif->vif->bss_conf.nontransmitted) { + if (ar->ah->hw->wiphy != ieee80211_vif_to_wdev(tx_vif)->wiphy) + return -EINVAL; + + *flags = WMI_VDEV_MBSSID_FLAGS_NON_TRANSMIT_AP; + *tx_vdev_id = tx_arvif->vdev_id; + } else if (tx_arvif == arvif) { + *flags = WMI_VDEV_MBSSID_FLAGS_TRANSMIT_AP; + } else { + return -EINVAL; + } + + if (arvif->vif->bss_conf.ema_ap) + *flags |= WMI_VDEV_MBSSID_FLAGS_EMA_MODE; + + return 0; +} + +static int ath12k_mac_setup_vdev_create_arg(struct ath12k_vif *arvif, + struct ath12k_wmi_vdev_create_arg *arg) { struct ath12k *ar = arvif->ar; struct ath12k_pdev *pdev = ar->pdev; + int ret; arg->if_id = arvif->vdev_id; arg->type = arvif->vdev_type; arg->subtype = arvif->vdev_subtype; arg->pdev_id = pdev->pdev_id; + arg->mbssid_flags = WMI_VDEV_MBSSID_FLAGS_NON_MBSSID_AP; + arg->mbssid_tx_vdev_id = 0; + if (!test_bit(WMI_TLV_SERVICE_MBSS_PARAM_IN_VDEV_START_SUPPORT, + ar->ab->wmi_ab.svc_map)) { + ret = ath12k_mac_setup_vdev_params_mbssid(arvif, + &arg->mbssid_flags, + &arg->mbssid_tx_vdev_id); + if (ret) + return ret; + } + if (pdev->cap.supported_bands & WMI_HOST_WLAN_2G_CAP) { arg->chains[NL80211_BAND_2GHZ].tx = ar->num_tx_chains; arg->chains[NL80211_BAND_2GHZ].rx = ar->num_rx_chains; @@ -5918,6 +6221,7 @@ static void ath12k_mac_setup_vdev_create_arg(struct ath12k_vif *arvif, } arg->if_stats_id = ath12k_mac_get_vdev_stats_id(arvif); + return 0; } static u32 @@ -6099,7 +6403,12 @@ static int ath12k_mac_vdev_create(struct ath12k *ar, struct ieee80211_vif *vif) for (i = 0; i < ARRAY_SIZE(vif->hw_queue); i++) vif->hw_queue[i] = i % (ATH12K_HW_MAX_QUEUES - 1); - ath12k_mac_setup_vdev_create_arg(arvif, &vdev_arg); + ret = ath12k_mac_setup_vdev_create_arg(arvif, &vdev_arg); + if (ret) { + ath12k_warn(ab, "failed to create vdev parameters %d: %d\n", + arvif->vdev_id, ret); + goto err; + } ret = ath12k_wmi_vdev_create(ar, vif->addr, &vdev_arg); if (ret) { @@ -6492,7 +6801,6 @@ err_vdev_del: /* Recalc txpower for remaining vdev */ ath12k_mac_txpower_recalc(ar); - clear_bit(ATH12K_FLAG_MONITOR_ENABLED, &ar->monitor_flags); /* TODO: recal traffic pause state based on the available vdevs */ arvif->is_created = false; @@ -6563,15 +6871,9 @@ static void ath12k_mac_configure_filter(struct ath12k *ar, reset_flag = !(ar->filter_flags & FIF_BCN_PRBRESP_PROMISC); ret = ath12k_dp_tx_htt_monitor_mode_ring_config(ar, reset_flag); - if (!ret) { - if (!reset_flag) - set_bit(ATH12K_FLAG_MONITOR_ENABLED, &ar->monitor_flags); - else - clear_bit(ATH12K_FLAG_MONITOR_ENABLED, &ar->monitor_flags); - } else { + if (ret) ath12k_warn(ar->ab, "fail to set monitor filter: %d\n", ret); - } ath12k_dbg(ar->ab, ATH12K_DBG_MAC, "total_flags:0x%x, reset_flag:%d\n", @@ -6848,10 +7150,16 @@ ath12k_mac_vdev_start_restart(struct ath12k_vif *arvif, arg.pref_tx_streams = ar->num_tx_chains; arg.pref_rx_streams = ar->num_rx_chains; - /* Fill the MBSSID flags to indicate AP is non MBSSID by default - * Corresponding flags would be updated with MBSSID support. - */ arg.mbssid_flags = WMI_VDEV_MBSSID_FLAGS_NON_MBSSID_AP; + arg.mbssid_tx_vdev_id = 0; + if (test_bit(WMI_TLV_SERVICE_MBSS_PARAM_IN_VDEV_START_SUPPORT, + ar->ab->wmi_ab.svc_map)) { + ret = ath12k_mac_setup_vdev_params_mbssid(arvif, + &arg.mbssid_flags, + &arg.mbssid_tx_vdev_id); + if (ret) + return ret; + } if (arvif->vdev_type == WMI_VDEV_TYPE_AP) { arg.ssid = arvif->u.ap.ssid; @@ -7045,7 +7353,9 @@ ath12k_mac_update_vif_chan(struct ath12k *ar, struct ieee80211_vif_chanctx_switch *vifs, int n_vifs) { + struct ath12k_wmi_vdev_up_params params = {}; struct ath12k_base *ab = ar->ab; + struct ieee80211_vif *vif; struct ath12k_vif *arvif; int ret; int i; @@ -7054,9 +7364,10 @@ ath12k_mac_update_vif_chan(struct ath12k *ar, lockdep_assert_held(&ar->conf_mutex); for (i = 0; i < n_vifs; i++) { - arvif = ath12k_vif_to_arvif(vifs[i].vif); + vif = vifs[i].vif; + arvif = ath12k_vif_to_arvif(vif); - if (vifs[i].vif->type == NL80211_IFTYPE_MONITOR) + if (vif->type == NL80211_IFTYPE_MONITOR) monitor_vif = true; ath12k_dbg(ab, ATH12K_DBG_MAC, @@ -7067,29 +7378,6 @@ ath12k_mac_update_vif_chan(struct ath12k *ar, vifs[i].old_ctx->def.width, vifs[i].new_ctx->def.width); - if (WARN_ON(!arvif->is_started)) - continue; - - if (WARN_ON(!arvif->is_up)) - continue; - - ret = ath12k_wmi_vdev_down(ar, arvif->vdev_id); - if (ret) { - ath12k_warn(ab, "failed to down vdev %d: %d\n", - arvif->vdev_id, ret); - continue; - } - } - - /* All relevant vdevs are downed and associated channel resources - * should be available for the channel switch now. - */ - - /* TODO: Update ar->rx_channel */ - - for (i = 0; i < n_vifs; i++) { - arvif = ath12k_vif_to_arvif(vifs[i].vif); - if (WARN_ON(!arvif->is_started)) continue; @@ -7125,8 +7413,16 @@ ath12k_mac_update_vif_chan(struct ath12k *ar, ath12k_warn(ab, "failed to update bcn tmpl during csa: %d\n", ret); - ret = ath12k_wmi_vdev_up(arvif->ar, arvif->vdev_id, arvif->aid, - arvif->bssid); + memset(¶ms, 0, sizeof(params)); + params.vdev_id = arvif->vdev_id; + params.aid = arvif->aid; + params.bssid = arvif->bssid; + if (vif->mbssid_tx_vif) { + params.tx_bssid = ath12k_vif_to_arvif(vif->mbssid_tx_vif)->bssid; + params.nontx_profile_idx = vif->bss_conf.bssid_index; + params.nontx_profile_cnt = 1 << vif->bss_conf.bssid_indicator; + } + ret = ath12k_wmi_vdev_up(arvif->ar, ¶ms); if (ret) { ath12k_warn(ab, "failed to bring vdev up %d: %d\n", arvif->vdev_id, ret); @@ -7259,7 +7555,6 @@ ath12k_mac_op_assign_vif_chanctx(struct ieee80211_hw *hw, struct ath12k_base *ab; struct ath12k_vif *arvif = ath12k_vif_to_arvif(vif); int ret; - struct ath12k_wmi_peer_create_arg param; /* For multi radio wiphy, the vdev was not created during add_interface * create now since we have a channel ctx now to assign to a specific ar/fw @@ -7295,21 +7590,6 @@ ath12k_mac_op_assign_vif_chanctx(struct ieee80211_hw *hw, goto out; } - if (ab->hw_params->vdev_start_delay && - arvif->vdev_type != WMI_VDEV_TYPE_AP && - arvif->vdev_type != WMI_VDEV_TYPE_MONITOR) { - param.vdev_id = arvif->vdev_id; - param.peer_type = WMI_PEER_TYPE_DEFAULT; - param.peer_addr = ar->mac_addr; - - ret = ath12k_peer_create(ar, arvif, NULL, ¶m); - if (ret) { - ath12k_warn(ab, "failed to create peer after vdev start delay: %d", - ret); - goto out; - } - } - if (arvif->vdev_type == WMI_VDEV_TYPE_MONITOR) { ret = ath12k_mac_monitor_start(ar); if (ret) @@ -7371,11 +7651,6 @@ ath12k_mac_op_unassign_vif_chanctx(struct ieee80211_hw *hw, WARN_ON(!arvif->is_started); - if (ab->hw_params->vdev_start_delay && - arvif->vdev_type == WMI_VDEV_TYPE_MONITOR && - ath12k_peer_find_by_addr(ab, ar->mac_addr)) - ath12k_peer_delete(ar, arvif->vdev_id, ar->mac_addr); - if (arvif->vdev_type == WMI_VDEV_TYPE_MONITOR) { ret = ath12k_mac_monitor_stop(ar); if (ret) { @@ -7386,7 +7661,8 @@ ath12k_mac_op_unassign_vif_chanctx(struct ieee80211_hw *hw, arvif->is_started = false; } - if (arvif->vdev_type != WMI_VDEV_TYPE_STA) { + if (arvif->vdev_type != WMI_VDEV_TYPE_STA && + arvif->vdev_type != WMI_VDEV_TYPE_MONITOR) { ath12k_bss_disassoc(ar, arvif); ret = ath12k_mac_vdev_stop(arvif); if (ret) @@ -7395,10 +7671,6 @@ ath12k_mac_op_unassign_vif_chanctx(struct ieee80211_hw *hw, } arvif->is_started = false; - if (ab->hw_params->vdev_start_delay && - arvif->vdev_type == WMI_VDEV_TYPE_MONITOR) - ath12k_wmi_vdev_down(ar, arvif->vdev_id); - if (arvif->vdev_type != WMI_VDEV_TYPE_MONITOR && ar->num_started_vdevs == 1 && ar->monitor_vdev_created) ath12k_mac_monitor_stop(ar); @@ -7920,26 +8192,33 @@ ath12k_mac_op_reconfig_complete(struct ieee80211_hw *hw, struct ath12k *ar; struct ath12k_base *ab; struct ath12k_vif *arvif; - int recovery_count; + int recovery_count, i; if (reconfig_type != IEEE80211_RECONFIG_TYPE_RESTART) return; - ar = ath12k_ah_to_ar(ah, 0); - ab = ar->ab; + guard(mutex)(&ah->hw_mutex); - mutex_lock(&ar->conf_mutex); + if (ah->state != ATH12K_HW_STATE_RESTARTED) + return; + + ah->state = ATH12K_HW_STATE_ON; + ieee80211_wake_queues(hw); + + for_each_ar(ah, ar, i) { + mutex_lock(&ar->conf_mutex); + + ab = ar->ab; - if (ar->state == ATH12K_STATE_RESTARTED) { ath12k_warn(ar->ab, "pdev %d successfully recovered\n", ar->pdev->pdev_id); - ar->state = ATH12K_STATE_ON; - ieee80211_wake_queues(hw); if (ab->is_reset) { recovery_count = atomic_inc_return(&ab->recovery_count); + ath12k_dbg(ab, ATH12K_DBG_BOOT, "recovery count %d\n", recovery_count); + /* When there are multiple radios in an SOC, * the recovery has to be done for each radio */ @@ -7958,6 +8237,7 @@ ath12k_mac_op_reconfig_complete(struct ieee80211_hw *hw, arvif->key_cipher, arvif->is_up, arvif->vdev_type); + /* After trigger disconnect, then upper layer will * trigger connect again, then the PN number of * upper layer will be reset to keep up with AP @@ -7967,13 +8247,14 @@ ath12k_mac_op_reconfig_complete(struct ieee80211_hw *hw, arvif->vdev_type == WMI_VDEV_TYPE_STA && arvif->vdev_subtype == WMI_VDEV_SUBTYPE_NONE) { ieee80211_hw_restart_disconnect(arvif->vif); + ath12k_dbg(ab, ATH12K_DBG_BOOT, "restart disconnect\n"); } } - } - mutex_unlock(&ar->conf_mutex); + mutex_unlock(&ar->conf_mutex); + } } static void @@ -8026,6 +8307,17 @@ static int ath12k_mac_op_get_survey(struct ieee80211_hw *hw, int idx, if (!sband) sband = hw->wiphy->bands[NL80211_BAND_5GHZ]; + if (sband && idx >= sband->n_channels) { + idx -= sband->n_channels; + sband = NULL; + } + + if (!sband) + sband = hw->wiphy->bands[NL80211_BAND_6GHZ]; + if (!sband || idx >= sband->n_channels) { + idx -= sband->n_channels; + sband = NULL; + } if (!sband || idx >= sband->n_channels) return -ENOENT; @@ -8488,19 +8780,23 @@ static int ath12k_mac_setup_iface_combinations(struct ath12k_hw *ah) static const u8 ath12k_if_types_ext_capa[] = { [0] = WLAN_EXT_CAPA1_EXT_CHANNEL_SWITCHING, + [2] = WLAN_EXT_CAPA3_MULTI_BSSID_SUPPORT, [7] = WLAN_EXT_CAPA8_OPMODE_NOTIF, }; static const u8 ath12k_if_types_ext_capa_sta[] = { [0] = WLAN_EXT_CAPA1_EXT_CHANNEL_SWITCHING, + [2] = WLAN_EXT_CAPA3_MULTI_BSSID_SUPPORT, [7] = WLAN_EXT_CAPA8_OPMODE_NOTIF, [9] = WLAN_EXT_CAPA10_TWT_REQUESTER_SUPPORT, }; static const u8 ath12k_if_types_ext_capa_ap[] = { [0] = WLAN_EXT_CAPA1_EXT_CHANNEL_SWITCHING, + [2] = WLAN_EXT_CAPA3_MULTI_BSSID_SUPPORT, [7] = WLAN_EXT_CAPA8_OPMODE_NOTIF, [9] = WLAN_EXT_CAPA10_TWT_RESPONDER_SUPPORT, + [10] = WLAN_EXT_CAPA11_EMA_SUPPORT, }; static const struct wiphy_iftype_ext_capab ath12k_iftypes_ext_capa[] = { @@ -8685,7 +8981,7 @@ static int ath12k_mac_hw_register(struct ath12k_hw *ah) ieee80211_hw_set(hw, SUPPORTS_TX_FRAG); ieee80211_hw_set(hw, REPORTS_LOW_ACK); - if (ht_cap & WMI_HT_CAP_ENABLED) { + if ((ht_cap & WMI_HT_CAP_ENABLED) || ar->supports_6ghz) { ieee80211_hw_set(hw, AMPDU_AGGREGATION); ieee80211_hw_set(hw, TX_AMPDU_SETUP_IN_HW); ieee80211_hw_set(hw, SUPPORTS_REORDERING_BUFFER); @@ -8700,7 +8996,8 @@ static int ath12k_mac_hw_register(struct ath12k_hw *ah) * for each band for a dual band capable radio. It will be tricky to * handle it when the ht capability different for each band. */ - if (ht_cap & WMI_HT_CAP_DYNAMIC_SMPS) + if (ht_cap & WMI_HT_CAP_DYNAMIC_SMPS || + (ar->supports_6ghz && ab->hw_params->supports_dynamic_smps_6ghz)) wiphy->features |= NL80211_FEATURE_DYNAMIC_SMPS; wiphy->max_scan_ssids = WLAN_SCAN_PARAMS_MAX_SSID; @@ -8739,6 +9036,9 @@ static int ath12k_mac_hw_register(struct ath12k_hw *ah) wiphy->iftype_ext_capab = ath12k_iftypes_ext_capa; wiphy->num_iftype_ext_capab = ARRAY_SIZE(ath12k_iftypes_ext_capa); + wiphy->mbssid_max_interfaces = TARGET_NUM_VDEVS; + wiphy->ema_max_profile_periodicity = TARGET_EMA_MAX_PROFILE_PERIOD; + if (is_6ghz) { wiphy_ext_feature_set(wiphy, NL80211_EXT_FEATURE_FILS_DISCOVERY); @@ -8842,7 +9142,6 @@ static void ath12k_mac_setup(struct ath12k *ar) INIT_WORK(&ar->wmi_mgmt_tx_work, ath12k_mgmt_over_wmi_tx_work); skb_queue_head_init(&ar->wmi_mgmt_tx_queue); - clear_bit(ATH12K_FLAG_MONITOR_ENABLED, &ar->monitor_flags); } int ath12k_mac_register(struct ath12k_base *ab) @@ -8917,8 +9216,11 @@ static struct ath12k_hw *ath12k_mac_hw_allocate(struct ath12k_base *ab, ah = ath12k_hw_to_ah(hw); ah->hw = hw; + ah->ab = ab; ah->num_radio = num_pdev_map; + mutex_init(&ah->hw_mutex); + for (i = 0; i < num_pdev_map; i++) { ab = pdev_map[i].ab; pdev_idx = pdev_map[i].pdev_idx; @@ -8927,7 +9229,7 @@ static struct ath12k_hw *ath12k_mac_hw_allocate(struct ath12k_base *ab, ar = ath12k_ah_to_ar(ah, i); ar->ah = ah; ar->ab = ab; - ar->hw_link_id = i; + ar->hw_link_id = pdev->hw_link_id; ar->pdev = pdev; ar->pdev_idx = pdev_idx; pdev->ar = ar; diff --git a/drivers/net/wireless/ath/ath12k/mhi.c b/drivers/net/wireless/ath/ath12k/mhi.c index fef2f76220336d8817f657cbdefff55f4956192d..df96b0f91f54c1f05126663e1c23f8d31f1de769 100644 --- a/drivers/net/wireless/ath/ath12k/mhi.c +++ b/drivers/net/wireless/ath/ath12k/mhi.c @@ -16,6 +16,7 @@ #define MHI_TIMEOUT_DEFAULT_MS 90000 #define OTP_INVALID_BOARD_ID 0xFFFF #define OTP_VALID_DUALMAC_BOARD_ID_MASK 0x1000 +#define MHI_CB_INVALID 0xff static const struct mhi_channel_config ath12k_mhi_channels_qcn9274[] = { { @@ -268,6 +269,7 @@ static void ath12k_mhi_op_status_cb(struct mhi_controller *mhi_cntrl, enum mhi_callback cb) { struct ath12k_base *ab = dev_get_drvdata(mhi_cntrl->cntrl_dev); + struct ath12k_pci *ab_pci = ath12k_pci_priv(ab); ath12k_dbg(ab, ATH12K_DBG_BOOT, "mhi notify status reason %s\n", ath12k_mhi_op_callback_to_str(cb)); @@ -277,12 +279,20 @@ static void ath12k_mhi_op_status_cb(struct mhi_controller *mhi_cntrl, ath12k_warn(ab, "firmware crashed: MHI_CB_SYS_ERROR\n"); break; case MHI_CB_EE_RDDM: + if (ab_pci->mhi_pre_cb == MHI_CB_EE_RDDM) { + ath12k_dbg(ab, ATH12K_DBG_BOOT, + "do not queue again for consecutive RDDM event\n"); + break; + } + if (!(test_bit(ATH12K_FLAG_UNREGISTERING, &ab->dev_flags))) queue_work(ab->workqueue_aux, &ab->reset_work); break; default: break; } + + ab_pci->mhi_pre_cb = cb; } static int ath12k_mhi_op_read_reg(struct mhi_controller *mhi_cntrl, @@ -313,6 +323,7 @@ int ath12k_mhi_register(struct ath12k_pci *ab_pci) if (!mhi_ctrl) return -ENOMEM; + ab_pci->mhi_pre_cb = MHI_CB_INVALID; ab_pci->mhi_ctrl = mhi_ctrl; mhi_ctrl->cntrl_dev = ab->dev; mhi_ctrl->regs = ab->mem; diff --git a/drivers/net/wireless/ath/ath12k/pci.c b/drivers/net/wireless/ath/ath12k/pci.c index 16af046c33d9eeed6ab9e1d1e89fa6e50cee0e54..876c029f58f62c3d9644dc44c80eabae8b084739 100644 --- a/drivers/net/wireless/ath/ath12k/pci.c +++ b/drivers/net/wireless/ath/ath12k/pci.c @@ -350,6 +350,7 @@ static void ath12k_pci_free_ext_irq(struct ath12k_base *ab) free_irq(ab->irq_num[irq_grp->irqs[j]], irq_grp); netif_napi_del(&irq_grp->napi); + free_netdev(irq_grp->napi_ndev); } } @@ -560,8 +561,9 @@ static irqreturn_t ath12k_pci_ext_interrupt_handler(int irq, void *arg) static int ath12k_pci_ext_irq_config(struct ath12k_base *ab) { struct ath12k_pci *ab_pci = ath12k_pci_priv(ab); - int i, j, ret, num_vectors = 0; + int i, j, n, ret, num_vectors = 0; u32 user_base_data = 0, base_vector = 0, base_idx; + struct ath12k_ext_irq_grp *irq_grp; base_idx = ATH12K_PCI_IRQ_CE0_OFFSET + CE_COUNT_MAX; ret = ath12k_pci_get_user_msi_assignment(ab, "DP", @@ -572,13 +574,18 @@ static int ath12k_pci_ext_irq_config(struct ath12k_base *ab) return ret; for (i = 0; i < ATH12K_EXT_IRQ_GRP_NUM_MAX; i++) { - struct ath12k_ext_irq_grp *irq_grp = &ab->ext_irq_grp[i]; + irq_grp = &ab->ext_irq_grp[i]; u32 num_irq = 0; irq_grp->ab = ab; irq_grp->grp_id = i; - init_dummy_netdev(&irq_grp->napi_ndev); - netif_napi_add(&irq_grp->napi_ndev, &irq_grp->napi, + irq_grp->napi_ndev = alloc_netdev_dummy(0); + if (!irq_grp->napi_ndev) { + ret = -ENOMEM; + goto fail_allocate; + } + + netif_napi_add(irq_grp->napi_ndev, &irq_grp->napi, ath12k_pci_ext_grp_napi_poll); if (ab->hw_params->ring_mask->tx[i] || @@ -611,13 +618,23 @@ static int ath12k_pci_ext_irq_config(struct ath12k_base *ab) if (ret) { ath12k_err(ab, "failed request irq %d: %d\n", vector, ret); - return ret; + goto fail_request; } } ath12k_pci_ext_grp_disable(irq_grp); } return 0; + +fail_request: + /* i ->napi_ndev was properly allocated. Free it also */ + i += 1; +fail_allocate: + for (n = 0; n < i; n++) { + irq_grp = &ab->ext_irq_grp[n]; + free_netdev(irq_grp->napi_ndev); + } + return ret; } static int ath12k_pci_set_irq_affinity_hint(struct ath12k_pci *ab_pci, @@ -1090,14 +1107,14 @@ void ath12k_pci_ext_irq_enable(struct ath12k_base *ab) { int i; - set_bit(ATH12K_FLAG_EXT_IRQ_ENABLED, &ab->dev_flags); - for (i = 0; i < ATH12K_EXT_IRQ_GRP_NUM_MAX; i++) { struct ath12k_ext_irq_grp *irq_grp = &ab->ext_irq_grp[i]; napi_enable(&irq_grp->napi); ath12k_pci_ext_grp_enable(irq_grp); } + + set_bit(ATH12K_FLAG_EXT_IRQ_ENABLED, &ab->dev_flags); } void ath12k_pci_ext_irq_disable(struct ath12k_base *ab) @@ -1285,6 +1302,13 @@ void ath12k_pci_power_down(struct ath12k_base *ab, bool is_suspend) ath12k_pci_sw_reset(ab_pci->ab, false); } +static int ath12k_pci_panic_handler(struct ath12k_base *ab) +{ + ath12k_pci_sw_reset(ab, false); + + return NOTIFY_OK; +} + static const struct ath12k_hif_ops ath12k_pci_hif_ops = { .start = ath12k_pci_start, .stop = ath12k_pci_stop, @@ -1302,6 +1326,7 @@ static const struct ath12k_hif_ops ath12k_pci_hif_ops = { .ce_irq_enable = ath12k_pci_hif_ce_irq_enable, .ce_irq_disable = ath12k_pci_hif_ce_irq_disable, .get_ce_msi_idx = ath12k_pci_get_ce_msi_idx, + .panic_handler = ath12k_pci_panic_handler, }; static diff --git a/drivers/net/wireless/ath/ath12k/pci.h b/drivers/net/wireless/ath/ath12k/pci.h index 6186a78038cf6e48be0d7d7ebafeee7187d075e8..31584a7ad80eb9f55423905039050eac247aa1d8 100644 --- a/drivers/net/wireless/ath/ath12k/pci.h +++ b/drivers/net/wireless/ath/ath12k/pci.h @@ -104,6 +104,7 @@ struct ath12k_pci { struct mhi_controller *mhi_ctrl; const struct ath12k_msi_config *msi_config; unsigned long mhi_state; + enum mhi_callback mhi_pre_cb; u32 register_window; /* protects register_window above */ diff --git a/drivers/net/wireless/ath/ath12k/qmi.c b/drivers/net/wireless/ath/ath12k/qmi.c index 5484112859a667e26d171eda47346cb7801f51a1..b93ce9f87f6129c0ed0cc6dbf79f7325004651a2 100644 --- a/drivers/net/wireless/ath/ath12k/qmi.c +++ b/drivers/net/wireless/ath/ath12k/qmi.c @@ -2041,7 +2041,7 @@ static void ath12k_host_cap_parse_mlo(struct ath12k_base *ab, req->mlo_capable_valid = 1; req->mlo_capable = 1; req->mlo_chip_id_valid = 1; - req->mlo_chip_id = 0; + req->mlo_chip_id = ab->device_id; req->mlo_group_id_valid = 1; req->mlo_group_id = 0; req->max_mlo_peer_valid = 1; @@ -2053,7 +2053,7 @@ static void ath12k_host_cap_parse_mlo(struct ath12k_base *ab, req->mlo_num_chips = 1; info = &req->mlo_chip_info[0]; - info->chip_id = 0; + info->chip_id = ab->device_id; info->num_local_links = ab->qmi.num_radios; for (i = 0; i < info->num_local_links; i++) { @@ -2503,7 +2503,7 @@ static int ath12k_qmi_request_target_cap(struct ath12k_base *ab) ab->qmi.dev_mem[i].size = resp.dev_mem[i].size; ath12k_dbg(ab, ATH12K_DBG_QMI, - "devmem [%d] start ox%llx size %llu\n", i, + "devmem [%d] start 0x%llx size %llu\n", i, ab->qmi.dev_mem[i].start, ab->qmi.dev_mem[i].size); } @@ -2538,7 +2538,7 @@ static int ath12k_qmi_load_file_target_mem(struct ath12k_base *ab, struct qmi_wlanfw_bdf_download_resp_msg_v01 resp = {}; struct qmi_txn txn; const u8 *temp = data; - int ret; + int ret = 0; u32 remaining = len; req = kzalloc(sizeof(*req), GFP_KERNEL); diff --git a/drivers/net/wireless/ath/ath12k/reg.c b/drivers/net/wireless/ath/ath12k/reg.c index fbf38044938c6245230e88b399352bfcc77ca7da..439d61f284d89222e79c05d6cff8e85d0d315aad 100644 --- a/drivers/net/wireless/ath/ath12k/reg.c +++ b/drivers/net/wireless/ath/ath12k/reg.c @@ -206,9 +206,9 @@ static void ath12k_copy_regd(struct ieee80211_regdomain *regd_orig, int ath12k_regd_update(struct ath12k *ar, bool init) { - struct ieee80211_hw *hw = ath12k_ar_to_hw(ar); + struct ath12k_hw *ah = ath12k_ar_to_ah(ar); + struct ieee80211_hw *hw = ah->hw; struct ieee80211_regdomain *regd, *regd_copy = NULL; - struct ath12k_hw *ah = ar->ah; int ret, regd_len, pdev_id; struct ath12k_base *ab; int i; @@ -286,19 +286,20 @@ int ath12k_regd_update(struct ath12k *ar, bool init) if (ret) goto err; + if (ah->state != ATH12K_HW_STATE_ON) + goto skip; + ah->regd_updated = true; /* Apply the new regd to all the radios, this is expected to be received only once * since we check for ah->regd_updated and allow here only once. */ for_each_ar(ah, ar, i) { - if (ar->state == ATH12K_STATE_ON) { - ab = ar->ab; - ret = ath12k_reg_update_chan_list(ar); - if (ret) - goto err; - } + ab = ar->ab; + ret = ath12k_reg_update_chan_list(ar); + if (ret) + goto err; } - +skip: return 0; err: ath12k_warn(ab, "failed to perform regd update : %d\n", ret); diff --git a/drivers/net/wireless/ath/ath12k/wmi.c b/drivers/net/wireless/ath/ath12k/wmi.c index 7a52d2082b792e70bb7e4388956d619fb6f04bc4..d6e1d1398cdbd911a04766a318d4c1c3cbcc32e5 100644 --- a/drivers/net/wireless/ath/ath12k/wmi.c +++ b/drivers/net/wireless/ath/ath12k/wmi.c @@ -228,6 +228,9 @@ void ath12k_wmi_init_qcn9274(struct ath12k_base *ab, config->peer_map_unmap_version = 0x32; config->twt_ap_pdev_count = ab->num_radios; config->twt_ap_sta_count = 1000; + config->ema_max_vap_cnt = ab->num_radios; + config->ema_max_profile_period = TARGET_EMA_MAX_PROFILE_PERIOD; + config->beacon_tx_offload_max_vdev += config->ema_max_vap_cnt; if (test_bit(WMI_TLV_SERVICE_PEER_METADATA_V1A_V1B_SUPPORT, ab->wmi_ab.svc_map)) config->dp_peer_meta_data_ver = TARGET_RX_PEER_METADATA_VER_V1B; @@ -497,6 +500,7 @@ ath12k_pull_mac_phy_cap_svc_ready_ext(struct ath12k_wmi_pdev *wmi_handle, mac_caps = wmi_mac_phy_caps + phy_idx; pdev->pdev_id = ath12k_wmi_mac_phy_get_pdev_id(mac_caps); + pdev->hw_link_id = ath12k_wmi_mac_phy_get_hw_link_id(mac_caps); pdev_cap->supported_bands |= le32_to_cpu(mac_caps->supported_bands); pdev_cap->ampdu_density = le32_to_cpu(mac_caps->ampdu_density); @@ -841,6 +845,8 @@ int ath12k_wmi_vdev_create(struct ath12k *ar, u8 *macaddr, cmd->vdev_subtype = cpu_to_le32(args->subtype); cmd->num_cfg_txrx_streams = cpu_to_le32(WMI_NUM_SUPPORTED_BAND_MAX); cmd->pdev_id = cpu_to_le32(args->pdev_id); + cmd->mbssid_flags = cpu_to_le32(args->mbssid_flags); + cmd->mbssid_tx_vdev_id = cpu_to_le32(args->mbssid_tx_vdev_id); cmd->vdev_stats_id = cpu_to_le32(args->if_stats_id); ether_addr_copy(cmd->vdev_macaddr.addr, macaddr); @@ -1046,6 +1052,7 @@ int ath12k_wmi_vdev_start(struct ath12k *ar, struct wmi_vdev_start_req_arg *arg, cmd->he_ops = cpu_to_le32(arg->he_ops); cmd->punct_bitmap = cpu_to_le32(arg->punct_bitmap); cmd->mbssid_flags = cpu_to_le32(arg->mbssid_flags); + cmd->mbssid_tx_vdev_id = cpu_to_le32(arg->mbssid_tx_vdev_id); if (!restart) { if (arg->ssid) { @@ -1097,7 +1104,7 @@ int ath12k_wmi_vdev_start(struct ath12k *ar, struct wmi_vdev_start_req_arg *arg, return ret; } -int ath12k_wmi_vdev_up(struct ath12k *ar, u32 vdev_id, u32 aid, const u8 *bssid) +int ath12k_wmi_vdev_up(struct ath12k *ar, struct ath12k_wmi_vdev_up_params *params) { struct ath12k_wmi_pdev *wmi = ar->wmi; struct wmi_vdev_up_cmd *cmd; @@ -1112,14 +1119,20 @@ int ath12k_wmi_vdev_up(struct ath12k *ar, u32 vdev_id, u32 aid, const u8 *bssid) cmd->tlv_header = ath12k_wmi_tlv_cmd_hdr(WMI_TAG_VDEV_UP_CMD, sizeof(*cmd)); - cmd->vdev_id = cpu_to_le32(vdev_id); - cmd->vdev_assoc_id = cpu_to_le32(aid); + cmd->vdev_id = cpu_to_le32(params->vdev_id); + cmd->vdev_assoc_id = cpu_to_le32(params->aid); + + ether_addr_copy(cmd->vdev_bssid.addr, params->bssid); - ether_addr_copy(cmd->vdev_bssid.addr, bssid); + if (params->tx_bssid) { + ether_addr_copy(cmd->tx_vdev_bssid.addr, params->tx_bssid); + cmd->nontx_profile_idx = cpu_to_le32(params->nontx_profile_idx); + cmd->nontx_profile_cnt = cpu_to_le32(params->nontx_profile_cnt); + } ath12k_dbg(ar->ab, ATH12K_DBG_WMI, "WMI mgmt vdev up id 0x%x assoc id %d bssid %pM\n", - vdev_id, aid, bssid); + params->vdev_id, params->aid, params->bssid); ret = ath12k_wmi_cmd_send(wmi, skb, WMI_VDEV_UP_CMDID); if (ret) { @@ -1776,13 +1789,15 @@ int ath12k_wmi_p2p_go_bcn_ie(struct ath12k *ar, u32 vdev_id, int ath12k_wmi_bcn_tmpl(struct ath12k *ar, u32 vdev_id, struct ieee80211_mutable_offsets *offs, - struct sk_buff *bcn) + struct sk_buff *bcn, + struct ath12k_wmi_bcn_tmpl_ema_arg *ema_args) { struct ath12k_wmi_pdev *wmi = ar->wmi; struct wmi_bcn_tmpl_cmd *cmd; struct ath12k_wmi_bcn_prb_info_params *bcn_prb_info; struct wmi_tlv *tlv; struct sk_buff *skb; + u32 ema_params = 0; void *ptr; int ret, len; size_t aligned_len = roundup(bcn->len, 4); @@ -1801,6 +1816,16 @@ int ath12k_wmi_bcn_tmpl(struct ath12k *ar, u32 vdev_id, cmd->csa_switch_count_offset = cpu_to_le32(offs->cntdwn_counter_offs[0]); cmd->ext_csa_switch_count_offset = cpu_to_le32(offs->cntdwn_counter_offs[1]); cmd->buf_len = cpu_to_le32(bcn->len); + cmd->mbssid_ie_offset = cpu_to_le32(offs->mbssid_off); + if (ema_args) { + u32p_replace_bits(&ema_params, ema_args->bcn_cnt, WMI_EMA_BEACON_CNT); + u32p_replace_bits(&ema_params, ema_args->bcn_index, WMI_EMA_BEACON_IDX); + if (ema_args->bcn_index == 0) + u32p_replace_bits(&ema_params, 1, WMI_EMA_BEACON_FIRST); + if (ema_args->bcn_index + 1 == ema_args->bcn_cnt) + u32p_replace_bits(&ema_params, 1, WMI_EMA_BEACON_LAST); + cmd->ema_params = cpu_to_le32(ema_params); + } ptr = skb->data + sizeof(*cmd); @@ -3475,9 +3500,11 @@ ath12k_wmi_copy_resource_config(struct ath12k_wmi_resource_config_params *wmi_cf wmi_cfg->twt_ap_sta_count = cpu_to_le32(tg_cfg->twt_ap_sta_count); wmi_cfg->flags2 = le32_encode_bits(tg_cfg->dp_peer_meta_data_ver, WMI_RSRC_CFG_FLAGS2_RX_PEER_METADATA_VERSION); - wmi_cfg->host_service_flags = cpu_to_le32(tg_cfg->is_reg_cc_ext_event_supported << WMI_RSRC_CFG_HOST_SVC_FLAG_REG_CC_EXT_SUPPORT_BIT); + wmi_cfg->ema_max_vap_cnt = cpu_to_le32(tg_cfg->ema_max_vap_cnt); + wmi_cfg->ema_max_profile_period = cpu_to_le32(tg_cfg->ema_max_profile_period); + wmi_cfg->flags2 |= cpu_to_le32(WMI_RSRC_CFG_FLAGS2_CALC_NEXT_DTIM_COUNT_SET); } static int ath12k_init_cmd_send(struct ath12k_wmi_pdev *wmi, @@ -3808,7 +3835,7 @@ int ath12k_wmi_pdev_dma_ring_cfg(struct ath12k *ar, cmd->tlv_header = ath12k_wmi_tlv_cmd_hdr(WMI_TAG_DMA_RING_CFG_REQ, sizeof(*cmd)); - cmd->pdev_id = cpu_to_le32(DP_SW2HW_MACID(arg->pdev_id)); + cmd->pdev_id = cpu_to_le32(arg->pdev_id); cmd->module_id = cpu_to_le32(arg->module_id); cmd->base_paddr_lo = cpu_to_le32(arg->base_paddr_lo); cmd->base_paddr_hi = cpu_to_le32(arg->base_paddr_hi); @@ -5693,7 +5720,7 @@ static int ath12k_reg_chan_list_event(struct ath12k_base *ab, struct sk_buff *sk * event. Otherwise, it goes to fallback. */ if (ab->hw_params->single_pdev_only && - pdev_idx < ab->hw_params->num_rxmda_per_pdev) + pdev_idx < ab->hw_params->num_rxdma_per_pdev) goto mem_free; else goto fallback; @@ -6022,8 +6049,10 @@ static void ath12k_mgmt_rx_event(struct ath12k_base *ab, struct sk_buff *skb) if (rx_ev.status & WMI_RX_STATUS_ERR_MIC) status->flag |= RX_FLAG_MMIC_ERROR; - if (rx_ev.chan_freq >= ATH12K_MIN_6G_FREQ) { + if (rx_ev.chan_freq >= ATH12K_MIN_6G_FREQ && + rx_ev.chan_freq <= ATH12K_MAX_6G_FREQ) { status->band = NL80211_BAND_6GHZ; + status->freq = rx_ev.chan_freq; } else if (rx_ev.channel >= 1 && rx_ev.channel <= 14) { status->band = NL80211_BAND_2GHZ; } else if (rx_ev.channel >= 36 && rx_ev.channel <= ATH12K_MAX_5G_CHAN) { @@ -6044,8 +6073,10 @@ static void ath12k_mgmt_rx_event(struct ath12k_base *ab, struct sk_buff *skb) sband = &ar->mac.sbands[status->band]; - status->freq = ieee80211_channel_to_frequency(rx_ev.channel, - status->band); + if (status->band != NL80211_BAND_6GHZ) + status->freq = ieee80211_channel_to_frequency(rx_ev.channel, + status->band); + status->signal = rx_ev.snr + ATH12K_DEFAULT_NOISE_FLOOR; status->rate_idx = ath12k_mac_bitrate_to_idx(sband, rx_ev.rate / 100); diff --git a/drivers/net/wireless/ath/ath12k/wmi.h b/drivers/net/wireless/ath/ath12k/wmi.h index 496866673aead975945bf3c11cab91255289a5b3..c2b86e187a03fe8637c3c85e791a06b91a25a929 100644 --- a/drivers/net/wireless/ath/ath12k/wmi.h +++ b/drivers/net/wireless/ath/ath12k/wmi.h @@ -2154,6 +2154,7 @@ enum wmi_tlv_service { WMI_TLV_SERVICE_PER_PEER_HTT_STATS_RESET = 213, WMI_TLV_SERVICE_FREQINFO_IN_METADATA = 219, WMI_TLV_SERVICE_EXT2_MSG = 220, + WMI_TLV_SERVICE_MBSS_PARAM_IN_VDEV_START_SUPPORT = 253, WMI_MAX_EXT_SERVICE = 256, @@ -2356,6 +2357,8 @@ struct ath12k_wmi_resource_config_arg { u32 twt_ap_sta_count; bool is_reg_cc_ext_event_supported; u8 dp_peer_meta_data_ver; + u32 ema_max_vap_cnt; + u32 ema_max_profile_period; }; struct ath12k_wmi_init_cmd_arg { @@ -2410,6 +2413,7 @@ struct wmi_init_cmd { #define WMI_RSRC_CFG_HOST_SVC_FLAG_REG_CC_EXT_SUPPORT_BIT 4 #define WMI_RSRC_CFG_FLAGS2_RX_PEER_METADATA_VERSION GENMASK(5, 4) #define WMI_RSRC_CFG_FLAG1_BSS_CHANNEL_INFO_64 BIT(5) +#define WMI_RSRC_CFG_FLAGS2_CALC_NEXT_DTIM_COUNT_SET BIT(9) struct ath12k_wmi_resource_config_params { __le32 tlv_header; @@ -2726,6 +2730,8 @@ struct ath12k_wmi_vdev_create_arg { } chains[NUM_NL80211_BANDS]; u32 pdev_id; u8 if_stats_id; + u32 mbssid_flags; + u32 mbssid_tx_vdev_id; }; #define ATH12K_MAX_VDEV_STATS_ID 0x30 @@ -2757,14 +2763,23 @@ struct wmi_vdev_delete_cmd { __le32 vdev_id; } __packed; +struct ath12k_wmi_vdev_up_params { + u32 vdev_id; + u32 aid; + const u8 *bssid; + const u8 *tx_bssid; + u32 nontx_profile_idx; + u32 nontx_profile_cnt; +}; + struct wmi_vdev_up_cmd { __le32 tlv_header; __le32 vdev_id; __le32 vdev_assoc_id; struct ath12k_wmi_mac_addr_params vdev_bssid; - struct ath12k_wmi_mac_addr_params trans_bssid; - __le32 profile_idx; - __le32 profile_num; + struct ath12k_wmi_mac_addr_params tx_vdev_bssid; + __le32 nontx_profile_idx; + __le32 nontx_profile_cnt; } __packed; struct wmi_vdev_stop_cmd { @@ -2792,6 +2807,10 @@ struct ath12k_wmi_ssid_params { enum wmi_vdev_mbssid_flags { WMI_VDEV_MBSSID_FLAGS_NON_MBSSID_AP = BIT(0), + WMI_VDEV_MBSSID_FLAGS_TRANSMIT_AP = BIT(1), + WMI_VDEV_MBSSID_FLAGS_NON_TRANSMIT_AP = BIT(2), + WMI_VDEV_MBSSID_FLAGS_EMA_MODE = BIT(3), + WMI_VDEV_MBSSID_FLAGS_SCAN_MODE_VAP = BIT(4), }; struct wmi_vdev_start_request_cmd { @@ -3514,6 +3533,16 @@ struct ath12k_wmi_p2p_noa_info { #define WMI_BEACON_TX_BUFFER_SIZE 512 +#define WMI_EMA_BEACON_CNT GENMASK(7, 0) +#define WMI_EMA_BEACON_IDX GENMASK(15, 8) +#define WMI_EMA_BEACON_FIRST GENMASK(23, 16) +#define WMI_EMA_BEACON_LAST GENMASK(31, 24) + +struct ath12k_wmi_bcn_tmpl_ema_arg { + u8 bcn_cnt; + u8 bcn_index; +}; + struct wmi_bcn_tmpl_cmd { __le32 tlv_header; __le32 vdev_id; @@ -3524,6 +3553,11 @@ struct wmi_bcn_tmpl_cmd { __le32 csa_event_bitmap; __le32 mbssid_ie_offset; __le32 esp_ie_offset; + __le32 csc_switch_count_offset; + __le32 csc_event_bitmap; + __le32 mu_edca_ie_offset; + __le32 feature_enable_bitmap; + __le32 ema_params; } __packed; struct wmi_p2p_go_set_beacon_ie_cmd { @@ -4770,7 +4804,7 @@ struct wmi_probe_tmpl_cmd { __le32 buf_len; } __packed; -#define MAX_RADIOS 3 +#define MAX_RADIOS 2 #define WMI_SERVICE_READY_TIMEOUT_HZ (5 * HZ) #define WMI_SEND_TIMEOUT_HZ (3 * HZ) @@ -4881,10 +4915,10 @@ int ath12k_wmi_p2p_go_bcn_ie(struct ath12k *ar, u32 vdev_id, const u8 *p2p_ie); int ath12k_wmi_bcn_tmpl(struct ath12k *ar, u32 vdev_id, struct ieee80211_mutable_offsets *offs, - struct sk_buff *bcn); + struct sk_buff *bcn, + struct ath12k_wmi_bcn_tmpl_ema_arg *ema_args); int ath12k_wmi_vdev_down(struct ath12k *ar, u8 vdev_id); -int ath12k_wmi_vdev_up(struct ath12k *ar, u32 vdev_id, u32 aid, - const u8 *bssid); +int ath12k_wmi_vdev_up(struct ath12k *ar, struct ath12k_wmi_vdev_up_params *params); int ath12k_wmi_vdev_stop(struct ath12k *ar, u8 vdev_id); int ath12k_wmi_vdev_start(struct ath12k *ar, struct wmi_vdev_start_req_arg *arg, bool restart); diff --git a/drivers/net/wireless/ath/wil6210/netdev.c b/drivers/net/wireless/ath/wil6210/netdev.c index ee7d7e9c27184b425956bb8f148ea72e6a2d5c54..d5d364683c0e9e738005bf932274446a8f3904ad 100644 --- a/drivers/net/wireless/ath/wil6210/netdev.c +++ b/drivers/net/wireless/ath/wil6210/netdev.c @@ -453,16 +453,21 @@ int wil_if_add(struct wil6210_priv *wil) return rc; } - init_dummy_netdev(&wil->napi_ndev); + wil->napi_ndev = alloc_netdev_dummy(0); + if (!wil->napi_ndev) { + wil_err(wil, "failed to allocate dummy netdev"); + rc = -ENOMEM; + goto out_wiphy; + } if (wil->use_enhanced_dma_hw) { - netif_napi_add(&wil->napi_ndev, &wil->napi_rx, + netif_napi_add(wil->napi_ndev, &wil->napi_rx, wil6210_netdev_poll_rx_edma); - netif_napi_add_tx(&wil->napi_ndev, + netif_napi_add_tx(wil->napi_ndev, &wil->napi_tx, wil6210_netdev_poll_tx_edma); } else { - netif_napi_add(&wil->napi_ndev, &wil->napi_rx, + netif_napi_add(wil->napi_ndev, &wil->napi_rx, wil6210_netdev_poll_rx); - netif_napi_add_tx(&wil->napi_ndev, + netif_napi_add_tx(wil->napi_ndev, &wil->napi_tx, wil6210_netdev_poll_tx); } @@ -474,10 +479,12 @@ int wil_if_add(struct wil6210_priv *wil) wiphy_unlock(wiphy); rtnl_unlock(); if (rc < 0) - goto out_wiphy; + goto free_dummy; return 0; +free_dummy: + free_netdev(wil->napi_ndev); out_wiphy: wiphy_unregister(wiphy); return rc; @@ -554,5 +561,7 @@ void wil_if_remove(struct wil6210_priv *wil) netif_napi_del(&wil->napi_tx); netif_napi_del(&wil->napi_rx); + free_netdev(wil->napi_ndev); + wiphy_unregister(wiphy); } diff --git a/drivers/net/wireless/ath/wil6210/wil6210.h b/drivers/net/wireless/ath/wil6210/wil6210.h index 22a6eb3e12b7ee3d27e723484e28da829e594e34..9bd1286d2857b09774bc1faf80fb87c6a4723f8c 100644 --- a/drivers/net/wireless/ath/wil6210/wil6210.h +++ b/drivers/net/wireless/ath/wil6210/wil6210.h @@ -983,7 +983,7 @@ struct wil6210_priv { spinlock_t eap_lock; /* guarding access to eap rekey fields */ struct napi_struct napi_rx; struct napi_struct napi_tx; - struct net_device napi_ndev; /* dummy net_device serving all VIFs */ + struct net_device *napi_ndev; /* dummy net_device serving all VIFs */ /* DMA related */ struct wil_ring ring_rx; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c index 13391c2d82aae21ee80a79617d3fe80129f73c60..d35262335eaf79fabb74c1e3363e916466d1ffc8 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c @@ -1061,10 +1061,10 @@ static int brcmf_ops_sdio_probe(struct sdio_func *func, if (func->num != 2) return -ENODEV; - bus_if = kzalloc(sizeof(struct brcmf_bus), GFP_KERNEL); + bus_if = kzalloc(sizeof(*bus_if), GFP_KERNEL); if (!bus_if) return -ENOMEM; - sdiodev = kzalloc(sizeof(struct brcmf_sdio_dev), GFP_KERNEL); + sdiodev = kzalloc(sizeof(*sdiodev), GFP_KERNEL); if (!sdiodev) { kfree(bus_if); return -ENOMEM; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/btcoex.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/btcoex.c index 7ea2631b80692dc50479f0abb753f85254835748..0c3d119d12199c9ef2507d2bc87a6ae838ec6486 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/btcoex.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/btcoex.c @@ -358,10 +358,10 @@ idle: */ int brcmf_btcoex_attach(struct brcmf_cfg80211_info *cfg) { - struct brcmf_btcoex_info *btci = NULL; + struct brcmf_btcoex_info *btci; brcmf_dbg(TRACE, "enter\n"); - btci = kmalloc(sizeof(struct brcmf_btcoex_info), GFP_KERNEL); + btci = kmalloc(sizeof(*btci), GFP_KERNEL); if (!btci) return -ENOMEM; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c index 06698a714b5233485ad9f45af41db0583c6d92d8..ce482a3877e90ac16373ec7b081261d509c8bc34 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/pcie.c @@ -313,11 +313,6 @@ struct brcmf_pcie_shared_info { u8 version; }; -struct brcmf_pcie_core_info { - u32 base; - u32 wrapbase; -}; - #define BRCMF_OTP_MAX_PARAM_LEN 16 struct brcmf_otp_params { diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c index 6b38d9de71af6ea481ad09f522c9e951795a8f64..1461dc453ac22e45766d2e8fd50b895e08dd0a5d 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c @@ -4450,7 +4450,7 @@ struct brcmf_sdio *brcmf_sdio_probe(struct brcmf_sdio_dev *sdiodev) brcmf_dbg(TRACE, "Enter\n"); /* Allocate private bus interface state */ - bus = kzalloc(sizeof(struct brcmf_sdio), GFP_ATOMIC); + bus = kzalloc(sizeof(*bus), GFP_ATOMIC); if (!bus) goto fail; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c index 9a105e6debe1f59ac26cf774a693df298709ffa4..8afbf529c745038151c778f6ea35e45084a8171f 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.c @@ -1236,8 +1236,8 @@ brcmf_usb_prepare_fw_request(struct brcmf_usbdev_info *devinfo) static int brcmf_usb_probe_cb(struct brcmf_usbdev_info *devinfo, enum brcmf_fwvendor fwvid) { - struct brcmf_bus *bus = NULL; - struct brcmf_usbdev *bus_pub = NULL; + struct brcmf_bus *bus; + struct brcmf_usbdev *bus_pub; struct device *dev = devinfo->dev; struct brcmf_fw_request *fwreq; int ret; @@ -1247,7 +1247,7 @@ static int brcmf_usb_probe_cb(struct brcmf_usbdev_info *devinfo, if (!bus_pub) return -ENODEV; - bus = kzalloc(sizeof(struct brcmf_bus), GFP_ATOMIC); + bus = kzalloc(sizeof(*bus), GFP_ATOMIC); if (!bus) { ret = -ENOMEM; goto fail; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/aiutils.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/aiutils.c index 2084b506a4500cf695a9dd2cdc4d5807154f8639..50d817485cf9fc4f93f63cb52990520dfdb3e1c0 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/aiutils.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/aiutils.c @@ -512,7 +512,7 @@ ai_attach(struct bcma_bus *pbus) struct si_info *sii; /* alloc struct si_info */ - sii = kzalloc(sizeof(struct si_info), GFP_ATOMIC); + sii = kzalloc(sizeof(*sii), GFP_ATOMIC); if (sii == NULL) return NULL; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/ampdu.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/ampdu.c index c3376f887114a4fb5e0f1b59ae91e11d7465208b..33d17b779201d2adfb172f1b18d3237f469005c1 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/ampdu.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/ampdu.c @@ -219,7 +219,7 @@ struct ampdu_info *brcms_c_ampdu_attach(struct brcms_c_info *wlc) struct ampdu_info *ampdu; int i; - ampdu = kzalloc(sizeof(struct ampdu_info), GFP_ATOMIC); + ampdu = kzalloc(sizeof(*ampdu), GFP_ATOMIC); if (!ampdu) return NULL; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/antsel.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/antsel.c index 54c616919590d2ebc89c5b15e3415c605104833f..f411bc6d795d613ad783d35aab147a955c9c7bf1 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/antsel.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/antsel.c @@ -111,7 +111,7 @@ struct antsel_info *brcms_c_antsel_attach(struct brcms_c_info *wlc) struct antsel_info *asi; struct ssb_sprom *sprom = &wlc->hw->d11core->bus->sprom; - asi = kzalloc(sizeof(struct antsel_info), GFP_ATOMIC); + asi = kzalloc(sizeof(*asi), GFP_ATOMIC); if (!asi) return NULL; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/channel.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/channel.c index f6962e558d7c73a8d4b604f9c04f4cea81f0b2fd..d1b9a18d0374034b0020283bd1513a7408f50615 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/channel.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/channel.c @@ -331,7 +331,7 @@ struct brcms_cm_info *brcms_c_channel_mgr_attach(struct brcms_c_info *wlc) const char *ccode = sprom->alpha2; int ccode_len = sizeof(sprom->alpha2); - wlc_cm = kzalloc(sizeof(struct brcms_cm_info), GFP_ATOMIC); + wlc_cm = kzalloc(sizeof(*wlc_cm), GFP_ATOMIC); if (wlc_cm == NULL) return NULL; wlc_cm->pub = pub; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/dma.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/dma.c index 3d5c1ef8f7f25b3a9802e0a37c27c7a6e1a9e18b..bd480239368a4bf817167c46f80d5eca96e2f7dc 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/dma.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/dma.c @@ -558,7 +558,7 @@ struct dma_pub *dma_attach(char *name, struct brcms_c_info *wlc, struct si_info *sii = container_of(sih, struct si_info, pub); /* allocate private info structure */ - di = kzalloc(sizeof(struct dma_info), GFP_ATOMIC); + di = kzalloc(sizeof(*di), GFP_ATOMIC); if (di == NULL) return NULL; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/mac80211_if.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/mac80211_if.c index 92860dc0a92eb70eb30fd5f1761bf5a6416d3a7c..860ef9c11c46d31d5e88f76bbce197dde4068300 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/mac80211_if.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/mac80211_if.c @@ -1496,7 +1496,7 @@ struct brcms_timer *brcms_init_timer(struct brcms_info *wl, { struct brcms_timer *t; - t = kzalloc(sizeof(struct brcms_timer), GFP_ATOMIC); + t = kzalloc(sizeof(*t), GFP_ATOMIC); if (!t) return NULL; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/main.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/main.c index 34460b5815d02ae9e2937e1b50bdf0e26862bb79..2738d4d6c60a914a50b47fd283005cd5bf574387 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/main.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/main.c @@ -234,12 +234,6 @@ /* max # tx status to process in wlc_txstatus() */ #define TXSBND 8 -/* brcmu_format_flags() bit description structure */ -struct brcms_c_bit_desc { - u32 bit; - const char *name; -}; - /* * The following table lists the buffer memory allocated to xmt fifos in HW. * the size is in units of 256bytes(one block), total size is HW dependent @@ -463,11 +457,11 @@ static struct brcms_bss_cfg *brcms_c_bsscfg_malloc(uint unit) { struct brcms_bss_cfg *cfg; - cfg = kzalloc(sizeof(struct brcms_bss_cfg), GFP_ATOMIC); + cfg = kzalloc(sizeof(*cfg), GFP_ATOMIC); if (cfg == NULL) goto fail; - cfg->current_bss = kzalloc(sizeof(struct brcms_bss_info), GFP_ATOMIC); + cfg->current_bss = kzalloc(sizeof(*cfg->current_bss), GFP_ATOMIC); if (cfg->current_bss == NULL) goto fail; @@ -483,14 +477,14 @@ brcms_c_attach_malloc(uint unit, uint *err, uint devid) { struct brcms_c_info *wlc; - wlc = kzalloc(sizeof(struct brcms_c_info), GFP_ATOMIC); + wlc = kzalloc(sizeof(*wlc), GFP_ATOMIC); if (wlc == NULL) { *err = 1002; goto fail; } /* allocate struct brcms_c_pub state structure */ - wlc->pub = kzalloc(sizeof(struct brcms_pub), GFP_ATOMIC); + wlc->pub = kzalloc(sizeof(*wlc->pub), GFP_ATOMIC); if (wlc->pub == NULL) { *err = 1003; goto fail; @@ -499,7 +493,7 @@ brcms_c_attach_malloc(uint unit, uint *err, uint devid) /* allocate struct brcms_hardware state structure */ - wlc->hw = kzalloc(sizeof(struct brcms_hardware), GFP_ATOMIC); + wlc->hw = kzalloc(sizeof(*wlc->hw), GFP_ATOMIC); if (wlc->hw == NULL) { *err = 1005; goto fail; @@ -528,7 +522,7 @@ brcms_c_attach_malloc(uint unit, uint *err, uint devid) goto fail; } - wlc->default_bss = kzalloc(sizeof(struct brcms_bss_info), GFP_ATOMIC); + wlc->default_bss = kzalloc(sizeof(*wlc->default_bss), GFP_ATOMIC); if (wlc->default_bss == NULL) { *err = 1010; goto fail; @@ -540,21 +534,20 @@ brcms_c_attach_malloc(uint unit, uint *err, uint devid) goto fail; } - wlc->protection = kzalloc(sizeof(struct brcms_protection), - GFP_ATOMIC); + wlc->protection = kzalloc(sizeof(*wlc->protection), GFP_ATOMIC); if (wlc->protection == NULL) { *err = 1016; goto fail; } - wlc->stf = kzalloc(sizeof(struct brcms_stf), GFP_ATOMIC); + wlc->stf = kzalloc(sizeof(*wlc->stf), GFP_ATOMIC); if (wlc->stf == NULL) { *err = 1017; goto fail; } wlc->bandstate[0] = - kcalloc(MAXBANDS, sizeof(struct brcms_band), GFP_ATOMIC); + kcalloc(MAXBANDS, sizeof(*wlc->bandstate[0]), GFP_ATOMIC); if (wlc->bandstate[0] == NULL) { *err = 1025; goto fail; @@ -567,14 +560,14 @@ brcms_c_attach_malloc(uint unit, uint *err, uint devid) + (sizeof(struct brcms_band)*i)); } - wlc->corestate = kzalloc(sizeof(struct brcms_core), GFP_ATOMIC); + wlc->corestate = kzalloc(sizeof(*wlc->corestate), GFP_ATOMIC); if (wlc->corestate == NULL) { *err = 1026; goto fail; } wlc->corestate->macstat_snapshot = - kzalloc(sizeof(struct macstat), GFP_ATOMIC); + kzalloc(sizeof(*wlc->corestate->macstat_snapshot), GFP_ATOMIC); if (wlc->corestate->macstat_snapshot == NULL) { *err = 1027; goto fail; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_cmn.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_cmn.c index a27d6f0b881911509e617db0c700d955e48e1cdc..c3d7aa570b4e204acae90792f9769ed51ceb66df 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_cmn.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_cmn.c @@ -355,7 +355,7 @@ struct shared_phy *wlc_phy_shared_attach(struct shared_phy_params *shp) { struct shared_phy *sh; - sh = kzalloc(sizeof(struct shared_phy), GFP_ATOMIC); + sh = kzalloc(sizeof(*sh), GFP_ATOMIC); if (sh == NULL) return NULL; @@ -442,7 +442,7 @@ wlc_phy_attach(struct shared_phy *sh, struct bcma_device *d11core, return &pi->pubpi_ro; } - pi = kzalloc(sizeof(struct brcms_phy), GFP_ATOMIC); + pi = kzalloc(sizeof(*pi), GFP_ATOMIC); if (pi == NULL) return NULL; pi->wiphy = wiphy; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_lcn.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_lcn.c index aae2cf95fe9588ce2f1045798dac9c0d77684b60..d0faba2405610558af5acd5321b455ab0cdb9faf 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_lcn.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_lcn.c @@ -2567,7 +2567,6 @@ wlc_lcnphy_tx_iqlo_cal(struct brcms_phy *pi, struct lcnphy_txgains cal_gains, temp_gains; u16 hash; - u8 band_idx; int j; u16 ncorr_override[5]; u16 syst_coeffs[] = { 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, 0x0000, @@ -2599,6 +2598,9 @@ wlc_lcnphy_tx_iqlo_cal(struct brcms_phy *pi, u16 *values_to_save; struct brcms_phy_lcnphy *pi_lcn = pi->u.pi_lcnphy; + if (WARN_ON(CHSPEC_IS5G(pi->radio_chanspec))) + return; + values_to_save = kmalloc_array(20, sizeof(u16), GFP_ATOMIC); if (NULL == values_to_save) return; @@ -2662,20 +2664,18 @@ wlc_lcnphy_tx_iqlo_cal(struct brcms_phy *pi, hash = (target_gains->gm_gain << 8) | (target_gains->pga_gain << 4) | (target_gains->pad_gain); - band_idx = (CHSPEC_IS5G(pi->radio_chanspec) ? 1 : 0); - cal_gains = *target_gains; memset(ncorr_override, 0, sizeof(ncorr_override)); - for (j = 0; j < iqcal_gainparams_numgains_lcnphy[band_idx]; j++) { - if (hash == tbl_iqcal_gainparams_lcnphy[band_idx][j][0]) { + for (j = 0; j < iqcal_gainparams_numgains_lcnphy[0]; j++) { + if (hash == tbl_iqcal_gainparams_lcnphy[0][j][0]) { cal_gains.gm_gain = - tbl_iqcal_gainparams_lcnphy[band_idx][j][1]; + tbl_iqcal_gainparams_lcnphy[0][j][1]; cal_gains.pga_gain = - tbl_iqcal_gainparams_lcnphy[band_idx][j][2]; + tbl_iqcal_gainparams_lcnphy[0][j][2]; cal_gains.pad_gain = - tbl_iqcal_gainparams_lcnphy[band_idx][j][3]; + tbl_iqcal_gainparams_lcnphy[0][j][3]; memcpy(ncorr_override, - &tbl_iqcal_gainparams_lcnphy[band_idx][j][3], + &tbl_iqcal_gainparams_lcnphy[0][j][3], sizeof(ncorr_override)); break; } @@ -4968,11 +4968,11 @@ bool wlc_phy_attach_lcnphy(struct brcms_phy *pi) { struct brcms_phy_lcnphy *pi_lcn; - pi->u.pi_lcnphy = kzalloc(sizeof(struct brcms_phy_lcnphy), GFP_ATOMIC); - if (pi->u.pi_lcnphy == NULL) + pi_lcn = kzalloc(sizeof(*pi_lcn), GFP_ATOMIC); + if (!pi_lcn) return false; - pi_lcn = pi->u.pi_lcnphy; + pi->u.pi_lcnphy = pi_lcn; if (0 == (pi->sh->boardflags & BFL_NOPA)) { pi->hwpwrctrl = true; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy_shim.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy_shim.c index b7238179153653f35e6480314ba12980fcb88f69..8b852581c4e471f11c12b2fd95a017743de9ef77 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy_shim.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy_shim.c @@ -38,9 +38,9 @@ struct phy_shim_info { struct phy_shim_info *wlc_phy_shim_attach(struct brcms_hardware *wlc_hw, struct brcms_info *wl, struct brcms_c_info *wlc) { - struct phy_shim_info *physhim = NULL; + struct phy_shim_info *physhim; - physhim = kzalloc(sizeof(struct phy_shim_info), GFP_ATOMIC); + physhim = kzalloc(sizeof(*physhim), GFP_ATOMIC); if (!physhim) return NULL; diff --git a/drivers/net/wireless/intel/iwlegacy/3945.c b/drivers/net/wireless/intel/iwlegacy/3945.c index a773939b8c2aac4a6eca6b701bc569062bdf4630..1fab7849f56d1a66bc242a4cae087f293d20a769 100644 --- a/drivers/net/wireless/intel/iwlegacy/3945.c +++ b/drivers/net/wireless/intel/iwlegacy/3945.c @@ -566,7 +566,7 @@ il3945_hdl_rx(struct il_priv *il, struct il_rx_buf *rxb) if (!(rx_end->status & RX_RES_STATUS_NO_CRC32_ERROR) || !(rx_end->status & RX_RES_STATUS_NO_RXE_OVERFLOW)) { D_RX("Bad CRC or FIFO: 0x%08X.\n", rx_end->status); - return; + rx_status.flag |= RX_FLAG_FAILED_FCS_CRC; } /* Convert 3945's rssi indicator to dBm */ diff --git a/drivers/net/wireless/intel/iwlegacy/4965-mac.c b/drivers/net/wireless/intel/iwlegacy/4965-mac.c index 4beb7be6d51df88a842226681ae92aa8f93415d9..d018f56be96645e073162ebed7d1ef2a80de38a5 100644 --- a/drivers/net/wireless/intel/iwlegacy/4965-mac.c +++ b/drivers/net/wireless/intel/iwlegacy/4965-mac.c @@ -664,7 +664,7 @@ il4965_hdl_rx(struct il_priv *il, struct il_rx_buf *rxb) if (!(rx_pkt_status & RX_RES_STATUS_NO_CRC32_ERROR) || !(rx_pkt_status & RX_RES_STATUS_NO_RXE_OVERFLOW)) { D_RX("Bad CRC or FIFO: 0x%08X.\n", le32_to_cpu(rx_pkt_status)); - return; + rx_status.flag |= RX_FLAG_FAILED_FCS_CRC; } /* This will be used in several places later */ diff --git a/drivers/net/wireless/intel/iwlwifi/Makefile b/drivers/net/wireless/intel/iwlwifi/Makefile index 8bb94a4c12cd5f7f6fd16438ba5fee0b5783b60d..93cce88f2e27dc62d7c75486c52a395371159ff4 100644 --- a/drivers/net/wireless/intel/iwlwifi/Makefile +++ b/drivers/net/wireless/intel/iwlwifi/Makefile @@ -4,7 +4,7 @@ obj-$(CONFIG_IWLWIFI) += iwlwifi.o iwlwifi-objs += iwl-io.o iwlwifi-objs += iwl-drv.o iwlwifi-objs += iwl-debug.o -iwlwifi-objs += iwl-eeprom-read.o iwl-eeprom-parse.o +iwlwifi-objs += iwl-nvm-utils.o iwlwifi-objs += iwl-phy-db.o iwl-nvm-parse.o iwlwifi-objs += pcie/drv.o pcie/rx.o pcie/tx.o pcie/trans.o iwlwifi-objs += pcie/ctxt-info.o pcie/ctxt-info-gen3.o diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/bz.c b/drivers/net/wireless/intel/iwlwifi/cfg/bz.c index bc98b87cf2a131b51c444be03b3a6a3d3c6ea1dd..45905e57e084a009ec54d2bc38609aa90b109ccc 100644 --- a/drivers/net/wireless/intel/iwlwifi/cfg/bz.c +++ b/drivers/net/wireless/intel/iwlwifi/cfg/bz.c @@ -10,7 +10,7 @@ #include "fw/api/txq.h" /* Highest firmware API version supported */ -#define IWL_BZ_UCODE_API_MAX 90 +#define IWL_BZ_UCODE_API_MAX 91 /* Lowest firmware API version supported */ #define IWL_BZ_UCODE_API_MIN 80 diff --git a/drivers/net/wireless/intel/iwlwifi/cfg/sc.c b/drivers/net/wireless/intel/iwlwifi/cfg/sc.c index 9b79279fd76cad94170d28614b7eee0729ac02f8..653279e089273f48c1a89df26f517e1c9ff227da 100644 --- a/drivers/net/wireless/intel/iwlwifi/cfg/sc.c +++ b/drivers/net/wireless/intel/iwlwifi/cfg/sc.c @@ -10,7 +10,7 @@ #include "fw/api/txq.h" /* Highest firmware API version supported */ -#define IWL_SC_UCODE_API_MAX 90 +#define IWL_SC_UCODE_API_MAX 91 /* Lowest firmware API version supported */ #define IWL_SC_UCODE_API_MIN 82 diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/Makefile b/drivers/net/wireless/intel/iwlwifi/dvm/Makefile index 6109d64006dbc0fd26a5dd1dccf38bec1090488e..abcf8aeb010d3c08f6655e8b3afd2e9b2aec3586 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/Makefile +++ b/drivers/net/wireless/intel/iwlwifi/dvm/Makefile @@ -2,7 +2,7 @@ # DVM obj-$(CONFIG_IWLDVM) += iwldvm.o iwldvm-objs += main.o rs.o mac80211.o ucode.o tx.o -iwldvm-objs += lib.o calib.o tt.o sta.o rx.o +iwldvm-objs += lib.o calib.o tt.o sta.o rx.o eeprom.o iwldvm-objs += power.o iwldvm-objs += scan.o diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/agn.h b/drivers/net/wireless/intel/iwlwifi/dvm/agn.h index fefaa414272b3a75ef0bffa1a36a7a4f2724b37f..a13add556a7b6c7071e5602de2976e4698c44afc 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/agn.h +++ b/drivers/net/wireless/intel/iwlwifi/dvm/agn.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */ /* - * Copyright (C) 2005-2014, 2021 Intel Corporation + * Copyright (C) 2005-2014, 2021, 2024 Intel Corporation */ #ifndef __iwl_agn_h__ #define __iwl_agn_h__ @@ -385,6 +385,25 @@ static inline void iwl_dvm_set_pmi(struct iwl_priv *priv, bool state) iwl_trans_set_pmi(priv->trans, state); } +/** + * iwl_parse_eeprom_data - parse EEPROM data and return values + * + * @trans: ransport we're parsing for, for debug only + * @cfg: device configuration for parsing and overrides + * @eeprom: the EEPROM data + * @eeprom_size: length of the EEPROM data + * + * This function parses all EEPROM values we need and then + * returns a (newly allocated) struct containing all the + * relevant values for driver use. The struct must be freed + * later with iwl_free_nvm_data(). + */ +struct iwl_nvm_data * +iwl_parse_eeprom_data(struct iwl_trans *trans, const struct iwl_cfg *cfg, + const u8 *eeprom, size_t eeprom_size); + +int iwl_read_eeprom(struct iwl_trans *trans, u8 **eeprom, size_t *eeprom_size); + #ifdef CONFIG_IWLWIFI_DEBUGFS void iwl_dbgfs_register(struct iwl_priv *priv, struct dentry *dbgfs_dir); #else diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/dev.h b/drivers/net/wireless/intel/iwlwifi/dvm/dev.h index 25283e4b849f22dc38eae0d0f722cf7bf8e15bb5..4ac8b862ad418536834cfc93459735fc504aae18 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/dev.h +++ b/drivers/net/wireless/intel/iwlwifi/dvm/dev.h @@ -19,7 +19,7 @@ #include #include "fw/img.h" -#include "iwl-eeprom-parse.h" +#include "iwl-nvm-utils.h" #include "iwl-csr.h" #include "iwl-debug.h" #include "iwl-agn-hw.h" diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/devices.c b/drivers/net/wireless/intel/iwlwifi/dvm/devices.c index 39e40901fa4647753c532c9afca9f9a5feb0351b..48a8349680fc331812d5cab3721d8e2fad137c3c 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/devices.c +++ b/drivers/net/wireless/intel/iwlwifi/dvm/devices.c @@ -12,7 +12,7 @@ */ #include "iwl-io.h" #include "iwl-prph.h" -#include "iwl-eeprom-parse.h" +#include "iwl-nvm-utils.h" #include "agn.h" #include "dev.h" diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.c b/drivers/net/wireless/intel/iwlwifi/dvm/eeprom.c similarity index 69% rename from drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.c rename to drivers/net/wireless/intel/iwlwifi/dvm/eeprom.c index 2b290fab1ef2b4a7b954de4b46c27e72db1c1eea..931aa3f5798d52e2e1f23614ce9a684b11a0cae5 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.c +++ b/drivers/net/wireless/intel/iwlwifi/dvm/eeprom.c @@ -1,16 +1,18 @@ // SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause /* - * Copyright (C) 2005-2014, 2018-2021, 2023 Intel Corporation - * Copyright (C) 2015 Intel Mobile Communications GmbH + * Copyright (C) 2005-2014, 2018-2019, 2021, 2024 Intel Corporation */ #include #include #include + #include "iwl-drv.h" -#include "iwl-modparams.h" -#include "iwl-eeprom-parse.h" +#include "iwl-debug.h" +#include "iwl-io.h" +#include "iwl-prph.h" +#include "iwl-csr.h" +#include "agn.h" -#if IS_ENABLED(CONFIG_IWLDVM) /* EEPROM offset definitions */ /* indirect access definitions */ @@ -79,7 +81,6 @@ enum eeprom_sku_bits { #define EEPROM_RF_CFG_TX_ANT_MSK(x) ((x >> 8) & 0xF) /* bits 8-11 */ #define EEPROM_RF_CFG_RX_ANT_MSK(x) ((x >> 12) & 0xF) /* bits 12-15 */ - /* * EEPROM bands * These are the channel numbers from each band in the order @@ -257,7 +258,6 @@ struct iwl_eeprom_channel { s8 max_power_avg; } __packed; - enum iwl_eeprom_enhanced_txpwr_flags { IWL_EEPROM_ENH_TXP_FL_VALID = BIT(0), IWL_EEPROM_ENH_TXP_FL_BAND_52G = BIT(1), @@ -648,114 +648,385 @@ static int iwl_init_channel_map(struct device *dev, const struct iwl_cfg *cfg, return n_channels; } -#endif +/* + * EEPROM access time values: + * + * Driver initiates EEPROM read by writing byte address << 1 to CSR_EEPROM_REG. + * Driver then polls CSR_EEPROM_REG for CSR_EEPROM_REG_READ_VALID_MSK (0x1). + * When polling, wait 10 uSec between polling loops, up to a maximum 5000 uSec. + * Driver reads 16-bit value from bits 31-16 of CSR_EEPROM_REG. + */ +#define IWL_EEPROM_ACCESS_TIMEOUT 5000 /* uSec */ -int iwl_init_sband_channels(struct iwl_nvm_data *data, - struct ieee80211_supported_band *sband, - int n_channels, enum nl80211_band band) +/* + * The device's EEPROM semaphore prevents conflicts between driver and uCode + * when accessing the EEPROM; each access is a series of pulses to/from the + * EEPROM chip, not a single event, so even reads could conflict if they + * weren't arbitrated by the semaphore. + */ +#define IWL_EEPROM_SEM_TIMEOUT 10 /* microseconds */ +#define IWL_EEPROM_SEM_RETRY_LIMIT 1000 /* number of attempts (not time) */ + + +static int iwl_eeprom_acquire_semaphore(struct iwl_trans *trans) { - struct ieee80211_channel *chan = &data->channels[0]; - int n = 0, idx = 0; + u16 count; + int ret; + + for (count = 0; count < IWL_EEPROM_SEM_RETRY_LIMIT; count++) { + /* Request semaphore */ + iwl_set_bit(trans, CSR_HW_IF_CONFIG_REG, + CSR_HW_IF_CONFIG_REG_BIT_EEPROM_OWN_SEM); + + /* See if we got it */ + ret = iwl_poll_bit(trans, CSR_HW_IF_CONFIG_REG, + CSR_HW_IF_CONFIG_REG_BIT_EEPROM_OWN_SEM, + CSR_HW_IF_CONFIG_REG_BIT_EEPROM_OWN_SEM, + IWL_EEPROM_SEM_TIMEOUT); + if (ret >= 0) { + IWL_DEBUG_EEPROM(trans->dev, + "Acquired semaphore after %d tries.\n", + count+1); + return ret; + } + } - while (idx < n_channels && chan->band != band) - chan = &data->channels[++idx]; + return ret; +} - sband->channels = &data->channels[idx]; +static void iwl_eeprom_release_semaphore(struct iwl_trans *trans) +{ + iwl_clear_bit(trans, CSR_HW_IF_CONFIG_REG, + CSR_HW_IF_CONFIG_REG_BIT_EEPROM_OWN_SEM); +} - while (idx < n_channels && chan->band == band) { - chan = &data->channels[++idx]; - n++; - } +static int iwl_eeprom_verify_signature(struct iwl_trans *trans, bool nvm_is_otp) +{ + u32 gp = iwl_read32(trans, CSR_EEPROM_GP) & CSR_EEPROM_GP_VALID_MSK; - sband->n_channels = n; + IWL_DEBUG_EEPROM(trans->dev, "EEPROM signature=0x%08x\n", gp); - return n; + switch (gp) { + case CSR_EEPROM_GP_BAD_SIG_EEP_GOOD_SIG_OTP: + if (!nvm_is_otp) { + IWL_ERR(trans, "EEPROM with bad signature: 0x%08x\n", + gp); + return -ENOENT; + } + return 0; + case CSR_EEPROM_GP_GOOD_SIG_EEP_LESS_THAN_4K: + case CSR_EEPROM_GP_GOOD_SIG_EEP_MORE_THAN_4K: + if (nvm_is_otp) { + IWL_ERR(trans, "OTP with bad signature: 0x%08x\n", gp); + return -ENOENT; + } + return 0; + case CSR_EEPROM_GP_BAD_SIGNATURE_BOTH_EEP_AND_OTP: + default: + IWL_ERR(trans, + "bad EEPROM/OTP signature, type=%s, EEPROM_GP=0x%08x\n", + nvm_is_otp ? "OTP" : "EEPROM", gp); + return -ENOENT; + } } -#define MAX_BIT_RATE_40_MHZ 150 /* Mbps */ -#define MAX_BIT_RATE_20_MHZ 72 /* Mbps */ +/****************************************************************************** + * + * OTP related functions + * +******************************************************************************/ -void iwl_init_ht_hw_capab(struct iwl_trans *trans, - struct iwl_nvm_data *data, - struct ieee80211_sta_ht_cap *ht_info, - enum nl80211_band band, - u8 tx_chains, u8 rx_chains) +static void iwl_set_otp_access_absolute(struct iwl_trans *trans) { - const struct iwl_cfg *cfg = trans->cfg; - int max_bit_rate = 0; - - tx_chains = hweight8(tx_chains); - if (cfg->rx_with_siso_diversity) - rx_chains = 1; - else - rx_chains = hweight8(rx_chains); - - if (!(data->sku_cap_11n_enable) || - (iwlwifi_mod_params.disable_11n & IWL_DISABLE_HT_ALL) || - !cfg->ht_params) { - ht_info->ht_supported = false; - return; + iwl_read32(trans, CSR_OTP_GP_REG); + + iwl_clear_bit(trans, CSR_OTP_GP_REG, + CSR_OTP_GP_REG_OTP_ACCESS_MODE); +} + +static int iwl_nvm_is_otp(struct iwl_trans *trans) +{ + u32 otpgp; + + /* OTP only valid for CP/PP and after */ + switch (trans->hw_rev & CSR_HW_REV_TYPE_MSK) { + case CSR_HW_REV_TYPE_NONE: + IWL_ERR(trans, "Unknown hardware type\n"); + return -EIO; + case CSR_HW_REV_TYPE_5300: + case CSR_HW_REV_TYPE_5350: + case CSR_HW_REV_TYPE_5100: + case CSR_HW_REV_TYPE_5150: + return 0; + default: + otpgp = iwl_read32(trans, CSR_OTP_GP_REG); + if (otpgp & CSR_OTP_GP_REG_DEVICE_SELECT) + return 1; + return 0; } +} + +static int iwl_init_otp_access(struct iwl_trans *trans) +{ + int ret; + + ret = iwl_finish_nic_init(trans); + if (ret) + return ret; + + iwl_set_bits_prph(trans, APMG_PS_CTRL_REG, + APMG_PS_CTRL_VAL_RESET_REQ); + udelay(5); + iwl_clear_bits_prph(trans, APMG_PS_CTRL_REG, + APMG_PS_CTRL_VAL_RESET_REQ); - if (data->sku_cap_mimo_disabled) - rx_chains = 1; + /* + * CSR auto clock gate disable bit - + * this is only applicable for HW with OTP shadow RAM + */ + if (trans->trans_cfg->base_params->shadow_ram_support) + iwl_set_bit(trans, CSR_DBG_LINK_PWR_MGMT_REG, + CSR_RESET_LINK_PWR_MGMT_DISABLED); - ht_info->ht_supported = true; - ht_info->cap = IEEE80211_HT_CAP_DSSSCCK40; + return 0; +} - if (cfg->ht_params->stbc) { - ht_info->cap |= (1 << IEEE80211_HT_CAP_RX_STBC_SHIFT); +static int iwl_read_otp_word(struct iwl_trans *trans, u16 addr, + __le16 *eeprom_data) +{ + int ret = 0; + u32 r; + u32 otpgp; + + iwl_write32(trans, CSR_EEPROM_REG, + CSR_EEPROM_REG_MSK_ADDR & (addr << 1)); + ret = iwl_poll_bit(trans, CSR_EEPROM_REG, + CSR_EEPROM_REG_READ_VALID_MSK, + CSR_EEPROM_REG_READ_VALID_MSK, + IWL_EEPROM_ACCESS_TIMEOUT); + if (ret < 0) { + IWL_ERR(trans, "Time out reading OTP[%d]\n", addr); + return ret; + } + r = iwl_read32(trans, CSR_EEPROM_REG); + /* check for ECC errors: */ + otpgp = iwl_read32(trans, CSR_OTP_GP_REG); + if (otpgp & CSR_OTP_GP_REG_ECC_UNCORR_STATUS_MSK) { + /* stop in this case */ + /* set the uncorrectable OTP ECC bit for acknowledgment */ + iwl_set_bit(trans, CSR_OTP_GP_REG, + CSR_OTP_GP_REG_ECC_UNCORR_STATUS_MSK); + IWL_ERR(trans, "Uncorrectable OTP ECC error, abort OTP read\n"); + return -EINVAL; + } + if (otpgp & CSR_OTP_GP_REG_ECC_CORR_STATUS_MSK) { + /* continue in this case */ + /* set the correctable OTP ECC bit for acknowledgment */ + iwl_set_bit(trans, CSR_OTP_GP_REG, + CSR_OTP_GP_REG_ECC_CORR_STATUS_MSK); + IWL_ERR(trans, "Correctable OTP ECC error, continue read\n"); + } + *eeprom_data = cpu_to_le16(r >> 16); + return 0; +} - if (tx_chains > 1) - ht_info->cap |= IEEE80211_HT_CAP_TX_STBC; +/* + * iwl_is_otp_empty: check for empty OTP + */ +static bool iwl_is_otp_empty(struct iwl_trans *trans) +{ + u16 next_link_addr = 0; + __le16 link_value; + bool is_empty = false; + + /* locate the beginning of OTP link list */ + if (!iwl_read_otp_word(trans, next_link_addr, &link_value)) { + if (!link_value) { + IWL_ERR(trans, "OTP is empty\n"); + is_empty = true; + } + } else { + IWL_ERR(trans, "Unable to read first block of OTP list.\n"); + is_empty = true; } - if (cfg->ht_params->ldpc) - ht_info->cap |= IEEE80211_HT_CAP_LDPC_CODING; + return is_empty; +} - if (trans->trans_cfg->mq_rx_supported || - iwlwifi_mod_params.amsdu_size >= IWL_AMSDU_8K) - ht_info->cap |= IEEE80211_HT_CAP_MAX_AMSDU; - ht_info->ampdu_factor = IEEE80211_HT_MAX_AMPDU_64K; - ht_info->ampdu_density = IEEE80211_HT_MPDU_DENSITY_4; +/* + * iwl_find_otp_image: find EEPROM image in OTP + * finding the OTP block that contains the EEPROM image. + * the last valid block on the link list (the block _before_ the last block) + * is the block we should read and used to configure the device. + * If all the available OTP blocks are full, the last block will be the block + * we should read and used to configure the device. + * only perform this operation if shadow RAM is disabled + */ +static int iwl_find_otp_image(struct iwl_trans *trans, + u16 *validblockaddr) +{ + u16 next_link_addr = 0, valid_addr; + __le16 link_value = 0; + int usedblocks = 0; - ht_info->mcs.rx_mask[0] = 0xFF; - ht_info->mcs.rx_mask[1] = 0x00; - ht_info->mcs.rx_mask[2] = 0x00; + /* set addressing mode to absolute to traverse the link list */ + iwl_set_otp_access_absolute(trans); - if (rx_chains >= 2) - ht_info->mcs.rx_mask[1] = 0xFF; - if (rx_chains >= 3) - ht_info->mcs.rx_mask[2] = 0xFF; + /* checking for empty OTP or error */ + if (iwl_is_otp_empty(trans)) + return -EINVAL; - if (cfg->ht_params->ht_greenfield_support) - ht_info->cap |= IEEE80211_HT_CAP_GRN_FLD; - ht_info->cap |= IEEE80211_HT_CAP_SGI_20; + /* + * start traverse link list + * until reach the max number of OTP blocks + * different devices have different number of OTP blocks + */ + do { + /* save current valid block address + * check for more block on the link list + */ + valid_addr = next_link_addr; + next_link_addr = le16_to_cpu(link_value) * sizeof(u16); + IWL_DEBUG_EEPROM(trans->dev, "OTP blocks %d addr 0x%x\n", + usedblocks, next_link_addr); + if (iwl_read_otp_word(trans, next_link_addr, &link_value)) + return -EINVAL; + if (!link_value) { + /* + * reach the end of link list, return success and + * set address point to the starting address + * of the image + */ + *validblockaddr = valid_addr; + /* skip first 2 bytes (link list pointer) */ + *validblockaddr += 2; + return 0; + } + /* more in the link list, continue */ + usedblocks++; + } while (usedblocks <= trans->trans_cfg->base_params->max_ll_items); - max_bit_rate = MAX_BIT_RATE_20_MHZ; + /* OTP has no valid blocks */ + IWL_DEBUG_EEPROM(trans->dev, "OTP has no valid blocks\n"); + return -EINVAL; +} - if (cfg->ht_params->ht40_bands & BIT(band)) { - ht_info->cap |= IEEE80211_HT_CAP_SUP_WIDTH_20_40; - ht_info->cap |= IEEE80211_HT_CAP_SGI_40; - max_bit_rate = MAX_BIT_RATE_40_MHZ; +/* + * iwl_read_eeprom - read EEPROM contents + * + * Load the EEPROM contents from adapter and return it + * and its size. + * + * NOTE: This routine uses the non-debug IO access functions. + */ +int iwl_read_eeprom(struct iwl_trans *trans, u8 **eeprom, size_t *eeprom_size) +{ + __le16 *e; + u32 gp = iwl_read32(trans, CSR_EEPROM_GP); + int sz; + int ret; + u16 addr; + u16 validblockaddr = 0; + u16 cache_addr = 0; + int nvm_is_otp; + + if (!eeprom || !eeprom_size) + return -EINVAL; + + nvm_is_otp = iwl_nvm_is_otp(trans); + if (nvm_is_otp < 0) + return nvm_is_otp; + + sz = trans->trans_cfg->base_params->eeprom_size; + IWL_DEBUG_EEPROM(trans->dev, "NVM size = %d\n", sz); + + e = kmalloc(sz, GFP_KERNEL); + if (!e) + return -ENOMEM; + + ret = iwl_eeprom_verify_signature(trans, nvm_is_otp); + if (ret < 0) { + IWL_ERR(trans, "EEPROM not found, EEPROM_GP=0x%08x\n", gp); + goto err_free; } - /* Highest supported Rx data rate */ - max_bit_rate *= rx_chains; - WARN_ON(max_bit_rate & ~IEEE80211_HT_MCS_RX_HIGHEST_MASK); - ht_info->mcs.rx_highest = cpu_to_le16(max_bit_rate); - - /* Tx MCS capabilities */ - ht_info->mcs.tx_params = IEEE80211_HT_MCS_TX_DEFINED; - if (tx_chains != rx_chains) { - ht_info->mcs.tx_params |= IEEE80211_HT_MCS_TX_RX_DIFF; - ht_info->mcs.tx_params |= ((tx_chains - 1) << - IEEE80211_HT_MCS_TX_MAX_STREAMS_SHIFT); + /* Make sure driver (instead of uCode) is allowed to read EEPROM */ + ret = iwl_eeprom_acquire_semaphore(trans); + if (ret < 0) { + IWL_ERR(trans, "Failed to acquire EEPROM semaphore.\n"); + goto err_free; } + + if (nvm_is_otp) { + ret = iwl_init_otp_access(trans); + if (ret) { + IWL_ERR(trans, "Failed to initialize OTP access.\n"); + goto err_unlock; + } + + iwl_write32(trans, CSR_EEPROM_GP, + iwl_read32(trans, CSR_EEPROM_GP) & + ~CSR_EEPROM_GP_IF_OWNER_MSK); + + iwl_set_bit(trans, CSR_OTP_GP_REG, + CSR_OTP_GP_REG_ECC_CORR_STATUS_MSK | + CSR_OTP_GP_REG_ECC_UNCORR_STATUS_MSK); + /* traversing the linked list if no shadow ram supported */ + if (!trans->trans_cfg->base_params->shadow_ram_support) { + ret = iwl_find_otp_image(trans, &validblockaddr); + if (ret) + goto err_unlock; + } + for (addr = validblockaddr; addr < validblockaddr + sz; + addr += sizeof(u16)) { + __le16 eeprom_data; + + ret = iwl_read_otp_word(trans, addr, &eeprom_data); + if (ret) + goto err_unlock; + e[cache_addr / 2] = eeprom_data; + cache_addr += sizeof(u16); + } + } else { + /* eeprom is an array of 16bit values */ + for (addr = 0; addr < sz; addr += sizeof(u16)) { + u32 r; + + iwl_write32(trans, CSR_EEPROM_REG, + CSR_EEPROM_REG_MSK_ADDR & (addr << 1)); + + ret = iwl_poll_bit(trans, CSR_EEPROM_REG, + CSR_EEPROM_REG_READ_VALID_MSK, + CSR_EEPROM_REG_READ_VALID_MSK, + IWL_EEPROM_ACCESS_TIMEOUT); + if (ret < 0) { + IWL_ERR(trans, + "Time out reading EEPROM[%d]\n", addr); + goto err_unlock; + } + r = iwl_read32(trans, CSR_EEPROM_REG); + e[addr / 2] = cpu_to_le16(r >> 16); + } + } + + IWL_DEBUG_EEPROM(trans->dev, "NVM Type: %s\n", + nvm_is_otp ? "OTP" : "EEPROM"); + + iwl_eeprom_release_semaphore(trans); + + *eeprom_size = sz; + *eeprom = (u8 *)e; + return 0; + + err_unlock: + iwl_eeprom_release_semaphore(trans); + err_free: + kfree(e); + + return ret; } -#if IS_ENABLED(CONFIG_IWLDVM) static void iwl_init_sbands(struct iwl_trans *trans, const struct iwl_cfg *cfg, struct iwl_nvm_data *data, const u8 *eeprom, size_t eeprom_size) @@ -790,7 +1061,6 @@ static void iwl_init_sbands(struct iwl_trans *trans, const struct iwl_cfg *cfg, } /* EEPROM data functions */ - struct iwl_nvm_data * iwl_parse_eeprom_data(struct iwl_trans *trans, const struct iwl_cfg *cfg, const u8 *eeprom, size_t eeprom_size) @@ -837,8 +1107,8 @@ iwl_parse_eeprom_data(struct iwl_trans *trans, const struct iwl_cfg *cfg, data->kelvin_temperature = *(__le16 *)tmp; data->kelvin_voltage = *((__le16 *)tmp + 1); - radio_cfg = iwl_eeprom_query16(eeprom, eeprom_size, - EEPROM_RADIO_CONFIG); + radio_cfg = + iwl_eeprom_query16(eeprom, eeprom_size, EEPROM_RADIO_CONFIG); data->radio_cfg_dash = EEPROM_RF_CFG_DASH_MSK(radio_cfg); data->radio_cfg_pnum = EEPROM_RF_CFG_PNUM_MSK(radio_cfg); data->radio_cfg_step = EEPROM_RF_CFG_STEP_MSK(radio_cfg); @@ -878,5 +1148,3 @@ iwl_parse_eeprom_data(struct iwl_trans *trans, const struct iwl_cfg *cfg, kfree(data); return NULL; } -IWL_EXPORT_SYMBOL(iwl_parse_eeprom_data); -#endif diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/main.c b/drivers/net/wireless/intel/iwlwifi/dvm/main.c index 8774dd7b921effb43568d09a92c6f7ec47260d16..65b7c68e5ca7731f9e0caeeb00af365938fdc543 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/main.c +++ b/drivers/net/wireless/intel/iwlwifi/dvm/main.c @@ -1,7 +1,8 @@ // SPDX-License-Identifier: GPL-2.0-only /****************************************************************************** * - * Copyright(c) 2003 - 2014, 2018 - 2022 Intel Corporation. All rights reserved. + * Copyright(c) 2003 - 2014, 2018 - 2022 Intel Corporation. All rights reserved. + * Copyright(c) 2024 Intel Corporation. All rights reserved. * Copyright(c) 2015 Intel Deutschland GmbH * * Portions of this file are derived from the ipw3945 project, as well @@ -25,8 +26,7 @@ #include -#include "iwl-eeprom-read.h" -#include "iwl-eeprom-parse.h" +#include "iwl-nvm-utils.h" #include "iwl-io.h" #include "iwl-trans.h" #include "iwl-op-mode.h" diff --git a/drivers/net/wireless/intel/iwlwifi/dvm/rs.c b/drivers/net/wireless/intel/iwlwifi/dvm/rs.c index f4a6f76cf193ab6886fc787911ef807e8acdbc8f..8879e668ef0da048766ce356229eb1191a7616e6 100644 --- a/drivers/net/wireless/intel/iwlwifi/dvm/rs.c +++ b/drivers/net/wireless/intel/iwlwifi/dvm/rs.c @@ -2673,20 +2673,16 @@ static void rs_get_rate(void *priv_r, struct ieee80211_sta *sta, void *priv_sta, IWL_DEBUG_RATE_LIMIT(priv, "rate scale calculate new rate for skb\n"); /* Get max rate if user set max rate */ - if (lq_sta) { - lq_sta->max_rate_idx = fls(txrc->rate_idx_mask) - 1; - if ((sband->band == NL80211_BAND_5GHZ) && - (lq_sta->max_rate_idx != -1)) - lq_sta->max_rate_idx += IWL_FIRST_OFDM_RATE; - if ((lq_sta->max_rate_idx < 0) || - (lq_sta->max_rate_idx >= IWL_RATE_COUNT)) - lq_sta->max_rate_idx = -1; - } + lq_sta->max_rate_idx = fls(txrc->rate_idx_mask) - 1; + if (sband->band == NL80211_BAND_5GHZ && lq_sta->max_rate_idx != -1) + lq_sta->max_rate_idx += IWL_FIRST_OFDM_RATE; + if (lq_sta->max_rate_idx < 0 || lq_sta->max_rate_idx >= IWL_RATE_COUNT) + lq_sta->max_rate_idx = -1; - /* Treat uninitialized rate scaling data same as non-existing. */ - if (lq_sta && !lq_sta->drv) { + if (!lq_sta->drv) { IWL_DEBUG_RATE(priv, "Rate scaling not initialized yet.\n"); - priv_sta = NULL; + /* mac80211 already set up the data for using low rates */ + return; } rate_idx = lq_sta->last_txrate_idx; @@ -2756,7 +2752,6 @@ void iwl_rs_rate_init(struct iwl_priv *priv, struct ieee80211_sta *sta, u8 sta_i lq_sta = &sta_priv->lq_sta; sband = hw->wiphy->bands[conf->chandef.chan->band]; - lq_sta->lq.sta_id = sta_id; for (j = 0; j < LQ_SIZE; j++) diff --git a/drivers/net/wireless/intel/iwlwifi/fw/acpi.c b/drivers/net/wireless/intel/iwlwifi/fw/acpi.c index fa339791223b8fe5cd1616e598cd9416b7954d2a..79774c8c7ff4526b9ca1e6bf215df9931d1ba071 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/acpi.c +++ b/drivers/net/wireless/intel/iwlwifi/fw/acpi.c @@ -27,6 +27,7 @@ static const size_t acpi_dsm_size[DSM_FUNC_NUM_FUNCS] = { [DSM_FUNC_FORCE_DISABLE_CHANNELS] = sizeof(u32), [DSM_FUNC_ENERGY_DETECTION_THRESHOLD] = sizeof(u32), [DSM_FUNC_RFI_CONFIG] = sizeof(u32), + [DSM_FUNC_ENABLE_11BE] = sizeof(u32), }; static int iwl_acpi_get_handle(struct device *dev, acpi_string method, diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h b/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h index 47c914de2992776b69cebe87ef11dca813875359..6cfe8a779cc736af30edcc530db1de2ec6e57caa 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/api/dbg-tlv.h @@ -470,6 +470,10 @@ enum iwl_fw_ini_region_device_memory_subtype { * @IWL_FW_INI_TIME_POINT_EAPOL_FAILED: EAPOL failed * @IWL_FW_INI_TIME_POINT_FAKE_TX: fake Tx * @IWL_FW_INI_TIME_POINT_DEASSOC: de association + * @IWL_FW_INI_TIME_POINT_PRESET_OVERRIDE_EXT_REQ: request to override preset + * @IWL_FW_INI_TIME_POINT_PRESET_OVERRIDE_START: start handling override preset + * request + * @IWL_FW_INI_TIME_SCAN_FAILURE: failed scan channel list * @IWL_FW_INI_TIME_POINT_NUM: number of time points */ enum iwl_fw_ini_time_point { @@ -500,6 +504,9 @@ enum iwl_fw_ini_time_point { IWL_FW_INI_TIME_POINT_EAPOL_FAILED, IWL_FW_INI_TIME_POINT_FAKE_TX, IWL_FW_INI_TIME_POINT_DEASSOC, + IWL_FW_INI_TIME_POINT_PRESET_OVERRIDE_EXT_REQ, + IWL_FW_INI_TIME_POINT_PRESET_OVERRIDE_START, + IWL_FW_INI_TIME_SCAN_FAILURE, IWL_FW_INI_TIME_POINT_NUM, }; /* FW_TLV_DEBUG_TIME_POINT_API_E */ diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/location.h b/drivers/net/wireless/intel/iwlwifi/fw/api/location.h index 25530a29317e87e22f3e5bf3f690ea0a08bd0d9a..7c158a8dfed03ffcffe9dac52e3605af2c7e9f34 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/api/location.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/api/location.h @@ -2,6 +2,7 @@ /* * Copyright (C) 2015-2017 Intel Deutschland GmbH * Copyright (C) 2018-2022 Intel Corporation + * Copyright (C) 2024 Intel Corporation */ #ifndef __iwl_fw_api_location_h__ #define __iwl_fw_api_location_h__ @@ -390,6 +391,58 @@ struct iwl_tof_responder_config_cmd_v9 { __le16 max_time_between_msr; } __packed; /* TOF_RESPONDER_CONFIG_CMD_API_S_VER_8 */ +/** + * struct iwl_tof_responder_config_cmd - ToF AP mode + * @cmd_valid_fields: &iwl_tof_responder_cmd_valid_field + * @responder_cfg_flags: &iwl_tof_responder_cfg_flags + * @format_bw: bits 0 - 3: &enum iwl_location_frame_format. + * bits 4 - 7: &enum iwl_location_bw. + * @bss_color: current AP bss_color + * @channel_num: current AP Channel + * @ctrl_ch_position: coding of the control channel position relative to + * the center frequency, see iwl_mvm_get_ctrl_pos() + * @sta_id: index of the AP STA when in AP mode + * @band: current AP band + * @toa_offset: Artificial addition [pSec] for the ToA - to be used for debug + * purposes, simulating station movement by adding various values + * to this field + * @common_calib: XVT: common calibration value + * @specific_calib: XVT: specific calibration value + * @bssid: Current AP BSSID + * @r2i_ndp_params: parameters for R2I NDP. + * bits 0 - 2: max number of LTF repetitions + * bits 3 - 5: max number of spatial streams (supported values are < 2) + * bits 6 - 7: max number of total LTFs see + * &enum ieee80211_range_params_max_total_ltf + * @i2r_ndp_params: parameters for I2R NDP. + * bits 0 - 2: max number of LTF repetitions + * bits 3 - 5: max number of spatial streams + * bits 6 - 7: max number of total LTFs see + * &enum ieee80211_range_params_max_total_ltf + * @min_time_between_msr: for non trigger based NDP ranging, minimum time + * between measurements in milliseconds. + * @max_time_between_msr: for non trigger based NDP ranging, maximum time + * between measurements in milliseconds. + */ +struct iwl_tof_responder_config_cmd { + __le32 cmd_valid_fields; + __le32 responder_cfg_flags; + u8 format_bw; + u8 bss_color; + u8 channel_num; + u8 ctrl_ch_position; + u8 sta_id; + u8 band; + __le16 toa_offset; + __le16 common_calib; + __le16 specific_calib; + u8 bssid[ETH_ALEN]; + u8 r2i_ndp_params; + u8 i2r_ndp_params; + __le16 min_time_between_msr; + __le16 max_time_between_msr; +} __packed; /* TOF_RESPONDER_CONFIG_CMD_API_S_VER_10 */ + #define IWL_LCI_CIVIC_IE_MAX_SIZE 400 /** @@ -561,6 +614,8 @@ struct iwl_tof_range_req_ap_entry_v2 { * the responder asked for LMR feedback although the initiator did not set * the LMR feedback bit in the FTM request. If not set, the initiator will * continue with the session and will provide the LMR feedback. + * @IWL_INITIATOR_AP_FLAGS_TEST_INCORRECT_SAC: send an incorrect SAC in the + * first NDP exchange. This is used for testing. */ enum iwl_initiator_ap_flags { IWL_INITIATOR_AP_FLAGS_ASAP = BIT(1), @@ -577,6 +632,7 @@ enum iwl_initiator_ap_flags { IWL_INITIATOR_AP_FLAGS_USE_CALIB = BIT(13), IWL_INITIATOR_AP_FLAGS_PMF = BIT(14), IWL_INITIATOR_AP_FLAGS_TERMINATE_ON_LMR_FEEDBACK = BIT(15), + IWL_INITIATOR_AP_FLAGS_TEST_INCORRECT_SAC = BIT(16), }; /** @@ -797,6 +853,7 @@ struct iwl_tof_range_req_ap_entry_v7 { } __packed; /* LOCATION_RANGE_REQ_AP_ENTRY_CMD_API_S_VER_7 */ #define IWL_LOCATION_MAX_STS_POS 3 +#define IWL_LOCATION_TOTAL_LTF_POS 6 /** * struct iwl_tof_range_req_ap_entry_v8 - AP configuration parameters @@ -953,6 +1010,78 @@ struct iwl_tof_range_req_ap_entry_v9 { __le16 min_time_between_msr; } __packed; /* LOCATION_RANGE_REQ_AP_ENTRY_CMD_API_S_VER_9 */ +/** + * struct iwl_tof_range_req_ap_entry_v10 - AP configuration parameters + * @initiator_ap_flags: see &enum iwl_initiator_ap_flags. + * @band: 0 for 5.2 GHz, 1 for 2.4 GHz, 2 for 6GHz + * @channel_num: AP Channel number + * @format_bw: bits 0 - 3: &enum iwl_location_frame_format. + * bits 4 - 7: &enum iwl_location_bw. + * @ctrl_ch_position: Coding of the control channel position relative to the + * center frequency, see iwl_mvm_get_ctrl_pos(). + * @bssid: AP's BSSID + * @burst_period: For EDCA based ranging: Recommended value to be sent to the + * AP. Measurement periodicity In units of 100ms. ignored if + * num_of_bursts_exp = 0. + * For non trigger based NDP ranging, the maximum time between + * measurements in units of milliseconds. + * @samples_per_burst: the number of FTMs pairs in single Burst (1-31); + * @num_of_bursts: Recommended value to be sent to the AP. 2s Exponent of + * the number of measurement iterations (min 2^0 = 1, max 2^14) + * @sta_id: the station id of the AP. Only relevant when associated to the AP, + * otherwise should be set to &IWL_MVM_INVALID_STA. + * @cipher: pairwise cipher suite for secured measurement. + * &enum iwl_location_cipher. + * @hltk: HLTK to be used for secured 11az measurement + * @tk: TK to be used for secured 11az measurement + * @calib: An array of calibration values per FTM rx bandwidth. + * If &IWL_INITIATOR_AP_FLAGS_USE_CALIB is set, the fw will use the + * calibration value that corresponds to the rx bandwidth of the FTM + * frame. + * @beacon_interval: beacon interval of the AP in TUs. Only required if + * &IWL_INITIATOR_AP_FLAGS_TB is set. + * @rx_pn: the next expected PN for protected management frames Rx. LE byte + * order. Only valid if &IWL_INITIATOR_AP_FLAGS_SECURED is set and sta_id + * is set to &IWL_MVM_INVALID_STA. + * @tx_pn: the next PN to use for protected management frames Tx. LE byte + * order. Only valid if &IWL_INITIATOR_AP_FLAGS_SECURED is set and sta_id + * is set to &IWL_MVM_INVALID_STA. + * @r2i_ndp_params: parameters for R2I NDP ranging negotiation. + * bits 0 - 2: max LTF repetitions + * bits 3 - 5: max number of spatial streams + * bits 6 - 7: max total LTFs. One of + * &enum ieee80211_range_params_max_total_ltf. + * @i2r_ndp_params: parameters for I2R NDP ranging negotiation. + * bits 0 - 2: max LTF repetitions + * bits 3 - 5: max number of spatial streams (supported values are < 2) + * bits 6 - 7: max total LTFs. One of + * &enum ieee80211_range_params_max_total_ltf. + * @min_time_between_msr: For non trigger based NDP ranging, the minimum time + * between measurements in units of milliseconds + */ +struct iwl_tof_range_req_ap_entry_v10 { + __le32 initiator_ap_flags; + u8 band; + u8 channel_num; + u8 format_bw; + u8 ctrl_ch_position; + u8 bssid[ETH_ALEN]; + __le16 burst_period; + u8 samples_per_burst; + u8 num_of_bursts; + u8 sta_id; + u8 cipher; + u8 hltk[HLTK_11AZ_LEN]; + u8 tk[TK_11AZ_LEN]; + __le16 calib[IWL_TOF_BW_NUM]; + __le16 beacon_interval; + u8 rx_pn[IEEE80211_CCMP_PN_LEN]; + u8 tx_pn[IEEE80211_CCMP_PN_LEN]; + u8 r2i_ndp_params; + u8 i2r_ndp_params; + __le16 min_time_between_msr; +} __packed; /* LOCATION_RANGE_REQ_AP_ENTRY_CMD_API_S_VER_9 */ + /** * enum iwl_tof_response_mode * @IWL_MVM_TOF_RESPONSE_ASAP: report each AP measurement separately as soon as @@ -1230,6 +1359,34 @@ struct iwl_tof_range_req_cmd_v13 { struct iwl_tof_range_req_ap_entry_v9 ap[IWL_MVM_TOF_MAX_APS]; } __packed; /* LOCATION_RANGE_REQ_CMD_API_S_VER_13 */ +/** + * struct iwl_tof_range_req_cmd_v14 - start measurement cmd + * @initiator_flags: see flags @ iwl_tof_initiator_flags + * @request_id: A Token incremented per request. The same Token will be + * sent back in the range response + * @num_of_ap: Number of APs to measure (error if > IWL_MVM_TOF_MAX_APS) + * @range_req_bssid: ranging request BSSID + * @macaddr_mask: Bits set to 0 shall be copied from the MAC address template. + * Bits set to 1 shall be randomized by the UMAC + * @macaddr_template: MAC address template to use for non-randomized bits + * @req_timeout_ms: Requested timeout of the response in units of milliseconds. + * This is the session time for completing the measurement. + * @tsf_mac_id: report the measurement start time for each ap in terms of the + * TSF of this mac id. 0xff to disable TSF reporting. + * @ap: per-AP request data, see &struct iwl_tof_range_req_ap_entry_v10. + */ +struct iwl_tof_range_req_cmd_v14 { + __le32 initiator_flags; + u8 request_id; + u8 num_of_ap; + u8 range_req_bssid[ETH_ALEN]; + u8 macaddr_mask[ETH_ALEN]; + u8 macaddr_template[ETH_ALEN]; + __le32 req_timeout_ms; + __le32 tsf_mac_id; + struct iwl_tof_range_req_ap_entry_v10 ap[IWL_MVM_TOF_MAX_APS]; +} __packed; /* LOCATION_RANGE_REQ_CMD_API_S_VER_13 */ + /* * enum iwl_tof_range_request_status - status of the sent request * @IWL_TOF_RANGE_REQUEST_STATUS_SUCCESSFUL - FW successfully received the diff --git a/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h b/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h index a08497a04733d3a3edfa7442f7f72cb5ef618e56..644c8df780bc52fbe38c524bc1596db2551072bb 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h @@ -23,7 +23,8 @@ enum iwl_regulatory_and_nvm_subcmd_ids { * &struct iwl_lari_config_change_cmd_v4, * &struct iwl_lari_config_change_cmd_v5, * &struct iwl_lari_config_change_cmd_v6, - * &struct iwl_lari_config_change_cmd_v7 or + * &struct iwl_lari_config_change_cmd_v7, + * &struct iwl_lari_config_change_cmd_v10 or * &struct iwl_lari_config_change_cmd */ LARI_CONFIG_CHANGE = 0x1, @@ -648,7 +649,7 @@ struct iwl_lari_config_change_cmd_v7 { /* LARI_CHANGE_CONF_CMD_S_VER_9 */ /** - * struct iwl_lari_config_change_cmd - change LARI configuration + * struct iwl_lari_config_change_cmd_v10 - change LARI configuration * @config_bitmap: Bitmap of the config commands. Each bit will trigger a * different predefined FW config operation. * @oem_uhb_allow_bitmap: Bitmap of UHB enabled MCC sets. @@ -674,7 +675,7 @@ struct iwl_lari_config_change_cmd_v7 { * bit1: enable 320Mhz in South Korea. * bit 2 - 31: reserved. */ -struct iwl_lari_config_change_cmd { +struct iwl_lari_config_change_cmd_v10 { __le32 config_bitmap; __le32 oem_uhb_allow_bitmap; __le32 oem_11ax_allow_bitmap; @@ -686,6 +687,51 @@ struct iwl_lari_config_change_cmd { } __packed; /* LARI_CHANGE_CONF_CMD_S_VER_10 */ +/** + * struct iwl_lari_config_change_cmd - change LARI configuration + * @config_bitmap: Bitmap of the config commands. Each bit will trigger a + * different predefined FW config operation. + * @oem_uhb_allow_bitmap: Bitmap of UHB enabled MCC sets. + * @oem_11ax_allow_bitmap: Bitmap of 11ax allowed MCCs. There are two bits + * per country, one to indicate whether to override and the other to + * indicate the value to use. + * @oem_unii4_allow_bitmap: Bitmap of unii4 allowed MCCs.There are two bits + * per country, one to indicate whether to override and the other to + * indicate allow/disallow unii4 channels. + * For LARI cmd version 11 - bits 0:5 are supported. + * @chan_state_active_bitmap: Bitmap to enable different bands per country + * or region. + * Each bit represents a country or region, and a band to activate + * according to the BIOS definitions. + * For LARI cmd version 11 - bits 0:4 are supported. + * @force_disable_channels_bitmap: Bitmap of disabled bands/channels. + * Each bit represents a set of channels in a specific band that should be + * disabled + * @edt_bitmap: Bitmap of energy detection threshold table. + * Disable/enable the EDT optimization method for different band. + * @oem_320mhz_allow_bitmap: 320Mhz bandwidth enablement bitmap per MCC. + * bit0: enable 320Mhz in Japan. + * bit1: enable 320Mhz in South Korea. + * bit 2 - 31: reserved. + * @oem_11be_allow_bitmap: Bitmap of 11be allowed MCCs. No need to mask out the + * unsupported bits + * bit0: enable 11be in China(CB/CN). + * bit1: enable 11be in South Korea. + * bit 2 - 31: reserved. + */ +struct iwl_lari_config_change_cmd { + __le32 config_bitmap; + __le32 oem_uhb_allow_bitmap; + __le32 oem_11ax_allow_bitmap; + __le32 oem_unii4_allow_bitmap; + __le32 chan_state_active_bitmap; + __le32 force_disable_channels_bitmap; + __le32 edt_bitmap; + __le32 oem_320mhz_allow_bitmap; + __le32 oem_11be_allow_bitmap; +} __packed; +/* LARI_CHANGE_CONF_CMD_S_VER_11 */ + /* Activate UNII-1 (5.2GHz) for World Wide */ #define ACTIVATE_5G2_IN_WW_MASK BIT(4) diff --git a/drivers/net/wireless/intel/iwlwifi/fw/debugfs.c b/drivers/net/wireless/intel/iwlwifi/fw/debugfs.c index 751a125a1566f87c8a734c6cfb55a5a4ea7c7381..893b21fcaf87c6322a7c2df950ab94155274fd63 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/debugfs.c +++ b/drivers/net/wireless/intel/iwlwifi/fw/debugfs.c @@ -230,8 +230,7 @@ static ssize_t iwl_dbgfs_send_hcmd_write(struct iwl_fw_runtime *fwrt, char *buf, .data = { NULL, }, }; - if (fwrt->ops && fwrt->ops->fw_running && - !fwrt->ops->fw_running(fwrt->ops_ctx)) + if (!iwl_trans_fw_running(fwrt->trans)) return -EIO; if (count < header_size + 1 || count > 1024 * 4) diff --git a/drivers/net/wireless/intel/iwlwifi/fw/init.c b/drivers/net/wireless/intel/iwlwifi/fw/init.c index 135bd48bfe9fa80fbd263b2c8ffa3348fa7440fd..d8b083be5b6b5e5a60a582870185d1788c1d38ee 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/init.c +++ b/drivers/net/wireless/intel/iwlwifi/fw/init.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause /* * Copyright (C) 2017 Intel Deutschland GmbH - * Copyright (C) 2019-2021 Intel Corporation + * Copyright (C) 2019-2021, 2024 Intel Corporation */ #include "iwl-drv.h" #include "runtime.h" @@ -135,7 +135,9 @@ int iwl_configure_rxq(struct iwl_fw_runtime *fwrt) struct iwl_trans_rxq_dma_data data; cmd->data[i].q_num = i + 1; - iwl_trans_get_rxq_dma_data(fwrt->trans, i + 1, &data); + ret = iwl_trans_get_rxq_dma_data(fwrt->trans, i + 1, &data); + if (ret) + goto out; cmd->data[i].fr_bd_cb = cpu_to_le64(data.fr_bd_cb); cmd->data[i].urbd_stts_wrptr = @@ -149,6 +151,7 @@ int iwl_configure_rxq(struct iwl_fw_runtime *fwrt) ret = iwl_trans_send_cmd(fwrt->trans, &hcmd); +out: kfree(cmd); if (ret) diff --git a/drivers/net/wireless/intel/iwlwifi/fw/regulatory.c b/drivers/net/wireless/intel/iwlwifi/fw/regulatory.c index b9bb3636e88f0342e534173a9be0983849b97326..5228b837a9ef68a11dd22e57234733b9151867c0 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/regulatory.c +++ b/drivers/net/wireless/intel/iwlwifi/fw/regulatory.c @@ -497,9 +497,12 @@ static size_t iwl_get_lari_config_cmd_size(u8 cmd_ver) size_t cmd_size; switch (cmd_ver) { - case 10: + case 11: cmd_size = sizeof(struct iwl_lari_config_change_cmd); break; + case 10: + cmd_size = sizeof(struct iwl_lari_config_change_cmd_v10); + break; case 9: case 8: case 7: @@ -580,6 +583,10 @@ int iwl_fill_lari_config(struct iwl_fw_runtime *fwrt, if (!ret) cmd->oem_320mhz_allow_bitmap = cpu_to_le32(value); + ret = iwl_bios_get_dsm(fwrt, DSM_FUNC_ENABLE_11BE, &value); + if (!ret) + cmd->oem_11be_allow_bitmap = cpu_to_le32(value); + if (cmd->config_bitmap || cmd->oem_uhb_allow_bitmap || cmd->oem_11ax_allow_bitmap || @@ -587,7 +594,8 @@ int iwl_fill_lari_config(struct iwl_fw_runtime *fwrt, cmd->chan_state_active_bitmap || cmd->force_disable_channels_bitmap || cmd->edt_bitmap || - cmd->oem_320mhz_allow_bitmap) { + cmd->oem_320mhz_allow_bitmap || + cmd->oem_11be_allow_bitmap) { IWL_DEBUG_RADIO(fwrt, "sending LARI_CONFIG_CHANGE, config_bitmap=0x%x, oem_11ax_allow_bitmap=0x%x\n", le32_to_cpu(cmd->config_bitmap), @@ -605,6 +613,9 @@ int iwl_fill_lari_config(struct iwl_fw_runtime *fwrt, "sending LARI_CONFIG_CHANGE, edt_bitmap=0x%x, oem_320mhz_allow_bitmap=0x%x\n", le32_to_cpu(cmd->edt_bitmap), le32_to_cpu(cmd->oem_320mhz_allow_bitmap)); + IWL_DEBUG_RADIO(fwrt, + "sending LARI_CONFIG_CHANGE, oem_11be_allow_bitmap=0x%x\n", + le32_to_cpu(cmd->oem_11be_allow_bitmap)); } else { return 1; } diff --git a/drivers/net/wireless/intel/iwlwifi/fw/regulatory.h b/drivers/net/wireless/intel/iwlwifi/fw/regulatory.h index 633c9ad9af841e39fb7ceb096b28a1fd500fb608..e2c056f483c1c70980d62a70c811ccba33ba28e5 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/regulatory.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/regulatory.h @@ -115,7 +115,8 @@ enum iwl_dsm_funcs { DSM_FUNC_FORCE_DISABLE_CHANNELS = 9, DSM_FUNC_ENERGY_DETECTION_THRESHOLD = 10, DSM_FUNC_RFI_CONFIG = 11, - DSM_FUNC_NUM_FUNCS = 12, + DSM_FUNC_ENABLE_11BE = 12, + DSM_FUNC_NUM_FUNCS = 13, }; enum iwl_dsm_values_srd { diff --git a/drivers/net/wireless/intel/iwlwifi/fw/runtime.h b/drivers/net/wireless/intel/iwlwifi/fw/runtime.h index 9122f9a1260ae0a505d2bb8f6310dba4bc3ee76c..3fcc08b3c0db007c789065cfeed45c6c62b2120e 100644 --- a/drivers/net/wireless/intel/iwlwifi/fw/runtime.h +++ b/drivers/net/wireless/intel/iwlwifi/fw/runtime.h @@ -12,14 +12,13 @@ #include "fw/api/debug.h" #include "fw/api/paging.h" #include "fw/api/power.h" -#include "iwl-eeprom-parse.h" +#include "iwl-nvm-utils.h" #include "fw/acpi.h" #include "fw/regulatory.h" struct iwl_fw_runtime_ops { void (*dump_start)(void *ctx); void (*dump_end)(void *ctx); - bool (*fw_running)(void *ctx); int (*send_hcmd)(void *ctx, struct iwl_host_cmd *host_cmd); bool (*d3_debug_enable)(void *ctx); }; diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-devtrace-data.h b/drivers/net/wireless/intel/iwlwifi/iwl-devtrace-data.h index 2c280a2fe3df323cd3ee3b7bdd49bd469300e883..0d4a0896a2c51b0efad624329bc8514bbffcd9f9 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-devtrace-data.h +++ b/drivers/net/wireless/intel/iwlwifi/iwl-devtrace-data.h @@ -3,7 +3,7 @@ * * Copyright(c) 2009 - 2014 Intel Corporation. All rights reserved. * Copyright(c) 2015 Intel Deutschland GmbH - * Copyright(c) 2018 - 2019, 2023 Intel Corporation + * Copyright(c) 2018 - 2019, 2023-2024 Intel Corporation *****************************************************************************/ #if !defined(__IWLWIFI_DEVICE_TRACE_DATA) || defined(TRACE_HEADER_MULTI_READ) @@ -28,7 +28,7 @@ TRACE_EVENT(iwlwifi_dev_tx_tb, TP_fast_assign( DEV_ASSIGN; __entry->phys = phys; - if (iwl_trace_data(skb)) + if (__get_dynamic_array_len(data)) memcpy(__get_dynamic_array(data), data_src, data_len); ), TP_printk("[%s] TX frame data", __get_str(dev)) diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-devtrace-iwlwifi.h b/drivers/net/wireless/intel/iwlwifi/iwl-devtrace-iwlwifi.h index e656bf6bc00338248e0a8ab61cf8afe38d46c6ae..ead72c3d33bdf02dd5bfef62e9284dc78777e600 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-devtrace-iwlwifi.h +++ b/drivers/net/wireless/intel/iwlwifi/iwl-devtrace-iwlwifi.h @@ -4,7 +4,7 @@ * Copyright(c) 2009 - 2014 Intel Corporation. All rights reserved. * Copyright(c) 2015 Intel Mobile Communications GmbH * Copyright(c) 2016 - 2017 Intel Deutschland GmbH - * Copyright(c) 2018, 2023 Intel Corporation + * Copyright(c) 2018, 2023-2024 Intel Corporation *****************************************************************************/ #if !defined(__IWLWIFI_DEVICE_TRACE_IWLWIFI) || defined(TRACE_HEADER_MULTI_READ) @@ -88,8 +88,8 @@ TRACE_EVENT(iwlwifi_dev_tx, * for the possible padding). */ __dynamic_array(u8, buf0, buf0_len) - __dynamic_array(u8, buf1, hdr_len > 0 && iwl_trace_data(skb) ? - 0 : skb->len - hdr_len) + __dynamic_array(u8, buf1, hdr_len > 0 && !iwl_trace_data(skb) ? + skb->len - hdr_len : 0) ), TP_fast_assign( DEV_ASSIGN; @@ -99,7 +99,7 @@ TRACE_EVENT(iwlwifi_dev_tx, __entry->framelen += skb->len - hdr_len; memcpy(__get_dynamic_array(tfd), tfd, tfdlen); memcpy(__get_dynamic_array(buf0), buf0, buf0_len); - if (hdr_len > 0 && !iwl_trace_data(skb)) + if (__get_dynamic_array_len(buf1)) skb_copy_bits(skb, hdr_len, __get_dynamic_array(buf1), skb->len - hdr_len); diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-drv.c b/drivers/net/wireless/intel/iwlwifi/iwl-drv.c index 33654f228ee871f7b8bbe1463351cc677904662a..249dfd207fcfc3b3e71768db2b409dc5c4c05db9 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-drv.c +++ b/drivers/net/wireless/intel/iwlwifi/iwl-drv.c @@ -982,16 +982,10 @@ static int iwl_parse_tlv_firmware(struct iwl_drv *drv, minor = le32_to_cpup(ptr++); local_comp = le32_to_cpup(ptr); - if (major >= 35) - snprintf(drv->fw.fw_version, - sizeof(drv->fw.fw_version), - "%u.%08x.%u %s", major, minor, - local_comp, iwl_reduced_fw_name(drv)); - else - snprintf(drv->fw.fw_version, - sizeof(drv->fw.fw_version), - "%u.%u.%u %s", major, minor, - local_comp, iwl_reduced_fw_name(drv)); + snprintf(drv->fw.fw_version, + sizeof(drv->fw.fw_version), + "%u.%08x.%u %s", major, minor, + local_comp, iwl_reduced_fw_name(drv)); break; } case IWL_UCODE_TLV_FW_DBG_DEST: { @@ -1815,8 +1809,8 @@ struct iwl_drv *iwl_drv_start(struct iwl_trans *trans) err_fw: #ifdef CONFIG_IWLWIFI_DEBUGFS debugfs_remove_recursive(drv->dbgfs_drv); - iwl_dbg_tlv_free(drv->trans); #endif + iwl_dbg_tlv_free(drv->trans); kfree(drv); err: return ERR_PTR(ret); diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-read.c b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-read.c deleted file mode 100644 index 5f386bb1a353946d41d559d123c8382105c3e441..0000000000000000000000000000000000000000 --- a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-read.c +++ /dev/null @@ -1,394 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause -/* - * Copyright (C) 2005-2014, 2018-2019, 2021 Intel Corporation - */ -#include -#include -#include - -#include "iwl-drv.h" -#include "iwl-debug.h" -#include "iwl-eeprom-read.h" -#include "iwl-io.h" -#include "iwl-prph.h" -#include "iwl-csr.h" - -/* - * EEPROM access time values: - * - * Driver initiates EEPROM read by writing byte address << 1 to CSR_EEPROM_REG. - * Driver then polls CSR_EEPROM_REG for CSR_EEPROM_REG_READ_VALID_MSK (0x1). - * When polling, wait 10 uSec between polling loops, up to a maximum 5000 uSec. - * Driver reads 16-bit value from bits 31-16 of CSR_EEPROM_REG. - */ -#define IWL_EEPROM_ACCESS_TIMEOUT 5000 /* uSec */ - -/* - * The device's EEPROM semaphore prevents conflicts between driver and uCode - * when accessing the EEPROM; each access is a series of pulses to/from the - * EEPROM chip, not a single event, so even reads could conflict if they - * weren't arbitrated by the semaphore. - */ -#define IWL_EEPROM_SEM_TIMEOUT 10 /* microseconds */ -#define IWL_EEPROM_SEM_RETRY_LIMIT 1000 /* number of attempts (not time) */ - - -static int iwl_eeprom_acquire_semaphore(struct iwl_trans *trans) -{ - u16 count; - int ret; - - for (count = 0; count < IWL_EEPROM_SEM_RETRY_LIMIT; count++) { - /* Request semaphore */ - iwl_set_bit(trans, CSR_HW_IF_CONFIG_REG, - CSR_HW_IF_CONFIG_REG_BIT_EEPROM_OWN_SEM); - - /* See if we got it */ - ret = iwl_poll_bit(trans, CSR_HW_IF_CONFIG_REG, - CSR_HW_IF_CONFIG_REG_BIT_EEPROM_OWN_SEM, - CSR_HW_IF_CONFIG_REG_BIT_EEPROM_OWN_SEM, - IWL_EEPROM_SEM_TIMEOUT); - if (ret >= 0) { - IWL_DEBUG_EEPROM(trans->dev, - "Acquired semaphore after %d tries.\n", - count+1); - return ret; - } - } - - return ret; -} - -static void iwl_eeprom_release_semaphore(struct iwl_trans *trans) -{ - iwl_clear_bit(trans, CSR_HW_IF_CONFIG_REG, - CSR_HW_IF_CONFIG_REG_BIT_EEPROM_OWN_SEM); -} - -static int iwl_eeprom_verify_signature(struct iwl_trans *trans, bool nvm_is_otp) -{ - u32 gp = iwl_read32(trans, CSR_EEPROM_GP) & CSR_EEPROM_GP_VALID_MSK; - - IWL_DEBUG_EEPROM(trans->dev, "EEPROM signature=0x%08x\n", gp); - - switch (gp) { - case CSR_EEPROM_GP_BAD_SIG_EEP_GOOD_SIG_OTP: - if (!nvm_is_otp) { - IWL_ERR(trans, "EEPROM with bad signature: 0x%08x\n", - gp); - return -ENOENT; - } - return 0; - case CSR_EEPROM_GP_GOOD_SIG_EEP_LESS_THAN_4K: - case CSR_EEPROM_GP_GOOD_SIG_EEP_MORE_THAN_4K: - if (nvm_is_otp) { - IWL_ERR(trans, "OTP with bad signature: 0x%08x\n", gp); - return -ENOENT; - } - return 0; - case CSR_EEPROM_GP_BAD_SIGNATURE_BOTH_EEP_AND_OTP: - default: - IWL_ERR(trans, - "bad EEPROM/OTP signature, type=%s, EEPROM_GP=0x%08x\n", - nvm_is_otp ? "OTP" : "EEPROM", gp); - return -ENOENT; - } -} - -/****************************************************************************** - * - * OTP related functions - * -******************************************************************************/ - -static void iwl_set_otp_access_absolute(struct iwl_trans *trans) -{ - iwl_read32(trans, CSR_OTP_GP_REG); - - iwl_clear_bit(trans, CSR_OTP_GP_REG, - CSR_OTP_GP_REG_OTP_ACCESS_MODE); -} - -static int iwl_nvm_is_otp(struct iwl_trans *trans) -{ - u32 otpgp; - - /* OTP only valid for CP/PP and after */ - switch (trans->hw_rev & CSR_HW_REV_TYPE_MSK) { - case CSR_HW_REV_TYPE_NONE: - IWL_ERR(trans, "Unknown hardware type\n"); - return -EIO; - case CSR_HW_REV_TYPE_5300: - case CSR_HW_REV_TYPE_5350: - case CSR_HW_REV_TYPE_5100: - case CSR_HW_REV_TYPE_5150: - return 0; - default: - otpgp = iwl_read32(trans, CSR_OTP_GP_REG); - if (otpgp & CSR_OTP_GP_REG_DEVICE_SELECT) - return 1; - return 0; - } -} - -static int iwl_init_otp_access(struct iwl_trans *trans) -{ - int ret; - - ret = iwl_finish_nic_init(trans); - if (ret) - return ret; - - iwl_set_bits_prph(trans, APMG_PS_CTRL_REG, - APMG_PS_CTRL_VAL_RESET_REQ); - udelay(5); - iwl_clear_bits_prph(trans, APMG_PS_CTRL_REG, - APMG_PS_CTRL_VAL_RESET_REQ); - - /* - * CSR auto clock gate disable bit - - * this is only applicable for HW with OTP shadow RAM - */ - if (trans->trans_cfg->base_params->shadow_ram_support) - iwl_set_bit(trans, CSR_DBG_LINK_PWR_MGMT_REG, - CSR_RESET_LINK_PWR_MGMT_DISABLED); - - return 0; -} - -static int iwl_read_otp_word(struct iwl_trans *trans, u16 addr, - __le16 *eeprom_data) -{ - int ret = 0; - u32 r; - u32 otpgp; - - iwl_write32(trans, CSR_EEPROM_REG, - CSR_EEPROM_REG_MSK_ADDR & (addr << 1)); - ret = iwl_poll_bit(trans, CSR_EEPROM_REG, - CSR_EEPROM_REG_READ_VALID_MSK, - CSR_EEPROM_REG_READ_VALID_MSK, - IWL_EEPROM_ACCESS_TIMEOUT); - if (ret < 0) { - IWL_ERR(trans, "Time out reading OTP[%d]\n", addr); - return ret; - } - r = iwl_read32(trans, CSR_EEPROM_REG); - /* check for ECC errors: */ - otpgp = iwl_read32(trans, CSR_OTP_GP_REG); - if (otpgp & CSR_OTP_GP_REG_ECC_UNCORR_STATUS_MSK) { - /* stop in this case */ - /* set the uncorrectable OTP ECC bit for acknowledgment */ - iwl_set_bit(trans, CSR_OTP_GP_REG, - CSR_OTP_GP_REG_ECC_UNCORR_STATUS_MSK); - IWL_ERR(trans, "Uncorrectable OTP ECC error, abort OTP read\n"); - return -EINVAL; - } - if (otpgp & CSR_OTP_GP_REG_ECC_CORR_STATUS_MSK) { - /* continue in this case */ - /* set the correctable OTP ECC bit for acknowledgment */ - iwl_set_bit(trans, CSR_OTP_GP_REG, - CSR_OTP_GP_REG_ECC_CORR_STATUS_MSK); - IWL_ERR(trans, "Correctable OTP ECC error, continue read\n"); - } - *eeprom_data = cpu_to_le16(r >> 16); - return 0; -} - -/* - * iwl_is_otp_empty: check for empty OTP - */ -static bool iwl_is_otp_empty(struct iwl_trans *trans) -{ - u16 next_link_addr = 0; - __le16 link_value; - bool is_empty = false; - - /* locate the beginning of OTP link list */ - if (!iwl_read_otp_word(trans, next_link_addr, &link_value)) { - if (!link_value) { - IWL_ERR(trans, "OTP is empty\n"); - is_empty = true; - } - } else { - IWL_ERR(trans, "Unable to read first block of OTP list.\n"); - is_empty = true; - } - - return is_empty; -} - - -/* - * iwl_find_otp_image: find EEPROM image in OTP - * finding the OTP block that contains the EEPROM image. - * the last valid block on the link list (the block _before_ the last block) - * is the block we should read and used to configure the device. - * If all the available OTP blocks are full, the last block will be the block - * we should read and used to configure the device. - * only perform this operation if shadow RAM is disabled - */ -static int iwl_find_otp_image(struct iwl_trans *trans, - u16 *validblockaddr) -{ - u16 next_link_addr = 0, valid_addr; - __le16 link_value = 0; - int usedblocks = 0; - - /* set addressing mode to absolute to traverse the link list */ - iwl_set_otp_access_absolute(trans); - - /* checking for empty OTP or error */ - if (iwl_is_otp_empty(trans)) - return -EINVAL; - - /* - * start traverse link list - * until reach the max number of OTP blocks - * different devices have different number of OTP blocks - */ - do { - /* save current valid block address - * check for more block on the link list - */ - valid_addr = next_link_addr; - next_link_addr = le16_to_cpu(link_value) * sizeof(u16); - IWL_DEBUG_EEPROM(trans->dev, "OTP blocks %d addr 0x%x\n", - usedblocks, next_link_addr); - if (iwl_read_otp_word(trans, next_link_addr, &link_value)) - return -EINVAL; - if (!link_value) { - /* - * reach the end of link list, return success and - * set address point to the starting address - * of the image - */ - *validblockaddr = valid_addr; - /* skip first 2 bytes (link list pointer) */ - *validblockaddr += 2; - return 0; - } - /* more in the link list, continue */ - usedblocks++; - } while (usedblocks <= trans->trans_cfg->base_params->max_ll_items); - - /* OTP has no valid blocks */ - IWL_DEBUG_EEPROM(trans->dev, "OTP has no valid blocks\n"); - return -EINVAL; -} - -/* - * iwl_read_eeprom - read EEPROM contents - * - * Load the EEPROM contents from adapter and return it - * and its size. - * - * NOTE: This routine uses the non-debug IO access functions. - */ -int iwl_read_eeprom(struct iwl_trans *trans, u8 **eeprom, size_t *eeprom_size) -{ - __le16 *e; - u32 gp = iwl_read32(trans, CSR_EEPROM_GP); - int sz; - int ret; - u16 addr; - u16 validblockaddr = 0; - u16 cache_addr = 0; - int nvm_is_otp; - - if (!eeprom || !eeprom_size) - return -EINVAL; - - nvm_is_otp = iwl_nvm_is_otp(trans); - if (nvm_is_otp < 0) - return nvm_is_otp; - - sz = trans->trans_cfg->base_params->eeprom_size; - IWL_DEBUG_EEPROM(trans->dev, "NVM size = %d\n", sz); - - e = kmalloc(sz, GFP_KERNEL); - if (!e) - return -ENOMEM; - - ret = iwl_eeprom_verify_signature(trans, nvm_is_otp); - if (ret < 0) { - IWL_ERR(trans, "EEPROM not found, EEPROM_GP=0x%08x\n", gp); - goto err_free; - } - - /* Make sure driver (instead of uCode) is allowed to read EEPROM */ - ret = iwl_eeprom_acquire_semaphore(trans); - if (ret < 0) { - IWL_ERR(trans, "Failed to acquire EEPROM semaphore.\n"); - goto err_free; - } - - if (nvm_is_otp) { - ret = iwl_init_otp_access(trans); - if (ret) { - IWL_ERR(trans, "Failed to initialize OTP access.\n"); - goto err_unlock; - } - - iwl_write32(trans, CSR_EEPROM_GP, - iwl_read32(trans, CSR_EEPROM_GP) & - ~CSR_EEPROM_GP_IF_OWNER_MSK); - - iwl_set_bit(trans, CSR_OTP_GP_REG, - CSR_OTP_GP_REG_ECC_CORR_STATUS_MSK | - CSR_OTP_GP_REG_ECC_UNCORR_STATUS_MSK); - /* traversing the linked list if no shadow ram supported */ - if (!trans->trans_cfg->base_params->shadow_ram_support) { - ret = iwl_find_otp_image(trans, &validblockaddr); - if (ret) - goto err_unlock; - } - for (addr = validblockaddr; addr < validblockaddr + sz; - addr += sizeof(u16)) { - __le16 eeprom_data; - - ret = iwl_read_otp_word(trans, addr, &eeprom_data); - if (ret) - goto err_unlock; - e[cache_addr / 2] = eeprom_data; - cache_addr += sizeof(u16); - } - } else { - /* eeprom is an array of 16bit values */ - for (addr = 0; addr < sz; addr += sizeof(u16)) { - u32 r; - - iwl_write32(trans, CSR_EEPROM_REG, - CSR_EEPROM_REG_MSK_ADDR & (addr << 1)); - - ret = iwl_poll_bit(trans, CSR_EEPROM_REG, - CSR_EEPROM_REG_READ_VALID_MSK, - CSR_EEPROM_REG_READ_VALID_MSK, - IWL_EEPROM_ACCESS_TIMEOUT); - if (ret < 0) { - IWL_ERR(trans, - "Time out reading EEPROM[%d]\n", addr); - goto err_unlock; - } - r = iwl_read32(trans, CSR_EEPROM_REG); - e[addr / 2] = cpu_to_le16(r >> 16); - } - } - - IWL_DEBUG_EEPROM(trans->dev, "NVM Type: %s\n", - nvm_is_otp ? "OTP" : "EEPROM"); - - iwl_eeprom_release_semaphore(trans); - - *eeprom_size = sz; - *eeprom = (u8 *)e; - return 0; - - err_unlock: - iwl_eeprom_release_semaphore(trans); - err_free: - kfree(e); - - return ret; -} -IWL_EXPORT_SYMBOL(iwl_read_eeprom); diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-read.h b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-read.h deleted file mode 100644 index 63b8e6c6659b686443dbdde2457fca0f8b9b23bc..0000000000000000000000000000000000000000 --- a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-read.h +++ /dev/null @@ -1,12 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */ -/* - * Copyright (C) 2005-2014 Intel Corporation - */ -#ifndef __iwl_eeprom_h__ -#define __iwl_eeprom_h__ - -#include "iwl-trans.h" - -int iwl_read_eeprom(struct iwl_trans *trans, u8 **eeprom, size_t *eeprom_size); - -#endif /* __iwl_eeprom_h__ */ diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-modparams.h b/drivers/net/wireless/intel/iwlwifi/iwl-modparams.h index 1cf26ab4f4887f27f1a30f220ad1f27deecdf954..21eabfc3ffc84b8ec3670feba7d3e4883469fdce 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-modparams.h +++ b/drivers/net/wireless/intel/iwlwifi/iwl-modparams.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */ /* - * Copyright (C) 2005-2014, 2018-2022 Intel Corporation + * Copyright (C) 2005-2014, 2018-2022, 2024 Intel Corporation */ #ifndef __iwl_modparams_h__ #define __iwl_modparams_h__ @@ -106,4 +106,23 @@ static inline bool iwl_enable_tx_ampdu(void) return true; } +/* Verify amsdu_size module parameter and convert it to a rxb size */ +static inline enum iwl_amsdu_size +iwl_amsdu_size_to_rxb_size(void) +{ + switch (iwlwifi_mod_params.amsdu_size) { + case IWL_AMSDU_8K: + return IWL_AMSDU_8K; + case IWL_AMSDU_12K: + return IWL_AMSDU_12K; + default: + pr_err("%s: Unsupported amsdu_size: %d\n", KBUILD_MODNAME, + iwlwifi_mod_params.amsdu_size); + fallthrough; + case IWL_AMSDU_DEF: + case IWL_AMSDU_4K: + return IWL_AMSDU_4K; + } +} + #endif /* #__iwl_modparams_h__ */ diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c index 149903f5256730f1644c9484f5beb0f6277936f9..712dabb30acc4831eeecfbb2a54d277601ff5fc9 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c +++ b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c @@ -38,16 +38,13 @@ enum nvm_offsets { N_HW_ADDRS = 3, NVM_CHANNELS = 0x1E0 - NVM_SW_SECTION, - /* NVM calibration section offset (in words) definitions */ - NVM_CALIB_SECTION = 0x2B8, - XTAL_CALIB = 0x316 - NVM_CALIB_SECTION, - /* NVM REGULATORY -Section offset (in words) definitions */ NVM_CHANNELS_SDP = 0, }; enum ext_nvm_offsets { /* NVM HW-Section offset (in words) definitions */ + MAC_ADDRESS_OVERRIDE_EXT_NVM = 1, /* NVM SW-Section offset (in words) definitions */ @@ -1574,9 +1571,6 @@ iwl_parse_nvm_data(struct iwl_trans *trans, const struct iwl_cfg *cfg, ®ulatory[NVM_CHANNELS_SDP] : &nvm_sw[NVM_CHANNELS]; - /* in family 8000 Xtal calibration values moved to OTP */ - data->xtal_calib[0] = *(nvm_calib + XTAL_CALIB); - data->xtal_calib[1] = *(nvm_calib + XTAL_CALIB + 1); lar_enabled = true; } else { u16 lar_offset = data->nvm_version < 0xE39 ? @@ -1625,11 +1619,15 @@ static u32 iwl_nvm_get_regdom_bw_flags(const u16 *nvm_chan, flags &= ~NL80211_RRF_NO_HT40PLUS; if (nvm_chan[ch_idx] >= FIRST_2GHZ_HT_MINUS) flags &= ~NL80211_RRF_NO_HT40MINUS; - } else if (nvm_flags & NVM_CHANNEL_40MHZ) { + } else if (ch_idx < NUM_2GHZ_CHANNELS + NUM_5GHZ_CHANNELS && + nvm_flags & NVM_CHANNEL_40MHZ) { if ((ch_idx - NUM_2GHZ_CHANNELS) % 2 == 0) flags &= ~NL80211_RRF_NO_HT40PLUS; else flags &= ~NL80211_RRF_NO_HT40MINUS; + } else if (nvm_flags & NVM_CHANNEL_40MHZ) { + flags &= ~NL80211_RRF_NO_HT40PLUS; + flags &= ~NL80211_RRF_NO_HT40MINUS; } if (!(nvm_flags & NVM_CHANNEL_80MHZ)) @@ -1732,7 +1730,6 @@ iwl_parse_nvm_mcc_info(struct device *dev, const struct iwl_cfg *cfg, const u16 *nvm_chan; struct ieee80211_regdomain *regd, *copy_rd; struct ieee80211_reg_rule *rule; - enum nl80211_band band; int center_freq, prev_center_freq = 0; int valid_rules = 0; bool new_rule; @@ -1776,8 +1773,10 @@ iwl_parse_nvm_mcc_info(struct device *dev, const struct iwl_cfg *cfg, reg_capa = iwl_get_reg_capa(cap, resp_ver); for (ch_idx = 0; ch_idx < num_of_ch; ch_idx++) { + enum nl80211_band band = + iwl_nl80211_band_from_channel_idx(ch_idx); + ch_flags = (u16)__le32_to_cpup(channels + ch_idx); - band = iwl_nl80211_band_from_channel_idx(ch_idx); center_freq = ieee80211_channel_to_frequency(nvm_chan[ch_idx], band); new_rule = false; diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.h b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.h index fd9c3bed9407983c5a8bc0be68900b591989d81e..5a1b5ab62de152d27903bcf5ca2a94c23908f659 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.h +++ b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.h @@ -7,7 +7,7 @@ #define __iwl_nvm_parse_h__ #include -#include "iwl-eeprom-parse.h" +#include "iwl-nvm-utils.h" #include "mei/iwl-mei.h" /** diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-utils.c b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-utils.c new file mode 100644 index 0000000000000000000000000000000000000000..b3c25acd3691bdd86ed9a14dfbc83ab06aad7134 --- /dev/null +++ b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-utils.c @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +/* + * Copyright (C) 2005-2014, 2018-2021, 2023 Intel Corporation + * Copyright (C) 2015 Intel Mobile Communications GmbH + */ +#include +#include +#include +#include "iwl-drv.h" +#include "iwl-modparams.h" +#include "iwl-nvm-utils.h" + +int iwl_init_sband_channels(struct iwl_nvm_data *data, + struct ieee80211_supported_band *sband, + int n_channels, enum nl80211_band band) +{ + struct ieee80211_channel *chan = &data->channels[0]; + int n = 0, idx = 0; + + while (idx < n_channels && chan->band != band) + chan = &data->channels[++idx]; + + sband->channels = &data->channels[idx]; + + while (idx < n_channels && chan->band == band) { + chan = &data->channels[++idx]; + n++; + } + + sband->n_channels = n; + + return n; +} +IWL_EXPORT_SYMBOL(iwl_init_sband_channels); + +#define MAX_BIT_RATE_40_MHZ 150 /* Mbps */ +#define MAX_BIT_RATE_20_MHZ 72 /* Mbps */ + +void iwl_init_ht_hw_capab(struct iwl_trans *trans, + struct iwl_nvm_data *data, + struct ieee80211_sta_ht_cap *ht_info, + enum nl80211_band band, + u8 tx_chains, u8 rx_chains) +{ + const struct iwl_cfg *cfg = trans->cfg; + int max_bit_rate = 0; + + tx_chains = hweight8(tx_chains); + if (cfg->rx_with_siso_diversity) + rx_chains = 1; + else + rx_chains = hweight8(rx_chains); + + if (!(data->sku_cap_11n_enable) || + (iwlwifi_mod_params.disable_11n & IWL_DISABLE_HT_ALL) || + !cfg->ht_params) { + ht_info->ht_supported = false; + return; + } + + if (data->sku_cap_mimo_disabled) + rx_chains = 1; + + ht_info->ht_supported = true; + ht_info->cap = IEEE80211_HT_CAP_DSSSCCK40; + + if (cfg->ht_params->stbc) { + ht_info->cap |= (1 << IEEE80211_HT_CAP_RX_STBC_SHIFT); + + if (tx_chains > 1) + ht_info->cap |= IEEE80211_HT_CAP_TX_STBC; + } + + if (cfg->ht_params->ldpc) + ht_info->cap |= IEEE80211_HT_CAP_LDPC_CODING; + + if (trans->trans_cfg->mq_rx_supported || + iwlwifi_mod_params.amsdu_size >= IWL_AMSDU_8K) + ht_info->cap |= IEEE80211_HT_CAP_MAX_AMSDU; + + ht_info->ampdu_factor = IEEE80211_HT_MAX_AMPDU_64K; + ht_info->ampdu_density = IEEE80211_HT_MPDU_DENSITY_4; + + ht_info->mcs.rx_mask[0] = 0xFF; + ht_info->mcs.rx_mask[1] = 0x00; + ht_info->mcs.rx_mask[2] = 0x00; + + if (rx_chains >= 2) + ht_info->mcs.rx_mask[1] = 0xFF; + if (rx_chains >= 3) + ht_info->mcs.rx_mask[2] = 0xFF; + + if (cfg->ht_params->ht_greenfield_support) + ht_info->cap |= IEEE80211_HT_CAP_GRN_FLD; + ht_info->cap |= IEEE80211_HT_CAP_SGI_20; + + max_bit_rate = MAX_BIT_RATE_20_MHZ; + + if (cfg->ht_params->ht40_bands & BIT(band)) { + ht_info->cap |= IEEE80211_HT_CAP_SUP_WIDTH_20_40; + ht_info->cap |= IEEE80211_HT_CAP_SGI_40; + max_bit_rate = MAX_BIT_RATE_40_MHZ; + } + + /* Highest supported Rx data rate */ + max_bit_rate *= rx_chains; + WARN_ON(max_bit_rate & ~IEEE80211_HT_MCS_RX_HIGHEST_MASK); + ht_info->mcs.rx_highest = cpu_to_le16(max_bit_rate); + + /* Tx MCS capabilities */ + ht_info->mcs.tx_params = IEEE80211_HT_MCS_TX_DEFINED; + if (tx_chains != rx_chains) { + ht_info->mcs.tx_params |= IEEE80211_HT_MCS_TX_RX_DIFF; + ht_info->mcs.tx_params |= ((tx_chains - 1) << + IEEE80211_HT_MCS_TX_MAX_STREAMS_SHIFT); + } +} +IWL_EXPORT_SYMBOL(iwl_init_ht_hw_capab); diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-utils.h similarity index 73% rename from drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h rename to drivers/net/wireless/intel/iwlwifi/iwl-nvm-utils.h index 34a178a2eb5d4b12655458f49fce88a108650951..ac0a29a1c31f5b0107be352e6450f3cb5eab807f 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h +++ b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-utils.h @@ -58,23 +58,6 @@ struct iwl_nvm_data { struct ieee80211_channel channels[]; }; -/** - * iwl_parse_eeprom_data - parse EEPROM data and return values - * - * @trans: ransport we're parsing for, for debug only - * @cfg: device configuration for parsing and overrides - * @eeprom: the EEPROM data - * @eeprom_size: length of the EEPROM data - * - * This function parses all EEPROM values we need and then - * returns a (newly allocated) struct containing all the - * relevant values for driver use. The struct must be freed - * later with iwl_free_nvm_data(). - */ -struct iwl_nvm_data * -iwl_parse_eeprom_data(struct iwl_trans *trans, const struct iwl_cfg *cfg, - const u8 *eeprom, size_t eeprom_size); - int iwl_init_sband_channels(struct iwl_nvm_data *data, struct ieee80211_supported_band *sband, int n_channels, enum nl80211_band band); diff --git a/drivers/net/wireless/intel/iwlwifi/mei/iwl-mei.h b/drivers/net/wireless/intel/iwlwifi/mei/iwl-mei.h index 1f3c885aeb6548246e62c604d750d66d9b398c64..ee48b86674a6e60139c3a7a16b76f22dce8415b0 100644 --- a/drivers/net/wireless/intel/iwlwifi/mei/iwl-mei.h +++ b/drivers/net/wireless/intel/iwlwifi/mei/iwl-mei.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0-only */ /* - * Copyright (C) 2021-2023 Intel Corporation + * Copyright (C) 2021-2024 Intel Corporation */ #ifndef __iwl_mei_h__ @@ -488,7 +488,7 @@ static inline void iwl_mei_set_nic_info(const u8 *mac_address, const u8 *nvm_add static inline void iwl_mei_set_country_code(u16 mcc) {} -static inline void iwl_mei_set_power_limit(__le16 *power_limit) +static inline void iwl_mei_set_power_limit(const __le16 *power_limit) {} static inline int iwl_mei_register(void *priv, diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/constants.h b/drivers/net/wireless/intel/iwlwifi/mvm/constants.h index 3cbeaddf4358655643b367593dffd1070297b0f5..1abe9e9fd46b7ae9681fb314e52cd449f6aac6bb 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/constants.h +++ b/drivers/net/wireless/intel/iwlwifi/mvm/constants.h @@ -23,7 +23,7 @@ #define IWL_MVM_WOWLAN_PS_RX_DATA_TIMEOUT (10 * USEC_PER_MSEC) #define IWL_MVM_SHORT_PS_TX_DATA_TIMEOUT (2 * 1024) /* defined in TU */ #define IWL_MVM_SHORT_PS_RX_DATA_TIMEOUT (40 * 1024) /* defined in TU */ -#define IWL_MVM_P2P_LOWLATENCY_PS_ENABLE 0 +#define IWL_MVM_P2P_LOWLATENCY_PS_ENABLE 1 #define IWL_MVM_UAPSD_RX_DATA_TIMEOUT (50 * USEC_PER_MSEC) #define IWL_MVM_UAPSD_TX_DATA_TIMEOUT (50 * USEC_PER_MSEC) #define IWL_MVM_UAPSD_QUEUES (IEEE80211_WMM_IE_STA_QOSINFO_AC_VO |\ @@ -100,6 +100,7 @@ #define IWL_MVM_FTM_INITIATOR_ALGO IWL_TOF_ALGO_TYPE_MAX_LIKE #define IWL_MVM_FTM_INITIATOR_DYNACK true #define IWL_MVM_FTM_LMR_FEEDBACK_TERMINATE false +#define IWL_MVM_FTM_TEST_INCORRECT_SAC false #define IWL_MVM_FTM_R2I_MAX_REP 7 #define IWL_MVM_FTM_I2R_MAX_REP 7 #define IWL_MVM_FTM_R2I_MAX_STS 1 @@ -114,7 +115,6 @@ #define IWL_MVM_D3_DEBUG false #define IWL_MVM_USE_TWT true #define IWL_MVM_AMPDU_CONSEC_DROPS_DELBA 20 -#define IWL_MVM_USE_NSSN_SYNC 0 #define IWL_MVM_FTM_INITIATOR_ENABLE_SMOOTH false #define IWL_MVM_FTM_INITIATOR_SMOOTH_ALPHA 40 /* 20016 pSec is 6 meter RTT, meaning 3 meter range */ @@ -124,6 +124,7 @@ #define IWL_MVM_DISABLE_AP_FILS false #define IWL_MVM_6GHZ_PASSIVE_SCAN_TIMEOUT 3000 /* in seconds */ #define IWL_MVM_6GHZ_PASSIVE_SCAN_ASSOC_TIMEOUT 60 /* in seconds */ +#define IWL_MVM_MIN_BEACON_INTERVAL_TU 16 #define IWL_MVM_AUTO_EML_ENABLE true #define IWL_MVM_MISSED_BEACONS_EXIT_ESR_THRESH 7 diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c index 71e6b06481a93b6bf6f6eb14b1b1656510cdbc92..54f4acbbd05bd4d2f7e8a25e748c02c432025de1 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c @@ -595,6 +595,12 @@ static void iwl_mvm_wowlan_gtk_type_iter(struct ieee80211_hw *hw, void *_data) { struct wowlan_key_gtk_type_iter *data = _data; + __le32 *cipher = NULL; + + if (key->keyidx == 4 || key->keyidx == 5) + cipher = &data->kek_kck_cmd->igtk_cipher; + if (key->keyidx == 6 || key->keyidx == 7) + cipher = &data->kek_kck_cmd->bigtk_cipher; switch (key->cipher) { default: @@ -606,10 +612,13 @@ static void iwl_mvm_wowlan_gtk_type_iter(struct ieee80211_hw *hw, return; case WLAN_CIPHER_SUITE_BIP_GMAC_256: case WLAN_CIPHER_SUITE_BIP_GMAC_128: - data->kek_kck_cmd->igtk_cipher = cpu_to_le32(STA_KEY_FLG_GCMP); + if (cipher) + *cipher = cpu_to_le32(STA_KEY_FLG_GCMP); return; case WLAN_CIPHER_SUITE_AES_CMAC: - data->kek_kck_cmd->igtk_cipher = cpu_to_le32(STA_KEY_FLG_CCM); + case WLAN_CIPHER_SUITE_BIP_CMAC_256: + if (cipher) + *cipher = cpu_to_le32(STA_KEY_FLG_CCM); return; case WLAN_CIPHER_SUITE_CCMP: if (!sta) @@ -2341,7 +2350,8 @@ static bool iwl_mvm_setup_connection_keep(struct iwl_mvm *mvm, out: if (iwl_fw_lookup_notif_ver(mvm->fw, LONG_GROUP, - WOWLAN_GET_STATUSES, 0) < 10) { + WOWLAN_GET_STATUSES, + IWL_FW_CMD_VER_UNKNOWN) < 10) { mvmvif->seqno_valid = true; /* +0x10 because the set API expects next-to-use, not last-used */ mvmvif->seqno = status->non_qos_seq_ctr + 0x10; diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs-vif.c b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs-vif.c index 17c97dfbc62aa25ad91d040aaef8d3bcb05e9e4e..25f07e00db42973a309f149be94a65407300d8cc 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs-vif.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs-vif.c @@ -692,6 +692,42 @@ static ssize_t iwl_dbgfs_quota_min_read(struct file *file, return simple_read_from_buffer(user_buf, count, ppos, buf, len); } +static ssize_t iwl_dbgfs_max_tx_op_write(struct ieee80211_vif *vif, char *buf, + size_t count, loff_t *ppos) +{ + struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif); + struct iwl_mvm *mvm = mvmvif->mvm; + u16 value; + int ret; + + ret = kstrtou16(buf, 0, &value); + if (ret) + return ret; + + mutex_lock(&mvm->mutex); + mvmvif->max_tx_op = value; + mutex_unlock(&mvm->mutex); + + return count; +} + +static ssize_t iwl_dbgfs_max_tx_op_read(struct file *file, + char __user *user_buf, + size_t count, loff_t *ppos) +{ + struct ieee80211_vif *vif = file->private_data; + struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif); + struct iwl_mvm *mvm = mvmvif->mvm; + char buf[10]; + int len; + + mutex_lock(&mvm->mutex); + len = scnprintf(buf, sizeof(buf), "%hu\n", mvmvif->max_tx_op); + mutex_unlock(&mvm->mutex); + + return simple_read_from_buffer(user_buf, count, ppos, buf, len); +} + static ssize_t iwl_dbgfs_int_mlo_scan_write(struct ieee80211_vif *vif, char *buf, size_t count, loff_t *ppos) @@ -801,6 +837,7 @@ MVM_DEBUGFS_READ_WRITE_FILE_OPS(uapsd_misbehaving, 20); MVM_DEBUGFS_READ_WRITE_FILE_OPS(rx_phyinfo, 10); MVM_DEBUGFS_READ_WRITE_FILE_OPS(quota_min, 32); MVM_DEBUGFS_READ_FILE_OPS(os_device_timediff); +MVM_DEBUGFS_READ_WRITE_FILE_OPS(max_tx_op, 10); MVM_DEBUGFS_WRITE_FILE_OPS(int_mlo_scan, 32); MVM_DEBUGFS_READ_WRITE_FILE_OPS(esr_disable_reason, 32); @@ -830,6 +867,7 @@ void iwl_mvm_vif_add_debugfs(struct ieee80211_hw *hw, struct ieee80211_vif *vif) MVM_DEBUGFS_ADD_FILE_VIF(rx_phyinfo, mvmvif->dbgfs_dir, 0600); MVM_DEBUGFS_ADD_FILE_VIF(quota_min, mvmvif->dbgfs_dir, 0600); MVM_DEBUGFS_ADD_FILE_VIF(os_device_timediff, mvmvif->dbgfs_dir, 0400); + MVM_DEBUGFS_ADD_FILE_VIF(max_tx_op, mvmvif->dbgfs_dir, 0600); debugfs_create_bool("ftm_unprotected", 0200, mvmvif->dbgfs_dir, &mvmvif->ftm_unprotected); MVM_DEBUGFS_ADD_FILE_VIF(int_mlo_scan, mvmvif->dbgfs_dir, 0200); diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c index 79f4ac8cbc729e9a6caaf393080fe99a4320892e..cb164b733cb30e057c882832b1166e95e98e32ca 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c @@ -754,7 +754,6 @@ static ssize_t iwl_dbgfs_fw_ver_read(struct file *file, char __user *user_buf, struct iwl_mvm *mvm = file->private_data; char *buff, *pos, *endpos; static const size_t bufsz = 1024; - char _fw_name_pre[FW_NAME_PRE_BUFSIZE]; int ret; buff = kmalloc(bufsz, GFP_KERNEL); @@ -764,8 +763,8 @@ static ssize_t iwl_dbgfs_fw_ver_read(struct file *file, char __user *user_buf, pos = buff; endpos = pos + bufsz; - pos += scnprintf(pos, endpos - pos, "FW prefix: %s\n", - iwl_drv_get_fwname_pre(mvm->trans, _fw_name_pre)); + pos += scnprintf(pos, endpos - pos, "FW id: %s\n", + mvm->fwrt.fw->fw_version); pos += scnprintf(pos, endpos - pos, "FW: %s\n", mvm->fwrt.fw->human_readable); pos += scnprintf(pos, endpos - pos, "Device: %s\n", @@ -1396,6 +1395,8 @@ static ssize_t iwl_dbgfs_fw_nmi_write(struct iwl_mvm *mvm, char *buf, if (!iwl_mvm_firmware_running(mvm)) return -EIO; + IWL_ERR(mvm, "Triggering an NMI from debugfs\n"); + if (count == 6 && !strcmp(buf, "nolog\n")) set_bit(IWL_MVM_STATUS_SUPPRESS_ERROR_LOG_ONCE, &mvm->status); @@ -1617,6 +1618,15 @@ static int _iwl_dbgfs_inject_beacon_ie(struct iwl_mvm *mvm, char *bin, int len) &beacon_cmd.tim_size, beacon->data, beacon->len); + if (iwl_fw_lookup_cmd_ver(mvm->fw, + BEACON_TEMPLATE_CMD, 0) >= 14) { + u32 offset = iwl_mvm_find_ie_offset(beacon->data, + WLAN_EID_S1G_TWT, + beacon->len); + + beacon_cmd.btwt_offset = cpu_to_le32(offset); + } + iwl_mvm_mac_ctxt_send_beacon_cmd(mvm, beacon, &beacon_cmd, sizeof(beacon_cmd)); } @@ -2439,6 +2449,9 @@ void iwl_mvm_dbgfs_register(struct iwl_mvm *mvm) debugfs_create_file("mem", 0600, mvm->debugfs_dir, mvm, &iwl_dbgfs_mem_ops); + debugfs_create_bool("rx_ts_ptp", 0600, mvm->debugfs_dir, + &mvm->rx_ts_ptp); + /* * Create a symlink with mac80211. It will be removed when mac80211 * exists (before the opmode exists which removes the target.) diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c index 72a3d71f46f045172ff75324e0ad1bbacc612255..00d447e40cc63c6f250a42cf6f7941966d1dd135 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c @@ -40,6 +40,12 @@ struct iwl_mvm_ftm_pasn_entry { u32 flags; }; +struct iwl_mvm_ftm_iter_data { + u8 *cipher; + u8 *bssid; + u8 *tk; +}; + int iwl_mvm_ftm_add_pasn_sta(struct iwl_mvm *mvm, struct ieee80211_vif *vif, u8 *addr, u32 cipher, u8 *tk, u32 tk_len, u8 *hltk, u32 hltk_len) @@ -431,47 +437,55 @@ iwl_mvm_ftm_put_target_v2(struct iwl_mvm *mvm, return 0; } -#define FTM_PUT_FLAG(flag) (target->initiator_ap_flags |= \ +#define FTM_SET_FLAG(flag) (*flags |= \ cpu_to_le32(IWL_INITIATOR_AP_FLAGS_##flag)) static void -iwl_mvm_ftm_put_target_common(struct iwl_mvm *mvm, - struct cfg80211_pmsr_request_peer *peer, - struct iwl_tof_range_req_ap_entry_v6 *target) +iwl_mvm_ftm_set_target_flags(struct iwl_mvm *mvm, + struct cfg80211_pmsr_request_peer *peer, + __le32 *flags) { - memcpy(target->bssid, peer->addr, ETH_ALEN); - target->burst_period = - cpu_to_le16(peer->ftm.burst_period); - target->samples_per_burst = peer->ftm.ftms_per_burst; - target->num_of_bursts = peer->ftm.num_bursts_exp; - target->ftmr_max_retries = peer->ftm.ftmr_retries; - target->initiator_ap_flags = cpu_to_le32(0); + *flags = cpu_to_le32(0); if (peer->ftm.asap) - FTM_PUT_FLAG(ASAP); + FTM_SET_FLAG(ASAP); if (peer->ftm.request_lci) - FTM_PUT_FLAG(LCI_REQUEST); + FTM_SET_FLAG(LCI_REQUEST); if (peer->ftm.request_civicloc) - FTM_PUT_FLAG(CIVIC_REQUEST); + FTM_SET_FLAG(CIVIC_REQUEST); if (IWL_MVM_FTM_INITIATOR_DYNACK) - FTM_PUT_FLAG(DYN_ACK); + FTM_SET_FLAG(DYN_ACK); if (IWL_MVM_FTM_INITIATOR_ALGO == IWL_TOF_ALGO_TYPE_LINEAR_REG) - FTM_PUT_FLAG(ALGO_LR); + FTM_SET_FLAG(ALGO_LR); else if (IWL_MVM_FTM_INITIATOR_ALGO == IWL_TOF_ALGO_TYPE_FFT) - FTM_PUT_FLAG(ALGO_FFT); + FTM_SET_FLAG(ALGO_FFT); if (peer->ftm.trigger_based) - FTM_PUT_FLAG(TB); + FTM_SET_FLAG(TB); else if (peer->ftm.non_trigger_based) - FTM_PUT_FLAG(NON_TB); + FTM_SET_FLAG(NON_TB); if ((peer->ftm.trigger_based || peer->ftm.non_trigger_based) && peer->ftm.lmr_feedback) - FTM_PUT_FLAG(LMR_FEEDBACK); + FTM_SET_FLAG(LMR_FEEDBACK); +} + +static void +iwl_mvm_ftm_put_target_common(struct iwl_mvm *mvm, + struct cfg80211_pmsr_request_peer *peer, + struct iwl_tof_range_req_ap_entry_v6 *target) +{ + memcpy(target->bssid, peer->addr, ETH_ALEN); + target->burst_period = + cpu_to_le16(peer->ftm.burst_period); + target->samples_per_burst = peer->ftm.ftms_per_burst; + target->num_of_bursts = peer->ftm.num_bursts_exp; + target->ftmr_max_retries = peer->ftm.ftmr_retries; + iwl_mvm_ftm_set_target_flags(mvm, peer, &target->initiator_ap_flags); } static int @@ -514,21 +528,10 @@ iwl_mvm_ftm_put_target_v4(struct iwl_mvm *mvm, return 0; } -static int -iwl_mvm_ftm_put_target(struct iwl_mvm *mvm, struct ieee80211_vif *vif, - struct cfg80211_pmsr_request_peer *peer, - struct iwl_tof_range_req_ap_entry_v6 *target) +static int iwl_mvm_ftm_set_sta(struct iwl_mvm *mvm, struct ieee80211_vif *vif, + struct cfg80211_pmsr_request_peer *peer, + u8 *sta_id, __le32 *flags) { - int ret; - - ret = iwl_mvm_ftm_target_chandef_v2(mvm, peer, &target->channel_num, - &target->format_bw, - &target->ctrl_ch_position); - if (ret) - return ret; - - iwl_mvm_ftm_put_target_common(mvm, peer, target); - if (vif->cfg.assoc) { struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif); struct ieee80211_sta *sta; @@ -540,8 +543,8 @@ iwl_mvm_ftm_put_target(struct iwl_mvm *mvm, struct ieee80211_vif *vif, if (memcmp(peer->addr, link_conf->bssid, ETH_ALEN)) continue; - target->sta_id = mvmvif->link[link_id]->ap_sta_id; - sta = rcu_dereference(mvm->fw_id_to_mac_id[target->sta_id]); + *sta_id = mvmvif->link[link_id]->ap_sta_id; + sta = rcu_dereference(mvm->fw_id_to_mac_id[*sta_id]); if (WARN_ON_ONCE(IS_ERR_OR_NULL(sta))) { rcu_read_unlock(); return PTR_ERR_OR_ZERO(sta); @@ -549,23 +552,42 @@ iwl_mvm_ftm_put_target(struct iwl_mvm *mvm, struct ieee80211_vif *vif, if (sta->mfp && (peer->ftm.trigger_based || peer->ftm.non_trigger_based)) - FTM_PUT_FLAG(PMF); + FTM_SET_FLAG(PMF); break; } rcu_read_unlock(); #ifdef CONFIG_IWLWIFI_DEBUGFS if (mvmvif->ftm_unprotected) { - target->sta_id = IWL_MVM_INVALID_STA; - target->initiator_ap_flags &= - ~cpu_to_le32(IWL_INITIATOR_AP_FLAGS_PMF); + *sta_id = IWL_MVM_INVALID_STA; + *flags &= ~cpu_to_le32(IWL_INITIATOR_AP_FLAGS_PMF); } - #endif } else { - target->sta_id = IWL_MVM_INVALID_STA; + *sta_id = IWL_MVM_INVALID_STA; } + return 0; +} + +static int +iwl_mvm_ftm_put_target(struct iwl_mvm *mvm, struct ieee80211_vif *vif, + struct cfg80211_pmsr_request_peer *peer, + struct iwl_tof_range_req_ap_entry_v6 *target) +{ + int ret; + + ret = iwl_mvm_ftm_target_chandef_v2(mvm, peer, &target->channel_num, + &target->format_bw, + &target->ctrl_ch_position); + if (ret) + return ret; + + iwl_mvm_ftm_put_target_common(mvm, peer, target); + + iwl_mvm_ftm_set_sta(mvm, vif, peer, &target->sta_id, + &target->initiator_ap_flags); + /* * TODO: Beacon interval is currently unknown, so use the common value * of 100 TUs. @@ -703,7 +725,7 @@ static void iter(struct ieee80211_hw *hw, struct ieee80211_key_conf *key, void *data) { - struct iwl_tof_range_req_ap_entry_v6 *target = data; + struct iwl_mvm_ftm_iter_data *target = data; if (!sta || memcmp(sta->addr, target->bssid, ETH_ALEN)) return; @@ -714,16 +736,16 @@ static void iter(struct ieee80211_hw *hw, return; memcpy(target->tk, key->key, key->keylen); - target->cipher = iwl_mvm_cipher_to_location_cipher(key->cipher); - WARN_ON(target->cipher == IWL_LOCATION_CIPHER_INVALID); + *target->cipher = iwl_mvm_cipher_to_location_cipher(key->cipher); + WARN_ON(*target->cipher == IWL_LOCATION_CIPHER_INVALID); } static void iwl_mvm_ftm_set_secured_ranging(struct iwl_mvm *mvm, struct ieee80211_vif *vif, - struct iwl_tof_range_req_ap_entry_v7 *target) + u8 *bssid, u8 *cipher, u8 *hltk, u8 *tk, + u8 *rx_pn, u8 *tx_pn, __le32 *flags) { struct iwl_mvm_ftm_pasn_entry *entry; - u32 flags = le32_to_cpu(target->initiator_ap_flags); #ifdef CONFIG_IWLWIFI_DEBUGFS struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif); @@ -731,35 +753,39 @@ iwl_mvm_ftm_set_secured_ranging(struct iwl_mvm *mvm, struct ieee80211_vif *vif, return; #endif - if (!(flags & (IWL_INITIATOR_AP_FLAGS_NON_TB | + if (!(le32_to_cpu(*flags) & (IWL_INITIATOR_AP_FLAGS_NON_TB | IWL_INITIATOR_AP_FLAGS_TB))) return; lockdep_assert_held(&mvm->mutex); list_for_each_entry(entry, &mvm->ftm_initiator.pasn_list, list) { - if (memcmp(entry->addr, target->bssid, sizeof(entry->addr))) + if (memcmp(entry->addr, bssid, sizeof(entry->addr))) continue; - target->cipher = entry->cipher; + *cipher = entry->cipher; if (entry->flags & IWL_MVM_PASN_FLAG_HAS_HLTK) - memcpy(target->hltk, entry->hltk, sizeof(target->hltk)); + memcpy(hltk, entry->hltk, sizeof(entry->hltk)); else - memset(target->hltk, 0, sizeof(target->hltk)); + memset(hltk, 0, sizeof(entry->hltk)); if (vif->cfg.assoc && - !memcmp(vif->bss_conf.bssid, target->bssid, - sizeof(target->bssid))) - ieee80211_iter_keys(mvm->hw, vif, iter, target); - else - memcpy(target->tk, entry->tk, sizeof(target->tk)); + !memcmp(vif->bss_conf.bssid, bssid, ETH_ALEN)) { + struct iwl_mvm_ftm_iter_data target; + + target.cipher = cipher; + target.bssid = bssid; + target.tk = tk; + ieee80211_iter_keys(mvm->hw, vif, iter, &target); + } else { + memcpy(tk, entry->tk, sizeof(entry->tk)); + } - memcpy(target->rx_pn, entry->rx_pn, sizeof(target->rx_pn)); - memcpy(target->tx_pn, entry->tx_pn, sizeof(target->tx_pn)); + memcpy(rx_pn, entry->rx_pn, sizeof(entry->rx_pn)); + memcpy(tx_pn, entry->tx_pn, sizeof(entry->tx_pn)); - target->initiator_ap_flags |= - cpu_to_le32(IWL_INITIATOR_AP_FLAGS_SECURED); + FTM_SET_FLAG(SECURED); return; } } @@ -773,7 +799,11 @@ iwl_mvm_ftm_put_target_v7(struct iwl_mvm *mvm, struct ieee80211_vif *vif, if (err) return err; - iwl_mvm_ftm_set_secured_ranging(mvm, vif, target); + iwl_mvm_ftm_set_secured_ranging(mvm, vif, target->bssid, + &target->cipher, target->hltk, + target->tk, target->rx_pn, + target->tx_pn, + &target->initiator_ap_flags); return err; } @@ -920,6 +950,105 @@ static int iwl_mvm_ftm_start_v13(struct iwl_mvm *mvm, return iwl_mvm_ftm_send_cmd(mvm, &hcmd); } +static int +iwl_mvm_ftm_put_target_v10(struct iwl_mvm *mvm, struct ieee80211_vif *vif, + struct cfg80211_pmsr_request_peer *peer, + struct iwl_tof_range_req_ap_entry_v10 *target) +{ + u32 i2r_max_sts, flags; + int ret; + + ret = iwl_mvm_ftm_target_chandef_v2(mvm, peer, &target->channel_num, + &target->format_bw, + &target->ctrl_ch_position); + if (ret) + return ret; + + memcpy(target->bssid, peer->addr, ETH_ALEN); + target->burst_period = + cpu_to_le16(peer->ftm.burst_period); + target->samples_per_burst = peer->ftm.ftms_per_burst; + target->num_of_bursts = peer->ftm.num_bursts_exp; + iwl_mvm_ftm_set_target_flags(mvm, peer, &target->initiator_ap_flags); + iwl_mvm_ftm_set_sta(mvm, vif, peer, &target->sta_id, + &target->initiator_ap_flags); + iwl_mvm_ftm_set_secured_ranging(mvm, vif, target->bssid, + &target->cipher, target->hltk, + target->tk, target->rx_pn, + target->tx_pn, + &target->initiator_ap_flags); + + i2r_max_sts = IWL_MVM_FTM_I2R_MAX_STS > 1 ? 1 : + IWL_MVM_FTM_I2R_MAX_STS; + + target->r2i_ndp_params = IWL_MVM_FTM_R2I_MAX_REP | + (IWL_MVM_FTM_R2I_MAX_STS << IWL_LOCATION_MAX_STS_POS) | + (IWL_MVM_FTM_R2I_MAX_TOTAL_LTF << IWL_LOCATION_TOTAL_LTF_POS); + target->i2r_ndp_params = IWL_MVM_FTM_I2R_MAX_REP | + (i2r_max_sts << IWL_LOCATION_MAX_STS_POS) | + (IWL_MVM_FTM_I2R_MAX_TOTAL_LTF << IWL_LOCATION_TOTAL_LTF_POS); + + if (peer->ftm.non_trigger_based) { + target->min_time_between_msr = + cpu_to_le16(IWL_MVM_FTM_NON_TB_MIN_TIME_BETWEEN_MSR); + target->burst_period = + cpu_to_le16(IWL_MVM_FTM_NON_TB_MAX_TIME_BETWEEN_MSR); + } else { + target->min_time_between_msr = cpu_to_le16(0); + } + + target->band = + iwl_mvm_phy_band_from_nl80211(peer->chandef.chan->band); + + /* + * TODO: Beacon interval is currently unknown, so use the common value + * of 100 TUs. + */ + target->beacon_interval = cpu_to_le16(100); + + /* + * If secure LTF is turned off, replace the flag with PMF only + */ + flags = le32_to_cpu(target->initiator_ap_flags); + if (flags & IWL_INITIATOR_AP_FLAGS_SECURED) { + if (!IWL_MVM_FTM_INITIATOR_SECURE_LTF) + flags &= ~IWL_INITIATOR_AP_FLAGS_SECURED; + + flags |= IWL_INITIATOR_AP_FLAGS_PMF; + target->initiator_ap_flags = cpu_to_le32(flags); + } + + return 0; +} + +static int iwl_mvm_ftm_start_v14(struct iwl_mvm *mvm, + struct ieee80211_vif *vif, + struct cfg80211_pmsr_request *req) +{ + struct iwl_tof_range_req_cmd_v14 cmd; + struct iwl_host_cmd hcmd = { + .id = WIDE_ID(LOCATION_GROUP, TOF_RANGE_REQ_CMD), + .dataflags[0] = IWL_HCMD_DFL_DUP, + .data[0] = &cmd, + .len[0] = sizeof(cmd), + }; + u8 i; + int err; + + iwl_mvm_ftm_cmd_common(mvm, vif, (void *)&cmd, req); + + for (i = 0; i < cmd.num_of_ap; i++) { + struct cfg80211_pmsr_request_peer *peer = &req->peers[i]; + struct iwl_tof_range_req_ap_entry_v10 *target = &cmd.ap[i]; + + err = iwl_mvm_ftm_put_target_v10(mvm, vif, peer, target); + if (err) + return err; + } + + return iwl_mvm_ftm_send_cmd(mvm, &hcmd); +} + int iwl_mvm_ftm_start(struct iwl_mvm *mvm, struct ieee80211_vif *vif, struct cfg80211_pmsr_request *req) { @@ -938,6 +1067,9 @@ int iwl_mvm_ftm_start(struct iwl_mvm *mvm, struct ieee80211_vif *vif, IWL_FW_CMD_VER_UNKNOWN); switch (cmd_ver) { + case 14: + err = iwl_mvm_ftm_start_v14(mvm, vif, req); + break; case 13: err = iwl_mvm_ftm_start_v13(mvm, vif, req); break; diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c index 8e760300a1ab474af738eca2dc334588de4cee95..e4caa362f597b0053b910400abf68111d1ac1426 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause /* * Copyright (C) 2015-2017 Intel Deutschland GmbH - * Copyright (C) 2018-2023 Intel Corporation + * Copyright (C) 2018-2024 Intel Corporation */ #include #include @@ -88,7 +88,7 @@ static int iwl_mvm_ftm_responder_set_bw_v2(struct cfg80211_chan_def *chandef, static void iwl_mvm_ftm_responder_set_ndp(struct iwl_mvm *mvm, - struct iwl_tof_responder_config_cmd_v9 *cmd) + struct iwl_tof_responder_config_cmd *cmd) { /* Up to 2 R2I STS are allowed on the responder */ u32 r2i_max_sts = IWL_MVM_FTM_R2I_MAX_STS < 2 ? @@ -117,7 +117,7 @@ iwl_mvm_ftm_responder_cmd(struct iwl_mvm *mvm, * field interpretation is different), so the same struct can be use * for all cases. */ - struct iwl_tof_responder_config_cmd_v9 cmd = { + struct iwl_tof_responder_config_cmd cmd = { .channel_num = chandef->chan->hw_value, .cmd_valid_fields = cpu_to_le32(IWL_TOF_RESPONDER_CMD_VALID_CHAN_INFO | @@ -131,8 +131,13 @@ iwl_mvm_ftm_responder_cmd(struct iwl_mvm *mvm, lockdep_assert_held(&mvm->mutex); + if (cmd_ver == 10) { + cmd.band = + iwl_mvm_phy_band_from_nl80211(chandef->chan->band); + } + /* Use a default of bss_color=1 for now */ - if (cmd_ver == 9) { + if (cmd_ver >= 9) { cmd.cmd_valid_fields |= cpu_to_le32(IWL_TOF_RESPONDER_CMD_VALID_BSS_COLOR | IWL_TOF_RESPONDER_CMD_VALID_MIN_MAX_TIME_BETWEEN_MSR); @@ -148,7 +153,7 @@ iwl_mvm_ftm_responder_cmd(struct iwl_mvm *mvm, } if (cmd_ver >= 8) - iwl_mvm_ftm_responder_set_ndp(mvm, &cmd); + iwl_mvm_ftm_responder_set_ndp(mvm, (void *)&cmd); if (cmd_ver >= 7) err = iwl_mvm_ftm_responder_set_bw_v2(chandef, &cmd.format_bw, diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c index e7f5978ef2d71950e2d8bf9a006c054fe910a3b9..b8a7850325134d3aabd5c3697aa61377ee2bf1c4 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c @@ -94,20 +94,10 @@ void iwl_mvm_mfu_assert_dump_notif(struct iwl_mvm *mvm, { struct iwl_rx_packet *pkt = rxb_addr(rxb); struct iwl_mfu_assert_dump_notif *mfu_dump_notif = (void *)pkt->data; - __le32 *dump_data = mfu_dump_notif->data; - int n_words = le32_to_cpu(mfu_dump_notif->data_size) / sizeof(__le32); - int i; if (mfu_dump_notif->index_num == 0) IWL_INFO(mvm, "MFUART assert id 0x%x occurred\n", le32_to_cpu(mfu_dump_notif->assert_id)); - - for (i = 0; i < n_words; i++) - IWL_DEBUG_INFO(mvm, - "MFUART assert dump, dword %u: 0x%08x\n", - le16_to_cpu(mfu_dump_notif->index_num) * - n_words + i, - le32_to_cpu(dump_data[i])); } static bool iwl_alive_fn(struct iwl_notif_wait_data *notif_wait, @@ -895,8 +885,8 @@ int iwl_mvm_sar_select_profile(struct iwl_mvm *mvm, int prof_a, int prof_b) int ret; u16 len = 0; u32 n_subbands; - u8 cmd_ver = iwl_fw_lookup_cmd_ver(mvm->fw, cmd_id, - IWL_FW_CMD_VER_UNKNOWN); + u8 cmd_ver = iwl_fw_lookup_cmd_ver(mvm->fw, cmd_id, 3); + if (cmd_ver >= 7) { len = sizeof(cmd.v7); n_subbands = IWL_NUM_SUB_BANDS_V2; @@ -1499,8 +1489,6 @@ int iwl_mvm_up(struct iwl_mvm *mvm) for (i = 0; i < IWL_MVM_FW_MAX_LINK_ID + 1; i++) RCU_INIT_POINTER(mvm->link_id_to_link_conf[i], NULL); - memset(&mvm->fw_link_ids_map, 0, sizeof(mvm->fw_link_ids_map)); - mvm->tdls_cs.peer.sta_id = IWL_MVM_INVALID_STA; /* reset quota debouncing buffer - 0xff will yield invalid data */ diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/link.c b/drivers/net/wireless/intel/iwlwifi/mvm/link.c index 6ec9a8e21a34ee024f2475b19cc91f2f191ffa05..02a475a1f1b8a535f48ae0c7d06661e64ae34b6e 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/link.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/link.c @@ -50,26 +50,15 @@ static void iwl_mvm_print_esr_state(struct iwl_mvm *mvm, u32 mask) static u32 iwl_mvm_get_free_fw_link_id(struct iwl_mvm *mvm, struct iwl_mvm_vif *mvm_vif) { - u32 link_id; + u32 i; lockdep_assert_held(&mvm->mutex); - link_id = ffz(mvm->fw_link_ids_map); + for (i = 0; i < ARRAY_SIZE(mvm->link_id_to_link_conf); i++) + if (!rcu_access_pointer(mvm->link_id_to_link_conf[i])) + return i; - /* this case can happen if there're deactivated but not removed links */ - if (link_id > IWL_MVM_FW_MAX_LINK_ID) - return IWL_MVM_FW_LINK_ID_INVALID; - - mvm->fw_link_ids_map |= BIT(link_id); - return link_id; -} - -static void iwl_mvm_release_fw_link_id(struct iwl_mvm *mvm, u32 link_id) -{ - lockdep_assert_held(&mvm->mutex); - - if (!WARN_ON(link_id > IWL_MVM_FW_MAX_LINK_ID)) - mvm->fw_link_ids_map &= ~BIT(link_id); + return IWL_MVM_FW_LINK_ID_INVALID; } static int iwl_mvm_link_cmd_send(struct iwl_mvm *mvm, @@ -380,7 +369,6 @@ int iwl_mvm_unset_link_mapping(struct iwl_mvm *mvm, struct ieee80211_vif *vif, RCU_INIT_POINTER(mvm->link_id_to_link_conf[link_info->fw_link_id], NULL); - iwl_mvm_release_fw_link_id(mvm, link_info->fw_link_id); return 0; } @@ -504,17 +492,27 @@ iwl_mvm_get_puncturing_factor(const struct ieee80211_bss_conf *link_conf) static unsigned int iwl_mvm_get_chan_load(struct ieee80211_bss_conf *link_conf) { + struct ieee80211_vif *vif = link_conf->vif; struct iwl_mvm_vif_link_info *mvm_link = iwl_mvm_vif_from_mac80211(link_conf->vif)->link[link_conf->link_id]; const struct element *bss_load_elem; const struct ieee80211_bss_load_elem *bss_load; enum nl80211_band band = link_conf->chanreq.oper.chan->band; + const struct cfg80211_bss_ies *ies; unsigned int chan_load; u32 chan_load_by_us; rcu_read_lock(); - bss_load_elem = ieee80211_bss_get_elem(link_conf->bss, - WLAN_EID_QBSS_LOAD); + if (ieee80211_vif_link_active(vif, link_conf->link_id)) + ies = rcu_dereference(link_conf->bss->beacon_ies); + else + ies = rcu_dereference(link_conf->bss->ies); + + if (ies) + bss_load_elem = cfg80211_find_elem(WLAN_EID_QBSS_LOAD, + ies->data, ies->len); + else + bss_load_elem = NULL; /* If there isn't BSS Load element, take the defaults */ if (!bss_load_elem || @@ -978,6 +976,9 @@ void iwl_mvm_exit_esr(struct iwl_mvm *mvm, struct ieee80211_vif *vif, lockdep_assert_held(&mvm->mutex); + if (!IWL_MVM_AUTO_EML_ENABLE) + return; + /* Nothing to do */ if (!mvmvif->esr_active) return; @@ -1025,6 +1026,9 @@ void iwl_mvm_block_esr(struct iwl_mvm *mvm, struct ieee80211_vif *vif, lockdep_assert_held(&mvm->mutex); + if (!IWL_MVM_AUTO_EML_ENABLE) + return; + /* This should be called only with disable reasons */ if (WARN_ON(!(reason & IWL_MVM_BLOCK_ESR_REASONS))) return; @@ -1082,6 +1086,13 @@ static void iwl_mvm_esr_unblocked(struct iwl_mvm *mvm, IWL_DEBUG_INFO(mvm, "EMLSR is unblocked\n"); + /* We exited due to an EXIT reason, so MLO scan was scheduled already */ + if (mvmvif->last_esr_exit.reason && + !(mvmvif->last_esr_exit.reason & IWL_MVM_BLOCK_ESR_REASONS)) { + IWL_DEBUG_INFO(mvm, "Wait for MLO scan\n"); + return; + } + /* * If EMLSR was blocked for more than 30 seconds, or the last link * selection decided to not enter EMLSR, trigger a new scan. @@ -1111,6 +1122,9 @@ void iwl_mvm_unblock_esr(struct iwl_mvm *mvm, struct ieee80211_vif *vif, lockdep_assert_held(&mvm->mutex); + if (!IWL_MVM_AUTO_EML_ENABLE) + return; + /* This should be called only with disable reasons */ if (WARN_ON(!(reason & IWL_MVM_BLOCK_ESR_REASONS))) return; diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c b/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c index 5a06f887769a6ad2de8f73c59afb1b75aa37aa1f..ef7ffeaf6ff21707b8dfee3808474f36b90e611a 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c @@ -873,7 +873,7 @@ void iwl_mvm_mac_ctxt_set_tim(struct iwl_mvm *mvm, } } -static u32 iwl_mvm_find_ie_offset(u8 *beacon, u8 eid, u32 frame_size) +u32 iwl_mvm_find_ie_offset(u8 *beacon, u8 eid, u32 frame_size) { struct ieee80211_mgmt *mgmt = (void *)beacon; const u8 *ie; @@ -1010,12 +1010,13 @@ static void iwl_mvm_mac_ctxt_set_tx(struct iwl_mvm *mvm, tx->tx_flags = cpu_to_le32(tx_flags); if (!fw_has_capa(&mvm->fw->ucode_capa, - IWL_UCODE_TLV_CAPA_BEACON_ANT_SELECTION)) + IWL_UCODE_TLV_CAPA_BEACON_ANT_SELECTION)) { iwl_mvm_toggle_tx_ant(mvm, &mvm->mgmt_last_antenna_idx); - tx->rate_n_flags = - cpu_to_le32(BIT(mvm->mgmt_last_antenna_idx) << - RATE_MCS_ANT_POS); + tx->rate_n_flags = + cpu_to_le32(BIT(mvm->mgmt_last_antenna_idx) << + RATE_MCS_ANT_POS); + } rate = iwl_mvm_mac_ctxt_get_beacon_rate(mvm, info, vif); diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c index 486a6b8f3c97f7540e95e68fa7dfc0fee8fb2af9..88bc0baabf7e2d9483e30ce25f848e7daa77d2fe 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c @@ -22,7 +22,7 @@ #include "mvm.h" #include "sta.h" #include "time-event.h" -#include "iwl-eeprom-parse.h" +#include "iwl-nvm-utils.h" #include "iwl-phy-db.h" #include "testmode.h" #include "fw/error-dump.h" @@ -1128,6 +1128,39 @@ static void iwl_mvm_cleanup_iterator(void *data, u8 *mac, RCU_INIT_POINTER(mvmvif->deflink.probe_resp_data, NULL); } +static void iwl_mvm_cleanup_sta_iterator(void *data, struct ieee80211_sta *sta) +{ + struct iwl_mvm *mvm = data; + struct iwl_mvm_sta *mvm_sta; + struct ieee80211_vif *vif; + int link_id; + + mvm_sta = iwl_mvm_sta_from_mac80211(sta); + vif = mvm_sta->vif; + + if (!sta->valid_links) + return; + + for (link_id = 0; link_id < ARRAY_SIZE((sta)->link); link_id++) { + struct iwl_mvm_link_sta *mvm_link_sta; + + mvm_link_sta = + rcu_dereference_check(mvm_sta->link[link_id], + lockdep_is_held(&mvm->mutex)); + if (mvm_link_sta && !(vif->active_links & BIT(link_id))) { + /* + * We have a link STA but the link is inactive in + * mac80211. This will happen if we failed to + * deactivate the link but mac80211 roll back the + * deactivation of the link. + * Delete the stale data to avoid issues later on. + */ + iwl_mvm_mld_free_sta_link(mvm, mvm_sta, mvm_link_sta, + link_id, false); + } + } +} + static void iwl_mvm_restart_cleanup(struct iwl_mvm *mvm) { iwl_mvm_stop_device(mvm); @@ -1150,6 +1183,10 @@ static void iwl_mvm_restart_cleanup(struct iwl_mvm *mvm) */ ieee80211_iterate_interfaces(mvm->hw, 0, iwl_mvm_cleanup_iterator, mvm); + /* cleanup stations as links may be gone after restart */ + ieee80211_iterate_stations_atomic(mvm->hw, + iwl_mvm_cleanup_sta_iterator, mvm); + mvm->p2p_device_vif = NULL; iwl_mvm_reset_phy_ctxts(mvm); @@ -1258,7 +1295,7 @@ static void iwl_mvm_restart_complete(struct iwl_mvm *mvm) { int ret; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); clear_bit(IWL_MVM_STATUS_IN_HW_RESTART, &mvm->status); @@ -1274,8 +1311,6 @@ static void iwl_mvm_restart_complete(struct iwl_mvm *mvm) * of packets the FW sent out, so we must reconnect. */ iwl_mvm_teardown_tdls_peers(mvm); - - mutex_unlock(&mvm->mutex); } void iwl_mvm_mac_reconfig_complete(struct ieee80211_hw *hw, @@ -1621,9 +1656,8 @@ static void iwl_mvm_prevent_esr_done_wk(struct wiphy *wiphy, struct iwl_mvm *mvm = mvmvif->mvm; struct ieee80211_vif *vif = iwl_mvm_get_bss_vif(mvm); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_unblock_esr(mvm, vif, IWL_MVM_ESR_BLOCKED_PREVENTION); - mutex_unlock(&mvm->mutex); } static void iwl_mvm_mlo_int_scan_wk(struct wiphy *wiphy, struct wiphy_work *wk) @@ -1633,11 +1667,8 @@ static void iwl_mvm_mlo_int_scan_wk(struct wiphy *wiphy, struct wiphy_work *wk) struct ieee80211_vif *vif = container_of((void *)mvmvif, struct ieee80211_vif, drv_priv); - mutex_lock(&mvmvif->mvm->mutex); - + guard(mvm)(mvmvif->mvm); iwl_mvm_int_mlo_scan(mvmvif->mvm, vif); - - mutex_unlock(&mvmvif->mvm->mutex); } static void iwl_mvm_unblock_esr_tpt(struct wiphy *wiphy, struct wiphy_work *wk) @@ -1647,9 +1678,8 @@ static void iwl_mvm_unblock_esr_tpt(struct wiphy *wiphy, struct wiphy_work *wk) struct iwl_mvm *mvm = mvmvif->mvm; struct ieee80211_vif *vif = iwl_mvm_get_bss_vif(mvm); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_unblock_esr(mvm, vif, IWL_MVM_ESR_BLOCKED_TPT); - mutex_unlock(&mvm->mutex); } void iwl_mvm_mac_init_mvmvif(struct iwl_mvm *mvm, struct iwl_mvm_vif *mvmvif) @@ -2039,7 +2069,7 @@ void iwl_mvm_configure_filter(struct ieee80211_hw *hw, struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); struct iwl_mcast_filter_cmd *cmd = (void *)(unsigned long)multicast; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); /* replace previous configuration */ kfree(mvm->mcast_filter_cmd); @@ -2056,7 +2086,6 @@ void iwl_mvm_configure_filter(struct ieee80211_hw *hw, iwl_mvm_recalc_multicast(mvm); out: - mutex_unlock(&mvm->mutex); *total_flags = 0; } @@ -2076,9 +2105,8 @@ static void iwl_mvm_config_iface_filter(struct ieee80211_hw *hw, !vif->p2p) return; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_mac_ctxt_changed(mvm, vif, false, NULL); - mutex_unlock(&mvm->mutex); } int iwl_mvm_update_mu_groups(struct iwl_mvm *mvm, struct ieee80211_vif *vif) @@ -2740,6 +2768,13 @@ iwl_mvm_bss_info_changed_station_common(struct iwl_mvm *mvm, if (changes & BSS_CHANGED_BANDWIDTH) iwl_mvm_update_link_smps(vif, link_conf); + + if (changes & BSS_CHANGED_TPE) { + IWL_DEBUG_CALIB(mvm, "Changing TPE\n"); + iwl_mvm_send_ap_tx_power_constraint_cmd(mvm, vif, + link_conf, + false); + } } static void iwl_mvm_bss_info_changed_station(struct iwl_mvm *mvm, @@ -3123,7 +3158,7 @@ static void iwl_mvm_stop_ap_ibss(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_stop_ap_ibss_common(mvm, vif); @@ -3153,8 +3188,6 @@ static void iwl_mvm_stop_ap_ibss(struct ieee80211_hw *hw, iwl_mvm_power_update_mac(mvm); iwl_mvm_mac_ctxt_remove(mvm, vif); - - mutex_unlock(&mvm->mutex); } static void iwl_mvm_stop_ap(struct ieee80211_hw *hw, @@ -3209,7 +3242,7 @@ static void iwl_mvm_bss_info_changed(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (changes & BSS_CHANGED_IDLE && !vif->cfg.idle) iwl_mvm_scan_stop(mvm, IWL_MVM_SCAN_SCHED, true); @@ -3236,25 +3269,19 @@ static void iwl_mvm_bss_info_changed(struct ieee80211_hw *hw, bss_conf->txpower); iwl_mvm_set_tx_power(mvm, vif, bss_conf->txpower); } - - mutex_unlock(&mvm->mutex); } int iwl_mvm_mac_hw_scan(struct ieee80211_hw *hw, struct ieee80211_vif *vif, struct ieee80211_scan_request *hw_req) { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - int ret; if (hw_req->req.n_channels == 0 || hw_req->req.n_channels > mvm->fw->ucode_capa.n_scan_channels) return -EINVAL; - mutex_lock(&mvm->mutex); - ret = iwl_mvm_reg_scan_start(mvm, vif, &hw_req->req, &hw_req->ies); - mutex_unlock(&mvm->mutex); - - return ret; + guard(mvm)(mvm); + return iwl_mvm_reg_scan_start(mvm, vif, &hw_req->req, &hw_req->ies); } void iwl_mvm_mac_cancel_hw_scan(struct ieee80211_hw *hw, @@ -3262,7 +3289,7 @@ void iwl_mvm_mac_cancel_hw_scan(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); /* Due to a race condition, it's possible that mac80211 asks * us to stop a hw_scan when it's already stopped. This can @@ -3273,8 +3300,6 @@ void iwl_mvm_mac_cancel_hw_scan(struct ieee80211_hw *hw, */ if (mvm->scan_status & IWL_MVM_SCAN_REGULAR) iwl_mvm_scan_stop(mvm, IWL_MVM_SCAN_REGULAR, true); - - mutex_unlock(&mvm->mutex); } void @@ -3443,7 +3468,7 @@ void iwl_mvm_sta_pre_rcu_remove(struct ieee80211_hw *hw, * Since there's mvm->mutex here, no need to have RCU lock for * mvm_sta->link access. */ - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); for (link_id = 0; link_id < ARRAY_SIZE(mvm_sta->link); link_id++) { struct iwl_mvm_link_sta *link_sta; u32 sta_id; @@ -3460,7 +3485,6 @@ void iwl_mvm_sta_pre_rcu_remove(struct ieee80211_hw *hw, RCU_INIT_POINTER(mvm->fw_id_to_link_sta[sta_id], NULL); } } - mutex_unlock(&mvm->mutex); } static void iwl_mvm_check_uapsd(struct iwl_mvm *mvm, struct ieee80211_vif *vif, @@ -3736,8 +3760,6 @@ static void iwl_mvm_rs_rate_init_all_links(struct iwl_mvm *mvm, } } -#define IWL_MVM_MIN_BEACON_INTERVAL_TU 16 - static bool iwl_mvm_vif_conf_from_sta(struct iwl_mvm *mvm, struct ieee80211_vif *vif, struct ieee80211_sta *sta) @@ -4207,12 +4229,8 @@ static int iwl_mvm_mac_conf_tx(struct ieee80211_hw *hw, * The exception is P2P_DEVICE interface which needs immediate update. */ if (vif->type == NL80211_IFTYPE_P2P_DEVICE) { - int ret; - - mutex_lock(&mvm->mutex); - ret = iwl_mvm_mac_ctxt_changed(mvm, vif, false, NULL); - mutex_unlock(&mvm->mutex); - return ret; + guard(mvm)(mvm); + return iwl_mvm_mac_ctxt_changed(mvm, vif, false, NULL); } return 0; } @@ -4223,9 +4241,8 @@ void iwl_mvm_mac_mgd_prepare_tx(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_protect_assoc(mvm, vif, info->duration, info->link_id); - mutex_unlock(&mvm->mutex); } void iwl_mvm_mac_mgd_complete_tx(struct ieee80211_hw *hw, @@ -4238,9 +4255,8 @@ void iwl_mvm_mac_mgd_complete_tx(struct ieee80211_hw *hw, if (info->success) return; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_stop_session_protection(mvm, vif); - mutex_unlock(&mvm->mutex); } int iwl_mvm_mac_sched_scan_start(struct ieee80211_hw *hw, @@ -4250,20 +4266,12 @@ int iwl_mvm_mac_sched_scan_start(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - int ret; - - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); - if (!vif->cfg.idle) { - ret = -EBUSY; - goto out; - } - - ret = iwl_mvm_sched_scan_start(mvm, vif, req, ies, IWL_MVM_SCAN_SCHED); + if (!vif->cfg.idle) + return -EBUSY; -out: - mutex_unlock(&mvm->mutex); - return ret; + return iwl_mvm_sched_scan_start(mvm, vif, req, ies, IWL_MVM_SCAN_SCHED); } int iwl_mvm_mac_sched_scan_stop(struct ieee80211_hw *hw, @@ -4541,13 +4549,9 @@ int iwl_mvm_mac_set_key(struct ieee80211_hw *hw, enum set_key_cmd cmd, struct ieee80211_key_conf *key) { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - int ret; - - mutex_lock(&mvm->mutex); - ret = __iwl_mvm_mac_set_key(hw, cmd, vif, sta, key); - mutex_unlock(&mvm->mutex); - return ret; + guard(mvm)(mvm); + return __iwl_mvm_mac_set_key(hw, cmd, vif, sta, key); } void iwl_mvm_mac_update_tkip_key(struct ieee80211_hw *hw, @@ -4838,7 +4842,7 @@ int iwl_mvm_roc_common(struct ieee80211_hw *hw, struct ieee80211_vif *vif, if (ret) return ret; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); switch (vif->type) { case NL80211_IFTYPE_STATION: @@ -4848,30 +4852,25 @@ int iwl_mvm_roc_common(struct ieee80211_hw *hw, struct ieee80211_vif *vif, ret = ops->add_aux_sta_for_hs20(mvm, lmac_id); if (!ret) ret = iwl_mvm_roc_station(mvm, channel, vif, duration); - goto out_unlock; + return ret; case NL80211_IFTYPE_P2P_DEVICE: /* handle below */ break; default: IWL_ERR(mvm, "ROC: Invalid vif type=%u\n", vif->type); - ret = -EINVAL; - goto out_unlock; + return -EINVAL; } ret = iwl_mvm_p2p_find_phy_ctxt(mvm, vif, channel); if (ret) - goto out_unlock; + return ret; ret = ops->link(mvm, vif); if (ret) - goto out_unlock; + return ret; - ret = iwl_mvm_start_p2p_roc(mvm, vif, duration, type); -out_unlock: - mutex_unlock(&mvm->mutex); - IWL_DEBUG_MAC80211(mvm, "leave\n"); - return ret; + return iwl_mvm_start_p2p_roc(mvm, vif, duration, type); } int iwl_mvm_cancel_roc(struct ieee80211_hw *hw, @@ -4952,13 +4951,9 @@ int iwl_mvm_add_chanctx(struct ieee80211_hw *hw, struct ieee80211_chanctx_conf *ctx) { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - int ret; - - mutex_lock(&mvm->mutex); - ret = __iwl_mvm_add_chanctx(mvm, ctx); - mutex_unlock(&mvm->mutex); - return ret; + guard(mvm)(mvm); + return __iwl_mvm_add_chanctx(mvm, ctx); } static void __iwl_mvm_remove_chanctx(struct iwl_mvm *mvm, @@ -4977,9 +4972,8 @@ void iwl_mvm_remove_chanctx(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); __iwl_mvm_remove_chanctx(mvm, ctx); - mutex_unlock(&mvm->mutex); } void iwl_mvm_change_chanctx(struct ieee80211_hw *hw, @@ -4999,26 +4993,23 @@ void iwl_mvm_change_chanctx(struct ieee80211_hw *hw, phy_ctxt->ref, changed)) return; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); /* we are only changing the min_width, may be a noop */ if (changed == IEEE80211_CHANCTX_CHANGE_MIN_WIDTH) { if (phy_ctxt->width == def->width) - goto out_unlock; + return; /* we are just toggling between 20_NOHT and 20 */ if (phy_ctxt->width <= NL80211_CHAN_WIDTH_20 && def->width <= NL80211_CHAN_WIDTH_20) - goto out_unlock; + return; } iwl_mvm_bt_coex_vif_change(mvm); iwl_mvm_phy_ctxt_changed(mvm, phy_ctxt, def, &ctx->ap, ctx->rx_chains_static, ctx->rx_chains_dynamic); - -out_unlock: - mutex_unlock(&mvm->mutex); } /* @@ -5138,6 +5129,10 @@ static int __iwl_mvm_assign_vif_chanctx(struct iwl_mvm *mvm, } iwl_mvm_update_quotas(mvm, false, NULL); + + iwl_mvm_send_ap_tx_power_constraint_cmd(mvm, vif, + link_conf, + false); } goto out; @@ -5157,13 +5152,9 @@ static int iwl_mvm_assign_vif_chanctx(struct ieee80211_hw *hw, struct ieee80211_chanctx_conf *ctx) { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - int ret; - mutex_lock(&mvm->mutex); - ret = __iwl_mvm_assign_vif_chanctx(mvm, vif, link_conf, ctx, false); - mutex_unlock(&mvm->mutex); - - return ret; + guard(mvm)(mvm); + return __iwl_mvm_assign_vif_chanctx(mvm, vif, link_conf, ctx, false); } /* @@ -5251,9 +5242,8 @@ static void iwl_mvm_unassign_vif_chanctx(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); __iwl_mvm_unassign_vif_chanctx(mvm, vif, link_conf, ctx, false); - mutex_unlock(&mvm->mutex); } static int @@ -5263,7 +5253,7 @@ iwl_mvm_switch_vif_chanctx_swap(struct iwl_mvm *mvm, { int ret; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); ops->__unassign_vif_chanctx(mvm, vifs[0].vif, vifs[0].link_conf, vifs[0].old_ctx, true); __iwl_mvm_remove_chanctx(mvm, vifs[0].old_ctx); @@ -5286,7 +5276,7 @@ iwl_mvm_switch_vif_chanctx_swap(struct iwl_mvm *mvm, if (iwl_mvm_phy_ctx_count(mvm) > 1) iwl_mvm_teardown_tdls_peers(mvm); - goto out; + return 0; out_remove: __iwl_mvm_remove_chanctx(mvm, vifs[0].new_ctx); @@ -5303,15 +5293,11 @@ out_reassign: goto out_restart; } - goto out; + return ret; out_restart: /* things keep failing, better restart the hw */ iwl_mvm_nic_restart(mvm, false); - -out: - mutex_unlock(&mvm->mutex); - return ret; } @@ -5322,7 +5308,7 @@ iwl_mvm_switch_vif_chanctx_reassign(struct iwl_mvm *mvm, { int ret; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); ops->__unassign_vif_chanctx(mvm, vifs[0].vif, vifs[0].link_conf, vifs[0].old_ctx, true); @@ -5334,7 +5320,7 @@ iwl_mvm_switch_vif_chanctx_reassign(struct iwl_mvm *mvm, goto out_reassign; } - goto out; + return 0; out_reassign: if (ops->__assign_vif_chanctx(mvm, vifs[0].vif, vifs[0].link_conf, @@ -5343,15 +5329,11 @@ out_reassign: goto out_restart; } - goto out; + return ret; out_restart: /* things keep failing, better restart the hw */ iwl_mvm_nic_restart(mvm, false); - -out: - mutex_unlock(&mvm->mutex); - return ret; } @@ -5478,13 +5460,9 @@ int iwl_mvm_mac_testmode_cmd(struct ieee80211_hw *hw, void *data, int len) { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - int err; - - mutex_lock(&mvm->mutex); - err = __iwl_mvm_mac_testmode_cmd(mvm, vif, data, len); - mutex_unlock(&mvm->mutex); - return err; + guard(mvm)(mvm); + return __iwl_mvm_mac_testmode_cmd(mvm, vif, data, len); } #endif @@ -5701,13 +5679,9 @@ static int iwl_mvm_mac_pre_channel_switch(struct ieee80211_hw *hw, struct ieee80211_channel_switch *chsw) { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - int ret; - - mutex_lock(&mvm->mutex); - ret = iwl_mvm_pre_channel_switch(mvm, vif, chsw); - mutex_unlock(&mvm->mutex); - return ret; + guard(mvm)(mvm); + return iwl_mvm_pre_channel_switch(mvm, vif, chsw); } void iwl_mvm_channel_switch_rx_beacon(struct ieee80211_hw *hw, @@ -5759,16 +5733,14 @@ void iwl_mvm_channel_switch_rx_beacon(struct ieee80211_hw *hw, } mvmvif->csa_count = chsw->count; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (mvmvif->csa_failed) - goto out_unlock; + return; WARN_ON(iwl_mvm_send_cmd_pdu(mvm, WIDE_ID(MAC_CONF_GROUP, CHANNEL_SWITCH_TIME_EVENT_CMD), 0, sizeof(cmd), &cmd)); -out_unlock: - mutex_unlock(&mvm->mutex); } static void iwl_mvm_flush_no_vif(struct iwl_mvm *mvm, u32 queues, bool drop) @@ -5777,17 +5749,16 @@ static void iwl_mvm_flush_no_vif(struct iwl_mvm *mvm, u32 queues, bool drop) if (!iwl_mvm_has_new_tx_api(mvm)) { if (drop) { - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_flush_tx_path(mvm, iwl_mvm_flushable_queues(mvm) & queues); - mutex_unlock(&mvm->mutex); } else { iwl_trans_wait_tx_queues_empty(mvm->trans, queues); } return; } - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); for (i = 0; i < mvm->fw->ucode_capa.num_stations; i++) { struct ieee80211_sta *sta; @@ -5802,7 +5773,6 @@ static void iwl_mvm_flush_no_vif(struct iwl_mvm *mvm, u32 queues, bool drop) iwl_mvm_wait_sta_queues_empty(mvm, iwl_mvm_sta_from_mac80211(sta)); } - mutex_unlock(&mvm->mutex); } void iwl_mvm_mac_flush(struct ieee80211_hw *hw, struct ieee80211_vif *vif, @@ -5885,7 +5855,7 @@ void iwl_mvm_mac_flush_sta(struct ieee80211_hw *hw, struct ieee80211_vif *vif, struct ieee80211_link_sta *link_sta; int link_id; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); for_each_sta_active_link(vif, sta, link_sta, link_id) { mvm_link_sta = rcu_dereference_protected(mvmsta->link[link_id], lockdep_is_held(&mvm->mutex)); @@ -5896,7 +5866,6 @@ void iwl_mvm_mac_flush_sta(struct ieee80211_hw *hw, struct ieee80211_vif *vif, mvmsta->tfd_queue_msk)) IWL_ERR(mvm, "flush request fail\n"); } - mutex_unlock(&mvm->mutex); } static int iwl_mvm_mac_get_acs_survey(struct iwl_mvm *mvm, int idx, @@ -5962,7 +5931,6 @@ int iwl_mvm_mac_get_survey(struct ieee80211_hw *hw, int idx, struct survey_info *survey) { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - int ret = 0; u8 cmd_ver = iwl_fw_lookup_cmd_ver(mvm->fw, WIDE_ID(SYSTEM_GROUP, SYSTEM_STATISTICS_CMD), @@ -5982,12 +5950,13 @@ int iwl_mvm_mac_get_survey(struct ieee80211_hw *hw, int idx, if (idx > 0) return iwl_mvm_mac_get_acs_survey(mvm, idx - 1, survey); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (iwl_mvm_firmware_running(mvm)) { - ret = iwl_mvm_request_statistics(mvm, false); + int ret = iwl_mvm_request_statistics(mvm, false); + if (ret) - goto out; + return ret; } survey->filled = SURVEY_INFO_TIME_RX | @@ -6003,7 +5972,7 @@ int iwl_mvm_mac_get_survey(struct ieee80211_hw *hw, int idx, /* the new fw api doesn't support the following fields */ if (cmd_ver != IWL_FW_CMD_VER_UNKNOWN) - goto out; + return 0; survey->filled |= SURVEY_INFO_TIME | SURVEY_INFO_TIME_SCAN; @@ -6015,9 +5984,7 @@ int iwl_mvm_mac_get_survey(struct ieee80211_hw *hw, int idx, mvm->radio_stats.on_time_scan; do_div(survey->time_scan, USEC_PER_MSEC); - out: - mutex_unlock(&mvm->mutex); - return ret; + return 0; } static void iwl_mvm_set_sta_rate(u32 rate_n_flags, struct rate_info *rinfo) @@ -6184,13 +6151,13 @@ void iwl_mvm_mac_sta_statistics(struct ieee80211_hw *hw, if (!vif->cfg.assoc) return; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (mvmvif->deflink.ap_sta_id != mvmsta->deflink.sta_id) - goto unlock; + return; if (iwl_mvm_request_statistics(mvm, false)) - goto unlock; + return; sinfo->rx_beacon = 0; for_each_mvm_vif_valid_link(mvmvif, i) @@ -6204,8 +6171,6 @@ void iwl_mvm_mac_sta_statistics(struct ieee80211_hw *hw, mvmvif->deflink.beacon_stats.avg_signal; sinfo->filled |= BIT_ULL(NL80211_STA_INFO_BEACON_SIGNAL_AVG); } - unlock: - mutex_unlock(&mvm->mutex); } static void iwl_mvm_event_mlme_callback_ini(struct iwl_mvm *mvm, @@ -6348,7 +6313,7 @@ void iwl_mvm_sync_rx_queues_internal(struct iwl_mvm *mvm, .len[0] = sizeof(cmd), .data[1] = data, .len[1] = size, - .flags = sync ? 0 : CMD_ASYNC, + .flags = CMD_SEND_IN_RFKILL | (sync ? 0 : CMD_ASYNC), }; int ret; @@ -6393,9 +6358,8 @@ void iwl_mvm_sync_rx_queues(struct ieee80211_hw *hw) { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_sync_rx_queues_internal(mvm, IWL_MVM_RXQ_EMPTY, true, NULL, 0); - mutex_unlock(&mvm->mutex); } int @@ -6431,13 +6395,9 @@ int iwl_mvm_start_pmsr(struct ieee80211_hw *hw, struct ieee80211_vif *vif, struct cfg80211_pmsr_request *request) { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - int ret; - mutex_lock(&mvm->mutex); - ret = iwl_mvm_ftm_start(mvm, vif, request); - mutex_unlock(&mvm->mutex); - - return ret; + guard(mvm)(mvm); + return iwl_mvm_ftm_start(mvm, vif, request); } void iwl_mvm_abort_pmsr(struct ieee80211_hw *hw, struct ieee80211_vif *vif, @@ -6445,9 +6405,8 @@ void iwl_mvm_abort_pmsr(struct ieee80211_hw *hw, struct ieee80211_vif *vif, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_ftm_abort(mvm, request); - mutex_unlock(&mvm->mutex); } static bool iwl_mvm_can_hw_csum(struct sk_buff *skb) @@ -6482,7 +6441,6 @@ int iwl_mvm_set_hw_timestamp(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); u32 protocols = 0; - int ret; /* HW timestamping is only supported for a specific station */ if (!hwts->macaddr) @@ -6492,11 +6450,8 @@ int iwl_mvm_set_hw_timestamp(struct ieee80211_hw *hw, protocols = IWL_TIME_SYNC_PROTOCOL_TM | IWL_TIME_SYNC_PROTOCOL_FTM; - mutex_lock(&mvm->mutex); - ret = iwl_mvm_time_sync_config(mvm, hwts->macaddr, protocols); - mutex_unlock(&mvm->mutex); - - return ret; + guard(mvm)(mvm); + return iwl_mvm_time_sync_config(mvm, hwts->macaddr, protocols); } const struct ieee80211_ops iwl_mvm_hw_ops = { diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c b/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c index 0a3b7284eeddf9f382752e7d12a88da6a0b0d654..ebf313e161f4d380b6c45da9d02727580ea88bf7 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/mld-mac80211.c @@ -12,7 +12,7 @@ static int iwl_mvm_mld_mac_add_interface(struct ieee80211_hw *hw, int ret; int i; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_mac_init_mvmvif(mvm, mvmvif); @@ -32,7 +32,7 @@ static int iwl_mvm_mld_mac_add_interface(struct ieee80211_hw *hw, /* Allocate resources for the MAC context, and add it to the fw */ ret = iwl_mvm_mac_ctxt_init(mvm, vif); if (ret) - goto out_unlock; + return ret; rcu_assign_pointer(mvm->vif_id_to_mac[mvmvif->id], vif); @@ -46,7 +46,7 @@ static int iwl_mvm_mld_mac_add_interface(struct ieee80211_hw *hw, ret = iwl_mvm_mld_mac_ctxt_add(mvm, vif); if (ret) - goto out_unlock; + return ret; /* beacon filtering */ ret = iwl_mvm_disable_beacon_filter(mvm, vif); @@ -75,8 +75,6 @@ static int iwl_mvm_mld_mac_add_interface(struct ieee80211_hw *hw, goto out_free_bf; iwl_mvm_tcm_add_vif(mvm, vif); - INIT_DELAYED_WORK(&mvmvif->csa_work, - iwl_mvm_channel_switch_disconnect_wk); if (vif->type == NL80211_IFTYPE_MONITOR) { mvm->monitor_on = true; @@ -97,7 +95,7 @@ static int iwl_mvm_mld_mac_add_interface(struct ieee80211_hw *hw, if (vif->p2p || iwl_fw_lookup_cmd_ver(mvm->fw, PHY_CONTEXT_CMD, 1) < 5) vif->driver_flags |= IEEE80211_VIF_IGNORE_OFDMA_WIDER_BW; - goto out_unlock; + return 0; out_free_bf: if (mvm->bf_allowed_vif == mvmvif) { @@ -108,9 +106,6 @@ static int iwl_mvm_mld_mac_add_interface(struct ieee80211_hw *hw, out_remove_mac: mvmvif->link[0] = NULL; iwl_mvm_mld_mac_ctxt_remove(mvm, vif); - out_unlock: - mutex_unlock(&mvm->mutex); - return ret; } @@ -127,7 +122,7 @@ static void iwl_mvm_mld_mac_remove_interface(struct ieee80211_hw *hw, vif->type == NL80211_IFTYPE_ADHOC)) iwl_mvm_tcm_rm_vif(mvm, vif); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (vif == mvm->csme_vif) { iwl_mei_set_netdev(NULL); @@ -190,8 +185,6 @@ static void iwl_mvm_mld_mac_remove_interface(struct ieee80211_hw *hw, mvm->monitor_on = false; __clear_bit(IEEE80211_HW_RX_INCLUDES_FCS, mvm->hw->flags); } - - mutex_unlock(&mvm->mutex); } static unsigned int iwl_mvm_mld_count_active_links(struct iwl_mvm_vif *mvmvif) @@ -229,6 +222,8 @@ static void iwl_mvm_restart_mpdu_count(struct iwl_mvm *mvm, mvmsta->mpdu_counters[q].window_start = jiffies; spin_unlock_bh(&mvmsta->mpdu_counters[q].lock); } + + IWL_DEBUG_STATS(mvm, "MPDU counters are cleared\n"); } static int iwl_mvm_esr_mode_active(struct iwl_mvm *mvm, @@ -352,6 +347,11 @@ __iwl_mvm_mld_assign_vif_chanctx(struct iwl_mvm *mvm, rcu_read_unlock(); } + if (vif->type == NL80211_IFTYPE_STATION) + iwl_mvm_send_ap_tx_power_constraint_cmd(mvm, vif, + link_conf, + false); + /* then activate */ ret = iwl_mvm_link_changed(mvm, vif, link_conf, LINK_CONTEXT_MODIFY_ACTIVE | @@ -389,10 +389,11 @@ static int iwl_mvm_mld_assign_vif_chanctx(struct ieee80211_hw *hw, struct ieee80211_chanctx_conf *ctx) { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - int ret; /* update EMLSR mode */ if (ieee80211_vif_type_p2p(vif) != NL80211_IFTYPE_STATION) { + int ret; + ret = iwl_mvm_esr_non_bss_link(mvm, vif, link_conf->link_id, true); /* @@ -403,11 +404,8 @@ static int iwl_mvm_mld_assign_vif_chanctx(struct ieee80211_hw *hw, return ret; } - mutex_lock(&mvm->mutex); - ret = __iwl_mvm_mld_assign_vif_chanctx(mvm, vif, link_conf, ctx, false); - mutex_unlock(&mvm->mutex); - - return ret; + guard(mvm)(mvm); + return __iwl_mvm_mld_assign_vif_chanctx(mvm, vif, link_conf, ctx, false); } static int iwl_mvm_esr_mode_inactive(struct iwl_mvm *mvm, @@ -533,9 +531,37 @@ static void iwl_mvm_mld_unassign_vif_chanctx(struct ieee80211_hw *hw, } static void +iwl_mvm_tpe_sta_cmd_data(struct iwl_txpower_constraints_cmd *cmd, + const struct ieee80211_bss_conf *bss_info) +{ + u8 i; + + /* + * NOTE: the 0 here is IEEE80211_TPE_CAT_6GHZ_DEFAULT, + * we fully ignore IEEE80211_TPE_CAT_6GHZ_SUBORDINATE + */ + + BUILD_BUG_ON(ARRAY_SIZE(cmd->psd_pwr) != + ARRAY_SIZE(bss_info->tpe.psd_local[0].power)); + + /* if not valid, mac80211 puts default (max value) */ + for (i = 0; i < ARRAY_SIZE(cmd->psd_pwr); i++) + cmd->psd_pwr[i] = min(bss_info->tpe.psd_local[0].power[i], + bss_info->tpe.psd_reg_client[0].power[i]); + + BUILD_BUG_ON(ARRAY_SIZE(cmd->eirp_pwr) != + ARRAY_SIZE(bss_info->tpe.max_local[0].power)); + + for (i = 0; i < ARRAY_SIZE(cmd->eirp_pwr); i++) + cmd->eirp_pwr[i] = min(bss_info->tpe.max_local[0].power[i], + bss_info->tpe.max_reg_client[0].power[i]); +} + +void iwl_mvm_send_ap_tx_power_constraint_cmd(struct iwl_mvm *mvm, struct ieee80211_vif *vif, - struct ieee80211_bss_conf *bss_conf) + struct ieee80211_bss_conf *bss_conf, + bool is_ap) { struct iwl_txpower_constraints_cmd cmd = {}; struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif); @@ -555,19 +581,22 @@ iwl_mvm_send_ap_tx_power_constraint_cmd(struct iwl_mvm *mvm, link_info->fw_link_id == IWL_MVM_FW_LINK_ID_INVALID) return; - if (bss_conf->chanreq.oper.chan->band != NL80211_BAND_6GHZ || - bss_conf->chanreq.oper.chan->flags & - IEEE80211_CHAN_NO_6GHZ_VLP_CLIENT) + if (bss_conf->chanreq.oper.chan->band != NL80211_BAND_6GHZ) return; cmd.link_id = cpu_to_le16(link_info->fw_link_id); - /* - * Currently supporting VLP Soft AP only. - */ - cmd.ap_type = cpu_to_le16(IWL_6GHZ_AP_TYPE_VLP); memset(cmd.psd_pwr, DEFAULT_TPE_TX_POWER, sizeof(cmd.psd_pwr)); memset(cmd.eirp_pwr, DEFAULT_TPE_TX_POWER, sizeof(cmd.eirp_pwr)); + if (is_ap) { + cmd.ap_type = cpu_to_le16(IWL_6GHZ_AP_TYPE_VLP); + } else if (bss_conf->power_type == IEEE80211_REG_UNSET_AP) { + return; + } else { + cmd.ap_type = cpu_to_le16(bss_conf->power_type - 1); + iwl_mvm_tpe_sta_cmd_data(&cmd, bss_conf); + } + ret = iwl_mvm_send_cmd_pdu(mvm, WIDE_ID(PHY_OPS_GROUP, AP_TX_POWER_CONSTRAINTS_CMD), @@ -586,15 +615,16 @@ static int iwl_mvm_mld_start_ap_ibss(struct ieee80211_hw *hw, struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif); int ret; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (vif->type == NL80211_IFTYPE_AP) - iwl_mvm_send_ap_tx_power_constraint_cmd(mvm, vif, link_conf); + iwl_mvm_send_ap_tx_power_constraint_cmd(mvm, vif, + link_conf, true); /* Send the beacon template */ ret = iwl_mvm_mac_ctxt_beacon_changed(mvm, vif, link_conf); if (ret) - goto out_unlock; + return ret; /* the link should be already activated when assigning chan context */ ret = iwl_mvm_link_changed(mvm, vif, link_conf, @@ -602,11 +632,11 @@ static int iwl_mvm_mld_start_ap_ibss(struct ieee80211_hw *hw, ~LINK_CONTEXT_MODIFY_ACTIVE, true); if (ret) - goto out_unlock; + return ret; ret = iwl_mvm_mld_add_mcast_sta(mvm, vif, link_conf); if (ret) - goto out_unlock; + return ret; /* Send the bcast station. At this stage the TBTT and DTIM time * events are added and applied to the scheduler @@ -630,7 +660,7 @@ static int iwl_mvm_mld_start_ap_ibss(struct ieee80211_hw *hw, iwl_mvm_ftm_restart_responder(mvm, vif, link_conf); - goto out_unlock; + return 0; out_failed: iwl_mvm_power_update_mac(mvm); @@ -638,8 +668,6 @@ out_failed: iwl_mvm_mld_rm_bcast_sta(mvm, vif, link_conf); out_rm_mcast: iwl_mvm_mld_rm_mcast_sta(mvm, vif, link_conf); -out_unlock: - mutex_unlock(&mvm->mutex); return ret; } @@ -662,7 +690,7 @@ static void iwl_mvm_mld_stop_ap_ibss(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_stop_ap_ibss_common(mvm, vif); @@ -676,7 +704,6 @@ static void iwl_mvm_mld_stop_ap_ibss(struct ieee80211_hw *hw, iwl_mvm_mld_rm_mcast_sta(mvm, vif, link_conf); iwl_mvm_power_update_mac(mvm); - mutex_unlock(&mvm->mutex); } static void iwl_mvm_mld_stop_ap(struct ieee80211_hw *hw, @@ -979,7 +1006,7 @@ static void iwl_mvm_mld_link_info_changed(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); switch (vif->type) { case NL80211_IFTYPE_STATION: @@ -1005,8 +1032,6 @@ static void iwl_mvm_mld_link_info_changed(struct ieee80211_hw *hw, link_conf->txpower); iwl_mvm_set_tx_power(mvm, vif, link_conf->txpower); } - - mutex_unlock(&mvm->mutex); } static void iwl_mvm_mld_vif_cfg_changed(struct ieee80211_hw *hw, @@ -1015,15 +1040,13 @@ static void iwl_mvm_mld_vif_cfg_changed(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (changes & BSS_CHANGED_IDLE && !vif->cfg.idle) iwl_mvm_scan_stop(mvm, IWL_MVM_SCAN_SCHED, true); if (vif->type == NL80211_IFTYPE_STATION) iwl_mvm_mld_vif_cfg_changed_station(mvm, vif, changes); - - mutex_unlock(&mvm->mutex); } static int @@ -1056,9 +1079,8 @@ static void iwl_mvm_mld_config_iface_filter(struct ieee80211_hw *hw, !vif->p2p) return; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); iwl_mvm_mld_mac_ctxt_changed(mvm, vif, false); - mutex_unlock(&mvm->mutex); } static int @@ -1080,14 +1102,10 @@ iwl_mvm_mld_mac_conf_tx(struct ieee80211_hw *hw, * The exception is P2P_DEVICE interface which needs immediate update. */ if (vif->type == NL80211_IFTYPE_P2P_DEVICE) { - int ret; - - mutex_lock(&mvm->mutex); - ret = iwl_mvm_link_changed(mvm, vif, &vif->bss_conf, - LINK_CONTEXT_MODIFY_QOS_PARAMS, - true); - mutex_unlock(&mvm->mutex); - return ret; + guard(mvm)(mvm); + return iwl_mvm_link_changed(mvm, vif, &vif->bss_conf, + LINK_CONTEXT_MODIFY_QOS_PARAMS, + true); } return 0; } @@ -1232,13 +1250,9 @@ iwl_mvm_mld_change_sta_links(struct ieee80211_hw *hw, u16 old_links, u16 new_links) { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); - int ret; - mutex_lock(&mvm->mutex); - ret = iwl_mvm_mld_update_sta_links(mvm, vif, sta, old_links, new_links); - mutex_unlock(&mvm->mutex); - - return ret; + guard(mvm)(mvm); + return iwl_mvm_mld_update_sta_links(mvm, vif, sta, old_links, new_links); } bool iwl_mvm_vif_has_esr_cap(struct iwl_mvm *mvm, struct ieee80211_vif *vif) @@ -1266,26 +1280,19 @@ static bool iwl_mvm_mld_can_activate_links(struct ieee80211_hw *hw, { struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw); int n_links = hweight16(desired_links); - bool ret = true; if (n_links <= 1) return true; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); /* Check if HW supports the wanted number of links */ - if (n_links > iwl_mvm_max_active_links(mvm, vif)) { - ret = false; - goto unlock; - } + if (n_links > iwl_mvm_max_active_links(mvm, vif)) + return false; /* If it is an eSR device, check that we can enter eSR */ - ret = iwl_mvm_is_esr_supported(mvm->fwrt.trans) && - iwl_mvm_vif_has_esr_cap(mvm, vif); - -unlock: - mutex_unlock(&mvm->mutex); - return ret; + return iwl_mvm_is_esr_supported(mvm->fwrt.trans) && + iwl_mvm_vif_has_esr_cap(mvm, vif); } static enum ieee80211_neg_ttlm_res diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mld-sta.c b/drivers/net/wireless/intel/iwlwifi/mvm/mld-sta.c index b7a461dba41ee3fe87a09f3ca10d5a1cdded0360..d5a204e520764bdcd64a4829b6203650da6b4c50 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/mld-sta.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/mld-sta.c @@ -241,7 +241,7 @@ int iwl_mvm_mld_add_bcast_sta(struct iwl_mvm *mvm, struct ieee80211_vif *vif, IWL_MAX_TID_COUNT, &wdg_timeout); } -/* Allocate a new station entry for the broadcast station to the given vif, +/* Allocate a new station entry for the multicast station to the given vif, * and send it to the FW. * Note that each AP/GO mac should have its own multicast station. */ @@ -470,7 +470,7 @@ static int iwl_mvm_mld_cfg_sta(struct iwl_mvm *mvm, struct ieee80211_sta *sta, break; } - switch (sta->deflink.smps_mode) { + switch (link_sta->smps_mode) { case IEEE80211_SMPS_AUTOMATIC: case IEEE80211_SMPS_NUM_MODES: WARN_ON(1); @@ -515,11 +515,11 @@ static int iwl_mvm_mld_cfg_sta(struct iwl_mvm *mvm, struct ieee80211_sta *sta, return iwl_mvm_mld_send_sta_cmd(mvm, &cmd); } -static void iwl_mvm_mld_free_sta_link(struct iwl_mvm *mvm, - struct iwl_mvm_sta *mvm_sta, - struct iwl_mvm_link_sta *mvm_sta_link, - unsigned int link_id, - bool is_in_fw) +void iwl_mvm_mld_free_sta_link(struct iwl_mvm *mvm, + struct iwl_mvm_sta *mvm_sta, + struct iwl_mvm_link_sta *mvm_sta_link, + unsigned int link_id, + bool is_in_fw) { RCU_INIT_POINTER(mvm->fw_id_to_mac_id[mvm_sta_link->sta_id], is_in_fw ? ERR_PTR(-EINVAL) : NULL); @@ -1014,7 +1014,8 @@ static int iwl_mvm_mld_update_sta_baids(struct iwl_mvm *mvm, cmd.modify.tid = cpu_to_le32(data->tid); - ret = iwl_mvm_send_cmd_pdu(mvm, cmd_id, 0, sizeof(cmd), &cmd); + ret = iwl_mvm_send_cmd_pdu(mvm, cmd_id, CMD_SEND_IN_RFKILL, + sizeof(cmd), &cmd); data->sta_mask = new_sta_mask; if (ret) return ret; diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h index 1f58c727fa632df016241ed1a64ba5731d0b1d79..9b939225990af4582b31d1e55c43fa87d294e370 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h +++ b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h @@ -9,6 +9,7 @@ #include #include +#include #include #include @@ -23,7 +24,7 @@ #include "iwl-op-mode.h" #include "iwl-trans.h" #include "fw/notif-wait.h" -#include "iwl-eeprom-parse.h" +#include "iwl-nvm-utils.h" #include "fw/file.h" #include "iwl-config.h" #include "sta.h" @@ -426,6 +427,7 @@ struct iwl_mvm_esr_exit { * @csa_bcn_pending: indicates that we are waiting for a beacon on a new channel * @csa_blocks_tx: CSA is blocking TX * @features: hw features active for this vif + * @max_tx_op: max TXOP in usecs for all ACs, zero for no limit. * @ap_beacon_time: AP beacon time for synchronisation (on older FW) * @bf_enabled: indicates if beacon filtering is enabled * @ba_enabled: indicated if beacon abort is enabled @@ -538,6 +540,8 @@ struct iwl_mvm_vif { struct ieee80211_key_conf __rcu *keys[2]; } bcn_prot; + u16 max_tx_op; + u16 link_selection_res; u8 link_selection_primary; u8 primary_link; @@ -1040,7 +1044,6 @@ struct iwl_mvm { struct iwl_rx_phy_info last_phy_info; struct ieee80211_sta __rcu *fw_id_to_mac_id[IWL_MVM_STATION_COUNT_MAX]; struct ieee80211_link_sta __rcu *fw_id_to_link_sta[IWL_MVM_STATION_COUNT_MAX]; - unsigned long fw_link_ids_map; u8 rx_ba_sessions; /* configured by mac80211 */ @@ -1306,6 +1309,9 @@ struct iwl_mvm { struct iwl_phy_specific_cfg phy_filters; #endif + /* report rx timestamp in ptp clock time */ + bool rx_ts_ptp; + unsigned long last_6ghz_passive_scan_jiffies; unsigned long last_reset_or_resume_time_jiffies; @@ -1330,6 +1336,8 @@ struct iwl_mvm { #define IWL_MAC80211_GET_MVM(_hw) \ IWL_OP_MODE_GET_MVM((struct iwl_op_mode *)((_hw)->priv)) +DEFINE_GUARD(mvm, struct iwl_mvm *, mutex_lock(&_T->mutex), mutex_unlock(&_T->mutex)) + /** * enum iwl_mvm_status - MVM status bits * @IWL_MVM_STATUS_HW_RFKILL: HW RF-kill is asserted @@ -1758,6 +1766,7 @@ u8 iwl_mvm_next_antenna(struct iwl_mvm *mvm, u8 valid, u8 last_idx); void iwl_mvm_get_sync_time(struct iwl_mvm *mvm, int clock_type, u32 *gp2, u64 *boottime, ktime_t *realtime); u32 iwl_mvm_get_systime(struct iwl_mvm *mvm); +u32 iwl_mvm_find_ie_offset(u8 *beacon, u8 eid, u32 frame_size); /* Tx / Host Commands */ int __must_check iwl_mvm_send_cmd(struct iwl_mvm *mvm, @@ -2953,4 +2962,10 @@ iwl_mvm_bt_coex_calculate_esr_mode(struct iwl_mvm *mvm, bool primary); int iwl_mvm_esr_non_bss_link(struct iwl_mvm *mvm, struct ieee80211_vif *vif, unsigned int link_id, bool active); + +void +iwl_mvm_send_ap_tx_power_constraint_cmd(struct iwl_mvm *mvm, + struct ieee80211_vif *vif, + struct ieee80211_bss_conf *bss_conf, + bool is_ap); #endif /* __IWL_MVM_H__ */ diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/nvm.c b/drivers/net/wireless/intel/iwlwifi/mvm/nvm.c index ae817722288134ee98bcaae0fee4c6c5bdd3c0f8..836ca22597bc1e7211d09a3bd02e9a482c8fa37a 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/nvm.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/nvm.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause /* - * Copyright (C) 2012-2014, 2018-2019, 2021-2023 Intel Corporation + * Copyright (C) 2012-2014, 2018-2019, 2021-2024 Intel Corporation * Copyright (C) 2013-2015 Intel Mobile Communications GmbH * Copyright (C) 2016-2017 Intel Deutschland GmbH */ @@ -9,8 +9,7 @@ #include "iwl-trans.h" #include "iwl-csr.h" #include "mvm.h" -#include "iwl-eeprom-parse.h" -#include "iwl-eeprom-read.h" +#include "iwl-nvm-utils.h" #include "iwl-nvm-parse.h" #include "iwl-prph.h" #include "fw/acpi.h" diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c index 53283d052e18904c4813147cf04768150cb684ef..85178fe1b7eac2b507579a85e74804fb781649f7 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c @@ -18,7 +18,7 @@ #include "iwl-modparams.h" #include "mvm.h" #include "iwl-phy-db.h" -#include "iwl-eeprom-parse.h" +#include "iwl-nvm-utils.h" #include "iwl-csr.h" #include "iwl-io.h" #include "iwl-prph.h" @@ -572,6 +572,7 @@ static const struct iwl_hcmd_names iwl_mvm_legacy_names[] = { HCMD_NAME(D0I3_END_CMD), HCMD_NAME(LTR_CONFIG), HCMD_NAME(LDBG_CONFIG_CMD), + HCMD_NAME(DEBUG_LOG_MSG), }; /* Please keep this array *SORTED* by hex value. @@ -579,6 +580,7 @@ static const struct iwl_hcmd_names iwl_mvm_legacy_names[] = { */ static const struct iwl_hcmd_names iwl_mvm_system_names[] = { HCMD_NAME(SHARED_MEM_CFG_CMD), + HCMD_NAME(SOC_CONFIGURATION_CMD), HCMD_NAME(INIT_EXTENDED_CFG_CMD), HCMD_NAME(FW_ERROR_RECOVERY_CMD), HCMD_NAME(RFI_CONFIG_CMD), @@ -593,8 +595,10 @@ static const struct iwl_hcmd_names iwl_mvm_system_names[] = { * Access is done through binary search */ static const struct iwl_hcmd_names iwl_mvm_mac_conf_names[] = { + HCMD_NAME(LOW_LATENCY_CMD), HCMD_NAME(CHANNEL_SWITCH_TIME_EVENT_CMD), HCMD_NAME(SESSION_PROTECTION_CMD), + HCMD_NAME(CANCEL_CHANNEL_SWITCH_CMD), HCMD_NAME(MAC_CONFIG_CMD), HCMD_NAME(LINK_CONFIG_CMD), HCMD_NAME(STA_CONFIG_CMD), @@ -603,7 +607,10 @@ static const struct iwl_hcmd_names iwl_mvm_mac_conf_names[] = { HCMD_NAME(STA_DISABLE_TX_CMD), HCMD_NAME(ROC_CMD), HCMD_NAME(ROC_NOTIF), + HCMD_NAME(CHANNEL_SWITCH_ERROR_NOTIF), + HCMD_NAME(MISSED_VAP_NOTIF), HCMD_NAME(SESSION_PROTECTION_NOTIF), + HCMD_NAME(PROBE_RESPONSE_DATA_NOTIF), HCMD_NAME(CHANNEL_SWITCH_START_NOTIF), }; @@ -627,6 +634,8 @@ static const struct iwl_hcmd_names iwl_mvm_data_path_names[] = { HCMD_NAME(DQA_ENABLE_CMD), HCMD_NAME(UPDATE_MU_GROUPS_CMD), HCMD_NAME(TRIGGER_RX_QUEUES_NOTIF_CMD), + HCMD_NAME(WNM_PLATFORM_PTM_REQUEST_CMD), + HCMD_NAME(WNM_80211V_TIMING_MEASUREMENT_CONFIG_CMD), HCMD_NAME(STA_HE_CTXT_CMD), HCMD_NAME(RLC_CONFIG_CMD), HCMD_NAME(RFH_QUEUE_CONFIG_CMD), @@ -650,6 +659,21 @@ static const struct iwl_hcmd_names iwl_mvm_statistics_names[] = { HCMD_NAME(STATISTICS_OPER_PART1_NOTIF), }; +/* Please keep this array *SORTED* by hex value. + * Access is done through binary search + */ +static const struct iwl_hcmd_names iwl_mvm_debug_names[] = { + HCMD_NAME(LMAC_RD_WR), + HCMD_NAME(UMAC_RD_WR), + HCMD_NAME(HOST_EVENT_CFG), + HCMD_NAME(DBGC_SUSPEND_RESUME), + HCMD_NAME(BUFFER_ALLOCATION), + HCMD_NAME(GET_TAS_STATUS), + HCMD_NAME(FW_DUMP_COMPLETE_CMD), + HCMD_NAME(FW_CLEAR_BUFFER), + HCMD_NAME(MFU_ASSERT_DUMP_NTF), +}; + /* Please keep this array *SORTED* by hex value. * Access is done through binary search */ @@ -705,6 +729,7 @@ static const struct iwl_hcmd_arr iwl_mvm_groups[] = { [PROT_OFFLOAD_GROUP] = HCMD_ARR(iwl_mvm_prot_offload_names), [REGULATORY_AND_NVM_GROUP] = HCMD_ARR(iwl_mvm_regulatory_and_nvm_names), + [DEBUG_GROUP] = HCMD_ARR(iwl_mvm_debug_names), [STATISTICS_GROUP] = HCMD_ARR(iwl_mvm_statistics_names), }; @@ -740,20 +765,18 @@ static void iwl_mvm_tx_unblock_dwork(struct work_struct *work) struct ieee80211_vif *tx_blocked_vif; struct iwl_mvm_vif *mvmvif; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); tx_blocked_vif = rcu_dereference_protected(mvm->csa_tx_blocked_vif, lockdep_is_held(&mvm->mutex)); if (!tx_blocked_vif) - goto unlock; + return; mvmvif = iwl_mvm_vif_from_mac80211(tx_blocked_vif); iwl_mvm_modify_all_sta_disable_tx(mvm, mvmvif, false); RCU_INIT_POINTER(mvm->csa_tx_blocked_vif, NULL); -unlock: - mutex_unlock(&mvm->mutex); } static void iwl_mvm_fwrt_dump_start(void *ctx) @@ -770,21 +793,12 @@ static void iwl_mvm_fwrt_dump_end(void *ctx) mutex_unlock(&mvm->mutex); } -static bool iwl_mvm_fwrt_fw_running(void *ctx) -{ - return iwl_mvm_firmware_running(ctx); -} - static int iwl_mvm_fwrt_send_hcmd(void *ctx, struct iwl_host_cmd *host_cmd) { struct iwl_mvm *mvm = (struct iwl_mvm *)ctx; - int ret; - mutex_lock(&mvm->mutex); - ret = iwl_mvm_send_cmd(mvm, host_cmd); - mutex_unlock(&mvm->mutex); - - return ret; + guard(mvm)(mvm); + return iwl_mvm_send_cmd(mvm, host_cmd); } static bool iwl_mvm_d3_debug_enable(void *ctx) @@ -795,7 +809,6 @@ static bool iwl_mvm_d3_debug_enable(void *ctx) static const struct iwl_fw_runtime_ops iwl_mvm_fwrt_ops = { .dump_start = iwl_mvm_fwrt_dump_start, .dump_end = iwl_mvm_fwrt_dump_end, - .fw_running = iwl_mvm_fwrt_fw_running, .send_hcmd = iwl_mvm_fwrt_send_hcmd, .d3_debug_enable = iwl_mvm_d3_debug_enable, }; @@ -1360,24 +1373,7 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg, trans_cfg.no_reclaim_cmds = no_reclaim_cmds; trans_cfg.n_no_reclaim_cmds = ARRAY_SIZE(no_reclaim_cmds); - switch (iwlwifi_mod_params.amsdu_size) { - case IWL_AMSDU_DEF: - trans_cfg.rx_buf_size = IWL_AMSDU_4K; - break; - case IWL_AMSDU_4K: - trans_cfg.rx_buf_size = IWL_AMSDU_4K; - break; - case IWL_AMSDU_8K: - trans_cfg.rx_buf_size = IWL_AMSDU_8K; - break; - case IWL_AMSDU_12K: - trans_cfg.rx_buf_size = IWL_AMSDU_12K; - break; - default: - pr_err("%s: Unsupported amsdu_size: %d\n", KBUILD_MODNAME, - iwlwifi_mod_params.amsdu_size); - trans_cfg.rx_buf_size = IWL_AMSDU_4K; - } + trans_cfg.rx_buf_size = iwl_amsdu_size_to_rxb_size(); trans->wide_cmd_header = true; trans_cfg.bc_table_dword = @@ -1437,9 +1433,6 @@ iwl_op_mode_mvm_start(struct iwl_trans *trans, const struct iwl_cfg *cfg, goto out_free; } - IWL_INFO(mvm, "Detected %s, REV=0x%X\n", - mvm->trans->name, mvm->trans->hw_rev); - if (iwlwifi_mod_params.nvm_file) mvm->nvm_file_name = iwlwifi_mod_params.nvm_file; else diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/power.c b/drivers/net/wireless/intel/iwlwifi/mvm/power.c index 568f53c5619984295fcde0c212f57a0a963df956..cc7176b0be4fc043f10b931ff0d5630919cc5374 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/power.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/power.c @@ -211,19 +211,37 @@ static void iwl_mvm_power_configure_uapsd(struct iwl_mvm *mvm, IWL_MVM_PS_HEAVY_RX_THLD_PERCENT; } -static void iwl_mvm_p2p_standalone_iterator(void *_data, u8 *mac, - struct ieee80211_vif *vif) +struct iwl_allow_uapsd_iface_iterator_data { + struct ieee80211_vif *current_vif; + bool allow_uapsd; +}; + +static void iwl_mvm_allow_uapsd_iterator(void *_data, u8 *mac, + struct ieee80211_vif *vif) { - bool *is_p2p_standalone = _data; + struct iwl_allow_uapsd_iface_iterator_data *data = _data; + struct iwl_mvm_vif *other_mvmvif = iwl_mvm_vif_from_mac80211(vif); + struct iwl_mvm_vif *curr_mvmvif = + iwl_mvm_vif_from_mac80211(data->current_vif); - switch (ieee80211_vif_type_p2p(vif)) { - case NL80211_IFTYPE_P2P_GO: + /* exclude the given vif */ + if (vif == data->current_vif) + return; + + switch (vif->type) { case NL80211_IFTYPE_AP: - *is_p2p_standalone = false; + case NL80211_IFTYPE_ADHOC: + case NL80211_IFTYPE_NAN: + data->allow_uapsd = false; break; case NL80211_IFTYPE_STATION: - if (vif->cfg.assoc) - *is_p2p_standalone = false; + /* allow UAPSD if P2P interface and BSS station interface share + * the same channel. + */ + if (vif->cfg.assoc && other_mvmvif->deflink.phy_ctxt && + curr_mvmvif->deflink.phy_ctxt && + other_mvmvif->deflink.phy_ctxt->id != curr_mvmvif->deflink.phy_ctxt->id) + data->allow_uapsd = false; break; default: @@ -235,6 +253,10 @@ static bool iwl_mvm_power_allow_uapsd(struct iwl_mvm *mvm, struct ieee80211_vif *vif) { struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif); + struct iwl_allow_uapsd_iface_iterator_data data = { + .current_vif = vif, + .allow_uapsd = true, + }; if (ether_addr_equal(mvmvif->uapsd_misbehaving_ap_addr, vif->cfg.ap_addr)) @@ -249,30 +271,15 @@ static bool iwl_mvm_power_allow_uapsd(struct iwl_mvm *mvm, IEEE80211_P2P_OPPPS_ENABLE_BIT)) return false; - /* - * Avoid using uAPSD if client is in DCM - - * low latency issue in Miracast - */ - if (iwl_mvm_phy_ctx_count(mvm) >= 2) + if (vif->p2p && !iwl_mvm_is_p2p_scm_uapsd_supported(mvm)) return false; - if (vif->p2p) { - /* Allow U-APSD only if p2p is stand alone */ - bool is_p2p_standalone = true; - - if (!iwl_mvm_is_p2p_scm_uapsd_supported(mvm)) - return false; - - ieee80211_iterate_active_interfaces_atomic(mvm->hw, - IEEE80211_IFACE_ITER_NORMAL, - iwl_mvm_p2p_standalone_iterator, - &is_p2p_standalone); - - if (!is_p2p_standalone) - return false; - } + ieee80211_iterate_active_interfaces_atomic(mvm->hw, + IEEE80211_IFACE_ITER_NORMAL, + iwl_mvm_allow_uapsd_iterator, + &data); - return true; + return data.allow_uapsd; } static bool iwl_mvm_power_is_radar(struct ieee80211_vif *vif) diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c index 3ba62fb2c85e2a23b3f501d4d1ac7a131a4d493e..05715e5af6ab3ad7dbbf1de5021a2faf626aa602 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c @@ -514,6 +514,8 @@ void iwl_mvm_tlc_update_notif(struct iwl_mvm *mvm, link_sta->agg.max_tid_amsdu_len[i] = 1; } + ieee80211_sta_recalc_aggregates(sta); + IWL_DEBUG_RATE(mvm, "AMSDU update. AMSDU size: %d, AMSDU selected size: %d, AMSDU TID bitmap 0x%X\n", le32_to_cpu(notif->amsdu_size), size, @@ -609,6 +611,7 @@ void iwl_mvm_rs_fw_rate_init(struct iwl_mvm *mvm, cpu_to_le16(max_amsdu_len) : 0, }; unsigned int link_id = link_conf->link_id; + struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(mvmsta->vif); int cmd_ver; int ret; @@ -652,7 +655,10 @@ void iwl_mvm_rs_fw_rate_init(struct iwl_mvm *mvm, * since TLC offload works with one mode we can assume * that only vht/ht is used and also set it as station max amsdu */ - sta->deflink.agg.max_amsdu_len = max_amsdu_len; + link_sta->agg.max_amsdu_len = max_amsdu_len; + ieee80211_sta_recalc_aggregates(sta); + + cfg_cmd.max_tx_op = cpu_to_le16(mvmvif->max_tx_op); cmd_ver = iwl_fw_lookup_cmd_ver(mvm->fw, cmd_id, 0); IWL_DEBUG_RATE(mvm, "TLC CONFIG CMD, sta_id=%d, max_ch_width=%d, mode=%d\n", diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rs.h b/drivers/net/wireless/intel/iwlwifi/mvm/rs.h index 376b23b409dcad2c3a9fffd9c1a4ce0d50b71e28..6cd4ec4d8f34411b7c4f53a2f03f468d1f1498b9 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/rs.h +++ b/drivers/net/wireless/intel/iwlwifi/mvm/rs.h @@ -122,13 +122,8 @@ enum { #define LINK_QUAL_AGG_FRAME_LIMIT_DEF (63) #define LINK_QUAL_AGG_FRAME_LIMIT_MAX (63) -/* - * FIXME - various places in firmware API still use u8, - * e.g. LQ command and SCD config command. - * This should be 256 instead. - */ -#define LINK_QUAL_AGG_FRAME_LIMIT_GEN2_DEF (255) -#define LINK_QUAL_AGG_FRAME_LIMIT_GEN2_MAX (255) +#define LINK_QUAL_AGG_FRAME_LIMIT_GEN2_DEF (64) +#define LINK_QUAL_AGG_FRAME_LIMIT_GEN2_MAX (64) #define LINK_QUAL_AGG_FRAME_LIMIT_MIN (0) #define LQ_SIZE 2 /* 2 mode tables: "Active" and "Search" */ diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rx.c b/drivers/net/wireless/intel/iwlwifi/mvm/rx.c index 4fa8066a89b6a196505efd2570c5f1773c2c06cb..b7185ddcca87ed144b2298a434f3e4967ce393c9 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/rx.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/rx.c @@ -1010,6 +1010,9 @@ static void iwl_mvm_update_esr_mode_tpt(struct iwl_mvm *mvm) spin_unlock_bh(&mvmsta->mpdu_counters[q].lock); } + IWL_DEBUG_STATS(mvm, "total Tx MPDUs: %ld. total Rx MPDUs: %ld\n", + total_tx, total_rx); + /* If we don't have enough MPDUs - exit EMLSR */ if (total_tx < IWL_MVM_ENTER_ESR_TPT_THRESH && total_rx < IWL_MVM_ENTER_ESR_TPT_THRESH) { diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c index d78af29281522ee756b3a9a3d412e673d70106ed..5370580f621084e449a89e02b62237f885c44aa6 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c @@ -1954,6 +1954,16 @@ static void iwl_mvm_rx_fill_status(struct iwl_mvm *mvm, iwl_mvm_decode_lsig(skb, phy_data); rx_status->device_timestamp = phy_data->gp2_on_air_rise; + + if (mvm->rx_ts_ptp && mvm->monitor_on) { + u64 adj_time = + iwl_mvm_ptp_get_adj_time(mvm, phy_data->gp2_on_air_rise * NSEC_PER_USEC); + + rx_status->mactime = div64_u64(adj_time, NSEC_PER_USEC); + rx_status->flag |= RX_FLAG_MACTIME_IS_RTAP_TS64; + rx_status->flag &= ~RX_FLAG_MACTIME; + } + rx_status->freq = ieee80211_channel_to_frequency(phy_data->channel, rx_status->band); iwl_mvm_get_signal_strength(mvm, rx_status, rate_n_flags, @@ -2032,7 +2042,6 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, u32 len; u32 pkt_len = iwl_rx_packet_payload_len(pkt); struct ieee80211_sta *sta = NULL; - struct ieee80211_link_sta *link_sta = NULL; struct sk_buff *skb; u8 crypt_len = 0; u8 sta_id = le32_get_bits(desc->status, IWL_RX_MPDU_STATUS_STA_ID); @@ -2185,6 +2194,8 @@ void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi, if (desc->status & cpu_to_le32(IWL_RX_MPDU_STATUS_SRC_STA_FOUND)) { if (!WARN_ON_ONCE(sta_id >= mvm->fw->ucode_capa.num_stations)) { + struct ieee80211_link_sta *link_sta; + sta = rcu_dereference(mvm->fw_id_to_mac_id[sta_id]); if (IS_ERR(sta)) sta = NULL; @@ -2360,7 +2371,6 @@ void iwl_mvm_rx_monitor_no_data(struct iwl_mvm *mvm, struct napi_struct *napi, struct iwl_rx_packet *pkt = rxb_addr(rxb); struct iwl_rx_no_data_ver_3 *desc = (void *)pkt->data; u32 rssi; - u32 info_type; struct ieee80211_sta *sta = NULL; struct sk_buff *skb; struct iwl_mvm_rx_phy_data phy_data; @@ -2373,7 +2383,6 @@ void iwl_mvm_rx_monitor_no_data(struct iwl_mvm *mvm, struct napi_struct *napi, return; rssi = le32_to_cpu(desc->rssi); - info_type = le32_to_cpu(desc->info) & RX_NO_DATA_INFO_TYPE_MSK; phy_data.d0 = desc->phy_info[0]; phy_data.d1 = desc->phy_info[1]; phy_data.phy_info = IWL_RX_MPDU_PHY_TSF_OVERLOAD; @@ -2425,7 +2434,12 @@ void iwl_mvm_rx_monitor_no_data(struct iwl_mvm *mvm, struct napi_struct *napi, /* 0-length PSDU */ rx_status->flag |= RX_FLAG_NO_PSDU; - switch (info_type) { + /* mark as failed PLCP on any errors to skip checks in mac80211 */ + if (le32_get_bits(desc->info, RX_NO_DATA_INFO_ERR_MSK) != + RX_NO_DATA_INFO_ERR_NONE) + rx_status->flag |= RX_FLAG_FAILED_PLCP_CRC; + + switch (le32_get_bits(desc->info, RX_NO_DATA_INFO_TYPE_MSK)) { case RX_NO_DATA_INFO_TYPE_NDP: rx_status->zero_length_psdu_type = IEEE80211_RADIOTAP_ZERO_LEN_PSDU_SOUNDING; @@ -2450,8 +2464,11 @@ void iwl_mvm_rx_monitor_no_data(struct iwl_mvm *mvm, struct napi_struct *napi, * * We mark it as mac header, for upper layers to know where * all radio tap header ends. + * + * Since data doesn't move data while putting data on skb and that is + * the only way we use, data + len is the next place that hdr would be put */ - skb_reset_mac_header(skb); + skb_set_mac_header(skb, skb->len); /* * Override the nss from the rx_vec since the rate_n_flags has diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c index a7ec172eeade8513f5c3a63d6c650a3473a9594b..23a8600553543adb68989f86d7cfd7e447b3e492 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/scan.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/scan.c @@ -208,7 +208,7 @@ static void iwl_mvm_scan_iterator(void *_data, u8 *mac, curr_mvmvif = iwl_mvm_vif_from_mac80211(data->current_vif); - if (vif->type == NL80211_IFTYPE_AP && vif->p2p && + if (ieee80211_vif_type_p2p(vif) == NL80211_IFTYPE_P2P_GO && mvmvif->deflink.phy_ctxt && curr_mvmvif->deflink.phy_ctxt && mvmvif->deflink.phy_ctxt->id != curr_mvmvif->deflink.phy_ctxt->id) data->is_dcm_with_p2p_go = true; @@ -1313,7 +1313,7 @@ static void iwl_mvm_scan_umac_dwell(struct iwl_mvm *mvm, if (IWL_MVM_ADWELL_MAX_BUDGET) cmd->v7.adwell_max_budget = cpu_to_le16(IWL_MVM_ADWELL_MAX_BUDGET); - else if (params->ssids && params->ssids[0].ssid_len) + else if (params->n_ssids && params->ssids[0].ssid_len) cmd->v7.adwell_max_budget = cpu_to_le16(IWL_SCAN_ADWELL_MAX_BUDGET_DIRECTED_SCAN); else @@ -1418,7 +1418,7 @@ iwl_mvm_scan_umac_dwell_v11(struct iwl_mvm *mvm, if (IWL_MVM_ADWELL_MAX_BUDGET) general_params->adwell_max_budget = cpu_to_le16(IWL_MVM_ADWELL_MAX_BUDGET); - else if (params->ssids && params->ssids[0].ssid_len) + else if (params->n_ssids && params->ssids[0].ssid_len) general_params->adwell_max_budget = cpu_to_le16(IWL_SCAN_ADWELL_MAX_BUDGET_DIRECTED_SCAN); else @@ -1730,7 +1730,10 @@ iwl_mvm_umac_scan_fill_6g_chan_list(struct iwl_mvm *mvm, break; } - if (k == idex_b && idex_b < SCAN_BSSID_MAX_SIZE) { + if (k == idex_b && idex_b < SCAN_BSSID_MAX_SIZE && + !WARN_ONCE(!is_valid_ether_addr(scan_6ghz_params[j].bssid), + "scan: invalid BSSID at index %u, index_b=%u\n", + j, idex_b)) { memcpy(&pp->bssid_array[idex_b++], scan_6ghz_params[j].bssid, ETH_ALEN); } @@ -2875,7 +2878,7 @@ static void iwl_mvm_scan_respect_p2p_go_iter(void *_data, u8 *mac, if (vif == data->current_vif) return; - if (vif->type == NL80211_IFTYPE_AP && vif->p2p) { + if (ieee80211_vif_type_p2p(vif) == NL80211_IFTYPE_P2P_GO) { u32 link_id; for (link_id = 0; @@ -3319,10 +3322,11 @@ static int iwl_mvm_umac_scan_abort(struct iwl_mvm *mvm, int type) ret = iwl_mvm_send_cmd_pdu(mvm, WIDE_ID(IWL_ALWAYS_LONG_GROUP, SCAN_ABORT_UMAC), - 0, sizeof(cmd), &cmd); + CMD_SEND_IN_RFKILL, sizeof(cmd), &cmd); if (!ret) mvm->scan_uid_status[uid] = type << IWL_MVM_SCAN_STOPPING_SHIFT; + IWL_DEBUG_SCAN(mvm, "Scan abort: ret=%d\n", ret); return ret; } diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c index 20d4968d692a3664387b14951ebdf01af147cae8..2a8ad718ce789347ce53eb94f0a7f97ab0e1f0a8 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c @@ -2848,7 +2848,12 @@ static int iwl_mvm_fw_baid_op_cmd(struct iwl_mvm *mvm, .action = start ? cpu_to_le32(IWL_RX_BAID_ACTION_ADD) : cpu_to_le32(IWL_RX_BAID_ACTION_REMOVE), }; - u32 cmd_id = WIDE_ID(DATA_PATH_GROUP, RX_BAID_ALLOCATION_CONFIG_CMD); + struct iwl_host_cmd hcmd = { + .id = WIDE_ID(DATA_PATH_GROUP, RX_BAID_ALLOCATION_CONFIG_CMD), + .flags = CMD_SEND_IN_RFKILL, + .len[0] = sizeof(cmd), + .data[0] = &cmd, + }; int ret; BUILD_BUG_ON(sizeof(struct iwl_rx_baid_cfg_resp) != sizeof(baid)); @@ -2860,7 +2865,7 @@ static int iwl_mvm_fw_baid_op_cmd(struct iwl_mvm *mvm, cmd.alloc.ssn = cpu_to_le16(ssn); cmd.alloc.win_size = cpu_to_le16(buf_size); baid = -EIO; - } else if (iwl_fw_lookup_cmd_ver(mvm->fw, cmd_id, 1) == 1) { + } else if (iwl_fw_lookup_cmd_ver(mvm->fw, hcmd.id, 1) == 1) { cmd.remove_v1.baid = cpu_to_le32(baid); BUILD_BUG_ON(sizeof(cmd.remove_v1) > sizeof(cmd.remove)); } else { @@ -2869,8 +2874,7 @@ static int iwl_mvm_fw_baid_op_cmd(struct iwl_mvm *mvm, cmd.remove.tid = cpu_to_le32(tid); } - ret = iwl_mvm_send_cmd_pdu_status(mvm, cmd_id, sizeof(cmd), - &cmd, &baid); + ret = iwl_mvm_send_cmd_status(mvm, &hcmd, &baid); if (ret) return ret; @@ -4429,6 +4433,7 @@ void iwl_mvm_count_mpdu(struct iwl_mvm_sta *mvm_sta, u8 fw_sta_id, u32 count, bool tx, int queue) { struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(mvm_sta->vif); + struct iwl_mvm *mvm = mvmvif->mvm; struct iwl_mvm_tpt_counter *queue_counter; struct iwl_mvm_mpdu_counter *link_counter; u32 total_mpdus = 0; @@ -4465,6 +4470,8 @@ void iwl_mvm_count_mpdu(struct iwl_mvm_sta *mvm_sta, u8 fw_sta_id, u32 count, memset(queue_counter->per_link, 0, sizeof(queue_counter->per_link)); queue_counter->window_start = jiffies; + + IWL_DEBUG_STATS(mvm, "MPDU counters are cleared\n"); } for (int i = 0; i < IWL_MVM_FW_MAX_LINK_ID; i++) diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.h b/drivers/net/wireless/intel/iwlwifi/mvm/sta.h index 264f1f9394b6de26af044f5d7dc971ad73d7e7c7..754a05a8c189bcb7e2af33bbe98d549e8547dc5a 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.h +++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.h @@ -662,6 +662,11 @@ int iwl_mvm_mld_update_sta(struct iwl_mvm *mvm, struct ieee80211_vif *vif, struct ieee80211_sta *sta); int iwl_mvm_mld_rm_sta(struct iwl_mvm *mvm, struct ieee80211_vif *vif, struct ieee80211_sta *sta); +void iwl_mvm_mld_free_sta_link(struct iwl_mvm *mvm, + struct iwl_mvm_sta *mvm_sta, + struct iwl_mvm_link_sta *mvm_sta_link, + unsigned int link_id, + bool is_in_fw); int iwl_mvm_mld_rm_sta_id(struct iwl_mvm *mvm, u8 sta_id); int iwl_mvm_mld_update_sta_links(struct iwl_mvm *mvm, struct ieee80211_vif *vif, diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tdls.c b/drivers/net/wireless/intel/iwlwifi/mvm/tdls.c index e7d5f4ebeb259623d34ff457b47fda06bc2ff7a1..3d25ff5cd7e8abe4e99f0dde6d0c89d481e0e79e 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/tdls.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/tdls.c @@ -2,7 +2,7 @@ /* * Copyright (C) 2014 Intel Mobile Communications GmbH * Copyright (C) 2017 Intel Deutschland GmbH - * Copyright (C) 2018-2020, 2022-2023 Intel Corporation + * Copyright (C) 2018-2020, 2022-2024 Intel Corporation */ #include #include "mvm.h" @@ -151,7 +151,7 @@ void iwl_mvm_mac_mgd_protect_tdls_discover(struct ieee80211_hw *hw, u32 duration = 2 * vif->bss_conf.dtim_period * vif->bss_conf.beacon_int; /* Protect the session to hear the TDLS setup response on the channel */ - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_SESSION_PROT_CMD)) iwl_mvm_schedule_session_protection(mvm, vif, duration, @@ -159,7 +159,6 @@ void iwl_mvm_mac_mgd_protect_tdls_discover(struct ieee80211_hw *hw, else iwl_mvm_protect_session(mvm, vif, duration, duration, 100, true); - mutex_unlock(&mvm->mutex); } static const char * @@ -460,21 +459,21 @@ void iwl_mvm_tdls_ch_switch_work(struct work_struct *work) int ret; mvm = container_of(work, struct iwl_mvm, tdls_cs.dwork.work); - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); /* called after an active channel switch has finished or timed-out */ iwl_mvm_tdls_update_cs_state(mvm, IWL_MVM_TDLS_SW_IDLE); /* station might be gone, in that case do nothing */ if (mvm->tdls_cs.peer.sta_id == IWL_MVM_INVALID_STA) - goto out; + return; sta = rcu_dereference_protected( mvm->fw_id_to_mac_id[mvm->tdls_cs.peer.sta_id], lockdep_is_held(&mvm->mutex)); /* the station may not be here, but if it is, it must be a TDLS peer */ if (!sta || IS_ERR(sta) || WARN_ON(!sta->tdls)) - goto out; + return; mvmsta = iwl_mvm_sta_from_mac80211(sta); vif = mvmsta->vif; @@ -493,8 +492,6 @@ void iwl_mvm_tdls_ch_switch_work(struct work_struct *work) /* retry after a DTIM if we failed sending now */ delay = TU_TO_MS(vif->bss_conf.dtim_period * vif->bss_conf.beacon_int); schedule_delayed_work(&mvm->tdls_cs.dwork, msecs_to_jiffies(delay)); -out: - mutex_unlock(&mvm->mutex); } int @@ -509,7 +506,7 @@ iwl_mvm_tdls_channel_switch(struct ieee80211_hw *hw, unsigned int delay; int ret; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); IWL_DEBUG_TDLS(mvm, "TDLS channel switch with %pM ch %d width %d\n", sta->addr, chandef->chan->center_freq, chandef->width); @@ -519,8 +516,7 @@ iwl_mvm_tdls_channel_switch(struct ieee80211_hw *hw, IWL_DEBUG_TDLS(mvm, "Existing peer. Can't start switch with %pM\n", sta->addr); - ret = -EBUSY; - goto out; + return -EBUSY; } ret = iwl_mvm_tdls_config_channel_switch(mvm, vif, @@ -529,17 +525,15 @@ iwl_mvm_tdls_channel_switch(struct ieee80211_hw *hw, oper_class, chandef, 0, 0, 0, tmpl_skb, ch_sw_tm_ie); if (ret) - goto out; + return ret; /* * Mark the peer as "in tdls switch" for this vif. We only allow a * single such peer per vif. */ mvm->tdls_cs.peer.skb = skb_copy(tmpl_skb, GFP_KERNEL); - if (!mvm->tdls_cs.peer.skb) { - ret = -ENOMEM; - goto out; - } + if (!mvm->tdls_cs.peer.skb) + return -ENOMEM; mvmsta = iwl_mvm_sta_from_mac80211(sta); mvm->tdls_cs.peer.sta_id = mvmsta->deflink.sta_id; @@ -556,10 +550,7 @@ iwl_mvm_tdls_channel_switch(struct ieee80211_hw *hw, vif->bss_conf.beacon_int); mod_delayed_work(system_wq, &mvm->tdls_cs.dwork, msecs_to_jiffies(delay)); - -out: - mutex_unlock(&mvm->mutex); - return ret; + return 0; } void iwl_mvm_tdls_cancel_channel_switch(struct ieee80211_hw *hw, @@ -626,7 +617,7 @@ iwl_mvm_tdls_recv_channel_switch(struct ieee80211_hw *hw, params->action_code == WLAN_TDLS_CHANNEL_SWITCH_REQUEST ? "REQ" : "RESP"; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); IWL_DEBUG_TDLS(mvm, "Received TDLS ch switch action %s from %pM status %d\n", @@ -670,5 +661,4 @@ retry: 1024 / 1000; mod_delayed_work(system_wq, &mvm->tdls_cs.dwork, msecs_to_jiffies(delay)); - mutex_unlock(&mvm->mutex); } diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tests/links.c b/drivers/net/wireless/intel/iwlwifi/mvm/tests/links.c index f49e3c98b1ba44cc27de70360ffc206cb1a4d06e..47b8e7b64ead46195ba4edf7945d89d5736cd102 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/tests/links.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/tests/links.c @@ -208,6 +208,7 @@ static void setup_link_conf(struct kunit *test) bss_load->channel_util = params->channel_util; rcu_assign_pointer(bss.ies, ies); + rcu_assign_pointer(bss.beacon_ies, ies); } static void test_link_grading(struct kunit *test) @@ -393,9 +394,6 @@ static void test_valid_link_pair(struct kunit *test) chandef_a.width = params->cw_a ?: NL80211_CHAN_WIDTH_20; chandef_b.width = params->cw_b ?: NL80211_CHAN_WIDTH_20; -#ifdef CONFIG_IWLWIFI_SUPPORT_DEBUG_OVERRIDES - trans->dbg_cfg = default_dbg_config; -#endif mvm.trans = trans; mvm.last_bt_notif.wifi_loss_low_rssi = params->bt; diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tt.c b/drivers/net/wireless/intel/iwlwifi/mvm/tt.c index 61a4638d1be2f2fdf952db252bb3755359f85eae..45b431ffbc977d51251abe5e9bf8b23dbb92c83f 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/tt.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/tt.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause /* - * Copyright (C) 2012-2014, 2019-2022 Intel Corporation + * Copyright (C) 2012-2014, 2019-2022, 2024 Intel Corporation * Copyright (C) 2013-2014 Intel Mobile Communications GmbH * Copyright (C) 2015-2016 Intel Deutschland GmbH */ @@ -618,48 +618,35 @@ static int iwl_mvm_tzone_get_temp(struct thermal_zone_device *device, int ret; int temp; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (!iwl_mvm_firmware_running(mvm) || - mvm->fwrt.cur_fw_img != IWL_UCODE_REGULAR) { - ret = -ENODATA; - goto out; - } + mvm->fwrt.cur_fw_img != IWL_UCODE_REGULAR) + return -ENODATA; ret = iwl_mvm_get_temp(mvm, &temp); if (ret) - goto out; + return ret; *temperature = temp * 1000; - -out: - mutex_unlock(&mvm->mutex); - return ret; + return 0; } static int iwl_mvm_tzone_set_trip_temp(struct thermal_zone_device *device, int trip, int temp) { struct iwl_mvm *mvm = thermal_zone_device_priv(device); - int ret; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (!iwl_mvm_firmware_running(mvm) || - mvm->fwrt.cur_fw_img != IWL_UCODE_REGULAR) { - ret = -EIO; - goto out; - } + mvm->fwrt.cur_fw_img != IWL_UCODE_REGULAR) + return -EIO; - if ((temp / 1000) > S16_MAX) { - ret = -EINVAL; - goto out; - } + if ((temp / 1000) > S16_MAX) + return -EINVAL; - ret = iwl_mvm_send_temp_report_ths_cmd(mvm); -out: - mutex_unlock(&mvm->mutex); - return ret; + return iwl_mvm_send_temp_report_ths_cmd(mvm); } static struct thermal_zone_device_ops tzone_ops = { @@ -733,27 +720,18 @@ static int iwl_mvm_tcool_set_cur_state(struct thermal_cooling_device *cdev, unsigned long new_state) { struct iwl_mvm *mvm = (struct iwl_mvm *)(cdev->devdata); - int ret; - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (!iwl_mvm_firmware_running(mvm) || - mvm->fwrt.cur_fw_img != IWL_UCODE_REGULAR) { - ret = -EIO; - goto unlock; - } + mvm->fwrt.cur_fw_img != IWL_UCODE_REGULAR) + return -EIO; - if (new_state >= ARRAY_SIZE(iwl_mvm_cdev_budgets)) { - ret = -EINVAL; - goto unlock; - } - - ret = iwl_mvm_ctdp_command(mvm, CTDP_CMD_OPERATION_START, - new_state); + if (new_state >= ARRAY_SIZE(iwl_mvm_cdev_budgets)) + return -EINVAL; -unlock: - mutex_unlock(&mvm->mutex); - return ret; + return iwl_mvm_ctdp_command(mvm, CTDP_CMD_OPERATION_START, + new_state); } static const struct thermal_cooling_device_ops tcooling_ops = { diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c index 1d695ece93e9e921213f6681d02c78f4af6cfac5..599550f311deaa2429f4c7220645601063f08215 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c @@ -12,7 +12,7 @@ #include #include "iwl-trans.h" -#include "iwl-eeprom-parse.h" +#include "iwl-nvm-utils.h" #include "mvm.h" #include "sta.h" #include "time-sync.h" diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/utils.c b/drivers/net/wireless/intel/iwlwifi/mvm/utils.c index 47283a358ffddc45fd065f412ee94bd163e921a7..0e5fa8374103f4f50adce2bf9dd27db510596937 100644 --- a/drivers/net/wireless/intel/iwlwifi/mvm/utils.c +++ b/drivers/net/wireless/intel/iwlwifi/mvm/utils.c @@ -892,7 +892,7 @@ static void iwl_mvm_tcm_iter(void *_data, u8 *mac, struct ieee80211_vif *vif) static void iwl_mvm_tcm_results(struct iwl_mvm *mvm) { - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); ieee80211_iterate_active_interfaces( mvm->hw, IEEE80211_IFACE_ITER_NORMAL, @@ -900,8 +900,6 @@ static void iwl_mvm_tcm_results(struct iwl_mvm *mvm) if (fw_has_capa(&mvm->fw->ucode_capa, IWL_UCODE_TLV_CAPA_UMAC_SCAN)) iwl_mvm_config_scan(mvm); - - mutex_unlock(&mvm->mutex); } static void iwl_mvm_tcm_uapsd_nonagg_detected_wk(struct work_struct *wk) @@ -1130,10 +1128,9 @@ void iwl_mvm_recalc_tcm(struct iwl_mvm *mvm) spin_unlock(&mvm->tcm.lock); if (handle_uapsd && iwl_mvm_has_new_rx_api(mvm)) { - mutex_lock(&mvm->mutex); + guard(mvm)(mvm); if (iwl_mvm_request_statistics(mvm, true)) handle_uapsd = false; - mutex_unlock(&mvm->mutex); } spin_lock(&mvm->tcm.lock); diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c index fed2754be68029fd21afa6442759e54ea8f2cc25..d6da25e24818f3377a8ac4a01edb5567c8b88f9c 100644 --- a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c +++ b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c @@ -997,32 +997,6 @@ VISIBLE_IF_IWLWIFI_KUNIT const struct iwl_dev_info iwl_dev_info_table[] = { IWL_CFG_160, IWL_CFG_ANY, IWL_CFG_CDB, iwlax411_2ax_cfg_so_gf4_a0, iwl_ax411_name), -/* Bz */ -/* FIXME: need to change the naming according to the actual CRF */ - _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, - IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY, - IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, - IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, - iwl_cfg_bz, iwl_fm_name), - - _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, - IWL_CFG_MAC_TYPE_BZ_W, IWL_CFG_ANY, - IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, - IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, - iwl_cfg_bz, iwl_fm_name), - -/* Ga (Gl) */ - _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, - IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY, - IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, IWL_CFG_ANY, - IWL_CFG_320, IWL_CFG_ANY, IWL_CFG_NO_CDB, - iwl_cfg_gl, iwl_gl_name), - _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, - IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY, - IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, IWL_CFG_ANY, - IWL_CFG_NO_320, IWL_CFG_ANY, IWL_CFG_NO_CDB, - iwl_cfg_gl, iwl_mtp_name), - /* SoF with JF2 */ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_MAC_TYPE_SOF, IWL_CFG_ANY, @@ -1103,6 +1077,32 @@ VISIBLE_IF_IWLWIFI_KUNIT const struct iwl_dev_info iwl_dev_info_table[] = { IWL_CFG_NO_160, IWL_CFG_CORES_BT, IWL_CFG_NO_CDB, iwlax210_2ax_cfg_so_jf_b0, iwl9462_name), +/* Bz */ +/* FIXME: need to change the naming according to the actual CRF */ + _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, + IWL_CFG_MAC_TYPE_BZ, IWL_CFG_ANY, + IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, + IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, + iwl_cfg_bz, iwl_fm_name), + + _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, + IWL_CFG_MAC_TYPE_BZ_W, IWL_CFG_ANY, + IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, + IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_ANY, + iwl_cfg_bz, iwl_fm_name), + +/* Ga (Gl) */ + _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, + IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY, + IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, IWL_CFG_ANY, + IWL_CFG_320, IWL_CFG_ANY, IWL_CFG_NO_CDB, + iwl_cfg_gl, iwl_gl_name), + _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, + IWL_CFG_MAC_TYPE_GL, IWL_CFG_ANY, + IWL_CFG_RF_TYPE_FM, IWL_CFG_ANY, IWL_CFG_ANY, + IWL_CFG_NO_320, IWL_CFG_ANY, IWL_CFG_NO_CDB, + iwl_cfg_gl, iwl_mtp_name), + /* Sc */ _IWL_DEV_INFO(IWL_CFG_ANY, IWL_CFG_ANY, IWL_CFG_MAC_TYPE_SC, IWL_CFG_ANY, @@ -1476,6 +1476,8 @@ static int iwl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) if (!iwl_trans->name) iwl_trans->name = iwl_trans->cfg->name; + IWL_INFO(iwl_trans, "Detected %s\n", iwl_trans->name); + if (iwl_trans->trans_cfg->mq_rx_supported) { if (WARN_ON(!iwl_trans->cfg->num_rbds)) { ret = -EINVAL; diff --git a/drivers/net/wireless/marvell/mwifiex/cfg80211.c b/drivers/net/wireless/marvell/mwifiex/cfg80211.c index b909a7665e9cc17c21b4c29afd2aabac1f878291..155eb0fab12a48b568ab189c45819d8df7bd3045 100644 --- a/drivers/net/wireless/marvell/mwifiex/cfg80211.c +++ b/drivers/net/wireless/marvell/mwifiex/cfg80211.c @@ -926,6 +926,8 @@ mwifiex_init_new_priv_params(struct mwifiex_private *priv, return -EOPNOTSUPP; } + priv->bss_num = mwifiex_get_unused_bss_num(adapter, priv->bss_type); + spin_lock_irqsave(&adapter->main_proc_lock, flags); adapter->main_locked = false; spin_unlock_irqrestore(&adapter->main_proc_lock, flags); diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/main.c b/drivers/net/wireless/mediatek/mt76/mt7615/main.c index 0971c164b57e926d2d22dd1ef4f0559d45420db2..c27acaf0eb1cf7e9698adec73868867f349620e8 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7615/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7615/main.c @@ -1326,6 +1326,10 @@ static void mt7615_set_rekey_data(struct ieee80211_hw *hw, #endif /* CONFIG_PM */ const struct ieee80211_ops mt7615_ops = { + .add_chanctx = ieee80211_emulate_add_chanctx, + .remove_chanctx = ieee80211_emulate_remove_chanctx, + .change_chanctx = ieee80211_emulate_change_chanctx, + .switch_vif_chanctx = ieee80211_emulate_switch_vif_chanctx, .tx = mt7615_tx, .start = mt7615_start, .stop = mt7615_stop, diff --git a/drivers/net/wireless/microchip/wilc1000/cfg80211.c b/drivers/net/wireless/microchip/wilc1000/cfg80211.c index 7d9fb9f2d52799b1e1c92ae285943bdd921114a0..ba02e6cfd3ae43f1a6740fc88981e1cb919a2415 100644 --- a/drivers/net/wireless/microchip/wilc1000/cfg80211.c +++ b/drivers/net/wireless/microchip/wilc1000/cfg80211.c @@ -237,11 +237,12 @@ static int set_channel(struct wiphy *wiphy, struct wilc_vif *vif; u32 channelnum; int result; + int srcu_idx; - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wl->srcu); vif = wilc_get_wl_to_vif(wl); if (IS_ERR(vif)) { - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); return PTR_ERR(vif); } @@ -252,7 +253,7 @@ static int set_channel(struct wiphy *wiphy, if (result) netdev_err(vif->ndev, "Error in setting channel\n"); - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); return result; } @@ -805,8 +806,9 @@ static int set_wiphy_params(struct wiphy *wiphy, u32 changed) struct wilc *wl = wiphy_priv(wiphy); struct wilc_vif *vif; struct wilc_priv *priv; + int srcu_idx; - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wl->srcu); vif = wilc_get_wl_to_vif(wl); if (IS_ERR(vif)) goto out; @@ -861,7 +863,7 @@ static int set_wiphy_params(struct wiphy *wiphy, u32 changed) netdev_err(priv->dev, "Error in setting WIPHY PARAMS\n"); out: - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); return ret; } @@ -1537,19 +1539,20 @@ static struct wireless_dev *add_virtual_intf(struct wiphy *wiphy, if (type == NL80211_IFTYPE_MONITOR) { struct net_device *ndev; + int srcu_idx; - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wl->srcu); vif = wilc_get_vif_from_type(wl, WILC_AP_MODE); if (!vif) { vif = wilc_get_vif_from_type(wl, WILC_GO_MODE); if (!vif) { - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); goto validate_interface; } } if (vif->monitor_flag) { - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); goto validate_interface; } @@ -1557,12 +1560,12 @@ static struct wireless_dev *add_virtual_intf(struct wiphy *wiphy, if (ndev) { vif->monitor_flag = 1; } else { - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); return ERR_PTR(-EINVAL); } wdev = &vif->priv.wdev; - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); return wdev; } @@ -1610,7 +1613,7 @@ static int del_virtual_intf(struct wiphy *wiphy, struct wireless_dev *wdev) list_del_rcu(&vif->list); wl->vif_num--; mutex_unlock(&wl->vif_mutex); - synchronize_rcu(); + synchronize_srcu(&wl->srcu); return 0; } @@ -1635,23 +1638,25 @@ static void wilc_set_wakeup(struct wiphy *wiphy, bool enabled) { struct wilc *wl = wiphy_priv(wiphy); struct wilc_vif *vif; + int srcu_idx; - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wl->srcu); vif = wilc_get_wl_to_vif(wl); if (IS_ERR(vif)) { - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); return; } netdev_info(vif->ndev, "cfg set wake up = %d\n", enabled); wilc_set_wowlan_trigger(vif, enabled); - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); } static int set_tx_power(struct wiphy *wiphy, struct wireless_dev *wdev, enum nl80211_tx_power_setting type, int mbm) { int ret; + int srcu_idx; s32 tx_power = MBM_TO_DBM(mbm); struct wilc *wl = wiphy_priv(wiphy); struct wilc_vif *vif; @@ -1659,10 +1664,10 @@ static int set_tx_power(struct wiphy *wiphy, struct wireless_dev *wdev, if (!wl->initialized) return -EIO; - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wl->srcu); vif = wilc_get_wl_to_vif(wl); if (IS_ERR(vif)) { - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); return -EINVAL; } @@ -1674,7 +1679,7 @@ static int set_tx_power(struct wiphy *wiphy, struct wireless_dev *wdev, ret = wilc_set_tx_power(vif, tx_power); if (ret) netdev_err(vif->ndev, "Failed to set tx power\n"); - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); return ret; } @@ -1757,6 +1762,7 @@ static void wlan_init_locks(struct wilc *wl) init_completion(&wl->cfg_event); init_completion(&wl->sync_event); init_completion(&wl->txq_thread_started); + init_srcu_struct(&wl->srcu); } void wlan_deinit_locks(struct wilc *wilc) @@ -1767,13 +1773,13 @@ void wlan_deinit_locks(struct wilc *wilc) mutex_destroy(&wilc->txq_add_to_head_cs); mutex_destroy(&wilc->vif_mutex); mutex_destroy(&wilc->deinit_lock); + cleanup_srcu_struct(&wilc->srcu); } int wilc_cfg80211_init(struct wilc **wilc, struct device *dev, int io_type, const struct wilc_hif_func *ops) { struct wilc *wl; - struct wilc_vif *vif; int ret, i; wl = wilc_create_wiphy(dev); @@ -1802,18 +1808,9 @@ int wilc_cfg80211_init(struct wilc **wilc, struct device *dev, int io_type, ret = -ENOMEM; goto free_cfg; } - vif = wilc_netdev_ifc_init(wl, "wlan%d", WILC_STATION_MODE, - NL80211_IFTYPE_STATION, false); - if (IS_ERR(vif)) { - ret = PTR_ERR(vif); - goto free_hq; - } return 0; -free_hq: - destroy_workqueue(wl->hif_workqueue); - free_cfg: wilc_wlan_cfg_deinit(wl); diff --git a/drivers/net/wireless/microchip/wilc1000/fw.h b/drivers/net/wireless/microchip/wilc1000/fw.h index 5c5cac4aab029bf0bf9b985a01f03527500cff14..7a930e89614caf9406c64d06dc892ed728e2d6b5 100644 --- a/drivers/net/wireless/microchip/wilc1000/fw.h +++ b/drivers/net/wireless/microchip/wilc1000/fw.h @@ -13,6 +13,12 @@ #define WILC_MAX_RATES_SUPPORTED 12 #define WILC_MAX_NUM_PMKIDS 16 #define WILC_MAX_NUM_SCANNED_CH 14 +#define WILC_NVMEM_MAX_NUM_BANK 6 +#define WILC_NVMEM_BANK_BASE 0x30000000 +#define WILC_NVMEM_LOW_BANK_OFFSET 0x102c +#define WILC_NVMEM_HIGH_BANK_OFFSET 0x1380 +#define WILC_NVMEM_IS_BANK_USED BIT(31) +#define WILC_NVMEM_IS_BANK_INVALID BIT(30) struct wilc_assoc_resp { __le16 capab_info; @@ -127,4 +133,11 @@ struct wilc_external_auth_param { __le32 key_mgmt_suites; __le16 status; } __packed; + +static inline u32 get_bank_offset_from_bank_index(unsigned int i) +{ + return (((i) < 2) ? WILC_NVMEM_LOW_BANK_OFFSET + ((i) * 32) : + WILC_NVMEM_HIGH_BANK_OFFSET + ((i) - 2) * 16); +} + #endif diff --git a/drivers/net/wireless/microchip/wilc1000/hif.c b/drivers/net/wireless/microchip/wilc1000/hif.c index 919de6ffb8217c54375df09f8ecde8e1bfa997bf..d67293142ffb5149ff09fa7926e73859abe219c5 100644 --- a/drivers/net/wireless/microchip/wilc1000/hif.c +++ b/drivers/net/wireless/microchip/wilc1000/hif.c @@ -1293,7 +1293,7 @@ int wilc_get_mac_address(struct wilc_vif *vif, u8 *mac_addr) return result; } -int wilc_set_mac_address(struct wilc_vif *vif, u8 *mac_addr) +int wilc_set_mac_address(struct wilc_vif *vif, const u8 *mac_addr) { struct wid wid; int result; @@ -1301,7 +1301,7 @@ int wilc_set_mac_address(struct wilc_vif *vif, u8 *mac_addr) wid.id = WID_MAC_ADDR; wid.type = WID_STR; wid.size = ETH_ALEN; - wid.val = mac_addr; + wid.val = (u8 *)mac_addr; result = wilc_send_config_pkt(vif, WILC_SET_CFG, &wid, 1); if (result) @@ -1570,11 +1570,12 @@ void wilc_network_info_received(struct wilc *wilc, u8 *buffer, u32 length) struct host_if_drv *hif_drv; struct host_if_msg *msg; struct wilc_vif *vif; + int srcu_idx; int result; int id; id = get_unaligned_le32(&buffer[length - 4]); - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wilc->srcu); vif = wilc_get_vif_from_idx(wilc, id); if (!vif) goto out; @@ -1593,7 +1594,7 @@ void wilc_network_info_received(struct wilc *wilc, u8 *buffer, u32 length) msg->body.net_info.rssi = buffer[8]; msg->body.net_info.mgmt = kmemdup(&buffer[9], msg->body.net_info.frame_len, - GFP_ATOMIC); + GFP_KERNEL); if (!msg->body.net_info.mgmt) { kfree(msg); goto out; @@ -1606,7 +1607,7 @@ void wilc_network_info_received(struct wilc *wilc, u8 *buffer, u32 length) kfree(msg); } out: - rcu_read_unlock(); + srcu_read_unlock(&wilc->srcu, srcu_idx); } void wilc_gnrl_async_info_received(struct wilc *wilc, u8 *buffer, u32 length) @@ -1614,13 +1615,14 @@ void wilc_gnrl_async_info_received(struct wilc *wilc, u8 *buffer, u32 length) struct host_if_drv *hif_drv; struct host_if_msg *msg; struct wilc_vif *vif; + int srcu_idx; int result; int id; mutex_lock(&wilc->deinit_lock); id = get_unaligned_le32(&buffer[length - 4]); - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wilc->srcu); vif = wilc_get_vif_from_idx(wilc, id); if (!vif) goto out; @@ -1647,7 +1649,7 @@ void wilc_gnrl_async_info_received(struct wilc *wilc, u8 *buffer, u32 length) kfree(msg); } out: - rcu_read_unlock(); + srcu_read_unlock(&wilc->srcu, srcu_idx); mutex_unlock(&wilc->deinit_lock); } @@ -1655,11 +1657,12 @@ void wilc_scan_complete_received(struct wilc *wilc, u8 *buffer, u32 length) { struct host_if_drv *hif_drv; struct wilc_vif *vif; + int srcu_idx; int result; int id; id = get_unaligned_le32(&buffer[length - 4]); - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wilc->srcu); vif = wilc_get_vif_from_idx(wilc, id); if (!vif) goto out; @@ -1684,7 +1687,7 @@ void wilc_scan_complete_received(struct wilc *wilc, u8 *buffer, u32 length) } } out: - rcu_read_unlock(); + srcu_read_unlock(&wilc->srcu, srcu_idx); } int wilc_remain_on_channel(struct wilc_vif *vif, u64 cookie, u16 chan, diff --git a/drivers/net/wireless/microchip/wilc1000/hif.h b/drivers/net/wireless/microchip/wilc1000/hif.h index 0d380586b1d9c91b71a37e9774ea6378c9276b5b..96eeaf31d23777e0392699240479b341529bfd42 100644 --- a/drivers/net/wireless/microchip/wilc1000/hif.h +++ b/drivers/net/wireless/microchip/wilc1000/hif.h @@ -167,7 +167,7 @@ int wilc_add_rx_gtk(struct wilc_vif *vif, const u8 *rx_gtk, u8 gtk_key_len, u8 cipher_mode); int wilc_set_pmkid_info(struct wilc_vif *vif, struct wilc_pmkid_attr *pmkid); int wilc_get_mac_address(struct wilc_vif *vif, u8 *mac_addr); -int wilc_set_mac_address(struct wilc_vif *vif, u8 *mac_addr); +int wilc_set_mac_address(struct wilc_vif *vif, const u8 *mac_addr); int wilc_set_join_req(struct wilc_vif *vif, u8 *bssid, const u8 *ies, size_t ies_len); int wilc_disconnect(struct wilc_vif *vif); diff --git a/drivers/net/wireless/microchip/wilc1000/netdev.c b/drivers/net/wireless/microchip/wilc1000/netdev.c index 73f56f7b002bf373277804db28c18cc8a4338a67..9ecf3fb29b558f611e3858d07771cb8fb9b7d64d 100644 --- a/drivers/net/wireless/microchip/wilc1000/netdev.c +++ b/drivers/net/wireless/microchip/wilc1000/netdev.c @@ -127,28 +127,30 @@ void wilc_wlan_set_bssid(struct net_device *wilc_netdev, const u8 *bssid, int wilc_wlan_get_num_conn_ifcs(struct wilc *wilc) { + int srcu_idx; u8 ret_val = 0; struct wilc_vif *vif; - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wilc->srcu); wilc_for_each_vif(wilc, vif) { if (!is_zero_ether_addr(vif->bssid)) ret_val++; } - rcu_read_unlock(); + srcu_read_unlock(&wilc->srcu, srcu_idx); return ret_val; } static void wilc_wake_tx_queues(struct wilc *wl) { + int srcu_idx; struct wilc_vif *ifc; - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wl->srcu); wilc_for_each_vif(wl, ifc) { if (ifc->mac_opened && netif_queue_stopped(ifc->ndev)) netif_wake_queue(ifc->ndev); } - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); } static int wilc_txq_task(void *vp) @@ -588,7 +590,6 @@ static int wilc_mac_open(struct net_device *ndev) struct wilc *wl = vif->wilc; int ret = 0; struct mgmt_frame_regs mgmt_regs = {}; - u8 addr[ETH_ALEN] __aligned(2); if (!wl || !wl->dev) { netdev_err(ndev, "device not ready\n"); @@ -607,25 +608,19 @@ static int wilc_mac_open(struct net_device *ndev) return ret; } - wilc_set_operation_mode(vif, wilc_get_vif_idx(vif), vif->iftype, - vif->idx); - - if (is_valid_ether_addr(ndev->dev_addr)) { - ether_addr_copy(addr, ndev->dev_addr); - wilc_set_mac_address(vif, addr); - } else { - wilc_get_mac_address(vif, addr); - eth_hw_addr_set(ndev, addr); - } netdev_dbg(ndev, "Mac address: %pM\n", ndev->dev_addr); - - if (!is_valid_ether_addr(ndev->dev_addr)) { - netdev_err(ndev, "Wrong MAC address\n"); + ret = wilc_set_mac_address(vif, ndev->dev_addr); + if (ret) { + netdev_err(ndev, "Failed to enforce MAC address in chip"); wilc_deinit_host_int(ndev); - wilc_wlan_deinitialize(ndev); - return -EINVAL; + if (!wl->open_ifcs) + wilc_wlan_deinitialize(ndev); + return ret; } + wilc_set_operation_mode(vif, wilc_get_vif_idx(vif), vif->iftype, + vif->idx); + mgmt_regs.interface_stypes = vif->mgmt_reg_stypes; /* so we detect a change */ vif->mgmt_reg_stypes = 0; @@ -653,6 +648,7 @@ static int wilc_set_mac_addr(struct net_device *dev, void *p) struct sockaddr *addr = (struct sockaddr *)p; unsigned char mac_addr[ETH_ALEN]; struct wilc_vif *tmp_vif; + int srcu_idx; if (!is_valid_ether_addr(addr->sa_data)) return -EADDRNOTAVAIL; @@ -664,21 +660,21 @@ static int wilc_set_mac_addr(struct net_device *dev, void *p) /* Verify MAC Address is not already in use: */ - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wilc->srcu); wilc_for_each_vif(wilc, tmp_vif) { wilc_get_mac_address(tmp_vif, mac_addr); if (ether_addr_equal(addr->sa_data, mac_addr)) { if (vif != tmp_vif) { - rcu_read_unlock(); + srcu_read_unlock(&wilc->srcu, srcu_idx); return -EADDRNOTAVAIL; } - rcu_read_unlock(); + srcu_read_unlock(&wilc->srcu, srcu_idx); return 0; } } - rcu_read_unlock(); + srcu_read_unlock(&wilc->srcu, srcu_idx); - result = wilc_set_mac_address(vif, (u8 *)addr->sa_data); + result = wilc_set_mac_address(vif, addr->sa_data); if (result) return result; @@ -764,14 +760,15 @@ netdev_tx_t wilc_mac_xmit(struct sk_buff *skb, struct net_device *ndev) wilc_tx_complete); if (queue_count > FLOW_CONTROL_UPPER_THRESHOLD) { + int srcu_idx; struct wilc_vif *vif; - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wilc->srcu); wilc_for_each_vif(wilc, vif) { if (vif->mac_opened) netif_stop_queue(vif->ndev); } - rcu_read_unlock(); + srcu_read_unlock(&wilc->srcu, srcu_idx); } return NETDEV_TX_OK; @@ -815,12 +812,13 @@ void wilc_frmw_to_host(struct wilc *wilc, u8 *buff, u32 size, unsigned int frame_len = 0; struct wilc_vif *vif; struct sk_buff *skb; + int srcu_idx; int stats; if (!wilc) return; - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wilc->srcu); wilc_netdev = get_if_handler(wilc, buff); if (!wilc_netdev) goto out; @@ -848,14 +846,15 @@ void wilc_frmw_to_host(struct wilc *wilc, u8 *buff, u32 size, netdev_dbg(wilc_netdev, "netif_rx ret value is: %d\n", stats); } out: - rcu_read_unlock(); + srcu_read_unlock(&wilc->srcu, srcu_idx); } void wilc_wfi_mgmt_rx(struct wilc *wilc, u8 *buff, u32 size, bool is_auth) { + int srcu_idx; struct wilc_vif *vif; - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wilc->srcu); wilc_for_each_vif(wilc, vif) { struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)buff; u16 type = le16_to_cpup((__le16 *)buff); @@ -876,7 +875,7 @@ void wilc_wfi_mgmt_rx(struct wilc *wilc, u8 *buff, u32 size, bool is_auth) if (vif->monitor_flag) wilc_wfi_monitor_rx(wilc->monitor_dev, buff, size); } - rcu_read_unlock(); + srcu_read_unlock(&wilc->srcu, srcu_idx); } static const struct net_device_ops wilc_netdev_ops = { @@ -906,7 +905,7 @@ void wilc_netdev_cleanup(struct wilc *wilc) list_del_rcu(&vif->list); wilc->vif_num--; mutex_unlock(&wilc->vif_mutex); - synchronize_rcu(); + synchronize_srcu(&wilc->srcu); if (vif->ndev) unregister_netdev(vif->ndev); } @@ -925,15 +924,16 @@ static u8 wilc_get_available_idx(struct wilc *wl) { int idx = 0; struct wilc_vif *vif; + int srcu_idx; - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wl->srcu); wilc_for_each_vif(wl, vif) { if (vif->idx == 0) idx = 1; else idx = 0; } - rcu_read_unlock(); + srcu_read_unlock(&wl->srcu, srcu_idx); return idx; } @@ -941,6 +941,7 @@ struct wilc_vif *wilc_netdev_ifc_init(struct wilc *wl, const char *name, int vif_type, enum nl80211_iftype type, bool rtnl_locked) { + u8 mac_address[ETH_ALEN]; struct net_device *ndev; struct wilc_vif *vif; int ret; @@ -965,36 +966,50 @@ struct wilc_vif *wilc_netdev_ifc_init(struct wilc *wl, const char *name, vif->priv.wdev.iftype = type; vif->priv.dev = ndev; - if (rtnl_locked) - ret = cfg80211_register_netdevice(ndev); - else - ret = register_netdev(ndev); - - if (ret) { - ret = -EFAULT; - goto error; - } - ndev->needs_free_netdev = true; vif->iftype = vif_type; vif->idx = wilc_get_available_idx(wl); vif->mac_opened = 0; + + memcpy(mac_address, wl->nv_mac_address, ETH_ALEN); + /* WILC firmware uses locally administered MAC address for the + * second virtual interface (bit 1 of first byte set), but + * since it is possibly not loaded/running yet, reproduce this behavior + * in the driver during interface creation. + */ + if (vif->idx) + mac_address[0] |= 0x2; + + eth_hw_addr_set(vif->ndev, mac_address); + mutex_lock(&wl->vif_mutex); list_add_tail_rcu(&vif->list, &wl->vif_list); wl->vif_num += 1; mutex_unlock(&wl->vif_mutex); - synchronize_rcu(); - - return vif; + synchronize_srcu(&wl->srcu); -error: if (rtnl_locked) - cfg80211_unregister_netdevice(ndev); + ret = cfg80211_register_netdevice(ndev); else - unregister_netdev(ndev); + ret = register_netdev(ndev); + + if (ret) { + ret = -EFAULT; + goto error_remove_vif; + } + + return vif; + +error_remove_vif: + mutex_lock(&wl->vif_mutex); + list_del_rcu(&vif->list); + wl->vif_num -= 1; + mutex_unlock(&wl->vif_mutex); + synchronize_srcu(&wl->srcu); free_netdev(ndev); return ERR_PTR(ret); } +EXPORT_SYMBOL_GPL(wilc_netdev_ifc_init); MODULE_DESCRIPTION("Atmel WILC1000 core wireless driver"); MODULE_LICENSE("GPL"); diff --git a/drivers/net/wireless/microchip/wilc1000/netdev.h b/drivers/net/wireless/microchip/wilc1000/netdev.h index eecee3973d6a420cce81f721ea1c9a24de0bcccd..e59d1b6f5d7d3224de05a715b4759d658c03e562 100644 --- a/drivers/net/wireless/microchip/wilc1000/netdev.h +++ b/drivers/net/wireless/microchip/wilc1000/netdev.h @@ -14,6 +14,7 @@ #include #include #include +#include #include "hif.h" #include "wlan.h" @@ -32,8 +33,8 @@ #define wilc_for_each_vif(w, v) \ struct wilc *_w = w; \ - list_for_each_entry_rcu(v, &_w->vif_list, list, \ - rcu_read_lock_held()) + list_for_each_entry_srcu(v, &_w->vif_list, list, \ + srcu_read_lock_held(&_w->srcu)) struct wilc_wfi_stats { unsigned long rx_packets; @@ -220,6 +221,14 @@ struct wilc { /* protect vif list */ struct mutex vif_mutex; + /* Sleepable RCU struct to manipulate vif list. Sleepable version is + * needed over the classic RCU version because the driver's current + * design involves some sleeping code while manipulating a vif + * retrieved from vif list (so in a SRCU critical section), like: + * - sending commands to the chip, using info from retrieved vif + * - registering a new monitoring net device + */ + struct srcu_struct srcu; u8 open_ifcs; /* protect head of transmit queue */ @@ -278,6 +287,7 @@ struct wilc { struct ieee80211_rate bitrates[ARRAY_SIZE(wilc_bitrates)]; struct ieee80211_supported_band band; u32 cipher_suites[ARRAY_SIZE(wilc_cipher_suites)]; + u8 nv_mac_address[ETH_ALEN]; }; struct wilc_wfi_mon_priv { diff --git a/drivers/net/wireless/microchip/wilc1000/sdio.c b/drivers/net/wireless/microchip/wilc1000/sdio.c index 52a770c5e76f38cc553d448bd064eff3fa4f9797..e6e20c86b791731ad908c5781dc5518c5a203d17 100644 --- a/drivers/net/wireless/microchip/wilc1000/sdio.c +++ b/drivers/net/wireless/microchip/wilc1000/sdio.c @@ -24,6 +24,9 @@ MODULE_DEVICE_TABLE(sdio, wilc_sdio_ids); #define WILC_SDIO_BLOCK_SIZE 512 +static int wilc_sdio_init(struct wilc *wilc, bool resume); +static int wilc_sdio_deinit(struct wilc *wilc); + struct wilc_sdio { bool irq_gpio; u32 block_size; @@ -136,9 +139,11 @@ out: static int wilc_sdio_probe(struct sdio_func *func, const struct sdio_device_id *id) { + struct wilc_sdio *sdio_priv; + struct wilc_vif *vif; struct wilc *wilc; int ret; - struct wilc_sdio *sdio_priv; + sdio_priv = kzalloc(sizeof(*sdio_priv), GFP_KERNEL); if (!sdio_priv) @@ -176,9 +181,28 @@ static int wilc_sdio_probe(struct sdio_func *func, } clk_prepare_enable(wilc->rtc_clk); + wilc_sdio_init(wilc, false); + + ret = wilc_load_mac_from_nv(wilc); + if (ret) { + pr_err("Can not retrieve MAC address from chip\n"); + goto clk_disable_unprepare; + } + + wilc_sdio_deinit(wilc); + + vif = wilc_netdev_ifc_init(wilc, "wlan%d", WILC_STATION_MODE, + NL80211_IFTYPE_STATION, false); + if (IS_ERR(vif)) { + ret = PTR_ERR(vif); + goto clk_disable_unprepare; + } + dev_info(&func->dev, "Driver Initializing success\n"); return 0; +clk_disable_unprepare: + clk_disable_unprepare(wilc->rtc_clk); dispose_irq: irq_dispose_mapping(wilc->dev_irq_num); wilc_netdev_cleanup(wilc); @@ -617,7 +641,52 @@ static int wilc_sdio_read(struct wilc *wilc, u32 addr, u8 *buf, u32 size) static int wilc_sdio_deinit(struct wilc *wilc) { + struct sdio_func *func = dev_to_sdio_func(wilc->dev); struct wilc_sdio *sdio_priv = wilc->bus_data; + struct sdio_cmd52 cmd; + int ret; + + cmd.read_write = 1; + cmd.function = 0; + cmd.raw = 1; + + /* Disable all functions interrupts */ + cmd.address = SDIO_CCCR_IENx; + cmd.data = 0; + ret = wilc_sdio_cmd52(wilc, &cmd); + if (ret) { + dev_err(&func->dev, "Failed to disable functions interrupts\n"); + return ret; + } + + /* Disable all functions */ + cmd.address = SDIO_CCCR_IOEx; + cmd.data = 0; + ret = wilc_sdio_cmd52(wilc, &cmd); + if (ret) { + dev_err(&func->dev, + "Failed to reset all functions\n"); + return ret; + } + + /* Disable CSA */ + cmd.read_write = 0; + cmd.address = SDIO_FBR_BASE(1); + ret = wilc_sdio_cmd52(wilc, &cmd); + if (ret) { + dev_err(&func->dev, + "Failed to read CSA for function 1\n"); + return ret; + } + cmd.read_write = 1; + cmd.address = SDIO_FBR_BASE(1); + cmd.data &= ~SDIO_FBR_ENABLE_CSA; + ret = wilc_sdio_cmd52(wilc, &cmd); + if (ret) { + dev_err(&func->dev, + "Failed to disable CSA for function 1\n"); + return ret; + } sdio_priv->isinit = false; return 0; diff --git a/drivers/net/wireless/microchip/wilc1000/spi.c b/drivers/net/wireless/microchip/wilc1000/spi.c index 61c3572ce32172b70e873a8c5b32c76924a222d8..5ff940c53ad9c06f792137957a82b013dbd5bea7 100644 --- a/drivers/net/wireless/microchip/wilc1000/spi.c +++ b/drivers/net/wireless/microchip/wilc1000/spi.c @@ -206,9 +206,10 @@ static void wilc_wlan_power(struct wilc *wilc, bool on) static int wilc_bus_probe(struct spi_device *spi) { - int ret; - struct wilc *wilc; struct wilc_spi *spi_priv; + struct wilc_vif *vif; + struct wilc *wilc; + int ret; spi_priv = kzalloc(sizeof(*spi_priv), GFP_KERNEL); if (!spi_priv) @@ -249,7 +250,19 @@ static int wilc_bus_probe(struct spi_device *spi) if (ret) goto power_down; + ret = wilc_load_mac_from_nv(wilc); + if (ret) { + pr_err("Can not retrieve MAC address from chip\n"); + goto power_down; + } + wilc_wlan_power(wilc, false); + vif = wilc_netdev_ifc_init(wilc, "wlan%d", WILC_STATION_MODE, + NL80211_IFTYPE_STATION, false); + if (IS_ERR(vif)) { + ret = PTR_ERR(vif); + goto power_down; + } return 0; power_down: diff --git a/drivers/net/wireless/microchip/wilc1000/wlan.c b/drivers/net/wireless/microchip/wilc1000/wlan.c index 37c32d17856ea70e4aa15146463c08807da59e84..41a5c0ed7480575e6a93cdda2dcb40488e28294d 100644 --- a/drivers/net/wireless/microchip/wilc1000/wlan.c +++ b/drivers/net/wireless/microchip/wilc1000/wlan.c @@ -712,6 +712,7 @@ int wilc_wlan_handle_txq(struct wilc *wilc, u32 *txq_count) u32 *vmm_table = wilc->vmm_table; u8 ac_pkt_num_to_chip[NQUEUES] = {0, 0, 0, 0}; const struct wilc_hif_func *func; + int srcu_idx; u8 *txb = wilc->tx_buffer; struct wilc_vif *vif; @@ -723,10 +724,10 @@ int wilc_wlan_handle_txq(struct wilc *wilc, u32 *txq_count) mutex_lock(&wilc->txq_add_to_head_cs); - rcu_read_lock(); + srcu_idx = srcu_read_lock(&wilc->srcu); wilc_for_each_vif(wilc, vif) wilc_wlan_txq_filter_dup_tcp_ack(vif->ndev); - rcu_read_unlock(); + srcu_read_unlock(&wilc->srcu, srcu_idx); for (ac = 0; ac < NQUEUES; ac++) tqe_q[ac] = wilc_wlan_txq_get_first(wilc, ac); @@ -1472,6 +1473,55 @@ u32 wilc_get_chipid(struct wilc *wilc, bool update) return wilc->chipid; } +int wilc_load_mac_from_nv(struct wilc *wl) +{ + int ret = -EINVAL; + unsigned int i; + + acquire_bus(wl, WILC_BUS_ACQUIRE_AND_WAKEUP); + + for (i = 0; i < WILC_NVMEM_MAX_NUM_BANK; i++) { + int bank_offset = get_bank_offset_from_bank_index(i); + u32 reg1, reg2; + u8 invalid; + u8 used; + + ret = wl->hif_func->hif_read_reg(wl, + WILC_NVMEM_BANK_BASE + bank_offset, + ®1); + if (ret) { + pr_err("Can not read address %d lower part", i); + break; + } + ret = wl->hif_func->hif_read_reg(wl, + WILC_NVMEM_BANK_BASE + bank_offset + 4, + ®2); + if (ret) { + pr_err("Can not read address %d upper part", i); + break; + } + + used = FIELD_GET(WILC_NVMEM_IS_BANK_USED, reg1); + invalid = FIELD_GET(WILC_NVMEM_IS_BANK_INVALID, reg1); + if (!used || invalid) + continue; + + wl->nv_mac_address[0] = FIELD_GET(GENMASK(23, 16), reg1); + wl->nv_mac_address[1] = FIELD_GET(GENMASK(15, 8), reg1); + wl->nv_mac_address[2] = FIELD_GET(GENMASK(7, 0), reg1); + wl->nv_mac_address[3] = FIELD_GET(GENMASK(31, 24), reg2); + wl->nv_mac_address[4] = FIELD_GET(GENMASK(23, 16), reg2); + wl->nv_mac_address[5] = FIELD_GET(GENMASK(15, 8), reg2); + + ret = 0; + break; + } + + release_bus(wl, WILC_BUS_RELEASE_ALLOW_SLEEP); + return ret; +} +EXPORT_SYMBOL_GPL(wilc_load_mac_from_nv); + int wilc_wlan_init(struct net_device *dev) { int ret = 0; diff --git a/drivers/net/wireless/microchip/wilc1000/wlan.h b/drivers/net/wireless/microchip/wilc1000/wlan.h index 54643d8fef04e45f834ff207131730be60b8812b..d72a0a81bbda2cd8cbd61eee0bf49af070ceb8da 100644 --- a/drivers/net/wireless/microchip/wilc1000/wlan.h +++ b/drivers/net/wireless/microchip/wilc1000/wlan.h @@ -445,4 +445,5 @@ int wilc_send_config_pkt(struct wilc_vif *vif, u8 mode, struct wid *wids, u32 count); int wilc_wlan_init(struct net_device *dev); u32 wilc_get_chipid(struct wilc *wilc, bool update); +int wilc_load_mac_from_nv(struct wilc *wilc); #endif diff --git a/drivers/net/wireless/ralink/rt2x00/rt2x00.h b/drivers/net/wireless/ralink/rt2x00/rt2x00.h index 82af01448a0a8d2da7a20ae0cbb0e614875957a6..bb648f95dfdd9d51259de9765288c8697dfe6d61 100644 --- a/drivers/net/wireless/ralink/rt2x00/rt2x00.h +++ b/drivers/net/wireless/ralink/rt2x00/rt2x00.h @@ -335,16 +335,6 @@ struct link { struct delayed_work watchdog_work; unsigned int watchdog_interval; unsigned int watchdog; - - /* - * Work structure for scheduling periodic AGC adjustments. - */ - struct delayed_work agc_work; - - /* - * Work structure for scheduling periodic VCO calibration. - */ - struct delayed_work vco_work; }; enum rt2x00_delayed_flags { diff --git a/drivers/net/wireless/realtek/rtlwifi/Kconfig b/drivers/net/wireless/realtek/rtlwifi/Kconfig index cfe63f7b28d9af56ee2884c5794fd9bf49d73f56..1e66c1bf7c8bb80a10c2eb2955d0db4c207157ea 100644 --- a/drivers/net/wireless/realtek/rtlwifi/Kconfig +++ b/drivers/net/wireless/realtek/rtlwifi/Kconfig @@ -119,6 +119,18 @@ config RTL8192CU If you choose to build it as a module, it will be called rtl8192cu +config RTL8192DU + tristate "Realtek RTL8192DU USB Wireless Network Adapter" + depends on USB + select RTLWIFI + select RTLWIFI_USB + select RTL8192D_COMMON + help + This is the driver for Realtek RTL8192DU 802.11n USB + wireless network adapters. + + If you choose to build it as a module, it will be called rtl8192du + config RTLWIFI tristate select FW_LOADER diff --git a/drivers/net/wireless/realtek/rtlwifi/Makefile b/drivers/net/wireless/realtek/rtlwifi/Makefile index 423981b148df212afcefebf0b4a90405a931d262..9cf32277c7f1b3c2c68c5c4036a4259d1b607434 100644 --- a/drivers/net/wireless/realtek/rtlwifi/Makefile +++ b/drivers/net/wireless/realtek/rtlwifi/Makefile @@ -25,6 +25,7 @@ obj-$(CONFIG_RTL8192CU) += rtl8192cu/ obj-$(CONFIG_RTL8192SE) += rtl8192se/ obj-$(CONFIG_RTL8192D_COMMON) += rtl8192d/ obj-$(CONFIG_RTL8192DE) += rtl8192de/ +obj-$(CONFIG_RTL8192DU) += rtl8192du/ obj-$(CONFIG_RTL8723AE) += rtl8723ae/ obj-$(CONFIG_RTL8723BE) += rtl8723be/ obj-$(CONFIG_RTL8188EE) += rtl8188ee/ diff --git a/drivers/net/wireless/realtek/rtlwifi/base.c b/drivers/net/wireless/realtek/rtlwifi/base.c index 1a8d715b7c07099357e699d380c382a839c83918..aab4605de9c47c0078b9e0199fc85a50a8b4ae02 100644 --- a/drivers/net/wireless/realtek/rtlwifi/base.c +++ b/drivers/net/wireless/realtek/rtlwifi/base.c @@ -2272,7 +2272,7 @@ static void rtl_c2h_content_parsing(struct ieee80211_hw *hw, struct sk_buff *skb) { struct rtl_priv *rtlpriv = rtl_priv(hw); - struct rtl_hal_ops *hal_ops = rtlpriv->cfg->ops; + const struct rtl_hal_ops *hal_ops = rtlpriv->cfg->ops; const struct rtl_btc_ops *btc_ops = rtlpriv->btcoexist.btc_ops; u8 cmd_id, cmd_len; u8 *cmd_buf = NULL; diff --git a/drivers/net/wireless/realtek/rtlwifi/core.c b/drivers/net/wireless/realtek/rtlwifi/core.c index 2e60a6991ca1665db8c4d5c3730500e9dc53eded..42b7db12b1bd41d0c3abbaea60bb2218cbab4894 100644 --- a/drivers/net/wireless/realtek/rtlwifi/core.c +++ b/drivers/net/wireless/realtek/rtlwifi/core.c @@ -633,21 +633,6 @@ static int rtl_op_config(struct ieee80211_hw *hw, u32 changed) } } - if (changed & IEEE80211_CONF_CHANGE_RETRY_LIMITS) { - rtl_dbg(rtlpriv, COMP_MAC80211, DBG_LOUD, - "IEEE80211_CONF_CHANGE_RETRY_LIMITS %x\n", - hw->conf.long_frame_max_tx_count); - /* brought up everything changes (changed == ~0) indicates first - * open, so use our default value instead of that of wiphy. - */ - if (changed != ~0) { - mac->retry_long = hw->conf.long_frame_max_tx_count; - mac->retry_short = hw->conf.long_frame_max_tx_count; - rtlpriv->cfg->ops->set_hw_reg(hw, HW_VAR_RETRY_LIMIT, - (u8 *)(&hw->conf.long_frame_max_tx_count)); - } - } - if (changed & IEEE80211_CONF_CHANGE_CHANNEL && !rtlpriv->proximity.proxim_on) { struct ieee80211_channel *channel = hw->conf.chandef.chan; diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/sw.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/sw.c index 48be7e346efcc1ad613fda7cc06195050af5abbc..c9b9e2bc90cc49cb6131f31a3101bdf299abe8fa 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/sw.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192cu/sw.c @@ -53,8 +53,6 @@ static int rtl92cu_init_sw_vars(struct ieee80211_hw *hw) } else { fw_name = "rtlwifi/rtl8192cufw_TMSC.bin"; } - /* provide name of alternative file */ - rtlpriv->cfg->alt_fw_name = "rtlwifi/rtl8192cufw.bin"; pr_info("Loading firmware %s\n", fw_name); rtlpriv->max_fw_size = 0x4000; err = request_firmware_nowait(THIS_MODULE, 1, @@ -160,6 +158,7 @@ static struct rtl_hal_usbint_cfg rtl92cu_interface_cfg = { static struct rtl_hal_cfg rtl92cu_hal_cfg = { .name = "rtl92c_usb", + .alt_fw_name = "rtlwifi/rtl8192cufw.bin", .ops = &rtl8192cu_hal_ops, .mod_params = &rtl92cu_mod_params, .usb_interface_cfg = &rtl92cu_interface_cfg, diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192d/hw_common.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192d/hw_common.c index 6570d5e168e91ec2ef7f10697c275f2c91326873..97e0d9c01e0acbca2ef212ed03e93a65b8806383 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192d/hw_common.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192d/hw_common.c @@ -14,7 +14,7 @@ #include "hw_common.h" #include "phy_common.h" -void rtl92de_stop_tx_beacon(struct ieee80211_hw *hw) +void rtl92d_stop_tx_beacon(struct ieee80211_hw *hw) { struct rtl_priv *rtlpriv = rtl_priv(hw); u8 tmp1byte; @@ -27,9 +27,9 @@ void rtl92de_stop_tx_beacon(struct ieee80211_hw *hw) tmp1byte &= ~(BIT(0)); rtl_write_byte(rtlpriv, REG_TBTT_PROHIBIT + 2, tmp1byte); } -EXPORT_SYMBOL_GPL(rtl92de_stop_tx_beacon); +EXPORT_SYMBOL_GPL(rtl92d_stop_tx_beacon); -void rtl92de_resume_tx_beacon(struct ieee80211_hw *hw) +void rtl92d_resume_tx_beacon(struct ieee80211_hw *hw) { struct rtl_priv *rtlpriv = rtl_priv(hw); u8 tmp1byte; @@ -42,7 +42,7 @@ void rtl92de_resume_tx_beacon(struct ieee80211_hw *hw) tmp1byte |= BIT(0); rtl_write_byte(rtlpriv, REG_TBTT_PROHIBIT + 2, tmp1byte); } -EXPORT_SYMBOL_GPL(rtl92de_resume_tx_beacon); +EXPORT_SYMBOL_GPL(rtl92d_resume_tx_beacon); void rtl92d_get_hw_reg(struct ieee80211_hw *hw, u8 variable, u8 *val) { @@ -285,7 +285,7 @@ void rtl92d_set_hw_reg(struct ieee80211_hw *hw, u8 variable, u8 *val) } EXPORT_SYMBOL_GPL(rtl92d_set_hw_reg); -bool rtl92de_llt_write(struct ieee80211_hw *hw, u32 address, u32 data) +bool rtl92d_llt_write(struct ieee80211_hw *hw, u32 address, u32 data) { struct rtl_priv *rtlpriv = rtl_priv(hw); bool status = true; @@ -307,9 +307,9 @@ bool rtl92de_llt_write(struct ieee80211_hw *hw, u32 address, u32 data) } while (++count); return status; } -EXPORT_SYMBOL_GPL(rtl92de_llt_write); +EXPORT_SYMBOL_GPL(rtl92d_llt_write); -void rtl92de_enable_hw_security_config(struct ieee80211_hw *hw) +void rtl92d_enable_hw_security_config(struct ieee80211_hw *hw) { struct rtl_priv *rtlpriv = rtl_priv(hw); u8 sec_reg_value; @@ -334,16 +334,16 @@ void rtl92de_enable_hw_security_config(struct ieee80211_hw *hw) "The SECR-value %x\n", sec_reg_value); rtlpriv->cfg->ops->set_hw_reg(hw, HW_VAR_WPA_CONFIG, &sec_reg_value); } -EXPORT_SYMBOL_GPL(rtl92de_enable_hw_security_config); +EXPORT_SYMBOL_GPL(rtl92d_enable_hw_security_config); /* don't set REG_EDCA_BE_PARAM here because * mac80211 will send pkt when scan */ -void rtl92de_set_qos(struct ieee80211_hw *hw, int aci) +void rtl92d_set_qos(struct ieee80211_hw *hw, int aci) { rtl92d_dm_init_edca_turbo(hw); } -EXPORT_SYMBOL_GPL(rtl92de_set_qos); +EXPORT_SYMBOL_GPL(rtl92d_set_qos); static enum version_8192d _rtl92d_read_chip_version(struct ieee80211_hw *hw) { @@ -362,8 +362,8 @@ static enum version_8192d _rtl92d_read_chip_version(struct ieee80211_hw *hw) return version; } -static void _rtl92de_readpowervalue_fromprom(struct txpower_info *pwrinfo, - u8 *efuse, bool autoloadfail) +static void _rtl92d_readpowervalue_fromprom(struct txpower_info *pwrinfo, + u8 *efuse, bool autoloadfail) { u32 rfpath, eeaddr, group, offset, offset1, offset2; u8 i, val8; @@ -500,8 +500,8 @@ static void _rtl92de_readpowervalue_fromprom(struct txpower_info *pwrinfo, } } -static void _rtl92de_read_txpower_info(struct ieee80211_hw *hw, - bool autoload_fail, u8 *hwinfo) +static void _rtl92d_read_txpower_info(struct ieee80211_hw *hw, + bool autoload_fail, u8 *hwinfo) { struct rtl_priv *rtlpriv = rtl_priv(hw); struct rtl_efuse *rtlefuse = rtl_efuse(rtl_priv(hw)); @@ -509,7 +509,7 @@ static void _rtl92de_read_txpower_info(struct ieee80211_hw *hw, u8 tempval[2], i, pwr, diff; u32 ch, rfpath, group; - _rtl92de_readpowervalue_fromprom(&pwrinfo, hwinfo, autoload_fail); + _rtl92d_readpowervalue_fromprom(&pwrinfo, hwinfo, autoload_fail); if (!autoload_fail) { /* bit0~2 */ rtlefuse->eeprom_regulatory = (hwinfo[EEPROM_RF_OPT1] & 0x7); @@ -613,8 +613,8 @@ static void _rtl92de_read_txpower_info(struct ieee80211_hw *hw, } } -static void _rtl92de_read_macphymode_from_prom(struct ieee80211_hw *hw, - u8 *content) +static void _rtl92d_read_macphymode_from_prom(struct ieee80211_hw *hw, + u8 *content) { struct rtl_priv *rtlpriv = rtl_priv(hw); struct rtl_hal *rtlhal = rtl_hal(rtl_priv(hw)); @@ -636,15 +636,15 @@ static void _rtl92de_read_macphymode_from_prom(struct ieee80211_hw *hw, } } -static void _rtl92de_read_macphymode_and_bandtype(struct ieee80211_hw *hw, - u8 *content) +static void _rtl92d_read_macphymode_and_bandtype(struct ieee80211_hw *hw, + u8 *content) { - _rtl92de_read_macphymode_from_prom(hw, content); + _rtl92d_read_macphymode_from_prom(hw, content); rtl92d_phy_config_macphymode(hw); rtl92d_phy_config_macphymode_info(hw); } -static void _rtl92de_efuse_update_chip_version(struct ieee80211_hw *hw) +static void _rtl92d_efuse_update_chip_version(struct ieee80211_hw *hw) { struct rtl_priv *rtlpriv = rtl_priv(hw); enum version_8192d chipver = rtlpriv->rtlhal.version; @@ -676,7 +676,7 @@ static void _rtl92de_efuse_update_chip_version(struct ieee80211_hw *hw) rtlpriv->rtlhal.version = chipver; } -static void _rtl92de_read_adapter_info(struct ieee80211_hw *hw) +static void _rtl92d_read_adapter_info(struct ieee80211_hw *hw) { static const int params_pci[] = { RTL8190_EEPROM_ID, EEPROM_VID, EEPROM_DID, @@ -706,8 +706,8 @@ static void _rtl92de_read_adapter_info(struct ieee80211_hw *hw) if (rtl_get_hwinfo(hw, rtlpriv, HWSET_MAX_SIZE, hwinfo, params)) goto exit; - _rtl92de_efuse_update_chip_version(hw); - _rtl92de_read_macphymode_and_bandtype(hw, hwinfo); + _rtl92d_efuse_update_chip_version(hw); + _rtl92d_read_macphymode_and_bandtype(hw, hwinfo); /* Read Permanent MAC address for 2nd interface */ if (rtlhal->interfaceindex != 0) @@ -717,7 +717,7 @@ static void _rtl92de_read_adapter_info(struct ieee80211_hw *hw) rtlpriv->cfg->ops->set_hw_reg(hw, HW_VAR_ETHER_ADDR, rtlefuse->dev_addr); rtl_dbg(rtlpriv, COMP_INIT, DBG_DMESG, "%pM\n", rtlefuse->dev_addr); - _rtl92de_read_txpower_info(hw, rtlefuse->autoload_failflag, hwinfo); + _rtl92d_read_txpower_info(hw, rtlefuse->autoload_failflag, hwinfo); /* Read Channel Plan */ switch (rtlhal->bandset) { @@ -739,7 +739,7 @@ exit: kfree(hwinfo); } -void rtl92de_read_eeprom_info(struct ieee80211_hw *hw) +void rtl92d_read_eeprom_info(struct ieee80211_hw *hw) { struct rtl_priv *rtlpriv = rtl_priv(hw); struct rtl_efuse *rtlefuse = rtl_efuse(rtl_priv(hw)); @@ -760,15 +760,15 @@ void rtl92de_read_eeprom_info(struct ieee80211_hw *hw) rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, "Autoload OK\n"); rtlefuse->autoload_failflag = false; - _rtl92de_read_adapter_info(hw); + _rtl92d_read_adapter_info(hw); } else { pr_err("Autoload ERR!!\n"); } } -EXPORT_SYMBOL_GPL(rtl92de_read_eeprom_info); +EXPORT_SYMBOL_GPL(rtl92d_read_eeprom_info); -static void rtl92de_update_hal_rate_table(struct ieee80211_hw *hw, - struct ieee80211_sta *sta) +static void rtl92d_update_hal_rate_table(struct ieee80211_hw *hw, + struct ieee80211_sta *sta) { struct rtl_hal *rtlhal = rtl_hal(rtl_priv(hw)); struct rtl_mac *mac = rtl_mac(rtl_priv(hw)); @@ -851,9 +851,9 @@ static void rtl92de_update_hal_rate_table(struct ieee80211_hw *hw, rtl_read_dword(rtlpriv, REG_ARFR0)); } -static void rtl92de_update_hal_rate_mask(struct ieee80211_hw *hw, - struct ieee80211_sta *sta, - u8 rssi_level, bool update_bw) +static void rtl92d_update_hal_rate_mask(struct ieee80211_hw *hw, + struct ieee80211_sta *sta, + u8 rssi_level, bool update_bw) { struct rtl_hal *rtlhal = rtl_hal(rtl_priv(hw)); struct rtl_mac *mac = rtl_mac(rtl_priv(hw)); @@ -1009,20 +1009,20 @@ static void rtl92de_update_hal_rate_mask(struct ieee80211_hw *hw, sta_entry->ratr_index = ratr_index; } -void rtl92de_update_hal_rate_tbl(struct ieee80211_hw *hw, - struct ieee80211_sta *sta, - u8 rssi_level, bool update_bw) +void rtl92d_update_hal_rate_tbl(struct ieee80211_hw *hw, + struct ieee80211_sta *sta, + u8 rssi_level, bool update_bw) { struct rtl_priv *rtlpriv = rtl_priv(hw); if (rtlpriv->dm.useramask) - rtl92de_update_hal_rate_mask(hw, sta, rssi_level, update_bw); + rtl92d_update_hal_rate_mask(hw, sta, rssi_level, update_bw); else - rtl92de_update_hal_rate_table(hw, sta); + rtl92d_update_hal_rate_table(hw, sta); } -EXPORT_SYMBOL_GPL(rtl92de_update_hal_rate_tbl); +EXPORT_SYMBOL_GPL(rtl92d_update_hal_rate_tbl); -void rtl92de_update_channel_access_setting(struct ieee80211_hw *hw) +void rtl92d_update_channel_access_setting(struct ieee80211_hw *hw) { struct rtl_priv *rtlpriv = rtl_priv(hw); struct rtl_mac *mac = rtl_mac(rtl_priv(hw)); @@ -1036,9 +1036,9 @@ void rtl92de_update_channel_access_setting(struct ieee80211_hw *hw) sifs_timer = 0x1010; rtlpriv->cfg->ops->set_hw_reg(hw, HW_VAR_SIFS, (u8 *)&sifs_timer); } -EXPORT_SYMBOL_GPL(rtl92de_update_channel_access_setting); +EXPORT_SYMBOL_GPL(rtl92d_update_channel_access_setting); -bool rtl92de_gpio_radio_on_off_checking(struct ieee80211_hw *hw, u8 *valid) +bool rtl92d_gpio_radio_on_off_checking(struct ieee80211_hw *hw, u8 *valid) { struct rtl_priv *rtlpriv = rtl_priv(hw); struct rtl_ps_ctl *ppsc = rtl_psc(rtl_priv(hw)); @@ -1093,11 +1093,11 @@ bool rtl92de_gpio_radio_on_off_checking(struct ieee80211_hw *hw, u8 *valid) *valid = 1; return !ppsc->hwradiooff; } -EXPORT_SYMBOL_GPL(rtl92de_gpio_radio_on_off_checking); +EXPORT_SYMBOL_GPL(rtl92d_gpio_radio_on_off_checking); -void rtl92de_set_key(struct ieee80211_hw *hw, u32 key_index, - u8 *p_macaddr, bool is_group, u8 enc_algo, - bool is_wepkey, bool clear_all) +void rtl92d_set_key(struct ieee80211_hw *hw, u32 key_index, + u8 *p_macaddr, bool is_group, u8 enc_algo, + bool is_wepkey, bool clear_all) { static const u8 cam_const_addr[4][6] = { {0x00, 0x00, 0x00, 0x00, 0x00, 0x00}, @@ -1222,4 +1222,4 @@ void rtl92de_set_key(struct ieee80211_hw *hw, u32 key_index, } } } -EXPORT_SYMBOL_GPL(rtl92de_set_key); +EXPORT_SYMBOL_GPL(rtl92d_set_key); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192d/hw_common.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192d/hw_common.h index 2c07f5cc57660fd2c7fe24f98dbc504b104c2a9a..4da1bab15f36a1179f852eff73498a3325a14dfc 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192d/hw_common.h +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192d/hw_common.h @@ -4,21 +4,21 @@ #ifndef __RTL92D_HW_COMMON_H__ #define __RTL92D_HW_COMMON_H__ -void rtl92de_stop_tx_beacon(struct ieee80211_hw *hw); -void rtl92de_resume_tx_beacon(struct ieee80211_hw *hw); +void rtl92d_stop_tx_beacon(struct ieee80211_hw *hw); +void rtl92d_resume_tx_beacon(struct ieee80211_hw *hw); void rtl92d_get_hw_reg(struct ieee80211_hw *hw, u8 variable, u8 *val); void rtl92d_set_hw_reg(struct ieee80211_hw *hw, u8 variable, u8 *val); -bool rtl92de_llt_write(struct ieee80211_hw *hw, u32 address, u32 data); -void rtl92de_enable_hw_security_config(struct ieee80211_hw *hw); -void rtl92de_set_qos(struct ieee80211_hw *hw, int aci); -void rtl92de_read_eeprom_info(struct ieee80211_hw *hw); -void rtl92de_update_hal_rate_tbl(struct ieee80211_hw *hw, - struct ieee80211_sta *sta, - u8 rssi_level, bool update_bw); -void rtl92de_update_channel_access_setting(struct ieee80211_hw *hw); -bool rtl92de_gpio_radio_on_off_checking(struct ieee80211_hw *hw, u8 *valid); -void rtl92de_set_key(struct ieee80211_hw *hw, u32 key_index, - u8 *p_macaddr, bool is_group, u8 enc_algo, - bool is_wepkey, bool clear_all); +bool rtl92d_llt_write(struct ieee80211_hw *hw, u32 address, u32 data); +void rtl92d_enable_hw_security_config(struct ieee80211_hw *hw); +void rtl92d_set_qos(struct ieee80211_hw *hw, int aci); +void rtl92d_read_eeprom_info(struct ieee80211_hw *hw); +void rtl92d_update_hal_rate_tbl(struct ieee80211_hw *hw, + struct ieee80211_sta *sta, + u8 rssi_level, bool update_bw); +void rtl92d_update_channel_access_setting(struct ieee80211_hw *hw); +bool rtl92d_gpio_radio_on_off_checking(struct ieee80211_hw *hw, u8 *valid); +void rtl92d_set_key(struct ieee80211_hw *hw, u32 key_index, + u8 *p_macaddr, bool is_group, u8 enc_algo, + bool is_wepkey, bool clear_all); #endif diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192d/trx_common.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192d/trx_common.c index 72d2b7426d82fd75172cd1d7e29cace2b48d376d..9f9a34492030c6ccedb8c8ad1bd29a5c7a13fabf 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192d/trx_common.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192d/trx_common.c @@ -7,8 +7,8 @@ #include "def.h" #include "trx_common.h" -static long _rtl92de_translate_todbm(struct ieee80211_hw *hw, - u8 signal_strength_index) +static long _rtl92d_translate_todbm(struct ieee80211_hw *hw, + u8 signal_strength_index) { long signal_power; @@ -17,13 +17,13 @@ static long _rtl92de_translate_todbm(struct ieee80211_hw *hw, return signal_power; } -static void _rtl92de_query_rxphystatus(struct ieee80211_hw *hw, - struct rtl_stats *pstats, - __le32 *pdesc, - struct rx_fwinfo_92d *p_drvinfo, - bool packet_match_bssid, - bool packet_toself, - bool packet_beacon) +static void _rtl92d_query_rxphystatus(struct ieee80211_hw *hw, + struct rtl_stats *pstats, + __le32 *pdesc, + struct rx_fwinfo_92d *p_drvinfo, + bool packet_match_bssid, + bool packet_toself, + bool packet_beacon) { struct rtl_priv *rtlpriv = rtl_priv(hw); struct rtl_phy *rtlphy = &rtlpriv->phy; @@ -203,8 +203,8 @@ static void rtl92d_loop_over_paths(struct ieee80211_hw *hw, } } -static void _rtl92de_process_ui_rssi(struct ieee80211_hw *hw, - struct rtl_stats *pstats) +static void _rtl92d_process_ui_rssi(struct ieee80211_hw *hw, + struct rtl_stats *pstats) { struct rtl_priv *rtlpriv = rtl_priv(hw); struct rt_smooth_data *ui_rssi; @@ -226,15 +226,15 @@ static void _rtl92de_process_ui_rssi(struct ieee80211_hw *hw, if (ui_rssi->index >= PHY_RSSI_SLID_WIN_MAX) ui_rssi->index = 0; tmpval = ui_rssi->total_val / ui_rssi->total_num; - rtlpriv->stats.signal_strength = _rtl92de_translate_todbm(hw, (u8)tmpval); + rtlpriv->stats.signal_strength = _rtl92d_translate_todbm(hw, (u8)tmpval); pstats->rssi = rtlpriv->stats.signal_strength; if (!pstats->is_cck && pstats->packet_toself) rtl92d_loop_over_paths(hw, pstats); } -static void _rtl92de_update_rxsignalstatistics(struct ieee80211_hw *hw, - struct rtl_stats *pstats) +static void _rtl92d_update_rxsignalstatistics(struct ieee80211_hw *hw, + struct rtl_stats *pstats) { struct rtl_priv *rtlpriv = rtl_priv(hw); int weighting = 0; @@ -249,8 +249,8 @@ static void _rtl92de_update_rxsignalstatistics(struct ieee80211_hw *hw, 5 + pstats->recvsignalpower + weighting) / 6; } -static void _rtl92de_process_pwdb(struct ieee80211_hw *hw, - struct rtl_stats *pstats) +static void _rtl92d_process_pwdb(struct ieee80211_hw *hw, + struct rtl_stats *pstats) { struct rtl_priv *rtlpriv = rtl_priv(hw); struct rtl_mac *mac = rtl_mac(rtl_priv(hw)); @@ -276,7 +276,7 @@ static void _rtl92de_process_pwdb(struct ieee80211_hw *hw, (pstats->rx_pwdb_all)) / (RX_SMOOTH_FACTOR); } rtlpriv->dm.undec_sm_pwdb = undec_sm_pwdb; - _rtl92de_update_rxsignalstatistics(hw, pstats); + _rtl92d_update_rxsignalstatistics(hw, pstats); } } @@ -301,8 +301,8 @@ static void rtl92d_loop_over_streams(struct ieee80211_hw *hw, } } -static void _rtl92de_process_ui_link_quality(struct ieee80211_hw *hw, - struct rtl_stats *pstats) +static void _rtl92d_process_ui_link_quality(struct ieee80211_hw *hw, + struct rtl_stats *pstats) { struct rtl_priv *rtlpriv = rtl_priv(hw); struct rt_smooth_data *ui_link_quality; @@ -330,24 +330,24 @@ static void _rtl92de_process_ui_link_quality(struct ieee80211_hw *hw, rtl92d_loop_over_streams(hw, pstats); } -static void _rtl92de_process_phyinfo(struct ieee80211_hw *hw, - u8 *buffer, - struct rtl_stats *pcurrent_stats) +static void _rtl92d_process_phyinfo(struct ieee80211_hw *hw, + u8 *buffer, + struct rtl_stats *pcurrent_stats) { if (!pcurrent_stats->packet_matchbssid && !pcurrent_stats->packet_beacon) return; - _rtl92de_process_ui_rssi(hw, pcurrent_stats); - _rtl92de_process_pwdb(hw, pcurrent_stats); - _rtl92de_process_ui_link_quality(hw, pcurrent_stats); + _rtl92d_process_ui_rssi(hw, pcurrent_stats); + _rtl92d_process_pwdb(hw, pcurrent_stats); + _rtl92d_process_ui_link_quality(hw, pcurrent_stats); } -static void _rtl92de_translate_rx_signal_stuff(struct ieee80211_hw *hw, - struct sk_buff *skb, - struct rtl_stats *pstats, - __le32 *pdesc, - struct rx_fwinfo_92d *p_drvinfo) +static void _rtl92d_translate_rx_signal_stuff(struct ieee80211_hw *hw, + struct sk_buff *skb, + struct rtl_stats *pstats, + __le32 *pdesc, + struct rx_fwinfo_92d *p_drvinfo) { struct rtl_efuse *rtlefuse = rtl_efuse(rtl_priv(hw)); struct rtl_mac *mac = rtl_mac(rtl_priv(hw)); @@ -375,15 +375,15 @@ static void _rtl92de_translate_rx_signal_stuff(struct ieee80211_hw *hw, packet_toself = packet_matchbssid && ether_addr_equal(praddr, rtlefuse->dev_addr); packet_beacon = ieee80211_is_beacon(fc); - _rtl92de_query_rxphystatus(hw, pstats, pdesc, p_drvinfo, - packet_matchbssid, packet_toself, - packet_beacon); - _rtl92de_process_phyinfo(hw, tmp_buf, pstats); + _rtl92d_query_rxphystatus(hw, pstats, pdesc, p_drvinfo, + packet_matchbssid, packet_toself, + packet_beacon); + _rtl92d_process_phyinfo(hw, tmp_buf, pstats); } -bool rtl92de_rx_query_desc(struct ieee80211_hw *hw, struct rtl_stats *stats, - struct ieee80211_rx_status *rx_status, - u8 *pdesc8, struct sk_buff *skb) +bool rtl92d_rx_query_desc(struct ieee80211_hw *hw, struct rtl_stats *stats, + struct ieee80211_rx_status *rx_status, + u8 *pdesc8, struct sk_buff *skb) { __le32 *pdesc = (__le32 *)pdesc8; struct rx_fwinfo_92d *p_drvinfo; @@ -423,17 +423,17 @@ bool rtl92de_rx_query_desc(struct ieee80211_hw *hw, struct rtl_stats *stats, if (phystatus) { p_drvinfo = (struct rx_fwinfo_92d *)(skb->data + stats->rx_bufshift); - _rtl92de_translate_rx_signal_stuff(hw, skb, stats, pdesc, - p_drvinfo); + _rtl92d_translate_rx_signal_stuff(hw, skb, stats, pdesc, + p_drvinfo); } /*rx_status->qual = stats->signal; */ rx_status->signal = stats->recvsignalpower + 10; return true; } -EXPORT_SYMBOL_GPL(rtl92de_rx_query_desc); +EXPORT_SYMBOL_GPL(rtl92d_rx_query_desc); -void rtl92de_set_desc(struct ieee80211_hw *hw, u8 *pdesc8, bool istx, - u8 desc_name, u8 *val) +void rtl92d_set_desc(struct ieee80211_hw *hw, u8 *pdesc8, bool istx, + u8 desc_name, u8 *val) { __le32 *pdesc = (__le32 *)pdesc8; @@ -473,10 +473,10 @@ void rtl92de_set_desc(struct ieee80211_hw *hw, u8 *pdesc8, bool istx, } } } -EXPORT_SYMBOL_GPL(rtl92de_set_desc); +EXPORT_SYMBOL_GPL(rtl92d_set_desc); -u64 rtl92de_get_desc(struct ieee80211_hw *hw, - u8 *p_desc8, bool istx, u8 desc_name) +u64 rtl92d_get_desc(struct ieee80211_hw *hw, + u8 *p_desc8, bool istx, u8 desc_name) { __le32 *p_desc = (__le32 *)p_desc8; u32 ret = 0; @@ -513,4 +513,4 @@ u64 rtl92de_get_desc(struct ieee80211_hw *hw, } return ret; } -EXPORT_SYMBOL_GPL(rtl92de_get_desc); +EXPORT_SYMBOL_GPL(rtl92d_get_desc); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192d/trx_common.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192d/trx_common.h index 87d956d771ebe14229bf3b940e1ccc78a3620360..528182b1eba68bf067eea109c8ab277be2794f97 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192d/trx_common.h +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192d/trx_common.h @@ -393,13 +393,13 @@ struct rx_fwinfo_92d { #endif } __packed; -bool rtl92de_rx_query_desc(struct ieee80211_hw *hw, - struct rtl_stats *stats, - struct ieee80211_rx_status *rx_status, - u8 *pdesc, struct sk_buff *skb); -void rtl92de_set_desc(struct ieee80211_hw *hw, u8 *pdesc, bool istx, - u8 desc_name, u8 *val); -u64 rtl92de_get_desc(struct ieee80211_hw *hw, - u8 *p_desc, bool istx, u8 desc_name); +bool rtl92d_rx_query_desc(struct ieee80211_hw *hw, + struct rtl_stats *stats, + struct ieee80211_rx_status *rx_status, + u8 *pdesc, struct sk_buff *skb); +void rtl92d_set_desc(struct ieee80211_hw *hw, u8 *pdesc, bool istx, + u8 desc_name, u8 *val); +u64 rtl92d_get_desc(struct ieee80211_hw *hw, + u8 *p_desc, bool istx, u8 desc_name); #endif diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/hw.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/hw.c index 73b81e60cfa90e9d6d5721ad80d0552010979054..03f4314bdb2e6e052cac8817472cdd661bbf2f25 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/hw.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/hw.c @@ -181,7 +181,7 @@ void rtl92de_set_hw_reg(struct ieee80211_hw *hw, u8 variable, u8 *val) u8 btype_ibss = val[0]; if (btype_ibss) - rtl92de_stop_tx_beacon(hw); + rtl92d_stop_tx_beacon(hw); _rtl92de_set_bcn_ctrl_reg(hw, 0, BIT(3)); rtl_write_dword(rtlpriv, REG_TSFTR, (u32) (mac->tsf & 0xffffffff)); @@ -189,7 +189,7 @@ void rtl92de_set_hw_reg(struct ieee80211_hw *hw, u8 variable, u8 *val) (u32) ((mac->tsf >> 32) & 0xffffffff)); _rtl92de_set_bcn_ctrl_reg(hw, BIT(3), 0); if (btype_ibss) - rtl92de_resume_tx_beacon(hw); + rtl92d_resume_tx_beacon(hw); break; } @@ -295,13 +295,13 @@ static bool _rtl92de_llt_table_init(struct ieee80211_hw *hw) /* 18. LLT_table_init(Adapter); */ for (i = 0; i < (txpktbuf_bndy - 1); i++) { - status = rtl92de_llt_write(hw, i, i + 1); + status = rtl92d_llt_write(hw, i, i + 1); if (!status) return status; } /* end of list */ - status = rtl92de_llt_write(hw, (txpktbuf_bndy - 1), 0xFF); + status = rtl92d_llt_write(hw, (txpktbuf_bndy - 1), 0xFF); if (!status) return status; @@ -310,13 +310,13 @@ static bool _rtl92de_llt_table_init(struct ieee80211_hw *hw) /* config this MAC as two MAC transfer. */ /* Otherwise used as local loopback buffer. */ for (i = txpktbuf_bndy; i < maxpage; i++) { - status = rtl92de_llt_write(hw, i, (i + 1)); + status = rtl92d_llt_write(hw, i, (i + 1)); if (!status) return status; } /* Let last entry point to the start entry of ring buffer */ - status = rtl92de_llt_write(hw, maxpage, txpktbuf_bndy); + status = rtl92d_llt_write(hw, maxpage, txpktbuf_bndy); if (!status) return status; @@ -688,7 +688,7 @@ int rtl92de_hw_init(struct ieee80211_hw *hw) /* reset hw sec */ rtl_cam_reset_all_entry(hw); - rtl92de_enable_hw_security_config(hw); + rtl92d_enable_hw_security_config(hw); /* Read EEPROM TX power index and PHY_REG_PG.txt to capture correct */ /* TX power index for different rate set. */ @@ -742,11 +742,11 @@ static int _rtl92de_set_media_status(struct ieee80211_hw *hw, if (type == NL80211_IFTYPE_UNSPECIFIED || type == NL80211_IFTYPE_STATION) { - rtl92de_stop_tx_beacon(hw); + rtl92d_stop_tx_beacon(hw); _rtl92de_enable_bcn_sub_func(hw); } else if (type == NL80211_IFTYPE_ADHOC || type == NL80211_IFTYPE_AP) { - rtl92de_resume_tx_beacon(hw); + rtl92d_resume_tx_beacon(hw); _rtl92de_disable_bcn_sub_func(hw); } else { rtl_dbg(rtlpriv, COMP_ERR, DBG_WARNING, diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/sw.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/sw.c index 5f6311c2aac477631b20077c0c1a17ce99643a11..f5ce4889523e33e2fbd40db15c30c89a2b29412c 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/sw.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/sw.c @@ -187,7 +187,7 @@ static void rtl92d_deinit_sw_vars(struct ieee80211_hw *hw) static struct rtl_hal_ops rtl8192de_hal_ops = { .init_sw_vars = rtl92d_init_sw_vars, .deinit_sw_vars = rtl92d_deinit_sw_vars, - .read_eeprom_info = rtl92de_read_eeprom_info, + .read_eeprom_info = rtl92d_read_eeprom_info, .interrupt_recognized = rtl92de_interrupt_recognized, .hw_init = rtl92de_hw_init, .hw_disable = rtl92de_card_disable, @@ -197,30 +197,30 @@ static struct rtl_hal_ops rtl8192de_hal_ops = { .disable_interrupt = rtl92de_disable_interrupt, .set_network_type = rtl92de_set_network_type, .set_chk_bssid = rtl92de_set_check_bssid, - .set_qos = rtl92de_set_qos, + .set_qos = rtl92d_set_qos, .set_bcn_reg = rtl92de_set_beacon_related_registers, .set_bcn_intv = rtl92de_set_beacon_interval, .update_interrupt_mask = rtl92de_update_interrupt_mask, .get_hw_reg = rtl92de_get_hw_reg, .set_hw_reg = rtl92de_set_hw_reg, - .update_rate_tbl = rtl92de_update_hal_rate_tbl, + .update_rate_tbl = rtl92d_update_hal_rate_tbl, .fill_tx_desc = rtl92de_tx_fill_desc, .fill_tx_cmddesc = rtl92de_tx_fill_cmddesc, - .query_rx_desc = rtl92de_rx_query_desc, - .set_channel_access = rtl92de_update_channel_access_setting, - .radio_onoff_checking = rtl92de_gpio_radio_on_off_checking, + .query_rx_desc = rtl92d_rx_query_desc, + .set_channel_access = rtl92d_update_channel_access_setting, + .radio_onoff_checking = rtl92d_gpio_radio_on_off_checking, .set_bw_mode = rtl92d_phy_set_bw_mode, .switch_channel = rtl92d_phy_sw_chnl, .dm_watchdog = rtl92de_dm_watchdog, .scan_operation_backup = rtl_phy_scan_operation_backup, .set_rf_power_state = rtl92d_phy_set_rf_power_state, .led_control = rtl92de_led_control, - .set_desc = rtl92de_set_desc, - .get_desc = rtl92de_get_desc, + .set_desc = rtl92d_set_desc, + .get_desc = rtl92d_get_desc, .is_tx_desc_closed = rtl92de_is_tx_desc_closed, .tx_polling = rtl92de_tx_polling, - .enable_hw_sec = rtl92de_enable_hw_security_config, - .set_key = rtl92de_set_key, + .enable_hw_sec = rtl92d_enable_hw_security_config, + .set_key = rtl92d_set_key, .get_bbreg = rtl92d_phy_query_bb_reg, .set_bbreg = rtl92d_phy_set_bb_reg, .get_rfreg = rtl92d_phy_query_rf_reg, diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/trx.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/trx.c index 2b9b352f778397c5c0f32d265770e54ae0053c6c..91bf399c9ef12ec532b5a8eaba62b3e3a06d1756 100644 --- a/drivers/net/wireless/realtek/rtlwifi/rtl8192de/trx.c +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192de/trx.c @@ -292,7 +292,7 @@ bool rtl92de_is_tx_desc_closed(struct ieee80211_hw *hw, struct rtl_pci *rtlpci = rtl_pcidev(rtl_pcipriv(hw)); struct rtl8192_tx_ring *ring = &rtlpci->tx_ring[hw_queue]; u8 *entry = (u8 *)(&ring->desc[ring->idx]); - u8 own = (u8)rtl92de_get_desc(hw, entry, true, HW_DESC_OWN); + u8 own = (u8)rtl92d_get_desc(hw, entry, true, HW_DESC_OWN); /* a beacon packet will only use the first * descriptor by defaut, and the own bit may not diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/Makefile b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/Makefile new file mode 100644 index 0000000000000000000000000000000000000000..569bfd3d50308e4ff56d6e65467bbe32dcdc71c9 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/Makefile @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0 +rtl8192du-objs := \ + dm.o \ + fw.o \ + hw.o \ + led.o \ + phy.o \ + rf.o \ + sw.o \ + table.o \ + trx.o + +obj-$(CONFIG_RTL8192DU) += rtl8192du.o diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/dm.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/dm.c new file mode 100644 index 0000000000000000000000000000000000000000..dd57707a9184b2c6f438e3c0060394f8fb99dac6 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/dm.c @@ -0,0 +1,120 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2024 Realtek Corporation.*/ + +#include "../wifi.h" +#include "../core.h" +#include "../rtl8192d/reg.h" +#include "../rtl8192d/def.h" +#include "../rtl8192d/dm_common.h" +#include "../rtl8192d/fw_common.h" +#include "dm.h" + +static void rtl92du_dm_init_1r_cca(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct ps_t *dm_pstable = &rtlpriv->dm_pstable; + + dm_pstable->pre_ccastate = CCA_MAX; + dm_pstable->cur_ccasate = CCA_MAX; +} + +static void rtl92du_dm_1r_cca(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct ps_t *dm_pstable = &rtlpriv->dm_pstable; + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + int pwdb = rtlpriv->dm_digtable.min_undec_pwdb_for_dm; + + if (rtlhal->macphymode != SINGLEMAC_SINGLEPHY || + rtlhal->current_bandtype != BAND_ON_5G) + return; + + if (pwdb != 0) { + if (dm_pstable->pre_ccastate == CCA_2R || + dm_pstable->pre_ccastate == CCA_MAX) + dm_pstable->cur_ccasate = (pwdb >= 35) ? CCA_1R : CCA_2R; + else + dm_pstable->cur_ccasate = (pwdb <= 30) ? CCA_2R : CCA_1R; + } else { + dm_pstable->cur_ccasate = CCA_MAX; + } + + if (dm_pstable->pre_ccastate == dm_pstable->cur_ccasate) + return; + + rtl_dbg(rtlpriv, COMP_BB_POWERSAVING, DBG_TRACE, + "Old CCA state: %d new CCA state: %d\n", + dm_pstable->pre_ccastate, dm_pstable->cur_ccasate); + + if (dm_pstable->cur_ccasate == CCA_1R) { + if (rtlpriv->phy.rf_type == RF_2T2R) + rtl_set_bbreg(hw, ROFDM0_TRXPATHENABLE, MASKBYTE0, 0x13); + else /* Is this branch reachable? */ + rtl_set_bbreg(hw, ROFDM0_TRXPATHENABLE, MASKBYTE0, 0x23); + } else { /* CCA_2R or CCA_MAX */ + rtl_set_bbreg(hw, ROFDM0_TRXPATHENABLE, MASKBYTE0, 0x33); + } +} + +static void rtl92du_dm_pwdb_monitor(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + const u32 max_macid = 32; + u32 temp; + + /* AP & ADHOC & MESH will return tmp */ + if (rtlpriv->mac80211.opmode != NL80211_IFTYPE_STATION) + return; + + /* Indicate Rx signal strength to FW. */ + if (rtlpriv->dm.useramask) { + temp = rtlpriv->dm.undec_sm_pwdb << 16; + temp |= max_macid << 8; + + rtl92d_fill_h2c_cmd(hw, H2C_RSSI_REPORT, 3, (u8 *)(&temp)); + } else { + rtl_write_byte(rtlpriv, 0x4fe, (u8)rtlpriv->dm.undec_sm_pwdb); + } +} + +void rtl92du_dm_init(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + + rtlpriv->dm.dm_type = DM_TYPE_BYDRIVER; + rtl_dm_diginit(hw, 0x20); + rtlpriv->dm_digtable.rx_gain_max = DM_DIG_FA_UPPER; + rtlpriv->dm_digtable.rx_gain_min = DM_DIG_FA_LOWER; + rtl92d_dm_init_edca_turbo(hw); + rtl92du_dm_init_1r_cca(hw); + rtl92d_dm_init_rate_adaptive_mask(hw); + rtl92d_dm_initialize_txpower_tracking(hw); +} + +void rtl92du_dm_watchdog(struct ieee80211_hw *hw) +{ + struct rtl_ps_ctl *ppsc = rtl_psc(rtl_priv(hw)); + bool fw_current_inpsmode = false; + bool fwps_awake = true; + + /* 1. RF is OFF. (No need to do DM.) + * 2. Fw is under power saving mode for FwLPS. + * (Prevent from SW/FW I/O racing.) + * 3. IPS workitem is scheduled. (Prevent from IPS sequence + * to be swapped with DM. + * 4. RFChangeInProgress is TRUE. + * (Prevent from broken by IPS/HW/SW Rf off.) + */ + + if (ppsc->rfpwr_state != ERFON || fw_current_inpsmode || + !fwps_awake || ppsc->rfchange_inprogress) + return; + + rtl92du_dm_pwdb_monitor(hw); + rtl92d_dm_false_alarm_counter_statistics(hw); + rtl92d_dm_find_minimum_rssi(hw); + rtl92d_dm_dig(hw); + rtl92d_dm_check_txpower_tracking_thermal_meter(hw); + rtl92d_dm_check_edca_turbo(hw); + rtl92du_dm_1r_cca(hw); +} diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/dm.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/dm.h new file mode 100644 index 0000000000000000000000000000000000000000..2f283bf1e4d8d81e6f011adabaa56d16f984c06b --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/dm.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2024 Realtek Corporation.*/ + +#ifndef __RTL92DU_DM_H__ +#define __RTL92DU_DM_H__ + +void rtl92du_dm_init(struct ieee80211_hw *hw); +void rtl92du_dm_watchdog(struct ieee80211_hw *hw); + +#endif diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/fw.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/fw.c new file mode 100644 index 0000000000000000000000000000000000000000..f74e4e84fe3914e814fd4474332b6c4de0ba1165 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/fw.c @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2024 Realtek Corporation.*/ + +#include "../wifi.h" +#include "../rtl8192d/reg.h" +#include "../rtl8192d/def.h" +#include "../rtl8192d/fw_common.h" +#include "fw.h" + +int rtl92du_download_fw(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + enum version_8192d version = rtlhal->version; + u8 *pfwheader; + u8 *pfwdata; + u32 fwsize; + int err; + + if (rtlpriv->max_fw_size == 0 || !rtlhal->pfirmware) + return 1; + + fwsize = rtlhal->fwsize; + pfwheader = rtlhal->pfirmware; + pfwdata = rtlhal->pfirmware; + rtlhal->fw_version = (u16)GET_FIRMWARE_HDR_VERSION(pfwheader); + rtlhal->fw_subversion = (u16)GET_FIRMWARE_HDR_SUB_VER(pfwheader); + + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + "FirmwareVersion(%d), FirmwareSubVersion(%d), Signature(%#x)\n", + rtlhal->fw_version, rtlhal->fw_subversion, + GET_FIRMWARE_HDR_SIGNATURE(pfwheader)); + + if (IS_FW_HEADER_EXIST(pfwheader)) { + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + "Shift 32 bytes for FW header!!\n"); + pfwdata = pfwdata + 32; + fwsize = fwsize - 32; + } + + if (rtl92d_is_fw_downloaded(rtlpriv)) + goto exit; + + /* If 8051 is running in RAM code, driver should + * inform Fw to reset by itself, or it will cause + * download Fw fail. + */ + if (rtl_read_byte(rtlpriv, REG_MCUFWDL) & BIT(7)) { + rtl92d_firmware_selfreset(hw); + rtl_write_byte(rtlpriv, REG_MCUFWDL, 0x00); + } + + rtl92d_enable_fw_download(hw, true); + rtl92d_write_fw(hw, version, pfwdata, fwsize); + rtl92d_enable_fw_download(hw, false); + + err = rtl92d_fw_free_to_go(hw); + if (err) + pr_err("fw is not ready to run!\n"); +exit: + err = rtl92d_fw_init(hw); + return err; +} diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/fw.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/fw.h new file mode 100644 index 0000000000000000000000000000000000000000..7904bfbda4ba90bc8f2a966c1442fdb78f0c3129 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/fw.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2024 Realtek Corporation.*/ + +#ifndef __RTL92DU_FW_H__ +#define __RTL92DU_FW_H__ + +int rtl92du_download_fw(struct ieee80211_hw *hw); + +#endif diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/hw.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/hw.c new file mode 100644 index 0000000000000000000000000000000000000000..700c6e2bcad19468e61075fa9b836348de0f637e --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/hw.c @@ -0,0 +1,1212 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2024 Realtek Corporation.*/ + +#include "../wifi.h" +#include "../cam.h" +#include "../usb.h" +#include "../rtl8192d/reg.h" +#include "../rtl8192d/def.h" +#include "../rtl8192d/dm_common.h" +#include "../rtl8192d/fw_common.h" +#include "../rtl8192d/hw_common.h" +#include "../rtl8192d/phy_common.h" +#include "phy.h" +#include "dm.h" +#include "fw.h" +#include "hw.h" +#include "trx.h" + +static void _rtl92du_set_bcn_ctrl_reg(struct ieee80211_hw *hw, + u8 set_bits, u8 clear_bits) +{ + struct rtl_usb *rtlusb = rtl_usbdev(rtl_usbpriv(hw)); + struct rtl_priv *rtlpriv = rtl_priv(hw); + + rtlusb->reg_bcn_ctrl_val |= set_bits; + rtlusb->reg_bcn_ctrl_val &= ~clear_bits; + rtl_write_byte(rtlpriv, REG_BCN_CTRL, (u8)rtlusb->reg_bcn_ctrl_val); +} + +static void _rtl92du_enable_bcn_sub_func(struct ieee80211_hw *hw) +{ + _rtl92du_set_bcn_ctrl_reg(hw, 0, BIT(1)); +} + +static void _rtl92du_disable_bcn_sub_func(struct ieee80211_hw *hw) +{ + _rtl92du_set_bcn_ctrl_reg(hw, BIT(1), 0); +} + +void rtl92du_get_hw_reg(struct ieee80211_hw *hw, u8 variable, u8 *val) +{ + struct rtl_mac *mac = rtl_mac(rtl_priv(hw)); + + switch (variable) { + case HW_VAR_RCR: + *((u32 *)val) = mac->rx_conf; + break; + default: + rtl92d_get_hw_reg(hw, variable, val); + break; + } +} + +void rtl92du_set_hw_reg(struct ieee80211_hw *hw, u8 variable, u8 *val) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_mac *mac = rtl_mac(rtlpriv); + + switch (variable) { + case HW_VAR_AC_PARAM: + rtl92d_dm_init_edca_turbo(hw); + break; + case HW_VAR_ACM_CTRL: { + u8 e_aci = *val; + union aci_aifsn *p_aci_aifsn = + (union aci_aifsn *)(&mac->ac[0].aifs); + u8 acm = p_aci_aifsn->f.acm; + u8 acm_ctrl = rtl_read_byte(rtlpriv, REG_ACMHWCTRL); + + if (acm) { + switch (e_aci) { + case AC0_BE: + acm_ctrl |= ACMHW_BEQEN; + break; + case AC2_VI: + acm_ctrl |= ACMHW_VIQEN; + break; + case AC3_VO: + acm_ctrl |= ACMHW_VOQEN; + break; + default: + rtl_dbg(rtlpriv, COMP_ERR, DBG_WARNING, + "HW_VAR_ACM_CTRL acm set failed: eACI is %d\n", + acm); + break; + } + } else { + switch (e_aci) { + case AC0_BE: + acm_ctrl &= (~ACMHW_BEQEN); + break; + case AC2_VI: + acm_ctrl &= (~ACMHW_VIQEN); + break; + case AC3_VO: + acm_ctrl &= (~ACMHW_VOQEN); + break; + default: + pr_err("%s:%d switch case %#x not processed\n", + __func__, __LINE__, e_aci); + break; + } + } + rtl_dbg(rtlpriv, COMP_QOS, DBG_TRACE, + "SetHwReg8190pci(): [HW_VAR_ACM_CTRL] Write 0x%X\n", + acm_ctrl); + rtl_write_byte(rtlpriv, REG_ACMHWCTRL, acm_ctrl); + break; + } + case HW_VAR_RCR: + mac->rx_conf = ((u32 *)val)[0]; + rtl_write_dword(rtlpriv, REG_RCR, mac->rx_conf); + break; + case HW_VAR_H2C_FW_JOINBSSRPT: { + u8 tmp_regcr, tmp_reg422; + bool recover = false; + u8 mstatus = *val; + + if (mstatus == RT_MEDIA_CONNECT) { + rtlpriv->cfg->ops->set_hw_reg(hw, + HW_VAR_AID, NULL); + tmp_regcr = rtl_read_byte(rtlpriv, REG_CR + 1); + rtl_write_byte(rtlpriv, REG_CR + 1, + tmp_regcr | ENSWBCN); + _rtl92du_set_bcn_ctrl_reg(hw, 0, EN_BCN_FUNCTION); + _rtl92du_set_bcn_ctrl_reg(hw, DIS_TSF_UDT, 0); + tmp_reg422 = rtl_read_byte(rtlpriv, + REG_FWHW_TXQ_CTRL + 2); + if (tmp_reg422 & (EN_BCNQ_DL >> 16)) + recover = true; + rtl_write_byte(rtlpriv, REG_FWHW_TXQ_CTRL + 2, + tmp_reg422 & ~(EN_BCNQ_DL >> 16)); + + /* We don't implement FW LPS so this is not needed. */ + /* rtl92d_set_fw_rsvdpagepkt(hw, 0); */ + + _rtl92du_set_bcn_ctrl_reg(hw, EN_BCN_FUNCTION, 0); + _rtl92du_set_bcn_ctrl_reg(hw, 0, DIS_TSF_UDT); + if (recover) + rtl_write_byte(rtlpriv, + REG_FWHW_TXQ_CTRL + 2, + tmp_reg422); + rtl_write_byte(rtlpriv, REG_CR + 1, + tmp_regcr & ~ENSWBCN); + } + rtl92d_set_fw_joinbss_report_cmd(hw, (*val)); + break; + } + case HW_VAR_CORRECT_TSF: { + u8 btype_ibss = val[0]; + + if (btype_ibss) + rtl92d_stop_tx_beacon(hw); + _rtl92du_set_bcn_ctrl_reg(hw, 0, EN_BCN_FUNCTION); + rtl_write_dword(rtlpriv, REG_TSFTR, + (u32)(mac->tsf & 0xffffffff)); + rtl_write_dword(rtlpriv, REG_TSFTR + 4, + (u32)((mac->tsf >> 32) & 0xffffffff)); + _rtl92du_set_bcn_ctrl_reg(hw, EN_BCN_FUNCTION, 0); + if (btype_ibss) + rtl92d_resume_tx_beacon(hw); + + break; + } + case HW_VAR_KEEP_ALIVE: + /* Avoid "switch case not processed" error. RTL8192DU doesn't + * need to do anything here, maybe. + */ + break; + default: + rtl92d_set_hw_reg(hw, variable, val); + break; + } +} + +static void _rtl92du_init_queue_reserved_page(struct ieee80211_hw *hw, + u8 out_ep_num, + u8 queue_sel) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + u32 txqpagenum, txqpageunit; + u32 txqremainingpage; + u32 numhq = 0; + u32 numlq = 0; + u32 numnq = 0; + u32 numpubq; + u32 value32; + + if (rtlhal->macphymode != SINGLEMAC_SINGLEPHY) { + numpubq = NORMAL_PAGE_NUM_PUBQ_92D_DUAL_MAC; + txqpagenum = TX_TOTAL_PAGE_NUMBER_92D_DUAL_MAC - numpubq; + } else { + numpubq = TEST_PAGE_NUM_PUBQ_92DU; + txqpagenum = TX_TOTAL_PAGE_NUMBER_92DU - numpubq; + } + + if (rtlhal->macphymode != SINGLEMAC_SINGLEPHY && out_ep_num == 3) { + numhq = NORMAL_PAGE_NUM_HPQ_92D_DUAL_MAC; + numlq = NORMAL_PAGE_NUM_LPQ_92D_DUAL_MAC; + numnq = NORMAL_PAGE_NUM_NORMALQ_92D_DUAL_MAC; + } else { + txqpageunit = txqpagenum / out_ep_num; + txqremainingpage = txqpagenum % out_ep_num; + + if (queue_sel & TX_SELE_HQ) + numhq = txqpageunit; + if (queue_sel & TX_SELE_LQ) + numlq = txqpageunit; + if (queue_sel & TX_SELE_NQ) + numnq = txqpageunit; + + /* HIGH priority queue always present in the + * configuration of 2 or 3 out-ep. Remainder pages + * assigned to High queue + */ + if (out_ep_num > 1 && txqremainingpage) + numhq += txqremainingpage; + } + + /* NOTE: This step done before writing REG_RQPN. */ + rtl_write_byte(rtlpriv, REG_RQPN_NPQ, (u8)numnq); + + /* TX DMA */ + u32p_replace_bits(&value32, numhq, HPQ_MASK); + u32p_replace_bits(&value32, numlq, LPQ_MASK); + u32p_replace_bits(&value32, numpubq, PUBQ_MASK); + value32 |= LD_RQPN; + rtl_write_dword(rtlpriv, REG_RQPN, value32); +} + +static void _rtl92du_init_tx_buffer_boundary(struct ieee80211_hw *hw, + u8 txpktbuf_bndy) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + + rtl_write_byte(rtlpriv, REG_TXPKTBUF_BCNQ_BDNY, txpktbuf_bndy); + rtl_write_byte(rtlpriv, REG_TXPKTBUF_MGQ_BDNY, txpktbuf_bndy); + + rtl_write_byte(rtlpriv, REG_TXPKTBUF_WMAC_LBK_BF_HD, txpktbuf_bndy); + + /* TXRKTBUG_PG_BNDY */ + rtl_write_byte(rtlpriv, REG_TRXFF_BNDY, txpktbuf_bndy); + + /* Beacon Head for TXDMA */ + rtl_write_byte(rtlpriv, REG_TDECTRL + 1, txpktbuf_bndy); +} + +static bool _rtl92du_llt_table_init(struct ieee80211_hw *hw, u8 txpktbuf_bndy) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + unsigned short i; + bool status; + u8 maxpage; + + if (rtlpriv->rtlhal.macphymode == SINGLEMAC_SINGLEPHY) + maxpage = 255; + else + maxpage = 127; + + for (i = 0; i < (txpktbuf_bndy - 1); i++) { + status = rtl92d_llt_write(hw, i, i + 1); + if (!status) + return status; + } + + /* end of list */ + status = rtl92d_llt_write(hw, txpktbuf_bndy - 1, 0xFF); + if (!status) + return status; + + /* Make the other pages as ring buffer + * This ring buffer is used as beacon buffer if we + * config this MAC as two MAC transfer. + * Otherwise used as local loopback buffer. + */ + for (i = txpktbuf_bndy; i < maxpage; i++) { + status = rtl92d_llt_write(hw, i, i + 1); + if (!status) + return status; + } + + /* Let last entry point to the start entry of ring buffer */ + status = rtl92d_llt_write(hw, maxpage, txpktbuf_bndy); + if (!status) + return status; + + return true; +} + +static void _rtl92du_init_chipn_reg_priority(struct ieee80211_hw *hw, u16 beq, + u16 bkq, u16 viq, u16 voq, + u16 mgtq, u16 hiq) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u16 value16; + + value16 = rtl_read_word(rtlpriv, REG_TRXDMA_CTRL) & 0x7; + u16p_replace_bits(&value16, beq, TXDMA_BEQ_MAP); + u16p_replace_bits(&value16, bkq, TXDMA_BKQ_MAP); + u16p_replace_bits(&value16, viq, TXDMA_VIQ_MAP); + u16p_replace_bits(&value16, voq, TXDMA_VOQ_MAP); + u16p_replace_bits(&value16, mgtq, TXDMA_MGQ_MAP); + u16p_replace_bits(&value16, hiq, TXDMA_HIQ_MAP); + rtl_write_word(rtlpriv, REG_TRXDMA_CTRL, value16); +} + +static void _rtl92du_init_chipn_one_out_ep_priority(struct ieee80211_hw *hw, + u8 queue_sel) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u16 value; + + switch (queue_sel) { + case TX_SELE_HQ: + value = QUEUE_HIGH; + break; + case TX_SELE_LQ: + value = QUEUE_LOW; + break; + case TX_SELE_NQ: + value = QUEUE_NORMAL; + break; + default: + WARN_ON(1); /* Shall not reach here! */ + return; + } + _rtl92du_init_chipn_reg_priority(hw, value, value, value, value, + value, value); + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + "Tx queue select: 0x%02x\n", queue_sel); +} + +static void _rtl92du_init_chipn_two_out_ep_priority(struct ieee80211_hw *hw, + u8 queue_sel) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u16 beq, bkq, viq, voq, mgtq, hiq; + u16 valuehi, valuelow; + + switch (queue_sel) { + default: + WARN_ON(1); + fallthrough; + case (TX_SELE_HQ | TX_SELE_LQ): + valuehi = QUEUE_HIGH; + valuelow = QUEUE_LOW; + break; + case (TX_SELE_NQ | TX_SELE_LQ): + valuehi = QUEUE_NORMAL; + valuelow = QUEUE_LOW; + break; + case (TX_SELE_HQ | TX_SELE_NQ): + valuehi = QUEUE_HIGH; + valuelow = QUEUE_NORMAL; + break; + } + + beq = valuelow; + bkq = valuelow; + viq = valuehi; + voq = valuehi; + mgtq = valuehi; + hiq = valuehi; + + _rtl92du_init_chipn_reg_priority(hw, beq, bkq, viq, voq, mgtq, hiq); + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + "Tx queue select: 0x%02x\n", queue_sel); +} + +static void _rtl92du_init_chipn_three_out_ep_priority(struct ieee80211_hw *hw, + u8 queue_sel) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u16 beq, bkq, viq, voq, mgtq, hiq; + + beq = QUEUE_LOW; + bkq = QUEUE_LOW; + viq = QUEUE_NORMAL; + voq = QUEUE_HIGH; + mgtq = QUEUE_HIGH; + hiq = QUEUE_HIGH; + + _rtl92du_init_chipn_reg_priority(hw, beq, bkq, viq, voq, mgtq, hiq); + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + "Tx queue select: 0x%02x\n", queue_sel); +} + +static void _rtl92du_init_queue_priority(struct ieee80211_hw *hw, + u8 out_ep_num, + u8 queue_sel) +{ + switch (out_ep_num) { + case 1: + _rtl92du_init_chipn_one_out_ep_priority(hw, queue_sel); + break; + case 2: + _rtl92du_init_chipn_two_out_ep_priority(hw, queue_sel); + break; + case 3: + _rtl92du_init_chipn_three_out_ep_priority(hw, queue_sel); + break; + default: + WARN_ON(1); /* Shall not reach here! */ + break; + } +} + +static void _rtl92du_init_wmac_setting(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_mac *mac = rtl_mac(rtlpriv); + + mac->rx_conf = RCR_APM | RCR_AM | RCR_AB | RCR_ADF | RCR_APP_ICV | + RCR_AMF | RCR_HTC_LOC_CTRL | RCR_APP_MIC | + RCR_APP_PHYST_RXFF | RCR_APPFCS; + + rtl_write_dword(rtlpriv, REG_RCR, mac->rx_conf); + + /* Set Multicast Address. */ + rtl_write_dword(rtlpriv, REG_MAR, 0xffffffff); + rtl_write_dword(rtlpriv, REG_MAR + 4, 0xffffffff); +} + +static void _rtl92du_init_adaptive_ctrl(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u32 val32; + + val32 = rtl_read_dword(rtlpriv, REG_RRSR); + val32 &= ~0xfffff; + if (rtlpriv->rtlhal.current_bandtype == BAND_ON_5G) + val32 |= 0xffff0; /* No CCK */ + else + val32 |= 0xffff1; + rtl_write_dword(rtlpriv, REG_RRSR, val32); + + /* Set Spec SIFS (used in NAV) */ + rtl_write_word(rtlpriv, REG_SPEC_SIFS, 0x1010); + + /* Retry limit 0x30 */ + rtl_write_word(rtlpriv, REG_RL, 0x3030); +} + +static void _rtl92du_init_edca(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u16 val16; + + /* Disable EDCCA count down, to reduce collison and retry */ + val16 = rtl_read_word(rtlpriv, REG_RD_CTRL); + val16 |= DIS_EDCA_CNT_DWN; + rtl_write_word(rtlpriv, REG_RD_CTRL, val16); + + /* CCK SIFS shall always be 10us. */ + rtl_write_word(rtlpriv, REG_SIFS_CTX, 0x0a0a); + /* Set SIFS for OFDM */ + rtl_write_word(rtlpriv, REG_SIFS_TRX, 0x1010); + + rtl_write_word(rtlpriv, REG_PROT_MODE_CTRL, 0x0204); + + rtl_write_dword(rtlpriv, REG_BAR_MODE_CTRL, 0x014004); + + /* TXOP */ + rtl_write_dword(rtlpriv, REG_EDCA_BE_PARAM, 0x005EA42B); + rtl_write_dword(rtlpriv, REG_EDCA_BK_PARAM, 0x0000A44F); + rtl_write_dword(rtlpriv, REG_EDCA_VI_PARAM, 0x005EA324); + rtl_write_dword(rtlpriv, REG_EDCA_VO_PARAM, 0x002FA226); + + rtl_write_byte(rtlpriv, REG_PIFS, 0x1C); + + rtl_write_byte(rtlpriv, REG_AGGR_BREAK_TIME, 0x16); + + rtl_write_word(rtlpriv, REG_NAV_PROT_LEN, 0x0040); + + rtl_write_byte(rtlpriv, REG_BCNDMATIM, 0x2); + rtl_write_byte(rtlpriv, REG_ATIMWND, 0x2); +} + +static void _rtl92du_init_retry_function(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u8 val8; + + val8 = rtl_read_byte(rtlpriv, REG_FWHW_TXQ_CTRL); + val8 |= EN_AMPDU_RTY_NEW; + rtl_write_byte(rtlpriv, REG_FWHW_TXQ_CTRL, val8); + + rtl_write_byte(rtlpriv, REG_ACKTO, 0x40); +} + +static void _rtl92du_init_operation_mode(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + + rtl_write_byte(rtlpriv, REG_BWOPMODE, BW_OPMODE_20MHZ); + + switch (rtlpriv->phy.rf_type) { + case RF_1T2R: + case RF_1T1R: + rtlhal->minspace_cfg = (MAX_MSS_DENSITY_1T << 3); + break; + case RF_2T2R: + case RF_2T2R_GREEN: + rtlhal->minspace_cfg = (MAX_MSS_DENSITY_2T << 3); + break; + } + rtl_write_byte(rtlpriv, REG_AMPDU_MIN_SPACE, rtlhal->minspace_cfg); +} + +static void _rtl92du_init_beacon_parameters(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + + rtl_write_word(rtlpriv, REG_BCN_CTRL, 0x1010); + + rtl_write_word(rtlpriv, REG_TBTT_PROHIBIT, 0x3c02); + rtl_write_byte(rtlpriv, REG_DRVERLYINT, 0x05); + rtl_write_byte(rtlpriv, REG_BCNDMATIM, 0x03); + + rtl_write_word(rtlpriv, REG_BCNTCFG, 0x660f); +} + +static void _rtl92du_init_ampdu_aggregation(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + + /* Aggregation threshold */ + if (rtlhal->macphymode == DUALMAC_DUALPHY) + rtl_write_dword(rtlpriv, REG_AGGLEN_LMT, 0x66525541); + else if (rtlhal->macphymode == DUALMAC_SINGLEPHY) + rtl_write_dword(rtlpriv, REG_AGGLEN_LMT, 0x44444441); + else + rtl_write_dword(rtlpriv, REG_AGGLEN_LMT, 0x88728841); + + rtl_write_byte(rtlpriv, REG_AGGR_BREAK_TIME, 0x16); +} + +static bool _rtl92du_init_power_on(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + unsigned short wordtmp; + unsigned char bytetmp; + u16 retry = 0; + + do { + if (rtl_read_byte(rtlpriv, REG_APS_FSMCO) & PFM_ALDN) + break; + + if (retry++ > 1000) + return false; + } while (true); + + /* Unlock ISO/CLK/Power control register */ + rtl_write_byte(rtlpriv, REG_RSV_CTRL, 0x00); + + /* SPS0_CTRL 0x11[7:0] = 0x2b enable SPS into PWM mode */ + rtl_write_byte(rtlpriv, REG_SPS0_CTRL, 0x2b); + + msleep(1); + + bytetmp = rtl_read_byte(rtlpriv, REG_LDOV12D_CTRL); + if ((bytetmp & LDV12_EN) == 0) { + bytetmp |= LDV12_EN; + rtl_write_byte(rtlpriv, REG_LDOV12D_CTRL, bytetmp); + + msleep(1); + + bytetmp = rtl_read_byte(rtlpriv, REG_SYS_ISO_CTRL); + bytetmp &= ~ISO_MD2PP; + rtl_write_byte(rtlpriv, REG_SYS_ISO_CTRL, bytetmp); + } + + /* Auto enable WLAN */ + wordtmp = rtl_read_word(rtlpriv, REG_APS_FSMCO); + wordtmp |= APFM_ONMAC; + rtl_write_word(rtlpriv, REG_APS_FSMCO, wordtmp); + + wordtmp = rtl_read_word(rtlpriv, REG_APS_FSMCO); + retry = 0; + while ((wordtmp & APFM_ONMAC) && retry < 1000) { + retry++; + wordtmp = rtl_read_word(rtlpriv, REG_APS_FSMCO); + } + + /* Release RF digital isolation */ + wordtmp = rtl_read_word(rtlpriv, REG_SYS_ISO_CTRL); + wordtmp &= ~ISO_DIOR; + rtl_write_word(rtlpriv, REG_SYS_ISO_CTRL, wordtmp); + + /* Enable MAC DMA/WMAC/SCHEDULE/SEC block */ + wordtmp = rtl_read_word(rtlpriv, REG_CR); + wordtmp |= HCI_TXDMA_EN | HCI_RXDMA_EN | TXDMA_EN | RXDMA_EN | + PROTOCOL_EN | SCHEDULE_EN | MACTXEN | MACRXEN | ENSEC; + rtl_write_word(rtlpriv, REG_CR, wordtmp); + + return true; +} + +static bool _rtl92du_init_mac(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u8 val8; + + rtl_write_byte(rtlpriv, REG_RSV_CTRL, 0x00); + + val8 = rtl_read_byte(rtlpriv, REG_SYS_FUNC_EN + 1); + val8 &= ~(FEN_MREGEN >> 8); + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN + 1, val8); + + /* For s3/s4 may reset mac, Reg0xf8 may be set to 0, + * so reset macphy control reg here. + */ + rtl92d_phy_config_macphymode(hw); + + rtl92du_phy_set_poweron(hw); + + if (!_rtl92du_init_power_on(hw)) { + pr_err("Failed to init power on!\n"); + return false; + } + + rtl92d_phy_config_maccoexist_rfpage(hw); + + return true; +} + +int rtl92du_hw_init(struct ieee80211_hw *hw) +{ + struct rtl_usb_priv *usb_priv = rtl_usbpriv(hw); + struct rtl_usb *rtlusb = rtl_usbdev(usb_priv); + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_ps_ctl *ppsc = rtl_psc(rtlpriv); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + struct rtl_mac *mac = rtl_mac(rtlpriv); + struct rtl_phy *rtlphy = &rtlpriv->phy; + u8 val8, txpktbuf_bndy; + int err, i; + u32 val32; + u16 val16; + + mutex_lock(rtlpriv->mutex_for_hw_init); + + /* we should do iqk after disable/enable */ + rtl92d_phy_reset_iqk_result(hw); + + if (!_rtl92du_init_mac(hw)) { + pr_err("Init MAC failed\n"); + mutex_unlock(rtlpriv->mutex_for_hw_init); + return 1; + } + + if (rtlhal->macphymode == SINGLEMAC_SINGLEPHY) + txpktbuf_bndy = 249; + else + txpktbuf_bndy = 123; + + if (!_rtl92du_llt_table_init(hw, txpktbuf_bndy)) { + pr_err("Init LLT failed\n"); + mutex_unlock(rtlpriv->mutex_for_hw_init); + return 1; + } + + err = rtl92du_download_fw(hw); + + /* return fail only when part number check fail */ + if (err && rtl_read_byte(rtlpriv, 0x1c5) == 0xe0) { + rtl_dbg(rtlpriv, COMP_ERR, DBG_WARNING, + "Failed to download FW. Init HW without FW..\n"); + mutex_unlock(rtlpriv->mutex_for_hw_init); + return 1; + } + rtlhal->last_hmeboxnum = 0; + rtlpriv->psc.fw_current_inpsmode = false; + + rtl92du_phy_mac_config(hw); + + /* Set reserved page for each queue */ + _rtl92du_init_queue_reserved_page(hw, rtlusb->out_ep_nums, + rtlusb->out_queue_sel); + + _rtl92du_init_tx_buffer_boundary(hw, txpktbuf_bndy); + + _rtl92du_init_queue_priority(hw, rtlusb->out_ep_nums, + rtlusb->out_queue_sel); + + /* Set Tx/Rx page size (Tx must be 128 Bytes, + * Rx can be 64, 128, 256, 512, 1024 bytes) + */ + rtl_write_byte(rtlpriv, REG_PBP, 0x11); + + /* Get Rx PHY status in order to report RSSI and others. */ + rtl_write_byte(rtlpriv, REG_RX_DRVINFO_SZ, 0x4); + + rtl_write_dword(rtlpriv, REG_HISR, 0xffffffff); + rtl_write_dword(rtlpriv, REG_HIMR, 0xffffffff); + + val8 = rtl_read_byte(rtlpriv, MSR); + val8 &= ~MSR_MASK; + val8 |= MSR_INFRA; + rtl_write_byte(rtlpriv, MSR, val8); + + _rtl92du_init_wmac_setting(hw); + _rtl92du_init_adaptive_ctrl(hw); + _rtl92du_init_edca(hw); + + rtl_write_dword(rtlpriv, REG_DARFRC, 0x00000000); + rtl_write_dword(rtlpriv, REG_DARFRC + 4, 0x10080404); + rtl_write_dword(rtlpriv, REG_RARFRC, 0x04030201); + rtl_write_dword(rtlpriv, REG_RARFRC + 4, 0x08070605); + + _rtl92du_init_retry_function(hw); + /* _InitUsbAggregationSetting(padapter); no aggregation for now */ + _rtl92du_init_operation_mode(hw); + _rtl92du_init_beacon_parameters(hw); + _rtl92du_init_ampdu_aggregation(hw); + + rtl_write_byte(rtlpriv, REG_BCN_MAX_ERR, 0xff); + + /* unit: 256us. 256ms */ + rtl_write_word(rtlpriv, REG_PKT_VO_VI_LIFE_TIME, 0x0400); + rtl_write_word(rtlpriv, REG_PKT_BE_BK_LIFE_TIME, 0x0400); + + /* Hardware-controlled blinking. */ + rtl_write_word(rtlpriv, REG_LEDCFG0, 0x8282); + rtl_write_byte(rtlpriv, REG_LEDCFG2, 0x82); + + val32 = rtl_read_dword(rtlpriv, REG_TXDMA_OFFSET_CHK); + val32 |= DROP_DATA_EN; + rtl_write_dword(rtlpriv, REG_TXDMA_OFFSET_CHK, val32); + + if (mac->rdg_en) { + rtl_write_byte(rtlpriv, REG_RD_CTRL, 0xff); + rtl_write_word(rtlpriv, REG_RD_NAV_NXT, 0x200); + rtl_write_byte(rtlpriv, REG_RD_RESP_PKT_TH, 0x05); + } + + for (i = 0; i < 4; i++) + rtl_write_dword(rtlpriv, REG_ARFR0 + i * 4, 0x1f8ffff0); + + if (rtlhal->macphymode == SINGLEMAC_SINGLEPHY) { + if (rtlusb->out_ep_nums == 2) + rtl_write_dword(rtlpriv, REG_FAST_EDCA_CTRL, 0x03066666); + else + rtl_write_word(rtlpriv, REG_FAST_EDCA_CTRL, 0x8888); + } else { + rtl_write_word(rtlpriv, REG_FAST_EDCA_CTRL, 0x5555); + } + + val8 = rtl_read_byte(rtlpriv, 0x605); + val8 |= 0xf0; + rtl_write_byte(rtlpriv, 0x605, val8); + + rtl_write_byte(rtlpriv, REG_RXTSF_OFFSET_CCK, 0x30); + rtl_write_byte(rtlpriv, REG_RXTSF_OFFSET_OFDM, 0x30); + rtl_write_byte(rtlpriv, 0x606, 0x30); + + /* temp for high queue and mgnt Queue corrupt in time; it may + * cause hang when sw beacon use high_Q, other frame use mgnt_Q; + * or, sw beacon use mgnt_Q, other frame use high_Q; + */ + rtl_write_byte(rtlpriv, REG_DIS_TXREQ_CLR, 0x10); + val16 = rtl_read_word(rtlpriv, REG_RD_CTRL); + val16 |= BIT(12); + rtl_write_word(rtlpriv, REG_RD_CTRL, val16); + + rtl_write_byte(rtlpriv, REG_TXPAUSE, 0); + + /* usb suspend idle time count for bitfile0927 */ + val8 = rtl_read_byte(rtlpriv, 0xfe56); + val8 |= BIT(0) | BIT(1); + rtl_write_byte(rtlpriv, 0xfe56, val8); + + if (rtlhal->earlymode_enable) { + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + "EarlyMode Enabled!!!\n"); + + val8 = rtl_read_byte(rtlpriv, REG_EARLY_MODE_CONTROL); + val8 |= 0x1f; + rtl_write_byte(rtlpriv, REG_EARLY_MODE_CONTROL, val8); + + rtl_write_byte(rtlpriv, REG_EARLY_MODE_CONTROL + 3, 0x80); + + val8 = rtl_read_byte(rtlpriv, 0x605); + val8 |= 0x40; + rtl_write_byte(rtlpriv, 0x605, val8); + } else { + rtl_write_byte(rtlpriv, REG_EARLY_MODE_CONTROL, 0); + } + + rtl92du_phy_bb_config(hw); + + rtlphy->rf_mode = RF_OP_BY_SW_3WIRE; + /* set before initialize RF */ + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER4, 0x00f00000, 0xf); + + /* config RF */ + rtl92du_phy_rf_config(hw); + + /* set default value after initialize RF */ + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER4, 0x00f00000, 0); + + /* After load BB, RF params, we need to do more for 92D. */ + rtl92du_update_bbrf_configuration(hw); + + rtlphy->rfreg_chnlval[0] = + rtl_get_rfreg(hw, RF90_PATH_A, RF_CHNLBW, RFREG_OFFSET_MASK); + rtlphy->rfreg_chnlval[1] = + rtl_get_rfreg(hw, RF90_PATH_B, RF_CHNLBW, RFREG_OFFSET_MASK); + + /*---- Set CCK and OFDM Block "ON"----*/ + if (rtlhal->current_bandtype == BAND_ON_2_4G) + rtl_set_bbreg(hw, RFPGA0_RFMOD, BCCKEN, 0x1); + rtl_set_bbreg(hw, RFPGA0_RFMOD, BOFDMEN, 0x1); + + /* reset hw sec */ + rtl_cam_reset_all_entry(hw); + rtl92d_enable_hw_security_config(hw); + + rtl_write_byte(rtlpriv, REG_HWSEQ_CTRL, 0xFF); + + /* schmitt trigger, improve tx evm for 92du */ + val8 = rtl_read_byte(rtlpriv, REG_AFE_XTAL_CTRL); + val8 |= BIT(1); + rtl_write_byte(rtlpriv, REG_AFE_XTAL_CTRL, val8); + + /* Disable bar */ + rtl_write_dword(rtlpriv, REG_BAR_MODE_CTRL, 0xffff); + + /* Nav limit */ + rtl_write_byte(rtlpriv, REG_NAV_CTRL + 2, 0); + rtl_write_byte(rtlpriv, ROFDM0_XATXAFE + 3, 0x50); + + /* Read EEPROM TX power index and PHY_REG_PG.txt to capture correct + * TX power index for different rate set. + */ + rtl92d_phy_get_hw_reg_originalvalue(hw); + + ppsc->rfpwr_state = ERFON; + + /* do IQK for 2.4G for better scan result */ + if (rtlhal->current_bandtype == BAND_ON_2_4G) + rtl92du_phy_iq_calibrate(hw); + + rtl92du_phy_lc_calibrate(hw, IS_92D_SINGLEPHY(rtlhal->version)); + + rtl92du_phy_init_pa_bias(hw); + + mutex_unlock(rtlpriv->mutex_for_hw_init); + + rtl92du_dm_init(hw); + + /* For 2 PORT TSF SYNC */ + rtl_write_word(rtlpriv, REG_BCN_CTRL, 0x1818); + rtlusb->reg_bcn_ctrl_val = 0x18; + + udelay(500); + + if (rtlhal->macphymode != DUALMAC_DUALPHY) { + rtl_write_dword(rtlpriv, RFPGA1_TXINFO, + rtl_read_dword(rtlpriv, RFPGA1_TXINFO) & ~BIT(30)); + + rtl_write_dword(rtlpriv, RFPGA0_TXGAINSTAGE, + rtl_read_dword(rtlpriv, RFPGA0_TXGAINSTAGE) & ~BIT(31)); + + rtl_write_dword(rtlpriv, ROFDM0_XBTXAFE, 0xa0e40000); + } + + val32 = rtl_read_dword(rtlpriv, REG_FWHW_TXQ_CTRL); + val32 |= BIT(12); + rtl_write_dword(rtlpriv, REG_FWHW_TXQ_CTRL, val32); + + return err; +} + +static int _rtl92du_set_media_status(struct ieee80211_hw *hw, + enum nl80211_iftype type) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + enum led_ctl_mode ledaction = LED_CTL_NO_LINK; + u8 bt_msr = rtl_read_byte(rtlpriv, MSR); + + bt_msr &= 0xfc; + + if (type == NL80211_IFTYPE_UNSPECIFIED || + type == NL80211_IFTYPE_STATION) { + rtl92d_stop_tx_beacon(hw); + _rtl92du_enable_bcn_sub_func(hw); + } else if (type == NL80211_IFTYPE_ADHOC || + type == NL80211_IFTYPE_AP) { + rtl92d_resume_tx_beacon(hw); + _rtl92du_disable_bcn_sub_func(hw); + } else { + rtl_dbg(rtlpriv, COMP_ERR, DBG_WARNING, + "Set HW_VAR_MEDIA_STATUS: No such media status(%x)\n", + type); + } + + switch (type) { + case NL80211_IFTYPE_UNSPECIFIED: + bt_msr |= MSR_NOLINK; + ledaction = LED_CTL_LINK; + rtl_dbg(rtlpriv, COMP_INIT, DBG_TRACE, + "Set Network type to NO LINK!\n"); + break; + case NL80211_IFTYPE_ADHOC: + bt_msr |= MSR_ADHOC; + rtl_dbg(rtlpriv, COMP_INIT, DBG_TRACE, + "Set Network type to Ad Hoc!\n"); + break; + case NL80211_IFTYPE_STATION: + bt_msr |= MSR_INFRA; + ledaction = LED_CTL_LINK; + rtl_dbg(rtlpriv, COMP_INIT, DBG_TRACE, + "Set Network type to STA!\n"); + break; + case NL80211_IFTYPE_AP: + bt_msr |= MSR_AP; + rtl_dbg(rtlpriv, COMP_INIT, DBG_TRACE, + "Set Network type to AP!\n"); + break; + default: + pr_err("Network type %d not supported!\n", type); + return 1; + } + rtl_write_byte(rtlpriv, MSR, bt_msr); + + rtlpriv->cfg->ops->led_control(hw, ledaction); + + if ((bt_msr & MSR_MASK) == MSR_AP) + rtl_write_byte(rtlpriv, REG_BCNTCFG + 1, 0x00); + else + rtl_write_byte(rtlpriv, REG_BCNTCFG + 1, 0x66); + + return 0; +} + +void rtl92du_set_check_bssid(struct ieee80211_hw *hw, bool check_bssid) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u32 reg_rcr; + + if (rtlpriv->psc.rfpwr_state != ERFON) + return; + + rtlpriv->cfg->ops->get_hw_reg(hw, HW_VAR_RCR, (u8 *)(®_rcr)); + + if (check_bssid) { + reg_rcr |= RCR_CBSSID_DATA | RCR_CBSSID_BCN; + rtlpriv->cfg->ops->set_hw_reg(hw, HW_VAR_RCR, (u8 *)®_rcr); + _rtl92du_set_bcn_ctrl_reg(hw, 0, DIS_TSF_UDT); + } else if (!check_bssid) { + reg_rcr &= ~(RCR_CBSSID_DATA | RCR_CBSSID_BCN); + _rtl92du_set_bcn_ctrl_reg(hw, DIS_TSF_UDT, 0); + rtlpriv->cfg->ops->set_hw_reg(hw, HW_VAR_RCR, (u8 *)®_rcr); + } +} + +int rtl92du_set_network_type(struct ieee80211_hw *hw, enum nl80211_iftype type) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + + if (_rtl92du_set_media_status(hw, type)) + return -EOPNOTSUPP; + + /* check bssid */ + if (rtlpriv->mac80211.link_state == MAC80211_LINKED) { + if (type != NL80211_IFTYPE_AP) + rtl92du_set_check_bssid(hw, true); + } else { + rtl92du_set_check_bssid(hw, false); + } + + return 0; +} + +/* do iqk or reload iqk */ +/* windows just rtl92d_phy_reload_iqk_setting in set channel, + * but it's very strict for time sequence so we add + * rtl92d_phy_reload_iqk_setting here + */ +void rtl92du_linked_set_reg(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_phy *rtlphy = &rtlpriv->phy; + u8 channel = rtlphy->current_channel; + u8 indexforchannel; + + indexforchannel = rtl92d_get_rightchnlplace_for_iqk(channel); + if (!rtlphy->iqk_matrix[indexforchannel].iqk_done) { + rtl_dbg(rtlpriv, COMP_SCAN | COMP_INIT, DBG_DMESG, + "Do IQK for channel:%d\n", channel); + rtl92du_phy_iq_calibrate(hw); + } +} + +void rtl92du_enable_interrupt(struct ieee80211_hw *hw) +{ + /* Nothing to do. */ +} + +void rtl92du_disable_interrupt(struct ieee80211_hw *hw) +{ + /* Nothing to do. */ +} + +static void _rtl92du_poweroff_adapter(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u8 retry = 100; + u8 u1b_tmp; + u16 val16; + u32 val32; + + rtl_write_byte(rtlpriv, REG_LDOA15_CTRL, 0x04); + + rtl_write_byte(rtlpriv, REG_RF_CTRL, 0x00); + + /* IF fw in RAM code, do reset */ + if (rtl_read_byte(rtlpriv, REG_MCUFWDL) & MCUFWDL_RDY) { + rtl_write_byte(rtlpriv, REG_FSIMR, 0); + + /* We need to disable other HRCV INT to influence 8051 reset. */ + rtl_write_byte(rtlpriv, REG_FWIMR, 0x20); + + /* Close mask to prevent incorrect FW write operation. */ + rtl_write_byte(rtlpriv, REG_FTIMR, 0); + + rtl_write_byte(rtlpriv, REG_MCUFWDL, 0); + + /* Set (REG_HMETFR + 3) to 0x20 is reset 8051 */ + rtl_write_byte(rtlpriv, REG_HMETFR + 3, 0x20); + val16 = rtl_read_word(rtlpriv, REG_SYS_FUNC_EN); + while (val16 & FEN_CPUEN) { + retry--; + if (retry == 0) + break; + udelay(50); + val16 = rtl_read_word(rtlpriv, REG_SYS_FUNC_EN); + } + + if (retry == 0) { + rtl_write_byte(rtlpriv, REG_FWIMR, 0); + + /* if 8051 reset fail, reset MAC directly. */ + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN + 1, 0x50); + + mdelay(10); + } + } + + /* reset MCU, MAC register, DCORE */ + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN + 1, 0x54); + + /* reset MCU ready status */ + rtl_write_byte(rtlpriv, REG_MCUFWDL, 0x00); + + /* Pull GPIO PIN to balance level and LED control */ + + /* Disable GPIO[7:0] */ + rtl_write_word(rtlpriv, REG_GPIO_PIN_CTRL + 2, 0x0000); + val32 = rtl_read_dword(rtlpriv, REG_GPIO_PIN_CTRL); + u32p_replace_bits(&val32, val32 & 0xff, 0x0000ff00); + u32p_replace_bits(&val32, 0xff, 0x00ff0000); + rtl_write_dword(rtlpriv, REG_GPIO_PIN_CTRL, val32); + + /* Disable GPIO[10:8] */ + rtl_write_byte(rtlpriv, REG_MAC_PINMUX_CFG, 0); + val16 = rtl_read_word(rtlpriv, REG_GPIO_IO_SEL); + u16p_replace_bits(&val16, val16 & 0xf, 0x00f0); + u16p_replace_bits(&val16, 0xf, 0x0780); + rtl_write_word(rtlpriv, REG_GPIO_IO_SEL, val16); + + /* Disable LED 0, 1, and 2 */ + rtl_write_word(rtlpriv, REG_LEDCFG0, 0x8888); + rtl_write_byte(rtlpriv, REG_LEDCFG2, 0x88); + + /* Disable analog sequence */ + + /* enter PFM mode */ + rtl_write_byte(rtlpriv, REG_SPS0_CTRL, 0x23); + + rtl_write_word(rtlpriv, REG_APS_FSMCO, + APDM_HOST | AFSM_HSUS | PFM_ALDN); + + /* lock ISO/CLK/Power control register */ + rtl_write_byte(rtlpriv, REG_RSV_CTRL, 0x0e); + + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + "In PowerOff,reg0x%x=%X\n", + REG_SPS0_CTRL, rtl_read_byte(rtlpriv, REG_SPS0_CTRL)); + + /* 0x17[7] 1b': power off in process 0b' : power off over */ + if (rtlpriv->rtlhal.macphymode != SINGLEMAC_SINGLEPHY) { + mutex_lock(rtlpriv->mutex_for_power_on_off); + u1b_tmp = rtl_read_byte(rtlpriv, REG_POWER_OFF_IN_PROCESS); + u1b_tmp &= ~BIT(7); + rtl_write_byte(rtlpriv, REG_POWER_OFF_IN_PROCESS, u1b_tmp); + mutex_unlock(rtlpriv->mutex_for_power_on_off); + } + + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, "<=======\n"); +} + +void rtl92du_card_disable(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_ps_ctl *ppsc = rtl_psc(rtl_priv(hw)); + struct rtl_hal *rtlhal = rtl_hal(rtl_priv(hw)); + struct rtl_mac *mac = rtl_mac(rtl_priv(hw)); + enum nl80211_iftype opmode; + u32 val32; + u16 val16; + u8 val8; + + mac->link_state = MAC80211_NOLINK; + opmode = NL80211_IFTYPE_UNSPECIFIED; + _rtl92du_set_media_status(hw, opmode); + + RT_SET_PS_LEVEL(ppsc, RT_RF_OFF_LEVL_HALT_NIC); + /* Power sequence for each MAC. */ + /* a. stop tx DMA */ + /* b. close RF */ + /* c. clear rx buf */ + /* d. stop rx DMA */ + /* e. reset MAC */ + + val16 = rtl_read_word(rtlpriv, REG_GPIO_MUXCFG); + val16 &= ~BIT(12); + rtl_write_word(rtlpriv, REG_GPIO_MUXCFG, val16); + + rtl_write_byte(rtlpriv, REG_TXPAUSE, 0xff); + udelay(500); + rtl_write_byte(rtlpriv, REG_CR, 0); + + /* RF OFF sequence */ + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER4, 0x00f00000, 0xf); + rtl_set_rfreg(hw, RF90_PATH_A, RF_AC, RFREG_OFFSET_MASK, 0x00); + + rtl_write_byte(rtlpriv, REG_APSD_CTRL, 0x40); + + val8 = FEN_USBD | FEN_USBA | FEN_BB_GLB_RSTN; + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN, val8); + + /* Mac0 can not do Global reset. Mac1 can do. */ + if (rtlhal->macphymode == SINGLEMAC_SINGLEPHY || + rtlhal->interfaceindex == 1) { + /* before BB reset should do clock gated */ + val32 = rtl_read_dword(rtlpriv, RFPGA0_XCD_RFPARAMETER); + val32 |= BIT(31); + rtl_write_dword(rtlpriv, RFPGA0_XCD_RFPARAMETER, val32); + + val8 &= ~FEN_BB_GLB_RSTN; + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN, val8); + } + + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, "==> Do power off.......\n"); + if (!rtl92du_phy_check_poweroff(hw)) + return; + + _rtl92du_poweroff_adapter(hw); +} + +void rtl92du_set_beacon_related_registers(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_mac *mac = rtl_mac(rtlpriv); + u16 bcn_interval, atim_window; + + bcn_interval = mac->beacon_interval; + atim_window = 2; + rtl92du_disable_interrupt(hw); + rtl_write_word(rtlpriv, REG_ATIMWND, atim_window); + rtl_write_word(rtlpriv, REG_BCN_INTERVAL, bcn_interval); + rtl_write_word(rtlpriv, REG_BCNTCFG, 0x660f); + rtl_write_byte(rtlpriv, REG_RXTSF_OFFSET_CCK, 0x20); + if (rtlpriv->rtlhal.current_bandtype == BAND_ON_5G) + rtl_write_byte(rtlpriv, REG_RXTSF_OFFSET_OFDM, 0x30); + else + rtl_write_byte(rtlpriv, REG_RXTSF_OFFSET_OFDM, 0x20); + rtl_write_byte(rtlpriv, 0x606, 0x30); +} + +void rtl92du_set_beacon_interval(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_mac *mac = rtl_mac(rtl_priv(hw)); + u16 bcn_interval = mac->beacon_interval; + + rtl_dbg(rtlpriv, COMP_BEACON, DBG_DMESG, + "beacon_interval:%d\n", bcn_interval); + rtl92du_disable_interrupt(hw); + rtl_write_word(rtlpriv, REG_BCN_INTERVAL, bcn_interval); + rtl92du_enable_interrupt(hw); +} + +void rtl92du_update_interrupt_mask(struct ieee80211_hw *hw, + u32 add_msr, u32 rm_msr) +{ + /* Nothing to do here. */ +} + +void rtl92du_read_chip_version(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + + /* Chip version reading is done in rtl92d_read_eeprom_info. */ + + rtlpriv->rtlhal.hw_type = HARDWARE_TYPE_RTL8192DU; +} diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/hw.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/hw.h new file mode 100644 index 0000000000000000000000000000000000000000..80ed00c90c16989315e2865c28d9f60bdf475aa3 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/hw.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2024 Realtek Corporation.*/ + +#ifndef __RTL92DU_HW_H__ +#define __RTL92DU_HW_H__ + +void rtl92du_get_hw_reg(struct ieee80211_hw *hw, u8 variable, u8 *val); +void rtl92du_set_hw_reg(struct ieee80211_hw *hw, u8 variable, u8 *val); +void rtl92du_read_chip_version(struct ieee80211_hw *hw); +int rtl92du_hw_init(struct ieee80211_hw *hw); +void rtl92du_card_disable(struct ieee80211_hw *hw); +void rtl92du_enable_interrupt(struct ieee80211_hw *hw); +void rtl92du_disable_interrupt(struct ieee80211_hw *hw); +int rtl92du_set_network_type(struct ieee80211_hw *hw, enum nl80211_iftype type); +void rtl92du_set_check_bssid(struct ieee80211_hw *hw, bool check_bssid); +void rtl92du_set_beacon_related_registers(struct ieee80211_hw *hw); +void rtl92du_set_beacon_interval(struct ieee80211_hw *hw); +void rtl92du_update_interrupt_mask(struct ieee80211_hw *hw, + u32 add_msr, u32 rm_msr); +void rtl92du_linked_set_reg(struct ieee80211_hw *hw); + +#endif diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/led.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/led.c new file mode 100644 index 0000000000000000000000000000000000000000..6c12dfbd6367df5b1c7f86fe9e7fd946b11bfbac --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/led.c @@ -0,0 +1,10 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2024 Realtek Corporation.*/ + +#include "../wifi.h" +#include "led.h" + +void rtl92du_led_control(struct ieee80211_hw *hw, enum led_ctl_mode ledaction) +{ + /* The hardware has control. */ +} diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/led.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/led.h new file mode 100644 index 0000000000000000000000000000000000000000..d7ebc8afcc7b1253053ec38b57e3e53517da1580 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/led.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2024 Realtek Corporation.*/ + +#ifndef __RTL92DU_LED_H__ +#define __RTL92DU_LED_H__ + +void rtl92du_led_control(struct ieee80211_hw *hw, enum led_ctl_mode ledaction); + +#endif diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/phy.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/phy.c new file mode 100644 index 0000000000000000000000000000000000000000..289ec71ce3e5641a51f3030432cf49756e215963 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/phy.c @@ -0,0 +1,3123 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2024 Realtek Corporation.*/ + +#include "../wifi.h" +#include "../ps.h" +#include "../core.h" +#include "../efuse.h" +#include "../usb.h" +#include "../rtl8192d/reg.h" +#include "../rtl8192d/def.h" +#include "../rtl8192d/phy_common.h" +#include "../rtl8192d/rf_common.h" +#include "phy.h" +#include "rf.h" +#include "table.h" + +#define MAX_RF_IMR_INDEX 12 +#define MAX_RF_IMR_INDEX_NORMAL 13 +#define RF_REG_NUM_FOR_C_CUT_5G 6 +#define RF_REG_NUM_FOR_C_CUT_5G_INTERNALPA 7 +#define RF_REG_NUM_FOR_C_CUT_2G 5 +#define RF_CHNL_NUM_5G 19 +#define RF_CHNL_NUM_5G_40M 17 +#define CV_CURVE_CNT 64 + +static const u32 rf_reg_for_5g_swchnl_normal[MAX_RF_IMR_INDEX_NORMAL] = { + 0, 0x2f, 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x0 +}; + +static const u8 rf_reg_for_c_cut_5g[RF_REG_NUM_FOR_C_CUT_5G] = { + RF_SYN_G1, RF_SYN_G2, RF_SYN_G3, RF_SYN_G4, RF_SYN_G5, RF_SYN_G6 +}; + +static const u8 rf_reg_for_c_cut_2g[RF_REG_NUM_FOR_C_CUT_2G] = { + RF_SYN_G1, RF_SYN_G2, RF_SYN_G3, RF_SYN_G7, RF_SYN_G8 +}; + +static const u8 rf_for_c_cut_5g_internal_pa[RF_REG_NUM_FOR_C_CUT_5G_INTERNALPA] = { + 0x0B, 0x48, 0x49, 0x4B, 0x03, 0x04, 0x0E +}; + +static const u32 rf_reg_mask_for_c_cut_2g[RF_REG_NUM_FOR_C_CUT_2G] = { + BIT(19) | BIT(18) | BIT(17) | BIT(14) | BIT(1), + BIT(10) | BIT(9), + BIT(18) | BIT(17) | BIT(16) | BIT(1), + BIT(2) | BIT(1), + BIT(15) | BIT(14) | BIT(13) | BIT(12) | BIT(11) +}; + +static const u8 rf_chnl_5g[RF_CHNL_NUM_5G] = { + 36, 40, 44, 48, 52, 56, 60, 64, 100, 104, 108, + 112, 116, 120, 124, 128, 132, 136, 140 +}; + +static const u8 rf_chnl_5g_40m[RF_CHNL_NUM_5G_40M] = { + 38, 42, 46, 50, 54, 58, 62, 102, 106, 110, 114, + 118, 122, 126, 130, 134, 138 +}; + +static const u32 rf_reg_pram_c_5g[5][RF_REG_NUM_FOR_C_CUT_5G] = { + {0xE43BE, 0xFC638, 0x77C0A, 0xDE471, 0xd7110, 0x8EB04}, + {0xE43BE, 0xFC078, 0xF7C1A, 0xE0C71, 0xD7550, 0xAEB04}, + {0xE43BF, 0xFF038, 0xF7C0A, 0xDE471, 0xE5550, 0xAEB04}, + {0xE43BF, 0xFF079, 0xF7C1A, 0xDE471, 0xE5550, 0xAEB04}, + {0xE43BF, 0xFF038, 0xF7C1A, 0xDE471, 0xd7550, 0xAEB04} +}; + +static const u32 rf_reg_param_for_c_cut_2g[3][RF_REG_NUM_FOR_C_CUT_2G] = { + {0x643BC, 0xFC038, 0x77C1A, 0x41289, 0x01840}, + {0x643BC, 0xFC038, 0x07C1A, 0x41289, 0x01840}, + {0x243BC, 0xFC438, 0x07C1A, 0x4128B, 0x0FC41} +}; + +static const u32 rf_syn_g4_for_c_cut_2g = 0xD1C31 & 0x7FF; + +static const u32 rf_pram_c_5g_int_pa[3][RF_REG_NUM_FOR_C_CUT_5G_INTERNALPA] = { + {0x01a00, 0x40443, 0x00eb5, 0x89bec, 0x94a12, 0x94a12, 0x94a12}, + {0x01800, 0xc0443, 0x00730, 0x896ee, 0x94a52, 0x94a52, 0x94a52}, + {0x01800, 0xc0443, 0x00730, 0x896ee, 0x94a12, 0x94a12, 0x94a12} +}; + +/* [patha+b][reg] */ +static const u32 rf_imr_param_normal[3][MAX_RF_IMR_INDEX_NORMAL] = { + /* channels 1-14. */ + { + 0x70000, 0x00ff0, 0x4400f, 0x00ff0, 0x0, 0x0, 0x0, + 0x0, 0x0, 0x64888, 0xe266c, 0x00090, 0x22fff + }, + /* channels 36-64 */ + { + 0x70000, 0x22880, 0x4470f, 0x55880, 0x00070, 0x88000, + 0x0, 0x88080, 0x70000, 0x64a82, 0xe466c, 0x00090, + 0x32c9a + }, + /* channels 100-165 */ + { + 0x70000, 0x44880, 0x4477f, 0x77880, 0x00070, 0x88000, + 0x0, 0x880b0, 0x0, 0x64b82, 0xe466c, 0x00090, 0x32c9a + } +}; + +static const u32 targetchnl_5g[TARGET_CHNL_NUM_5G] = { + 25141, 25116, 25091, 25066, 25041, + 25016, 24991, 24966, 24941, 24917, + 24892, 24867, 24843, 24818, 24794, + 24770, 24765, 24721, 24697, 24672, + 24648, 24624, 24600, 24576, 24552, + 24528, 24504, 24480, 24457, 24433, + 24409, 24385, 24362, 24338, 24315, + 24291, 24268, 24245, 24221, 24198, + 24175, 24151, 24128, 24105, 24082, + 24059, 24036, 24013, 23990, 23967, + 23945, 23922, 23899, 23876, 23854, + 23831, 23809, 23786, 23764, 23741, + 23719, 23697, 23674, 23652, 23630, + 23608, 23586, 23564, 23541, 23519, + 23498, 23476, 23454, 23432, 23410, + 23388, 23367, 23345, 23323, 23302, + 23280, 23259, 23237, 23216, 23194, + 23173, 23152, 23130, 23109, 23088, + 23067, 23046, 23025, 23003, 22982, + 22962, 22941, 22920, 22899, 22878, + 22857, 22837, 22816, 22795, 22775, + 22754, 22733, 22713, 22692, 22672, + 22652, 22631, 22611, 22591, 22570, + 22550, 22530, 22510, 22490, 22469, + 22449, 22429, 22409, 22390, 22370, + 22350, 22336, 22310, 22290, 22271, + 22251, 22231, 22212, 22192, 22173, + 22153, 22134, 22114, 22095, 22075, + 22056, 22037, 22017, 21998, 21979, + 21960, 21941, 21921, 21902, 21883, + 21864, 21845, 21826, 21807, 21789, + 21770, 21751, 21732, 21713, 21695, + 21676, 21657, 21639, 21620, 21602, + 21583, 21565, 21546, 21528, 21509, + 21491, 21473, 21454, 21436, 21418, + 21400, 21381, 21363, 21345, 21327, + 21309, 21291, 21273, 21255, 21237, + 21219, 21201, 21183, 21166, 21148, + 21130, 21112, 21095, 21077, 21059, + 21042, 21024, 21007, 20989, 20972, + 25679, 25653, 25627, 25601, 25575, + 25549, 25523, 25497, 25471, 25446, + 25420, 25394, 25369, 25343, 25318, + 25292, 25267, 25242, 25216, 25191, + 25166 +}; + +/* channel 1~14 */ +static const u32 targetchnl_2g[TARGET_CHNL_NUM_2G] = { + 26084, 26030, 25976, 25923, 25869, 25816, 25764, + 25711, 25658, 25606, 25554, 25502, 25451, 25328 +}; + +u32 rtl92du_phy_query_bb_reg(struct ieee80211_hw *hw, u32 regaddr, u32 bitmask) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + u32 returnvalue, originalvalue, bitshift; + + rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "regaddr(%#x), bitmask(%#x)\n", + regaddr, bitmask); + + if (rtlhal->during_mac1init_radioa) + regaddr |= MAC1_ACCESS_PHY0; + else if (rtlhal->during_mac0init_radiob) + regaddr |= MAC0_ACCESS_PHY1; + + originalvalue = rtl_read_dword(rtlpriv, regaddr); + bitshift = calculate_bit_shift(bitmask); + returnvalue = (originalvalue & bitmask) >> bitshift; + rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, + "BBR MASK=0x%x Addr[0x%x]=0x%x\n", + bitmask, regaddr, originalvalue); + return returnvalue; +} + +void rtl92du_phy_set_bb_reg(struct ieee80211_hw *hw, + u32 regaddr, u32 bitmask, u32 data) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + u32 originalvalue, bitshift; + + rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, + "regaddr(%#x), bitmask(%#x), data(%#x)\n", + regaddr, bitmask, data); + + if (rtlhal->during_mac1init_radioa) + regaddr |= MAC1_ACCESS_PHY0; + else if (rtlhal->during_mac0init_radiob) + regaddr |= MAC0_ACCESS_PHY1; + + if (bitmask != MASKDWORD) { + originalvalue = rtl_read_dword(rtlpriv, regaddr); + bitshift = calculate_bit_shift(bitmask); + data = (originalvalue & (~bitmask)) | + ((data << bitshift) & bitmask); + } + + rtl_write_dword(rtlpriv, regaddr, data); + rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, + "regaddr(%#x), bitmask(%#x), data(%#x)\n", + regaddr, bitmask, data); +} + +/* To avoid miswrite Reg0x800 for 92D */ +static void rtl92du_phy_set_bb_reg_1byte(struct ieee80211_hw *hw, + u32 regaddr, u32 bitmask, u32 data) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u32 originalvalue, bitshift, offset; + u8 value; + + /* BitMask only support bit0~bit7 or bit8~bit15, bit16~bit23, + * bit24~bit31, should be in 1 byte scale; + */ + bitshift = calculate_bit_shift(bitmask); + offset = bitshift / 8; + + originalvalue = rtl_read_dword(rtlpriv, regaddr); + data = (originalvalue & (~bitmask)) | ((data << bitshift) & bitmask); + + value = data >> (8 * offset); + + rtl_write_byte(rtlpriv, regaddr + offset, value); +} + +bool rtl92du_phy_mac_config(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u32 arraylength; + const u32 *ptrarray; + u32 i; + + rtl_dbg(rtlpriv, COMP_INIT, DBG_TRACE, "Read Rtl819XMACPHY_Array\n"); + + arraylength = MAC_2T_ARRAYLENGTH; + ptrarray = rtl8192du_mac_2tarray; + + for (i = 0; i < arraylength; i = i + 2) + rtl_write_byte(rtlpriv, ptrarray[i], (u8)ptrarray[i + 1]); + + if (rtlpriv->rtlhal.macphymode == SINGLEMAC_SINGLEPHY) { + /* improve 2-stream TX EVM */ + /* rtl_write_byte(rtlpriv, 0x14,0x71); */ + /* AMPDU aggregation number 9 */ + /* rtl_write_word(rtlpriv, REG_MAX_AGGR_NUM, MAX_AGGR_NUM); */ + rtl_write_byte(rtlpriv, REG_MAX_AGGR_NUM, 0x0B); + } else { + /* 92D need to test to decide the num. */ + rtl_write_byte(rtlpriv, REG_MAX_AGGR_NUM, 0x07); + } + + return true; +} + +static bool _rtl92du_phy_config_bb(struct ieee80211_hw *hw, u8 configtype) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + u16 phy_reg_arraylen, agctab_arraylen = 0; + const u32 *agctab_array_table = NULL; + const u32 *phy_regarray_table; + int i; + + /* Normal chip, Mac0 use AGC_TAB.txt for 2G and 5G band. */ + if (rtlhal->interfaceindex == 0) { + agctab_arraylen = AGCTAB_ARRAYLENGTH; + agctab_array_table = rtl8192du_agctab_array; + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + " ===> phy:MAC0, Rtl819XAGCTAB_Array\n"); + } else { + if (rtlhal->current_bandtype == BAND_ON_2_4G) { + agctab_arraylen = AGCTAB_2G_ARRAYLENGTH; + agctab_array_table = rtl8192du_agctab_2garray; + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + " ===> phy:MAC1, Rtl819XAGCTAB_2GArray\n"); + } else { + agctab_arraylen = AGCTAB_5G_ARRAYLENGTH; + agctab_array_table = rtl8192du_agctab_5garray; + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + " ===> phy:MAC1, Rtl819XAGCTAB_5GArray\n"); + } + } + phy_reg_arraylen = PHY_REG_2T_ARRAYLENGTH; + phy_regarray_table = rtl8192du_phy_reg_2tarray; + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + " ===> phy:Rtl819XPHY_REG_Array_PG\n"); + + if (configtype == BASEBAND_CONFIG_PHY_REG) { + for (i = 0; i < phy_reg_arraylen; i = i + 2) { + rtl_addr_delay(phy_regarray_table[i]); + rtl_set_bbreg(hw, phy_regarray_table[i], MASKDWORD, + phy_regarray_table[i + 1]); + udelay(1); + rtl_dbg(rtlpriv, COMP_INIT, DBG_TRACE, + "The phy_regarray_table[0] is %x Rtl819XPHY_REGArray[1] is %x\n", + phy_regarray_table[i], + phy_regarray_table[i + 1]); + } + } else if (configtype == BASEBAND_CONFIG_AGC_TAB) { + for (i = 0; i < agctab_arraylen; i = i + 2) { + rtl_set_bbreg(hw, agctab_array_table[i], + MASKDWORD, agctab_array_table[i + 1]); + + /* Add 1us delay between BB/RF register setting. */ + udelay(1); + + rtl_dbg(rtlpriv, COMP_INIT, DBG_TRACE, + "AGC table %u %u\n", + agctab_array_table[i], + agctab_array_table[i + 1]); + } + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + "Normal Chip, loaded AGC table\n"); + } + return true; +} + +static bool _rtl92du_phy_config_bb_pg(struct ieee80211_hw *hw, u8 configtype) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + const u32 *phy_regarray_table_pg; + u16 phy_regarray_pg_len; + int i; + + phy_regarray_pg_len = PHY_REG_ARRAY_PG_LENGTH; + phy_regarray_table_pg = rtl8192du_phy_reg_array_pg; + + if (configtype == BASEBAND_CONFIG_PHY_REG) { + for (i = 0; i < phy_regarray_pg_len; i = i + 3) { + rtl_addr_delay(phy_regarray_table_pg[i]); + rtl92d_store_pwrindex_diffrate_offset(hw, + phy_regarray_table_pg[i], + phy_regarray_table_pg[i + 1], + phy_regarray_table_pg[i + 2]); + } + } else { + rtl_dbg(rtlpriv, COMP_SEND, DBG_TRACE, + "configtype != BaseBand_Config_PHY_REG\n"); + } + return true; +} + +static bool _rtl92du_phy_bb_config(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_efuse *rtlefuse = rtl_efuse(rtlpriv); + struct rtl_phy *rtlphy = &rtlpriv->phy; + bool ret; + + rtl_dbg(rtlpriv, COMP_INIT, DBG_TRACE, "==>\n"); + ret = _rtl92du_phy_config_bb(hw, BASEBAND_CONFIG_PHY_REG); + if (!ret) { + pr_err("Write BB Reg Fail!!\n"); + return false; + } + + if (!rtlefuse->autoload_failflag) { + rtlphy->pwrgroup_cnt = 0; + ret = _rtl92du_phy_config_bb_pg(hw, BASEBAND_CONFIG_PHY_REG); + } + if (!ret) { + pr_err("BB_PG Reg Fail!!\n"); + return false; + } + + ret = _rtl92du_phy_config_bb(hw, BASEBAND_CONFIG_AGC_TAB); + if (!ret) { + pr_err("AGC Table Fail\n"); + return false; + } + + rtlphy->cck_high_power = (bool)rtl_get_bbreg(hw, + RFPGA0_XA_HSSIPARAMETER2, + 0x200); + + return true; +} + +bool rtl92du_phy_bb_config(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + bool rtstatus; + u32 regvaldw; + u16 regval; + u8 value; + + rtl92d_phy_init_bb_rf_register_definition(hw); + + regval = rtl_read_word(rtlpriv, REG_SYS_FUNC_EN); + rtl_write_word(rtlpriv, REG_SYS_FUNC_EN, + regval | BIT(13) | BIT(0) | BIT(1)); + + rtl_write_byte(rtlpriv, REG_AFE_PLL_CTRL, 0x83); + rtl_write_byte(rtlpriv, REG_AFE_PLL_CTRL + 1, 0xdb); + + /* 0x1f bit7 bit6 represent for mac0/mac1 driver ready */ + value = rtl_read_byte(rtlpriv, REG_RF_CTRL); + rtl_write_byte(rtlpriv, REG_RF_CTRL, value | RF_EN | RF_RSTB | + RF_SDMRSTB); + + value = FEN_BB_GLB_RSTN | FEN_BBRSTB; + if (rtlhal->interface == INTF_PCI) + value |= FEN_PPLL | FEN_PCIEA | FEN_DIO_PCIE; + else if (rtlhal->interface == INTF_USB) + value |= FEN_USBA | FEN_USBD; + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN, value); + + regvaldw = rtl_read_dword(rtlpriv, RFPGA0_XCD_RFPARAMETER); + regvaldw &= ~BIT(31); + rtl_write_dword(rtlpriv, RFPGA0_XCD_RFPARAMETER, regvaldw); + + /* To Fix MAC loopback mode fail. */ + rtl_write_byte(rtlpriv, REG_LDOHCI12_CTRL, 0x0f); + rtl_write_byte(rtlpriv, 0x15, 0xe9); + + rtl_write_byte(rtlpriv, REG_AFE_XTAL_CTRL + 1, 0x80); + if (!(IS_92D_SINGLEPHY(rtlpriv->rtlhal.version)) && + rtlhal->interface == INTF_PCI) { + regvaldw = rtl_read_dword(rtlpriv, REG_LEDCFG0); + rtl_write_dword(rtlpriv, REG_LEDCFG0, regvaldw | BIT(23)); + } + + rtstatus = _rtl92du_phy_bb_config(hw); + + /* Crystal calibration */ + rtl_set_bbreg(hw, REG_AFE_XTAL_CTRL, 0xf0, + rtlpriv->efuse.crystalcap & 0x0f); + rtl_set_bbreg(hw, REG_AFE_PLL_CTRL, 0xf0000000, + (rtlpriv->efuse.crystalcap & 0xf0) >> 4); + + return rtstatus; +} + +bool rtl92du_phy_rf_config(struct ieee80211_hw *hw) +{ + return rtl92du_phy_rf6052_config(hw); +} + +bool rtl92du_phy_config_rf_with_headerfile(struct ieee80211_hw *hw, + enum rf_content content, + enum radio_path rfpath) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u16 radioa_arraylen, radiob_arraylen; + const u32 *radioa_array_table; + const u32 *radiob_array_table; + int i; + + radioa_arraylen = RADIOA_2T_ARRAYLENGTH; + radioa_array_table = rtl8192du_radioa_2tarray; + radiob_arraylen = RADIOB_2T_ARRAYLENGTH; + radiob_array_table = rtl8192du_radiob_2tarray; + if (rtlpriv->efuse.internal_pa_5g[0]) { + radioa_arraylen = RADIOA_2T_INT_PA_ARRAYLENGTH; + radioa_array_table = rtl8192du_radioa_2t_int_paarray; + } + if (rtlpriv->efuse.internal_pa_5g[1]) { + radiob_arraylen = RADIOB_2T_INT_PA_ARRAYLENGTH; + radiob_array_table = rtl8192du_radiob_2t_int_paarray; + } + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + "PHY_ConfigRFWithHeaderFile() Radio_A:Rtl819XRadioA_1TArray\n"); + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + "PHY_ConfigRFWithHeaderFile() Radio_B:Rtl819XRadioB_1TArray\n"); + rtl_dbg(rtlpriv, COMP_INIT, DBG_TRACE, "Radio No %x\n", rfpath); + + /* this only happens when DMDP, mac0 start on 2.4G, + * mac1 start on 5G, mac 0 has to set phy0 & phy1 + * pathA or mac1 has to set phy0 & phy1 pathA + */ + if (content == radiob_txt && rfpath == RF90_PATH_A) { + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, + " ===> althougth Path A, we load radiob.txt\n"); + radioa_arraylen = radiob_arraylen; + radioa_array_table = radiob_array_table; + } + + switch (rfpath) { + case RF90_PATH_A: + for (i = 0; i < radioa_arraylen; i = i + 2) { + rtl_rfreg_delay(hw, rfpath, radioa_array_table[i], + RFREG_OFFSET_MASK, + radioa_array_table[i + 1]); + } + break; + case RF90_PATH_B: + for (i = 0; i < radiob_arraylen; i = i + 2) { + rtl_rfreg_delay(hw, rfpath, radiob_array_table[i], + RFREG_OFFSET_MASK, + radiob_array_table[i + 1]); + } + break; + case RF90_PATH_C: + case RF90_PATH_D: + pr_err("switch case %#x not processed\n", rfpath); + break; + } + + return true; +} + +void rtl92du_phy_set_bw_mode(struct ieee80211_hw *hw, + enum nl80211_channel_type ch_type) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + struct rtl_phy *rtlphy = &rtlpriv->phy; + struct rtl_mac *mac = rtl_mac(rtlpriv); + u8 reg_bw_opmode; + u8 reg_prsr_rsc; + + if (rtlphy->set_bwmode_inprogress) + return; + + if ((is_hal_stop(rtlhal)) || (RT_CANNOT_IO(hw))) { + rtl_dbg(rtlpriv, COMP_ERR, DBG_WARNING, + "FALSE driver sleep or unload\n"); + return; + } + + rtlphy->set_bwmode_inprogress = true; + + rtl_dbg(rtlpriv, COMP_SCAN, DBG_TRACE, "Switch to %s bandwidth\n", + rtlphy->current_chan_bw == HT_CHANNEL_WIDTH_20 ? + "20MHz" : "40MHz"); + + reg_bw_opmode = rtl_read_byte(rtlpriv, REG_BWOPMODE); + reg_prsr_rsc = rtl_read_byte(rtlpriv, REG_RRSR + 2); + + switch (rtlphy->current_chan_bw) { + case HT_CHANNEL_WIDTH_20: + reg_bw_opmode |= BW_OPMODE_20MHZ; + rtl_write_byte(rtlpriv, REG_BWOPMODE, reg_bw_opmode); + break; + case HT_CHANNEL_WIDTH_20_40: + reg_bw_opmode &= ~BW_OPMODE_20MHZ; + rtl_write_byte(rtlpriv, REG_BWOPMODE, reg_bw_opmode); + + reg_prsr_rsc = (reg_prsr_rsc & 0x90) | + (mac->cur_40_prime_sc << 5); + rtl_write_byte(rtlpriv, REG_RRSR + 2, reg_prsr_rsc); + break; + default: + pr_err("unknown bandwidth: %#X\n", + rtlphy->current_chan_bw); + break; + } + + switch (rtlphy->current_chan_bw) { + case HT_CHANNEL_WIDTH_20: + rtl92du_phy_set_bb_reg_1byte(hw, RFPGA0_RFMOD, BRFMOD, 0x0); + rtl_set_bbreg(hw, RFPGA1_RFMOD, BRFMOD, 0x0); + /* SET BIT10 BIT11 for receive cck */ + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER2, BIT(10) | BIT(11), 3); + break; + case HT_CHANNEL_WIDTH_20_40: + rtl92du_phy_set_bb_reg_1byte(hw, RFPGA0_RFMOD, BRFMOD, 0x1); + rtl_set_bbreg(hw, RFPGA1_RFMOD, BRFMOD, 0x1); + /* Set Control channel to upper or lower. + * These settings are required only for 40MHz + */ + if (rtlhal->current_bandtype == BAND_ON_2_4G) + rtl_set_bbreg(hw, RCCK0_SYSTEM, BCCKSIDEBAND, + mac->cur_40_prime_sc >> 1); + rtl_set_bbreg(hw, ROFDM1_LSTF, 0xC00, mac->cur_40_prime_sc); + /* SET BIT10 BIT11 for receive cck */ + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER2, + BIT(10) | BIT(11), 0); + rtl_set_bbreg(hw, 0x818, BIT(26) | BIT(27), + mac->cur_40_prime_sc == + HAL_PRIME_CHNL_OFFSET_LOWER ? 2 : 1); + break; + default: + pr_err("unknown bandwidth: %#X\n", + rtlphy->current_chan_bw); + break; + } + + rtl92d_phy_rf6052_set_bandwidth(hw, rtlphy->current_chan_bw); + + rtlphy->set_bwmode_inprogress = false; + rtl_dbg(rtlpriv, COMP_SCAN, DBG_TRACE, "<==\n"); +} + +static void _rtl92du_phy_stop_trx_before_changeband(struct ieee80211_hw *hw) +{ + rtl92du_phy_set_bb_reg_1byte(hw, RFPGA0_RFMOD, BCCKEN | BOFDMEN, 0); + rtl_set_bbreg(hw, ROFDM0_TRXPATHENABLE, MASKBYTE0, 0x00); + rtl_set_bbreg(hw, ROFDM1_TRXPATHENABLE, BDWORD, 0x0); +} + +static void rtl92du_phy_switch_wirelessband(struct ieee80211_hw *hw, u8 band) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + u16 basic_rates; + u32 reg_mac; + u8 value8; + + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, "==>\n"); + rtlhal->bandset = band; + rtlhal->current_bandtype = band; + if (IS_92D_SINGLEPHY(rtlhal->version)) + rtlhal->bandset = BAND_ON_BOTH; + + /* stop RX/Tx */ + _rtl92du_phy_stop_trx_before_changeband(hw); + + /* reconfig BB/RF according to wireless mode */ + if (rtlhal->current_bandtype == BAND_ON_2_4G) + /* BB & RF Config */ + rtl_dbg(rtlpriv, COMP_CMD, DBG_DMESG, "====>2.4G\n"); + else + /* 5G band */ + rtl_dbg(rtlpriv, COMP_CMD, DBG_DMESG, "====>5G\n"); + + if (rtlhal->interfaceindex == 1) + _rtl92du_phy_config_bb(hw, BASEBAND_CONFIG_AGC_TAB); + + rtl92du_update_bbrf_configuration(hw); + + basic_rates = RRSR_6M | RRSR_12M | RRSR_24M; + if (rtlhal->current_bandtype == BAND_ON_2_4G) + basic_rates |= RRSR_1M | RRSR_2M | RRSR_5_5M | RRSR_11M; + rtlpriv->cfg->ops->set_hw_reg(hw, HW_VAR_BASIC_RATE, + (u8 *)&basic_rates); + + rtl92du_phy_set_bb_reg_1byte(hw, RFPGA0_RFMOD, BCCKEN | BOFDMEN, 0x3); + + /* 20M BW. */ + /* rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER2, BIT(10), 1); */ + rtlhal->reloadtxpowerindex = true; + + reg_mac = rtlhal->interfaceindex == 0 ? REG_MAC0 : REG_MAC1; + + /* notice fw know band status 0x81[1]/0x53[1] = 0: 5G, 1: 2G */ + if (rtlhal->current_bandtype == BAND_ON_2_4G) { + value8 = rtl_read_byte(rtlpriv, reg_mac); + value8 |= BIT(1); + rtl_write_byte(rtlpriv, reg_mac, value8); + } else { + value8 = rtl_read_byte(rtlpriv, reg_mac); + value8 &= ~BIT(1); + rtl_write_byte(rtlpriv, reg_mac, value8); + } + mdelay(1); + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, "<==Switch Band OK\n"); +} + +static void _rtl92du_phy_reload_imr_setting(struct ieee80211_hw *hw, + u8 channel, u8 rfpath) +{ + struct rtl_usb *rtlusb = rtl_usbdev(rtl_usbpriv(hw)); + struct rtl_priv *rtlpriv = rtl_priv(hw); + u8 group, i; + + if (rtlusb->udev->speed != USB_SPEED_HIGH) + return; + + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "====>path %d\n", rfpath); + if (rtlpriv->rtlhal.current_bandtype == BAND_ON_5G) { + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "====>5G\n"); + rtl92du_phy_set_bb_reg_1byte(hw, RFPGA0_RFMOD, + BOFDMEN | BCCKEN, 0); + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER4, 0x00f00000, 0xf); + + /* fc area 0xd2c */ + if (channel >= 149) + rtl_set_bbreg(hw, ROFDM1_CFOTRACKING, BIT(13) | + BIT(14), 2); + else + rtl_set_bbreg(hw, ROFDM1_CFOTRACKING, BIT(13) | + BIT(14), 1); + + /* leave 0 for channel1-14. */ + group = channel <= 64 ? 1 : 2; + for (i = 0; i < MAX_RF_IMR_INDEX_NORMAL; i++) + rtl_set_rfreg(hw, (enum radio_path)rfpath, + rf_reg_for_5g_swchnl_normal[i], + RFREG_OFFSET_MASK, + rf_imr_param_normal[group][i]); + + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER4, 0x00f00000, 0); + rtl92du_phy_set_bb_reg_1byte(hw, RFPGA0_RFMOD, + BOFDMEN | BCCKEN, 3); + } else { + /* G band. */ + rtl_dbg(rtlpriv, COMP_SCAN, DBG_LOUD, + "Load RF IMR parameters for G band. IMR already setting %d\n", + rtlpriv->rtlhal.load_imrandiqk_setting_for2g); + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "====>2.4G\n"); + + if (!rtlpriv->rtlhal.load_imrandiqk_setting_for2g) { + rtl_dbg(rtlpriv, COMP_SCAN, DBG_LOUD, + "Load RF IMR parameters for G band. %d\n", + rfpath); + rtl92du_phy_set_bb_reg_1byte(hw, RFPGA0_RFMOD, + BOFDMEN | BCCKEN, 0); + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER4, + 0x00f00000, 0xf); + + for (i = 0; i < MAX_RF_IMR_INDEX_NORMAL; i++) { + rtl_set_rfreg(hw, (enum radio_path)rfpath, + rf_reg_for_5g_swchnl_normal[i], + RFREG_OFFSET_MASK, + rf_imr_param_normal[0][i]); + } + + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER4, + 0x00f00000, 0); + rtl92du_phy_set_bb_reg_1byte(hw, RFPGA0_RFMOD, + BOFDMEN | BCCKEN, 3); + } + } + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "<====\n"); +} + +static void _rtl92du_phy_switch_rf_setting(struct ieee80211_hw *hw, u8 channel) +{ + struct rtl_usb *rtlusb = rtl_usbdev(rtl_usbpriv(hw)); + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + struct rtl_phy *rtlphy = &rtlpriv->phy; + u8 path = rtlhal->current_bandtype == BAND_ON_5G ? RF90_PATH_A + : RF90_PATH_B; + u32 u4regvalue, mask = 0x1C000, value = 0, u4tmp, u4tmp2; + bool need_pwr_down = false, internal_pa = false; + u32 regb30 = rtl_get_bbreg(hw, 0xb30, BIT(27)); + u8 index = 0, i, rfpath; + + if (rtlusb->udev->speed != USB_SPEED_HIGH) + return; + + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "====>\n"); + /* config path A for 5G */ + if (rtlhal->current_bandtype == BAND_ON_5G) { + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "====>5G\n"); + u4tmp = rtlpriv->curveindex_5g[channel - 1]; + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "ver 1 set RF-A, 5G, 0x28 = 0x%x !!\n", u4tmp); + + for (i = 0; i < RF_CHNL_NUM_5G; i++) { + if (channel == rf_chnl_5g[i] && channel <= 140) + index = 0; + } + for (i = 0; i < RF_CHNL_NUM_5G_40M; i++) { + if (channel == rf_chnl_5g_40m[i] && channel <= 140) + index = 1; + } + if (channel == 149 || channel == 155 || channel == 161) + index = 2; + else if (channel == 151 || channel == 153 || channel == 163 || + channel == 165) + index = 3; + else if (channel == 157 || channel == 159) + index = 4; + + if (rtlhal->macphymode == DUALMAC_DUALPHY && + rtlhal->interfaceindex == 1) { + need_pwr_down = rtl92du_phy_enable_anotherphy(hw, false); + rtlhal->during_mac1init_radioa = true; + /* asume no this case */ + if (need_pwr_down) + rtl92d_phy_enable_rf_env(hw, path, + &u4regvalue); + } + + /* DMDP, if band = 5G, Mac0 need to set PHY1 when regB30[27]=1 */ + if (regb30 && rtlhal->interfaceindex == 0) { + need_pwr_down = rtl92du_phy_enable_anotherphy(hw, true); + rtlhal->during_mac0init_radiob = true; + if (need_pwr_down) + rtl92d_phy_enable_rf_env(hw, path, + &u4regvalue); + } + + for (i = 0; i < RF_REG_NUM_FOR_C_CUT_5G; i++) { + if (i == 0 && rtlhal->macphymode == DUALMAC_DUALPHY) { + rtl_set_rfreg(hw, (enum radio_path)path, + rf_reg_for_c_cut_5g[i], + RFREG_OFFSET_MASK, 0xE439D); + } else if (rf_reg_for_c_cut_5g[i] == RF_SYN_G4) { + u4tmp2 = (rf_reg_pram_c_5g[index][i] & + 0x7FF) | (u4tmp << 11); + if (channel == 36) + u4tmp2 &= ~(BIT(7) | BIT(6)); + rtl_set_rfreg(hw, (enum radio_path)path, + rf_reg_for_c_cut_5g[i], + RFREG_OFFSET_MASK, u4tmp2); + } else { + rtl_set_rfreg(hw, (enum radio_path)path, + rf_reg_for_c_cut_5g[i], + RFREG_OFFSET_MASK, + rf_reg_pram_c_5g[index][i]); + } + rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, + "offset 0x%x value 0x%x path %d index %d readback 0x%x\n", + rf_reg_for_c_cut_5g[i], + rf_reg_pram_c_5g[index][i], + path, index, + rtl_get_rfreg(hw, (enum radio_path)path, + rf_reg_for_c_cut_5g[i], + RFREG_OFFSET_MASK)); + } + if (rtlhal->macphymode == DUALMAC_DUALPHY && + rtlhal->interfaceindex == 1) { + if (need_pwr_down) + rtl92d_phy_restore_rf_env(hw, path, &u4regvalue); + + rtl92du_phy_powerdown_anotherphy(hw, false); + } + + if (regb30 && rtlhal->interfaceindex == 0) { + if (need_pwr_down) + rtl92d_phy_restore_rf_env(hw, path, &u4regvalue); + + rtl92du_phy_powerdown_anotherphy(hw, true); + } + + if (channel < 149) + value = 0x07; + else if (channel >= 149) + value = 0x02; + if (channel >= 36 && channel <= 64) + index = 0; + else if (channel >= 100 && channel <= 140) + index = 1; + else + index = 2; + + for (rfpath = RF90_PATH_A; rfpath < rtlphy->num_total_rfpath; + rfpath++) { + if (rtlhal->macphymode == DUALMAC_DUALPHY && + rtlhal->interfaceindex == 1) /* MAC 1 5G */ + internal_pa = rtlpriv->efuse.internal_pa_5g[1]; + else + internal_pa = + rtlpriv->efuse.internal_pa_5g[rfpath]; + + if (internal_pa) { + for (i = 0; + i < RF_REG_NUM_FOR_C_CUT_5G_INTERNALPA; + i++) { + if (rf_for_c_cut_5g_internal_pa[i] == 0x03 && + channel >= 36 && channel <= 64) + rtl_set_rfreg(hw, rfpath, + rf_for_c_cut_5g_internal_pa[i], + RFREG_OFFSET_MASK, + 0x7bdef); + else + rtl_set_rfreg(hw, rfpath, + rf_for_c_cut_5g_internal_pa[i], + RFREG_OFFSET_MASK, + rf_pram_c_5g_int_pa[index][i]); + rtl_dbg(rtlpriv, COMP_RF, DBG_LOUD, + "offset 0x%x value 0x%x path %d index %d\n", + rf_for_c_cut_5g_internal_pa[i], + rf_pram_c_5g_int_pa[index][i], + rfpath, index); + } + } else { + rtl_set_rfreg(hw, (enum radio_path)rfpath, RF_TXPA_AG, + mask, value); + } + } + } else if (rtlhal->current_bandtype == BAND_ON_2_4G) { + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "====>2.4G\n"); + u4tmp = rtlpriv->curveindex_2g[channel - 1]; + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "ver 3 set RF-B, 2G, 0x28 = 0x%x !!\n", u4tmp); + + if (channel == 1 || channel == 2 || channel == 4 || + channel == 9 || channel == 10 || channel == 11 || + channel == 12) + index = 0; + else if (channel == 3 || channel == 13 || channel == 14) + index = 1; + else if (channel >= 5 && channel <= 8) + index = 2; + + if (rtlhal->macphymode == DUALMAC_DUALPHY) { + path = RF90_PATH_A; + if (rtlhal->interfaceindex == 0) { + need_pwr_down = + rtl92du_phy_enable_anotherphy(hw, true); + rtlhal->during_mac0init_radiob = true; + + if (need_pwr_down) + rtl92d_phy_enable_rf_env(hw, path, + &u4regvalue); + } + + /* DMDP, if band = 2G, MAC1 need to set PHY0 when regB30[27]=1 */ + if (regb30 && rtlhal->interfaceindex == 1) { + need_pwr_down = + rtl92du_phy_enable_anotherphy(hw, false); + rtlhal->during_mac1init_radioa = true; + + if (need_pwr_down) + rtl92d_phy_enable_rf_env(hw, path, + &u4regvalue); + } + } + + for (i = 0; i < RF_REG_NUM_FOR_C_CUT_2G; i++) { + if (rf_reg_for_c_cut_2g[i] == RF_SYN_G7) + rtl_set_rfreg(hw, (enum radio_path)path, + rf_reg_for_c_cut_2g[i], + RFREG_OFFSET_MASK, + rf_reg_param_for_c_cut_2g[index][i] | + BIT(17)); + else + rtl_set_rfreg(hw, (enum radio_path)path, + rf_reg_for_c_cut_2g[i], + RFREG_OFFSET_MASK, + rf_reg_param_for_c_cut_2g + [index][i]); + + rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, + "offset 0x%x value 0x%x mak 0x%x path %d index %d readback 0x%x\n", + rf_reg_for_c_cut_2g[i], + rf_reg_param_for_c_cut_2g[index][i], + rf_reg_mask_for_c_cut_2g[i], path, index, + rtl_get_rfreg(hw, (enum radio_path)path, + rf_reg_for_c_cut_2g[i], + RFREG_OFFSET_MASK)); + } + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "cosa ver 3 set RF-B, 2G, 0x28 = 0x%x !!\n", + rf_syn_g4_for_c_cut_2g | (u4tmp << 11)); + + rtl_set_rfreg(hw, (enum radio_path)path, RF_SYN_G4, + RFREG_OFFSET_MASK, + rf_syn_g4_for_c_cut_2g | (u4tmp << 11)); + + if (rtlhal->macphymode == DUALMAC_DUALPHY && + rtlhal->interfaceindex == 0) { + if (need_pwr_down) + rtl92d_phy_restore_rf_env(hw, path, &u4regvalue); + + rtl92du_phy_powerdown_anotherphy(hw, true); + } + + if (regb30 && rtlhal->interfaceindex == 1) { + if (need_pwr_down) + rtl92d_phy_restore_rf_env(hw, path, &u4regvalue); + + rtl92du_phy_powerdown_anotherphy(hw, false); + } + } + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "<====\n"); +} + +/* bit0 = 1 => Tx OK, bit1 = 1 => Rx OK */ +static u8 _rtl92du_phy_patha_iqk(struct ieee80211_hw *hw, bool configpathb) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtl_priv(hw)); + u32 regeac, rege94, rege9c, regea4; + u8 result = 0; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path-A IQK setting!\n"); + + if (rtlhal->interfaceindex == 0) { + rtl_set_bbreg(hw, RTX_IQK_TONE_A, MASKDWORD, 0x10008c1f); + rtl_set_bbreg(hw, RRX_IQK_TONE_A, MASKDWORD, 0x10008c1f); + } else { + rtl_set_bbreg(hw, RTX_IQK_TONE_A, MASKDWORD, 0x10008c22); + rtl_set_bbreg(hw, RRX_IQK_TONE_A, MASKDWORD, 0x10008c22); + } + rtl_set_bbreg(hw, RTX_IQK_PI_A, MASKDWORD, 0x82140102); + rtl_set_bbreg(hw, RRX_IQK_PI_A, MASKDWORD, + configpathb ? 0x28160202 : 0x28160502); + /* path-B IQK setting */ + if (configpathb) { + rtl_set_bbreg(hw, RTX_IQK_TONE_B, MASKDWORD, 0x10008c22); + rtl_set_bbreg(hw, RRX_IQK_TONE_B, MASKDWORD, 0x10008c22); + rtl_set_bbreg(hw, RTX_IQK_PI_B, MASKDWORD, 0x82140102); + rtl_set_bbreg(hw, RRX_IQK_PI_B, MASKDWORD, 0x28160206); + } + + /* LO calibration setting */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, "LO calibration setting!\n"); + rtl_set_bbreg(hw, RIQK_AGC_RSP, MASKDWORD, 0x00462911); + + /* One shot, path A LOK & IQK */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, "One shot, path A LOK & IQK!\n"); + rtl_set_bbreg(hw, RIQK_AGC_PTS, MASKDWORD, 0xf9000000); + rtl_set_bbreg(hw, RIQK_AGC_PTS, MASKDWORD, 0xf8000000); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Delay %d ms for One shot, path A LOK & IQK\n", + IQK_DELAY_TIME); + mdelay(IQK_DELAY_TIME); + + /* Check failed */ + regeac = rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_A_2, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xeac = 0x%x\n", regeac); + rege94 = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_A, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xe94 = 0x%x\n", rege94); + rege9c = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_A, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xe9c = 0x%x\n", rege9c); + regea4 = rtl_get_bbreg(hw, RRX_POWER_BEFORE_IQK_A_2, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xea4 = 0x%x\n", regea4); + + if (!(regeac & BIT(28)) && + (((rege94 & 0x03FF0000) >> 16) != 0x142) && + (((rege9c & 0x03FF0000) >> 16) != 0x42)) + result |= 0x01; + else /* if Tx not OK, ignore Rx */ + return result; + + /* if Tx is OK, check whether Rx is OK */ + if (!(regeac & BIT(27)) && + (((regea4 & 0x03FF0000) >> 16) != 0x132) && + (((regeac & 0x03FF0000) >> 16) != 0x36)) + result |= 0x02; + else + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path A Rx IQK fail!!\n"); + + return result; +} + +/* bit0 = 1 => Tx OK, bit1 = 1 => Rx OK */ +static u8 _rtl92du_phy_patha_iqk_5g_normal(struct ieee80211_hw *hw, + bool configpathb) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + struct rtl_phy *rtlphy = &rtlpriv->phy; + u32 TXOKBIT = BIT(28), RXOKBIT = BIT(27); + u32 regeac, rege94, rege9c, regea4; + u8 timeout = 20, timecount = 0; + u8 retrycount = 2; + u8 result = 0; + u8 i; + + if (rtlhal->interfaceindex == 1) { /* PHY1 */ + TXOKBIT = BIT(31); + RXOKBIT = BIT(30); + } + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path-A IQK setting!\n"); + rtl_set_bbreg(hw, RTX_IQK_TONE_A, MASKDWORD, 0x18008c1f); + rtl_set_bbreg(hw, RRX_IQK_TONE_A, MASKDWORD, 0x18008c1f); + rtl_set_bbreg(hw, RTX_IQK_PI_A, MASKDWORD, 0x82140307); + rtl_set_bbreg(hw, RRX_IQK_PI_A, MASKDWORD, 0x68160960); + /* path-B IQK setting */ + if (configpathb) { + rtl_set_bbreg(hw, RTX_IQK_TONE_B, MASKDWORD, 0x18008c2f); + rtl_set_bbreg(hw, RRX_IQK_TONE_B, MASKDWORD, 0x18008c2f); + rtl_set_bbreg(hw, RTX_IQK_PI_B, MASKDWORD, 0x82110000); + rtl_set_bbreg(hw, RRX_IQK_PI_B, MASKDWORD, 0x68110000); + } + + /* LO calibration setting */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, "LO calibration setting!\n"); + rtl_set_bbreg(hw, RIQK_AGC_RSP, MASKDWORD, 0x00462911); + + /* path-A PA on */ + rtl_set_bbreg(hw, RFPGA0_XAB_RFINTERFACESW, MASKDWORD, 0x07000f60); + rtl_set_bbreg(hw, RFPGA0_XA_RFINTERFACEOE, MASKDWORD, 0x66e60e30); + + for (i = 0; i < retrycount; i++) { + /* One shot, path A LOK & IQK */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "One shot, path A LOK & IQK!\n"); + rtl_set_bbreg(hw, RIQK_AGC_PTS, MASKDWORD, 0xf9000000); + rtl_set_bbreg(hw, RIQK_AGC_PTS, MASKDWORD, 0xf8000000); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Delay %d ms for One shot, path A LOK & IQK.\n", + IQK_DELAY_TIME); + mdelay(IQK_DELAY_TIME * 10); + + while (timecount < timeout && + rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_A_2, BIT(26)) == 0) { + udelay(IQK_DELAY_TIME * 1000 * 2); + timecount++; + } + + timecount = 0; + while (timecount < timeout && + rtl_get_bbreg(hw, RRX_POWER_BEFORE_IQK_A_2, MASK_IQK_RESULT) == 0) { + udelay(IQK_DELAY_TIME * 1000 * 2); + timecount++; + } + + /* Check failed */ + regeac = rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_A_2, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xeac = 0x%x\n", regeac); + rege94 = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_A, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xe94 = 0x%x\n", rege94); + rege9c = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_A, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xe9c = 0x%x\n", rege9c); + regea4 = rtl_get_bbreg(hw, RRX_POWER_BEFORE_IQK_A_2, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xea4 = 0x%x\n", regea4); + + if (!(regeac & TXOKBIT) && + (((rege94 & 0x03FF0000) >> 16) != 0x142)) { + result |= 0x01; + } else { /* if Tx not OK, ignore Rx */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path A Tx IQK fail!!\n"); + continue; + } + + /* if Tx is OK, check whether Rx is OK */ + if (!(regeac & RXOKBIT) && + (((regea4 & 0x03FF0000) >> 16) != 0x132)) { + result |= 0x02; + break; + } + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path A Rx IQK fail!!\n"); + } + + /* path A PA off */ + rtl_set_bbreg(hw, RFPGA0_XAB_RFINTERFACESW, MASKDWORD, + rtlphy->iqk_bb_backup[0]); + rtl_set_bbreg(hw, RFPGA0_XA_RFINTERFACEOE, MASKDWORD, + rtlphy->iqk_bb_backup[1]); + + if (!(result & 0x01)) /* Tx IQK fail */ + rtl_set_bbreg(hw, RTX_IQK_TONE_A, MASKDWORD, 0x19008c00); + + if (!(result & 0x02)) { /* Rx IQK fail */ + rtl_set_bbreg(hw, ROFDM0_XARXIQIMBALANCE, MASKDWORD, 0x40000100); + rtl_set_bbreg(hw, RRX_IQK_TONE_A, MASKDWORD, 0x19008c00); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path A Rx IQK fail!! 0xe34 = %#x\n", + rtl_get_bbreg(hw, RRX_IQK_TONE_A, MASKDWORD)); + } + + return result; +} + +/* bit0 = 1 => Tx OK, bit1 = 1 => Rx OK */ +static u8 _rtl92du_phy_pathb_iqk(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u32 regeac, regeb4, regebc, regec4, regecc; + u8 result = 0; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "One shot, path B LOK & IQK!\n"); + rtl_set_bbreg(hw, RIQK_AGC_CONT, MASKDWORD, 0x00000002); + rtl_set_bbreg(hw, RIQK_AGC_CONT, MASKDWORD, 0x00000000); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Delay %d ms for One shot, path B LOK & IQK\n", IQK_DELAY_TIME); + mdelay(IQK_DELAY_TIME); + + /* Check failed */ + regeac = rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_A_2, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xeac = 0x%x\n", regeac); + regeb4 = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_B, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xeb4 = 0x%x\n", regeb4); + regebc = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_B, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xebc = 0x%x\n", regebc); + regec4 = rtl_get_bbreg(hw, RRX_POWER_BEFORE_IQK_B_2, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xec4 = 0x%x\n", regec4); + regecc = rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_B_2, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xecc = 0x%x\n", regecc); + + if (!(regeac & BIT(31)) && + (((regeb4 & 0x03FF0000) >> 16) != 0x142) && + (((regebc & 0x03FF0000) >> 16) != 0x42)) + result |= 0x01; + else + return result; + + if (!(regeac & BIT(30)) && + (((regec4 & 0x03FF0000) >> 16) != 0x132) && + (((regecc & 0x03FF0000) >> 16) != 0x36)) + result |= 0x02; + else + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path B Rx IQK fail!!\n"); + + return result; +} + +/* bit0 = 1 => Tx OK, bit1 = 1 => Rx OK */ +static u8 _rtl92du_phy_pathb_iqk_5g_normal(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_phy *rtlphy = &rtlpriv->phy; + u32 regeac, regeb4, regebc, regec4, regecc; + u8 timeout = 20, timecount = 0; + u8 retrycount = 2; + u8 result = 0; + u8 i; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path-B IQK setting!\n"); + rtl_set_bbreg(hw, RTX_IQK_TONE_A, MASKDWORD, 0x18008c1f); + rtl_set_bbreg(hw, RRX_IQK_TONE_A, MASKDWORD, 0x18008c1f); + rtl_set_bbreg(hw, RTX_IQK_PI_A, MASKDWORD, 0x82110000); + rtl_set_bbreg(hw, RRX_IQK_PI_A, MASKDWORD, 0x68110000); + + /* path-B IQK setting */ + rtl_set_bbreg(hw, RTX_IQK_TONE_B, MASKDWORD, 0x18008c2f); + rtl_set_bbreg(hw, RRX_IQK_TONE_B, MASKDWORD, 0x18008c2f); + rtl_set_bbreg(hw, RTX_IQK_PI_B, MASKDWORD, 0x82140307); + rtl_set_bbreg(hw, RRX_IQK_PI_B, MASKDWORD, 0x68160960); + + /* LO calibration setting */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, "LO calibration setting!\n"); + rtl_set_bbreg(hw, RIQK_AGC_RSP, MASKDWORD, 0x00462911); + + /* path-B PA on */ + rtl_set_bbreg(hw, RFPGA0_XAB_RFINTERFACESW, MASKDWORD, 0x0f600700); + rtl_set_bbreg(hw, RFPGA0_XB_RFINTERFACEOE, MASKDWORD, 0x061f0d30); + + for (i = 0; i < retrycount; i++) { + /* One shot, path B LOK & IQK */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "One shot, path A LOK & IQK!\n"); + rtl_set_bbreg(hw, RIQK_AGC_PTS, MASKDWORD, 0xfa000000); + rtl_set_bbreg(hw, RIQK_AGC_PTS, MASKDWORD, 0xf8000000); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Delay %d ms for One shot, path B LOK & IQK.\n", 10); + mdelay(IQK_DELAY_TIME * 10); + + while (timecount < timeout && + rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_A_2, BIT(29)) == 0) { + udelay(IQK_DELAY_TIME * 1000 * 2); + timecount++; + } + + timecount = 0; + while (timecount < timeout && + rtl_get_bbreg(hw, RRX_POWER_BEFORE_IQK_B_2, MASK_IQK_RESULT) == 0) { + udelay(IQK_DELAY_TIME * 1000 * 2); + timecount++; + } + + /* Check failed */ + regeac = rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_A_2, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xeac = 0x%x\n", regeac); + regeb4 = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_B, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xeb4 = 0x%x\n", regeb4); + regebc = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_B, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xebc = 0x%x\n", regebc); + regec4 = rtl_get_bbreg(hw, RRX_POWER_BEFORE_IQK_B_2, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xec4 = 0x%x\n", regec4); + regecc = rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_B_2, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xecc = 0x%x\n", regecc); + + if (!(regeac & BIT(31)) && + (((regeb4 & 0x03FF0000) >> 16) != 0x142)) + result |= 0x01; + else + continue; + + if (!(regeac & BIT(30)) && + (((regec4 & 0x03FF0000) >> 16) != 0x132)) { + result |= 0x02; + break; + } + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path B Rx IQK fail!!\n"); + } + + /* path B PA off */ + rtl_set_bbreg(hw, RFPGA0_XAB_RFINTERFACESW, MASKDWORD, + rtlphy->iqk_bb_backup[0]); + rtl_set_bbreg(hw, RFPGA0_XB_RFINTERFACEOE, MASKDWORD, + rtlphy->iqk_bb_backup[2]); + + if (!(result & 0x01)) + rtl_set_bbreg(hw, RTX_IQK_TONE_B, MASKDWORD, 0x19008c00); + + if (!(result & 0x02)) { + rtl_set_bbreg(hw, ROFDM0_XBRXIQIMBALANCE, MASKDWORD, 0x40000100); + rtl_set_bbreg(hw, RRX_IQK_TONE_B, MASKDWORD, 0x19008c00); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path B Rx IQK fail!! 0xe54 = %#x\n", + rtl_get_bbreg(hw, RRX_IQK_TONE_B, MASKDWORD)); + } + + return result; +} + +static void _rtl92du_phy_reload_adda_registers(struct ieee80211_hw *hw, + const u32 *adda_reg, + u32 *adda_backup, u32 regnum) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u32 i; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Reload ADDA power saving parameters !\n"); + for (i = 0; i < regnum; i++) { + /* path-A/B BB to initial gain */ + if (adda_reg[i] == ROFDM0_XAAGCCORE1 || + adda_reg[i] == ROFDM0_XBAGCCORE1) + rtl_set_bbreg(hw, adda_reg[i], MASKDWORD, 0x50); + + rtl_set_bbreg(hw, adda_reg[i], MASKDWORD, adda_backup[i]); + } +} + +static void _rtl92du_phy_reload_mac_registers(struct ieee80211_hw *hw, + const u32 *macreg, u32 *macbackup) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u32 i; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Reload MAC parameters !\n"); + for (i = 0; i < (IQK_MAC_REG_NUM - 1); i++) + rtl_write_byte(rtlpriv, macreg[i], (u8)macbackup[i]); + rtl_write_dword(rtlpriv, macreg[i], macbackup[i]); +} + +static void _rtl92du_phy_patha_standby(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path-A standby mode!\n"); + + rtl_set_bbreg(hw, RFPGA0_IQK, MASKH3BYTES, 0x0); + rtl_set_bbreg(hw, RFPGA0_XA_LSSIPARAMETER, MASKDWORD, 0x00010000); + rtl_set_bbreg(hw, RFPGA0_IQK, MASKH3BYTES, 0x808000); +} + +static void _rtl92du_phy_pimode_switch(struct ieee80211_hw *hw, bool pi_mode) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u32 mode; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "BB Switch to %s mode!\n", pi_mode ? "PI" : "SI"); + mode = pi_mode ? 0x01000100 : 0x01000000; + rtl_set_bbreg(hw, RFPGA0_XA_HSSIPARAMETER1, MASKDWORD, mode); + rtl_set_bbreg(hw, RFPGA0_XB_HSSIPARAMETER1, MASKDWORD, mode); +} + +static void _rtl92du_phy_iq_calibrate(struct ieee80211_hw *hw, long result[][8], + u8 t, bool is2t) +{ + static const u32 adda_reg[IQK_ADDA_REG_NUM] = { + RFPGA0_XCD_SWITCHCONTROL, RBLUE_TOOTH, RRX_WAIT_CCA, + RTX_CCK_RFON, RTX_CCK_BBON, RTX_OFDM_RFON, RTX_OFDM_BBON, + RTX_TO_RX, RTX_TO_TX, RRX_CCK, RRX_OFDM, RRX_WAIT_RIFS, + RRX_TO_RX, RSTANDBY, RSLEEP, RPMPD_ANAEN + }; + static const u32 iqk_mac_reg[IQK_MAC_REG_NUM] = { + REG_TXPAUSE, REG_BCN_CTRL, REG_BCN_CTRL_1, REG_GPIO_MUXCFG + }; + static const u32 iqk_bb_reg[IQK_BB_REG_NUM] = { + RFPGA0_XAB_RFINTERFACESW, RFPGA0_XA_RFINTERFACEOE, + RFPGA0_XB_RFINTERFACEOE, ROFDM0_TRMUXPAR, + RFPGA0_XCD_RFINTERFACESW, ROFDM0_TRXPATHENABLE, + RFPGA0_RFMOD, RFPGA0_ANALOGPARAMETER4, + ROFDM0_XAAGCCORE1, ROFDM0_XBAGCCORE1 + }; + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_phy *rtlphy = &rtlpriv->phy; + const u32 retrycount = 2; + u8 patha_ok, pathb_ok; + u32 bbvalue; + u32 i; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "IQK for 2.4G :Start!!!\n"); + if (t == 0) { + bbvalue = rtl_get_bbreg(hw, RFPGA0_RFMOD, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "==>0x%08x\n", bbvalue); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "IQ Calibration for %s\n", + is2t ? "2T2R" : "1T1R"); + + /* Save ADDA parameters, turn Path A ADDA on */ + rtl92d_phy_save_adda_registers(hw, adda_reg, + rtlphy->adda_backup, + IQK_ADDA_REG_NUM); + rtl92d_phy_save_mac_registers(hw, iqk_mac_reg, + rtlphy->iqk_mac_backup); + rtl92d_phy_save_adda_registers(hw, iqk_bb_reg, + rtlphy->iqk_bb_backup, + IQK_BB_REG_NUM); + } + rtl92d_phy_path_adda_on(hw, adda_reg, true, is2t); + + rtl_set_bbreg(hw, RPDP_ANTA, MASKDWORD, 0x01017038); + + if (t == 0) + rtlphy->rfpi_enable = (u8)rtl_get_bbreg(hw, + RFPGA0_XA_HSSIPARAMETER1, BIT(8)); + + /* Switch BB to PI mode to do IQ Calibration. */ + if (!rtlphy->rfpi_enable) + _rtl92du_phy_pimode_switch(hw, true); + + rtl92du_phy_set_bb_reg_1byte(hw, RFPGA0_RFMOD, BCCKEN, 0x00); + rtl_set_bbreg(hw, ROFDM0_TRXPATHENABLE, MASKDWORD, 0x03a05600); + rtl_set_bbreg(hw, ROFDM0_TRMUXPAR, MASKDWORD, 0x000800e4); + rtl_set_bbreg(hw, RFPGA0_XCD_RFINTERFACESW, MASKDWORD, 0x22204000); + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER4, 0xf00000, 0x0f); + if (is2t) { + rtl_set_bbreg(hw, RFPGA0_XA_LSSIPARAMETER, MASKDWORD, + 0x00010000); + rtl_set_bbreg(hw, RFPGA0_XB_LSSIPARAMETER, MASKDWORD, + 0x00010000); + } + + /* MAC settings */ + rtl92d_phy_mac_setting_calibration(hw, iqk_mac_reg, + rtlphy->iqk_mac_backup); + + /* Page B init */ + rtl_set_bbreg(hw, RCONFIG_ANTA, MASKDWORD, 0x0f600000); + if (is2t) + rtl_set_bbreg(hw, RCONFIG_ANTB, MASKDWORD, 0x0f600000); + + /* IQ calibration setting */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, "IQK setting!\n"); + rtl_set_bbreg(hw, RFPGA0_IQK, MASKH3BYTES, 0x808000); + rtl_set_bbreg(hw, RTX_IQK, MASKDWORD, 0x01007c00); + rtl_set_bbreg(hw, RRX_IQK, MASKDWORD, 0x01004800); + + for (i = 0; i < retrycount; i++) { + patha_ok = _rtl92du_phy_patha_iqk(hw, is2t); + if (patha_ok == 0x03) { + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path A IQK Success!!\n"); + result[t][0] = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_A, + MASK_IQK_RESULT); + result[t][1] = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_A, + MASK_IQK_RESULT); + result[t][2] = rtl_get_bbreg(hw, RRX_POWER_BEFORE_IQK_A_2, + MASK_IQK_RESULT); + result[t][3] = rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_A_2, + MASK_IQK_RESULT); + break; + } else if (i == (retrycount - 1) && patha_ok == 0x01) { + /* Tx IQK OK */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path A IQK Only Tx Success!!\n"); + + result[t][0] = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_A, + MASK_IQK_RESULT); + result[t][1] = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_A, + MASK_IQK_RESULT); + } + } + if (patha_ok == 0x00) + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path A IQK failed!!\n"); + + if (is2t) { + _rtl92du_phy_patha_standby(hw); + /* Turn Path B ADDA on */ + rtl92d_phy_path_adda_on(hw, adda_reg, false, is2t); + + for (i = 0; i < retrycount; i++) { + pathb_ok = _rtl92du_phy_pathb_iqk(hw); + if (pathb_ok == 0x03) { + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path B IQK Success!!\n"); + result[t][4] = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_B, + MASK_IQK_RESULT); + result[t][5] = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_B, + MASK_IQK_RESULT); + result[t][6] = rtl_get_bbreg(hw, RRX_POWER_BEFORE_IQK_B_2, + MASK_IQK_RESULT); + result[t][7] = rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_B_2, + MASK_IQK_RESULT); + break; + } else if (i == (retrycount - 1) && pathb_ok == 0x01) { + /* Tx IQK OK */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path B Only Tx IQK Success!!\n"); + result[t][4] = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_B, + MASK_IQK_RESULT); + result[t][5] = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_B, + MASK_IQK_RESULT); + } + } + if (pathb_ok == 0x00) + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path B IQK failed!!\n"); + } + + /* Back to BB mode, load original value */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "IQK:Back to BB mode, load original value!\n"); + + rtl_set_bbreg(hw, RFPGA0_IQK, MASKH3BYTES, 0x000000); + + if (t != 0) { + /* Switch back BB to SI mode after finish IQ Calibration. */ + if (!rtlphy->rfpi_enable) + _rtl92du_phy_pimode_switch(hw, false); + + /* Reload ADDA power saving parameters */ + _rtl92du_phy_reload_adda_registers(hw, adda_reg, + rtlphy->adda_backup, + IQK_ADDA_REG_NUM); + + /* Reload MAC parameters */ + _rtl92du_phy_reload_mac_registers(hw, iqk_mac_reg, + rtlphy->iqk_mac_backup); + + if (is2t) + _rtl92du_phy_reload_adda_registers(hw, iqk_bb_reg, + rtlphy->iqk_bb_backup, + IQK_BB_REG_NUM); + else + _rtl92du_phy_reload_adda_registers(hw, iqk_bb_reg, + rtlphy->iqk_bb_backup, + IQK_BB_REG_NUM - 1); + + /* load 0xe30 IQC default value */ + rtl_set_bbreg(hw, RTX_IQK_TONE_A, MASKDWORD, 0x01008c00); + rtl_set_bbreg(hw, RRX_IQK_TONE_A, MASKDWORD, 0x01008c00); + } + RTPRINT(rtlpriv, FINIT, INIT_IQK, "<==\n"); +} + +static void _rtl92du_phy_iq_calibrate_5g_normal(struct ieee80211_hw *hw, + long result[][8], u8 t) +{ + static const u32 adda_reg[IQK_ADDA_REG_NUM] = { + RFPGA0_XCD_SWITCHCONTROL, RBLUE_TOOTH, RRX_WAIT_CCA, + RTX_CCK_RFON, RTX_CCK_BBON, RTX_OFDM_RFON, RTX_OFDM_BBON, + RTX_TO_RX, RTX_TO_TX, RRX_CCK, RRX_OFDM, RRX_WAIT_RIFS, + RRX_TO_RX, RSTANDBY, RSLEEP, RPMPD_ANAEN + }; + static const u32 iqk_mac_reg[IQK_MAC_REG_NUM] = { + REG_TXPAUSE, REG_BCN_CTRL, REG_BCN_CTRL_1, REG_GPIO_MUXCFG + }; + static const u32 iqk_bb_reg[IQK_BB_REG_NUM] = { + RFPGA0_XAB_RFINTERFACESW, RFPGA0_XA_RFINTERFACEOE, + RFPGA0_XB_RFINTERFACEOE, ROFDM0_TRMUXPAR, + RFPGA0_XCD_RFINTERFACESW, ROFDM0_TRXPATHENABLE, + RFPGA0_RFMOD, RFPGA0_ANALOGPARAMETER4, + ROFDM0_XAAGCCORE1, ROFDM0_XBAGCCORE1 + }; + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_phy *rtlphy = &rtlpriv->phy; + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + bool is2t = IS_92D_SINGLEPHY(rtlhal->version); + u8 patha_ok, pathb_ok; + bool rf_path_div; + u32 bbvalue; + + /* Note: IQ calibration must be performed after loading + * PHY_REG.txt , and radio_a, radio_b.txt + */ + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "IQK for 5G NORMAL:Start!!!\n"); + + mdelay(IQK_DELAY_TIME * 20); + + if (t == 0) { + bbvalue = rtl_get_bbreg(hw, RFPGA0_RFMOD, MASKDWORD); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "==>0x%08x\n", bbvalue); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "IQ Calibration for %s\n", + is2t ? "2T2R" : "1T1R"); + + /* Save ADDA parameters, turn Path A ADDA on */ + rtl92d_phy_save_adda_registers(hw, adda_reg, + rtlphy->adda_backup, + IQK_ADDA_REG_NUM); + rtl92d_phy_save_mac_registers(hw, iqk_mac_reg, + rtlphy->iqk_mac_backup); + if (is2t) + rtl92d_phy_save_adda_registers(hw, iqk_bb_reg, + rtlphy->iqk_bb_backup, + IQK_BB_REG_NUM); + else + rtl92d_phy_save_adda_registers(hw, iqk_bb_reg, + rtlphy->iqk_bb_backup, + IQK_BB_REG_NUM - 1); + } + + rf_path_div = rtl_get_bbreg(hw, 0xb30, BIT(27)); + rtl92d_phy_path_adda_on(hw, adda_reg, !rf_path_div, is2t); + + if (t == 0) + rtlphy->rfpi_enable = rtl_get_bbreg(hw, + RFPGA0_XA_HSSIPARAMETER1, + BIT(8)); + + /* Switch BB to PI mode to do IQ Calibration. */ + if (!rtlphy->rfpi_enable) + _rtl92du_phy_pimode_switch(hw, true); + + /* MAC settings */ + rtl92d_phy_mac_setting_calibration(hw, iqk_mac_reg, + rtlphy->iqk_mac_backup); + + rtl92du_phy_set_bb_reg_1byte(hw, RFPGA0_RFMOD, BCCKEN, 0x00); + rtl_set_bbreg(hw, ROFDM0_TRXPATHENABLE, MASKDWORD, 0x03a05600); + rtl_set_bbreg(hw, ROFDM0_TRMUXPAR, MASKDWORD, 0x000800e4); + rtl_set_bbreg(hw, RFPGA0_XCD_RFINTERFACESW, MASKDWORD, 0x22208000); + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER4, 0xf00000, 0x0f); + + /* Page A AP setting for IQK */ + rtl_set_bbreg(hw, RPDP_ANTA, MASKDWORD, 0); + rtl_set_bbreg(hw, RCONFIG_ANTA, MASKDWORD, 0x20000000); + if (is2t) { + /* Page B AP setting for IQK */ + rtl_set_bbreg(hw, RPDP_ANTB, MASKDWORD, 0); + rtl_set_bbreg(hw, RCONFIG_ANTB, MASKDWORD, 0x20000000); + } + + /* IQ calibration setting */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, "IQK setting!\n"); + rtl_set_bbreg(hw, RFPGA0_IQK, MASKH3BYTES, 0x808000); + rtl_set_bbreg(hw, RTX_IQK, MASKDWORD, 0x10007c00); + rtl_set_bbreg(hw, RRX_IQK, MASKDWORD, 0x01004800); + + patha_ok = _rtl92du_phy_patha_iqk_5g_normal(hw, is2t); + if (patha_ok == 0x03) { + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path A IQK Success!!\n"); + result[t][0] = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_A, + MASK_IQK_RESULT); + result[t][1] = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_A, + MASK_IQK_RESULT); + result[t][2] = rtl_get_bbreg(hw, RRX_POWER_BEFORE_IQK_A_2, + MASK_IQK_RESULT); + result[t][3] = rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_A_2, + MASK_IQK_RESULT); + } else if (patha_ok == 0x01) { /* Tx IQK OK */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path A IQK Only Tx Success!!\n"); + + result[t][0] = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_A, + MASK_IQK_RESULT); + result[t][1] = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_A, + MASK_IQK_RESULT); + } else { + rtl_set_bbreg(hw, RFPGA0_IQK, MASKH3BYTES, 0x000000); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xe70 = %#x\n", + rtl_get_bbreg(hw, RRX_WAIT_CCA, MASKDWORD)); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "RF path A 0x0 = %#x\n", + rtl_get_rfreg(hw, RF90_PATH_A, RF_AC, RFREG_OFFSET_MASK)); + rtl_set_bbreg(hw, RFPGA0_IQK, MASKH3BYTES, 0x808000); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path A IQK Fail!!\n"); + } + + if (is2t) { + /* _rtl92d_phy_patha_standby(hw); */ + /* Turn Path B ADDA on */ + rtl92d_phy_path_adda_on(hw, adda_reg, false, is2t); + + pathb_ok = _rtl92du_phy_pathb_iqk_5g_normal(hw); + if (pathb_ok == 0x03) { + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path B IQK Success!!\n"); + result[t][4] = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_B, + MASK_IQK_RESULT); + result[t][5] = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_B, + MASK_IQK_RESULT); + result[t][6] = rtl_get_bbreg(hw, RRX_POWER_BEFORE_IQK_B_2, + MASK_IQK_RESULT); + result[t][7] = rtl_get_bbreg(hw, RRX_POWER_AFTER_IQK_B_2, + MASK_IQK_RESULT); + } else if (pathb_ok == 0x01) { /* Tx IQK OK */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path B Only Tx IQK Success!!\n"); + result[t][4] = rtl_get_bbreg(hw, RTX_POWER_BEFORE_IQK_B, + MASK_IQK_RESULT); + result[t][5] = rtl_get_bbreg(hw, RTX_POWER_AFTER_IQK_B, + MASK_IQK_RESULT); + } else { + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path B IQK failed!!\n"); + } + } + + /* Back to BB mode, load original value */ + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "IQK:Back to BB mode, load original value!\n"); + rtl_set_bbreg(hw, RFPGA0_IQK, MASKH3BYTES, 0); + + if (is2t) + _rtl92du_phy_reload_adda_registers(hw, iqk_bb_reg, + rtlphy->iqk_bb_backup, + IQK_BB_REG_NUM); + else + _rtl92du_phy_reload_adda_registers(hw, iqk_bb_reg, + rtlphy->iqk_bb_backup, + IQK_BB_REG_NUM - 1); + + /* path A IQ path to DP block */ + rtl_set_bbreg(hw, RPDP_ANTA, MASKDWORD, 0x010170b8); + if (is2t) /* path B IQ path to DP block */ + rtl_set_bbreg(hw, RPDP_ANTB, MASKDWORD, 0x010170b8); + + /* Reload MAC parameters */ + _rtl92du_phy_reload_mac_registers(hw, iqk_mac_reg, + rtlphy->iqk_mac_backup); + + /* Switch back BB to SI mode after finish IQ Calibration. */ + if (!rtlphy->rfpi_enable) + _rtl92du_phy_pimode_switch(hw, false); + + /* Reload ADDA power saving parameters */ + _rtl92du_phy_reload_adda_registers(hw, adda_reg, + rtlphy->adda_backup, + IQK_ADDA_REG_NUM); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "<==\n"); +} + +static bool _rtl92du_phy_simularity_compare(struct ieee80211_hw *hw, + long result[][8], u8 c1, u8 c2) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + u32 i, j, diff, sim_bitmap, bound, u4temp = 0; + u8 final_candidate[2] = {0xFF, 0xFF}; /* for path A and path B */ + bool is2t = IS_92D_SINGLEPHY(rtlhal->version); + bool bresult = true; + + if (is2t) + bound = 8; + else + bound = 4; + + sim_bitmap = 0; + + for (i = 0; i < bound; i++) { + diff = abs_diff(result[c1][i], result[c2][i]); + + if (diff > MAX_TOLERANCE_92D) { + if ((i == 2 || i == 6) && !sim_bitmap) { + if (result[c1][i] + result[c1][i + 1] == 0) + final_candidate[(i / 4)] = c2; + else if (result[c2][i] + result[c2][i + 1] == 0) + final_candidate[(i / 4)] = c1; + else + sim_bitmap = sim_bitmap | (1 << i); + } else { + sim_bitmap = sim_bitmap | (1 << i); + } + } + } + + if (sim_bitmap == 0) { + for (i = 0; i < (bound / 4); i++) { + if (final_candidate[i] != 0xFF) { + for (j = i * 4; j < (i + 1) * 4 - 2; j++) + result[3][j] = + result[final_candidate[i]][j]; + bresult = false; + } + } + + for (i = 0; i < bound; i++) + u4temp += result[c1][i] + result[c2][i]; + + if (u4temp == 0) /* IQK fail for c1 & c2 */ + bresult = false; + + return bresult; + } + + if (!(sim_bitmap & 0x0F)) { /* path A OK */ + for (i = 0; i < 4; i++) + result[3][i] = result[c1][i]; + } else if (!(sim_bitmap & 0x03)) { /* path A, Tx OK */ + for (i = 0; i < 2; i++) + result[3][i] = result[c1][i]; + } + + if (!(sim_bitmap & 0xF0) && is2t) { /* path B OK */ + for (i = 4; i < 8; i++) + result[3][i] = result[c1][i]; + } else if (!(sim_bitmap & 0x30)) { /* path B, Tx OK */ + for (i = 4; i < 6; i++) + result[3][i] = result[c1][i]; + } + + return false; +} + +static void _rtl92du_phy_patha_fill_iqk_matrix_5g_normal(struct ieee80211_hw *hw, + bool iqk_ok, + long result[][8], + u8 final_candidate, + bool txonly) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + u32 val_x, reg; + int val_y; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path A IQ Calibration %s !\n", iqk_ok ? "Success" : "Failed"); + if (iqk_ok && final_candidate != 0xFF) { + val_x = result[final_candidate][0]; + if ((val_x & 0x00000200) != 0) + val_x = val_x | 0xFFFFFC00; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "X = 0x%x\n", val_x); + rtl_set_bbreg(hw, RTX_IQK_TONE_A, 0x3FF0000, val_x); + rtl_set_bbreg(hw, ROFDM0_ECCATHRESHOLD, BIT(24), 0); + + val_y = result[final_candidate][1]; + if ((val_y & 0x00000200) != 0) + val_y = val_y | 0xFFFFFC00; + + /* path B IQK result + 3 */ + if (rtlhal->current_bandtype == BAND_ON_5G) + val_y += 3; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Y = 0x%x\n", val_y); + + rtl_set_bbreg(hw, RTX_IQK_TONE_A, 0x3FF, val_y); + rtl_set_bbreg(hw, ROFDM0_ECCATHRESHOLD, BIT(26), 0); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xe30 = 0x%x\n", + rtl_get_bbreg(hw, RTX_IQK_TONE_A, MASKDWORD)); + + if (txonly) { + RTPRINT(rtlpriv, FINIT, INIT_IQK, "only Tx OK\n"); + return; + } + + reg = result[final_candidate][2]; + rtl_set_bbreg(hw, ROFDM0_XARXIQIMBALANCE, 0x3FF, reg); + reg = result[final_candidate][3] & 0x3F; + rtl_set_bbreg(hw, ROFDM0_XARXIQIMBALANCE, 0xFC00, reg); + reg = (result[final_candidate][3] >> 6) & 0xF; + rtl_set_bbreg(hw, ROFDM0_RXIQEXTANTA, 0xF0000000, reg); + } else { + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "%s: Tx/Rx fail restore default value\n", __func__); + + rtl_set_bbreg(hw, RTX_IQK_TONE_A, MASKDWORD, 0x19008c00); + rtl_set_bbreg(hw, ROFDM0_XARXIQIMBALANCE, MASKDWORD, 0x40000100); + rtl_set_bbreg(hw, RRX_IQK_TONE_A, MASKDWORD, 0x19008c00); + } +} + +static void _rtl92du_phy_patha_fill_iqk_matrix(struct ieee80211_hw *hw, + bool iqk_ok, long result[][8], + u8 final_candidate, bool txonly) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + u32 oldval_0, val_x, tx0_a, reg; + long val_y, tx0_c; + bool is2t = IS_92D_SINGLEPHY(rtlhal->version) || + rtlhal->macphymode == DUALMAC_DUALPHY; + + if (rtlhal->current_bandtype == BAND_ON_5G) { + _rtl92du_phy_patha_fill_iqk_matrix_5g_normal(hw, iqk_ok, result, + final_candidate, + txonly); + return; + } + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path A IQ Calibration %s !\n", iqk_ok ? "Success" : "Failed"); + if (final_candidate == 0xFF || !iqk_ok) + return; + + /* OFDM0_D */ + oldval_0 = rtl_get_bbreg(hw, ROFDM0_XATXIQIMBALANCE, 0xffc00000); + + val_x = result[final_candidate][0]; + if ((val_x & 0x00000200) != 0) + val_x = val_x | 0xFFFFFC00; + + tx0_a = (val_x * oldval_0) >> 8; + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "X = 0x%x, tx0_a = 0x%x, oldval_0 0x%x\n", + val_x, tx0_a, oldval_0); + rtl_set_bbreg(hw, ROFDM0_XATXIQIMBALANCE, 0x3FF, tx0_a); + rtl_set_bbreg(hw, ROFDM0_ECCATHRESHOLD, BIT(24), + ((val_x * oldval_0 >> 7) & 0x1)); + + val_y = result[final_candidate][1]; + if ((val_y & 0x00000200) != 0) + val_y = val_y | 0xFFFFFC00; + + /* path B IQK result + 3 */ + if (rtlhal->interfaceindex == 1 && + rtlhal->current_bandtype == BAND_ON_5G) + val_y += 3; + + tx0_c = (val_y * oldval_0) >> 8; + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Y = 0x%lx, tx0_c = 0x%lx\n", + val_y, tx0_c); + + rtl_set_bbreg(hw, ROFDM0_XCTXAFE, 0xF0000000, (tx0_c & 0x3C0) >> 6); + rtl_set_bbreg(hw, ROFDM0_XATXIQIMBALANCE, 0x003F0000, tx0_c & 0x3F); + if (is2t) + rtl_set_bbreg(hw, ROFDM0_ECCATHRESHOLD, BIT(26), + (val_y * oldval_0 >> 7) & 0x1); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xC80 = 0x%x\n", + rtl_get_bbreg(hw, ROFDM0_XATXIQIMBALANCE, + MASKDWORD)); + + if (txonly) { + RTPRINT(rtlpriv, FINIT, INIT_IQK, "only Tx OK\n"); + return; + } + + reg = result[final_candidate][2]; + rtl_set_bbreg(hw, ROFDM0_XARXIQIMBALANCE, 0x3FF, reg); + reg = result[final_candidate][3] & 0x3F; + rtl_set_bbreg(hw, ROFDM0_XARXIQIMBALANCE, 0xFC00, reg); + reg = (result[final_candidate][3] >> 6) & 0xF; + rtl_set_bbreg(hw, ROFDM0_RXIQEXTANTA, 0xF0000000, reg); +} + +static void _rtl92du_phy_pathb_fill_iqk_matrix_5g_normal(struct ieee80211_hw *hw, + bool iqk_ok, + long result[][8], + u8 final_candidate, + bool txonly) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + u32 val_x, reg; + int val_y; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "Path B IQ Calibration %s !\n", iqk_ok ? "Success" : "Failed"); + if (iqk_ok && final_candidate != 0xFF) { + val_x = result[final_candidate][4]; + if ((val_x & 0x00000200) != 0) + val_x = val_x | 0xFFFFFC00; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "X = 0x%x\n", val_x); + rtl_set_bbreg(hw, RTX_IQK_TONE_B, 0x3FF0000, val_x); + rtl_set_bbreg(hw, ROFDM0_ECCATHRESHOLD, BIT(28), 0); + + val_y = result[final_candidate][5]; + if ((val_y & 0x00000200) != 0) + val_y = val_y | 0xFFFFFC00; + + /* path B IQK result + 3 */ + if (rtlhal->current_bandtype == BAND_ON_5G) + val_y += 3; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Y = 0x%x\n", val_y); + + rtl_set_bbreg(hw, RTX_IQK_TONE_B, 0x3FF, val_y); + rtl_set_bbreg(hw, ROFDM0_ECCATHRESHOLD, BIT(30), 0); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "0xe50 = 0x%x\n", + rtl_get_bbreg(hw, RTX_IQK_TONE_B, MASKDWORD)); + + if (txonly) { + RTPRINT(rtlpriv, FINIT, INIT_IQK, "only Tx OK\n"); + return; + } + + reg = result[final_candidate][6]; + rtl_set_bbreg(hw, ROFDM0_XBRXIQIMBALANCE, 0x3FF, reg); + reg = result[final_candidate][7] & 0x3F; + rtl_set_bbreg(hw, ROFDM0_XBRXIQIMBALANCE, 0xFC00, reg); + reg = (result[final_candidate][7] >> 6) & 0xF; + rtl_set_bbreg(hw, ROFDM0_AGCRSSITABLE, 0x0000F000, reg); + } else { + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "%s: Tx/Rx fail restore default value\n", __func__); + + rtl_set_bbreg(hw, RTX_IQK_TONE_B, MASKDWORD, 0x19008c00); + rtl_set_bbreg(hw, ROFDM0_XBRXIQIMBALANCE, MASKDWORD, 0x40000100); + rtl_set_bbreg(hw, RRX_IQK_TONE_B, MASKDWORD, 0x19008c00); + } +} + +static void _rtl92du_phy_pathb_fill_iqk_matrix(struct ieee80211_hw *hw, + bool iqk_ok, long result[][8], + u8 final_candidate, bool txonly) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + u32 oldval_1, val_x, tx1_a, reg; + long val_y, tx1_c; + + if (rtlhal->current_bandtype == BAND_ON_5G) { + _rtl92du_phy_pathb_fill_iqk_matrix_5g_normal(hw, iqk_ok, result, + final_candidate, + txonly); + return; + } + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Path B IQ Calibration %s !\n", + iqk_ok ? "Success" : "Failed"); + + if (final_candidate == 0xFF || !iqk_ok) + return; + + oldval_1 = rtl_get_bbreg(hw, ROFDM0_XBTXIQIMBALANCE, 0xffc00000); + + val_x = result[final_candidate][4]; + if ((val_x & 0x00000200) != 0) + val_x = val_x | 0xFFFFFC00; + + tx1_a = (val_x * oldval_1) >> 8; + RTPRINT(rtlpriv, FINIT, INIT_IQK, "X = 0x%x, tx1_a = 0x%x\n", + val_x, tx1_a); + rtl_set_bbreg(hw, ROFDM0_XBTXIQIMBALANCE, 0x3FF, tx1_a); + rtl_set_bbreg(hw, ROFDM0_ECCATHRESHOLD, BIT(28), + (val_x * oldval_1 >> 7) & 0x1); + + val_y = result[final_candidate][5]; + if ((val_y & 0x00000200) != 0) + val_y = val_y | 0xFFFFFC00; + + if (rtlhal->current_bandtype == BAND_ON_5G) + val_y += 3; + + tx1_c = (val_y * oldval_1) >> 8; + RTPRINT(rtlpriv, FINIT, INIT_IQK, "Y = 0x%lx, tx1_c = 0x%lx\n", + val_y, tx1_c); + + rtl_set_bbreg(hw, ROFDM0_XDTXAFE, 0xF0000000, (tx1_c & 0x3C0) >> 6); + rtl_set_bbreg(hw, ROFDM0_XBTXIQIMBALANCE, 0x003F0000, tx1_c & 0x3F); + rtl_set_bbreg(hw, ROFDM0_ECCATHRESHOLD, BIT(30), + (val_y * oldval_1 >> 7) & 0x1); + + if (txonly) + return; + + reg = result[final_candidate][6]; + rtl_set_bbreg(hw, ROFDM0_XBRXIQIMBALANCE, 0x3FF, reg); + reg = result[final_candidate][7] & 0x3F; + rtl_set_bbreg(hw, ROFDM0_XBRXIQIMBALANCE, 0xFC00, reg); + reg = (result[final_candidate][7] >> 6) & 0xF; + rtl_set_bbreg(hw, ROFDM0_AGCRSSITABLE, 0x0000F000, reg); +} + +void rtl92du_phy_iq_calibrate(struct ieee80211_hw *hw) +{ + long rege94, rege9c, regea4, regeac, regeb4; + bool is12simular, is13simular, is23simular; + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + long regebc, regec4, regecc, regtmp = 0; + struct rtl_phy *rtlphy = &rtlpriv->phy; + u8 i, final_candidate, indexforchannel; + bool patha_ok, pathb_ok; + long result[4][8] = {}; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "IQK:Start!!!channel %d\n", rtlphy->current_channel); + + final_candidate = 0xff; + patha_ok = false; + pathb_ok = false; + is12simular = false; + is23simular = false; + is13simular = false; + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "IQK !!!currentband %d\n", rtlhal->current_bandtype); + + for (i = 0; i < 3; i++) { + if (rtlhal->current_bandtype == BAND_ON_5G) { + _rtl92du_phy_iq_calibrate_5g_normal(hw, result, i); + } else if (rtlhal->current_bandtype == BAND_ON_2_4G) { + if (IS_92D_SINGLEPHY(rtlhal->version)) + _rtl92du_phy_iq_calibrate(hw, result, i, true); + else + _rtl92du_phy_iq_calibrate(hw, result, i, false); + } + + if (i == 1) { + is12simular = _rtl92du_phy_simularity_compare(hw, result, + 0, 1); + if (is12simular) { + final_candidate = 0; + break; + } + } + + if (i == 2) { + is13simular = _rtl92du_phy_simularity_compare(hw, result, + 0, 2); + if (is13simular) { + final_candidate = 0; + break; + } + + is23simular = _rtl92du_phy_simularity_compare(hw, result, + 1, 2); + if (is23simular) { + final_candidate = 1; + } else { + for (i = 0; i < 8; i++) + regtmp += result[3][i]; + + if (regtmp != 0) + final_candidate = 3; + else + final_candidate = 0xFF; + } + } + } + + for (i = 0; i < 4; i++) { + rege94 = result[i][0]; + rege9c = result[i][1]; + regea4 = result[i][2]; + regeac = result[i][3]; + regeb4 = result[i][4]; + regebc = result[i][5]; + regec4 = result[i][6]; + regecc = result[i][7]; + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "IQK: rege94=%lx rege9c=%lx regea4=%lx regeac=%lx regeb4=%lx regebc=%lx regec4=%lx regecc=%lx\n", + rege94, rege9c, regea4, regeac, regeb4, regebc, regec4, + regecc); + } + + if (final_candidate != 0xff) { + rege94 = result[final_candidate][0]; + rtlphy->reg_e94 = rege94; + rege9c = result[final_candidate][1]; + rtlphy->reg_e9c = rege9c; + regea4 = result[final_candidate][2]; + regeac = result[final_candidate][3]; + regeb4 = result[final_candidate][4]; + rtlphy->reg_eb4 = regeb4; + regebc = result[final_candidate][5]; + rtlphy->reg_ebc = regebc; + regec4 = result[final_candidate][6]; + regecc = result[final_candidate][7]; + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "IQK: final_candidate is %x\n", final_candidate); + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "IQK: rege94=%lx rege9c=%lx regea4=%lx regeac=%lx regeb4=%lx regebc=%lx regec4=%lx regecc=%lx\n", + rege94, rege9c, regea4, regeac, regeb4, regebc, regec4, + regecc); + + patha_ok = true; + pathb_ok = true; + } else { + rtlphy->reg_e94 = 0x100; + rtlphy->reg_eb4 = 0x100; /* X default value */ + rtlphy->reg_e9c = 0x0; + rtlphy->reg_ebc = 0x0; /* Y default value */ + } + if (rege94 != 0 /*&& regea4 != 0*/) + _rtl92du_phy_patha_fill_iqk_matrix(hw, patha_ok, result, + final_candidate, + regea4 == 0); + if (IS_92D_SINGLEPHY(rtlhal->version) && + regeb4 != 0 /*&& regec4 != 0*/) + _rtl92du_phy_pathb_fill_iqk_matrix(hw, pathb_ok, result, + final_candidate, + regec4 == 0); + + if (final_candidate != 0xFF) { + indexforchannel = + rtl92d_get_rightchnlplace_for_iqk(rtlphy->current_channel); + + for (i = 0; i < IQK_MATRIX_REG_NUM; i++) + rtlphy->iqk_matrix[indexforchannel].value[0][i] = + result[final_candidate][i]; + + rtlphy->iqk_matrix[indexforchannel].iqk_done = true; + + rtl_dbg(rtlpriv, COMP_SCAN | COMP_MLME, DBG_LOUD, + "IQK OK indexforchannel %d\n", indexforchannel); + } +} + +void rtl92du_phy_reload_iqk_setting(struct ieee80211_hw *hw, u8 channel) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + struct rtl_phy *rtlphy = &rtlpriv->phy; + struct rtl_mac *mac = rtl_mac(rtlpriv); + u8 indexforchannel; + bool need_iqk; + + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "channel %d\n", channel); + /*------Do IQK for normal chip and test chip 5G band------- */ + + indexforchannel = rtl92d_get_rightchnlplace_for_iqk(channel); + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "indexforchannel %d done %d\n", + indexforchannel, + rtlphy->iqk_matrix[indexforchannel].iqk_done); + + /* We need to do IQK if we're about to connect to a network on 5 GHz. + * On 5 GHz a channel switch outside of scanning happens only before + * connecting. + */ + need_iqk = !mac->act_scanning; + + if (!rtlphy->iqk_matrix[indexforchannel].iqk_done && need_iqk) { + rtl_dbg(rtlpriv, COMP_SCAN | COMP_INIT, DBG_LOUD, + "Do IQK Matrix reg for channel:%d....\n", channel); + rtl92du_phy_iq_calibrate(hw); + return; + } + + /* Just load the value. */ + /* 2G band just load once. */ + if ((!rtlhal->load_imrandiqk_setting_for2g && indexforchannel == 0) || + indexforchannel > 0) { + rtl_dbg(rtlpriv, COMP_SCAN, DBG_LOUD, + "Just Read IQK Matrix reg for channel:%d....\n", + channel); + + if (rtlphy->iqk_matrix[indexforchannel].value[0][0] != 0) + _rtl92du_phy_patha_fill_iqk_matrix(hw, true, + rtlphy->iqk_matrix[indexforchannel].value, 0, + rtlphy->iqk_matrix[indexforchannel].value[0][2] == 0); + + if (IS_92D_SINGLEPHY(rtlhal->version) && + rtlphy->iqk_matrix[indexforchannel].value[0][4] != 0) + _rtl92du_phy_pathb_fill_iqk_matrix(hw, true, + rtlphy->iqk_matrix[indexforchannel].value, 0, + rtlphy->iqk_matrix[indexforchannel].value[0][6] == 0); + } + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "<====\n"); +} + +static void _rtl92du_phy_reload_lck_setting(struct ieee80211_hw *hw, u8 channel) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + u8 erfpath = rtlhal->current_bandtype == BAND_ON_5G ? RF90_PATH_A : + IS_92D_SINGLEPHY(rtlhal->version) ? RF90_PATH_B : RF90_PATH_A; + bool bneed_powerdown_radio = false; + u32 u4tmp, u4regvalue; + + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "path %d\n", erfpath); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "band type = %d\n", + rtlpriv->rtlhal.current_bandtype); + RTPRINT(rtlpriv, FINIT, INIT_IQK, "channel = %d\n", channel); + + if (rtlpriv->rtlhal.current_bandtype == BAND_ON_5G) {/* Path-A for 5G */ + u4tmp = rtlpriv->curveindex_5g[channel - 1]; + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "ver 1 set RF-A, 5G, 0x28 = 0x%x !!\n", u4tmp); + + if (rtlpriv->rtlhal.macphymode == DUALMAC_DUALPHY && + rtlpriv->rtlhal.interfaceindex == 1) { + bneed_powerdown_radio = + rtl92du_phy_enable_anotherphy(hw, false); + rtlpriv->rtlhal.during_mac1init_radioa = true; + /* asume no this case */ + if (bneed_powerdown_radio) + rtl92d_phy_enable_rf_env(hw, erfpath, + &u4regvalue); + } + + rtl_set_rfreg(hw, erfpath, RF_SYN_G4, 0x3f800, u4tmp); + + if (bneed_powerdown_radio) { + rtl92d_phy_restore_rf_env(hw, erfpath, &u4regvalue); + rtl92du_phy_powerdown_anotherphy(hw, false); + } + } else if (rtlpriv->rtlhal.current_bandtype == BAND_ON_2_4G) { + u4tmp = rtlpriv->curveindex_2g[channel - 1]; + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "ver 3 set RF-B, 2G, 0x28 = 0x%x !!\n", u4tmp); + + if (rtlpriv->rtlhal.macphymode == DUALMAC_DUALPHY && + rtlpriv->rtlhal.interfaceindex == 0) { + bneed_powerdown_radio = + rtl92du_phy_enable_anotherphy(hw, true); + rtlpriv->rtlhal.during_mac0init_radiob = true; + if (bneed_powerdown_radio) + rtl92d_phy_enable_rf_env(hw, erfpath, + &u4regvalue); + } + + rtl_set_rfreg(hw, erfpath, RF_SYN_G4, 0x3f800, u4tmp); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "ver 3 set RF-B, 2G, 0x28 = 0x%x !!\n", + rtl_get_rfreg(hw, erfpath, RF_SYN_G4, 0x3f800)); + + if (bneed_powerdown_radio) { + rtl92d_phy_restore_rf_env(hw, erfpath, &u4regvalue); + rtl92du_phy_powerdown_anotherphy(hw, true); + } + } + rtl_dbg(rtlpriv, COMP_CMD, DBG_LOUD, "<====\n"); +} + +static void _rtl92du_phy_lc_calibrate_sw(struct ieee80211_hw *hw, bool is2t) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + u32 curvecount_val[CV_CURVE_CNT * 2]; + u16 timeout = 800, timecount = 0; + u32 u4tmp, offset, rf_syn_g4[2]; + u8 tmpreg, index, rf_mode[2]; + u8 path = is2t ? 2 : 1; + u8 i; + + /* Check continuous TX and Packet TX */ + tmpreg = rtl_read_byte(rtlpriv, 0xd03); + if ((tmpreg & 0x70) != 0) + /* if Deal with contisuous TX case, disable all continuous TX */ + rtl_write_byte(rtlpriv, 0xd03, tmpreg & 0x8F); + else + /* if Deal with Packet TX case, block all queues */ + rtl_write_byte(rtlpriv, REG_TXPAUSE, 0xFF); + + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER4, 0xF00000, 0x0F); + + for (index = 0; index < path; index++) { + /* 1. Read original RF mode */ + offset = index == 0 ? ROFDM0_XAAGCCORE1 : ROFDM0_XBAGCCORE1; + rf_mode[index] = rtl_read_byte(rtlpriv, offset); + + /* 2. Set RF mode = standby mode */ + rtl_set_rfreg(hw, (enum radio_path)index, RF_AC, + RFREG_OFFSET_MASK, 0x010000); + + rf_syn_g4[index] = rtl_get_rfreg(hw, index, RF_SYN_G4, + RFREG_OFFSET_MASK); + rtl_set_rfreg(hw, index, RF_SYN_G4, 0x700, 0x7); + + /* switch CV-curve control by LC-calibration */ + rtl_set_rfreg(hw, (enum radio_path)index, RF_SYN_G7, + BIT(17), 0x0); + + /* 4. Set LC calibration begin */ + rtl_set_rfreg(hw, (enum radio_path)index, RF_CHNLBW, + 0x08000, 0x01); + } + + for (index = 0; index < path; index++) { + u4tmp = rtl_get_rfreg(hw, (enum radio_path)index, RF_SYN_G6, + RFREG_OFFSET_MASK); + + while ((!(u4tmp & BIT(11))) && timecount <= timeout) { + mdelay(50); + timecount += 50; + u4tmp = rtl_get_rfreg(hw, (enum radio_path)index, + RF_SYN_G6, RFREG_OFFSET_MASK); + } + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "PHY_LCK finish delay for %d ms=2\n", timecount); + } + + if ((tmpreg & 0x70) != 0) + rtl_write_byte(rtlpriv, 0xd03, tmpreg); + else /* Deal with Packet TX case */ + rtl_write_byte(rtlpriv, REG_TXPAUSE, 0x00); + + rtl_set_bbreg(hw, RFPGA0_ANALOGPARAMETER4, 0xF00000, 0x00); + + for (index = 0; index < path; index++) { + rtl_get_rfreg(hw, index, RF_SYN_G4, RFREG_OFFSET_MASK); + + if (index == 0 && rtlhal->interfaceindex == 0) { + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "path-A / 5G LCK\n"); + } else { + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "path-B / 2.4G LCK\n"); + } + + memset(curvecount_val, 0, sizeof(curvecount_val)); + + /* Set LC calibration off */ + rtl_set_rfreg(hw, (enum radio_path)index, RF_CHNLBW, + 0x08000, 0x0); + + RTPRINT(rtlpriv, FINIT, INIT_IQK, "set RF 0x18[15] = 0\n"); + + /* save Curve-counting number */ + for (i = 0; i < CV_CURVE_CNT; i++) { + u32 readval = 0, readval2 = 0; + + rtl_set_rfreg(hw, (enum radio_path)index, 0x3F, + 0x7f, i); + + rtl_set_rfreg(hw, (enum radio_path)index, 0x4D, + RFREG_OFFSET_MASK, 0x0); + + readval = rtl_get_rfreg(hw, (enum radio_path)index, + 0x4F, RFREG_OFFSET_MASK); + curvecount_val[2 * i + 1] = (readval & 0xfffe0) >> 5; + + /* reg 0x4f [4:0] */ + /* reg 0x50 [19:10] */ + readval2 = rtl_get_rfreg(hw, (enum radio_path)index, + 0x50, 0xffc00); + curvecount_val[2 * i] = (((readval & 0x1F) << 10) | + readval2); + } + + if (index == 0 && rtlhal->interfaceindex == 0) + rtl92d_phy_calc_curvindex(hw, targetchnl_5g, + curvecount_val, + true, rtlpriv->curveindex_5g); + else + rtl92d_phy_calc_curvindex(hw, targetchnl_2g, + curvecount_val, + false, rtlpriv->curveindex_2g); + + /* switch CV-curve control mode */ + rtl_set_rfreg(hw, (enum radio_path)index, RF_SYN_G7, + BIT(17), 0x1); + } + + /* Restore original situation */ + for (index = 0; index < path; index++) { + rtl_set_rfreg(hw, index, RF_SYN_G4, RFREG_OFFSET_MASK, + rf_syn_g4[index]); + + offset = index == 0 ? ROFDM0_XAAGCCORE1 : ROFDM0_XBAGCCORE1; + rtl_write_byte(rtlpriv, offset, 0x50); + rtl_write_byte(rtlpriv, offset, rf_mode[index]); + } + + _rtl92du_phy_reload_lck_setting(hw, rtlpriv->phy.current_channel); +} + +void rtl92du_phy_lc_calibrate(struct ieee80211_hw *hw, bool is2t) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + struct rtl_phy *rtlphy = &rtlpriv->phy; + u32 timeout = 2000, timecount = 0; + + while (rtlpriv->mac80211.act_scanning && timecount < timeout) { + udelay(50); + timecount += 50; + } + + rtlphy->lck_inprogress = true; + RTPRINT(rtlpriv, FINIT, INIT_IQK, + "LCK:Start!!! currentband %x delay %d ms\n", + rtlhal->current_bandtype, timecount); + + _rtl92du_phy_lc_calibrate_sw(hw, is2t); + + rtlphy->lck_inprogress = false; + RTPRINT(rtlpriv, FINIT, INIT_IQK, "LCK:Finish!!!\n"); +} + +void rtl92du_phy_ap_calibrate(struct ieee80211_hw *hw, s8 delta) +{ + /* Nothing to do. */ +} + +u8 rtl92du_phy_sw_chnl(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + struct rtl_phy *rtlphy = &rtlpriv->phy; + u8 num_total_rfpath = rtlphy->num_total_rfpath; + u8 channel = rtlphy->current_channel; + u32 timeout = 1000, timecount = 0; + u32 ret_value; + u8 rfpath; + + if (rtlphy->sw_chnl_inprogress) + return 0; + if (rtlphy->set_bwmode_inprogress) + return 0; + + if ((is_hal_stop(rtlhal)) || (RT_CANNOT_IO(hw))) { + rtl_dbg(rtlpriv, COMP_CHAN, DBG_LOUD, + "sw_chnl_inprogress false driver sleep or unload\n"); + return 0; + } + + while (rtlphy->lck_inprogress && timecount < timeout) { + mdelay(50); + timecount += 50; + } + + if (rtlhal->macphymode == SINGLEMAC_SINGLEPHY && + rtlhal->bandset == BAND_ON_BOTH) { + ret_value = rtl_get_bbreg(hw, RFPGA0_XAB_RFPARAMETER, + MASKDWORD); + if (rtlphy->current_channel > 14 && !(ret_value & BIT(0))) + rtl92du_phy_switch_wirelessband(hw, BAND_ON_5G); + else if (rtlphy->current_channel <= 14 && (ret_value & BIT(0))) + rtl92du_phy_switch_wirelessband(hw, BAND_ON_2_4G); + } + + switch (rtlhal->current_bandtype) { + case BAND_ON_5G: + /* Get first channel error when change between + * 5G and 2.4G band. + */ + if (WARN_ONCE(channel <= 14, "rtl8192du: 5G but channel<=14\n")) + return 0; + break; + case BAND_ON_2_4G: + /* Get first channel error when change between + * 5G and 2.4G band. + */ + if (WARN_ONCE(channel > 14, "rtl8192du: 2G but channel>14\n")) + return 0; + break; + default: + WARN_ONCE(true, "rtl8192du: Invalid WirelessMode(%#x)!!\n", + rtlpriv->mac80211.mode); + break; + } + + rtlphy->sw_chnl_inprogress = true; + + rtl_dbg(rtlpriv, COMP_SCAN, DBG_TRACE, + "switch to channel%d\n", rtlphy->current_channel); + + rtl92d_phy_set_txpower_level(hw, channel); + + for (rfpath = 0; rfpath < num_total_rfpath; rfpath++) { + u32p_replace_bits(&rtlphy->rfreg_chnlval[rfpath], + channel, 0xff); + + if (rtlpriv->rtlhal.current_bandtype == BAND_ON_5G) { + if (channel > 99) + rtlphy->rfreg_chnlval[rfpath] |= (BIT(18)); + else + rtlphy->rfreg_chnlval[rfpath] &= ~BIT(18); + rtlphy->rfreg_chnlval[rfpath] |= (BIT(16) | BIT(8)); + } else { + rtlphy->rfreg_chnlval[rfpath] &= + ~(BIT(8) | BIT(16) | BIT(18)); + } + rtl_set_rfreg(hw, rfpath, RF_CHNLBW, RFREG_OFFSET_MASK, + rtlphy->rfreg_chnlval[rfpath]); + + _rtl92du_phy_reload_imr_setting(hw, channel, rfpath); + } + + _rtl92du_phy_switch_rf_setting(hw, channel); + + /* do IQK when all parameters are ready */ + rtl92du_phy_reload_iqk_setting(hw, channel); + + rtl_dbg(rtlpriv, COMP_SCAN, DBG_TRACE, "<==\n"); + rtlphy->sw_chnl_inprogress = false; + return 1; +} + +static void _rtl92du_phy_set_rfon(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + + /* a. SYS_CLKR 0x08[11] = 1 restore MAC clock */ + /* b. SPS_CTRL 0x11[7:0] = 0x2b */ + if (rtlpriv->rtlhal.macphymode == SINGLEMAC_SINGLEPHY) + rtl_write_byte(rtlpriv, REG_SPS0_CTRL, 0x2b); + + /* c. For PCIE: SYS_FUNC_EN 0x02[7:0] = 0xE3 enable BB TRX function */ + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN, 0xE3); + + /* RF_ON_EXCEP(d~g): */ + /* d. APSD_CTRL 0x600[7:0] = 0x00 */ + rtl_write_byte(rtlpriv, REG_APSD_CTRL, 0x00); + + /* e. SYS_FUNC_EN 0x02[7:0] = 0xE2 reset BB TRX function again */ + /* f. SYS_FUNC_EN 0x02[7:0] = 0xE3 enable BB TRX function*/ + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN, 0xE2); + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN, 0xE3); + + /* g. txpause 0x522[7:0] = 0x00 enable mac tx queue */ + rtl_write_byte(rtlpriv, REG_TXPAUSE, 0x00); +} + +static void _rtl92du_phy_set_rfsleep(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + u32 u4btmp; + u8 retry = 5; + + /* a. TXPAUSE 0x522[7:0] = 0xFF Pause MAC TX queue */ + rtl_write_byte(rtlpriv, REG_TXPAUSE, 0xFF); + + /* b. RF path 0 offset 0x00 = 0x00 disable RF */ + rtl_set_rfreg(hw, RF90_PATH_A, 0x00, RFREG_OFFSET_MASK, 0x00); + + /* c. APSD_CTRL 0x600[7:0] = 0x40 */ + rtl_write_byte(rtlpriv, REG_APSD_CTRL, 0x40); + + /* d. APSD_CTRL 0x600[7:0] = 0x00 + * APSD_CTRL 0x600[7:0] = 0x00 + * RF path 0 offset 0x00 = 0x00 + * APSD_CTRL 0x600[7:0] = 0x40 + */ + u4btmp = rtl_get_rfreg(hw, RF90_PATH_A, 0, RFREG_OFFSET_MASK); + while (u4btmp != 0 && retry > 0) { + rtl_write_byte(rtlpriv, REG_APSD_CTRL, 0x0); + rtl_set_rfreg(hw, RF90_PATH_A, 0x00, RFREG_OFFSET_MASK, 0x00); + rtl_write_byte(rtlpriv, REG_APSD_CTRL, 0x40); + u4btmp = rtl_get_rfreg(hw, RF90_PATH_A, 0, RFREG_OFFSET_MASK); + retry--; + } + if (retry == 0) { + /* Jump out the LPS turn off sequence to RF_ON_EXCEP */ + rtl_write_byte(rtlpriv, REG_APSD_CTRL, 0x00); + + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN, 0xE2); + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN, 0xE3); + rtl_write_byte(rtlpriv, REG_TXPAUSE, 0x00); + rtl_dbg(rtlpriv, COMP_POWER, DBG_LOUD, + "Fail !!! Switch RF timeout\n"); + return; + } + + /* e. For PCIE: SYS_FUNC_EN 0x02[7:0] = 0xE2 reset BB TRX function */ + rtl_write_byte(rtlpriv, REG_SYS_FUNC_EN, 0xE2); + + /* f. SPS_CTRL 0x11[7:0] = 0x22 */ + if (rtlpriv->rtlhal.macphymode == SINGLEMAC_SINGLEPHY) + rtl_write_byte(rtlpriv, REG_SPS0_CTRL, 0x22); +} + +bool rtl92du_phy_set_rf_power_state(struct ieee80211_hw *hw, + enum rf_pwrstate rfpwr_state) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_ps_ctl *ppsc = rtl_psc(rtlpriv); + struct rtl_mac *mac = rtl_mac(rtlpriv); + bool bresult = true; + + if (rfpwr_state == ppsc->rfpwr_state) + return false; + + switch (rfpwr_state) { + case ERFON: + if (ppsc->rfpwr_state == ERFOFF && + RT_IN_PS_LEVEL(ppsc, RT_RF_OFF_LEVL_HALT_NIC)) { + u32 initializecount = 0; + bool rtstatus; + + do { + initializecount++; + rtl_dbg(rtlpriv, COMP_RF, DBG_DMESG, + "IPS Set eRf nic enable\n"); + rtstatus = rtl_ps_enable_nic(hw); + } while (!rtstatus && (initializecount < 10)); + + RT_CLEAR_PS_LEVEL(ppsc, + RT_RF_OFF_LEVL_HALT_NIC); + } else { + rtl_dbg(rtlpriv, COMP_POWER, DBG_DMESG, + "awake, slept:%d ms state_inap:%x\n", + jiffies_to_msecs(jiffies - + ppsc->last_sleep_jiffies), + rtlpriv->psc.state_inap); + ppsc->last_awake_jiffies = jiffies; + _rtl92du_phy_set_rfon(hw); + } + + if (mac->link_state == MAC80211_LINKED) + rtlpriv->cfg->ops->led_control(hw, LED_CTL_LINK); + else + rtlpriv->cfg->ops->led_control(hw, LED_CTL_NO_LINK); + break; + case ERFOFF: + if (ppsc->reg_rfps_level & RT_RF_OFF_LEVL_HALT_NIC) { + rtl_dbg(rtlpriv, COMP_RF, DBG_DMESG, + "IPS Set eRf nic disable\n"); + rtl_ps_disable_nic(hw); + RT_SET_PS_LEVEL(ppsc, RT_RF_OFF_LEVL_HALT_NIC); + } else { + if (ppsc->rfoff_reason == RF_CHANGE_BY_IPS) + rtlpriv->cfg->ops->led_control(hw, LED_CTL_NO_LINK); + else + rtlpriv->cfg->ops->led_control(hw, LED_CTL_POWER_OFF); + } + break; + case ERFSLEEP: + if (ppsc->rfpwr_state == ERFOFF) + return false; + + rtl_dbg(rtlpriv, COMP_POWER, DBG_DMESG, + "sleep awakened:%d ms state_inap:%x\n", + jiffies_to_msecs(jiffies - + ppsc->last_awake_jiffies), + rtlpriv->psc.state_inap); + ppsc->last_sleep_jiffies = jiffies; + _rtl92du_phy_set_rfsleep(hw); + break; + default: + pr_err("switch case %#x not processed\n", + rfpwr_state); + return false; + } + + if (bresult) + ppsc->rfpwr_state = rfpwr_state; + + return bresult; +} + +void rtl92du_phy_set_poweron(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtl_priv(hw)); + u32 mac_reg = (rtlhal->interfaceindex == 0 ? REG_MAC0 : REG_MAC1); + u8 value8; + u16 i; + + /* notice fw know band status 0x81[1]/0x53[1] = 0: 5G, 1: 2G */ + if (rtlhal->current_bandtype == BAND_ON_2_4G) { + value8 = rtl_read_byte(rtlpriv, mac_reg); + value8 |= BIT(1); + rtl_write_byte(rtlpriv, mac_reg, value8); + } else { + value8 = rtl_read_byte(rtlpriv, mac_reg); + value8 &= ~BIT(1); + rtl_write_byte(rtlpriv, mac_reg, value8); + } + + if (rtlhal->macphymode == SINGLEMAC_SINGLEPHY) { + value8 = rtl_read_byte(rtlpriv, REG_MAC0); + rtl_write_byte(rtlpriv, REG_MAC0, value8 | MAC0_ON); + } else { + mutex_lock(rtlpriv->mutex_for_power_on_off); + if (rtlhal->interfaceindex == 0) { + value8 = rtl_read_byte(rtlpriv, REG_MAC0); + rtl_write_byte(rtlpriv, REG_MAC0, value8 | MAC0_ON); + } else { + value8 = rtl_read_byte(rtlpriv, REG_MAC1); + rtl_write_byte(rtlpriv, REG_MAC1, value8 | MAC1_ON); + } + value8 = rtl_read_byte(rtlpriv, REG_POWER_OFF_IN_PROCESS); + mutex_unlock(rtlpriv->mutex_for_power_on_off); + + for (i = 0; i < 200; i++) { + if ((value8 & BIT(7)) == 0) + break; + + udelay(500); + mutex_lock(rtlpriv->mutex_for_power_on_off); + value8 = rtl_read_byte(rtlpriv, + REG_POWER_OFF_IN_PROCESS); + mutex_unlock(rtlpriv->mutex_for_power_on_off); + } + if (i == 200) + WARN_ONCE(true, "rtl8192du: Another mac power off over time\n"); + } +} + +void rtl92du_update_bbrf_configuration(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_efuse *rtlefuse = rtl_efuse(rtlpriv); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + struct rtl_phy *rtlphy = &rtlpriv->phy; + u8 rfpath, i; + + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, "==>\n"); + /* r_select_5G for path_A/B 0 for 2.4G, 1 for 5G */ + if (rtlhal->current_bandtype == BAND_ON_2_4G) { + /* r_select_5G for path_A/B, 0x878 */ + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, BIT(0), 0x0); + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, BIT(15), 0x0); + if (rtlhal->macphymode != DUALMAC_DUALPHY) { + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, BIT(16), 0x0); + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, BIT(31), 0x0); + } + + /* rssi_table_select: index 0 for 2.4G. 1~3 for 5G, 0xc78 */ + rtl_set_bbreg(hw, ROFDM0_AGCRSSITABLE, BIT(6) | BIT(7), 0x0); + + /* fc_area 0xd2c */ + rtl_set_bbreg(hw, ROFDM1_CFOTRACKING, BIT(14) | BIT(13), 0x0); + + /* 5G LAN ON */ + rtl_set_bbreg(hw, 0xB30, 0x00F00000, 0xa); + + /* TX BB gain shift*1, Just for testchip, 0xc80, 0xc88 */ + rtl_set_bbreg(hw, ROFDM0_XATXIQIMBALANCE, MASKDWORD, 0x40000100); + rtl_set_bbreg(hw, ROFDM0_XBTXIQIMBALANCE, MASKDWORD, 0x40000100); + if (rtlhal->macphymode == DUALMAC_DUALPHY) { + rtl_set_bbreg(hw, RFPGA0_XAB_RFINTERFACESW, + BIT(10) | BIT(6) | BIT(5), + ((rtlefuse->eeprom_c9 & BIT(3)) >> 3) | + (rtlefuse->eeprom_c9 & BIT(1)) | + ((rtlefuse->eeprom_cc & BIT(1)) << 4)); + rtl_set_bbreg(hw, RFPGA0_XA_RFINTERFACEOE, + BIT(10) | BIT(6) | BIT(5), + ((rtlefuse->eeprom_c9 & BIT(2)) >> 2) | + ((rtlefuse->eeprom_c9 & BIT(0)) << 1) | + ((rtlefuse->eeprom_cc & BIT(0)) << 5)); + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, BIT(15), 0); + + rtl_set_bbreg(hw, RPDP_ANTA, MASKDWORD, 0x01017038); + rtl_set_bbreg(hw, RCONFIG_ANTA, MASKDWORD, 0x0f600000); + } else { + rtl_set_bbreg(hw, RFPGA0_XAB_RFINTERFACESW, + BIT(26) | BIT(22) | BIT(21) | BIT(10) | + BIT(6) | BIT(5), + ((rtlefuse->eeprom_c9 & BIT(3)) >> 3) | + (rtlefuse->eeprom_c9 & BIT(1)) | + ((rtlefuse->eeprom_cc & BIT(1)) << 4) | + ((rtlefuse->eeprom_c9 & BIT(7)) << 9) | + ((rtlefuse->eeprom_c9 & BIT(5)) << 12) | + ((rtlefuse->eeprom_cc & BIT(3)) << 18)); + rtl_set_bbreg(hw, RFPGA0_XA_RFINTERFACEOE, + BIT(10) | BIT(6) | BIT(5), + ((rtlefuse->eeprom_c9 & BIT(2)) >> 2) | + ((rtlefuse->eeprom_c9 & BIT(0)) << 1) | + ((rtlefuse->eeprom_cc & BIT(0)) << 5)); + rtl_set_bbreg(hw, RFPGA0_XB_RFINTERFACEOE, + BIT(10) | BIT(6) | BIT(5), + ((rtlefuse->eeprom_c9 & BIT(6)) >> 6) | + ((rtlefuse->eeprom_c9 & BIT(4)) >> 3) | + ((rtlefuse->eeprom_cc & BIT(2)) << 3)); + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, + BIT(31) | BIT(15), 0); + + rtl_set_bbreg(hw, RPDP_ANTA, MASKDWORD, 0x01017038); + rtl_set_bbreg(hw, RPDP_ANTB, MASKDWORD, 0x01017038); + rtl_set_bbreg(hw, RCONFIG_ANTA, MASKDWORD, 0x0f600000); + rtl_set_bbreg(hw, RCONFIG_ANTB, MASKDWORD, 0x0f600000); + } + /* 1.5V_LDO */ + } else { + /* r_select_5G for path_A/B */ + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, BIT(0), 0x1); + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, BIT(15), 0x1); + if (rtlhal->macphymode != DUALMAC_DUALPHY) { + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, BIT(16), 0x1); + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, BIT(31), 0x1); + } + + /* rssi_table_select: index 0 for 2.4G. 1~3 for 5G */ + rtl_set_bbreg(hw, ROFDM0_AGCRSSITABLE, BIT(6) | BIT(7), 0x1); + + /* fc_area */ + rtl_set_bbreg(hw, ROFDM1_CFOTRACKING, BIT(14) | BIT(13), 0x1); + + /* 5G LAN ON */ + rtl_set_bbreg(hw, 0xB30, 0x00F00000, 0x0); + + /* TX BB gain shift, Just for testchip, 0xc80, 0xc88 */ + if (rtlefuse->internal_pa_5g[rtlhal->interfaceindex]) + rtl_set_bbreg(hw, ROFDM0_XATXIQIMBALANCE, MASKDWORD, + 0x2d4000b5); + else + rtl_set_bbreg(hw, ROFDM0_XATXIQIMBALANCE, MASKDWORD, + 0x20000080); + + if (rtlhal->macphymode != DUALMAC_DUALPHY) { + if (rtlefuse->internal_pa_5g[1]) + rtl_set_bbreg(hw, ROFDM0_XBTXIQIMBALANCE, + MASKDWORD, 0x2d4000b5); + else + rtl_set_bbreg(hw, ROFDM0_XBTXIQIMBALANCE, + MASKDWORD, 0x20000080); + } + + rtl_set_bbreg(hw, 0xB30, BIT(27), 0); + + if (rtlhal->macphymode == DUALMAC_DUALPHY) { + rtl_set_bbreg(hw, RFPGA0_XAB_RFINTERFACESW, + BIT(10) | BIT(6) | BIT(5), + (rtlefuse->eeprom_cc & BIT(5))); + rtl_set_bbreg(hw, RFPGA0_XA_RFINTERFACEOE, BIT(10), + ((rtlefuse->eeprom_cc & BIT(4)) >> 4)); + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, BIT(15), + (rtlefuse->eeprom_cc & BIT(4)) >> 4); + + rtl_set_bbreg(hw, RPDP_ANTA, MASKDWORD, 0x01017098); + rtl_set_bbreg(hw, RCONFIG_ANTA, MASKDWORD, 0x20000000); + } else { + rtl_set_bbreg(hw, RFPGA0_XAB_RFINTERFACESW, + BIT(26) | BIT(22) | BIT(21) | BIT(10) | + BIT(6) | BIT(5), + (rtlefuse->eeprom_cc & BIT(5)) | + ((rtlefuse->eeprom_cc & BIT(7)) << 14)); + rtl_set_bbreg(hw, RFPGA0_XA_RFINTERFACEOE, BIT(10), + ((rtlefuse->eeprom_cc & BIT(4)) >> 4)); + rtl_set_bbreg(hw, RFPGA0_XB_RFINTERFACEOE, BIT(10), + ((rtlefuse->eeprom_cc & BIT(6)) >> 6)); + rtl_set_bbreg(hw, RFPGA0_XAB_RFPARAMETER, + BIT(31) | BIT(15), + ((rtlefuse->eeprom_cc & BIT(4)) >> 4) | + ((rtlefuse->eeprom_cc & BIT(6)) << 10)); + + rtl_set_bbreg(hw, RPDP_ANTA, MASKDWORD, 0x01017098); + rtl_set_bbreg(hw, RPDP_ANTB, MASKDWORD, 0x01017098); + rtl_set_bbreg(hw, RCONFIG_ANTA, MASKDWORD, 0x20000000); + rtl_set_bbreg(hw, RCONFIG_ANTB, MASKDWORD, 0x20000000); + } + } + + /* update IQK related settings */ + rtl_set_bbreg(hw, ROFDM0_XARXIQIMBALANCE, MASKDWORD, 0x40000100); + rtl_set_bbreg(hw, ROFDM0_XBRXIQIMBALANCE, MASKDWORD, 0x40000100); + rtl_set_bbreg(hw, ROFDM0_XCTXAFE, 0xF0000000, 0x00); + rtl_set_bbreg(hw, ROFDM0_ECCATHRESHOLD, BIT(30) | BIT(28) | + BIT(26) | BIT(24), 0x00); + rtl_set_bbreg(hw, ROFDM0_XDTXAFE, 0xF0000000, 0x00); + rtl_set_bbreg(hw, ROFDM0_RXIQEXTANTA, 0xF0000000, 0x00); + rtl_set_bbreg(hw, ROFDM0_AGCRSSITABLE, 0x0000F000, 0x00); + + /* Update RF */ + for (rfpath = RF90_PATH_A; rfpath < rtlphy->num_total_rfpath; + rfpath++) { + if (rtlhal->current_bandtype == BAND_ON_2_4G) { + /* MOD_AG for RF path_A 0x18 BIT8,BIT16 */ + rtl_set_rfreg(hw, rfpath, RF_CHNLBW, BIT(8) | BIT(16) | + BIT(18) | 0xff, 1); + + /* RF0x0b[16:14] =3b'111 */ + rtl_set_rfreg(hw, (enum radio_path)rfpath, 0x0B, + 0x1c000, 0x07); + } else { + /* MOD_AG for RF path_A 0x18 BIT8,BIT16 */ + rtl_set_rfreg(hw, RF90_PATH_A, RF_CHNLBW, RFREG_OFFSET_MASK, + 0x97524); + } + + /* Set right channel on RF reg0x18 for another mac. */ + if (rtlhal->interfaceindex == 0 && rtlhal->bandset == BAND_ON_2_4G) { + /* Set MAC1 default channel if MAC1 not up. */ + if (!(rtl_read_byte(rtlpriv, REG_MAC1) & MAC1_ON)) { + rtl92du_phy_enable_anotherphy(hw, true); + rtlhal->during_mac0init_radiob = true; + rtl_set_rfreg(hw, RF90_PATH_A, RF_CHNLBW, + RFREG_OFFSET_MASK, 0x97524); + rtl92du_phy_powerdown_anotherphy(hw, true); + } + } else if (rtlhal->interfaceindex == 1 && rtlhal->bandset == BAND_ON_5G) { + /* Set MAC0 default channel */ + if (!(rtl_read_byte(rtlpriv, REG_MAC0) & MAC0_ON)) { + rtl92du_phy_enable_anotherphy(hw, false); + rtlhal->during_mac1init_radioa = true; + rtl_set_rfreg(hw, RF90_PATH_A, RF_CHNLBW, + RFREG_OFFSET_MASK, 0x87401); + rtl92du_phy_powerdown_anotherphy(hw, false); + } + } + } + + /* Update for all band. */ + /* DMDP */ + if (rtlphy->rf_type == RF_1T1R) { + /* Use antenna 0, 0xc04, 0xd04 */ + rtl_set_bbreg(hw, ROFDM0_TRXPATHENABLE, MASKBYTE0, 0x11); + rtl_set_bbreg(hw, ROFDM1_TRXPATHENABLE, BDWORD, 0x1); + + /* enable ad/da clock1 for dual-phy reg0x888 */ + if (rtlhal->interfaceindex == 0) { + rtl_set_bbreg(hw, RFPGA0_ADDALLOCKEN, BIT(12) | + BIT(13), 0x3); + } else if (rtl92du_phy_enable_anotherphy(hw, false)) { + rtlhal->during_mac1init_radioa = true; + rtl_set_bbreg(hw, RFPGA0_ADDALLOCKEN, + BIT(12) | BIT(13), 0x3); + rtl92du_phy_powerdown_anotherphy(hw, false); + } + + rtl_set_bbreg(hw, ROFDM1_LSTF, BIT(19) | BIT(20), 0x0); + } else { + /* Single PHY */ + /* Use antenna 0 & 1, 0xc04, 0xd04 */ + rtl_set_bbreg(hw, ROFDM0_TRXPATHENABLE, MASKBYTE0, 0x33); + rtl_set_bbreg(hw, ROFDM1_TRXPATHENABLE, BDWORD, 0x3); + /* disable ad/da clock1,0x888 */ + rtl_set_bbreg(hw, RFPGA0_ADDALLOCKEN, BIT(12) | BIT(13), 0); + + rtl_set_bbreg(hw, ROFDM1_LSTF, BIT(19) | BIT(20), 0x1); + } + + for (rfpath = RF90_PATH_A; rfpath < rtlphy->num_total_rfpath; + rfpath++) { + rtlphy->rfreg_chnlval[rfpath] = rtl_get_rfreg(hw, rfpath, + RF_CHNLBW, + RFREG_OFFSET_MASK); + rtlphy->reg_rf3c[rfpath] = rtl_get_rfreg(hw, rfpath, 0x3C, + RFREG_OFFSET_MASK); + } + + for (i = 0; i < 2; i++) + rtl_dbg(rtlpriv, COMP_RF, DBG_LOUD, "RF 0x18 = 0x%x\n", + rtlphy->rfreg_chnlval[i]); + + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, "<==\n"); +} + +bool rtl92du_phy_check_poweroff(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtl_priv(hw)); + u8 u1btmp; + + if (rtlhal->macphymode == SINGLEMAC_SINGLEPHY) { + u1btmp = rtl_read_byte(rtlpriv, REG_MAC0); + rtl_write_byte(rtlpriv, REG_MAC0, u1btmp & ~MAC0_ON); + return true; + } + + mutex_lock(rtlpriv->mutex_for_power_on_off); + if (rtlhal->interfaceindex == 0) { + u1btmp = rtl_read_byte(rtlpriv, REG_MAC0); + rtl_write_byte(rtlpriv, REG_MAC0, u1btmp & ~MAC0_ON); + u1btmp = rtl_read_byte(rtlpriv, REG_MAC1); + u1btmp &= MAC1_ON; + } else { + u1btmp = rtl_read_byte(rtlpriv, REG_MAC1); + rtl_write_byte(rtlpriv, REG_MAC1, u1btmp & ~MAC1_ON); + u1btmp = rtl_read_byte(rtlpriv, REG_MAC0); + u1btmp &= MAC0_ON; + } + if (u1btmp) { + mutex_unlock(rtlpriv->mutex_for_power_on_off); + return false; + } + u1btmp = rtl_read_byte(rtlpriv, REG_POWER_OFF_IN_PROCESS); + u1btmp |= BIT(7); + rtl_write_byte(rtlpriv, REG_POWER_OFF_IN_PROCESS, u1btmp); + mutex_unlock(rtlpriv->mutex_for_power_on_off); + + return true; +} + +void rtl92du_phy_init_pa_bias(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + bool is_single_mac = rtlhal->macphymode == SINGLEMAC_SINGLEPHY; + enum radio_path rf_path; + u8 val8; + + read_efuse_byte(hw, 0x3FA, &val8); + + rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "%s: 0x3FA %#x\n", + __func__, val8); + + if (!(val8 & BIT(0)) && (is_single_mac || rtlhal->interfaceindex == 0)) { + rtl_set_rfreg(hw, RF90_PATH_A, RF_CHNLBW, RFREG_OFFSET_MASK, 0x07401); + rtl_set_rfreg(hw, RF90_PATH_A, RF_AC, RFREG_OFFSET_MASK, 0x70000); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x0F425); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x4F425); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x8F425); + + /* Back to RX Mode */ + rtl_set_rfreg(hw, RF90_PATH_A, RF_AC, RFREG_OFFSET_MASK, 0x30000); + + rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "2G PA BIAS path A\n"); + } + + if (!(val8 & BIT(1)) && (is_single_mac || rtlhal->interfaceindex == 1)) { + rf_path = rtlhal->interfaceindex == 1 ? RF90_PATH_A : RF90_PATH_B; + + rtl_set_rfreg(hw, rf_path, RF_CHNLBW, RFREG_OFFSET_MASK, 0x07401); + rtl_set_rfreg(hw, rf_path, RF_AC, RFREG_OFFSET_MASK, 0x70000); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x0F425); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x4F425); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x8F425); + + /* Back to RX Mode */ + rtl_set_rfreg(hw, rf_path, RF_AC, RFREG_OFFSET_MASK, 0x30000); + + rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "2G PA BIAS path B\n"); + } + + if (!(val8 & BIT(2)) && (is_single_mac || rtlhal->interfaceindex == 0)) { + /* 5GL_channel */ + rtl_set_rfreg(hw, RF90_PATH_A, RF_CHNLBW, RFREG_OFFSET_MASK, 0x17524); + rtl_set_rfreg(hw, RF90_PATH_A, RF_AC, RFREG_OFFSET_MASK, 0x70000); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x0F496); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x4F496); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x8F496); + + /* 5GM_channel */ + rtl_set_rfreg(hw, RF90_PATH_A, RF_CHNLBW, RFREG_OFFSET_MASK, 0x37564); + rtl_set_rfreg(hw, RF90_PATH_A, RF_AC, RFREG_OFFSET_MASK, 0x70000); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x0F496); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x4F496); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x8F496); + + /* 5GH_channel */ + rtl_set_rfreg(hw, RF90_PATH_A, RF_CHNLBW, RFREG_OFFSET_MASK, 0x57595); + rtl_set_rfreg(hw, RF90_PATH_A, RF_AC, RFREG_OFFSET_MASK, 0x70000); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x0F496); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x4F496); + rtl_set_rfreg(hw, RF90_PATH_A, RF_IPA, RFREG_OFFSET_MASK, 0x8F496); + + /* Back to RX Mode */ + rtl_set_rfreg(hw, RF90_PATH_A, RF_AC, RFREG_OFFSET_MASK, 0x30000); + + rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "5G PA BIAS path A\n"); + } + + if (!(val8 & BIT(3)) && (is_single_mac || rtlhal->interfaceindex == 1)) { + rf_path = rtlhal->interfaceindex == 1 ? RF90_PATH_A : RF90_PATH_B; + + /* 5GL_channel */ + rtl_set_rfreg(hw, rf_path, RF_CHNLBW, RFREG_OFFSET_MASK, 0x17524); + rtl_set_rfreg(hw, rf_path, RF_AC, RFREG_OFFSET_MASK, 0x70000); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x0F496); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x4F496); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x8F496); + + /* 5GM_channel */ + rtl_set_rfreg(hw, rf_path, RF_CHNLBW, RFREG_OFFSET_MASK, 0x37564); + rtl_set_rfreg(hw, rf_path, RF_AC, RFREG_OFFSET_MASK, 0x70000); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x0F496); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x4F496); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x8F496); + + /* 5GH_channel */ + rtl_set_rfreg(hw, rf_path, RF_CHNLBW, RFREG_OFFSET_MASK, 0x57595); + rtl_set_rfreg(hw, rf_path, RF_AC, RFREG_OFFSET_MASK, 0x70000); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x0F496); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x4F496); + rtl_set_rfreg(hw, rf_path, RF_IPA, RFREG_OFFSET_MASK, 0x8F496); + + /* Back to RX Mode */ + rtl_set_rfreg(hw, rf_path, RF_AC, RFREG_OFFSET_MASK, 0x30000); + + rtl_dbg(rtlpriv, COMP_RF, DBG_TRACE, "5G PA BIAS path B\n"); + } +} diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/phy.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/phy.h new file mode 100644 index 0000000000000000000000000000000000000000..090a6203db7ecde24f7680d24fda66abf8bed447 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/phy.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2024 Realtek Corporation.*/ + +#ifndef __RTL92DU_PHY_H__ +#define __RTL92DU_PHY_H__ + +u32 rtl92du_phy_query_bb_reg(struct ieee80211_hw *hw, + u32 regaddr, u32 bitmask); +void rtl92du_phy_set_bb_reg(struct ieee80211_hw *hw, + u32 regaddr, u32 bitmask, u32 data); +bool rtl92du_phy_mac_config(struct ieee80211_hw *hw); +bool rtl92du_phy_bb_config(struct ieee80211_hw *hw); +bool rtl92du_phy_rf_config(struct ieee80211_hw *hw); +void rtl92du_phy_set_bw_mode(struct ieee80211_hw *hw, + enum nl80211_channel_type ch_type); +u8 rtl92du_phy_sw_chnl(struct ieee80211_hw *hw); +bool rtl92du_phy_config_rf_with_headerfile(struct ieee80211_hw *hw, + enum rf_content content, + enum radio_path rfpath); +bool rtl92du_phy_set_rf_power_state(struct ieee80211_hw *hw, + enum rf_pwrstate rfpwr_state); + +void rtl92du_phy_set_poweron(struct ieee80211_hw *hw); +bool rtl92du_phy_check_poweroff(struct ieee80211_hw *hw); +void rtl92du_phy_lc_calibrate(struct ieee80211_hw *hw, bool is2t); +void rtl92du_update_bbrf_configuration(struct ieee80211_hw *hw); +void rtl92du_phy_ap_calibrate(struct ieee80211_hw *hw, s8 delta); +void rtl92du_phy_iq_calibrate(struct ieee80211_hw *hw); +void rtl92du_phy_reload_iqk_setting(struct ieee80211_hw *hw, u8 channel); +void rtl92du_phy_init_pa_bias(struct ieee80211_hw *hw); + +#endif diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/rf.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/rf.c new file mode 100644 index 0000000000000000000000000000000000000000..044dd65eafd04af51b5e604ce97e56577e7297fb --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/rf.c @@ -0,0 +1,240 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2024 Realtek Corporation.*/ + +#include "../wifi.h" +#include "../rtl8192d/reg.h" +#include "../rtl8192d/phy_common.h" +#include "phy.h" +#include "rf.h" + +bool rtl92du_phy_enable_anotherphy(struct ieee80211_hw *hw, bool bmac0) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + u8 mac_on_bit = bmac0 ? MAC1_ON : MAC0_ON; + u8 mac_reg = bmac0 ? REG_MAC1 : REG_MAC0; + bool bresult = true; /* true: need to enable BB/RF power */ + u32 maskforphyset = 0; + u16 val16; + u8 u1btmp; + + rtlhal->during_mac0init_radiob = false; + rtlhal->during_mac1init_radioa = false; + rtl_dbg(rtlpriv, COMP_RF, DBG_LOUD, "===>\n"); + + /* MAC0 Need PHY1 load radio_b.txt . Driver use DBI to write. */ + u1btmp = rtl_read_byte(rtlpriv, mac_reg); + if (!(u1btmp & mac_on_bit)) { + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, "enable BB & RF\n"); + /* Enable BB and RF power */ + + maskforphyset = bmac0 ? MAC0_ACCESS_PHY1 : MAC1_ACCESS_PHY0; + + val16 = rtl_read_word(rtlpriv, REG_SYS_FUNC_EN | maskforphyset); + val16 &= 0xfffc; + rtl_write_word(rtlpriv, REG_SYS_FUNC_EN | maskforphyset, val16); + + val16 = rtl_read_word(rtlpriv, REG_SYS_FUNC_EN | maskforphyset); + val16 |= BIT(13) | BIT(0) | BIT(1); + rtl_write_word(rtlpriv, REG_SYS_FUNC_EN | maskforphyset, val16); + } else { + /* We think if MAC1 is ON,then radio_a.txt + * and radio_b.txt has been load. + */ + bresult = false; + } + rtl_dbg(rtlpriv, COMP_RF, DBG_LOUD, "<===\n"); + return bresult; +} + +void rtl92du_phy_powerdown_anotherphy(struct ieee80211_hw *hw, bool bmac0) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + u8 mac_on_bit = bmac0 ? MAC1_ON : MAC0_ON; + u8 mac_reg = bmac0 ? REG_MAC1 : REG_MAC0; + u32 maskforphyset = 0; + u8 u1btmp; + + rtlhal->during_mac0init_radiob = false; + rtlhal->during_mac1init_radioa = false; + rtl_dbg(rtlpriv, COMP_RF, DBG_LOUD, "====>\n"); + + /* check MAC0 enable or not again now, if + * enabled, not power down radio A. + */ + u1btmp = rtl_read_byte(rtlpriv, mac_reg); + if (!(u1btmp & mac_on_bit)) { + rtl_dbg(rtlpriv, COMP_INIT, DBG_LOUD, "power down\n"); + /* power down RF radio A according to YuNan's advice. */ + maskforphyset = bmac0 ? MAC0_ACCESS_PHY1 : MAC1_ACCESS_PHY0; + rtl_write_dword(rtlpriv, RFPGA0_XA_LSSIPARAMETER | maskforphyset, + 0x00000000); + } + rtl_dbg(rtlpriv, COMP_RF, DBG_LOUD, "<====\n"); +} + +bool rtl92du_phy_rf6052_config(struct ieee80211_hw *hw) +{ + bool mac1_initradioa_first = false, mac0_initradiob_first = false; + bool need_pwrdown_radioa = false, need_pwrdown_radiob = false; + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = &rtlpriv->rtlhal; + struct rtl_phy *rtlphy = &rtlpriv->phy; + struct bb_reg_def *pphyreg; + bool true_bpath = false; + bool rtstatus = true; + u32 u4_regvalue = 0; + u8 rfpath; + + if (rtlphy->rf_type == RF_1T1R) + rtlphy->num_total_rfpath = 1; + else + rtlphy->num_total_rfpath = 2; + + /* Single phy mode: use radio_a radio_b config path_A path_B + * separately by MAC0, and MAC1 needn't configure RF; + * Dual PHY mode: MAC0 use radio_a config 1st phy path_A, + * MAC1 use radio_b config 2nd PHY path_A. + * DMDP, MAC0 on G band, MAC1 on A band. + */ + if (rtlhal->macphymode == DUALMAC_DUALPHY) { + if (rtlhal->current_bandtype == BAND_ON_2_4G && + rtlhal->interfaceindex == 0) { + /* MAC0 needs PHY1 load radio_b.txt. */ + if (rtl92du_phy_enable_anotherphy(hw, true)) { + rtlphy->num_total_rfpath = 2; + mac0_initradiob_first = true; + } else { + /* We think if MAC1 is ON,then radio_a.txt and + * radio_b.txt has been load. + */ + return rtstatus; + } + } else if (rtlhal->current_bandtype == BAND_ON_5G && + rtlhal->interfaceindex == 1) { + /* MAC1 needs PHY0 load radio_a.txt. */ + if (rtl92du_phy_enable_anotherphy(hw, false)) { + rtlphy->num_total_rfpath = 2; + mac1_initradioa_first = true; + } else { + /* We think if MAC0 is ON, then radio_a.txt and + * radio_b.txt has been load. + */ + return rtstatus; + } + } else if (rtlhal->interfaceindex == 1) { + /* MAC0 enabled, only init radia B. */ + true_bpath = true; + } + } + + for (rfpath = 0; rfpath < rtlphy->num_total_rfpath; rfpath++) { + /* Mac1 use PHY0 write */ + if (mac1_initradioa_first) { + if (rfpath == RF90_PATH_A) { + rtlhal->during_mac1init_radioa = true; + need_pwrdown_radioa = true; + } else if (rfpath == RF90_PATH_B) { + rtlhal->during_mac1init_radioa = false; + mac1_initradioa_first = false; + rfpath = RF90_PATH_A; + true_bpath = true; + rtlphy->num_total_rfpath = 1; + } + } else if (mac0_initradiob_first) { + /* Mac0 use PHY1 write */ + if (rfpath == RF90_PATH_A) + rtlhal->during_mac0init_radiob = false; + if (rfpath == RF90_PATH_B) { + rtlhal->during_mac0init_radiob = true; + mac0_initradiob_first = false; + need_pwrdown_radiob = true; + rfpath = RF90_PATH_A; + true_bpath = true; + rtlphy->num_total_rfpath = 1; + } + } + + pphyreg = &rtlphy->phyreg_def[rfpath]; + + switch (rfpath) { + case RF90_PATH_A: + case RF90_PATH_C: + u4_regvalue = rtl_get_bbreg(hw, pphyreg->rfintfs, + BRFSI_RFENV); + break; + case RF90_PATH_B: + case RF90_PATH_D: + u4_regvalue = rtl_get_bbreg(hw, pphyreg->rfintfs, + BRFSI_RFENV << 16); + break; + } + + rtl_set_bbreg(hw, pphyreg->rfintfe, BRFSI_RFENV << 16, 0x1); + udelay(1); + rtl_set_bbreg(hw, pphyreg->rfintfo, BRFSI_RFENV, 0x1); + udelay(1); + + /* Set bit number of Address and Data for RF register */ + rtl_set_bbreg(hw, pphyreg->rfhssi_para2, + B3WIREADDRESSLENGTH, 0x0); + udelay(1); + rtl_set_bbreg(hw, pphyreg->rfhssi_para2, B3WIREDATALENGTH, 0x0); + udelay(1); + + switch (rfpath) { + case RF90_PATH_A: + if (true_bpath) + rtstatus = rtl92du_phy_config_rf_with_headerfile( + hw, radiob_txt, + (enum radio_path)rfpath); + else + rtstatus = rtl92du_phy_config_rf_with_headerfile( + hw, radioa_txt, + (enum radio_path)rfpath); + break; + case RF90_PATH_B: + rtstatus = + rtl92du_phy_config_rf_with_headerfile(hw, radiob_txt, + (enum radio_path)rfpath); + break; + case RF90_PATH_C: + break; + case RF90_PATH_D: + break; + } + + switch (rfpath) { + case RF90_PATH_A: + case RF90_PATH_C: + rtl_set_bbreg(hw, pphyreg->rfintfs, BRFSI_RFENV, + u4_regvalue); + break; + case RF90_PATH_B: + case RF90_PATH_D: + rtl_set_bbreg(hw, pphyreg->rfintfs, BRFSI_RFENV << 16, + u4_regvalue); + break; + } + + if (!rtstatus) { + rtl_dbg(rtlpriv, COMP_INIT, DBG_TRACE, + "Radio[%d] Fail!!\n", rfpath); + return rtstatus; + } + } + + /* check MAC0 enable or not again, if enabled, + * not power down radio A. + * check MAC1 enable or not again, if enabled, + * not power down radio B. + */ + if (need_pwrdown_radioa) + rtl92du_phy_powerdown_anotherphy(hw, false); + else if (need_pwrdown_radiob) + rtl92du_phy_powerdown_anotherphy(hw, true); + rtl_dbg(rtlpriv, COMP_INIT, DBG_TRACE, "<---\n"); + + return rtstatus; +} diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/rf.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/rf.h new file mode 100644 index 0000000000000000000000000000000000000000..4a92cbdd00c0e0678a003621cbc05314f3850059 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/rf.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2024 Realtek Corporation.*/ + +#ifndef __RTL92DU_RF_H__ +#define __RTL92DU_RF_H__ + +bool rtl92du_phy_rf6052_config(struct ieee80211_hw *hw); +bool rtl92du_phy_enable_anotherphy(struct ieee80211_hw *hw, bool bmac0); +void rtl92du_phy_powerdown_anotherphy(struct ieee80211_hw *hw, bool bmac0); + +#endif diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/sw.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/sw.c new file mode 100644 index 0000000000000000000000000000000000000000..d069a81ac617521a8ad44b96b65743b358ede837 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/sw.c @@ -0,0 +1,395 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2024 Realtek Corporation.*/ + +#include "../wifi.h" +#include "../core.h" +#include "../usb.h" +#include "../base.h" +#include "../rtl8192d/reg.h" +#include "../rtl8192d/def.h" +#include "../rtl8192d/fw_common.h" +#include "../rtl8192d/hw_common.h" +#include "../rtl8192d/phy_common.h" +#include "../rtl8192d/trx_common.h" +#include "phy.h" +#include "dm.h" +#include "hw.h" +#include "trx.h" +#include "led.h" + +#include + +static struct usb_interface *rtl92du_get_other_intf(struct ieee80211_hw *hw) +{ + struct usb_interface *intf; + struct usb_device *udev; + u8 other_interfaceindex; + + /* See SET_IEEE80211_DEV(hw, &intf->dev); in usb.c */ + intf = container_of_const(wiphy_dev(hw->wiphy), struct usb_interface, dev); + + if (intf->altsetting[0].desc.bInterfaceNumber == 0) + other_interfaceindex = 1; + else + other_interfaceindex = 0; + + udev = interface_to_usbdev(intf); + + return usb_ifnum_to_if(udev, other_interfaceindex); +} + +static int rtl92du_init_shared_data(struct ieee80211_hw *hw) +{ + struct usb_interface *other_intf = rtl92du_get_other_intf(hw); + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_priv *other_rtlpriv = NULL; + struct ieee80211_hw *other_hw = NULL; + + if (other_intf) + other_hw = usb_get_intfdata(other_intf); + + if (other_hw) { + /* The other interface was already probed. */ + other_rtlpriv = rtl_priv(other_hw); + rtlpriv->curveindex_2g = other_rtlpriv->curveindex_2g; + rtlpriv->curveindex_5g = other_rtlpriv->curveindex_5g; + rtlpriv->mutex_for_power_on_off = other_rtlpriv->mutex_for_power_on_off; + rtlpriv->mutex_for_hw_init = other_rtlpriv->mutex_for_hw_init; + + if (!rtlpriv->curveindex_2g || !rtlpriv->curveindex_5g || + !rtlpriv->mutex_for_power_on_off || !rtlpriv->mutex_for_hw_init) + return -ENOMEM; + + return 0; + } + + /* The other interface doesn't exist or was not probed yet. */ + rtlpriv->curveindex_2g = kcalloc(TARGET_CHNL_NUM_2G, + sizeof(*rtlpriv->curveindex_2g), + GFP_KERNEL); + rtlpriv->curveindex_5g = kcalloc(TARGET_CHNL_NUM_5G, + sizeof(*rtlpriv->curveindex_5g), + GFP_KERNEL); + rtlpriv->mutex_for_power_on_off = + kzalloc(sizeof(*rtlpriv->mutex_for_power_on_off), GFP_KERNEL); + rtlpriv->mutex_for_hw_init = + kzalloc(sizeof(*rtlpriv->mutex_for_hw_init), GFP_KERNEL); + + if (!rtlpriv->curveindex_2g || !rtlpriv->curveindex_5g || + !rtlpriv->mutex_for_power_on_off || !rtlpriv->mutex_for_hw_init) { + kfree(rtlpriv->curveindex_2g); + kfree(rtlpriv->curveindex_5g); + kfree(rtlpriv->mutex_for_power_on_off); + kfree(rtlpriv->mutex_for_hw_init); + rtlpriv->curveindex_2g = NULL; + rtlpriv->curveindex_5g = NULL; + rtlpriv->mutex_for_power_on_off = NULL; + rtlpriv->mutex_for_hw_init = NULL; + return -ENOMEM; + } + + mutex_init(rtlpriv->mutex_for_power_on_off); + mutex_init(rtlpriv->mutex_for_hw_init); + + return 0; +} + +static void rtl92du_deinit_shared_data(struct ieee80211_hw *hw) +{ + struct usb_interface *other_intf = rtl92du_get_other_intf(hw); + struct rtl_priv *rtlpriv = rtl_priv(hw); + + if (!other_intf || !usb_get_intfdata(other_intf)) { + /* The other interface doesn't exist or was already disconnected. */ + kfree(rtlpriv->curveindex_2g); + kfree(rtlpriv->curveindex_5g); + if (rtlpriv->mutex_for_power_on_off) + mutex_destroy(rtlpriv->mutex_for_power_on_off); + if (rtlpriv->mutex_for_hw_init) + mutex_destroy(rtlpriv->mutex_for_hw_init); + kfree(rtlpriv->mutex_for_power_on_off); + kfree(rtlpriv->mutex_for_hw_init); + } +} + +static int rtl92du_init_sw_vars(struct ieee80211_hw *hw) +{ + const char *fw_name = "rtlwifi/rtl8192dufw.bin"; + struct rtl_priv *rtlpriv = rtl_priv(hw); + int err; + + err = rtl92du_init_shared_data(hw); + if (err) + return err; + + rtlpriv->dm.dm_initialgain_enable = true; + rtlpriv->dm.dm_flag = 0; + rtlpriv->dm.disable_framebursting = false; + rtlpriv->dm.thermalvalue = 0; + rtlpriv->dm.useramask = true; + + /* dual mac */ + if (rtlpriv->rtlhal.current_bandtype == BAND_ON_5G) + rtlpriv->phy.current_channel = 36; + else + rtlpriv->phy.current_channel = 1; + + if (rtlpriv->rtlhal.macphymode != SINGLEMAC_SINGLEPHY) + rtlpriv->rtlhal.disable_amsdu_8k = true; + + /* for LPS & IPS */ + rtlpriv->psc.inactiveps = rtlpriv->cfg->mod_params->inactiveps; + rtlpriv->psc.swctrl_lps = rtlpriv->cfg->mod_params->swctrl_lps; + rtlpriv->psc.fwctrl_lps = rtlpriv->cfg->mod_params->fwctrl_lps; + + /* for early mode */ + rtlpriv->rtlhal.earlymode_enable = false; + + /* for firmware buf */ + rtlpriv->rtlhal.pfirmware = kmalloc(0x8000, GFP_KERNEL); + if (!rtlpriv->rtlhal.pfirmware) + return -ENOMEM; + + rtlpriv->max_fw_size = 0x8000; + pr_info("Driver for Realtek RTL8192DU WLAN interface\n"); + pr_info("Loading firmware file %s\n", fw_name); + + /* request fw */ + err = request_firmware_nowait(THIS_MODULE, 1, fw_name, + rtlpriv->io.dev, GFP_KERNEL, hw, + rtl_fw_cb); + if (err) { + pr_err("Failed to request firmware!\n"); + kfree(rtlpriv->rtlhal.pfirmware); + rtlpriv->rtlhal.pfirmware = NULL; + return err; + } + + return 0; +} + +static void rtl92du_deinit_sw_vars(struct ieee80211_hw *hw) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + + kfree(rtlpriv->rtlhal.pfirmware); + rtlpriv->rtlhal.pfirmware = NULL; + + rtl92du_deinit_shared_data(hw); +} + +static const struct rtl_hal_ops rtl8192du_hal_ops = { + .init_sw_vars = rtl92du_init_sw_vars, + .deinit_sw_vars = rtl92du_deinit_sw_vars, + .read_chip_version = rtl92du_read_chip_version, + .read_eeprom_info = rtl92d_read_eeprom_info, + .hw_init = rtl92du_hw_init, + .hw_disable = rtl92du_card_disable, + .enable_interrupt = rtl92du_enable_interrupt, + .disable_interrupt = rtl92du_disable_interrupt, + .set_network_type = rtl92du_set_network_type, + .set_chk_bssid = rtl92du_set_check_bssid, + .set_qos = rtl92d_set_qos, + .set_bcn_reg = rtl92du_set_beacon_related_registers, + .set_bcn_intv = rtl92du_set_beacon_interval, + .update_interrupt_mask = rtl92du_update_interrupt_mask, + .get_hw_reg = rtl92du_get_hw_reg, + .set_hw_reg = rtl92du_set_hw_reg, + .update_rate_tbl = rtl92d_update_hal_rate_tbl, + .fill_tx_desc = rtl92du_tx_fill_desc, + .query_rx_desc = rtl92d_rx_query_desc, + .set_channel_access = rtl92d_update_channel_access_setting, + .radio_onoff_checking = rtl92d_gpio_radio_on_off_checking, + .set_bw_mode = rtl92du_phy_set_bw_mode, + .switch_channel = rtl92du_phy_sw_chnl, + .dm_watchdog = rtl92du_dm_watchdog, + .scan_operation_backup = rtl_phy_scan_operation_backup, + .set_rf_power_state = rtl92du_phy_set_rf_power_state, + .led_control = rtl92du_led_control, + .set_desc = rtl92d_set_desc, + .get_desc = rtl92d_get_desc, + .enable_hw_sec = rtl92d_enable_hw_security_config, + .set_key = rtl92d_set_key, + .get_bbreg = rtl92du_phy_query_bb_reg, + .set_bbreg = rtl92du_phy_set_bb_reg, + .get_rfreg = rtl92d_phy_query_rf_reg, + .set_rfreg = rtl92d_phy_set_rf_reg, + .linked_set_reg = rtl92du_linked_set_reg, + .fill_h2c_cmd = rtl92d_fill_h2c_cmd, + .get_btc_status = rtl_btc_status_false, + .phy_iq_calibrate = rtl92du_phy_iq_calibrate, + .phy_lc_calibrate = rtl92du_phy_lc_calibrate, +}; + +static struct rtl_mod_params rtl92du_mod_params = { + .sw_crypto = false, + .inactiveps = false, + .swctrl_lps = false, + .debug_level = 0, + .debug_mask = 0, +}; + +static const struct rtl_hal_usbint_cfg rtl92du_interface_cfg = { + /* rx */ + .rx_urb_num = 8, + .rx_max_size = 15360, + .usb_rx_hdl = NULL, + .usb_rx_segregate_hdl = NULL, + /* tx */ + .usb_tx_cleanup = rtl92du_tx_cleanup, + .usb_tx_post_hdl = rtl92du_tx_post_hdl, + .usb_tx_aggregate_hdl = rtl92du_tx_aggregate_hdl, + .usb_endpoint_mapping = rtl92du_endpoint_mapping, + .usb_mq_to_hwq = rtl92du_mq_to_hwq, +}; + +static const struct rtl_hal_cfg rtl92du_hal_cfg = { + .name = "rtl8192du", + .ops = &rtl8192du_hal_ops, + .mod_params = &rtl92du_mod_params, + .usb_interface_cfg = &rtl92du_interface_cfg, + + .maps[SYS_ISO_CTRL] = REG_SYS_ISO_CTRL, + .maps[SYS_FUNC_EN] = REG_SYS_FUNC_EN, + .maps[SYS_CLK] = REG_SYS_CLKR, + .maps[MAC_RCR_AM] = RCR_AM, + .maps[MAC_RCR_AB] = RCR_AB, + .maps[MAC_RCR_ACRC32] = RCR_ACRC32, + .maps[MAC_RCR_ACF] = RCR_ACF, + .maps[MAC_RCR_AAP] = RCR_AAP, + + .maps[EFUSE_TEST] = REG_EFUSE_TEST, + .maps[EFUSE_ACCESS] = REG_EFUSE_ACCESS, + .maps[EFUSE_CTRL] = REG_EFUSE_CTRL, + .maps[EFUSE_CLK] = 0, /* just for 92se */ + .maps[EFUSE_CLK_CTRL] = REG_EFUSE_CTRL, + .maps[EFUSE_PWC_EV12V] = PWC_EV12V, + .maps[EFUSE_FEN_ELDR] = FEN_ELDR, + .maps[EFUSE_LOADER_CLK_EN] = 0, + .maps[EFUSE_ANA8M] = 0, /* just for 92se */ + .maps[EFUSE_HWSET_MAX_SIZE] = HWSET_MAX_SIZE, + .maps[EFUSE_MAX_SECTION_MAP] = EFUSE_MAX_SECTION, + .maps[EFUSE_REAL_CONTENT_SIZE] = EFUSE_REAL_CONTENT_LEN, + + .maps[RWCAM] = REG_CAMCMD, + .maps[WCAMI] = REG_CAMWRITE, + .maps[RCAMO] = REG_CAMREAD, + .maps[CAMDBG] = REG_CAMDBG, + .maps[SECR] = REG_SECCFG, + .maps[SEC_CAM_NONE] = CAM_NONE, + .maps[SEC_CAM_WEP40] = CAM_WEP40, + .maps[SEC_CAM_TKIP] = CAM_TKIP, + .maps[SEC_CAM_AES] = CAM_AES, + .maps[SEC_CAM_WEP104] = CAM_WEP104, + + .maps[RTL_IMR_BCNDMAINT6] = IMR_BCNDMAINT6, + .maps[RTL_IMR_BCNDMAINT5] = IMR_BCNDMAINT5, + .maps[RTL_IMR_BCNDMAINT4] = IMR_BCNDMAINT4, + .maps[RTL_IMR_BCNDMAINT3] = IMR_BCNDMAINT3, + .maps[RTL_IMR_BCNDMAINT2] = IMR_BCNDMAINT2, + .maps[RTL_IMR_BCNDMAINT1] = IMR_BCNDMAINT1, + .maps[RTL_IMR_BCNDOK8] = IMR_BCNDOK8, + .maps[RTL_IMR_BCNDOK7] = IMR_BCNDOK7, + .maps[RTL_IMR_BCNDOK6] = IMR_BCNDOK6, + .maps[RTL_IMR_BCNDOK5] = IMR_BCNDOK5, + .maps[RTL_IMR_BCNDOK4] = IMR_BCNDOK4, + .maps[RTL_IMR_BCNDOK3] = IMR_BCNDOK3, + .maps[RTL_IMR_BCNDOK2] = IMR_BCNDOK2, + .maps[RTL_IMR_BCNDOK1] = IMR_BCNDOK1, + .maps[RTL_IMR_TIMEOUT2] = IMR_TIMEOUT2, + .maps[RTL_IMR_TIMEOUT1] = IMR_TIMEOUT1, + + .maps[RTL_IMR_TXFOVW] = IMR_TXFOVW, + .maps[RTL_IMR_PSTIMEOUT] = IMR_PSTIMEOUT, + .maps[RTL_IMR_BCNINT] = IMR_BCNINT, + .maps[RTL_IMR_RXFOVW] = IMR_RXFOVW, + .maps[RTL_IMR_RDU] = IMR_RDU, + .maps[RTL_IMR_ATIMEND] = IMR_ATIMEND, + .maps[RTL_IMR_BDOK] = IMR_BDOK, + .maps[RTL_IMR_MGNTDOK] = IMR_MGNTDOK, + .maps[RTL_IMR_TBDER] = IMR_TBDER, + .maps[RTL_IMR_HIGHDOK] = IMR_HIGHDOK, + .maps[RTL_IMR_TBDOK] = IMR_TBDOK, + .maps[RTL_IMR_BKDOK] = IMR_BKDOK, + .maps[RTL_IMR_BEDOK] = IMR_BEDOK, + .maps[RTL_IMR_VIDOK] = IMR_VIDOK, + .maps[RTL_IMR_VODOK] = IMR_VODOK, + .maps[RTL_IMR_ROK] = IMR_ROK, + .maps[RTL_IBSS_INT_MASKS] = (IMR_BCNINT | IMR_TBDOK | IMR_TBDER), + + .maps[RTL_RC_CCK_RATE1M] = DESC_RATE1M, + .maps[RTL_RC_CCK_RATE2M] = DESC_RATE2M, + .maps[RTL_RC_CCK_RATE5_5M] = DESC_RATE5_5M, + .maps[RTL_RC_CCK_RATE11M] = DESC_RATE11M, + .maps[RTL_RC_OFDM_RATE6M] = DESC_RATE6M, + .maps[RTL_RC_OFDM_RATE9M] = DESC_RATE9M, + .maps[RTL_RC_OFDM_RATE12M] = DESC_RATE12M, + .maps[RTL_RC_OFDM_RATE18M] = DESC_RATE18M, + .maps[RTL_RC_OFDM_RATE24M] = DESC_RATE24M, + .maps[RTL_RC_OFDM_RATE36M] = DESC_RATE36M, + .maps[RTL_RC_OFDM_RATE48M] = DESC_RATE48M, + .maps[RTL_RC_OFDM_RATE54M] = DESC_RATE54M, + + .maps[RTL_RC_HT_RATEMCS7] = DESC_RATEMCS7, + .maps[RTL_RC_HT_RATEMCS15] = DESC_RATEMCS15, +}; + +module_param_named(swenc, rtl92du_mod_params.sw_crypto, bool, 0444); +module_param_named(debug_level, rtl92du_mod_params.debug_level, int, 0644); +module_param_named(ips, rtl92du_mod_params.inactiveps, bool, 0444); +module_param_named(swlps, rtl92du_mod_params.swctrl_lps, bool, 0444); +module_param_named(debug_mask, rtl92du_mod_params.debug_mask, ullong, 0644); +MODULE_PARM_DESC(swenc, "Set to 1 for software crypto (default 0)\n"); +MODULE_PARM_DESC(ips, "Set to 0 to not use link power save (default 0)\n"); +MODULE_PARM_DESC(swlps, "Set to 1 to use SW control power save (default 0)\n"); +MODULE_PARM_DESC(debug_level, "Set debug level (0-5) (default 0)"); +MODULE_PARM_DESC(debug_mask, "Set debug mask (default 0)"); + +#define USB_VENDOR_ID_REALTEK 0x0bda + +static const struct usb_device_id rtl8192d_usb_ids[] = { + {RTL_USB_DEVICE(USB_VENDOR_ID_REALTEK, 0x8193, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(USB_VENDOR_ID_REALTEK, 0x8194, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(USB_VENDOR_ID_REALTEK, 0x8111, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(USB_VENDOR_ID_REALTEK, 0x0193, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(USB_VENDOR_ID_REALTEK, 0x8171, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(USB_VENDOR_ID_REALTEK, 0xe194, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x2019, 0xab2c, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x2019, 0xab2d, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x2019, 0x4903, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x2019, 0x4904, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x07b8, 0x8193, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x20f4, 0x664b, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x04dd, 0x954f, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x04dd, 0x96a6, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x050d, 0x110a, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x050d, 0x1105, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x050d, 0x120a, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x1668, 0x8102, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x0930, 0x0a0a, rtl92du_hal_cfg)}, + {RTL_USB_DEVICE(0x2001, 0x330c, rtl92du_hal_cfg)}, + {} +}; + +MODULE_DEVICE_TABLE(usb, rtl8192d_usb_ids); + +static int rtl8192du_probe(struct usb_interface *intf, + const struct usb_device_id *id) +{ + return rtl_usb_probe(intf, id, &rtl92du_hal_cfg); +} + +static struct usb_driver rtl8192du_driver = { + .name = "rtl8192du", + .probe = rtl8192du_probe, + .disconnect = rtl_usb_disconnect, + .id_table = rtl8192d_usb_ids, + .disable_hub_initiated_lpm = 1, +}; + +module_usb_driver(rtl8192du_driver); + +MODULE_AUTHOR("Bitterblue Smith "); +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Realtek 8192DU 802.11n Dual Mac USB wireless"); +MODULE_FIRMWARE("rtlwifi/rtl8192dufw.bin"); diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/table.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/table.c new file mode 100644 index 0000000000000000000000000000000000000000..036701433d853d2f182c20a28e066ea018e86a68 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/table.c @@ -0,0 +1,1675 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2024 Realtek Corporation.*/ + +#include + +#include "table.h" + +const u32 rtl8192du_phy_reg_2tarray[PHY_REG_2T_ARRAYLENGTH] = { + 0x800, 0x80040002, + 0x804, 0x00000003, + 0x808, 0x0000fc00, + 0x80c, 0x0000000a, + 0x810, 0x10001331, + 0x814, 0x020c3d10, + 0x818, 0x02200385, + 0x81c, 0x00000000, + 0x820, 0x01000100, + 0x824, 0x00390004, + 0x828, 0x01000100, + 0x82c, 0x00390004, + 0x830, 0x27272727, + 0x834, 0x27272727, + 0x838, 0x27272727, + 0x83c, 0x27272727, + 0x840, 0x00010000, + 0x844, 0x00010000, + 0x848, 0x27272727, + 0x84c, 0x27272727, + 0x850, 0x00000000, + 0x854, 0x00000000, + 0x858, 0x569a569a, + 0x85c, 0x0c1b25a4, + 0x860, 0x66e60250, + 0x864, 0x061f0150, + 0x868, 0x27272727, + 0x86c, 0x272b2b2b, + 0x870, 0x07000700, + 0x874, 0x22188000, + 0x878, 0x08080808, + 0x87c, 0x0001fff8, + 0x880, 0xc0083070, + 0x884, 0x00000cd5, + 0x888, 0x00000000, + 0x88c, 0xcc0000c0, + 0x890, 0x00000800, + 0x894, 0xfffffffe, + 0x898, 0x40302010, + 0x89c, 0x00706050, + 0x900, 0x00000000, + 0x904, 0x00000023, + 0x908, 0x00000000, + 0x90c, 0x81121313, + 0xa00, 0x00d047c8, + 0xa04, 0x80ff000c, + 0xa08, 0x8c8a8300, + 0xa0c, 0x2e68120f, + 0xa10, 0x9500bb78, + 0xa14, 0x11144028, + 0xa18, 0x00881117, + 0xa1c, 0x89140f00, + 0xa20, 0x1a1b0000, + 0xa24, 0x090e1317, + 0xa28, 0x00000204, + 0xa2c, 0x00d30000, + 0xa70, 0x101fff00, + 0xa74, 0x00000007, + 0xc00, 0x40071d40, + 0xc04, 0x03a05633, + 0xc08, 0x001000e4, + 0xc0c, 0x6c6c6c6c, + 0xc10, 0x08800000, + 0xc14, 0x40000100, + 0xc18, 0x08800000, + 0xc1c, 0x40000100, + 0xc20, 0x00000000, + 0xc24, 0x00000000, + 0xc28, 0x00000000, + 0xc2c, 0x00000000, + 0xc30, 0x69e9ac44, + 0xc34, 0x469652af, + 0xc38, 0x49795994, + 0xc3c, 0x0a979718, + 0xc40, 0x1f7c403f, + 0xc44, 0x000100b7, + 0xc48, 0xec020107, + 0xc4c, 0x007f037f, + 0xc50, 0x69543420, + 0xc54, 0x43bc009e, + 0xc58, 0x69543420, + 0xc5c, 0x433c00a8, + 0xc60, 0x00000000, + 0xc64, 0x7112848b, + 0xc68, 0x47c00bff, + 0xc6c, 0x00000036, + 0xc70, 0x2c7f000d, + 0xc74, 0x258610db, + 0xc78, 0x0000001f, + 0xc7c, 0x40b95612, + 0xc80, 0x40000100, + 0xc84, 0x20f60000, + 0xc88, 0x40000100, + 0xc8c, 0xa0e40000, + 0xc90, 0x00121820, + 0xc94, 0x00000007, + 0xc98, 0x00121820, + 0xc9c, 0x00007f7f, + 0xca0, 0x00000000, + 0xca4, 0x00000080, + 0xca8, 0x00000000, + 0xcac, 0x00000000, + 0xcb0, 0x00000000, + 0xcb4, 0x00000000, + 0xcb8, 0x00000000, + 0xcbc, 0x28000000, + 0xcc0, 0x00000000, + 0xcc4, 0x00000000, + 0xcc8, 0x00000000, + 0xccc, 0x00000000, + 0xcd0, 0x00000000, + 0xcd4, 0x00000000, + 0xcd8, 0x64b11e20, + 0xcdc, 0xe0767533, + 0xce0, 0x00222222, + 0xce4, 0x00000000, + 0xce8, 0x37644302, + 0xcec, 0x2f97d40c, + 0xd00, 0x00080740, + 0xd04, 0x00020403, + 0xd08, 0x0000907f, + 0xd0c, 0x20010201, + 0xd10, 0xa0633333, + 0xd14, 0x3333bc43, + 0xd18, 0x7a8f5b6b, + 0xd2c, 0xcc979975, + 0xd30, 0x00000000, + 0xd34, 0x80608404, + 0xd38, 0x00000000, + 0xd3c, 0x00027353, + 0xd40, 0x00000000, + 0xd44, 0x00000000, + 0xd48, 0x00000000, + 0xd4c, 0x00000000, + 0xd50, 0x6437140a, + 0xd54, 0x00000000, + 0xd58, 0x00000000, + 0xd5c, 0x30032064, + 0xd60, 0x4653de68, + 0xd64, 0x04518a3c, + 0xd68, 0x00002101, + 0xd6c, 0x2a201c16, + 0xd70, 0x1812362e, + 0xd74, 0x322c2220, + 0xd78, 0x000e3c24, + 0xe00, 0x2a2a2a2a, + 0xe04, 0x2a2a2a2a, + 0xe08, 0x03902a2a, + 0xe10, 0x2a2a2a2a, + 0xe14, 0x2a2a2a2a, + 0xe18, 0x2a2a2a2a, + 0xe1c, 0x2a2a2a2a, + 0xe28, 0x00000000, + 0xe30, 0x1000dc1f, + 0xe34, 0x10008c1f, + 0xe38, 0x02140102, + 0xe3c, 0x681604c2, + 0xe40, 0x01007c00, + 0xe44, 0x01004800, + 0xe48, 0xfb000000, + 0xe4c, 0x000028d1, + 0xe50, 0x1000dc1f, + 0xe54, 0x10008c1f, + 0xe58, 0x02140102, + 0xe5c, 0x28160d05, + 0xe60, 0x00000010, + 0xe68, 0x001b25a4, + 0xe6c, 0x63db25a4, + 0xe70, 0x63db25a4, + 0xe74, 0x0c126da4, + 0xe78, 0x0c126da4, + 0xe7c, 0x0c126da4, + 0xe80, 0x0c126da4, + 0xe84, 0x63db25a4, + 0xe88, 0x0c126da4, + 0xe8c, 0x63db25a4, + 0xed0, 0x63db25a4, + 0xed4, 0x63db25a4, + 0xed8, 0x63db25a4, + 0xedc, 0x001b25a4, + 0xee0, 0x001b25a4, + 0xeec, 0x6fdb25a4, + 0xf14, 0x00000003, + 0xf1c, 0x00000064, + 0xf4c, 0x00000004, + 0xf00, 0x00000300, +}; + +const u32 rtl8192du_phy_reg_array_pg[PHY_REG_ARRAY_PG_LENGTH] = { + 0xe00, 0xffffffff, 0x07090c0c, + 0xe04, 0xffffffff, 0x01020405, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x0b0c0c0e, + 0xe14, 0xffffffff, 0x01030506, + 0xe18, 0xffffffff, 0x0b0c0d0e, + 0xe1c, 0xffffffff, 0x01030509, + 0x830, 0xffffffff, 0x07090c0c, + 0x834, 0xffffffff, 0x01020405, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x0b0c0c0e, + 0x848, 0xffffffff, 0x01030506, + 0x84c, 0xffffffff, 0x0b0c0d0e, + 0x868, 0xffffffff, 0x01030509, + 0xe00, 0xffffffff, 0x00000000, + 0xe04, 0xffffffff, 0x00000000, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x00000000, + 0xe14, 0xffffffff, 0x00000000, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x00000000, + 0x834, 0xffffffff, 0x00000000, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x00000000, + 0x848, 0xffffffff, 0x00000000, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, + 0xe00, 0xffffffff, 0x04040404, + 0xe04, 0xffffffff, 0x00020204, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x06060606, + 0xe14, 0xffffffff, 0x00020406, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x04040404, + 0x834, 0xffffffff, 0x00020204, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x06060606, + 0x848, 0xffffffff, 0x00020406, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, + 0xe00, 0xffffffff, 0x00000000, + 0xe04, 0xffffffff, 0x00000000, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x00000000, + 0xe14, 0xffffffff, 0x00000000, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x00000000, + 0x834, 0xffffffff, 0x00000000, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x00000000, + 0x848, 0xffffffff, 0x00000000, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, + 0xe00, 0xffffffff, 0x00000000, + 0xe04, 0xffffffff, 0x00000000, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x00000000, + 0xe14, 0xffffffff, 0x00000000, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x00000000, + 0x834, 0xffffffff, 0x00000000, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x00000000, + 0x848, 0xffffffff, 0x00000000, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, + 0xe00, 0xffffffff, 0x04040404, + 0xe04, 0xffffffff, 0x00020204, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x00000000, + 0xe14, 0xffffffff, 0x00000000, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x04040404, + 0x834, 0xffffffff, 0x00020204, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x00000000, + 0x848, 0xffffffff, 0x00000000, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, + 0xe00, 0xffffffff, 0x00000000, + 0xe04, 0xffffffff, 0x00000000, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x00000000, + 0xe14, 0xffffffff, 0x00000000, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x00000000, + 0x834, 0xffffffff, 0x00000000, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x00000000, + 0x848, 0xffffffff, 0x00000000, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, + 0xe00, 0xffffffff, 0x04040404, + 0xe04, 0xffffffff, 0x00020204, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x08080808, + 0xe14, 0xffffffff, 0x00040408, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x04040404, + 0x834, 0xffffffff, 0x00020204, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x08080808, + 0x848, 0xffffffff, 0x00040408, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, + 0xe00, 0xffffffff, 0x04040404, + 0xe04, 0xffffffff, 0x00020204, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x08080808, + 0xe14, 0xffffffff, 0x00040408, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x04040404, + 0x834, 0xffffffff, 0x00020204, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x08080808, + 0x848, 0xffffffff, 0x00040408, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, + 0xe00, 0xffffffff, 0x04040404, + 0xe04, 0xffffffff, 0x00020204, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x08080808, + 0xe14, 0xffffffff, 0x00040408, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x04040404, + 0x834, 0xffffffff, 0x00020204, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x08080808, + 0x848, 0xffffffff, 0x00040408, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, + 0xe00, 0xffffffff, 0x04040404, + 0xe04, 0xffffffff, 0x00020204, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x08080808, + 0xe14, 0xffffffff, 0x00040408, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x04040404, + 0x834, 0xffffffff, 0x00020204, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x08080808, + 0x848, 0xffffffff, 0x00040408, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, + 0xe00, 0xffffffff, 0x04040404, + 0xe04, 0xffffffff, 0x00020204, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x08080808, + 0xe14, 0xffffffff, 0x00040408, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x04040404, + 0x834, 0xffffffff, 0x00020204, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x08080808, + 0x848, 0xffffffff, 0x00040408, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, + 0xe00, 0xffffffff, 0x04040404, + 0xe04, 0xffffffff, 0x00020204, + 0xe08, 0x0000ff00, 0x00000000, + 0x86c, 0xffffff00, 0x00000000, + 0xe10, 0xffffffff, 0x08080808, + 0xe14, 0xffffffff, 0x00040408, + 0xe18, 0xffffffff, 0x00000000, + 0xe1c, 0xffffffff, 0x00000000, + 0x830, 0xffffffff, 0x04040404, + 0x834, 0xffffffff, 0x00020204, + 0x838, 0xffffff00, 0x00000000, + 0x86c, 0x000000ff, 0x00000000, + 0x83c, 0xffffffff, 0x08080808, + 0x848, 0xffffffff, 0x00040408, + 0x84c, 0xffffffff, 0x00000000, + 0x868, 0xffffffff, 0x00000000, +}; + +const u32 rtl8192du_radioa_2tarray[RADIOA_2T_ARRAYLENGTH] = { + 0x000, 0x00030000, + 0x001, 0x00030000, + 0x002, 0x00000000, + 0x003, 0x00018c63, + 0x004, 0x00018c63, + 0x008, 0x00084000, + 0x00b, 0x0001c000, + 0x00e, 0x00018c67, + 0x00f, 0x00000851, + 0x014, 0x00021440, + 0x018, 0x00017524, + 0x019, 0x00000000, + 0x01d, 0x000a1290, + 0x023, 0x00001558, + 0x01a, 0x00030a99, + 0x01b, 0x00040b00, + 0x01c, 0x000fc339, + 0x03a, 0x000a57eb, + 0x03b, 0x00020000, + 0x03c, 0x000ff454, + 0x020, 0x0000aa52, + 0x021, 0x00054000, + 0x040, 0x0000aa52, + 0x041, 0x00014000, + 0x025, 0x000803be, + 0x026, 0x000fc638, + 0x027, 0x00077c18, + 0x028, 0x000de471, + 0x029, 0x000d7110, + 0x02a, 0x0008cb04, + 0x02b, 0x0004128b, + 0x02c, 0x00001840, + 0x043, 0x0002444f, + 0x044, 0x0001adb0, + 0x045, 0x00056467, + 0x046, 0x0008992c, + 0x047, 0x0000452c, + 0x048, 0x000f9c43, + 0x049, 0x00002e0c, + 0x04a, 0x000546eb, + 0x04b, 0x0008966c, + 0x04c, 0x0000dde9, + 0x018, 0x00007401, + 0x000, 0x00070000, + 0x012, 0x000dc000, + 0x012, 0x00090000, + 0x012, 0x00051000, + 0x012, 0x00012000, + 0x013, 0x000287b7, + 0x013, 0x000247ab, + 0x013, 0x0002079f, + 0x013, 0x0001c793, + 0x013, 0x0001839b, + 0x013, 0x00014392, + 0x013, 0x0001019a, + 0x013, 0x0000c191, + 0x013, 0x00008194, + 0x013, 0x000040a0, + 0x013, 0x00000018, + 0x015, 0x0000f424, + 0x015, 0x0004f424, + 0x015, 0x0008f424, + 0x016, 0x000e1330, + 0x016, 0x000a1330, + 0x016, 0x00061330, + 0x016, 0x00021330, + 0x018, 0x00017524, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bc, + 0x013, 0x000247b0, + 0x013, 0x000203b4, + 0x013, 0x0001c3a8, + 0x013, 0x000181b4, + 0x013, 0x000141a8, + 0x013, 0x000100b4, + 0x013, 0x0000c0a8, + 0x013, 0x0000b030, + 0x013, 0x00004024, + 0x013, 0x00000018, + 0x015, 0x0000f4c3, + 0x015, 0x0004f4c3, + 0x015, 0x0008f4c3, + 0x016, 0x000e085f, + 0x016, 0x000a085f, + 0x016, 0x0006085f, + 0x016, 0x0002085f, + 0x018, 0x00037524, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bc, + 0x013, 0x000247b0, + 0x013, 0x000203b4, + 0x013, 0x0001c3a8, + 0x013, 0x000181b4, + 0x013, 0x000141a8, + 0x013, 0x000100b4, + 0x013, 0x0000c0a8, + 0x013, 0x0000b030, + 0x013, 0x00004024, + 0x013, 0x00000018, + 0x015, 0x0000f4c3, + 0x015, 0x0004f4c3, + 0x015, 0x0008f4c3, + 0x016, 0x000e085f, + 0x016, 0x000a085f, + 0x016, 0x0006085f, + 0x016, 0x0002085f, + 0x018, 0x00057568, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bc, + 0x013, 0x000247b0, + 0x013, 0x000203b4, + 0x013, 0x0001c3a8, + 0x013, 0x000181b4, + 0x013, 0x000141a8, + 0x013, 0x000100b4, + 0x013, 0x0000c0a8, + 0x013, 0x0000b030, + 0x013, 0x00004024, + 0x013, 0x00000018, + 0x015, 0x0000f4c3, + 0x015, 0x0004f4c3, + 0x015, 0x0008f4c3, + 0x016, 0x000e085f, + 0x016, 0x000a085f, + 0x016, 0x0006085f, + 0x016, 0x0002085f, + 0x030, 0x0004470f, + 0x031, 0x00044ff0, + 0x032, 0x00000070, + 0x033, 0x000dd480, + 0x034, 0x000ffac0, + 0x035, 0x000b80c0, + 0x036, 0x00077000, + 0x037, 0x00064ff2, + 0x038, 0x000e7661, + 0x039, 0x00000e90, + 0x000, 0x00030000, + 0x018, 0x0000f401, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x01e, 0x00088009, + 0x01f, 0x00080003, + 0x0fe, 0x00000000, + 0x01e, 0x00088001, + 0x01f, 0x00080000, + 0x0fe, 0x00000000, + 0x018, 0x00097524, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x02b, 0x00041289, + 0x0fe, 0x00000000, + 0x02d, 0x0006aaaa, + 0x02e, 0x000b4d01, + 0x02d, 0x00080000, + 0x02e, 0x00004d02, + 0x02d, 0x00095555, + 0x02e, 0x00054d03, + 0x02d, 0x000aaaaa, + 0x02e, 0x000b4d04, + 0x02d, 0x000c0000, + 0x02e, 0x00004d05, + 0x02d, 0x000d5555, + 0x02e, 0x00054d06, + 0x02d, 0x000eaaaa, + 0x02e, 0x000b4d07, + 0x02d, 0x00000000, + 0x02e, 0x00005108, + 0x02d, 0x00015555, + 0x02e, 0x00055109, + 0x02d, 0x0002aaaa, + 0x02e, 0x000b510a, + 0x02d, 0x00040000, + 0x02e, 0x0000510b, + 0x02d, 0x00055555, + 0x02e, 0x0005510c, +}; + +const u32 rtl8192du_radiob_2tarray[RADIOB_2T_ARRAYLENGTH] = { + 0x000, 0x00030000, + 0x001, 0x00030000, + 0x002, 0x00000000, + 0x003, 0x00018c63, + 0x004, 0x00018c63, + 0x008, 0x00084000, + 0x00b, 0x0001c000, + 0x00e, 0x00018c67, + 0x00f, 0x00000851, + 0x014, 0x00021440, + 0x018, 0x00007401, + 0x019, 0x00000060, + 0x01d, 0x000a1290, + 0x023, 0x00001558, + 0x01a, 0x00030a99, + 0x01b, 0x00040b00, + 0x01c, 0x000fc339, + 0x03a, 0x000a57eb, + 0x03b, 0x00020000, + 0x03c, 0x000ff454, + 0x020, 0x0000aa52, + 0x021, 0x00054000, + 0x040, 0x0000aa52, + 0x041, 0x00014000, + 0x025, 0x000803be, + 0x026, 0x000fc638, + 0x027, 0x00077c18, + 0x028, 0x000d1c31, + 0x029, 0x000d7110, + 0x02a, 0x000aeb04, + 0x02b, 0x0004128b, + 0x02c, 0x00001840, + 0x043, 0x0002444f, + 0x044, 0x0001adb0, + 0x045, 0x00056467, + 0x046, 0x0008992c, + 0x047, 0x0000452c, + 0x048, 0x000f9c43, + 0x049, 0x00002e0c, + 0x04a, 0x000546eb, + 0x04b, 0x0008966c, + 0x04c, 0x0000dde9, + 0x018, 0x00007401, + 0x000, 0x00070000, + 0x012, 0x000dc000, + 0x012, 0x00090000, + 0x012, 0x00051000, + 0x012, 0x00012000, + 0x013, 0x000287b7, + 0x013, 0x000247ab, + 0x013, 0x0002079f, + 0x013, 0x0001c793, + 0x013, 0x0001839b, + 0x013, 0x00014392, + 0x013, 0x0001019a, + 0x013, 0x0000c191, + 0x013, 0x00008194, + 0x013, 0x000040a0, + 0x013, 0x00000018, + 0x015, 0x0000f424, + 0x015, 0x0004f424, + 0x015, 0x0008f424, + 0x016, 0x000e1330, + 0x016, 0x000a1330, + 0x016, 0x00061330, + 0x016, 0x00021330, + 0x018, 0x00017524, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bc, + 0x013, 0x000247b0, + 0x013, 0x000203b4, + 0x013, 0x0001c3a8, + 0x013, 0x000181b4, + 0x013, 0x000141a8, + 0x013, 0x000100b4, + 0x013, 0x0000c0a8, + 0x013, 0x0000b030, + 0x013, 0x00004024, + 0x013, 0x00000018, + 0x015, 0x0000f4c3, + 0x015, 0x0004f4c3, + 0x015, 0x0008f4c3, + 0x016, 0x000e085f, + 0x016, 0x000a085f, + 0x016, 0x0006085f, + 0x016, 0x0002085f, + 0x018, 0x00037524, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bc, + 0x013, 0x000247b0, + 0x013, 0x000203b4, + 0x013, 0x0001c3a8, + 0x013, 0x000181b4, + 0x013, 0x000141a8, + 0x013, 0x000100b4, + 0x013, 0x0000c0a8, + 0x013, 0x0000b030, + 0x013, 0x00004024, + 0x013, 0x00000018, + 0x015, 0x0000f4c3, + 0x015, 0x0004f4c3, + 0x015, 0x0008f4c3, + 0x016, 0x000e085f, + 0x016, 0x000a085f, + 0x016, 0x0006085f, + 0x016, 0x0002085f, + 0x018, 0x00057524, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bc, + 0x013, 0x000247b0, + 0x013, 0x000203b4, + 0x013, 0x0001c3a8, + 0x013, 0x000181b4, + 0x013, 0x000141a8, + 0x013, 0x000100b4, + 0x013, 0x0000c0a8, + 0x013, 0x0000b030, + 0x013, 0x00004024, + 0x013, 0x00000018, + 0x015, 0x0000f4c3, + 0x015, 0x0004f4c3, + 0x015, 0x0008f4c3, + 0x016, 0x000e085f, + 0x016, 0x000a085f, + 0x016, 0x0006085f, + 0x016, 0x0002085f, + 0x030, 0x0004470f, + 0x031, 0x00044ff0, + 0x032, 0x00000070, + 0x033, 0x000dd480, + 0x034, 0x000ffac0, + 0x035, 0x000b80c0, + 0x036, 0x00077000, + 0x037, 0x00064ff2, + 0x038, 0x000e7661, + 0x039, 0x00000e90, + 0x000, 0x00030000, + 0x018, 0x0000f401, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x01e, 0x00088009, + 0x01f, 0x00080003, + 0x0fe, 0x00000000, + 0x01e, 0x00088001, + 0x01f, 0x00080000, + 0x0fe, 0x00000000, + 0x018, 0x00087401, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x02b, 0x00041289, + 0x0fe, 0x00000000, + 0x02d, 0x00066666, + 0x02e, 0x00064001, + 0x02d, 0x00091111, + 0x02e, 0x00014002, + 0x02d, 0x000bbbbb, + 0x02e, 0x000b4003, + 0x02d, 0x000e6666, + 0x02e, 0x00064004, + 0x02d, 0x00088888, + 0x02e, 0x00084005, + 0x02d, 0x0009dddd, + 0x02e, 0x000d4006, + 0x02d, 0x000b3333, + 0x02e, 0x00034007, + 0x02d, 0x00048888, + 0x02e, 0x00084408, + 0x02d, 0x000bbbbb, + 0x02e, 0x000b4409, + 0x02d, 0x000e6666, + 0x02e, 0x0006440a, + 0x02d, 0x00011111, + 0x02e, 0x0001480b, + 0x02d, 0x0003bbbb, + 0x02e, 0x000b480c, + 0x02d, 0x00066666, + 0x02e, 0x0006480d, + 0x02d, 0x000ccccc, + 0x02e, 0x000c480e, +}; + +const u32 rtl8192du_radioa_2t_int_paarray[RADIOA_2T_INT_PA_ARRAYLENGTH] = { + 0x000, 0x00030000, + 0x001, 0x00030000, + 0x002, 0x00000000, + 0x003, 0x00018c63, + 0x004, 0x00018c63, + 0x008, 0x00084000, + 0x00b, 0x0001c000, + 0x00e, 0x00018c67, + 0x00f, 0x00000851, + 0x014, 0x00021440, + 0x018, 0x00017524, + 0x019, 0x00000000, + 0x01d, 0x000a1290, + 0x023, 0x00001558, + 0x01a, 0x00030a99, + 0x01b, 0x00040b00, + 0x01c, 0x000fc339, + 0x03a, 0x000a57eb, + 0x03b, 0x00020000, + 0x03c, 0x000ff455, + 0x020, 0x0000aa52, + 0x021, 0x00054000, + 0x040, 0x0000aa52, + 0x041, 0x00014000, + 0x025, 0x000803be, + 0x026, 0x000fc638, + 0x027, 0x00077c18, + 0x028, 0x000de471, + 0x029, 0x000d7110, + 0x02a, 0x0008eb04, + 0x02b, 0x0004128b, + 0x02c, 0x00001840, + 0x043, 0x0002444f, + 0x044, 0x0001adb0, + 0x045, 0x00056467, + 0x046, 0x0008992c, + 0x047, 0x0000452c, + 0x048, 0x000c0443, + 0x049, 0x00000730, + 0x04a, 0x00050f0f, + 0x04b, 0x000896ef, + 0x04c, 0x0000ddee, + 0x018, 0x00007401, + 0x000, 0x00070000, + 0x012, 0x000dc000, + 0x012, 0x00090000, + 0x012, 0x00051000, + 0x012, 0x00012000, + 0x013, 0x000287b7, + 0x013, 0x000247ab, + 0x013, 0x0002079f, + 0x013, 0x0001c793, + 0x013, 0x0001839b, + 0x013, 0x00014392, + 0x013, 0x0001019a, + 0x013, 0x0000c191, + 0x013, 0x00008194, + 0x013, 0x000040a0, + 0x013, 0x00000018, + 0x015, 0x0000f424, + 0x015, 0x0004f424, + 0x015, 0x0008f424, + 0x016, 0x000e1330, + 0x016, 0x000a1330, + 0x016, 0x00061330, + 0x016, 0x00021330, + 0x018, 0x00017524, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bf, + 0x013, 0x000247b3, + 0x013, 0x000207a7, + 0x013, 0x0001c79b, + 0x013, 0x0001839f, + 0x013, 0x00014393, + 0x013, 0x00010399, + 0x013, 0x0000c38d, + 0x013, 0x00008199, + 0x013, 0x0000418d, + 0x013, 0x00000099, + 0x015, 0x0000f495, + 0x015, 0x0004f495, + 0x015, 0x0008f495, + 0x016, 0x000e1874, + 0x016, 0x000a1874, + 0x016, 0x00061874, + 0x016, 0x00021874, + 0x018, 0x00037564, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bf, + 0x013, 0x000247b3, + 0x013, 0x000207a7, + 0x013, 0x0001c79b, + 0x013, 0x0001839f, + 0x013, 0x00014393, + 0x013, 0x00010399, + 0x013, 0x0000c38d, + 0x013, 0x00008199, + 0x013, 0x0000418d, + 0x013, 0x00000099, + 0x015, 0x0000f495, + 0x015, 0x0004f495, + 0x015, 0x0008f495, + 0x016, 0x000e1874, + 0x016, 0x000a1874, + 0x016, 0x00061874, + 0x016, 0x00021874, + 0x018, 0x00057595, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bf, + 0x013, 0x000247b3, + 0x013, 0x000207a7, + 0x013, 0x0001c79b, + 0x013, 0x0001839f, + 0x013, 0x00014393, + 0x013, 0x00010399, + 0x013, 0x0000c38d, + 0x013, 0x00008199, + 0x013, 0x0000418d, + 0x013, 0x00000099, + 0x015, 0x0000f495, + 0x015, 0x0004f495, + 0x015, 0x0008f495, + 0x016, 0x000e1874, + 0x016, 0x000a1874, + 0x016, 0x00061874, + 0x016, 0x00021874, + 0x030, 0x0004470f, + 0x031, 0x00044ff0, + 0x032, 0x00000070, + 0x033, 0x000dd480, + 0x034, 0x000ffac0, + 0x035, 0x000b80c0, + 0x036, 0x00077000, + 0x037, 0x00064ff2, + 0x038, 0x000e7661, + 0x039, 0x00000e90, + 0x000, 0x00030000, + 0x018, 0x0000f401, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x01e, 0x00088009, + 0x01f, 0x00080003, + 0x0fe, 0x00000000, + 0x01e, 0x00088001, + 0x01f, 0x00080000, + 0x0fe, 0x00000000, + 0x018, 0x00097524, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x02b, 0x00041289, + 0x0fe, 0x00000000, + 0x02d, 0x0006aaaa, + 0x02e, 0x000b4d01, + 0x02d, 0x00080000, + 0x02e, 0x00004d02, + 0x02d, 0x00095555, + 0x02e, 0x00054d03, + 0x02d, 0x000aaaaa, + 0x02e, 0x000b4d04, + 0x02d, 0x000c0000, + 0x02e, 0x00004d05, + 0x02d, 0x000d5555, + 0x02e, 0x00054d06, + 0x02d, 0x000eaaaa, + 0x02e, 0x000b4d07, + 0x02d, 0x00000000, + 0x02e, 0x00005108, + 0x02d, 0x00015555, + 0x02e, 0x00055109, + 0x02d, 0x0002aaaa, + 0x02e, 0x000b510a, + 0x02d, 0x00040000, + 0x02e, 0x0000510b, + 0x02d, 0x00055555, + 0x02e, 0x0005510c, +}; + +const u32 rtl8192du_radiob_2t_int_paarray[RADIOB_2T_INT_PA_ARRAYLENGTH] = { + 0x000, 0x00030000, + 0x001, 0x00030000, + 0x002, 0x00000000, + 0x003, 0x00018c63, + 0x004, 0x00018c63, + 0x008, 0x00084000, + 0x00b, 0x0001c000, + 0x00e, 0x00018c67, + 0x00f, 0x00000851, + 0x014, 0x00021440, + 0x018, 0x00007401, + 0x019, 0x00000060, + 0x01d, 0x000a1290, + 0x023, 0x00001558, + 0x01a, 0x00030a99, + 0x01b, 0x00040b00, + 0x01c, 0x000fc339, + 0x03a, 0x000a57eb, + 0x03b, 0x00020000, + 0x03c, 0x000ff455, + 0x020, 0x0000aa52, + 0x021, 0x00054000, + 0x040, 0x0000aa52, + 0x041, 0x00014000, + 0x025, 0x000803be, + 0x026, 0x000fc638, + 0x027, 0x00077c18, + 0x028, 0x000d1c31, + 0x029, 0x000d7110, + 0x02a, 0x000aeb04, + 0x02b, 0x0004128b, + 0x02c, 0x00001840, + 0x043, 0x0002444f, + 0x044, 0x0001adb0, + 0x045, 0x00056467, + 0x046, 0x0008992c, + 0x047, 0x0000452c, + 0x048, 0x000c0443, + 0x049, 0x00000730, + 0x04a, 0x00050f0f, + 0x04b, 0x000896ef, + 0x04c, 0x0000ddee, + 0x018, 0x00007401, + 0x000, 0x00070000, + 0x012, 0x000dc000, + 0x012, 0x00090000, + 0x012, 0x00051000, + 0x012, 0x00012000, + 0x013, 0x000287b7, + 0x013, 0x000247ab, + 0x013, 0x0002079f, + 0x013, 0x0001c793, + 0x013, 0x0001839b, + 0x013, 0x00014392, + 0x013, 0x0001019a, + 0x013, 0x0000c191, + 0x013, 0x00008194, + 0x013, 0x000040a0, + 0x013, 0x00000018, + 0x015, 0x0000f424, + 0x015, 0x0004f424, + 0x015, 0x0008f424, + 0x016, 0x000e1330, + 0x016, 0x000a1330, + 0x016, 0x00061330, + 0x016, 0x00021330, + 0x018, 0x00017524, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bf, + 0x013, 0x000247b3, + 0x013, 0x000207a7, + 0x013, 0x0001c79b, + 0x013, 0x0001839f, + 0x013, 0x00014393, + 0x013, 0x00010399, + 0x013, 0x0000c38d, + 0x013, 0x00008199, + 0x013, 0x0000418d, + 0x013, 0x00000099, + 0x015, 0x0000f495, + 0x015, 0x0004f495, + 0x015, 0x0008f495, + 0x016, 0x000e1874, + 0x016, 0x000a1874, + 0x016, 0x00061874, + 0x016, 0x00021874, + 0x018, 0x00037564, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bf, + 0x013, 0x000247b3, + 0x013, 0x000207a7, + 0x013, 0x0001c79b, + 0x013, 0x0001839f, + 0x013, 0x00014393, + 0x013, 0x00010399, + 0x013, 0x0000c38d, + 0x013, 0x00008199, + 0x013, 0x0000418d, + 0x013, 0x00000099, + 0x015, 0x0000f495, + 0x015, 0x0004f495, + 0x015, 0x0008f495, + 0x016, 0x000e1874, + 0x016, 0x000a1874, + 0x016, 0x00061874, + 0x016, 0x00021874, + 0x018, 0x00057595, + 0x000, 0x00070000, + 0x012, 0x000cf000, + 0x012, 0x000bc000, + 0x012, 0x00078000, + 0x012, 0x00000000, + 0x013, 0x000287bf, + 0x013, 0x000247b3, + 0x013, 0x000207a7, + 0x013, 0x0001c79b, + 0x013, 0x0001839f, + 0x013, 0x00014393, + 0x013, 0x00010399, + 0x013, 0x0000c38d, + 0x013, 0x00008199, + 0x013, 0x0000418d, + 0x013, 0x00000099, + 0x015, 0x0000f495, + 0x015, 0x0004f495, + 0x015, 0x0008f495, + 0x016, 0x000e1874, + 0x016, 0x000a1874, + 0x016, 0x00061874, + 0x016, 0x00021874, + 0x030, 0x0004470f, + 0x031, 0x00044ff0, + 0x032, 0x00000070, + 0x033, 0x000dd480, + 0x034, 0x000ffac0, + 0x035, 0x000b80c0, + 0x036, 0x00077000, + 0x037, 0x00064ff2, + 0x038, 0x000e7661, + 0x039, 0x00000e90, + 0x000, 0x00030000, + 0x018, 0x0000f401, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x01e, 0x00088009, + 0x01f, 0x00080003, + 0x0fe, 0x00000000, + 0x01e, 0x00088001, + 0x01f, 0x00080000, + 0x0fe, 0x00000000, + 0x018, 0x00087401, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x0fe, 0x00000000, + 0x02b, 0x00041289, + 0x0fe, 0x00000000, + 0x02d, 0x00066666, + 0x02e, 0x00064001, + 0x02d, 0x00091111, + 0x02e, 0x00014002, + 0x02d, 0x000bbbbb, + 0x02e, 0x000b4003, + 0x02d, 0x000e6666, + 0x02e, 0x00064004, + 0x02d, 0x00088888, + 0x02e, 0x00084005, + 0x02d, 0x0009dddd, + 0x02e, 0x000d4006, + 0x02d, 0x000b3333, + 0x02e, 0x00034007, + 0x02d, 0x00048888, + 0x02e, 0x00084408, + 0x02d, 0x000bbbbb, + 0x02e, 0x000b4409, + 0x02d, 0x000e6666, + 0x02e, 0x0006440a, + 0x02d, 0x00011111, + 0x02e, 0x0001480b, + 0x02d, 0x0003bbbb, + 0x02e, 0x000b480c, + 0x02d, 0x00066666, + 0x02e, 0x0006480d, + 0x02d, 0x000ccccc, + 0x02e, 0x000c480e, +}; + +const u32 rtl8192du_mac_2tarray[MAC_2T_ARRAYLENGTH] = { + 0x420, 0x00000080, + 0x423, 0x00000000, + 0x430, 0x00000000, + 0x431, 0x00000000, + 0x432, 0x00000000, + 0x433, 0x00000001, + 0x434, 0x00000004, + 0x435, 0x00000005, + 0x436, 0x00000006, + 0x437, 0x00000007, + 0x438, 0x00000000, + 0x439, 0x00000000, + 0x43a, 0x00000000, + 0x43b, 0x00000001, + 0x43c, 0x00000004, + 0x43d, 0x00000005, + 0x43e, 0x00000006, + 0x43f, 0x00000007, + 0x440, 0x00000050, + 0x441, 0x00000001, + 0x442, 0x00000000, + 0x444, 0x00000015, + 0x445, 0x000000f0, + 0x446, 0x0000000f, + 0x447, 0x00000000, + 0x462, 0x00000008, + 0x463, 0x00000003, + 0x4c8, 0x000000ff, + 0x4c9, 0x00000008, + 0x4cc, 0x000000ff, + 0x4cd, 0x000000ff, + 0x4ce, 0x00000001, + 0x500, 0x00000026, + 0x501, 0x000000a2, + 0x502, 0x0000002f, + 0x503, 0x00000000, + 0x504, 0x00000028, + 0x505, 0x000000a3, + 0x506, 0x0000005e, + 0x507, 0x00000000, + 0x508, 0x0000002b, + 0x509, 0x000000a4, + 0x50a, 0x0000005e, + 0x50b, 0x00000000, + 0x50c, 0x0000004f, + 0x50d, 0x000000a4, + 0x50e, 0x00000000, + 0x50f, 0x00000000, + 0x512, 0x0000001c, + 0x514, 0x0000000a, + 0x515, 0x00000010, + 0x516, 0x0000000a, + 0x517, 0x00000010, + 0x51a, 0x00000016, + 0x524, 0x0000000f, + 0x525, 0x0000004f, + 0x546, 0x00000040, + 0x547, 0x00000000, + 0x550, 0x00000010, + 0x551, 0x00000010, + 0x559, 0x00000002, + 0x55a, 0x00000002, + 0x55d, 0x000000ff, + 0x605, 0x00000080, + 0x608, 0x0000000e, + 0x609, 0x0000002a, + 0x652, 0x00000020, + 0x63c, 0x0000000a, + 0x63d, 0x0000000a, + 0x63e, 0x0000000e, + 0x63f, 0x0000000e, + 0x66e, 0x00000005, + 0x700, 0x00000021, + 0x701, 0x00000043, + 0x702, 0x00000065, + 0x703, 0x00000087, + 0x708, 0x00000021, + 0x709, 0x00000043, + 0x70a, 0x00000065, + 0x70b, 0x00000087, + 0x024, 0x0000000d, + 0x025, 0x00000080, + 0x026, 0x00000011, + 0x027, 0x00000000, + 0x028, 0x00000083, + 0x029, 0x000000db, + 0x02a, 0x000000ff, + 0x02b, 0x00000000, + 0x014, 0x00000055, + 0x015, 0x000000a9, + 0x016, 0x0000008b, + 0x017, 0x00000008, + 0x010, 0x00000003, + 0x011, 0x0000002b, + 0x012, 0x00000002, + 0x013, 0x00000049, +}; + +const u32 rtl8192du_agctab_array[AGCTAB_ARRAYLENGTH] = { + 0xc78, 0x7b000001, + 0xc78, 0x7b010001, + 0xc78, 0x7b020001, + 0xc78, 0x7b030001, + 0xc78, 0x7b040001, + 0xc78, 0x7b050001, + 0xc78, 0x7b060001, + 0xc78, 0x7a070001, + 0xc78, 0x79080001, + 0xc78, 0x78090001, + 0xc78, 0x770a0001, + 0xc78, 0x760b0001, + 0xc78, 0x750c0001, + 0xc78, 0x740d0001, + 0xc78, 0x730e0001, + 0xc78, 0x720f0001, + 0xc78, 0x71100001, + 0xc78, 0x70110001, + 0xc78, 0x6f120001, + 0xc78, 0x6e130001, + 0xc78, 0x6d140001, + 0xc78, 0x6c150001, + 0xc78, 0x6b160001, + 0xc78, 0x6a170001, + 0xc78, 0x69180001, + 0xc78, 0x68190001, + 0xc78, 0x671a0001, + 0xc78, 0x661b0001, + 0xc78, 0x651c0001, + 0xc78, 0x641d0001, + 0xc78, 0x631e0001, + 0xc78, 0x621f0001, + 0xc78, 0x61200001, + 0xc78, 0x60210001, + 0xc78, 0x49220001, + 0xc78, 0x48230001, + 0xc78, 0x47240001, + 0xc78, 0x46250001, + 0xc78, 0x45260001, + 0xc78, 0x44270001, + 0xc78, 0x43280001, + 0xc78, 0x42290001, + 0xc78, 0x412a0001, + 0xc78, 0x402b0001, + 0xc78, 0x262c0001, + 0xc78, 0x252d0001, + 0xc78, 0x242e0001, + 0xc78, 0x232f0001, + 0xc78, 0x22300001, + 0xc78, 0x21310001, + 0xc78, 0x20320001, + 0xc78, 0x06330001, + 0xc78, 0x05340001, + 0xc78, 0x04350001, + 0xc78, 0x03360001, + 0xc78, 0x02370001, + 0xc78, 0x01380001, + 0xc78, 0x00390001, + 0xc78, 0x003a0001, + 0xc78, 0x003b0001, + 0xc78, 0x003c0001, + 0xc78, 0x003d0001, + 0xc78, 0x003e0001, + 0xc78, 0x003f0001, + 0xc78, 0x7b400001, + 0xc78, 0x7b410001, + 0xc78, 0x7a420001, + 0xc78, 0x79430001, + 0xc78, 0x78440001, + 0xc78, 0x77450001, + 0xc78, 0x76460001, + 0xc78, 0x75470001, + 0xc78, 0x74480001, + 0xc78, 0x73490001, + 0xc78, 0x724a0001, + 0xc78, 0x714b0001, + 0xc78, 0x704c0001, + 0xc78, 0x6f4d0001, + 0xc78, 0x6e4e0001, + 0xc78, 0x6d4f0001, + 0xc78, 0x6c500001, + 0xc78, 0x6b510001, + 0xc78, 0x6a520001, + 0xc78, 0x69530001, + 0xc78, 0x68540001, + 0xc78, 0x67550001, + 0xc78, 0x66560001, + 0xc78, 0x65570001, + 0xc78, 0x64580001, + 0xc78, 0x63590001, + 0xc78, 0x625a0001, + 0xc78, 0x615b0001, + 0xc78, 0x605c0001, + 0xc78, 0x485d0001, + 0xc78, 0x475e0001, + 0xc78, 0x465f0001, + 0xc78, 0x45600001, + 0xc78, 0x44610001, + 0xc78, 0x43620001, + 0xc78, 0x42630001, + 0xc78, 0x41640001, + 0xc78, 0x40650001, + 0xc78, 0x27660001, + 0xc78, 0x26670001, + 0xc78, 0x25680001, + 0xc78, 0x24690001, + 0xc78, 0x236a0001, + 0xc78, 0x226b0001, + 0xc78, 0x216c0001, + 0xc78, 0x206d0001, + 0xc78, 0x206e0001, + 0xc78, 0x206f0001, + 0xc78, 0x20700001, + 0xc78, 0x20710001, + 0xc78, 0x20720001, + 0xc78, 0x20730001, + 0xc78, 0x20740001, + 0xc78, 0x20750001, + 0xc78, 0x20760001, + 0xc78, 0x20770001, + 0xc78, 0x20780001, + 0xc78, 0x20790001, + 0xc78, 0x207a0001, + 0xc78, 0x207b0001, + 0xc78, 0x207c0001, + 0xc78, 0x207d0001, + 0xc78, 0x207e0001, + 0xc78, 0x207f0001, + 0xc78, 0x38000002, + 0xc78, 0x38010002, + 0xc78, 0x38020002, + 0xc78, 0x38030002, + 0xc78, 0x38040002, + 0xc78, 0x38050002, + 0xc78, 0x38060002, + 0xc78, 0x38070002, + 0xc78, 0x38080002, + 0xc78, 0x3c090002, + 0xc78, 0x3e0a0002, + 0xc78, 0x400b0002, + 0xc78, 0x440c0002, + 0xc78, 0x480d0002, + 0xc78, 0x4c0e0002, + 0xc78, 0x500f0002, + 0xc78, 0x52100002, + 0xc78, 0x56110002, + 0xc78, 0x5a120002, + 0xc78, 0x5e130002, + 0xc78, 0x60140002, + 0xc78, 0x60150002, + 0xc78, 0x60160002, + 0xc78, 0x62170002, + 0xc78, 0x62180002, + 0xc78, 0x62190002, + 0xc78, 0x621a0002, + 0xc78, 0x621b0002, + 0xc78, 0x621c0002, + 0xc78, 0x621d0002, + 0xc78, 0x621e0002, + 0xc78, 0x621f0002, + 0xc78, 0x32000044, + 0xc78, 0x32010044, + 0xc78, 0x32020044, + 0xc78, 0x32030044, + 0xc78, 0x32040044, + 0xc78, 0x32050044, + 0xc78, 0x32060044, + 0xc78, 0x34070044, + 0xc78, 0x35080044, + 0xc78, 0x36090044, + 0xc78, 0x370a0044, + 0xc78, 0x380b0044, + 0xc78, 0x390c0044, + 0xc78, 0x3a0d0044, + 0xc78, 0x3e0e0044, + 0xc78, 0x420f0044, + 0xc78, 0x44100044, + 0xc78, 0x46110044, + 0xc78, 0x4a120044, + 0xc78, 0x4e130044, + 0xc78, 0x50140044, + 0xc78, 0x55150044, + 0xc78, 0x5a160044, + 0xc78, 0x5e170044, + 0xc78, 0x64180044, + 0xc78, 0x6e190044, + 0xc78, 0x6e1a0044, + 0xc78, 0x6e1b0044, + 0xc78, 0x6e1c0044, + 0xc78, 0x6e1d0044, + 0xc78, 0x6e1e0044, + 0xc78, 0x6e1f0044, + 0xc78, 0x6e1f0000, +}; + +const u32 rtl8192du_agctab_5garray[AGCTAB_5G_ARRAYLENGTH] = { + 0xc78, 0x7b000001, + 0xc78, 0x7b010001, + 0xc78, 0x7a020001, + 0xc78, 0x79030001, + 0xc78, 0x78040001, + 0xc78, 0x77050001, + 0xc78, 0x76060001, + 0xc78, 0x75070001, + 0xc78, 0x74080001, + 0xc78, 0x73090001, + 0xc78, 0x720a0001, + 0xc78, 0x710b0001, + 0xc78, 0x700c0001, + 0xc78, 0x6f0d0001, + 0xc78, 0x6e0e0001, + 0xc78, 0x6d0f0001, + 0xc78, 0x6c100001, + 0xc78, 0x6b110001, + 0xc78, 0x6a120001, + 0xc78, 0x69130001, + 0xc78, 0x68140001, + 0xc78, 0x67150001, + 0xc78, 0x66160001, + 0xc78, 0x65170001, + 0xc78, 0x64180001, + 0xc78, 0x63190001, + 0xc78, 0x621a0001, + 0xc78, 0x611b0001, + 0xc78, 0x601c0001, + 0xc78, 0x481d0001, + 0xc78, 0x471e0001, + 0xc78, 0x461f0001, + 0xc78, 0x45200001, + 0xc78, 0x44210001, + 0xc78, 0x43220001, + 0xc78, 0x42230001, + 0xc78, 0x41240001, + 0xc78, 0x40250001, + 0xc78, 0x27260001, + 0xc78, 0x26270001, + 0xc78, 0x25280001, + 0xc78, 0x24290001, + 0xc78, 0x232a0001, + 0xc78, 0x222b0001, + 0xc78, 0x212c0001, + 0xc78, 0x202d0001, + 0xc78, 0x202e0001, + 0xc78, 0x202f0001, + 0xc78, 0x20300001, + 0xc78, 0x20310001, + 0xc78, 0x20320001, + 0xc78, 0x20330001, + 0xc78, 0x20340001, + 0xc78, 0x20350001, + 0xc78, 0x20360001, + 0xc78, 0x20370001, + 0xc78, 0x20380001, + 0xc78, 0x20390001, + 0xc78, 0x203a0001, + 0xc78, 0x203b0001, + 0xc78, 0x203c0001, + 0xc78, 0x203d0001, + 0xc78, 0x203e0001, + 0xc78, 0x203f0001, + 0xc78, 0x32000044, + 0xc78, 0x32010044, + 0xc78, 0x32020044, + 0xc78, 0x32030044, + 0xc78, 0x32040044, + 0xc78, 0x32050044, + 0xc78, 0x32060044, + 0xc78, 0x34070044, + 0xc78, 0x35080044, + 0xc78, 0x36090044, + 0xc78, 0x370a0044, + 0xc78, 0x380b0044, + 0xc78, 0x390c0044, + 0xc78, 0x3a0d0044, + 0xc78, 0x3e0e0044, + 0xc78, 0x420f0044, + 0xc78, 0x44100044, + 0xc78, 0x46110044, + 0xc78, 0x4a120044, + 0xc78, 0x4e130044, + 0xc78, 0x50140044, + 0xc78, 0x55150044, + 0xc78, 0x5a160044, + 0xc78, 0x5e170044, + 0xc78, 0x64180044, + 0xc78, 0x6e190044, + 0xc78, 0x6e1a0044, + 0xc78, 0x6e1b0044, + 0xc78, 0x6e1c0044, + 0xc78, 0x6e1d0044, + 0xc78, 0x6e1e0044, + 0xc78, 0x6e1f0044, + 0xc78, 0x6e1f0000, +}; + +const u32 rtl8192du_agctab_2garray[AGCTAB_2G_ARRAYLENGTH] = { + 0xc78, 0x7b000001, + 0xc78, 0x7b010001, + 0xc78, 0x7b020001, + 0xc78, 0x7b030001, + 0xc78, 0x7b040001, + 0xc78, 0x7b050001, + 0xc78, 0x7b060001, + 0xc78, 0x7a070001, + 0xc78, 0x79080001, + 0xc78, 0x78090001, + 0xc78, 0x770a0001, + 0xc78, 0x760b0001, + 0xc78, 0x750c0001, + 0xc78, 0x740d0001, + 0xc78, 0x730e0001, + 0xc78, 0x720f0001, + 0xc78, 0x71100001, + 0xc78, 0x70110001, + 0xc78, 0x6f120001, + 0xc78, 0x6e130001, + 0xc78, 0x6d140001, + 0xc78, 0x6c150001, + 0xc78, 0x6b160001, + 0xc78, 0x6a170001, + 0xc78, 0x69180001, + 0xc78, 0x68190001, + 0xc78, 0x671a0001, + 0xc78, 0x661b0001, + 0xc78, 0x651c0001, + 0xc78, 0x641d0001, + 0xc78, 0x631e0001, + 0xc78, 0x621f0001, + 0xc78, 0x61200001, + 0xc78, 0x60210001, + 0xc78, 0x49220001, + 0xc78, 0x48230001, + 0xc78, 0x47240001, + 0xc78, 0x46250001, + 0xc78, 0x45260001, + 0xc78, 0x44270001, + 0xc78, 0x43280001, + 0xc78, 0x42290001, + 0xc78, 0x412a0001, + 0xc78, 0x402b0001, + 0xc78, 0x262c0001, + 0xc78, 0x252d0001, + 0xc78, 0x242e0001, + 0xc78, 0x232f0001, + 0xc78, 0x22300001, + 0xc78, 0x21310001, + 0xc78, 0x20320001, + 0xc78, 0x06330001, + 0xc78, 0x05340001, + 0xc78, 0x04350001, + 0xc78, 0x03360001, + 0xc78, 0x02370001, + 0xc78, 0x01380001, + 0xc78, 0x00390001, + 0xc78, 0x003a0001, + 0xc78, 0x003b0001, + 0xc78, 0x003c0001, + 0xc78, 0x003d0001, + 0xc78, 0x003e0001, + 0xc78, 0x003f0001, + 0xc78, 0x38000002, + 0xc78, 0x38010002, + 0xc78, 0x38020002, + 0xc78, 0x38030002, + 0xc78, 0x38040002, + 0xc78, 0x38050002, + 0xc78, 0x38060002, + 0xc78, 0x38070002, + 0xc78, 0x38080002, + 0xc78, 0x3c090002, + 0xc78, 0x3e0a0002, + 0xc78, 0x400b0002, + 0xc78, 0x440c0002, + 0xc78, 0x480d0002, + 0xc78, 0x4c0e0002, + 0xc78, 0x500f0002, + 0xc78, 0x52100002, + 0xc78, 0x56110002, + 0xc78, 0x5a120002, + 0xc78, 0x5e130002, + 0xc78, 0x60140002, + 0xc78, 0x60150002, + 0xc78, 0x60160002, + 0xc78, 0x62170002, + 0xc78, 0x62180002, + 0xc78, 0x62190002, + 0xc78, 0x621a0002, + 0xc78, 0x621b0002, + 0xc78, 0x621c0002, + 0xc78, 0x621d0002, + 0xc78, 0x621e0002, + 0xc78, 0x621f0002, + 0xc78, 0x6e1f0000, +}; diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/table.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/table.h new file mode 100644 index 0000000000000000000000000000000000000000..b809ba5113201eec07cf3da495cb17bb1030bd1a --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/table.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2024 Realtek Corporation.*/ + +#ifndef __RTL92DU_TABLE_H__ +#define __RTL92DU_TABLE_H__ + +#define PHY_REG_2T_ARRAYLENGTH 372 +#define PHY_REG_ARRAY_PG_LENGTH 624 +#define RADIOA_2T_ARRAYLENGTH 378 +#define RADIOB_2T_ARRAYLENGTH 384 +#define RADIOA_2T_INT_PA_ARRAYLENGTH 378 +#define RADIOB_2T_INT_PA_ARRAYLENGTH 384 +#define MAC_2T_ARRAYLENGTH 192 +#define AGCTAB_ARRAYLENGTH 386 +#define AGCTAB_5G_ARRAYLENGTH 194 +#define AGCTAB_2G_ARRAYLENGTH 194 + +extern const u32 rtl8192du_phy_reg_2tarray[PHY_REG_2T_ARRAYLENGTH]; +extern const u32 rtl8192du_phy_reg_array_pg[PHY_REG_ARRAY_PG_LENGTH]; +extern const u32 rtl8192du_radioa_2tarray[RADIOA_2T_ARRAYLENGTH]; +extern const u32 rtl8192du_radiob_2tarray[RADIOB_2T_ARRAYLENGTH]; +extern const u32 rtl8192du_radioa_2t_int_paarray[RADIOA_2T_INT_PA_ARRAYLENGTH]; +extern const u32 rtl8192du_radiob_2t_int_paarray[RADIOB_2T_INT_PA_ARRAYLENGTH]; +extern const u32 rtl8192du_mac_2tarray[MAC_2T_ARRAYLENGTH]; +extern const u32 rtl8192du_agctab_array[AGCTAB_ARRAYLENGTH]; +extern const u32 rtl8192du_agctab_5garray[AGCTAB_5G_ARRAYLENGTH]; +extern const u32 rtl8192du_agctab_2garray[AGCTAB_2G_ARRAYLENGTH]; + +#endif diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/trx.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/trx.c new file mode 100644 index 0000000000000000000000000000000000000000..743ce0cfffe6dda9d04209bc5060c0eaa2f16a99 --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/trx.c @@ -0,0 +1,372 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2024 Realtek Corporation.*/ + +#include "../wifi.h" +#include "../base.h" +#include "../usb.h" +#include "../rtl8192d/reg.h" +#include "../rtl8192d/def.h" +#include "../rtl8192d/trx_common.h" +#include "trx.h" + +void rtl92du_tx_cleanup(struct ieee80211_hw *hw, struct sk_buff *skb) +{ +} + +int rtl92du_tx_post_hdl(struct ieee80211_hw *hw, struct urb *urb, + struct sk_buff *skb) +{ + return 0; +} + +struct sk_buff *rtl92du_tx_aggregate_hdl(struct ieee80211_hw *hw, + struct sk_buff_head *list) +{ + return skb_dequeue(list); +} + +static enum rtl_desc_qsel _rtl92du_hwq_to_descq(u16 queue_index) +{ + switch (queue_index) { + case RTL_TXQ_BCN: + return QSLT_BEACON; + case RTL_TXQ_MGT: + return QSLT_MGNT; + case RTL_TXQ_VO: + return QSLT_VO; + case RTL_TXQ_VI: + return QSLT_VI; + case RTL_TXQ_BK: + return QSLT_BK; + default: + case RTL_TXQ_BE: + return QSLT_BE; + } +} + +/* For HW recovery information */ +static void _rtl92du_tx_desc_checksum(__le32 *txdesc) +{ + __le16 *ptr = (__le16 *)txdesc; + u16 checksum = 0; + u32 index; + + /* Clear first */ + set_tx_desc_tx_desc_checksum(txdesc, 0); + for (index = 0; index < 16; index++) + checksum = checksum ^ le16_to_cpu(*(ptr + index)); + set_tx_desc_tx_desc_checksum(txdesc, checksum); +} + +void rtl92du_tx_fill_desc(struct ieee80211_hw *hw, + struct ieee80211_hdr *hdr, u8 *pdesc_tx, + u8 *pbd_desc_tx, struct ieee80211_tx_info *info, + struct ieee80211_sta *sta, + struct sk_buff *skb, + u8 queue_index, + struct rtl_tcb_desc *tcb_desc) +{ + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_ps_ctl *ppsc = rtl_psc(rtlpriv); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + struct rtl_mac *mac = rtl_mac(rtlpriv); + struct rtl_sta_info *sta_entry; + __le16 fc = hdr->frame_control; + u8 agg_state = RTL_AGG_STOP; + u16 pktlen = skb->len; + u32 rts_en, hw_rts_en; + u8 ampdu_density = 0; + u16 seq_number; + __le32 *txdesc; + u8 rate_flag; + u8 tid; + + rtl_get_tcb_desc(hw, info, sta, skb, tcb_desc); + + txdesc = (__le32 *)skb_push(skb, RTL_TX_HEADER_SIZE); + memset(txdesc, 0, RTL_TX_HEADER_SIZE); + + set_tx_desc_pkt_size(txdesc, pktlen); + set_tx_desc_linip(txdesc, 0); + set_tx_desc_pkt_offset(txdesc, RTL_DUMMY_OFFSET); + set_tx_desc_offset(txdesc, RTL_TX_HEADER_SIZE); + /* 5G have no CCK rate */ + if (rtlhal->current_bandtype == BAND_ON_5G) + if (tcb_desc->hw_rate < DESC_RATE6M) + tcb_desc->hw_rate = DESC_RATE6M; + + set_tx_desc_tx_rate(txdesc, tcb_desc->hw_rate); + if (tcb_desc->use_shortgi || tcb_desc->use_shortpreamble) + set_tx_desc_data_shortgi(txdesc, 1); + + if (rtlhal->macphymode == DUALMAC_DUALPHY && + tcb_desc->hw_rate == DESC_RATEMCS7) + set_tx_desc_data_shortgi(txdesc, 1); + + if (sta) { + sta_entry = (struct rtl_sta_info *)sta->drv_priv; + tid = ieee80211_get_tid(hdr); + agg_state = sta_entry->tids[tid].agg.agg_state; + ampdu_density = sta->deflink.ht_cap.ampdu_density; + } + + if (agg_state == RTL_AGG_OPERATIONAL && + info->flags & IEEE80211_TX_CTL_AMPDU) { + set_tx_desc_agg_enable(txdesc, 1); + set_tx_desc_max_agg_num(txdesc, 0x14); + set_tx_desc_ampdu_density(txdesc, ampdu_density); + tcb_desc->rts_enable = 1; + tcb_desc->rts_rate = DESC_RATE24M; + } else { + set_tx_desc_agg_break(txdesc, 1); + } + seq_number = (le16_to_cpu(hdr->seq_ctrl) & IEEE80211_SCTL_SEQ) >> 4; + set_tx_desc_seq(txdesc, seq_number); + + rts_en = tcb_desc->rts_enable && !tcb_desc->cts_enable; + hw_rts_en = tcb_desc->rts_enable || tcb_desc->cts_enable; + set_tx_desc_rts_enable(txdesc, rts_en); + set_tx_desc_hw_rts_enable(txdesc, hw_rts_en); + set_tx_desc_cts2self(txdesc, tcb_desc->cts_enable); + set_tx_desc_rts_stbc(txdesc, tcb_desc->rts_stbc); + /* 5G have no CCK rate */ + if (rtlhal->current_bandtype == BAND_ON_5G) + if (tcb_desc->rts_rate < DESC_RATE6M) + tcb_desc->rts_rate = DESC_RATE6M; + set_tx_desc_rts_rate(txdesc, tcb_desc->rts_rate); + set_tx_desc_rts_bw(txdesc, 0); + set_tx_desc_rts_sc(txdesc, tcb_desc->rts_sc); + set_tx_desc_rts_short(txdesc, tcb_desc->rts_use_shortpreamble); + + rate_flag = info->control.rates[0].flags; + if (mac->bw_40) { + if (rate_flag & IEEE80211_TX_RC_DUP_DATA) { + set_tx_desc_data_bw(txdesc, 1); + set_tx_desc_tx_sub_carrier(txdesc, 3); + } else if (rate_flag & IEEE80211_TX_RC_40_MHZ_WIDTH) { + set_tx_desc_data_bw(txdesc, 1); + set_tx_desc_tx_sub_carrier(txdesc, mac->cur_40_prime_sc); + } else { + set_tx_desc_data_bw(txdesc, 0); + set_tx_desc_tx_sub_carrier(txdesc, 0); + } + } else { + set_tx_desc_data_bw(txdesc, 0); + set_tx_desc_tx_sub_carrier(txdesc, 0); + } + + if (info->control.hw_key) { + struct ieee80211_key_conf *keyconf = info->control.hw_key; + + switch (keyconf->cipher) { + case WLAN_CIPHER_SUITE_WEP40: + case WLAN_CIPHER_SUITE_WEP104: + case WLAN_CIPHER_SUITE_TKIP: + set_tx_desc_sec_type(txdesc, 0x1); + break; + case WLAN_CIPHER_SUITE_CCMP: + set_tx_desc_sec_type(txdesc, 0x3); + break; + default: + set_tx_desc_sec_type(txdesc, 0x0); + break; + } + } + + set_tx_desc_pkt_id(txdesc, 0); + set_tx_desc_queue_sel(txdesc, _rtl92du_hwq_to_descq(queue_index)); + set_tx_desc_data_rate_fb_limit(txdesc, 0x1F); + set_tx_desc_rts_rate_fb_limit(txdesc, 0xF); + set_tx_desc_disable_fb(txdesc, 0); + set_tx_desc_use_rate(txdesc, tcb_desc->use_driver_rate); + + if (ieee80211_is_data_qos(fc)) { + if (mac->rdg_en) { + rtl_dbg(rtlpriv, COMP_SEND, DBG_TRACE, + "Enable RDG function\n"); + set_tx_desc_rdg_enable(txdesc, 1); + set_tx_desc_htc(txdesc, 1); + } + set_tx_desc_qos(txdesc, 1); + } + + if (rtlpriv->dm.useramask) { + set_tx_desc_rate_id(txdesc, tcb_desc->ratr_index); + set_tx_desc_macid(txdesc, tcb_desc->mac_id); + } else { + set_tx_desc_rate_id(txdesc, 0xC + tcb_desc->ratr_index); + set_tx_desc_macid(txdesc, tcb_desc->ratr_index); + } + + if (!ieee80211_is_data_qos(fc) && ppsc->leisure_ps && + ppsc->fwctrl_lps) { + set_tx_desc_hwseq_en(txdesc, 1); + set_tx_desc_pkt_id(txdesc, 8); + } + + if (ieee80211_has_morefrags(fc)) + set_tx_desc_more_frag(txdesc, 1); + if (is_multicast_ether_addr(ieee80211_get_DA(hdr)) || + is_broadcast_ether_addr(ieee80211_get_DA(hdr))) + set_tx_desc_bmc(txdesc, 1); + + set_tx_desc_own(txdesc, 1); + set_tx_desc_last_seg(txdesc, 1); + set_tx_desc_first_seg(txdesc, 1); + _rtl92du_tx_desc_checksum(txdesc); + + rtl_dbg(rtlpriv, COMP_SEND, DBG_TRACE, "==>\n"); +} + +static void _rtl92du_config_out_ep(struct ieee80211_hw *hw, u8 num_out_pipe) +{ + struct rtl_usb *rtlusb = rtl_usbdev(rtl_usbpriv(hw)); + struct rtl_priv *rtlpriv = rtl_priv(hw); + struct rtl_hal *rtlhal = rtl_hal(rtlpriv); + u16 ep_cfg; + + rtlusb->out_queue_sel = 0; + rtlusb->out_ep_nums = 0; + + if (rtlhal->interfaceindex == 0) + ep_cfg = rtl_read_word(rtlpriv, REG_USB_Queue_Select_MAC0); + else + ep_cfg = rtl_read_word(rtlpriv, REG_USB_Queue_Select_MAC1); + + if (ep_cfg & 0x00f) { + rtlusb->out_queue_sel |= TX_SELE_HQ; + rtlusb->out_ep_nums++; + } + if (ep_cfg & 0x0f0) { + rtlusb->out_queue_sel |= TX_SELE_NQ; + rtlusb->out_ep_nums++; + } + if (ep_cfg & 0xf00) { + rtlusb->out_queue_sel |= TX_SELE_LQ; + rtlusb->out_ep_nums++; + } + + switch (num_out_pipe) { + case 3: + rtlusb->out_queue_sel = TX_SELE_HQ | TX_SELE_NQ | TX_SELE_LQ; + rtlusb->out_ep_nums = 3; + break; + case 2: + rtlusb->out_queue_sel = TX_SELE_HQ | TX_SELE_NQ; + rtlusb->out_ep_nums = 2; + break; + case 1: + rtlusb->out_queue_sel = TX_SELE_HQ; + rtlusb->out_ep_nums = 1; + break; + default: + break; + } +} + +static void _rtl92du_one_out_ep_mapping(struct rtl_usb *rtlusb, + struct rtl_ep_map *ep_map) +{ + ep_map->ep_mapping[RTL_TXQ_BE] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_BK] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_VI] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_VO] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_MGT] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_BCN] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_HI] = rtlusb->out_eps[0]; +} + +static void _rtl92du_two_out_ep_mapping(struct rtl_usb *rtlusb, + struct rtl_ep_map *ep_map) +{ + ep_map->ep_mapping[RTL_TXQ_BE] = rtlusb->out_eps[1]; + ep_map->ep_mapping[RTL_TXQ_BK] = rtlusb->out_eps[1]; + ep_map->ep_mapping[RTL_TXQ_VI] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_VO] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_MGT] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_BCN] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_HI] = rtlusb->out_eps[0]; +} + +static void _rtl92du_three_out_ep_mapping(struct rtl_usb *rtlusb, + struct rtl_ep_map *ep_map) +{ + ep_map->ep_mapping[RTL_TXQ_BE] = rtlusb->out_eps[2]; + ep_map->ep_mapping[RTL_TXQ_BK] = rtlusb->out_eps[2]; + ep_map->ep_mapping[RTL_TXQ_VI] = rtlusb->out_eps[1]; + ep_map->ep_mapping[RTL_TXQ_VO] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_MGT] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_BCN] = rtlusb->out_eps[0]; + ep_map->ep_mapping[RTL_TXQ_HI] = rtlusb->out_eps[0]; +} + +static int _rtl92du_out_ep_mapping(struct ieee80211_hw *hw) +{ + struct rtl_usb *rtlusb = rtl_usbdev(rtl_usbpriv(hw)); + struct rtl_ep_map *ep_map = &rtlusb->ep_map; + + switch (rtlusb->out_ep_nums) { + case 1: + _rtl92du_one_out_ep_mapping(rtlusb, ep_map); + break; + case 2: + _rtl92du_two_out_ep_mapping(rtlusb, ep_map); + break; + case 3: + _rtl92du_three_out_ep_mapping(rtlusb, ep_map); + break; + default: + return -EINVAL; + } + + return 0; +} + +int rtl92du_endpoint_mapping(struct ieee80211_hw *hw) +{ + struct rtl_usb *rtlusb = rtl_usbdev(rtl_usbpriv(hw)); + + _rtl92du_config_out_ep(hw, rtlusb->out_ep_nums); + + /* Normal chip with one IN and one OUT doesn't have interrupt IN EP. */ + if (rtlusb->out_ep_nums == 1 && rtlusb->in_ep_nums != 1) + return -EINVAL; + + return _rtl92du_out_ep_mapping(hw); +} + +u16 rtl92du_mq_to_hwq(__le16 fc, u16 mac80211_queue_index) +{ + u16 hw_queue_index; + + if (unlikely(ieee80211_is_beacon(fc))) { + hw_queue_index = RTL_TXQ_BCN; + goto out; + } + if (ieee80211_is_mgmt(fc)) { + hw_queue_index = RTL_TXQ_MGT; + goto out; + } + + switch (mac80211_queue_index) { + case 0: + hw_queue_index = RTL_TXQ_VO; + break; + case 1: + hw_queue_index = RTL_TXQ_VI; + break; + case 2: + hw_queue_index = RTL_TXQ_BE; + break; + case 3: + hw_queue_index = RTL_TXQ_BK; + break; + default: + hw_queue_index = RTL_TXQ_BE; + WARN_ONCE(true, "rtl8192du: QSLT_BE queue, skb_queue:%d\n", + mac80211_queue_index); + break; + } +out: + return hw_queue_index; +} diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192du/trx.h b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/trx.h new file mode 100644 index 0000000000000000000000000000000000000000..8c3d24622fa7cefa17ba347ba7f9aa40e2fc125f --- /dev/null +++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192du/trx.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2024 Realtek Corporation.*/ + +#ifndef __RTL92DU_TRX_H__ +#define __RTL92DU_TRX_H__ + +#define TX_SELE_HQ BIT(0) /* High Queue */ +#define TX_SELE_LQ BIT(1) /* Low Queue */ +#define TX_SELE_NQ BIT(2) /* Normal Queue */ + +#define TX_TOTAL_PAGE_NUMBER_92DU 0xF8 +#define TEST_PAGE_NUM_PUBQ_92DU 0x89 +#define TX_TOTAL_PAGE_NUMBER_92D_DUAL_MAC 0x7A +#define NORMAL_PAGE_NUM_PUBQ_92D_DUAL_MAC 0x5A +#define NORMAL_PAGE_NUM_HPQ_92D_DUAL_MAC 0x10 +#define NORMAL_PAGE_NUM_LPQ_92D_DUAL_MAC 0x10 +#define NORMAL_PAGE_NUM_NORMALQ_92D_DUAL_MAC 0 + +#define WMM_NORMAL_TX_TOTAL_PAGE_NUMBER 0xF5 + +#define WMM_NORMAL_PAGE_NUM_PUBQ_92D 0x65 +#define WMM_NORMAL_PAGE_NUM_HPQ_92D 0x30 +#define WMM_NORMAL_PAGE_NUM_LPQ_92D 0x30 +#define WMM_NORMAL_PAGE_NUM_NPQ_92D 0x30 + +#define WMM_NORMAL_PAGE_NUM_PUBQ_92D_DUAL_MAC 0x32 +#define WMM_NORMAL_PAGE_NUM_HPQ_92D_DUAL_MAC 0x18 +#define WMM_NORMAL_PAGE_NUM_LPQ_92D_DUAL_MAC 0x18 +#define WMM_NORMAL_PAGE_NUM_NPQ_92D_DUAL_MAC 0x18 + +static inline void set_tx_desc_bmc(__le32 *__txdesc, u32 __value) +{ + le32p_replace_bits(__txdesc, __value, BIT(24)); +} + +static inline void set_tx_desc_agg_break(__le32 *__txdesc, u32 __value) +{ + le32p_replace_bits((__txdesc + 1), __value, BIT(6)); +} + +static inline void set_tx_desc_tx_desc_checksum(__le32 *__txdesc, u32 __value) +{ + le32p_replace_bits((__txdesc + 7), __value, GENMASK(15, 0)); +} + +void rtl92du_tx_fill_desc(struct ieee80211_hw *hw, + struct ieee80211_hdr *hdr, u8 *pdesc, + u8 *pbd_desc_tx, struct ieee80211_tx_info *info, + struct ieee80211_sta *sta, + struct sk_buff *skb, u8 hw_queue, + struct rtl_tcb_desc *ptcb_desc); +int rtl92du_endpoint_mapping(struct ieee80211_hw *hw); +u16 rtl92du_mq_to_hwq(__le16 fc, u16 mac80211_queue_index); +struct sk_buff *rtl92du_tx_aggregate_hdl(struct ieee80211_hw *hw, + struct sk_buff_head *list); +void rtl92du_tx_cleanup(struct ieee80211_hw *hw, struct sk_buff *skb); +int rtl92du_tx_post_hdl(struct ieee80211_hw *hw, struct urb *urb, + struct sk_buff *skb); + +#endif diff --git a/drivers/net/wireless/realtek/rtlwifi/usb.c b/drivers/net/wireless/realtek/rtlwifi/usb.c index 2ea72d9e39577ede7b763f78aa9d1e3f468c7aa5..d37a017b2b814f83a0cc1689abb8a2ecaba8f509 100644 --- a/drivers/net/wireless/realtek/rtlwifi/usb.c +++ b/drivers/net/wireless/realtek/rtlwifi/usb.c @@ -23,6 +23,8 @@ MODULE_DESCRIPTION("USB basic driver for rtlwifi"); #define MAX_USBCTRL_VENDORREQ_TIMES 10 +static void _rtl_usb_cleanup_tx(struct ieee80211_hw *hw); + static void _usbctrl_vendorreq_sync(struct usb_device *udev, u8 reqtype, u16 value, void *pdata, u16 len) { @@ -285,9 +287,23 @@ static int _rtl_usb_init(struct ieee80211_hw *hw) } /* usb endpoint mapping */ err = rtlpriv->cfg->usb_interface_cfg->usb_endpoint_mapping(hw); - rtlusb->usb_mq_to_hwq = rtlpriv->cfg->usb_interface_cfg->usb_mq_to_hwq; - _rtl_usb_init_tx(hw); - _rtl_usb_init_rx(hw); + if (err) + return err; + + rtlusb->usb_mq_to_hwq = rtlpriv->cfg->usb_interface_cfg->usb_mq_to_hwq; + + err = _rtl_usb_init_tx(hw); + if (err) + return err; + + err = _rtl_usb_init_rx(hw); + if (err) + goto err_out; + + return 0; + +err_out: + _rtl_usb_cleanup_tx(hw); return err; } @@ -691,17 +707,13 @@ static int rtl_usb_start(struct ieee80211_hw *hw) } /*======================= tx =========================================*/ -static void rtl_usb_cleanup(struct ieee80211_hw *hw) +static void _rtl_usb_cleanup_tx(struct ieee80211_hw *hw) { u32 i; struct sk_buff *_skb; struct rtl_usb *rtlusb = rtl_usbdev(rtl_usbpriv(hw)); struct ieee80211_tx_info *txinfo; - /* clean up rx stuff. */ - _rtl_usb_cleanup_rx(hw); - - /* clean up tx stuff */ for (i = 0; i < RTL_USB_MAX_EP_NUM; i++) { while ((_skb = skb_dequeue(&rtlusb->tx_skb_queue[i]))) { rtlusb->usb_tx_cleanup(hw, _skb); @@ -715,6 +727,12 @@ static void rtl_usb_cleanup(struct ieee80211_hw *hw) usb_kill_anchored_urbs(&rtlusb->tx_submitted); } +static void rtl_usb_cleanup(struct ieee80211_hw *hw) +{ + _rtl_usb_cleanup_rx(hw); + _rtl_usb_cleanup_tx(hw); +} + /* We may add some struct into struct rtl_usb later. Do deinit here. */ static void rtl_usb_deinit(struct ieee80211_hw *hw) { @@ -937,7 +955,7 @@ static const struct rtl_intf_ops rtl_usb_ops = { int rtl_usb_probe(struct usb_interface *intf, const struct usb_device_id *id, - struct rtl_hal_cfg *rtl_hal_cfg) + const struct rtl_hal_cfg *rtl_hal_cfg) { int err; struct ieee80211_hw *hw = NULL; diff --git a/drivers/net/wireless/realtek/rtlwifi/usb.h b/drivers/net/wireless/realtek/rtlwifi/usb.h index 12529afc0510b623d01ad3f8e7b4662256266304..b66d6f9ae564eb4826b8ec21ed96b78e76399cdb 100644 --- a/drivers/net/wireless/realtek/rtlwifi/usb.h +++ b/drivers/net/wireless/realtek/rtlwifi/usb.h @@ -136,7 +136,7 @@ struct rtl_usb_priv { int rtl_usb_probe(struct usb_interface *intf, const struct usb_device_id *id, - struct rtl_hal_cfg *rtl92cu_hal_cfg); + const struct rtl_hal_cfg *rtl92cu_hal_cfg); void rtl_usb_disconnect(struct usb_interface *intf); int rtl_usb_suspend(struct usb_interface *pusb_intf, pm_message_t message); int rtl_usb_resume(struct usb_interface *pusb_intf); diff --git a/drivers/net/wireless/realtek/rtlwifi/wifi.h b/drivers/net/wireless/realtek/rtlwifi/wifi.h index 442419568734b25737c2c5e5aa3975b72135bcc9..ae6e351bc83c9158256696cd2f301fece689cf01 100644 --- a/drivers/net/wireless/realtek/rtlwifi/wifi.h +++ b/drivers/net/wireless/realtek/rtlwifi/wifi.h @@ -2356,9 +2356,9 @@ struct rtl_hal_cfg { bool write_readback; char *name; char *alt_fw_name; - struct rtl_hal_ops *ops; + const struct rtl_hal_ops *ops; struct rtl_mod_params *mod_params; - struct rtl_hal_usbint_cfg *usb_interface_cfg; + const struct rtl_hal_usbint_cfg *usb_interface_cfg; enum rtl_spec_ver spec_ver; /*this map used for some registers or vars @@ -2707,7 +2707,7 @@ struct rtl_priv { /* hal_cfg : for diff cards * intf_ops : for diff interrface usb/pcie */ - struct rtl_hal_cfg *cfg; + const struct rtl_hal_cfg *cfg; const struct rtl_intf_ops *intf_ops; /* this var will be set by set_bit, @@ -2746,6 +2746,12 @@ struct rtl_priv { */ bool use_new_trx_flow; + /* For dual MAC RTL8192DU, things shared by the 2 USB interfaces */ + u32 *curveindex_2g; + u32 *curveindex_5g; + struct mutex *mutex_for_power_on_off; /* for power on/off */ + struct mutex *mutex_for_hw_init; /* for hardware init */ + #ifdef CONFIG_PM struct wiphy_wowlan_support wowlan; #endif diff --git a/drivers/net/wireless/realtek/rtw88/pci.c b/drivers/net/wireless/realtek/rtw88/pci.c index 30232f7e3ec543c456271e815ffea8adb464c080..a5b9d6c7be37e619de11e5f8d274cb278aaa3b79 100644 --- a/drivers/net/wireless/realtek/rtw88/pci.c +++ b/drivers/net/wireless/realtek/rtw88/pci.c @@ -1682,12 +1682,16 @@ static int rtw_pci_napi_poll(struct napi_struct *napi, int budget) return work_done; } -static void rtw_pci_napi_init(struct rtw_dev *rtwdev) +static int rtw_pci_napi_init(struct rtw_dev *rtwdev) { struct rtw_pci *rtwpci = (struct rtw_pci *)rtwdev->priv; - init_dummy_netdev(&rtwpci->netdev); - netif_napi_add(&rtwpci->netdev, &rtwpci->napi, rtw_pci_napi_poll); + rtwpci->netdev = alloc_netdev_dummy(0); + if (!rtwpci->netdev) + return -ENOMEM; + + netif_napi_add(rtwpci->netdev, &rtwpci->napi, rtw_pci_napi_poll); + return 0; } static void rtw_pci_napi_deinit(struct rtw_dev *rtwdev) @@ -1696,6 +1700,7 @@ static void rtw_pci_napi_deinit(struct rtw_dev *rtwdev) rtw_pci_napi_stop(rtwdev); netif_napi_del(&rtwpci->napi); + free_netdev(rtwpci->netdev); } int rtw_pci_probe(struct pci_dev *pdev, @@ -1745,7 +1750,11 @@ int rtw_pci_probe(struct pci_dev *pdev, goto err_pci_declaim; } - rtw_pci_napi_init(rtwdev); + ret = rtw_pci_napi_init(rtwdev); + if (ret) { + rtw_err(rtwdev, "failed to setup NAPI\n"); + goto err_pci_declaim; + } ret = rtw_chip_info_setup(rtwdev); if (ret) { diff --git a/drivers/net/wireless/realtek/rtw88/pci.h b/drivers/net/wireless/realtek/rtw88/pci.h index 0c37efd8c66fa776be6362250a28ec154f19de58..13988db1cb4c4e3a3d9080223c3fd286312cae3c 100644 --- a/drivers/net/wireless/realtek/rtw88/pci.h +++ b/drivers/net/wireless/realtek/rtw88/pci.h @@ -215,7 +215,7 @@ struct rtw_pci { bool running; /* napi structure */ - struct net_device netdev; + struct net_device *netdev; struct napi_struct napi; u16 rx_tag; diff --git a/drivers/net/wireless/realtek/rtw88/usb.c b/drivers/net/wireless/realtek/rtw88/usb.c index a0188511099a1bdfa28bc8882141fb7390f1c851..d204d138afe298a1e993a376018efa4977a06f57 100644 --- a/drivers/net/wireless/realtek/rtw88/usb.c +++ b/drivers/net/wireless/realtek/rtw88/usb.c @@ -273,6 +273,8 @@ static void rtw_usb_write_port_tx_complete(struct urb *urb) info = IEEE80211_SKB_CB(skb); tx_data = rtw_usb_get_tx_data(skb); + skb_pull(skb, rtwdev->chip->tx_pkt_desc_sz); + /* enqueue to wait for tx report */ if (info->flags & IEEE80211_TX_CTL_REQ_TX_STATUS) { rtw_tx_report_enqueue(rtwdev, skb, tx_data->sn); @@ -433,23 +435,21 @@ static int rtw_usb_write_data(struct rtw_dev *rtwdev, { const struct rtw_chip_info *chip = rtwdev->chip; struct sk_buff *skb; - unsigned int desclen, headsize, size; + unsigned int size; u8 qsel; int ret = 0; size = pkt_info->tx_pkt_size; qsel = pkt_info->qsel; - desclen = chip->tx_pkt_desc_sz; - headsize = pkt_info->offset ? pkt_info->offset : desclen; - skb = dev_alloc_skb(headsize + size); + skb = dev_alloc_skb(chip->tx_pkt_desc_sz + size); if (unlikely(!skb)) return -ENOMEM; - skb_reserve(skb, headsize); + skb_reserve(skb, chip->tx_pkt_desc_sz); skb_put_data(skb, buf, size); - skb_push(skb, headsize); - memset(skb->data, 0, headsize); + skb_push(skb, chip->tx_pkt_desc_sz); + memset(skb->data, 0, chip->tx_pkt_desc_sz); rtw_tx_fill_tx_desc(pkt_info, skb); rtw_tx_fill_txdesc_checksum(rtwdev, pkt_info, skb->data); diff --git a/drivers/net/wireless/realtek/rtw89/cam.c b/drivers/net/wireless/realtek/rtw89/cam.c index 1864f543a6c67eda4eb8bcb74328f88b7301b528..35291efbbae9f4ce7ae84b67db70b5f3eb0f4313 100644 --- a/drivers/net/wireless/realtek/rtw89/cam.c +++ b/drivers/net/wireless/realtek/rtw89/cam.c @@ -211,6 +211,46 @@ static int rtw89_cam_get_addr_cam_key_idx(struct rtw89_addr_cam_entry *addr_cam, return 0; } +static int rtw89_cam_detach_sec_cam(struct rtw89_dev *rtwdev, + struct ieee80211_vif *vif, + struct ieee80211_sta *sta, + const struct rtw89_sec_cam_entry *sec_cam, + bool inform_fw) +{ + struct rtw89_sta *rtwsta = sta_to_rtwsta_safe(sta); + struct rtw89_vif *rtwvif; + struct rtw89_addr_cam_entry *addr_cam; + unsigned int i; + int ret = 0; + + if (!vif) { + rtw89_err(rtwdev, "No iface for deleting sec cam\n"); + return -EINVAL; + } + + rtwvif = (struct rtw89_vif *)vif->drv_priv; + addr_cam = rtw89_get_addr_cam_of(rtwvif, rtwsta); + + for_each_set_bit(i, addr_cam->sec_cam_map, RTW89_SEC_CAM_IN_ADDR_CAM) { + if (addr_cam->sec_ent[i] != sec_cam->sec_cam_idx) + continue; + + clear_bit(i, addr_cam->sec_cam_map); + } + + if (inform_fw) { + ret = rtw89_chip_h2c_dctl_sec_cam(rtwdev, rtwvif, rtwsta); + if (ret) + rtw89_err(rtwdev, + "failed to update dctl cam del key: %d\n", ret); + ret = rtw89_fw_h2c_cam(rtwdev, rtwvif, rtwsta, NULL); + if (ret) + rtw89_err(rtwdev, "failed to update cam del key: %d\n", ret); + } + + return ret; +} + static int rtw89_cam_attach_sec_cam(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif, struct ieee80211_sta *sta, @@ -242,10 +282,8 @@ static int rtw89_cam_attach_sec_cam(struct rtw89_dev *rtwdev, return ret; } - key->hw_key_idx = key_idx; addr_cam->sec_ent_keyid[key_idx] = key->keyidx; addr_cam->sec_ent[key_idx] = sec_cam->sec_cam_idx; - addr_cam->sec_entries[key_idx] = sec_cam; set_bit(key_idx, addr_cam->sec_cam_map); ret = rtw89_chip_h2c_dctl_sec_cam(rtwdev, rtwvif, rtwsta); if (ret) { @@ -258,7 +296,6 @@ static int rtw89_cam_attach_sec_cam(struct rtw89_dev *rtwdev, rtw89_err(rtwdev, "failed to update addr cam sec entry: %d\n", ret); clear_bit(key_idx, addr_cam->sec_cam_map); - addr_cam->sec_entries[key_idx] = NULL; return ret; } @@ -295,6 +332,9 @@ static int rtw89_cam_sec_key_install(struct rtw89_dev *rtwdev, goto err_release_cam; } + key->hw_key_idx = sec_cam_idx; + cam_info->sec_entries[sec_cam_idx] = sec_cam; + sec_cam->sec_cam_idx = sec_cam_idx; sec_cam->type = hw_key_type; sec_cam->len = RTW89_SEC_CAM_LEN; @@ -316,6 +356,7 @@ static int rtw89_cam_sec_key_install(struct rtw89_dev *rtwdev, return 0; err_release_cam: + cam_info->sec_entries[sec_cam_idx] = NULL; kfree(sec_cam); clear_bit(sec_cam_idx, cam_info->sec_cam_map); if (ext_key) @@ -386,42 +427,22 @@ int rtw89_cam_sec_key_del(struct rtw89_dev *rtwdev, struct ieee80211_key_conf *key, bool inform_fw) { - struct rtw89_sta *rtwsta = sta_to_rtwsta_safe(sta); struct rtw89_cam_info *cam_info = &rtwdev->cam_info; - struct rtw89_vif *rtwvif; - struct rtw89_addr_cam_entry *addr_cam; - struct rtw89_sec_cam_entry *sec_cam; - u8 key_idx = key->hw_key_idx; + const struct rtw89_sec_cam_entry *sec_cam; u8 sec_cam_idx; - int ret = 0; - - if (!vif) { - rtw89_err(rtwdev, "No iface for deleting sec cam\n"); - return -EINVAL; - } + int ret; - rtwvif = (struct rtw89_vif *)vif->drv_priv; - addr_cam = rtw89_get_addr_cam_of(rtwvif, rtwsta); - sec_cam = addr_cam->sec_entries[key_idx]; + sec_cam_idx = key->hw_key_idx; + sec_cam = cam_info->sec_entries[sec_cam_idx]; if (!sec_cam) return -EINVAL; - /* detach sec cam from addr cam */ - clear_bit(key_idx, addr_cam->sec_cam_map); - addr_cam->sec_entries[key_idx] = NULL; - if (inform_fw) { - ret = rtw89_chip_h2c_dctl_sec_cam(rtwdev, rtwvif, rtwsta); - if (ret) - rtw89_err(rtwdev, "failed to update dctl cam del key: %d\n", ret); - ret = rtw89_fw_h2c_cam(rtwdev, rtwvif, rtwsta, NULL); - if (ret) - rtw89_err(rtwdev, "failed to update cam del key: %d\n", ret); - } + ret = rtw89_cam_detach_sec_cam(rtwdev, vif, sta, sec_cam, inform_fw); /* clear valid bit in addr cam will disable sec cam, * so we don't need to send H2C command again */ - sec_cam_idx = sec_cam->sec_cam_idx; + cam_info->sec_entries[sec_cam_idx] = NULL; clear_bit(sec_cam_idx, cam_info->sec_cam_map); if (sec_cam->ext_key) clear_bit(sec_cam_idx + 1, cam_info->sec_cam_map); diff --git a/drivers/net/wireless/realtek/rtw89/chan.c b/drivers/net/wireless/realtek/rtw89/chan.c index 051a3cad6101b6459381a89f1623383793688c3c..3b1997223cc52268d23c67bc3de54ffd27d1eae6 100644 --- a/drivers/net/wireless/realtek/rtw89/chan.c +++ b/drivers/net/wireless/realtek/rtw89/chan.c @@ -2322,7 +2322,6 @@ static void rtw89_swap_sub_entity(struct rtw89_dev *rtwdev, enum rtw89_sub_entity_idx idx2) { struct rtw89_hal *hal = &rtwdev->hal; - struct rtw89_sub_entity tmp; struct rtw89_vif *rtwvif; u8 cur; @@ -2332,9 +2331,7 @@ static void rtw89_swap_sub_entity(struct rtw89_dev *rtwdev, hal->sub[idx1].cfg->idx = idx2; hal->sub[idx2].cfg->idx = idx1; - tmp = hal->sub[idx1]; - hal->sub[idx1] = hal->sub[idx2]; - hal->sub[idx2] = tmp; + swap(hal->sub[idx1], hal->sub[idx2]); rtw89_for_each_rtwvif(rtwdev, rtwvif) { if (!rtwvif->chanctx_assigned) diff --git a/drivers/net/wireless/realtek/rtw89/core.c b/drivers/net/wireless/realtek/rtw89/core.c index ddc390d24ec1a6e6e230574a570ade00a115ea90..3e99b63a7995951556570a6162b643f517dfa26f 100644 --- a/drivers/net/wireless/realtek/rtw89/core.c +++ b/drivers/net/wireless/realtek/rtw89/core.c @@ -499,31 +499,21 @@ static void rtw89_core_tx_update_sec_key(struct rtw89_dev *rtwdev, struct rtw89_core_tx_request *tx_req) { + struct rtw89_cam_info *cam_info = &rtwdev->cam_info; const struct rtw89_chip_info *chip = rtwdev->chip; - struct ieee80211_vif *vif = tx_req->vif; - struct ieee80211_sta *sta = tx_req->sta; + const struct rtw89_sec_cam_entry *sec_cam; struct ieee80211_tx_info *info; struct ieee80211_key_conf *key; - struct rtw89_vif *rtwvif; - struct rtw89_sta *rtwsta = sta_to_rtwsta_safe(sta); - struct rtw89_addr_cam_entry *addr_cam; - struct rtw89_sec_cam_entry *sec_cam; struct rtw89_tx_desc_info *desc_info = &tx_req->desc_info; struct sk_buff *skb = tx_req->skb; u8 sec_type = RTW89_SEC_KEY_TYPE_NONE; + u8 sec_cam_idx; u64 pn64; - if (!vif) { - rtw89_warn(rtwdev, "cannot set sec key without vif\n"); - return; - } - - rtwvif = (struct rtw89_vif *)vif->drv_priv; - addr_cam = rtw89_get_addr_cam_of(rtwvif, rtwsta); - info = IEEE80211_SKB_CB(skb); key = info->control.hw_key; - sec_cam = addr_cam->sec_entries[key->hw_key_idx]; + sec_cam_idx = key->hw_key_idx; + sec_cam = cam_info->sec_entries[sec_cam_idx]; if (!sec_cam) { rtw89_warn(rtwdev, "sec cam entry is empty\n"); return; @@ -823,6 +813,8 @@ rtw89_core_tx_update_data_info(struct rtw89_dev *rtwdev, desc_info->mac_id = rtw89_core_tx_get_mac_id(rtwdev, tx_req); desc_info->port = desc_info->hiq ? rtwvif->port : 0; desc_info->er_cap = rtwsta ? rtwsta->er_cap : false; + desc_info->stbc = rtwsta ? rtwsta->ra.stbc_cap : false; + desc_info->ldpc = rtwsta ? rtwsta->ra.ldpc_cap : false; /* enable wd_info for AMPDU */ desc_info->en_wd_info = true; @@ -1137,6 +1129,8 @@ static __le32 rtw89_build_txwd_info0(struct rtw89_tx_desc_info *desc_info) { u32 dword = FIELD_PREP(RTW89_TXWD_INFO0_USE_RATE, desc_info->use_rate) | FIELD_PREP(RTW89_TXWD_INFO0_DATA_RATE, desc_info->data_rate) | + FIELD_PREP(RTW89_TXWD_INFO0_DATA_STBC, desc_info->stbc) | + FIELD_PREP(RTW89_TXWD_INFO0_DATA_LDPC, desc_info->ldpc) | FIELD_PREP(RTW89_TXWD_INFO0_DISDATAFB, desc_info->dis_data_fb) | FIELD_PREP(RTW89_TXWD_INFO0_MULTIPORT_ID, desc_info->port); @@ -1145,7 +1139,9 @@ static __le32 rtw89_build_txwd_info0(struct rtw89_tx_desc_info *desc_info) static __le32 rtw89_build_txwd_info0_v1(struct rtw89_tx_desc_info *desc_info) { - u32 dword = FIELD_PREP(RTW89_TXWD_INFO0_DISDATAFB, desc_info->dis_data_fb) | + u32 dword = FIELD_PREP(RTW89_TXWD_INFO0_DATA_STBC, desc_info->stbc) | + FIELD_PREP(RTW89_TXWD_INFO0_DATA_LDPC, desc_info->ldpc) | + FIELD_PREP(RTW89_TXWD_INFO0_DISDATAFB, desc_info->dis_data_fb) | FIELD_PREP(RTW89_TXWD_INFO0_MULTIPORT_ID, desc_info->port) | FIELD_PREP(RTW89_TXWD_INFO0_DATA_ER, desc_info->er_cap) | FIELD_PREP(RTW89_TXWD_INFO0_DATA_BW_ER, 0); @@ -1311,7 +1307,9 @@ static __le32 rtw89_build_txwd_body7_v2(struct rtw89_tx_desc_info *desc_info) static __le32 rtw89_build_txwd_info0_v2(struct rtw89_tx_desc_info *desc_info) { - u32 dword = FIELD_PREP(BE_TXD_INFO0_DISDATAFB, desc_info->dis_data_fb) | + u32 dword = FIELD_PREP(BE_TXD_INFO0_DATA_STBC, desc_info->stbc) | + FIELD_PREP(BE_TXD_INFO0_DATA_LDPC, desc_info->ldpc) | + FIELD_PREP(BE_TXD_INFO0_DISDATAFB, desc_info->dis_data_fb) | FIELD_PREP(BE_TXD_INFO0_MULTIPORT_ID, desc_info->port); return cpu_to_le32(dword); @@ -1559,6 +1557,12 @@ static void rtw89_core_parse_phy_status_ie01(struct rtw89_dev *rtwdev, u32 t; phy_ppdu->chan_idx = le32_get_bits(ie->w0, RTW89_PHY_STS_IE01_W0_CH_IDX); + + if (rtwdev->hw->conf.flags & IEEE80211_CONF_MONITOR) { + phy_ppdu->ldpc = le32_get_bits(ie->w2, RTW89_PHY_STS_IE01_W2_LDPC); + phy_ppdu->stbc = le32_get_bits(ie->w2, RTW89_PHY_STS_IE01_W2_STBC); + } + if (phy_ppdu->rate < RTW89_HW_RATE_OFDM6) return; @@ -1984,6 +1988,23 @@ static void rtw89_core_hw_to_sband_rate(struct ieee80211_rx_status *rx_status) rx_status->rate_idx -= 4; } +static +void rtw89_core_update_rx_status_by_ppdu(struct rtw89_dev *rtwdev, + struct ieee80211_rx_status *rx_status, + struct rtw89_rx_phy_ppdu *phy_ppdu) +{ + if (!(rtwdev->hw->conf.flags & IEEE80211_CONF_MONITOR)) + return; + + if (!phy_ppdu) + return; + + if (phy_ppdu->ldpc) + rx_status->enc_flags |= RX_ENC_FLAG_LDPC; + if (phy_ppdu->stbc) + rx_status->enc_flags |= u8_encode_bits(1, RX_ENC_FLAG_STBC_MASK); +} + static const u8 rx_status_bw_to_radiotap_eht_usig[] = { [RATE_INFO_BW_20] = IEEE80211_RADIOTAP_EHT_USIG_COMMON_BW_20MHZ, [RATE_INFO_BW_5] = U8_MAX, @@ -2027,10 +2048,14 @@ static void rtw89_core_update_radiotap_eht(struct rtw89_dev *rtwdev, eht->user_info[0] = cpu_to_le32(IEEE80211_RADIOTAP_EHT_USER_INFO_MCS_KNOWN | - IEEE80211_RADIOTAP_EHT_USER_INFO_NSS_KNOWN_O); + IEEE80211_RADIOTAP_EHT_USER_INFO_NSS_KNOWN_O | + IEEE80211_RADIOTAP_EHT_USER_INFO_CODING_KNOWN); eht->user_info[0] |= le32_encode_bits(rx_status->rate_idx, IEEE80211_RADIOTAP_EHT_USER_INFO_MCS) | le32_encode_bits(rx_status->nss, IEEE80211_RADIOTAP_EHT_USER_INFO_NSS_O); + if (rx_status->enc_flags & RX_ENC_FLAG_LDPC) + eht->user_info[0] |= + cpu_to_le32(IEEE80211_RADIOTAP_EHT_USER_INFO_CODING); /* U-SIG */ tlv = (void *)tlv + sizeof(*tlv) + ALIGN(eht_len, 4); @@ -2056,6 +2081,8 @@ static void rtw89_core_update_radiotap(struct rtw89_dev *rtwdev, { static const struct ieee80211_radiotap_he known_he = { .data1 = cpu_to_le16(IEEE80211_RADIOTAP_HE_DATA1_DATA_MCS_KNOWN | + IEEE80211_RADIOTAP_HE_DATA1_CODING_KNOWN | + IEEE80211_RADIOTAP_HE_DATA1_STBC_KNOWN | IEEE80211_RADIOTAP_HE_DATA1_BW_RU_ALLOC_KNOWN), .data2 = cpu_to_le16(IEEE80211_RADIOTAP_HE_DATA2_GI_KNOWN), }; @@ -2087,6 +2114,7 @@ static void rtw89_core_rx_to_mac80211(struct rtw89_dev *rtwdev, rtw89_core_hw_to_sband_rate(rx_status); rtw89_core_rx_stats(rtwdev, phy_ppdu, desc_info, skb_ppdu); + rtw89_core_update_rx_status_by_ppdu(rtwdev, rx_status, phy_ppdu); rtw89_core_update_radiotap(rtwdev, skb_ppdu, rx_status); /* In low power mode, it does RX in thread context. */ local_bh_disable(); @@ -2492,11 +2520,15 @@ void rtw89_core_napi_stop(struct rtw89_dev *rtwdev) } EXPORT_SYMBOL(rtw89_core_napi_stop); -void rtw89_core_napi_init(struct rtw89_dev *rtwdev) +int rtw89_core_napi_init(struct rtw89_dev *rtwdev) { - init_dummy_netdev(&rtwdev->netdev); - netif_napi_add(&rtwdev->netdev, &rtwdev->napi, + rtwdev->netdev = alloc_netdev_dummy(0); + if (!rtwdev->netdev) + return -ENOMEM; + + netif_napi_add(rtwdev->netdev, &rtwdev->napi, rtwdev->hci.ops->napi_poll); + return 0; } EXPORT_SYMBOL(rtw89_core_napi_init); @@ -2504,6 +2536,7 @@ void rtw89_core_napi_deinit(struct rtw89_dev *rtwdev) { rtw89_core_napi_stop(rtwdev); netif_napi_del(&rtwdev->napi); + free_netdev(rtwdev->netdev); } EXPORT_SYMBOL(rtw89_core_napi_deinit); @@ -3348,14 +3381,13 @@ int rtw89_core_sta_add(struct rtw89_dev *rtwdev, BTC_ROLE_MSTS_STA_CONN_START); rtw89_chip_rfk_channel(rtwdev); } else if (vif->type == NL80211_IFTYPE_AP || sta->tdls) { - rtwsta->mac_id = rtw89_core_acquire_bit_map(rtwdev->mac_id_map, - RTW89_MAX_MAC_ID_NUM); + rtwsta->mac_id = rtw89_acquire_mac_id(rtwdev); if (rtwsta->mac_id == RTW89_MAX_MAC_ID_NUM) return -ENOSPC; ret = rtw89_mac_set_macid_pause(rtwdev, rtwsta->mac_id, false); if (ret) { - rtw89_core_release_bit_map(rtwdev->mac_id_map, rtwsta->mac_id); + rtw89_release_mac_id(rtwdev, rtwsta->mac_id); rtw89_warn(rtwdev, "failed to send h2c macid pause\n"); return ret; } @@ -3363,7 +3395,7 @@ int rtw89_core_sta_add(struct rtw89_dev *rtwdev, ret = rtw89_fw_h2c_role_maintain(rtwdev, rtwvif, rtwsta, RTW89_ROLE_CREATE); if (ret) { - rtw89_core_release_bit_map(rtwdev->mac_id_map, rtwsta->mac_id); + rtw89_release_mac_id(rtwdev, rtwsta->mac_id); rtw89_warn(rtwdev, "failed to send h2c role info\n"); return ret; } @@ -3536,7 +3568,7 @@ int rtw89_core_sta_remove(struct rtw89_dev *rtwdev, rtw89_btc_ntfy_role_info(rtwdev, rtwvif, rtwsta, BTC_ROLE_MSTS_STA_DIS_CONN); } else if (vif->type == NL80211_IFTYPE_AP || sta->tdls) { - rtw89_core_release_bit_map(rtwdev->mac_id_map, rtwsta->mac_id); + rtw89_release_mac_id(rtwdev, rtwsta->mac_id); ret = rtw89_fw_h2c_role_maintain(rtwdev, rtwvif, rtwsta, RTW89_ROLE_REMOVE); @@ -4186,6 +4218,25 @@ void rtw89_core_stop(struct rtw89_dev *rtwdev) rtw89_hci_reset(rtwdev); } +u8 rtw89_acquire_mac_id(struct rtw89_dev *rtwdev) +{ + const struct rtw89_chip_info *chip = rtwdev->chip; + u8 mac_id_num = chip->support_macid_num; + u8 mac_id; + + mac_id = find_first_zero_bit(rtwdev->mac_id_map, mac_id_num); + if (mac_id == mac_id_num) + return RTW89_MAX_MAC_ID_NUM; + + set_bit(mac_id, rtwdev->mac_id_map); + return mac_id; +} + +void rtw89_release_mac_id(struct rtw89_dev *rtwdev, u8 mac_id) +{ + clear_bit(mac_id, rtwdev->mac_id_map); +} + int rtw89_core_init(struct rtw89_dev *rtwdev) { struct rtw89_btc *btc = &rtwdev->btc; @@ -4475,6 +4526,10 @@ static int rtw89_core_register_hw(struct rtw89_dev *rtwdev) hw->max_tx_aggregation_subframes = RTW89_MAX_TX_AGG_NUM; hw->uapsd_max_sp_len = IEEE80211_WMM_IE_STA_QOSINFO_SP_ALL; + hw->radiotap_mcs_details |= IEEE80211_RADIOTAP_MCS_HAVE_FEC | + IEEE80211_RADIOTAP_MCS_HAVE_STBC; + hw->radiotap_vht_details |= IEEE80211_RADIOTAP_VHT_KNOWN_STBC; + ieee80211_hw_set(hw, SIGNAL_DBM); ieee80211_hw_set(hw, HAS_RATE_CONTROL); ieee80211_hw_set(hw, MFP_CAPABLE); diff --git a/drivers/net/wireless/realtek/rtw89/core.h b/drivers/net/wireless/realtek/rtw89/core.h index 112bdd95fc6ead4a3c0981ebe9d10a9998390764..68221f0b156e2789b851a1bdebf1484e4996422c 100644 --- a/drivers/net/wireless/realtek/rtw89/core.h +++ b/drivers/net/wireless/realtek/rtw89/core.h @@ -793,6 +793,8 @@ struct rtw89_rx_phy_ppdu { u8 evm_max; u8 evm_min; } ofdm; + bool ldpc; + bool stbc; bool to_self; bool valid; }; @@ -884,6 +886,13 @@ enum rtw89_ps_mode { #define RTW89_BYR_BW_NUM (RTW89_CHANNEL_WIDTH_320 + 1) #define RTW89_PPE_BW_NUM (RTW89_CHANNEL_WIDTH_320 + 1) +enum rtw89_pe_duration { + RTW89_PE_DURATION_0 = 0, + RTW89_PE_DURATION_8 = 1, + RTW89_PE_DURATION_16 = 2, + RTW89_PE_DURATION_16_20 = 3, +}; + enum rtw89_ru_bandwidth { RTW89_RU26 = 0, RTW89_RU52 = 1, @@ -1129,6 +1138,8 @@ struct rtw89_tx_desc_info { bool hiq; u8 port; bool er_cap; + bool stbc; + bool ldpc; }; struct rtw89_core_tx_request { @@ -3249,7 +3260,6 @@ struct rtw89_addr_cam_entry { DECLARE_BITMAP(sec_cam_map, RTW89_SEC_CAM_IN_ADDR_CAM); u8 sec_ent_keyid[RTW89_SEC_CAM_IN_ADDR_CAM]; u8 sec_ent[RTW89_SEC_CAM_IN_ADDR_CAM]; - struct rtw89_sec_cam_entry *sec_entries[RTW89_SEC_CAM_IN_ADDR_CAM]; }; struct rtw89_bssid_cam_entry { @@ -4144,6 +4154,7 @@ struct rtw89_chip_info { u8 wde_qempty_acq_grpnum; u8 wde_qempty_mgq_grpsel; u32 rf_base_addr[2]; + u8 support_macid_num; u8 support_chanctx_num; u8 support_bands; u16 support_bandwidths; @@ -4436,6 +4447,7 @@ struct rtw89_cam_info { DECLARE_BITMAP(sec_cam_map, RTW89_MAX_SEC_CAM_NUM); DECLARE_BITMAP(ba_cam_map, RTW89_MAX_BA_CAM_NUM); struct rtw89_ba_cam_entry ba_cam_entry[RTW89_MAX_BA_CAM_NUM]; + const struct rtw89_sec_cam_entry *sec_entries[RTW89_MAX_SEC_CAM_NUM]; }; enum rtw89_sar_sources { @@ -5469,7 +5481,7 @@ struct rtw89_dev { struct rtw89_wow_param wow; /* napi structure */ - struct net_device netdev; + struct net_device *netdev; struct napi_struct napi; int napi_budget_countdown; @@ -6441,7 +6453,7 @@ void rtw89_core_query_rxdesc_v2(struct rtw89_dev *rtwdev, u8 *data, u32 data_offset); void rtw89_core_napi_start(struct rtw89_dev *rtwdev); void rtw89_core_napi_stop(struct rtw89_dev *rtwdev); -void rtw89_core_napi_init(struct rtw89_dev *rtwdev); +int rtw89_core_napi_init(struct rtw89_dev *rtwdev); void rtw89_core_napi_deinit(struct rtw89_dev *rtwdev); int rtw89_core_sta_add(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif, @@ -6470,6 +6482,8 @@ struct rtw89_dev *rtw89_alloc_ieee80211_hw(struct device *device, u32 bus_data_size, const struct rtw89_chip_info *chip); void rtw89_free_ieee80211_hw(struct rtw89_dev *rtwdev); +u8 rtw89_acquire_mac_id(struct rtw89_dev *rtwdev); +void rtw89_release_mac_id(struct rtw89_dev *rtwdev, u8 mac_id); void rtw89_core_set_chip_txpwr(struct rtw89_dev *rtwdev); void rtw89_get_default_chandef(struct cfg80211_chan_def *chandef); void rtw89_get_channel_params(const struct cfg80211_chan_def *chandef, diff --git a/drivers/net/wireless/realtek/rtw89/debug.c b/drivers/net/wireless/realtek/rtw89/debug.c index affffc4092ba32b128d6277c4e6687bf0603c8ab..210f192ac9ec488d8128c79f2515f2361608e081 100644 --- a/drivers/net/wireless/realtek/rtw89/debug.c +++ b/drivers/net/wireless/realtek/rtw89/debug.c @@ -3645,17 +3645,21 @@ static int rtw89_debug_priv_phy_info_get(struct seq_file *m, void *v) } static void rtw89_dump_addr_cam(struct seq_file *m, + struct rtw89_dev *rtwdev, struct rtw89_addr_cam_entry *addr_cam) { - struct rtw89_sec_cam_entry *sec_entry; + struct rtw89_cam_info *cam_info = &rtwdev->cam_info; + const struct rtw89_sec_cam_entry *sec_entry; + u8 sec_cam_idx; int i; seq_printf(m, "\taddr_cam_idx=%u\n", addr_cam->addr_cam_idx); seq_printf(m, "\t-> bssid_cam_idx=%u\n", addr_cam->bssid_cam_idx); seq_printf(m, "\tsec_cam_bitmap=%*ph\n", (int)sizeof(addr_cam->sec_cam_map), addr_cam->sec_cam_map); - for (i = 0; i < RTW89_SEC_CAM_IN_ADDR_CAM; i++) { - sec_entry = addr_cam->sec_entries[i]; + for_each_set_bit(i, addr_cam->sec_cam_map, RTW89_SEC_CAM_IN_ADDR_CAM) { + sec_cam_idx = addr_cam->sec_ent[i]; + sec_entry = cam_info->sec_entries[sec_cam_idx]; if (!sec_entry) continue; seq_printf(m, "\tsec[%d]: sec_cam_idx %u", i, sec_entry->sec_cam_idx); @@ -3694,12 +3698,13 @@ static void rtw89_vif_ids_get_iter(void *data, u8 *mac, struct ieee80211_vif *vif) { struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; + struct rtw89_dev *rtwdev = rtwvif->rtwdev; struct seq_file *m = (struct seq_file *)data; struct rtw89_bssid_cam_entry *bssid_cam = &rtwvif->bssid_cam; seq_printf(m, "VIF [%d] %pM\n", rtwvif->mac_id, rtwvif->mac_addr); seq_printf(m, "\tbssid_cam_idx=%u\n", bssid_cam->bssid_cam_idx); - rtw89_dump_addr_cam(m, &rtwvif->addr_cam); + rtw89_dump_addr_cam(m, rtwdev, &rtwvif->addr_cam); rtw89_dump_pkt_offload(m, &rtwvif->general_pkt_list, "\tpkt_ofld[GENERAL]: "); } @@ -3726,11 +3731,12 @@ static void rtw89_dump_ba_cam(struct seq_file *m, struct rtw89_sta *rtwsta) static void rtw89_sta_ids_get_iter(void *data, struct ieee80211_sta *sta) { struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv; + struct rtw89_dev *rtwdev = rtwsta->rtwdev; struct seq_file *m = (struct seq_file *)data; seq_printf(m, "STA [%d] %pM %s\n", rtwsta->mac_id, sta->addr, sta->tdls ? "(TDLS)" : ""); - rtw89_dump_addr_cam(m, &rtwsta->addr_cam); + rtw89_dump_addr_cam(m, rtwdev, &rtwsta->addr_cam); rtw89_dump_ba_cam(m, rtwsta); } diff --git a/drivers/net/wireless/realtek/rtw89/fw.c b/drivers/net/wireless/realtek/rtw89/fw.c index 044a5b90c7f4e04a7e5670056a1027d5c3408262..be39a8468d32b806f6353a74c5029d9a859377a2 100644 --- a/drivers/net/wireless/realtek/rtw89/fw.c +++ b/drivers/net/wireless/realtek/rtw89/fw.c @@ -2752,11 +2752,11 @@ static void __get_sta_he_pkt_padding(struct rtw89_dev *rtwdev, ppe8 = (ppe >> sh) & IEEE80211_PPE_THRES_NSS_MASK; if (ppe16 != 7 && ppe8 == 7) - pads[i] = 2; + pads[i] = RTW89_PE_DURATION_16; else if (ppe8 != 7) - pads[i] = 1; + pads[i] = RTW89_PE_DURATION_8; else - pads[i] = 0; + pads[i] = RTW89_PE_DURATION_0; } } @@ -2889,11 +2889,11 @@ static void __get_sta_eht_pkt_padding(struct rtw89_dev *rtwdev, ppe8 = (ppe >> sh) & IEEE80211_PPE_THRES_NSS_MASK; if (ppe16 != 7 && ppe8 == 7) - pads[i] = 2; + pads[i] = RTW89_PE_DURATION_16_20; else if (ppe8 != 7) - pads[i] = 1; + pads[i] = RTW89_PE_DURATION_8; else - pads[i] = 0; + pads[i] = RTW89_PE_DURATION_0; } } @@ -4850,6 +4850,7 @@ int rtw89_fw_h2c_scan_offload_be(struct rtw89_dev *rtwdev, { struct rtw89_hw_scan_info *scan_info = &rtwdev->scan_info; struct rtw89_wait_info *wait = &rtwdev->mac.fw_ofld_wait; + struct cfg80211_scan_request *req = rtwvif->scan_req; struct rtw89_h2c_scanofld_be_macc_role *macc_role; struct rtw89_chan *op = &scan_info->op_chan; struct rtw89_h2c_scanofld_be_opch *opch; @@ -4923,6 +4924,15 @@ int rtw89_fw_h2c_scan_offload_be(struct rtw89_dev *rtwdev, RTW89_H2C_SCANOFLD_BE_W6_CHAN_PROHIB_LOW); h2c->w7 = le32_encode_bits(option->prohib_chan >> 32, RTW89_H2C_SCANOFLD_BE_W7_CHAN_PROHIB_HIGH); + if (req->no_cck) { + h2c->w0 |= le32_encode_bits(true, RTW89_H2C_SCANOFLD_BE_W0_PROBE_WITH_RATE); + h2c->w8 = le32_encode_bits(RTW89_HW_RATE_OFDM6, + RTW89_H2C_SCANOFLD_BE_W8_PROBE_RATE_2GHZ) | + le32_encode_bits(RTW89_HW_RATE_OFDM6, + RTW89_H2C_SCANOFLD_BE_W8_PROBE_RATE_5GHZ) | + le32_encode_bits(RTW89_HW_RATE_OFDM6, + RTW89_H2C_SCANOFLD_BE_W8_PROBE_RATE_6GHZ); + } ptr += sizeof(*h2c); for (i = 0; i < option->num_macc_role; i++) { @@ -6245,7 +6255,14 @@ void rtw89_hw_scan_abort(struct rtw89_dev *rtwdev, struct ieee80211_vif *vif) ret = rtw89_hw_scan_offload(rtwdev, vif, false); if (ret) - rtw89_hw_scan_complete(rtwdev, vif, true); + rtw89_warn(rtwdev, "rtw89_hw_scan_offload failed ret %d\n", ret); + + /* Indicate ieee80211_scan_completed() before returning, which is safe + * because scan abort command always waits for completion of + * RTW89_SCAN_END_SCAN_NOTIFY, so that ieee80211_stop() can flush scan + * work properly. + */ + rtw89_hw_scan_complete(rtwdev, vif, true); } static bool rtw89_is_any_vif_connected_or_connecting(struct rtw89_dev *rtwdev) diff --git a/drivers/net/wireless/realtek/rtw89/fw.h b/drivers/net/wireless/realtek/rtw89/fw.h index 4151c9d566bd6ae5441334942f553766c63cd69e..01fea0b004d3598ea9a6b10446fff6deb5fe6e93 100644 --- a/drivers/net/wireless/realtek/rtw89/fw.h +++ b/drivers/net/wireless/realtek/rtw89/fw.h @@ -2722,6 +2722,7 @@ struct rtw89_h2c_scanofld_be { #define RTW89_H2C_SCANOFLD_BE_W0_MACID GENMASK(23, 8) #define RTW89_H2C_SCANOFLD_BE_W0_PORT GENMASK(26, 24) #define RTW89_H2C_SCANOFLD_BE_W0_BAND GENMASK(28, 27) +#define RTW89_H2C_SCANOFLD_BE_W0_PROBE_WITH_RATE BIT(29) #define RTW89_H2C_SCANOFLD_BE_W1_NUM_MACC_ROLE GENMASK(7, 0) #define RTW89_H2C_SCANOFLD_BE_W1_NUM_OP GENMASK(15, 8) #define RTW89_H2C_SCANOFLD_BE_W1_NORM_PD GENMASK(31, 16) @@ -2738,6 +2739,9 @@ struct rtw89_h2c_scanofld_be { #define RTW89_H2C_SCANOFLD_BE_W5_MLO_MODE GENMASK(31, 0) #define RTW89_H2C_SCANOFLD_BE_W6_CHAN_PROHIB_LOW GENMASK(31, 0) #define RTW89_H2C_SCANOFLD_BE_W7_CHAN_PROHIB_HIGH GENMASK(31, 0) +#define RTW89_H2C_SCANOFLD_BE_W8_PROBE_RATE_2GHZ GENMASK(7, 0) +#define RTW89_H2C_SCANOFLD_BE_W8_PROBE_RATE_5GHZ GENMASK(15, 8) +#define RTW89_H2C_SCANOFLD_BE_W8_PROBE_RATE_6GHZ GENMASK(23, 16) static inline void RTW89_SET_FWCMD_P2P_MACID(void *cmd, u32 val) { diff --git a/drivers/net/wireless/realtek/rtw89/mac.c b/drivers/net/wireless/realtek/rtw89/mac.c index 3fe0046f6eaa24998c1eb9dd90fe61e3d96af608..824ece03d92d2998121c6bb7247c881aad4e7635 100644 --- a/drivers/net/wireless/realtek/rtw89/mac.c +++ b/drivers/net/wireless/realtek/rtw89/mac.c @@ -4664,8 +4664,7 @@ int rtw89_mac_add_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) { int ret; - rtwvif->mac_id = rtw89_core_acquire_bit_map(rtwdev->mac_id_map, - RTW89_MAX_MAC_ID_NUM); + rtwvif->mac_id = rtw89_acquire_mac_id(rtwdev); if (rtwvif->mac_id == RTW89_MAX_MAC_ID_NUM) return -ENOSPC; @@ -4676,7 +4675,7 @@ int rtw89_mac_add_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) return 0; release_mac_id: - rtw89_core_release_bit_map(rtwdev->mac_id_map, rtwvif->mac_id); + rtw89_release_mac_id(rtwdev, rtwvif->mac_id); return ret; } @@ -4686,7 +4685,7 @@ int rtw89_mac_remove_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) int ret; ret = rtw89_mac_vif_deinit(rtwdev, rtwvif); - rtw89_core_release_bit_map(rtwdev->mac_id_map, rtwvif->mac_id); + rtw89_release_mac_id(rtwdev, rtwvif->mac_id); return ret; } @@ -4757,6 +4756,9 @@ rtw89_mac_c2h_scanofld_rsp(struct rtw89_dev *rtwdev, struct sk_buff *skb, } return; case RTW89_SCAN_END_SCAN_NOTIFY: + if (rtwdev->scan_info.abort) + return; + if (rtwvif && rtwvif->scan_req && last_chan < rtwvif->scan_req->n_channels) { ret = rtw89_hw_scan_offload(rtwdev, vif, true); @@ -4765,7 +4767,7 @@ rtw89_mac_c2h_scanofld_rsp(struct rtw89_dev *rtwdev, struct sk_buff *skb, rtw89_warn(rtwdev, "HW scan failed: %d\n", ret); } } else { - rtw89_hw_scan_complete(rtwdev, vif, rtwdev->scan_info.abort); + rtw89_hw_scan_complete(rtwdev, vif, false); } break; case RTW89_SCAN_ENTER_OP_NOTIFY: diff --git a/drivers/net/wireless/realtek/rtw89/mac80211.c b/drivers/net/wireless/realtek/rtw89/mac80211.c index 1ec97250e88e54679b28baad33a72d51ad43d3c4..41b286da3d591661e89d4d7133703c0b0f1b8e28 100644 --- a/drivers/net/wireless/realtek/rtw89/mac80211.c +++ b/drivers/net/wireless/realtek/rtw89/mac80211.c @@ -397,15 +397,14 @@ static void rtw89_conf_tx(struct rtw89_dev *rtwdev, } static void rtw89_station_mode_sta_assoc(struct rtw89_dev *rtwdev, - struct ieee80211_vif *vif, - struct ieee80211_bss_conf *conf) + struct ieee80211_vif *vif) { struct ieee80211_sta *sta; if (vif->type != NL80211_IFTYPE_STATION) return; - sta = ieee80211_find_sta(vif, conf->bssid); + sta = ieee80211_find_sta(vif, vif->cfg.ap_addr); if (!sta) { rtw89_err(rtwdev, "can't find sta to set sta_assoc state\n"); return; @@ -416,10 +415,8 @@ static void rtw89_station_mode_sta_assoc(struct rtw89_dev *rtwdev, rtw89_core_sta_assoc(rtwdev, vif, sta); } -static void rtw89_ops_bss_info_changed(struct ieee80211_hw *hw, - struct ieee80211_vif *vif, - struct ieee80211_bss_conf *conf, - u64 changed) +static void rtw89_ops_vif_cfg_changed(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, u64 changed) { struct rtw89_dev *rtwdev = hw->priv; struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; @@ -429,7 +426,7 @@ static void rtw89_ops_bss_info_changed(struct ieee80211_hw *hw, if (changed & BSS_CHANGED_ASSOC) { if (vif->cfg.assoc) { - rtw89_station_mode_sta_assoc(rtwdev, vif, conf); + rtw89_station_mode_sta_assoc(rtwdev, vif); rtw89_phy_set_bss_color(rtwdev, vif); rtw89_chip_cfg_txpwr_ul_tb_offset(rtwdev, vif); rtw89_mac_port_update(rtwdev, rtwvif); @@ -445,6 +442,26 @@ static void rtw89_ops_bss_info_changed(struct ieee80211_hw *hw, } } + if (changed & BSS_CHANGED_PS) + rtw89_recalc_lps(rtwdev); + + if (changed & BSS_CHANGED_ARP_FILTER) + rtwvif->ip_addr = vif->cfg.arp_addr_list[0]; + + mutex_unlock(&rtwdev->mutex); +} + +static void rtw89_ops_link_info_changed(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, + struct ieee80211_bss_conf *conf, + u64 changed) +{ + struct rtw89_dev *rtwdev = hw->priv; + struct rtw89_vif *rtwvif = (struct rtw89_vif *)vif->drv_priv; + + mutex_lock(&rtwdev->mutex); + rtw89_leave_ps_mode(rtwdev); + if (changed & BSS_CHANGED_BSSID) { ether_addr_copy(rtwvif->bssid, conf->bssid); rtw89_cam_bssid_changed(rtwdev, rtwvif); @@ -470,12 +487,6 @@ static void rtw89_ops_bss_info_changed(struct ieee80211_hw *hw, if (changed & BSS_CHANGED_CQM) rtw89_fw_h2c_set_bcn_fltr_cfg(rtwdev, vif, true); - if (changed & BSS_CHANGED_PS) - rtw89_recalc_lps(rtwdev); - - if (changed & BSS_CHANGED_ARP_FILTER) - rtwvif->ip_addr = vif->cfg.arp_addr_list[0]; - mutex_unlock(&rtwdev->mutex); } @@ -1143,7 +1154,8 @@ const struct ieee80211_ops rtw89_ops = { .change_interface = rtw89_ops_change_interface, .remove_interface = rtw89_ops_remove_interface, .configure_filter = rtw89_ops_configure_filter, - .bss_info_changed = rtw89_ops_bss_info_changed, + .vif_cfg_changed = rtw89_ops_vif_cfg_changed, + .link_info_changed = rtw89_ops_link_info_changed, .start_ap = rtw89_ops_start_ap, .stop_ap = rtw89_ops_stop_ap, .set_tim = rtw89_ops_set_tim, diff --git a/drivers/net/wireless/realtek/rtw89/pci.c b/drivers/net/wireless/realtek/rtw89/pci.c index 03bbcf9b6737c7899ad7b30d9a40854748abc37d..36cb0e351eea81525da2d09a3e609cc007024962 100644 --- a/drivers/net/wireless/realtek/rtw89/pci.c +++ b/drivers/net/wireless/realtek/rtw89/pci.c @@ -2330,21 +2330,20 @@ static void rtw89_pci_disable_eq(struct rtw89_dev *rtwdev) u32 backup_aspm; u32 phy_offset; u16 oobs_val; - u16 val16; int ret; if (rtwdev->chip->chip_id != RTL8852C) return; - backup_aspm = rtw89_read32(rtwdev, R_AX_PCIE_MIX_CFG_V1); - rtw89_write32_clr(rtwdev, R_AX_PCIE_MIX_CFG_V1, B_AX_ASPM_CTRL_MASK); - g1_oobs = rtw89_read16_mask(rtwdev, R_RAC_DIRECT_OFFSET_G1 + RAC_ANA09 * RAC_MULT, BAC_OOBS_SEL); g2_oobs = rtw89_read16_mask(rtwdev, R_RAC_DIRECT_OFFSET_G2 + RAC_ANA09 * RAC_MULT, BAC_OOBS_SEL); if (g1_oobs && g2_oobs) - goto out; + return; + + backup_aspm = rtw89_read32(rtwdev, R_AX_PCIE_MIX_CFG_V1); + rtw89_write32_clr(rtwdev, R_AX_PCIE_MIX_CFG_V1, B_AX_ASPM_CTRL_MASK); ret = rtw89_pci_get_phy_offset_by_link_speed(rtwdev, &phy_offset); if (ret) @@ -2354,15 +2353,16 @@ static void rtw89_pci_disable_eq(struct rtw89_dev *rtwdev) rtw89_write16(rtwdev, phy_offset + RAC_ANA10 * RAC_MULT, ADDR_SEL_PINOUT_DIS_VAL); rtw89_write16_set(rtwdev, phy_offset + RAC_ANA19 * RAC_MULT, B_PCIE_BIT_RD_SEL); - val16 = rtw89_read16_mask(rtwdev, phy_offset + RAC_ANA1F * RAC_MULT, - OOBS_LEVEL_MASK); - oobs_val = u16_encode_bits(val16, OOBS_SEN_MASK); + oobs_val = rtw89_read16_mask(rtwdev, phy_offset + RAC_ANA1F * RAC_MULT, + OOBS_LEVEL_MASK); - rtw89_write16(rtwdev, R_RAC_DIRECT_OFFSET_G1 + RAC_ANA03 * RAC_MULT, oobs_val); + rtw89_write16_mask(rtwdev, R_RAC_DIRECT_OFFSET_G1 + RAC_ANA03 * RAC_MULT, + OOBS_SEN_MASK, oobs_val); rtw89_write16_set(rtwdev, R_RAC_DIRECT_OFFSET_G1 + RAC_ANA09 * RAC_MULT, BAC_OOBS_SEL); - rtw89_write16(rtwdev, R_RAC_DIRECT_OFFSET_G2 + RAC_ANA03 * RAC_MULT, oobs_val); + rtw89_write16_mask(rtwdev, R_RAC_DIRECT_OFFSET_G2 + RAC_ANA03 * RAC_MULT, + OOBS_SEN_MASK, oobs_val); rtw89_write16_set(rtwdev, R_RAC_DIRECT_OFFSET_G2 + RAC_ANA09 * RAC_MULT, BAC_OOBS_SEL); @@ -2783,7 +2783,6 @@ static int rtw89_pci_ops_mac_pre_init_ax(struct rtw89_dev *rtwdev) const struct rtw89_pci_info *info = rtwdev->pci_info; int ret; - rtw89_pci_disable_eq(rtwdev); rtw89_pci_ber(rtwdev); rtw89_pci_rxdma_prefth(rtwdev); rtw89_pci_l1off_pwroff(rtwdev); @@ -4155,6 +4154,7 @@ static int __maybe_unused rtw89_pci_resume(struct device *dev) B_AX_SEL_REQ_ENTR_L1); } rtw89_pci_l2_hci_ldo(rtwdev); + rtw89_pci_disable_eq(rtwdev); rtw89_pci_filter_out(rtwdev); rtw89_pci_link_cfg(rtwdev); rtw89_pci_l1ss_cfg(rtwdev); @@ -4289,11 +4289,16 @@ int rtw89_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) goto err_clear_resource; } + rtw89_pci_disable_eq(rtwdev); rtw89_pci_filter_out(rtwdev); rtw89_pci_link_cfg(rtwdev); rtw89_pci_l1ss_cfg(rtwdev); - rtw89_core_napi_init(rtwdev); + ret = rtw89_core_napi_init(rtwdev); + if (ret) { + rtw89_err(rtwdev, "failed to init napi\n"); + goto err_clear_resource; + } ret = rtw89_pci_request_irq(rtwdev, pdev); if (ret) { diff --git a/drivers/net/wireless/realtek/rtw89/rtw8851b.c b/drivers/net/wireless/realtek/rtw89/rtw8851b.c index 87b51823244dcf805fa26dae87c53363c50ccc47..78e276f321c205da6fbfc5b1c32d47d105eb65d6 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8851b.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8851b.c @@ -2447,6 +2447,7 @@ const struct rtw89_chip_info rtw8851b_chip_info = { .dig_table = NULL, .dig_regs = &rtw8851b_dig_regs, .tssi_dbw_table = NULL, + .support_macid_num = RTW89_MAX_MAC_ID_NUM, .support_chanctx_num = 0, .support_rnr = false, .support_bands = BIT(NL80211_BAND_2GHZ) | diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852a.c b/drivers/net/wireless/realtek/rtw89/rtw8852a.c index e93cee1456bda33b99e5d41e3669ec355064f6cc..aebbceea93f84516964c69da043cb6d2c3580081 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8852a.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8852a.c @@ -2162,6 +2162,7 @@ const struct rtw89_chip_info rtw8852a_chip_info = { .dig_table = &rtw89_8852a_phy_dig_table, .dig_regs = &rtw8852a_dig_regs, .tssi_dbw_table = NULL, + .support_macid_num = RTW89_MAX_MAC_ID_NUM, .support_chanctx_num = 1, .support_rnr = false, .support_bands = BIT(NL80211_BAND_2GHZ) | diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852b.c b/drivers/net/wireless/realtek/rtw89/rtw8852b.c index d351096fa4b413f2048dbb4259b80fb650543698..517ad1a763fa417bcb9f8e9139fb0b6d4e502bfb 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8852b.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8852b.c @@ -403,6 +403,8 @@ static int rtw8852b_pwr_on_func(struct rtw89_dev *rtwdev) u32 val32; u32 ret; + rtw8852b_pwr_sps_ana(rtwdev); + rtw89_write32_clr(rtwdev, R_AX_SYS_PW_CTRL, B_AX_AFSM_WLSUS_EN | B_AX_AFSM_PCIE_SUS_EN); rtw89_write32_set(rtwdev, R_AX_SYS_PW_CTRL, B_AX_DIS_WLBT_PDNSUSEN_SOPC); @@ -530,9 +532,7 @@ static int rtw8852b_pwr_off_func(struct rtw89_dev *rtwdev) u32 val32; u32 ret; - /* Only do once during probe stage after reading efuse */ - if (!test_bit(RTW89_FLAG_PROBE_DONE, rtwdev->flags)) - rtw8852b_pwr_sps_ana(rtwdev); + rtw8852b_pwr_sps_ana(rtwdev); ret = rtw89_mac_write_xtal_si(rtwdev, XTAL_SI_ANAPAR_WL, XTAL_SI_RFC2RF, XTAL_SI_RFC2RF); @@ -2610,6 +2610,7 @@ const struct rtw89_chip_info rtw8852b_chip_info = { .dig_table = NULL, .dig_regs = &rtw8852b_dig_regs, .tssi_dbw_table = NULL, + .support_macid_num = RTW89_MAX_MAC_ID_NUM, .support_chanctx_num = 0, .support_rnr = false, .support_bands = BIT(NL80211_BAND_2GHZ) | diff --git a/drivers/net/wireless/realtek/rtw89/rtw8852c.c b/drivers/net/wireless/realtek/rtw89/rtw8852c.c index 3571b41786d7cedb1b259c65a664cb8e3cb2cdf5..544ea1d3b577fbb9238dc6166fd08314bb702ae3 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8852c.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8852c.c @@ -2941,6 +2941,7 @@ const struct rtw89_chip_info rtw8852c_chip_info = { .dig_table = NULL, .dig_regs = &rtw8852c_dig_regs, .tssi_dbw_table = &rtw89_8852c_tssi_dbw_table, + .support_macid_num = RTW89_MAX_MAC_ID_NUM, .support_chanctx_num = 2, .support_rnr = false, .support_bands = BIT(NL80211_BAND_2GHZ) | diff --git a/drivers/net/wireless/realtek/rtw89/rtw8922a.c b/drivers/net/wireless/realtek/rtw89/rtw8922a.c index 3b3ea3a7c19ae8f4b4ffccaf1ecba3765046c7a6..cc4f251caadd6648a22900d0c9067d7996b5afcd 100644 --- a/drivers/net/wireless/realtek/rtw89/rtw8922a.c +++ b/drivers/net/wireless/realtek/rtw89/rtw8922a.c @@ -2544,6 +2544,7 @@ const struct rtw89_chip_info rtw8922a_chip_info = { .dig_table = NULL, .dig_regs = &rtw8922a_dig_regs, .tssi_dbw_table = NULL, + .support_macid_num = 32, .support_chanctx_num = 2, .support_rnr = true, .support_bands = BIT(NL80211_BAND_2GHZ) | diff --git a/drivers/net/wireless/realtek/rtw89/ser.c b/drivers/net/wireless/realtek/rtw89/ser.c index 99896d85d2f81e4a5c37192a838736e0422306fa..5fc2faa9ba5a7ec58ad7ed1f8fb74570775b3680 100644 --- a/drivers/net/wireless/realtek/rtw89/ser.c +++ b/drivers/net/wireless/realtek/rtw89/ser.c @@ -308,9 +308,13 @@ static void ser_reset_vif(struct rtw89_dev *rtwdev, struct rtw89_vif *rtwvif) static void ser_sta_deinit_cam_iter(void *data, struct ieee80211_sta *sta) { - struct rtw89_vif *rtwvif = (struct rtw89_vif *)data; - struct rtw89_dev *rtwdev = rtwvif->rtwdev; + struct rtw89_vif *target_rtwvif = (struct rtw89_vif *)data; struct rtw89_sta *rtwsta = (struct rtw89_sta *)sta->drv_priv; + struct rtw89_vif *rtwvif = rtwsta->rtwvif; + struct rtw89_dev *rtwdev = rtwvif->rtwdev; + + if (rtwvif != target_rtwvif) + return; if (rtwvif->net_type == RTW89_NET_TYPE_AP_MODE || sta->tdls) rtw89_cam_deinit_addr_cam(rtwdev, &rtwsta->addr_cam); diff --git a/drivers/net/wireless/realtek/rtw89/txrx.h b/drivers/net/wireless/realtek/rtw89/txrx.h index c467a80ffa88d43e588b539df254d908ff14c7cc..3882938c089318cf3c8c66aa501bd320abafa9f9 100644 --- a/drivers/net/wireless/realtek/rtw89/txrx.h +++ b/drivers/net/wireless/realtek/rtw89/txrx.h @@ -113,6 +113,8 @@ static inline u8 rtw89_get_data_nss(struct rtw89_dev *rtwdev, u16 hw_rate) #define RTW89_TXWD_INFO0_GI_LTF GENMASK(27, 25) #define RTW89_TXWD_INFO0_DATA_RATE GENMASK(24, 16) #define RTW89_TXWD_INFO0_DATA_ER BIT(15) +#define RTW89_TXWD_INFO0_DATA_STBC BIT(12) +#define RTW89_TXWD_INFO0_DATA_LDPC BIT(11) #define RTW89_TXWD_INFO0_DISDATAFB BIT(10) #define RTW89_TXWD_INFO0_DATA_BW_ER BIT(8) #define RTW89_TXWD_INFO0_MULTIPORT_ID GENMASK(6, 4) @@ -556,6 +558,8 @@ struct rtw89_phy_sts_ie0 { #define RTW89_PHY_STS_IE01_W2_AVG_SNR GENMASK(5, 0) #define RTW89_PHY_STS_IE01_W2_EVM_MAX GENMASK(15, 8) #define RTW89_PHY_STS_IE01_W2_EVM_MIN GENMASK(23, 16) +#define RTW89_PHY_STS_IE01_W2_LDPC BIT(28) +#define RTW89_PHY_STS_IE01_W2_STBC BIT(30) enum rtw89_tx_channel { RTW89_TXCH_ACH0 = 0, diff --git a/drivers/net/wwan/iosm/iosm_ipc_devlink.c b/drivers/net/wwan/iosm/iosm_ipc_devlink.c index bef6819986e93914edcf659e7e2591e3058ba8a2..33d6342124bc339ab403c3046c32db10573aabf9 100644 --- a/drivers/net/wwan/iosm/iosm_ipc_devlink.c +++ b/drivers/net/wwan/iosm/iosm_ipc_devlink.c @@ -211,7 +211,7 @@ static int ipc_devlink_create_region(struct iosm_devlink *devlink) rc = PTR_ERR(devlink->cd_regions[i]); dev_err(devlink->dev, "Devlink region fail,err %d", rc); /* Delete previously created regions */ - for ( ; i >= 0; i--) + for (i--; i >= 0; i--) devlink_region_destroy(devlink->cd_regions[i]); goto region_create_fail; } diff --git a/drivers/nfc/virtual_ncidev.c b/drivers/nfc/virtual_ncidev.c index 590b038e449e5c61c9950ce6074fe94020915207..6b89d596ba9afe7f5d1225ec2681be6705137532 100644 --- a/drivers/nfc/virtual_ncidev.c +++ b/drivers/nfc/virtual_ncidev.c @@ -125,6 +125,10 @@ static ssize_t virtual_ncidev_write(struct file *file, kfree_skb(skb); return -EFAULT; } + if (strnlen(skb->data, count) != count) { + kfree_skb(skb); + return -EINVAL; + } nci_recv_frame(vdev->ndev, skb); return count; diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 954f850f113a1277ea2566fe98ae474fd4075f87..f5d150c62955d8e706b004ad1d9dfff93909fbd5 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -414,7 +414,15 @@ static inline void nvme_end_req_zoned(struct request *req) } } -static inline void nvme_end_req(struct request *req) +static inline void __nvme_end_req(struct request *req) +{ + nvme_end_req_zoned(req); + nvme_trace_bio_complete(req); + if (req->cmd_flags & REQ_NVME_MPATH) + nvme_mpath_end_request(req); +} + +void nvme_end_req(struct request *req) { blk_status_t status = nvme_error_status(nvme_req(req)->status); @@ -424,10 +432,7 @@ static inline void nvme_end_req(struct request *req) else nvme_log_error(req); } - nvme_end_req_zoned(req); - nvme_trace_bio_complete(req); - if (req->cmd_flags & REQ_NVME_MPATH) - nvme_mpath_end_request(req); + __nvme_end_req(req); blk_mq_end_request(req, status); } @@ -476,7 +481,7 @@ void nvme_complete_batch_req(struct request *req) { trace_nvme_complete_rq(req); nvme_cleanup_cmd(req); - nvme_end_req_zoned(req); + __nvme_end_req(req); } EXPORT_SYMBOL_GPL(nvme_complete_batch_req); @@ -673,7 +678,7 @@ static void nvme_free_ns(struct kref *kref) kfree(ns); } -static inline bool nvme_get_ns(struct nvme_ns *ns) +bool nvme_get_ns(struct nvme_ns *ns) { return kref_get_unless_zero(&ns->kref); } @@ -3679,9 +3684,10 @@ out_unlock: struct nvme_ns *nvme_find_get_ns(struct nvme_ctrl *ctrl, unsigned nsid) { struct nvme_ns *ns, *ret = NULL; + int srcu_idx; - down_read(&ctrl->namespaces_rwsem); - list_for_each_entry(ns, &ctrl->namespaces, list) { + srcu_idx = srcu_read_lock(&ctrl->srcu); + list_for_each_entry_rcu(ns, &ctrl->namespaces, list) { if (ns->head->ns_id == nsid) { if (!nvme_get_ns(ns)) continue; @@ -3691,7 +3697,7 @@ struct nvme_ns *nvme_find_get_ns(struct nvme_ctrl *ctrl, unsigned nsid) if (ns->head->ns_id > nsid) break; } - up_read(&ctrl->namespaces_rwsem); + srcu_read_unlock(&ctrl->srcu, srcu_idx); return ret; } EXPORT_SYMBOL_NS_GPL(nvme_find_get_ns, NVME_TARGET_PASSTHRU); @@ -3705,7 +3711,7 @@ static void nvme_ns_add_to_ctrl_list(struct nvme_ns *ns) list_for_each_entry_reverse(tmp, &ns->ctrl->namespaces, list) { if (tmp->head->ns_id < ns->head->ns_id) { - list_add(&ns->list, &tmp->list); + list_add_rcu(&ns->list, &tmp->list); return; } } @@ -3771,17 +3777,18 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, struct nvme_ns_info *info) if (nvme_update_ns_info(ns, info)) goto out_unlink_ns; - down_write(&ctrl->namespaces_rwsem); + mutex_lock(&ctrl->namespaces_lock); /* * Ensure that no namespaces are added to the ctrl list after the queues * are frozen, thereby avoiding a deadlock between scan and reset. */ if (test_bit(NVME_CTRL_FROZEN, &ctrl->flags)) { - up_write(&ctrl->namespaces_rwsem); + mutex_unlock(&ctrl->namespaces_lock); goto out_unlink_ns; } nvme_ns_add_to_ctrl_list(ns); - up_write(&ctrl->namespaces_rwsem); + mutex_unlock(&ctrl->namespaces_lock); + synchronize_srcu(&ctrl->srcu); nvme_get_ctrl(ctrl); if (device_add_disk(ctrl->device, ns->disk, nvme_ns_attr_groups)) @@ -3804,9 +3811,10 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, struct nvme_ns_info *info) out_cleanup_ns_from_list: nvme_put_ctrl(ctrl); - down_write(&ctrl->namespaces_rwsem); - list_del_init(&ns->list); - up_write(&ctrl->namespaces_rwsem); + mutex_lock(&ctrl->namespaces_lock); + list_del_rcu(&ns->list); + mutex_unlock(&ctrl->namespaces_lock); + synchronize_srcu(&ctrl->srcu); out_unlink_ns: mutex_lock(&ctrl->subsys->lock); list_del_rcu(&ns->siblings); @@ -3856,9 +3864,10 @@ static void nvme_ns_remove(struct nvme_ns *ns) nvme_cdev_del(&ns->cdev, &ns->cdev_device); del_gendisk(ns->disk); - down_write(&ns->ctrl->namespaces_rwsem); - list_del_init(&ns->list); - up_write(&ns->ctrl->namespaces_rwsem); + mutex_lock(&ns->ctrl->namespaces_lock); + list_del_rcu(&ns->list); + mutex_unlock(&ns->ctrl->namespaces_lock); + synchronize_srcu(&ns->ctrl->srcu); if (last_path) nvme_mpath_shutdown_disk(ns->head); @@ -3948,16 +3957,17 @@ static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl, struct nvme_ns *ns, *next; LIST_HEAD(rm_list); - down_write(&ctrl->namespaces_rwsem); + mutex_lock(&ctrl->namespaces_lock); list_for_each_entry_safe(ns, next, &ctrl->namespaces, list) { if (ns->head->ns_id > nsid) - list_move_tail(&ns->list, &rm_list); + list_splice_init_rcu(&ns->list, &rm_list, + synchronize_rcu); } - up_write(&ctrl->namespaces_rwsem); + mutex_unlock(&ctrl->namespaces_lock); + synchronize_srcu(&ctrl->srcu); list_for_each_entry_safe(ns, next, &rm_list, list) nvme_ns_remove(ns); - } static int nvme_scan_ns_list(struct nvme_ctrl *ctrl) @@ -4127,9 +4137,10 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl) /* this is a no-op when called from the controller reset handler */ nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING_NOIO); - down_write(&ctrl->namespaces_rwsem); - list_splice_init(&ctrl->namespaces, &ns_list); - up_write(&ctrl->namespaces_rwsem); + mutex_lock(&ctrl->namespaces_lock); + list_splice_init_rcu(&ctrl->namespaces, &ns_list, synchronize_rcu); + mutex_unlock(&ctrl->namespaces_lock); + synchronize_srcu(&ctrl->srcu); list_for_each_entry_safe(ns, next, &ns_list, list) nvme_ns_remove(ns); @@ -4577,6 +4588,7 @@ static void nvme_free_ctrl(struct device *dev) key_put(ctrl->tls_key); nvme_free_cels(ctrl); nvme_mpath_uninit(ctrl); + cleanup_srcu_struct(&ctrl->srcu); nvme_auth_stop(ctrl); nvme_auth_free(ctrl); __free_page(ctrl->discard_page); @@ -4609,10 +4621,15 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev, ctrl->passthru_err_log_enabled = false; clear_bit(NVME_CTRL_FAILFAST_EXPIRED, &ctrl->flags); spin_lock_init(&ctrl->lock); + mutex_init(&ctrl->namespaces_lock); + + ret = init_srcu_struct(&ctrl->srcu); + if (ret) + return ret; + mutex_init(&ctrl->scan_lock); INIT_LIST_HEAD(&ctrl->namespaces); xa_init(&ctrl->cels); - init_rwsem(&ctrl->namespaces_rwsem); ctrl->dev = dev; ctrl->ops = ops; ctrl->quirks = quirks; @@ -4692,6 +4709,7 @@ out_release_instance: out: if (ctrl->discard_page) __free_page(ctrl->discard_page); + cleanup_srcu_struct(&ctrl->srcu); return ret; } EXPORT_SYMBOL_GPL(nvme_init_ctrl); @@ -4700,22 +4718,24 @@ EXPORT_SYMBOL_GPL(nvme_init_ctrl); void nvme_mark_namespaces_dead(struct nvme_ctrl *ctrl) { struct nvme_ns *ns; + int srcu_idx; - down_read(&ctrl->namespaces_rwsem); - list_for_each_entry(ns, &ctrl->namespaces, list) + srcu_idx = srcu_read_lock(&ctrl->srcu); + list_for_each_entry_rcu(ns, &ctrl->namespaces, list) blk_mark_disk_dead(ns->disk); - up_read(&ctrl->namespaces_rwsem); + srcu_read_unlock(&ctrl->srcu, srcu_idx); } EXPORT_SYMBOL_GPL(nvme_mark_namespaces_dead); void nvme_unfreeze(struct nvme_ctrl *ctrl) { struct nvme_ns *ns; + int srcu_idx; - down_read(&ctrl->namespaces_rwsem); - list_for_each_entry(ns, &ctrl->namespaces, list) + srcu_idx = srcu_read_lock(&ctrl->srcu); + list_for_each_entry_rcu(ns, &ctrl->namespaces, list) blk_mq_unfreeze_queue(ns->queue); - up_read(&ctrl->namespaces_rwsem); + srcu_read_unlock(&ctrl->srcu, srcu_idx); clear_bit(NVME_CTRL_FROZEN, &ctrl->flags); } EXPORT_SYMBOL_GPL(nvme_unfreeze); @@ -4723,14 +4743,15 @@ EXPORT_SYMBOL_GPL(nvme_unfreeze); int nvme_wait_freeze_timeout(struct nvme_ctrl *ctrl, long timeout) { struct nvme_ns *ns; + int srcu_idx; - down_read(&ctrl->namespaces_rwsem); - list_for_each_entry(ns, &ctrl->namespaces, list) { + srcu_idx = srcu_read_lock(&ctrl->srcu); + list_for_each_entry_rcu(ns, &ctrl->namespaces, list) { timeout = blk_mq_freeze_queue_wait_timeout(ns->queue, timeout); if (timeout <= 0) break; } - up_read(&ctrl->namespaces_rwsem); + srcu_read_unlock(&ctrl->srcu, srcu_idx); return timeout; } EXPORT_SYMBOL_GPL(nvme_wait_freeze_timeout); @@ -4738,23 +4759,25 @@ EXPORT_SYMBOL_GPL(nvme_wait_freeze_timeout); void nvme_wait_freeze(struct nvme_ctrl *ctrl) { struct nvme_ns *ns; + int srcu_idx; - down_read(&ctrl->namespaces_rwsem); - list_for_each_entry(ns, &ctrl->namespaces, list) + srcu_idx = srcu_read_lock(&ctrl->srcu); + list_for_each_entry_rcu(ns, &ctrl->namespaces, list) blk_mq_freeze_queue_wait(ns->queue); - up_read(&ctrl->namespaces_rwsem); + srcu_read_unlock(&ctrl->srcu, srcu_idx); } EXPORT_SYMBOL_GPL(nvme_wait_freeze); void nvme_start_freeze(struct nvme_ctrl *ctrl) { struct nvme_ns *ns; + int srcu_idx; set_bit(NVME_CTRL_FROZEN, &ctrl->flags); - down_read(&ctrl->namespaces_rwsem); - list_for_each_entry(ns, &ctrl->namespaces, list) + srcu_idx = srcu_read_lock(&ctrl->srcu); + list_for_each_entry_rcu(ns, &ctrl->namespaces, list) blk_freeze_queue_start(ns->queue); - up_read(&ctrl->namespaces_rwsem); + srcu_read_unlock(&ctrl->srcu, srcu_idx); } EXPORT_SYMBOL_GPL(nvme_start_freeze); @@ -4797,11 +4820,12 @@ EXPORT_SYMBOL_GPL(nvme_unquiesce_admin_queue); void nvme_sync_io_queues(struct nvme_ctrl *ctrl) { struct nvme_ns *ns; + int srcu_idx; - down_read(&ctrl->namespaces_rwsem); - list_for_each_entry(ns, &ctrl->namespaces, list) + srcu_idx = srcu_read_lock(&ctrl->srcu); + list_for_each_entry_rcu(ns, &ctrl->namespaces, list) blk_sync_queue(ns->queue); - up_read(&ctrl->namespaces_rwsem); + srcu_read_unlock(&ctrl->srcu, srcu_idx); } EXPORT_SYMBOL_GPL(nvme_sync_io_queues); diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 499a8bb7cac7d13e618021f9c6b95d94d974f0bf..9d9d2a127c4ec2e846dd862ddbc09cd41359b309 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -789,15 +789,15 @@ static int nvme_dev_user_cmd(struct nvme_ctrl *ctrl, void __user *argp, bool open_for_write) { struct nvme_ns *ns; - int ret; + int ret, srcu_idx; - down_read(&ctrl->namespaces_rwsem); + srcu_idx = srcu_read_lock(&ctrl->srcu); if (list_empty(&ctrl->namespaces)) { ret = -ENOTTY; goto out_unlock; } - ns = list_first_entry(&ctrl->namespaces, struct nvme_ns, list); + ns = list_first_or_null_rcu(&ctrl->namespaces, struct nvme_ns, list); if (ns != list_last_entry(&ctrl->namespaces, struct nvme_ns, list)) { dev_warn(ctrl->device, "NVME_IOCTL_IO_CMD not supported when multiple namespaces present!\n"); @@ -807,15 +807,18 @@ static int nvme_dev_user_cmd(struct nvme_ctrl *ctrl, void __user *argp, dev_warn(ctrl->device, "using deprecated NVME_IOCTL_IO_CMD ioctl on the char device!\n"); - kref_get(&ns->kref); - up_read(&ctrl->namespaces_rwsem); + if (!nvme_get_ns(ns)) { + ret = -ENXIO; + goto out_unlock; + } + srcu_read_unlock(&ctrl->srcu, srcu_idx); ret = nvme_user_cmd(ctrl, ns, argp, 0, open_for_write); nvme_put_ns(ns); return ret; out_unlock: - up_read(&ctrl->namespaces_rwsem); + srcu_read_unlock(&ctrl->srcu, srcu_idx); return ret; } diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index d16e976ae1a4732efef32a9da1da88be6567f621..d8b6b4648eaff91aa89c92aca03907fafa97b2ad 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -118,7 +118,8 @@ void nvme_failover_req(struct request *req) blk_steal_bios(&ns->head->requeue_list, req); spin_unlock_irqrestore(&ns->head->requeue_lock, flags); - blk_mq_end_request(req, 0); + nvme_req(req)->status = 0; + nvme_end_req(req); kblockd_schedule_work(&ns->head->requeue_work); } @@ -150,16 +151,17 @@ void nvme_mpath_end_request(struct request *rq) void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl) { struct nvme_ns *ns; + int srcu_idx; - down_read(&ctrl->namespaces_rwsem); - list_for_each_entry(ns, &ctrl->namespaces, list) { + srcu_idx = srcu_read_lock(&ctrl->srcu); + list_for_each_entry_rcu(ns, &ctrl->namespaces, list) { if (!ns->head->disk) continue; kblockd_schedule_work(&ns->head->requeue_work); if (nvme_ctrl_state(ns->ctrl) == NVME_CTRL_LIVE) disk_uevent(ns->head->disk, KOBJ_CHANGE); } - up_read(&ctrl->namespaces_rwsem); + srcu_read_unlock(&ctrl->srcu, srcu_idx); } static const char *nvme_ana_state_names[] = { @@ -193,13 +195,14 @@ out: void nvme_mpath_clear_ctrl_paths(struct nvme_ctrl *ctrl) { struct nvme_ns *ns; + int srcu_idx; - down_read(&ctrl->namespaces_rwsem); - list_for_each_entry(ns, &ctrl->namespaces, list) { + srcu_idx = srcu_read_lock(&ctrl->srcu); + list_for_each_entry_rcu(ns, &ctrl->namespaces, list) { nvme_mpath_clear_current_path(ns); kblockd_schedule_work(&ns->head->requeue_work); } - up_read(&ctrl->namespaces_rwsem); + srcu_read_unlock(&ctrl->srcu, srcu_idx); } void nvme_mpath_revalidate_paths(struct nvme_ns *ns) @@ -595,7 +598,7 @@ static void nvme_mpath_set_live(struct nvme_ns *ns) int node, srcu_idx; srcu_idx = srcu_read_lock(&head->srcu); - for_each_node(node) + for_each_online_node(node) __nvme_find_path(head, node); srcu_read_unlock(&head->srcu, srcu_idx); } @@ -680,6 +683,7 @@ static int nvme_update_ana_state(struct nvme_ctrl *ctrl, u32 nr_nsids = le32_to_cpu(desc->nnsids), n = 0; unsigned *nr_change_groups = data; struct nvme_ns *ns; + int srcu_idx; dev_dbg(ctrl->device, "ANA group %d: %s.\n", le32_to_cpu(desc->grpid), @@ -691,8 +695,8 @@ static int nvme_update_ana_state(struct nvme_ctrl *ctrl, if (!nr_nsids) return 0; - down_read(&ctrl->namespaces_rwsem); - list_for_each_entry(ns, &ctrl->namespaces, list) { + srcu_idx = srcu_read_lock(&ctrl->srcu); + list_for_each_entry_rcu(ns, &ctrl->namespaces, list) { unsigned nsid; again: nsid = le32_to_cpu(desc->nsids[n]); @@ -705,7 +709,7 @@ again: if (ns->head->ns_id > nsid) goto again; } - up_read(&ctrl->namespaces_rwsem); + srcu_read_unlock(&ctrl->srcu, srcu_idx); return 0; } diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index cacc56f4bbf4423a101e2689ad9c75c85aa2afd7..f3a41133ac3f9745ee98fcc109710ee8a2988cc2 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -282,7 +282,8 @@ struct nvme_ctrl { struct blk_mq_tag_set *tagset; struct blk_mq_tag_set *admin_tagset; struct list_head namespaces; - struct rw_semaphore namespaces_rwsem; + struct mutex namespaces_lock; + struct srcu_struct srcu; struct device ctrl_device; struct device *device; /* char device */ #ifdef CONFIG_NVME_HWMON @@ -471,8 +472,6 @@ struct nvme_ns_head { u8 pi_type; u8 pi_offset; u8 guard_type; - u16 sgs; - u32 sws; #ifdef CONFIG_BLK_DEV_ZONED u64 zsze; #endif @@ -767,6 +766,7 @@ static inline bool nvme_state_terminal(struct nvme_ctrl *ctrl) } } +void nvme_end_req(struct request *req); void nvme_complete_rq(struct request *req); void nvme_complete_batch_req(struct request *req); @@ -1161,6 +1161,7 @@ void nvme_passthru_end(struct nvme_ctrl *ctrl, struct nvme_ns *ns, u32 effects, struct nvme_command *cmd, int status); struct nvme_ctrl *nvme_ctrl_from_file(struct file *file); struct nvme_ns *nvme_find_get_ns(struct nvme_ctrl *ctrl, unsigned nsid); +bool nvme_get_ns(struct nvme_ns *ns); void nvme_put_ns(struct nvme_ns *ns); static inline bool nvme_multi_css(struct nvme_ctrl *ctrl) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 710043086dffa5bdcc2e41665817760b4f3c2528..102a9fb0c65fff7dcbbba97f01d93f54d547b063 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -778,7 +778,8 @@ static blk_status_t nvme_map_data(struct nvme_dev *dev, struct request *req, struct bio_vec bv = req_bvec(req); if (!is_pci_p2pdma_page(bv.bv_page)) { - if (bv.bv_offset + bv.bv_len <= NVME_CTRL_PAGE_SIZE * 2) + if ((bv.bv_offset & (NVME_CTRL_PAGE_SIZE - 1)) + + bv.bv_len <= NVME_CTRL_PAGE_SIZE * 2) return nvme_setup_prp_simple(dev, req, &cmnd->rw, &bv); diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c index 7c43a0ad68771fad38591125dd91261b85cc83d6..bd87dfd173a4cbbd91ead9f689962b3106b90b55 100644 --- a/drivers/nvme/target/configfs.c +++ b/drivers/nvme/target/configfs.c @@ -676,10 +676,18 @@ static ssize_t nvmet_ns_enable_store(struct config_item *item, if (kstrtobool(page, &enable)) return -EINVAL; + /* + * take a global nvmet_config_sem because the disable routine has a + * window where it releases the subsys-lock, giving a chance to + * a parallel enable to concurrently execute causing the disable to + * have a misaccounting of the ns percpu_ref. + */ + down_write(&nvmet_config_sem); if (enable) ret = nvmet_ns_enable(ns); else nvmet_ns_disable(ns); + up_write(&nvmet_config_sem); return ret ? ret : count; } diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c index 2fde22323622e44366e6a194eb9af028799014e3..06f0c587f3437b8bdd57276a3929b91e2319e013 100644 --- a/drivers/nvme/target/core.c +++ b/drivers/nvme/target/core.c @@ -818,6 +818,15 @@ void nvmet_sq_destroy(struct nvmet_sq *sq) percpu_ref_exit(&sq->ref); nvmet_auth_sq_free(sq); + /* + * we must reference the ctrl again after waiting for inflight IO + * to complete. Because admin connect may have sneaked in after we + * store sq->ctrl locally, but before we killed the percpu_ref. the + * admin connect allocates and assigns sq->ctrl, which now needs a + * final ref put, as this ctrl is going away. + */ + ctrl = sq->ctrl; + if (ctrl) { /* * The teardown flow may take some time, and the host may not diff --git a/drivers/of/irq.c b/drivers/of/irq.c index 174900072c18cd7c255fe603c68966a1ca7ea4fb..462375b293e47577a68ab79a51d00101d59b1e9d 100644 --- a/drivers/of/irq.c +++ b/drivers/of/irq.c @@ -25,6 +25,8 @@ #include #include +#include "of_private.h" + /** * irq_of_parse_and_map - Parse and map an interrupt into linux virq space * @dev: Device node of the device whose interrupt is to be mapped @@ -96,6 +98,57 @@ static const char * const of_irq_imap_abusers[] = { NULL, }; +const __be32 *of_irq_parse_imap_parent(const __be32 *imap, int len, struct of_phandle_args *out_irq) +{ + u32 intsize, addrsize; + struct device_node *np; + + /* Get the interrupt parent */ + if (of_irq_workarounds & OF_IMAP_NO_PHANDLE) + np = of_node_get(of_irq_dflt_pic); + else + np = of_find_node_by_phandle(be32_to_cpup(imap)); + imap++; + + /* Check if not found */ + if (!np) { + pr_debug(" -> imap parent not found !\n"); + return NULL; + } + + /* Get #interrupt-cells and #address-cells of new parent */ + if (of_property_read_u32(np, "#interrupt-cells", + &intsize)) { + pr_debug(" -> parent lacks #interrupt-cells!\n"); + of_node_put(np); + return NULL; + } + if (of_property_read_u32(np, "#address-cells", + &addrsize)) + addrsize = 0; + + pr_debug(" -> intsize=%d, addrsize=%d\n", + intsize, addrsize); + + /* Check for malformed properties */ + if (WARN_ON(addrsize + intsize > MAX_PHANDLE_ARGS) + || (len < (addrsize + intsize))) { + of_node_put(np); + return NULL; + } + + pr_debug(" -> imaplen=%d\n", len); + + imap += addrsize + intsize; + + out_irq->np = np; + for (int i = 0; i < intsize; i++) + out_irq->args[i] = be32_to_cpup(imap - intsize + i); + out_irq->args_count = intsize; + + return imap; +} + /** * of_irq_parse_raw - Low level interrupt tree parsing * @addr: address specifier (start of "reg" property of the device) in be32 format @@ -112,12 +165,12 @@ static const char * const of_irq_imap_abusers[] = { */ int of_irq_parse_raw(const __be32 *addr, struct of_phandle_args *out_irq) { - struct device_node *ipar, *tnode, *old = NULL, *newpar = NULL; + struct device_node *ipar, *tnode, *old = NULL; __be32 initial_match_array[MAX_PHANDLE_ARGS]; const __be32 *match_array = initial_match_array; - const __be32 *tmp, *imap, *imask, dummy_imask[] = { [0 ... MAX_PHANDLE_ARGS] = cpu_to_be32(~0) }; - u32 intsize = 1, addrsize, newintsize = 0, newaddrsize = 0; - int imaplen, match, i, rc = -EINVAL; + const __be32 *tmp, dummy_imask[] = { [0 ... MAX_PHANDLE_ARGS] = cpu_to_be32(~0) }; + u32 intsize = 1, addrsize; + int i, rc = -EINVAL; #ifdef DEBUG of_print_phandle_args("of_irq_parse_raw: ", out_irq); @@ -176,6 +229,9 @@ int of_irq_parse_raw(const __be32 *addr, struct of_phandle_args *out_irq) /* Now start the actual "proper" walk of the interrupt tree */ while (ipar != NULL) { + int imaplen, match; + const __be32 *imap, *oldimap, *imask; + struct device_node *newpar; /* * Now check if cursor is an interrupt-controller and * if it is then we are done, unless there is an @@ -216,7 +272,7 @@ int of_irq_parse_raw(const __be32 *addr, struct of_phandle_args *out_irq) /* Parse interrupt-map */ match = 0; - while (imaplen > (addrsize + intsize + 1) && !match) { + while (imaplen > (addrsize + intsize + 1)) { /* Compare specifiers */ match = 1; for (i = 0; i < (addrsize + intsize); i++, imaplen--) @@ -224,48 +280,17 @@ int of_irq_parse_raw(const __be32 *addr, struct of_phandle_args *out_irq) pr_debug(" -> match=%d (imaplen=%d)\n", match, imaplen); - /* Get the interrupt parent */ - if (of_irq_workarounds & OF_IMAP_NO_PHANDLE) - newpar = of_node_get(of_irq_dflt_pic); - else - newpar = of_find_node_by_phandle(be32_to_cpup(imap)); - imap++; - --imaplen; - - /* Check if not found */ - if (newpar == NULL) { - pr_debug(" -> imap parent not found !\n"); - goto fail; - } - - if (!of_device_is_available(newpar)) - match = 0; - - /* Get #interrupt-cells and #address-cells of new - * parent - */ - if (of_property_read_u32(newpar, "#interrupt-cells", - &newintsize)) { - pr_debug(" -> parent lacks #interrupt-cells!\n"); - goto fail; - } - if (of_property_read_u32(newpar, "#address-cells", - &newaddrsize)) - newaddrsize = 0; - - pr_debug(" -> newintsize=%d, newaddrsize=%d\n", - newintsize, newaddrsize); - - /* Check for malformed properties */ - if (WARN_ON(newaddrsize + newintsize > MAX_PHANDLE_ARGS) - || (imaplen < (newaddrsize + newintsize))) { - rc = -EFAULT; + oldimap = imap; + imap = of_irq_parse_imap_parent(oldimap, imaplen, out_irq); + if (!imap) goto fail; - } - imap += newaddrsize + newintsize; - imaplen -= newaddrsize + newintsize; + match &= of_device_is_available(out_irq->np); + if (match) + break; + of_node_put(out_irq->np); + imaplen -= imap - oldimap; pr_debug(" -> imaplen=%d\n", imaplen); } if (!match) { @@ -287,11 +312,11 @@ int of_irq_parse_raw(const __be32 *addr, struct of_phandle_args *out_irq) * Successfully parsed an interrupt-map translation; copy new * interrupt specifier into the out_irq structure */ - match_array = imap - newaddrsize - newintsize; - for (i = 0; i < newintsize; i++) - out_irq->args[i] = be32_to_cpup(imap - newintsize + i); - out_irq->args_count = intsize = newintsize; - addrsize = newaddrsize; + match_array = oldimap + 1; + + newpar = out_irq->np; + intsize = out_irq->args_count; + addrsize = (imap - match_array) - intsize; if (ipar == newpar) { pr_debug("%pOF interrupt-map entry to self\n", ipar); @@ -300,7 +325,6 @@ int of_irq_parse_raw(const __be32 *addr, struct of_phandle_args *out_irq) skiplevel: /* Iterate again with new parent */ - out_irq->np = newpar; pr_debug(" -> new parent: %pOF\n", newpar); of_node_put(ipar); ipar = newpar; @@ -310,7 +334,6 @@ int of_irq_parse_raw(const __be32 *addr, struct of_phandle_args *out_irq) fail: of_node_put(ipar); - of_node_put(newpar); return rc; } diff --git a/drivers/of/of_private.h b/drivers/of/of_private.h index 94fc0aa07af9e383a1e0d844ac620f26a025bfd1..04aa2a91f851acd9b2da47c5b53edb025b9a1f8f 100644 --- a/drivers/of/of_private.h +++ b/drivers/of/of_private.h @@ -159,6 +159,9 @@ extern void __of_sysfs_remove_bin_file(struct device_node *np, extern int of_bus_n_addr_cells(struct device_node *np); extern int of_bus_n_size_cells(struct device_node *np); +const __be32 *of_irq_parse_imap_parent(const __be32 *imap, int len, + struct of_phandle_args *out_irq); + struct bus_dma_region; #if defined(CONFIG_OF_ADDRESS) && defined(CONFIG_HAS_DMA) int of_dma_get_range(struct device_node *np, diff --git a/drivers/of/of_test.c b/drivers/of/of_test.c index a9301d293f014cb2e088b1826397c2001331866e..c85a258bc6ae64b83b76fdfb99533914b02ff8b3 100644 --- a/drivers/of/of_test.c +++ b/drivers/of/of_test.c @@ -54,4 +54,5 @@ static struct kunit_suite of_dtb_suite = { kunit_test_suites( &of_dtb_suite, ); +MODULE_DESCRIPTION("KUnit tests for OF APIs"); MODULE_LICENSE("GPL"); diff --git a/drivers/of/property.c b/drivers/of/property.c index 1c83e68f805baa50af98da953a6434740e8892c9..164d77cb944585d6da7d4931a9fe5a121892841e 100644 --- a/drivers/of/property.c +++ b/drivers/of/property.c @@ -1306,10 +1306,10 @@ static struct device_node *parse_interrupts(struct device_node *np, static struct device_node *parse_interrupt_map(struct device_node *np, const char *prop_name, int index) { - const __be32 *imap, *imap_end, *addr; + const __be32 *imap, *imap_end; struct of_phandle_args sup_args; u32 addrcells, intcells; - int i, imaplen; + int imaplen; if (!IS_ENABLED(CONFIG_OF_IRQ)) return NULL; @@ -1322,33 +1322,23 @@ static struct device_node *parse_interrupt_map(struct device_node *np, addrcells = of_bus_n_addr_cells(np); imap = of_get_property(np, "interrupt-map", &imaplen); - if (!imap || imaplen <= (addrcells + intcells)) + imaplen /= sizeof(*imap); + if (!imap) return NULL; - imap_end = imap + imaplen; - while (imap < imap_end) { - addr = imap; - imap += addrcells; + imap_end = imap + imaplen; - sup_args.np = np; - sup_args.args_count = intcells; - for (i = 0; i < intcells; i++) - sup_args.args[i] = be32_to_cpu(imap[i]); - imap += intcells; + for (int i = 0; imap + addrcells + intcells + 1 < imap_end; i++) { + imap += addrcells + intcells; - /* - * Upon success, the function of_irq_parse_raw() returns - * interrupt controller DT node pointer in sup_args.np. - */ - if (of_irq_parse_raw(addr, &sup_args)) + imap = of_irq_parse_imap_parent(imap, imap_end - imap, &sup_args); + if (!imap) return NULL; - if (!index) + if (i == index) return sup_args.np; of_node_put(sup_args.np); - imap += sup_args.args_count + 1; - index--; } return NULL; diff --git a/drivers/platform/x86/Kconfig b/drivers/platform/x86/Kconfig index 0ec952b5d03e770fbc1c6cf6427e9cf535762359..1953317541eab8354d38e46282ef106313466c45 100644 --- a/drivers/platform/x86/Kconfig +++ b/drivers/platform/x86/Kconfig @@ -515,6 +515,7 @@ config THINKPAD_ACPI select NVRAM select NEW_LEDS select LEDS_CLASS + select INPUT_SPARSEKMAP help This is a driver for the IBM and Lenovo ThinkPad laptops. It adds support for Fn-Fx key combinations, Bluetooth control, video diff --git a/drivers/platform/x86/intel/speed_select_if/isst_tpmi_core.c b/drivers/platform/x86/intel/speed_select_if/isst_tpmi_core.c index 7bac7841ff0abaa101252428601b23752beace07..7fa360073f6ef48c2d6607c7f75c022ce9dbdf4e 100644 --- a/drivers/platform/x86/intel/speed_select_if/isst_tpmi_core.c +++ b/drivers/platform/x86/intel/speed_select_if/isst_tpmi_core.c @@ -1610,8 +1610,8 @@ void tpmi_sst_dev_remove(struct auxiliary_device *auxdev) tpmi_sst->partition_mask_current &= ~BIT(plat_info->partition); /* Free the package instance when the all partitions are removed */ if (!tpmi_sst->partition_mask_current) { - kfree(tpmi_sst); isst_common.sst_inst[tpmi_sst->package_id] = NULL; + kfree(tpmi_sst); } mutex_unlock(&isst_tpmi_dev_lock); } diff --git a/drivers/platform/x86/touchscreen_dmi.c b/drivers/platform/x86/touchscreen_dmi.c index c6a10ec2c83f666b936247abbc17fe4bf2d87304..2d9ca2292ea19d1d53e2bd50d50174571438d35f 100644 --- a/drivers/platform/x86/touchscreen_dmi.c +++ b/drivers/platform/x86/touchscreen_dmi.c @@ -9,10 +9,13 @@ */ #include +#include #include #include #include #include +#include +#include #include #include #include @@ -897,6 +900,22 @@ static const struct ts_dmi_data schneider_sct101ctm_data = { .properties = schneider_sct101ctm_props, }; +static const struct property_entry globalspace_solt_ivw116_props[] = { + PROPERTY_ENTRY_U32("touchscreen-min-x", 7), + PROPERTY_ENTRY_U32("touchscreen-min-y", 22), + PROPERTY_ENTRY_U32("touchscreen-size-x", 1723), + PROPERTY_ENTRY_U32("touchscreen-size-y", 1077), + PROPERTY_ENTRY_STRING("firmware-name", "gsl1680-globalspace-solt-ivw116.fw"), + PROPERTY_ENTRY_U32("silead,max-fingers", 10), + PROPERTY_ENTRY_BOOL("silead,home-button"), + { } +}; + +static const struct ts_dmi_data globalspace_solt_ivw116_data = { + .acpi_name = "MSSL1680:00", + .properties = globalspace_solt_ivw116_props, +}; + static const struct property_entry techbite_arc_11_6_props[] = { PROPERTY_ENTRY_U32("touchscreen-min-x", 5), PROPERTY_ENTRY_U32("touchscreen-min-y", 7), @@ -1385,6 +1404,17 @@ const struct dmi_system_id touchscreen_dmi_table[] = { DMI_MATCH(DMI_BIOS_DATE, "04/24/2018"), }, }, + { + /* Jumper EZpad 6s Pro */ + .driver_data = (void *)&jumper_ezpad_6_pro_b_data, + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "Jumper"), + DMI_MATCH(DMI_PRODUCT_NAME, "Ezpad"), + /* Above matches are too generic, add bios match */ + DMI_MATCH(DMI_BIOS_VERSION, "E.WSA116_8.E1.042.bin"), + DMI_MATCH(DMI_BIOS_DATE, "01/08/2020"), + }, + }, { /* Jumper EZpad 6 m4 */ .driver_data = (void *)&jumper_ezpad_6_m4_data, @@ -1624,6 +1654,15 @@ const struct dmi_system_id touchscreen_dmi_table[] = { DMI_MATCH(DMI_PRODUCT_NAME, "SCT101CTM"), }, }, + { + /* GlobalSpace SoLT IVW 11.6" */ + .driver_data = (void *)&globalspace_solt_ivw116_data, + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "Globalspace Tech Pvt Ltd"), + DMI_MATCH(DMI_PRODUCT_NAME, "SolTIVW"), + DMI_MATCH(DMI_PRODUCT_SKU, "PN20170413488"), + }, + }, { /* Techbite Arc 11.6 */ .driver_data = (void *)&techbite_arc_11_6_data, @@ -1817,7 +1856,7 @@ const struct dmi_system_id touchscreen_dmi_table[] = { { } }; -static const struct ts_dmi_data *ts_data; +static struct ts_dmi_data *ts_data; static void ts_dmi_add_props(struct i2c_client *client) { @@ -1852,6 +1891,64 @@ static int ts_dmi_notifier_call(struct notifier_block *nb, return 0; } +#define MAX_CMDLINE_PROPS 16 + +static struct property_entry ts_cmdline_props[MAX_CMDLINE_PROPS + 1]; + +static struct ts_dmi_data ts_cmdline_data = { + .properties = ts_cmdline_props, +}; + +static int __init ts_parse_props(char *str) +{ + /* Save the original str to show it on syntax errors */ + char orig_str[256]; + char *name, *value; + u32 u32val; + int i, ret; + + strscpy(orig_str, str, sizeof(orig_str)); + + /* + * str is part of the static_command_line from init/main.c and poking + * holes in that by writing 0 to it is allowed, as is taking long + * lasting references to it. + */ + ts_cmdline_data.acpi_name = strsep(&str, ":"); + + for (i = 0; i < MAX_CMDLINE_PROPS; i++) { + name = strsep(&str, ":"); + if (!name || !name[0]) + break; + + /* Replace '=' with 0 and make value point past '=' or NULL */ + value = name; + strsep(&value, "="); + if (!value) { + ts_cmdline_props[i] = PROPERTY_ENTRY_BOOL(name); + } else if (isdigit(value[0])) { + ret = kstrtou32(value, 0, &u32val); + if (ret) + goto syntax_error; + + ts_cmdline_props[i] = PROPERTY_ENTRY_U32(name, u32val); + } else { + ts_cmdline_props[i] = PROPERTY_ENTRY_STRING(name, value); + } + } + + if (!i || str) + goto syntax_error; + + ts_data = &ts_cmdline_data; + return 1; + +syntax_error: + pr_err("Invalid '%s' value for 'i2c_touchscreen_props='\n", orig_str); + return 1; /* "i2c_touchscreen_props=" is still a known parameter */ +} +__setup("i2c_touchscreen_props=", ts_parse_props); + static struct notifier_block ts_dmi_notifier = { .notifier_call = ts_dmi_notifier_call, }; @@ -1859,13 +1956,25 @@ static struct notifier_block ts_dmi_notifier = { static int __init ts_dmi_init(void) { const struct dmi_system_id *dmi_id; + struct ts_dmi_data *ts_data_dmi; int error; dmi_id = dmi_first_match(touchscreen_dmi_table); - if (!dmi_id) + ts_data_dmi = dmi_id ? dmi_id->driver_data : NULL; + + if (ts_data) { + /* + * Kernel cmdline provided data takes precedence, copy over + * DMI efi_embedded_fw info if available. + */ + if (ts_data_dmi) + ts_data->embedded_fw = ts_data_dmi->embedded_fw; + } else if (ts_data_dmi) { + ts_data = ts_data_dmi; + } else { return 0; /* Not an error */ + } - ts_data = dmi_id->driver_data; /* Some dmi table entries only provide an efi_embedded_fw_desc */ if (!ts_data->properties) return 0; diff --git a/drivers/platform/x86/x86-android-tablets/Kconfig b/drivers/platform/x86/x86-android-tablets/Kconfig index 6603461d427300ab07e80c3f76698c81cbe12856..b591419de80c301dc22c625dc82c00e1522f8035 100644 --- a/drivers/platform/x86/x86-android-tablets/Kconfig +++ b/drivers/platform/x86/x86-android-tablets/Kconfig @@ -6,6 +6,8 @@ config X86_ANDROID_TABLETS tristate "X86 Android tablet support" depends on I2C && SPI && SERIAL_DEV_BUS && ACPI && EFI && GPIOLIB && PMIC_OPREGION + select NEW_LEDS + select LEDS_CLASS help X86 tablets which ship with Android as (part of) the factory image typically have various problems with their DSDTs. The factory kernels diff --git a/drivers/pmdomain/imx/gpcv2.c b/drivers/pmdomain/imx/gpcv2.c index 4b828d74a6064c0d8ba8a1e153c6c3bbea678ddd..856eaac0ec140d9668a814dbf84b8e649cb26abf 100644 --- a/drivers/pmdomain/imx/gpcv2.c +++ b/drivers/pmdomain/imx/gpcv2.c @@ -393,6 +393,17 @@ static int imx_pgc_power_up(struct generic_pm_domain *genpd) * automatically there. Just add a delay and suppose the handshake finish * after that. */ + + /* + * For some BLK-CTL module (eg. AudioMix on i.MX8MP) doesn't have BUS + * clk-en bit, it is better to add delay here, as the BLK-CTL module + * doesn't need to care about how it is powered up. + * + * regmap_read_bypassed() is to make sure the above write IO transaction + * already reaches target before udelay() + */ + regmap_read_bypassed(domain->regmap, domain->regs->hsk, ®_val); + udelay(5); } /* Disable reset clocks for all devices in the domain */ diff --git a/drivers/pnp/base.h b/drivers/pnp/base.h index e74a0f6a31572bf53c87f4dfbb34cf7d76208f8f..4e80273dfb1ec9c90053faee5e1330875f3508fe 100644 --- a/drivers/pnp/base.h +++ b/drivers/pnp/base.h @@ -6,6 +6,7 @@ extern struct mutex pnp_lock; extern const struct attribute_group *pnp_dev_groups[]; +extern const struct bus_type pnp_bus_type; int pnp_register_protocol(struct pnp_protocol *protocol); void pnp_unregister_protocol(struct pnp_protocol *protocol); diff --git a/drivers/pnp/driver.c b/drivers/pnp/driver.c index 0a5d0d8befa8408d02b3eaaf278d6a936a83e102..3483e52e3a81d13d6dbc3e68882fa8b7535e03bf 100644 --- a/drivers/pnp/driver.c +++ b/drivers/pnp/driver.c @@ -266,6 +266,12 @@ const struct bus_type pnp_bus_type = { .dev_groups = pnp_dev_groups, }; +bool dev_is_pnp(const struct device *dev) +{ + return dev->bus == &pnp_bus_type; +} +EXPORT_SYMBOL_GPL(dev_is_pnp); + int pnp_register_driver(struct pnp_driver *drv) { drv->driver.name = drv->name; diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c index 7513018c9f9ac72d5c1b0055b55ae9ff36e710b0..2067b0120d083d868e0f8fee2119fb0a4cd6a2b4 100644 --- a/drivers/ptp/ptp_chardev.c +++ b/drivers/ptp/ptp_chardev.c @@ -85,7 +85,8 @@ int ptp_set_pinfunc(struct ptp_clock *ptp, unsigned int pin, } if (info->verify(info, pin, func, chan)) { - pr_err("driver cannot use function %u on pin %u\n", func, chan); + pr_err("driver cannot use function %u and channel %u on pin %u\n", + func, chan, pin); return -EOPNOTSUPP; } diff --git a/drivers/regulator/rtq2208-regulator.c b/drivers/regulator/rtq2208-regulator.c index b90e53d922d6d186dc05b5b009a903359425f677..c31b6dc3229cabbe436f6efb601be482ce62e06b 100644 --- a/drivers/regulator/rtq2208-regulator.c +++ b/drivers/regulator/rtq2208-regulator.c @@ -228,6 +228,11 @@ static const struct regulator_ops rtq2208_regulator_ldo_ops = { .set_suspend_disable = rtq2208_set_suspend_disable, }; +static struct of_regulator_match rtq2208_ldo_match[] = { + {.name = "ldo2", }, + {.name = "ldo1", }, +}; + static unsigned int rtq2208_of_map_mode(unsigned int mode) { switch (mode) { @@ -322,8 +327,7 @@ static irqreturn_t rtq2208_irq_handler(int irqno, void *devid) return IRQ_HANDLED; } -static int rtq2208_of_get_fixed_voltage(struct device *dev, - struct of_regulator_match *rtq2208_ldo_match, int n_fixed) +static int rtq2208_of_get_ldo_dvs_ability(struct device *dev) { struct device_node *np; struct of_regulator_match *match; @@ -338,14 +342,14 @@ static int rtq2208_of_get_fixed_voltage(struct device *dev, if (!np) np = dev->of_node; - ret = of_regulator_match(dev, np, rtq2208_ldo_match, n_fixed); + ret = of_regulator_match(dev, np, rtq2208_ldo_match, ARRAY_SIZE(rtq2208_ldo_match)); of_node_put(np); if (ret < 0) return ret; - for (i = 0; i < n_fixed; i++) { + for (i = 0; i < ARRAY_SIZE(rtq2208_ldo_match); i++) { match = rtq2208_ldo_match + i; init_data = match->init_data; rdesc = (struct rtq2208_regulator_desc *)match->driver_data; @@ -388,8 +392,7 @@ static const struct linear_range rtq2208_vout_range[] = { REGULATOR_LINEAR_RANGE(1310000, 181, 255, 10000), }; -static void rtq2208_init_regulator_desc(struct rtq2208_regulator_desc *rdesc, int mtp_sel, - int idx, struct of_regulator_match *rtq2208_ldo_match, int *ldo_idx) +static void rtq2208_init_regulator_desc(struct rtq2208_regulator_desc *rdesc, int mtp_sel, int idx) { struct regulator_desc *desc; static const struct { @@ -461,8 +464,7 @@ static void rtq2208_init_regulator_desc(struct rtq2208_regulator_desc *rdesc, in static int rtq2208_parse_regulator_dt_data(int n_regulator, const unsigned int *regulator_idx_table, struct rtq2208_regulator_desc *rdesc[RTQ2208_LDO_MAX], struct device *dev) { - struct of_regulator_match rtq2208_ldo_match[2]; - int mtp_sel, ret, i, idx, ldo_idx = 0; + int mtp_sel, i, idx, ret; /* get mtp_sel0 or mtp_sel1 */ mtp_sel = device_property_read_bool(dev, "richtek,mtp-sel-high"); @@ -474,7 +476,7 @@ static int rtq2208_parse_regulator_dt_data(int n_regulator, const unsigned int * if (!rdesc[i]) return -ENOMEM; - rtq2208_init_regulator_desc(rdesc[i], mtp_sel, idx, rtq2208_ldo_match, &ldo_idx); + rtq2208_init_regulator_desc(rdesc[i], mtp_sel, idx); /* init ldo dvs ability */ if (idx >= RTQ2208_LDO2) @@ -482,7 +484,7 @@ static int rtq2208_parse_regulator_dt_data(int n_regulator, const unsigned int * } /* init ldo fixed_uV */ - ret = rtq2208_of_get_fixed_voltage(dev, rtq2208_ldo_match, ldo_idx); + ret = rtq2208_of_get_ldo_dvs_ability(dev); if (ret) return dev_err_probe(dev, ret, "Failed to get ldo fixed_uV\n"); diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index 258647fc6bddb42b6c484bfb73055f6e40cb1c2a..1320e06727df19188e285ec99276b0f66a0704ab 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -4774,7 +4774,7 @@ _base_display_ioc_capabilities(struct MPT3SAS_ADAPTER *ioc) char desc[17] = {0}; u32 iounit_pg1_flags; - strscpy(desc, ioc->manu_pg0.ChipName, sizeof(desc)); + memtostr(desc, ioc->manu_pg0.ChipName); ioc_info(ioc, "%s: FWVersion(%02d.%02d.%02d.%02d), ChipRevision(0x%02x)\n", desc, (ioc->facts.FWVersion.Word & 0xFF000000) >> 24, diff --git a/drivers/scsi/mpt3sas/mpt3sas_transport.c b/drivers/scsi/mpt3sas/mpt3sas_transport.c index 76f9a91771985baed3ede562c19c5df830696608..d84413b77d84999d5c3d43490575122123794d5d 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_transport.c +++ b/drivers/scsi/mpt3sas/mpt3sas_transport.c @@ -458,17 +458,13 @@ _transport_expander_report_manufacture(struct MPT3SAS_ADAPTER *ioc, goto out; manufacture_reply = data_out + sizeof(struct rep_manu_request); - strscpy(edev->vendor_id, manufacture_reply->vendor_id, - sizeof(edev->vendor_id)); - strscpy(edev->product_id, manufacture_reply->product_id, - sizeof(edev->product_id)); - strscpy(edev->product_rev, manufacture_reply->product_rev, - sizeof(edev->product_rev)); + memtostr(edev->vendor_id, manufacture_reply->vendor_id); + memtostr(edev->product_id, manufacture_reply->product_id); + memtostr(edev->product_rev, manufacture_reply->product_rev); edev->level = manufacture_reply->sas_format & 1; if (edev->level) { - strscpy(edev->component_vendor_id, - manufacture_reply->component_vendor_id, - sizeof(edev->component_vendor_id)); + memtostr(edev->component_vendor_id, + manufacture_reply->component_vendor_id); tmp = (u8 *)&manufacture_reply->component_id; edev->component_id = tmp[0] << 8 | tmp[1]; edev->component_revision_id = diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index 332eb9dac22d91e66e1ee768f66d1f7e83713470..f6c822c9cbd2d3ef0cfdf584bc307cffa6ace353 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -3700,8 +3700,10 @@ static int sd_revalidate_disk(struct gendisk *disk) */ if (sdkp->first_scan || q->limits.max_sectors > q->limits.max_dev_sectors || - q->limits.max_sectors > q->limits.max_hw_sectors) + q->limits.max_sectors > q->limits.max_hw_sectors) { q->limits.max_sectors = rw_max; + q->limits.max_user_sectors = rw_max; + } sdkp->first_scan = 0; diff --git a/drivers/spi/spi-cadence-xspi.c b/drivers/spi/spi-cadence-xspi.c index 2209e9fc378fa94998f0c2fc152d5d472263e146..2e3eacd46b7232e91508453594accb2bb912ab96 100644 --- a/drivers/spi/spi-cadence-xspi.c +++ b/drivers/spi/spi-cadence-xspi.c @@ -145,6 +145,9 @@ #define CDNS_XSPI_STIG_DONE_FLAG BIT(0) #define CDNS_XSPI_TRD_STATUS 0x0104 +#define MODE_NO_OF_BYTES GENMASK(25, 24) +#define MODEBYTES_COUNT 1 + /* Helper macros for filling command registers */ #define CDNS_XSPI_CMD_FLD_P1_INSTR_CMD_1(op, data_phase) ( \ FIELD_PREP(CDNS_XSPI_CMD_INSTR_TYPE, (data_phase) ? \ @@ -157,9 +160,10 @@ FIELD_PREP(CDNS_XSPI_CMD_P1_R2_ADDR3, ((op)->addr.val >> 24) & 0xFF) | \ FIELD_PREP(CDNS_XSPI_CMD_P1_R2_ADDR4, ((op)->addr.val >> 32) & 0xFF)) -#define CDNS_XSPI_CMD_FLD_P1_INSTR_CMD_3(op) ( \ +#define CDNS_XSPI_CMD_FLD_P1_INSTR_CMD_3(op, modebytes) ( \ FIELD_PREP(CDNS_XSPI_CMD_P1_R3_ADDR5, ((op)->addr.val >> 40) & 0xFF) | \ FIELD_PREP(CDNS_XSPI_CMD_P1_R3_CMD, (op)->cmd.opcode) | \ + FIELD_PREP(MODE_NO_OF_BYTES, modebytes) | \ FIELD_PREP(CDNS_XSPI_CMD_P1_R3_NUM_ADDR_BYTES, (op)->addr.nbytes)) #define CDNS_XSPI_CMD_FLD_P1_INSTR_CMD_4(op, chipsel) ( \ @@ -173,12 +177,12 @@ #define CDNS_XSPI_CMD_FLD_DSEQ_CMD_2(op) \ FIELD_PREP(CDNS_XSPI_CMD_DSEQ_R2_DCNT_L, (op)->data.nbytes & 0xFFFF) -#define CDNS_XSPI_CMD_FLD_DSEQ_CMD_3(op) ( \ +#define CDNS_XSPI_CMD_FLD_DSEQ_CMD_3(op, dummybytes) ( \ FIELD_PREP(CDNS_XSPI_CMD_DSEQ_R3_DCNT_H, \ ((op)->data.nbytes >> 16) & 0xffff) | \ FIELD_PREP(CDNS_XSPI_CMD_DSEQ_R3_NUM_OF_DUMMY, \ (op)->dummy.buswidth != 0 ? \ - (((op)->dummy.nbytes * 8) / (op)->dummy.buswidth) : \ + (((dummybytes) * 8) / (op)->dummy.buswidth) : \ 0)) #define CDNS_XSPI_CMD_FLD_DSEQ_CMD_4(op, chipsel) ( \ @@ -351,6 +355,7 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi, u32 cmd_regs[6]; u32 cmd_status; int ret; + int dummybytes = op->dummy.nbytes; ret = cdns_xspi_wait_for_controller_idle(cdns_xspi); if (ret < 0) @@ -365,7 +370,12 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi, memset(cmd_regs, 0, sizeof(cmd_regs)); cmd_regs[1] = CDNS_XSPI_CMD_FLD_P1_INSTR_CMD_1(op, data_phase); cmd_regs[2] = CDNS_XSPI_CMD_FLD_P1_INSTR_CMD_2(op); - cmd_regs[3] = CDNS_XSPI_CMD_FLD_P1_INSTR_CMD_3(op); + if (dummybytes != 0) { + cmd_regs[3] = CDNS_XSPI_CMD_FLD_P1_INSTR_CMD_3(op, 1); + dummybytes--; + } else { + cmd_regs[3] = CDNS_XSPI_CMD_FLD_P1_INSTR_CMD_3(op, 0); + } cmd_regs[4] = CDNS_XSPI_CMD_FLD_P1_INSTR_CMD_4(op, cdns_xspi->cur_cs); @@ -375,7 +385,7 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi, cmd_regs[0] = CDNS_XSPI_STIG_DONE_FLAG; cmd_regs[1] = CDNS_XSPI_CMD_FLD_DSEQ_CMD_1(op); cmd_regs[2] = CDNS_XSPI_CMD_FLD_DSEQ_CMD_2(op); - cmd_regs[3] = CDNS_XSPI_CMD_FLD_DSEQ_CMD_3(op); + cmd_regs[3] = CDNS_XSPI_CMD_FLD_DSEQ_CMD_3(op, dummybytes); cmd_regs[4] = CDNS_XSPI_CMD_FLD_DSEQ_CMD_4(op, cdns_xspi->cur_cs); diff --git a/drivers/spi/spi-stm32.c b/drivers/spi/spi-stm32.c index 4a68abcdcc35351dc1fab0a79b4180387caaa09b..4c4ff074e3f6f84caa2c9aa617f9808405093b04 100644 --- a/drivers/spi/spi-stm32.c +++ b/drivers/spi/spi-stm32.c @@ -1016,8 +1016,10 @@ end_irq: static irqreturn_t stm32fx_spi_irq_thread(int irq, void *dev_id) { struct spi_controller *ctrl = dev_id; + struct stm32_spi *spi = spi_controller_get_devdata(ctrl); spi_finalize_current_transfer(ctrl); + stm32fx_spi_disable(spi); return IRQ_HANDLED; } @@ -1055,7 +1057,7 @@ static irqreturn_t stm32h7_spi_irq_thread(int irq, void *dev_id) mask |= STM32H7_SPI_SR_TXP | STM32H7_SPI_SR_RXP; if (!(sr & mask)) { - dev_warn(spi->dev, "spurious IT (sr=0x%08x, ier=0x%08x)\n", + dev_vdbg(spi->dev, "spurious IT (sr=0x%08x, ier=0x%08x)\n", sr, ier); spin_unlock_irqrestore(&spi->lock, flags); return IRQ_NONE; @@ -1185,8 +1187,6 @@ static int stm32_spi_prepare_msg(struct spi_controller *ctrl, ~clrb) | setb, spi->base + spi->cfg->regs->cpol.reg); - stm32_spi_enable(spi); - spin_unlock_irqrestore(&spi->lock, flags); return 0; @@ -1204,6 +1204,7 @@ static void stm32fx_spi_dma_tx_cb(void *data) if (spi->cur_comm == SPI_SIMPLEX_TX || spi->cur_comm == SPI_3WIRE_TX) { spi_finalize_current_transfer(spi->ctrl); + stm32fx_spi_disable(spi); } } @@ -1218,6 +1219,7 @@ static void stm32_spi_dma_rx_cb(void *data) struct stm32_spi *spi = data; spi_finalize_current_transfer(spi->ctrl); + spi->cfg->disable(spi); } /** @@ -1305,6 +1307,8 @@ static int stm32fx_spi_transfer_one_irq(struct stm32_spi *spi) stm32_spi_set_bits(spi, STM32FX_SPI_CR2, cr2); + stm32_spi_enable(spi); + /* starting data transfer when buffer is loaded */ if (spi->tx_buf) spi->cfg->write_tx(spi); @@ -1341,6 +1345,8 @@ static int stm32h7_spi_transfer_one_irq(struct stm32_spi *spi) spin_lock_irqsave(&spi->lock, flags); + stm32_spi_enable(spi); + /* Be sure to have data in fifo before starting data transfer */ if (spi->tx_buf) stm32h7_spi_write_txfifo(spi); @@ -1372,6 +1378,8 @@ static void stm32fx_spi_transfer_one_dma_start(struct stm32_spi *spi) */ stm32_spi_set_bits(spi, STM32FX_SPI_CR2, STM32FX_SPI_CR2_ERRIE); } + + stm32_spi_enable(spi); } /** @@ -1405,6 +1413,8 @@ static void stm32h7_spi_transfer_one_dma_start(struct stm32_spi *spi) stm32_spi_set_bits(spi, STM32H7_SPI_IER, ier); + stm32_spi_enable(spi); + if (STM32_SPI_HOST_MODE(spi)) stm32_spi_set_bits(spi, STM32H7_SPI_CR1, STM32H7_SPI_CR1_CSTART); } diff --git a/drivers/spi/spi.c b/drivers/spi/spi.c index 289feccca37607a3ffe982ab0ca0dfdd7daf62d6..9bc9fd10d538d2b61643f35608a93e445645968c 100644 --- a/drivers/spi/spi.c +++ b/drivers/spi/spi.c @@ -1220,6 +1220,11 @@ void spi_unmap_buf(struct spi_controller *ctlr, struct device *dev, spi_unmap_buf_attrs(ctlr, dev, sgt, dir, 0); } +/* Dummy SG for unidirect transfers */ +static struct scatterlist dummy_sg = { + .page_link = SG_END, +}; + static int __spi_map_msg(struct spi_controller *ctlr, struct spi_message *msg) { struct device *tx_dev, *rx_dev; @@ -1243,6 +1248,7 @@ static int __spi_map_msg(struct spi_controller *ctlr, struct spi_message *msg) else rx_dev = ctlr->dev.parent; + ret = -ENOMSG; list_for_each_entry(xfer, &msg->transfers, transfer_list) { /* The sync is done before each transfer. */ unsigned long attrs = DMA_ATTR_SKIP_CPU_SYNC; @@ -1257,6 +1263,8 @@ static int __spi_map_msg(struct spi_controller *ctlr, struct spi_message *msg) attrs); if (ret != 0) return ret; + } else { + xfer->tx_sg.sgl = &dummy_sg; } if (xfer->rx_buf != NULL) { @@ -1270,8 +1278,13 @@ static int __spi_map_msg(struct spi_controller *ctlr, struct spi_message *msg) return ret; } + } else { + xfer->rx_sg.sgl = &dummy_sg; } } + /* No transfer has been mapped, bail out with success */ + if (ret) + return 0; ctlr->cur_rx_dma_dev = rx_dev; ctlr->cur_tx_dma_dev = tx_dev; @@ -1307,7 +1320,7 @@ static int __spi_unmap_msg(struct spi_controller *ctlr, struct spi_message *msg) return 0; } -static void spi_dma_sync_for_device(struct spi_controller *ctlr, +static void spi_dma_sync_for_device(struct spi_controller *ctlr, struct spi_message *msg, struct spi_transfer *xfer) { struct device *rx_dev = ctlr->cur_rx_dma_dev; @@ -1316,11 +1329,14 @@ static void spi_dma_sync_for_device(struct spi_controller *ctlr, if (!ctlr->cur_msg_mapped) return; + if (!ctlr->can_dma(ctlr, msg->spi, xfer)) + return; + dma_sync_sgtable_for_device(tx_dev, &xfer->tx_sg, DMA_TO_DEVICE); dma_sync_sgtable_for_device(rx_dev, &xfer->rx_sg, DMA_FROM_DEVICE); } -static void spi_dma_sync_for_cpu(struct spi_controller *ctlr, +static void spi_dma_sync_for_cpu(struct spi_controller *ctlr, struct spi_message *msg, struct spi_transfer *xfer) { struct device *rx_dev = ctlr->cur_rx_dma_dev; @@ -1329,6 +1345,9 @@ static void spi_dma_sync_for_cpu(struct spi_controller *ctlr, if (!ctlr->cur_msg_mapped) return; + if (!ctlr->can_dma(ctlr, msg->spi, xfer)) + return; + dma_sync_sgtable_for_cpu(rx_dev, &xfer->rx_sg, DMA_FROM_DEVICE); dma_sync_sgtable_for_cpu(tx_dev, &xfer->tx_sg, DMA_TO_DEVICE); } @@ -1346,11 +1365,13 @@ static inline int __spi_unmap_msg(struct spi_controller *ctlr, } static void spi_dma_sync_for_device(struct spi_controller *ctrl, + struct spi_message *msg, struct spi_transfer *xfer) { } static void spi_dma_sync_for_cpu(struct spi_controller *ctrl, + struct spi_message *msg, struct spi_transfer *xfer) { } @@ -1622,10 +1643,10 @@ static int spi_transfer_one_message(struct spi_controller *ctlr, reinit_completion(&ctlr->xfer_completion); fallback_pio: - spi_dma_sync_for_device(ctlr, xfer); + spi_dma_sync_for_device(ctlr, msg, xfer); ret = ctlr->transfer_one(ctlr, msg->spi, xfer); if (ret < 0) { - spi_dma_sync_for_cpu(ctlr, xfer); + spi_dma_sync_for_cpu(ctlr, msg, xfer); if (ctlr->cur_msg_mapped && (xfer->error & SPI_TRANS_FAIL_NO_START)) { @@ -1650,7 +1671,7 @@ fallback_pio: msg->status = ret; } - spi_dma_sync_for_cpu(ctlr, xfer); + spi_dma_sync_for_cpu(ctlr, msg, xfer); } else { if (xfer->len) dev_err(&msg->spi->dev, diff --git a/drivers/thermal/thermal_core.c b/drivers/thermal/thermal_core.c index 54cce4e523bc510a0b25621317f2a56a31c32e97..30567b4994551b174e2b96bdfe1189816be115e6 100644 --- a/drivers/thermal/thermal_core.c +++ b/drivers/thermal/thermal_core.c @@ -467,6 +467,21 @@ static void thermal_governor_trip_crossed(struct thermal_governor *governor, governor->trip_crossed(tz, trip, crossed_up); } +static void thermal_trip_crossed(struct thermal_zone_device *tz, + const struct thermal_trip *trip, + struct thermal_governor *governor, + bool crossed_up) +{ + if (crossed_up) { + thermal_notify_tz_trip_up(tz, trip); + thermal_debug_tz_trip_up(tz, trip); + } else { + thermal_notify_tz_trip_down(tz, trip); + thermal_debug_tz_trip_down(tz, trip); + } + thermal_governor_trip_crossed(governor, tz, trip, crossed_up); +} + static int thermal_trip_notify_cmp(void *ascending, const struct list_head *a, const struct list_head *b) { @@ -506,18 +521,12 @@ void __thermal_zone_device_update(struct thermal_zone_device *tz, handle_thermal_trip(tz, td, &way_up_list, &way_down_list); list_sort(&way_up_list, &way_up_list, thermal_trip_notify_cmp); - list_for_each_entry(td, &way_up_list, notify_list_node) { - thermal_notify_tz_trip_up(tz, &td->trip); - thermal_debug_tz_trip_up(tz, &td->trip); - thermal_governor_trip_crossed(governor, tz, &td->trip, true); - } + list_for_each_entry(td, &way_up_list, notify_list_node) + thermal_trip_crossed(tz, &td->trip, governor, true); list_sort(NULL, &way_down_list, thermal_trip_notify_cmp); - list_for_each_entry(td, &way_down_list, notify_list_node) { - thermal_notify_tz_trip_down(tz, &td->trip); - thermal_debug_tz_trip_down(tz, &td->trip); - thermal_governor_trip_crossed(governor, tz, &td->trip, false); - } + list_for_each_entry(td, &way_down_list, notify_list_node) + thermal_trip_crossed(tz, &td->trip, governor, false); if (governor->manage) governor->manage(tz); @@ -593,6 +602,12 @@ void thermal_zone_device_update(struct thermal_zone_device *tz, } EXPORT_SYMBOL_GPL(thermal_zone_device_update); +void thermal_zone_trip_down(struct thermal_zone_device *tz, + const struct thermal_trip *trip) +{ + thermal_trip_crossed(tz, trip, thermal_get_tz_governor(tz), false); +} + int for_each_thermal_governor(int (*cb)(struct thermal_governor *, void *), void *data) { diff --git a/drivers/thermal/thermal_core.h b/drivers/thermal/thermal_core.h index d9785e5bbb08cde8b2c5a672aa1c14936a4fc06c..20e7b45673d683064209f63ecb0d4cb1b2f5fbdb 100644 --- a/drivers/thermal/thermal_core.h +++ b/drivers/thermal/thermal_core.h @@ -246,6 +246,8 @@ int thermal_zone_trip_id(const struct thermal_zone_device *tz, void thermal_zone_trip_updated(struct thermal_zone_device *tz, const struct thermal_trip *trip); int __thermal_zone_get_temp(struct thermal_zone_device *tz, int *temp); +void thermal_zone_trip_down(struct thermal_zone_device *tz, + const struct thermal_trip *trip); /* sysfs I/F */ int thermal_zone_create_device_groups(struct thermal_zone_device *tz); diff --git a/drivers/thermal/thermal_debugfs.c b/drivers/thermal/thermal_debugfs.c index 91f9c21235a8a9dcb05d51a45ddb4ee0f2ecf983..9424472291570861de5e68a27f8e0d7fa00f8964 100644 --- a/drivers/thermal/thermal_debugfs.c +++ b/drivers/thermal/thermal_debugfs.c @@ -91,6 +91,8 @@ struct cdev_record { * * @timestamp: the trip crossing timestamp * @duration: total time when the zone temperature was above the trip point + * @trip_temp: trip temperature at mitigation start + * @trip_hyst: trip hysteresis at mitigation start * @count: the number of times the zone temperature was above the trip point * @max: maximum recorded temperature above the trip point * @min: minimum recorded temperature above the trip point @@ -99,6 +101,8 @@ struct cdev_record { struct trip_stats { ktime_t timestamp; ktime_t duration; + int trip_temp; + int trip_hyst; int count; int max; int min; @@ -574,6 +578,7 @@ void thermal_debug_tz_trip_up(struct thermal_zone_device *tz, struct thermal_debugfs *thermal_dbg = tz->debugfs; int trip_id = thermal_zone_trip_id(tz, trip); ktime_t now = ktime_get(); + struct trip_stats *trip_stats; if (!thermal_dbg) return; @@ -639,7 +644,10 @@ void thermal_debug_tz_trip_up(struct thermal_zone_device *tz, tz_dbg->trips_crossed[tz_dbg->nr_trips++] = trip_id; tze = list_first_entry(&tz_dbg->tz_episodes, struct tz_episode, node); - tze->trip_stats[trip_id].timestamp = now; + trip_stats = &tze->trip_stats[trip_id]; + trip_stats->trip_temp = trip->temperature; + trip_stats->trip_hyst = trip->hysteresis; + trip_stats->timestamp = now; unlock: mutex_unlock(&thermal_dbg->lock); @@ -794,10 +802,6 @@ static int tze_seq_show(struct seq_file *s, void *v) const struct thermal_trip *trip = &td->trip; struct trip_stats *trip_stats; - /* Skip invalid trips. */ - if (trip->temperature == THERMAL_TEMP_INVALID) - continue; - /* * There is no possible mitigation happening at the * critical trip point, so the stats will be always @@ -836,8 +840,8 @@ static int tze_seq_show(struct seq_file *s, void *v) seq_printf(s, "| %*d | %*s | %*d | %*d | %c%*lld | %*d | %*d | %*d |\n", 4 , trip_id, 8, type, - 9, trip->temperature, - 9, trip->hysteresis, + 9, trip_stats->trip_temp, + 9, trip_stats->trip_hyst, c, 10, duration_ms, 9, trip_stats->avg, 9, trip_stats->min, diff --git a/drivers/thermal/thermal_trip.c b/drivers/thermal/thermal_trip.c index d6a6acc78ddb6c1586bd32c878e59a128933464f..49e63db685172e44a7f3c83a2df958bc5d839d90 100644 --- a/drivers/thermal/thermal_trip.c +++ b/drivers/thermal/thermal_trip.c @@ -152,17 +152,23 @@ void thermal_zone_set_trip_temp(struct thermal_zone_device *tz, if (trip->temperature == temp) return; + trip->temperature = temp; + thermal_notify_tz_trip_change(tz, trip); + if (temp == THERMAL_TEMP_INVALID) { struct thermal_trip_desc *td = trip_to_trip_desc(trip); - if (trip->type == THERMAL_TRIP_PASSIVE && - tz->temperature >= td->threshold) { + if (tz->temperature >= td->threshold) { /* - * The trip has been crossed, so the thermal zone's - * passive count needs to be adjusted. + * The trip has been crossed on the way up, so some + * adjustments are needed to compensate for the lack + * of it going forward. */ - tz->passive--; - WARN_ON_ONCE(tz->passive < 0); + if (trip->type == THERMAL_TRIP_PASSIVE) { + tz->passive--; + WARN_ON_ONCE(tz->passive < 0); + } + thermal_zone_trip_down(tz, trip); } /* * Invalidate the threshold to avoid triggering a spurious @@ -170,7 +176,5 @@ void thermal_zone_set_trip_temp(struct thermal_zone_device *tz, */ td->threshold = INT_MAX; } - trip->temperature = temp; - thermal_notify_tz_trip_change(tz, trip); } EXPORT_SYMBOL_GPL(thermal_zone_set_trip_temp); diff --git a/fs/9p/vfs_dentry.c b/fs/9p/vfs_dentry.c index f16f7358163490f633f67d3e93a82eb8da327da5..01338d4c2d9e6fde95e8e69816781ce968522ef6 100644 --- a/fs/9p/vfs_dentry.c +++ b/fs/9p/vfs_dentry.c @@ -48,12 +48,17 @@ static int v9fs_cached_dentry_delete(const struct dentry *dentry) static void v9fs_dentry_release(struct dentry *dentry) { struct hlist_node *p, *n; + struct hlist_head head; p9_debug(P9_DEBUG_VFS, " dentry: %pd (%p)\n", dentry, dentry); - hlist_for_each_safe(p, n, (struct hlist_head *)&dentry->d_fsdata) + + spin_lock(&dentry->d_lock); + hlist_move_list((struct hlist_head *)&dentry->d_fsdata, &head); + spin_unlock(&dentry->d_lock); + + hlist_for_each_safe(p, n, &head) p9_fid_put(hlist_entry(p, struct p9_fid, dlist)); - dentry->d_fsdata = NULL; } static int v9fs_lookup_revalidate(struct dentry *dentry, unsigned int flags) diff --git a/fs/9p/vfs_inode.c b/fs/9p/vfs_inode.c index 7a3308d776060e2e2565af09d358f2cf33416b6b..fd72fc38c8f5b1ff971414388f6fab16ce53ac4c 100644 --- a/fs/9p/vfs_inode.c +++ b/fs/9p/vfs_inode.c @@ -348,6 +348,7 @@ void v9fs_evict_inode(struct inode *inode) __le32 __maybe_unused version; if (!is_bad_inode(inode)) { + netfs_wait_for_outstanding_io(inode); truncate_inode_pages_final(&inode->i_data); version = cpu_to_le32(v9inode->qid.version); diff --git a/fs/afs/inode.c b/fs/afs/inode.c index 94fc049aff584f43e622d164a13fc30962dd04f1..15bb7989c387ae59a4e1e4fd49c9b9f99e4f0ecf 100644 --- a/fs/afs/inode.c +++ b/fs/afs/inode.c @@ -648,6 +648,7 @@ void afs_evict_inode(struct inode *inode) ASSERTCMP(inode->i_ino, ==, vnode->fid.vnode); + netfs_wait_for_outstanding_io(inode); truncate_inode_pages_final(&inode->i_data); afs_set_cache_aux(vnode, &aux); diff --git a/fs/afs/mntpt.c b/fs/afs/mntpt.c index 97f50e9fd9eb017384e5790a33519662d3cca978..297487ee832317a43608b38db09186b47cb1d583 100644 --- a/fs/afs/mntpt.c +++ b/fs/afs/mntpt.c @@ -140,6 +140,11 @@ static int afs_mntpt_set_params(struct fs_context *fc, struct dentry *mntpt) put_page(page); if (ret < 0) return ret; + + /* Don't cross a backup volume mountpoint from a backup volume */ + if (src_as->volume && src_as->volume->type == AFSVL_BACKVOL && + ctx->type == AFSVL_BACKVOL) + return -ENODEV; } return 0; diff --git a/fs/bcachefs/backpointers.c b/fs/bcachefs/backpointers.c index 692b1c7d5018c876bb41883e8443e3517dbd3be8..4321f9fb73bd9f7e098b663e20f516b0c505f6d4 100644 --- a/fs/bcachefs/backpointers.c +++ b/fs/bcachefs/backpointers.c @@ -690,7 +690,7 @@ static int check_extent_to_backpointers(struct btree_trans *trans, ptrs = bch2_bkey_ptrs_c(k); bkey_for_each_ptr_decode(k.k, ptrs, p, entry) { - struct bpos bucket_pos; + struct bpos bucket_pos = POS_MIN; struct bch_backpointer bp; if (p.ptr.cached) diff --git a/fs/bcachefs/bcachefs.h b/fs/bcachefs/bcachefs.h index bc0ea2c4efef25fce089f4b3ca24e3cf6b2f5bdb..2a538eb2af110c6e8bc5fbef4301b7eed29841c0 100644 --- a/fs/bcachefs/bcachefs.h +++ b/fs/bcachefs/bcachefs.h @@ -457,6 +457,7 @@ enum bch_time_stats { }; #include "alloc_types.h" +#include "btree_gc_types.h" #include "btree_types.h" #include "btree_node_scan_types.h" #include "btree_write_buffer_types.h" @@ -488,49 +489,6 @@ enum bch_time_stats { struct btree; -enum gc_phase { - GC_PHASE_NOT_RUNNING, - GC_PHASE_START, - GC_PHASE_SB, - - GC_PHASE_BTREE_stripes, - GC_PHASE_BTREE_extents, - GC_PHASE_BTREE_inodes, - GC_PHASE_BTREE_dirents, - GC_PHASE_BTREE_xattrs, - GC_PHASE_BTREE_alloc, - GC_PHASE_BTREE_quotas, - GC_PHASE_BTREE_reflink, - GC_PHASE_BTREE_subvolumes, - GC_PHASE_BTREE_snapshots, - GC_PHASE_BTREE_lru, - GC_PHASE_BTREE_freespace, - GC_PHASE_BTREE_need_discard, - GC_PHASE_BTREE_backpointers, - GC_PHASE_BTREE_bucket_gens, - GC_PHASE_BTREE_snapshot_trees, - GC_PHASE_BTREE_deleted_inodes, - GC_PHASE_BTREE_logged_ops, - GC_PHASE_BTREE_rebalance_work, - GC_PHASE_BTREE_subvolume_children, - - GC_PHASE_PENDING_DELETE, -}; - -struct gc_pos { - enum gc_phase phase; - u16 level; - struct bpos pos; -}; - -struct reflink_gc { - u64 offset; - u32 size; - u32 refcount; -}; - -typedef GENRADIX(struct reflink_gc) reflink_gc_table; - struct io_count { u64 sectors[2][BCH_DATA_NR]; }; diff --git a/fs/bcachefs/bcachefs_format.h b/fs/bcachefs/bcachefs_format.h index d801e19cb4890e014589f44de53e1c1aa6355aaa..90c12fe2a2cd3b030f0c6eb6aee1149262cc8dd1 100644 --- a/fs/bcachefs/bcachefs_format.h +++ b/fs/bcachefs/bcachefs_format.h @@ -503,16 +503,22 @@ struct bch_sb_field { #include "alloc_background_format.h" #include "extents_format.h" -#include "reflink_format.h" #include "ec_format.h" -#include "inode_format.h" #include "dirent_format.h" -#include "xattr_format.h" -#include "quota_format.h" +#include "disk_groups_format.h" +#include "inode_format.h" +#include "journal_seq_blacklist_format.h" #include "logged_ops_format.h" +#include "quota_format.h" +#include "reflink_format.h" +#include "replicas_format.h" #include "snapshot_format.h" #include "subvolume_format.h" #include "sb-counters_format.h" +#include "sb-downgrade_format.h" +#include "sb-errors_format.h" +#include "sb-members_format.h" +#include "xattr_format.h" enum bch_sb_field_type { #define x(f, nr) BCH_SB_FIELD_##f = nr, @@ -545,107 +551,6 @@ struct bch_sb_field_journal_v2 { } d[]; }; -/* BCH_SB_FIELD_members_v1: */ - -#define BCH_MIN_NR_NBUCKETS (1 << 6) - -#define BCH_IOPS_MEASUREMENTS() \ - x(seqread, 0) \ - x(seqwrite, 1) \ - x(randread, 2) \ - x(randwrite, 3) - -enum bch_iops_measurement { -#define x(t, n) BCH_IOPS_##t = n, - BCH_IOPS_MEASUREMENTS() -#undef x - BCH_IOPS_NR -}; - -#define BCH_MEMBER_ERROR_TYPES() \ - x(read, 0) \ - x(write, 1) \ - x(checksum, 2) - -enum bch_member_error_type { -#define x(t, n) BCH_MEMBER_ERROR_##t = n, - BCH_MEMBER_ERROR_TYPES() -#undef x - BCH_MEMBER_ERROR_NR -}; - -struct bch_member { - __uuid_t uuid; - __le64 nbuckets; /* device size */ - __le16 first_bucket; /* index of first bucket used */ - __le16 bucket_size; /* sectors */ - __u8 btree_bitmap_shift; - __u8 pad[3]; - __le64 last_mount; /* time_t */ - - __le64 flags; - __le32 iops[4]; - __le64 errors[BCH_MEMBER_ERROR_NR]; - __le64 errors_at_reset[BCH_MEMBER_ERROR_NR]; - __le64 errors_reset_time; - __le64 seq; - __le64 btree_allocated_bitmap; - /* - * On recovery from a clean shutdown we don't normally read the journal, - * but we still want to resume writing from where we left off so we - * don't overwrite more than is necessary, for list journal debugging: - */ - __le32 last_journal_bucket; - __le32 last_journal_bucket_offset; -}; - -/* - * This limit comes from the bucket_gens array - it's a single allocation, and - * kernel allocation are limited to INT_MAX - */ -#define BCH_MEMBER_NBUCKETS_MAX (INT_MAX - 64) - -#define BCH_MEMBER_V1_BYTES 56 - -LE64_BITMASK(BCH_MEMBER_STATE, struct bch_member, flags, 0, 4) -/* 4-14 unused, was TIER, HAS_(META)DATA, REPLACEMENT */ -LE64_BITMASK(BCH_MEMBER_DISCARD, struct bch_member, flags, 14, 15) -LE64_BITMASK(BCH_MEMBER_DATA_ALLOWED, struct bch_member, flags, 15, 20) -LE64_BITMASK(BCH_MEMBER_GROUP, struct bch_member, flags, 20, 28) -LE64_BITMASK(BCH_MEMBER_DURABILITY, struct bch_member, flags, 28, 30) -LE64_BITMASK(BCH_MEMBER_FREESPACE_INITIALIZED, - struct bch_member, flags, 30, 31) - -#if 0 -LE64_BITMASK(BCH_MEMBER_NR_READ_ERRORS, struct bch_member, flags[1], 0, 20); -LE64_BITMASK(BCH_MEMBER_NR_WRITE_ERRORS,struct bch_member, flags[1], 20, 40); -#endif - -#define BCH_MEMBER_STATES() \ - x(rw, 0) \ - x(ro, 1) \ - x(failed, 2) \ - x(spare, 3) - -enum bch_member_state { -#define x(t, n) BCH_MEMBER_STATE_##t = n, - BCH_MEMBER_STATES() -#undef x - BCH_MEMBER_STATE_NR -}; - -struct bch_sb_field_members_v1 { - struct bch_sb_field field; - struct bch_member _members[]; //Members are now variable size -}; - -struct bch_sb_field_members_v2 { - struct bch_sb_field field; - __le16 member_bytes; //size of single member entry - u8 pad[6]; - struct bch_member _members[]; -}; - /* BCH_SB_FIELD_crypt: */ struct nonce { @@ -694,8 +599,6 @@ LE64_BITMASK(BCH_KDF_SCRYPT_N, struct bch_sb_field_crypt, kdf_flags, 0, 16); LE64_BITMASK(BCH_KDF_SCRYPT_R, struct bch_sb_field_crypt, kdf_flags, 16, 32); LE64_BITMASK(BCH_KDF_SCRYPT_P, struct bch_sb_field_crypt, kdf_flags, 32, 48); -/* BCH_SB_FIELD_replicas: */ - #define BCH_DATA_TYPES() \ x(free, 0) \ x(sb, 1) \ @@ -738,50 +641,6 @@ static inline bool data_type_is_hidden(enum bch_data_type type) } } -struct bch_replicas_entry_v0 { - __u8 data_type; - __u8 nr_devs; - __u8 devs[]; -} __packed; - -struct bch_sb_field_replicas_v0 { - struct bch_sb_field field; - struct bch_replicas_entry_v0 entries[]; -} __packed __aligned(8); - -struct bch_replicas_entry_v1 { - __u8 data_type; - __u8 nr_devs; - __u8 nr_required; - __u8 devs[]; -} __packed; - -#define replicas_entry_bytes(_i) \ - (offsetof(typeof(*(_i)), devs) + (_i)->nr_devs) - -struct bch_sb_field_replicas { - struct bch_sb_field field; - struct bch_replicas_entry_v1 entries[]; -} __packed __aligned(8); - -/* BCH_SB_FIELD_disk_groups: */ - -#define BCH_SB_LABEL_SIZE 32 - -struct bch_disk_group { - __u8 label[BCH_SB_LABEL_SIZE]; - __le64 flags[2]; -} __packed __aligned(8); - -LE64_BITMASK(BCH_GROUP_DELETED, struct bch_disk_group, flags[0], 0, 1) -LE64_BITMASK(BCH_GROUP_DATA_ALLOWED, struct bch_disk_group, flags[0], 1, 6) -LE64_BITMASK(BCH_GROUP_PARENT, struct bch_disk_group, flags[0], 6, 24) - -struct bch_sb_field_disk_groups { - struct bch_sb_field field; - struct bch_disk_group entries[]; -} __packed __aligned(8); - /* * On clean shutdown, store btree roots and current journal sequence number in * the superblock: @@ -809,27 +668,6 @@ struct bch_sb_field_clean { __u64 _data[]; }; -struct journal_seq_blacklist_entry { - __le64 start; - __le64 end; -}; - -struct bch_sb_field_journal_seq_blacklist { - struct bch_sb_field field; - struct journal_seq_blacklist_entry start[]; -}; - -struct bch_sb_field_errors { - struct bch_sb_field field; - struct bch_sb_field_error_entry { - __le64 v; - __le64 last_error_time; - } entries[]; -}; - -LE64_BITMASK(BCH_SB_ERROR_ENTRY_ID, struct bch_sb_field_error_entry, v, 0, 16); -LE64_BITMASK(BCH_SB_ERROR_ENTRY_NR, struct bch_sb_field_error_entry, v, 16, 64); - struct bch_sb_field_ext { struct bch_sb_field field; __le64 recovery_passes_required[2]; @@ -837,18 +675,6 @@ struct bch_sb_field_ext { __le64 btrees_lost_data; }; -struct bch_sb_field_downgrade_entry { - __le16 version; - __le64 recovery_passes[2]; - __le16 nr_errors; - __le16 errors[] __counted_by(nr_errors); -} __packed __aligned(2); - -struct bch_sb_field_downgrade { - struct bch_sb_field field; - struct bch_sb_field_downgrade_entry entries[]; -}; - /* Superblock: */ /* @@ -909,7 +735,6 @@ unsigned bcachefs_metadata_required_upgrade_below = bcachefs_metadata_version_re #define bcachefs_metadata_version_current (bcachefs_metadata_version_max - 1) #define BCH_SB_SECTOR 8 -#define BCH_SB_MEMBERS_MAX 64 /* XXX kill */ #define BCH_SB_LAYOUT_SIZE_BITS_MAX 16 /* 32 MB */ diff --git a/fs/bcachefs/btree_gc.c b/fs/bcachefs/btree_gc.c index 8035c8b797ab37658ebd10b55fd1b28ee5304271..dc97991bcd6adad823121fd27f6f89889ce9a0d6 100644 --- a/fs/bcachefs/btree_gc.c +++ b/fs/bcachefs/btree_gc.c @@ -585,16 +585,17 @@ static int bch2_gc_mark_key(struct btree_trans *trans, enum btree_id btree_id, if (fsck_err_on(k.k->version.lo > atomic64_read(&c->key_version), c, bkey_version_in_future, - "key version number higher than recorded: %llu > %llu", - k.k->version.lo, - atomic64_read(&c->key_version))) + "key version number higher than recorded %llu\n %s", + atomic64_read(&c->key_version), + (bch2_bkey_val_to_text(&buf, c, k), buf.buf))) atomic64_set(&c->key_version, k.k->version.lo); } if (mustfix_fsck_err_on(level && !bch2_dev_btree_bitmap_marked(c, k), c, btree_bitmap_not_marked, "btree ptr not marked in member info btree allocated bitmap\n %s", - (bch2_bkey_val_to_text(&buf, c, k), + (printbuf_reset(&buf), + bch2_bkey_val_to_text(&buf, c, k), buf.buf))) { mutex_lock(&c->sb_lock); bch2_dev_btree_bitmap_mark(c, k); @@ -673,8 +674,7 @@ static int bch2_gc_btree(struct btree_trans *trans, enum btree_id btree, bool in static inline int btree_id_gc_phase_cmp(enum btree_id l, enum btree_id r) { - return (int) btree_id_to_gc_phase(l) - - (int) btree_id_to_gc_phase(r); + return cmp_int(gc_btree_order(l), gc_btree_order(r)); } static int bch2_gc_btrees(struct bch_fs *c) @@ -711,7 +711,7 @@ fsck_err: static int bch2_mark_superblocks(struct bch_fs *c) { mutex_lock(&c->sb_lock); - gc_pos_set(c, gc_phase(GC_PHASE_SB)); + gc_pos_set(c, gc_phase(GC_PHASE_sb)); int ret = bch2_trans_mark_dev_sbs_flags(c, BTREE_TRIGGER_gc); mutex_unlock(&c->sb_lock); @@ -1209,7 +1209,7 @@ int bch2_check_allocations(struct bch_fs *c) if (ret) goto out; - gc_pos_set(c, gc_phase(GC_PHASE_START)); + gc_pos_set(c, gc_phase(GC_PHASE_start)); ret = bch2_mark_superblocks(c); BUG_ON(ret); @@ -1231,7 +1231,7 @@ out: percpu_down_write(&c->mark_lock); /* Indicates that gc is no longer in progress: */ - __gc_pos_set(c, gc_phase(GC_PHASE_NOT_RUNNING)); + __gc_pos_set(c, gc_phase(GC_PHASE_not_running)); bch2_gc_free(c); percpu_up_write(&c->mark_lock); diff --git a/fs/bcachefs/btree_gc.h b/fs/bcachefs/btree_gc.h index 1b6489d8e0f4fa4a953ffb02cbc424abe9b60415..876d81e2017d73e3240aa239c20a2defb5132981 100644 --- a/fs/bcachefs/btree_gc.h +++ b/fs/bcachefs/btree_gc.h @@ -3,6 +3,7 @@ #define _BCACHEFS_BTREE_GC_H #include "bkey.h" +#include "btree_gc_types.h" #include "btree_types.h" int bch2_check_topology(struct bch_fs *); @@ -32,36 +33,15 @@ int bch2_check_allocations(struct bch_fs *); /* Position of (the start of) a gc phase: */ static inline struct gc_pos gc_phase(enum gc_phase phase) { - return (struct gc_pos) { - .phase = phase, - .level = 0, - .pos = POS_MIN, - }; -} - -static inline int gc_pos_cmp(struct gc_pos l, struct gc_pos r) -{ - return cmp_int(l.phase, r.phase) ?: - -cmp_int(l.level, r.level) ?: - bpos_cmp(l.pos, r.pos); -} - -static inline enum gc_phase btree_id_to_gc_phase(enum btree_id id) -{ - switch (id) { -#define x(name, v, ...) case BTREE_ID_##name: return GC_PHASE_BTREE_##name; - BCH_BTREE_IDS() -#undef x - default: - BUG(); - } + return (struct gc_pos) { .phase = phase, }; } static inline struct gc_pos gc_pos_btree(enum btree_id btree, unsigned level, struct bpos pos) { return (struct gc_pos) { - .phase = btree_id_to_gc_phase(btree), + .phase = GC_PHASE_btree, + .btree = btree, .level = level, .pos = pos, }; @@ -76,6 +56,22 @@ static inline struct gc_pos gc_pos_btree_node(struct btree *b) return gc_pos_btree(b->c.btree_id, b->c.level, b->key.k.p); } +static inline int gc_btree_order(enum btree_id btree) +{ + if (btree == BTREE_ID_stripes) + return -1; + return btree; +} + +static inline int gc_pos_cmp(struct gc_pos l, struct gc_pos r) +{ + return cmp_int(l.phase, r.phase) ?: + cmp_int(gc_btree_order(l.btree), + gc_btree_order(r.btree)) ?: + -cmp_int(l.level, r.level) ?: + bpos_cmp(l.pos, r.pos); +} + static inline bool gc_visited(struct bch_fs *c, struct gc_pos pos) { unsigned seq; diff --git a/fs/bcachefs/btree_gc_types.h b/fs/bcachefs/btree_gc_types.h new file mode 100644 index 0000000000000000000000000000000000000000..b82c24bcc0880d1725ef3ef55cdc156a0b0f5118 --- /dev/null +++ b/fs/bcachefs/btree_gc_types.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _BCACHEFS_BTREE_GC_TYPES_H +#define _BCACHEFS_BTREE_GC_TYPES_H + +#include + +enum gc_phase { + GC_PHASE_not_running, + GC_PHASE_start, + GC_PHASE_sb, + GC_PHASE_btree, +}; + +struct gc_pos { + enum gc_phase phase:8; + enum btree_id btree:8; + u16 level; + struct bpos pos; +}; + +struct reflink_gc { + u64 offset; + u32 size; + u32 refcount; +}; + +typedef GENRADIX(struct reflink_gc) reflink_gc_table; + +#endif /* _BCACHEFS_BTREE_GC_TYPES_H */ diff --git a/fs/bcachefs/btree_io.c b/fs/bcachefs/btree_io.c index cbf8f5d90602115576a4bdb975450cdb9199926d..829c1b91477d796bf1e3176807271ff8760e6cf7 100644 --- a/fs/bcachefs/btree_io.c +++ b/fs/bcachefs/btree_io.c @@ -519,7 +519,7 @@ void bch2_btree_init_next(struct btree_trans *trans, struct btree *b) static void btree_err_msg(struct printbuf *out, struct bch_fs *c, struct bch_dev *ca, - struct btree *b, struct bset *i, + struct btree *b, struct bset *i, struct bkey_packed *k, unsigned offset, int write) { prt_printf(out, bch2_log_msg(c, "%s"), @@ -537,15 +537,20 @@ static void btree_err_msg(struct printbuf *out, struct bch_fs *c, b->written, btree_ptr_sectors_written(&b->key)); if (i) prt_printf(out, " bset u64s %u", le16_to_cpu(i->u64s)); + if (k) + prt_printf(out, " bset byte offset %lu", + (unsigned long)(void *)k - + ((unsigned long)(void *)i & ~511UL)); prt_str(out, ": "); } -__printf(9, 10) +__printf(10, 11) static int __btree_err(int ret, struct bch_fs *c, struct bch_dev *ca, struct btree *b, struct bset *i, + struct bkey_packed *k, int write, bool have_retry, enum bch_sb_error_id err_type, @@ -555,7 +560,7 @@ static int __btree_err(int ret, bool silent = c->curr_recovery_pass == BCH_RECOVERY_PASS_scan_for_btree_nodes; va_list args; - btree_err_msg(&out, c, ca, b, i, b->written, write); + btree_err_msg(&out, c, ca, b, i, k, b->written, write); va_start(args, fmt); prt_vprintf(&out, fmt, args); @@ -611,9 +616,9 @@ fsck_err: return ret; } -#define btree_err(type, c, ca, b, i, _err_type, msg, ...) \ +#define btree_err(type, c, ca, b, i, k, _err_type, msg, ...) \ ({ \ - int _ret = __btree_err(type, c, ca, b, i, write, have_retry, \ + int _ret = __btree_err(type, c, ca, b, i, k, write, have_retry, \ BCH_FSCK_ERR_##_err_type, \ msg, ##__VA_ARGS__); \ \ @@ -690,7 +695,7 @@ static int validate_bset(struct bch_fs *c, struct bch_dev *ca, btree_err_on(!bch2_version_compatible(version), -BCH_ERR_btree_node_read_err_incompatible, - c, ca, b, i, + c, ca, b, i, NULL, btree_node_unsupported_version, "unsupported bset version %u.%u", BCH_VERSION_MAJOR(version), @@ -698,7 +703,7 @@ static int validate_bset(struct bch_fs *c, struct bch_dev *ca, if (btree_err_on(version < c->sb.version_min, -BCH_ERR_btree_node_read_err_fixable, - c, NULL, b, i, + c, NULL, b, i, NULL, btree_node_bset_older_than_sb_min, "bset version %u older than superblock version_min %u", version, c->sb.version_min)) { @@ -711,7 +716,7 @@ static int validate_bset(struct bch_fs *c, struct bch_dev *ca, if (btree_err_on(BCH_VERSION_MAJOR(version) > BCH_VERSION_MAJOR(c->sb.version), -BCH_ERR_btree_node_read_err_fixable, - c, NULL, b, i, + c, NULL, b, i, NULL, btree_node_bset_newer_than_sb, "bset version %u newer than superblock version %u", version, c->sb.version)) { @@ -723,13 +728,13 @@ static int validate_bset(struct bch_fs *c, struct bch_dev *ca, btree_err_on(BSET_SEPARATE_WHITEOUTS(i), -BCH_ERR_btree_node_read_err_incompatible, - c, ca, b, i, + c, ca, b, i, NULL, btree_node_unsupported_version, "BSET_SEPARATE_WHITEOUTS no longer supported"); if (btree_err_on(offset + sectors > btree_sectors(c), -BCH_ERR_btree_node_read_err_fixable, - c, ca, b, i, + c, ca, b, i, NULL, bset_past_end_of_btree_node, "bset past end of btree node")) { i->u64s = 0; @@ -739,13 +744,13 @@ static int validate_bset(struct bch_fs *c, struct bch_dev *ca, btree_err_on(offset && !i->u64s, -BCH_ERR_btree_node_read_err_fixable, - c, ca, b, i, + c, ca, b, i, NULL, bset_empty, "empty bset"); btree_err_on(BSET_OFFSET(i) && BSET_OFFSET(i) != offset, -BCH_ERR_btree_node_read_err_want_retry, - c, ca, b, i, + c, ca, b, i, NULL, bset_wrong_sector_offset, "bset at wrong sector offset"); @@ -761,20 +766,20 @@ static int validate_bset(struct bch_fs *c, struct bch_dev *ca, /* XXX endianness */ btree_err_on(bp->seq != bn->keys.seq, -BCH_ERR_btree_node_read_err_must_retry, - c, ca, b, NULL, + c, ca, b, NULL, NULL, bset_bad_seq, "incorrect sequence number (wrong btree node)"); } btree_err_on(BTREE_NODE_ID(bn) != b->c.btree_id, -BCH_ERR_btree_node_read_err_must_retry, - c, ca, b, i, + c, ca, b, i, NULL, btree_node_bad_btree, "incorrect btree id"); btree_err_on(BTREE_NODE_LEVEL(bn) != b->c.level, -BCH_ERR_btree_node_read_err_must_retry, - c, ca, b, i, + c, ca, b, i, NULL, btree_node_bad_level, "incorrect level"); @@ -793,7 +798,7 @@ static int validate_bset(struct bch_fs *c, struct bch_dev *ca, btree_err_on(!bpos_eq(b->data->min_key, bp->min_key), -BCH_ERR_btree_node_read_err_must_retry, - c, ca, b, NULL, + c, ca, b, NULL, NULL, btree_node_bad_min_key, "incorrect min_key: got %s should be %s", (printbuf_reset(&buf1), @@ -804,7 +809,7 @@ static int validate_bset(struct bch_fs *c, struct bch_dev *ca, btree_err_on(!bpos_eq(bn->max_key, b->key.k.p), -BCH_ERR_btree_node_read_err_must_retry, - c, ca, b, i, + c, ca, b, i, NULL, btree_node_bad_max_key, "incorrect max key %s", (printbuf_reset(&buf1), @@ -816,7 +821,7 @@ static int validate_bset(struct bch_fs *c, struct bch_dev *ca, btree_err_on(bch2_bkey_format_invalid(c, &bn->format, write, &buf1), -BCH_ERR_btree_node_read_err_bad_node, - c, ca, b, i, + c, ca, b, i, NULL, btree_node_bad_format, "invalid bkey format: %s\n %s", buf1.buf, (printbuf_reset(&buf2), @@ -883,7 +888,7 @@ static int validate_bset_keys(struct bch_fs *c, struct btree *b, if (btree_err_on(bkey_p_next(k) > vstruct_last(i), -BCH_ERR_btree_node_read_err_fixable, - c, NULL, b, i, + c, NULL, b, i, k, btree_node_bkey_past_bset_end, "key extends past end of bset")) { i->u64s = cpu_to_le16((u64 *) k - i->_data); @@ -892,14 +897,14 @@ static int validate_bset_keys(struct bch_fs *c, struct btree *b, if (btree_err_on(k->format > KEY_FORMAT_CURRENT, -BCH_ERR_btree_node_read_err_fixable, - c, NULL, b, i, + c, NULL, b, i, k, btree_node_bkey_bad_format, "invalid bkey format %u", k->format)) goto drop_this_key; if (btree_err_on(!bkeyp_u64s_valid(&b->format, k), -BCH_ERR_btree_node_read_err_fixable, - c, NULL, b, i, + c, NULL, b, i, k, btree_node_bkey_bad_u64s, "bad k->u64s %u (min %u max %zu)", k->u64s, bkeyp_key_u64s(&b->format, k), @@ -921,7 +926,7 @@ static int validate_bset_keys(struct bch_fs *c, struct btree *b, bch2_bkey_val_to_text(&buf, c, u.s_c); btree_err(-BCH_ERR_btree_node_read_err_fixable, - c, NULL, b, i, + c, NULL, b, i, k, btree_node_bad_bkey, "invalid bkey: %s", buf.buf); goto drop_this_key; @@ -942,7 +947,7 @@ static int validate_bset_keys(struct bch_fs *c, struct btree *b, bch2_bkey_to_text(&buf, u.k); if (btree_err(-BCH_ERR_btree_node_read_err_fixable, - c, NULL, b, i, + c, NULL, b, i, k, btree_node_bkey_out_of_order, "%s", buf.buf)) goto drop_this_key; @@ -1011,13 +1016,13 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, if (bch2_meta_read_fault("btree")) btree_err(-BCH_ERR_btree_node_read_err_must_retry, - c, ca, b, NULL, + c, ca, b, NULL, NULL, btree_node_fault_injected, "dynamic fault"); btree_err_on(le64_to_cpu(b->data->magic) != bset_magic(c), -BCH_ERR_btree_node_read_err_must_retry, - c, ca, b, NULL, + c, ca, b, NULL, NULL, btree_node_bad_magic, "bad magic: want %llx, got %llx", bset_magic(c), le64_to_cpu(b->data->magic)); @@ -1032,7 +1037,7 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, btree_err_on(b->data->keys.seq != bp->seq, -BCH_ERR_btree_node_read_err_must_retry, - c, ca, b, NULL, + c, ca, b, NULL, NULL, btree_node_bad_seq, "got wrong btree node: got\n%s", (printbuf_reset(&buf), @@ -1041,7 +1046,7 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, } else { btree_err_on(!b->data->keys.seq, -BCH_ERR_btree_node_read_err_must_retry, - c, ca, b, NULL, + c, ca, b, NULL, NULL, btree_node_bad_seq, "bad btree header: seq 0\n%s", (printbuf_reset(&buf), @@ -1060,7 +1065,7 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, btree_err_on(!bch2_checksum_type_valid(c, BSET_CSUM_TYPE(i)), -BCH_ERR_btree_node_read_err_want_retry, - c, ca, b, i, + c, ca, b, i, NULL, bset_unknown_csum, "unknown checksum type %llu", BSET_CSUM_TYPE(i)); @@ -1073,7 +1078,7 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, btree_err_on(csum_bad, -BCH_ERR_btree_node_read_err_want_retry, - c, ca, b, i, + c, ca, b, i, NULL, bset_bad_csum, "%s", (printbuf_reset(&buf), @@ -1088,7 +1093,7 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, btree_err_on(btree_node_type_is_extents(btree_node_type(b)) && !BTREE_NODE_NEW_EXTENT_OVERWRITE(b->data), -BCH_ERR_btree_node_read_err_incompatible, - c, NULL, b, NULL, + c, NULL, b, NULL, NULL, btree_node_unsupported_version, "btree node does not have NEW_EXTENT_OVERWRITE set"); @@ -1102,7 +1107,7 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, btree_err_on(!bch2_checksum_type_valid(c, BSET_CSUM_TYPE(i)), -BCH_ERR_btree_node_read_err_want_retry, - c, ca, b, i, + c, ca, b, i, NULL, bset_unknown_csum, "unknown checksum type %llu", BSET_CSUM_TYPE(i)); @@ -1114,7 +1119,7 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, btree_err_on(csum_bad, -BCH_ERR_btree_node_read_err_want_retry, - c, ca, b, i, + c, ca, b, i, NULL, bset_bad_csum, "%s", (printbuf_reset(&buf), @@ -1152,14 +1157,14 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, btree_err_on(blacklisted && first, -BCH_ERR_btree_node_read_err_fixable, - c, ca, b, i, + c, ca, b, i, NULL, bset_blacklisted_journal_seq, "first btree node bset has blacklisted journal seq (%llu)", le64_to_cpu(i->journal_seq)); btree_err_on(blacklisted && ptr_written, -BCH_ERR_btree_node_read_err_fixable, - c, ca, b, i, + c, ca, b, i, NULL, first_bset_blacklisted_journal_seq, "found blacklisted bset (journal seq %llu) in btree node at offset %u-%u/%u", le64_to_cpu(i->journal_seq), @@ -1178,7 +1183,7 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, if (ptr_written) { btree_err_on(b->written < ptr_written, -BCH_ERR_btree_node_read_err_want_retry, - c, ca, b, NULL, + c, ca, b, NULL, NULL, btree_node_data_missing, "btree node data missing: expected %u sectors, found %u", ptr_written, b->written); @@ -1191,7 +1196,7 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, le64_to_cpu(bne->keys.journal_seq), true), -BCH_ERR_btree_node_read_err_want_retry, - c, ca, b, NULL, + c, ca, b, NULL, NULL, btree_node_bset_after_end, "found bset signature after last bset"); } @@ -1235,7 +1240,7 @@ int bch2_btree_node_read_done(struct bch_fs *c, struct bch_dev *ca, bch2_bkey_val_to_text(&buf, c, u.s_c); btree_err(-BCH_ERR_btree_node_read_err_fixable, - c, NULL, b, i, + c, NULL, b, i, k, btree_node_bad_bkey, "%s", buf.buf); @@ -1471,18 +1476,18 @@ static CLOSURE_CALLBACK(btree_node_read_all_replicas_done) written2 = btree_node_sectors_written(c, ra->buf[i]); if (btree_err_on(written2 != written, -BCH_ERR_btree_node_read_err_fixable, - c, NULL, b, NULL, + c, NULL, b, NULL, NULL, btree_node_replicas_sectors_written_mismatch, "btree node sectors written mismatch: %u != %u", written, written2) || btree_err_on(btree_node_has_extra_bsets(c, written2, ra->buf[i]), -BCH_ERR_btree_node_read_err_fixable, - c, NULL, b, NULL, + c, NULL, b, NULL, NULL, btree_node_bset_after_end, "found bset signature after last bset") || btree_err_on(memcmp(ra->buf[best], ra->buf[i], written << 9), -BCH_ERR_btree_node_read_err_fixable, - c, NULL, b, NULL, + c, NULL, b, NULL, NULL, btree_node_replicas_data_mismatch, "btree node replicas content mismatch")) dump_bset_maps = true; diff --git a/fs/bcachefs/btree_key_cache.c b/fs/bcachefs/btree_key_cache.c index 75f5e6fe46349f0ed0a2d3b3b4611883d69b6700..34056aaece009e80c1ef2bd39d304f8b185ce68d 100644 --- a/fs/bcachefs/btree_key_cache.c +++ b/fs/bcachefs/btree_key_cache.c @@ -424,16 +424,16 @@ static int btree_key_cache_fill(struct btree_trans *trans, goto err; } - if (!bch2_btree_node_relock(trans, ck_path, 0)) { + ret = bch2_trans_relock(trans); + if (ret) { kfree(new_k); - trace_and_count(trans->c, trans_restart_relock_key_cache_fill, trans, _THIS_IP_, ck_path); - ret = btree_trans_restart(trans, BCH_ERR_transaction_restart_key_cache_fill); goto err; } - ret = bch2_trans_relock(trans); - if (ret) { + if (!bch2_btree_node_relock(trans, ck_path, 0)) { kfree(new_k); + trace_and_count(trans->c, trans_restart_relock_key_cache_fill, trans, _THIS_IP_, ck_path); + ret = btree_trans_restart(trans, BCH_ERR_transaction_restart_key_cache_fill); goto err; } } diff --git a/fs/bcachefs/btree_locking.c b/fs/bcachefs/btree_locking.c index c3e9b0cc7bbdc9ece6ee8da4966a43838f64df9e..d66fff22109ae791b3a803209bc263c8d899a864 100644 --- a/fs/bcachefs/btree_locking.c +++ b/fs/bcachefs/btree_locking.c @@ -215,6 +215,7 @@ static noinline int break_cycle(struct lock_graph *g, struct printbuf *cycle) if (unlikely(!best)) { struct printbuf buf = PRINTBUF; + buf.atomic++; prt_printf(&buf, bch2_fmt(g->g->trans->c, "cycle of nofail locks")); diff --git a/fs/bcachefs/buckets.c b/fs/bcachefs/buckets.c index b469586517a86dbbf60110838208d3633f55cbe6..ed97712d0db1edc1b9564700342f2f8ffca06361 100644 --- a/fs/bcachefs/buckets.c +++ b/fs/bcachefs/buckets.c @@ -1134,7 +1134,7 @@ static int __trigger_extent(struct btree_trans *trans, r.e.nr_required = 1; bkey_for_each_ptr_decode(k.k, ptrs, p, entry) { - s64 disk_sectors; + s64 disk_sectors = 0; ret = bch2_trigger_pointer(trans, btree_id, level, k, p, entry, &disk_sectors, flags); if (ret < 0) return ret; diff --git a/fs/bcachefs/disk_groups_format.h b/fs/bcachefs/disk_groups_format.h new file mode 100644 index 0000000000000000000000000000000000000000..698990bbf1d20671cb25905efb76309b30f3747b --- /dev/null +++ b/fs/bcachefs/disk_groups_format.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _BCACHEFS_DISK_GROUPS_FORMAT_H +#define _BCACHEFS_DISK_GROUPS_FORMAT_H + +#define BCH_SB_LABEL_SIZE 32 + +struct bch_disk_group { + __u8 label[BCH_SB_LABEL_SIZE]; + __le64 flags[2]; +} __packed __aligned(8); + +LE64_BITMASK(BCH_GROUP_DELETED, struct bch_disk_group, flags[0], 0, 1) +LE64_BITMASK(BCH_GROUP_DATA_ALLOWED, struct bch_disk_group, flags[0], 1, 6) +LE64_BITMASK(BCH_GROUP_PARENT, struct bch_disk_group, flags[0], 6, 24) + +struct bch_sb_field_disk_groups { + struct bch_sb_field field; + struct bch_disk_group entries[]; +} __packed __aligned(8); + +#endif /* _BCACHEFS_DISK_GROUPS_FORMAT_H */ diff --git a/fs/bcachefs/ec.c b/fs/bcachefs/ec.c index b26dc7424662390b07c4d205d245403fbd1cde69..d8b9beca377627200d1bda8ffd75136ff5fdaa25 100644 --- a/fs/bcachefs/ec.c +++ b/fs/bcachefs/ec.c @@ -908,7 +908,7 @@ static int __ec_stripe_mem_alloc(struct bch_fs *c, size_t idx, gfp_t gfp) if (!genradix_ptr_alloc(&c->stripes, idx, gfp)) return -BCH_ERR_ENOMEM_ec_stripe_mem_alloc; - if (c->gc_pos.phase != GC_PHASE_NOT_RUNNING && + if (c->gc_pos.phase != GC_PHASE_not_running && !genradix_ptr_alloc(&c->gc_stripes, idx, gfp)) return -BCH_ERR_ENOMEM_ec_stripe_mem_alloc; diff --git a/fs/bcachefs/fs-io-buffered.c b/fs/bcachefs/fs-io-buffered.c index 6b69e5cd68dd514bc51c3d7ae65b7977fad7fb3d..54873ecc635cb03176feb921badda3229ed525fd 100644 --- a/fs/bcachefs/fs-io-buffered.c +++ b/fs/bcachefs/fs-io-buffered.c @@ -437,8 +437,8 @@ static void bch2_writepage_io_done(struct bch_write_op *op) */ /* - * PageWriteback is effectively our ref on the inode - fixup i_blocks - * before calling end_page_writeback: + * The writeback flag is effectively our ref on the inode - + * fixup i_blocks before calling folio_end_writeback: */ bch2_i_sectors_acct(c, io->inode, NULL, io->op.i_sectors_delta); @@ -898,7 +898,7 @@ static int __bch2_buffered_write(struct bch_inode_info *inode, darray_for_each(fs, fi) { f = *fi; f_len = min(end, folio_end_pos(f)) - f_pos; - f_copied = copy_page_from_iter_atomic(&f->page, f_offset, f_len, iter); + f_copied = copy_folio_from_iter_atomic(f, f_offset, f_len, iter); if (!f_copied) { folios_trunc(&fs, fi); break; diff --git a/fs/bcachefs/fs-io-direct.c b/fs/bcachefs/fs-io-direct.c index 09d21aef879af89bbdadbfabcc71bc630604e069..049b61bc9a5b38e491c367d9a6b5a0f9c6567426 100644 --- a/fs/bcachefs/fs-io-direct.c +++ b/fs/bcachefs/fs-io-direct.c @@ -609,8 +609,10 @@ ssize_t bch2_direct_write(struct kiocb *req, struct iov_iter *iter) if (unlikely(ret)) goto err_put_write_ref; - if (unlikely((req->ki_pos|iter->count) & (block_bytes(c) - 1))) + if (unlikely((req->ki_pos|iter->count) & (block_bytes(c) - 1))) { + ret = -EINVAL; goto err_put_write_ref; + } inode_dio_begin(&inode->v); bch2_pagecache_block_get(inode); diff --git a/fs/bcachefs/fs.c b/fs/bcachefs/fs.c index 96040a95cf4667069ad8dac4cc02a56f45382dc8..cd388f1702dc85a98f2d5b05db518ea66e77243a 100644 --- a/fs/bcachefs/fs.c +++ b/fs/bcachefs/fs.c @@ -1939,8 +1939,7 @@ got_sb: if (IS_ERR(sb)) { ret = PTR_ERR(sb); - ret = bch2_err_class(ret); - return ERR_PTR(ret); + goto err; } c = sb->s_fs_info; @@ -2016,6 +2015,15 @@ out: err_put_super: __bch2_fs_stop(c); deactivate_locked_super(sb); +err: + /* + * On an inconsistency error in recovery we might see an -EROFS derived + * errorcode (from the journal), but we don't want to return that to + * userspace as that causes util-linux to retry the mount RO - which is + * confusing: + */ + if (bch2_err_matches(ret, EROFS) && ret != -EROFS) + ret = -EIO; return ERR_PTR(bch2_err_class(ret)); } diff --git a/fs/bcachefs/fsck.c b/fs/bcachefs/fsck.c index c8f57465131c54becd0cb745d7ce2b54b1e87cd1..fd277bd58ed34afe3ad5a86f93210ea621756b62 100644 --- a/fs/bcachefs/fsck.c +++ b/fs/bcachefs/fsck.c @@ -77,21 +77,17 @@ static int lookup_first_inode(struct btree_trans *trans, u64 inode_nr, struct bkey_s_c k; int ret; - bch2_trans_iter_init(trans, &iter, BTREE_ID_inodes, - POS(0, inode_nr), - BTREE_ITER_all_snapshots); - k = bch2_btree_iter_peek(&iter); - ret = bkey_err(k); - if (ret) - goto err; - - if (!k.k || !bkey_eq(k.k->p, POS(0, inode_nr))) { - ret = -BCH_ERR_ENOENT_inode; - goto err; + for_each_btree_key_norestart(trans, iter, BTREE_ID_inodes, POS(0, inode_nr), + BTREE_ITER_all_snapshots, k, ret) { + if (k.k->p.offset != inode_nr) + break; + if (!bkey_is_inode(k.k)) + continue; + ret = bch2_inode_unpack(k, inode); + goto found; } - - ret = bch2_inode_unpack(k, inode); -err: + ret = -BCH_ERR_ENOENT_inode; +found: bch_err_msg(trans->c, ret, "fetching inode %llu", inode_nr); bch2_trans_iter_exit(trans, &iter); return ret; @@ -770,25 +766,6 @@ static int get_visible_inodes(struct btree_trans *trans, return ret; } -static int check_key_has_snapshot(struct btree_trans *trans, - struct btree_iter *iter, - struct bkey_s_c k) -{ - struct bch_fs *c = trans->c; - struct printbuf buf = PRINTBUF; - int ret = 0; - - if (mustfix_fsck_err_on(!bch2_snapshot_equiv(c, k.k->p.snapshot), c, - bkey_in_missing_snapshot, - "key in missing snapshot: %s", - (bch2_bkey_val_to_text(&buf, c, k), buf.buf))) - ret = bch2_btree_delete_at(trans, iter, - BTREE_UPDATE_internal_snapshot_node) ?: 1; -fsck_err: - printbuf_exit(&buf); - return ret; -} - static int hash_redo_key(struct btree_trans *trans, const struct bch_hash_desc desc, struct bch_hash_info *hash_info, @@ -983,7 +960,7 @@ static int check_inode(struct btree_trans *trans, bool do_update = false; int ret; - ret = check_key_has_snapshot(trans, iter, k); + ret = bch2_check_key_has_snapshot(trans, iter, k); if (ret < 0) goto err; if (ret) @@ -1487,7 +1464,7 @@ static int check_extent(struct btree_trans *trans, struct btree_iter *iter, struct printbuf buf = PRINTBUF; int ret = 0; - ret = check_key_has_snapshot(trans, iter, k); + ret = bch2_check_key_has_snapshot(trans, iter, k); if (ret) { ret = ret < 0 ? ret : 0; goto out; @@ -2010,7 +1987,7 @@ static int check_dirent(struct btree_trans *trans, struct btree_iter *iter, struct printbuf buf = PRINTBUF; int ret = 0; - ret = check_key_has_snapshot(trans, iter, k); + ret = bch2_check_key_has_snapshot(trans, iter, k); if (ret) { ret = ret < 0 ? ret : 0; goto out; @@ -2165,7 +2142,7 @@ static int check_xattr(struct btree_trans *trans, struct btree_iter *iter, struct inode_walker_entry *i; int ret; - ret = check_key_has_snapshot(trans, iter, k); + ret = bch2_check_key_has_snapshot(trans, iter, k); if (ret < 0) return ret; if (ret) diff --git a/fs/bcachefs/journal_seq_blacklist_format.h b/fs/bcachefs/journal_seq_blacklist_format.h new file mode 100644 index 0000000000000000000000000000000000000000..2566b12dbc045ea7892a73d777f0d503a32fba5f --- /dev/null +++ b/fs/bcachefs/journal_seq_blacklist_format.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _BCACHEFS_JOURNAL_SEQ_BLACKLIST_FORMAT_H +#define _BCACHEFS_JOURNAL_SEQ_BLACKLIST_FORMAT_H + +struct journal_seq_blacklist_entry { + __le64 start; + __le64 end; +}; + +struct bch_sb_field_journal_seq_blacklist { + struct bch_sb_field field; + struct journal_seq_blacklist_entry start[]; +}; + +#endif /* _BCACHEFS_JOURNAL_SEQ_BLACKLIST_FORMAT_H */ diff --git a/fs/bcachefs/mean_and_variance_test.c b/fs/bcachefs/mean_and_variance_test.c index 4c298e74723db3023b9120cf1f823e46bfbaec4c..e9d9c0212e44b155322df9d2d6c9996042f8a452 100644 --- a/fs/bcachefs/mean_and_variance_test.c +++ b/fs/bcachefs/mean_and_variance_test.c @@ -217,4 +217,5 @@ static struct kunit_suite mean_and_variance_test_suite = { kunit_test_suite(mean_and_variance_test_suite); MODULE_AUTHOR("Daniel B. Hill"); +MODULE_DESCRIPTION("bcachefs filesystem mean and variance unit tests"); MODULE_LICENSE("GPL"); diff --git a/fs/bcachefs/move.c b/fs/bcachefs/move.c index 8171f947fac8f8d75f376d2df4a4ac341a0645d1..6e477fadaa2a5b546e191f4da614e8d74722bd47 100644 --- a/fs/bcachefs/move.c +++ b/fs/bcachefs/move.c @@ -547,6 +547,7 @@ static int bch2_move_data_btree(struct moving_context *ctxt, ctxt->stats->pos = BBPOS(btree_id, start); } + bch2_trans_begin(trans); bch2_trans_iter_init(trans, &iter, btree_id, start, BTREE_ITER_prefetch| BTREE_ITER_all_snapshots); @@ -920,7 +921,20 @@ static bool rereplicate_pred(struct bch_fs *c, void *arg, ? c->opts.metadata_replicas : io_opts->data_replicas; - if (!nr_good || nr_good >= replicas) + rcu_read_lock(); + struct bkey_ptrs_c ptrs = bch2_bkey_ptrs_c(k); + unsigned i = 0; + bkey_for_each_ptr(ptrs, ptr) { + struct bch_dev *ca = bch2_dev_rcu(c, ptr->dev); + if (!ptr->cached && + (!ca || !ca->mi.durability)) + data_opts->kill_ptrs |= BIT(i); + i++; + } + rcu_read_unlock(); + + if (!data_opts->kill_ptrs && + (!nr_good || nr_good >= replicas)) return false; data_opts->target = 0; diff --git a/fs/bcachefs/replicas_format.h b/fs/bcachefs/replicas_format.h new file mode 100644 index 0000000000000000000000000000000000000000..b97208195d064344f8509de7aceb59ef146c6fac --- /dev/null +++ b/fs/bcachefs/replicas_format.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _BCACHEFS_REPLICAS_FORMAT_H +#define _BCACHEFS_REPLICAS_FORMAT_H + +struct bch_replicas_entry_v0 { + __u8 data_type; + __u8 nr_devs; + __u8 devs[]; +} __packed; + +struct bch_sb_field_replicas_v0 { + struct bch_sb_field field; + struct bch_replicas_entry_v0 entries[]; +} __packed __aligned(8); + +struct bch_replicas_entry_v1 { + __u8 data_type; + __u8 nr_devs; + __u8 nr_required; + __u8 devs[]; +} __packed; + +struct bch_sb_field_replicas { + struct bch_sb_field field; + struct bch_replicas_entry_v1 entries[]; +} __packed __aligned(8); + +#define replicas_entry_bytes(_i) \ + (offsetof(typeof(*(_i)), devs) + (_i)->nr_devs) + +#endif /* _BCACHEFS_REPLICAS_FORMAT_H */ diff --git a/fs/bcachefs/sb-downgrade.c b/fs/bcachefs/sb-downgrade.c index 390a1bbd2567ac271f3c69b505753f969d0508eb..3fb23e399ffb3e6c021b8825fe3403e5a048ba4c 100644 --- a/fs/bcachefs/sb-downgrade.c +++ b/fs/bcachefs/sb-downgrade.c @@ -146,10 +146,17 @@ static int bch2_sb_downgrade_validate(struct bch_sb *sb, struct bch_sb_field *f, for (const struct bch_sb_field_downgrade_entry *i = e->entries; (void *) i < vstruct_end(&e->field); i = downgrade_entry_next_c(i)) { + /* + * Careful: sb_field_downgrade_entry is only 2 byte aligned, but + * section sizes are 8 byte aligned - an empty entry spanning + * the end of the section is allowed (and ignored): + */ + if ((void *) &i->errors[0] > vstruct_end(&e->field)) + break; + if (flags & BCH_VALIDATE_write && - ((void *) &i->errors[0] > vstruct_end(&e->field) || - (void *) downgrade_entry_next_c(i) > vstruct_end(&e->field))) { - prt_printf(err, "downgrade entry overruns end of superblock section)"); + (void *) downgrade_entry_next_c(i) > vstruct_end(&e->field)) { + prt_printf(err, "downgrade entry overruns end of superblock section"); return -BCH_ERR_invalid_sb_downgrade; } diff --git a/fs/bcachefs/sb-downgrade_format.h b/fs/bcachefs/sb-downgrade_format.h new file mode 100644 index 0000000000000000000000000000000000000000..cffd932be3eca00d5aa4aebfc8dd0ab11259073c --- /dev/null +++ b/fs/bcachefs/sb-downgrade_format.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _BCACHEFS_SB_DOWNGRADE_FORMAT_H +#define _BCACHEFS_SB_DOWNGRADE_FORMAT_H + +struct bch_sb_field_downgrade_entry { + __le16 version; + __le64 recovery_passes[2]; + __le16 nr_errors; + __le16 errors[] __counted_by(nr_errors); +} __packed __aligned(2); + +struct bch_sb_field_downgrade { + struct bch_sb_field field; + struct bch_sb_field_downgrade_entry entries[]; +}; + +#endif /* _BCACHEFS_SB_DOWNGRADE_FORMAT_H */ diff --git a/fs/bcachefs/sb-errors_format.h b/fs/bcachefs/sb-errors_format.h new file mode 100644 index 0000000000000000000000000000000000000000..84d2763bd597bdcaa8c80b912032d75ac35107bf --- /dev/null +++ b/fs/bcachefs/sb-errors_format.h @@ -0,0 +1,296 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _BCACHEFS_SB_ERRORS_FORMAT_H +#define _BCACHEFS_SB_ERRORS_FORMAT_H + +#define BCH_SB_ERRS() \ + x(clean_but_journal_not_empty, 0) \ + x(dirty_but_no_journal_entries, 1) \ + x(dirty_but_no_journal_entries_post_drop_nonflushes, 2) \ + x(sb_clean_journal_seq_mismatch, 3) \ + x(sb_clean_btree_root_mismatch, 4) \ + x(sb_clean_missing, 5) \ + x(jset_unsupported_version, 6) \ + x(jset_unknown_csum, 7) \ + x(jset_last_seq_newer_than_seq, 8) \ + x(jset_past_bucket_end, 9) \ + x(jset_seq_blacklisted, 10) \ + x(journal_entries_missing, 11) \ + x(journal_entry_replicas_not_marked, 12) \ + x(journal_entry_past_jset_end, 13) \ + x(journal_entry_replicas_data_mismatch, 14) \ + x(journal_entry_bkey_u64s_0, 15) \ + x(journal_entry_bkey_past_end, 16) \ + x(journal_entry_bkey_bad_format, 17) \ + x(journal_entry_bkey_invalid, 18) \ + x(journal_entry_btree_root_bad_size, 19) \ + x(journal_entry_blacklist_bad_size, 20) \ + x(journal_entry_blacklist_v2_bad_size, 21) \ + x(journal_entry_blacklist_v2_start_past_end, 22) \ + x(journal_entry_usage_bad_size, 23) \ + x(journal_entry_data_usage_bad_size, 24) \ + x(journal_entry_clock_bad_size, 25) \ + x(journal_entry_clock_bad_rw, 26) \ + x(journal_entry_dev_usage_bad_size, 27) \ + x(journal_entry_dev_usage_bad_dev, 28) \ + x(journal_entry_dev_usage_bad_pad, 29) \ + x(btree_node_unreadable, 30) \ + x(btree_node_fault_injected, 31) \ + x(btree_node_bad_magic, 32) \ + x(btree_node_bad_seq, 33) \ + x(btree_node_unsupported_version, 34) \ + x(btree_node_bset_older_than_sb_min, 35) \ + x(btree_node_bset_newer_than_sb, 36) \ + x(btree_node_data_missing, 37) \ + x(btree_node_bset_after_end, 38) \ + x(btree_node_replicas_sectors_written_mismatch, 39) \ + x(btree_node_replicas_data_mismatch, 40) \ + x(bset_unknown_csum, 41) \ + x(bset_bad_csum, 42) \ + x(bset_past_end_of_btree_node, 43) \ + x(bset_wrong_sector_offset, 44) \ + x(bset_empty, 45) \ + x(bset_bad_seq, 46) \ + x(bset_blacklisted_journal_seq, 47) \ + x(first_bset_blacklisted_journal_seq, 48) \ + x(btree_node_bad_btree, 49) \ + x(btree_node_bad_level, 50) \ + x(btree_node_bad_min_key, 51) \ + x(btree_node_bad_max_key, 52) \ + x(btree_node_bad_format, 53) \ + x(btree_node_bkey_past_bset_end, 54) \ + x(btree_node_bkey_bad_format, 55) \ + x(btree_node_bad_bkey, 56) \ + x(btree_node_bkey_out_of_order, 57) \ + x(btree_root_bkey_invalid, 58) \ + x(btree_root_read_error, 59) \ + x(btree_root_bad_min_key, 60) \ + x(btree_root_bad_max_key, 61) \ + x(btree_node_read_error, 62) \ + x(btree_node_topology_bad_min_key, 63) \ + x(btree_node_topology_bad_max_key, 64) \ + x(btree_node_topology_overwritten_by_prev_node, 65) \ + x(btree_node_topology_overwritten_by_next_node, 66) \ + x(btree_node_topology_interior_node_empty, 67) \ + x(fs_usage_hidden_wrong, 68) \ + x(fs_usage_btree_wrong, 69) \ + x(fs_usage_data_wrong, 70) \ + x(fs_usage_cached_wrong, 71) \ + x(fs_usage_reserved_wrong, 72) \ + x(fs_usage_persistent_reserved_wrong, 73) \ + x(fs_usage_nr_inodes_wrong, 74) \ + x(fs_usage_replicas_wrong, 75) \ + x(dev_usage_buckets_wrong, 76) \ + x(dev_usage_sectors_wrong, 77) \ + x(dev_usage_fragmented_wrong, 78) \ + x(dev_usage_buckets_ec_wrong, 79) \ + x(bkey_version_in_future, 80) \ + x(bkey_u64s_too_small, 81) \ + x(bkey_invalid_type_for_btree, 82) \ + x(bkey_extent_size_zero, 83) \ + x(bkey_extent_size_greater_than_offset, 84) \ + x(bkey_size_nonzero, 85) \ + x(bkey_snapshot_nonzero, 86) \ + x(bkey_snapshot_zero, 87) \ + x(bkey_at_pos_max, 88) \ + x(bkey_before_start_of_btree_node, 89) \ + x(bkey_after_end_of_btree_node, 90) \ + x(bkey_val_size_nonzero, 91) \ + x(bkey_val_size_too_small, 92) \ + x(alloc_v1_val_size_bad, 93) \ + x(alloc_v2_unpack_error, 94) \ + x(alloc_v3_unpack_error, 95) \ + x(alloc_v4_val_size_bad, 96) \ + x(alloc_v4_backpointers_start_bad, 97) \ + x(alloc_key_data_type_bad, 98) \ + x(alloc_key_empty_but_have_data, 99) \ + x(alloc_key_dirty_sectors_0, 100) \ + x(alloc_key_data_type_inconsistency, 101) \ + x(alloc_key_to_missing_dev_bucket, 102) \ + x(alloc_key_cached_inconsistency, 103) \ + x(alloc_key_cached_but_read_time_zero, 104) \ + x(alloc_key_to_missing_lru_entry, 105) \ + x(alloc_key_data_type_wrong, 106) \ + x(alloc_key_gen_wrong, 107) \ + x(alloc_key_dirty_sectors_wrong, 108) \ + x(alloc_key_cached_sectors_wrong, 109) \ + x(alloc_key_stripe_wrong, 110) \ + x(alloc_key_stripe_redundancy_wrong, 111) \ + x(bucket_sector_count_overflow, 112) \ + x(bucket_metadata_type_mismatch, 113) \ + x(need_discard_key_wrong, 114) \ + x(freespace_key_wrong, 115) \ + x(freespace_hole_missing, 116) \ + x(bucket_gens_val_size_bad, 117) \ + x(bucket_gens_key_wrong, 118) \ + x(bucket_gens_hole_wrong, 119) \ + x(bucket_gens_to_invalid_dev, 120) \ + x(bucket_gens_to_invalid_buckets, 121) \ + x(bucket_gens_nonzero_for_invalid_buckets, 122) \ + x(need_discard_freespace_key_to_invalid_dev_bucket, 123) \ + x(need_discard_freespace_key_bad, 124) \ + x(backpointer_bucket_offset_wrong, 125) \ + x(backpointer_to_missing_device, 126) \ + x(backpointer_to_missing_alloc, 127) \ + x(backpointer_to_missing_ptr, 128) \ + x(lru_entry_at_time_0, 129) \ + x(lru_entry_to_invalid_bucket, 130) \ + x(lru_entry_bad, 131) \ + x(btree_ptr_val_too_big, 132) \ + x(btree_ptr_v2_val_too_big, 133) \ + x(btree_ptr_has_non_ptr, 134) \ + x(extent_ptrs_invalid_entry, 135) \ + x(extent_ptrs_no_ptrs, 136) \ + x(extent_ptrs_too_many_ptrs, 137) \ + x(extent_ptrs_redundant_crc, 138) \ + x(extent_ptrs_redundant_stripe, 139) \ + x(extent_ptrs_unwritten, 140) \ + x(extent_ptrs_written_and_unwritten, 141) \ + x(ptr_to_invalid_device, 142) \ + x(ptr_to_duplicate_device, 143) \ + x(ptr_after_last_bucket, 144) \ + x(ptr_before_first_bucket, 145) \ + x(ptr_spans_multiple_buckets, 146) \ + x(ptr_to_missing_backpointer, 147) \ + x(ptr_to_missing_alloc_key, 148) \ + x(ptr_to_missing_replicas_entry, 149) \ + x(ptr_to_missing_stripe, 150) \ + x(ptr_to_incorrect_stripe, 151) \ + x(ptr_gen_newer_than_bucket_gen, 152) \ + x(ptr_too_stale, 153) \ + x(stale_dirty_ptr, 154) \ + x(ptr_bucket_data_type_mismatch, 155) \ + x(ptr_cached_and_erasure_coded, 156) \ + x(ptr_crc_uncompressed_size_too_small, 157) \ + x(ptr_crc_csum_type_unknown, 158) \ + x(ptr_crc_compression_type_unknown, 159) \ + x(ptr_crc_redundant, 160) \ + x(ptr_crc_uncompressed_size_too_big, 161) \ + x(ptr_crc_nonce_mismatch, 162) \ + x(ptr_stripe_redundant, 163) \ + x(reservation_key_nr_replicas_invalid, 164) \ + x(reflink_v_refcount_wrong, 165) \ + x(reflink_p_to_missing_reflink_v, 166) \ + x(stripe_pos_bad, 167) \ + x(stripe_val_size_bad, 168) \ + x(stripe_sector_count_wrong, 169) \ + x(snapshot_tree_pos_bad, 170) \ + x(snapshot_tree_to_missing_snapshot, 171) \ + x(snapshot_tree_to_missing_subvol, 172) \ + x(snapshot_tree_to_wrong_subvol, 173) \ + x(snapshot_tree_to_snapshot_subvol, 174) \ + x(snapshot_pos_bad, 175) \ + x(snapshot_parent_bad, 176) \ + x(snapshot_children_not_normalized, 177) \ + x(snapshot_child_duplicate, 178) \ + x(snapshot_child_bad, 179) \ + x(snapshot_skiplist_not_normalized, 180) \ + x(snapshot_skiplist_bad, 181) \ + x(snapshot_should_not_have_subvol, 182) \ + x(snapshot_to_bad_snapshot_tree, 183) \ + x(snapshot_bad_depth, 184) \ + x(snapshot_bad_skiplist, 185) \ + x(subvol_pos_bad, 186) \ + x(subvol_not_master_and_not_snapshot, 187) \ + x(subvol_to_missing_root, 188) \ + x(subvol_root_wrong_bi_subvol, 189) \ + x(bkey_in_missing_snapshot, 190) \ + x(inode_pos_inode_nonzero, 191) \ + x(inode_pos_blockdev_range, 192) \ + x(inode_unpack_error, 193) \ + x(inode_str_hash_invalid, 194) \ + x(inode_v3_fields_start_bad, 195) \ + x(inode_snapshot_mismatch, 196) \ + x(inode_unlinked_but_clean, 197) \ + x(inode_unlinked_but_nlink_nonzero, 198) \ + x(inode_checksum_type_invalid, 199) \ + x(inode_compression_type_invalid, 200) \ + x(inode_subvol_root_but_not_dir, 201) \ + x(inode_i_size_dirty_but_clean, 202) \ + x(inode_i_sectors_dirty_but_clean, 203) \ + x(inode_i_sectors_wrong, 204) \ + x(inode_dir_wrong_nlink, 205) \ + x(inode_dir_multiple_links, 206) \ + x(inode_multiple_links_but_nlink_0, 207) \ + x(inode_wrong_backpointer, 208) \ + x(inode_wrong_nlink, 209) \ + x(inode_unreachable, 210) \ + x(deleted_inode_but_clean, 211) \ + x(deleted_inode_missing, 212) \ + x(deleted_inode_is_dir, 213) \ + x(deleted_inode_not_unlinked, 214) \ + x(extent_overlapping, 215) \ + x(extent_in_missing_inode, 216) \ + x(extent_in_non_reg_inode, 217) \ + x(extent_past_end_of_inode, 218) \ + x(dirent_empty_name, 219) \ + x(dirent_val_too_big, 220) \ + x(dirent_name_too_long, 221) \ + x(dirent_name_embedded_nul, 222) \ + x(dirent_name_dot_or_dotdot, 223) \ + x(dirent_name_has_slash, 224) \ + x(dirent_d_type_wrong, 225) \ + x(inode_bi_parent_wrong, 226) \ + x(dirent_in_missing_dir_inode, 227) \ + x(dirent_in_non_dir_inode, 228) \ + x(dirent_to_missing_inode, 229) \ + x(dirent_to_missing_subvol, 230) \ + x(dirent_to_itself, 231) \ + x(quota_type_invalid, 232) \ + x(xattr_val_size_too_small, 233) \ + x(xattr_val_size_too_big, 234) \ + x(xattr_invalid_type, 235) \ + x(xattr_name_invalid_chars, 236) \ + x(xattr_in_missing_inode, 237) \ + x(root_subvol_missing, 238) \ + x(root_dir_missing, 239) \ + x(root_inode_not_dir, 240) \ + x(dir_loop, 241) \ + x(hash_table_key_duplicate, 242) \ + x(hash_table_key_wrong_offset, 243) \ + x(unlinked_inode_not_on_deleted_list, 244) \ + x(reflink_p_front_pad_bad, 245) \ + x(journal_entry_dup_same_device, 246) \ + x(inode_bi_subvol_missing, 247) \ + x(inode_bi_subvol_wrong, 248) \ + x(inode_points_to_missing_dirent, 249) \ + x(inode_points_to_wrong_dirent, 250) \ + x(inode_bi_parent_nonzero, 251) \ + x(dirent_to_missing_parent_subvol, 252) \ + x(dirent_not_visible_in_parent_subvol, 253) \ + x(subvol_fs_path_parent_wrong, 254) \ + x(subvol_root_fs_path_parent_nonzero, 255) \ + x(subvol_children_not_set, 256) \ + x(subvol_children_bad, 257) \ + x(subvol_loop, 258) \ + x(subvol_unreachable, 259) \ + x(btree_node_bkey_bad_u64s, 260) \ + x(btree_node_topology_empty_interior_node, 261) \ + x(btree_ptr_v2_min_key_bad, 262) \ + x(btree_root_unreadable_and_scan_found_nothing, 263) \ + x(snapshot_node_missing, 264) \ + x(dup_backpointer_to_bad_csum_extent, 265) \ + x(btree_bitmap_not_marked, 266) \ + x(sb_clean_entry_overrun, 267) \ + x(btree_ptr_v2_written_0, 268) \ + x(subvol_snapshot_bad, 269) \ + x(subvol_inode_bad, 270) + +enum bch_sb_error_id { +#define x(t, n) BCH_FSCK_ERR_##t = n, + BCH_SB_ERRS() +#undef x + BCH_SB_ERR_MAX +}; + +struct bch_sb_field_errors { + struct bch_sb_field field; + struct bch_sb_field_error_entry { + __le64 v; + __le64 last_error_time; + } entries[]; +}; + +LE64_BITMASK(BCH_SB_ERROR_ENTRY_ID, struct bch_sb_field_error_entry, v, 0, 16); +LE64_BITMASK(BCH_SB_ERROR_ENTRY_NR, struct bch_sb_field_error_entry, v, 16, 64); + +#endif /* _BCACHEFS_SB_ERRORS_FORMAT_H */ diff --git a/fs/bcachefs/sb-errors_types.h b/fs/bcachefs/sb-errors_types.h index 666599d3fb9d7d1a684a1efa94f77eab2d9dc0bb..40325239c3b0f23572e67c722b13e631d9a6d78f 100644 --- a/fs/bcachefs/sb-errors_types.h +++ b/fs/bcachefs/sb-errors_types.h @@ -4,286 +4,6 @@ #include "darray.h" -#define BCH_SB_ERRS() \ - x(clean_but_journal_not_empty, 0) \ - x(dirty_but_no_journal_entries, 1) \ - x(dirty_but_no_journal_entries_post_drop_nonflushes, 2) \ - x(sb_clean_journal_seq_mismatch, 3) \ - x(sb_clean_btree_root_mismatch, 4) \ - x(sb_clean_missing, 5) \ - x(jset_unsupported_version, 6) \ - x(jset_unknown_csum, 7) \ - x(jset_last_seq_newer_than_seq, 8) \ - x(jset_past_bucket_end, 9) \ - x(jset_seq_blacklisted, 10) \ - x(journal_entries_missing, 11) \ - x(journal_entry_replicas_not_marked, 12) \ - x(journal_entry_past_jset_end, 13) \ - x(journal_entry_replicas_data_mismatch, 14) \ - x(journal_entry_bkey_u64s_0, 15) \ - x(journal_entry_bkey_past_end, 16) \ - x(journal_entry_bkey_bad_format, 17) \ - x(journal_entry_bkey_invalid, 18) \ - x(journal_entry_btree_root_bad_size, 19) \ - x(journal_entry_blacklist_bad_size, 20) \ - x(journal_entry_blacklist_v2_bad_size, 21) \ - x(journal_entry_blacklist_v2_start_past_end, 22) \ - x(journal_entry_usage_bad_size, 23) \ - x(journal_entry_data_usage_bad_size, 24) \ - x(journal_entry_clock_bad_size, 25) \ - x(journal_entry_clock_bad_rw, 26) \ - x(journal_entry_dev_usage_bad_size, 27) \ - x(journal_entry_dev_usage_bad_dev, 28) \ - x(journal_entry_dev_usage_bad_pad, 29) \ - x(btree_node_unreadable, 30) \ - x(btree_node_fault_injected, 31) \ - x(btree_node_bad_magic, 32) \ - x(btree_node_bad_seq, 33) \ - x(btree_node_unsupported_version, 34) \ - x(btree_node_bset_older_than_sb_min, 35) \ - x(btree_node_bset_newer_than_sb, 36) \ - x(btree_node_data_missing, 37) \ - x(btree_node_bset_after_end, 38) \ - x(btree_node_replicas_sectors_written_mismatch, 39) \ - x(btree_node_replicas_data_mismatch, 40) \ - x(bset_unknown_csum, 41) \ - x(bset_bad_csum, 42) \ - x(bset_past_end_of_btree_node, 43) \ - x(bset_wrong_sector_offset, 44) \ - x(bset_empty, 45) \ - x(bset_bad_seq, 46) \ - x(bset_blacklisted_journal_seq, 47) \ - x(first_bset_blacklisted_journal_seq, 48) \ - x(btree_node_bad_btree, 49) \ - x(btree_node_bad_level, 50) \ - x(btree_node_bad_min_key, 51) \ - x(btree_node_bad_max_key, 52) \ - x(btree_node_bad_format, 53) \ - x(btree_node_bkey_past_bset_end, 54) \ - x(btree_node_bkey_bad_format, 55) \ - x(btree_node_bad_bkey, 56) \ - x(btree_node_bkey_out_of_order, 57) \ - x(btree_root_bkey_invalid, 58) \ - x(btree_root_read_error, 59) \ - x(btree_root_bad_min_key, 60) \ - x(btree_root_bad_max_key, 61) \ - x(btree_node_read_error, 62) \ - x(btree_node_topology_bad_min_key, 63) \ - x(btree_node_topology_bad_max_key, 64) \ - x(btree_node_topology_overwritten_by_prev_node, 65) \ - x(btree_node_topology_overwritten_by_next_node, 66) \ - x(btree_node_topology_interior_node_empty, 67) \ - x(fs_usage_hidden_wrong, 68) \ - x(fs_usage_btree_wrong, 69) \ - x(fs_usage_data_wrong, 70) \ - x(fs_usage_cached_wrong, 71) \ - x(fs_usage_reserved_wrong, 72) \ - x(fs_usage_persistent_reserved_wrong, 73) \ - x(fs_usage_nr_inodes_wrong, 74) \ - x(fs_usage_replicas_wrong, 75) \ - x(dev_usage_buckets_wrong, 76) \ - x(dev_usage_sectors_wrong, 77) \ - x(dev_usage_fragmented_wrong, 78) \ - x(dev_usage_buckets_ec_wrong, 79) \ - x(bkey_version_in_future, 80) \ - x(bkey_u64s_too_small, 81) \ - x(bkey_invalid_type_for_btree, 82) \ - x(bkey_extent_size_zero, 83) \ - x(bkey_extent_size_greater_than_offset, 84) \ - x(bkey_size_nonzero, 85) \ - x(bkey_snapshot_nonzero, 86) \ - x(bkey_snapshot_zero, 87) \ - x(bkey_at_pos_max, 88) \ - x(bkey_before_start_of_btree_node, 89) \ - x(bkey_after_end_of_btree_node, 90) \ - x(bkey_val_size_nonzero, 91) \ - x(bkey_val_size_too_small, 92) \ - x(alloc_v1_val_size_bad, 93) \ - x(alloc_v2_unpack_error, 94) \ - x(alloc_v3_unpack_error, 95) \ - x(alloc_v4_val_size_bad, 96) \ - x(alloc_v4_backpointers_start_bad, 97) \ - x(alloc_key_data_type_bad, 98) \ - x(alloc_key_empty_but_have_data, 99) \ - x(alloc_key_dirty_sectors_0, 100) \ - x(alloc_key_data_type_inconsistency, 101) \ - x(alloc_key_to_missing_dev_bucket, 102) \ - x(alloc_key_cached_inconsistency, 103) \ - x(alloc_key_cached_but_read_time_zero, 104) \ - x(alloc_key_to_missing_lru_entry, 105) \ - x(alloc_key_data_type_wrong, 106) \ - x(alloc_key_gen_wrong, 107) \ - x(alloc_key_dirty_sectors_wrong, 108) \ - x(alloc_key_cached_sectors_wrong, 109) \ - x(alloc_key_stripe_wrong, 110) \ - x(alloc_key_stripe_redundancy_wrong, 111) \ - x(bucket_sector_count_overflow, 112) \ - x(bucket_metadata_type_mismatch, 113) \ - x(need_discard_key_wrong, 114) \ - x(freespace_key_wrong, 115) \ - x(freespace_hole_missing, 116) \ - x(bucket_gens_val_size_bad, 117) \ - x(bucket_gens_key_wrong, 118) \ - x(bucket_gens_hole_wrong, 119) \ - x(bucket_gens_to_invalid_dev, 120) \ - x(bucket_gens_to_invalid_buckets, 121) \ - x(bucket_gens_nonzero_for_invalid_buckets, 122) \ - x(need_discard_freespace_key_to_invalid_dev_bucket, 123) \ - x(need_discard_freespace_key_bad, 124) \ - x(backpointer_bucket_offset_wrong, 125) \ - x(backpointer_to_missing_device, 126) \ - x(backpointer_to_missing_alloc, 127) \ - x(backpointer_to_missing_ptr, 128) \ - x(lru_entry_at_time_0, 129) \ - x(lru_entry_to_invalid_bucket, 130) \ - x(lru_entry_bad, 131) \ - x(btree_ptr_val_too_big, 132) \ - x(btree_ptr_v2_val_too_big, 133) \ - x(btree_ptr_has_non_ptr, 134) \ - x(extent_ptrs_invalid_entry, 135) \ - x(extent_ptrs_no_ptrs, 136) \ - x(extent_ptrs_too_many_ptrs, 137) \ - x(extent_ptrs_redundant_crc, 138) \ - x(extent_ptrs_redundant_stripe, 139) \ - x(extent_ptrs_unwritten, 140) \ - x(extent_ptrs_written_and_unwritten, 141) \ - x(ptr_to_invalid_device, 142) \ - x(ptr_to_duplicate_device, 143) \ - x(ptr_after_last_bucket, 144) \ - x(ptr_before_first_bucket, 145) \ - x(ptr_spans_multiple_buckets, 146) \ - x(ptr_to_missing_backpointer, 147) \ - x(ptr_to_missing_alloc_key, 148) \ - x(ptr_to_missing_replicas_entry, 149) \ - x(ptr_to_missing_stripe, 150) \ - x(ptr_to_incorrect_stripe, 151) \ - x(ptr_gen_newer_than_bucket_gen, 152) \ - x(ptr_too_stale, 153) \ - x(stale_dirty_ptr, 154) \ - x(ptr_bucket_data_type_mismatch, 155) \ - x(ptr_cached_and_erasure_coded, 156) \ - x(ptr_crc_uncompressed_size_too_small, 157) \ - x(ptr_crc_csum_type_unknown, 158) \ - x(ptr_crc_compression_type_unknown, 159) \ - x(ptr_crc_redundant, 160) \ - x(ptr_crc_uncompressed_size_too_big, 161) \ - x(ptr_crc_nonce_mismatch, 162) \ - x(ptr_stripe_redundant, 163) \ - x(reservation_key_nr_replicas_invalid, 164) \ - x(reflink_v_refcount_wrong, 165) \ - x(reflink_p_to_missing_reflink_v, 166) \ - x(stripe_pos_bad, 167) \ - x(stripe_val_size_bad, 168) \ - x(stripe_sector_count_wrong, 169) \ - x(snapshot_tree_pos_bad, 170) \ - x(snapshot_tree_to_missing_snapshot, 171) \ - x(snapshot_tree_to_missing_subvol, 172) \ - x(snapshot_tree_to_wrong_subvol, 173) \ - x(snapshot_tree_to_snapshot_subvol, 174) \ - x(snapshot_pos_bad, 175) \ - x(snapshot_parent_bad, 176) \ - x(snapshot_children_not_normalized, 177) \ - x(snapshot_child_duplicate, 178) \ - x(snapshot_child_bad, 179) \ - x(snapshot_skiplist_not_normalized, 180) \ - x(snapshot_skiplist_bad, 181) \ - x(snapshot_should_not_have_subvol, 182) \ - x(snapshot_to_bad_snapshot_tree, 183) \ - x(snapshot_bad_depth, 184) \ - x(snapshot_bad_skiplist, 185) \ - x(subvol_pos_bad, 186) \ - x(subvol_not_master_and_not_snapshot, 187) \ - x(subvol_to_missing_root, 188) \ - x(subvol_root_wrong_bi_subvol, 189) \ - x(bkey_in_missing_snapshot, 190) \ - x(inode_pos_inode_nonzero, 191) \ - x(inode_pos_blockdev_range, 192) \ - x(inode_unpack_error, 193) \ - x(inode_str_hash_invalid, 194) \ - x(inode_v3_fields_start_bad, 195) \ - x(inode_snapshot_mismatch, 196) \ - x(inode_unlinked_but_clean, 197) \ - x(inode_unlinked_but_nlink_nonzero, 198) \ - x(inode_checksum_type_invalid, 199) \ - x(inode_compression_type_invalid, 200) \ - x(inode_subvol_root_but_not_dir, 201) \ - x(inode_i_size_dirty_but_clean, 202) \ - x(inode_i_sectors_dirty_but_clean, 203) \ - x(inode_i_sectors_wrong, 204) \ - x(inode_dir_wrong_nlink, 205) \ - x(inode_dir_multiple_links, 206) \ - x(inode_multiple_links_but_nlink_0, 207) \ - x(inode_wrong_backpointer, 208) \ - x(inode_wrong_nlink, 209) \ - x(inode_unreachable, 210) \ - x(deleted_inode_but_clean, 211) \ - x(deleted_inode_missing, 212) \ - x(deleted_inode_is_dir, 213) \ - x(deleted_inode_not_unlinked, 214) \ - x(extent_overlapping, 215) \ - x(extent_in_missing_inode, 216) \ - x(extent_in_non_reg_inode, 217) \ - x(extent_past_end_of_inode, 218) \ - x(dirent_empty_name, 219) \ - x(dirent_val_too_big, 220) \ - x(dirent_name_too_long, 221) \ - x(dirent_name_embedded_nul, 222) \ - x(dirent_name_dot_or_dotdot, 223) \ - x(dirent_name_has_slash, 224) \ - x(dirent_d_type_wrong, 225) \ - x(inode_bi_parent_wrong, 226) \ - x(dirent_in_missing_dir_inode, 227) \ - x(dirent_in_non_dir_inode, 228) \ - x(dirent_to_missing_inode, 229) \ - x(dirent_to_missing_subvol, 230) \ - x(dirent_to_itself, 231) \ - x(quota_type_invalid, 232) \ - x(xattr_val_size_too_small, 233) \ - x(xattr_val_size_too_big, 234) \ - x(xattr_invalid_type, 235) \ - x(xattr_name_invalid_chars, 236) \ - x(xattr_in_missing_inode, 237) \ - x(root_subvol_missing, 238) \ - x(root_dir_missing, 239) \ - x(root_inode_not_dir, 240) \ - x(dir_loop, 241) \ - x(hash_table_key_duplicate, 242) \ - x(hash_table_key_wrong_offset, 243) \ - x(unlinked_inode_not_on_deleted_list, 244) \ - x(reflink_p_front_pad_bad, 245) \ - x(journal_entry_dup_same_device, 246) \ - x(inode_bi_subvol_missing, 247) \ - x(inode_bi_subvol_wrong, 248) \ - x(inode_points_to_missing_dirent, 249) \ - x(inode_points_to_wrong_dirent, 250) \ - x(inode_bi_parent_nonzero, 251) \ - x(dirent_to_missing_parent_subvol, 252) \ - x(dirent_not_visible_in_parent_subvol, 253) \ - x(subvol_fs_path_parent_wrong, 254) \ - x(subvol_root_fs_path_parent_nonzero, 255) \ - x(subvol_children_not_set, 256) \ - x(subvol_children_bad, 257) \ - x(subvol_loop, 258) \ - x(subvol_unreachable, 259) \ - x(btree_node_bkey_bad_u64s, 260) \ - x(btree_node_topology_empty_interior_node, 261) \ - x(btree_ptr_v2_min_key_bad, 262) \ - x(btree_root_unreadable_and_scan_found_nothing, 263) \ - x(snapshot_node_missing, 264) \ - x(dup_backpointer_to_bad_csum_extent, 265) \ - x(btree_bitmap_not_marked, 266) \ - x(sb_clean_entry_overrun, 267) \ - x(btree_ptr_v2_written_0, 268) \ - x(subvol_snapshot_bad, 269) \ - x(subvol_inode_bad, 270) - -enum bch_sb_error_id { -#define x(t, n) BCH_FSCK_ERR_##t = n, - BCH_SB_ERRS() -#undef x - BCH_SB_ERR_MAX -}; - struct bch_sb_error_entry_cpu { u64 id:16, nr:48; @@ -293,4 +13,3 @@ struct bch_sb_error_entry_cpu { typedef DARRAY(struct bch_sb_error_entry_cpu) bch_sb_errors_cpu; #endif /* _BCACHEFS_SB_ERRORS_TYPES_H */ - diff --git a/fs/bcachefs/sb-members_format.h b/fs/bcachefs/sb-members_format.h new file mode 100644 index 0000000000000000000000000000000000000000..e2630548c0f681dc35edef4f7b0f5228f2f5d9a3 --- /dev/null +++ b/fs/bcachefs/sb-members_format.h @@ -0,0 +1,110 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _BCACHEFS_SB_MEMBERS_FORMAT_H +#define _BCACHEFS_SB_MEMBERS_FORMAT_H + +/* + * We refer to members with bitmasks in various places - but we need to get rid + * of this limit: + */ +#define BCH_SB_MEMBERS_MAX 64 + +#define BCH_MIN_NR_NBUCKETS (1 << 6) + +#define BCH_IOPS_MEASUREMENTS() \ + x(seqread, 0) \ + x(seqwrite, 1) \ + x(randread, 2) \ + x(randwrite, 3) + +enum bch_iops_measurement { +#define x(t, n) BCH_IOPS_##t = n, + BCH_IOPS_MEASUREMENTS() +#undef x + BCH_IOPS_NR +}; + +#define BCH_MEMBER_ERROR_TYPES() \ + x(read, 0) \ + x(write, 1) \ + x(checksum, 2) + +enum bch_member_error_type { +#define x(t, n) BCH_MEMBER_ERROR_##t = n, + BCH_MEMBER_ERROR_TYPES() +#undef x + BCH_MEMBER_ERROR_NR +}; + +struct bch_member { + __uuid_t uuid; + __le64 nbuckets; /* device size */ + __le16 first_bucket; /* index of first bucket used */ + __le16 bucket_size; /* sectors */ + __u8 btree_bitmap_shift; + __u8 pad[3]; + __le64 last_mount; /* time_t */ + + __le64 flags; + __le32 iops[4]; + __le64 errors[BCH_MEMBER_ERROR_NR]; + __le64 errors_at_reset[BCH_MEMBER_ERROR_NR]; + __le64 errors_reset_time; + __le64 seq; + __le64 btree_allocated_bitmap; + /* + * On recovery from a clean shutdown we don't normally read the journal, + * but we still want to resume writing from where we left off so we + * don't overwrite more than is necessary, for list journal debugging: + */ + __le32 last_journal_bucket; + __le32 last_journal_bucket_offset; +}; + +/* + * This limit comes from the bucket_gens array - it's a single allocation, and + * kernel allocation are limited to INT_MAX + */ +#define BCH_MEMBER_NBUCKETS_MAX (INT_MAX - 64) + +#define BCH_MEMBER_V1_BYTES 56 + +LE64_BITMASK(BCH_MEMBER_STATE, struct bch_member, flags, 0, 4) +/* 4-14 unused, was TIER, HAS_(META)DATA, REPLACEMENT */ +LE64_BITMASK(BCH_MEMBER_DISCARD, struct bch_member, flags, 14, 15) +LE64_BITMASK(BCH_MEMBER_DATA_ALLOWED, struct bch_member, flags, 15, 20) +LE64_BITMASK(BCH_MEMBER_GROUP, struct bch_member, flags, 20, 28) +LE64_BITMASK(BCH_MEMBER_DURABILITY, struct bch_member, flags, 28, 30) +LE64_BITMASK(BCH_MEMBER_FREESPACE_INITIALIZED, + struct bch_member, flags, 30, 31) + +#if 0 +LE64_BITMASK(BCH_MEMBER_NR_READ_ERRORS, struct bch_member, flags[1], 0, 20); +LE64_BITMASK(BCH_MEMBER_NR_WRITE_ERRORS,struct bch_member, flags[1], 20, 40); +#endif + +#define BCH_MEMBER_STATES() \ + x(rw, 0) \ + x(ro, 1) \ + x(failed, 2) \ + x(spare, 3) + +enum bch_member_state { +#define x(t, n) BCH_MEMBER_STATE_##t = n, + BCH_MEMBER_STATES() +#undef x + BCH_MEMBER_STATE_NR +}; + +struct bch_sb_field_members_v1 { + struct bch_sb_field field; + struct bch_member _members[]; //Members are now variable size +}; + +struct bch_sb_field_members_v2 { + struct bch_sb_field field; + __le16 member_bytes; //size of single member entry + u8 pad[6]; + struct bch_member _members[]; +}; + +#endif /* _BCACHEFS_SB_MEMBERS_FORMAT_H */ diff --git a/fs/bcachefs/snapshot.c b/fs/bcachefs/snapshot.c index 629900a5e6411c2ee78e9be528646c973b500b09..51918acfd72681cc2249a0b123f1e020ecf6e3d5 100644 --- a/fs/bcachefs/snapshot.c +++ b/fs/bcachefs/snapshot.c @@ -1042,6 +1042,25 @@ err: return ret; } +int bch2_check_key_has_snapshot(struct btree_trans *trans, + struct btree_iter *iter, + struct bkey_s_c k) +{ + struct bch_fs *c = trans->c; + struct printbuf buf = PRINTBUF; + int ret = 0; + + if (fsck_err_on(!bch2_snapshot_equiv(c, k.k->p.snapshot), c, + bkey_in_missing_snapshot, + "key in missing snapshot %s, delete?", + (bch2_bkey_val_to_text(&buf, c, k), buf.buf))) + ret = bch2_btree_delete_at(trans, iter, + BTREE_UPDATE_internal_snapshot_node) ?: 1; +fsck_err: + printbuf_exit(&buf); + return ret; +} + /* * Mark a snapshot as deleted, for future cleanup: */ @@ -1351,35 +1370,39 @@ int bch2_snapshot_node_create(struct btree_trans *trans, u32 parent, * that key to snapshot leaf nodes, where we can mutate it */ -static int snapshot_delete_key(struct btree_trans *trans, +static int delete_dead_snapshots_process_key(struct btree_trans *trans, struct btree_iter *iter, struct bkey_s_c k, snapshot_id_list *deleted, snapshot_id_list *equiv_seen, struct bpos *last_pos) { + int ret = bch2_check_key_has_snapshot(trans, iter, k); + if (ret) + return ret < 0 ? ret : 0; + struct bch_fs *c = trans->c; u32 equiv = bch2_snapshot_equiv(c, k.k->p.snapshot); + if (!equiv) /* key for invalid snapshot node, but we chose not to delete */ + return 0; if (!bkey_eq(k.k->p, *last_pos)) equiv_seen->nr = 0; - *last_pos = k.k->p; - if (snapshot_list_has_id(deleted, k.k->p.snapshot) || - snapshot_list_has_id(equiv_seen, equiv)) { + if (snapshot_list_has_id(deleted, k.k->p.snapshot)) return bch2_btree_delete_at(trans, iter, BTREE_UPDATE_internal_snapshot_node); - } else { - return snapshot_list_add(c, equiv_seen, equiv); - } -} -static int move_key_to_correct_snapshot(struct btree_trans *trans, - struct btree_iter *iter, - struct bkey_s_c k) -{ - struct bch_fs *c = trans->c; - u32 equiv = bch2_snapshot_equiv(c, k.k->p.snapshot); + if (!bpos_eq(*last_pos, k.k->p) && + snapshot_list_has_id(equiv_seen, equiv)) + return bch2_btree_delete_at(trans, iter, + BTREE_UPDATE_internal_snapshot_node); + + *last_pos = k.k->p; + + ret = snapshot_list_add_nodup(c, equiv_seen, equiv); + if (ret) + return ret; /* * When we have a linear chain of snapshot nodes, we consider @@ -1389,21 +1412,20 @@ static int move_key_to_correct_snapshot(struct btree_trans *trans, * * If there are multiple keys in different snapshots at the same * position, we're only going to keep the one in the newest - * snapshot - the rest have been overwritten and are redundant, - * and for the key we're going to keep we need to move it to the - * equivalance class ID if it's not there already. + * snapshot (we delete the others above) - the rest have been + * overwritten and are redundant, and for the key we're going to keep we + * need to move it to the equivalance class ID if it's not there + * already. */ if (equiv != k.k->p.snapshot) { struct bkey_i *new = bch2_bkey_make_mut_noupdate(trans, k); - struct btree_iter new_iter; - int ret; - - ret = PTR_ERR_OR_ZERO(new); + int ret = PTR_ERR_OR_ZERO(new); if (ret) return ret; new->k.p.snapshot = equiv; + struct btree_iter new_iter; bch2_trans_iter_init(trans, &new_iter, iter->btree_id, new->k.p, BTREE_ITER_all_snapshots| BTREE_ITER_cached| @@ -1538,7 +1560,6 @@ int bch2_delete_dead_snapshots(struct bch_fs *c) struct btree_trans *trans; snapshot_id_list deleted = { 0 }; snapshot_id_list deleted_interior = { 0 }; - u32 id; int ret = 0; if (!test_and_clear_bit(BCH_FS_need_delete_dead_snapshots, &c->flags)) @@ -1585,33 +1606,20 @@ int bch2_delete_dead_snapshots(struct bch_fs *c) if (ret) goto err; - for (id = 0; id < BTREE_ID_NR; id++) { + for (unsigned btree = 0; btree < BTREE_ID_NR; btree++) { struct bpos last_pos = POS_MIN; snapshot_id_list equiv_seen = { 0 }; struct disk_reservation res = { 0 }; - if (!btree_type_has_snapshots(id)) - continue; - - /* - * deleted inodes btree is maintained by a trigger on the inodes - * btree - no work for us to do here, and it's not safe to scan - * it because we'll see out of date keys due to the btree write - * buffer: - */ - if (id == BTREE_ID_deleted_inodes) + if (!btree_type_has_snapshots(btree)) continue; ret = for_each_btree_key_commit(trans, iter, - id, POS_MIN, - BTREE_ITER_prefetch|BTREE_ITER_all_snapshots, k, - &res, NULL, BCH_TRANS_COMMIT_no_enospc, - snapshot_delete_key(trans, &iter, k, &deleted, &equiv_seen, &last_pos)) ?: - for_each_btree_key_commit(trans, iter, - id, POS_MIN, + btree, POS_MIN, BTREE_ITER_prefetch|BTREE_ITER_all_snapshots, k, &res, NULL, BCH_TRANS_COMMIT_no_enospc, - move_key_to_correct_snapshot(trans, &iter, k)); + delete_dead_snapshots_process_key(trans, &iter, k, &deleted, + &equiv_seen, &last_pos)); bch2_disk_reservation_put(c, &res); darray_exit(&equiv_seen); diff --git a/fs/bcachefs/snapshot.h b/fs/bcachefs/snapshot.h index ab13d8f4b41e1e934c1b5f5c21975358df128019..31b0ee03e96288e8dd525fab7f65e9eb56020990 100644 --- a/fs/bcachefs/snapshot.h +++ b/fs/bcachefs/snapshot.h @@ -242,6 +242,7 @@ int bch2_snapshot_node_create(struct btree_trans *, u32, int bch2_check_snapshot_trees(struct bch_fs *); int bch2_check_snapshots(struct bch_fs *); int bch2_reconstruct_snapshots(struct bch_fs *); +int bch2_check_key_has_snapshot(struct btree_trans *, struct btree_iter *, struct bkey_s_c); int bch2_snapshot_node_set_deleted(struct btree_trans *, u32); void bch2_delete_dead_snapshots_work(struct work_struct *); diff --git a/fs/bcachefs/super-io.c b/fs/bcachefs/super-io.c index f1bee6c5222d24b0dd32da210c7924b350c05d59..d73a0222f709541e5fadd288360c202d787d04e3 100644 --- a/fs/bcachefs/super-io.c +++ b/fs/bcachefs/super-io.c @@ -1132,18 +1132,12 @@ bool bch2_check_version_downgrade(struct bch_fs *c) * c->sb will be checked before we write the superblock, so update it as * well: */ - if (BCH_SB_VERSION_UPGRADE_COMPLETE(c->disk_sb.sb) > bcachefs_metadata_version_current) { + if (BCH_SB_VERSION_UPGRADE_COMPLETE(c->disk_sb.sb) > bcachefs_metadata_version_current) SET_BCH_SB_VERSION_UPGRADE_COMPLETE(c->disk_sb.sb, bcachefs_metadata_version_current); - c->sb.version_upgrade_complete = bcachefs_metadata_version_current; - } - if (c->sb.version > bcachefs_metadata_version_current) { + if (c->sb.version > bcachefs_metadata_version_current) c->disk_sb.sb->version = cpu_to_le16(bcachefs_metadata_version_current); - c->sb.version = bcachefs_metadata_version_current; - } - if (c->sb.version_min > bcachefs_metadata_version_current) { + if (c->sb.version_min > bcachefs_metadata_version_current) c->disk_sb.sb->version_min = cpu_to_le16(bcachefs_metadata_version_current); - c->sb.version_min = bcachefs_metadata_version_current; - } c->disk_sb.sb->compat[0] &= cpu_to_le64((1ULL << BCH_COMPAT_NR) - 1); return ret; } diff --git a/fs/bcachefs/super.c b/fs/bcachefs/super.c index 2206a8dee693ab1ce183ec621860c72fcf5bb7e9..df2bea38e83f0ff8996b4df57fec86374dfdf34c 100644 --- a/fs/bcachefs/super.c +++ b/fs/bcachefs/super.c @@ -564,7 +564,7 @@ static void __bch2_fs_free(struct bch_fs *c) BUG_ON(atomic_read(&c->journal_keys.ref)); bch2_fs_btree_write_buffer_exit(c); percpu_free_rwsem(&c->mark_lock); - EBUG_ON(percpu_u64_get(c->online_reserved)); + EBUG_ON(c->online_reserved && percpu_u64_get(c->online_reserved)); free_percpu(c->online_reserved); darray_exit(&c->btree_roots_extra); diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h index 91c994b569f3b805600e5979c3c07aff12d102a0..6ed495ca7a311dd974e3a4359ca4ada94ec22aec 100644 --- a/fs/btrfs/btrfs_inode.h +++ b/fs/btrfs/btrfs_inode.h @@ -89,6 +89,16 @@ enum { BTRFS_INODE_FREE_SPACE_INODE, /* Set when there are no capabilities in XATTs for the inode. */ BTRFS_INODE_NO_CAP_XATTR, + /* + * Set if an error happened when doing a COW write before submitting a + * bio or during writeback. Used for both buffered writes and direct IO + * writes. This is to signal a fast fsync that it has to wait for + * ordered extents to complete and therefore not log extent maps that + * point to unwritten extents (when an ordered extent completes and it + * has the BTRFS_ORDERED_IOERR flag set, it drops extent maps in its + * range). + */ + BTRFS_INODE_COW_WRITE_ERROR, }; /* in memory btrfs inode */ diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index e764ac3f22e23674b8969ee2641634d1916e14fb..d90138683a0a3031ed9e04060d6d1fc0c84a03ce 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -1885,6 +1885,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) */ if (full_sync || btrfs_is_zoned(fs_info)) { ret = btrfs_wait_ordered_range(inode, start, len); + clear_bit(BTRFS_INODE_COW_WRITE_ERROR, &BTRFS_I(inode)->runtime_flags); } else { /* * Get our ordered extents as soon as possible to avoid doing @@ -1894,6 +1895,21 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) btrfs_get_ordered_extents_for_logging(BTRFS_I(inode), &ctx.ordered_extents); ret = filemap_fdatawait_range(inode->i_mapping, start, end); + if (ret) + goto out_release_extents; + + /* + * Check and clear the BTRFS_INODE_COW_WRITE_ERROR now after + * starting and waiting for writeback, because for buffered IO + * it may have been set during the end IO callback + * (end_bbio_data_write() -> btrfs_finish_ordered_extent()) in + * case an error happened and we need to wait for ordered + * extents to complete so that any extent maps that point to + * unwritten locations are dropped and we don't log them. + */ + if (test_and_clear_bit(BTRFS_INODE_COW_WRITE_ERROR, + &BTRFS_I(inode)->runtime_flags)) + ret = btrfs_wait_ordered_range(inode, start, len); } if (ret) diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c index c5bdd674f55c534f03c52c7c6d8c8b3f6cf116ba..35a413ce935d9e6ce67bf103940b879e43ab3ead 100644 --- a/fs/btrfs/ordered-data.c +++ b/fs/btrfs/ordered-data.c @@ -388,6 +388,37 @@ bool btrfs_finish_ordered_extent(struct btrfs_ordered_extent *ordered, ret = can_finish_ordered_extent(ordered, page, file_offset, len, uptodate); spin_unlock_irqrestore(&inode->ordered_tree_lock, flags); + /* + * If this is a COW write it means we created new extent maps for the + * range and they point to unwritten locations if we got an error either + * before submitting a bio or during IO. + * + * We have marked the ordered extent with BTRFS_ORDERED_IOERR, and we + * are queuing its completion below. During completion, at + * btrfs_finish_one_ordered(), we will drop the extent maps for the + * unwritten extents. + * + * However because completion runs in a work queue we can end up having + * a fast fsync running before that. In the case of direct IO, once we + * unlock the inode the fsync might start, and we queue the completion + * before unlocking the inode. In the case of buffered IO when writeback + * finishes (end_bbio_data_write()) we queue the completion, so if the + * writeback was triggered by a fast fsync, the fsync might start + * logging before ordered extent completion runs in the work queue. + * + * The fast fsync will log file extent items based on the extent maps it + * finds, so if by the time it collects extent maps the ordered extent + * completion didn't happen yet, it will log file extent items that + * point to unwritten extents, resulting in a corruption if a crash + * happens and the log tree is replayed. Note that a fast fsync does not + * wait for completion of ordered extents in order to reduce latency. + * + * Set a flag in the inode so that the next fast fsync will wait for + * ordered extents to complete before starting to log. + */ + if (!uptodate && !test_bit(BTRFS_ORDERED_NOCOW, &ordered->flags)) + set_bit(BTRFS_INODE_COW_WRITE_ERROR, &inode->runtime_flags); + if (ret) btrfs_queue_ordered_fn(ordered); return ret; diff --git a/fs/dcache.c b/fs/dcache.c index 1ee6404b430baa5bd66972f64315db0ee1944f3a..407095188f83a7faeec0a8c0542fb6598952b7be 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -2360,17 +2360,19 @@ EXPORT_SYMBOL(d_hash_and_lookup); * - unhash this dentry and free it. * * Usually, we want to just turn this into - * a negative dentry, but certain workloads can - * generate a large number of negative dentries. - * Therefore, it would be better to simply - * unhash it. + * a negative dentry, but if anybody else is + * currently using the dentry or the inode + * we can't do that and we fall back on removing + * it from the hash queues and waiting for + * it to be deleted later when it has no users */ /** * d_delete - delete a dentry * @dentry: The dentry to delete * - * Remove the dentry from the hash queues so it can be deleted later. + * Turn the dentry into a negative dentry if possible, otherwise + * remove it from the hash queues so it can be deleted later */ void d_delete(struct dentry * dentry) @@ -2379,8 +2381,6 @@ void d_delete(struct dentry * dentry) spin_lock(&inode->i_lock); spin_lock(&dentry->d_lock); - __d_drop(dentry); - /* * Are we the only user? */ @@ -2388,6 +2388,7 @@ void d_delete(struct dentry * dentry) dentry->d_flags &= ~DCACHE_CANT_MOUNT; dentry_unlink_inode(dentry); } else { + __d_drop(dentry); spin_unlock(&dentry->d_lock); spin_unlock(&inode->i_lock); } diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 41c8f0c68ef564960a4341f605d9da135e9f14ef..c5802a4593345a9de8c3b2685b04b6ef0e7fdc41 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -898,11 +898,11 @@ static bool iomap_write_end(struct iomap_iter *iter, loff_t pos, size_t len, static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) { loff_t length = iomap_length(iter); - size_t chunk = PAGE_SIZE << MAX_PAGECACHE_ORDER; loff_t pos = iter->pos; ssize_t total_written = 0; long status = 0; struct address_space *mapping = iter->inode->i_mapping; + size_t chunk = mapping_max_folio_size(mapping); unsigned int bdp_flags = (iter->flags & IOMAP_NOWAIT) ? BDP_ASYNC : 0; do { diff --git a/fs/netfs/buffered_write.c b/fs/netfs/buffered_write.c index 1121601536d1844eddff472160adb61d1982040b..07bc1fd4353092acb9697f76b487c59f290a6f6c 100644 --- a/fs/netfs/buffered_write.c +++ b/fs/netfs/buffered_write.c @@ -181,7 +181,7 @@ ssize_t netfs_perform_write(struct kiocb *iocb, struct iov_iter *iter, struct folio *folio, *writethrough = NULL; enum netfs_how_to_modify howto; enum netfs_folio_trace trace; - unsigned int bdp_flags = (iocb->ki_flags & IOCB_SYNC) ? 0: BDP_ASYNC; + unsigned int bdp_flags = (iocb->ki_flags & IOCB_NOWAIT) ? BDP_ASYNC : 0; ssize_t written = 0, ret, ret2; loff_t i_size, pos = iocb->ki_pos, from, to; size_t max_chunk = PAGE_SIZE << MAX_PAGECACHE_ORDER; diff --git a/fs/netfs/direct_write.c b/fs/netfs/direct_write.c index f516460e994e2ad142c0fd26ccb2528c7f1dae64..e14cd53ac9fd7559337036699fc39a8c49030a39 100644 --- a/fs/netfs/direct_write.c +++ b/fs/netfs/direct_write.c @@ -12,7 +12,7 @@ static void netfs_cleanup_dio_write(struct netfs_io_request *wreq) { struct inode *inode = wreq->inode; - unsigned long long end = wreq->start + wreq->len; + unsigned long long end = wreq->start + wreq->transferred; if (!wreq->error && i_size_read(inode) < end) { diff --git a/fs/netfs/objects.c b/fs/netfs/objects.c index c90d482b16505d319dede502bcf23c7bed38e254..f4a642727479252de0212f80fe32ef23468e0ebf 100644 --- a/fs/netfs/objects.c +++ b/fs/netfs/objects.c @@ -72,6 +72,7 @@ struct netfs_io_request *netfs_alloc_request(struct address_space *mapping, } } + atomic_inc(&ctx->io_count); trace_netfs_rreq_ref(rreq->debug_id, 1, netfs_rreq_trace_new); netfs_proc_add_rreq(rreq); netfs_stat(&netfs_n_rh_rreq); @@ -124,6 +125,7 @@ static void netfs_free_request(struct work_struct *work) { struct netfs_io_request *rreq = container_of(work, struct netfs_io_request, work); + struct netfs_inode *ictx = netfs_inode(rreq->inode); unsigned int i; trace_netfs_rreq(rreq, netfs_rreq_trace_free); @@ -142,6 +144,9 @@ static void netfs_free_request(struct work_struct *work) } kvfree(rreq->direct_bv); } + + if (atomic_dec_and_test(&ictx->io_count)) + wake_up_var(&ictx->io_count); call_rcu(&rreq->rcu, netfs_free_request_rcu); } diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c index 60112e4b2c5eb70e350a873feb2cc397517534a7..426cf87aaf2ecd97f82ea43eabb326b2c83b8080 100644 --- a/fs/netfs/write_collect.c +++ b/fs/netfs/write_collect.c @@ -510,7 +510,7 @@ reassess_streams: * stream has a gap that can be jumped. */ if (notes & SOME_EMPTY) { - unsigned long long jump_to = wreq->start + wreq->len; + unsigned long long jump_to = wreq->start + READ_ONCE(wreq->submitted); for (s = 0; s < NR_IO_STREAMS; s++) { stream = &wreq->io_streams[s]; @@ -690,10 +690,11 @@ void netfs_write_collection_worker(struct work_struct *work) wake_up_bit(&wreq->flags, NETFS_RREQ_IN_PROGRESS); if (wreq->iocb) { - wreq->iocb->ki_pos += wreq->transferred; + size_t written = min(wreq->transferred, wreq->len); + wreq->iocb->ki_pos += written; if (wreq->iocb->ki_complete) wreq->iocb->ki_complete( - wreq->iocb, wreq->error ? wreq->error : wreq->transferred); + wreq->iocb, wreq->error ? wreq->error : written); wreq->iocb = VFS_PTR_POISON; } diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c index e190043bc0daa724f749003a2a80e151e02806d3..3aa86e268f40d4a7b01a85715d41b7fd4a270da6 100644 --- a/fs/netfs/write_issue.c +++ b/fs/netfs/write_issue.c @@ -254,7 +254,7 @@ static void netfs_issue_write(struct netfs_io_request *wreq, stream->construct = NULL; if (subreq->start + subreq->len > wreq->start + wreq->submitted) - wreq->len = wreq->submitted = subreq->start + subreq->len - wreq->start; + WRITE_ONCE(wreq->submitted, subreq->start + subreq->len - wreq->start); netfs_do_issue_write(stream, subreq); } @@ -636,7 +636,12 @@ int netfs_end_writethrough(struct netfs_io_request *wreq, struct writeback_contr mutex_unlock(&ictx->wb_lock); - ret = wreq->error; + if (wreq->iocb) { + ret = -EIOCBQUEUED; + } else { + wait_on_bit(&wreq->flags, NETFS_RREQ_IN_PROGRESS, TASK_UNINTERRUPTIBLE); + ret = wreq->error; + } netfs_put_request(wreq, false, netfs_rreq_trace_put_return); return ret; } diff --git a/fs/signalfd.c b/fs/signalfd.c index 4a5614442dbfa733ff2dc9fa704532e8b5cc147c..ec7b2da2477a41c3b7ac04bbf85835febafd08f1 100644 --- a/fs/signalfd.c +++ b/fs/signalfd.c @@ -282,14 +282,10 @@ static int do_signalfd4(int ufd, sigset_t *mask, int flags) if (IS_ERR(file)) { put_unused_fd(ufd); kfree(ctx); - return ufd; + return PTR_ERR(file); } file->f_mode |= FMODE_NOWAIT; - /* - * When we call this, the initialization must be complete, since - * anon_inode_getfd() will install the fd. - */ fd_install(ufd, file); } else { struct fd f = fdget(ufd); diff --git a/fs/smb/client/cifsfs.c b/fs/smb/client/cifsfs.c index a665aac9be9f4bcef7c017a01577c45809dcefed..bb86fc0641d83ab3757fd50fb074a082894ae088 100644 --- a/fs/smb/client/cifsfs.c +++ b/fs/smb/client/cifsfs.c @@ -431,6 +431,7 @@ cifs_free_inode(struct inode *inode) static void cifs_evict_inode(struct inode *inode) { + netfs_wait_for_outstanding_io(inode); truncate_inode_pages_final(&inode->i_data); if (inode->i_state & I_PINNING_NETFS_WB) cifs_fscache_unuse_inode_cookie(inode, true); diff --git a/fs/smb/client/cifspdu.h b/fs/smb/client/cifspdu.h index c46d418c1c0c3ea065eaa5d4f4f751750df90080..a2072ab9e586d0d3ba34afe5bcab985b38537202 100644 --- a/fs/smb/client/cifspdu.h +++ b/fs/smb/client/cifspdu.h @@ -2574,7 +2574,7 @@ typedef struct { struct win_dev { - unsigned char type[8]; /* IntxCHR or IntxBLK or LnxFIFO*/ + unsigned char type[8]; /* IntxCHR or IntxBLK or LnxFIFO or LnxSOCK */ __le64 major; __le64 minor; } __attribute__((packed)); diff --git a/fs/smb/client/inode.c b/fs/smb/client/inode.c index 262576573eb5182b8a1729677e1b98e7bdee66a3..4a8aa1de95223dfc487e577e476be18d677b6477 100644 --- a/fs/smb/client/inode.c +++ b/fs/smb/client/inode.c @@ -606,6 +606,10 @@ cifs_sfu_type(struct cifs_fattr *fattr, const char *path, mnr = le64_to_cpu(*(__le64 *)(pbuf+16)); fattr->cf_rdev = MKDEV(mjr, mnr); } + } else if (memcmp("LnxSOCK", pbuf, 8) == 0) { + cifs_dbg(FYI, "Socket\n"); + fattr->cf_mode |= S_IFSOCK; + fattr->cf_dtype = DT_SOCK; } else if (memcmp("IntxLNK", pbuf, 7) == 0) { cifs_dbg(FYI, "Symlink\n"); fattr->cf_mode |= S_IFLNK; diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c index 4ce6c3121a7ef17866be79561fa4589360967a17..c8e536540895a2e6378ec16b640752134742d134 100644 --- a/fs/smb/client/smb2ops.c +++ b/fs/smb/client/smb2ops.c @@ -4997,6 +4997,9 @@ static int __cifs_sfu_make_node(unsigned int xid, struct inode *inode, pdev.major = cpu_to_le64(MAJOR(dev)); pdev.minor = cpu_to_le64(MINOR(dev)); break; + case S_IFSOCK: + strscpy(pdev.type, "LnxSOCK"); + break; case S_IFIFO: strscpy(pdev.type, "LnxFIFO"); break; diff --git a/fs/smb/common/cifs_arc4.c b/fs/smb/common/cifs_arc4.c index 043e4cb839fa235f7c2e05fac3fe558e82e9d4f2..df360ca47826ab7fa62f284ec6c71017732c6455 100644 --- a/fs/smb/common/cifs_arc4.c +++ b/fs/smb/common/cifs_arc4.c @@ -10,6 +10,7 @@ #include #include "arc4.h" +MODULE_DESCRIPTION("ARC4 Cipher Algorithm"); MODULE_LICENSE("GPL"); int cifs_arc4_setkey(struct arc4_ctx *ctx, const u8 *in_key, unsigned int key_len) diff --git a/fs/smb/common/cifs_md4.c b/fs/smb/common/cifs_md4.c index 50f78cfc6ce921efd8e8ea38e5237ac1fe7fe19a..7ee7f4dad90c00f5c15fe00a6874fbeeb631665d 100644 --- a/fs/smb/common/cifs_md4.c +++ b/fs/smb/common/cifs_md4.c @@ -24,6 +24,7 @@ #include #include "md4.h" +MODULE_DESCRIPTION("MD4 Message Digest Algorithm (RFC1320)"); MODULE_LICENSE("GPL"); static inline u32 lshift(u32 x, unsigned int s) diff --git a/fs/verity/measure.c b/fs/verity/measure.c index 3969d54158d1286c66aaf7a7890e7ab4f59749fa..175d2f1bc0893cbc669e0037117189f079c223ff 100644 --- a/fs/verity/measure.c +++ b/fs/verity/measure.c @@ -111,14 +111,15 @@ __bpf_kfunc_start_defs(); /** * bpf_get_fsverity_digest: read fsverity digest of file * @file: file to get digest from - * @digest_ptr: (out) dynptr for struct fsverity_digest + * @digest_p: (out) dynptr for struct fsverity_digest * * Read fsverity_digest of *file* into *digest_ptr*. * * Return: 0 on success, a negative value on error. */ -__bpf_kfunc int bpf_get_fsverity_digest(struct file *file, struct bpf_dynptr_kern *digest_ptr) +__bpf_kfunc int bpf_get_fsverity_digest(struct file *file, struct bpf_dynptr *digest_p) { + struct bpf_dynptr_kern *digest_ptr = (struct bpf_dynptr_kern *)digest_p; const struct inode *inode = file_inode(file); u32 dynptr_sz = __bpf_dynptr_size(digest_ptr); struct fsverity_digest *arg; diff --git a/fs/xfs/libxfs/xfs_alloc.c b/fs/xfs/libxfs/xfs_alloc.c index 6cb8b2ddc541b4b28d553ffee414b903725e97f8..6c55a6e88eba3ffb8ac76803b2b14ea03b45b467 100644 --- a/fs/xfs/libxfs/xfs_alloc.c +++ b/fs/xfs/libxfs/xfs_alloc.c @@ -1008,13 +1008,12 @@ xfs_alloc_cur_finish( struct xfs_alloc_arg *args, struct xfs_alloc_cur *acur) { - struct xfs_agf __maybe_unused *agf = args->agbp->b_addr; int error; ASSERT(acur->cnt && acur->bnolt); ASSERT(acur->bno >= acur->rec_bno); ASSERT(acur->bno + acur->len <= acur->rec_bno + acur->rec_len); - ASSERT(acur->rec_bno + acur->rec_len <= be32_to_cpu(agf->agf_length)); + ASSERT(xfs_verify_agbext(args->pag, acur->rec_bno, acur->rec_len)); error = xfs_alloc_fixup_trees(acur->cnt, acur->bnolt, acur->rec_bno, acur->rec_len, acur->bno, acur->len, 0); @@ -1217,7 +1216,6 @@ STATIC int /* error */ xfs_alloc_ag_vextent_exact( xfs_alloc_arg_t *args) /* allocation argument structure */ { - struct xfs_agf __maybe_unused *agf = args->agbp->b_addr; struct xfs_btree_cur *bno_cur;/* by block-number btree cursor */ struct xfs_btree_cur *cnt_cur;/* by count btree cursor */ int error; @@ -1297,7 +1295,7 @@ xfs_alloc_ag_vextent_exact( */ cnt_cur = xfs_cntbt_init_cursor(args->mp, args->tp, args->agbp, args->pag); - ASSERT(args->agbno + args->len <= be32_to_cpu(agf->agf_length)); + ASSERT(xfs_verify_agbext(args->pag, args->agbno, args->len)); error = xfs_alloc_fixup_trees(cnt_cur, bno_cur, fbno, flen, args->agbno, args->len, XFSA_FIXUP_BNO_OK); if (error) { diff --git a/fs/xfs/libxfs/xfs_attr.c b/fs/xfs/libxfs/xfs_attr.c index 430cd3244c143dc065f0490b1ec3c3feef47e2e3..f30bcc64100d56b7199fc6b73ee777f6a6e06aa9 100644 --- a/fs/xfs/libxfs/xfs_attr.c +++ b/fs/xfs/libxfs/xfs_attr.c @@ -329,26 +329,20 @@ xfs_attr_calc_size( return nblks; } -/* Initialize transaction reservation for attr operations */ -void -xfs_init_attr_trans( - struct xfs_da_args *args, - struct xfs_trans_res *tres, - unsigned int *total) +/* Initialize transaction reservation for an xattr set/replace/upsert */ +inline struct xfs_trans_res +xfs_attr_set_resv( + const struct xfs_da_args *args) { - struct xfs_mount *mp = args->dp->i_mount; - - if (args->value) { - tres->tr_logres = M_RES(mp)->tr_attrsetm.tr_logres + - M_RES(mp)->tr_attrsetrt.tr_logres * - args->total; - tres->tr_logcount = XFS_ATTRSET_LOG_COUNT; - tres->tr_logflags = XFS_TRANS_PERM_LOG_RES; - *total = args->total; - } else { - *tres = M_RES(mp)->tr_attrrm; - *total = XFS_ATTRRM_SPACE_RES(mp); - } + struct xfs_mount *mp = args->dp->i_mount; + struct xfs_trans_res ret = { + .tr_logres = M_RES(mp)->tr_attrsetm.tr_logres + + M_RES(mp)->tr_attrsetrt.tr_logres * args->total, + .tr_logcount = XFS_ATTRSET_LOG_COUNT, + .tr_logflags = XFS_TRANS_PERM_LOG_RES, + }; + + return ret; } /* @@ -1006,7 +1000,7 @@ xfs_attr_set( struct xfs_trans_res tres; int error, local; int rmt_blks = 0; - unsigned int total; + unsigned int total = 0; ASSERT(!args->trans); @@ -1033,10 +1027,15 @@ xfs_attr_set( if (!local) rmt_blks = xfs_attr3_rmt_blocks(mp, args->valuelen); + + tres = xfs_attr_set_resv(args); + total = args->total; break; case XFS_ATTRUPDATE_REMOVE: XFS_STATS_INC(mp, xs_attr_remove); rmt_blks = xfs_attr3_max_rmt_blocks(mp); + tres = M_RES(mp)->tr_attrrm; + total = XFS_ATTRRM_SPACE_RES(mp); break; } @@ -1044,7 +1043,6 @@ xfs_attr_set( * Root fork attributes can use reserved data blocks for this * operation if necessary */ - xfs_init_attr_trans(args, &tres, &total); error = xfs_trans_alloc_inode(dp, &tres, total, 0, rsvd, &args->trans); if (error) return error; diff --git a/fs/xfs/libxfs/xfs_attr.h b/fs/xfs/libxfs/xfs_attr.h index 088cb7b301680ca024d58cf2ca8d7bbdc5a4ef8d..0e51d0723f9aa36c1519f6ad2c1881489a26dd8a 100644 --- a/fs/xfs/libxfs/xfs_attr.h +++ b/fs/xfs/libxfs/xfs_attr.h @@ -565,8 +565,7 @@ bool xfs_attr_check_namespace(unsigned int attr_flags); bool xfs_attr_namecheck(unsigned int attr_flags, const void *name, size_t length); int xfs_attr_calc_size(struct xfs_da_args *args, int *local); -void xfs_init_attr_trans(struct xfs_da_args *args, struct xfs_trans_res *tres, - unsigned int *total); +struct xfs_trans_res xfs_attr_set_resv(const struct xfs_da_args *args); /* * Check to see if the attr should be upgraded from non-existent or shortform to diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c index 3b3206d312d6fe157ff90e14f28e96ab39f2e583..c101cf266bc4dbf605e5ebe40cfe124dfc1ab8e2 100644 --- a/fs/xfs/libxfs/xfs_bmap.c +++ b/fs/xfs/libxfs/xfs_bmap.c @@ -6383,6 +6383,7 @@ xfs_bunmapi_range( error = xfs_defer_finish(tpp); if (error) goto out; + cond_resched(); } out: return error; diff --git a/fs/xfs/libxfs/xfs_inode_buf.c b/fs/xfs/libxfs/xfs_inode_buf.c index d79002343d0b631290d7dfdaf69e715075111c52..e7a7bfbe75b46a2145f7ed7b47d00bdab0afdfca 100644 --- a/fs/xfs/libxfs/xfs_inode_buf.c +++ b/fs/xfs/libxfs/xfs_inode_buf.c @@ -374,17 +374,37 @@ xfs_dinode_verify_fork( /* * For fork types that can contain local data, check that the fork * format matches the size of local data contained within the fork. - * - * For all types, check that when the size says the should be in extent - * or btree format, the inode isn't claiming it is in local format. */ if (whichfork == XFS_DATA_FORK) { - if (S_ISDIR(mode) || S_ISLNK(mode)) { + /* + * A directory small enough to fit in the inode must be stored + * in local format. The directory sf <-> extents conversion + * code updates the directory size accordingly. + */ + if (S_ISDIR(mode)) { + if (be64_to_cpu(dip->di_size) <= fork_size && + fork_format != XFS_DINODE_FMT_LOCAL) + return __this_address; + } + + /* + * A symlink with a target small enough to fit in the inode can + * be stored in extents format if xattrs were added (thus + * converting the data fork from shortform to remote format) + * and then removed. + */ + if (S_ISLNK(mode)) { if (be64_to_cpu(dip->di_size) <= fork_size && + fork_format != XFS_DINODE_FMT_EXTENTS && fork_format != XFS_DINODE_FMT_LOCAL) return __this_address; } + /* + * For all types, check that when the size says the fork should + * be in extent or btree format, the inode isn't claiming to be + * in local format. + */ if (be64_to_cpu(dip->di_size) > fork_size && fork_format == XFS_DINODE_FMT_LOCAL) return __this_address; diff --git a/fs/xfs/scrub/scrub.c b/fs/xfs/scrub/scrub.c index c013f0ba4f36b2cfc9a3fe7c8131b14954fcc10a..4cbcf7a86dbec5920af6b2070d9e5e5e287225e8 100644 --- a/fs/xfs/scrub/scrub.c +++ b/fs/xfs/scrub/scrub.c @@ -856,7 +856,7 @@ xfs_ioc_scrubv_metadata( if (vec_bytes > PAGE_SIZE) return -ENOMEM; - uvectors = (void __user *)(uintptr_t)head.svh_vectors; + uvectors = u64_to_user_ptr(head.svh_vectors); vectors = memdup_user(uvectors, vec_bytes); if (IS_ERR(vectors)) return PTR_ERR(vectors); diff --git a/fs/xfs/scrub/xfarray.c b/fs/xfs/scrub/xfarray.c index 9185ae7088d49a9dd5b1f8c133e92205f45fe1a3..cdd13ed9c569a7b46e385f234b32faa2ac7e3cd6 100644 --- a/fs/xfs/scrub/xfarray.c +++ b/fs/xfs/scrub/xfarray.c @@ -822,12 +822,14 @@ xfarray_sort_scan( /* Grab the first folio that backs this array element. */ if (!si->folio) { + struct folio *folio; loff_t next_pos; - si->folio = xfile_get_folio(si->array->xfile, idx_pos, + folio = xfile_get_folio(si->array->xfile, idx_pos, si->array->obj_size, XFILE_ALLOC); - if (IS_ERR(si->folio)) - return PTR_ERR(si->folio); + if (IS_ERR(folio)) + return PTR_ERR(folio); + si->folio = folio; si->first_folio_idx = xfarray_idx(si->array, folio_pos(si->folio) + si->array->obj_size - 1); @@ -1048,6 +1050,7 @@ xfarray_sort( out_free: trace_xfarray_sort_stats(si, error); + xfarray_sort_scan_done(si); kvfree(si); return error; } diff --git a/fs/xfs/xfs_attr_item.c b/fs/xfs/xfs_attr_item.c index 2b10ac4c5fce24d0df38b1fb2445e491c1aa389b..f683b7a9323f16146b1c8cad6e54a40ca056997d 100644 --- a/fs/xfs/xfs_attr_item.c +++ b/fs/xfs/xfs_attr_item.c @@ -746,7 +746,7 @@ xfs_attr_recover_work( struct xfs_attri_log_format *attrp; struct xfs_attri_log_nameval *nv = attrip->attri_nameval; int error; - int total; + unsigned int total = 0; /* * First check the validity of the attr described by the ATTRI. If any @@ -763,7 +763,20 @@ xfs_attr_recover_work( return PTR_ERR(attr); args = attr->xattri_da_args; - xfs_init_attr_trans(args, &resv, &total); + switch (xfs_attr_intent_op(attr)) { + case XFS_ATTRI_OP_FLAGS_PPTR_SET: + case XFS_ATTRI_OP_FLAGS_PPTR_REPLACE: + case XFS_ATTRI_OP_FLAGS_SET: + case XFS_ATTRI_OP_FLAGS_REPLACE: + resv = xfs_attr_set_resv(args); + total = args->total; + break; + case XFS_ATTRI_OP_FLAGS_PPTR_REMOVE: + case XFS_ATTRI_OP_FLAGS_REMOVE: + resv = M_RES(mp)->tr_attrrm; + total = XFS_ATTRRM_SPACE_RES(mp); + break; + } resv = xlog_recover_resv(&resv); error = xfs_trans_alloc(mp, &resv, total, 0, XFS_TRANS_RESERVE, &tp); if (error) diff --git a/fs/xfs/xfs_handle.c b/fs/xfs/xfs_handle.c index c8785ed595434223086affd3b0a71d18e25662bb..a3f16e9b6fe5b53baf7e75e6391435772d51044b 100644 --- a/fs/xfs/xfs_handle.c +++ b/fs/xfs/xfs_handle.c @@ -773,11 +773,6 @@ xfs_getparents_expand_lastrec( trace_xfs_getparents_expand_lastrec(gpx->ip, gp, &gpx->context, gpr); } -static inline void __user *u64_to_uptr(u64 val) -{ - return (void __user *)(uintptr_t)val; -} - /* Retrieve the parent pointers for a given inode. */ STATIC int xfs_getparents( @@ -862,7 +857,7 @@ xfs_getparents( ASSERT(gpx->context.firstu <= gpx->gph.gph_request.gp_bufsize); /* Copy the records to userspace. */ - if (copy_to_user(u64_to_uptr(gpx->gph.gph_request.gp_buffer), + if (copy_to_user(u64_to_user_ptr(gpx->gph.gph_request.gp_buffer), gpx->krecords, gpx->context.firstu)) error = -EFAULT; diff --git a/fs/xfs/xfs_iwalk.c b/fs/xfs/xfs_iwalk.c index 730c8d48da2827714127324cc5f8954465aef4c4..86f14ec7c31fede6fc22d9b9d7a5eec26b34a79f 100644 --- a/fs/xfs/xfs_iwalk.c +++ b/fs/xfs/xfs_iwalk.c @@ -351,7 +351,6 @@ xfs_iwalk_run_callbacks( int *has_more) { struct xfs_mount *mp = iwag->mp; - struct xfs_inobt_rec_incore *irec; xfs_agino_t next_agino; int error; @@ -361,8 +360,8 @@ xfs_iwalk_run_callbacks( /* Delete cursor but remember the last record we cached... */ xfs_iwalk_del_inobt(iwag->tp, curpp, agi_bpp, 0); - irec = &iwag->recs[iwag->nr_recs - 1]; - ASSERT(next_agino >= irec->ir_startino + XFS_INODES_PER_CHUNK); + ASSERT(next_agino >= iwag->recs[iwag->nr_recs - 1].ir_startino + + XFS_INODES_PER_CHUNK); if (iwag->drop_trans) { xfs_trans_cancel(iwag->tp); diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/xfs_reflink.c index 063a2e00d1691564c4bd5bce9bef31955299aa11..265a2a418bc7b098a595237dbbf3a7931a66e94d 100644 --- a/fs/xfs/xfs_reflink.c +++ b/fs/xfs/xfs_reflink.c @@ -1387,6 +1387,7 @@ xfs_reflink_remap_blocks( destoff += imap.br_blockcount; len -= imap.br_blockcount; remapped_len += imap.br_blockcount; + cond_resched(); } if (error) diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h index 82570f77e8172a5d88b63170ba7b44fee1dc6486..2a74fa9d0ce554831bb3fa43b79a8a17aec91562 100644 --- a/include/drm/drm_buddy.h +++ b/include/drm/drm_buddy.h @@ -56,8 +56,8 @@ struct drm_buddy_block { struct list_head tmp_link; }; -/* Order-zero must be at least PAGE_SIZE */ -#define DRM_BUDDY_MAX_ORDER (63 - PAGE_SHIFT) +/* Order-zero must be at least SZ_4K */ +#define DRM_BUDDY_MAX_ORDER (63 - 12) /* * Binary Buddy System. @@ -85,7 +85,7 @@ struct drm_buddy { unsigned int n_roots; unsigned int max_order; - /* Must be at least PAGE_SIZE */ + /* Must be at least SZ_4K */ u64 chunk_size; u64 size; u64 avail; diff --git a/include/linux/blk-integrity.h b/include/linux/blk-integrity.h index e253e7bd0d1793f23f1de2fb03a9032c66bbe659..7428cb43952da0d31b8c09858f732f71c1d848aa 100644 --- a/include/linux/blk-integrity.h +++ b/include/linux/blk-integrity.h @@ -66,12 +66,6 @@ blk_integrity_queue_supports_integrity(struct request_queue *q) return q->integrity.profile; } -static inline void blk_queue_max_integrity_segments(struct request_queue *q, - unsigned int segs) -{ - q->limits.max_integrity_segments = segs; -} - static inline unsigned short queue_max_integrity_segments(const struct request_queue *q) { @@ -151,10 +145,6 @@ static inline void blk_integrity_register(struct gendisk *d, static inline void blk_integrity_unregister(struct gendisk *d) { } -static inline void blk_queue_max_integrity_segments(struct request_queue *q, - unsigned int segs) -{ -} static inline unsigned short queue_max_integrity_segments(const struct request_queue *q) { diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index aefdda9f4ec711cba939e15722f0372cef1ac035..24c36929920b76934fe32eea7f95a3808542dcb0 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -186,6 +186,7 @@ struct gendisk { */ unsigned int nr_zones; unsigned int zone_capacity; + unsigned int last_zone_capacity; unsigned long *conv_zones_bitmap; unsigned int zone_wplugs_hash_bits; spinlock_t zone_wplugs_lock; diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 5e694a308081aa157f19a54b4552dbef695db28c..f636b4998bf71857825a5b3d691de257f73660d4 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1612,6 +1612,7 @@ struct bpf_link_ops { struct bpf_link_info *info); int (*update_map)(struct bpf_link *link, struct bpf_map *new_map, struct bpf_map *old_map); + __poll_t (*poll)(struct file *file, struct poll_table_struct *pts); }; struct bpf_tramp_link { @@ -1730,9 +1731,9 @@ struct bpf_struct_ops { int (*init_member)(const struct btf_type *t, const struct btf_member *member, void *kdata, const void *udata); - int (*reg)(void *kdata); - void (*unreg)(void *kdata); - int (*update)(void *kdata, void *old_kdata); + int (*reg)(void *kdata, struct bpf_link *link); + void (*unreg)(void *kdata, struct bpf_link *link); + int (*update)(void *kdata, void *old_kdata, struct bpf_link *link); int (*validate)(void *kdata); void *cfi_stubs; struct module *owner; @@ -2333,6 +2334,7 @@ int bpf_link_prime(struct bpf_link *link, struct bpf_link_primer *primer); int bpf_link_settle(struct bpf_link_primer *primer); void bpf_link_cleanup(struct bpf_link_primer *primer); void bpf_link_inc(struct bpf_link *link); +struct bpf_link *bpf_link_inc_not_zero(struct bpf_link *link); void bpf_link_put(struct bpf_link *link); int bpf_link_new_fd(struct bpf_link *link); struct bpf_link *bpf_link_get_from_fd(u32 ufd); @@ -2704,6 +2706,11 @@ static inline void bpf_link_inc(struct bpf_link *link) { } +static inline struct bpf_link *bpf_link_inc_not_zero(struct bpf_link *link) +{ + return NULL; +} + static inline void bpf_link_put(struct bpf_link *link) { } @@ -3258,8 +3265,8 @@ u32 bpf_sock_convert_ctx_access(enum bpf_access_type type, struct bpf_insn *insn_buf, struct bpf_prog *prog, u32 *target_size); -int bpf_dynptr_from_skb_rdonly(struct sk_buff *skb, u64 flags, - struct bpf_dynptr_kern *ptr); +int bpf_dynptr_from_skb_rdonly(struct __sk_buff *skb, u64 flags, + struct bpf_dynptr *ptr); #else static inline bool bpf_sock_common_is_valid_access(int off, int size, enum bpf_access_type type, @@ -3281,8 +3288,8 @@ static inline u32 bpf_sock_convert_ctx_access(enum bpf_access_type type, { return 0; } -static inline int bpf_dynptr_from_skb_rdonly(struct sk_buff *skb, u64 flags, - struct bpf_dynptr_kern *ptr) +static inline int bpf_dynptr_from_skb_rdonly(struct __sk_buff *skb, u64 flags, + struct bpf_dynptr *ptr) { return -EOPNOTSUPP; } diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 50aa87f8d77ff6c85d5e362ec9e4ecac15e6dc2f..2b54e25d236458c6538ac4a0fc6c43349c62bbc4 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -73,7 +73,10 @@ enum bpf_iter_state { struct bpf_reg_state { /* Ordering of fields matters. See states_equal() */ enum bpf_reg_type type; - /* Fixed part of pointer offset, pointer types only */ + /* + * Fixed part of pointer offset, pointer types only. + * Or constant delta between "linked" scalars with the same ID. + */ s32 off; union { /* valid when type == PTR_TO_PACKET */ @@ -167,6 +170,13 @@ struct bpf_reg_state { * Similarly to dynptrs, we use ID to track "belonging" of a reference * to a specific instance of bpf_iter. */ + /* + * Upper bit of ID is used to remember relationship between "linked" + * registers. Example: + * r1 = r2; both will have r1->id == r2->id == N + * r1 += 10; r1->id == N | BPF_ADD_CONST and r1->off == 10 + */ +#define BPF_ADD_CONST (1U << 31) u32 id; /* PTR_TO_SOCKET and PTR_TO_TCP_SOCK could be a ptr returned * from a pointer-cast helper, bpf_sk_fullsock() and diff --git a/include/linux/btf.h b/include/linux/btf.h index f9e56fd12a9fd52d94347da85f7ae5b37f5ef330..56d91daacdba303f4a6d891660563d2b1a6e28d8 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -531,6 +531,7 @@ s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id); int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_cnt, struct module *owner); struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf, u32 btf_id); +bool btf_is_projection_of(const char *pname, const char *tname); bool btf_is_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, const struct btf_type *t, enum bpf_prog_type prog_type, int arg); diff --git a/include/linux/etherdevice.h b/include/linux/etherdevice.h index 2ad1ffa4ccb9fec92cfb8d37897a4c5696d9dff6..0ed47d00549ba72d4c39451ce8d0c3fcf0fee108 100644 --- a/include/linux/etherdevice.h +++ b/include/linux/etherdevice.h @@ -636,6 +636,14 @@ static inline void eth_skb_pkt_type(struct sk_buff *skb, } } +static inline struct ethhdr *eth_skb_pull_mac(struct sk_buff *skb) +{ + struct ethhdr *eth = (struct ethhdr *)skb->data; + + skb_pull_inline(skb, ETH_HLEN); + return eth; +} + /** * eth_skb_pad - Pad buffer to mininum number of octets for Ethernet frame * @skb: Buffer to pad diff --git a/include/linux/filter.h b/include/linux/filter.h index 0f12cf01070e34581c6f1a8d519d777304a649d9..b02aea291b7e8ea22468d6a6ecb3e0c188dff026 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -1406,7 +1406,7 @@ struct bpf_sock_ops_kern { struct bpf_sysctl_kern { struct ctl_table_header *head; - struct ctl_table *table; + const struct ctl_table *table; void *cur_val; size_t cur_len; void *new_val; diff --git a/include/linux/i2c.h b/include/linux/i2c.h index 5e6cd43a6dbdd9fcd29e30d59492c736a149e25a..9709537370ee9b993e7baf1118588755e1f98017 100644 --- a/include/linux/i2c.h +++ b/include/linux/i2c.h @@ -852,7 +852,6 @@ static inline void i2c_mark_adapter_resumed(struct i2c_adapter *adap) /* i2c adapter classes (bitmask) */ #define I2C_CLASS_HWMON (1<<0) /* lm_sensors, ... */ -#define I2C_CLASS_SPD (1<<7) /* Memory modules */ /* Warn users that the adapter doesn't support classes anymore */ #define I2C_CLASS_DEPRECATED (1<<8) diff --git a/include/linux/ieee80211.h b/include/linux/ieee80211.h index de2dce743ee2cbbaf2929a4575c913bb8bddef52..769008a518094e930172e593972d225a7950fc6a 100644 --- a/include/linux/ieee80211.h +++ b/include/linux/ieee80211.h @@ -373,6 +373,7 @@ struct ieee80211_trigger { /** * ieee80211_has_tods - check if IEEE80211_FCTL_TODS is set * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame has to-DS set */ static inline bool ieee80211_has_tods(__le16 fc) { @@ -382,6 +383,7 @@ static inline bool ieee80211_has_tods(__le16 fc) /** * ieee80211_has_fromds - check if IEEE80211_FCTL_FROMDS is set * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame has from-DS set */ static inline bool ieee80211_has_fromds(__le16 fc) { @@ -391,6 +393,7 @@ static inline bool ieee80211_has_fromds(__le16 fc) /** * ieee80211_has_a4 - check if IEEE80211_FCTL_TODS and IEEE80211_FCTL_FROMDS are set * @fc: frame control bytes in little-endian byteorder + * Return: whether or not it's a 4-address frame (from-DS and to-DS set) */ static inline bool ieee80211_has_a4(__le16 fc) { @@ -401,6 +404,7 @@ static inline bool ieee80211_has_a4(__le16 fc) /** * ieee80211_has_morefrags - check if IEEE80211_FCTL_MOREFRAGS is set * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame has more fragments (more frags bit set) */ static inline bool ieee80211_has_morefrags(__le16 fc) { @@ -410,6 +414,7 @@ static inline bool ieee80211_has_morefrags(__le16 fc) /** * ieee80211_has_retry - check if IEEE80211_FCTL_RETRY is set * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the retry flag is set */ static inline bool ieee80211_has_retry(__le16 fc) { @@ -419,6 +424,7 @@ static inline bool ieee80211_has_retry(__le16 fc) /** * ieee80211_has_pm - check if IEEE80211_FCTL_PM is set * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the power management flag is set */ static inline bool ieee80211_has_pm(__le16 fc) { @@ -428,6 +434,7 @@ static inline bool ieee80211_has_pm(__le16 fc) /** * ieee80211_has_moredata - check if IEEE80211_FCTL_MOREDATA is set * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the more data flag is set */ static inline bool ieee80211_has_moredata(__le16 fc) { @@ -437,6 +444,7 @@ static inline bool ieee80211_has_moredata(__le16 fc) /** * ieee80211_has_protected - check if IEEE80211_FCTL_PROTECTED is set * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the protected flag is set */ static inline bool ieee80211_has_protected(__le16 fc) { @@ -446,6 +454,7 @@ static inline bool ieee80211_has_protected(__le16 fc) /** * ieee80211_has_order - check if IEEE80211_FCTL_ORDER is set * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the order flag is set */ static inline bool ieee80211_has_order(__le16 fc) { @@ -455,6 +464,7 @@ static inline bool ieee80211_has_order(__le16 fc) /** * ieee80211_is_mgmt - check if type is IEEE80211_FTYPE_MGMT * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame type is management */ static inline bool ieee80211_is_mgmt(__le16 fc) { @@ -465,6 +475,7 @@ static inline bool ieee80211_is_mgmt(__le16 fc) /** * ieee80211_is_ctl - check if type is IEEE80211_FTYPE_CTL * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame type is control */ static inline bool ieee80211_is_ctl(__le16 fc) { @@ -475,6 +486,7 @@ static inline bool ieee80211_is_ctl(__le16 fc) /** * ieee80211_is_data - check if type is IEEE80211_FTYPE_DATA * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a data frame */ static inline bool ieee80211_is_data(__le16 fc) { @@ -485,6 +497,7 @@ static inline bool ieee80211_is_data(__le16 fc) /** * ieee80211_is_ext - check if type is IEEE80211_FTYPE_EXT * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame type is extended */ static inline bool ieee80211_is_ext(__le16 fc) { @@ -496,6 +509,7 @@ static inline bool ieee80211_is_ext(__le16 fc) /** * ieee80211_is_data_qos - check if type is IEEE80211_FTYPE_DATA and IEEE80211_STYPE_QOS_DATA is set * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a QoS data frame */ static inline bool ieee80211_is_data_qos(__le16 fc) { @@ -510,6 +524,8 @@ static inline bool ieee80211_is_data_qos(__le16 fc) /** * ieee80211_is_data_present - check if type is IEEE80211_FTYPE_DATA and has data * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a QoS data frame that has data + * (i.e. is not null data) */ static inline bool ieee80211_is_data_present(__le16 fc) { @@ -524,6 +540,7 @@ static inline bool ieee80211_is_data_present(__le16 fc) /** * ieee80211_is_assoc_req - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_ASSOC_REQ * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is an association request */ static inline bool ieee80211_is_assoc_req(__le16 fc) { @@ -534,6 +551,7 @@ static inline bool ieee80211_is_assoc_req(__le16 fc) /** * ieee80211_is_assoc_resp - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_ASSOC_RESP * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is an association response */ static inline bool ieee80211_is_assoc_resp(__le16 fc) { @@ -544,6 +562,7 @@ static inline bool ieee80211_is_assoc_resp(__le16 fc) /** * ieee80211_is_reassoc_req - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_REASSOC_REQ * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a reassociation request */ static inline bool ieee80211_is_reassoc_req(__le16 fc) { @@ -554,6 +573,7 @@ static inline bool ieee80211_is_reassoc_req(__le16 fc) /** * ieee80211_is_reassoc_resp - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_REASSOC_RESP * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a reassociation response */ static inline bool ieee80211_is_reassoc_resp(__le16 fc) { @@ -564,6 +584,7 @@ static inline bool ieee80211_is_reassoc_resp(__le16 fc) /** * ieee80211_is_probe_req - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_PROBE_REQ * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a probe request */ static inline bool ieee80211_is_probe_req(__le16 fc) { @@ -574,6 +595,7 @@ static inline bool ieee80211_is_probe_req(__le16 fc) /** * ieee80211_is_probe_resp - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_PROBE_RESP * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a probe response */ static inline bool ieee80211_is_probe_resp(__le16 fc) { @@ -584,6 +606,7 @@ static inline bool ieee80211_is_probe_resp(__le16 fc) /** * ieee80211_is_beacon - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_BEACON * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a (regular, not S1G) beacon */ static inline bool ieee80211_is_beacon(__le16 fc) { @@ -595,6 +618,7 @@ static inline bool ieee80211_is_beacon(__le16 fc) * ieee80211_is_s1g_beacon - check if IEEE80211_FTYPE_EXT && * IEEE80211_STYPE_S1G_BEACON * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is an S1G beacon */ static inline bool ieee80211_is_s1g_beacon(__le16 fc) { @@ -604,30 +628,21 @@ static inline bool ieee80211_is_s1g_beacon(__le16 fc) } /** - * ieee80211_next_tbtt_present - check if IEEE80211_FTYPE_EXT && - * IEEE80211_STYPE_S1G_BEACON && IEEE80211_S1G_BCN_NEXT_TBTT - * @fc: frame control bytes in little-endian byteorder - */ -static inline bool ieee80211_next_tbtt_present(__le16 fc) -{ - return (fc & cpu_to_le16(IEEE80211_FCTL_FTYPE | IEEE80211_FCTL_STYPE)) == - cpu_to_le16(IEEE80211_FTYPE_EXT | IEEE80211_STYPE_S1G_BEACON) && - fc & cpu_to_le16(IEEE80211_S1G_BCN_NEXT_TBTT); -} - -/** - * ieee80211_is_s1g_short_beacon - check if next tbtt present bit is set. Only - * true for S1G beacons when they're short. + * ieee80211_is_s1g_short_beacon - check if frame is an S1G short beacon * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is an S1G short beacon, + * i.e. it is an S1G beacon with 'next TBTT' flag set */ static inline bool ieee80211_is_s1g_short_beacon(__le16 fc) { - return ieee80211_is_s1g_beacon(fc) && ieee80211_next_tbtt_present(fc); + return ieee80211_is_s1g_beacon(fc) && + (fc & cpu_to_le16(IEEE80211_S1G_BCN_NEXT_TBTT)); } /** * ieee80211_is_atim - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_ATIM * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is an ATIM frame */ static inline bool ieee80211_is_atim(__le16 fc) { @@ -638,6 +653,7 @@ static inline bool ieee80211_is_atim(__le16 fc) /** * ieee80211_is_disassoc - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_DISASSOC * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a disassociation frame */ static inline bool ieee80211_is_disassoc(__le16 fc) { @@ -648,6 +664,7 @@ static inline bool ieee80211_is_disassoc(__le16 fc) /** * ieee80211_is_auth - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_AUTH * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is an authentication frame */ static inline bool ieee80211_is_auth(__le16 fc) { @@ -658,6 +675,7 @@ static inline bool ieee80211_is_auth(__le16 fc) /** * ieee80211_is_deauth - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_DEAUTH * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a deauthentication frame */ static inline bool ieee80211_is_deauth(__le16 fc) { @@ -668,6 +686,7 @@ static inline bool ieee80211_is_deauth(__le16 fc) /** * ieee80211_is_action - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_ACTION * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is an action frame */ static inline bool ieee80211_is_action(__le16 fc) { @@ -678,6 +697,7 @@ static inline bool ieee80211_is_action(__le16 fc) /** * ieee80211_is_back_req - check if IEEE80211_FTYPE_CTL && IEEE80211_STYPE_BACK_REQ * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a block-ACK request frame */ static inline bool ieee80211_is_back_req(__le16 fc) { @@ -688,6 +708,7 @@ static inline bool ieee80211_is_back_req(__le16 fc) /** * ieee80211_is_back - check if IEEE80211_FTYPE_CTL && IEEE80211_STYPE_BACK * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a block-ACK frame */ static inline bool ieee80211_is_back(__le16 fc) { @@ -698,6 +719,7 @@ static inline bool ieee80211_is_back(__le16 fc) /** * ieee80211_is_pspoll - check if IEEE80211_FTYPE_CTL && IEEE80211_STYPE_PSPOLL * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a PS-poll frame */ static inline bool ieee80211_is_pspoll(__le16 fc) { @@ -708,6 +730,7 @@ static inline bool ieee80211_is_pspoll(__le16 fc) /** * ieee80211_is_rts - check if IEEE80211_FTYPE_CTL && IEEE80211_STYPE_RTS * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is an RTS frame */ static inline bool ieee80211_is_rts(__le16 fc) { @@ -718,6 +741,7 @@ static inline bool ieee80211_is_rts(__le16 fc) /** * ieee80211_is_cts - check if IEEE80211_FTYPE_CTL && IEEE80211_STYPE_CTS * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a CTS frame */ static inline bool ieee80211_is_cts(__le16 fc) { @@ -728,6 +752,7 @@ static inline bool ieee80211_is_cts(__le16 fc) /** * ieee80211_is_ack - check if IEEE80211_FTYPE_CTL && IEEE80211_STYPE_ACK * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is an ACK frame */ static inline bool ieee80211_is_ack(__le16 fc) { @@ -738,6 +763,7 @@ static inline bool ieee80211_is_ack(__le16 fc) /** * ieee80211_is_cfend - check if IEEE80211_FTYPE_CTL && IEEE80211_STYPE_CFEND * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a CF-end frame */ static inline bool ieee80211_is_cfend(__le16 fc) { @@ -748,6 +774,7 @@ static inline bool ieee80211_is_cfend(__le16 fc) /** * ieee80211_is_cfendack - check if IEEE80211_FTYPE_CTL && IEEE80211_STYPE_CFENDACK * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a CF-end-ack frame */ static inline bool ieee80211_is_cfendack(__le16 fc) { @@ -758,6 +785,7 @@ static inline bool ieee80211_is_cfendack(__le16 fc) /** * ieee80211_is_nullfunc - check if frame is a regular (non-QoS) nullfunc frame * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a nullfunc frame */ static inline bool ieee80211_is_nullfunc(__le16 fc) { @@ -768,6 +796,7 @@ static inline bool ieee80211_is_nullfunc(__le16 fc) /** * ieee80211_is_qos_nullfunc - check if frame is a QoS nullfunc frame * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a QoS nullfunc frame */ static inline bool ieee80211_is_qos_nullfunc(__le16 fc) { @@ -778,6 +807,7 @@ static inline bool ieee80211_is_qos_nullfunc(__le16 fc) /** * ieee80211_is_trigger - check if frame is trigger frame * @fc: frame control field in little-endian byteorder + * Return: whether or not the frame is a trigger frame */ static inline bool ieee80211_is_trigger(__le16 fc) { @@ -788,6 +818,7 @@ static inline bool ieee80211_is_trigger(__le16 fc) /** * ieee80211_is_any_nullfunc - check if frame is regular or QoS nullfunc frame * @fc: frame control bytes in little-endian byteorder + * Return: whether or not the frame is a nullfunc or QoS nullfunc frame */ static inline bool ieee80211_is_any_nullfunc(__le16 fc) { @@ -797,6 +828,8 @@ static inline bool ieee80211_is_any_nullfunc(__le16 fc) /** * ieee80211_is_first_frag - check if IEEE80211_SCTL_FRAG is not set * @seq_ctrl: frame sequence control bytes in little-endian byteorder + * Return: whether or not the frame is the first fragment (also true if + * it's not fragmented at all) */ static inline bool ieee80211_is_first_frag(__le16 seq_ctrl) { @@ -806,6 +839,7 @@ static inline bool ieee80211_is_first_frag(__le16 seq_ctrl) /** * ieee80211_is_frag - check if a frame is a fragment * @hdr: 802.11 header of the frame + * Return: whether or not the frame is a fragment */ static inline bool ieee80211_is_frag(struct ieee80211_hdr *hdr) { @@ -1101,7 +1135,7 @@ enum ieee80211_vht_opmode_bits { }; /** - * enum ieee80211_s1g_chanwidth + * enum ieee80211_s1g_chanwidth - S1G channel widths * These are defined in IEEE802.11-2016ah Table 10-20 * as BSS Channel Width * @@ -2359,6 +2393,8 @@ struct ieee80211_eht_operation_info { * @max_vht_nss: current maximum NSS as advertised by the STA in * operating mode notification, can be 0 in which case the * capability data will be used to derive this (from MCS support) + * Return: The maximum NSS that can be used for the given bandwidth/MCS + * combination * * Due to the VHT Extended NSS Bandwidth Support, the maximum NSS can * vary for a given BW/MCS. This function parses the data. @@ -2789,22 +2825,6 @@ struct ieee80211_he_6ghz_oper { u8 minrate; } __packed; -/* - * In "9.4.2.161 Transmit Power Envelope element" of "IEEE Std 802.11ax-2021", - * it show four types in "Table 9-275a-Maximum Transmit Power Interpretation - * subfield encoding", and two category for each type in "Table E-12-Regulatory - * Info subfield encoding in the United States". - * So it it totally max 8 Transmit Power Envelope element. - */ -#define IEEE80211_TPE_MAX_IE_COUNT 8 -/* - * In "Table 9-277—Meaning of Maximum Transmit Power Count subfield" - * of "IEEE Std 802.11ax™‐2021", the max power level is 8. - */ -#define IEEE80211_MAX_NUM_PWR_LEVEL 8 - -#define IEEE80211_TPE_MAX_POWER_COUNT 8 - /* transmit power interpretation type of transmit power envelope element */ enum ieee80211_tx_power_intrpt_type { IEEE80211_TPE_LOCAL_EIRP, @@ -2813,24 +2833,107 @@ enum ieee80211_tx_power_intrpt_type { IEEE80211_TPE_REG_CLIENT_EIRP_PSD, }; +/* category type of transmit power envelope element */ +enum ieee80211_tx_power_category_6ghz { + IEEE80211_TPE_CAT_6GHZ_DEFAULT = 0, + IEEE80211_TPE_CAT_6GHZ_SUBORDINATE = 1, +}; + +/* + * For IEEE80211_TPE_LOCAL_EIRP / IEEE80211_TPE_REG_CLIENT_EIRP, + * setting to 63.5 dBm means no constraint. + */ +#define IEEE80211_TPE_MAX_TX_PWR_NO_CONSTRAINT 127 + +/* + * For IEEE80211_TPE_LOCAL_EIRP_PSD / IEEE80211_TPE_REG_CLIENT_EIRP_PSD, + * setting to 127 indicates no PSD limit for the 20 MHz channel. + */ +#define IEEE80211_TPE_PSD_NO_LIMIT 127 + /** * struct ieee80211_tx_pwr_env - Transmit Power Envelope - * @tx_power_info: Transmit Power Information field - * @tx_power: Maximum Transmit Power field + * @info: Transmit Power Information field + * @variable: Maximum Transmit Power field * * This structure represents the payload of the "Transmit Power * Envelope element" as described in IEEE Std 802.11ax-2021 section * 9.4.2.161 */ struct ieee80211_tx_pwr_env { - u8 tx_power_info; - s8 tx_power[IEEE80211_TPE_MAX_POWER_COUNT]; + u8 info; + u8 variable[]; } __packed; #define IEEE80211_TX_PWR_ENV_INFO_COUNT 0x7 #define IEEE80211_TX_PWR_ENV_INFO_INTERPRET 0x38 #define IEEE80211_TX_PWR_ENV_INFO_CATEGORY 0xC0 +#define IEEE80211_TX_PWR_ENV_EXT_COUNT 0xF + +static inline bool ieee80211_valid_tpe_element(const u8 *data, u8 len) +{ + const struct ieee80211_tx_pwr_env *env = (const void *)data; + u8 count, interpret, category; + u8 needed = sizeof(*env); + u8 N; /* also called N in the spec */ + + if (len < needed) + return false; + + count = u8_get_bits(env->info, IEEE80211_TX_PWR_ENV_INFO_COUNT); + interpret = u8_get_bits(env->info, IEEE80211_TX_PWR_ENV_INFO_INTERPRET); + category = u8_get_bits(env->info, IEEE80211_TX_PWR_ENV_INFO_CATEGORY); + + switch (category) { + case IEEE80211_TPE_CAT_6GHZ_DEFAULT: + case IEEE80211_TPE_CAT_6GHZ_SUBORDINATE: + break; + default: + return false; + } + + switch (interpret) { + case IEEE80211_TPE_LOCAL_EIRP: + case IEEE80211_TPE_REG_CLIENT_EIRP: + if (count > 3) + return false; + + /* count == 0 encodes 1 value for 20 MHz, etc. */ + needed += count + 1; + + if (len < needed) + return false; + + /* there can be extension fields not accounted for in 'count' */ + + return true; + case IEEE80211_TPE_LOCAL_EIRP_PSD: + case IEEE80211_TPE_REG_CLIENT_EIRP_PSD: + if (count > 4) + return false; + + N = count ? 1 << (count - 1) : 1; + needed += N; + + if (len < needed) + return false; + + if (len > needed) { + u8 K = u8_get_bits(env->variable[N], + IEEE80211_TX_PWR_ENV_EXT_COUNT); + + needed += 1 + K; + if (len < needed) + return false; + } + + return true; + } + + return false; +} + /* * ieee80211_he_oper_size - calculate 802.11ax HE Operations IE size * @he_oper_ie: byte data of the He Operations IE, stating from the byte @@ -4145,7 +4248,7 @@ enum ieee80211_idle_options { }; /** - * struct ieee80211_bss_max_idle_period_ie + * struct ieee80211_bss_max_idle_period_ie - BSS max idle period element struct * * This structure refers to "BSS Max idle period element" * @@ -4180,7 +4283,7 @@ enum ieee80211_sa_query_action { }; /** - * struct ieee80211_bssid_index + * struct ieee80211_bssid_index - multiple BSSID index element structure * * This structure refers to "Multiple BSSID-index element" * @@ -4195,7 +4298,8 @@ struct ieee80211_bssid_index { }; /** - * struct ieee80211_multiple_bssid_configuration + * struct ieee80211_multiple_bssid_configuration - multiple BSSID configuration + * element structure * * This structure refers to "Multiple BSSID Configuration element" * @@ -4326,6 +4430,7 @@ struct ieee80211_he_6ghz_capa { /** * ieee80211_get_qos_ctl - get pointer to qos control bytes * @hdr: the frame + * Return: a pointer to the QoS control field in the frame header * * The qos ctrl bytes come after the frame_control, duration, seq_num * and 3 or 4 addresses of length ETH_ALEN. Checks frame_control to choose @@ -4348,6 +4453,7 @@ static inline u8 *ieee80211_get_qos_ctl(struct ieee80211_hdr *hdr) /** * ieee80211_get_tid - get qos TID * @hdr: the frame + * Return: the TID from the QoS control field */ static inline u8 ieee80211_get_tid(struct ieee80211_hdr *hdr) { @@ -4359,6 +4465,7 @@ static inline u8 ieee80211_get_tid(struct ieee80211_hdr *hdr) /** * ieee80211_get_SA - get pointer to SA * @hdr: the frame + * Return: a pointer to the source address (SA) * * Given an 802.11 frame, this function returns the offset * to the source address (SA). It does not verify that the @@ -4378,6 +4485,7 @@ static inline u8 *ieee80211_get_SA(struct ieee80211_hdr *hdr) /** * ieee80211_get_DA - get pointer to DA * @hdr: the frame + * Return: a pointer to the destination address (DA) * * Given an 802.11 frame, this function returns the offset * to the destination address (DA). It does not verify that @@ -4396,6 +4504,7 @@ static inline u8 *ieee80211_get_DA(struct ieee80211_hdr *hdr) /** * ieee80211_is_bufferable_mmpdu - check if frame is bufferable MMPDU * @skb: the skb to check, starting with the 802.11 header + * Return: whether or not the MMPDU is bufferable */ static inline bool ieee80211_is_bufferable_mmpdu(struct sk_buff *skb) { @@ -4434,6 +4543,7 @@ static inline bool ieee80211_is_bufferable_mmpdu(struct sk_buff *skb) /** * _ieee80211_is_robust_mgmt_frame - check if frame is a robust management frame * @hdr: the frame (buffer must include at least the first octet of payload) + * Return: whether or not the frame is a robust management frame */ static inline bool _ieee80211_is_robust_mgmt_frame(struct ieee80211_hdr *hdr) { @@ -4470,6 +4580,7 @@ static inline bool _ieee80211_is_robust_mgmt_frame(struct ieee80211_hdr *hdr) /** * ieee80211_is_robust_mgmt_frame - check if skb contains a robust mgmt frame * @skb: the skb containing the frame, length will be checked + * Return: whether or not the frame is a robust management frame */ static inline bool ieee80211_is_robust_mgmt_frame(struct sk_buff *skb) { @@ -4482,6 +4593,7 @@ static inline bool ieee80211_is_robust_mgmt_frame(struct sk_buff *skb) * ieee80211_is_public_action - check if frame is a public action frame * @hdr: the frame * @len: length of the frame + * Return: whether or not the frame is a public action frame */ static inline bool ieee80211_is_public_action(struct ieee80211_hdr *hdr, size_t len) @@ -4527,8 +4639,9 @@ ieee80211_is_protected_dual_of_public_action(struct sk_buff *skb) /** * _ieee80211_is_group_privacy_action - check if frame is a group addressed - * privacy action frame + * privacy action frame * @hdr: the frame + * Return: whether or not the frame is a group addressed privacy action frame */ static inline bool _ieee80211_is_group_privacy_action(struct ieee80211_hdr *hdr) { @@ -4544,8 +4657,9 @@ static inline bool _ieee80211_is_group_privacy_action(struct ieee80211_hdr *hdr) /** * ieee80211_is_group_privacy_action - check if frame is a group addressed - * privacy action frame + * privacy action frame * @skb: the skb containing the frame, length will be checked + * Return: whether or not the frame is a group addressed privacy action frame */ static inline bool ieee80211_is_group_privacy_action(struct sk_buff *skb) { @@ -4557,6 +4671,7 @@ static inline bool ieee80211_is_group_privacy_action(struct sk_buff *skb) /** * ieee80211_tu_to_usec - convert time units (TU) to microseconds * @tu: the TUs + * Return: the time value converted to microseconds */ static inline unsigned long ieee80211_tu_to_usec(unsigned long tu) { @@ -4568,6 +4683,7 @@ static inline unsigned long ieee80211_tu_to_usec(unsigned long tu) * @tim: the TIM IE * @tim_len: length of the TIM IE * @aid: the AID to look for + * Return: whether or not traffic is indicated in the TIM for the given AID */ static inline bool ieee80211_check_tim(const struct ieee80211_tim_ie *tim, u8 tim_len, u16 aid) @@ -4594,8 +4710,10 @@ static inline bool ieee80211_check_tim(const struct ieee80211_tim_ie *tim, } /** - * ieee80211_get_tdls_action - get tdls packet action (or -1, if not tdls packet) + * ieee80211_get_tdls_action - get TDLS action code * @skb: the skb containing the frame, length will not be checked + * Return: the TDLS action code, or -1 if it's not an encapsulated TDLS action + * frame * * This function assumes the frame is a data frame, and that the network header * is in the correct place. @@ -4635,6 +4753,7 @@ static inline int ieee80211_get_tdls_action(struct sk_buff *skb) /** * ieee80211_action_contains_tpc - checks if the frame contains TPC element * @skb: the skb containing the frame, length will be checked + * Return: %true if the frame contains a TPC element, %false otherwise * * This function checks if it's either TPC report action frame or Link * Measurement report action frame as defined in IEEE Std. 802.11-2012 8.5.2.5 @@ -4679,6 +4798,11 @@ static inline bool ieee80211_action_contains_tpc(struct sk_buff *skb) return true; } +/** + * ieee80211_is_timing_measurement - check if frame is timing measurement response + * @skb: the SKB to check + * Return: whether or not the frame is a valid timing measurement response + */ static inline bool ieee80211_is_timing_measurement(struct sk_buff *skb) { struct ieee80211_mgmt *mgmt = (void *)skb->data; @@ -4698,6 +4822,11 @@ static inline bool ieee80211_is_timing_measurement(struct sk_buff *skb) return false; } +/** + * ieee80211_is_ftm - check if frame is FTM response + * @skb: the SKB to check + * Return: whether or not the frame is a valid FTM response action frame + */ static inline bool ieee80211_is_ftm(struct sk_buff *skb) { struct ieee80211_mgmt *mgmt = (void *)skb->data; @@ -4752,6 +4881,7 @@ struct element { * @element: element pointer after for_each_element() or friends * @data: same data pointer as passed to for_each_element() or friends * @datalen: same data length as passed to for_each_element() or friends + * Return: %true if all elements were iterated, %false otherwise; see notes * * This function returns %true if all the data was parsed or considered * while walking the elements. Only use this if your for_each_element() @@ -4955,6 +5085,7 @@ struct ieee80211_mle_tdls_common_info { * ieee80211_mle_common_size - check multi-link element common size * @data: multi-link element, must already be checked for size using * ieee80211_mle_size_ok() + * Return: the size of the multi-link element's "common" subfield */ static inline u8 ieee80211_mle_common_size(const u8 *data) { @@ -4987,11 +5118,10 @@ static inline u8 ieee80211_mle_common_size(const u8 *data) /** * ieee80211_mle_get_link_id - returns the link ID * @data: the basic multi link element + * Return: the link ID, or -1 if not present * * The element is assumed to be of the correct type (BASIC) and big enough, * this must be checked using ieee80211_mle_type_ok(). - * - * If the BSS link ID can't be found, -1 will be returned */ static inline int ieee80211_mle_get_link_id(const u8 *data) { @@ -5011,12 +5141,10 @@ static inline int ieee80211_mle_get_link_id(const u8 *data) /** * ieee80211_mle_get_bss_param_ch_cnt - returns the BSS parameter change count * @data: pointer to the basic multi link element + * Return: the BSS Parameter Change Count field value, or -1 if not present * * The element is assumed to be of the correct type (BASIC) and big enough, * this must be checked using ieee80211_mle_type_ok(). - * - * If the BSS parameter change count value can't be found (the presence bit - * for it is clear), -1 will be returned. */ static inline int ieee80211_mle_get_bss_param_ch_cnt(const u8 *data) @@ -5039,13 +5167,13 @@ ieee80211_mle_get_bss_param_ch_cnt(const u8 *data) /** * ieee80211_mle_get_eml_med_sync_delay - returns the medium sync delay - * @data: pointer to the multi link EHT IE + * @data: pointer to the multi-link element + * Return: the medium synchronization delay field value from the multi-link + * element, or the default value (%IEEE80211_MED_SYNC_DELAY_DEFAULT) + * if not present * * The element is assumed to be of the correct type (BASIC) and big enough, * this must be checked using ieee80211_mle_type_ok(). - * - * If the medium synchronization is not present, then the default value is - * returned. */ static inline u16 ieee80211_mle_get_eml_med_sync_delay(const u8 *data) { @@ -5069,12 +5197,12 @@ static inline u16 ieee80211_mle_get_eml_med_sync_delay(const u8 *data) /** * ieee80211_mle_get_eml_cap - returns the EML capability - * @data: pointer to the multi link EHT IE + * @data: pointer to the multi-link element + * Return: the EML capability field value from the multi-link element, + * or 0 if not present * * The element is assumed to be of the correct type (BASIC) and big enough, * this must be checked using ieee80211_mle_type_ok(). - * - * If the EML capability is not present, 0 will be returned. */ static inline u16 ieee80211_mle_get_eml_cap(const u8 *data) { @@ -5100,13 +5228,12 @@ static inline u16 ieee80211_mle_get_eml_cap(const u8 *data) /** * ieee80211_mle_get_mld_capa_op - returns the MLD capabilities and operations. - * @data: pointer to the multi link EHT IE + * @data: pointer to the multi-link element + * Return: the MLD capabilities and operations field value from the multi-link + * element, or 0 if not present * * The element is assumed to be of the correct type (BASIC) and big enough, * this must be checked using ieee80211_mle_type_ok(). - * - * If the MLD capabilities and operations field is not present, 0 will be - * returned. */ static inline u16 ieee80211_mle_get_mld_capa_op(const u8 *data) { @@ -5137,12 +5264,11 @@ static inline u16 ieee80211_mle_get_mld_capa_op(const u8 *data) /** * ieee80211_mle_get_mld_id - returns the MLD ID - * @data: pointer to the multi link element + * @data: pointer to the multi-link element + * Return: The MLD ID in the given multi-link element, or 0 if not present * * The element is assumed to be of the correct type (BASIC) and big enough, * this must be checked using ieee80211_mle_type_ok(). - * - * If the MLD ID is not present, 0 will be returned. */ static inline u8 ieee80211_mle_get_mld_id(const u8 *data) { @@ -5177,6 +5303,7 @@ static inline u8 ieee80211_mle_get_mld_id(const u8 *data) * ieee80211_mle_size_ok - validate multi-link element size * @data: pointer to the element data * @len: length of the containing element + * Return: whether or not the multi-link element size is OK */ static inline bool ieee80211_mle_size_ok(const u8 *data, size_t len) { @@ -5246,6 +5373,7 @@ static inline bool ieee80211_mle_size_ok(const u8 *data, size_t len) * @data: pointer to the element data * @type: expected type of the element * @len: length of the containing element + * Return: whether or not the multi-link element type matches and size is OK */ static inline bool ieee80211_mle_type_ok(const u8 *data, u8 type, size_t len) { @@ -5289,6 +5417,7 @@ struct ieee80211_mle_per_sta_profile { * profile size * @data: pointer to the sub element data * @len: length of the containing sub element + * Return: %true if the STA profile is large enough, %false otherwise */ static inline bool ieee80211_mle_basic_sta_prof_size_ok(const u8 *data, size_t len) @@ -5373,6 +5502,7 @@ ieee80211_mle_basic_sta_prof_bss_param_ch_cnt(const struct ieee80211_mle_per_sta * element sta profile size. * @data: pointer to the sub element data * @len: length of the containing sub element + * Return: %true if the STA profile is large enough, %false otherwise */ static inline bool ieee80211_mle_reconf_sta_prof_size_ok(const u8 *data, size_t len) diff --git a/include/linux/math64.h b/include/linux/math64.h index d34def7f9a8cd3b28584e6c2b70cb260213ea495..6aaccc1626ab4c6d38d7b4cb6cd3820ba41f628f 100644 --- a/include/linux/math64.h +++ b/include/linux/math64.h @@ -297,6 +297,19 @@ u64 mul_u64_u64_div_u64(u64 a, u64 mul, u64 div); #define DIV64_U64_ROUND_UP(ll, d) \ ({ u64 _tmp = (d); div64_u64((ll) + _tmp - 1, _tmp); }) +/** + * DIV_U64_ROUND_UP - unsigned 64bit divide with 32bit divisor rounded up + * @ll: unsigned 64bit dividend + * @d: unsigned 32bit divisor + * + * Divide unsigned 64bit dividend by unsigned 32bit divisor + * and round up. + * + * Return: dividend / divisor rounded up + */ +#define DIV_U64_ROUND_UP(ll, d) \ + ({ u32 _tmp = (d); div_u64((ll) + _tmp - 1, _tmp); }) + /** * DIV64_U64_ROUND_CLOSEST - unsigned 64bit divide with 64bit divisor rounded to nearest integer * @dividend: unsigned 64bit dividend @@ -342,4 +355,19 @@ u64 mul_u64_u64_div_u64(u64 a, u64 mul, u64 div); div_s64((__x - (__d / 2)), __d); \ } \ ) + +/** + * roundup_u64 - Round up a 64bit value to the next specified 32bit multiple + * @x: the value to up + * @y: 32bit multiple to round up to + * + * Rounds @x to the next multiple of @y. For 32bit @x values, see roundup and + * the faster round_up() for powers of 2. + * + * Return: rounded up value. + */ +static inline u64 roundup_u64(u64 x, u32 y) +{ + return DIV_U64_ROUND_UP(x, y) * y; +} #endif /* _LINUX_MATH64_H */ diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h index d7bb31d9a4463abf132df7b5d928d9a98c71bf76..da09bfaa7b813c1d1541b8493249d52c6973e741 100644 --- a/include/linux/mlx5/device.h +++ b/include/linux/mlx5/device.h @@ -294,6 +294,7 @@ enum { #define MLX5_UMR_FLEX_ALIGNMENT 0x40 #define MLX5_UMR_MTT_NUM_ENTRIES_ALIGNMENT (MLX5_UMR_FLEX_ALIGNMENT / sizeof(struct mlx5_mtt)) #define MLX5_UMR_KLM_NUM_ENTRIES_ALIGNMENT (MLX5_UMR_FLEX_ALIGNMENT / sizeof(struct mlx5_klm)) +#define MLX5_UMR_KSM_NUM_ENTRIES_ALIGNMENT (MLX5_UMR_FLEX_ALIGNMENT / sizeof(struct mlx5_ksm)) #define MLX5_USER_INDEX_LEN (MLX5_FLD_SZ_BYTES(qpc, user_index) * 8) diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index f468763478ae61229f88ba1aeafd3f14191a9d81..17acd0f3ca8ed543774ae32a62a6d594bad63366 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -1526,8 +1526,7 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 ts_cqe_to_dest_cqn[0x1]; u8 reserved_at_b3[0x6]; u8 go_back_n[0x1]; - u8 shampo[0x1]; - u8 reserved_at_bb[0x5]; + u8 reserved_at_ba[0x6]; u8 max_sgl_for_optimized_performance[0x8]; u8 log_max_cq_sz[0x8]; @@ -1744,7 +1743,9 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 reserved_at_280[0x10]; u8 max_wqe_sz_sq[0x10]; - u8 reserved_at_2a0[0x10]; + u8 reserved_at_2a0[0xb]; + u8 shampo[0x1]; + u8 reserved_at_2ac[0x4]; u8 max_wqe_sz_rq[0x10]; u8 max_flow_counter_31_16[0x10]; @@ -2017,7 +2018,8 @@ struct mlx5_ifc_cmd_hca_cap_2_bits { u8 reserved_at_250[0x10]; u8 reserved_at_260[0x120]; - u8 reserved_at_380[0x10]; + u8 reserved_at_380[0xb]; + u8 min_mkey_log_entity_size_fixed_buffer[0x5]; u8 ec_vf_vport_base[0x10]; u8 reserved_at_3a0[0x10]; @@ -2029,7 +2031,11 @@ struct mlx5_ifc_cmd_hca_cap_2_bits { u8 pcc_ifa2[0x1]; u8 reserved_at_3f1[0xf]; - u8 reserved_at_400[0x400]; + u8 reserved_at_400[0x1]; + u8 min_mkey_log_entity_size_fixed_buffer_valid[0x1]; + u8 reserved_at_402[0x1e]; + + u8 reserved_at_420[0x3e0]; }; enum mlx5_ifc_flow_destination_type { @@ -10308,9 +10314,9 @@ struct mlx5_ifc_mcam_access_reg_bits { u8 mfrl[0x1]; u8 regs_39_to_32[0x8]; - u8 regs_31_to_10[0x16]; + u8 regs_31_to_11[0x15]; u8 mtmp[0x1]; - u8 regs_8_to_0[0x9]; + u8 regs_9_to_0[0xa]; }; struct mlx5_ifc_mcam_access_reg_bits1 { diff --git a/include/linux/netfs.h b/include/linux/netfs.h index d2d291a9cdadba94743daf38cb7ad56bad99ebf3..3ca3906bb8da661c9514d17356004dcf349b2dd1 100644 --- a/include/linux/netfs.h +++ b/include/linux/netfs.h @@ -68,6 +68,7 @@ struct netfs_inode { loff_t remote_i_size; /* Size of the remote file */ loff_t zero_point; /* Size after which we assume there's no data * on the server */ + atomic_t io_count; /* Number of outstanding reqs */ unsigned long flags; #define NETFS_ICTX_ODIRECT 0 /* The file has DIO in progress */ #define NETFS_ICTX_UNBUFFERED 1 /* I/O should not use the pagecache */ @@ -474,6 +475,7 @@ static inline void netfs_inode_init(struct netfs_inode *ctx, ctx->remote_i_size = i_size_read(&ctx->inode); ctx->zero_point = LLONG_MAX; ctx->flags = 0; + atomic_set(&ctx->io_count, 0); #if IS_ENABLED(CONFIG_FSCACHE) ctx->cache = NULL; #endif @@ -517,4 +519,20 @@ static inline struct fscache_cookie *netfs_i_cookie(struct netfs_inode *ctx) #endif } +/** + * netfs_wait_for_outstanding_io - Wait for outstanding I/O to complete + * @ctx: The netfs inode to wait on + * + * Wait for outstanding I/O requests of any type to complete. This is intended + * to be called from inode eviction routines. This makes sure that any + * resources held by those requests are cleaned up before we let the inode get + * cleaned up. + */ +static inline void netfs_wait_for_outstanding_io(struct inode *inode) +{ + struct netfs_inode *ictx = netfs_inode(inode); + + wait_var_event(&ictx->io_count, atomic_read(&ictx->io_count) == 0); +} + #endif /* _LINUX_NETFS_H */ diff --git a/include/linux/netlink.h b/include/linux/netlink.h index 5df7340d4dabc0c0b1728dafde43b5522dacd024..b332c2048c75576cd265a2f7940d8cab715beacd 100644 --- a/include/linux/netlink.h +++ b/include/linux/netlink.h @@ -47,7 +47,6 @@ struct netlink_kernel_cfg { unsigned int groups; unsigned int flags; void (*input)(struct sk_buff *skb); - struct mutex *cb_mutex; int (*bind)(struct net *net, int group); void (*unbind)(struct net *net, int group); void (*release) (struct sock *sk, unsigned long *groups); diff --git a/include/linux/objagg.h b/include/linux/objagg.h index 78021777df46264af5279d56bf9628d2448ce279..6df5b887dc547cd0ac772e6c37b684f3e035837a 100644 --- a/include/linux/objagg.h +++ b/include/linux/objagg.h @@ -8,7 +8,6 @@ struct objagg_ops { size_t obj_size; bool (*delta_check)(void *priv, const void *parent_obj, const void *obj); - int (*hints_obj_cmp)(const void *obj1, const void *obj2); void * (*delta_create)(void *priv, void *parent_obj, void *obj); void (*delta_destroy)(void *priv, void *delta_priv); void * (*root_create)(void *priv, void *obj, unsigned int root_id); diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 3d69589c00a4bb4d9ee7a69172e7fb9a65a1ca90..ee633712bba0b9b89a8715a3ee92b59d3499bdf3 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -346,6 +346,19 @@ static inline void mapping_set_gfp_mask(struct address_space *m, gfp_t mask) m->gfp_mask = mask; } +/* + * There are some parts of the kernel which assume that PMD entries + * are exactly HPAGE_PMD_ORDER. Those should be fixed, but until then, + * limit the maximum allocation order to PMD size. I'm not aware of any + * assumptions about maximum order if THP are disabled, but 8 seems like + * a good order (that's 1MB if you're using 4kB pages) + */ +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#define MAX_PAGECACHE_ORDER HPAGE_PMD_ORDER +#else +#define MAX_PAGECACHE_ORDER 8 +#endif + /** * mapping_set_large_folios() - Indicate the file supports large folios. * @mapping: The file. @@ -372,6 +385,14 @@ static inline bool mapping_large_folio_support(struct address_space *mapping) test_bit(AS_LARGE_FOLIO_SUPPORT, &mapping->flags); } +/* Return the maximum folio size for this pagecache mapping, in bytes. */ +static inline size_t mapping_max_folio_size(struct address_space *mapping) +{ + if (mapping_large_folio_support(mapping)) + return PAGE_SIZE << MAX_PAGECACHE_ORDER; + return PAGE_SIZE; +} + static inline int filemap_nr_thps(struct address_space *mapping) { #ifdef CONFIG_READ_ONLY_THP_FOR_FS @@ -530,19 +551,6 @@ static inline void *detach_page_private(struct page *page) return folio_detach_private(page_folio(page)); } -/* - * There are some parts of the kernel which assume that PMD entries - * are exactly HPAGE_PMD_ORDER. Those should be fixed, but until then, - * limit the maximum allocation order to PMD size. I'm not aware of any - * assumptions about maximum order if THP are disabled, but 8 seems like - * a good order (that's 1MB if you're using 4kB pages) - */ -#ifdef CONFIG_TRANSPARENT_HUGEPAGE -#define MAX_PAGECACHE_ORDER HPAGE_PMD_ORDER -#else -#define MAX_PAGECACHE_ORDER 8 -#endif - #ifdef CONFIG_NUMA struct folio *filemap_alloc_folio_noprof(gfp_t gfp, unsigned int order); #else diff --git a/include/linux/phylink.h b/include/linux/phylink.h index 5ea6b2ad2396390ba7738ff8de36ef85595a378f..a30a692acc327dbf08485766dcd535cc9bbe9b48 100644 --- a/include/linux/phylink.h +++ b/include/linux/phylink.h @@ -141,7 +141,8 @@ enum phylink_op_type { * @mac_requires_rxc: if true, the MAC always requires a receive clock from PHY. * The PHY driver should start the clock signal as soon as * possible and avoid stopping it during suspend events. - * @ovr_an_inband: if true, override PCS to MLO_AN_INBAND + * @default_an_inband: if true, defaults to MLO_AN_INBAND rather than + * MLO_AN_PHY. A fixed-link specification will override. * @get_fixed_state: callback to execute to determine the fixed link state, * if MAC link is at %MLO_AN_FIXED mode. * @supported_interfaces: bitmap describing which PHY_INTERFACE_MODE_xxx @@ -154,7 +155,7 @@ struct phylink_config { bool poll_fixed_state; bool mac_managed_pm; bool mac_requires_rxc; - bool ovr_an_inband; + bool default_an_inband; void (*get_fixed_state)(struct phylink_config *config, struct phylink_link_state *state); DECLARE_PHY_INTERFACE_MASK(supported_interfaces); diff --git a/include/linux/pnp.h b/include/linux/pnp.h index 82561242cda4281a109bcef3bcba3a3e9739f5b7..7f2ff95d2deb5e4a79dad6cf96ae9626e9ef1bd0 100644 --- a/include/linux/pnp.h +++ b/include/linux/pnp.h @@ -435,8 +435,6 @@ struct pnp_protocol { #define protocol_for_each_dev(protocol, dev) \ list_for_each_entry(dev, &(protocol)->devices, protocol_list) -extern const struct bus_type pnp_bus_type; - #if defined(CONFIG_PNP) /* device management */ @@ -469,7 +467,7 @@ int compare_pnp_id(struct pnp_id *pos, const char *id); int pnp_register_driver(struct pnp_driver *drv); void pnp_unregister_driver(struct pnp_driver *drv); -#define dev_is_pnp(d) ((d)->bus == &pnp_bus_type) +bool dev_is_pnp(const struct device *dev); #else @@ -502,7 +500,7 @@ static inline int compare_pnp_id(struct pnp_id *pos, const char *id) { return -E static inline int pnp_register_driver(struct pnp_driver *drv) { return -ENODEV; } static inline void pnp_unregister_driver(struct pnp_driver *drv) { } -#define dev_is_pnp(d) false +static inline bool dev_is_pnp(const struct device *dev) { return false; } #endif /* CONFIG_PNP */ diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 1c2902eaebd376c3e6a9212ce5d6f9edb87a3bfa..fe7d8dbef77e14fa7bf35a68d7a4969808650f93 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -706,6 +706,13 @@ typedef unsigned int sk_buff_data_t; typedef unsigned char *sk_buff_data_t; #endif +enum skb_tstamp_type { + SKB_CLOCK_REALTIME, + SKB_CLOCK_MONOTONIC, + SKB_CLOCK_TAI, + __SKB_CLOCK_MAX = SKB_CLOCK_TAI, +}; + /** * DOC: Basic sk_buff geometry * @@ -823,10 +830,8 @@ typedef unsigned char *sk_buff_data_t; * @dst_pending_confirm: need to confirm neighbour * @decrypted: Decrypted SKB * @slow_gro: state present at GRO time, slower prepare step required - * @mono_delivery_time: When set, skb->tstamp has the - * delivery_time in mono clock base (i.e. EDT). Otherwise, the - * skb->tstamp has the (rcv) timestamp at ingress and - * delivery_time at egress. + * @tstamp_type: When set, skb->tstamp has the + * delivery_time clock base of skb->tstamp. * @napi_id: id of the NAPI struct this skb came from * @sender_cpu: (aka @napi_id) source CPU in XPS * @alloc_cpu: CPU which did the skb allocation. @@ -954,7 +959,7 @@ struct sk_buff { /* private: */ __u8 __mono_tc_offset[0]; /* public: */ - __u8 mono_delivery_time:1; /* See SKB_MONO_DELIVERY_TIME_MASK */ + __u8 tstamp_type:2; /* See skb_tstamp_type */ #ifdef CONFIG_NET_XGRESS __u8 tc_at_ingress:1; /* See TC_AT_INGRESS_MASK */ __u8 tc_skip_classify:1; @@ -1084,15 +1089,16 @@ struct sk_buff { #endif #define PKT_TYPE_OFFSET offsetof(struct sk_buff, __pkt_type_offset) -/* if you move tc_at_ingress or mono_delivery_time +/* if you move tc_at_ingress or tstamp_type * around, you also must adapt these constants. */ #ifdef __BIG_ENDIAN_BITFIELD -#define SKB_MONO_DELIVERY_TIME_MASK (1 << 7) -#define TC_AT_INGRESS_MASK (1 << 6) +#define SKB_TSTAMP_TYPE_MASK (3 << 6) +#define SKB_TSTAMP_TYPE_RSHIFT (6) +#define TC_AT_INGRESS_MASK (1 << 5) #else -#define SKB_MONO_DELIVERY_TIME_MASK (1 << 0) -#define TC_AT_INGRESS_MASK (1 << 1) +#define SKB_TSTAMP_TYPE_MASK (3) +#define TC_AT_INGRESS_MASK (1 << 2) #endif #define SKB_BF_MONO_TC_OFFSET offsetof(struct sk_buff, __mono_tc_offset) @@ -4179,7 +4185,7 @@ static inline void skb_get_new_timestampns(const struct sk_buff *skb, static inline void __net_timestamp(struct sk_buff *skb) { skb->tstamp = ktime_get_real(); - skb->mono_delivery_time = 0; + skb->tstamp_type = SKB_CLOCK_REALTIME; } static inline ktime_t net_timedelta(ktime_t t) @@ -4188,10 +4194,36 @@ static inline ktime_t net_timedelta(ktime_t t) } static inline void skb_set_delivery_time(struct sk_buff *skb, ktime_t kt, - bool mono) + u8 tstamp_type) { skb->tstamp = kt; - skb->mono_delivery_time = kt && mono; + + if (kt) + skb->tstamp_type = tstamp_type; + else + skb->tstamp_type = SKB_CLOCK_REALTIME; +} + +static inline void skb_set_delivery_type_by_clockid(struct sk_buff *skb, + ktime_t kt, clockid_t clockid) +{ + u8 tstamp_type = SKB_CLOCK_REALTIME; + + switch (clockid) { + case CLOCK_REALTIME: + break; + case CLOCK_MONOTONIC: + tstamp_type = SKB_CLOCK_MONOTONIC; + break; + case CLOCK_TAI: + tstamp_type = SKB_CLOCK_TAI; + break; + default: + WARN_ON_ONCE(1); + kt = 0; + } + + skb_set_delivery_time(skb, kt, tstamp_type); } DECLARE_STATIC_KEY_FALSE(netstamp_needed_key); @@ -4201,8 +4233,8 @@ DECLARE_STATIC_KEY_FALSE(netstamp_needed_key); */ static inline void skb_clear_delivery_time(struct sk_buff *skb) { - if (skb->mono_delivery_time) { - skb->mono_delivery_time = 0; + if (skb->tstamp_type) { + skb->tstamp_type = SKB_CLOCK_REALTIME; if (static_branch_unlikely(&netstamp_needed_key)) skb->tstamp = ktime_get_real(); else @@ -4212,7 +4244,7 @@ static inline void skb_clear_delivery_time(struct sk_buff *skb) static inline void skb_clear_tstamp(struct sk_buff *skb) { - if (skb->mono_delivery_time) + if (skb->tstamp_type) return; skb->tstamp = 0; @@ -4220,7 +4252,7 @@ static inline void skb_clear_tstamp(struct sk_buff *skb) static inline ktime_t skb_tstamp(const struct sk_buff *skb) { - if (skb->mono_delivery_time) + if (skb->tstamp_type) return 0; return skb->tstamp; @@ -4228,7 +4260,7 @@ static inline ktime_t skb_tstamp(const struct sk_buff *skb) static inline ktime_t skb_tstamp_cond(const struct sk_buff *skb, bool cond) { - if (!skb->mono_delivery_time && skb->tstamp) + if (skb->tstamp_type != SKB_CLOCK_MONOTONIC && skb->tstamp) return skb->tstamp; if (static_branch_unlikely(&netstamp_needed_key) || cond) diff --git a/include/linux/stmmac.h b/include/linux/stmmac.h index f92c195c76ed23812c39dad45116798a41e4f599..8f0f156d50d35cc98953069f1d4da3bd57d106fb 100644 --- a/include/linux/stmmac.h +++ b/include/linux/stmmac.h @@ -83,7 +83,7 @@ struct stmmac_priv; struct stmmac_mdio_bus_data { unsigned int phy_mask; unsigned int has_xpcs; - unsigned int xpcs_an_inband; + unsigned int default_an_inband; int *irqs; int probed_phy_irq; bool needs_reset; diff --git a/include/linux/tpm.h b/include/linux/tpm.h index c17e4efbb2e5cd367f880fec4c7e263517e631ed..21a67dc9efe80b2d0dc6bc2851b2b4ab67933818 100644 --- a/include/linux/tpm.h +++ b/include/linux/tpm.h @@ -394,21 +394,6 @@ enum tpm2_object_attributes { TPM2_OA_SIGN = BIT(18), }; -/* - * definitions for the canonical template. These are mandated - * by the TCG key template documents - */ - -#define AES_KEY_BYTES AES_KEYSIZE_128 -#define AES_KEY_BITS (AES_KEY_BYTES*8) -#define TPM2_OA_TMPL (TPM2_OA_NO_DA | \ - TPM2_OA_FIXED_TPM | \ - TPM2_OA_FIXED_PARENT | \ - TPM2_OA_SENSITIVE_DATA_ORIGIN | \ - TPM2_OA_USER_WITH_AUTH | \ - TPM2_OA_DECRYPT | \ - TPM2_OA_RESTRICTED) - enum tpm2_session_attributes { TPM2_SA_CONTINUE_SESSION = BIT(0), TPM2_SA_AUDIT_EXCLUSIVE = BIT(1), @@ -437,8 +422,6 @@ u8 tpm_buf_read_u8(struct tpm_buf *buf, off_t *offset); u16 tpm_buf_read_u16(struct tpm_buf *buf, off_t *offset); u32 tpm_buf_read_u32(struct tpm_buf *buf, off_t *offset); -u8 *tpm_buf_parameters(struct tpm_buf *buf); - /* * Check if TPM device is in the firmware upgrade mode. */ diff --git a/include/net/caif/caif_layer.h b/include/net/caif/caif_layer.h index 51f7bb42a93611a74ecf0fb85a537b8a61ac46f6..0f45d875905f88bd2bc08f9c6ef5c55dc1130b4d 100644 --- a/include/net/caif/caif_layer.h +++ b/include/net/caif/caif_layer.h @@ -11,9 +11,7 @@ struct cflayer; struct cfpkt; -struct cfpktq; struct caif_payload_info; -struct caif_packet_funcs; #define CAIF_LAYER_NAME_SZ 16 diff --git a/include/net/cfg80211.h b/include/net/cfg80211.h index cbf1664dc569cdc2bda51164d21e2f5afb18edb0..5da9bb0ac6a4782fc843597090c4b4724e5dbc29 100644 --- a/include/net/cfg80211.h +++ b/include/net/cfg80211.h @@ -2200,7 +2200,7 @@ struct cfg80211_sar_sub_specs { struct cfg80211_sar_specs { enum nl80211_sar_type type; u32 num_sub_specs; - struct cfg80211_sar_sub_specs sub_specs[]; + struct cfg80211_sar_sub_specs sub_specs[] __counted_by(num_sub_specs); }; @@ -2838,7 +2838,7 @@ struct cfg80211_sched_scan_request { struct list_head list; /* keep last */ - struct ieee80211_channel *channels[]; + struct ieee80211_channel *channels[] __counted_by(n_channels); }; /** @@ -3566,8 +3566,8 @@ struct cfg80211_coalesce_rules { * @n_rules: number of rules */ struct cfg80211_coalesce { - struct cfg80211_coalesce_rules *rules; int n_rules; + struct cfg80211_coalesce_rules rules[] __counted_by(n_rules); }; /** @@ -3582,7 +3582,7 @@ struct cfg80211_coalesce { struct cfg80211_wowlan_nd_match { struct cfg80211_ssid ssid; int n_channels; - u32 channels[]; + u32 channels[] __counted_by(n_channels); }; /** @@ -3596,7 +3596,7 @@ struct cfg80211_wowlan_nd_match { */ struct cfg80211_wowlan_nd_info { int n_matches; - struct cfg80211_wowlan_nd_match *matches[]; + struct cfg80211_wowlan_nd_match *matches[] __counted_by(n_matches); }; /** diff --git a/include/net/devlink.h b/include/net/devlink.h index 35eb0f884386175259231855b2c3287b3ae36d00..db5eff6cb60fa0c095ae7bac937d823281389499 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -352,7 +352,7 @@ struct devlink_dpipe_table { bool resource_valid; u64 resource_id; u64 resource_units; - struct devlink_dpipe_table_ops *table_ops; + const struct devlink_dpipe_table_ops *table_ops; struct rcu_head rcu; }; @@ -1751,7 +1751,7 @@ void devl_sb_unregister(struct devlink *devlink, unsigned int sb_index); void devlink_sb_unregister(struct devlink *devlink, unsigned int sb_index); int devl_dpipe_table_register(struct devlink *devlink, const char *table_name, - struct devlink_dpipe_table_ops *table_ops, + const struct devlink_dpipe_table_ops *table_ops, void *priv, bool counter_control_extern); void devl_dpipe_table_unregister(struct devlink *devlink, const char *table_name); diff --git a/include/net/dsa.h b/include/net/dsa.h index b60e7e410aba78321d2e5e0ed446be1286146989..f9ae3ca66b6f19792749367599103244b063c8da 100644 --- a/include/net/dsa.h +++ b/include/net/dsa.h @@ -882,15 +882,9 @@ struct dsa_switch_ops { struct phylink_pcs *(*phylink_mac_select_pcs)(struct dsa_switch *ds, int port, phy_interface_t iface); - int (*phylink_mac_prepare)(struct dsa_switch *ds, int port, - unsigned int mode, - phy_interface_t interface); void (*phylink_mac_config)(struct dsa_switch *ds, int port, unsigned int mode, const struct phylink_link_state *state); - int (*phylink_mac_finish)(struct dsa_switch *ds, int port, - unsigned int mode, - phy_interface_t interface); void (*phylink_mac_link_down)(struct dsa_switch *ds, int port, unsigned int mode, phy_interface_t interface); diff --git a/include/net/dst_ops.h b/include/net/dst_ops.h index 6d1c8541183dbe7bd6d3e5bd6c57174de9524a50..3a9001a042a5c392a79cfc59af528ef410a28668 100644 --- a/include/net/dst_ops.h +++ b/include/net/dst_ops.h @@ -24,7 +24,7 @@ struct dst_ops { void (*destroy)(struct dst_entry *); void (*ifdown)(struct dst_entry *, struct net_device *dev); - struct dst_entry * (*negative_advice)(struct dst_entry *); + void (*negative_advice)(struct sock *sk, struct dst_entry *); void (*link_failure)(struct sk_buff *); void (*update_pmtu)(struct dst_entry *dst, struct sock *sk, struct sk_buff *skb, u32 mtu, diff --git a/include/net/flow_dissector.h b/include/net/flow_dissector.h index 9ab376d1a6771a53a4f602262340e57a67847fb2..99626475c3f4aac3132fd3552e30f56bbf1b5052 100644 --- a/include/net/flow_dissector.h +++ b/include/net/flow_dissector.h @@ -329,6 +329,14 @@ struct flow_dissector_key_cfm { #define FLOW_DIS_CFM_MDL_MASK GENMASK(7, 5) #define FLOW_DIS_CFM_MDL_MAX 7 +/** + * struct flow_dissector_key_enc_flags: tunnel metadata control flags + * @flags: tunnel control flags + */ +struct flow_dissector_key_enc_flags { + u32 flags; +}; + enum flow_dissector_key_id { FLOW_DISSECTOR_KEY_CONTROL, /* struct flow_dissector_key_control */ FLOW_DISSECTOR_KEY_BASIC, /* struct flow_dissector_key_basic */ @@ -363,6 +371,7 @@ enum flow_dissector_key_id { FLOW_DISSECTOR_KEY_L2TPV3, /* struct flow_dissector_key_l2tpv3 */ FLOW_DISSECTOR_KEY_CFM, /* struct flow_dissector_key_cfm */ FLOW_DISSECTOR_KEY_IPSEC, /* struct flow_dissector_key_ipsec */ + FLOW_DISSECTOR_KEY_ENC_FLAGS, /* struct flow_dissector_key_enc_flags */ FLOW_DISSECTOR_KEY_MAX, }; diff --git a/include/net/ieee80211_radiotap.h b/include/net/ieee80211_radiotap.h index 925bac726a92bba8489b07f8bf32cf00090bf11a..91762faecc13d55508a77bd20265884609dcfa5f 100644 --- a/include/net/ieee80211_radiotap.h +++ b/include/net/ieee80211_radiotap.h @@ -582,6 +582,7 @@ enum ieee80211_radiotap_eht_usig_tb { /** * ieee80211_get_radiotap_len - get radiotap header length * @data: pointer to the header + * Return: the radiotap header length */ static inline u16 ieee80211_get_radiotap_len(const char *data) { diff --git a/include/net/inet_frag.h b/include/net/inet_frag.h index 153960663ce4c2389259e0dc2bb4ba0b011dc698..5af6eb14c5db1533d559f565deb607934626219d 100644 --- a/include/net/inet_frag.h +++ b/include/net/inet_frag.h @@ -76,7 +76,7 @@ struct frag_v6_compare_key { * @stamp: timestamp of the last received fragment * @len: total length of the original datagram * @meat: length of received fragments so far - * @mono_delivery_time: stamp has a mono delivery time (EDT) + * @tstamp_type: stamp has a mono delivery time (EDT) * @flags: fragment queue flags * @max_size: maximum received fragment size * @fqdir: pointer to struct fqdir @@ -97,7 +97,7 @@ struct inet_frag_queue { ktime_t stamp; int len; int meat; - u8 mono_delivery_time; + u8 tstamp_type; __u8 flags; u16 max_size; struct fqdir *fqdir; diff --git a/include/net/inet_timewait_sock.h b/include/net/inet_timewait_sock.h index 2a536eea9424ea4fad65a1321217b07d2346e638..f88b6826901275c899e79272ae39c5c4aeaaf09b 100644 --- a/include/net/inet_timewait_sock.h +++ b/include/net/inet_timewait_sock.h @@ -93,17 +93,14 @@ struct inet_timewait_sock *inet_twsk_alloc(const struct sock *sk, struct inet_timewait_death_row *dr, const int state); -void inet_twsk_hashdance(struct inet_timewait_sock *tw, struct sock *sk, - struct inet_hashinfo *hashinfo); +void inet_twsk_hashdance_schedule(struct inet_timewait_sock *tw, + struct sock *sk, + struct inet_hashinfo *hashinfo, + int timeo); void __inet_twsk_schedule(struct inet_timewait_sock *tw, int timeo, bool rearm); -static inline void inet_twsk_schedule(struct inet_timewait_sock *tw, int timeo) -{ - __inet_twsk_schedule(tw, timeo, false); -} - static inline void inet_twsk_reschedule(struct inet_timewait_sock *tw, int timeo) { __inet_twsk_schedule(tw, timeo, true); diff --git a/include/net/ip.h b/include/net/ip.h index 6d735e00d3f3ec4b01dd8e4fca93432496c0b62c..c5606cadb1a552f3e282a5e1e721fd47b07432b2 100644 --- a/include/net/ip.h +++ b/include/net/ip.h @@ -506,8 +506,7 @@ static inline unsigned int ip_skb_dst_mtu(struct sock *sk, return mtu - lwtunnel_headroom(skb_dst(skb)->lwtstate, mtu); } -struct dst_metrics *ip_fib_metrics_init(struct net *net, struct nlattr *fc_mx, - int fc_mx_len, +struct dst_metrics *ip_fib_metrics_init(struct nlattr *fc_mx, int fc_mx_len, struct netlink_ext_ack *extack); static inline void ip_fib_metrics_put(struct dst_metrics *fib_metrics) { diff --git a/include/net/ip_tunnels.h b/include/net/ip_tunnels.h index 9a6a08ec77139c9bfb1bf70d393fc45bfad8b3e0..5a530d4fb02c68591377f8751520ef4541cb66ae 100644 --- a/include/net/ip_tunnels.h +++ b/include/net/ip_tunnels.h @@ -247,6 +247,18 @@ static inline bool ip_tunnel_is_options_present(const unsigned long *flags) return ip_tunnel_flags_intersect(flags, present); } +static inline void ip_tunnel_set_encflags_present(unsigned long *flags) +{ + IP_TUNNEL_DECLARE_FLAGS(present) = { }; + + __set_bit(IP_TUNNEL_CSUM_BIT, present); + __set_bit(IP_TUNNEL_DONT_FRAGMENT_BIT, present); + __set_bit(IP_TUNNEL_OAM_BIT, present); + __set_bit(IP_TUNNEL_CRIT_OPT_BIT, present); + + ip_tunnel_flags_or(flags, flags, present); +} + static inline bool ip_tunnel_flags_is_be16_compat(const unsigned long *flags) { IP_TUNNEL_DECLARE_FLAGS(supp) = { }; diff --git a/include/net/mac80211.h b/include/net/mac80211.h index cafc664ee53185dd89104892ad66bbd962635651..ecfa65ade226252092ae18b5de93e8db9d30c43f 100644 --- a/include/net/mac80211.h +++ b/include/net/mac80211.h @@ -362,6 +362,7 @@ struct ieee80211_vif_chanctx_switch { * status changed. * @BSS_CHANGED_MLD_VALID_LINKS: MLD valid links status changed. * @BSS_CHANGED_MLD_TTLM: negotiated TID to link mapping was changed + * @BSS_CHANGED_TPE: transmit power envelope changed */ enum ieee80211_bss_change { BSS_CHANGED_ASSOC = 1<<0, @@ -398,6 +399,7 @@ enum ieee80211_bss_change { BSS_CHANGED_UNSOL_BCAST_PROBE_RESP = 1<<31, BSS_CHANGED_MLD_VALID_LINKS = BIT_ULL(33), BSS_CHANGED_MLD_TTLM = BIT_ULL(34), + BSS_CHANGED_TPE = BIT_ULL(35), /* when adding here, make sure to change ieee80211_reconfig */ }; @@ -550,6 +552,39 @@ struct ieee80211_fils_discovery { u32 max_interval; }; +#define IEEE80211_TPE_EIRP_ENTRIES_320MHZ 5 +struct ieee80211_parsed_tpe_eirp { + bool valid; + s8 power[IEEE80211_TPE_EIRP_ENTRIES_320MHZ]; + u8 count; +}; + +#define IEEE80211_TPE_PSD_ENTRIES_320MHZ 16 +struct ieee80211_parsed_tpe_psd { + bool valid; + s8 power[IEEE80211_TPE_PSD_ENTRIES_320MHZ]; + u8 count, n; +}; + +/** + * struct ieee80211_parsed_tpe - parsed transmit power envelope information + * @max_local: maximum local EIRP, one value for 20, 40, 80, 160, 320 MHz each + * (indexed by TX power category) + * @max_reg_client: maximum regulatory client EIRP, one value for 20, 40, 80, + * 160, 320 MHz each + * (indexed by TX power category) + * @psd_local: maximum local power spectral density, one value for each 20 MHz + * subchannel per bss_conf's chanreq.oper + * (indexed by TX power category) + * @psd_reg_client: maximum regulatory power spectral density, one value for + * each 20 MHz subchannel per bss_conf's chanreq.oper + * (indexed by TX power category) + */ +struct ieee80211_parsed_tpe { + struct ieee80211_parsed_tpe_eirp max_local[2], max_reg_client[2]; + struct ieee80211_parsed_tpe_psd psd_local[2], psd_reg_client[2]; +}; + /** * struct ieee80211_bss_conf - holds the BSS's changing parameters * @@ -662,8 +697,7 @@ struct ieee80211_fils_discovery { * @beacon_tx_rate: The configured beacon transmit rate that needs to be passed * to driver when rate control is offloaded to firmware. * @power_type: power type of BSS for 6 GHz - * @tx_pwr_env: transmit power envelope array of BSS. - * @tx_pwr_env_num: number of @tx_pwr_env. + * @tpe: transmit power envelope information * @pwr_reduction: power constraint of BSS. * @eht_support: does this BSS support EHT * @csa_active: marks whether a channel switch is going on. @@ -701,6 +735,9 @@ struct ieee80211_fils_discovery { * beamformee * @eht_mu_beamformer: in AP-mode, does this BSS enable operation as an EHT MU * beamformer + * @eht_80mhz_full_bw_ul_mumimo: in AP-mode, does this BSS support the + * reception of an EHT TB PPDU on an RU that spans the entire PPDU + * bandwidth */ struct ieee80211_bss_conf { struct ieee80211_vif *vif; @@ -766,8 +803,9 @@ struct ieee80211_bss_conf { u32 unsol_bcast_probe_resp_interval; struct cfg80211_bitrate_mask beacon_tx_rate; enum ieee80211_ap_reg_power power_type; - struct ieee80211_tx_pwr_env tx_pwr_env[IEEE80211_TPE_MAX_IE_COUNT]; - u8 tx_pwr_env_num; + + struct ieee80211_parsed_tpe tpe; + u8 pwr_reduction; bool eht_support; @@ -793,6 +831,7 @@ struct ieee80211_bss_conf { bool eht_su_beamformer; bool eht_su_beamformee; bool eht_mu_beamformer; + bool eht_80mhz_full_bw_ul_mumimo; }; /** diff --git a/include/net/netdev_queues.h b/include/net/netdev_queues.h index a8a7e48dfa6caf352ad78767dc271f34206af797..5ca019d294ca3dbfba831c01f46ccc5c2814f15d 100644 --- a/include/net/netdev_queues.h +++ b/include/net/netdev_queues.h @@ -62,6 +62,8 @@ struct netdev_queue_stats_tx { * statistics will not generally add up to the total number of events for * the device. The @get_base_stats callback allows filling in the delta * between events for currently live queues and overall device history. + * @get_base_stats can also be used to report any miscellaneous packets + * transferred outside of the main set of queues used by the networking stack. * When the statistics for the entire device are queried, first @get_base_stats * is issued to collect the delta, and then a series of per-queue callbacks. * Only statistics which are set in @get_base_stats will be reported diff --git a/include/net/netns/ipv4.h b/include/net/netns/ipv4.h index c356c458b3409f966c3c2a1feddc0fa26b5f7088..a91bb971f9015d25151523eb53b205edbd65f003 100644 --- a/include/net/netns/ipv4.h +++ b/include/net/netns/ipv4.h @@ -170,6 +170,7 @@ struct netns_ipv4 { u8 sysctl_tcp_sack; u8 sysctl_tcp_window_scaling; u8 sysctl_tcp_timestamps; + int sysctl_tcp_rto_min_us; u8 sysctl_tcp_recovery; u8 sysctl_tcp_thin_linear_timeouts; u8 sysctl_tcp_slow_start_after_idle; diff --git a/include/net/page_pool/types.h b/include/net/page_pool/types.h index b088d131aeb0d039ce2c34fc6674c616f2f2559d..7e8477057f3d14103f9132c9366a2ae9ed5f0724 100644 --- a/include/net/page_pool/types.h +++ b/include/net/page_pool/types.h @@ -45,16 +45,17 @@ struct pp_alloc_cache { /** * struct page_pool_params - page pool parameters + * @fast: params accessed frequently on hotpath * @order: 2^order pages on allocation * @pool_size: size of the ptr_ring * @nid: NUMA node id to allocate from pages from * @dev: device, for DMA pre-mapping purposes - * @netdev: netdev this pool will serve (leave as NULL if none or multiple) * @napi: NAPI which is the sole consumer of pages, otherwise NULL * @dma_dir: DMA mapping direction * @max_len: max DMA sync memory size for PP_FLAG_DMA_SYNC_DEV * @offset: DMA sync address offset for PP_FLAG_DMA_SYNC_DEV - * @netdev: corresponding &net_device for Netlink introspection + * @slow: params with slowpath access only (initialization and Netlink) + * @netdev: netdev this pool will serve (leave as NULL if none or multiple) * @flags: PP_FLAG_DMA_MAP, PP_FLAG_DMA_SYNC_DEV, PP_FLAG_SYSTEM_POOL */ struct page_pool_params { diff --git a/include/net/regulatory.h b/include/net/regulatory.h index ebf9e028d1ef90661605f694e212df55701e9376..a103f4c8cf756e57386c32cda40b6e261d8de46e 100644 --- a/include/net/regulatory.h +++ b/include/net/regulatory.h @@ -71,8 +71,6 @@ enum environment_cap { * CRDA and can be used by other regulatory requests. When a * the last request is not yet processed we must yield until it * is processed before processing any new requests. - * @country_ie_checksum: checksum of the last processed and accepted - * country IE * @country_ie_env: lets us know if the AP is telling us we are outdoor, * indoor, or if it doesn't matter * @list: used to insert into the reg_requests_list linked list diff --git a/include/net/request_sock.h b/include/net/request_sock.h index d88c0dfc2d46831e63492becfa15603624fc457d..b07b1cd14e9f4d41d2b4fbad6015bbfae4190636 100644 --- a/include/net/request_sock.h +++ b/include/net/request_sock.h @@ -128,39 +128,6 @@ static inline struct sock *skb_steal_sock(struct sk_buff *skb, return sk; } -static inline struct request_sock * -reqsk_alloc_noprof(const struct request_sock_ops *ops, struct sock *sk_listener, - bool attach_listener) -{ - struct request_sock *req; - - req = kmem_cache_alloc_noprof(ops->slab, GFP_ATOMIC | __GFP_NOWARN); - if (!req) - return NULL; - req->rsk_listener = NULL; - if (attach_listener) { - if (unlikely(!refcount_inc_not_zero(&sk_listener->sk_refcnt))) { - kmem_cache_free(ops->slab, req); - return NULL; - } - req->rsk_listener = sk_listener; - } - req->rsk_ops = ops; - req_to_sk(req)->sk_prot = sk_listener->sk_prot; - sk_node_init(&req_to_sk(req)->sk_node); - sk_tx_queue_clear(req_to_sk(req)); - req->saved_syn = NULL; - req->syncookie = 0; - req->timeout = 0; - req->num_timeout = 0; - req->num_retrans = 0; - req->sk = NULL; - refcount_set(&req->rsk_refcnt, 0); - - return req; -} -#define reqsk_alloc(...) alloc_hooks(reqsk_alloc_noprof(__VA_ARGS__)) - static inline void __reqsk_free(struct request_sock *req) { req->rsk_ops->destructor(req); @@ -172,14 +139,14 @@ static inline void __reqsk_free(struct request_sock *req) static inline void reqsk_free(struct request_sock *req) { - WARN_ON_ONCE(refcount_read(&req->rsk_refcnt) != 0); + DEBUG_NET_WARN_ON_ONCE(refcount_read(&req->rsk_refcnt) != 0); __reqsk_free(req); } static inline void reqsk_put(struct request_sock *req) { if (refcount_dec_and_test(&req->rsk_refcnt)) - reqsk_free(req); + __reqsk_free(req); } /* @@ -285,4 +252,16 @@ static inline int reqsk_queue_len_young(const struct request_sock_queue *queue) return atomic_read(&queue->young); } +/* RFC 7323 2.3 Using the Window Scale Option + * The window field (SEG.WND) of every outgoing segment, with the + * exception of segments, MUST be right-shifted by + * Rcv.Wind.Shift bits. + * + * This means the SEG.WND carried in SYNACK can not exceed 65535. + * We use this property to harden TCP stack while in NEW_SYN_RECV state. + */ +static inline u32 tcp_synack_window(const struct request_sock *req) +{ + return min(req->rsk_rcv_wnd, 65535U); +} #endif /* _REQUEST_SOCK_H */ diff --git a/include/net/rtnetlink.h b/include/net/rtnetlink.h index 3bfb80bad1739d244a3906fa7f0e1a606dfaf868..b45d57b5968af4d17a1fc002a75a0923aefabf12 100644 --- a/include/net/rtnetlink.h +++ b/include/net/rtnetlink.h @@ -13,6 +13,7 @@ enum rtnl_link_flags { RTNL_FLAG_DOIT_UNLOCKED = BIT(0), RTNL_FLAG_BULK_DEL_SUPPORTED = BIT(1), RTNL_FLAG_DUMP_UNLOCKED = BIT(2), + RTNL_FLAG_DUMP_SPLIT_NLM_DONE = BIT(3), /* legacy behavior */ }; enum rtnl_kinds { diff --git a/include/net/seg6.h b/include/net/seg6.h index af668f17b3988d890fed907a22339cfc1d3fd25c..82b3fbbcbb93511b5aba8be366bd04ea4c027ca8 100644 --- a/include/net/seg6.h +++ b/include/net/seg6.h @@ -52,10 +52,17 @@ static inline struct seg6_pernet_data *seg6_pernet(struct net *net) extern int seg6_init(void); extern void seg6_exit(void); +#ifdef CONFIG_IPV6_SEG6_LWTUNNEL extern int seg6_iptunnel_init(void); extern void seg6_iptunnel_exit(void); extern int seg6_local_init(void); extern void seg6_local_exit(void); +#else +static inline int seg6_iptunnel_init(void) { return 0; } +static inline void seg6_iptunnel_exit(void) {} +static inline int seg6_local_init(void) { return 0; } +static inline void seg6_local_exit(void) {} +#endif extern bool seg6_validate_srh(struct ipv6_sr_hdr *srh, int len, bool reduced); extern struct ipv6_sr_hdr *seg6_get_srh(struct sk_buff *skb, int flags); diff --git a/include/net/seg6_hmac.h b/include/net/seg6_hmac.h index 2b5d2ee5613e83a2b1d83fb806ab990fffb1f36c..24f733b3e3fe9c4afa14a017124026b36815630d 100644 --- a/include/net/seg6_hmac.h +++ b/include/net/seg6_hmac.h @@ -49,9 +49,16 @@ extern int seg6_hmac_info_del(struct net *net, u32 key); extern int seg6_push_hmac(struct net *net, struct in6_addr *saddr, struct ipv6_sr_hdr *srh); extern bool seg6_hmac_validate_skb(struct sk_buff *skb); +#ifdef CONFIG_IPV6_SEG6_HMAC extern int seg6_hmac_init(void); extern void seg6_hmac_exit(void); extern int seg6_hmac_net_init(struct net *net); extern void seg6_hmac_net_exit(struct net *net); +#else +static inline int seg6_hmac_init(void) { return 0; } +static inline void seg6_hmac_exit(void) {} +static inline int seg6_hmac_net_init(struct net *net) { return 0; } +static inline void seg6_hmac_net_exit(struct net *net) {} +#endif #endif diff --git a/include/net/sock.h b/include/net/sock.h index 5f4d0629348f3fcb7b8d5e5e0796a35a9b913101..b30ea0c342a652b371c6ad45ab6e5f835ba31aeb 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2063,17 +2063,10 @@ sk_dst_get(const struct sock *sk) static inline void __dst_negative_advice(struct sock *sk) { - struct dst_entry *ndst, *dst = __sk_dst_get(sk); + struct dst_entry *dst = __sk_dst_get(sk); - if (dst && dst->ops->negative_advice) { - ndst = dst->ops->negative_advice(dst); - - if (ndst != dst) { - rcu_assign_pointer(sk->sk_dst_cache, ndst); - sk_tx_queue_clear(sk); - WRITE_ONCE(sk->sk_dst_pending_confirm, 0); - } - } + if (dst && dst->ops->negative_advice) + dst->ops->negative_advice(sk, dst); } static inline void dst_negative_advice(struct sock *sk) @@ -2102,7 +2095,7 @@ sk_dst_set(struct sock *sk, struct dst_entry *dst) sk_tx_queue_clear(sk); WRITE_ONCE(sk->sk_dst_pending_confirm, 0); - old_dst = xchg((__force struct dst_entry **)&sk->sk_dst_cache, dst); + old_dst = unrcu_pointer(xchg(&sk->sk_dst_cache, RCU_INITIALIZER(dst))); dst_release(old_dst); } diff --git a/include/net/tcp.h b/include/net/tcp.h index 060e95b331a286ad7c355be11dc03250d2944920..a70fc39090fe86e892137fa5b0b3362262e1c127 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -677,6 +677,7 @@ void tcp_skb_collapse_tstamp(struct sk_buff *skb, /* tcp_input.c */ void tcp_rearm_rto(struct sock *sk); void tcp_synack_rtt_meas(struct sock *sk, struct request_sock *req); +void tcp_done_with_error(struct sock *sk, int err); void tcp_reset(struct sock *sk, struct sk_buff *skb); void tcp_fin(struct sock *sk); void tcp_check_space(struct sock *sk); @@ -1065,11 +1066,19 @@ static inline bool tcp_skb_can_collapse_to(const struct sk_buff *skb) static inline bool tcp_skb_can_collapse(const struct sk_buff *to, const struct sk_buff *from) { + /* skb_cmp_decrypted() not needed, use tcp_write_collapse_fence() */ return likely(tcp_skb_can_collapse_to(to) && mptcp_skb_can_collapse(to, from) && skb_pure_zcopy_same(to, from)); } +static inline bool tcp_skb_can_collapse_rx(const struct sk_buff *to, + const struct sk_buff *from) +{ + return likely(mptcp_skb_can_collapse(to, from) && + !skb_cmp_decrypted(to, from)); +} + /* Events passed to congestion control interface */ enum tcp_ca_event { CA_EVENT_TX_START, /* first transmit when no packets in flight */ @@ -1215,7 +1224,7 @@ extern struct tcp_congestion_ops tcp_reno; struct tcp_congestion_ops *tcp_ca_find(const char *name); struct tcp_congestion_ops *tcp_ca_find_key(u32 key); -u32 tcp_ca_get_key_by_name(struct net *net, const char *name, bool *ecn_ca); +u32 tcp_ca_get_key_by_name(const char *name, bool *ecn_ca); #ifdef CONFIG_INET char *tcp_ca_get_name_by_key(u32 key, char *buffer); #else @@ -2094,6 +2103,14 @@ static inline void tcp_rtx_queue_unlink_and_free(struct sk_buff *skb, struct soc tcp_wmem_free_skb(sk, skb); } +static inline void tcp_write_collapse_fence(struct sock *sk) +{ + struct sk_buff *skb = tcp_write_queue_tail(sk); + + if (skb) + TCP_SKB_CB(skb)->eor = 1; +} + static inline void tcp_push_pending_frames(struct sock *sk) { if (tcp_send_head(sk)) { diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h index 471e177362b4c02ed37a85d4b5b1174aed67f18c..5d8e9ed2c0056f8437baf6b7d1853d30f079a64a 100644 --- a/include/net/tcp_ao.h +++ b/include/net/tcp_ao.h @@ -86,7 +86,8 @@ static inline int tcp_ao_sizeof_key(const struct tcp_ao_key *key) struct tcp_ao_info { /* List of tcp_ao_key's */ struct hlist_head head; - /* current_key and rnext_key aren't maintained on listen sockets. + /* current_key and rnext_key are maintained on sockets + * in TCP_AO_ESTABLISHED states. * Their purpose is to cache keys on established connections, * saving needless lookups. Never dereference any of them from * listen sockets. @@ -201,9 +202,9 @@ struct tcp6_ao_context { }; struct tcp_sigpool; +/* Established states are fast-path and there always is current_key/rnext_key */ #define TCP_AO_ESTABLISHED (TCPF_ESTABLISHED | TCPF_FIN_WAIT1 | TCPF_FIN_WAIT2 | \ - TCPF_CLOSE | TCPF_CLOSE_WAIT | \ - TCPF_LAST_ACK | TCPF_CLOSING) + TCPF_CLOSE_WAIT | TCPF_LAST_ACK | TCPF_CLOSING) int tcp_ao_transmit_skb(struct sock *sk, struct sk_buff *skb, struct tcp_ao_key *key, struct tcphdr *th, diff --git a/include/sound/pcm.h b/include/sound/pcm.h index 61c6054618c8ead951b15390ec91bb40b4a63ca2..3edd7a7346daaaf30ddb44e1151d8caae8371a64 100644 --- a/include/sound/pcm.h +++ b/include/sound/pcm.h @@ -124,7 +124,7 @@ struct snd_pcm_ops { #define SNDRV_PCM_RATE_768000 (1U<<16) /* 768000Hz */ #define SNDRV_PCM_RATE_CONTINUOUS (1U<<30) /* continuous range */ -#define SNDRV_PCM_RATE_KNOT (1U<<31) /* supports more non-continuos rates */ +#define SNDRV_PCM_RATE_KNOT (1U<<31) /* supports more non-continuous rates */ #define SNDRV_PCM_RATE_8000_44100 (SNDRV_PCM_RATE_8000|SNDRV_PCM_RATE_11025|\ SNDRV_PCM_RATE_16000|SNDRV_PCM_RATE_22050|\ diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 90706a47f6ffe974d346502c5511d9ac63a4f86e..35bcf52dbc6526469a933f67af8a2374cad88f50 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1425,6 +1425,8 @@ enum { #define BPF_F_TEST_RUN_ON_CPU (1U << 0) /* If set, XDP frames will be transmitted after processing */ #define BPF_F_TEST_XDP_LIVE_FRAMES (1U << 1) +/* If set, apply CHECKSUM_COMPLETE to skb and validate the checksum */ +#define BPF_F_TEST_SKB_CHECKSUM_COMPLETE (1U << 2) /* type for BPF_ENABLE_STATS */ enum bpf_stats_type { @@ -6207,12 +6209,17 @@ union { \ __u64 :64; \ } __attribute__((aligned(8))) +/* The enum used in skb->tstamp_type. It specifies the clock type + * of the time stored in the skb->tstamp. + */ enum { - BPF_SKB_TSTAMP_UNSPEC, - BPF_SKB_TSTAMP_DELIVERY_MONO, /* tstamp has mono delivery time */ - /* For any BPF_SKB_TSTAMP_* that the bpf prog cannot handle, - * the bpf prog should handle it like BPF_SKB_TSTAMP_UNSPEC - * and try to deduce it by ingress, egress or skb->sk->sk_clockid. + BPF_SKB_TSTAMP_UNSPEC = 0, /* DEPRECATED */ + BPF_SKB_TSTAMP_DELIVERY_MONO = 1, /* DEPRECATED */ + BPF_SKB_CLOCK_REALTIME = 0, + BPF_SKB_CLOCK_MONOTONIC = 1, + BPF_SKB_CLOCK_TAI = 2, + /* For any future BPF_SKB_CLOCK_* that the bpf prog cannot handle, + * the bpf prog can try to deduce it by ingress/egress/skb->sk->sk_clockid. */ }; diff --git a/include/uapi/linux/cn_proc.h b/include/uapi/linux/cn_proc.h index f2afb7cc4926cdc048541a2ca2e18b8e474ea69d..18e3745b86cd481f97fe7330a297373a25c29209 100644 --- a/include/uapi/linux/cn_proc.h +++ b/include/uapi/linux/cn_proc.h @@ -69,8 +69,7 @@ struct proc_input { static inline enum proc_cn_event valid_event(enum proc_cn_event ev_type) { - ev_type &= PROC_EVENT_ALL; - return ev_type; + return (enum proc_cn_event)(ev_type & PROC_EVENT_ALL); } /* diff --git a/include/uapi/linux/kd.h b/include/uapi/linux/kd.h index 8ddb2219a84bdfa6412097e0375e7c3f92a19560..6b384065c01356efbe8db490fd3f85c9cebbe6d3 100644 --- a/include/uapi/linux/kd.h +++ b/include/uapi/linux/kd.h @@ -5,61 +5,60 @@ #include /* 0x4B is 'K', to avoid collision with termios and vt */ -#define KD_IOCTL_BASE 'K' -#define GIO_FONT _IO(KD_IOCTL_BASE, 0x60) /* gets font in expanded form */ -#define PIO_FONT _IO(KD_IOCTL_BASE, 0x61) /* use font in expanded form */ +#define GIO_FONT 0x4B60 /* gets font in expanded form */ +#define PIO_FONT 0x4B61 /* use font in expanded form */ -#define GIO_FONTX _IO(KD_IOCTL_BASE, 0x6B) /* get font using struct consolefontdesc */ -#define PIO_FONTX _IO(KD_IOCTL_BASE, 0x6C) /* set font using struct consolefontdesc */ +#define GIO_FONTX 0x4B6B /* get font using struct consolefontdesc */ +#define PIO_FONTX 0x4B6C /* set font using struct consolefontdesc */ struct consolefontdesc { unsigned short charcount; /* characters in font (256 or 512) */ unsigned short charheight; /* scan lines per character (1-32) */ char __user *chardata; /* font data in expanded form */ }; -#define PIO_FONTRESET _IO(KD_IOCTL_BASE, 0x6D) /* reset to default font */ +#define PIO_FONTRESET 0x4B6D /* reset to default font */ -#define GIO_CMAP _IO(KD_IOCTL_BASE, 0x70) /* gets colour palette on VGA+ */ -#define PIO_CMAP _IO(KD_IOCTL_BASE, 0x71) /* sets colour palette on VGA+ */ +#define GIO_CMAP 0x4B70 /* gets colour palette on VGA+ */ +#define PIO_CMAP 0x4B71 /* sets colour palette on VGA+ */ -#define KIOCSOUND _IO(KD_IOCTL_BASE, 0x2F) /* start sound generation (0 for off) */ -#define KDMKTONE _IO(KD_IOCTL_BASE, 0x30) /* generate tone */ +#define KIOCSOUND 0x4B2F /* start sound generation (0 for off) */ +#define KDMKTONE 0x4B30 /* generate tone */ -#define KDGETLED _IO(KD_IOCTL_BASE, 0x31) /* return current led state */ -#define KDSETLED _IO(KD_IOCTL_BASE, 0x32) /* set led state [lights, not flags] */ +#define KDGETLED 0x4B31 /* return current led state */ +#define KDSETLED 0x4B32 /* set led state [lights, not flags] */ #define LED_SCR 0x01 /* scroll lock led */ #define LED_NUM 0x02 /* num lock led */ #define LED_CAP 0x04 /* caps lock led */ -#define KDGKBTYPE _IO(KD_IOCTL_BASE, 0x33) /* get keyboard type */ +#define KDGKBTYPE 0x4B33 /* get keyboard type */ #define KB_84 0x01 #define KB_101 0x02 /* this is what we always answer */ #define KB_OTHER 0x03 -#define KDADDIO _IO(KD_IOCTL_BASE, 0x34) /* add i/o port as valid */ -#define KDDELIO _IO(KD_IOCTL_BASE, 0x35) /* del i/o port as valid */ -#define KDENABIO _IO(KD_IOCTL_BASE, 0x36) /* enable i/o to video board */ -#define KDDISABIO _IO(KD_IOCTL_BASE, 0x37) /* disable i/o to video board */ +#define KDADDIO 0x4B34 /* add i/o port as valid */ +#define KDDELIO 0x4B35 /* del i/o port as valid */ +#define KDENABIO 0x4B36 /* enable i/o to video board */ +#define KDDISABIO 0x4B37 /* disable i/o to video board */ -#define KDSETMODE _IO(KD_IOCTL_BASE, 0x3A) /* set text/graphics mode */ +#define KDSETMODE 0x4B3A /* set text/graphics mode */ #define KD_TEXT 0x00 #define KD_GRAPHICS 0x01 #define KD_TEXT0 0x02 /* obsolete */ #define KD_TEXT1 0x03 /* obsolete */ -#define KDGETMODE _IO(KD_IOCTL_BASE, 0x3B) /* get current mode */ +#define KDGETMODE 0x4B3B /* get current mode */ -#define KDMAPDISP _IO(KD_IOCTL_BASE, 0x3C) /* map display into address space */ -#define KDUNMAPDISP _IO(KD_IOCTL_BASE, 0x3D) /* unmap display from address space */ +#define KDMAPDISP 0x4B3C /* map display into address space */ +#define KDUNMAPDISP 0x4B3D /* unmap display from address space */ typedef char scrnmap_t; #define E_TABSZ 256 -#define GIO_SCRNMAP _IO(KD_IOCTL_BASE, 0x40) /* get screen mapping from kernel */ -#define PIO_SCRNMAP _IO(KD_IOCTL_BASE, 0x41) /* put screen mapping table in kernel */ -#define GIO_UNISCRNMAP _IO(KD_IOCTL_BASE, 0x69) /* get full Unicode screen mapping */ -#define PIO_UNISCRNMAP _IO(KD_IOCTL_BASE, 0x6A) /* set full Unicode screen mapping */ +#define GIO_SCRNMAP 0x4B40 /* get screen mapping from kernel */ +#define PIO_SCRNMAP 0x4B41 /* put screen mapping table in kernel */ +#define GIO_UNISCRNMAP 0x4B69 /* get full Unicode screen mapping */ +#define PIO_UNISCRNMAP 0x4B6A /* set full Unicode screen mapping */ -#define GIO_UNIMAP _IO(KD_IOCTL_BASE, 0x66) /* get unicode-to-font mapping from kernel */ +#define GIO_UNIMAP 0x4B66 /* get unicode-to-font mapping from kernel */ struct unipair { unsigned short unicode; unsigned short fontpos; @@ -68,8 +67,8 @@ struct unimapdesc { unsigned short entry_ct; struct unipair __user *entries; }; -#define PIO_UNIMAP _IO(KD_IOCTL_BASE, 0x67) /* put unicode-to-font mapping in kernel */ -#define PIO_UNIMAPCLR _IO(KD_IOCTL_BASE, 0x68) /* clear table, possibly advise hash algorithm */ +#define PIO_UNIMAP 0x4B67 /* put unicode-to-font mapping in kernel */ +#define PIO_UNIMAPCLR 0x4B68 /* clear table, possibly advise hash algorithm */ struct unimapinit { unsigned short advised_hashsize; /* 0 if no opinion */ unsigned short advised_hashstep; /* 0 if no opinion */ @@ -84,19 +83,19 @@ struct unimapinit { #define K_MEDIUMRAW 0x02 #define K_UNICODE 0x03 #define K_OFF 0x04 -#define KDGKBMODE _IO(KD_IOCTL_BASE, 0x44) /* gets current keyboard mode */ -#define KDSKBMODE _IO(KD_IOCTL_BASE, 0x45) /* sets current keyboard mode */ +#define KDGKBMODE 0x4B44 /* gets current keyboard mode */ +#define KDSKBMODE 0x4B45 /* sets current keyboard mode */ #define K_METABIT 0x03 #define K_ESCPREFIX 0x04 -#define KDGKBMETA _IO(KD_IOCTL_BASE, 0x62) /* gets meta key handling mode */ -#define KDSKBMETA _IO(KD_IOCTL_BASE, 0x63) /* sets meta key handling mode */ +#define KDGKBMETA 0x4B62 /* gets meta key handling mode */ +#define KDSKBMETA 0x4B63 /* sets meta key handling mode */ #define K_SCROLLLOCK 0x01 #define K_NUMLOCK 0x02 #define K_CAPSLOCK 0x04 -#define KDGKBLED _IO(KD_IOCTL_BASE, 0x64) /* get led flags (not lights) */ -#define KDSKBLED _IO(KD_IOCTL_BASE, 0x65) /* set led flags (not lights) */ +#define KDGKBLED 0x4B64 /* get led flags (not lights) */ +#define KDSKBLED 0x4B65 /* set led flags (not lights) */ struct kbentry { unsigned char kb_table; @@ -108,15 +107,15 @@ struct kbentry { #define K_ALTTAB 0x02 #define K_ALTSHIFTTAB 0x03 -#define KDGKBENT _IO(KD_IOCTL_BASE, 0x46) /* gets one entry in translation table */ -#define KDSKBENT _IO(KD_IOCTL_BASE, 0x47) /* sets one entry in translation table */ +#define KDGKBENT 0x4B46 /* gets one entry in translation table */ +#define KDSKBENT 0x4B47 /* sets one entry in translation table */ struct kbsentry { unsigned char kb_func; unsigned char kb_string[512]; }; -#define KDGKBSENT _IO(KD_IOCTL_BASE, 0x48) /* gets one function key string entry */ -#define KDSKBSENT _IO(KD_IOCTL_BASE, 0x49) /* sets one function key string entry */ +#define KDGKBSENT 0x4B48 /* gets one function key string entry */ +#define KDSKBSENT 0x4B49 /* sets one function key string entry */ struct kbdiacr { unsigned char diacr, base, result; @@ -125,8 +124,8 @@ struct kbdiacrs { unsigned int kb_cnt; /* number of entries in following array */ struct kbdiacr kbdiacr[256]; /* MAX_DIACR from keyboard.h */ }; -#define KDGKBDIACR _IO(KD_IOCTL_BASE, 0x4A) /* read kernel accent table */ -#define KDSKBDIACR _IO(KD_IOCTL_BASE, 0x4B) /* write kernel accent table */ +#define KDGKBDIACR 0x4B4A /* read kernel accent table */ +#define KDSKBDIACR 0x4B4B /* write kernel accent table */ struct kbdiacruc { unsigned int diacr, base, result; @@ -135,16 +134,16 @@ struct kbdiacrsuc { unsigned int kb_cnt; /* number of entries in following array */ struct kbdiacruc kbdiacruc[256]; /* MAX_DIACR from keyboard.h */ }; -#define KDGKBDIACRUC _IO(KD_IOCTL_BASE, 0xFA) /* read kernel accent table - UCS */ -#define KDSKBDIACRUC _IO(KD_IOCTL_BASE, 0xFB) /* write kernel accent table - UCS */ +#define KDGKBDIACRUC 0x4BFA /* read kernel accent table - UCS */ +#define KDSKBDIACRUC 0x4BFB /* write kernel accent table - UCS */ struct kbkeycode { unsigned int scancode, keycode; }; -#define KDGETKEYCODE _IO(KD_IOCTL_BASE, 0x4C) /* read kernel keycode table entry */ -#define KDSETKEYCODE _IO(KD_IOCTL_BASE, 0x4D) /* write kernel keycode table entry */ +#define KDGETKEYCODE 0x4B4C /* read kernel keycode table entry */ +#define KDSETKEYCODE 0x4B4D /* write kernel keycode table entry */ -#define KDSIGACCEPT _IO(KD_IOCTL_BASE, 0x4E) /* accept kbd generated signals */ +#define KDSIGACCEPT 0x4B4E /* accept kbd generated signals */ struct kbd_repeat { int delay; /* in msec; <= 0: don't change */ @@ -152,11 +151,10 @@ struct kbd_repeat { /* earlier this field was misnamed "rate" */ }; -#define KDKBDREP _IO(KD_IOCTL_BASE, 0x52) /* set keyboard delay/repeat rate; - * actually used values are returned - */ +#define KDKBDREP 0x4B52 /* set keyboard delay/repeat rate; + * actually used values are returned */ -#define KDFONTOP _IO(KD_IOCTL_BASE, 0x72) /* font operations */ +#define KDFONTOP 0x4B72 /* font operations */ struct console_font_op { unsigned int op; /* operation code KD_FONT_OP_* */ diff --git a/include/uapi/linux/netdev.h b/include/uapi/linux/netdev.h index a8188202413ec4cab8ea0c1df92637a64750cb4e..43742ac5b00da0aa66660d86cb627561d8eaf298 100644 --- a/include/uapi/linux/netdev.h +++ b/include/uapi/linux/netdev.h @@ -148,6 +148,7 @@ enum { NETDEV_A_QSTATS_RX_ALLOC_FAIL, NETDEV_A_QSTATS_RX_HW_DROPS, NETDEV_A_QSTATS_RX_HW_DROP_OVERRUNS, + NETDEV_A_QSTATS_RX_CSUM_COMPLETE, NETDEV_A_QSTATS_RX_CSUM_UNNECESSARY, NETDEV_A_QSTATS_RX_CSUM_NONE, NETDEV_A_QSTATS_RX_CSUM_BAD, diff --git a/include/uapi/linux/pkt_cls.h b/include/uapi/linux/pkt_cls.h index 229fc925ec3a299ece747c77076feb9afe193c90..b6d38f5fd7c05aaed160c93081d9d630f4056f1d 100644 --- a/include/uapi/linux/pkt_cls.h +++ b/include/uapi/linux/pkt_cls.h @@ -554,6 +554,9 @@ enum { TCA_FLOWER_KEY_SPI, /* be32 */ TCA_FLOWER_KEY_SPI_MASK, /* be32 */ + TCA_FLOWER_KEY_ENC_FLAGS, /* u32 */ + TCA_FLOWER_KEY_ENC_FLAGS_MASK, /* u32 */ + __TCA_FLOWER_MAX, }; diff --git a/io_uring/memmap.c b/io_uring/memmap.c index 4785d6af5fee9298940fa6c9a6cfb68c06df9d24..a0f32a255fd1e1a0e3bd0c2f20421d66fc0b2a25 100644 --- a/io_uring/memmap.c +++ b/io_uring/memmap.c @@ -244,6 +244,7 @@ __cold int io_uring_mmap(struct file *file, struct vm_area_struct *vma) struct io_ring_ctx *ctx = file->private_data; size_t sz = vma->vm_end - vma->vm_start; long offset = vma->vm_pgoff << PAGE_SHIFT; + unsigned int npages; void *ptr; ptr = io_uring_validate_mmap_request(file, vma->vm_pgoff, sz); @@ -253,8 +254,8 @@ __cold int io_uring_mmap(struct file *file, struct vm_area_struct *vma) switch (offset & IORING_OFF_MMAP_MASK) { case IORING_OFF_SQ_RING: case IORING_OFF_CQ_RING: - return io_uring_mmap_pages(ctx, vma, ctx->ring_pages, - ctx->n_ring_pages); + npages = min(ctx->n_ring_pages, (sz + PAGE_SIZE - 1) >> PAGE_SHIFT); + return io_uring_mmap_pages(ctx, vma, ctx->ring_pages, npages); case IORING_OFF_SQES: return io_uring_mmap_pages(ctx, vma, ctx->sqe_pages, ctx->n_sqe_pages); diff --git a/io_uring/net.c b/io_uring/net.c index 0a48596429d9f53edd252a58435619654f7ee7a0..7c98c4d5094633cb948bb761d2345e65cfb380f2 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -1127,6 +1127,9 @@ int io_recv(struct io_kiocb *req, unsigned int issue_flags) flags |= MSG_DONTWAIT; retry_multishot: + kmsg->msg.msg_inq = -1; + kmsg->msg.msg_flags = 0; + if (io_do_buffer_select(req)) { ret = io_recv_buf_select(req, kmsg, &len, issue_flags); if (unlikely(ret)) @@ -1134,9 +1137,6 @@ retry_multishot: sr->buf = NULL; } - kmsg->msg.msg_inq = -1; - kmsg->msg.msg_flags = 0; - if (flags & MSG_WAITALL) min_ret = iov_iter_count(&kmsg->msg.msg_iter); diff --git a/io_uring/opdef.c b/io_uring/opdef.c index 2de5cca9504eb4bc9be4002b0f9a7f27e9c605fc..2e3b7b16effb34154b2be88dc07af5e6e0a4699a 100644 --- a/io_uring/opdef.c +++ b/io_uring/opdef.c @@ -516,10 +516,12 @@ const struct io_cold_def io_cold_defs[] = { }, [IORING_OP_READ_FIXED] = { .name = "READ_FIXED", + .cleanup = io_readv_writev_cleanup, .fail = io_rw_fail, }, [IORING_OP_WRITE_FIXED] = { .name = "WRITE_FIXED", + .cleanup = io_readv_writev_cleanup, .fail = io_rw_fail, }, [IORING_OP_POLL_ADD] = { @@ -582,10 +584,12 @@ const struct io_cold_def io_cold_defs[] = { }, [IORING_OP_READ] = { .name = "READ", + .cleanup = io_readv_writev_cleanup, .fail = io_rw_fail, }, [IORING_OP_WRITE] = { .name = "WRITE", + .cleanup = io_readv_writev_cleanup, .fail = io_rw_fail, }, [IORING_OP_FADVISE] = { @@ -692,6 +696,7 @@ const struct io_cold_def io_cold_defs[] = { }, [IORING_OP_READ_MULTISHOT] = { .name = "READ_MULTISHOT", + .cleanup = io_readv_writev_cleanup, }, [IORING_OP_WAITID] = { .name = "WAITID", diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 976cb258a0edb2bd7762bb9d77fc2a0dd37dc33f..c938dea5ddbf3a7900ee25eeedc076e2bd69c0b5 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -782,8 +782,8 @@ bpf_local_storage_map_alloc(union bpf_attr *attr, nbuckets = max_t(u32, 2, nbuckets); smap->bucket_log = ilog2(nbuckets); - smap->buckets = bpf_map_kvcalloc(&smap->map, sizeof(*smap->buckets), - nbuckets, GFP_USER | __GFP_NOWARN); + smap->buckets = bpf_map_kvcalloc(&smap->map, nbuckets, + sizeof(*smap->buckets), GFP_USER | __GFP_NOWARN); if (!smap->buckets) { err = -ENOMEM; goto free_smap; diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 86c7884abaf87ab1cc0d4b83b564bba584772205..a2cf31b14be46de4128e7dbe37b333f413eb7b8d 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -12,6 +12,7 @@ #include #include #include +#include struct bpf_struct_ops_value { struct bpf_struct_ops_common_value common; @@ -56,6 +57,7 @@ struct bpf_struct_ops_map { struct bpf_struct_ops_link { struct bpf_link link; struct bpf_map __rcu *map; + wait_queue_head_t wait_hup; }; static DEFINE_MUTEX(update_mutex); @@ -757,7 +759,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, goto unlock; } - err = st_ops->reg(kdata); + err = st_ops->reg(kdata, NULL); if (likely(!err)) { /* This refcnt increment on the map here after * 'st_ops->reg()' is secure since the state of the @@ -805,7 +807,7 @@ static long bpf_struct_ops_map_delete_elem(struct bpf_map *map, void *key) BPF_STRUCT_OPS_STATE_TOBEFREE); switch (prev_state) { case BPF_STRUCT_OPS_STATE_INUSE: - st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data); + st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data, NULL); bpf_map_put(map); return 0; case BPF_STRUCT_OPS_STATE_TOBEFREE: @@ -1057,10 +1059,7 @@ static void bpf_struct_ops_map_link_dealloc(struct bpf_link *link) st_map = (struct bpf_struct_ops_map *) rcu_dereference_protected(st_link->map, true); if (st_map) { - /* st_link->map can be NULL if - * bpf_struct_ops_link_create() fails to register. - */ - st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data); + st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data, link); bpf_map_put(&st_map->map); } kfree(st_link); @@ -1075,7 +1074,8 @@ static void bpf_struct_ops_map_link_show_fdinfo(const struct bpf_link *link, st_link = container_of(link, struct bpf_struct_ops_link, link); rcu_read_lock(); map = rcu_dereference(st_link->map); - seq_printf(seq, "map_id:\t%d\n", map->id); + if (map) + seq_printf(seq, "map_id:\t%d\n", map->id); rcu_read_unlock(); } @@ -1088,7 +1088,8 @@ static int bpf_struct_ops_map_link_fill_link_info(const struct bpf_link *link, st_link = container_of(link, struct bpf_struct_ops_link, link); rcu_read_lock(); map = rcu_dereference(st_link->map); - info->struct_ops.map_id = map->id; + if (map) + info->struct_ops.map_id = map->id; rcu_read_unlock(); return 0; } @@ -1113,6 +1114,10 @@ static int bpf_struct_ops_map_link_update(struct bpf_link *link, struct bpf_map mutex_lock(&update_mutex); old_map = rcu_dereference_protected(st_link->map, lockdep_is_held(&update_mutex)); + if (!old_map) { + err = -ENOLINK; + goto err_out; + } if (expected_old_map && old_map != expected_old_map) { err = -EPERM; goto err_out; @@ -1125,7 +1130,7 @@ static int bpf_struct_ops_map_link_update(struct bpf_link *link, struct bpf_map goto err_out; } - err = st_map->st_ops_desc->st_ops->update(st_map->kvalue.data, old_st_map->kvalue.data); + err = st_map->st_ops_desc->st_ops->update(st_map->kvalue.data, old_st_map->kvalue.data, link); if (err) goto err_out; @@ -1139,11 +1144,53 @@ err_out: return err; } +static int bpf_struct_ops_map_link_detach(struct bpf_link *link) +{ + struct bpf_struct_ops_link *st_link = container_of(link, struct bpf_struct_ops_link, link); + struct bpf_struct_ops_map *st_map; + struct bpf_map *map; + + mutex_lock(&update_mutex); + + map = rcu_dereference_protected(st_link->map, lockdep_is_held(&update_mutex)); + if (!map) { + mutex_unlock(&update_mutex); + return 0; + } + st_map = container_of(map, struct bpf_struct_ops_map, map); + + st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data, link); + + RCU_INIT_POINTER(st_link->map, NULL); + /* Pair with bpf_map_get() in bpf_struct_ops_link_create() or + * bpf_map_inc() in bpf_struct_ops_map_link_update(). + */ + bpf_map_put(&st_map->map); + + mutex_unlock(&update_mutex); + + wake_up_interruptible_poll(&st_link->wait_hup, EPOLLHUP); + + return 0; +} + +static __poll_t bpf_struct_ops_map_link_poll(struct file *file, + struct poll_table_struct *pts) +{ + struct bpf_struct_ops_link *st_link = file->private_data; + + poll_wait(file, &st_link->wait_hup, pts); + + return rcu_access_pointer(st_link->map) ? 0 : EPOLLHUP; +} + static const struct bpf_link_ops bpf_struct_ops_map_lops = { .dealloc = bpf_struct_ops_map_link_dealloc, + .detach = bpf_struct_ops_map_link_detach, .show_fdinfo = bpf_struct_ops_map_link_show_fdinfo, .fill_link_info = bpf_struct_ops_map_link_fill_link_info, .update_map = bpf_struct_ops_map_link_update, + .poll = bpf_struct_ops_map_link_poll, }; int bpf_struct_ops_link_create(union bpf_attr *attr) @@ -1176,13 +1223,21 @@ int bpf_struct_ops_link_create(union bpf_attr *attr) if (err) goto err_out; - err = st_map->st_ops_desc->st_ops->reg(st_map->kvalue.data); + init_waitqueue_head(&link->wait_hup); + + /* Hold the update_mutex such that the subsystem cannot + * do link->ops->detach() before the link is fully initialized. + */ + mutex_lock(&update_mutex); + err = st_map->st_ops_desc->st_ops->reg(st_map->kvalue.data, &link->link); if (err) { + mutex_unlock(&update_mutex); bpf_link_cleanup(&link_primer); link = NULL; goto err_out; } RCU_INIT_POINTER(link->map, map); + mutex_unlock(&update_mutex); return bpf_link_settle(&link_primer); diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 821063660d9f9db1e4a5efac43cc9b59d8327378..ce47079682176195f5ae62e8bb4748d92bae5189 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3442,10 +3442,12 @@ btf_find_graph_root(const struct btf *btf, const struct btf_type *pt, goto end; \ } -static int btf_get_field_type(const char *name, u32 field_mask, u32 *seen_mask, +static int btf_get_field_type(const struct btf *btf, const struct btf_type *var_type, + u32 field_mask, u32 *seen_mask, int *align, int *sz) { int type = 0; + const char *name = __btf_name_by_offset(btf, var_type->name_off); if (field_mask & BPF_SPIN_LOCK) { if (!strcmp(name, "bpf_spin_lock")) { @@ -3481,7 +3483,7 @@ static int btf_get_field_type(const char *name, u32 field_mask, u32 *seen_mask, field_mask_test_name(BPF_REFCOUNT, "bpf_refcount"); /* Only return BPF_KPTR when all other types with matchable names fail */ - if (field_mask & BPF_KPTR) { + if (field_mask & BPF_KPTR && !__btf_type_is_struct(var_type)) { type = BPF_KPTR_REF; goto end; } @@ -3494,140 +3496,232 @@ end: #undef field_mask_test_name +/* Repeat a number of fields for a specified number of times. + * + * Copy the fields starting from the first field and repeat them for + * repeat_cnt times. The fields are repeated by adding the offset of each + * field with + * (i + 1) * elem_size + * where i is the repeat index and elem_size is the size of an element. + */ +static int btf_repeat_fields(struct btf_field_info *info, + u32 field_cnt, u32 repeat_cnt, u32 elem_size) +{ + u32 i, j; + u32 cur; + + /* Ensure not repeating fields that should not be repeated. */ + for (i = 0; i < field_cnt; i++) { + switch (info[i].type) { + case BPF_KPTR_UNREF: + case BPF_KPTR_REF: + case BPF_KPTR_PERCPU: + case BPF_LIST_HEAD: + case BPF_RB_ROOT: + break; + default: + return -EINVAL; + } + } + + cur = field_cnt; + for (i = 0; i < repeat_cnt; i++) { + memcpy(&info[cur], &info[0], field_cnt * sizeof(info[0])); + for (j = 0; j < field_cnt; j++) + info[cur++].off += (i + 1) * elem_size; + } + + return 0; +} + static int btf_find_struct_field(const struct btf *btf, const struct btf_type *t, u32 field_mask, - struct btf_field_info *info, int info_cnt) + struct btf_field_info *info, int info_cnt, + u32 level); + +/* Find special fields in the struct type of a field. + * + * This function is used to find fields of special types that is not a + * global variable or a direct field of a struct type. It also handles the + * repetition if it is the element type of an array. + */ +static int btf_find_nested_struct(const struct btf *btf, const struct btf_type *t, + u32 off, u32 nelems, + u32 field_mask, struct btf_field_info *info, + int info_cnt, u32 level) { - int ret, idx = 0, align, sz, field_type; - const struct btf_member *member; + int ret, err, i; + + level++; + if (level >= MAX_RESOLVE_DEPTH) + return -E2BIG; + + ret = btf_find_struct_field(btf, t, field_mask, info, info_cnt, level); + + if (ret <= 0) + return ret; + + /* Shift the offsets of the nested struct fields to the offsets + * related to the container. + */ + for (i = 0; i < ret; i++) + info[i].off += off; + + if (nelems > 1) { + err = btf_repeat_fields(info, ret, nelems - 1, t->size); + if (err == 0) + ret *= nelems; + else + ret = err; + } + + return ret; +} + +static int btf_find_field_one(const struct btf *btf, + const struct btf_type *var, + const struct btf_type *var_type, + int var_idx, + u32 off, u32 expected_size, + u32 field_mask, u32 *seen_mask, + struct btf_field_info *info, int info_cnt, + u32 level) +{ + int ret, align, sz, field_type; struct btf_field_info tmp; + const struct btf_array *array; + u32 i, nelems = 1; + + /* Walk into array types to find the element type and the number of + * elements in the (flattened) array. + */ + for (i = 0; i < MAX_RESOLVE_DEPTH && btf_type_is_array(var_type); i++) { + array = btf_array(var_type); + nelems *= array->nelems; + var_type = btf_type_by_id(btf, array->type); + } + if (i == MAX_RESOLVE_DEPTH) + return -E2BIG; + if (nelems == 0) + return 0; + + field_type = btf_get_field_type(btf, var_type, + field_mask, seen_mask, &align, &sz); + /* Look into variables of struct types */ + if (!field_type && __btf_type_is_struct(var_type)) { + sz = var_type->size; + if (expected_size && expected_size != sz * nelems) + return 0; + ret = btf_find_nested_struct(btf, var_type, off, nelems, field_mask, + &info[0], info_cnt, level); + return ret; + } + + if (field_type == 0) + return 0; + if (field_type < 0) + return field_type; + + if (expected_size && expected_size != sz * nelems) + return 0; + if (off % align) + return 0; + + switch (field_type) { + case BPF_SPIN_LOCK: + case BPF_TIMER: + case BPF_WORKQUEUE: + case BPF_LIST_NODE: + case BPF_RB_NODE: + case BPF_REFCOUNT: + ret = btf_find_struct(btf, var_type, off, sz, field_type, + info_cnt ? &info[0] : &tmp); + if (ret < 0) + return ret; + break; + case BPF_KPTR_UNREF: + case BPF_KPTR_REF: + case BPF_KPTR_PERCPU: + ret = btf_find_kptr(btf, var_type, off, sz, + info_cnt ? &info[0] : &tmp); + if (ret < 0) + return ret; + break; + case BPF_LIST_HEAD: + case BPF_RB_ROOT: + ret = btf_find_graph_root(btf, var, var_type, + var_idx, off, sz, + info_cnt ? &info[0] : &tmp, + field_type); + if (ret < 0) + return ret; + break; + default: + return -EFAULT; + } + + if (ret == BTF_FIELD_IGNORE) + return 0; + if (nelems > info_cnt) + return -E2BIG; + if (nelems > 1) { + ret = btf_repeat_fields(info, 1, nelems - 1, sz); + if (ret < 0) + return ret; + } + return nelems; +} + +static int btf_find_struct_field(const struct btf *btf, + const struct btf_type *t, u32 field_mask, + struct btf_field_info *info, int info_cnt, + u32 level) +{ + int ret, idx = 0; + const struct btf_member *member; u32 i, off, seen_mask = 0; for_each_member(i, t, member) { const struct btf_type *member_type = btf_type_by_id(btf, member->type); - field_type = btf_get_field_type(__btf_name_by_offset(btf, member_type->name_off), - field_mask, &seen_mask, &align, &sz); - if (field_type == 0) - continue; - if (field_type < 0) - return field_type; - off = __btf_member_bit_offset(t, member); if (off % 8) /* valid C code cannot generate such BTF */ return -EINVAL; off /= 8; - if (off % align) - continue; - switch (field_type) { - case BPF_SPIN_LOCK: - case BPF_TIMER: - case BPF_WORKQUEUE: - case BPF_LIST_NODE: - case BPF_RB_NODE: - case BPF_REFCOUNT: - ret = btf_find_struct(btf, member_type, off, sz, field_type, - idx < info_cnt ? &info[idx] : &tmp); - if (ret < 0) - return ret; - break; - case BPF_KPTR_UNREF: - case BPF_KPTR_REF: - case BPF_KPTR_PERCPU: - ret = btf_find_kptr(btf, member_type, off, sz, - idx < info_cnt ? &info[idx] : &tmp); - if (ret < 0) - return ret; - break; - case BPF_LIST_HEAD: - case BPF_RB_ROOT: - ret = btf_find_graph_root(btf, t, member_type, - i, off, sz, - idx < info_cnt ? &info[idx] : &tmp, - field_type); - if (ret < 0) - return ret; - break; - default: - return -EFAULT; - } - - if (ret == BTF_FIELD_IGNORE) - continue; - if (idx >= info_cnt) - return -E2BIG; - ++idx; + ret = btf_find_field_one(btf, t, member_type, i, + off, 0, + field_mask, &seen_mask, + &info[idx], info_cnt - idx, level); + if (ret < 0) + return ret; + idx += ret; } return idx; } static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t, u32 field_mask, struct btf_field_info *info, - int info_cnt) + int info_cnt, u32 level) { - int ret, idx = 0, align, sz, field_type; + int ret, idx = 0; const struct btf_var_secinfo *vsi; - struct btf_field_info tmp; u32 i, off, seen_mask = 0; for_each_vsi(i, t, vsi) { const struct btf_type *var = btf_type_by_id(btf, vsi->type); const struct btf_type *var_type = btf_type_by_id(btf, var->type); - field_type = btf_get_field_type(__btf_name_by_offset(btf, var_type->name_off), - field_mask, &seen_mask, &align, &sz); - if (field_type == 0) - continue; - if (field_type < 0) - return field_type; - off = vsi->offset; - if (vsi->size != sz) - continue; - if (off % align) - continue; - - switch (field_type) { - case BPF_SPIN_LOCK: - case BPF_TIMER: - case BPF_WORKQUEUE: - case BPF_LIST_NODE: - case BPF_RB_NODE: - case BPF_REFCOUNT: - ret = btf_find_struct(btf, var_type, off, sz, field_type, - idx < info_cnt ? &info[idx] : &tmp); - if (ret < 0) - return ret; - break; - case BPF_KPTR_UNREF: - case BPF_KPTR_REF: - case BPF_KPTR_PERCPU: - ret = btf_find_kptr(btf, var_type, off, sz, - idx < info_cnt ? &info[idx] : &tmp); - if (ret < 0) - return ret; - break; - case BPF_LIST_HEAD: - case BPF_RB_ROOT: - ret = btf_find_graph_root(btf, var, var_type, - -1, off, sz, - idx < info_cnt ? &info[idx] : &tmp, - field_type); - if (ret < 0) - return ret; - break; - default: - return -EFAULT; - } - - if (ret == BTF_FIELD_IGNORE) - continue; - if (idx >= info_cnt) - return -E2BIG; - ++idx; + ret = btf_find_field_one(btf, var, var_type, -1, off, vsi->size, + field_mask, &seen_mask, + &info[idx], info_cnt - idx, + level); + if (ret < 0) + return ret; + idx += ret; } return idx; } @@ -3637,9 +3731,9 @@ static int btf_find_field(const struct btf *btf, const struct btf_type *t, int info_cnt) { if (__btf_type_is_struct(t)) - return btf_find_struct_field(btf, t, field_mask, info, info_cnt); + return btf_find_struct_field(btf, t, field_mask, info, info_cnt, 0); else if (btf_type_is_datasec(t)) - return btf_find_datasec_var(btf, t, field_mask, info, info_cnt); + return btf_find_datasec_var(btf, t, field_mask, info, info_cnt, 0); return -EINVAL; } @@ -5726,6 +5820,15 @@ static int find_kern_ctx_type_id(enum bpf_prog_type prog_type) return ctx_type->type; } +bool btf_is_projection_of(const char *pname, const char *tname) +{ + if (strcmp(pname, "__sk_buff") == 0 && strcmp(tname, "sk_buff") == 0) + return true; + if (strcmp(pname, "xdp_md") == 0 && strcmp(tname, "xdp_buff") == 0) + return true; + return false; +} + bool btf_is_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, const struct btf_type *t, enum bpf_prog_type prog_type, int arg) @@ -5788,9 +5891,7 @@ again: * int socket_filter_bpf_prog(struct __sk_buff *skb) * { // no fields of skb are ever used } */ - if (strcmp(ctx_tname, "__sk_buff") == 0 && strcmp(tname, "sk_buff") == 0) - return true; - if (strcmp(ctx_tname, "xdp_md") == 0 && strcmp(tname, "xdp_buff") == 0) + if (btf_is_projection_of(ctx_tname, tname)) return true; if (strcmp(ctx_tname, tname)) { /* bpf_user_pt_regs_t is a typedef, so resolve it to @@ -6693,7 +6794,7 @@ int btf_struct_access(struct bpf_verifier_log *log, for (i = 0; i < rec->cnt; i++) { struct btf_field *field = &rec->fields[i]; u32 offset = field->offset; - if (off < offset + btf_field_type_size(field->type) && offset < off + size) { + if (off < offset + field->size && offset < off + size) { bpf_log(log, "direct access to %s is disallowed\n", btf_field_type_name(field->type)); diff --git a/kernel/bpf/crypto.c b/kernel/bpf/crypto.c index 2bee4af91e383bb61c0090662e4eaae8fcc6fa20..94854cd9c4cc32217b94aac81b47a6ec760e9913 100644 --- a/kernel/bpf/crypto.c +++ b/kernel/bpf/crypto.c @@ -275,7 +275,7 @@ static int bpf_crypto_crypt(const struct bpf_crypto_ctx *ctx, if (__bpf_dynptr_is_rdonly(dst)) return -EINVAL; - siv_len = __bpf_dynptr_size(siv); + siv_len = siv ? __bpf_dynptr_size(siv) : 0; src_len = __bpf_dynptr_size(src); dst_len = __bpf_dynptr_size(dst); if (!src_len || !dst_len) @@ -303,36 +303,44 @@ static int bpf_crypto_crypt(const struct bpf_crypto_ctx *ctx, /** * bpf_crypto_decrypt() - Decrypt buffer using configured context and IV provided. - * @ctx: The crypto context being used. The ctx must be a trusted pointer. - * @src: bpf_dynptr to the encrypted data. Must be a trusted pointer. - * @dst: bpf_dynptr to the buffer where to store the result. Must be a trusted pointer. - * @siv: bpf_dynptr to IV data and state data to be used by decryptor. + * @ctx: The crypto context being used. The ctx must be a trusted pointer. + * @src: bpf_dynptr to the encrypted data. Must be a trusted pointer. + * @dst: bpf_dynptr to the buffer where to store the result. Must be a trusted pointer. + * @siv__nullable: bpf_dynptr to IV data and state data to be used by decryptor. May be NULL. * * Decrypts provided buffer using IV data and the crypto context. Crypto context must be configured. */ __bpf_kfunc int bpf_crypto_decrypt(struct bpf_crypto_ctx *ctx, - const struct bpf_dynptr_kern *src, - const struct bpf_dynptr_kern *dst, - const struct bpf_dynptr_kern *siv) + const struct bpf_dynptr *src, + const struct bpf_dynptr *dst, + const struct bpf_dynptr *siv__nullable) { - return bpf_crypto_crypt(ctx, src, dst, siv, true); + const struct bpf_dynptr_kern *src_kern = (struct bpf_dynptr_kern *)src; + const struct bpf_dynptr_kern *dst_kern = (struct bpf_dynptr_kern *)dst; + const struct bpf_dynptr_kern *siv_kern = (struct bpf_dynptr_kern *)siv__nullable; + + return bpf_crypto_crypt(ctx, src_kern, dst_kern, siv_kern, true); } /** * bpf_crypto_encrypt() - Encrypt buffer using configured context and IV provided. - * @ctx: The crypto context being used. The ctx must be a trusted pointer. - * @src: bpf_dynptr to the plain data. Must be a trusted pointer. - * @dst: bpf_dynptr to buffer where to store the result. Must be a trusted pointer. - * @siv: bpf_dynptr to IV data and state data to be used by decryptor. + * @ctx: The crypto context being used. The ctx must be a trusted pointer. + * @src: bpf_dynptr to the plain data. Must be a trusted pointer. + * @dst: bpf_dynptr to the buffer where to store the result. Must be a trusted pointer. + * @siv__nullable: bpf_dynptr to IV data and state data to be used by decryptor. May be NULL. * * Encrypts provided buffer using IV data and the crypto context. Crypto context must be configured. */ __bpf_kfunc int bpf_crypto_encrypt(struct bpf_crypto_ctx *ctx, - const struct bpf_dynptr_kern *src, - const struct bpf_dynptr_kern *dst, - const struct bpf_dynptr_kern *siv) + const struct bpf_dynptr *src, + const struct bpf_dynptr *dst, + const struct bpf_dynptr *siv__nullable) { - return bpf_crypto_crypt(ctx, src, dst, siv, false); + const struct bpf_dynptr_kern *src_kern = (struct bpf_dynptr_kern *)src; + const struct bpf_dynptr_kern *dst_kern = (struct bpf_dynptr_kern *)dst; + const struct bpf_dynptr_kern *siv_kern = (struct bpf_dynptr_kern *)siv__nullable; + + return bpf_crypto_crypt(ctx, src_kern, dst_kern, siv_kern, false); } __bpf_kfunc_end_defs(); diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 4e2cdbb5629f22fc1464c5cba296688c45fc8d26..7f3b34452243c83b6e113b5a5d831fc088854f54 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -760,9 +760,6 @@ int dev_map_redirect_multi(struct net_device *dev, struct sk_buff *skb, for (i = 0; i < dtab->n_buckets; i++) { head = dev_map_index_hash(dtab, i); hlist_for_each_entry_safe(dst, next, head, index_hlist) { - if (!dst) - continue; - if (is_ifindex_excluded(excluded_devices, num_excluded, dst->dev->ifindex)) continue; diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 2a69a9a36c0fcc3284d899060bcd8fbbec83e04f..229396172026a4ecc035c5fa7fffcbb9ddda24a3 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2433,7 +2433,7 @@ __bpf_kfunc struct task_struct *bpf_task_from_pid(s32 pid) /** * bpf_dynptr_slice() - Obtain a read-only pointer to the dynptr data. - * @ptr: The dynptr whose data slice to retrieve + * @p: The dynptr whose data slice to retrieve * @offset: Offset into the dynptr * @buffer__opt: User-provided buffer to copy contents into. May be NULL * @buffer__szk: Size (in bytes) of the buffer if present. This is the @@ -2459,9 +2459,10 @@ __bpf_kfunc struct task_struct *bpf_task_from_pid(s32 pid) * provided buffer, with its contents containing the data, if unable to obtain * direct pointer) */ -__bpf_kfunc void *bpf_dynptr_slice(const struct bpf_dynptr_kern *ptr, u32 offset, +__bpf_kfunc void *bpf_dynptr_slice(const struct bpf_dynptr *p, u32 offset, void *buffer__opt, u32 buffer__szk) { + const struct bpf_dynptr_kern *ptr = (struct bpf_dynptr_kern *)p; enum bpf_dynptr_type type; u32 len = buffer__szk; int err; @@ -2503,7 +2504,7 @@ __bpf_kfunc void *bpf_dynptr_slice(const struct bpf_dynptr_kern *ptr, u32 offset /** * bpf_dynptr_slice_rdwr() - Obtain a writable pointer to the dynptr data. - * @ptr: The dynptr whose data slice to retrieve + * @p: The dynptr whose data slice to retrieve * @offset: Offset into the dynptr * @buffer__opt: User-provided buffer to copy contents into. May be NULL * @buffer__szk: Size (in bytes) of the buffer if present. This is the @@ -2543,9 +2544,11 @@ __bpf_kfunc void *bpf_dynptr_slice(const struct bpf_dynptr_kern *ptr, u32 offset * provided buffer, with its contents containing the data, if unable to obtain * direct pointer) */ -__bpf_kfunc void *bpf_dynptr_slice_rdwr(const struct bpf_dynptr_kern *ptr, u32 offset, +__bpf_kfunc void *bpf_dynptr_slice_rdwr(const struct bpf_dynptr *p, u32 offset, void *buffer__opt, u32 buffer__szk) { + const struct bpf_dynptr_kern *ptr = (struct bpf_dynptr_kern *)p; + if (!ptr->data || __bpf_dynptr_is_rdonly(ptr)) return NULL; @@ -2571,11 +2574,12 @@ __bpf_kfunc void *bpf_dynptr_slice_rdwr(const struct bpf_dynptr_kern *ptr, u32 o * will be copied out into the buffer and the user will need to call * bpf_dynptr_write() to commit changes. */ - return bpf_dynptr_slice(ptr, offset, buffer__opt, buffer__szk); + return bpf_dynptr_slice(p, offset, buffer__opt, buffer__szk); } -__bpf_kfunc int bpf_dynptr_adjust(struct bpf_dynptr_kern *ptr, u32 start, u32 end) +__bpf_kfunc int bpf_dynptr_adjust(const struct bpf_dynptr *p, u32 start, u32 end) { + struct bpf_dynptr_kern *ptr = (struct bpf_dynptr_kern *)p; u32 size; if (!ptr->data || start > end) @@ -2592,36 +2596,45 @@ __bpf_kfunc int bpf_dynptr_adjust(struct bpf_dynptr_kern *ptr, u32 start, u32 en return 0; } -__bpf_kfunc bool bpf_dynptr_is_null(struct bpf_dynptr_kern *ptr) +__bpf_kfunc bool bpf_dynptr_is_null(const struct bpf_dynptr *p) { + struct bpf_dynptr_kern *ptr = (struct bpf_dynptr_kern *)p; + return !ptr->data; } -__bpf_kfunc bool bpf_dynptr_is_rdonly(struct bpf_dynptr_kern *ptr) +__bpf_kfunc bool bpf_dynptr_is_rdonly(const struct bpf_dynptr *p) { + struct bpf_dynptr_kern *ptr = (struct bpf_dynptr_kern *)p; + if (!ptr->data) return false; return __bpf_dynptr_is_rdonly(ptr); } -__bpf_kfunc __u32 bpf_dynptr_size(const struct bpf_dynptr_kern *ptr) +__bpf_kfunc __u32 bpf_dynptr_size(const struct bpf_dynptr *p) { + struct bpf_dynptr_kern *ptr = (struct bpf_dynptr_kern *)p; + if (!ptr->data) return -EINVAL; return __bpf_dynptr_size(ptr); } -__bpf_kfunc int bpf_dynptr_clone(struct bpf_dynptr_kern *ptr, - struct bpf_dynptr_kern *clone__uninit) +__bpf_kfunc int bpf_dynptr_clone(const struct bpf_dynptr *p, + struct bpf_dynptr *clone__uninit) { + struct bpf_dynptr_kern *clone = (struct bpf_dynptr_kern *)clone__uninit; + struct bpf_dynptr_kern *ptr = (struct bpf_dynptr_kern *)p; + if (!ptr->data) { - bpf_dynptr_set_null(clone__uninit); + bpf_dynptr_set_null(clone); return -EINVAL; } - *clone__uninit = *ptr; + *clone = *ptr; return 0; } @@ -2744,6 +2757,122 @@ __bpf_kfunc void bpf_preempt_enable(void) preempt_enable(); } +struct bpf_iter_bits { + __u64 __opaque[2]; +} __aligned(8); + +struct bpf_iter_bits_kern { + union { + unsigned long *bits; + unsigned long bits_copy; + }; + u32 nr_bits; + int bit; +} __aligned(8); + +/** + * bpf_iter_bits_new() - Initialize a new bits iterator for a given memory area + * @it: The new bpf_iter_bits to be created + * @unsafe_ptr__ign: A pointer pointing to a memory area to be iterated over + * @nr_words: The size of the specified memory area, measured in 8-byte units. + * Due to the limitation of memalloc, it can't be greater than 512. + * + * This function initializes a new bpf_iter_bits structure for iterating over + * a memory area which is specified by the @unsafe_ptr__ign and @nr_words. It + * copies the data of the memory area to the newly created bpf_iter_bits @it for + * subsequent iteration operations. + * + * On success, 0 is returned. On failure, ERR is returned. + */ +__bpf_kfunc int +bpf_iter_bits_new(struct bpf_iter_bits *it, const u64 *unsafe_ptr__ign, u32 nr_words) +{ + struct bpf_iter_bits_kern *kit = (void *)it; + u32 nr_bytes = nr_words * sizeof(u64); + u32 nr_bits = BYTES_TO_BITS(nr_bytes); + int err; + + BUILD_BUG_ON(sizeof(struct bpf_iter_bits_kern) != sizeof(struct bpf_iter_bits)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_bits_kern) != + __alignof__(struct bpf_iter_bits)); + + kit->nr_bits = 0; + kit->bits_copy = 0; + kit->bit = -1; + + if (!unsafe_ptr__ign || !nr_words) + return -EINVAL; + + /* Optimization for u64 mask */ + if (nr_bits == 64) { + err = bpf_probe_read_kernel_common(&kit->bits_copy, nr_bytes, unsafe_ptr__ign); + if (err) + return -EFAULT; + + kit->nr_bits = nr_bits; + return 0; + } + + /* Fallback to memalloc */ + kit->bits = bpf_mem_alloc(&bpf_global_ma, nr_bytes); + if (!kit->bits) + return -ENOMEM; + + err = bpf_probe_read_kernel_common(kit->bits, nr_bytes, unsafe_ptr__ign); + if (err) { + bpf_mem_free(&bpf_global_ma, kit->bits); + return err; + } + + kit->nr_bits = nr_bits; + return 0; +} + +/** + * bpf_iter_bits_next() - Get the next bit in a bpf_iter_bits + * @it: The bpf_iter_bits to be checked + * + * This function returns a pointer to a number representing the value of the + * next bit in the bits. + * + * If there are no further bits available, it returns NULL. + */ +__bpf_kfunc int *bpf_iter_bits_next(struct bpf_iter_bits *it) +{ + struct bpf_iter_bits_kern *kit = (void *)it; + u32 nr_bits = kit->nr_bits; + const unsigned long *bits; + int bit; + + if (nr_bits == 0) + return NULL; + + bits = nr_bits == 64 ? &kit->bits_copy : kit->bits; + bit = find_next_bit(bits, nr_bits, kit->bit + 1); + if (bit >= nr_bits) { + kit->nr_bits = 0; + return NULL; + } + + kit->bit = bit; + return &kit->bit; +} + +/** + * bpf_iter_bits_destroy() - Destroy a bpf_iter_bits + * @it: The bpf_iter_bits to be destroyed + * + * Destroy the resource associated with the bpf_iter_bits. + */ +__bpf_kfunc void bpf_iter_bits_destroy(struct bpf_iter_bits *it) +{ + struct bpf_iter_bits_kern *kit = (void *)it; + + if (kit->nr_bits <= 64) + return; + bpf_mem_free(&bpf_global_ma, kit->bits); +} + __bpf_kfunc_end_defs(); BTF_KFUNCS_START(generic_btf_ids) @@ -2826,6 +2955,9 @@ BTF_ID_FLAGS(func, bpf_wq_set_callback_impl) BTF_ID_FLAGS(func, bpf_wq_start) BTF_ID_FLAGS(func, bpf_preempt_disable) BTF_ID_FLAGS(func, bpf_preempt_enable) +BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) +BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_KFUNCS_END(common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = { @@ -2867,7 +2999,9 @@ late_initcall(kfunc_init); */ const void *__bpf_dynptr_data(const struct bpf_dynptr_kern *ptr, u32 len) { - return bpf_dynptr_slice(ptr, 0, NULL, len); + const struct bpf_dynptr *p = (struct bpf_dynptr *)ptr; + + return bpf_dynptr_slice(p, 0, NULL, len); } /* Get a pointer to dynptr data up to len bytes for read write access. If diff --git a/kernel/bpf/log.c b/kernel/bpf/log.c index 4bd8f17a9f240114f5ca592c1b32d181c12b97b3..3f4ae92e549ffae411dfe620ecfa8bd9cec2a450 100644 --- a/kernel/bpf/log.c +++ b/kernel/bpf/log.c @@ -708,7 +708,9 @@ static void print_reg_state(struct bpf_verifier_env *env, verbose(env, "%s", btf_type_name(reg->btf, reg->btf_id)); verbose(env, "("); if (reg->id) - verbose_a("id=%d", reg->id); + verbose_a("id=%d", reg->id & ~BPF_ADD_CONST); + if (reg->id & BPF_ADD_CONST) + verbose(env, "%+d", reg->off); if (reg->ref_obj_id) verbose_a("ref_obj_id=%d", reg->ref_obj_id); if (type_is_non_owning_ref(reg->type)) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 2222c3ff88e7fd639390112de66581b934584457..869265852d51571a3c60c62f50c1514f22b2ae15 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -2998,6 +2998,7 @@ static int bpf_obj_get(const union bpf_attr *attr) void bpf_link_init(struct bpf_link *link, enum bpf_link_type type, const struct bpf_link_ops *ops, struct bpf_prog *prog) { + WARN_ON(ops->dealloc && ops->dealloc_deferred); atomic64_set(&link->refcnt, 1); link->type = type; link->id = 0; @@ -3056,16 +3057,17 @@ static void bpf_link_defer_dealloc_mult_rcu_gp(struct rcu_head *rcu) /* bpf_link_free is guaranteed to be called from process context */ static void bpf_link_free(struct bpf_link *link) { + const struct bpf_link_ops *ops = link->ops; bool sleepable = false; bpf_link_free_id(link->id); if (link->prog) { sleepable = link->prog->sleepable; /* detach BPF program, clean up used resources */ - link->ops->release(link); + ops->release(link); bpf_prog_put(link->prog); } - if (link->ops->dealloc_deferred) { + if (ops->dealloc_deferred) { /* schedule BPF link deallocation; if underlying BPF program * is sleepable, we need to first wait for RCU tasks trace * sync, then go through "classic" RCU grace period @@ -3074,9 +3076,8 @@ static void bpf_link_free(struct bpf_link *link) call_rcu_tasks_trace(&link->rcu, bpf_link_defer_dealloc_mult_rcu_gp); else call_rcu(&link->rcu, bpf_link_defer_dealloc_rcu_gp); - } - if (link->ops->dealloc) - link->ops->dealloc(link); + } else if (ops->dealloc) + ops->dealloc(link); } static void bpf_link_put_deferred(struct work_struct *work) @@ -3150,6 +3151,13 @@ static void bpf_link_show_fdinfo(struct seq_file *m, struct file *filp) } #endif +static __poll_t bpf_link_poll(struct file *file, struct poll_table_struct *pts) +{ + struct bpf_link *link = file->private_data; + + return link->ops->poll(file, pts); +} + static const struct file_operations bpf_link_fops = { #ifdef CONFIG_PROC_FS .show_fdinfo = bpf_link_show_fdinfo, @@ -3159,6 +3167,16 @@ static const struct file_operations bpf_link_fops = { .write = bpf_dummy_write, }; +static const struct file_operations bpf_link_fops_poll = { +#ifdef CONFIG_PROC_FS + .show_fdinfo = bpf_link_show_fdinfo, +#endif + .release = bpf_link_release, + .read = bpf_dummy_read, + .write = bpf_dummy_write, + .poll = bpf_link_poll, +}; + static int bpf_link_alloc_id(struct bpf_link *link) { int id; @@ -3201,7 +3219,9 @@ int bpf_link_prime(struct bpf_link *link, struct bpf_link_primer *primer) return id; } - file = anon_inode_getfile("bpf_link", &bpf_link_fops, link, O_CLOEXEC); + file = anon_inode_getfile("bpf_link", + link->ops->poll ? &bpf_link_fops_poll : &bpf_link_fops, + link, O_CLOEXEC); if (IS_ERR(file)) { bpf_link_free_id(id); put_unused_fd(fd); @@ -3229,7 +3249,9 @@ int bpf_link_settle(struct bpf_link_primer *primer) int bpf_link_new_fd(struct bpf_link *link) { - return anon_inode_getfd("bpf-link", &bpf_link_fops, link, O_CLOEXEC); + return anon_inode_getfd("bpf-link", + link->ops->poll ? &bpf_link_fops_poll : &bpf_link_fops, + link, O_CLOEXEC); } struct bpf_link *bpf_link_get_from_fd(u32 ufd) @@ -3239,7 +3261,7 @@ struct bpf_link *bpf_link_get_from_fd(u32 ufd) if (!f.file) return ERR_PTR(-EBADF); - if (f.file->f_op != &bpf_link_fops) { + if (f.file->f_op != &bpf_link_fops && f.file->f_op != &bpf_link_fops_poll) { fdput(f); return ERR_PTR(-EINVAL); } @@ -4971,7 +4993,7 @@ static int bpf_obj_get_info_by_fd(const union bpf_attr *attr, uattr); else if (f.file->f_op == &btf_fops) err = bpf_btf_get_info_by_fd(f.file, f.file->private_data, attr, uattr); - else if (f.file->f_op == &bpf_link_fops) + else if (f.file->f_op == &bpf_link_fops || f.file->f_op == &bpf_link_fops_poll) err = bpf_link_get_info_by_fd(f.file, f.file->private_data, attr, uattr); else @@ -5106,7 +5128,7 @@ static int bpf_task_fd_query(const union bpf_attr *attr, if (!file) return -EBADF; - if (file->f_op == &bpf_link_fops) { + if (file->f_op == &bpf_link_fops || file->f_op == &bpf_link_fops_poll) { struct bpf_link *link = file->private_data; if (link->ops == &bpf_raw_tp_link_lops) { @@ -5416,10 +5438,11 @@ static int link_detach(union bpf_attr *attr) return ret; } -static struct bpf_link *bpf_link_inc_not_zero(struct bpf_link *link) +struct bpf_link *bpf_link_inc_not_zero(struct bpf_link *link) { return atomic64_fetch_add_unless(&link->refcnt, 1, 0) ? link : ERR_PTR(-ENOENT); } +EXPORT_SYMBOL(bpf_link_inc_not_zero); struct bpf_link *bpf_link_by_id(u32 id) { diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 77da1f438beccec71f59a4b129323a1064389e12..dcbbf5f64c5de79eb541d346a72b3b6839565c30 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -3991,7 +3991,7 @@ static bool idset_contains(struct bpf_idset *s, u32 id) u32 i; for (i = 0; i < s->count; ++i) - if (s->ids[i] == id) + if (s->ids[i] == (id & ~BPF_ADD_CONST)) return true; return false; @@ -4001,7 +4001,7 @@ static int idset_push(struct bpf_idset *s, u32 id) { if (WARN_ON_ONCE(s->count >= ARRAY_SIZE(s->ids))) return -EFAULT; - s->ids[s->count++] = id; + s->ids[s->count++] = id & ~BPF_ADD_CONST; return 0; } @@ -4438,8 +4438,20 @@ static bool __is_pointer_value(bool allow_ptr_leaks, static void assign_scalar_id_before_mov(struct bpf_verifier_env *env, struct bpf_reg_state *src_reg) { - if (src_reg->type == SCALAR_VALUE && !src_reg->id && - !tnum_is_const(src_reg->var_off)) + if (src_reg->type != SCALAR_VALUE) + return; + + if (src_reg->id & BPF_ADD_CONST) { + /* + * The verifier is processing rX = rY insn and + * rY->id has special linked register already. + * Cleared it, since multiple rX += const are not supported. + */ + src_reg->id = 0; + src_reg->off = 0; + } + + if (!src_reg->id && !tnum_is_const(src_reg->var_off)) /* Ensure that src_reg has a valid ID that will be copied to * dst_reg and then will be used by find_equal_scalars() to * propagate min/max range. @@ -5448,7 +5460,7 @@ static int check_map_access(struct bpf_verifier_env *env, u32 regno, * this program. To check that [x1, x2) overlaps with [y1, y2), * it is sufficient to check x1 < y2 && y1 < x2. */ - if (reg->smin_value + off < p + btf_field_type_size(field->type) && + if (reg->smin_value + off < p + field->size && p < reg->umax_value + off + size) { switch (field->type) { case BPF_KPTR_UNREF: @@ -8882,7 +8894,8 @@ static bool may_update_sockmap(struct bpf_verifier_env *env, int func_id) enum bpf_attach_type eatype = env->prog->expected_attach_type; enum bpf_prog_type type = resolve_prog_type(env->prog); - if (func_id != BPF_FUNC_map_update_elem) + if (func_id != BPF_FUNC_map_update_elem && + func_id != BPF_FUNC_map_delete_elem) return false; /* It's not possible to get access to a locked struct sock in these @@ -8893,6 +8906,11 @@ static bool may_update_sockmap(struct bpf_verifier_env *env, int func_id) if (eatype == BPF_TRACE_ITER) return true; break; + case BPF_PROG_TYPE_SOCK_OPS: + /* map_update allowed only via dedicated helpers with event type checks */ + if (func_id == BPF_FUNC_map_delete_elem) + return true; + break; case BPF_PROG_TYPE_SOCKET_FILTER: case BPF_PROG_TYPE_SCHED_CLS: case BPF_PROG_TYPE_SCHED_ACT: @@ -8988,7 +9006,6 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env, case BPF_MAP_TYPE_SOCKMAP: if (func_id != BPF_FUNC_sk_redirect_map && func_id != BPF_FUNC_sock_map_update && - func_id != BPF_FUNC_map_delete_elem && func_id != BPF_FUNC_msg_redirect_map && func_id != BPF_FUNC_sk_select_reuseport && func_id != BPF_FUNC_map_lookup_elem && @@ -8998,7 +9015,6 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env, case BPF_MAP_TYPE_SOCKHASH: if (func_id != BPF_FUNC_sk_redirect_hash && func_id != BPF_FUNC_sock_hash_update && - func_id != BPF_FUNC_map_delete_elem && func_id != BPF_FUNC_msg_redirect_hash && func_id != BPF_FUNC_sk_select_reuseport && func_id != BPF_FUNC_map_lookup_elem && @@ -10910,7 +10926,7 @@ enum { }; BTF_ID_LIST(kf_arg_btf_ids) -BTF_ID(struct, bpf_dynptr_kern) +BTF_ID(struct, bpf_dynptr) BTF_ID(struct, bpf_list_head) BTF_ID(struct, bpf_list_node) BTF_ID(struct, bpf_rb_root) @@ -11124,7 +11140,11 @@ BTF_ID(func, bpf_iter_css_task_new) #else BTF_ID_UNUSED #endif +#ifdef CONFIG_BPF_EVENTS BTF_ID(func, bpf_session_cookie) +#else +BTF_ID_UNUSED +#endif static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { @@ -11179,6 +11199,9 @@ get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, if (btf_is_prog_ctx_type(&env->log, meta->btf, t, resolve_prog_type(env->prog), argno)) return KF_ARG_PTR_TO_CTX; + if (is_kfunc_arg_nullable(meta->btf, &args[argno]) && register_is_null(reg)) + return KF_ARG_PTR_TO_NULL; + if (is_kfunc_arg_alloc_obj(meta->btf, &args[argno])) return KF_ARG_PTR_TO_ALLOC_BTF_ID; @@ -11224,9 +11247,6 @@ get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, if (is_kfunc_arg_callback(env, meta->btf, &args[argno])) return KF_ARG_PTR_TO_CALLBACK; - if (is_kfunc_arg_nullable(meta->btf, &args[argno]) && register_is_null(reg)) - return KF_ARG_PTR_TO_NULL; - if (argno + 1 < nargs && (is_kfunc_arg_mem_size(meta->btf, &args[argno + 1], ®s[regno + 1]) || is_kfunc_arg_const_mem_size(meta->btf, &args[argno + 1], ®s[regno + 1]))) @@ -11257,6 +11277,8 @@ static int process_kf_arg_ptr_to_btf_id(struct bpf_verifier_env *env, bool strict_type_match = false; const struct btf *reg_btf; const char *reg_ref_tname; + bool taking_projection; + bool struct_same; u32 reg_ref_id; if (base_type(reg->type) == PTR_TO_BTF_ID) { @@ -11300,7 +11322,13 @@ static int process_kf_arg_ptr_to_btf_id(struct bpf_verifier_env *env, reg_ref_t = btf_type_skip_modifiers(reg_btf, reg_ref_id, ®_ref_id); reg_ref_tname = btf_name_by_offset(reg_btf, reg_ref_t->name_off); - if (!btf_struct_ids_match(&env->log, reg_btf, reg_ref_id, reg->off, meta->btf, ref_id, strict_type_match)) { + struct_same = btf_struct_ids_match(&env->log, reg_btf, reg_ref_id, reg->off, meta->btf, ref_id, strict_type_match); + /* If kfunc is accepting a projection type (ie. __sk_buff), it cannot + * actually use it -- it must cast to the underlying type. So we allow + * caller to pass in the underlying type. + */ + taking_projection = btf_is_projection_of(ref_tname, reg_ref_tname); + if (!taking_projection && !struct_same) { verbose(env, "kernel function %s args#%d expected pointer to %s %s but R%d has a pointer to %s %s\n", meta->func_name, argno, btf_type_str(ref_t), ref_tname, argno + 1, btf_type_str(reg_ref_t), reg_ref_tname); @@ -11640,7 +11668,7 @@ __process_kf_arg_ptr_to_graph_node(struct bpf_verifier_env *env, node_off = reg->off + reg->var_off.value; field = reg_find_field_offset(reg, node_off, node_field_type); - if (!field || field->offset != node_off) { + if (!field) { verbose(env, "%s not found at offset=%u\n", node_type_name, node_off); return -EINVAL; } @@ -14026,6 +14054,7 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, struct bpf_func_state *state = vstate->frame[vstate->curframe]; struct bpf_reg_state *regs = state->regs, *dst_reg, *src_reg; struct bpf_reg_state *ptr_reg = NULL, off_reg = {0}; + bool alu32 = (BPF_CLASS(insn->code) != BPF_ALU64); u8 opcode = BPF_OP(insn->code); int err; @@ -14048,11 +14077,7 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, if (dst_reg->type != SCALAR_VALUE) ptr_reg = dst_reg; - else - /* Make sure ID is cleared otherwise dst_reg min/max could be - * incorrectly propagated into other registers by find_equal_scalars() - */ - dst_reg->id = 0; + if (BPF_SRC(insn->code) == BPF_X) { src_reg = ®s[insn->src_reg]; if (src_reg->type != SCALAR_VALUE) { @@ -14116,7 +14141,43 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, verbose(env, "verifier internal error: no src_reg\n"); return -EINVAL; } - return adjust_scalar_min_max_vals(env, insn, dst_reg, *src_reg); + err = adjust_scalar_min_max_vals(env, insn, dst_reg, *src_reg); + if (err) + return err; + /* + * Compilers can generate the code + * r1 = r2 + * r1 += 0x1 + * if r2 < 1000 goto ... + * use r1 in memory access + * So remember constant delta between r2 and r1 and update r1 after + * 'if' condition. + */ + if (env->bpf_capable && BPF_OP(insn->code) == BPF_ADD && + dst_reg->id && is_reg_const(src_reg, alu32)) { + u64 val = reg_const_value(src_reg, alu32); + + if ((dst_reg->id & BPF_ADD_CONST) || + /* prevent overflow in find_equal_scalars() later */ + val > (u32)S32_MAX) { + /* + * If the register already went through rX += val + * we cannot accumulate another val into rx->off. + */ + dst_reg->off = 0; + dst_reg->id = 0; + } else { + dst_reg->id |= BPF_ADD_CONST; + dst_reg->off = val; + } + } else { + /* + * Make sure ID is cleared otherwise dst_reg min/max could be + * incorrectly propagated into other registers by find_equal_scalars() + */ + dst_reg->id = 0; + } + return 0; } /* check validity of 32-bit and 64-bit arithmetic operations */ @@ -15088,12 +15149,36 @@ static bool try_match_pkt_pointers(const struct bpf_insn *insn, static void find_equal_scalars(struct bpf_verifier_state *vstate, struct bpf_reg_state *known_reg) { + struct bpf_reg_state fake_reg; struct bpf_func_state *state; struct bpf_reg_state *reg; bpf_for_each_reg_in_vstate(vstate, state, reg, ({ - if (reg->type == SCALAR_VALUE && reg->id == known_reg->id) + if (reg->type != SCALAR_VALUE || reg == known_reg) + continue; + if ((reg->id & ~BPF_ADD_CONST) != (known_reg->id & ~BPF_ADD_CONST)) + continue; + if ((!(reg->id & BPF_ADD_CONST) && !(known_reg->id & BPF_ADD_CONST)) || + reg->off == known_reg->off) { + copy_register_state(reg, known_reg); + } else { + s32 saved_off = reg->off; + + fake_reg.type = SCALAR_VALUE; + __mark_reg_known(&fake_reg, (s32)reg->off - (s32)known_reg->off); + + /* reg = known_reg; reg += delta */ copy_register_state(reg, known_reg); + /* + * Must preserve off, id and add_const flag, + * otherwise another find_equal_scalars() will be incorrect. + */ + reg->off = saved_off; + + scalar32_min_max_add(reg, &fake_reg); + scalar_min_max_add(reg, &fake_reg); + reg->var_off = tnum_add(reg->var_off, fake_reg.var_off); + } })); } @@ -16722,6 +16807,10 @@ static bool regsafe(struct bpf_verifier_env *env, struct bpf_reg_state *rold, } if (!rold->precise && exact == NOT_EXACT) return true; + if ((rold->id & BPF_ADD_CONST) != (rcur->id & BPF_ADD_CONST)) + return false; + if ((rold->id & BPF_ADD_CONST) && (rold->off != rcur->off)) + return false; /* Why check_ids() for scalar registers? * * Consider the following BPF code: diff --git a/kernel/dma/map_benchmark.c b/kernel/dma/map_benchmark.c index 02205ab53b7e93b2122fcfd3a4ab811fa2d97166..4950e0b622b1f3af9e985bb67624401193b8b1db 100644 --- a/kernel/dma/map_benchmark.c +++ b/kernel/dma/map_benchmark.c @@ -101,7 +101,6 @@ static int do_map_benchmark(struct map_benchmark_data *map) struct task_struct **tsk; int threads = map->bparam.threads; int node = map->bparam.node; - const cpumask_t *cpu_mask = cpumask_of_node(node); u64 loops; int ret = 0; int i; @@ -118,11 +117,13 @@ static int do_map_benchmark(struct map_benchmark_data *map) if (IS_ERR(tsk[i])) { pr_err("create dma_map thread failed\n"); ret = PTR_ERR(tsk[i]); + while (--i >= 0) + kthread_stop(tsk[i]); goto out; } if (node != NUMA_NO_NODE) - kthread_bind_mask(tsk[i], cpu_mask); + kthread_bind_mask(tsk[i], cpumask_of_node(node)); } /* clear the old value in the previous benchmark */ @@ -139,13 +140,17 @@ static int do_map_benchmark(struct map_benchmark_data *map) msleep_interruptible(map->bparam.seconds * 1000); - /* wait for the completion of benchmark threads */ + /* wait for the completion of all started benchmark threads */ for (i = 0; i < threads; i++) { - ret = kthread_stop(tsk[i]); - if (ret) - goto out; + int kthread_ret = kthread_stop_put(tsk[i]); + + if (kthread_ret) + ret = kthread_ret; } + if (ret) + goto out; + loops = atomic64_read(&map->loops); if (likely(loops > 0)) { u64 map_variance, unmap_variance; @@ -170,8 +175,6 @@ static int do_map_benchmark(struct map_benchmark_data *map) } out: - for (i = 0; i < threads; i++) - put_task_struct(tsk[i]); put_device(map->dev); kfree(tsk); return ret; @@ -208,7 +211,8 @@ static long map_benchmark_ioctl(struct file *file, unsigned int cmd, } if (map->bparam.node != NUMA_NO_NODE && - !node_possible(map->bparam.node)) { + (map->bparam.node < 0 || map->bparam.node >= MAX_NUMNODES || + !node_possible(map->bparam.node))) { pr_err("invalid numa node\n"); return -EINVAL; } @@ -252,6 +256,9 @@ static long map_benchmark_ioctl(struct file *file, unsigned int cmd, * dma_mask changed by benchmark */ dma_set_mask(map->dev, old_dma_mask); + + if (ret) + return ret; break; default: return -EINVAL; diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh index 6d443ea22bb7325beb1f7b6e2807497cbdd35cac..383fd43ac6122247310720afc391ab2610bd9847 100755 --- a/kernel/gen_kheaders.sh +++ b/kernel/gen_kheaders.sh @@ -14,7 +14,12 @@ include/ arch/$SRCARCH/include/ " -type cpio > /dev/null +if ! command -v cpio >/dev/null; then + echo >&2 "***" + echo >&2 "*** 'cpio' could not be found." + echo >&2 "***" + exit 1 +fi # Support incremental builds by skipping archive generation # if timestamps of files being archived are not changed. @@ -84,7 +89,7 @@ find $cpio_dir -type f -print0 | # Create archive and try to normalize metadata for reproducibility. tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \ - --owner=0 --group=0 --sort=name --numeric-owner \ + --owner=0 --group=0 --sort=name --numeric-owner --mode=u=rw,go=r,a+X \ -I $XZ -cf $tarfile -C $cpio_dir/ . > /dev/null echo $headers_md5 > kernel/kheaders.md5 diff --git a/kernel/power/swap.c b/kernel/power/swap.c index d9abb7ab031dd5f8dc6950ba26ac0a4d0bb4d7da..753b8dd42a59a5beddf5e13cece6d8ecc1846157 100644 --- a/kernel/power/swap.c +++ b/kernel/power/swap.c @@ -1595,7 +1595,7 @@ int swsusp_check(bool exclusive) put: if (error) - fput(hib_resume_bdev_file); + bdev_fput(hib_resume_bdev_file); else pr_debug("Image signature found, resuming\n"); } else { diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index f5154c051d2c105dea6d42735872a34a8d41150c..4b3fda4562999bbbc01f7432aac8d6ed1e2fc1c4 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1369,8 +1369,8 @@ __bpf_kfunc void bpf_key_put(struct bpf_key *bkey) #ifdef CONFIG_SYSTEM_DATA_VERIFICATION /** * bpf_verify_pkcs7_signature - verify a PKCS#7 signature - * @data_ptr: data to verify - * @sig_ptr: signature of the data + * @data_p: data to verify + * @sig_p: signature of the data * @trusted_keyring: keyring with keys trusted for signature verification * * Verify the PKCS#7 signature *sig_ptr* against the supplied *data_ptr* @@ -1378,10 +1378,12 @@ __bpf_kfunc void bpf_key_put(struct bpf_key *bkey) * * Return: 0 on success, a negative value on error. */ -__bpf_kfunc int bpf_verify_pkcs7_signature(struct bpf_dynptr_kern *data_ptr, - struct bpf_dynptr_kern *sig_ptr, +__bpf_kfunc int bpf_verify_pkcs7_signature(struct bpf_dynptr *data_p, + struct bpf_dynptr *sig_p, struct bpf_key *trusted_keyring) { + struct bpf_dynptr_kern *data_ptr = (struct bpf_dynptr_kern *)data_p; + struct bpf_dynptr_kern *sig_ptr = (struct bpf_dynptr_kern *)sig_p; const void *data, *sig; u32 data_len, sig_len; int ret; @@ -1444,7 +1446,7 @@ __bpf_kfunc_start_defs(); * bpf_get_file_xattr - get xattr of a file * @file: file to get xattr from * @name__str: name of the xattr - * @value_ptr: output buffer of the xattr value + * @value_p: output buffer of the xattr value * * Get xattr *name__str* of *file* and store the output in *value_ptr*. * @@ -1453,8 +1455,9 @@ __bpf_kfunc_start_defs(); * Return: 0 on success, a negative value on error. */ __bpf_kfunc int bpf_get_file_xattr(struct file *file, const char *name__str, - struct bpf_dynptr_kern *value_ptr) + struct bpf_dynptr *value_p) { + struct bpf_dynptr_kern *value_ptr = (struct bpf_dynptr_kern *)value_p; struct dentry *dentry; u32 value_len; void *value; @@ -3295,7 +3298,7 @@ static int uprobe_prog_run(struct bpf_uprobe *uprobe, struct bpf_run_ctx *old_run_ctx; int err = 0; - if (link->task && current != link->task) + if (link->task && current->mm != link->task->mm) return 0; if (sleepable) @@ -3396,8 +3399,9 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr upath = u64_to_user_ptr(attr->link_create.uprobe_multi.path); uoffsets = u64_to_user_ptr(attr->link_create.uprobe_multi.offsets); cnt = attr->link_create.uprobe_multi.cnt; + pid = attr->link_create.uprobe_multi.pid; - if (!upath || !uoffsets || !cnt) + if (!upath || !uoffsets || !cnt || pid < 0) return -EINVAL; if (cnt > MAX_UPROBE_MULTI_CNT) return -E2BIG; @@ -3421,11 +3425,8 @@ int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr goto error_path_put; } - pid = attr->link_create.uprobe_multi.pid; if (pid) { - rcu_read_lock(); - task = get_pid_task(find_vpid(pid), PIDTYPE_PID); - rcu_read_unlock(); + task = get_pid_task(find_vpid(pid), PIDTYPE_TGID); if (!task) { err = -ESRCH; goto error_path_put; @@ -3519,7 +3520,6 @@ static u64 bpf_uprobe_multi_entry_ip(struct bpf_run_ctx *ctx) } #endif /* CONFIG_UPROBES */ -#ifdef CONFIG_FPROBE __bpf_kfunc_start_defs(); __bpf_kfunc bool bpf_session_is_return(void) @@ -3530,7 +3530,7 @@ __bpf_kfunc bool bpf_session_is_return(void) return session_ctx->is_return; } -__bpf_kfunc __u64 *bpf_session_cookie(void) +__bpf_kfunc long *bpf_session_cookie(void) { struct bpf_session_run_ctx *session_ctx; @@ -3568,4 +3568,3 @@ static int __init bpf_kprobe_multi_kfuncs_init(void) } late_initcall(bpf_kprobe_multi_kfuncs_init); -#endif diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c index 5e263c14157403cd9ad65be5bf543fe525fb09ba..39877c80d6cb9ace8163e2bfd16525629562ba82 100644 --- a/kernel/trace/trace_probe.c +++ b/kernel/trace/trace_probe.c @@ -554,6 +554,10 @@ static int parse_btf_field(char *fieldname, const struct btf_type *type, anon_offs = 0; field = btf_find_struct_member(ctx->btf, type, fieldname, &anon_offs); + if (IS_ERR(field)) { + trace_probe_log_err(ctx->offset, BAD_BTF_TID); + return PTR_ERR(field); + } if (!field) { trace_probe_log_err(ctx->offset, NO_BTF_FIELD); return -ENOENT; diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c index 8541fa1494ae3d7a22e18fd37b99434b9042f5f0..c98e3b3386badb60a187e86e557ed4c26f76c966 100644 --- a/kernel/trace/trace_uprobe.c +++ b/kernel/trace/trace_uprobe.c @@ -970,19 +970,17 @@ static struct uprobe_cpu_buffer *prepare_uprobe_buffer(struct trace_uprobe *tu, static void __uprobe_trace_func(struct trace_uprobe *tu, unsigned long func, struct pt_regs *regs, - struct uprobe_cpu_buffer **ucbp, + struct uprobe_cpu_buffer *ucb, struct trace_event_file *trace_file) { struct uprobe_trace_entry_head *entry; struct trace_event_buffer fbuffer; - struct uprobe_cpu_buffer *ucb; void *data; int size, esize; struct trace_event_call *call = trace_probe_event_call(&tu->tp); WARN_ON(call != trace_file->event_call); - ucb = prepare_uprobe_buffer(tu, regs, ucbp); if (WARN_ON_ONCE(ucb->dsize > PAGE_SIZE)) return; @@ -1014,13 +1012,16 @@ static int uprobe_trace_func(struct trace_uprobe *tu, struct pt_regs *regs, struct uprobe_cpu_buffer **ucbp) { struct event_file_link *link; + struct uprobe_cpu_buffer *ucb; if (is_ret_probe(tu)) return 0; + ucb = prepare_uprobe_buffer(tu, regs, ucbp); + rcu_read_lock(); trace_probe_for_each_link_rcu(link, &tu->tp) - __uprobe_trace_func(tu, 0, regs, ucbp, link->file); + __uprobe_trace_func(tu, 0, regs, ucb, link->file); rcu_read_unlock(); return 0; @@ -1031,10 +1032,13 @@ static void uretprobe_trace_func(struct trace_uprobe *tu, unsigned long func, struct uprobe_cpu_buffer **ucbp) { struct event_file_link *link; + struct uprobe_cpu_buffer *ucb; + + ucb = prepare_uprobe_buffer(tu, regs, ucbp); rcu_read_lock(); trace_probe_for_each_link_rcu(link, &tu->tp) - __uprobe_trace_func(tu, func, regs, ucbp, link->file); + __uprobe_trace_func(tu, func, regs, ucb, link->file); rcu_read_unlock(); } diff --git a/lib/fortify_kunit.c b/lib/fortify_kunit.c index 39da5b3bc649c28896e60d9dac974b98973da8d0..f9cc467334ce3de06fbf101ace3dcab87566cbf0 100644 --- a/lib/fortify_kunit.c +++ b/lib/fortify_kunit.c @@ -235,9 +235,6 @@ static void fortify_test_alloc_size_##allocator##_dynamic(struct kunit *test) \ kmalloc_array_node(alloc_size, 1, gfp, NUMA_NO_NODE), \ kfree(p)); \ checker(expected_size, __kmalloc(alloc_size, gfp), \ - kfree(p)); \ - checker(expected_size, \ - __kmalloc_node(alloc_size, gfp, NUMA_NO_NODE), \ kfree(p)); \ \ orig = kmalloc(alloc_size, gfp); \ diff --git a/lib/objagg.c b/lib/objagg.c index 1e248629ed6431052384c8327df55ca5a17cefe7..363e43e849ac3198f63bdcb97ec57b0eb652c73d 100644 --- a/lib/objagg.c +++ b/lib/objagg.c @@ -167,6 +167,9 @@ static int objagg_obj_parent_assign(struct objagg *objagg, { void *delta_priv; + if (WARN_ON(!objagg_obj_is_root(parent))) + return -EINVAL; + delta_priv = objagg->ops->delta_create(objagg->priv, parent->obj, objagg_obj->obj); if (IS_ERR(delta_priv)) @@ -421,7 +424,7 @@ static struct objagg_obj *__objagg_obj_get(struct objagg *objagg, void *obj) * * There are 3 main options this function wraps: * 1) The object according to "obj" already exist. In that case - * the reference counter is incrementes and the object is returned. + * the reference counter is incremented and the object is returned. * 2) The object does not exist, but it can be aggregated within * another object. In that case, user ops->delta_create() is called * to obtain delta data and a new object is created with returned @@ -903,20 +906,6 @@ static const struct objagg_opt_algo *objagg_opt_algos[] = { [OBJAGG_OPT_ALGO_SIMPLE_GREEDY] = &objagg_opt_simple_greedy, }; -static int objagg_hints_obj_cmp(struct rhashtable_compare_arg *arg, - const void *obj) -{ - struct rhashtable *ht = arg->ht; - struct objagg_hints *objagg_hints = - container_of(ht, struct objagg_hints, node_ht); - const struct objagg_ops *ops = objagg_hints->ops; - const char *ptr = obj; - - ptr += ht->p.key_offset; - return ops->hints_obj_cmp ? ops->hints_obj_cmp(ptr, arg->key) : - memcmp(ptr, arg->key, ht->p.key_len); -} - /** * objagg_hints_get - obtains hints instance * @objagg: objagg instance @@ -955,7 +944,6 @@ struct objagg_hints *objagg_hints_get(struct objagg *objagg, offsetof(struct objagg_hints_node, obj); objagg_hints->ht_params.head_offset = offsetof(struct objagg_hints_node, ht_node); - objagg_hints->ht_params.obj_cmpfn = objagg_hints_obj_cmp; err = rhashtable_init(&objagg_hints->node_ht, &objagg_hints->ht_params); if (err) diff --git a/lib/test_bpf.c b/lib/test_bpf.c index 207ff87194db74cd00edba9e96ee55c14727ef99..ce5716c3999a470a5e03bfbfb4cda899d3c0354b 100644 --- a/lib/test_bpf.c +++ b/lib/test_bpf.c @@ -15706,4 +15706,5 @@ static void __exit test_bpf_exit(void) module_init(test_bpf_init); module_exit(test_bpf_exit); +MODULE_DESCRIPTION("Testsuite for BPF interpreter and BPF JIT compiler"); MODULE_LICENSE("GPL"); diff --git a/lib/test_objagg.c b/lib/test_objagg.c index c0c957c50635415b00b5ce526c2b25efc4ac370e..d34df4306b874f6d338acf5787fe880f96c3144c 100644 --- a/lib/test_objagg.c +++ b/lib/test_objagg.c @@ -60,7 +60,7 @@ static struct objagg_obj *world_obj_get(struct world *world, if (!world->key_refs[key_id_index(key_id)]) { world->objagg_objs[key_id_index(key_id)] = objagg_obj; } else if (world->objagg_objs[key_id_index(key_id)] != objagg_obj) { - pr_err("Key %u: God another object for the same key.\n", + pr_err("Key %u: Got another object for the same key.\n", key_id); err = -EINVAL; goto err_key_id_check; diff --git a/lib/test_rhashtable.c b/lib/test_rhashtable.c index 42b585208249c6d8731cb02e81ff62c5939c121d..c63db03ebb9dcfd5730c7363bf4585e2424af4bc 100644 --- a/lib/test_rhashtable.c +++ b/lib/test_rhashtable.c @@ -811,4 +811,5 @@ static void __exit test_rht_exit(void) module_init(test_rht_init); module_exit(test_rht_exit); +MODULE_DESCRIPTION("Resizable, Scalable, Concurrent Hash Table test module"); MODULE_LICENSE("GPL v2"); diff --git a/net/9p/client.c b/net/9p/client.c index 00774656eeac8368f538e59cb8b456af405a3469..5cd94721d974f4428d9a642c13bf325ee15fd2be 100644 --- a/net/9p/client.c +++ b/net/9p/client.c @@ -236,6 +236,8 @@ static int p9_fcall_init(struct p9_client *c, struct p9_fcall *fc, if (!fc->sdata) return -ENOMEM; fc->capacity = alloc_msize; + fc->id = 0; + fc->tag = P9_NOTAG; return 0; } diff --git a/net/Kconfig b/net/Kconfig index f0a8692496ffa63a2fb96bd2ae764df9f9fe311b..9fe65fa26e48c5e72f809b407edaa42c0d3db41d 100644 --- a/net/Kconfig +++ b/net/Kconfig @@ -290,15 +290,21 @@ config MAX_SKB_FRAGS If unsure, say 17. config RPS - bool + bool "Receive packet steering" depends on SMP && SYSFS default y + help + Software receive side packet steering (RPS) distributes the + load of received packet processing across multiple CPUs. config RFS_ACCEL - bool + bool "Hardware acceleration of RFS" depends on RPS select CPU_RMAP default y + help + Allowing drivers for multiqueue hardware with flow filter tables to + accelerate RFS. config SOCK_RX_QUEUE_MAPPING bool @@ -351,7 +357,7 @@ config BPF_STREAM_PARSER BPF_MAP_TYPE_SOCKMAP. config NET_FLOW_LIMIT - bool + bool "Net flow limit" depends on RPS default y help diff --git a/net/ax25/af_ax25.c b/net/ax25/af_ax25.c index 8077cf2ee448038fab65e842785f6d18bf0e4dc4..d6f9fae06a9d8139ec0505358327e3876af228ae 100644 --- a/net/ax25/af_ax25.c +++ b/net/ax25/af_ax25.c @@ -1378,8 +1378,10 @@ static int ax25_accept(struct socket *sock, struct socket *newsock, { struct sk_buff *skb; struct sock *newsk; + ax25_dev *ax25_dev; DEFINE_WAIT(wait); struct sock *sk; + ax25_cb *ax25; int err = 0; if (sock->state != SS_UNCONNECTED) @@ -1434,6 +1436,10 @@ static int ax25_accept(struct socket *sock, struct socket *newsock, kfree_skb(skb); sk_acceptq_removed(sk); newsock->state = SS_CONNECTED; + ax25 = sk_to_ax25(newsk); + ax25_dev = ax25->ax25_dev; + netdev_hold(ax25_dev->dev, &ax25->dev_tracker, GFP_ATOMIC); + ax25_dev_hold(ax25_dev); out: release_sock(sk); diff --git a/net/ax25/ax25_dev.c b/net/ax25/ax25_dev.c index 742d7c68e7e7e9fb6c1734f75d20402b2e4ad5a9..9efd6690b3443653a2f2ef421080aa48b214a8ba 100644 --- a/net/ax25/ax25_dev.c +++ b/net/ax25/ax25_dev.c @@ -196,7 +196,7 @@ void __exit ax25_dev_free(void) list_for_each_entry_safe(s, n, &ax25_dev_list, list) { netdev_put(s->dev, &s->dev_tracker); list_del(&s->list); - kfree(s); + ax25_dev_put(s); } spin_unlock_bh(&ax25_dev_lock); } diff --git a/net/bpf/bpf_dummy_struct_ops.c b/net/bpf/bpf_dummy_struct_ops.c index 891cdf61c65ae2f850ed0218f2d9b8c110ee7bd1..3ea52b05adfbd8a22aa126b17e2751e938d1dccc 100644 --- a/net/bpf/bpf_dummy_struct_ops.c +++ b/net/bpf/bpf_dummy_struct_ops.c @@ -272,12 +272,12 @@ static int bpf_dummy_init_member(const struct btf_type *t, return -EOPNOTSUPP; } -static int bpf_dummy_reg(void *kdata) +static int bpf_dummy_reg(void *kdata, struct bpf_link *link) { return -EOPNOTSUPP; } -static void bpf_dummy_unreg(void *kdata) +static void bpf_dummy_unreg(void *kdata, struct bpf_link *link) { } diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c index f6aad4ed2ab2f6b09ad2f6442cde26b39949871a..3c965e32fc3332ad3e2c7e3c1e7c5208c9a0e9bb 100644 --- a/net/bpf/test_run.c +++ b/net/bpf/test_run.c @@ -727,10 +727,16 @@ static void __bpf_prog_test_run_raw_tp(void *data) { struct bpf_raw_tp_test_run_info *info = data; + struct bpf_trace_run_ctx run_ctx = {}; + struct bpf_run_ctx *old_run_ctx; + + old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx); rcu_read_lock(); info->retval = bpf_prog_run(info->prog, info->ctx); rcu_read_unlock(); + + bpf_reset_run_ctx(old_run_ctx); } int bpf_prog_test_run_raw_tp(struct bpf_prog *prog, @@ -977,7 +983,8 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, void *data; int ret; - if (kattr->test.flags || kattr->test.cpu || kattr->test.batch_size) + if ((kattr->test.flags & ~BPF_F_TEST_SKB_CHECKSUM_COMPLETE) || + kattr->test.cpu || kattr->test.batch_size) return -EINVAL; data = bpf_test_init(kattr, kattr->test.data_size_in, @@ -1025,6 +1032,7 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, skb_reserve(skb, NET_SKB_PAD + NET_IP_ALIGN); __skb_put(skb, size); + if (ctx && ctx->ifindex > 1) { dev = dev_get_by_index(net, ctx->ifindex); if (!dev) { @@ -1060,9 +1068,19 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, __skb_push(skb, hh_len); if (is_direct_pkt_access) bpf_compute_data_pointers(skb); + ret = convert___skb_to_skb(skb, ctx); if (ret) goto out; + + if (kattr->test.flags & BPF_F_TEST_SKB_CHECKSUM_COMPLETE) { + const int off = skb_network_offset(skb); + int len = skb->len - off; + + skb->csum = skb_checksum(skb, off, len, 0); + skb->ip_summed = CHECKSUM_COMPLETE; + } + ret = bpf_test_run(prog, skb, repeat, &retval, &duration, false); if (ret) goto out; @@ -1077,6 +1095,20 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, } memset(__skb_push(skb, hh_len), 0, hh_len); } + + if (kattr->test.flags & BPF_F_TEST_SKB_CHECKSUM_COMPLETE) { + const int off = skb_network_offset(skb); + int len = skb->len - off; + __wsum csum; + + csum = skb_checksum(skb, off, len, 0); + + if (csum_fold(skb->csum) != csum_fold(csum)) { + ret = -EBADMSG; + goto out; + } + } + convert_skb_to___skb(skb, ctx); size = skb->len; diff --git a/net/bridge/br_netlink_tunnel.c b/net/bridge/br_netlink_tunnel.c index 17abf092f7cac1fd1ba2b0e3818ae0fc210e00b3..71a12da30004c795736aadf5d3d535b439aa5ce7 100644 --- a/net/bridge/br_netlink_tunnel.c +++ b/net/bridge/br_netlink_tunnel.c @@ -315,8 +315,8 @@ int br_process_vlan_tunnel_info(const struct net_bridge *br, if (curr_change) *changed = curr_change; - __vlan_tunnel_handle_range(p, &v_start, &v_end, v, - curr_change); + __vlan_tunnel_handle_range(p, &v_start, &v_end, v, + curr_change); } if (v_start && v_end) br_vlan_notify(br, p, v_start->vid, v_end->vid, diff --git a/net/bridge/netfilter/nf_conntrack_bridge.c b/net/bridge/netfilter/nf_conntrack_bridge.c index c3c51b9a68265b443326432274e7fd75675e0e28..816bb0fde718edcf6be9e775120861be74cb915c 100644 --- a/net/bridge/netfilter/nf_conntrack_bridge.c +++ b/net/bridge/netfilter/nf_conntrack_bridge.c @@ -32,7 +32,7 @@ static int nf_br_ip_fragment(struct net *net, struct sock *sk, struct sk_buff *)) { int frag_max_size = BR_INPUT_SKB_CB(skb)->frag_max_size; - bool mono_delivery_time = skb->mono_delivery_time; + u8 tstamp_type = skb->tstamp_type; unsigned int hlen, ll_rs, mtu; ktime_t tstamp = skb->tstamp; struct ip_frag_state state; @@ -82,7 +82,7 @@ static int nf_br_ip_fragment(struct net *net, struct sock *sk, if (iter.frag) ip_fraglist_prepare(skb, &iter); - skb_set_delivery_time(skb, tstamp, mono_delivery_time); + skb_set_delivery_time(skb, tstamp, tstamp_type); err = output(net, sk, data, skb); if (err || !iter.frag) break; @@ -113,7 +113,7 @@ slow_path: goto blackhole; } - skb_set_delivery_time(skb2, tstamp, mono_delivery_time); + skb_set_delivery_time(skb2, tstamp, tstamp_type); err = output(net, sk, data, skb2); if (err) goto blackhole; diff --git a/net/caif/cfpkt_skbuff.c b/net/caif/cfpkt_skbuff.c index 7796414d47e52a30ce8c0518fb2c52ffc8070bd1..2ae8cfa3df8845dc85ee28619740c7e8fc9743ea 100644 --- a/net/caif/cfpkt_skbuff.c +++ b/net/caif/cfpkt_skbuff.c @@ -21,13 +21,6 @@ do { \ pr_warn(errmsg); \ } while (0) -struct cfpktq { - struct sk_buff_head head; - atomic_t count; - /* Lock protects count updates */ - spinlock_t lock; -}; - /* * net/caif/ is generic and does not * understand SKB, so we do this typecast diff --git a/net/core/dev.c b/net/core/dev.c index e1bb6d7856d922dfca1499df91bdade86bf391c1..e62698c7a0e63bdcbf9589a2d59398eca56a7b46 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2160,7 +2160,7 @@ EXPORT_SYMBOL(net_disable_timestamp); static inline void net_timestamp_set(struct sk_buff *skb) { skb->tstamp = 0; - skb->mono_delivery_time = 0; + skb->tstamp_type = SKB_CLOCK_REALTIME; if (static_branch_unlikely(&netstamp_needed_key)) skb->tstamp = ktime_get_real(); } @@ -4516,12 +4516,13 @@ set_rps_cpu(struct net_device *dev, struct sk_buff *skb, struct rps_dev_flow *rflow, u16 next_cpu) { if (next_cpu < nr_cpu_ids) { + u32 head; #ifdef CONFIG_RFS_ACCEL struct netdev_rx_queue *rxqueue; struct rps_dev_flow_table *flow_table; struct rps_dev_flow *old_rflow; - u32 flow_id, head; u16 rxq_index; + u32 flow_id; int rc; /* Should we steer this flow to a different hardware queue? */ diff --git a/net/core/dst_cache.c b/net/core/dst_cache.c index 6a0482e676d379f1f9bffdda51c7535243b3ec38..70c634b9e7b02300188582a1634d5977838db132 100644 --- a/net/core/dst_cache.c +++ b/net/core/dst_cache.c @@ -27,6 +27,7 @@ struct dst_cache_pcpu { static void dst_cache_per_cpu_dst_set(struct dst_cache_pcpu *dst_cache, struct dst_entry *dst, u32 cookie) { + DEBUG_NET_WARN_ON_ONCE(!in_softirq()); dst_release(dst_cache->dst); if (dst) dst_hold(dst); @@ -40,6 +41,7 @@ static struct dst_entry *dst_cache_per_cpu_get(struct dst_cache *dst_cache, { struct dst_entry *dst; + DEBUG_NET_WARN_ON_ONCE(!in_softirq()); dst = idst->dst; if (!dst) goto fail; diff --git a/net/core/filter.c b/net/core/filter.c index 2510464692af0f1ecb03c414a3dfc90e11cc954f..f1c37c85b8580aa44fc79b8f74ea7241cba4883d 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -2274,12 +2274,12 @@ static int __bpf_redirect_neigh_v6(struct sk_buff *skb, struct net_device *dev, err = bpf_out_neigh_v6(net, skb, dev, nh); if (unlikely(net_xmit_eval(err))) - dev->stats.tx_errors++; + DEV_STATS_INC(dev, tx_errors); else ret = NET_XMIT_SUCCESS; goto out_xmit; out_drop: - dev->stats.tx_errors++; + DEV_STATS_INC(dev, tx_errors); kfree_skb(skb); out_xmit: return ret; @@ -2380,12 +2380,12 @@ static int __bpf_redirect_neigh_v4(struct sk_buff *skb, struct net_device *dev, err = bpf_out_neigh_v4(net, skb, dev, nh); if (unlikely(net_xmit_eval(err))) - dev->stats.tx_errors++; + DEV_STATS_INC(dev, tx_errors); else ret = NET_XMIT_SUCCESS; goto out_xmit; out_drop: - dev->stats.tx_errors++; + DEV_STATS_INC(dev, tx_errors); kfree_skb(skb); out_xmit: return ret; @@ -6815,7 +6815,7 @@ static const struct bpf_func_proto bpf_skc_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCK_COMMON_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6834,7 +6834,7 @@ static const struct bpf_func_proto bpf_sk_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6853,7 +6853,7 @@ static const struct bpf_func_proto bpf_sk_lookup_udp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6877,7 +6877,7 @@ static const struct bpf_func_proto bpf_tc_skc_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCK_COMMON_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6901,7 +6901,7 @@ static const struct bpf_func_proto bpf_tc_sk_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6925,7 +6925,7 @@ static const struct bpf_func_proto bpf_tc_sk_lookup_udp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6963,7 +6963,7 @@ static const struct bpf_func_proto bpf_xdp_sk_lookup_udp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -6987,7 +6987,7 @@ static const struct bpf_func_proto bpf_xdp_skc_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCK_COMMON_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -7011,7 +7011,7 @@ static const struct bpf_func_proto bpf_xdp_sk_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -7031,7 +7031,7 @@ static const struct bpf_func_proto bpf_sock_addr_skc_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCK_COMMON_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -7050,7 +7050,7 @@ static const struct bpf_func_proto bpf_sock_addr_sk_lookup_tcp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -7069,7 +7069,7 @@ static const struct bpf_func_proto bpf_sock_addr_sk_lookup_udp_proto = { .ret_type = RET_PTR_TO_SOCKET_OR_NULL, .arg1_type = ARG_PTR_TO_CTX, .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, - .arg3_type = ARG_CONST_SIZE, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; @@ -7726,17 +7726,21 @@ BPF_CALL_3(bpf_skb_set_tstamp, struct sk_buff *, skb, return -EOPNOTSUPP; switch (tstamp_type) { - case BPF_SKB_TSTAMP_DELIVERY_MONO: + case BPF_SKB_CLOCK_REALTIME: + skb->tstamp = tstamp; + skb->tstamp_type = SKB_CLOCK_REALTIME; + break; + case BPF_SKB_CLOCK_MONOTONIC: if (!tstamp) return -EINVAL; skb->tstamp = tstamp; - skb->mono_delivery_time = 1; + skb->tstamp_type = SKB_CLOCK_MONOTONIC; break; - case BPF_SKB_TSTAMP_UNSPEC: - if (tstamp) + case BPF_SKB_CLOCK_TAI: + if (!tstamp) return -EINVAL; - skb->tstamp = 0; - skb->mono_delivery_time = 0; + skb->tstamp = tstamp; + skb->tstamp_type = SKB_CLOCK_TAI; break; default: return -EINVAL; @@ -9387,16 +9391,17 @@ static struct bpf_insn *bpf_convert_tstamp_type_read(const struct bpf_insn *si, { __u8 value_reg = si->dst_reg; __u8 skb_reg = si->src_reg; - /* AX is needed because src_reg and dst_reg could be the same */ - __u8 tmp_reg = BPF_REG_AX; - - *insn++ = BPF_LDX_MEM(BPF_B, tmp_reg, skb_reg, - SKB_BF_MONO_TC_OFFSET); - *insn++ = BPF_JMP32_IMM(BPF_JSET, tmp_reg, - SKB_MONO_DELIVERY_TIME_MASK, 2); - *insn++ = BPF_MOV32_IMM(value_reg, BPF_SKB_TSTAMP_UNSPEC); - *insn++ = BPF_JMP_A(1); - *insn++ = BPF_MOV32_IMM(value_reg, BPF_SKB_TSTAMP_DELIVERY_MONO); + BUILD_BUG_ON(__SKB_CLOCK_MAX != (int)BPF_SKB_CLOCK_TAI); + BUILD_BUG_ON(SKB_CLOCK_REALTIME != (int)BPF_SKB_CLOCK_REALTIME); + BUILD_BUG_ON(SKB_CLOCK_MONOTONIC != (int)BPF_SKB_CLOCK_MONOTONIC); + BUILD_BUG_ON(SKB_CLOCK_TAI != (int)BPF_SKB_CLOCK_TAI); + *insn++ = BPF_LDX_MEM(BPF_B, value_reg, skb_reg, SKB_BF_MONO_TC_OFFSET); + *insn++ = BPF_ALU32_IMM(BPF_AND, value_reg, SKB_TSTAMP_TYPE_MASK); +#ifdef __BIG_ENDIAN_BITFIELD + *insn++ = BPF_ALU32_IMM(BPF_RSH, value_reg, SKB_TSTAMP_TYPE_RSHIFT); +#else + BUILD_BUG_ON(!(SKB_TSTAMP_TYPE_MASK & 0x1)); +#endif return insn; } @@ -9439,11 +9444,12 @@ static struct bpf_insn *bpf_convert_tstamp_read(const struct bpf_prog *prog, __u8 tmp_reg = BPF_REG_AX; *insn++ = BPF_LDX_MEM(BPF_B, tmp_reg, skb_reg, SKB_BF_MONO_TC_OFFSET); - *insn++ = BPF_ALU32_IMM(BPF_AND, tmp_reg, - TC_AT_INGRESS_MASK | SKB_MONO_DELIVERY_TIME_MASK); - *insn++ = BPF_JMP32_IMM(BPF_JNE, tmp_reg, - TC_AT_INGRESS_MASK | SKB_MONO_DELIVERY_TIME_MASK, 2); - /* skb->tc_at_ingress && skb->mono_delivery_time, + /* check if ingress mask bits is set */ + *insn++ = BPF_JMP32_IMM(BPF_JSET, tmp_reg, TC_AT_INGRESS_MASK, 1); + *insn++ = BPF_JMP_A(4); + *insn++ = BPF_JMP32_IMM(BPF_JSET, tmp_reg, SKB_TSTAMP_TYPE_MASK, 1); + *insn++ = BPF_JMP_A(2); + /* skb->tc_at_ingress && skb->tstamp_type, * read 0 as the (rcv) timestamp. */ *insn++ = BPF_MOV64_IMM(value_reg, 0); @@ -9468,7 +9474,7 @@ static struct bpf_insn *bpf_convert_tstamp_write(const struct bpf_prog *prog, * the bpf prog is aware the tstamp could have delivery time. * Thus, write skb->tstamp as is if tstamp_type_access is true. * Otherwise, writing at ingress will have to clear the - * mono_delivery_time bit also. + * skb->tstamp_type bit also. */ if (!prog->tstamp_type_access) { __u8 tmp_reg = BPF_REG_AX; @@ -9478,8 +9484,8 @@ static struct bpf_insn *bpf_convert_tstamp_write(const struct bpf_prog *prog, *insn++ = BPF_JMP32_IMM(BPF_JSET, tmp_reg, TC_AT_INGRESS_MASK, 1); /* goto */ *insn++ = BPF_JMP_A(2); - /* : mono_delivery_time */ - *insn++ = BPF_ALU32_IMM(BPF_AND, tmp_reg, ~SKB_MONO_DELIVERY_TIME_MASK); + /* : skb->tstamp_type */ + *insn++ = BPF_ALU32_IMM(BPF_AND, tmp_reg, ~SKB_TSTAMP_TYPE_MASK); *insn++ = BPF_STX_MEM(BPF_B, skb_reg, tmp_reg, SKB_BF_MONO_TC_OFFSET); } #endif @@ -11853,28 +11859,34 @@ bpf_sk_base_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) } __bpf_kfunc_start_defs(); -__bpf_kfunc int bpf_dynptr_from_skb(struct sk_buff *skb, u64 flags, - struct bpf_dynptr_kern *ptr__uninit) +__bpf_kfunc int bpf_dynptr_from_skb(struct __sk_buff *s, u64 flags, + struct bpf_dynptr *ptr__uninit) { + struct bpf_dynptr_kern *ptr = (struct bpf_dynptr_kern *)ptr__uninit; + struct sk_buff *skb = (struct sk_buff *)s; + if (flags) { - bpf_dynptr_set_null(ptr__uninit); + bpf_dynptr_set_null(ptr); return -EINVAL; } - bpf_dynptr_init(ptr__uninit, skb, BPF_DYNPTR_TYPE_SKB, 0, skb->len); + bpf_dynptr_init(ptr, skb, BPF_DYNPTR_TYPE_SKB, 0, skb->len); return 0; } -__bpf_kfunc int bpf_dynptr_from_xdp(struct xdp_buff *xdp, u64 flags, - struct bpf_dynptr_kern *ptr__uninit) +__bpf_kfunc int bpf_dynptr_from_xdp(struct xdp_md *x, u64 flags, + struct bpf_dynptr *ptr__uninit) { + struct bpf_dynptr_kern *ptr = (struct bpf_dynptr_kern *)ptr__uninit; + struct xdp_buff *xdp = (struct xdp_buff *)x; + if (flags) { - bpf_dynptr_set_null(ptr__uninit); + bpf_dynptr_set_null(ptr); return -EINVAL; } - bpf_dynptr_init(ptr__uninit, xdp, BPF_DYNPTR_TYPE_XDP, 0, xdp_get_buff_len(xdp)); + bpf_dynptr_init(ptr, xdp, BPF_DYNPTR_TYPE_XDP, 0, xdp_get_buff_len(xdp)); return 0; } @@ -11900,10 +11912,11 @@ __bpf_kfunc int bpf_sock_addr_set_sun_path(struct bpf_sock_addr_kern *sa_kern, return 0; } -__bpf_kfunc int bpf_sk_assign_tcp_reqsk(struct sk_buff *skb, struct sock *sk, +__bpf_kfunc int bpf_sk_assign_tcp_reqsk(struct __sk_buff *s, struct sock *sk, struct bpf_tcp_req_attrs *attrs, int attrs__sz) { #if IS_ENABLED(CONFIG_SYN_COOKIES) + struct sk_buff *skb = (struct sk_buff *)s; const struct request_sock_ops *ops; struct inet_request_sock *ireq; struct tcp_request_sock *treq; @@ -11998,16 +12011,17 @@ __bpf_kfunc int bpf_sk_assign_tcp_reqsk(struct sk_buff *skb, struct sock *sk, __bpf_kfunc_end_defs(); -int bpf_dynptr_from_skb_rdonly(struct sk_buff *skb, u64 flags, - struct bpf_dynptr_kern *ptr__uninit) +int bpf_dynptr_from_skb_rdonly(struct __sk_buff *skb, u64 flags, + struct bpf_dynptr *ptr__uninit) { + struct bpf_dynptr_kern *ptr = (struct bpf_dynptr_kern *)ptr__uninit; int err; err = bpf_dynptr_from_skb(skb, flags, ptr__uninit); if (err) return err; - bpf_dynptr_set_rdonly(ptr__uninit); + bpf_dynptr_set_rdonly(ptr); return 0; } diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c index f82e9a7d3b379b328c81c5af23be884a05cabaea..59fe46077b3ca311e59a786932e6cda1e9b65446 100644 --- a/net/core/flow_dissector.c +++ b/net/core/flow_dissector.c @@ -382,7 +382,9 @@ skb_flow_dissect_tunnel_info(const struct sk_buff *skb, !dissector_uses_key(flow_dissector, FLOW_DISSECTOR_KEY_ENC_IP) && !dissector_uses_key(flow_dissector, - FLOW_DISSECTOR_KEY_ENC_OPTS)) + FLOW_DISSECTOR_KEY_ENC_OPTS) && + !dissector_uses_key(flow_dissector, + FLOW_DISSECTOR_KEY_ENC_FLAGS)) return; info = skb_tunnel_info(skb); @@ -475,6 +477,18 @@ skb_flow_dissect_tunnel_info(const struct sk_buff *skb, IP_TUNNEL_GENEVE_OPT_BIT); enc_opt->dst_opt_type = val < __IP_TUNNEL_FLAG_NUM ? val : 0; } + + if (dissector_uses_key(flow_dissector, FLOW_DISSECTOR_KEY_ENC_FLAGS)) { + struct flow_dissector_key_enc_flags *enc_flags; + IP_TUNNEL_DECLARE_FLAGS(flags) = {}; + + enc_flags = skb_flow_dissector_target(flow_dissector, + FLOW_DISSECTOR_KEY_ENC_FLAGS, + target_container); + ip_tunnel_set_encflags_present(flags); + ip_tunnel_flags_and(flags, flags, info->key.tun_flags); + enc_flags->flags = bitmap_read(flags, IP_TUNNEL_CSUM_BIT, 32); + } } EXPORT_SYMBOL(skb_flow_dissect_tunnel_info); diff --git a/net/core/gen_estimator.c b/net/core/gen_estimator.c index fae9c4694186eaaac3295f32c7cb2f6b7e5e0906..412816076b8bc5543ed1eb32838438d1966cd473 100644 --- a/net/core/gen_estimator.c +++ b/net/core/gen_estimator.c @@ -206,7 +206,7 @@ void gen_kill_estimator(struct net_rate_estimator __rcu **rate_est) { struct net_rate_estimator *est; - est = xchg((__force struct net_rate_estimator **)rate_est, NULL); + est = unrcu_pointer(xchg(rate_est, NULL)); if (est) { timer_shutdown_sync(&est->timer); kfree_rcu(est, rcu); diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 45fd88405b6b89d373328e44276adf54be194dc1..277751375b0a5a7322691390d6d9f8ec5574efd4 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -3578,7 +3578,7 @@ static void neigh_copy_dflt_parms(struct net *net, struct neigh_parms *p, rcu_read_unlock(); } -static void neigh_proc_update(struct ctl_table *ctl, int write) +static void neigh_proc_update(const struct ctl_table *ctl, int write) { struct net_device *dev = ctl->extra1; struct neigh_parms *p = ctl->extra2; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index f4444b4e39e63a4fa0ad24f36db24123068a0152..3927a0a7fa9a890bf6afca3dcea9fa7641017eb4 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -1027,8 +1027,8 @@ static void page_pool_disable_direct_recycling(struct page_pool *pool) /* To avoid races with recycling and additional barriers make sure * pool and NAPI are unlinked when NAPI is disabled. */ - WARN_ON(!test_bit(NAPI_STATE_SCHED, &pool->p.napi->state) || - READ_ONCE(pool->p.napi->list_owner) != -1); + WARN_ON(!test_bit(NAPI_STATE_SCHED, &pool->p.napi->state)); + WARN_ON(READ_ONCE(pool->p.napi->list_owner) != -1); WRITE_ONCE(pool->p.napi, NULL); } diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c index b86b0a87367dd53b5bdef80cfddad558a186bb7c..eabfc8290f5e29f2ef3e5c1481715ae9056ea689 100644 --- a/net/core/rtnetlink.c +++ b/net/core/rtnetlink.c @@ -6484,6 +6484,52 @@ static int rtnl_mdb_del(struct sk_buff *skb, struct nlmsghdr *nlh, /* Process one rtnetlink message. */ +static int rtnl_dumpit(struct sk_buff *skb, struct netlink_callback *cb) +{ + const bool needs_lock = !(cb->flags & RTNL_FLAG_DUMP_UNLOCKED); + rtnl_dumpit_func dumpit = cb->data; + int err; + + /* Previous iteration have already finished, avoid calling->dumpit() + * again, it may not expect to be called after it reached the end. + */ + if (!dumpit) + return 0; + + if (needs_lock) + rtnl_lock(); + err = dumpit(skb, cb); + if (needs_lock) + rtnl_unlock(); + + /* Old dump handlers used to send NLM_DONE as in a separate recvmsg(). + * Some applications which parse netlink manually depend on this. + */ + if (cb->flags & RTNL_FLAG_DUMP_SPLIT_NLM_DONE) { + if (err < 0 && err != -EMSGSIZE) + return err; + if (!err) + cb->data = NULL; + + return skb->len; + } + return err; +} + +static int rtnetlink_dump_start(struct sock *ssk, struct sk_buff *skb, + const struct nlmsghdr *nlh, + struct netlink_dump_control *control) +{ + if (control->flags & RTNL_FLAG_DUMP_SPLIT_NLM_DONE || + !(control->flags & RTNL_FLAG_DUMP_UNLOCKED)) { + WARN_ON(control->data); + control->data = control->dump; + control->dump = rtnl_dumpit; + } + + return netlink_dump_start(ssk, skb, nlh, control); +} + static int rtnetlink_rcv_msg(struct sk_buff *skb, struct nlmsghdr *nlh, struct netlink_ext_ack *extack) { @@ -6548,7 +6594,7 @@ static int rtnetlink_rcv_msg(struct sk_buff *skb, struct nlmsghdr *nlh, .module = owner, .flags = flags, }; - err = netlink_dump_start(rtnl, skb, nlh, &c); + err = rtnetlink_dump_start(rtnl, skb, nlh, &c); /* netlink_dump_start() will keep a reference on * module if dump is still in progress. */ @@ -6663,7 +6709,6 @@ static int __net_init rtnetlink_net_init(struct net *net) struct netlink_kernel_cfg cfg = { .groups = RTNLGRP_MAX, .input = rtnetlink_rcv, - .cb_mutex = &rtnl_mutex, .flags = NL_CFG_F_NONROOT_RECV, .bind = rtnetlink_bind, }; @@ -6694,7 +6739,7 @@ void __init rtnetlink_init(void) register_netdevice_notifier(&rtnetlink_dev_notifier); rtnl_register(PF_UNSPEC, RTM_GETLINK, rtnl_getlink, - rtnl_dump_ifinfo, 0); + rtnl_dump_ifinfo, RTNL_FLAG_DUMP_SPLIT_NLM_DONE); rtnl_register(PF_UNSPEC, RTM_SETLINK, rtnl_setlink, NULL, 0); rtnl_register(PF_UNSPEC, RTM_NEWLINK, rtnl_newlink, NULL, 0); rtnl_register(PF_UNSPEC, RTM_DELLINK, rtnl_dellink, NULL, 0); diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 466999a7515e6c50873f84f694a74188a592cc2d..c8ac79851cd67954205cf6e8dcad6d246ce17d7e 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -4139,6 +4139,9 @@ int skb_shift(struct sk_buff *tgt, struct sk_buff *skb, int shiftlen) if (skb_zcopy(tgt) || skb_zcopy(skb)) return 0; + DEBUG_NET_WARN_ON_ONCE(tgt->pp_recycle != skb->pp_recycle); + DEBUG_NET_WARN_ON_ONCE(skb_cmp_decrypted(tgt, skb)); + todo = shiftlen; from = 0; to = skb_shinfo(tgt)->nr_frags; diff --git a/net/core/sock.c b/net/core/sock.c index 8629f9aecf91a36cdd17d91371b0480894969f2b..69baddcfbd8c123b4566dd6a0d07db6fc83660f2 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -1083,6 +1083,17 @@ bool sockopt_capable(int cap) } EXPORT_SYMBOL(sockopt_capable); +static int sockopt_validate_clockid(__kernel_clockid_t value) +{ + switch (value) { + case CLOCK_REALTIME: + case CLOCK_MONOTONIC: + case CLOCK_TAI: + return 0; + } + return -EINVAL; +} + /* * This is meant for all protocols to use and covers goings on * at the socket level. Everything here is generic. @@ -1497,6 +1508,11 @@ set_sndbuf: ret = -EPERM; break; } + + ret = sockopt_validate_clockid(sk_txtime.clockid); + if (ret) + break; + sock_valbool_flag(sk, SOCK_TXTIME, true); sk->sk_clockid = sk_txtime.clockid; sk->sk_txtime_deadline_mode = @@ -2262,7 +2278,12 @@ static void sk_init_common(struct sock *sk) lockdep_set_class_and_name(&sk->sk_error_queue.lock, af_elock_keys + sk->sk_family, af_family_elock_key_strings[sk->sk_family]); - lockdep_set_class_and_name(&sk->sk_callback_lock, + if (sk->sk_kern_sock) + lockdep_set_class_and_name(&sk->sk_callback_lock, + af_kern_callback_keys + sk->sk_family, + af_family_kern_clock_key_strings[sk->sk_family]); + else + lockdep_set_class_and_name(&sk->sk_callback_lock, af_callback_keys + sk->sk_family, af_family_clock_key_strings[sk->sk_family]); } @@ -3460,18 +3481,6 @@ void sock_init_data_uid(struct socket *sock, struct sock *sk, kuid_t uid) } sk->sk_uid = uid; - rwlock_init(&sk->sk_callback_lock); - if (sk->sk_kern_sock) - lockdep_set_class_and_name( - &sk->sk_callback_lock, - af_kern_callback_keys + sk->sk_family, - af_family_kern_clock_key_strings[sk->sk_family]); - else - lockdep_set_class_and_name( - &sk->sk_callback_lock, - af_callback_keys + sk->sk_family, - af_family_clock_key_strings[sk->sk_family]); - sk->sk_state_change = sock_def_wakeup; sk->sk_data_ready = sock_def_readable; sk->sk_write_space = sock_def_write_space; diff --git a/net/core/sock_diag.c b/net/core/sock_diag.c index 6541228380252d597821b084df34176bff4ada83..a08eed9b9142e38d5f4a6a16c139b37d1a84cad1 100644 --- a/net/core/sock_diag.c +++ b/net/core/sock_diag.c @@ -18,7 +18,7 @@ static const struct sock_diag_handler __rcu *sock_diag_handlers[AF_MAX]; -static struct sock_diag_inet_compat __rcu *inet_rcv_compat; +static const struct sock_diag_inet_compat __rcu *inet_rcv_compat; static struct workqueue_struct *broadcast_wq; @@ -187,8 +187,7 @@ void sock_diag_broadcast_destroy(struct sock *sk) void sock_diag_register_inet_compat(const struct sock_diag_inet_compat *ptr) { - xchg((__force const struct sock_diag_inet_compat **)&inet_rcv_compat, - ptr); + xchg(&inet_rcv_compat, RCU_INITIALIZER(ptr)); } EXPORT_SYMBOL_GPL(sock_diag_register_inet_compat); @@ -196,8 +195,7 @@ void sock_diag_unregister_inet_compat(const struct sock_diag_inet_compat *ptr) { const struct sock_diag_inet_compat *old; - old = xchg((__force const struct sock_diag_inet_compat **)&inet_rcv_compat, - NULL); + old = unrcu_pointer(xchg(&inet_rcv_compat, NULL)); WARN_ON_ONCE(old != ptr); } EXPORT_SYMBOL_GPL(sock_diag_unregister_inet_compat); diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 9402889840bf7e4fe2adb743d387b9dcdbe17024..d3dbb92153f2fe7f1ddc8e35b495533fbf60a8cb 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -423,9 +423,6 @@ static int __sock_map_delete(struct bpf_stab *stab, struct sock *sk_test, struct sock *sk; int err = 0; - if (irqs_disabled()) - return -EOPNOTSUPP; /* locks here are hardirq-unsafe */ - spin_lock_bh(&stab->lock); sk = *psk; if (!sk_test || sk_test == sk) @@ -948,9 +945,6 @@ static long sock_hash_delete_elem(struct bpf_map *map, void *key) struct bpf_shtab_elem *elem; int ret = -ENOENT; - if (irqs_disabled()) - return -EOPNOTSUPP; /* locks here are hardirq-unsafe */ - hash = sock_hash_bucket_hash(key, key_size); bucket = sock_hash_select_bucket(htab, hash); @@ -1680,19 +1674,23 @@ void sock_map_close(struct sock *sk, long timeout) lock_sock(sk); rcu_read_lock(); - psock = sk_psock_get(sk); - if (unlikely(!psock)) { - rcu_read_unlock(); - release_sock(sk); - saved_close = READ_ONCE(sk->sk_prot)->close; - } else { + psock = sk_psock(sk); + if (likely(psock)) { saved_close = psock->saved_close; sock_map_remove_links(sk, psock); + psock = sk_psock_get(sk); + if (unlikely(!psock)) + goto no_psock; rcu_read_unlock(); sk_psock_stop(psock); release_sock(sk); cancel_delayed_work_sync(&psock->work); sk_psock_put(sk, psock); + } else { + saved_close = READ_ONCE(sk->sk_prot)->close; +no_psock: + rcu_read_unlock(); + release_sock(sk); } /* Make sure we do not recurse. This is a bug. diff --git a/net/core/sysctl_net_core.c b/net/core/sysctl_net_core.c index c9fb9ad874857311537cd50924e29caa6d202e07..2079000691e286ef753063b69586c7d297d175cb 100644 --- a/net/core/sysctl_net_core.c +++ b/net/core/sysctl_net_core.c @@ -382,38 +382,6 @@ proc_dolongvec_minmax_bpf_restricted(struct ctl_table *table, int write, #endif static struct ctl_table net_core_table[] = { - { - .procname = "wmem_max", - .data = &sysctl_wmem_max, - .maxlen = sizeof(int), - .mode = 0644, - .proc_handler = proc_dointvec_minmax, - .extra1 = &min_sndbuf, - }, - { - .procname = "rmem_max", - .data = &sysctl_rmem_max, - .maxlen = sizeof(int), - .mode = 0644, - .proc_handler = proc_dointvec_minmax, - .extra1 = &min_rcvbuf, - }, - { - .procname = "wmem_default", - .data = &sysctl_wmem_default, - .maxlen = sizeof(int), - .mode = 0644, - .proc_handler = proc_dointvec_minmax, - .extra1 = &min_sndbuf, - }, - { - .procname = "rmem_default", - .data = &sysctl_rmem_default, - .maxlen = sizeof(int), - .mode = 0644, - .proc_handler = proc_dointvec_minmax, - .extra1 = &min_rcvbuf, - }, { .procname = "mem_pcpu_rsv", .data = &net_hotdata.sysctl_mem_pcpu_rsv, @@ -697,6 +665,41 @@ static struct ctl_table netns_core_table[] = { .extra2 = SYSCTL_ONE, .proc_handler = proc_dou8vec_minmax, }, + /* sysctl_core_net_init() will set the values after this + * to readonly in network namespaces + */ + { + .procname = "wmem_max", + .data = &sysctl_wmem_max, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = &min_sndbuf, + }, + { + .procname = "rmem_max", + .data = &sysctl_rmem_max, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = &min_rcvbuf, + }, + { + .procname = "wmem_default", + .data = &sysctl_wmem_default, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = &min_sndbuf, + }, + { + .procname = "rmem_default", + .data = &sysctl_rmem_default, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = &min_rcvbuf, + }, }; static int __init fb_tunnels_only_for_init_net_sysctl_setup(char *str) @@ -724,8 +727,14 @@ static __net_init int sysctl_core_net_init(struct net *net) if (tbl == NULL) goto err_dup; - for (i = 0; i < table_size; ++i) + for (i = 0; i < table_size; ++i) { + if (tbl[i].data == &sysctl_wmem_max) + break; + tbl[i].data += (char *)net - (char *)&init_net; + } + for (; i < table_size; ++i) + tbl[i].mode &= ~0222; } net->core.sysctl_hdr = register_net_sysctl_sz(net, "net/core", tbl, table_size); diff --git a/net/dccp/minisocks.c b/net/dccp/minisocks.c index 251a57cf58223b07a7a29e81e049532c46fe74e6..fecc8190064f865e8f52951a863f978193abca49 100644 --- a/net/dccp/minisocks.c +++ b/net/dccp/minisocks.c @@ -54,17 +54,10 @@ void dccp_time_wait(struct sock *sk, int state, int timeo) if (state == DCCP_TIME_WAIT) timeo = DCCP_TIMEWAIT_LEN; - /* tw_timer is pinned, so we need to make sure BH are disabled - * in following section, otherwise timer handler could run before - * we complete the initialization. - */ - local_bh_disable(); - inet_twsk_schedule(tw, timeo); /* Linkage updates. * Note that access to tw after this point is illegal. */ - inet_twsk_hashdance(tw, sk, &dccp_hashinfo); - local_bh_enable(); + inet_twsk_hashdance_schedule(tw, sk, &dccp_hashinfo, timeo); } else { /* Sorry, if we're out of memory, just CLOSE this * socket up. We've got bigger problems than diff --git a/net/devlink/dpipe.c b/net/devlink/dpipe.c index a72a9292efc5cfc1a8548d04a7f404941a24874c..55009b377447fe7734a1533828185e8e2e162bd8 100644 --- a/net/devlink/dpipe.c +++ b/net/devlink/dpipe.c @@ -839,7 +839,7 @@ EXPORT_SYMBOL_GPL(devlink_dpipe_table_counter_enabled); */ int devl_dpipe_table_register(struct devlink *devlink, const char *table_name, - struct devlink_dpipe_table_ops *table_ops, + const struct devlink_dpipe_table_ops *table_ops, void *priv, bool counter_control_extern) { struct devlink_dpipe_table *table; diff --git a/net/dsa/dsa.c b/net/dsa/dsa.c index 12521a7d404810c7399e3f8c0baa408f0a8a4b8b..668c729946ea676591a641e9a87e9b2277e07819 100644 --- a/net/dsa/dsa.c +++ b/net/dsa/dsa.c @@ -1507,9 +1507,7 @@ static int dsa_switch_probe(struct dsa_switch *ds) if (ds->phylink_mac_ops) { if (ds->ops->phylink_mac_select_pcs || - ds->ops->phylink_mac_prepare || ds->ops->phylink_mac_config || - ds->ops->phylink_mac_finish || ds->ops->phylink_mac_link_down || ds->ops->phylink_mac_link_up) return -EINVAL; diff --git a/net/dsa/port.c b/net/dsa/port.c index 9a249d4ac3a5d9dffb0e7fa899ed71e5499deb22..25258b33e59e015db7c4be9a7d403e318b21af71 100644 --- a/net/dsa/port.c +++ b/net/dsa/port.c @@ -1467,10 +1467,34 @@ int dsa_port_change_conduit(struct dsa_port *dp, struct net_device *conduit, */ dsa_user_unsync_ha(dev); + /* If live-changing, we also need to uninstall the user device address + * from the port FDB and the conduit interface. + */ + if (dev->flags & IFF_UP) + dsa_user_host_uc_uninstall(dev); + err = dsa_port_assign_conduit(dp, conduit, extack, true); if (err) goto rewind_old_addrs; + /* If the port doesn't have its own MAC address and relies on the DSA + * conduit's one, inherit it again from the new DSA conduit. + */ + if (is_zero_ether_addr(dp->mac)) + eth_hw_addr_inherit(dev, conduit); + + /* If live-changing, we need to install the user device address to the + * port FDB and the conduit interface. + */ + if (dev->flags & IFF_UP) { + err = dsa_user_host_uc_install(dev, dev->dev_addr); + if (err) { + NL_SET_ERR_MSG_MOD(extack, + "Failed to install host UC address"); + goto rewind_addr_inherit; + } + } + dsa_user_sync_ha(dev); if (vlan_filtering) { @@ -1500,10 +1524,26 @@ rewind_new_vlan: rewind_new_addrs: dsa_user_unsync_ha(dev); + if (dev->flags & IFF_UP) + dsa_user_host_uc_uninstall(dev); + +rewind_addr_inherit: + if (is_zero_ether_addr(dp->mac)) + eth_hw_addr_inherit(dev, old_conduit); + dsa_port_assign_conduit(dp, old_conduit, NULL, false); /* Restore the objects on the old CPU port */ rewind_old_addrs: + if (dev->flags & IFF_UP) { + tmp = dsa_user_host_uc_install(dev, dev->dev_addr); + if (tmp) { + dev_err(ds->dev, + "port %d failed to restore host UC address: %pe\n", + dp->index, ERR_PTR(tmp)); + } + } + dsa_user_sync_ha(dev); if (vlan_filtering) { @@ -1549,21 +1589,6 @@ dsa_port_phylink_mac_select_pcs(struct phylink_config *config, return pcs; } -static int dsa_port_phylink_mac_prepare(struct phylink_config *config, - unsigned int mode, - phy_interface_t interface) -{ - struct dsa_port *dp = dsa_phylink_to_port(config); - struct dsa_switch *ds = dp->ds; - int err = 0; - - if (ds->ops->phylink_mac_prepare) - err = ds->ops->phylink_mac_prepare(ds, dp->index, mode, - interface); - - return err; -} - static void dsa_port_phylink_mac_config(struct phylink_config *config, unsigned int mode, const struct phylink_link_state *state) @@ -1577,21 +1602,6 @@ static void dsa_port_phylink_mac_config(struct phylink_config *config, ds->ops->phylink_mac_config(ds, dp->index, mode, state); } -static int dsa_port_phylink_mac_finish(struct phylink_config *config, - unsigned int mode, - phy_interface_t interface) -{ - struct dsa_port *dp = dsa_phylink_to_port(config); - struct dsa_switch *ds = dp->ds; - int err = 0; - - if (ds->ops->phylink_mac_finish) - err = ds->ops->phylink_mac_finish(ds, dp->index, mode, - interface); - - return err; -} - static void dsa_port_phylink_mac_link_down(struct phylink_config *config, unsigned int mode, phy_interface_t interface) @@ -1624,9 +1634,7 @@ static void dsa_port_phylink_mac_link_up(struct phylink_config *config, static const struct phylink_mac_ops dsa_port_phylink_mac_ops = { .mac_select_pcs = dsa_port_phylink_mac_select_pcs, - .mac_prepare = dsa_port_phylink_mac_prepare, .mac_config = dsa_port_phylink_mac_config, - .mac_finish = dsa_port_phylink_mac_finish, .mac_link_down = dsa_port_phylink_mac_link_down, .mac_link_up = dsa_port_phylink_mac_link_up, }; diff --git a/net/dsa/user.c b/net/dsa/user.c index 867c5fe9a4dafbb4796b5521bc3ed1a503d2fecf..e8f56a40b6141500dd0e3b26716579c6618e7e66 100644 --- a/net/dsa/user.c +++ b/net/dsa/user.c @@ -355,60 +355,82 @@ static int dsa_user_get_iflink(const struct net_device *dev) return READ_ONCE(dsa_user_to_conduit(dev)->ifindex); } -static int dsa_user_open(struct net_device *dev) +int dsa_user_host_uc_install(struct net_device *dev, const u8 *addr) { struct net_device *conduit = dsa_user_to_conduit(dev); struct dsa_port *dp = dsa_user_to_port(dev); struct dsa_switch *ds = dp->ds; int err; - err = dev_open(conduit, NULL); - if (err < 0) { - netdev_err(dev, "failed to open conduit %s\n", conduit->name); - goto out; - } - if (dsa_switch_supports_uc_filtering(ds)) { - err = dsa_port_standalone_host_fdb_add(dp, dev->dev_addr, 0); + err = dsa_port_standalone_host_fdb_add(dp, addr, 0); if (err) goto out; } - if (!ether_addr_equal(dev->dev_addr, conduit->dev_addr)) { - err = dev_uc_add(conduit, dev->dev_addr); + if (!ether_addr_equal(addr, conduit->dev_addr)) { + err = dev_uc_add(conduit, addr); if (err < 0) goto del_host_addr; } - err = dsa_port_enable_rt(dp, dev->phydev); - if (err) - goto del_unicast; - return 0; -del_unicast: - if (!ether_addr_equal(dev->dev_addr, conduit->dev_addr)) - dev_uc_del(conduit, dev->dev_addr); del_host_addr: if (dsa_switch_supports_uc_filtering(ds)) - dsa_port_standalone_host_fdb_del(dp, dev->dev_addr, 0); + dsa_port_standalone_host_fdb_del(dp, addr, 0); out: return err; } -static int dsa_user_close(struct net_device *dev) +void dsa_user_host_uc_uninstall(struct net_device *dev) { struct net_device *conduit = dsa_user_to_conduit(dev); struct dsa_port *dp = dsa_user_to_port(dev); struct dsa_switch *ds = dp->ds; - dsa_port_disable_rt(dp); - if (!ether_addr_equal(dev->dev_addr, conduit->dev_addr)) dev_uc_del(conduit, dev->dev_addr); if (dsa_switch_supports_uc_filtering(ds)) dsa_port_standalone_host_fdb_del(dp, dev->dev_addr, 0); +} + +static int dsa_user_open(struct net_device *dev) +{ + struct net_device *conduit = dsa_user_to_conduit(dev); + struct dsa_port *dp = dsa_user_to_port(dev); + int err; + + err = dev_open(conduit, NULL); + if (err < 0) { + netdev_err(dev, "failed to open conduit %s\n", conduit->name); + goto out; + } + + err = dsa_user_host_uc_install(dev, dev->dev_addr); + if (err) + goto out; + + err = dsa_port_enable_rt(dp, dev->phydev); + if (err) + goto out_del_host_uc; + + return 0; + +out_del_host_uc: + dsa_user_host_uc_uninstall(dev); +out: + return err; +} + +static int dsa_user_close(struct net_device *dev) +{ + struct dsa_port *dp = dsa_user_to_port(dev); + + dsa_port_disable_rt(dp); + + dsa_user_host_uc_uninstall(dev); return 0; } @@ -448,7 +470,6 @@ static void dsa_user_set_rx_mode(struct net_device *dev) static int dsa_user_set_mac_address(struct net_device *dev, void *a) { - struct net_device *conduit = dsa_user_to_conduit(dev); struct dsa_port *dp = dsa_user_to_port(dev); struct dsa_switch *ds = dp->ds; struct sockaddr *addr = a; @@ -470,34 +491,16 @@ static int dsa_user_set_mac_address(struct net_device *dev, void *a) if (!(dev->flags & IFF_UP)) goto out_change_dev_addr; - if (dsa_switch_supports_uc_filtering(ds)) { - err = dsa_port_standalone_host_fdb_add(dp, addr->sa_data, 0); - if (err) - return err; - } - - if (!ether_addr_equal(addr->sa_data, conduit->dev_addr)) { - err = dev_uc_add(conduit, addr->sa_data); - if (err < 0) - goto del_unicast; - } + err = dsa_user_host_uc_install(dev, addr->sa_data); + if (err) + return err; - if (!ether_addr_equal(dev->dev_addr, conduit->dev_addr)) - dev_uc_del(conduit, dev->dev_addr); - - if (dsa_switch_supports_uc_filtering(ds)) - dsa_port_standalone_host_fdb_del(dp, dev->dev_addr, 0); + dsa_user_host_uc_uninstall(dev); out_change_dev_addr: eth_hw_addr_set(dev, addr->sa_data); return 0; - -del_unicast: - if (dsa_switch_supports_uc_filtering(ds)) - dsa_port_standalone_host_fdb_del(dp, addr->sa_data, 0); - - return err; } struct dsa_user_dump_ctx { @@ -2879,12 +2882,6 @@ int dsa_user_change_conduit(struct net_device *dev, struct net_device *conduit, ERR_PTR(err)); } - /* If the port doesn't have its own MAC address and relies on the DSA - * conduit's one, inherit it again from the new DSA conduit. - */ - if (is_zero_ether_addr(dp->mac)) - eth_hw_addr_inherit(dev, conduit); - return 0; out_revert_conduit_link: diff --git a/net/dsa/user.h b/net/dsa/user.h index 996069130beae81afdf5184e87677c302786b6f7..016884bead3cd7c147b91d4c3e902e926a917b04 100644 --- a/net/dsa/user.h +++ b/net/dsa/user.h @@ -42,6 +42,8 @@ int dsa_user_suspend(struct net_device *user_dev); int dsa_user_resume(struct net_device *user_dev); int dsa_user_register_notifier(void); void dsa_user_unregister_notifier(void); +int dsa_user_host_uc_install(struct net_device *dev, const u8 *addr); +void dsa_user_host_uc_uninstall(struct net_device *dev); void dsa_user_sync_ha(struct net_device *dev); void dsa_user_unsync_ha(struct net_device *dev); void dsa_user_setup_tagger(struct net_device *user); diff --git a/net/ethernet/eth.c b/net/ethernet/eth.c index 049c3adeb85044ac78e5adf7dcfb389d21e75652..4e3651101b8660064a29a24b74263c1d522c9d01 100644 --- a/net/ethernet/eth.c +++ b/net/ethernet/eth.c @@ -161,9 +161,7 @@ __be16 eth_type_trans(struct sk_buff *skb, struct net_device *dev) skb->dev = dev; skb_reset_mac_header(skb); - eth = (struct ethhdr *)skb->data; - skb_pull_inline(skb, ETH_HLEN); - + eth = eth_skb_pull_mac(skb); eth_skb_pkt_type(skb, dev); /* diff --git a/net/ethtool/cabletest.c b/net/ethtool/cabletest.c index 06a151165c31ce59928d4a4769b5fce2749c53c7..f6f136ec7ddfc4efd293544b25d1c2f64df20fa0 100644 --- a/net/ethtool/cabletest.c +++ b/net/ethtool/cabletest.c @@ -207,10 +207,6 @@ err: } EXPORT_SYMBOL_GPL(ethnl_cable_test_fault_length); -struct cable_test_tdr_req_info { - struct ethnl_req_info base; -}; - static const struct nla_policy cable_test_tdr_act_cfg_policy[] = { [ETHTOOL_A_CABLE_TEST_TDR_CFG_FIRST] = { .type = NLA_U32 }, [ETHTOOL_A_CABLE_TEST_TDR_CFG_LAST] = { .type = NLA_U32 }, diff --git a/net/ethtool/ioctl.c b/net/ethtool/ioctl.c index 5a55270aa86e8804f467bd094f8bca9171bc2617..e645d751a5e8998064ef5fa239d465f66c044e6f 100644 --- a/net/ethtool/ioctl.c +++ b/net/ethtool/ioctl.c @@ -2220,7 +2220,7 @@ static int ethtool_get_phy_stats_ethtool(struct net_device *dev, const struct ethtool_ops *ops = dev->ethtool_ops; int n_stats, ret; - if (!ops || !ops->get_sset_count || ops->get_ethtool_phy_stats) + if (!ops || !ops->get_sset_count || !ops->get_ethtool_phy_stats) return -EOPNOTSUPP; n_stats = ops->get_sset_count(dev, ETH_SS_PHY_STATS); diff --git a/net/ethtool/tsinfo.c b/net/ethtool/tsinfo.c index be2755c8d8fde7a2e74225aed4a276a39d72d0f1..57d496287e523797b05f85fc26dca111dd701c6a 100644 --- a/net/ethtool/tsinfo.c +++ b/net/ethtool/tsinfo.c @@ -38,11 +38,11 @@ static int tsinfo_prepare_data(const struct ethnl_req_info *req_base, ret = ethnl_ops_begin(dev); if (ret < 0) return ret; - if (req_base->flags & ETHTOOL_FLAG_STATS && - dev->ethtool_ops->get_ts_stats) { + if (req_base->flags & ETHTOOL_FLAG_STATS) { ethtool_stats_init((u64 *)&data->stats, sizeof(data->stats) / sizeof(u64)); - dev->ethtool_ops->get_ts_stats(dev, &data->stats); + if (dev->ethtool_ops->get_ts_stats) + dev->ethtool_ops->get_ts_stats(dev, &data->stats); } ret = __ethtool_get_ts_info(dev, &data->ts_info); ethnl_ops_complete(dev); diff --git a/net/ieee802154/6lowpan/reassembly.c b/net/ieee802154/6lowpan/reassembly.c index 56ef873828f40e9d4681d67f6a7b67b5c11b6e95..867d637d86f08887b8312ebb2c6fc64da42d7df4 100644 --- a/net/ieee802154/6lowpan/reassembly.c +++ b/net/ieee802154/6lowpan/reassembly.c @@ -130,7 +130,7 @@ static int lowpan_frag_queue(struct lowpan_frag_queue *fq, goto err; fq->q.stamp = skb->tstamp; - fq->q.mono_delivery_time = skb->mono_delivery_time; + fq->q.tstamp_type = skb->tstamp_type; if (frag_type == LOWPAN_DISPATCH_FRAG1) fq->q.flags |= INET_FRAG_FIRST_IN; diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c index e03ba4a21c39048b2e78547a0815c50d2a620849..b24d74616637a0e5fc1183cad927fe7c862104c2 100644 --- a/net/ipv4/af_inet.c +++ b/net/ipv4/af_inet.c @@ -1532,7 +1532,7 @@ struct sk_buff *inet_gro_receive(struct list_head *head, struct sk_buff *skb) } NAPI_GRO_CB(skb)->flush |= flush; - NAPI_GRO_CB(skb)->inner_network_offset = off; + NAPI_GRO_CB(skb)->network_offsets[NAPI_GRO_CB(skb)->encap_mark] = off; /* Note : No need to call skb_gro_postpull_rcsum() here, * as we already checked checksum over ipv4 header was 0 diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index 18227757ec0cebea5847bb636733445580a1385d..3f88d0961e5b2f40994b9d2cfaedb87f5103fab3 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -260,17 +260,17 @@ static int bpf_tcp_ca_check_member(const struct btf_type *t, return 0; } -static int bpf_tcp_ca_reg(void *kdata) +static int bpf_tcp_ca_reg(void *kdata, struct bpf_link *link) { return tcp_register_congestion_control(kdata); } -static void bpf_tcp_ca_unreg(void *kdata) +static void bpf_tcp_ca_unreg(void *kdata, struct bpf_link *link) { tcp_unregister_congestion_control(kdata); } -static int bpf_tcp_ca_update(void *kdata, void *old_kdata) +static int bpf_tcp_ca_update(void *kdata, void *old_kdata, struct bpf_link *link) { return tcp_update_congestion_control(kdata, old_kdata); } diff --git a/net/ipv4/cipso_ipv4.c b/net/ipv4/cipso_ipv4.c index dd6d4601505806c79f008db69a98c0aff20eb516..3a95c0f13ce33a29d0289373255a9c8302d63edb 100644 --- a/net/ipv4/cipso_ipv4.c +++ b/net/ipv4/cipso_ipv4.c @@ -1953,7 +1953,7 @@ int cipso_v4_req_setattr(struct request_sock *req, buf = NULL; req_inet = inet_rsk(req); - opt = xchg((__force struct ip_options_rcu **)&req_inet->ireq_opt, opt); + opt = unrcu_pointer(xchg(&req_inet->ireq_opt, RCU_INITIALIZER(opt))); if (opt) kfree_rcu(opt, rcu); diff --git a/net/ipv4/devinet.c b/net/ipv4/devinet.c index 96accde527da2484417b14d96ca0222ef5f67c2f..d09f557eaa7790baf83cf567bf7874d32f010de0 100644 --- a/net/ipv4/devinet.c +++ b/net/ipv4/devinet.c @@ -1887,10 +1887,11 @@ static int inet_dump_ifaddr(struct sk_buff *skb, struct netlink_callback *cb) goto done; if (fillargs.ifindex) { - err = -ENODEV; dev = dev_get_by_index_rcu(tgt_net, fillargs.ifindex); - if (!dev) + if (!dev) { + err = -ENODEV; goto done; + } in_dev = __in_dev_get_rcu(dev); if (!in_dev) goto done; @@ -1902,7 +1903,7 @@ static int inet_dump_ifaddr(struct sk_buff *skb, struct netlink_callback *cb) cb->seq = inet_base_seq(tgt_net); - for_each_netdev_dump(net, dev, ctx->ifindex) { + for_each_netdev_dump(tgt_net, dev, ctx->ifindex) { in_dev = __in_dev_get_rcu(dev); if (!in_dev) continue; @@ -2804,7 +2805,7 @@ void __init devinet_init(void) rtnl_register(PF_INET, RTM_NEWADDR, inet_rtm_newaddr, NULL, 0); rtnl_register(PF_INET, RTM_DELADDR, inet_rtm_deladdr, NULL, 0); rtnl_register(PF_INET, RTM_GETADDR, NULL, inet_dump_ifaddr, - RTNL_FLAG_DUMP_UNLOCKED); + RTNL_FLAG_DUMP_UNLOCKED | RTNL_FLAG_DUMP_SPLIT_NLM_DONE); rtnl_register(PF_INET, RTM_GETNETCONF, inet_netconf_get_devconf, inet_netconf_dump_devconf, RTNL_FLAG_DOIT_UNLOCKED | RTNL_FLAG_DUMP_UNLOCKED); diff --git a/net/ipv4/fib_frontend.c b/net/ipv4/fib_frontend.c index c484b1c0fc00a79a45a1c3e7fde230ce59cb67a3..7ad2cafb927634fd60935c36ad68ded45e52dbab 100644 --- a/net/ipv4/fib_frontend.c +++ b/net/ipv4/fib_frontend.c @@ -1050,11 +1050,6 @@ next: e++; } } - - /* Don't let NLM_DONE coalesce into a message, even if it could. - * Some user space expects NLM_DONE in a separate recv(). - */ - err = skb->len; out: cb->args[1] = e; @@ -1665,5 +1660,5 @@ void __init ip_fib_init(void) rtnl_register(PF_INET, RTM_NEWROUTE, inet_rtm_newroute, NULL, 0); rtnl_register(PF_INET, RTM_DELROUTE, inet_rtm_delroute, NULL, 0); rtnl_register(PF_INET, RTM_GETROUTE, NULL, inet_dump_fib, - RTNL_FLAG_DUMP_UNLOCKED); + RTNL_FLAG_DUMP_UNLOCKED | RTNL_FLAG_DUMP_SPLIT_NLM_DONE); } diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c index f669da98d11d8f2a1182e6e84d98c2b92f774a88..7b6b042208bd24ab95b944199d6c2a16c69ee870 100644 --- a/net/ipv4/fib_semantics.c +++ b/net/ipv4/fib_semantics.c @@ -1030,7 +1030,7 @@ bool fib_metrics_match(struct fib_config *cfg, struct fib_info *fi) bool ecn_ca = false; nla_strscpy(tmp, nla, sizeof(tmp)); - val = tcp_ca_get_key_by_name(fi->fib_net, tmp, &ecn_ca); + val = tcp_ca_get_key_by_name(tmp, &ecn_ca); } else { if (nla_len(nla) != sizeof(u32)) return false; @@ -1459,8 +1459,7 @@ struct fib_info *fib_create_info(struct fib_config *cfg, fi = kzalloc(struct_size(fi, fib_nh, nhs), GFP_KERNEL); if (!fi) goto failure; - fi->fib_metrics = ip_fib_metrics_init(fi->fib_net, cfg->fc_mx, - cfg->fc_mx_len, extack); + fi->fib_metrics = ip_fib_metrics_init(cfg->fc_mx, cfg->fc_mx_len, extack); if (IS_ERR(fi->fib_metrics)) { err = PTR_ERR(fi->fib_metrics); kfree(fi); diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c index d81f74ce0f02e6a9432df49339b9a32a87965624..7ced569778ab44796fe883e48c78c8cccd6a909d 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -911,6 +911,64 @@ int inet_rtx_syn_ack(const struct sock *parent, struct request_sock *req) } EXPORT_SYMBOL(inet_rtx_syn_ack); +static struct request_sock * +reqsk_alloc_noprof(const struct request_sock_ops *ops, struct sock *sk_listener, + bool attach_listener) +{ + struct request_sock *req; + + req = kmem_cache_alloc_noprof(ops->slab, GFP_ATOMIC | __GFP_NOWARN); + if (!req) + return NULL; + req->rsk_listener = NULL; + if (attach_listener) { + if (unlikely(!refcount_inc_not_zero(&sk_listener->sk_refcnt))) { + kmem_cache_free(ops->slab, req); + return NULL; + } + req->rsk_listener = sk_listener; + } + req->rsk_ops = ops; + req_to_sk(req)->sk_prot = sk_listener->sk_prot; + sk_node_init(&req_to_sk(req)->sk_node); + sk_tx_queue_clear(req_to_sk(req)); + req->saved_syn = NULL; + req->syncookie = 0; + req->timeout = 0; + req->num_timeout = 0; + req->num_retrans = 0; + req->sk = NULL; + refcount_set(&req->rsk_refcnt, 0); + + return req; +} +#define reqsk_alloc(...) alloc_hooks(reqsk_alloc_noprof(__VA_ARGS__)) + +struct request_sock *inet_reqsk_alloc(const struct request_sock_ops *ops, + struct sock *sk_listener, + bool attach_listener) +{ + struct request_sock *req = reqsk_alloc(ops, sk_listener, + attach_listener); + + if (req) { + struct inet_request_sock *ireq = inet_rsk(req); + + ireq->ireq_opt = NULL; +#if IS_ENABLED(CONFIG_IPV6) + ireq->pktopts = NULL; +#endif + atomic64_set(&ireq->ir_cookie, 0); + ireq->ireq_state = TCP_NEW_SYN_RECV; + write_pnet(&ireq->ireq_net, sock_net(sk_listener)); + ireq->ireq_family = sk_listener->sk_family; + req->timeout = TCP_TIMEOUT_INIT; + } + + return req; +} +EXPORT_SYMBOL(inet_reqsk_alloc); + static struct request_sock *inet_reqsk_clone(struct request_sock *req, struct sock *sk) { diff --git a/net/ipv4/inet_fragment.c b/net/ipv4/inet_fragment.c index faaec92a46ac03e1fd4d1dfaf9a7fa080148ca2b..d179a2c8422276f99d6e8be599a36b50e1163380 100644 --- a/net/ipv4/inet_fragment.c +++ b/net/ipv4/inet_fragment.c @@ -619,7 +619,7 @@ void inet_frag_reasm_finish(struct inet_frag_queue *q, struct sk_buff *head, skb_mark_not_on_list(head); head->prev = NULL; head->tstamp = q->stamp; - head->mono_delivery_time = q->mono_delivery_time; + head->tstamp_type = q->tstamp_type; if (sk) refcount_add(sum_truesize - head_truesize, &sk->sk_wmem_alloc); diff --git a/net/ipv4/inet_timewait_sock.c b/net/ipv4/inet_timewait_sock.c index e28075f0006e333897ad379ebc8c87fc3f9643bd..337390ba85b4082701f78f1a0913ba47c1741378 100644 --- a/net/ipv4/inet_timewait_sock.c +++ b/net/ipv4/inet_timewait_sock.c @@ -92,13 +92,22 @@ static void inet_twsk_add_node_rcu(struct inet_timewait_sock *tw, hlist_nulls_add_head_rcu(&tw->tw_node, list); } +static void inet_twsk_schedule(struct inet_timewait_sock *tw, int timeo) +{ + __inet_twsk_schedule(tw, timeo, false); +} + /* - * Enter the time wait state. This is called with locally disabled BH. + * Enter the time wait state. * Essentially we whip up a timewait bucket, copy the relevant info into it * from the SK, and mess with hash chains and list linkage. + * + * The caller must not access @tw anymore after this function returns. */ -void inet_twsk_hashdance(struct inet_timewait_sock *tw, struct sock *sk, - struct inet_hashinfo *hashinfo) +void inet_twsk_hashdance_schedule(struct inet_timewait_sock *tw, + struct sock *sk, + struct inet_hashinfo *hashinfo, + int timeo) { const struct inet_sock *inet = inet_sk(sk); const struct inet_connection_sock *icsk = inet_csk(sk); @@ -114,6 +123,7 @@ void inet_twsk_hashdance(struct inet_timewait_sock *tw, struct sock *sk, hashinfo->bhash_size)]; bhead2 = inet_bhashfn_portaddr(hashinfo, sk, twsk_net(tw), inet->inet_num); + local_bh_disable(); spin_lock(&bhead->lock); spin_lock(&bhead2->lock); @@ -129,26 +139,34 @@ void inet_twsk_hashdance(struct inet_timewait_sock *tw, struct sock *sk, spin_lock(lock); + /* Step 2: Hash TW into tcp ehash chain */ inet_twsk_add_node_rcu(tw, &ehead->chain); /* Step 3: Remove SK from hash chain */ if (__sk_nulls_del_node_init_rcu(sk)) sock_prot_inuse_add(sock_net(sk), sk->sk_prot, -1); - spin_unlock(lock); + /* Ensure above writes are committed into memory before updating the + * refcount. + * Provides ordering vs later refcount_inc(). + */ + smp_wmb(); /* tw_refcnt is set to 3 because we have : * - one reference for bhash chain. * - one reference for ehash chain. * - one reference for timer. - * We can use atomic_set() because prior spin_lock()/spin_unlock() - * committed into memory all tw fields. * Also note that after this point, we lost our implicit reference * so we are not allowed to use tw anymore. */ refcount_set(&tw->tw_refcnt, 3); + + inet_twsk_schedule(tw, timeo); + + spin_unlock(lock); + local_bh_enable(); } -EXPORT_SYMBOL_GPL(inet_twsk_hashdance); +EXPORT_SYMBOL_GPL(inet_twsk_hashdance_schedule); static void tw_timer_handler(struct timer_list *t) { @@ -192,7 +210,7 @@ struct inet_timewait_sock *inet_twsk_alloc(const struct sock *sk, tw->tw_prot = sk->sk_prot_creator; atomic64_set(&tw->tw_cookie, atomic64_read(&sk->sk_cookie)); twsk_net_set(tw, sock_net(sk)); - timer_setup(&tw->tw_timer, tw_timer_handler, TIMER_PINNED); + timer_setup(&tw->tw_timer, tw_timer_handler, 0); /* * Because we use RCU lookups, we should not set tw_refcnt * to a non null value before everything is setup for this @@ -217,7 +235,34 @@ EXPORT_SYMBOL_GPL(inet_twsk_alloc); */ void inet_twsk_deschedule_put(struct inet_timewait_sock *tw) { - if (del_timer_sync(&tw->tw_timer)) + struct inet_hashinfo *hashinfo = tw->tw_dr->hashinfo; + spinlock_t *lock = inet_ehash_lockp(hashinfo, tw->tw_hash); + + /* inet_twsk_purge() walks over all sockets, including tw ones, + * and removes them via inet_twsk_deschedule_put() after a + * refcount_inc_not_zero(). + * + * inet_twsk_hashdance_schedule() must (re)init the refcount before + * arming the timer, i.e. inet_twsk_purge can obtain a reference to + * a twsk that did not yet schedule the timer. + * + * The ehash lock synchronizes these two: + * After acquiring the lock, the timer is always scheduled (else + * timer_shutdown returns false), because hashdance_schedule releases + * the ehash lock only after completing the timer initialization. + * + * Without grabbing the ehash lock, we get: + * 1) cpu x sets twsk refcount to 3 + * 2) cpu y bumps refcount to 4 + * 3) cpu y calls inet_twsk_deschedule_put() and shuts timer down + * 4) cpu x tries to start timer, but mod_timer is a noop post-shutdown + * -> timer refcount is never decremented. + */ + spin_lock(lock); + /* Makes sure hashdance_schedule() has completed */ + spin_unlock(lock); + + if (timer_shutdown_sync(&tw->tw_timer)) inet_twsk_kill(tw); inet_twsk_put(tw); } diff --git a/net/ipv4/ip_fragment.c b/net/ipv4/ip_fragment.c index 08e2c92e25ab5626a0fb84bbe39c4e4d85cf41db..a92664a5ef2efe0c2ced9984886d865f9067f721 100644 --- a/net/ipv4/ip_fragment.c +++ b/net/ipv4/ip_fragment.c @@ -355,7 +355,7 @@ static int ip_frag_queue(struct ipq *qp, struct sk_buff *skb) qp->iif = dev->ifindex; qp->q.stamp = skb->tstamp; - qp->q.mono_delivery_time = skb->mono_delivery_time; + qp->q.tstamp_type = skb->tstamp_type; qp->q.meat += skb->len; qp->ecn |= ecn; add_frag_mem_limit(qp->q.fqdir, skb->truesize); diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 9500031a1f55b7d5f988d4dca51b71c687da0398..b90d0f78ac8080f6e5aba2382b11075253f85d61 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -764,7 +764,7 @@ int ip_do_fragment(struct net *net, struct sock *sk, struct sk_buff *skb, { struct iphdr *iph; struct sk_buff *skb2; - bool mono_delivery_time = skb->mono_delivery_time; + u8 tstamp_type = skb->tstamp_type; struct rtable *rt = skb_rtable(skb); unsigned int mtu, hlen, ll_rs; struct ip_fraglist_iter iter; @@ -856,7 +856,7 @@ int ip_do_fragment(struct net *net, struct sock *sk, struct sk_buff *skb, } } - skb_set_delivery_time(skb, tstamp, mono_delivery_time); + skb_set_delivery_time(skb, tstamp, tstamp_type); err = output(net, sk, skb); if (!err) @@ -912,7 +912,7 @@ slow_path: /* * Put this fragment into the sending queue. */ - skb_set_delivery_time(skb2, tstamp, mono_delivery_time); + skb_set_delivery_time(skb2, tstamp, tstamp_type); err = output(net, sk, skb2); if (err) goto fail; @@ -1457,7 +1457,10 @@ struct sk_buff *__ip_make_skb(struct sock *sk, skb->priority = (cork->tos != -1) ? cork->priority: READ_ONCE(sk->sk_priority); skb->mark = cork->mark; - skb->tstamp = cork->transmit_time; + if (sk_is_tcp(sk)) + skb_set_delivery_time(skb, cork->transmit_time, SKB_CLOCK_MONOTONIC); + else + skb_set_delivery_type_by_clockid(skb, cork->transmit_time, sk->sk_clockid); /* * Steal rt from cork.dst to avoid a pair of atomic_inc/atomic_dec * on dst refcount @@ -1649,7 +1652,8 @@ void ip_send_unicast_reply(struct sock *sk, struct sk_buff *skb, arg->csumoffset) = csum_fold(csum_add(nskb->csum, arg->csum)); nskb->ip_summed = CHECKSUM_NONE; - nskb->mono_delivery_time = !!transmit_time; + if (transmit_time) + nskb->tstamp_type = SKB_CLOCK_MONOTONIC; if (txhash) skb_set_hash(nskb, txhash, PKT_HASH_TYPE_L4); ip_push_pending_frames(sk, &fl4); diff --git a/net/ipv4/metrics.c b/net/ipv4/metrics.c index 0e3ee1532848c8f49e0a342b7c8ecc1c27684e67..8ddac1f595ed8cc6077e027f53db7db3c7387843 100644 --- a/net/ipv4/metrics.c +++ b/net/ipv4/metrics.c @@ -7,7 +7,7 @@ #include #include -static int ip_metrics_convert(struct net *net, struct nlattr *fc_mx, +static int ip_metrics_convert(struct nlattr *fc_mx, int fc_mx_len, u32 *metrics, struct netlink_ext_ack *extack) { @@ -31,7 +31,7 @@ static int ip_metrics_convert(struct net *net, struct nlattr *fc_mx, char tmp[TCP_CA_NAME_MAX]; nla_strscpy(tmp, nla, sizeof(tmp)); - val = tcp_ca_get_key_by_name(net, tmp, &ecn_ca); + val = tcp_ca_get_key_by_name(tmp, &ecn_ca); if (val == TCP_CA_UNSPEC) { NL_SET_ERR_MSG(extack, "Unknown tcp congestion algorithm"); return -EINVAL; @@ -63,7 +63,7 @@ static int ip_metrics_convert(struct net *net, struct nlattr *fc_mx, return 0; } -struct dst_metrics *ip_fib_metrics_init(struct net *net, struct nlattr *fc_mx, +struct dst_metrics *ip_fib_metrics_init(struct nlattr *fc_mx, int fc_mx_len, struct netlink_ext_ack *extack) { @@ -77,7 +77,7 @@ struct dst_metrics *ip_fib_metrics_init(struct net *net, struct nlattr *fc_mx, if (unlikely(!fib_metrics)) return ERR_PTR(-ENOMEM); - err = ip_metrics_convert(net, fc_mx, fc_mx_len, fib_metrics->metrics, + err = ip_metrics_convert(fc_mx, fc_mx_len, fib_metrics->metrics, extack); if (!err) { refcount_set(&fib_metrics->refcnt, 1); diff --git a/net/ipv4/netfilter/nf_tproxy_ipv4.c b/net/ipv4/netfilter/nf_tproxy_ipv4.c index 69e331799604304078367011040b0dea69fcacf0..73e66a088e25eb179569d88a0362452b7d06ccd2 100644 --- a/net/ipv4/netfilter/nf_tproxy_ipv4.c +++ b/net/ipv4/netfilter/nf_tproxy_ipv4.c @@ -58,6 +58,8 @@ __be32 nf_tproxy_laddr4(struct sk_buff *skb, __be32 user_laddr, __be32 daddr) laddr = 0; indev = __in_dev_get_rcu(skb->dev); + if (!indev) + return daddr; in_dev_for_each_ifa_rcu(ifa, indev) { if (ifa->ifa_flags & IFA_F_SECONDARY) diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c index 4cb43401e0e06c5003c268c28bf0882ac4e8b4e0..1a095365035645e9370f3eb94219f88f552c9e60 100644 --- a/net/ipv4/raw.c +++ b/net/ipv4/raw.c @@ -360,7 +360,7 @@ static int raw_send_hdrinc(struct sock *sk, struct flowi4 *fl4, skb->protocol = htons(ETH_P_IP); skb->priority = READ_ONCE(sk->sk_priority); skb->mark = sockc->mark; - skb->tstamp = sockc->transmit_time; + skb_set_delivery_type_by_clockid(skb, sockc->transmit_time, sk->sk_clockid); skb_dst_set(skb, &rt->dst); *rtp = NULL; diff --git a/net/ipv4/route.c b/net/ipv4/route.c index 5fd54103174f72d24d8015ad69029cebdd50740f..cb0bdf34ed50c92688a3c0fe14c3e0c06d78b47c 100644 --- a/net/ipv4/route.c +++ b/net/ipv4/route.c @@ -129,7 +129,8 @@ struct dst_entry *ipv4_dst_check(struct dst_entry *dst, u32 cookie); static unsigned int ipv4_default_advmss(const struct dst_entry *dst); INDIRECT_CALLABLE_SCOPE unsigned int ipv4_mtu(const struct dst_entry *dst); -static struct dst_entry *ipv4_negative_advice(struct dst_entry *dst); +static void ipv4_negative_advice(struct sock *sk, + struct dst_entry *dst); static void ipv4_link_failure(struct sk_buff *skb); static void ip_rt_update_pmtu(struct dst_entry *dst, struct sock *sk, struct sk_buff *skb, u32 mtu, @@ -825,22 +826,15 @@ static void ip_do_redirect(struct dst_entry *dst, struct sock *sk, struct sk_buf __ip_do_redirect(rt, skb, &fl4, true); } -static struct dst_entry *ipv4_negative_advice(struct dst_entry *dst) +static void ipv4_negative_advice(struct sock *sk, + struct dst_entry *dst) { struct rtable *rt = dst_rtable(dst); - struct dst_entry *ret = dst; - if (rt) { - if (dst->obsolete > 0) { - ip_rt_put(rt); - ret = NULL; - } else if ((rt->rt_flags & RTCF_REDIRECTED) || - rt->dst.expires) { - ip_rt_put(rt); - ret = NULL; - } - } - return ret; + if ((dst->obsolete > 0) || + (rt->rt_flags & RTCF_REDIRECTED) || + rt->dst.expires) + sk_dst_reset(sk); } /* @@ -1487,7 +1481,6 @@ static bool rt_cache_route(struct fib_nh_common *nhc, struct rtable *rt) struct uncached_list { spinlock_t lock; struct list_head head; - struct list_head quarantine; }; static DEFINE_PER_CPU_ALIGNED(struct uncached_list, rt_uncached_list); @@ -1538,7 +1531,7 @@ void rt_flush_dev(struct net_device *dev) rt->dst.dev = blackhole_netdev; netdev_ref_replace(dev, blackhole_netdev, &rt->dst.dev_tracker, GFP_ATOMIC); - list_move(&rt->dst.rt_uncached, &ul->quarantine); + list_del_init(&rt->dst.rt_uncached); } spin_unlock_bh(&ul->lock); } @@ -3667,7 +3660,6 @@ int __init ip_rt_init(void) struct uncached_list *ul = &per_cpu(rt_uncached_list, cpu); INIT_LIST_HEAD(&ul->head); - INIT_LIST_HEAD(&ul->quarantine); spin_lock_init(&ul->lock); } #ifdef CONFIG_IP_ROUTE_CLASSID diff --git a/net/ipv4/sysctl_net_ipv4.c b/net/ipv4/sysctl_net_ipv4.c index 162a0a3b6ba52b6c5893dc403c615f407ae9bbf3..bb64c0ef092dbf839eba2d9c2d21a8e57b8e7b6d 100644 --- a/net/ipv4/sysctl_net_ipv4.c +++ b/net/ipv4/sysctl_net_ipv4.c @@ -130,7 +130,8 @@ static int ipv4_privileged_ports(struct ctl_table *table, int write, return ret; } -static void inet_get_ping_group_range_table(struct ctl_table *table, kgid_t *low, kgid_t *high) +static void inet_get_ping_group_range_table(const struct ctl_table *table, + kgid_t *low, kgid_t *high) { kgid_t *data = table->data; struct net *net = @@ -145,7 +146,8 @@ static void inet_get_ping_group_range_table(struct ctl_table *table, kgid_t *low } /* Update system visible IP port range */ -static void set_ping_group_range(struct ctl_table *table, kgid_t low, kgid_t high) +static void set_ping_group_range(const struct ctl_table *table, + kgid_t low, kgid_t high) { kgid_t *data = table->data; struct net *net = @@ -1501,6 +1503,14 @@ static struct ctl_table ipv4_net_table[] = { .proc_handler = proc_dou8vec_minmax, .extra1 = SYSCTL_ONE, }, + { + .procname = "tcp_rto_min_us", + .data = &init_net.ipv4.sysctl_tcp_rto_min_us, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = SYSCTL_ONE, + }, }; static __net_init int ipv4_sysctl_init_net(struct net *net) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 681b54e1f3a64387787738ab6495531b8abe1771..6553221694ec282a498c66f7028a8b862fae0e66 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -420,6 +420,7 @@ void tcp_init_sock(struct sock *sk) { struct inet_connection_sock *icsk = inet_csk(sk); struct tcp_sock *tp = tcp_sk(sk); + int rto_min_us; tp->out_of_order_queue = RB_ROOT; sk->tcp_rtx_queue = RB_ROOT; @@ -428,7 +429,8 @@ void tcp_init_sock(struct sock *sk) INIT_LIST_HEAD(&tp->tsorted_sent_queue); icsk->icsk_rto = TCP_TIMEOUT_INIT; - icsk->icsk_rto_min = TCP_RTO_MIN; + rto_min_us = READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_rto_min_us); + icsk->icsk_rto_min = usecs_to_jiffies(rto_min_us); icsk->icsk_delack_max = TCP_DELACK_MAX; tp->mdev_us = jiffies_to_usecs(TCP_TIMEOUT_INIT); minmax_reset(&tp->rtt_min, tcp_jiffies32, ~0U); @@ -598,7 +600,7 @@ __poll_t tcp_poll(struct file *file, struct socket *sock, poll_table *wait) */ mask |= EPOLLOUT | EPOLLWRNORM; } - /* This barrier is coupled with smp_wmb() in tcp_reset() */ + /* This barrier is coupled with smp_wmb() in tcp_done_with_error() */ smp_rmb(); if (READ_ONCE(sk->sk_err) || !skb_queue_empty_lockless(&sk->sk_error_queue)) @@ -1165,6 +1167,9 @@ new_segment: process_backlog++; +#ifdef CONFIG_SKB_DECRYPTED + skb->decrypted = !!(flags & MSG_SENDPAGE_DECRYPTED); +#endif tcp_skb_entail(sk, skb); copy = size_goal; @@ -2646,6 +2651,10 @@ void tcp_set_state(struct sock *sk, int state) if (oldstate != TCP_ESTABLISHED) TCP_INC_STATS(sock_net(sk), TCP_MIB_CURRESTAB); break; + case TCP_CLOSE_WAIT: + if (oldstate == TCP_SYN_RECV) + TCP_INC_STATS(sock_net(sk), TCP_MIB_CURRESTAB); + break; case TCP_CLOSE: if (oldstate == TCP_CLOSE_WAIT || oldstate == TCP_ESTABLISHED) @@ -2657,7 +2666,7 @@ void tcp_set_state(struct sock *sk, int state) inet_put_port(sk); fallthrough; default: - if (oldstate == TCP_ESTABLISHED) + if (oldstate == TCP_ESTABLISHED || oldstate == TCP_CLOSE_WAIT) TCP_DEC_STATS(sock_net(sk), TCP_MIB_CURRESTAB); } @@ -3079,7 +3088,7 @@ int tcp_disconnect(struct sock *sk, int flags) icsk->icsk_ack.rcv_mss = TCP_MIN_MSS; memset(&tp->rx_opt, 0, sizeof(tp->rx_opt)); __sk_dst_reset(sk); - dst_release(xchg((__force struct dst_entry **)&sk->sk_rx_dst, NULL)); + dst_release(unrcu_pointer(xchg(&sk->sk_rx_dst, NULL))); tcp_saved_syn_free(tp); tp->compressed_ack = 0; tp->segs_in = 0; @@ -4576,14 +4585,10 @@ int tcp_abort(struct sock *sk, int err) bh_lock_sock(sk); if (!sock_flag(sk, SOCK_DEAD)) { - WRITE_ONCE(sk->sk_err, err); - /* This barrier is coupled with smp_rmb() in tcp_poll() */ - smp_wmb(); - sk_error_report(sk); if (tcp_need_reset(sk->sk_state)) tcp_send_active_reset(sk, GFP_ATOMIC, SK_RST_REASON_NOT_SPECIFIED); - tcp_done(sk); + tcp_done_with_error(sk, err); } bh_unlock_sock(sk); diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c index 781b67a525719a42f21b713eb424427670d7afb2..37c42b63ff993466b52c5eea7270312149ca913b 100644 --- a/net/ipv4/tcp_ao.c +++ b/net/ipv4/tcp_ao.c @@ -933,6 +933,7 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb, struct tcp_ao_key *key; __be32 sisn, disn; u8 *traffic_key; + int state; u32 sne = 0; info = rcu_dereference(tcp_sk(sk)->ao_info); @@ -948,8 +949,9 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb, disn = 0; } + state = READ_ONCE(sk->sk_state); /* Fast-path */ - if (likely((1 << sk->sk_state) & TCP_AO_ESTABLISHED)) { + if (likely((1 << state) & TCP_AO_ESTABLISHED)) { enum skb_drop_reason err; struct tcp_ao_key *current_key; @@ -988,6 +990,9 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb, return SKB_NOT_DROPPED_YET; } + if (unlikely(state == TCP_CLOSE)) + return SKB_DROP_REASON_TCP_CLOSE; + /* Lookup key based on peer address and keyid. * current_key and rnext_key must not be used on tcp listen * sockets as otherwise: @@ -1001,7 +1006,7 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb, if (th->syn && !th->ack) goto verify_hash; - if ((1 << sk->sk_state) & (TCPF_LISTEN | TCPF_NEW_SYN_RECV)) { + if ((1 << state) & (TCPF_LISTEN | TCPF_NEW_SYN_RECV)) { /* Make the initial syn the likely case here */ if (unlikely(req)) { sne = tcp_ao_compute_sne(0, tcp_rsk(req)->rcv_isn, @@ -1018,14 +1023,14 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb, /* no way to figure out initial sisn/disn - drop */ return SKB_DROP_REASON_TCP_FLAGS; } - } else if ((1 << sk->sk_state) & (TCPF_SYN_SENT | TCPF_SYN_RECV)) { + } else if ((1 << state) & (TCPF_SYN_SENT | TCPF_SYN_RECV)) { disn = info->lisn; if (th->syn || th->rst) sisn = th->seq; else sisn = info->risn; } else { - WARN_ONCE(1, "TCP-AO: Unexpected sk_state %d", sk->sk_state); + WARN_ONCE(1, "TCP-AO: Unexpected sk_state %d", state); return SKB_DROP_REASON_TCP_AOFAILURE; } verify_hash: diff --git a/net/ipv4/tcp_cong.c b/net/ipv4/tcp_cong.c index 28ffcfbeef14e5bb57e62f1610cdf52caaafba89..48617d99abb0d4be003913e40d7a69d95db0138d 100644 --- a/net/ipv4/tcp_cong.c +++ b/net/ipv4/tcp_cong.c @@ -46,8 +46,7 @@ void tcp_set_ca_state(struct sock *sk, const u8 ca_state) } /* Must be called with rcu lock held */ -static struct tcp_congestion_ops *tcp_ca_find_autoload(struct net *net, - const char *name) +static struct tcp_congestion_ops *tcp_ca_find_autoload(const char *name) { struct tcp_congestion_ops *ca = tcp_ca_find(name); @@ -178,7 +177,7 @@ int tcp_update_congestion_control(struct tcp_congestion_ops *ca, struct tcp_cong return ret; } -u32 tcp_ca_get_key_by_name(struct net *net, const char *name, bool *ecn_ca) +u32 tcp_ca_get_key_by_name(const char *name, bool *ecn_ca) { const struct tcp_congestion_ops *ca; u32 key = TCP_CA_UNSPEC; @@ -186,7 +185,7 @@ u32 tcp_ca_get_key_by_name(struct net *net, const char *name, bool *ecn_ca) might_sleep(); rcu_read_lock(); - ca = tcp_ca_find_autoload(net, name); + ca = tcp_ca_find_autoload(name); if (ca) { key = ca->key; *ecn_ca = ca->flags & TCP_CONG_NEEDS_ECN; @@ -283,7 +282,7 @@ int tcp_set_default_congestion_control(struct net *net, const char *name) int ret; rcu_read_lock(); - ca = tcp_ca_find_autoload(net, name); + ca = tcp_ca_find_autoload(name); if (!ca) { ret = -ENOENT; } else if (!bpf_try_module_get(ca, ca->owner)) { @@ -421,7 +420,7 @@ int tcp_set_congestion_control(struct sock *sk, const char *name, bool load, if (!load) ca = tcp_ca_find(name); else - ca = tcp_ca_find_autoload(sock_net(sk), name); + ca = tcp_ca_find_autoload(name); /* No change asking for existing value */ if (ca == icsk->icsk_ca_ops) { diff --git a/net/ipv4/tcp_fastopen.c b/net/ipv4/tcp_fastopen.c index 8ed54e7334a9c646dfbbc6dc41b9ef11b925de0a..0f523cbfe329efeaee2ef206b0779e9911ef22cd 100644 --- a/net/ipv4/tcp_fastopen.c +++ b/net/ipv4/tcp_fastopen.c @@ -49,7 +49,7 @@ void tcp_fastopen_ctx_destroy(struct net *net) { struct tcp_fastopen_context *ctxt; - ctxt = xchg((__force struct tcp_fastopen_context **)&net->ipv4.tcp_fastopen_ctx, NULL); + ctxt = unrcu_pointer(xchg(&net->ipv4.tcp_fastopen_ctx, NULL)); if (ctxt) call_rcu(&ctxt->rcu, tcp_fastopen_ctx_free); @@ -80,9 +80,10 @@ int tcp_fastopen_reset_cipher(struct net *net, struct sock *sk, if (sk) { q = &inet_csk(sk)->icsk_accept_queue.fastopenq; - octx = xchg((__force struct tcp_fastopen_context **)&q->ctx, ctx); + octx = unrcu_pointer(xchg(&q->ctx, RCU_INITIALIZER(ctx))); } else { - octx = xchg((__force struct tcp_fastopen_context **)&net->ipv4.tcp_fastopen_ctx, ctx); + octx = unrcu_pointer(xchg(&net->ipv4.tcp_fastopen_ctx, + RCU_INITIALIZER(ctx))); } if (octx) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 9c04a9c8be9dfaa0ec2437b3748284e57588b216..eb187450e4d7666e6ff84f10be13d00f4400a25a 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -4436,9 +4436,26 @@ static enum skb_drop_reason tcp_sequence(const struct tcp_sock *tp, return SKB_NOT_DROPPED_YET; } + +void tcp_done_with_error(struct sock *sk, int err) +{ + /* This barrier is coupled with smp_rmb() in tcp_poll() */ + WRITE_ONCE(sk->sk_err, err); + smp_wmb(); + + tcp_write_queue_purge(sk); + tcp_done(sk); + + if (!sock_flag(sk, SOCK_DEAD)) + sk_error_report(sk); +} +EXPORT_SYMBOL(tcp_done_with_error); + /* When we get a reset we do this. */ void tcp_reset(struct sock *sk, struct sk_buff *skb) { + int err; + trace_tcp_receive_reset(sk); /* mptcp can't tell us to ignore reset pkts, @@ -4450,24 +4467,17 @@ void tcp_reset(struct sock *sk, struct sk_buff *skb) /* We want the right error as BSD sees it (and indeed as we do). */ switch (sk->sk_state) { case TCP_SYN_SENT: - WRITE_ONCE(sk->sk_err, ECONNREFUSED); + err = ECONNREFUSED; break; case TCP_CLOSE_WAIT: - WRITE_ONCE(sk->sk_err, EPIPE); + err = EPIPE; break; case TCP_CLOSE: return; default: - WRITE_ONCE(sk->sk_err, ECONNRESET); + err = ECONNRESET; } - /* This barrier is coupled with smp_rmb() in tcp_poll() */ - smp_wmb(); - - tcp_write_queue_purge(sk); - tcp_done(sk); - - if (!sock_flag(sk, SOCK_DEAD)) - sk_error_report(sk); + tcp_done_with_error(sk, err); } /* @@ -4803,10 +4813,7 @@ static bool tcp_try_coalesce(struct sock *sk, if (TCP_SKB_CB(from)->seq != TCP_SKB_CB(to)->end_seq) return false; - if (!mptcp_skb_can_collapse(to, from)) - return false; - - if (skb_cmp_decrypted(from, to)) + if (!tcp_skb_can_collapse_rx(to, from)) return false; if (!skb_try_coalesce(to, from, fragstolen, &delta)) @@ -5362,7 +5369,7 @@ restart: break; } - if (n && n != tail && mptcp_skb_can_collapse(skb, n) && + if (n && n != tail && tcp_skb_can_collapse_rx(skb, n) && TCP_SKB_CB(skb)->end_seq != TCP_SKB_CB(n)->seq) { end_of_skbs = false; break; @@ -5413,11 +5420,9 @@ restart: skb = tcp_collapse_one(sk, skb, list, root); if (!skb || skb == tail || - !mptcp_skb_can_collapse(nskb, skb) || + !tcp_skb_can_collapse_rx(nskb, skb) || (TCP_SKB_CB(skb)->tcp_flags & (TCPHDR_SYN | TCPHDR_FIN))) goto end; - if (skb_cmp_decrypted(skb, nskb)) - goto end; } } } @@ -6981,31 +6986,6 @@ static void tcp_openreq_init(struct request_sock *req, #endif } -struct request_sock *inet_reqsk_alloc(const struct request_sock_ops *ops, - struct sock *sk_listener, - bool attach_listener) -{ - struct request_sock *req = reqsk_alloc(ops, sk_listener, - attach_listener); - - if (req) { - struct inet_request_sock *ireq = inet_rsk(req); - - ireq->ireq_opt = NULL; -#if IS_ENABLED(CONFIG_IPV6) - ireq->pktopts = NULL; -#endif - atomic64_set(&ireq->ir_cookie, 0); - ireq->ireq_state = TCP_NEW_SYN_RECV; - write_pnet(&ireq->ireq_net, sock_net(sk_listener)); - ireq->ireq_family = sk_listener->sk_family; - req->timeout = TCP_TIMEOUT_INIT; - } - - return req; -} -EXPORT_SYMBOL(inet_reqsk_alloc); - /* * Return true if a syncookie should be sent */ diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 30ef0c8f5e92d301c31ea1a05f662c1fc4cf37af..e61c7c97474548363959b43bd75c13d96ea17ad8 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -114,6 +114,7 @@ int tcp_twsk_unique(struct sock *sk, struct sock *sktw, void *twp) const struct inet_timewait_sock *tw = inet_twsk(sktw); const struct tcp_timewait_sock *tcptw = tcp_twsk(sktw); struct tcp_sock *tp = tcp_sk(sk); + int ts_recent_stamp; if (reuse == 2) { /* Still does not detect *everything* that goes through @@ -152,10 +153,11 @@ int tcp_twsk_unique(struct sock *sk, struct sock *sktw, void *twp) If TW bucket has been already destroyed we fall back to VJ's scheme and use initial timestamp retrieved from peer table. */ - if (tcptw->tw_ts_recent_stamp && + ts_recent_stamp = READ_ONCE(tcptw->tw_ts_recent_stamp); + if (ts_recent_stamp && (!twp || (reuse && time_after32(ktime_get_seconds(), - tcptw->tw_ts_recent_stamp)))) { - /* inet_twsk_hashdance() sets sk_refcnt after putting twsk + ts_recent_stamp)))) { + /* inet_twsk_hashdance_schedule() sets sk_refcnt after putting twsk * and releasing the bucket lock. */ if (unlikely(!refcount_inc_not_zero(&sktw->sk_refcnt))) @@ -178,8 +180,8 @@ int tcp_twsk_unique(struct sock *sk, struct sock *sktw, void *twp) if (!seq) seq = 1; WRITE_ONCE(tp->write_seq, seq); - tp->rx_opt.ts_recent = tcptw->tw_ts_recent; - tp->rx_opt.ts_recent_stamp = tcptw->tw_ts_recent_stamp; + tp->rx_opt.ts_recent = READ_ONCE(tcptw->tw_ts_recent); + tp->rx_opt.ts_recent_stamp = ts_recent_stamp; } return 1; @@ -611,15 +613,10 @@ int tcp_v4_err(struct sk_buff *skb, u32 info) ip_icmp_error(sk, skb, err, th->dest, info, (u8 *)th); - if (!sock_owned_by_user(sk)) { - WRITE_ONCE(sk->sk_err, err); - - sk_error_report(sk); - - tcp_done(sk); - } else { + if (!sock_owned_by_user(sk)) + tcp_done_with_error(sk, err); + else WRITE_ONCE(sk->sk_err_soft, err); - } goto out; } @@ -1069,7 +1066,7 @@ static void tcp_v4_timewait_ack(struct sock *sk, struct sk_buff *skb) tcptw->tw_snd_nxt, tcptw->tw_rcv_nxt, tcptw->tw_rcv_wnd >> tw->tw_rcv_wscale, tcp_tw_tsval(tcptw), - tcptw->tw_ts_recent, + READ_ONCE(tcptw->tw_ts_recent), tw->tw_bound_dev_if, &key, tw->tw_transparent ? IP_REPLY_ARG_NOSRCCHECK : 0, tw->tw_tos, @@ -1144,14 +1141,9 @@ static void tcp_v4_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb, #endif } - /* RFC 7323 2.3 - * The window field (SEG.WND) of every outgoing segment, with the - * exception of segments, MUST be right-shifted by - * Rcv.Wind.Shift bits: - */ tcp_v4_send_ack(sk, skb, seq, tcp_rsk(req)->rcv_nxt, - req->rsk_rcv_wnd >> inet_rsk(req)->rcv_wscale, + tcp_synack_window(req) >> inet_rsk(req)->rcv_wscale, tcp_rsk_tsval(tcp_rsk(req)), READ_ONCE(req->ts_recent), 0, &key, @@ -2054,8 +2046,7 @@ bool tcp_add_backlog(struct sock *sk, struct sk_buff *skb, TCP_SKB_CB(skb)->tcp_flags) & TCPHDR_ACK) || ((TCP_SKB_CB(tail)->tcp_flags ^ TCP_SKB_CB(skb)->tcp_flags) & (TCPHDR_ECE | TCPHDR_CWR)) || - !mptcp_skb_can_collapse(tail, skb) || - skb_cmp_decrypted(tail, skb) || + !tcp_skb_can_collapse_rx(tail, skb) || thtail->doff != th->doff || memcmp(thtail + 1, th + 1, hdrlen - sizeof(*th))) goto no_coalesce; @@ -3511,6 +3502,7 @@ static int __net_init tcp_sk_init(struct net *net) net->ipv4.sysctl_tcp_shrink_window = 0; net->ipv4.sysctl_tcp_pingpong_thresh = 1; + net->ipv4.sysctl_tcp_rto_min_us = jiffies_to_usecs(TCP_RTO_MIN); return 0; } @@ -3625,6 +3617,8 @@ void __init tcp_v4_init(void) */ inet_sk(sk)->pmtudisc = IP_PMTUDISC_DO; + sk->sk_clockid = CLOCK_MONOTONIC; + per_cpu(ipv4_tcp_sk, cpu) = sk; } if (register_pernet_subsys(&tcp_sk_ops)) diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c index b93619b2384b3735ecb6e40238f8367d9afb7e15..bc67f6b9efae46f738f782464dedee1ea0c0cd32 100644 --- a/net/ipv4/tcp_minisocks.c +++ b/net/ipv4/tcp_minisocks.c @@ -101,16 +101,18 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb, struct tcp_options_received tmp_opt; struct tcp_timewait_sock *tcptw = tcp_twsk((struct sock *)tw); bool paws_reject = false; + int ts_recent_stamp; tmp_opt.saw_tstamp = 0; - if (th->doff > (sizeof(*th) >> 2) && tcptw->tw_ts_recent_stamp) { + ts_recent_stamp = READ_ONCE(tcptw->tw_ts_recent_stamp); + if (th->doff > (sizeof(*th) >> 2) && ts_recent_stamp) { tcp_parse_options(twsk_net(tw), skb, &tmp_opt, 0, NULL); if (tmp_opt.saw_tstamp) { if (tmp_opt.rcv_tsecr) tmp_opt.rcv_tsecr -= tcptw->tw_ts_offset; - tmp_opt.ts_recent = tcptw->tw_ts_recent; - tmp_opt.ts_recent_stamp = tcptw->tw_ts_recent_stamp; + tmp_opt.ts_recent = READ_ONCE(tcptw->tw_ts_recent); + tmp_opt.ts_recent_stamp = ts_recent_stamp; paws_reject = tcp_paws_reject(&tmp_opt, th->rst); } } @@ -152,8 +154,10 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb, twsk_rcv_nxt_update(tcptw, TCP_SKB_CB(skb)->end_seq); if (tmp_opt.saw_tstamp) { - tcptw->tw_ts_recent_stamp = ktime_get_seconds(); - tcptw->tw_ts_recent = tmp_opt.rcv_tsval; + WRITE_ONCE(tcptw->tw_ts_recent_stamp, + ktime_get_seconds()); + WRITE_ONCE(tcptw->tw_ts_recent, + tmp_opt.rcv_tsval); } inet_twsk_reschedule(tw, TCP_TIMEWAIT_LEN); @@ -197,8 +201,10 @@ kill: } if (tmp_opt.saw_tstamp) { - tcptw->tw_ts_recent = tmp_opt.rcv_tsval; - tcptw->tw_ts_recent_stamp = ktime_get_seconds(); + WRITE_ONCE(tcptw->tw_ts_recent, + tmp_opt.rcv_tsval); + WRITE_ONCE(tcptw->tw_ts_recent_stamp, + ktime_get_seconds()); } inet_twsk_put(tw); @@ -225,7 +231,7 @@ kill: if (th->syn && !th->rst && !th->ack && !paws_reject && (after(TCP_SKB_CB(skb)->seq, tcptw->tw_rcv_nxt) || (tmp_opt.saw_tstamp && - (s32)(tcptw->tw_ts_recent - tmp_opt.rcv_tsval) < 0))) { + (s32)(READ_ONCE(tcptw->tw_ts_recent) - tmp_opt.rcv_tsval) < 0))) { u32 isn = tcptw->tw_snd_nxt + 65535 + 2; if (isn == 0) isn++; @@ -339,17 +345,10 @@ void tcp_time_wait(struct sock *sk, int state, int timeo) if (state == TCP_TIME_WAIT) timeo = TCP_TIMEWAIT_LEN; - /* tw_timer is pinned, so we need to make sure BH are disabled - * in following section, otherwise timer handler could run before - * we complete the initialization. - */ - local_bh_disable(); - inet_twsk_schedule(tw, timeo); /* Linkage updates. * Note that access to tw after this point is illegal. */ - inet_twsk_hashdance(tw, sk, net->ipv4.tcp_death_row.hashinfo); - local_bh_enable(); + inet_twsk_hashdance_schedule(tw, sk, net->ipv4.tcp_death_row.hashinfo, timeo); } else { /* Sorry, if we're out of memory, just CLOSE this * socket up. We've got bigger problems than @@ -783,8 +782,11 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb, /* RFC793: "first check sequence number". */ - if (paws_reject || !tcp_in_window(TCP_SKB_CB(skb)->seq, TCP_SKB_CB(skb)->end_seq, - tcp_rsk(req)->rcv_nxt, tcp_rsk(req)->rcv_nxt + req->rsk_rcv_wnd)) { + if (paws_reject || !tcp_in_window(TCP_SKB_CB(skb)->seq, + TCP_SKB_CB(skb)->end_seq, + tcp_rsk(req)->rcv_nxt, + tcp_rsk(req)->rcv_nxt + + tcp_synack_window(req))) { /* Out of window: send ACK and drop. */ if (!(flg & TCP_FLAG_RST) && !tcp_oow_rate_limited(sock_net(sk), skb, diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 95618d0e78e495dd66a5852b214f8381caaa5fe6..090fb0c24599680c17a70b5c81fc9bc69d9be6a0 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -1301,7 +1301,7 @@ static int __tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, tp = tcp_sk(sk); prior_wstamp = tp->tcp_wstamp_ns; tp->tcp_wstamp_ns = max(tp->tcp_wstamp_ns, tp->tcp_clock_cache); - skb_set_delivery_time(skb, tp->tcp_wstamp_ns, true); + skb_set_delivery_time(skb, tp->tcp_wstamp_ns, SKB_CLOCK_MONOTONIC); if (clone_it) { oskb = skb; @@ -1655,7 +1655,7 @@ int tcp_fragment(struct sock *sk, enum tcp_queue tcp_queue, skb_split(skb, buff, len); - skb_set_delivery_time(buff, skb->tstamp, true); + skb_set_delivery_time(buff, skb->tstamp, SKB_CLOCK_MONOTONIC); tcp_fragment_tstamp(skb, buff); old_factor = tcp_skb_pcount(skb); @@ -2764,7 +2764,7 @@ static bool tcp_write_xmit(struct sock *sk, unsigned int mss_now, int nonagle, if (unlikely(tp->repair) && tp->repair_queue == TCP_SEND_QUEUE) { /* "skb_mstamp_ns" is used as a start point for the retransmit timer */ tp->tcp_wstamp_ns = tp->tcp_clock_cache; - skb_set_delivery_time(skb, tp->tcp_wstamp_ns, true); + skb_set_delivery_time(skb, tp->tcp_wstamp_ns, SKB_CLOCK_MONOTONIC); list_move_tail(&skb->tcp_tsorted_anchor, &tp->tsorted_sent_queue); tcp_init_tso_segs(skb, mss_now); goto repair; /* Skip network transmission */ @@ -3752,11 +3752,11 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst, #ifdef CONFIG_SYN_COOKIES if (unlikely(synack_type == TCP_SYNACK_COOKIE && ireq->tstamp_ok)) skb_set_delivery_time(skb, cookie_init_timestamp(req, now), - true); + SKB_CLOCK_MONOTONIC); else #endif { - skb_set_delivery_time(skb, now, true); + skb_set_delivery_time(skb, now, SKB_CLOCK_MONOTONIC); if (!tcp_rsk(req)->snt_synack) /* Timestamp first SYNACK */ tcp_rsk(req)->snt_synack = tcp_skb_timestamp_us(skb); } @@ -3843,7 +3843,7 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst, bpf_skops_write_hdr_opt((struct sock *)sk, skb, req, syn_skb, synack_type, &opts); - skb_set_delivery_time(skb, now, true); + skb_set_delivery_time(skb, now, SKB_CLOCK_MONOTONIC); tcp_add_tx_delay(skb, tp); return skb; @@ -4027,7 +4027,7 @@ static int tcp_send_syn_data(struct sock *sk, struct sk_buff *syn) err = tcp_transmit_skb(sk, syn_data, 1, sk->sk_allocation); - skb_set_delivery_time(syn, syn_data->skb_mstamp_ns, true); + skb_set_delivery_time(syn, syn_data->skb_mstamp_ns, SKB_CLOCK_MONOTONIC); /* Now full SYN+DATA was cloned and sent (or not), * remove the SYN from the original skb (syn_data) @@ -4163,16 +4163,9 @@ EXPORT_SYMBOL(tcp_connect); u32 tcp_delack_max(const struct sock *sk) { - const struct dst_entry *dst = __sk_dst_get(sk); - u32 delack_max = inet_csk(sk)->icsk_delack_max; - - if (dst && dst_metric_locked(dst, RTAX_RTO_MIN)) { - u32 rto_min = dst_metric_rtt(dst, RTAX_RTO_MIN); - u32 delack_from_rto_min = max_t(int, 1, rto_min - 1); + u32 delack_from_rto_min = max(tcp_rto_min(sk), 2) - 1; - delack_max = min_t(u32, delack_max, delack_from_rto_min); - } - return delack_max; + return min(inet_csk(sk)->icsk_delack_max, delack_from_rto_min); } /* Send out a delayed ack, the caller does the policy checking diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c index 83fe7f62f7f10ab111512a3ef15a97a04c79cb4a..3e8604ae7d06c5b010a2034e3295675a7d358f13 100644 --- a/net/ipv4/tcp_timer.c +++ b/net/ipv4/tcp_timer.c @@ -74,11 +74,7 @@ u32 tcp_clamp_probe0_to_user_timeout(const struct sock *sk, u32 when) static void tcp_write_err(struct sock *sk) { - WRITE_ONCE(sk->sk_err, READ_ONCE(sk->sk_err_soft) ? : ETIMEDOUT); - sk_error_report(sk); - - tcp_write_queue_purge(sk); - tcp_done(sk); + tcp_done_with_error(sk, READ_ONCE(sk->sk_err_soft) ? : ETIMEDOUT); __NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONTIMEOUT); } diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index 189c9113fe9a180cc2788632892cbd12a829a500..c9ca6d285347b8c3032e2a779b39aa426bd6ee4a 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -2230,7 +2230,7 @@ bool udp_sk_rx_dst_set(struct sock *sk, struct dst_entry *dst) struct dst_entry *old; if (dst_hold_safe(dst)) { - old = xchg((__force struct dst_entry **)&sk->sk_rx_dst, dst); + old = unrcu_pointer(xchg(&sk->sk_rx_dst, RCU_INITIALIZER(dst))); dst_release(old); return old != dst; } diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c index 5c424a0e7232faa854f5b661319b8f7f182a60b8..1e69756d53d9c0dc775bde19a085c03e73c4871e 100644 --- a/net/ipv6/addrconf.c +++ b/net/ipv6/addrconf.c @@ -863,7 +863,7 @@ static void addrconf_forward_change(struct net *net, __s32 newf) } } -static int addrconf_fixup_forwarding(struct ctl_table *table, int *p, int newf) +static int addrconf_fixup_forwarding(const struct ctl_table *table, int *p, int newf) { struct net *net; int old; @@ -931,7 +931,7 @@ static void addrconf_linkdown_change(struct net *net, __s32 newf) } } -static int addrconf_fixup_linkdown(struct ctl_table *table, int *p, int newf) +static int addrconf_fixup_linkdown(const struct ctl_table *table, int *p, int newf) { struct net *net; int old; @@ -6378,7 +6378,7 @@ static void addrconf_disable_change(struct net *net, __s32 newf) } } -static int addrconf_disable_ipv6(struct ctl_table *table, int *p, int newf) +static int addrconf_disable_ipv6(const struct ctl_table *table, int *p, int newf) { struct net *net = (struct net *)table->extra2; int old; @@ -6669,7 +6669,7 @@ void addrconf_disable_policy_idev(struct inet6_dev *idev, int val) } static -int addrconf_disable_policy(struct ctl_table *ctl, int *valp, int val) +int addrconf_disable_policy(const struct ctl_table *ctl, int *valp, int val) { struct net *net = (struct net *)ctl->extra2; struct inet6_dev *idev; diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c index 8041dc181bd42e5e1af2d9e9fe6af50057e5b58f..e03fb9a1dbeb46081b50e016064bdf296002e092 100644 --- a/net/ipv6/af_inet6.c +++ b/net/ipv6/af_inet6.c @@ -509,7 +509,7 @@ void inet6_cleanup_sock(struct sock *sk) /* Free tx options */ - opt = xchg((__force struct ipv6_txoptions **)&np->opt, NULL); + opt = unrcu_pointer(xchg(&np->opt, NULL)); if (opt) { atomic_sub(opt->tot_len, &sk->sk_omem_alloc); txopt_put(opt); diff --git a/net/ipv6/ila/ila_lwt.c b/net/ipv6/ila/ila_lwt.c index 0601bad798221389fe83318fbb17f192cec880d4..ff7e734e335b06f03c4c8815163a706136e42a3e 100644 --- a/net/ipv6/ila/ila_lwt.c +++ b/net/ipv6/ila/ila_lwt.c @@ -58,7 +58,9 @@ static int ila_output(struct net *net, struct sock *sk, struct sk_buff *skb) return orig_dst->lwtstate->orig_output(net, sk, skb); } + local_bh_disable(); dst = dst_cache_get(&ilwt->dst_cache); + local_bh_enable(); if (unlikely(!dst)) { struct ipv6hdr *ip6h = ipv6_hdr(skb); struct flowi6 fl6; @@ -86,8 +88,11 @@ static int ila_output(struct net *net, struct sock *sk, struct sk_buff *skb) goto drop; } - if (ilwt->connected) + if (ilwt->connected) { + local_bh_disable(); dst_cache_set_ip6(&ilwt->dst_cache, dst, &fl6.saddr); + local_bh_enable(); + } } skb_dst_set(skb, dst); diff --git a/net/ipv6/ioam6_iptunnel.c b/net/ipv6/ioam6_iptunnel.c index 7563f8c6aa87cf9f7841ee78dcea2a16f60ac344..bf7120ecea1ebe834e70073710be0c1692d7ad1d 100644 --- a/net/ipv6/ioam6_iptunnel.c +++ b/net/ipv6/ioam6_iptunnel.c @@ -351,9 +351,9 @@ do_encap: goto drop; if (!ipv6_addr_equal(&orig_daddr, &ipv6_hdr(skb)->daddr)) { - preempt_disable(); + local_bh_disable(); dst = dst_cache_get(&ilwt->cache); - preempt_enable(); + local_bh_enable(); if (unlikely(!dst)) { struct ipv6hdr *hdr = ipv6_hdr(skb); @@ -373,9 +373,9 @@ do_encap: goto drop; } - preempt_disable(); + local_bh_disable(); dst_cache_set_ip6(&ilwt->cache, dst, &fl6.saddr); - preempt_enable(); + local_bh_enable(); } skb_dst_drop(skb); diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c index 31d77885bcae3e3843b6d486cfc21cdbe709bcf0..d9c9a542a414b6cfc82517f80a38bc0b4cbef68d 100644 --- a/net/ipv6/ip6_fib.c +++ b/net/ipv6/ip6_fib.c @@ -966,6 +966,7 @@ static void __fib6_drop_pcpu_from(struct fib6_nh *fib6_nh, if (!fib6_nh->rt6i_pcpu) return; + rcu_read_lock(); /* release the reference to this fib entry from * all of its cached pcpu routes */ @@ -974,7 +975,9 @@ static void __fib6_drop_pcpu_from(struct fib6_nh *fib6_nh, struct rt6_info *pcpu_rt; ppcpu_rt = per_cpu_ptr(fib6_nh->rt6i_pcpu, cpu); - pcpu_rt = *ppcpu_rt; + + /* Paired with xchg() in rt6_get_pcpu_route() */ + pcpu_rt = READ_ONCE(*ppcpu_rt); /* only dropping the 'from' reference if the cached route * is using 'match'. The cached pcpu_rt->from only changes @@ -984,10 +987,11 @@ static void __fib6_drop_pcpu_from(struct fib6_nh *fib6_nh, if (pcpu_rt && rcu_access_pointer(pcpu_rt->from) == match) { struct fib6_info *from; - from = xchg((__force struct fib6_info **)&pcpu_rt->from, NULL); + from = unrcu_pointer(xchg(&pcpu_rt->from, NULL)); fib6_info_release(from); } } + rcu_read_unlock(); } struct fib6_nh_pcpu_arg { diff --git a/net/ipv6/ip6_offload.c b/net/ipv6/ip6_offload.c index bd5aff97d8b1d821329f067d15dd5143844612ce..9822163428b028d6dbf3f48abe4674bd6c581725 100644 --- a/net/ipv6/ip6_offload.c +++ b/net/ipv6/ip6_offload.c @@ -236,7 +236,7 @@ INDIRECT_CALLABLE_SCOPE struct sk_buff *ipv6_gro_receive(struct list_head *head, if (unlikely(!iph)) goto out; - NAPI_GRO_CB(skb)->inner_network_offset = off; + NAPI_GRO_CB(skb)->network_offsets[NAPI_GRO_CB(skb)->encap_mark] = off; flush += ntohs(iph->payload_len) != skb->len - hlen; diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 27d8725445e3532be2559ae10cb11078e59a0347..e7a19df3125e4390a401d59a3e354eee58c8da93 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -859,7 +859,7 @@ int ip6_fragment(struct net *net, struct sock *sk, struct sk_buff *skb, struct rt6_info *rt = dst_rt6_info(skb_dst(skb)); struct ipv6_pinfo *np = skb->sk && !dev_recursion_level() ? inet6_sk(skb->sk) : NULL; - bool mono_delivery_time = skb->mono_delivery_time; + u8 tstamp_type = skb->tstamp_type; struct ip6_frag_state state; unsigned int mtu, hlen, nexthdr_offset; ktime_t tstamp = skb->tstamp; @@ -955,7 +955,7 @@ int ip6_fragment(struct net *net, struct sock *sk, struct sk_buff *skb, if (iter.frag) ip6_fraglist_prepare(skb, &iter); - skb_set_delivery_time(skb, tstamp, mono_delivery_time); + skb_set_delivery_time(skb, tstamp, tstamp_type); err = output(net, sk, skb); if (!err) IP6_INC_STATS(net, ip6_dst_idev(&rt->dst), @@ -1016,7 +1016,7 @@ slow_path: /* * Put this fragment into the sending queue. */ - skb_set_delivery_time(frag, tstamp, mono_delivery_time); + skb_set_delivery_time(frag, tstamp, tstamp_type); err = output(net, sk, frag); if (err) goto fail; @@ -1924,7 +1924,10 @@ struct sk_buff *__ip6_make_skb(struct sock *sk, skb->priority = READ_ONCE(sk->sk_priority); skb->mark = cork->base.mark; - skb->tstamp = cork->base.transmit_time; + if (sk_is_tcp(sk)) + skb_set_delivery_time(skb, cork->base.transmit_time, SKB_CLOCK_MONOTONIC); + else + skb_set_delivery_type_by_clockid(skb, cork->base.transmit_time, sk->sk_clockid); ip6_cork_steal_dst(skb, cork); IP6_INC_STATS(net, rt->rt6i_idev, IPSTATS_MIB_OUTREQUESTS); diff --git a/net/ipv6/ipv6_sockglue.c b/net/ipv6/ipv6_sockglue.c index d4c28ec1bc517b33498b08b59bcda5fe960f652f..cd342d5015c6fb36bd03d4f3fcae4a3995ff6097 100644 --- a/net/ipv6/ipv6_sockglue.c +++ b/net/ipv6/ipv6_sockglue.c @@ -111,8 +111,7 @@ struct ipv6_txoptions *ipv6_update_options(struct sock *sk, icsk->icsk_sync_mss(sk, icsk->icsk_pmtu_cookie); } } - opt = xchg((__force struct ipv6_txoptions **)&inet6_sk(sk)->opt, - opt); + opt = unrcu_pointer(xchg(&inet6_sk(sk)->opt, RCU_INITIALIZER(opt))); sk_dst_reset(sk); return opt; diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c index d914b23256ce6372cc764236edcf224273f5257a..254b192c5705082d291c1c8cebad8a247a90cc2c 100644 --- a/net/ipv6/ndisc.c +++ b/net/ipv6/ndisc.c @@ -1936,7 +1936,7 @@ static struct notifier_block ndisc_netdev_notifier = { }; #ifdef CONFIG_SYSCTL -static void ndisc_warn_deprecated_sysctl(struct ctl_table *ctl, +static void ndisc_warn_deprecated_sysctl(const struct ctl_table *ctl, const char *func, const char *dev_name) { static char warncomm[TASK_COMM_LEN]; diff --git a/net/ipv6/netfilter.c b/net/ipv6/netfilter.c index 53d255838e6ab5f8ac4985c6d824966c3475ec97..e0c2347b4dc685fa986d9fda47c29a54caebb1e9 100644 --- a/net/ipv6/netfilter.c +++ b/net/ipv6/netfilter.c @@ -126,7 +126,7 @@ int br_ip6_fragment(struct net *net, struct sock *sk, struct sk_buff *skb, struct sk_buff *)) { int frag_max_size = BR_INPUT_SKB_CB(skb)->frag_max_size; - bool mono_delivery_time = skb->mono_delivery_time; + u8 tstamp_type = skb->tstamp_type; ktime_t tstamp = skb->tstamp; struct ip6_frag_state state; u8 *prevhdr, nexthdr = 0; @@ -192,7 +192,7 @@ int br_ip6_fragment(struct net *net, struct sock *sk, struct sk_buff *skb, if (iter.frag) ip6_fraglist_prepare(skb, &iter); - skb_set_delivery_time(skb, tstamp, mono_delivery_time); + skb_set_delivery_time(skb, tstamp, tstamp_type); err = output(net, sk, data, skb); if (err || !iter.frag) break; @@ -225,7 +225,7 @@ slow_path: goto blackhole; } - skb_set_delivery_time(skb2, tstamp, mono_delivery_time); + skb_set_delivery_time(skb2, tstamp, tstamp_type); err = output(net, sk, data, skb2); if (err) goto blackhole; diff --git a/net/ipv6/netfilter/nf_conntrack_reasm.c b/net/ipv6/netfilter/nf_conntrack_reasm.c index 5e1b50c6a44d238c1c5e9c9e081cd86e985ff90b..6f0844c9315d1819c0b1c48dd767148fca244b81 100644 --- a/net/ipv6/netfilter/nf_conntrack_reasm.c +++ b/net/ipv6/netfilter/nf_conntrack_reasm.c @@ -263,7 +263,7 @@ static int nf_ct_frag6_queue(struct frag_queue *fq, struct sk_buff *skb, fq->iif = dev->ifindex; fq->q.stamp = skb->tstamp; - fq->q.mono_delivery_time = skb->mono_delivery_time; + fq->q.tstamp_type = skb->tstamp_type; fq->q.meat += skb->len; fq->ecn |= ecn; if (payload_len > fq->q.max_size) diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c index 2eedf255600b9ec93cdb1f381d8e1fbf41524bfc..f838366e825672a6e95e67c116fb4c33146fbd0f 100644 --- a/net/ipv6/raw.c +++ b/net/ipv6/raw.c @@ -621,7 +621,7 @@ static int rawv6_send_hdrinc(struct sock *sk, struct msghdr *msg, int length, skb->protocol = htons(ETH_P_IPV6); skb->priority = READ_ONCE(sk->sk_priority); skb->mark = sockc->mark; - skb->tstamp = sockc->transmit_time; + skb_set_delivery_type_by_clockid(skb, sockc->transmit_time, sk->sk_clockid); skb_put(skb, length); skb_reset_network_header(skb); diff --git a/net/ipv6/reassembly.c b/net/ipv6/reassembly.c index 327caca642576b92b6f8f22ebc44081cd14822db..a48be617a8ab5e0ece9b01b427abe410cafef8bc 100644 --- a/net/ipv6/reassembly.c +++ b/net/ipv6/reassembly.c @@ -198,7 +198,7 @@ static int ip6_frag_queue(struct frag_queue *fq, struct sk_buff *skb, fq->iif = dev->ifindex; fq->q.stamp = skb->tstamp; - fq->q.mono_delivery_time = skb->mono_delivery_time; + fq->q.tstamp_type = skb->tstamp_type; fq->q.meat += skb->len; fq->ecn |= ecn; add_frag_mem_limit(fq->q.fqdir, skb->truesize); diff --git a/net/ipv6/route.c b/net/ipv6/route.c index bbc2a0dd931429e7f8c68df0df48bce6d604fb56..ad5fff5a210c0d345a44a112f6d9f69ed3058716 100644 --- a/net/ipv6/route.c +++ b/net/ipv6/route.c @@ -87,7 +87,8 @@ struct dst_entry *ip6_dst_check(struct dst_entry *dst, u32 cookie); static unsigned int ip6_default_advmss(const struct dst_entry *dst); INDIRECT_CALLABLE_SCOPE unsigned int ip6_mtu(const struct dst_entry *dst); -static struct dst_entry *ip6_negative_advice(struct dst_entry *); +static void ip6_negative_advice(struct sock *sk, + struct dst_entry *dst); static void ip6_dst_destroy(struct dst_entry *); static void ip6_dst_ifdown(struct dst_entry *, struct net_device *dev); @@ -130,7 +131,6 @@ static struct fib6_info *rt6_get_route_info(struct net *net, struct uncached_list { spinlock_t lock; struct list_head head; - struct list_head quarantine; }; static DEFINE_PER_CPU_ALIGNED(struct uncached_list, rt6_uncached_list); @@ -188,8 +188,7 @@ static void rt6_uncached_list_flush_dev(struct net_device *dev) handled = true; } if (handled) - list_move(&rt->dst.rt_uncached, - &ul->quarantine); + list_del_init(&rt->dst.rt_uncached); } spin_unlock_bh(&ul->lock); } @@ -367,7 +366,7 @@ static void ip6_dst_destroy(struct dst_entry *dst) in6_dev_put(idev); } - from = xchg((__force struct fib6_info **)&rt->from, NULL); + from = unrcu_pointer(xchg(&rt->from, NULL)); fib6_info_release(from); } @@ -1408,6 +1407,7 @@ static struct rt6_info *rt6_get_pcpu_route(const struct fib6_result *res) struct rt6_info *prev, **p; p = this_cpu_ptr(res->nh->rt6i_pcpu); + /* Paired with READ_ONCE() in __fib6_drop_pcpu_from() */ prev = xchg(p, NULL); if (prev) { dst_dev_put(&prev->dst); @@ -1436,7 +1436,7 @@ static struct rt6_info *rt6_make_pcpu_route(struct net *net, if (res->f6i->fib6_destroying) { struct fib6_info *from; - from = xchg((__force struct fib6_info **)&pcpu_rt->from, NULL); + from = unrcu_pointer(xchg(&pcpu_rt->from, NULL)); fib6_info_release(from); } @@ -1465,7 +1465,7 @@ static void rt6_remove_exception(struct rt6_exception_bucket *bucket, /* purge completely the exception to allow releasing the held resources: * some [sk] cache may keep the dst around for unlimited time */ - from = xchg((__force struct fib6_info **)&rt6_ex->rt6i->from, NULL); + from = unrcu_pointer(xchg(&rt6_ex->rt6i->from, NULL)); fib6_info_release(from); dst_dev_put(&rt6_ex->rt6i->dst); @@ -2770,24 +2770,24 @@ INDIRECT_CALLABLE_SCOPE struct dst_entry *ip6_dst_check(struct dst_entry *dst, } EXPORT_INDIRECT_CALLABLE(ip6_dst_check); -static struct dst_entry *ip6_negative_advice(struct dst_entry *dst) +static void ip6_negative_advice(struct sock *sk, + struct dst_entry *dst) { struct rt6_info *rt = dst_rt6_info(dst); - if (rt) { - if (rt->rt6i_flags & RTF_CACHE) { - rcu_read_lock(); - if (rt6_check_expired(rt)) { - rt6_remove_exception_rt(rt); - dst = NULL; - } - rcu_read_unlock(); - } else { - dst_release(dst); - dst = NULL; + if (rt->rt6i_flags & RTF_CACHE) { + rcu_read_lock(); + if (rt6_check_expired(rt)) { + /* counteract the dst_release() in sk_dst_reset() */ + dst_hold(dst); + sk_dst_reset(sk); + + rt6_remove_exception_rt(rt); } + rcu_read_unlock(); + return; } - return dst; + sk_dst_reset(sk); } static void ip6_link_failure(struct sk_buff *skb) @@ -3760,7 +3760,7 @@ static struct fib6_info *ip6_route_info_create(struct fib6_config *cfg, if (!rt) goto out; - rt->fib6_metrics = ip_fib_metrics_init(net, cfg->fc_mx, cfg->fc_mx_len, + rt->fib6_metrics = ip_fib_metrics_init(cfg->fc_mx, cfg->fc_mx_len, extack); if (IS_ERR(rt->fib6_metrics)) { err = PTR_ERR(rt->fib6_metrics); @@ -6754,7 +6754,6 @@ int __init ip6_route_init(void) struct uncached_list *ul = per_cpu_ptr(&rt6_uncached_list, cpu); INIT_LIST_HEAD(&ul->head); - INIT_LIST_HEAD(&ul->quarantine); spin_lock_init(&ul->lock); } diff --git a/net/ipv6/rpl_iptunnel.c b/net/ipv6/rpl_iptunnel.c index a013b92cbb860aa36a23f50d3d5c5963857d601c..2c83b7586422ddd2ae877f98e47698410e47b233 100644 --- a/net/ipv6/rpl_iptunnel.c +++ b/net/ipv6/rpl_iptunnel.c @@ -212,9 +212,9 @@ static int rpl_output(struct net *net, struct sock *sk, struct sk_buff *skb) if (unlikely(err)) goto drop; - preempt_disable(); + local_bh_disable(); dst = dst_cache_get(&rlwt->cache); - preempt_enable(); + local_bh_enable(); if (unlikely(!dst)) { struct ipv6hdr *hdr = ipv6_hdr(skb); @@ -234,9 +234,9 @@ static int rpl_output(struct net *net, struct sock *sk, struct sk_buff *skb) goto drop; } - preempt_disable(); + local_bh_disable(); dst_cache_set_ip6(&rlwt->cache, dst, &fl6.saddr); - preempt_enable(); + local_bh_enable(); } skb_dst_drop(skb); @@ -268,23 +268,21 @@ static int rpl_input(struct sk_buff *skb) return err; } - preempt_disable(); + local_bh_disable(); dst = dst_cache_get(&rlwt->cache); - preempt_enable(); if (!dst) { ip6_route_input(skb); dst = skb_dst(skb); if (!dst->error) { - preempt_disable(); dst_cache_set_ip6(&rlwt->cache, dst, &ipv6_hdr(skb)->saddr); - preempt_enable(); } } else { skb_dst_drop(skb); skb_dst_set(skb, dst); } + local_bh_enable(); err = skb_cow_head(skb, LL_RESERVED_SPACE(dst->dev)); if (unlikely(err)) diff --git a/net/ipv6/seg6.c b/net/ipv6/seg6.c index a31521e270f785691140d7ecfebcc3d472f690b2..180da19c148c1d4a24ca9b117b4ad60037cc066c 100644 --- a/net/ipv6/seg6.c +++ b/net/ipv6/seg6.c @@ -21,9 +21,7 @@ #include #include #include -#ifdef CONFIG_IPV6_SEG6_HMAC #include -#endif bool seg6_validate_srh(struct ipv6_sr_hdr *srh, int len, bool reduced) { @@ -437,13 +435,11 @@ static int __net_init seg6_net_init(struct net *net) net->ipv6.seg6_data = sdata; -#ifdef CONFIG_IPV6_SEG6_HMAC if (seg6_hmac_net_init(net)) { kfree(rcu_dereference_raw(sdata->tun_src)); kfree(sdata); return -ENOMEM; } -#endif return 0; } @@ -452,9 +448,7 @@ static void __net_exit seg6_net_exit(struct net *net) { struct seg6_pernet_data *sdata = seg6_pernet(net); -#ifdef CONFIG_IPV6_SEG6_HMAC seg6_hmac_net_exit(net); -#endif kfree(rcu_dereference_raw(sdata->tun_src)); kfree(sdata); @@ -520,41 +514,28 @@ int __init seg6_init(void) if (err) goto out_unregister_pernet; -#ifdef CONFIG_IPV6_SEG6_LWTUNNEL err = seg6_iptunnel_init(); if (err) goto out_unregister_genl; err = seg6_local_init(); - if (err) { - seg6_iptunnel_exit(); - goto out_unregister_genl; - } -#endif + if (err) + goto out_unregister_iptun; -#ifdef CONFIG_IPV6_SEG6_HMAC err = seg6_hmac_init(); if (err) - goto out_unregister_iptun; -#endif + goto out_unregister_seg6; pr_info("Segment Routing with IPv6\n"); out: return err; -#ifdef CONFIG_IPV6_SEG6_HMAC -out_unregister_iptun: -#ifdef CONFIG_IPV6_SEG6_LWTUNNEL +out_unregister_seg6: seg6_local_exit(); +out_unregister_iptun: seg6_iptunnel_exit(); -#endif -#endif -#ifdef CONFIG_IPV6_SEG6_LWTUNNEL out_unregister_genl: -#endif -#if IS_ENABLED(CONFIG_IPV6_SEG6_LWTUNNEL) || IS_ENABLED(CONFIG_IPV6_SEG6_HMAC) genl_unregister_family(&seg6_genl_family); -#endif out_unregister_pernet: unregister_pernet_subsys(&ip6_segments_ops); goto out; @@ -562,13 +543,9 @@ out_unregister_pernet: void seg6_exit(void) { -#ifdef CONFIG_IPV6_SEG6_HMAC seg6_hmac_exit(); -#endif -#ifdef CONFIG_IPV6_SEG6_LWTUNNEL seg6_local_exit(); seg6_iptunnel_exit(); -#endif genl_unregister_family(&seg6_genl_family); unregister_pernet_subsys(&ip6_segments_ops); } diff --git a/net/ipv6/seg6_iptunnel.c b/net/ipv6/seg6_iptunnel.c index a75df2ec8db0d369a4e3481576fc09f511a4dd36..098632adc9b5afa69e4b65439ee54c3fc0a8d668 100644 --- a/net/ipv6/seg6_iptunnel.c +++ b/net/ipv6/seg6_iptunnel.c @@ -464,23 +464,21 @@ static int seg6_input_core(struct net *net, struct sock *sk, slwt = seg6_lwt_lwtunnel(orig_dst->lwtstate); - preempt_disable(); + local_bh_disable(); dst = dst_cache_get(&slwt->cache); - preempt_enable(); if (!dst) { ip6_route_input(skb); dst = skb_dst(skb); if (!dst->error) { - preempt_disable(); dst_cache_set_ip6(&slwt->cache, dst, &ipv6_hdr(skb)->saddr); - preempt_enable(); } } else { skb_dst_drop(skb); skb_dst_set(skb, dst); } + local_bh_enable(); err = skb_cow_head(skb, LL_RESERVED_SPACE(dst->dev)); if (unlikely(err)) @@ -536,9 +534,9 @@ static int seg6_output_core(struct net *net, struct sock *sk, slwt = seg6_lwt_lwtunnel(orig_dst->lwtstate); - preempt_disable(); + local_bh_disable(); dst = dst_cache_get(&slwt->cache); - preempt_enable(); + local_bh_enable(); if (unlikely(!dst)) { struct ipv6hdr *hdr = ipv6_hdr(skb); @@ -558,9 +556,9 @@ static int seg6_output_core(struct net *net, struct sock *sk, goto drop; } - preempt_disable(); + local_bh_disable(); dst_cache_set_ip6(&slwt->cache, dst, &fl6.saddr); - preempt_enable(); + local_bh_enable(); } skb_dst_drop(skb); diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c index 4c3605485b68e7c333a0144df3d685b3db9ff45d..f4777c03cdd7a1f43c50901eca8e185d1e3a394c 100644 --- a/net/ipv6/tcp_ipv6.c +++ b/net/ipv6/tcp_ipv6.c @@ -490,14 +490,10 @@ static int tcp_v6_err(struct sk_buff *skb, struct inet6_skb_parm *opt, ipv6_icmp_error(sk, skb, err, th->dest, ntohl(info), (u8 *)th); - if (!sock_owned_by_user(sk)) { - WRITE_ONCE(sk->sk_err, err); - sk_error_report(sk); /* Wake people up to see the error (see connect in sock.c) */ - - tcp_done(sk); - } else { + if (!sock_owned_by_user(sk)) + tcp_done_with_error(sk, err); + else WRITE_ONCE(sk->sk_err_soft, err); - } goto out; case TCP_LISTEN: break; @@ -975,7 +971,7 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32 mark = inet_twsk(sk)->tw_mark; else mark = READ_ONCE(sk->sk_mark); - skb_set_delivery_time(buff, tcp_transmit_time(sk), true); + skb_set_delivery_time(buff, tcp_transmit_time(sk), SKB_CLOCK_MONOTONIC); } if (txhash) { /* autoflowlabel/skb_get_hash_flowi6 rely on buff->hash */ @@ -1200,9 +1196,9 @@ static void tcp_v6_timewait_ack(struct sock *sk, struct sk_buff *skb) tcp_v6_send_ack(sk, skb, tcptw->tw_snd_nxt, tcptw->tw_rcv_nxt, tcptw->tw_rcv_wnd >> tw->tw_rcv_wscale, tcp_tw_tsval(tcptw), - tcptw->tw_ts_recent, tw->tw_bound_dev_if, &key, - tw->tw_tclass, cpu_to_be32(tw->tw_flowlabel), tw->tw_priority, - tw->tw_txhash); + READ_ONCE(tcptw->tw_ts_recent), tw->tw_bound_dev_if, + &key, tw->tw_tclass, cpu_to_be32(tw->tw_flowlabel), + tw->tw_priority, tw->tw_txhash); #ifdef CONFIG_TCP_AO out: @@ -1272,15 +1268,10 @@ static void tcp_v6_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb, /* sk->sk_state == TCP_LISTEN -> for regular TCP_SYN_RECV * sk->sk_state == TCP_SYN_RECV -> for Fast Open. */ - /* RFC 7323 2.3 - * The window field (SEG.WND) of every outgoing segment, with the - * exception of segments, MUST be right-shifted by - * Rcv.Wind.Shift bits: - */ tcp_v6_send_ack(sk, skb, (sk->sk_state == TCP_LISTEN) ? tcp_rsk(req)->snt_isn + 1 : tcp_sk(sk)->snd_nxt, tcp_rsk(req)->rcv_nxt, - req->rsk_rcv_wnd >> inet_rsk(req)->rcv_wscale, + tcp_synack_window(req) >> inet_rsk(req)->rcv_wscale, tcp_rsk_tsval(tcp_rsk(req)), READ_ONCE(req->ts_recent), sk->sk_bound_dev_if, &key, ipv6_get_dsfield(ipv6_hdr(skb)), 0, @@ -2387,8 +2378,14 @@ static struct inet_protosw tcpv6_protosw = { static int __net_init tcpv6_net_init(struct net *net) { - return inet_ctl_sock_create(&net->ipv6.tcp_sk, PF_INET6, - SOCK_RAW, IPPROTO_TCP, net); + int res; + + res = inet_ctl_sock_create(&net->ipv6.tcp_sk, PF_INET6, + SOCK_RAW, IPPROTO_TCP, net); + if (!res) + net->ipv6.tcp_sk->sk_clockid = CLOCK_MONOTONIC; + + return res; } static void __net_exit tcpv6_net_exit(struct net *net) diff --git a/net/mac80211/cfg.c b/net/mac80211/cfg.c index b08e5d7687e3fcfb855b3db7ea916491d5c10c1b..afb361a043d92d90cdeb6c896dfd06ab3750ff29 100644 --- a/net/mac80211/cfg.c +++ b/net/mac80211/cfg.c @@ -1379,6 +1379,11 @@ static int ieee80211_start_ap(struct wiphy *wiphy, struct net_device *dev, (IEEE80211_EHT_PHY_CAP7_MU_BEAMFORMER_80MHZ | IEEE80211_EHT_PHY_CAP7_MU_BEAMFORMER_160MHZ | IEEE80211_EHT_PHY_CAP7_MU_BEAMFORMER_320MHZ); + link_conf->eht_80mhz_full_bw_ul_mumimo = + params->eht_cap->fixed.phy_cap_info[7] & + (IEEE80211_EHT_PHY_CAP7_NON_OFDMA_UL_MU_MIMO_80MHZ | + IEEE80211_EHT_PHY_CAP7_NON_OFDMA_UL_MU_MIMO_160MHZ | + IEEE80211_EHT_PHY_CAP7_NON_OFDMA_UL_MU_MIMO_320MHZ); } else { link_conf->eht_su_beamformer = false; link_conf->eht_su_beamformee = false; @@ -1666,7 +1671,7 @@ static int ieee80211_stop_ap(struct wiphy *wiphy, struct net_device *dev, if (sdata->wdev.cac_started) { chandef = link_conf->chanreq.oper; - wiphy_delayed_work_cancel(wiphy, &link->dfs_cac_timer_work); + wiphy_delayed_work_cancel(wiphy, &sdata->dfs_cac_timer_work); cfg80211_cac_event(sdata->dev, &chandef, NL80211_RADAR_CAC_ABORTED, GFP_KERNEL); @@ -2958,8 +2963,9 @@ static int ieee80211_set_mcast_rate(struct wiphy *wiphy, struct net_device *dev, memcpy(sdata->vif.bss_conf.mcast_rate, rate, sizeof(int) * NUM_NL80211_BANDS); - ieee80211_link_info_change_notify(sdata, &sdata->deflink, - BSS_CHANGED_MCAST_RATE); + if (ieee80211_sdata_running(sdata)) + ieee80211_link_info_change_notify(sdata, &sdata->deflink, + BSS_CHANGED_MCAST_RATE); return 0; } @@ -3466,7 +3472,7 @@ static int ieee80211_start_radar_detection(struct wiphy *wiphy, if (err) goto out_unlock; - wiphy_delayed_work_queue(wiphy, &sdata->deflink.dfs_cac_timer_work, + wiphy_delayed_work_queue(wiphy, &sdata->dfs_cac_timer_work, msecs_to_jiffies(cac_time_ms)); out_unlock: @@ -3482,12 +3488,8 @@ static void ieee80211_end_cac(struct wiphy *wiphy, lockdep_assert_wiphy(local->hw.wiphy); list_for_each_entry(sdata, &local->interfaces, list) { - /* it might be waiting for the local->mtx, but then - * by the time it gets it, sdata->wdev.cac_started - * will no longer be true - */ wiphy_delayed_work_cancel(wiphy, - &sdata->deflink.dfs_cac_timer_work); + &sdata->dfs_cac_timer_work); if (sdata->wdev.cac_started) { ieee80211_link_release_channel(&sdata->deflink); @@ -3637,10 +3639,10 @@ void ieee80211_csa_finish(struct ieee80211_vif *vif, unsigned int link_id) continue; wiphy_work_queue(iter->local->hw.wiphy, - &iter->deflink.csa_finalize_work); + &iter->deflink.csa.finalize_work); } } - wiphy_work_queue(local->hw.wiphy, &link_data->csa_finalize_work); + wiphy_work_queue(local->hw.wiphy, &link_data->csa.finalize_work); rcu_read_unlock(); } @@ -3727,7 +3729,7 @@ static int __ieee80211_csa_finalize(struct ieee80211_link_data *link_data) } if (!cfg80211_chandef_identical(&link_conf->chanreq.oper, - &link_data->csa_chanreq.oper)) + &link_data->csa.chanreq.oper)) return -EINVAL; link_conf->csa_active = false; @@ -3748,7 +3750,7 @@ static int __ieee80211_csa_finalize(struct ieee80211_link_data *link_data) if (err) return err; - cfg80211_ch_switch_notify(sdata->dev, &link_data->csa_chanreq.oper, + cfg80211_ch_switch_notify(sdata->dev, &link_data->csa.chanreq.oper, link_data->link_id); return 0; @@ -3769,7 +3771,7 @@ static void ieee80211_csa_finalize(struct ieee80211_link_data *link_data) void ieee80211_csa_finalize_work(struct wiphy *wiphy, struct wiphy_work *work) { struct ieee80211_link_data *link = - container_of(work, struct ieee80211_link_data, csa_finalize_work); + container_of(work, struct ieee80211_link_data, csa.finalize_work); struct ieee80211_sub_if_data *sdata = link->sdata; struct ieee80211_local *local = sdata->local; @@ -4016,7 +4018,7 @@ __ieee80211_channel_switch(struct wiphy *wiphy, struct net_device *dev, goto out; } - link_data->csa_chanreq = chanreq; + link_data->csa.chanreq = chanreq; link_conf->csa_active = true; if (params->block_tx && @@ -4027,12 +4029,12 @@ __ieee80211_channel_switch(struct wiphy *wiphy, struct net_device *dev, } cfg80211_ch_switch_started_notify(sdata->dev, - &link_data->csa_chanreq.oper, 0, + &link_data->csa.chanreq.oper, link_id, params->count, params->block_tx); if (changed) { ieee80211_link_info_change_notify(sdata, link_data, changed); - drv_channel_switch_beacon(sdata, &link_data->csa_chanreq.oper); + drv_channel_switch_beacon(sdata, &link_data->csa.chanreq.oper); } else { /* if the beacon didn't change, we can finalize immediately */ ieee80211_csa_finalize(link_data); @@ -4978,13 +4980,17 @@ static void ieee80211_del_intf_link(struct wiphy *wiphy, ieee80211_vif_set_links(sdata, wdev->valid_links, 0); } -static int sta_add_link_station(struct ieee80211_local *local, - struct ieee80211_sub_if_data *sdata, - struct link_station_parameters *params) +static int +ieee80211_add_link_station(struct wiphy *wiphy, struct net_device *dev, + struct link_station_parameters *params) { + struct ieee80211_sub_if_data *sdata = IEEE80211_DEV_TO_SUB_IF(dev); + struct ieee80211_local *local = wiphy_priv(wiphy); struct sta_info *sta; int ret; + lockdep_assert_wiphy(local->hw.wiphy); + sta = sta_info_get_bss(sdata, params->mld_mac); if (!sta) return -ENOENT; @@ -5010,23 +5016,15 @@ static int sta_add_link_station(struct ieee80211_local *local, } static int -ieee80211_add_link_station(struct wiphy *wiphy, struct net_device *dev, +ieee80211_mod_link_station(struct wiphy *wiphy, struct net_device *dev, struct link_station_parameters *params) { struct ieee80211_sub_if_data *sdata = IEEE80211_DEV_TO_SUB_IF(dev); struct ieee80211_local *local = wiphy_priv(wiphy); - - lockdep_assert_wiphy(sdata->local->hw.wiphy); - - return sta_add_link_station(local, sdata, params); -} - -static int sta_mod_link_station(struct ieee80211_local *local, - struct ieee80211_sub_if_data *sdata, - struct link_station_parameters *params) -{ struct sta_info *sta; + lockdep_assert_wiphy(local->hw.wiphy); + sta = sta_info_get_bss(sdata, params->mld_mac); if (!sta) return -ENOENT; @@ -5038,22 +5036,14 @@ static int sta_mod_link_station(struct ieee80211_local *local, } static int -ieee80211_mod_link_station(struct wiphy *wiphy, struct net_device *dev, - struct link_station_parameters *params) +ieee80211_del_link_station(struct wiphy *wiphy, struct net_device *dev, + struct link_station_del_parameters *params) { struct ieee80211_sub_if_data *sdata = IEEE80211_DEV_TO_SUB_IF(dev); - struct ieee80211_local *local = wiphy_priv(wiphy); + struct sta_info *sta; lockdep_assert_wiphy(sdata->local->hw.wiphy); - return sta_mod_link_station(local, sdata, params); -} - -static int sta_del_link_station(struct ieee80211_sub_if_data *sdata, - struct link_station_del_parameters *params) -{ - struct sta_info *sta; - sta = sta_info_get_bss(sdata, params->mld_mac); if (!sta) return -ENOENT; @@ -5070,17 +5060,6 @@ static int sta_del_link_station(struct ieee80211_sub_if_data *sdata, return 0; } -static int -ieee80211_del_link_station(struct wiphy *wiphy, struct net_device *dev, - struct link_station_del_parameters *params) -{ - struct ieee80211_sub_if_data *sdata = IEEE80211_DEV_TO_SUB_IF(dev); - - lockdep_assert_wiphy(sdata->local->hw.wiphy); - - return sta_del_link_station(sdata, params); -} - static int ieee80211_set_hw_timestamp(struct wiphy *wiphy, struct net_device *dev, struct cfg80211_set_hw_timestamp *hwts) diff --git a/net/mac80211/chan.c b/net/mac80211/chan.c index 380695fdc32fa58d9aaa7c1ac9bf54b29cf73a30..ec16d7676088ce7008ae7fab618557f1121f4e3a 100644 --- a/net/mac80211/chan.c +++ b/net/mac80211/chan.c @@ -1162,11 +1162,11 @@ ieee80211_link_chanctx_reservation_complete(struct ieee80211_link_data *link) case NL80211_IFTYPE_MESH_POINT: case NL80211_IFTYPE_OCB: wiphy_work_queue(sdata->local->hw.wiphy, - &link->csa_finalize_work); + &link->csa.finalize_work); break; case NL80211_IFTYPE_STATION: wiphy_delayed_work_queue(sdata->local->hw.wiphy, - &link->u.mgd.chswitch_work, 0); + &link->u.mgd.csa.switch_work, 0); break; case NL80211_IFTYPE_UNSPECIFIED: case NL80211_IFTYPE_AP_VLAN: diff --git a/net/mac80211/driver-ops.h b/net/mac80211/driver-ops.h index 5d078c0a23236cf796a93090ff139ddf1a8e04a2..d4e73d3630e0be39a1574bbff18092b2d05ff3db 100644 --- a/net/mac80211/driver-ops.h +++ b/net/mac80211/driver-ops.h @@ -1150,6 +1150,9 @@ drv_pre_channel_switch(struct ieee80211_sub_if_data *sdata, if (!check_sdata_in_driver(sdata)) return -EIO; + if (!ieee80211_vif_link_active(&sdata->vif, ch_switch->link_id)) + return 0; + trace_drv_pre_channel_switch(local, sdata, ch_switch); if (local->ops->pre_channel_switch) ret = local->ops->pre_channel_switch(&local->hw, &sdata->vif, @@ -1171,6 +1174,9 @@ drv_post_channel_switch(struct ieee80211_link_data *link) if (!check_sdata_in_driver(sdata)) return -EIO; + if (!ieee80211_vif_link_active(&sdata->vif, link->link_id)) + return 0; + trace_drv_post_channel_switch(local, sdata); if (local->ops->post_channel_switch) ret = local->ops->post_channel_switch(&local->hw, &sdata->vif, @@ -1191,6 +1197,9 @@ drv_abort_channel_switch(struct ieee80211_link_data *link) if (!check_sdata_in_driver(sdata)) return; + if (!ieee80211_vif_link_active(&sdata->vif, link->link_id)) + return; + trace_drv_abort_channel_switch(local, sdata); if (local->ops->abort_channel_switch) @@ -1210,6 +1219,9 @@ drv_channel_switch_rx_beacon(struct ieee80211_sub_if_data *sdata, if (!check_sdata_in_driver(sdata)) return; + if (!ieee80211_vif_link_active(&sdata->vif, ch_switch->link_id)) + return; + trace_drv_channel_switch_rx_beacon(local, sdata, ch_switch); if (local->ops->channel_switch_rx_beacon) local->ops->channel_switch_rx_beacon(&local->hw, &sdata->vif, diff --git a/net/mac80211/he.c b/net/mac80211/he.c index 9f5ffdc9db284a73be270c48f40d77a94b301e79..ecbb042dd0433e2224237dc7e8ae127bf7e97449 100644 --- a/net/mac80211/he.c +++ b/net/mac80211/he.c @@ -230,15 +230,21 @@ ieee80211_he_spr_ie_to_bss_conf(struct ieee80211_vif *vif, if (!he_spr_ie_elem) return; + + he_obss_pd->sr_ctrl = he_spr_ie_elem->he_sr_control; data = he_spr_ie_elem->optional; if (he_spr_ie_elem->he_sr_control & IEEE80211_HE_SPR_NON_SRG_OFFSET_PRESENT) - data++; + he_obss_pd->non_srg_max_offset = *data++; + if (he_spr_ie_elem->he_sr_control & IEEE80211_HE_SPR_SRG_INFORMATION_PRESENT) { - he_obss_pd->max_offset = *data++; he_obss_pd->min_offset = *data++; + he_obss_pd->max_offset = *data++; + memcpy(he_obss_pd->bss_color_bitmap, data, 8); + data += 8; + memcpy(he_obss_pd->partial_bssid_bitmap, data, 8); he_obss_pd->enable = true; } } diff --git a/net/mac80211/ht.c b/net/mac80211/ht.c index d7e8cf8e48b78f7d801e1bf5548eec2a7647c153..79caeb485fd5d38df4b41f4cbecc873e4861dbf0 100644 --- a/net/mac80211/ht.c +++ b/net/mac80211/ht.c @@ -475,7 +475,7 @@ void ieee80211_send_delba(struct ieee80211_sub_if_data *sdata, sdata->vif.type == NL80211_IFTYPE_MESH_POINT) memcpy(mgmt->bssid, sdata->vif.addr, ETH_ALEN); else if (sdata->vif.type == NL80211_IFTYPE_STATION) - memcpy(mgmt->bssid, sdata->deflink.u.mgd.bssid, ETH_ALEN); + memcpy(mgmt->bssid, sdata->vif.cfg.ap_addr, ETH_ALEN); else if (sdata->vif.type == NL80211_IFTYPE_ADHOC) memcpy(mgmt->bssid, sdata->u.ibss.bssid, ETH_ALEN); diff --git a/net/mac80211/ibss.c b/net/mac80211/ibss.c index 7ace5cdc6c2651c98e754f188ff3669194ac9d64..bf338f3d4dd329abd2b53627d881611859f6f90f 100644 --- a/net/mac80211/ibss.c +++ b/net/mac80211/ibss.c @@ -9,7 +9,7 @@ * Copyright 2009, Johannes Berg * Copyright 2013-2014 Intel Mobile Communications GmbH * Copyright(c) 2016 Intel Deutschland GmbH - * Copyright(c) 2018-2023 Intel Corporation + * Copyright(c) 2018-2024 Intel Corporation */ #include @@ -533,12 +533,12 @@ int ieee80211_ibss_finish_csa(struct ieee80211_sub_if_data *sdata, u64 *changed) IEEE80211_PRIVACY(ifibss->privacy)); /* XXX: should not really modify cfg80211 data */ if (cbss) { - cbss->channel = sdata->deflink.csa_chanreq.oper.chan; + cbss->channel = sdata->deflink.csa.chanreq.oper.chan; cfg80211_put_bss(sdata->local->hw.wiphy, cbss); } } - ifibss->chandef = sdata->deflink.csa_chanreq.oper; + ifibss->chandef = sdata->deflink.csa.chanreq.oper; /* generate the beacon */ return ieee80211_ibss_csa_beacon(sdata, NULL, changed); diff --git a/net/mac80211/ieee80211_i.h b/net/mac80211/ieee80211_i.h index eb62b7d4b4f7e298ce70b87c45fac16a28a03c94..3e735c9436d38427832733732fc0cea308a94a17 100644 --- a/net/mac80211/ieee80211_i.h +++ b/net/mac80211/ieee80211_i.h @@ -26,6 +26,7 @@ #include #include #include +#include #include #include #include @@ -974,10 +975,15 @@ struct ieee80211_link_data_managed { bool disable_wmm_tracking; bool operating_11g_mode; - bool csa_waiting_bcn; - bool csa_ignored_same_chan; - bool csa_blocked_tx; - struct wiphy_delayed_work chswitch_work; + struct { + struct wiphy_delayed_work switch_work; + struct cfg80211_chan_def ap_chandef; + struct ieee80211_parsed_tpe tpe; + unsigned long time; + bool waiting_bcn; + bool ignored_same_chan; + bool blocked_tx; + } csa; struct wiphy_work request_smps_work; /* used to reconfigure hardware SM PS */ @@ -1036,11 +1042,13 @@ struct ieee80211_link_data { struct ieee80211_key __rcu *default_mgmt_key; struct ieee80211_key __rcu *default_beacon_key; - struct wiphy_work csa_finalize_work; bool operating_11g_mode; - struct ieee80211_chan_req csa_chanreq; + struct { + struct wiphy_work finalize_work; + struct ieee80211_chan_req chanreq; + } csa; struct wiphy_work color_change_finalize_work; struct delayed_work color_collision_detect_work; @@ -1059,7 +1067,6 @@ struct ieee80211_link_data { int ap_power_level; /* in dBm */ bool radar_required; - struct wiphy_delayed_work dfs_cac_timer_work; union { struct ieee80211_link_data_managed mgd; @@ -1158,6 +1165,8 @@ struct ieee80211_sub_if_data { struct ieee80211_link_data deflink; struct ieee80211_link_data __rcu *link[IEEE80211_MLD_MAX_NUM_LINKS]; + struct wiphy_delayed_work dfs_cac_timer_work; + /* for ieee80211_set_active_links_async() */ struct wiphy_work activate_links_work; u16 desired_active_links; @@ -1708,7 +1717,6 @@ struct ieee802_11_elems { const struct ieee80211_he_spr *he_spr; const struct ieee80211_mu_edca_param_set *mu_edca_param_set; const struct ieee80211_he_6ghz_capa *he_6ghz_capa; - const struct ieee80211_tx_pwr_env *tx_pwr_env[IEEE80211_TPE_MAX_IE_COUNT]; const u8 *uora_element; const u8 *mesh_id; const u8 *peering; @@ -1746,6 +1754,10 @@ struct ieee802_11_elems { const struct ieee80211_bandwidth_indication *bandwidth_indication; const struct ieee80211_ttlm_elem *ttlm[IEEE80211_TTLM_MAX_CNT]; + /* not the order in the psd values is per element, not per chandef */ + struct ieee80211_parsed_tpe tpe; + struct ieee80211_parsed_tpe csa_tpe; + /* length of them, respectively */ u8 ext_capab_len; u8 ssid_len; @@ -1764,8 +1776,6 @@ struct ieee802_11_elems { u8 perr_len; u8 country_elem_len; u8 bssid_index_len; - u8 tx_pwr_env_len[IEEE80211_TPE_MAX_IE_COUNT]; - u8 tx_pwr_env_num; u8 eht_cap_len; /* mult-link element can be de-fragmented and thus u8 is not sufficient */ @@ -1845,6 +1855,8 @@ void ieee80211_link_info_change_notify(struct ieee80211_sub_if_data *sdata, void ieee80211_configure_filter(struct ieee80211_local *local); u64 ieee80211_reset_erp_info(struct ieee80211_sub_if_data *sdata); +void ieee80211_handle_queued_frames(struct ieee80211_local *local); + u64 ieee80211_mgmt_tx_cookie(struct ieee80211_local *local); int ieee80211_attach_ack_skb(struct ieee80211_local *local, struct sk_buff *skb, u64 *cookie, gfp_t gfp); @@ -1968,6 +1980,7 @@ void ieee80211_offchannel_stop_vifs(struct ieee80211_local *local); void ieee80211_offchannel_return(struct ieee80211_local *local); void ieee80211_roc_setup(struct ieee80211_local *local); void ieee80211_start_next_roc(struct ieee80211_local *local); +void ieee80211_reconfig_roc(struct ieee80211_local *local); void ieee80211_roc_purge(struct ieee80211_local *local, struct ieee80211_sub_if_data *sdata); int ieee80211_remain_on_channel(struct wiphy *wiphy, struct wireless_dev *wdev, @@ -2243,6 +2256,7 @@ int ieee80211_frame_duration(enum nl80211_band band, size_t len, void ieee80211_regulatory_limit_wmm_params(struct ieee80211_sub_if_data *sdata, struct ieee80211_tx_queue_params *qparam, int ac); +void ieee80211_clear_tpe(struct ieee80211_parsed_tpe *tpe); void ieee80211_set_wmm_default(struct ieee80211_link_data *link, bool bss_notify, bool enable_qos); void ieee80211_xmit(struct ieee80211_sub_if_data *sdata, @@ -2681,6 +2695,11 @@ void ieee80211_remove_wbrf(struct ieee80211_local *local, struct cfg80211_chan_d #define VISIBLE_IF_MAC80211_KUNIT ieee80211_rx_result ieee80211_drop_unencrypted_mgmt(struct ieee80211_rx_data *rx); +int ieee80211_calc_chandef_subchan_offset(const struct cfg80211_chan_def *ap, + u8 n_partial_subchans); +void ieee80211_rearrange_tpe_psd(struct ieee80211_parsed_tpe_psd *psd, + const struct cfg80211_chan_def *ap, + const struct cfg80211_chan_def *used); #else #define EXPORT_SYMBOL_IF_MAC80211_KUNIT(sym) #define VISIBLE_IF_MAC80211_KUNIT static diff --git a/net/mac80211/iface.c b/net/mac80211/iface.c index dc42902e269359a1a7d95ee85b2f4817df47a0d4..d1a49ee4a194f760b0d5c5562302b4b4c8f6e1f2 100644 --- a/net/mac80211/iface.c +++ b/net/mac80211/iface.c @@ -543,18 +543,18 @@ static void ieee80211_do_stop(struct ieee80211_sub_if_data *sdata, bool going_do sdata->vif.bss_conf.csa_active = false; if (sdata->vif.type == NL80211_IFTYPE_STATION) - sdata->deflink.u.mgd.csa_waiting_bcn = false; + sdata->deflink.u.mgd.csa.waiting_bcn = false; if (sdata->csa_blocked_queues) { ieee80211_wake_vif_queues(local, sdata, IEEE80211_QUEUE_STOP_REASON_CSA); sdata->csa_blocked_queues = false; } - wiphy_work_cancel(local->hw.wiphy, &sdata->deflink.csa_finalize_work); + wiphy_work_cancel(local->hw.wiphy, &sdata->deflink.csa.finalize_work); wiphy_work_cancel(local->hw.wiphy, &sdata->deflink.color_change_finalize_work); wiphy_delayed_work_cancel(local->hw.wiphy, - &sdata->deflink.dfs_cac_timer_work); + &sdata->dfs_cac_timer_work); if (sdata->wdev.cac_started) { chandef = sdata->vif.bss_conf.chanreq.oper; @@ -1746,6 +1746,8 @@ static void ieee80211_setup_sdata(struct ieee80211_sub_if_data *sdata, wiphy_work_init(&sdata->work, ieee80211_iface_work); wiphy_work_init(&sdata->activate_links_work, ieee80211_activate_links_work); + wiphy_delayed_work_init(&sdata->dfs_cac_timer_work, + ieee80211_dfs_cac_timer_work); switch (type) { case NL80211_IFTYPE_P2P_GO: diff --git a/net/mac80211/link.c b/net/mac80211/link.c index af0321408a97af1bf1dccd5eccc4bfb502d599d7..2e6e92defbca65a699f75ea2a07dc0a3aeefdfe5 100644 --- a/net/mac80211/link.c +++ b/net/mac80211/link.c @@ -37,7 +37,7 @@ void ieee80211_link_init(struct ieee80211_sub_if_data *sdata, link_conf->link_id = link_id; link_conf->vif = &sdata->vif; - wiphy_work_init(&link->csa_finalize_work, + wiphy_work_init(&link->csa.finalize_work, ieee80211_csa_finalize_work); wiphy_work_init(&link->color_change_finalize_work, ieee80211_color_change_finalize_work); @@ -45,8 +45,6 @@ void ieee80211_link_init(struct ieee80211_sub_if_data *sdata, ieee80211_color_collision_detection_work); INIT_LIST_HEAD(&link->assigned_chanctx_list); INIT_LIST_HEAD(&link->reserved_chanctx_list); - wiphy_delayed_work_init(&link->dfs_cac_timer_work, - ieee80211_dfs_cac_timer_work); if (!deflink) { switch (sdata->vif.type) { @@ -74,7 +72,7 @@ void ieee80211_link_stop(struct ieee80211_link_data *link) cancel_delayed_work_sync(&link->color_collision_detect_work); wiphy_work_cancel(link->sdata->local->hw.wiphy, - &link->csa_finalize_work); + &link->csa.finalize_work); ieee80211_link_release_channel(link); } @@ -359,6 +357,18 @@ static int _ieee80211_set_active_links(struct ieee80211_sub_if_data *sdata, ieee80211_teardown_tdls_peers(link); __ieee80211_link_release_channel(link, true); + + /* + * If CSA is (still) active while the link is deactivated, + * just schedule the channel switch work for the time we + * had previously calculated, and we'll take the process + * from there. + */ + if (link->conf->csa_active) + wiphy_delayed_work_queue(local->hw.wiphy, + &link->u.mgd.csa.switch_work, + link->u.mgd.csa.time - + jiffies); } list_for_each_entry(sta, &local->sta_list, list) { diff --git a/net/mac80211/main.c b/net/mac80211/main.c index 4eaea0a9975b474b0b6b314b3244a88d8e0f1d77..c8cb091b5ea37405c66eb80758501dffa478e877 100644 --- a/net/mac80211/main.c +++ b/net/mac80211/main.c @@ -337,6 +337,8 @@ void ieee80211_bss_info_change_notify(struct ieee80211_sub_if_data *sdata, might_sleep(); + WARN_ON_ONCE(ieee80211_vif_is_mld(&sdata->vif)); + if (!changed || sdata->vif.type == NL80211_IFTYPE_AP_VLAN) return; @@ -369,7 +371,6 @@ void ieee80211_bss_info_change_notify(struct ieee80211_sub_if_data *sdata, if (changed & ~BSS_CHANGED_VIF_CFG_FLAGS) { u64 ch = changed & ~BSS_CHANGED_VIF_CFG_FLAGS; - /* FIXME: should be for each link */ trace_drv_link_info_changed(local, sdata, &sdata->vif.bss_conf, changed); if (local->ops->link_info_changed) @@ -423,9 +424,8 @@ u64 ieee80211_reset_erp_info(struct ieee80211_sub_if_data *sdata) BSS_CHANGED_ERP_SLOT; } -static void ieee80211_tasklet_handler(struct tasklet_struct *t) +void ieee80211_handle_queued_frames(struct ieee80211_local *local) { - struct ieee80211_local *local = from_tasklet(local, t, tasklet); struct sk_buff *skb; while ((skb = skb_dequeue(&local->skb_queue)) || @@ -450,6 +450,13 @@ static void ieee80211_tasklet_handler(struct tasklet_struct *t) } } +static void ieee80211_tasklet_handler(struct tasklet_struct *t) +{ + struct ieee80211_local *local = from_tasklet(local, t, tasklet); + + ieee80211_handle_queued_frames(local); +} + static void ieee80211_restart_work(struct work_struct *work) { struct ieee80211_local *local = diff --git a/net/mac80211/mesh.c b/net/mac80211/mesh.c index cbc9b5e40cb35e81fb80dd55016c3afc8c31deb7..6d4510221c98e695ea140e40e5e9ec5cbf385a5e 100644 --- a/net/mac80211/mesh.c +++ b/net/mac80211/mesh.c @@ -1776,6 +1776,7 @@ void ieee80211_mesh_init_sdata(struct ieee80211_sub_if_data *sdata) ifmsh->last_preq = jiffies; ifmsh->next_perr = jiffies; ifmsh->csa_role = IEEE80211_MESH_CSA_ROLE_NONE; + ifmsh->nonpeer_pm = NL80211_MESH_POWER_ACTIVE; /* Allocate all mesh structures when creating the first mesh interface. */ if (!mesh_allocated) ieee80211s_init(); diff --git a/net/mac80211/mesh_pathtbl.c b/net/mac80211/mesh_pathtbl.c index a6b62169f08483c5aa481f4f8f59f67fa56a4ef7..c0a5c75cddcb9f1c3f509b909e443ea0e74fb19d 100644 --- a/net/mac80211/mesh_pathtbl.c +++ b/net/mac80211/mesh_pathtbl.c @@ -1017,10 +1017,23 @@ void mesh_path_discard_frame(struct ieee80211_sub_if_data *sdata, */ void mesh_path_flush_pending(struct mesh_path *mpath) { + struct ieee80211_sub_if_data *sdata = mpath->sdata; + struct ieee80211_if_mesh *ifmsh = &sdata->u.mesh; + struct mesh_preq_queue *preq, *tmp; struct sk_buff *skb; while ((skb = skb_dequeue(&mpath->frame_queue)) != NULL) mesh_path_discard_frame(mpath->sdata, skb); + + spin_lock_bh(&ifmsh->mesh_preq_queue_lock); + list_for_each_entry_safe(preq, tmp, &ifmsh->preq_queue.list, list) { + if (ether_addr_equal(mpath->dst, preq->dst)) { + list_del(&preq->list); + kfree(preq); + --ifmsh->preq_queue_len; + } + } + spin_unlock_bh(&ifmsh->mesh_preq_queue_lock); } /** diff --git a/net/mac80211/mlme.c b/net/mac80211/mlme.c index a5f2d3cfe60d2bf454129489c30e6517ca032126..ef3280fafbe994ab9736a52d3b79d8887e238dd1 100644 --- a/net/mac80211/mlme.c +++ b/net/mac80211/mlme.c @@ -606,11 +606,218 @@ static bool ieee80211_chandef_usable(struct ieee80211_sub_if_data *sdata, return true; } +static int ieee80211_chandef_num_subchans(const struct cfg80211_chan_def *c) +{ + if (c->width == NL80211_CHAN_WIDTH_80P80) + return 4 + 4; + + return nl80211_chan_width_to_mhz(c->width) / 20; +} + +static int ieee80211_chandef_num_widths(const struct cfg80211_chan_def *c) +{ + switch (c->width) { + case NL80211_CHAN_WIDTH_20: + case NL80211_CHAN_WIDTH_20_NOHT: + return 1; + case NL80211_CHAN_WIDTH_40: + return 2; + case NL80211_CHAN_WIDTH_80P80: + case NL80211_CHAN_WIDTH_80: + return 3; + case NL80211_CHAN_WIDTH_160: + return 4; + case NL80211_CHAN_WIDTH_320: + return 5; + default: + WARN_ON(1); + return 0; + } +} + +VISIBLE_IF_MAC80211_KUNIT int +ieee80211_calc_chandef_subchan_offset(const struct cfg80211_chan_def *ap, + u8 n_partial_subchans) +{ + int n = ieee80211_chandef_num_subchans(ap); + struct cfg80211_chan_def tmp = *ap; + int offset = 0; + + /* + * Given a chandef (in this context, it's the AP's) and a number + * of subchannels that we want to look at ('n_partial_subchans'), + * calculate the offset in number of subchannels between the full + * and the subset with the desired width. + */ + + /* same number of subchannels means no offset, obviously */ + if (n == n_partial_subchans) + return 0; + + /* don't WARN - misconfigured APs could cause this if their N > width */ + if (n < n_partial_subchans) + return 0; + + while (ieee80211_chandef_num_subchans(&tmp) > n_partial_subchans) { + u32 prev = tmp.center_freq1; + + ieee80211_chandef_downgrade(&tmp, NULL); + + /* + * if center_freq moved up, half the original channels + * are gone now but were below, so increase offset + */ + if (prev < tmp.center_freq1) + offset += ieee80211_chandef_num_subchans(&tmp); + } + + /* + * 80+80 with secondary 80 below primary - four subchannels for it + * (we cannot downgrade *to* 80+80, so no need to consider 'tmp') + */ + if (ap->width == NL80211_CHAN_WIDTH_80P80 && + ap->center_freq2 < ap->center_freq1) + offset += 4; + + return offset; +} +EXPORT_SYMBOL_IF_MAC80211_KUNIT(ieee80211_calc_chandef_subchan_offset); + +VISIBLE_IF_MAC80211_KUNIT void +ieee80211_rearrange_tpe_psd(struct ieee80211_parsed_tpe_psd *psd, + const struct cfg80211_chan_def *ap, + const struct cfg80211_chan_def *used) +{ + u8 needed = ieee80211_chandef_num_subchans(used); + u8 have = ieee80211_chandef_num_subchans(ap); + u8 tmp[IEEE80211_TPE_PSD_ENTRIES_320MHZ]; + u8 offset; + + if (!psd->valid) + return; + + /* if N is zero, all defaults were used, no point in rearranging */ + if (!psd->n) + goto out; + + BUILD_BUG_ON(sizeof(tmp) != sizeof(psd->power)); + + /* + * This assumes that 'N' is consistent with the HE channel, as + * it should be (otherwise the AP is broken). + * + * In psd->power we have values in the order 0..N, 0..K, where + * N+K should cover the entire channel per 'ap', but even if it + * doesn't then we've pre-filled 'unlimited' as defaults. + * + * But this is all the wrong order, we want to have them in the + * order of the 'used' channel. + * + * So for example, we could have a 320 MHz EHT AP, which has the + * HE channel as 80 MHz (e.g. due to puncturing, which doesn't + * seem to be considered for the TPE), as follows: + * + * EHT 320: | | | | | | | | | | | | | | | | | + * HE 80: | | | | | + * used 160: | | | | | | | | | + * + * N entries: |--|--|--|--| + * K entries: |--|--|--|--|--|--|--|--| |--|--|--|--| + * power idx: 4 5 6 7 8 9 10 11 0 1 2 3 12 13 14 15 + * full chan: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 + * used chan: 0 1 2 3 4 5 6 7 + * + * The idx in the power array ('power idx') is like this since it + * comes directly from the element's N and K entries in their + * element order, and those are this way for HE compatibility. + * + * Rearrange them as desired here, first by putting them into the + * 'full chan' order, and then selecting the necessary subset for + * the 'used chan'. + */ + + /* first reorder according to AP channel */ + offset = ieee80211_calc_chandef_subchan_offset(ap, psd->n); + for (int i = 0; i < have; i++) { + if (i < offset) + tmp[i] = psd->power[i + psd->n]; + else if (i < offset + psd->n) + tmp[i] = psd->power[i - offset]; + else + tmp[i] = psd->power[i]; + } + + /* + * and then select the subset for the used channel + * (set everything to defaults first in case a driver is confused) + */ + memset(psd->power, IEEE80211_TPE_PSD_NO_LIMIT, sizeof(psd->power)); + offset = ieee80211_calc_chandef_subchan_offset(ap, needed); + for (int i = 0; i < needed; i++) + psd->power[i] = tmp[offset + i]; + +out: + /* limit, but don't lie if there are defaults in the data */ + if (needed < psd->count) + psd->count = needed; +} +EXPORT_SYMBOL_IF_MAC80211_KUNIT(ieee80211_rearrange_tpe_psd); + +static void ieee80211_rearrange_tpe(struct ieee80211_parsed_tpe *tpe, + const struct cfg80211_chan_def *ap, + const struct cfg80211_chan_def *used) +{ + /* ignore this completely for narrow/invalid channels */ + if (!ieee80211_chandef_num_subchans(ap) || + !ieee80211_chandef_num_subchans(used)) { + ieee80211_clear_tpe(tpe); + return; + } + + for (int i = 0; i < 2; i++) { + int needed_pwr_count; + + ieee80211_rearrange_tpe_psd(&tpe->psd_local[i], ap, used); + ieee80211_rearrange_tpe_psd(&tpe->psd_reg_client[i], ap, used); + + /* limit this to the widths we actually need */ + needed_pwr_count = ieee80211_chandef_num_widths(used); + if (needed_pwr_count < tpe->max_local[i].count) + tpe->max_local[i].count = needed_pwr_count; + if (needed_pwr_count < tpe->max_reg_client[i].count) + tpe->max_reg_client[i].count = needed_pwr_count; + } +} + +/* + * The AP part of the channel request is used to distinguish settings + * to the device used for wider bandwidth OFDMA. This is used in the + * channel context code to assign two channel contexts even if they're + * both for the same channel, if the AP bandwidths are incompatible. + * If not EHT (or driver override) then ap.chan == NULL indicates that + * there's no wider BW OFDMA used. + */ +static void ieee80211_set_chanreq_ap(struct ieee80211_sub_if_data *sdata, + struct ieee80211_chan_req *chanreq, + struct ieee80211_conn_settings *conn, + struct cfg80211_chan_def *ap_chandef) +{ + chanreq->ap.chan = NULL; + + if (conn->mode < IEEE80211_CONN_MODE_EHT) + return; + if (sdata->vif.driver_flags & IEEE80211_VIF_IGNORE_OFDMA_WIDER_BW) + return; + + chanreq->ap = *ap_chandef; +} + static struct ieee802_11_elems * ieee80211_determine_chan_mode(struct ieee80211_sub_if_data *sdata, struct ieee80211_conn_settings *conn, struct cfg80211_bss *cbss, int link_id, - struct ieee80211_chan_req *chanreq) + struct ieee80211_chan_req *chanreq, + struct cfg80211_chan_def *ap_chandef) { const struct cfg80211_bss_ies *ies = rcu_dereference(cbss->ies); struct ieee80211_bss *bss = (void *)cbss->priv; @@ -623,7 +830,6 @@ ieee80211_determine_chan_mode(struct ieee80211_sub_if_data *sdata, }; struct ieee802_11_elems *elems; struct ieee80211_supported_band *sband; - struct cfg80211_chan_def ap_chandef; enum ieee80211_conn_mode ap_mode; int ret; @@ -634,7 +840,7 @@ again: return ERR_PTR(-ENOMEM); ap_mode = ieee80211_determine_ap_chan(sdata, channel, bss->vht_cap_info, - elems, false, conn, &ap_chandef); + elems, false, conn, ap_chandef); /* this should be impossible since parsing depends on our mode */ if (WARN_ON(ap_mode > conn->mode)) { @@ -701,14 +907,9 @@ again: break; } - chanreq->oper = ap_chandef; + chanreq->oper = *ap_chandef; - /* wider-bandwidth OFDMA is only done in EHT */ - if (conn->mode >= IEEE80211_CONN_MODE_EHT && - !(sdata->vif.driver_flags & IEEE80211_VIF_IGNORE_OFDMA_WIDER_BW)) - chanreq->ap = ap_chandef; - else - chanreq->ap.chan = NULL; + ieee80211_set_chanreq_ap(sdata, chanreq, conn, ap_chandef); while (!ieee80211_chandef_usable(sdata, &chanreq->oper, IEEE80211_CHAN_DISABLED)) { @@ -738,7 +939,7 @@ again: IEEE80211_CONN_BW_LIMIT_160); } - if (chanreq->oper.width != ap_chandef.width || ap_mode != conn->mode) + if (chanreq->oper.width != ap_chandef->width || ap_mode != conn->mode) sdata_info(sdata, "regulatory prevented using AP config, downgraded\n"); @@ -790,6 +991,7 @@ static int ieee80211_config_bw(struct ieee80211_link_data *link, struct ieee80211_channel *channel = link->conf->chanreq.oper.chan; struct ieee80211_sub_if_data *sdata = link->sdata; struct ieee80211_chan_req chanreq = {}; + struct cfg80211_chan_def ap_chandef; enum ieee80211_conn_mode ap_mode; u32 vht_cap_info = 0; u16 ht_opmode; @@ -805,7 +1007,7 @@ static int ieee80211_config_bw(struct ieee80211_link_data *link, ap_mode = ieee80211_determine_ap_chan(sdata, channel, vht_cap_info, elems, true, &link->u.mgd.conn, - &chanreq.ap); + &ap_chandef); if (ap_mode != link->u.mgd.conn.mode) { link_info(link, @@ -815,10 +1017,9 @@ static int ieee80211_config_bw(struct ieee80211_link_data *link, return -EINVAL; } - chanreq.oper = chanreq.ap; - if (link->u.mgd.conn.mode < IEEE80211_CONN_MODE_EHT || - sdata->vif.driver_flags & IEEE80211_VIF_IGNORE_OFDMA_WIDER_BW) - chanreq.ap.chan = NULL; + chanreq.oper = ap_chandef; + ieee80211_set_chanreq_ap(sdata, &chanreq, &link->u.mgd.conn, + &ap_chandef); /* * if HT operation mode changed store the new one - @@ -843,6 +1044,16 @@ static int ieee80211_config_bw(struct ieee80211_link_data *link, ieee80211_min_bw_limit_from_chandef(&chanreq.oper)) ieee80211_chandef_downgrade(&chanreq.oper, NULL); + if (ap_chandef.chan->band == NL80211_BAND_6GHZ && + link->u.mgd.conn.mode >= IEEE80211_CONN_MODE_HE) { + ieee80211_rearrange_tpe(&elems->tpe, &ap_chandef, + &chanreq.oper); + if (memcmp(&link->conf->tpe, &elems->tpe, sizeof(elems->tpe))) { + link->conf->tpe = elems->tpe; + *changed |= BSS_CHANGED_TPE; + } + } + if (ieee80211_chanreq_identical(&chanreq, &link->conf->chanreq)) return 0; @@ -1862,12 +2073,12 @@ void ieee80211_send_4addr_nullfunc(struct ieee80211_local *local, } /* spectrum management related things */ -static void ieee80211_chswitch_work(struct wiphy *wiphy, - struct wiphy_work *work) +static void ieee80211_csa_switch_work(struct wiphy *wiphy, + struct wiphy_work *work) { struct ieee80211_link_data *link = container_of(work, struct ieee80211_link_data, - u.mgd.chswitch_work.work); + u.mgd.csa.switch_work.work); struct ieee80211_sub_if_data *sdata = link->sdata; struct ieee80211_local *local = sdata->local; struct ieee80211_if_managed *ifmgd = &sdata->u.mgd; @@ -1884,6 +2095,18 @@ static void ieee80211_chswitch_work(struct wiphy *wiphy, if (!link->conf->csa_active) return; + /* + * If the link isn't active (now), we cannot wait for beacons, won't + * have a reserved chanctx, etc. Just switch over the chandef and + * update cfg80211 directly. + */ + if (!ieee80211_vif_link_active(&sdata->vif, link->link_id)) { + link->conf->chanreq = link->csa.chanreq; + cfg80211_ch_switch_notify(sdata->dev, &link->csa.chanreq.oper, + link->link_id); + return; + } + /* * using reservation isn't immediate as it may be deferred until later * with multi-vif. once reservation is complete it will re-schedule the @@ -1902,9 +2125,9 @@ static void ieee80211_chswitch_work(struct wiphy *wiphy, ret = ieee80211_link_use_reserved_context(link); if (ret) { - sdata_info(sdata, - "failed to use reserved channel context, disconnecting (err=%d)\n", - ret); + link_info(link, + "failed to use reserved channel context, disconnecting (err=%d)\n", + ret); wiphy_work_queue(sdata->local->hw.wiphy, &ifmgd->csa_connection_drop_work); } @@ -1912,15 +2135,29 @@ static void ieee80211_chswitch_work(struct wiphy *wiphy, } if (!ieee80211_chanreq_identical(&link->conf->chanreq, - &link->csa_chanreq)) { - sdata_info(sdata, - "failed to finalize channel switch, disconnecting\n"); + &link->csa.chanreq)) { + link_info(link, + "failed to finalize channel switch, disconnecting\n"); wiphy_work_queue(sdata->local->hw.wiphy, &ifmgd->csa_connection_drop_work); return; } - link->u.mgd.csa_waiting_bcn = true; + link->u.mgd.csa.waiting_bcn = true; + + /* apply new TPE restrictions immediately on the new channel */ + if (link->u.mgd.csa.ap_chandef.chan->band == NL80211_BAND_6GHZ && + link->u.mgd.conn.mode >= IEEE80211_CONN_MODE_HE) { + ieee80211_rearrange_tpe(&link->u.mgd.csa.tpe, + &link->u.mgd.csa.ap_chandef, + &link->conf->chanreq.oper); + if (memcmp(&link->conf->tpe, &link->u.mgd.csa.tpe, + sizeof(link->u.mgd.csa.tpe))) { + link->conf->tpe = link->u.mgd.csa.tpe; + ieee80211_link_info_change_notify(sdata, link, + BSS_CHANGED_TPE); + } + } ieee80211_sta_reset_beacon_monitor(sdata); ieee80211_sta_reset_conn_monitor(sdata); @@ -1944,19 +2181,19 @@ static void ieee80211_chswitch_post_beacon(struct ieee80211_link_data *link) } link->conf->csa_active = false; - link->u.mgd.csa_blocked_tx = false; - link->u.mgd.csa_waiting_bcn = false; + link->u.mgd.csa.blocked_tx = false; + link->u.mgd.csa.waiting_bcn = false; ret = drv_post_channel_switch(link); if (ret) { - sdata_info(sdata, - "driver post channel switch failed, disconnecting\n"); + link_info(link, + "driver post channel switch failed, disconnecting\n"); wiphy_work_queue(sdata->local->hw.wiphy, &ifmgd->csa_connection_drop_work); return; } - cfg80211_ch_switch_notify(sdata->dev, &link->reserved.oper, + cfg80211_ch_switch_notify(sdata->dev, &link->conf->chanreq.oper, link->link_id); } @@ -1971,7 +2208,8 @@ void ieee80211_chswitch_done(struct ieee80211_vif *vif, bool success, if (!success) { sdata_info(sdata, - "driver channel switch failed, disconnecting\n"); + "driver channel switch failed (link %d), disconnecting\n", + link_id); wiphy_work_queue(sdata->local->hw.wiphy, &sdata->u.mgd.csa_connection_drop_work); } else { @@ -1984,7 +2222,7 @@ void ieee80211_chswitch_done(struct ieee80211_vif *vif, bool success, } wiphy_delayed_work_queue(sdata->local->hw.wiphy, - &link->u.mgd.chswitch_work, 0); + &link->u.mgd.csa.switch_work, 0); } rcu_read_unlock(); @@ -2011,74 +2249,228 @@ ieee80211_sta_abort_chanswitch(struct ieee80211_link_data *link) } link->conf->csa_active = false; - link->u.mgd.csa_blocked_tx = false; + link->u.mgd.csa.blocked_tx = false; drv_abort_channel_switch(link); } +struct sta_csa_rnr_iter_data { + struct ieee80211_link_data *link; + struct ieee80211_channel *chan; + u8 mld_id; +}; + +static enum cfg80211_rnr_iter_ret +ieee80211_sta_csa_rnr_iter(void *_data, u8 type, + const struct ieee80211_neighbor_ap_info *info, + const u8 *tbtt_info, u8 tbtt_info_len) +{ + struct sta_csa_rnr_iter_data *data = _data; + struct ieee80211_link_data *link = data->link; + struct ieee80211_sub_if_data *sdata = link->sdata; + struct ieee80211_if_managed *ifmgd = &sdata->u.mgd; + const struct ieee80211_tbtt_info_ge_11 *ti; + enum nl80211_band band; + unsigned int center_freq; + int link_id; + + if (type != IEEE80211_TBTT_INFO_TYPE_TBTT) + return RNR_ITER_CONTINUE; + + if (tbtt_info_len < sizeof(*ti)) + return RNR_ITER_CONTINUE; + + ti = (const void *)tbtt_info; + + if (ti->mld_params.mld_id != data->mld_id) + return RNR_ITER_CONTINUE; + + link_id = le16_get_bits(ti->mld_params.params, + IEEE80211_RNR_MLD_PARAMS_LINK_ID); + if (link_id != data->link->link_id) + return RNR_ITER_CONTINUE; + + /* we found the entry for our link! */ + + /* this AP is confused, it had this right before ... just disconnect */ + if (!ieee80211_operating_class_to_band(info->op_class, &band)) { + link_info(link, + "AP now has invalid operating class in RNR, disconnect\n"); + wiphy_work_queue(sdata->local->hw.wiphy, + &ifmgd->csa_connection_drop_work); + return RNR_ITER_BREAK; + } + + center_freq = ieee80211_channel_to_frequency(info->channel, band); + data->chan = ieee80211_get_channel(sdata->local->hw.wiphy, center_freq); + + return RNR_ITER_BREAK; +} + +static void +ieee80211_sta_other_link_csa_disappeared(struct ieee80211_link_data *link, + struct ieee802_11_elems *elems) +{ + struct ieee80211_sub_if_data *sdata = link->sdata; + struct ieee80211_if_managed *ifmgd = &sdata->u.mgd; + struct sta_csa_rnr_iter_data data = { + .link = link, + }; + + /* + * If we get here, we see a beacon from another link without + * CSA still being reported for it, so now we have to check + * if the CSA was aborted or completed. This may not even be + * perfectly possible if the CSA was only done for changing + * the puncturing, but in that case if the link in inactive + * we don't really care, and if it's an active link (or when + * it's activated later) we'll get a beacon and adjust. + */ + + if (WARN_ON(!elems->ml_basic)) + return; + + data.mld_id = ieee80211_mle_get_mld_id((const void *)elems->ml_basic); + + /* + * So in order to do this, iterate the RNR element(s) and see + * what channel is reported now. + */ + cfg80211_iter_rnr(elems->ie_start, elems->total_len, + ieee80211_sta_csa_rnr_iter, &data); + + if (!data.chan) { + link_info(link, + "couldn't find (valid) channel in RNR for CSA, disconnect\n"); + wiphy_work_queue(sdata->local->hw.wiphy, + &ifmgd->csa_connection_drop_work); + return; + } + + /* + * If it doesn't match the CSA, then assume it aborted. This + * may erroneously detect that it was _not_ aborted when it + * was in fact aborted, but only changed the bandwidth or the + * puncturing configuration, but we don't have enough data to + * detect that. + */ + if (data.chan != link->csa.chanreq.oper.chan) + ieee80211_sta_abort_chanswitch(link); +} + +enum ieee80211_csa_source { + IEEE80211_CSA_SOURCE_BEACON, + IEEE80211_CSA_SOURCE_OTHER_LINK, + IEEE80211_CSA_SOURCE_ACTION, +}; + static void ieee80211_sta_process_chanswitch(struct ieee80211_link_data *link, u64 timestamp, u32 device_timestamp, - struct ieee802_11_elems *elems, - bool beacon) + struct ieee802_11_elems *full_elems, + struct ieee802_11_elems *csa_elems, + enum ieee80211_csa_source source) { struct ieee80211_sub_if_data *sdata = link->sdata; struct ieee80211_local *local = sdata->local; struct ieee80211_if_managed *ifmgd = &sdata->u.mgd; - struct cfg80211_bss *cbss = link->conf->bss; + struct ieee80211_chanctx *chanctx = NULL; struct ieee80211_chanctx_conf *conf; - struct ieee80211_chanctx *chanctx; - enum nl80211_band current_band; - struct ieee80211_csa_ie csa_ie; + struct ieee80211_csa_ie csa_ie = {}; struct ieee80211_channel_switch ch_switch = { .link_id = link->link_id, + .timestamp = timestamp, + .device_timestamp = device_timestamp, }; - struct ieee80211_bss *bss; - unsigned long timeout; + unsigned long now; int res; lockdep_assert_wiphy(local->hw.wiphy); - if (!cbss) - return; + if (csa_elems) { + struct cfg80211_bss *cbss = link->conf->bss; + enum nl80211_band current_band; + struct ieee80211_bss *bss; - current_band = cbss->channel->band; - bss = (void *)cbss->priv; - res = ieee80211_parse_ch_switch_ie(sdata, elems, current_band, - bss->vht_cap_info, - &link->u.mgd.conn, - link->u.mgd.bssid, &csa_ie); + if (WARN_ON(!cbss)) + return; + + current_band = cbss->channel->band; + bss = (void *)cbss->priv; + + res = ieee80211_parse_ch_switch_ie(sdata, csa_elems, + current_band, + bss->vht_cap_info, + &link->u.mgd.conn, + link->u.mgd.bssid, &csa_ie); + if (res == 0) { + ch_switch.block_tx = csa_ie.mode; + ch_switch.chandef = csa_ie.chanreq.oper; + ch_switch.count = csa_ie.count; + ch_switch.delay = csa_ie.max_switch_time; + } - if (!res) { - ch_switch.timestamp = timestamp; - ch_switch.device_timestamp = device_timestamp; - ch_switch.block_tx = csa_ie.mode; - ch_switch.chandef = csa_ie.chanreq.oper; - ch_switch.count = csa_ie.count; - ch_switch.delay = csa_ie.max_switch_time; + link->u.mgd.csa.tpe = csa_elems->csa_tpe; + } else { + /* + * If there was no per-STA profile for this link, we + * get called with csa_elems == NULL. This of course means + * there are no CSA elements, so set res=1 indicating + * no more CSA. + */ + res = 1; } if (res < 0) goto drop_connection; if (link->conf->csa_active) { - /* already processing - disregard action frames */ - if (!beacon) + switch (source) { + case IEEE80211_CSA_SOURCE_ACTION: + /* already processing - disregard action frames */ return; + case IEEE80211_CSA_SOURCE_BEACON: + if (link->u.mgd.csa.waiting_bcn) { + ieee80211_chswitch_post_beacon(link); + /* + * If the CSA is still present after the switch + * we need to consider it as a new CSA (possibly + * to self). This happens by not returning here + * so we'll get to the check below. + */ + } else if (res) { + ieee80211_sta_abort_chanswitch(link); + return; + } else { + drv_channel_switch_rx_beacon(sdata, &ch_switch); + return; + } + break; + case IEEE80211_CSA_SOURCE_OTHER_LINK: + /* active link: we want to see the beacon to continue */ + if (ieee80211_vif_link_active(&sdata->vif, + link->link_id)) + return; - if (link->u.mgd.csa_waiting_bcn) { - ieee80211_chswitch_post_beacon(link); - /* - * If the CSA IE is still present in the beacon after - * the switch, we need to consider it as a new CSA - * (possibly to self) - this happens by not returning - * here so we'll get to the check below. - */ - } else if (res) { - ieee80211_sta_abort_chanswitch(link); - return; - } else { - drv_channel_switch_rx_beacon(sdata, &ch_switch); + /* switch work ran, so just complete the process */ + if (link->u.mgd.csa.waiting_bcn) { + ieee80211_chswitch_post_beacon(link); + /* + * If the CSA is still present after the switch + * we need to consider it as a new CSA (possibly + * to self). This happens by not returning here + * so we'll get to the check below. + */ + break; + } + + /* link still has CSA but we already know, do nothing */ + if (!res) + return; + + /* check in the RNR if the CSA aborted */ + ieee80211_sta_other_link_csa_disappeared(link, + full_elems); return; } } @@ -2089,41 +2481,39 @@ ieee80211_sta_process_chanswitch(struct ieee80211_link_data *link, if (link->conf->chanreq.oper.chan->band != csa_ie.chanreq.oper.chan->band) { - sdata_info(sdata, - "AP %pM switches to different band (%d MHz, width:%d, CF1/2: %d/%d MHz), disconnecting\n", - link->u.mgd.bssid, - csa_ie.chanreq.oper.chan->center_freq, - csa_ie.chanreq.oper.width, - csa_ie.chanreq.oper.center_freq1, - csa_ie.chanreq.oper.center_freq2); + link_info(link, + "AP %pM switches to different band (%d MHz, width:%d, CF1/2: %d/%d MHz), disconnecting\n", + link->u.mgd.bssid, + csa_ie.chanreq.oper.chan->center_freq, + csa_ie.chanreq.oper.width, + csa_ie.chanreq.oper.center_freq1, + csa_ie.chanreq.oper.center_freq2); goto drop_connection; } if (!cfg80211_chandef_usable(local->hw.wiphy, &csa_ie.chanreq.oper, IEEE80211_CHAN_DISABLED)) { - sdata_info(sdata, - "AP %pM switches to unsupported channel " - "(%d.%03d MHz, width:%d, CF1/2: %d.%03d/%d MHz), " - "disconnecting\n", - link->u.mgd.bssid, - csa_ie.chanreq.oper.chan->center_freq, - csa_ie.chanreq.oper.chan->freq_offset, - csa_ie.chanreq.oper.width, - csa_ie.chanreq.oper.center_freq1, - csa_ie.chanreq.oper.freq1_offset, - csa_ie.chanreq.oper.center_freq2); + link_info(link, + "AP %pM switches to unsupported channel (%d.%03d MHz, width:%d, CF1/2: %d.%03d/%d MHz), disconnecting\n", + link->u.mgd.bssid, + csa_ie.chanreq.oper.chan->center_freq, + csa_ie.chanreq.oper.chan->freq_offset, + csa_ie.chanreq.oper.width, + csa_ie.chanreq.oper.center_freq1, + csa_ie.chanreq.oper.freq1_offset, + csa_ie.chanreq.oper.center_freq2); goto drop_connection; } if (cfg80211_chandef_identical(&csa_ie.chanreq.oper, &link->conf->chanreq.oper) && - (!csa_ie.mode || !beacon)) { - if (link->u.mgd.csa_ignored_same_chan) + (!csa_ie.mode || source != IEEE80211_CSA_SOURCE_BEACON)) { + if (link->u.mgd.csa.ignored_same_chan) return; - sdata_info(sdata, - "AP %pM tries to chanswitch to same channel, ignore\n", - link->u.mgd.bssid); - link->u.mgd.csa_ignored_same_chan = true; + link_info(link, + "AP %pM tries to chanswitch to same channel, ignore\n", + link->u.mgd.bssid); + link->u.mgd.csa.ignored_same_chan = true; return; } @@ -2138,40 +2528,48 @@ ieee80211_sta_process_chanswitch(struct ieee80211_link_data *link, conf = rcu_dereference_protected(link->conf->chanctx_conf, lockdep_is_held(&local->hw.wiphy->mtx)); - if (!conf) { - sdata_info(sdata, - "no channel context assigned to vif?, disconnecting\n"); + if (ieee80211_vif_link_active(&sdata->vif, link->link_id) && !conf) { + link_info(link, + "no channel context assigned to vif?, disconnecting\n"); goto drop_connection; } - chanctx = container_of(conf, struct ieee80211_chanctx, conf); + if (conf) + chanctx = container_of(conf, struct ieee80211_chanctx, conf); if (!ieee80211_hw_check(&local->hw, CHANCTX_STA_CSA)) { - sdata_info(sdata, - "driver doesn't support chan-switch with channel contexts\n"); + link_info(link, + "driver doesn't support chan-switch with channel contexts\n"); goto drop_connection; } if (drv_pre_channel_switch(sdata, &ch_switch)) { - sdata_info(sdata, - "preparing for channel switch failed, disconnecting\n"); + link_info(link, + "preparing for channel switch failed, disconnecting\n"); goto drop_connection; } - res = ieee80211_link_reserve_chanctx(link, &csa_ie.chanreq, - chanctx->mode, false); - if (res) { - sdata_info(sdata, - "failed to reserve channel context for channel switch, disconnecting (err=%d)\n", - res); - goto drop_connection; + link->u.mgd.csa.ap_chandef = csa_ie.chanreq.ap; + + link->csa.chanreq.oper = csa_ie.chanreq.oper; + ieee80211_set_chanreq_ap(sdata, &link->csa.chanreq, &link->u.mgd.conn, + &csa_ie.chanreq.ap); + + if (chanctx) { + res = ieee80211_link_reserve_chanctx(link, &link->csa.chanreq, + chanctx->mode, false); + if (res) { + link_info(link, + "failed to reserve channel context for channel switch, disconnecting (err=%d)\n", + res); + goto drop_connection; + } } link->conf->csa_active = true; - link->csa_chanreq = csa_ie.chanreq; - link->u.mgd.csa_ignored_same_chan = false; + link->u.mgd.csa.ignored_same_chan = false; link->u.mgd.beacon_crc_valid = false; - link->u.mgd.csa_blocked_tx = csa_ie.mode; + link->u.mgd.csa.blocked_tx = csa_ie.mode; if (csa_ie.mode && !ieee80211_hw_check(&local->hw, HANDLES_QUIET_CSA)) { @@ -2184,18 +2582,28 @@ ieee80211_sta_process_chanswitch(struct ieee80211_link_data *link, link->link_id, csa_ie.count, csa_ie.mode); - if (local->ops->channel_switch) { - /* use driver's channel switch callback */ + /* we may have to handle timeout for deactivated link in software */ + now = jiffies; + link->u.mgd.csa.time = now + + TU_TO_JIFFIES((max_t(int, csa_ie.count, 1) - 1) * + link->conf->beacon_int); + + if (ieee80211_vif_link_active(&sdata->vif, link->link_id) && + local->ops->channel_switch) { + /* + * Use driver's channel switch callback, the driver will + * later call ieee80211_chswitch_done(). It may deactivate + * the link as well, we handle that elsewhere and queue + * the csa.switch_work for the calculated time then. + */ drv_channel_switch(local, sdata, &ch_switch); return; } /* channel switch handled in software */ - timeout = TU_TO_JIFFIES((max_t(int, csa_ie.count, 1) - 1) * - cbss->beacon_interval); wiphy_delayed_work_queue(local->hw.wiphy, - &link->u.mgd.chswitch_work, - timeout); + &link->u.mgd.csa.switch_work, + link->u.mgd.csa.time - now); return; drop_connection: /* @@ -2206,7 +2614,7 @@ ieee80211_sta_process_chanswitch(struct ieee80211_link_data *link, * reset when the disconnection worker runs. */ link->conf->csa_active = true; - link->u.mgd.csa_blocked_tx = csa_ie.mode; + link->u.mgd.csa.blocked_tx = csa_ie.mode; sdata->csa_blocked_queues = csa_ie.mode && !ieee80211_hw_check(&local->hw, HANDLES_QUIET_CSA); @@ -2602,16 +3010,15 @@ void ieee80211_dynamic_ps_timer(struct timer_list *t) void ieee80211_dfs_cac_timer_work(struct wiphy *wiphy, struct wiphy_work *work) { - struct ieee80211_link_data *link = - container_of(work, struct ieee80211_link_data, + struct ieee80211_sub_if_data *sdata = + container_of(work, struct ieee80211_sub_if_data, dfs_cac_timer_work.work); - struct cfg80211_chan_def chandef = link->conf->chanreq.oper; - struct ieee80211_sub_if_data *sdata = link->sdata; + struct cfg80211_chan_def chandef = sdata->vif.bss_conf.chanreq.oper; lockdep_assert_wiphy(sdata->local->hw.wiphy); if (sdata->wdev.cac_started) { - ieee80211_link_release_channel(link); + ieee80211_link_release_channel(&sdata->deflink); cfg80211_cac_event(sdata->dev, &chandef, NL80211_RADAR_CAC_FINISHED, GFP_KERNEL); @@ -3260,9 +3667,9 @@ static void ieee80211_set_disassoc(struct ieee80211_sub_if_data *sdata, } sdata->vif.bss_conf.csa_active = false; - sdata->deflink.u.mgd.csa_blocked_tx = false; - sdata->deflink.u.mgd.csa_waiting_bcn = false; - sdata->deflink.u.mgd.csa_ignored_same_chan = false; + sdata->deflink.u.mgd.csa.blocked_tx = false; + sdata->deflink.u.mgd.csa.waiting_bcn = false; + sdata->deflink.u.mgd.csa.ignored_same_chan = false; if (sdata->csa_blocked_queues) { ieee80211_wake_vif_queues(local, sdata, IEEE80211_QUEUE_STOP_REASON_CSA); @@ -3275,9 +3682,7 @@ static void ieee80211_set_disassoc(struct ieee80211_sub_if_data *sdata, sdata->vif.bss_conf.power_type = IEEE80211_REG_UNSET_AP; sdata->vif.bss_conf.pwr_reduction = 0; - sdata->vif.bss_conf.tx_pwr_env_num = 0; - memset(sdata->vif.bss_conf.tx_pwr_env, 0, - sizeof(sdata->vif.bss_conf.tx_pwr_env)); + ieee80211_clear_tpe(&sdata->vif.bss_conf.tpe); sdata->vif.cfg.eml_cap = 0; sdata->vif.cfg.eml_med_sync_delay = 0; @@ -3287,8 +3692,17 @@ static void ieee80211_set_disassoc(struct ieee80211_sub_if_data *sdata, sizeof(sdata->u.mgd.ttlm_info)); wiphy_delayed_work_cancel(sdata->local->hw.wiphy, &ifmgd->ttlm_work); + memset(&sdata->vif.neg_ttlm, 0, sizeof(sdata->vif.neg_ttlm)); wiphy_delayed_work_cancel(sdata->local->hw.wiphy, &ifmgd->neg_ttlm_timeout_work); + + sdata->u.mgd.removed_links = 0; + wiphy_delayed_work_cancel(sdata->local->hw.wiphy, + &sdata->u.mgd.ml_reconf_work); + + wiphy_work_cancel(sdata->local->hw.wiphy, + &ifmgd->teardown_ttlm_work); + ieee80211_vif_set_links(sdata, 0, 0); ifmgd->mcast_seq_last = IEEE80211_SN_MODULO; @@ -3592,7 +4006,7 @@ static void __ieee80211_disconnect(struct ieee80211_sub_if_data *sdata) if (WARN_ON_ONCE(!link)) continue; - if (link->u.mgd.csa_blocked_tx) + if (link->u.mgd.csa.blocked_tx) continue; tx = true; @@ -3629,8 +4043,8 @@ static void __ieee80211_disconnect(struct ieee80211_sub_if_data *sdata) tx, frame_buf); /* the other links will be destroyed */ sdata->vif.bss_conf.csa_active = false; - sdata->deflink.u.mgd.csa_waiting_bcn = false; - sdata->deflink.u.mgd.csa_blocked_tx = false; + sdata->deflink.u.mgd.csa.waiting_bcn = false; + sdata->deflink.u.mgd.csa.blocked_tx = false; if (sdata->csa_blocked_queues) { ieee80211_wake_vif_queues(local, sdata, IEEE80211_QUEUE_STOP_REASON_CSA); @@ -4445,40 +4859,12 @@ static bool ieee80211_assoc_config_link(struct ieee80211_link_data *link, if (elems->he_operation && link->u.mgd.conn.mode >= IEEE80211_CONN_MODE_HE && elems->he_cap) { - const struct ieee80211_he_6ghz_oper *he_6ghz_oper; - ieee80211_he_cap_ie_to_sta_he_cap(sdata, sband, elems->he_cap, elems->he_cap_len, elems->he_6ghz_capa, link_sta); - he_6ghz_oper = ieee80211_he_6ghz_oper(elems->he_operation); - - if (is_6ghz && he_6ghz_oper) { - switch (u8_get_bits(he_6ghz_oper->control, - IEEE80211_HE_6GHZ_OPER_CTRL_REG_INFO)) { - case IEEE80211_6GHZ_CTRL_REG_LPI_AP: - case IEEE80211_6GHZ_CTRL_REG_INDOOR_LPI_AP: - bss_conf->power_type = IEEE80211_REG_LPI_AP; - break; - case IEEE80211_6GHZ_CTRL_REG_SP_AP: - case IEEE80211_6GHZ_CTRL_REG_INDOOR_SP_AP: - bss_conf->power_type = IEEE80211_REG_SP_AP; - break; - case IEEE80211_6GHZ_CTRL_REG_VLP_AP: - bss_conf->power_type = IEEE80211_REG_VLP_AP; - break; - default: - bss_conf->power_type = IEEE80211_REG_UNSET_AP; - break; - } - } else if (is_6ghz) { - link_info(link, - "HE 6 GHz operation missing (on %d MHz), expect issues\n", - bss_conf->chanreq.oper.chan->center_freq); - } - bss_conf->he_support = link_sta->pub->he_cap.has_he; if (elems->rsnx && elems->rsnx_len && (elems->rsnx[0] & WLAN_RSNX_CAPA_PROTECTED_TWT) && @@ -5020,6 +5406,23 @@ ieee80211_determine_our_sta_mode_assoc(struct ieee80211_sub_if_data *sdata, conn->bw_limit, tmp.bw_limit); } +static enum ieee80211_ap_reg_power +ieee80211_ap_power_type(u8 control) +{ + switch (u8_get_bits(control, IEEE80211_HE_6GHZ_OPER_CTRL_REG_INFO)) { + case IEEE80211_6GHZ_CTRL_REG_LPI_AP: + case IEEE80211_6GHZ_CTRL_REG_INDOOR_LPI_AP: + return IEEE80211_REG_LPI_AP; + case IEEE80211_6GHZ_CTRL_REG_SP_AP: + case IEEE80211_6GHZ_CTRL_REG_INDOOR_SP_AP: + return IEEE80211_REG_SP_AP; + case IEEE80211_6GHZ_CTRL_REG_VLP_AP: + return IEEE80211_REG_VLP_AP; + default: + return IEEE80211_REG_UNSET_AP; + } +} + static int ieee80211_prep_channel(struct ieee80211_sub_if_data *sdata, struct ieee80211_link_data *link, int link_id, @@ -5029,15 +5432,15 @@ static int ieee80211_prep_channel(struct ieee80211_sub_if_data *sdata, struct ieee80211_local *local = sdata->local; bool is_6ghz = cbss->channel->band == NL80211_BAND_6GHZ; struct ieee80211_chan_req chanreq = {}; + struct cfg80211_chan_def ap_chandef; struct ieee802_11_elems *elems; int ret; - u32 i; lockdep_assert_wiphy(local->hw.wiphy); rcu_read_lock(); elems = ieee80211_determine_chan_mode(sdata, conn, cbss, link_id, - &chanreq); + &chanreq, &ap_chandef); if (IS_ERR(elems)) { rcu_read_unlock(); @@ -5052,26 +5455,23 @@ static int ieee80211_prep_channel(struct ieee80211_sub_if_data *sdata, } if (link && is_6ghz && conn->mode >= IEEE80211_CONN_MODE_HE) { - struct ieee80211_bss_conf *bss_conf; - u8 j = 0; - - bss_conf = link->conf; + const struct ieee80211_he_6ghz_oper *he_6ghz_oper; if (elems->pwr_constr_elem) - bss_conf->pwr_reduction = *elems->pwr_constr_elem; + link->conf->pwr_reduction = *elems->pwr_constr_elem; - BUILD_BUG_ON(ARRAY_SIZE(bss_conf->tx_pwr_env) != - ARRAY_SIZE(elems->tx_pwr_env)); - - for (i = 0; i < elems->tx_pwr_env_num; i++) { - if (elems->tx_pwr_env_len[i] > sizeof(bss_conf->tx_pwr_env[j])) - continue; + he_6ghz_oper = ieee80211_he_6ghz_oper(elems->he_operation); + if (he_6ghz_oper) + link->conf->power_type = + ieee80211_ap_power_type(he_6ghz_oper->control); + else + link_info(link, + "HE 6 GHz operation missing (on %d MHz), expect issues\n", + cbss->channel->center_freq); - bss_conf->tx_pwr_env_num++; - memcpy(&bss_conf->tx_pwr_env[j], elems->tx_pwr_env[i], - elems->tx_pwr_env_len[i]); - j++; - } + link->conf->tpe = elems->tpe; + ieee80211_rearrange_tpe(&link->conf->tpe, &ap_chandef, + &chanreq.oper); } rcu_read_unlock(); /* the element data was RCU protected so no longer valid anyway */ @@ -6150,6 +6550,110 @@ static void ieee80211_process_adv_ttlm(struct ieee80211_sub_if_data *sdata, } } +static void +ieee80211_mgd_check_cross_link_csa(struct ieee80211_sub_if_data *sdata, + int reporting_link_id, + struct ieee802_11_elems *elems) +{ + const struct element *sta_profiles[IEEE80211_MLD_MAX_NUM_LINKS] = {}; + ssize_t sta_profiles_len[IEEE80211_MLD_MAX_NUM_LINKS] = {}; + const struct element *sub; + const u8 *subelems; + size_t subelems_len; + u8 common_size; + int link_id; + + if (!ieee80211_mle_size_ok((u8 *)elems->ml_basic, elems->ml_basic_len)) + return; + + common_size = ieee80211_mle_common_size((u8 *)elems->ml_basic); + subelems = (u8 *)elems->ml_basic + common_size; + subelems_len = elems->ml_basic_len - common_size; + + for_each_element_id(sub, IEEE80211_MLE_SUBELEM_PER_STA_PROFILE, + subelems, subelems_len) { + struct ieee80211_mle_per_sta_profile *prof = (void *)sub->data; + struct ieee80211_link_data *link; + ssize_t len; + + if (!ieee80211_mle_basic_sta_prof_size_ok(sub->data, + sub->datalen)) + continue; + + link_id = le16_get_bits(prof->control, + IEEE80211_MLE_STA_CONTROL_LINK_ID); + /* need a valid link ID, but also not our own, both AP bugs */ + if (link_id == reporting_link_id || + link_id >= IEEE80211_MLD_MAX_NUM_LINKS) + continue; + + link = sdata_dereference(sdata->link[link_id], sdata); + if (!link) + continue; + + len = cfg80211_defragment_element(sub, subelems, subelems_len, + NULL, 0, + IEEE80211_MLE_SUBELEM_FRAGMENT); + if (WARN_ON(len < 0)) + continue; + + sta_profiles[link_id] = sub; + sta_profiles_len[link_id] = len; + } + + for (link_id = 0; link_id < IEEE80211_MLD_MAX_NUM_LINKS; link_id++) { + struct ieee80211_mle_per_sta_profile *prof; + struct ieee802_11_elems *prof_elems; + struct ieee80211_link_data *link; + ssize_t len; + + if (link_id == reporting_link_id) + continue; + + link = sdata_dereference(sdata->link[link_id], sdata); + if (!link) + continue; + + if (!sta_profiles[link_id]) { + prof_elems = NULL; + goto handle; + } + + /* we can defragment in-place, won't use the buffer again */ + len = cfg80211_defragment_element(sta_profiles[link_id], + subelems, subelems_len, + (void *)sta_profiles[link_id], + sta_profiles_len[link_id], + IEEE80211_MLE_SUBELEM_FRAGMENT); + if (WARN_ON(len != sta_profiles_len[link_id])) + continue; + + prof = (void *)sta_profiles[link_id]; + prof_elems = ieee802_11_parse_elems(prof->variable + + (prof->sta_info_len - 1), + len - + (prof->sta_info_len - 1), + false, NULL); + + /* memory allocation failed - let's hope that's transient */ + if (!prof_elems) + continue; + +handle: + /* + * FIXME: the timings here are obviously incorrect, + * but only older Intel drivers seem to care, and + * those don't have MLO. If you really need this, + * the problem is having to calculate it with the + * TSF offset etc. The device_timestamp is still + * correct, of course. + */ + ieee80211_sta_process_chanswitch(link, 0, 0, elems, prof_elems, + IEEE80211_CSA_SOURCE_OTHER_LINK); + kfree(prof_elems); + } +} + static void ieee80211_rx_mgmt_beacon(struct ieee80211_link_data *link, struct ieee80211_hdr *hdr, size_t len, struct ieee80211_rx_status *rx_status) @@ -6374,7 +6878,11 @@ static void ieee80211_rx_mgmt_beacon(struct ieee80211_link_data *link, ieee80211_sta_process_chanswitch(link, rx_status->mactime, rx_status->device_timestamp, - elems, true); + elems, elems, + IEEE80211_CSA_SOURCE_BEACON); + + /* note that after this elems->ml_basic can no longer be used fully */ + ieee80211_mgd_check_cross_link_csa(sdata, rx_status->link_id, elems); if (!link->u.mgd.disable_wmm_tracking && ieee80211_sta_wmm_params(local, link, elems->wmm_param, @@ -6834,7 +7342,7 @@ static void ieee80211_teardown_ttlm_work(struct wiphy *wiphy, u16 new_dormant_links; struct ieee80211_sub_if_data *sdata = container_of(work, struct ieee80211_sub_if_data, - u.mgd.neg_ttlm_timeout_work.work); + u.mgd.teardown_ttlm_work); if (!sdata->vif.neg_ttlm.valid) return; @@ -6970,7 +7478,8 @@ void ieee80211_sta_rx_queued_mgmt(struct ieee80211_sub_if_data *sdata, ieee80211_sta_process_chanswitch(link, rx_status->mactime, rx_status->device_timestamp, - elems, false); + elems, elems, + IEEE80211_CSA_SOURCE_ACTION); kfree(elems); } else if (mgmt->u.action.category == WLAN_CATEGORY_PUBLIC) { struct ieee802_11_elems *elems; @@ -6998,7 +7507,8 @@ void ieee80211_sta_rx_queued_mgmt(struct ieee80211_sub_if_data *sdata, ieee80211_sta_process_chanswitch(link, rx_status->mactime, rx_status->device_timestamp, - elems, false); + elems, elems, + IEEE80211_CSA_SOURCE_ACTION); } kfree(elems); @@ -7321,7 +7831,7 @@ static void ieee80211_sta_bcn_mon_timer(struct timer_list *t) return; if (sdata->vif.bss_conf.csa_active && - !sdata->deflink.u.mgd.csa_waiting_bcn) + !sdata->deflink.u.mgd.csa.waiting_bcn) return; if (sdata->vif.driver_flags & IEEE80211_VIF_BEACON_FILTER) @@ -7345,7 +7855,7 @@ static void ieee80211_sta_conn_mon_timer(struct timer_list *t) return; if (sdata->vif.bss_conf.csa_active && - !sdata->deflink.u.mgd.csa_waiting_bcn) + !sdata->deflink.u.mgd.csa.waiting_bcn) return; sta = sta_info_get(sdata, sdata->vif.cfg.ap_addr); @@ -7556,8 +8066,10 @@ void ieee80211_mgd_setup_link(struct ieee80211_link_data *link) else link->u.mgd.req_smps = IEEE80211_SMPS_OFF; - wiphy_delayed_work_init(&link->u.mgd.chswitch_work, - ieee80211_chswitch_work); + wiphy_delayed_work_init(&link->u.mgd.csa.switch_work, + ieee80211_csa_switch_work); + + ieee80211_clear_tpe(&link->conf->tpe); if (sdata->u.mgd.assoc_data) ether_addr_copy(link->conf->addr, @@ -8686,7 +9198,7 @@ void ieee80211_mgd_stop_link(struct ieee80211_link_data *link) wiphy_work_cancel(link->sdata->local->hw.wiphy, &link->u.mgd.recalc_smps); wiphy_delayed_work_cancel(link->sdata->local->hw.wiphy, - &link->u.mgd.chswitch_work); + &link->u.mgd.csa.switch_work); } void ieee80211_mgd_stop(struct ieee80211_sub_if_data *sdata) @@ -8704,15 +9216,8 @@ void ieee80211_mgd_stop(struct ieee80211_sub_if_data *sdata) &ifmgd->beacon_connection_loss_work); wiphy_work_cancel(sdata->local->hw.wiphy, &ifmgd->csa_connection_drop_work); - wiphy_work_cancel(sdata->local->hw.wiphy, - &ifmgd->teardown_ttlm_work); wiphy_delayed_work_cancel(sdata->local->hw.wiphy, &ifmgd->tdls_peer_del_work); - wiphy_delayed_work_cancel(sdata->local->hw.wiphy, - &ifmgd->ml_reconf_work); - wiphy_delayed_work_cancel(sdata->local->hw.wiphy, &ifmgd->ttlm_work); - wiphy_delayed_work_cancel(sdata->local->hw.wiphy, - &ifmgd->neg_ttlm_timeout_work); if (ifmgd->assoc_data) ieee80211_destroy_assoc_data(sdata, ASSOC_TIMEOUT); diff --git a/net/mac80211/offchannel.c b/net/mac80211/offchannel.c index 65e1e9e971fd69893b5231185a397756a9d9bbf7..28d03196ef75a7c15f5244684c6d11202c96d9db 100644 --- a/net/mac80211/offchannel.c +++ b/net/mac80211/offchannel.c @@ -8,7 +8,7 @@ * Copyright 2006-2007 Jiri Benc * Copyright 2007, Michael Wu * Copyright 2009 Johannes Berg - * Copyright (C) 2019, 2022-2023 Intel Corporation + * Copyright (C) 2019, 2022-2024 Intel Corporation */ #include #include @@ -413,6 +413,39 @@ void ieee80211_start_next_roc(struct ieee80211_local *local) } } +void ieee80211_reconfig_roc(struct ieee80211_local *local) +{ + struct ieee80211_roc_work *roc, *tmp; + + /* + * In the software implementation can just continue with the + * interruption due to reconfig, roc_work is still queued if + * needed. + */ + if (!local->ops->remain_on_channel) + return; + + /* flush work so nothing from the driver is still pending */ + wiphy_work_flush(local->hw.wiphy, &local->hw_roc_start); + wiphy_work_flush(local->hw.wiphy, &local->hw_roc_done); + + list_for_each_entry_safe(roc, tmp, &local->roc_list, list) { + if (!roc->started) + break; + + if (!roc->hw_begun) { + /* it didn't start in HW yet, so we can restart it */ + roc->started = false; + continue; + } + + /* otherwise destroy it and tell userspace */ + ieee80211_roc_notify_destroy(roc); + } + + ieee80211_start_next_roc(local); +} + static void __ieee80211_roc_work(struct ieee80211_local *local) { struct ieee80211_roc_work *roc; diff --git a/net/mac80211/parse.c b/net/mac80211/parse.c index 55e5497f89781c0fc2c74965b30b966366672d25..28aae14db8a9883c31dc11635c0f9e5dc54546bb 100644 --- a/net/mac80211/parse.c +++ b/net/mac80211/parse.c @@ -111,7 +111,7 @@ ieee80211_parse_extension_element(u32 *crc, if (params->mode < IEEE80211_CONN_MODE_HE) break; if (len >= sizeof(*elems->he_spr) && - len >= ieee80211_he_spr_size(data)) + len >= ieee80211_he_spr_size(data) - 1) elems->he_spr = data; break; case WLAN_EID_EXT_HE_6GHZ_CAPA: @@ -187,6 +187,84 @@ ieee80211_parse_extension_element(u32 *crc, *crc = crc32_be(*crc, (void *)elem, elem->datalen + 2); } +static void ieee80211_parse_tpe(struct ieee80211_parsed_tpe *tpe, + const u8 *data, u8 len) +{ + const struct ieee80211_tx_pwr_env *env = (const void *)data; + u8 count, interpret, category; + u8 *out, N, *cnt_out = NULL, *N_out = NULL; + + if (!ieee80211_valid_tpe_element(data, len)) + return; + + count = u8_get_bits(env->info, IEEE80211_TX_PWR_ENV_INFO_COUNT); + interpret = u8_get_bits(env->info, IEEE80211_TX_PWR_ENV_INFO_INTERPRET); + category = u8_get_bits(env->info, IEEE80211_TX_PWR_ENV_INFO_CATEGORY); + + switch (interpret) { + case IEEE80211_TPE_LOCAL_EIRP: + out = tpe->max_local[category].power; + cnt_out = &tpe->max_local[category].count; + tpe->max_local[category].valid = true; + break; + case IEEE80211_TPE_REG_CLIENT_EIRP: + out = tpe->max_reg_client[category].power; + cnt_out = &tpe->max_reg_client[category].count; + tpe->max_reg_client[category].valid = true; + break; + case IEEE80211_TPE_LOCAL_EIRP_PSD: + out = tpe->psd_local[category].power; + cnt_out = &tpe->psd_local[category].count; + N_out = &tpe->psd_local[category].n; + tpe->psd_local[category].valid = true; + break; + case IEEE80211_TPE_REG_CLIENT_EIRP_PSD: + out = tpe->psd_reg_client[category].power; + cnt_out = &tpe->psd_reg_client[category].count; + N_out = &tpe->psd_reg_client[category].n; + tpe->psd_reg_client[category].valid = true; + break; + } + + switch (interpret) { + case IEEE80211_TPE_LOCAL_EIRP: + case IEEE80211_TPE_REG_CLIENT_EIRP: + /* count was validated <= 3, plus 320 MHz */ + BUILD_BUG_ON(IEEE80211_TPE_EIRP_ENTRIES_320MHZ < 5); + memcpy(out, env->variable, count + 1); + *cnt_out = count + 1; + /* separately take 320 MHz if present */ + if (count == 3 && len > sizeof(*env) + count + 1) { + out[4] = env->variable[count + 2]; + *cnt_out = 5; + } + break; + case IEEE80211_TPE_LOCAL_EIRP_PSD: + case IEEE80211_TPE_REG_CLIENT_EIRP_PSD: + if (!count) { + memset(out, env->variable[0], + IEEE80211_TPE_PSD_ENTRIES_320MHZ); + *cnt_out = IEEE80211_TPE_PSD_ENTRIES_320MHZ; + break; + } + + N = 1 << (count - 1); + memcpy(out, env->variable, N); + *cnt_out = N; + *N_out = N; + + if (len > sizeof(*env) + N) { + int K = u8_get_bits(env->variable[N], + IEEE80211_TX_PWR_ENV_EXT_COUNT); + + K = min(K, IEEE80211_TPE_PSD_ENTRIES_320MHZ - N); + memcpy(out + N, env->variable + N + 1, K); + (*cnt_out) += K; + } + break; + } +} + static u32 _ieee802_11_parse_elems_full(struct ieee80211_elems_parse_params *params, struct ieee80211_elems_parse *elems_parse, @@ -529,6 +607,13 @@ _ieee802_11_parse_elems_full(struct ieee80211_elems_parse_params *params, elem_parse_failed = IEEE80211_PARSE_ERR_BAD_ELEM_SIZE; } + + subelem = cfg80211_find_ext_elem(WLAN_EID_TX_POWER_ENVELOPE, + pos, elen); + if (subelem) + ieee80211_parse_tpe(&elems->csa_tpe, + subelem->data + 1, + subelem->datalen - 1); break; case WLAN_EID_COUNTRY: elems->country_elem = pos; @@ -593,16 +678,9 @@ _ieee802_11_parse_elems_full(struct ieee80211_elems_parse_params *params, elems->rsnx_len = elen; break; case WLAN_EID_TX_POWER_ENVELOPE: - if (elen < 1 || - elen > sizeof(struct ieee80211_tx_pwr_env)) + if (params->mode < IEEE80211_CONN_MODE_HE) break; - - if (elems->tx_pwr_env_num >= ARRAY_SIZE(elems->tx_pwr_env)) - break; - - elems->tx_pwr_env[elems->tx_pwr_env_num] = (void *)pos; - elems->tx_pwr_env_len[elems->tx_pwr_env_num] = elen; - elems->tx_pwr_env_num++; + ieee80211_parse_tpe(&elems->tpe, pos, elen); break; case WLAN_EID_EXTENSION: ieee80211_parse_extension_element(calc_crc ? @@ -889,6 +967,10 @@ ieee802_11_parse_elems_full(struct ieee80211_elems_parse_params *params) elems->ie_start = params->start; elems->total_len = params->len; + /* set all TPE entries to unlimited (but invalid) */ + ieee80211_clear_tpe(&elems->tpe); + ieee80211_clear_tpe(&elems->csa_tpe); + nontransmitted_profile = elems_parse->scratch_pos; nontransmitted_profile_len = ieee802_11_find_bssid_profile(params->start, params->len, diff --git a/net/mac80211/scan.c b/net/mac80211/scan.c index 3da1c5c450358cb77422d48b9e9a0dd5542a7508..8ecc4b710b0e64449c2a03339b3d3093ec939b7b 100644 --- a/net/mac80211/scan.c +++ b/net/mac80211/scan.c @@ -744,15 +744,21 @@ static int __ieee80211_start_scan(struct ieee80211_sub_if_data *sdata, local->hw_scan_ies_bufsize *= n_bands; } - local->hw_scan_req = kmalloc( - sizeof(*local->hw_scan_req) + - req->n_channels * sizeof(req->channels[0]) + - local->hw_scan_ies_bufsize, GFP_KERNEL); + local->hw_scan_req = kmalloc(struct_size(local->hw_scan_req, + req.channels, + req->n_channels) + + local->hw_scan_ies_bufsize, + GFP_KERNEL); if (!local->hw_scan_req) return -ENOMEM; local->hw_scan_req->req.ssids = req->ssids; local->hw_scan_req->req.n_ssids = req->n_ssids; + /* None of the channels are actually set + * up but let UBSAN know the boundaries. + */ + local->hw_scan_req->req.n_channels = req->n_channels; + ies = (u8 *)local->hw_scan_req + sizeof(*local->hw_scan_req) + req->n_channels * sizeof(req->channels[0]); diff --git a/net/mac80211/spectmgmt.c b/net/mac80211/spectmgmt.c index b2de4c6fb808860b7ae5a7f94c21a064fb877f5e..df96d3db1c0e6ed106e7b0032657c6f7e4f591ce 100644 --- a/net/mac80211/spectmgmt.c +++ b/net/mac80211/spectmgmt.c @@ -9,7 +9,7 @@ * Copyright 2007, Michael Wu * Copyright 2007-2008, Intel Corporation * Copyright 2008, Johannes Berg - * Copyright (C) 2018, 2020, 2022-2023 Intel Corporation + * Copyright (C) 2018, 2020, 2022-2024 Intel Corporation */ #include @@ -366,6 +366,9 @@ int ieee80211_parse_ch_switch_ie(struct ieee80211_sub_if_data *sdata, validate_chandef_by_ht_vht_oper(sdata, conn, vht_cap_info, &new_chandef); + /* capture the AP chandef before (potential) downgrading */ + csa_ie->chanreq.ap = new_chandef; + /* if data is there validate the bandwidth & use it */ if (new_chandef.chan) { if (conn->bw_limit < IEEE80211_CONN_BW_LIMIT_320 && diff --git a/net/mac80211/sta_info.c b/net/mac80211/sta_info.c index da5fdd6f5c852ba5514f1d04e3ef71398b126b9c..aa22f09e6d145f3e7fb40b95005fb434856d6d26 100644 --- a/net/mac80211/sta_info.c +++ b/net/mac80211/sta_info.c @@ -1724,7 +1724,7 @@ void ieee80211_sta_ps_deliver_wakeup(struct sta_info *sta) skb_queue_head_init(&pending); /* sync with ieee80211_tx_h_unicast_ps_buf */ - spin_lock(&sta->ps_lock); + spin_lock_bh(&sta->ps_lock); /* Send all buffered frames to the station */ for (ac = 0; ac < IEEE80211_NUM_ACS; ac++) { int count = skb_queue_len(&pending), tmp; @@ -1753,7 +1753,7 @@ void ieee80211_sta_ps_deliver_wakeup(struct sta_info *sta) */ clear_sta_flag(sta, WLAN_STA_PSPOLL); clear_sta_flag(sta, WLAN_STA_UAPSD); - spin_unlock(&sta->ps_lock); + spin_unlock_bh(&sta->ps_lock); atomic_dec(&ps->num_sta_ps); diff --git a/net/mac80211/tests/Makefile b/net/mac80211/tests/Makefile index 4fdaf3feaca3105d744ff1095f101615416ede67..511dfa226699430ae500b320b5a2dcf249af1e76 100644 --- a/net/mac80211/tests/Makefile +++ b/net/mac80211/tests/Makefile @@ -1,3 +1,3 @@ -mac80211-tests-y += module.o elems.o mfp.o +mac80211-tests-y += module.o elems.o mfp.o tpe.o obj-$(CONFIG_MAC80211_KUNIT_TEST) += mac80211-tests.o diff --git a/net/mac80211/tests/tpe.c b/net/mac80211/tests/tpe.c new file mode 100644 index 0000000000000000000000000000000000000000..dd63303a2985b96372957711cf3119094574d82d --- /dev/null +++ b/net/mac80211/tests/tpe.c @@ -0,0 +1,284 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * KUnit tests for TPE element handling + * + * Copyright (C) 2024 Intel Corporation + */ +#include +#include "../ieee80211_i.h" + +MODULE_IMPORT_NS(EXPORTED_FOR_KUNIT_TESTING); + +static struct ieee80211_channel chan6g_1 = { + .band = NL80211_BAND_6GHZ, + .center_freq = 5955, +}; + +static struct ieee80211_channel chan6g_33 = { + .band = NL80211_BAND_6GHZ, + .center_freq = 6115, +}; + +static struct ieee80211_channel chan6g_61 = { + .band = NL80211_BAND_6GHZ, + .center_freq = 6255, +}; + +static const struct subchan_test_case { + const char *desc; + struct cfg80211_chan_def c; + u8 n; + int expect; +} subchan_offset_cases[] = { + { + .desc = "identical 20 MHz", + .c.width = NL80211_CHAN_WIDTH_20, + .c.chan = &chan6g_1, + .c.center_freq1 = 5955, + .n = 1, + .expect = 0, + }, + { + .desc = "identical 40 MHz", + .c.width = NL80211_CHAN_WIDTH_40, + .c.chan = &chan6g_1, + .c.center_freq1 = 5965, + .n = 2, + .expect = 0, + }, + { + .desc = "identical 80+80 MHz", + /* not really is valid? doesn't matter for the test */ + .c.width = NL80211_CHAN_WIDTH_80P80, + .c.chan = &chan6g_1, + .c.center_freq1 = 5985, + .c.center_freq2 = 6225, + .n = 16, + .expect = 0, + }, + { + .desc = "identical 320 MHz", + .c.width = NL80211_CHAN_WIDTH_320, + .c.chan = &chan6g_1, + .c.center_freq1 = 6105, + .n = 16, + .expect = 0, + }, + { + .desc = "lower 160 MHz of 320 MHz", + .c.width = NL80211_CHAN_WIDTH_320, + .c.chan = &chan6g_1, + .c.center_freq1 = 6105, + .n = 8, + .expect = 0, + }, + { + .desc = "upper 160 MHz of 320 MHz", + .c.width = NL80211_CHAN_WIDTH_320, + .c.chan = &chan6g_61, + .c.center_freq1 = 6105, + .n = 8, + .expect = 8, + }, + { + .desc = "upper 160 MHz of 320 MHz, go to 40", + .c.width = NL80211_CHAN_WIDTH_320, + .c.chan = &chan6g_61, + .c.center_freq1 = 6105, + .n = 2, + .expect = 8 + 4 + 2, + }, + { + .desc = "secondary 80 above primary in 80+80 MHz", + /* not really is valid? doesn't matter for the test */ + .c.width = NL80211_CHAN_WIDTH_80P80, + .c.chan = &chan6g_1, + .c.center_freq1 = 5985, + .c.center_freq2 = 6225, + .n = 4, + .expect = 0, + }, + { + .desc = "secondary 80 below primary in 80+80 MHz", + /* not really is valid? doesn't matter for the test */ + .c.width = NL80211_CHAN_WIDTH_80P80, + .c.chan = &chan6g_61, + .c.center_freq1 = 6225, + .c.center_freq2 = 5985, + .n = 4, + .expect = 4, + }, + { + .desc = "secondary 80 below primary in 80+80 MHz, go to 20", + /* not really is valid? doesn't matter for the test */ + .c.width = NL80211_CHAN_WIDTH_80P80, + .c.chan = &chan6g_61, + .c.center_freq1 = 6225, + .c.center_freq2 = 5985, + .n = 1, + .expect = 7, + }, +}; + +KUNIT_ARRAY_PARAM_DESC(subchan_offset, subchan_offset_cases, desc); + +static void subchan_offset(struct kunit *test) +{ + const struct subchan_test_case *params = test->param_value; + int offset; + + KUNIT_ASSERT_EQ(test, cfg80211_chandef_valid(¶ms->c), true); + + offset = ieee80211_calc_chandef_subchan_offset(¶ms->c, params->n); + + KUNIT_EXPECT_EQ(test, params->expect, offset); +} + +static const struct psd_reorder_test_case { + const char *desc; + struct cfg80211_chan_def ap, used; + struct ieee80211_parsed_tpe_psd psd, out; +} psd_reorder_cases[] = { + { + .desc = "no changes, 320 MHz", + + .ap.width = NL80211_CHAN_WIDTH_320, + .ap.chan = &chan6g_1, + .ap.center_freq1 = 6105, + + .used.width = NL80211_CHAN_WIDTH_320, + .used.chan = &chan6g_1, + .used.center_freq1 = 6105, + + .psd.valid = true, + .psd.count = 16, + .psd.n = 8, + .psd.power = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 }, + + .out.valid = true, + .out.count = 16, + .out.n = 8, + .out.power = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 }, + }, + { + .desc = "no changes, 320 MHz, 160 MHz used, n=0", + + .ap.width = NL80211_CHAN_WIDTH_320, + .ap.chan = &chan6g_1, + .ap.center_freq1 = 6105, + + .used.width = NL80211_CHAN_WIDTH_160, + .used.chan = &chan6g_1, + .used.center_freq1 = 6025, + + .psd.valid = true, + .psd.count = 16, + .psd.n = 0, + .psd.power = { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, }, + + .out.valid = true, + .out.count = 8, + .out.n = 0, + .out.power = { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, }, + }, + { + .desc = "320 MHz, HE is 80, used 160, all lower", + + .ap.width = NL80211_CHAN_WIDTH_320, + .ap.chan = &chan6g_1, + .ap.center_freq1 = 6105, + + .used.width = NL80211_CHAN_WIDTH_160, + .used.chan = &chan6g_1, + .used.center_freq1 = 6025, + + .psd.valid = true, + .psd.count = 16, + .psd.n = 4, + .psd.power = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 }, + + .out.valid = true, + .out.count = 8, + .out.n = 4, + .out.power = { 0, 1, 2, 3, 4, 5, 6, 7, 127, 127, 127, 127, 127, 127, 127, 127}, + }, + { + .desc = "320 MHz, HE is 80, used 160, all upper", + /* + * EHT: | | | | | | | | | | | | | | | | | + * HE: | | | | | + * used: | | | | | | | | | + */ + + .ap.width = NL80211_CHAN_WIDTH_320, + .ap.chan = &chan6g_61, + .ap.center_freq1 = 6105, + + .used.width = NL80211_CHAN_WIDTH_160, + .used.chan = &chan6g_61, + .used.center_freq1 = 6185, + + .psd.valid = true, + .psd.count = 16, + .psd.n = 4, + .psd.power = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 }, + + .out.valid = true, + .out.count = 8, + .out.n = 4, + .out.power = { 12, 13, 14, 15, 0, 1, 2, 3, 127, 127, 127, 127, 127, 127, 127, 127}, + }, + { + .desc = "320 MHz, HE is 80, used 160, split", + /* + * EHT: | | | | | | | | | | | | | | | | | + * HE: | | | | | + * used: | | | | | | | | | + */ + + .ap.width = NL80211_CHAN_WIDTH_320, + .ap.chan = &chan6g_33, + .ap.center_freq1 = 6105, + + .used.width = NL80211_CHAN_WIDTH_160, + .used.chan = &chan6g_33, + .used.center_freq1 = 6185, + + .psd.valid = true, + .psd.count = 16, + .psd.n = 4, + .psd.power = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 }, + + .out.valid = true, + .out.count = 8, + .out.n = 4, + .out.power = { 0, 1, 2, 3, 12, 13, 14, 15, 127, 127, 127, 127, 127, 127, 127, 127}, + }, +}; + +KUNIT_ARRAY_PARAM_DESC(psd_reorder, psd_reorder_cases, desc); + +static void psd_reorder(struct kunit *test) +{ + const struct psd_reorder_test_case *params = test->param_value; + struct ieee80211_parsed_tpe_psd tmp = params->psd; + + KUNIT_ASSERT_EQ(test, cfg80211_chandef_valid(¶ms->ap), true); + KUNIT_ASSERT_EQ(test, cfg80211_chandef_valid(¶ms->used), true); + + ieee80211_rearrange_tpe_psd(&tmp, ¶ms->ap, ¶ms->used); + KUNIT_EXPECT_MEMEQ(test, &tmp, ¶ms->out, sizeof(tmp)); +} + +static struct kunit_case tpe_test_cases[] = { + KUNIT_CASE_PARAM(subchan_offset, subchan_offset_gen_params), + KUNIT_CASE_PARAM(psd_reorder, psd_reorder_gen_params), + {} +}; + +static struct kunit_suite tpe = { + .name = "mac80211-tpe", + .test_cases = tpe_test_cases, +}; + +kunit_test_suite(tpe); diff --git a/net/mac80211/util.c b/net/mac80211/util.c index 0b893e958959440e4b918e284533933a5c6145da..c73ff7dfbdba3ae32805c92fc010587014436002 100644 --- a/net/mac80211/util.c +++ b/net/mac80211/util.c @@ -1567,6 +1567,8 @@ u32 ieee80211_sta_get_rates(struct ieee80211_sub_if_data *sdata, void ieee80211_stop_device(struct ieee80211_local *local) { + ieee80211_handle_queued_frames(local); + ieee80211_led_radio(local, false); ieee80211_mod_tpt_led_trig(local, 0, IEEE80211_TPT_LEDTRIG_FL_RADIO); @@ -2175,8 +2177,7 @@ int ieee80211_reconfig(struct ieee80211_local *local) local->in_reconfig = false; barrier(); - /* Restart deferred ROCs */ - ieee80211_start_next_roc(local); + ieee80211_reconfig_roc(local); /* Requeue all works */ list_for_each_entry(sdata, &local->interfaces, list) @@ -3456,12 +3457,8 @@ void ieee80211_dfs_cac_cancel(struct ieee80211_local *local) lockdep_assert_wiphy(local->hw.wiphy); list_for_each_entry(sdata, &local->interfaces, list) { - /* it might be waiting for the local->mtx, but then - * by the time it gets it, sdata->wdev.cac_started - * will no longer be true - */ wiphy_delayed_work_cancel(local->hw.wiphy, - &sdata->deflink.dfs_cac_timer_work); + &sdata->dfs_cac_timer_work); if (sdata->wdev.cac_started) { chandef = sdata->vif.bss_conf.chanreq.oper; @@ -4338,3 +4335,28 @@ ieee80211_min_bw_limit_from_chandef(struct cfg80211_chan_def *chandef) return IEEE80211_CONN_BW_LIMIT_20; } } + +void ieee80211_clear_tpe(struct ieee80211_parsed_tpe *tpe) +{ + for (int i = 0; i < 2; i++) { + tpe->max_local[i].valid = false; + memset(tpe->max_local[i].power, + IEEE80211_TPE_MAX_TX_PWR_NO_CONSTRAINT, + sizeof(tpe->max_local[i].power)); + + tpe->max_reg_client[i].valid = false; + memset(tpe->max_reg_client[i].power, + IEEE80211_TPE_MAX_TX_PWR_NO_CONSTRAINT, + sizeof(tpe->max_reg_client[i].power)); + + tpe->psd_local[i].valid = false; + memset(tpe->psd_local[i].power, + IEEE80211_TPE_PSD_NO_LIMIT, + sizeof(tpe->psd_local[i].power)); + + tpe->psd_reg_client[i].valid = false; + memset(tpe->psd_reg_client[i].power, + IEEE80211_TPE_PSD_NO_LIMIT, + sizeof(tpe->psd_reg_client[i].power)); + } +} diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index 7d44196ec5b630500d28815fde9f92c0890c8314..cacad912747af8319bffc8c0dd0091a7369f046c 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -2040,13 +2040,13 @@ static void mptcp_rcv_space_adjust(struct mptcp_sock *msk, int copied) do_div(grow, msk->rcvq_space.space); rcvwin += (grow << 1); - rcvbuf = min_t(u64, __tcp_space_from_win(scaling_ratio, rcvwin), + rcvbuf = min_t(u64, mptcp_space_from_win(sk, rcvwin), READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_rmem[2])); if (rcvbuf > sk->sk_rcvbuf) { u32 window_clamp; - window_clamp = __tcp_win_from_space(scaling_ratio, rcvbuf); + window_clamp = mptcp_win_from_space(sk, rcvbuf); WRITE_ONCE(sk->sk_rcvbuf, rcvbuf); /* Make subflows follow along. If we do not do this, we @@ -2202,7 +2202,7 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, if (skb_queue_empty(&msk->receive_queue) && __mptcp_move_skbs(msk)) continue; - /* only the master socket status is relevant here. The exit + /* only the MPTCP socket status is relevant here. The exit * conditions mirror closely tcp_recvmsg() */ if (copied >= target) @@ -2916,9 +2916,14 @@ void mptcp_set_state(struct sock *sk, int state) if (oldstate != TCP_ESTABLISHED) MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_CURRESTAB); break; - + case TCP_CLOSE_WAIT: + /* Unlike TCP, MPTCP sk would not have the TCP_SYN_RECV state: + * MPTCP "accepted" sockets will be created later on. So no + * transition from TCP_SYN_RECV to TCP_CLOSE_WAIT. + */ + break; default: - if (oldstate == TCP_ESTABLISHED) + if (oldstate == TCP_ESTABLISHED || oldstate == TCP_CLOSE_WAIT) MPTCP_DEC_STATS(sock_net(sk), MPTCP_MIB_CURRESTAB); } @@ -3521,7 +3526,7 @@ void mptcp_subflow_process_delegated(struct sock *ssk, long status) static int mptcp_hash(struct sock *sk) { /* should never be called, - * we hash the TCP subflows not the master socket + * we hash the TCP subflows not the MPTCP socket */ WARN_ON_ONCE(1); return 0; diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index 7aa47e2dd52b158f022da0aa2747713a811ac596..b11a4e50d52b6f1950660654eee022d387b6dc4b 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -386,6 +386,11 @@ static inline int mptcp_win_from_space(const struct sock *sk, int space) return __tcp_win_from_space(mptcp_sk(sk)->scaling_ratio, space); } +static inline int mptcp_space_from_win(const struct sock *sk, int win) +{ + return __tcp_space_from_win(mptcp_sk(sk)->scaling_ratio, win); +} + static inline int __mptcp_space(const struct sock *sk) { return mptcp_win_from_space(sk, READ_ONCE(sk->sk_rcvbuf) - __mptcp_rmem(sk)); diff --git a/net/mptcp/sockopt.c b/net/mptcp/sockopt.c index f9a4fb17b5b78d5945243a326e5222000c6fa2c6..2026a9a36f804da74bb1a6209a7d4c8911d78a5d 100644 --- a/net/mptcp/sockopt.c +++ b/net/mptcp/sockopt.c @@ -1579,7 +1579,7 @@ int mptcp_set_rcvlowat(struct sock *sk, int val) if (sk->sk_userlocks & SOCK_RCVBUF_LOCK) return 0; - space = __tcp_space_from_win(mptcp_sk(sk)->scaling_ratio, val); + space = mptcp_space_from_win(sk, val); if (space <= sk->sk_rcvbuf) return 0; diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index 612c38570a6426655dd7be8b7e140b8a72dac7ac..39e2cbdf38019cc99e31e54ac221dbc7ef825110 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -1719,7 +1719,7 @@ int mptcp_subflow_create_socket(struct sock *sk, unsigned short family, mptcp_sockopt_sync_locked(mptcp_sk(sk), sf->sk); release_sock(sf->sk); - /* the newly created socket really belongs to the owning MPTCP master + /* the newly created socket really belongs to the owning MPTCP * socket, even if for additional subflows the allocation is performed * by a kernel workqueue. Adjust inode references, so that the * procfs/diag interfaces really show this one belonging to the correct diff --git a/net/ncsi/internal.h b/net/ncsi/internal.h index 374412ed780b6ed601e63e2af23cb0a364df80fc..ef0f8f73826f53b8995b7b11ae13da5f3bf82af9 100644 --- a/net/ncsi/internal.h +++ b/net/ncsi/internal.h @@ -325,6 +325,7 @@ struct ncsi_dev_priv { spinlock_t lock; /* Protect the NCSI device */ unsigned int package_probe_id;/* Current ID during probe */ unsigned int package_num; /* Number of packages */ + unsigned int channel_probe_id;/* Current cahnnel ID during probe */ struct list_head packages; /* List of packages */ struct ncsi_channel *hot_channel; /* Channel was ever active */ struct ncsi_request requests[256]; /* Request table */ @@ -343,6 +344,7 @@ struct ncsi_dev_priv { bool multi_package; /* Enable multiple packages */ bool mlx_multi_host; /* Enable multi host Mellanox */ u32 package_whitelist; /* Packages to configure */ + unsigned char channel_count; /* Num of channels to probe */ }; struct ncsi_cmd_arg { diff --git a/net/ncsi/ncsi-manage.c b/net/ncsi/ncsi-manage.c index 745c788f1d1dfcd3a0467f08271d2ce44dacf2a8..5ecf611c882009d647909e52a686286e090ba6c3 100644 --- a/net/ncsi/ncsi-manage.c +++ b/net/ncsi/ncsi-manage.c @@ -510,17 +510,19 @@ static void ncsi_suspend_channel(struct ncsi_dev_priv *ndp) break; case ncsi_dev_state_suspend_gls: - ndp->pending_req_num = np->channel_num; + ndp->pending_req_num = 1; nca.type = NCSI_PKT_CMD_GLS; nca.package = np->id; + nca.channel = ndp->channel_probe_id; + ret = ncsi_xmit_cmd(&nca); + if (ret) + goto error; + ndp->channel_probe_id++; - nd->state = ncsi_dev_state_suspend_dcnt; - NCSI_FOR_EACH_CHANNEL(np, nc) { - nca.channel = nc->id; - ret = ncsi_xmit_cmd(&nca); - if (ret) - goto error; + if (ndp->channel_probe_id == ndp->channel_count) { + ndp->channel_probe_id = 0; + nd->state = ncsi_dev_state_suspend_dcnt; } break; @@ -1345,7 +1347,6 @@ static void ncsi_probe_channel(struct ncsi_dev_priv *ndp) { struct ncsi_dev *nd = &ndp->ndev; struct ncsi_package *np; - struct ncsi_channel *nc; struct ncsi_cmd_arg nca; unsigned char index; int ret; @@ -1423,23 +1424,6 @@ static void ncsi_probe_channel(struct ncsi_dev_priv *ndp) nd->state = ncsi_dev_state_probe_cis; break; - case ncsi_dev_state_probe_cis: - ndp->pending_req_num = NCSI_RESERVED_CHANNEL; - - /* Clear initial state */ - nca.type = NCSI_PKT_CMD_CIS; - nca.package = ndp->active_package->id; - for (index = 0; index < NCSI_RESERVED_CHANNEL; index++) { - nca.channel = index; - ret = ncsi_xmit_cmd(&nca); - if (ret) - goto error; - } - - nd->state = ncsi_dev_state_probe_gvi; - if (IS_ENABLED(CONFIG_NCSI_OEM_CMD_KEEP_PHY)) - nd->state = ncsi_dev_state_probe_keep_phy; - break; case ncsi_dev_state_probe_keep_phy: ndp->pending_req_num = 1; @@ -1452,14 +1436,17 @@ static void ncsi_probe_channel(struct ncsi_dev_priv *ndp) nd->state = ncsi_dev_state_probe_gvi; break; + case ncsi_dev_state_probe_cis: case ncsi_dev_state_probe_gvi: case ncsi_dev_state_probe_gc: case ncsi_dev_state_probe_gls: np = ndp->active_package; - ndp->pending_req_num = np->channel_num; + ndp->pending_req_num = 1; - /* Retrieve version, capability or link status */ - if (nd->state == ncsi_dev_state_probe_gvi) + /* Clear initial state Retrieve version, capability or link status */ + if (nd->state == ncsi_dev_state_probe_cis) + nca.type = NCSI_PKT_CMD_CIS; + else if (nd->state == ncsi_dev_state_probe_gvi) nca.type = NCSI_PKT_CMD_GVI; else if (nd->state == ncsi_dev_state_probe_gc) nca.type = NCSI_PKT_CMD_GC; @@ -1467,19 +1454,29 @@ static void ncsi_probe_channel(struct ncsi_dev_priv *ndp) nca.type = NCSI_PKT_CMD_GLS; nca.package = np->id; - NCSI_FOR_EACH_CHANNEL(np, nc) { - nca.channel = nc->id; - ret = ncsi_xmit_cmd(&nca); - if (ret) - goto error; - } + nca.channel = ndp->channel_probe_id; - if (nd->state == ncsi_dev_state_probe_gvi) + ret = ncsi_xmit_cmd(&nca); + if (ret) + goto error; + + if (nd->state == ncsi_dev_state_probe_cis) { + nd->state = ncsi_dev_state_probe_gvi; + if (IS_ENABLED(CONFIG_NCSI_OEM_CMD_KEEP_PHY) && ndp->channel_probe_id == 0) + nd->state = ncsi_dev_state_probe_keep_phy; + } else if (nd->state == ncsi_dev_state_probe_gvi) { nd->state = ncsi_dev_state_probe_gc; - else if (nd->state == ncsi_dev_state_probe_gc) + } else if (nd->state == ncsi_dev_state_probe_gc) { nd->state = ncsi_dev_state_probe_gls; - else + } else { + nd->state = ncsi_dev_state_probe_cis; + ndp->channel_probe_id++; + } + + if (ndp->channel_probe_id == ndp->channel_count) { + ndp->channel_probe_id = 0; nd->state = ncsi_dev_state_probe_dp; + } break; case ncsi_dev_state_probe_dp: ndp->pending_req_num = 1; @@ -1780,6 +1777,7 @@ struct ncsi_dev *ncsi_register_dev(struct net_device *dev, ndp->requests[i].ndp = ndp; timer_setup(&ndp->requests[i].timer, ncsi_request_timeout, 0); } + ndp->channel_count = NCSI_RESERVED_CHANNEL; spin_lock_irqsave(&ncsi_dev_lock, flags); list_add_tail_rcu(&ndp->node, &ncsi_dev_list); @@ -1813,6 +1811,7 @@ int ncsi_start_dev(struct ncsi_dev *nd) if (!(ndp->flags & NCSI_DEV_PROBED)) { ndp->package_probe_id = 0; + ndp->channel_probe_id = 0; nd->state = ncsi_dev_state_probe; schedule_work(&ndp->work); return 0; diff --git a/net/ncsi/ncsi-rsp.c b/net/ncsi/ncsi-rsp.c index bee290d0f48b6f25701312fd2de9b1b796761c7a..e28be33bdf2c487c0fbfe3a1b4de6f52c8f923cc 100644 --- a/net/ncsi/ncsi-rsp.c +++ b/net/ncsi/ncsi-rsp.c @@ -795,12 +795,13 @@ static int ncsi_rsp_handler_gc(struct ncsi_request *nr) struct ncsi_rsp_gc_pkt *rsp; struct ncsi_dev_priv *ndp = nr->ndp; struct ncsi_channel *nc; + struct ncsi_package *np; size_t size; /* Find the channel */ rsp = (struct ncsi_rsp_gc_pkt *)skb_network_header(nr->rsp); ncsi_find_package_and_channel(ndp, rsp->rsp.common.channel, - NULL, &nc); + &np, &nc); if (!nc) return -ENODEV; @@ -835,6 +836,7 @@ static int ncsi_rsp_handler_gc(struct ncsi_request *nr) */ nc->vlan_filter.bitmap = U64_MAX; nc->vlan_filter.n_vids = rsp->vlan_cnt; + np->ndp->channel_count = rsp->channel_cnt; return 0; } diff --git a/net/netfilter/ipset/ip_set_list_set.c b/net/netfilter/ipset/ip_set_list_set.c index 6c3f28bc59b3259f0033cd4adc0ba5711db08c26..54e2a1dd7f5f5163fef436dc6ed000ce25e837b7 100644 --- a/net/netfilter/ipset/ip_set_list_set.c +++ b/net/netfilter/ipset/ip_set_list_set.c @@ -549,6 +549,9 @@ list_set_cancel_gc(struct ip_set *set) if (SET_WITH_TIMEOUT(set)) timer_shutdown_sync(&map->gc); + + /* Flush list to drop references to other ipsets */ + list_set_flush(set); } static const struct ip_set_type_variant set_variant = { diff --git a/net/netfilter/ipvs/ip_vs_ctl.c b/net/netfilter/ipvs/ip_vs_ctl.c index b6d0dcf3a5c3458c0967f97e6ccffeca971aa066..78a1cc72dc38ab4f43c1d6c1273a206664af1104 100644 --- a/net/netfilter/ipvs/ip_vs_ctl.c +++ b/net/netfilter/ipvs/ip_vs_ctl.c @@ -1924,7 +1924,8 @@ proc_do_sync_ports(struct ctl_table *table, int write, return rc; } -static int ipvs_proc_est_cpumask_set(struct ctl_table *table, void *buffer) +static int ipvs_proc_est_cpumask_set(const struct ctl_table *table, + void *buffer) { struct netns_ipvs *ipvs = table->extra2; cpumask_var_t *valp = table->data; @@ -1962,8 +1963,8 @@ out: return ret; } -static int ipvs_proc_est_cpumask_get(struct ctl_table *table, void *buffer, - size_t size) +static int ipvs_proc_est_cpumask_get(const struct ctl_table *table, + void *buffer, size_t size) { struct netns_ipvs *ipvs = table->extra2; cpumask_var_t *valp = table->data; diff --git a/net/netfilter/nf_conntrack_bpf.c b/net/netfilter/nf_conntrack_bpf.c index d2492d050fe601fbb1ce8e3cb598603914712a83..4a136fc3a9c0f57b63bb751cae64833b427345ad 100644 --- a/net/netfilter/nf_conntrack_bpf.c +++ b/net/netfilter/nf_conntrack_bpf.c @@ -32,7 +32,9 @@ * -EINVAL - Passed NULL for bpf_tuple pointer * -EINVAL - opts->reserved is not 0 * -EINVAL - netns_id is less than -1 - * -EINVAL - opts__sz isn't NF_BPF_CT_OPTS_SZ (12) + * -EINVAL - opts__sz isn't NF_BPF_CT_OPTS_SZ (16) or 12 + * -EINVAL - opts->ct_zone_id set when + opts__sz isn't NF_BPF_CT_OPTS_SZ (16) * -EPROTO - l4proto isn't one of IPPROTO_TCP or IPPROTO_UDP * -ENONET - No network namespace found for netns_id * -ENOENT - Conntrack lookup could not find entry for tuple @@ -42,6 +44,8 @@ * Values: * IPPROTO_TCP, IPPROTO_UDP * @dir: - connection tracking tuple direction. + * @ct_zone_id - connection tracking zone id. + * @ct_zone_dir - connection tracking zone direction. * @reserved - Reserved member, will be reused for more options in future * Values: * 0 @@ -51,11 +55,13 @@ struct bpf_ct_opts { s32 error; u8 l4proto; u8 dir; - u8 reserved[2]; + u16 ct_zone_id; + u8 ct_zone_dir; + u8 reserved[3]; }; enum { - NF_BPF_CT_OPTS_SZ = 12, + NF_BPF_CT_OPTS_SZ = 16, }; static int bpf_nf_ct_tuple_parse(struct bpf_sock_tuple *bpf_tuple, @@ -104,12 +110,21 @@ __bpf_nf_ct_alloc_entry(struct net *net, struct bpf_sock_tuple *bpf_tuple, u32 timeout) { struct nf_conntrack_tuple otuple, rtuple; + struct nf_conntrack_zone ct_zone; struct nf_conn *ct; int err; - if (!opts || !bpf_tuple || opts->reserved[0] || opts->reserved[1] || - opts_len != NF_BPF_CT_OPTS_SZ) + if (!opts || !bpf_tuple) return ERR_PTR(-EINVAL); + if (!(opts_len == NF_BPF_CT_OPTS_SZ || opts_len == 12)) + return ERR_PTR(-EINVAL); + if (opts_len == NF_BPF_CT_OPTS_SZ) { + if (opts->reserved[0] || opts->reserved[1] || opts->reserved[2]) + return ERR_PTR(-EINVAL); + } else { + if (opts->ct_zone_id) + return ERR_PTR(-EINVAL); + } if (unlikely(opts->netns_id < BPF_F_CURRENT_NETNS)) return ERR_PTR(-EINVAL); @@ -130,7 +145,16 @@ __bpf_nf_ct_alloc_entry(struct net *net, struct bpf_sock_tuple *bpf_tuple, return ERR_PTR(-ENONET); } - ct = nf_conntrack_alloc(net, &nf_ct_zone_dflt, &otuple, &rtuple, + if (opts_len == NF_BPF_CT_OPTS_SZ) { + if (opts->ct_zone_dir == 0) + opts->ct_zone_dir = NF_CT_DEFAULT_ZONE_DIR; + nf_ct_zone_init(&ct_zone, + opts->ct_zone_id, opts->ct_zone_dir, 0); + } else { + ct_zone = nf_ct_zone_dflt; + } + + ct = nf_conntrack_alloc(net, &ct_zone, &otuple, &rtuple, GFP_ATOMIC); if (IS_ERR(ct)) goto out; @@ -152,12 +176,21 @@ static struct nf_conn *__bpf_nf_ct_lookup(struct net *net, { struct nf_conntrack_tuple_hash *hash; struct nf_conntrack_tuple tuple; + struct nf_conntrack_zone ct_zone; struct nf_conn *ct; int err; - if (!opts || !bpf_tuple || opts->reserved[0] || opts->reserved[1] || - opts_len != NF_BPF_CT_OPTS_SZ) + if (!opts || !bpf_tuple) return ERR_PTR(-EINVAL); + if (!(opts_len == NF_BPF_CT_OPTS_SZ || opts_len == 12)) + return ERR_PTR(-EINVAL); + if (opts_len == NF_BPF_CT_OPTS_SZ) { + if (opts->reserved[0] || opts->reserved[1] || opts->reserved[2]) + return ERR_PTR(-EINVAL); + } else { + if (opts->ct_zone_id) + return ERR_PTR(-EINVAL); + } if (unlikely(opts->l4proto != IPPROTO_TCP && opts->l4proto != IPPROTO_UDP)) return ERR_PTR(-EPROTO); if (unlikely(opts->netns_id < BPF_F_CURRENT_NETNS)) @@ -174,7 +207,16 @@ static struct nf_conn *__bpf_nf_ct_lookup(struct net *net, return ERR_PTR(-ENONET); } - hash = nf_conntrack_find_get(net, &nf_ct_zone_dflt, &tuple); + if (opts_len == NF_BPF_CT_OPTS_SZ) { + if (opts->ct_zone_dir == 0) + opts->ct_zone_dir = NF_CT_DEFAULT_ZONE_DIR; + nf_ct_zone_init(&ct_zone, + opts->ct_zone_id, opts->ct_zone_dir, 0); + } else { + ct_zone = nf_ct_zone_dflt; + } + + hash = nf_conntrack_find_get(net, &ct_zone, &tuple); if (opts->netns_id >= 0) put_net(net); if (!hash) @@ -245,7 +287,7 @@ __bpf_kfunc_start_defs(); * @opts - Additional options for allocation (documented above) * Cannot be NULL * @opts__sz - Length of the bpf_ct_opts structure - * Must be NF_BPF_CT_OPTS_SZ (12) + * Must be NF_BPF_CT_OPTS_SZ (16) or 12 */ __bpf_kfunc struct nf_conn___init * bpf_xdp_ct_alloc(struct xdp_md *xdp_ctx, struct bpf_sock_tuple *bpf_tuple, @@ -279,7 +321,7 @@ bpf_xdp_ct_alloc(struct xdp_md *xdp_ctx, struct bpf_sock_tuple *bpf_tuple, * @opts - Additional options for lookup (documented above) * Cannot be NULL * @opts__sz - Length of the bpf_ct_opts structure - * Must be NF_BPF_CT_OPTS_SZ (12) + * Must be NF_BPF_CT_OPTS_SZ (16) or 12 */ __bpf_kfunc struct nf_conn * bpf_xdp_ct_lookup(struct xdp_md *xdp_ctx, struct bpf_sock_tuple *bpf_tuple, @@ -312,7 +354,7 @@ bpf_xdp_ct_lookup(struct xdp_md *xdp_ctx, struct bpf_sock_tuple *bpf_tuple, * @opts - Additional options for allocation (documented above) * Cannot be NULL * @opts__sz - Length of the bpf_ct_opts structure - * Must be NF_BPF_CT_OPTS_SZ (12) + * Must be NF_BPF_CT_OPTS_SZ (16) or 12 */ __bpf_kfunc struct nf_conn___init * bpf_skb_ct_alloc(struct __sk_buff *skb_ctx, struct bpf_sock_tuple *bpf_tuple, @@ -347,7 +389,7 @@ bpf_skb_ct_alloc(struct __sk_buff *skb_ctx, struct bpf_sock_tuple *bpf_tuple, * @opts - Additional options for lookup (documented above) * Cannot be NULL * @opts__sz - Length of the bpf_ct_opts structure - * Must be NF_BPF_CT_OPTS_SZ (12) + * Must be NF_BPF_CT_OPTS_SZ (16) or 12 */ __bpf_kfunc struct nf_conn * bpf_skb_ct_lookup(struct __sk_buff *skb_ctx, struct bpf_sock_tuple *bpf_tuple, diff --git a/net/netfilter/nfnetlink_queue.c b/net/netfilter/nfnetlink_queue.c index 00f4bd21c59b419e96794127693c21ccb05e45b0..f1c31757e4969e8f975c7a1ebbc3b96148ec9724 100644 --- a/net/netfilter/nfnetlink_queue.c +++ b/net/netfilter/nfnetlink_queue.c @@ -169,7 +169,9 @@ instance_destroy_rcu(struct rcu_head *head) struct nfqnl_instance *inst = container_of(head, struct nfqnl_instance, rcu); + rcu_read_lock(); nfqnl_flush(inst, NULL, 0); + rcu_read_unlock(); kfree(inst); module_put(THIS_MODULE); } diff --git a/net/netfilter/nft_fib.c b/net/netfilter/nft_fib.c index 37cfe6dd712d8b138fc290abe66fd8d9b69963d6..b58f62195ff3ee04aebcf62a166349ff3a1f1285 100644 --- a/net/netfilter/nft_fib.c +++ b/net/netfilter/nft_fib.c @@ -35,11 +35,9 @@ int nft_fib_validate(const struct nft_ctx *ctx, const struct nft_expr *expr, switch (priv->result) { case NFT_FIB_RESULT_OIF: case NFT_FIB_RESULT_OIFNAME: - hooks = (1 << NF_INET_PRE_ROUTING); - if (priv->flags & NFTA_FIB_F_IIF) { - hooks |= (1 << NF_INET_LOCAL_IN) | - (1 << NF_INET_FORWARD); - } + hooks = (1 << NF_INET_PRE_ROUTING) | + (1 << NF_INET_LOCAL_IN) | + (1 << NF_INET_FORWARD); break; case NFT_FIB_RESULT_ADDRTYPE: if (priv->flags & NFTA_FIB_F_IIF) diff --git a/net/netfilter/nft_payload.c b/net/netfilter/nft_payload.c index 0a689c8e0295dfed3511997abcac53fad4b56fd5..0c43d748e23ae1537eb9043e2209054584e4a352 100644 --- a/net/netfilter/nft_payload.c +++ b/net/netfilter/nft_payload.c @@ -45,36 +45,27 @@ nft_payload_copy_vlan(u32 *d, const struct sk_buff *skb, u8 offset, u8 len) int mac_off = skb_mac_header(skb) - skb->data; u8 *vlanh, *dst_u8 = (u8 *) d; struct vlan_ethhdr veth; - u8 vlan_hlen = 0; - - if ((skb->protocol == htons(ETH_P_8021AD) || - skb->protocol == htons(ETH_P_8021Q)) && - offset >= VLAN_ETH_HLEN && offset < VLAN_ETH_HLEN + VLAN_HLEN) - vlan_hlen += VLAN_HLEN; vlanh = (u8 *) &veth; - if (offset < VLAN_ETH_HLEN + vlan_hlen) { + if (offset < VLAN_ETH_HLEN) { u8 ethlen = len; - if (vlan_hlen && - skb_copy_bits(skb, mac_off, &veth, VLAN_ETH_HLEN) < 0) - return false; - else if (!nft_payload_rebuild_vlan_hdr(skb, mac_off, &veth)) + if (!nft_payload_rebuild_vlan_hdr(skb, mac_off, &veth)) return false; - if (offset + len > VLAN_ETH_HLEN + vlan_hlen) - ethlen -= offset + len - VLAN_ETH_HLEN - vlan_hlen; + if (offset + len > VLAN_ETH_HLEN) + ethlen -= offset + len - VLAN_ETH_HLEN; - memcpy(dst_u8, vlanh + offset - vlan_hlen, ethlen); + memcpy(dst_u8, vlanh + offset, ethlen); len -= ethlen; if (len == 0) return true; dst_u8 += ethlen; - offset = ETH_HLEN + vlan_hlen; + offset = ETH_HLEN; } else { - offset -= VLAN_HLEN + vlan_hlen; + offset -= VLAN_HLEN; } return skb_copy_bits(skb, offset + mac_off, dst_u8, len) == 0; @@ -154,12 +145,12 @@ int nft_payload_inner_offset(const struct nft_pktinfo *pkt) return pkt->inneroff; } -static bool nft_payload_need_vlan_copy(const struct nft_payload *priv) +static bool nft_payload_need_vlan_adjust(u32 offset, u32 len) { - unsigned int len = priv->offset + priv->len; + unsigned int boundary = offset + len; /* data past ether src/dst requested, copy needed */ - if (len > offsetof(struct ethhdr, h_proto)) + if (boundary > offsetof(struct ethhdr, h_proto)) return true; return false; @@ -183,7 +174,7 @@ void nft_payload_eval(const struct nft_expr *expr, goto err; if (skb_vlan_tag_present(skb) && - nft_payload_need_vlan_copy(priv)) { + nft_payload_need_vlan_adjust(priv->offset, priv->len)) { if (!nft_payload_copy_vlan(dest, skb, priv->offset, priv->len)) goto err; @@ -810,21 +801,79 @@ struct nft_payload_set { u8 csum_flags; }; +/* This is not struct vlan_hdr. */ +struct nft_payload_vlan_hdr { + __be16 h_vlan_proto; + __be16 h_vlan_TCI; +}; + +static bool +nft_payload_set_vlan(const u32 *src, struct sk_buff *skb, u8 offset, u8 len, + int *vlan_hlen) +{ + struct nft_payload_vlan_hdr *vlanh; + __be16 vlan_proto; + u16 vlan_tci; + + if (offset >= offsetof(struct vlan_ethhdr, h_vlan_encapsulated_proto)) { + *vlan_hlen = VLAN_HLEN; + return true; + } + + switch (offset) { + case offsetof(struct vlan_ethhdr, h_vlan_proto): + if (len == 2) { + vlan_proto = nft_reg_load_be16(src); + skb->vlan_proto = vlan_proto; + } else if (len == 4) { + vlanh = (struct nft_payload_vlan_hdr *)src; + __vlan_hwaccel_put_tag(skb, vlanh->h_vlan_proto, + ntohs(vlanh->h_vlan_TCI)); + } else { + return false; + } + break; + case offsetof(struct vlan_ethhdr, h_vlan_TCI): + if (len != 2) + return false; + + vlan_tci = ntohs(nft_reg_load_be16(src)); + skb->vlan_tci = vlan_tci; + break; + default: + return false; + } + + return true; +} + static void nft_payload_set_eval(const struct nft_expr *expr, struct nft_regs *regs, const struct nft_pktinfo *pkt) { const struct nft_payload_set *priv = nft_expr_priv(expr); - struct sk_buff *skb = pkt->skb; const u32 *src = ®s->data[priv->sreg]; - int offset, csum_offset; + int offset, csum_offset, vlan_hlen = 0; + struct sk_buff *skb = pkt->skb; __wsum fsum, tsum; switch (priv->base) { case NFT_PAYLOAD_LL_HEADER: if (!skb_mac_header_was_set(skb)) goto err; - offset = skb_mac_header(skb) - skb->data; + + if (skb_vlan_tag_present(skb) && + nft_payload_need_vlan_adjust(priv->offset, priv->len)) { + if (!nft_payload_set_vlan(src, skb, + priv->offset, priv->len, + &vlan_hlen)) + goto err; + + if (!vlan_hlen) + return; + } + + offset = skb_mac_header(skb) - skb->data - vlan_hlen; break; case NFT_PAYLOAD_NETWORK_HEADER: offset = skb_network_offset(skb); diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c index fa9c090cf629e6e92c097285b262ed90324c7656..0b7a89db3ab74e13837ec697d0343cd9839f38b1 100644 --- a/net/netlink/af_netlink.c +++ b/net/netlink/af_netlink.c @@ -636,8 +636,7 @@ static struct proto netlink_proto = { }; static int __netlink_create(struct net *net, struct socket *sock, - struct mutex *dump_cb_mutex, int protocol, - int kern) + int protocol, int kern) { struct sock *sk; struct netlink_sock *nlk; @@ -655,7 +654,6 @@ static int __netlink_create(struct net *net, struct socket *sock, lockdep_set_class_and_name(&nlk->nl_cb_mutex, nlk_cb_mutex_keys + protocol, nlk_cb_mutex_key_strings[protocol]); - nlk->dump_cb_mutex = dump_cb_mutex; init_waitqueue_head(&nlk->wait); sk->sk_destruct = netlink_sock_destruct; @@ -667,7 +665,6 @@ static int netlink_create(struct net *net, struct socket *sock, int protocol, int kern) { struct module *module = NULL; - struct mutex *cb_mutex; struct netlink_sock *nlk; int (*bind)(struct net *net, int group); void (*unbind)(struct net *net, int group); @@ -696,7 +693,6 @@ static int netlink_create(struct net *net, struct socket *sock, int protocol, module = nl_table[protocol].module; else err = -EPROTONOSUPPORT; - cb_mutex = nl_table[protocol].cb_mutex; bind = nl_table[protocol].bind; unbind = nl_table[protocol].unbind; release = nl_table[protocol].release; @@ -705,7 +701,7 @@ static int netlink_create(struct net *net, struct socket *sock, int protocol, if (err < 0) goto out; - err = __netlink_create(net, sock, cb_mutex, protocol, kern); + err = __netlink_create(net, sock, protocol, kern); if (err < 0) goto out_module; @@ -2016,7 +2012,6 @@ __netlink_kernel_create(struct net *net, int unit, struct module *module, struct sock *sk; struct netlink_sock *nlk; struct listeners *listeners = NULL; - struct mutex *cb_mutex = cfg ? cfg->cb_mutex : NULL; unsigned int groups; BUG_ON(!nl_table); @@ -2027,7 +2022,7 @@ __netlink_kernel_create(struct net *net, int unit, struct module *module, if (sock_create_lite(PF_NETLINK, SOCK_DGRAM, unit, &sock)) return NULL; - if (__netlink_create(net, sock, cb_mutex, unit, 1) < 0) + if (__netlink_create(net, sock, unit, 1) < 0) goto out_sock_release_nosk; sk = sock->sk; @@ -2055,7 +2050,6 @@ __netlink_kernel_create(struct net *net, int unit, struct module *module, if (!nl_table[unit].registered) { nl_table[unit].groups = groups; rcu_assign_pointer(nl_table[unit].listeners, listeners); - nl_table[unit].cb_mutex = cb_mutex; nl_table[unit].module = module; if (cfg) { nl_table[unit].bind = cfg->bind; @@ -2326,17 +2320,9 @@ static int netlink_dump(struct sock *sk, bool lock_taken) netlink_skb_set_owner_r(skb, sk); if (nlk->dump_done_errno > 0) { - struct mutex *extra_mutex = nlk->dump_cb_mutex; - cb->extack = &extack; - if (cb->flags & RTNL_FLAG_DUMP_UNLOCKED) - extra_mutex = NULL; - if (extra_mutex) - mutex_lock(extra_mutex); nlk->dump_done_errno = cb->dump(skb, cb); - if (extra_mutex) - mutex_unlock(extra_mutex); /* EMSGSIZE plus something already in the skb means * that there's more to dump but current skb has filled up. diff --git a/net/openvswitch/vport-internal_dev.c b/net/openvswitch/vport-internal_dev.c index 74c88a6baa43a4995d92bff60588e09dd38f4549..4b33133cbdfffa9eb1cf8044fb11743898bb53a9 100644 --- a/net/openvswitch/vport-internal_dev.c +++ b/net/openvswitch/vport-internal_dev.c @@ -85,7 +85,6 @@ static const struct net_device_ops internal_dev_netdev_ops = { .ndo_stop = internal_dev_stop, .ndo_start_xmit = internal_dev_xmit, .ndo_set_mac_address = eth_mac_addr, - .ndo_get_stats64 = dev_get_tstats64, }; static struct rtnl_link_ops internal_dev_link_ops __read_mostly = { @@ -140,11 +139,7 @@ static struct vport *internal_dev_create(const struct vport_parms *parms) err = -ENOMEM; goto error_free_vport; } - vport->dev->tstats = netdev_alloc_pcpu_stats(struct pcpu_sw_netstats); - if (!vport->dev->tstats) { - err = -ENOMEM; - goto error_free_netdev; - } + dev->pcpu_stat_type = NETDEV_PCPU_STAT_TSTATS; dev_net_set(vport->dev, ovs_dp_get_net(vport->dp)); dev->ifindex = parms->desired_ifindex; @@ -169,8 +164,6 @@ static struct vport *internal_dev_create(const struct vport_parms *parms) error_unlock: rtnl_unlock(); - free_percpu(dev->tstats); -error_free_netdev: free_netdev(dev); error_free_vport: ovs_vport_free(vport); @@ -186,7 +179,6 @@ static void internal_dev_destroy(struct vport *vport) /* unregister_netdevice() waits for an RCU grace period. */ unregister_netdevice(vport->dev); - free_percpu(vport->dev->tstats); rtnl_unlock(); } diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c index ea3ebc160e25cc661901717a755f47db927c304d..fce3908875912f67551cf090a524f96427c09ab3 100644 --- a/net/packet/af_packet.c +++ b/net/packet/af_packet.c @@ -2056,8 +2056,7 @@ retry: skb->dev = dev; skb->priority = READ_ONCE(sk->sk_priority); skb->mark = READ_ONCE(sk->sk_mark); - skb->tstamp = sockc.transmit_time; - + skb_set_delivery_type_by_clockid(skb, sockc.transmit_time, sk->sk_clockid); skb_setup_tx_timestamp(skb, sockc.tsflags); if (unlikely(extra_len == 4)) @@ -2584,7 +2583,7 @@ static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb, skb->dev = dev; skb->priority = READ_ONCE(po->sk.sk_priority); skb->mark = READ_ONCE(po->sk.sk_mark); - skb->tstamp = sockc->transmit_time; + skb_set_delivery_type_by_clockid(skb, sockc->transmit_time, po->sk.sk_clockid); skb_setup_tx_timestamp(skb, sockc->tsflags); skb_zcopy_set_nouarg(skb, ph.raw); @@ -3062,7 +3061,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) skb->dev = dev; skb->priority = READ_ONCE(sk->sk_priority); skb->mark = sockc.mark; - skb->tstamp = sockc.transmit_time; + skb_set_delivery_type_by_clockid(skb, sockc.transmit_time, sk->sk_clockid); if (unlikely(extra_len == 4)) skb->no_fcs = 1; diff --git a/net/sched/act_api.c b/net/sched/act_api.c index 9ee622fb1160fe5de8df9a5fca0c9a412e40e31a..7458b3154426279123a8b178fe40f05ce657b5c2 100644 --- a/net/sched/act_api.c +++ b/net/sched/act_api.c @@ -62,7 +62,7 @@ static void tcf_set_action_cookie(struct tc_cookie __rcu **old_cookie, { struct tc_cookie *old; - old = xchg((__force struct tc_cookie **)old_cookie, new_cookie); + old = unrcu_pointer(xchg(old_cookie, RCU_INITIALIZER(new_cookie))); if (old) call_rcu(&old->rcu, tcf_free_cookie_rcu); } diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c index 0e3cf11ae5fc042f1e6f960fb5159153cc67fb27..396b576390d00aad56bca6a18b7796e5324c0aef 100644 --- a/net/sched/act_bpf.c +++ b/net/sched/act_bpf.c @@ -54,8 +54,8 @@ TC_INDIRECT_SCOPE int tcf_bpf_act(struct sk_buff *skb, bpf_compute_data_pointers(skb); filter_res = bpf_prog_run(filter, skb); } - if (unlikely(!skb->tstamp && skb->mono_delivery_time)) - skb->mono_delivery_time = 0; + if (unlikely(!skb->tstamp && skb->tstamp_type)) + skb->tstamp_type = SKB_CLOCK_REALTIME; if (skb_sk_is_prefetched(skb) && filter_res != TC_ACT_OK) skb_orphan(skb); diff --git a/net/sched/cls_bpf.c b/net/sched/cls_bpf.c index 5e83e890f6a4b74a449a2f3eb06864efc24b38ba..1941ebec23ff9cfafe979dffca3e01f3b376fd07 100644 --- a/net/sched/cls_bpf.c +++ b/net/sched/cls_bpf.c @@ -104,8 +104,8 @@ TC_INDIRECT_SCOPE int cls_bpf_classify(struct sk_buff *skb, bpf_compute_data_pointers(skb); filter_res = bpf_prog_run(prog->filter, skb); } - if (unlikely(!skb->tstamp && skb->mono_delivery_time)) - skb->mono_delivery_time = 0; + if (unlikely(!skb->tstamp && skb->tstamp_type)) + skb->tstamp_type = SKB_CLOCK_REALTIME; if (prog->exts_integrated) { res->class = 0; diff --git a/net/sched/cls_flower.c b/net/sched/cls_flower.c index fd9a6f20b60b463286dec0ff171d3519617d4a7c..eef570c577ac7e35505492383e49554caf728f7b 100644 --- a/net/sched/cls_flower.c +++ b/net/sched/cls_flower.c @@ -41,6 +41,12 @@ #define TCA_FLOWER_KEY_CT_FLAGS_MASK \ (TCA_FLOWER_KEY_CT_FLAGS_MAX - 1) +#define TUNNEL_FLAGS_PRESENT (\ + _BITUL(IP_TUNNEL_CSUM_BIT) | \ + _BITUL(IP_TUNNEL_DONT_FRAGMENT_BIT) | \ + _BITUL(IP_TUNNEL_OAM_BIT) | \ + _BITUL(IP_TUNNEL_CRIT_OPT_BIT)) + struct fl_flow_key { struct flow_dissector_key_meta meta; struct flow_dissector_key_control control; @@ -75,6 +81,7 @@ struct fl_flow_key { struct flow_dissector_key_l2tpv3 l2tpv3; struct flow_dissector_key_ipsec ipsec; struct flow_dissector_key_cfm cfm; + struct flow_dissector_key_enc_flags enc_flags; } __aligned(BITS_PER_LONG / 8); /* Ensure that we can do comparisons as longs. */ struct fl_flow_mask_range { @@ -732,6 +739,10 @@ static const struct nla_policy fl_policy[TCA_FLOWER_MAX + 1] = { [TCA_FLOWER_KEY_SPI_MASK] = { .type = NLA_U32 }, [TCA_FLOWER_L2_MISS] = NLA_POLICY_MAX(NLA_U8, 1), [TCA_FLOWER_KEY_CFM] = { .type = NLA_NESTED }, + [TCA_FLOWER_KEY_ENC_FLAGS] = NLA_POLICY_MASK(NLA_U32, + TUNNEL_FLAGS_PRESENT), + [TCA_FLOWER_KEY_ENC_FLAGS_MASK] = NLA_POLICY_MASK(NLA_U32, + TUNNEL_FLAGS_PRESENT), }; static const struct nla_policy @@ -1825,6 +1836,21 @@ static int fl_set_key_cfm(struct nlattr **tb, return 0; } +static int fl_set_key_enc_flags(struct nlattr **tb, u32 *flags_key, + u32 *flags_mask, struct netlink_ext_ack *extack) +{ + /* mask is mandatory for flags */ + if (NL_REQ_ATTR_CHECK(extack, NULL, tb, TCA_FLOWER_KEY_ENC_FLAGS_MASK)) { + NL_SET_ERR_MSG(extack, "missing enc_flags mask"); + return -EINVAL; + } + + *flags_key = nla_get_u32(tb[TCA_FLOWER_KEY_ENC_FLAGS]); + *flags_mask = nla_get_u32(tb[TCA_FLOWER_KEY_ENC_FLAGS_MASK]); + + return 0; +} + static int fl_set_key(struct net *net, struct nlattr **tb, struct fl_flow_key *key, struct fl_flow_key *mask, struct netlink_ext_ack *extack) @@ -2059,9 +2085,16 @@ static int fl_set_key(struct net *net, struct nlattr **tb, if (ret) return ret; - if (tb[TCA_FLOWER_KEY_FLAGS]) + if (tb[TCA_FLOWER_KEY_FLAGS]) { ret = fl_set_key_flags(tb, &key->control.flags, &mask->control.flags, extack); + if (ret) + return ret; + } + + if (tb[TCA_FLOWER_KEY_ENC_FLAGS]) + ret = fl_set_key_enc_flags(tb, &key->enc_flags.flags, + &mask->enc_flags.flags, extack); return ret; } @@ -2175,6 +2208,8 @@ static void fl_init_dissector(struct flow_dissector *dissector, FLOW_DISSECTOR_KEY_IPSEC, ipsec); FL_KEY_SET_IF_MASKED(mask, keys, cnt, FLOW_DISSECTOR_KEY_CFM, cfm); + FL_KEY_SET_IF_MASKED(mask, keys, cnt, + FLOW_DISSECTOR_KEY_ENC_FLAGS, enc_flags); skb_flow_dissector_init(dissector, keys, cnt); } @@ -3291,6 +3326,22 @@ err_cfm_opts: return err; } +static int fl_dump_key_enc_flags(struct sk_buff *skb, + struct flow_dissector_key_enc_flags *key, + struct flow_dissector_key_enc_flags *mask) +{ + if (!memchr_inv(mask, 0, sizeof(*mask))) + return 0; + + if (nla_put_u32(skb, TCA_FLOWER_KEY_ENC_FLAGS, key->flags)) + return -EMSGSIZE; + + if (nla_put_u32(skb, TCA_FLOWER_KEY_ENC_FLAGS_MASK, mask->flags)) + return -EMSGSIZE; + + return 0; +} + static int fl_dump_key_options(struct sk_buff *skb, int enc_opt_type, struct flow_dissector_key_enc_opts *enc_opts) { @@ -3592,6 +3643,9 @@ static int fl_dump_key(struct sk_buff *skb, struct net *net, if (fl_dump_key_cfm(skb, &key->cfm, &mask->cfm)) goto nla_put_failure; + if (fl_dump_key_enc_flags(skb, &key->enc_flags, &mask->enc_flags)) + goto nla_put_failure; + return 0; nla_put_failure: diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c index 2a637a17061b91fe8e534823a8b031699bb64f4a..1417f1991452f4cd63eea9a74906e0eac3a7c19c 100644 --- a/net/sched/sch_generic.c +++ b/net/sched/sch_generic.c @@ -633,6 +633,7 @@ EXPORT_SYMBOL_GPL(netif_carrier_event); static int noop_enqueue(struct sk_buff *skb, struct Qdisc *qdisc, struct sk_buff **to_free) { + dev_core_stats_tx_dropped_inc(skb->dev); __qdisc_drop(skb, to_free); return NET_XMIT_CN; } diff --git a/net/sched/sch_multiq.c b/net/sched/sch_multiq.c index 79e93a19d5fabeb5ee91e20388d67ef019064064..06e03f5cd7ce182590c142e4c83f8d93f800b89e 100644 --- a/net/sched/sch_multiq.c +++ b/net/sched/sch_multiq.c @@ -185,7 +185,7 @@ static int multiq_tune(struct Qdisc *sch, struct nlattr *opt, qopt->bands = qdisc_dev(sch)->real_num_tx_queues; - removed = kmalloc(sizeof(*removed) * (q->max_bands - q->bands), + removed = kmalloc(sizeof(*removed) * (q->max_bands - qopt->bands), GFP_KERNEL); if (!removed) return -ENOMEM; diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c index 1ab17e8a72605385280fad9b7f656a6771236acc..b284a06b5a75fa3408c4a6515b71ed1a6ec63e78 100644 --- a/net/sched/sch_taprio.c +++ b/net/sched/sch_taprio.c @@ -1151,11 +1151,6 @@ static int parse_taprio_schedule(struct taprio_sched *q, struct nlattr **tb, list_for_each_entry(entry, &new->entries, list) cycle = ktime_add_ns(cycle, entry->interval); - if (!cycle) { - NL_SET_ERR_MSG(extack, "'cycle_time' can never be 0"); - return -EINVAL; - } - if (cycle < 0 || cycle > INT_MAX) { NL_SET_ERR_MSG(extack, "'cycle_time' is too big"); return -EINVAL; @@ -1164,6 +1159,11 @@ static int parse_taprio_schedule(struct taprio_sched *q, struct nlattr **tb, new->cycle_time = cycle; } + if (new->cycle_time < new->num_entries * length_to_duration(q, ETH_ZLEN)) { + NL_SET_ERR_MSG(extack, "'cycle_time' is too small"); + return -EINVAL; + } + taprio_calculate_gate_durations(q, new); return 0; @@ -1176,16 +1176,13 @@ static int taprio_parse_mqprio_opt(struct net_device *dev, { bool allow_overlapping_txqs = TXTIME_ASSIST_IS_ENABLED(taprio_flags); - if (!qopt && !dev->num_tc) { - NL_SET_ERR_MSG(extack, "'mqprio' configuration is necessary"); - return -EINVAL; - } - - /* If num_tc is already set, it means that the user already - * configured the mqprio part - */ - if (dev->num_tc) + if (!qopt) { + if (!dev->num_tc) { + NL_SET_ERR_MSG(extack, "'mqprio' configuration is necessary"); + return -EINVAL; + } return 0; + } /* taprio imposes that traffic classes map 1:n to tx queues */ if (qopt->num_tc > dev->num_tx_queues) { @@ -1848,6 +1845,9 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt, } q->flags = taprio_flags; + /* Needed for length_to_duration() during netlink attribute parsing */ + taprio_set_picos_per_byte(dev, q); + err = taprio_parse_mqprio_opt(dev, mqprio, extack, q->flags); if (err < 0) return err; @@ -1907,7 +1907,6 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt, if (err < 0) goto free_sched; - taprio_set_picos_per_byte(dev, q); taprio_update_queue_max_sdu(q, new_admin, stab); if (FULL_OFFLOAD_IS_ENABLED(q->flags)) diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index e50a286fd0fb77dfe2644d794ffb11d9db332906..c5f98c6b25613f542066f8fab1bb0d630d46a77d 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -459,29 +459,11 @@ out: static void smc_adjust_sock_bufsizes(struct sock *nsk, struct sock *osk, unsigned long mask) { - struct net *nnet = sock_net(nsk); - nsk->sk_userlocks = osk->sk_userlocks; - if (osk->sk_userlocks & SOCK_SNDBUF_LOCK) { + if (osk->sk_userlocks & SOCK_SNDBUF_LOCK) nsk->sk_sndbuf = osk->sk_sndbuf; - } else { - if (mask == SK_FLAGS_SMC_TO_CLC) - WRITE_ONCE(nsk->sk_sndbuf, - READ_ONCE(nnet->ipv4.sysctl_tcp_wmem[1])); - else - WRITE_ONCE(nsk->sk_sndbuf, - 2 * READ_ONCE(nnet->smc.sysctl_wmem)); - } - if (osk->sk_userlocks & SOCK_RCVBUF_LOCK) { + if (osk->sk_userlocks & SOCK_RCVBUF_LOCK) nsk->sk_rcvbuf = osk->sk_rcvbuf; - } else { - if (mask == SK_FLAGS_SMC_TO_CLC) - WRITE_ONCE(nsk->sk_rcvbuf, - READ_ONCE(nnet->ipv4.sysctl_tcp_rmem[1])); - else - WRITE_ONCE(nsk->sk_rcvbuf, - 2 * READ_ONCE(nnet->smc.sysctl_rmem)); - } } static void smc_copy_sock_settings(struct sock *nsk, struct sock *osk, diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c index fafdb97adfad9f897c989ac6ddae0197f3318b57..3b95828d997605a2432fe10a3aaa333598fd64ac 100644 --- a/net/smc/smc_core.c +++ b/net/smc/smc_core.c @@ -2006,7 +2006,7 @@ out: } #define SMCD_DMBE_SIZES 6 /* 0 -> 16KB, 1 -> 32KB, .. 6 -> 1MB */ -#define SMCR_RMBE_SIZES 5 /* 0 -> 16KB, 1 -> 32KB, .. 5 -> 512KB */ +#define SMCR_RMBE_SIZES 15 /* 0 -> 16KB, 1 -> 32KB, .. 15 -> 512MB */ /* convert the RMB size into the compressed notation (minimum 16K, see * SMCD/R_DMBE_SIZES. @@ -2015,7 +2015,6 @@ out: */ static u8 smc_compress_bufsize(int size, bool is_smcd, bool is_rmb) { - const unsigned int max_scat = SG_MAX_SINGLE_ALLOC * PAGE_SIZE; u8 compressed; if (size <= SMC_BUF_MIN_SIZE) @@ -2025,9 +2024,11 @@ static u8 smc_compress_bufsize(int size, bool is_smcd, bool is_rmb) compressed = min_t(u8, ilog2(size) + 1, is_smcd ? SMCD_DMBE_SIZES : SMCR_RMBE_SIZES); +#ifdef CONFIG_ARCH_NO_SG_CHAIN if (!is_smcd && is_rmb) /* RMBs are backed by & limited to max size of scatterlists */ - compressed = min_t(u8, compressed, ilog2(max_scat >> 14)); + compressed = min_t(u8, compressed, ilog2((SG_MAX_SINGLE_ALLOC * PAGE_SIZE) >> 14)); +#endif return compressed; } diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index ab6e694f7bc2582ddc8fbbf53d4d534b13bb5fb3..dc063c2c7950edc28961d877712827bc56bb58e6 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -231,14 +231,10 @@ static void tls_device_resync_tx(struct sock *sk, struct tls_context *tls_ctx, u32 seq) { struct net_device *netdev; - struct sk_buff *skb; int err = 0; u8 *rcd_sn; - skb = tcp_write_queue_tail(sk); - if (skb) - TCP_SKB_CB(skb)->eor = 1; - + tcp_write_collapse_fence(sk); rcd_sn = tls_ctx->tx.rec_seq; trace_tls_device_tx_resync_send(sk, seq, rcd_sn); @@ -1067,7 +1063,6 @@ int tls_set_device_offload(struct sock *sk) struct tls_prot_info *prot; struct net_device *netdev; struct tls_context *ctx; - struct sk_buff *skb; char *iv, *rec_seq; int rc; @@ -1138,9 +1133,7 @@ int tls_set_device_offload(struct sock *sk) * SKBs where only part of the payload needs to be encrypted. * So mark the last skb in the write queue as end of record. */ - skb = tcp_write_queue_tail(sk); - if (skb) - TCP_SKB_CB(skb)->eor = 1; + tcp_write_collapse_fence(sk); /* Avoid offloading if the device is down * We don't want to offload new flows after diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index e4af6616e1dff1da32772eddfdf645fb4c9fa45c..3821f8945b1e201f053d41283362c1e896d821cd 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -221,15 +221,9 @@ static inline int unix_may_send(struct sock *sk, struct sock *osk) return unix_peer(osk) == NULL || unix_our_peer(sk, osk); } -static inline int unix_recvq_full(const struct sock *sk) -{ - return skb_queue_len(&sk->sk_receive_queue) > sk->sk_max_ack_backlog; -} - static inline int unix_recvq_full_lockless(const struct sock *sk) { - return skb_queue_len_lockless(&sk->sk_receive_queue) > - READ_ONCE(sk->sk_max_ack_backlog); + return skb_queue_len_lockless(&sk->sk_receive_queue) > sk->sk_max_ack_backlog; } struct sock *unix_peer_get(struct sock *s) @@ -530,10 +524,10 @@ static int unix_dgram_peer_wake_me(struct sock *sk, struct sock *other) return 0; } -static int unix_writable(const struct sock *sk) +static int unix_writable(const struct sock *sk, unsigned char state) { - return sk->sk_state != TCP_LISTEN && - (refcount_read(&sk->sk_wmem_alloc) << 2) <= sk->sk_sndbuf; + return state != TCP_LISTEN && + (refcount_read(&sk->sk_wmem_alloc) << 2) <= READ_ONCE(sk->sk_sndbuf); } static void unix_write_space(struct sock *sk) @@ -541,7 +535,7 @@ static void unix_write_space(struct sock *sk) struct socket_wq *wq; rcu_read_lock(); - if (unix_writable(sk)) { + if (unix_writable(sk, READ_ONCE(sk->sk_state))) { wq = rcu_dereference(sk->sk_wq); if (skwq_has_sleeper(wq)) wake_up_interruptible_sync_poll(&wq->wait, @@ -570,7 +564,6 @@ static void unix_dgram_disconnected(struct sock *sk, struct sock *other) sk_error_report(other); } } - other->sk_state = TCP_CLOSE; } static void unix_sock_destructor(struct sock *sk) @@ -617,7 +610,7 @@ static void unix_release_sock(struct sock *sk, int embrion) u->path.dentry = NULL; u->path.mnt = NULL; state = sk->sk_state; - sk->sk_state = TCP_CLOSE; + WRITE_ONCE(sk->sk_state, TCP_CLOSE); skpair = unix_peer(sk); unix_peer(sk) = NULL; @@ -638,7 +631,7 @@ static void unix_release_sock(struct sock *sk, int embrion) unix_state_lock(skpair); /* No more writes */ WRITE_ONCE(skpair->sk_shutdown, SHUTDOWN_MASK); - if (!skb_queue_empty(&sk->sk_receive_queue) || embrion) + if (!skb_queue_empty_lockless(&sk->sk_receive_queue) || embrion) WRITE_ONCE(skpair->sk_err, ECONNRESET); unix_state_unlock(skpair); skpair->sk_state_change(skpair); @@ -654,8 +647,8 @@ static void unix_release_sock(struct sock *sk, int embrion) while ((skb = skb_dequeue(&sk->sk_receive_queue)) != NULL) { if (state == TCP_LISTEN) unix_release_sock(skb->sk, 1); + /* passed fds are erased in the kfree_skb hook */ - UNIXCB(skb).consumed = skb->len; kfree_skb(skb); } @@ -731,7 +724,7 @@ static int unix_listen(struct socket *sock, int backlog) if (sock->type != SOCK_STREAM && sock->type != SOCK_SEQPACKET) goto out; /* Only stream/seqpacket sockets accept */ err = -EINVAL; - if (!u->addr) + if (!READ_ONCE(u->addr)) goto out; /* No listens on an unbound socket */ unix_state_lock(sk); if (sk->sk_state != TCP_CLOSE && sk->sk_state != TCP_LISTEN) @@ -739,7 +732,8 @@ static int unix_listen(struct socket *sock, int backlog) if (backlog > sk->sk_max_ack_backlog) wake_up_interruptible_all(&u->peer_wait); sk->sk_max_ack_backlog = backlog; - sk->sk_state = TCP_LISTEN; + WRITE_ONCE(sk->sk_state, TCP_LISTEN); + /* set credentials so connect can copy them */ init_peercred(sk); err = 0; @@ -976,7 +970,7 @@ static struct sock *unix_create1(struct net *net, struct socket *sock, int kern, sk->sk_hash = unix_unbound_hash(sk); sk->sk_allocation = GFP_KERNEL_ACCOUNT; sk->sk_write_space = unix_write_space; - sk->sk_max_ack_backlog = net->unx.sysctl_max_dgram_qlen; + sk->sk_max_ack_backlog = READ_ONCE(net->unx.sysctl_max_dgram_qlen); sk->sk_destruct = unix_sock_destructor; u = unix_sk(sk); u->listener = NULL; @@ -1131,8 +1125,8 @@ static struct sock *unix_find_other(struct net *net, static int unix_autobind(struct sock *sk) { - unsigned int new_hash, old_hash = sk->sk_hash; struct unix_sock *u = unix_sk(sk); + unsigned int new_hash, old_hash; struct net *net = sock_net(sk); struct unix_address *addr; u32 lastnum, ordernum; @@ -1155,6 +1149,7 @@ static int unix_autobind(struct sock *sk) addr->name->sun_family = AF_UNIX; refcount_set(&addr->refcnt, 1); + old_hash = sk->sk_hash; ordernum = get_random_u32(); lastnum = ordernum & 0xFFFFF; retry: @@ -1195,8 +1190,8 @@ static int unix_bind_bsd(struct sock *sk, struct sockaddr_un *sunaddr, { umode_t mode = S_IFSOCK | (SOCK_INODE(sk->sk_socket)->i_mode & ~current_umask()); - unsigned int new_hash, old_hash = sk->sk_hash; struct unix_sock *u = unix_sk(sk); + unsigned int new_hash, old_hash; struct net *net = sock_net(sk); struct mnt_idmap *idmap; struct unix_address *addr; @@ -1234,6 +1229,7 @@ static int unix_bind_bsd(struct sock *sk, struct sockaddr_un *sunaddr, if (u->addr) goto out_unlock; + old_hash = sk->sk_hash; new_hash = unix_bsd_hash(d_backing_inode(dentry)); unix_table_double_lock(net, old_hash, new_hash); u->path.mnt = mntget(parent.mnt); @@ -1261,8 +1257,8 @@ out: static int unix_bind_abstract(struct sock *sk, struct sockaddr_un *sunaddr, int addr_len) { - unsigned int new_hash, old_hash = sk->sk_hash; struct unix_sock *u = unix_sk(sk); + unsigned int new_hash, old_hash; struct net *net = sock_net(sk); struct unix_address *addr; int err; @@ -1280,6 +1276,7 @@ static int unix_bind_abstract(struct sock *sk, struct sockaddr_un *sunaddr, goto out_mutex; } + old_hash = sk->sk_hash; new_hash = unix_abstract_hash(addr->name, addr->len, sk->sk_type); unix_table_double_lock(net, old_hash, new_hash); @@ -1369,7 +1366,7 @@ static int unix_dgram_connect(struct socket *sock, struct sockaddr *addr, if ((test_bit(SOCK_PASSCRED, &sock->flags) || test_bit(SOCK_PASSPIDFD, &sock->flags)) && - !unix_sk(sk)->addr) { + !READ_ONCE(unix_sk(sk)->addr)) { err = unix_autobind(sk); if (err) goto out; @@ -1399,7 +1396,8 @@ restart: if (err) goto out_unlock; - sk->sk_state = other->sk_state = TCP_ESTABLISHED; + WRITE_ONCE(sk->sk_state, TCP_ESTABLISHED); + WRITE_ONCE(other->sk_state, TCP_ESTABLISHED); } else { /* * 1003.1g breaking connected state with AF_UNSPEC @@ -1416,13 +1414,20 @@ restart: unix_peer(sk) = other; if (!other) - sk->sk_state = TCP_CLOSE; + WRITE_ONCE(sk->sk_state, TCP_CLOSE); unix_dgram_peer_wake_disconnect_wakeup(sk, old_peer); unix_state_double_unlock(sk, other); - if (other != old_peer) + if (other != old_peer) { unix_dgram_disconnected(sk, old_peer); + + unix_state_lock(old_peer); + if (!unix_peer(old_peer)) + WRITE_ONCE(old_peer->sk_state, TCP_CLOSE); + unix_state_unlock(old_peer); + } + sock_put(old_peer); } else { unix_peer(sk) = other; @@ -1470,7 +1475,6 @@ static int unix_stream_connect(struct socket *sock, struct sockaddr *uaddr, struct sk_buff *skb = NULL; long timeo; int err; - int st; err = unix_validate_addr(sunaddr, addr_len); if (err) @@ -1481,7 +1485,8 @@ static int unix_stream_connect(struct socket *sock, struct sockaddr *uaddr, goto out; if ((test_bit(SOCK_PASSCRED, &sock->flags) || - test_bit(SOCK_PASSPIDFD, &sock->flags)) && !u->addr) { + test_bit(SOCK_PASSPIDFD, &sock->flags)) && + !READ_ONCE(u->addr)) { err = unix_autobind(sk); if (err) goto out; @@ -1534,7 +1539,7 @@ restart: if (other->sk_shutdown & RCV_SHUTDOWN) goto out_unlock; - if (unix_recvq_full(other)) { + if (unix_recvq_full_lockless(other)) { err = -EAGAIN; if (!timeo) goto out_unlock; @@ -1559,9 +1564,7 @@ restart: Well, and we have to recheck the state after socket locked. */ - st = sk->sk_state; - - switch (st) { + switch (READ_ONCE(sk->sk_state)) { case TCP_CLOSE: /* This is ok... continue with connect */ break; @@ -1576,7 +1579,7 @@ restart: unix_state_lock_nested(sk, U_LOCK_SECOND); - if (sk->sk_state != st) { + if (sk->sk_state != TCP_CLOSE) { unix_state_unlock(sk); unix_state_unlock(other); sock_put(other); @@ -1629,7 +1632,7 @@ restart: copy_peercred(sk, other); sock->state = SS_CONNECTED; - sk->sk_state = TCP_ESTABLISHED; + WRITE_ONCE(sk->sk_state, TCP_ESTABLISHED); sock_hold(newsk); smp_mb__after_atomic(); /* sock_hold() does an atomic_inc() */ @@ -1701,7 +1704,7 @@ static int unix_accept(struct socket *sock, struct socket *newsock, goto out; arg->err = -EINVAL; - if (sk->sk_state != TCP_LISTEN) + if (READ_ONCE(sk->sk_state) != TCP_LISTEN) goto out; /* If socket state is TCP_LISTEN it cannot change (for now...), @@ -1950,14 +1953,15 @@ static int unix_dgram_sendmsg(struct socket *sock, struct msghdr *msg, } if ((test_bit(SOCK_PASSCRED, &sock->flags) || - test_bit(SOCK_PASSPIDFD, &sock->flags)) && !u->addr) { + test_bit(SOCK_PASSPIDFD, &sock->flags)) && + !READ_ONCE(u->addr)) { err = unix_autobind(sk); if (err) goto out; } err = -EMSGSIZE; - if (len > sk->sk_sndbuf - 32) + if (len > READ_ONCE(sk->sk_sndbuf) - 32) goto out; if (len > SKB_MAX_ALLOC) { @@ -2039,7 +2043,7 @@ restart_locked: unix_peer(sk) = NULL; unix_dgram_peer_wake_disconnect_wakeup(sk, other); - sk->sk_state = TCP_CLOSE; + WRITE_ONCE(sk->sk_state, TCP_CLOSE); unix_state_unlock(sk); unix_dgram_disconnected(sk, other); @@ -2216,7 +2220,7 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg, } if (msg->msg_namelen) { - err = sk->sk_state == TCP_ESTABLISHED ? -EISCONN : -EOPNOTSUPP; + err = READ_ONCE(sk->sk_state) == TCP_ESTABLISHED ? -EISCONN : -EOPNOTSUPP; goto out_err; } else { err = -ENOTCONN; @@ -2237,7 +2241,7 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg, &err, 0); } else { /* Keep two messages in the pipe so it schedules better */ - size = min_t(int, size, (sk->sk_sndbuf >> 1) - 64); + size = min_t(int, size, (READ_ONCE(sk->sk_sndbuf) >> 1) - 64); /* allow fallback to order-0 allocations */ size = min_t(int, size, SKB_MAX_HEAD(0) + UNIX_SKB_FRAGS_SZ); @@ -2330,7 +2334,7 @@ static int unix_seqpacket_sendmsg(struct socket *sock, struct msghdr *msg, if (err) return err; - if (sk->sk_state != TCP_ESTABLISHED) + if (READ_ONCE(sk->sk_state) != TCP_ESTABLISHED) return -ENOTCONN; if (msg->msg_namelen) @@ -2344,7 +2348,7 @@ static int unix_seqpacket_recvmsg(struct socket *sock, struct msghdr *msg, { struct sock *sk = sock->sk; - if (sk->sk_state != TCP_ESTABLISHED) + if (READ_ONCE(sk->sk_state) != TCP_ESTABLISHED) return -ENOTCONN; return unix_dgram_recvmsg(sock, msg, size, flags); @@ -2649,7 +2653,7 @@ static struct sk_buff *manage_oob(struct sk_buff *skb, struct sock *sk, static int unix_stream_read_skb(struct sock *sk, skb_read_actor_t recv_actor) { - if (unlikely(sk->sk_state != TCP_ESTABLISHED)) + if (unlikely(READ_ONCE(sk->sk_state) != TCP_ESTABLISHED)) return -ENOTCONN; return unix_read_skb(sk, recv_actor); @@ -2673,7 +2677,7 @@ static int unix_stream_read_generic(struct unix_stream_read_state *state, size_t size = state->size; unsigned int last_len; - if (unlikely(sk->sk_state != TCP_ESTABLISHED)) { + if (unlikely(READ_ONCE(sk->sk_state) != TCP_ESTABLISHED)) { err = -EINVAL; goto out; } @@ -2699,9 +2703,8 @@ static int unix_stream_read_generic(struct unix_stream_read_state *state, skip = max(sk_peek_offset(sk, flags), 0); do { - int chunk; - bool drop_skb; struct sk_buff *skb, *last; + int chunk; redo: unix_state_lock(sk); @@ -2797,11 +2800,7 @@ unlock: } chunk = min_t(unsigned int, unix_skb_len(skb) - skip, size); - skb_get(skb); chunk = state->recv_actor(skb, skip, chunk, state); - drop_skb = !unix_skb_len(skb); - /* skb is only safe to use if !drop_skb */ - consume_skb(skb); if (chunk < 0) { if (copied == 0) copied = -EFAULT; @@ -2810,18 +2809,6 @@ unlock: copied += chunk; size -= chunk; - if (drop_skb) { - /* the skb was touched by a concurrent reader; - * we should not expect anything from this skb - * anymore and assume it invalid - we can be - * sure it was dropped from the socket queue - * - * let's report a short read - */ - err = 0; - break; - } - /* Mark read part of skb as used */ if (!(flags & MSG_PEEK)) { UNIXCB(skb).consumed += chunk; @@ -3004,7 +2991,7 @@ long unix_inq_len(struct sock *sk) struct sk_buff *skb; long amount = 0; - if (sk->sk_state == TCP_LISTEN) + if (READ_ONCE(sk->sk_state) == TCP_LISTEN) return -EINVAL; spin_lock(&sk->sk_receive_queue.lock); @@ -3116,12 +3103,14 @@ static int unix_compat_ioctl(struct socket *sock, unsigned int cmd, unsigned lon static __poll_t unix_poll(struct file *file, struct socket *sock, poll_table *wait) { struct sock *sk = sock->sk; + unsigned char state; __poll_t mask; u8 shutdown; sock_poll_wait(file, sock, wait); mask = 0; shutdown = READ_ONCE(sk->sk_shutdown); + state = READ_ONCE(sk->sk_state); /* exceptional events? */ if (READ_ONCE(sk->sk_err)) @@ -3143,14 +3132,14 @@ static __poll_t unix_poll(struct file *file, struct socket *sock, poll_table *wa /* Connection-based need to check for termination and startup */ if ((sk->sk_type == SOCK_STREAM || sk->sk_type == SOCK_SEQPACKET) && - sk->sk_state == TCP_CLOSE) + state == TCP_CLOSE) mask |= EPOLLHUP; /* * we set writable also when the other side has shut down the * connection. This prevents stuck sockets. */ - if (unix_writable(sk)) + if (unix_writable(sk, state)) mask |= EPOLLOUT | EPOLLWRNORM | EPOLLWRBAND; return mask; @@ -3161,12 +3150,14 @@ static __poll_t unix_dgram_poll(struct file *file, struct socket *sock, { struct sock *sk = sock->sk, *other; unsigned int writable; + unsigned char state; __poll_t mask; u8 shutdown; sock_poll_wait(file, sock, wait); mask = 0; shutdown = READ_ONCE(sk->sk_shutdown); + state = READ_ONCE(sk->sk_state); /* exceptional events? */ if (READ_ONCE(sk->sk_err) || @@ -3186,19 +3177,14 @@ static __poll_t unix_dgram_poll(struct file *file, struct socket *sock, mask |= EPOLLIN | EPOLLRDNORM; /* Connection-based need to check for termination and startup */ - if (sk->sk_type == SOCK_SEQPACKET) { - if (sk->sk_state == TCP_CLOSE) - mask |= EPOLLHUP; - /* connection hasn't started yet? */ - if (sk->sk_state == TCP_SYN_SENT) - return mask; - } + if (sk->sk_type == SOCK_SEQPACKET && state == TCP_CLOSE) + mask |= EPOLLHUP; /* No write status requested, avoid expensive OUT tests. */ if (!(poll_requested_events(wait) & (EPOLLWRBAND|EPOLLWRNORM|EPOLLOUT))) return mask; - writable = unix_writable(sk); + writable = unix_writable(sk, state); if (writable) { unix_state_lock(sk); diff --git a/net/unix/diag.c b/net/unix/diag.c index ae39538c5042b34e864e709004bf554841eaf4c9..937edf4afed41339afce117bd08f9a58bb2a6118 100644 --- a/net/unix/diag.c +++ b/net/unix/diag.c @@ -65,7 +65,7 @@ static int sk_diag_dump_icons(struct sock *sk, struct sk_buff *nlskb) u32 *buf; int i; - if (sk->sk_state == TCP_LISTEN) { + if (READ_ONCE(sk->sk_state) == TCP_LISTEN) { spin_lock(&sk->sk_receive_queue.lock); attr = nla_reserve(nlskb, UNIX_DIAG_ICONS, @@ -103,8 +103,8 @@ static int sk_diag_show_rqlen(struct sock *sk, struct sk_buff *nlskb) { struct unix_diag_rqlen rql; - if (sk->sk_state == TCP_LISTEN) { - rql.udiag_rqueue = sk->sk_receive_queue.qlen; + if (READ_ONCE(sk->sk_state) == TCP_LISTEN) { + rql.udiag_rqueue = skb_queue_len_lockless(&sk->sk_receive_queue); rql.udiag_wqueue = sk->sk_max_ack_backlog; } else { rql.udiag_rqueue = (u32) unix_inq_len(sk); @@ -136,7 +136,7 @@ static int sk_diag_fill(struct sock *sk, struct sk_buff *skb, struct unix_diag_r rep = nlmsg_data(nlh); rep->udiag_family = AF_UNIX; rep->udiag_type = sk->sk_type; - rep->udiag_state = sk->sk_state; + rep->udiag_state = READ_ONCE(sk->sk_state); rep->pad = 0; rep->udiag_ino = sk_ino; sock_diag_save_cookie(sk, rep->udiag_cookie); @@ -165,7 +165,7 @@ static int sk_diag_fill(struct sock *sk, struct sk_buff *skb, struct unix_diag_r sock_diag_put_meminfo(sk, skb, UNIX_DIAG_MEMINFO)) goto out_nlmsg_trim; - if (nla_put_u8(skb, UNIX_DIAG_SHUTDOWN, sk->sk_shutdown)) + if (nla_put_u8(skb, UNIX_DIAG_SHUTDOWN, READ_ONCE(sk->sk_shutdown))) goto out_nlmsg_trim; if ((req->udiag_show & UDIAG_SHOW_UID) && @@ -215,7 +215,7 @@ static int unix_diag_dump(struct sk_buff *skb, struct netlink_callback *cb) sk_for_each(sk, &net->unx.table.buckets[slot]) { if (num < s_num) goto next; - if (!(req->udiag_states & (1 << sk->sk_state))) + if (!(req->udiag_states & (1 << READ_ONCE(sk->sk_state)))) goto next; if (sk_diag_dump(sk, skb, req, sk_user_ns(skb->sk), NETLINK_CB(cb->skb).portid, diff --git a/net/wireless/core.c b/net/wireless/core.c index 3fb1b637352a9d0b469206d890601031ffd4c68f..4d5d351bd0b51e6d74230d946740af9da18e6633 100644 --- a/net/wireless/core.c +++ b/net/wireless/core.c @@ -421,6 +421,8 @@ static void cfg80211_wiphy_work(struct work_struct *work) rdev = container_of(work, struct cfg80211_registered_device, wiphy_work); + trace_wiphy_work_worker_start(&rdev->wiphy); + wiphy_lock(&rdev->wiphy); if (rdev->suspended) goto out; @@ -431,9 +433,10 @@ static void cfg80211_wiphy_work(struct work_struct *work) if (wk) { list_del_init(&wk->entry); if (!list_empty(&rdev->wiphy_work_list)) - schedule_work(work); + queue_work(system_unbound_wq, work); spin_unlock_irq(&rdev->wiphy_work_lock); + trace_wiphy_work_run(&rdev->wiphy, wk); wk->func(&rdev->wiphy, wk); } else { spin_unlock_irq(&rdev->wiphy_work_lock); @@ -1066,6 +1069,7 @@ void cfg80211_process_wiphy_works(struct cfg80211_registered_device *rdev, list_del_init(&wk->entry); spin_unlock_irqrestore(&rdev->wiphy_work_lock, flags); + trace_wiphy_work_run(&rdev->wiphy, wk); wk->func(&rdev->wiphy, wk); spin_lock_irqsave(&rdev->wiphy_work_lock, flags); @@ -1141,7 +1145,8 @@ void wiphy_unregister(struct wiphy *wiphy) flush_work(&rdev->background_cac_abort_wk); cfg80211_rdev_free_wowlan(rdev); - cfg80211_rdev_free_coalesce(rdev); + cfg80211_free_coalesce(rdev->coalesce); + rdev->coalesce = NULL; } EXPORT_SYMBOL(wiphy_unregister); @@ -1610,6 +1615,8 @@ void wiphy_work_queue(struct wiphy *wiphy, struct wiphy_work *work) struct cfg80211_registered_device *rdev = wiphy_to_rdev(wiphy); unsigned long flags; + trace_wiphy_work_queue(wiphy, work); + spin_lock_irqsave(&rdev->wiphy_work_lock, flags); if (list_empty(&work->entry)) list_add_tail(&work->entry, &rdev->wiphy_work_list); @@ -1626,6 +1633,8 @@ void wiphy_work_cancel(struct wiphy *wiphy, struct wiphy_work *work) lockdep_assert_held(&wiphy->mtx); + trace_wiphy_work_cancel(wiphy, work); + spin_lock_irqsave(&rdev->wiphy_work_lock, flags); if (!list_empty(&work->entry)) list_del_init(&work->entry); @@ -1639,6 +1648,8 @@ void wiphy_work_flush(struct wiphy *wiphy, struct wiphy_work *work) unsigned long flags; bool run; + trace_wiphy_work_flush(wiphy, work); + spin_lock_irqsave(&rdev->wiphy_work_lock, flags); run = !work || !list_empty(&work->entry); spin_unlock_irqrestore(&rdev->wiphy_work_lock, flags); @@ -1660,6 +1671,8 @@ void wiphy_delayed_work_queue(struct wiphy *wiphy, struct wiphy_delayed_work *dwork, unsigned long delay) { + trace_wiphy_delayed_work_queue(wiphy, &dwork->work, delay); + if (!delay) { del_timer(&dwork->timer); wiphy_work_queue(wiphy, &dwork->work); diff --git a/net/wireless/ibss.c b/net/wireless/ibss.c index 9f02ee5f08beb40a83f6133f22c5f90343cdf442..34e5acff393510f5622d8891f68a1dd0bf3ab27e 100644 --- a/net/wireless/ibss.c +++ b/net/wireless/ibss.c @@ -3,7 +3,7 @@ * Some IBSS support code for cfg80211. * * Copyright 2009 Johannes Berg - * Copyright (C) 2020-2023 Intel Corporation + * Copyright (C) 2020-2024 Intel Corporation */ #include @@ -94,6 +94,9 @@ int __cfg80211_join_ibss(struct cfg80211_registered_device *rdev, lockdep_assert_held(&rdev->wiphy.mtx); + if (wdev->cac_started) + return -EBUSY; + if (wdev->u.ibss.ssid_len) return -EALREADY; diff --git a/net/wireless/mesh.c b/net/wireless/mesh.c index 83306979fbe21894d51ea117b2c077635e076f58..aaca65b66af48e2c27c748c971265304d402e305 100644 --- a/net/wireless/mesh.c +++ b/net/wireless/mesh.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* * Portions - * Copyright (C) 2022-2023 Intel Corporation + * Copyright (C) 2022-2024 Intel Corporation */ #include #include @@ -127,6 +127,9 @@ int __cfg80211_join_mesh(struct cfg80211_registered_device *rdev, if (!rdev->ops->join_mesh) return -EOPNOTSUPP; + if (wdev->cac_started) + return -EBUSY; + if (!setup->chandef.chan) { /* if no channel explicitly given, use preset channel */ setup->chandef = wdev->u.mesh.preset_chandef; diff --git a/net/wireless/nl80211.c b/net/wireless/nl80211.c index 3c0bca4238d357c01b6fe92bb0f2b2b8a2917725..8ff5f79d446ae1fa104059a72d42e55cc49fa679 100644 --- a/net/wireless/nl80211.c +++ b/net/wireless/nl80211.c @@ -3415,6 +3415,33 @@ static int __nl80211_set_channel(struct cfg80211_registered_device *rdev, if (chandef.chan != cur_chan) return -EBUSY; + /* only allow this for regular channel widths */ + switch (wdev->links[link_id].ap.chandef.width) { + case NL80211_CHAN_WIDTH_20_NOHT: + case NL80211_CHAN_WIDTH_20: + case NL80211_CHAN_WIDTH_40: + case NL80211_CHAN_WIDTH_80: + case NL80211_CHAN_WIDTH_80P80: + case NL80211_CHAN_WIDTH_160: + case NL80211_CHAN_WIDTH_320: + break; + default: + return -EINVAL; + } + + switch (chandef.width) { + case NL80211_CHAN_WIDTH_20_NOHT: + case NL80211_CHAN_WIDTH_20: + case NL80211_CHAN_WIDTH_40: + case NL80211_CHAN_WIDTH_80: + case NL80211_CHAN_WIDTH_80P80: + case NL80211_CHAN_WIDTH_160: + case NL80211_CHAN_WIDTH_320: + break; + default: + return -EINVAL; + } + result = rdev_set_ap_chanwidth(rdev, dev, link_id, &chandef); if (result) @@ -5937,6 +5964,9 @@ static int nl80211_start_ap(struct sk_buff *skb, struct genl_info *info) if (!rdev->ops->start_ap) return -EOPNOTSUPP; + if (wdev->cac_started) + return -EBUSY; + if (wdev->links[link_id].ap.beacon_interval) return -EALREADY; @@ -9929,6 +9959,17 @@ static int nl80211_start_radar_detection(struct sk_buff *skb, flush_delayed_work(&rdev->dfs_update_channels_wk); + switch (wdev->iftype) { + case NL80211_IFTYPE_AP: + case NL80211_IFTYPE_P2P_GO: + case NL80211_IFTYPE_MESH_POINT: + case NL80211_IFTYPE_ADHOC: + break; + default: + /* caution - see cfg80211_beaconing_iface_active() below */ + return -EINVAL; + } + wiphy_lock(wiphy); dfs_region = reg_get_dfs_region(wiphy); @@ -9959,12 +10000,7 @@ static int nl80211_start_radar_detection(struct sk_buff *skb, goto unlock; } - if (netif_carrier_ok(dev)) { - err = -EBUSY; - goto unlock; - } - - if (wdev->cac_started) { + if (cfg80211_beaconing_iface_active(wdev) || wdev->cac_started) { err = -EBUSY; goto unlock; } @@ -13861,9 +13897,8 @@ nla_put_failure: return -ENOBUFS; } -void cfg80211_rdev_free_coalesce(struct cfg80211_registered_device *rdev) +void cfg80211_free_coalesce(struct cfg80211_coalesce *coalesce) { - struct cfg80211_coalesce *coalesce = rdev->coalesce; int i, j; struct cfg80211_coalesce_rules *rule; @@ -13872,13 +13907,13 @@ void cfg80211_rdev_free_coalesce(struct cfg80211_registered_device *rdev) for (i = 0; i < coalesce->n_rules; i++) { rule = &coalesce->rules[i]; + if (!rule) + continue; for (j = 0; j < rule->n_patterns; j++) kfree(rule->patterns[j].mask); kfree(rule->patterns); } - kfree(coalesce->rules); kfree(coalesce); - rdev->coalesce = NULL; } static int nl80211_parse_coalesce_rule(struct cfg80211_registered_device *rdev, @@ -13976,17 +14011,16 @@ static int nl80211_set_coalesce(struct sk_buff *skb, struct genl_info *info) { struct cfg80211_registered_device *rdev = info->user_ptr[0]; const struct wiphy_coalesce_support *coalesce = rdev->wiphy.coalesce; - struct cfg80211_coalesce new_coalesce = {}; - struct cfg80211_coalesce *n_coalesce; - int err, rem_rule, n_rules = 0, i, j; + struct cfg80211_coalesce *new_coalesce; + int err, rem_rule, n_rules = 0, i; struct nlattr *rule; - struct cfg80211_coalesce_rules *tmp_rule; if (!rdev->wiphy.coalesce || !rdev->ops->set_coalesce) return -EOPNOTSUPP; if (!info->attrs[NL80211_ATTR_COALESCE_RULE]) { - cfg80211_rdev_free_coalesce(rdev); + cfg80211_free_coalesce(rdev->coalesce); + rdev->coalesce = NULL; rdev_set_coalesce(rdev, NULL); return 0; } @@ -13997,47 +14031,34 @@ static int nl80211_set_coalesce(struct sk_buff *skb, struct genl_info *info) if (n_rules > coalesce->n_rules) return -EINVAL; - new_coalesce.rules = kcalloc(n_rules, sizeof(new_coalesce.rules[0]), - GFP_KERNEL); - if (!new_coalesce.rules) + new_coalesce = kzalloc(struct_size(new_coalesce, rules, n_rules), + GFP_KERNEL); + if (!new_coalesce) return -ENOMEM; - new_coalesce.n_rules = n_rules; + new_coalesce->n_rules = n_rules; i = 0; nla_for_each_nested(rule, info->attrs[NL80211_ATTR_COALESCE_RULE], rem_rule) { err = nl80211_parse_coalesce_rule(rdev, rule, - &new_coalesce.rules[i]); + &new_coalesce->rules[i]); if (err) goto error; i++; } - err = rdev_set_coalesce(rdev, &new_coalesce); + err = rdev_set_coalesce(rdev, new_coalesce); if (err) goto error; - n_coalesce = kmemdup(&new_coalesce, sizeof(new_coalesce), GFP_KERNEL); - if (!n_coalesce) { - err = -ENOMEM; - goto error; - } - cfg80211_rdev_free_coalesce(rdev); - rdev->coalesce = n_coalesce; + cfg80211_free_coalesce(rdev->coalesce); + rdev->coalesce = new_coalesce; return 0; error: - for (i = 0; i < new_coalesce.n_rules; i++) { - tmp_rule = &new_coalesce.rules[i]; - if (!tmp_rule) - continue; - for (j = 0; j < tmp_rule->n_patterns; j++) - kfree(tmp_rule->patterns[j].mask); - kfree(tmp_rule->patterns); - } - kfree(new_coalesce.rules); + cfg80211_free_coalesce(new_coalesce); return err; } diff --git a/net/wireless/nl80211.h b/net/wireless/nl80211.h index 6376f3a87f8abf6e2007c06c287c111f65d499b2..ffaab9a92e5bc75ddbce5fddec81e3a26a05f32b 100644 --- a/net/wireless/nl80211.h +++ b/net/wireless/nl80211.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* * Portions of this file - * Copyright (C) 2018, 2020-2022 Intel Corporation + * Copyright (C) 2018, 2020-2024 Intel Corporation */ #ifndef __NET_WIRELESS_NL80211_H #define __NET_WIRELESS_NL80211_H @@ -119,7 +119,7 @@ nl80211_radar_notify(struct cfg80211_registered_device *rdev, void nl80211_send_ap_stopped(struct wireless_dev *wdev, unsigned int link_id); -void cfg80211_rdev_free_coalesce(struct cfg80211_registered_device *rdev); +void cfg80211_free_coalesce(struct cfg80211_coalesce *coalesce); /* peer measurement */ int nl80211_pmsr_start(struct sk_buff *skb, struct genl_info *info); diff --git a/net/wireless/pmsr.c b/net/wireless/pmsr.c index e106dcea3977828456992ca3ca9af97d7c9775c7..c569c37da31758a1b7f4051541537588140ef45d 100644 --- a/net/wireless/pmsr.c +++ b/net/wireless/pmsr.c @@ -56,7 +56,7 @@ static int pmsr_parse_ftm(struct cfg80211_registered_device *rdev, out->ftm.burst_period = 0; if (tb[NL80211_PMSR_FTM_REQ_ATTR_BURST_PERIOD]) out->ftm.burst_period = - nla_get_u32(tb[NL80211_PMSR_FTM_REQ_ATTR_BURST_PERIOD]); + nla_get_u16(tb[NL80211_PMSR_FTM_REQ_ATTR_BURST_PERIOD]); out->ftm.asap = !!tb[NL80211_PMSR_FTM_REQ_ATTR_ASAP]; if (out->ftm.asap && !capa->ftm.asap) { @@ -75,7 +75,7 @@ static int pmsr_parse_ftm(struct cfg80211_registered_device *rdev, out->ftm.num_bursts_exp = 0; if (tb[NL80211_PMSR_FTM_REQ_ATTR_NUM_BURSTS_EXP]) out->ftm.num_bursts_exp = - nla_get_u32(tb[NL80211_PMSR_FTM_REQ_ATTR_NUM_BURSTS_EXP]); + nla_get_u8(tb[NL80211_PMSR_FTM_REQ_ATTR_NUM_BURSTS_EXP]); if (capa->ftm.max_bursts_exponent >= 0 && out->ftm.num_bursts_exp > capa->ftm.max_bursts_exponent) { @@ -88,7 +88,7 @@ static int pmsr_parse_ftm(struct cfg80211_registered_device *rdev, out->ftm.burst_duration = 15; if (tb[NL80211_PMSR_FTM_REQ_ATTR_BURST_DURATION]) out->ftm.burst_duration = - nla_get_u32(tb[NL80211_PMSR_FTM_REQ_ATTR_BURST_DURATION]); + nla_get_u8(tb[NL80211_PMSR_FTM_REQ_ATTR_BURST_DURATION]); out->ftm.ftms_per_burst = 0; if (tb[NL80211_PMSR_FTM_REQ_ATTR_FTMS_PER_BURST]) @@ -107,7 +107,7 @@ static int pmsr_parse_ftm(struct cfg80211_registered_device *rdev, out->ftm.ftmr_retries = 3; if (tb[NL80211_PMSR_FTM_REQ_ATTR_NUM_FTMR_RETRIES]) out->ftm.ftmr_retries = - nla_get_u32(tb[NL80211_PMSR_FTM_REQ_ATTR_NUM_FTMR_RETRIES]); + nla_get_u8(tb[NL80211_PMSR_FTM_REQ_ATTR_NUM_FTMR_RETRIES]); out->ftm.request_lci = !!tb[NL80211_PMSR_FTM_REQ_ATTR_REQUEST_LCI]; if (out->ftm.request_lci && !capa->ftm.request_lci) { diff --git a/net/wireless/rdev-ops.h b/net/wireless/rdev-ops.h index 43897a5269b6a9e011e8e31bd882f55cd19480c5..755af47b88b91a6a315daecbc47341b9f4c71eb8 100644 --- a/net/wireless/rdev-ops.h +++ b/net/wireless/rdev-ops.h @@ -2,7 +2,7 @@ /* * Portions of this file * Copyright(c) 2016-2017 Intel Deutschland GmbH - * Copyright (C) 2018, 2021-2023 Intel Corporation + * Copyright (C) 2018, 2021-2024 Intel Corporation */ #ifndef __CFG80211_RDEV_OPS #define __CFG80211_RDEV_OPS @@ -458,6 +458,10 @@ static inline int rdev_scan(struct cfg80211_registered_device *rdev, struct cfg80211_scan_request *request) { int ret; + + if (WARN_ON_ONCE(!request->n_ssids && request->ssids)) + return -EINVAL; + trace_rdev_scan(&rdev->wiphy, request); ret = rdev->ops->scan(&rdev->wiphy, request); trace_rdev_return_int(&rdev->wiphy, ret); diff --git a/net/wireless/scan.c b/net/wireless/scan.c index 127853877a0ad14fec0f92615976c8fc8d33c3ef..2f2a3163968a7cc5c44ce85f73068e0a2765df76 100644 --- a/net/wireless/scan.c +++ b/net/wireless/scan.c @@ -812,6 +812,7 @@ static int cfg80211_scan_6ghz(struct cfg80211_registered_device *rdev) LIST_HEAD(coloc_ap_list); bool need_scan_psc = true; const struct ieee80211_sband_iftype_data *iftd; + size_t size, offs_ssids, offs_6ghz_params, offs_ies; rdev_req->scan_6ghz = true; @@ -877,10 +878,15 @@ static int cfg80211_scan_6ghz(struct cfg80211_registered_device *rdev) spin_unlock_bh(&rdev->bss_lock); } - request = kzalloc(struct_size(request, channels, n_channels) + - sizeof(*request->scan_6ghz_params) * count + - sizeof(*request->ssids) * rdev_req->n_ssids, - GFP_KERNEL); + size = struct_size(request, channels, n_channels); + offs_ssids = size; + size += sizeof(*request->ssids) * rdev_req->n_ssids; + offs_6ghz_params = size; + size += sizeof(*request->scan_6ghz_params) * count; + offs_ies = size; + size += rdev_req->ie_len; + + request = kzalloc(size, GFP_KERNEL); if (!request) { cfg80211_free_coloc_ap_list(&coloc_ap_list); return -ENOMEM; @@ -888,8 +894,26 @@ static int cfg80211_scan_6ghz(struct cfg80211_registered_device *rdev) *request = *rdev_req; request->n_channels = 0; - request->scan_6ghz_params = - (void *)&request->channels[n_channels]; + request->n_6ghz_params = 0; + if (rdev_req->n_ssids) { + /* + * Add the ssids from the parent scan request to the new + * scan request, so the driver would be able to use them + * in its probe requests to discover hidden APs on PSC + * channels. + */ + request->ssids = (void *)request + offs_ssids; + memcpy(request->ssids, rdev_req->ssids, + sizeof(*request->ssids) * request->n_ssids); + } + request->scan_6ghz_params = (void *)request + offs_6ghz_params; + + if (rdev_req->ie_len) { + void *ie = (void *)request + offs_ies; + + memcpy(ie, rdev_req->ie, rdev_req->ie_len); + request->ie = ie; + } /* * PSC channels should not be scanned in case of direct scan with 1 SSID @@ -978,17 +1002,8 @@ skip: if (request->n_channels) { struct cfg80211_scan_request *old = rdev->int_scan_req; - rdev->int_scan_req = request; - /* - * Add the ssids from the parent scan request to the new scan - * request, so the driver would be able to use them in its - * probe requests to discover hidden APs on PSC channels. - */ - request->ssids = (void *)&request->channels[request->n_channels]; - request->n_ssids = rdev_req->n_ssids; - memcpy(request->ssids, rdev_req->ssids, sizeof(*request->ssids) * - request->n_ssids); + rdev->int_scan_req = request; /* * If this scan follows a previous scan, save the scan start @@ -2128,7 +2143,8 @@ static bool cfg80211_6ghz_power_type_valid(const u8 *ie, size_t ielen, struct ieee80211_he_operation *he_oper; tmp = cfg80211_find_ext_elem(WLAN_EID_EXT_HE_OPERATION, ie, ielen); - if (tmp && tmp->datalen >= sizeof(*he_oper) + 1) { + if (tmp && tmp->datalen >= sizeof(*he_oper) + 1 && + tmp->datalen >= ieee80211_he_oper_size(tmp->data + 1)) { const struct ieee80211_he_6ghz_oper *he_6ghz_oper; he_oper = (void *)&tmp->data[1]; diff --git a/net/wireless/sysfs.c b/net/wireless/sysfs.c index 565511a3f461ed6872db387439a196351d02f00a..62f26618f674741a5163fa8e8d14c5319b2ceff8 100644 --- a/net/wireless/sysfs.c +++ b/net/wireless/sysfs.c @@ -5,7 +5,7 @@ * * Copyright 2005-2006 Jiri Benc * Copyright 2006 Johannes Berg - * Copyright (C) 2020-2021, 2023 Intel Corporation + * Copyright (C) 2020-2021, 2023-2024 Intel Corporation */ #include @@ -137,7 +137,7 @@ static int wiphy_resume(struct device *dev) if (rdev->wiphy.registered && rdev->ops->resume) ret = rdev_resume(rdev); rdev->suspended = false; - schedule_work(&rdev->wiphy_work); + queue_work(system_unbound_wq, &rdev->wiphy_work); wiphy_unlock(&rdev->wiphy); if (ret) diff --git a/net/wireless/trace.h b/net/wireless/trace.h index 87986170d1b10d259581e6d12a9d7e7db0204482..6ef9294747e3dc244ad22cc67101b0ff78b7491f 100644 --- a/net/wireless/trace.h +++ b/net/wireless/trace.h @@ -242,6 +242,80 @@ } \ } while (0) +/************************************************************* + * wiphy work traces * + *************************************************************/ + +DECLARE_EVENT_CLASS(wiphy_work_event, + TP_PROTO(struct wiphy *wiphy, struct wiphy_work *work), + TP_ARGS(wiphy, work), + TP_STRUCT__entry( + WIPHY_ENTRY + __field(void *, instance) + __field(void *, func) + ), + TP_fast_assign( + WIPHY_ASSIGN; + __entry->instance = work; + __entry->func = work ? work->func : NULL; + ), + TP_printk(WIPHY_PR_FMT " instance=%p func=%pS", + WIPHY_PR_ARG, __entry->instance, __entry->func) +); + +DEFINE_EVENT(wiphy_work_event, wiphy_work_queue, + TP_PROTO(struct wiphy *wiphy, struct wiphy_work *work), + TP_ARGS(wiphy, work) +); + +DEFINE_EVENT(wiphy_work_event, wiphy_work_run, + TP_PROTO(struct wiphy *wiphy, struct wiphy_work *work), + TP_ARGS(wiphy, work) +); + +DEFINE_EVENT(wiphy_work_event, wiphy_work_cancel, + TP_PROTO(struct wiphy *wiphy, struct wiphy_work *work), + TP_ARGS(wiphy, work) +); + +DEFINE_EVENT(wiphy_work_event, wiphy_work_flush, + TP_PROTO(struct wiphy *wiphy, struct wiphy_work *work), + TP_ARGS(wiphy, work) +); + +TRACE_EVENT(wiphy_delayed_work_queue, + TP_PROTO(struct wiphy *wiphy, struct wiphy_work *work, + unsigned long delay), + TP_ARGS(wiphy, work, delay), + TP_STRUCT__entry( + WIPHY_ENTRY + __field(void *, instance) + __field(void *, func) + __field(unsigned long, delay) + ), + TP_fast_assign( + WIPHY_ASSIGN; + __entry->instance = work; + __entry->func = work->func; + __entry->delay = delay; + ), + TP_printk(WIPHY_PR_FMT " instance=%p func=%pS delay=%ld", + WIPHY_PR_ARG, __entry->instance, __entry->func, + __entry->delay) +); + +TRACE_EVENT(wiphy_work_worker_start, + TP_PROTO(struct wiphy *wiphy), + TP_ARGS(wiphy), + TP_STRUCT__entry( + WIPHY_ENTRY + ), + TP_fast_assign( + WIPHY_ASSIGN; + ), + TP_printk(WIPHY_PR_FMT, WIPHY_PR_ARG) +); + /************************************************************* * rdev->ops traces * *************************************************************/ @@ -2889,6 +2963,75 @@ DEFINE_EVENT(wiphy_wdev_link_evt, rdev_del_intf_link, TP_ARGS(wiphy, wdev, link_id) ); +TRACE_EVENT(rdev_del_link_station, + TP_PROTO(struct wiphy *wiphy, struct net_device *netdev, + struct link_station_del_parameters *params), + TP_ARGS(wiphy, netdev, params), + TP_STRUCT__entry( + WIPHY_ENTRY + NETDEV_ENTRY + __array(u8, mld_mac, 6) + __field(u32, link_id) + ), + TP_fast_assign( + WIPHY_ASSIGN; + NETDEV_ASSIGN; + memset(__entry->mld_mac, 0, 6); + if (params->mld_mac) + memcpy(__entry->mld_mac, params->mld_mac, 6); + __entry->link_id = params->link_id; + ), + TP_printk(WIPHY_PR_FMT ", " NETDEV_PR_FMT ", station mac: %pM" + ", link id: %u", + WIPHY_PR_ARG, NETDEV_PR_ARG, __entry->mld_mac, + __entry->link_id) +); + +TRACE_EVENT(rdev_set_hw_timestamp, + TP_PROTO(struct wiphy *wiphy, struct net_device *netdev, + struct cfg80211_set_hw_timestamp *hwts), + + TP_ARGS(wiphy, netdev, hwts), + + TP_STRUCT__entry( + WIPHY_ENTRY + NETDEV_ENTRY + MAC_ENTRY(macaddr) + __field(bool, enable) + ), + + TP_fast_assign( + WIPHY_ASSIGN; + NETDEV_ASSIGN; + MAC_ASSIGN(macaddr, hwts->macaddr); + __entry->enable = hwts->enable; + ), + + TP_printk(WIPHY_PR_FMT ", " NETDEV_PR_FMT ", mac %pM, enable: %u", + WIPHY_PR_ARG, NETDEV_PR_ARG, __entry->macaddr, + __entry->enable) +); + +TRACE_EVENT(rdev_set_ttlm, + TP_PROTO(struct wiphy *wiphy, struct net_device *netdev, + struct cfg80211_ttlm_params *params), + TP_ARGS(wiphy, netdev, params), + TP_STRUCT__entry( + WIPHY_ENTRY + NETDEV_ENTRY + __array(u8, dlink, sizeof(u16) * 8) + __array(u8, ulink, sizeof(u16) * 8) + ), + TP_fast_assign( + WIPHY_ASSIGN; + NETDEV_ASSIGN; + memcpy(__entry->dlink, params->dlink, sizeof(params->dlink)); + memcpy(__entry->ulink, params->ulink, sizeof(params->ulink)); + ), + TP_printk(WIPHY_PR_FMT ", " NETDEV_PR_FMT, + WIPHY_PR_ARG, NETDEV_PR_ARG) +); + /************************************************************* * cfg80211 exported functions traces * *************************************************************/ @@ -3923,55 +4066,6 @@ DEFINE_EVENT(link_station_add_mod, rdev_mod_link_station, TP_ARGS(wiphy, netdev, params) ); -TRACE_EVENT(rdev_del_link_station, - TP_PROTO(struct wiphy *wiphy, struct net_device *netdev, - struct link_station_del_parameters *params), - TP_ARGS(wiphy, netdev, params), - TP_STRUCT__entry( - WIPHY_ENTRY - NETDEV_ENTRY - __array(u8, mld_mac, 6) - __field(u32, link_id) - ), - TP_fast_assign( - WIPHY_ASSIGN; - NETDEV_ASSIGN; - memset(__entry->mld_mac, 0, 6); - if (params->mld_mac) - memcpy(__entry->mld_mac, params->mld_mac, 6); - __entry->link_id = params->link_id; - ), - TP_printk(WIPHY_PR_FMT ", " NETDEV_PR_FMT ", station mac: %pM" - ", link id: %u", - WIPHY_PR_ARG, NETDEV_PR_ARG, __entry->mld_mac, - __entry->link_id) -); - -TRACE_EVENT(rdev_set_hw_timestamp, - TP_PROTO(struct wiphy *wiphy, struct net_device *netdev, - struct cfg80211_set_hw_timestamp *hwts), - - TP_ARGS(wiphy, netdev, hwts), - - TP_STRUCT__entry( - WIPHY_ENTRY - NETDEV_ENTRY - MAC_ENTRY(macaddr) - __field(bool, enable) - ), - - TP_fast_assign( - WIPHY_ASSIGN; - NETDEV_ASSIGN; - MAC_ASSIGN(macaddr, hwts->macaddr); - __entry->enable = hwts->enable; - ), - - TP_printk(WIPHY_PR_FMT ", " NETDEV_PR_FMT ", mac %pM, enable: %u", - WIPHY_PR_ARG, NETDEV_PR_ARG, __entry->macaddr, - __entry->enable) -); - TRACE_EVENT(cfg80211_links_removed, TP_PROTO(struct net_device *netdev, u16 link_mask), TP_ARGS(netdev, link_mask), @@ -3987,26 +4081,6 @@ TRACE_EVENT(cfg80211_links_removed, __entry->link_mask) ); -TRACE_EVENT(rdev_set_ttlm, - TP_PROTO(struct wiphy *wiphy, struct net_device *netdev, - struct cfg80211_ttlm_params *params), - TP_ARGS(wiphy, netdev, params), - TP_STRUCT__entry( - WIPHY_ENTRY - NETDEV_ENTRY - __array(u8, dlink, sizeof(u16) * 8) - __array(u8, ulink, sizeof(u16) * 8) - ), - TP_fast_assign( - WIPHY_ASSIGN; - NETDEV_ASSIGN; - memcpy(__entry->dlink, params->dlink, sizeof(params->dlink)); - memcpy(__entry->ulink, params->ulink, sizeof(params->ulink)); - ), - TP_printk(WIPHY_PR_FMT ", " NETDEV_PR_FMT, - WIPHY_PR_ARG, NETDEV_PR_ARG) -); - #endif /* !__RDEV_OPS_TRACE || TRACE_HEADER_MULTI_READ */ #undef TRACE_INCLUDE_PATH diff --git a/net/wireless/util.c b/net/wireless/util.c index 2bde8a35463132d32dd0a185566ad4e72012e96a..082c6f9c5416eb5da26282f6a030440c79ed223c 100644 --- a/net/wireless/util.c +++ b/net/wireless/util.c @@ -2549,6 +2549,7 @@ int cfg80211_get_station(struct net_device *dev, const u8 *mac_addr, { struct cfg80211_registered_device *rdev; struct wireless_dev *wdev; + int ret; wdev = dev->ieee80211_ptr; if (!wdev) @@ -2560,7 +2561,11 @@ int cfg80211_get_station(struct net_device *dev, const u8 *mac_addr, memset(sinfo, 0, sizeof(*sinfo)); - return rdev_get_station(rdev, dev, mac_addr, sinfo); + wiphy_lock(&rdev->wiphy); + ret = rdev_get_station(rdev, dev, mac_addr, sinfo); + wiphy_unlock(&rdev->wiphy); + + return ret; } EXPORT_SYMBOL(cfg80211_get_station); diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index 727aa20be4bde8dc63a544a44a5cdeb19cac7dcb..7d1c0986f9bb354aa5a562f3edc72e54837b8ac5 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -313,13 +313,10 @@ static bool xsk_is_bound(struct xdp_sock *xs) static int xsk_rcv_check(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len) { - struct net_device *dev = xdp->rxq->dev; - u32 qid = xdp->rxq->queue_index; - if (!xsk_is_bound(xs)) return -ENXIO; - if (!dev->_rx[qid].pool || xs->umem != dev->_rx[qid].pool->umem) + if (xs->dev != xdp->rxq->dev || xs->queue_id != xdp->rxq->queue_index) return -EINVAL; if (len > xsk_pool_get_rx_frame_size(xs->pool) && !xs->sg) { diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c index 475b904fe68b8fa0c4e06f265309a52332a582e8..66e07de2de35cd1b5d3b4b5771e152dde6660b0d 100644 --- a/net/xfrm/xfrm_policy.c +++ b/net/xfrm/xfrm_policy.c @@ -3910,15 +3910,10 @@ static void xfrm_link_failure(struct sk_buff *skb) /* Impossible. Such dst must be popped before reaches point of failure. */ } -static struct dst_entry *xfrm_negative_advice(struct dst_entry *dst) +static void xfrm_negative_advice(struct sock *sk, struct dst_entry *dst) { - if (dst) { - if (dst->obsolete) { - dst_release(dst); - dst = NULL; - } - } - return dst; + if (dst->obsolete) + sk_dst_reset(sk); } static void xfrm_init_pmtu(struct xfrm_dst **bundle, int nr) diff --git a/samples/bpf/cpustat_kern.c b/samples/bpf/cpustat_kern.c index 944f13fe164a83c2b2d33bca2266cdb2f8e2f009..7ec7143e27579427ce587a291a9bbd3b6721b6bb 100644 --- a/samples/bpf/cpustat_kern.c +++ b/samples/bpf/cpustat_kern.c @@ -211,7 +211,7 @@ int bpf_prog1(struct cpu_args *ctx) SEC("tracepoint/power/cpu_frequency") int bpf_prog2(struct cpu_args *ctx) { - u64 *pts, *cstate, *pstate, prev_state, cur_ts, delta; + u64 *pts, *cstate, *pstate, cur_ts, delta; u32 key, cpu, pstate_idx; u64 *val; @@ -232,7 +232,6 @@ int bpf_prog2(struct cpu_args *ctx) if (!cstate) return 0; - prev_state = *pstate; *pstate = ctx->state; if (!*pts) { diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf index 2d6e5ed9081e3aef98faae3e17cb88a65d9dd002..2597e3d4d6e0118403e4f30f5329ef1ed0a08829 100644 --- a/scripts/Makefile.btf +++ b/scripts/Makefile.btf @@ -14,14 +14,12 @@ pahole-flags-$(call test-ge, $(pahole-ver), 121) += --btf_gen_floats pahole-flags-$(call test-ge, $(pahole-ver), 122) += -j -ifeq ($(pahole-ver), 125) -pahole-flags-y += --skip_encoding_btf_inconsistent_proto --btf_gen_optimized -endif +pahole-flags-$(call test-ge, $(pahole-ver), 125) += --skip_encoding_btf_inconsistent_proto --btf_gen_optimized else # Switch to using --btf_features for v1.26 and later. -pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func +pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs endif diff --git a/scripts/dtc/Makefile b/scripts/dtc/Makefile index a186570725412f80f4d89df7ebd190ee5e8c01ab..b47f4daa4515dfa9ac5010a29968eeda7a177ca7 100644 --- a/scripts/dtc/Makefile +++ b/scripts/dtc/Makefile @@ -3,7 +3,7 @@ # *** Also keep .gitignore in sync when changing *** hostprogs-always-$(CONFIG_DTC) += dtc fdtoverlay -hostprogs-always-$(CHECK_DT_BINDING) += dtc +hostprogs-always-$(CHECK_DTBS) += dtc dtc-objs := dtc.o flattree.o fstree.o data.o livetree.o treesource.o \ srcpos.o checks.o util.o diff --git a/scripts/gdb/linux/Makefile b/scripts/gdb/linux/Makefile index d77ad9079d0f92e2bb8065717e928fc40db5431a..fd1402c0a1a18fb7176260d08989413411252c04 100644 --- a/scripts/gdb/linux/Makefile +++ b/scripts/gdb/linux/Makefile @@ -5,7 +5,7 @@ ifdef building_out_of_srctree symlinks := $(patsubst $(src)/%,%,$(wildcard $(src)/*.py)) quiet_cmd_symlink = SYMLINK $@ - cmd_symlink = ln -fsn $(patsubst $(obj)/%,$(abspath $(srctree))/$(src)/%,$@) $@ + cmd_symlink = ln -fsn $(patsubst $(obj)/%,$(src)/%,$@) $@ always-y += $(symlinks) $(addprefix $(obj)/, $(symlinks)): FORCE diff --git a/scripts/kconfig/expr.c b/scripts/kconfig/expr.c index a290de36307ba8abe184a915fb0a6b6a3b29bbb6..4d95fce5f9a7ab7c421c2829c59305738e66a010 100644 --- a/scripts/kconfig/expr.c +++ b/scripts/kconfig/expr.c @@ -476,7 +476,7 @@ static struct expr *expr_join_or(struct expr *e1, struct expr *e2) return expr_alloc_comp(E_UNEQUAL, sym1, &symbol_yes); } } - if (sym1->type == S_BOOLEAN && sym1 == sym2) { + if (sym1->type == S_BOOLEAN) { if ((e1->type == E_NOT && e1->left.expr->type == E_SYMBOL && e2->type == E_SYMBOL) || (e2->type == E_NOT && e2->left.expr->type == E_SYMBOL && e1->type == E_SYMBOL)) return expr_alloc_symbol(&symbol_yes); diff --git a/scripts/kconfig/expr.h b/scripts/kconfig/expr.h index d965e427753eb7f7d94c40600603ddb21bef19e7..fa50fc45622e38e717901e71c54532432c7f1c0c 100644 --- a/scripts/kconfig/expr.h +++ b/scripts/kconfig/expr.h @@ -302,11 +302,6 @@ static inline int expr_is_yes(struct expr *e) return !e || (e->type == E_SYMBOL && e->left.sym == &symbol_yes); } -static inline int expr_is_no(struct expr *e) -{ - return e && (e->type == E_SYMBOL && e->left.sym == &symbol_no); -} - #ifdef __cplusplus } #endif diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c index aa0e25ee5119ef2d2f993f309980fc27f1e3e00b..0e439d3d48d11b63ed4175923c360edfb7de6f56 100644 --- a/scripts/kconfig/symbol.c +++ b/scripts/kconfig/symbol.c @@ -14,6 +14,7 @@ struct symbol symbol_yes = { .name = "y", + .type = S_TRISTATE, .curr = { "y", yes }, .menus = LIST_HEAD_INIT(symbol_yes.menus), .flags = SYMBOL_CONST|SYMBOL_VALID, @@ -21,6 +22,7 @@ struct symbol symbol_yes = { struct symbol symbol_mod = { .name = "m", + .type = S_TRISTATE, .curr = { "m", mod }, .menus = LIST_HEAD_INIT(symbol_mod.menus), .flags = SYMBOL_CONST|SYMBOL_VALID, @@ -28,6 +30,7 @@ struct symbol symbol_mod = { struct symbol symbol_no = { .name = "n", + .type = S_TRISTATE, .curr = { "n", no }, .menus = LIST_HEAD_INIT(symbol_no.menus), .flags = SYMBOL_CONST|SYMBOL_VALID, @@ -820,8 +823,7 @@ const char *sym_get_string_value(struct symbol *sym) case no: return "n"; case mod: - sym_calc_value(modules_sym); - return (modules_sym->curr.tri == no) ? "n" : "m"; + return "m"; case yes: return "y"; } diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 7862a81017477daec1f702fdf46166e381bd396d..46ce5d04dbeb100ddbe524baa36ccd63415e8738 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -179,10 +179,10 @@ kallsyms_step() kallsyms_S=${kallsyms_vmlinux}.S vmlinux_link ${kallsyms_vmlinux} "${kallsymso_prev}" ${btf_vmlinux_bin_o} - mksysmap ${kallsyms_vmlinux} ${kallsyms_vmlinux}.syms ${kallsymso_prev} + mksysmap ${kallsyms_vmlinux} ${kallsyms_vmlinux}.syms kallsyms ${kallsyms_vmlinux}.syms ${kallsyms_S} - info AS ${kallsyms_S} + info AS ${kallsymso} ${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \ ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \ -c -o ${kallsymso} ${kallsyms_S} @@ -193,7 +193,7 @@ kallsyms_step() mksysmap() { info NM ${2} - ${CONFIG_SHELL} "${srctree}/scripts/mksysmap" ${1} ${2} ${3} + ${NM} -n "${1}" | "${srctree}/scripts/mksysmap" > "${2}" } sorttable() @@ -201,7 +201,6 @@ sorttable() ${objtree}/scripts/sorttable ${1} } -# Delete output files in case of error cleanup() { rm -f .btf.* @@ -282,7 +281,7 @@ if is_enabled CONFIG_DEBUG_INFO_BTF && is_enabled CONFIG_BPF; then ${RESOLVE_BTFIDS} vmlinux fi -mksysmap vmlinux System.map ${kallsymso} +mksysmap vmlinux System.map if is_enabled CONFIG_BUILDTIME_TABLE_SORT; then info SORTTAB vmlinux diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 3de90c5a094b7aacd8e5d18449374c2691113717..263147df80a445aaed75101fcb8179662f0de380 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -190,7 +190,7 @@ def output_dtb(fsw, seq, fname, arch, compress): Args: fsw (libfdt.FdtSw): Object to use for writing seq (int): Sequence number (1 for first) - fmame (str): Filename containing the DTB + fname (str): Filename containing the DTB arch: FIT architecture, e.g. 'arm64' compress (str): Compressed algorithm, e.g. 'gzip' @@ -211,7 +211,6 @@ def output_dtb(fsw, seq, fname, arch, compress): fsw.property_string('type', 'flat_dt') fsw.property_string('arch', arch) fsw.property_string('compression', compress) - fsw.property('compatible', bytes(compat)) with open(fname, 'rb') as inf: compressed = compress_data(inf, compress) diff --git a/scripts/mksysmap b/scripts/mksysmap index 57ff5656d566fbc659801bdb4fd445b7b9ef2b86..c12723a0465562c782bfdaafc3ce4800671f959b 100755 --- a/scripts/mksysmap +++ b/scripts/mksysmap @@ -1,22 +1,16 @@ -#!/bin/sh -x -# Based on the vmlinux file create the System.map file +#!/bin/sed -f +# SPDX-License-Identifier: GPL-2.0-only +# +# sed script to filter out symbols that are not needed for System.map, +# or not suitable for kallsyms. The input should be 'nm -n '. +# # System.map is used by module-init tools and some debugging # tools to retrieve the actual addresses of symbols in the kernel. # -# Usage -# mksysmap vmlinux System.map [exclude] - - -##### -# Generate System.map (actual filename passed as second argument) -# The following refers to the symbol type as per nm(1). - # readprofile starts reading symbols when _stext is found, and # continue until it finds a symbol which is not either of 'T', 't', # 'W' or 'w'. # - -${NM} -n ${1} | sed >${2} -e " # --------------------------------------------------------------------------- # Ignored symbol types # @@ -92,13 +86,3 @@ ${NM} -n ${1} | sed >${2} -e " # ppc stub /\.long_branch\./d /\.plt_branch\./d - -# --------------------------------------------------------------------------- -# Ignored kallsyms symbols -# -# If the 3rd parameter exists, symbols from it will be omitted from the output. -# This makes kallsyms have the identical symbol lists in the step 1 and 2. -# Without this, the step2 would get new symbols generated by scripts/kallsyms.c -# when CONFIG_KALLSYMS_ALL is enabled. That might require one more pass. -$(if [ $# -ge 3 ]; then ${NM} ${3} | sed -n '/ U /!s:.* \([^ ]*\)$:/ \1$/d:p'; fi) -" diff --git a/security/landlock/fs.c b/security/landlock/fs.c index 22d8b7c28074e056738a6cfb5307dc80c9044c55..7877a64cc6b87c0930e679ab07ee469c5bec22e5 100644 --- a/security/landlock/fs.c +++ b/security/landlock/fs.c @@ -1110,6 +1110,7 @@ static int current_check_refer_path(struct dentry *const old_dentry, bool allow_parent1, allow_parent2; access_mask_t access_request_parent1, access_request_parent2; struct path mnt_dir; + struct dentry *old_parent; layer_mask_t layer_masks_parent1[LANDLOCK_NUM_ACCESS_FS] = {}, layer_masks_parent2[LANDLOCK_NUM_ACCESS_FS] = {}; @@ -1157,9 +1158,17 @@ static int current_check_refer_path(struct dentry *const old_dentry, mnt_dir.mnt = new_dir->mnt; mnt_dir.dentry = new_dir->mnt->mnt_root; + /* + * old_dentry may be the root of the common mount point and + * !IS_ROOT(old_dentry) at the same time (e.g. with open_tree() and + * OPEN_TREE_CLONE). We do not need to call dget(old_parent) because + * we keep a reference to old_dentry. + */ + old_parent = (old_dentry == mnt_dir.dentry) ? old_dentry : + old_dentry->d_parent; + /* new_dir->dentry is equal to new_dentry->d_parent */ - allow_parent1 = collect_domain_accesses(dom, mnt_dir.dentry, - old_dentry->d_parent, + allow_parent1 = collect_domain_accesses(dom, mnt_dir.dentry, old_parent, &layer_masks_parent1); allow_parent2 = collect_domain_accesses( dom, mnt_dir.dentry, new_dir->dentry, &layer_masks_parent2); diff --git a/security/tomoyo/Kconfig b/security/tomoyo/Kconfig index fad75be5f381d6e0d16466fa8b0b47fcfd092a23..1e0dd1a6d0b0e9aa01b6fe4cadb43b8c6b7de058 100644 --- a/security/tomoyo/Kconfig +++ b/security/tomoyo/Kconfig @@ -10,7 +10,7 @@ config SECURITY_TOMOYO help This selects TOMOYO Linux, pathname-based access control. Required userspace tools and further information may be - found at . + found at . If you are unsure how to answer this question, answer N. config SECURITY_TOMOYO_MAX_ACCEPT_ENTRY diff --git a/security/tomoyo/common.c b/security/tomoyo/common.c index ea3140d510ecbfee06666df588a795b9f5bfc5ce..5c7b059a332aac494a95b1f99415dd2eef6a2900 100644 --- a/security/tomoyo/common.c +++ b/security/tomoyo/common.c @@ -2787,7 +2787,7 @@ void tomoyo_check_profile(void) else continue; pr_err("Userland tools for TOMOYO 2.6 must be installed and policy must be initialized.\n"); - pr_err("Please see https://tomoyo.osdn.jp/2.6/ for more information.\n"); + pr_err("Please see https://tomoyo.sourceforge.net/2.6/ for more information.\n"); panic("STOP!"); } tomoyo_read_unlock(idx); diff --git a/sound/core/init.c b/sound/core/init.c index 4e52bbe32786ba9945f03fd921b9d177692b8cda..b9b708cf980d6dbf9d80992b7af86e761a7bdf71 100644 --- a/sound/core/init.c +++ b/sound/core/init.c @@ -537,6 +537,11 @@ void snd_card_disconnect(struct snd_card *card) synchronize_irq(card->sync_irq); snd_info_card_disconnect(card); +#ifdef CONFIG_SND_DEBUG + debugfs_remove(card->debugfs_root); + card->debugfs_root = NULL; +#endif + if (card->registered) { device_del(&card->card_dev); card->registered = false; @@ -586,10 +591,6 @@ static int snd_card_do_free(struct snd_card *card) dev_warn(card->dev, "unable to free card info\n"); /* Not fatal error */ } -#ifdef CONFIG_SND_DEBUG - debugfs_remove(card->debugfs_root); - card->debugfs_root = NULL; -#endif if (card->release_completion) complete(card->release_completion); if (!card->managed) diff --git a/sound/core/jack.c b/sound/core/jack.c index e08b2c4fbd1a577380d31781d28b222b404c5c2c..e4bcecdf89b7ec0bccb0d113784c32ecf2620a68 100644 --- a/sound/core/jack.c +++ b/sound/core/jack.c @@ -37,11 +37,15 @@ static const int jack_switch_types[SND_JACK_SWITCH_TYPES] = { }; #endif /* CONFIG_SND_JACK_INPUT_DEV */ +static void snd_jack_remove_debugfs(struct snd_jack *jack); + static int snd_jack_dev_disconnect(struct snd_device *device) { -#ifdef CONFIG_SND_JACK_INPUT_DEV struct snd_jack *jack = device->device_data; + snd_jack_remove_debugfs(jack); + +#ifdef CONFIG_SND_JACK_INPUT_DEV guard(mutex)(&jack->input_dev_lock); if (!jack->input_dev) return 0; @@ -381,10 +385,14 @@ static int snd_jack_debugfs_add_inject_node(struct snd_jack *jack, return 0; } -static void snd_jack_debugfs_clear_inject_node(struct snd_jack_kctl *jack_kctl) +static void snd_jack_remove_debugfs(struct snd_jack *jack) { - debugfs_remove(jack_kctl->jack_debugfs_root); - jack_kctl->jack_debugfs_root = NULL; + struct snd_jack_kctl *jack_kctl; + + list_for_each_entry(jack_kctl, &jack->kctl_list, list) { + debugfs_remove(jack_kctl->jack_debugfs_root); + jack_kctl->jack_debugfs_root = NULL; + } } #else /* CONFIG_SND_JACK_INJECTION_DEBUG */ static int snd_jack_debugfs_add_inject_node(struct snd_jack *jack, @@ -393,7 +401,7 @@ static int snd_jack_debugfs_add_inject_node(struct snd_jack *jack, return 0; } -static void snd_jack_debugfs_clear_inject_node(struct snd_jack_kctl *jack_kctl) +static void snd_jack_remove_debugfs(struct snd_jack *jack) { } #endif /* CONFIG_SND_JACK_INJECTION_DEBUG */ @@ -404,7 +412,6 @@ static void snd_jack_kctl_private_free(struct snd_kcontrol *kctl) jack_kctl = kctl->private_data; if (jack_kctl) { - snd_jack_debugfs_clear_inject_node(jack_kctl); list_del(&jack_kctl->list); kfree(jack_kctl); } @@ -497,8 +504,8 @@ int snd_jack_new(struct snd_card *card, const char *id, int type, .dev_free = snd_jack_dev_free, #ifdef CONFIG_SND_JACK_INPUT_DEV .dev_register = snd_jack_dev_register, - .dev_disconnect = snd_jack_dev_disconnect, #endif /* CONFIG_SND_JACK_INPUT_DEV */ + .dev_disconnect = snd_jack_dev_disconnect, }; if (initial_kctl) { diff --git a/sound/core/seq/seq_ump_convert.c b/sound/core/seq/seq_ump_convert.c index ee6ac649df836d695133f93da9cfd0518cc1ae23..171fb75267afae6f799d21350233256211932a6d 100644 --- a/sound/core/seq/seq_ump_convert.c +++ b/sound/core/seq/seq_ump_convert.c @@ -157,7 +157,7 @@ static void ump_system_to_one_param_ev(const union snd_ump_midi1_msg *val, static void ump_system_to_songpos_ev(const union snd_ump_midi1_msg *val, struct snd_seq_event *ev) { - ev->data.control.value = (val->system.parm1 << 7) | val->system.parm2; + ev->data.control.value = (val->system.parm2 << 7) | val->system.parm1; } /* Encoders for 0xf0 - 0xff */ @@ -368,6 +368,7 @@ static int cvt_ump_midi1_to_midi2(struct snd_seq_client *dest, struct snd_seq_ump_event ev_cvt; const union snd_ump_midi1_msg *midi1 = (const union snd_ump_midi1_msg *)event->ump; union snd_ump_midi2_msg *midi2 = (union snd_ump_midi2_msg *)ev_cvt.ump; + struct snd_seq_ump_midi2_bank *cc; ev_cvt = *event; memset(&ev_cvt.ump, 0, sizeof(ev_cvt.ump)); @@ -387,11 +388,29 @@ static int cvt_ump_midi1_to_midi2(struct snd_seq_client *dest, midi2->paf.data = upscale_7_to_32bit(midi1->paf.data); break; case UMP_MSG_STATUS_CC: + cc = &dest_port->midi2_bank[midi1->note.channel]; + switch (midi1->cc.index) { + case UMP_CC_BANK_SELECT: + cc->bank_set = 1; + cc->cc_bank_msb = midi1->cc.data; + return 0; // skip + case UMP_CC_BANK_SELECT_LSB: + cc->bank_set = 1; + cc->cc_bank_lsb = midi1->cc.data; + return 0; // skip + } midi2->cc.index = midi1->cc.index; midi2->cc.data = upscale_7_to_32bit(midi1->cc.data); break; case UMP_MSG_STATUS_PROGRAM: midi2->pg.program = midi1->pg.program; + cc = &dest_port->midi2_bank[midi1->note.channel]; + if (cc->bank_set) { + midi2->pg.bank_valid = 1; + midi2->pg.bank_msb = cc->cc_bank_msb; + midi2->pg.bank_lsb = cc->cc_bank_lsb; + cc->bank_set = 0; + } break; case UMP_MSG_STATUS_CHANNEL_PRESSURE: midi2->caf.data = upscale_7_to_32bit(midi1->caf.data); @@ -419,6 +438,7 @@ static int cvt_ump_midi2_to_midi1(struct snd_seq_client *dest, struct snd_seq_ump_event ev_cvt; union snd_ump_midi1_msg *midi1 = (union snd_ump_midi1_msg *)ev_cvt.ump; const union snd_ump_midi2_msg *midi2 = (const union snd_ump_midi2_msg *)event->ump; + int err; u16 v; ev_cvt = *event; @@ -443,6 +463,24 @@ static int cvt_ump_midi2_to_midi1(struct snd_seq_client *dest, midi1->cc.data = downscale_32_to_7bit(midi2->cc.data); break; case UMP_MSG_STATUS_PROGRAM: + if (midi2->pg.bank_valid) { + midi1->cc.status = UMP_MSG_STATUS_CC; + midi1->cc.index = UMP_CC_BANK_SELECT; + midi1->cc.data = midi2->pg.bank_msb; + err = __snd_seq_deliver_single_event(dest, dest_port, + (struct snd_seq_event *)&ev_cvt, + atomic, hop); + if (err < 0) + return err; + midi1->cc.index = UMP_CC_BANK_SELECT_LSB; + midi1->cc.data = midi2->pg.bank_lsb; + err = __snd_seq_deliver_single_event(dest, dest_port, + (struct snd_seq_event *)&ev_cvt, + atomic, hop); + if (err < 0) + return err; + midi1->note.status = midi2->note.status; + } midi1->pg.program = midi2->pg.program; break; case UMP_MSG_STATUS_CHANNEL_PRESSURE: @@ -691,6 +729,7 @@ static int system_ev_to_ump_midi1(const struct snd_seq_event *event, union snd_ump_midi1_msg *data, unsigned char status) { + data->system.type = UMP_MSG_TYPE_SYSTEM; // override data->system.status = status; return 1; } @@ -701,6 +740,7 @@ static int system_1p_ev_to_ump_midi1(const struct snd_seq_event *event, union snd_ump_midi1_msg *data, unsigned char status) { + data->system.type = UMP_MSG_TYPE_SYSTEM; // override data->system.status = status; data->system.parm1 = event->data.control.value & 0x7f; return 1; @@ -712,9 +752,10 @@ static int system_2p_ev_to_ump_midi1(const struct snd_seq_event *event, union snd_ump_midi1_msg *data, unsigned char status) { + data->system.type = UMP_MSG_TYPE_SYSTEM; // override data->system.status = status; - data->system.parm1 = (event->data.control.value >> 7) & 0x7f; - data->system.parm2 = event->data.control.value & 0x7f; + data->system.parm1 = event->data.control.value & 0x7f; + data->system.parm2 = (event->data.control.value >> 7) & 0x7f; return 1; } @@ -854,7 +895,6 @@ static int pgm_ev_to_ump_midi2(const struct snd_seq_event *event, data->pg.bank_msb = cc->cc_bank_msb; data->pg.bank_lsb = cc->cc_bank_lsb; cc->bank_set = 0; - cc->cc_bank_msb = cc->cc_bank_lsb = 0; } return 1; } diff --git a/sound/core/ump.c b/sound/core/ump.c index fd6a68a5427886d11e443132a74568d933f2f163..3f61220c23b4ec028faddd63bff776a91cc8067c 100644 --- a/sound/core/ump.c +++ b/sound/core/ump.c @@ -685,10 +685,17 @@ static void seq_notify_protocol(struct snd_ump_endpoint *ump) */ int snd_ump_switch_protocol(struct snd_ump_endpoint *ump, unsigned int protocol) { + unsigned int type; + protocol &= ump->info.protocol_caps; if (protocol == ump->info.protocol) return 0; + type = protocol & SNDRV_UMP_EP_INFO_PROTO_MIDI_MASK; + if (type != SNDRV_UMP_EP_INFO_PROTO_MIDI1 && + type != SNDRV_UMP_EP_INFO_PROTO_MIDI2) + return 0; + ump->info.protocol = protocol; ump_dbg(ump, "New protocol = %x (caps = %x)\n", protocol, ump->info.protocol_caps); @@ -960,6 +967,14 @@ int snd_ump_parse_endpoint(struct snd_ump_endpoint *ump) if (err < 0) ump_dbg(ump, "Unable to get UMP EP stream config\n"); + /* If no protocol is set by some reason, assume the valid one */ + if (!(ump->info.protocol & SNDRV_UMP_EP_INFO_PROTO_MIDI_MASK)) { + if (ump->info.protocol_caps & SNDRV_UMP_EP_INFO_PROTO_MIDI2) + ump->info.protocol |= SNDRV_UMP_EP_INFO_PROTO_MIDI2; + else if (ump->info.protocol_caps & SNDRV_UMP_EP_INFO_PROTO_MIDI1) + ump->info.protocol |= SNDRV_UMP_EP_INFO_PROTO_MIDI1; + } + /* Query and create blocks from Function Blocks */ for (blk = 0; blk < ump->info.num_blocks; blk++) { err = create_block_from_fb_info(ump, blk); diff --git a/sound/core/ump_convert.c b/sound/core/ump_convert.c index de04799fdb69aaf3b08de639739e52f09ebb55a3..f67c44c83fde445258c59f34897b73a913c17cf1 100644 --- a/sound/core/ump_convert.c +++ b/sound/core/ump_convert.c @@ -404,7 +404,6 @@ static int cvt_legacy_cmd_to_ump(struct ump_cvt_to_ump *cvt, midi2->pg.bank_msb = cc->cc_bank_msb; midi2->pg.bank_lsb = cc->cc_bank_lsb; cc->bank_set = 0; - cc->cc_bank_msb = cc->cc_bank_lsb = 0; } break; case UMP_MSG_STATUS_CHANNEL_PRESSURE: diff --git a/sound/hda/intel-dsp-config.c b/sound/hda/intel-dsp-config.c index cfdb1b73c88c2011a2f977af4d53138cd0f9912e..537863447358e078d1d605d29f548352b8c80934 100644 --- a/sound/hda/intel-dsp-config.c +++ b/sound/hda/intel-dsp-config.c @@ -668,7 +668,7 @@ int snd_intel_dsp_driver_probe(struct pci_dev *pci) return SND_INTEL_DSP_DRIVER_LEGACY; } - dev_info(&pci->dev, "DSP detected with PCI class/subclass/prog-if info 0x%06x\n", pci->class); + dev_dbg(&pci->dev, "DSP detected with PCI class/subclass/prog-if info 0x%06x\n", pci->class); /* find the configuration for the specific device */ cfg = snd_intel_dsp_find_config(pci, config_table, ARRAY_SIZE(config_table)); @@ -678,12 +678,12 @@ int snd_intel_dsp_driver_probe(struct pci_dev *pci) if (cfg->flags & FLAG_SOF) { if (cfg->flags & FLAG_SOF_ONLY_IF_SOUNDWIRE && snd_intel_dsp_check_soundwire(pci) > 0) { - dev_info(&pci->dev, "SoundWire enabled on CannonLake+ platform, using SOF driver\n"); + dev_info_once(&pci->dev, "SoundWire enabled on CannonLake+ platform, using SOF driver\n"); return SND_INTEL_DSP_DRIVER_SOF; } if (cfg->flags & FLAG_SOF_ONLY_IF_DMIC && snd_intel_dsp_check_dmic(pci)) { - dev_info(&pci->dev, "Digital mics found on Skylake+ platform, using SOF driver\n"); + dev_info_once(&pci->dev, "Digital mics found on Skylake+ platform, using SOF driver\n"); return SND_INTEL_DSP_DRIVER_SOF; } if (!(cfg->flags & FLAG_SOF_ONLY_IF_DMIC_OR_SOUNDWIRE)) @@ -694,7 +694,7 @@ int snd_intel_dsp_driver_probe(struct pci_dev *pci) if (cfg->flags & FLAG_SST) { if (cfg->flags & FLAG_SST_ONLY_IF_DMIC) { if (snd_intel_dsp_check_dmic(pci)) { - dev_info(&pci->dev, "Digital mics found on Skylake+ platform, using SST driver\n"); + dev_info_once(&pci->dev, "Digital mics found on Skylake+ platform, using SST driver\n"); return SND_INTEL_DSP_DRIVER_SST; } } else { diff --git a/sound/pci/hda/patch_realtek.c b/sound/pci/hda/patch_realtek.c index e3c0b9d5552d957869de671704b4b567dd4854a6..aa76d1c885895a22e2745ab77ef6b11f5414c513 100644 --- a/sound/pci/hda/patch_realtek.c +++ b/sound/pci/hda/patch_realtek.c @@ -10310,7 +10310,7 @@ static const struct snd_pci_quirk alc269_fixup_tbl[] = { SND_PCI_QUIRK(0x1043, 0x3030, "ASUS ZN270IE", ALC256_FIXUP_ASUS_AIO_GPIO2), SND_PCI_QUIRK(0x1043, 0x3a20, "ASUS G614JZR", ALC245_FIXUP_CS35L41_SPI_2), SND_PCI_QUIRK(0x1043, 0x3a30, "ASUS G814JVR/JIR", ALC245_FIXUP_CS35L41_SPI_2), - SND_PCI_QUIRK(0x1043, 0x3a40, "ASUS G814JZR", ALC245_FIXUP_CS35L41_SPI_2), + SND_PCI_QUIRK(0x1043, 0x3a40, "ASUS G814JZR", ALC285_FIXUP_ASUS_SPI_REAR_SPEAKERS), SND_PCI_QUIRK(0x1043, 0x3a50, "ASUS G834JYR/JZR", ALC245_FIXUP_CS35L41_SPI_2), SND_PCI_QUIRK(0x1043, 0x3a60, "ASUS G634JYR/JZR", ALC285_FIXUP_ASUS_SPI_REAR_SPEAKERS), SND_PCI_QUIRK(0x1043, 0x831a, "ASUS P901", ALC269_FIXUP_STEREO_DMIC), diff --git a/sound/soc/codecs/cs42l43.c b/sound/soc/codecs/cs42l43.c index 94685449f0f48c9b7bd534b1947da07ab26fad53..92674314227c4623af36a6b1642e83c8a33572d6 100644 --- a/sound/soc/codecs/cs42l43.c +++ b/sound/soc/codecs/cs42l43.c @@ -310,8 +310,9 @@ static int cs42l43_startup(struct snd_pcm_substream *substream, struct snd_soc_d struct snd_soc_component *component = dai->component; struct cs42l43_codec *priv = snd_soc_component_get_drvdata(component); struct cs42l43 *cs42l43 = priv->core; - int provider = !!regmap_test_bits(cs42l43->regmap, CS42L43_ASP_CLK_CONFIG2, - CS42L43_ASP_MASTER_MODE_MASK); + int provider = !dai->id || !!regmap_test_bits(cs42l43->regmap, + CS42L43_ASP_CLK_CONFIG2, + CS42L43_ASP_MASTER_MODE_MASK); if (provider) priv->constraint.mask = CS42L43_PROVIDER_RATE_MASK; diff --git a/sound/soc/codecs/wm_adsp.c b/sound/soc/codecs/wm_adsp.c index c9d9a7b28efb08d28f206d585ae92b8ceb1be508..68d2d6444533f5d2f602dbe81fbf54b00979c4c2 100644 --- a/sound/soc/codecs/wm_adsp.c +++ b/sound/soc/codecs/wm_adsp.c @@ -2085,5 +2085,6 @@ static const struct cs_dsp_client_ops wm_adsp2_client_ops = { .watchdog_expired = wm_adsp_fatal_error, }; +MODULE_DESCRIPTION("Cirrus Logic ASoC DSP Support"); MODULE_LICENSE("GPL v2"); MODULE_IMPORT_NS(FW_CS_DSP); diff --git a/sound/soc/intel/boards/Kconfig b/sound/soc/intel/boards/Kconfig index 3ed81ab649c53b20818155bed4e2c9e59216f07a..4e0586034de4b2f0700324615844655ef42e6e43 100644 --- a/sound/soc/intel/boards/Kconfig +++ b/sound/soc/intel/boards/Kconfig @@ -652,7 +652,7 @@ if SND_SOC_SOF_INTEL_SOUNDWIRE config SND_SOC_INTEL_SOUNDWIRE_SOF_MACH tristate "SoundWire generic machine driver" - depends on I2C && ACPI + depends on I2C && SPI_MASTER && ACPI depends on MFD_INTEL_LPSS || COMPILE_TEST depends on SND_SOC_INTEL_USER_FRIENDLY_LONG_NAMES || COMPILE_TEST depends on SOUNDWIRE diff --git a/sound/soc/sof/amd/acp-common.c b/sound/soc/sof/amd/acp-common.c index b26fa471b431fff7c73934cede3e5a9c16eb58b2..81bb93e9835864267e1e8464806b6cdb8c02911f 100644 --- a/sound/soc/sof/amd/acp-common.c +++ b/sound/soc/sof/amd/acp-common.c @@ -258,8 +258,8 @@ const struct snd_sof_dsp_ops sof_acp_common_ops = { }; EXPORT_SYMBOL_NS(sof_acp_common_ops, SND_SOC_SOF_AMD_COMMON); +MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("ACP SOF COMMON Driver"); MODULE_IMPORT_NS(SND_SOC_SOF_AMD_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); MODULE_IMPORT_NS(SOUNDWIRE_AMD_INIT); -MODULE_DESCRIPTION("ACP SOF COMMON Driver"); -MODULE_LICENSE("Dual BSD/GPL"); diff --git a/sound/soc/sof/amd/acp.c b/sound/soc/sof/amd/acp.c index c12c7f820529476de0273474082b8174ab0ae052..74fd5f2b148b8545c1229731239aedf87af3e0a0 100644 --- a/sound/soc/sof/amd/acp.c +++ b/sound/soc/sof/amd/acp.c @@ -801,7 +801,7 @@ void amd_sof_acp_remove(struct snd_sof_dev *sdev) } EXPORT_SYMBOL_NS(amd_sof_acp_remove, SND_SOC_SOF_AMD_COMMON); +MODULE_LICENSE("Dual BSD/GPL"); MODULE_DESCRIPTION("AMD ACP sof driver"); MODULE_IMPORT_NS(SOUNDWIRE_AMD_INIT); MODULE_IMPORT_NS(SND_AMD_SOUNDWIRE_ACPI); -MODULE_LICENSE("Dual BSD/GPL"); diff --git a/sound/soc/sof/amd/acp63.c b/sound/soc/sof/amd/acp63.c index 9fb645079c3ad3e97aa006994a4177864295f640..9e6eb4bfc805b21e9b0607d8ea26843307338d57 100644 --- a/sound/soc/sof/amd/acp63.c +++ b/sound/soc/sof/amd/acp63.c @@ -140,7 +140,3 @@ int sof_acp63_ops_init(struct snd_sof_dev *sdev) return 0; } - -MODULE_IMPORT_NS(SND_SOC_SOF_AMD_COMMON); -MODULE_DESCRIPTION("ACP63 SOF Driver"); -MODULE_LICENSE("Dual BSD/GPL"); diff --git a/sound/soc/sof/amd/pci-acp63.c b/sound/soc/sof/amd/pci-acp63.c index eeaa12cceb23311312d5104ae4452c0e745fb22a..fc8984447365718410b4320e8cd05fbf3802d804 100644 --- a/sound/soc/sof/amd/pci-acp63.c +++ b/sound/soc/sof/amd/pci-acp63.c @@ -109,5 +109,6 @@ static struct pci_driver snd_sof_pci_amd_acp63_driver = { module_pci_driver(snd_sof_pci_amd_acp63_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("ACP63 SOF Driver"); MODULE_IMPORT_NS(SND_SOC_SOF_AMD_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_PCI_DEV); diff --git a/sound/soc/sof/amd/pci-rmb.c b/sound/soc/sof/amd/pci-rmb.c index 2f288545c4260525a09b62bc01a1bb733d178ef3..4bc30951f8b0d711fc8baa6afa285a16a6ad330c 100644 --- a/sound/soc/sof/amd/pci-rmb.c +++ b/sound/soc/sof/amd/pci-rmb.c @@ -99,5 +99,6 @@ static struct pci_driver snd_sof_pci_amd_rmb_driver = { module_pci_driver(snd_sof_pci_amd_rmb_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("REMBRANDT SOF Driver"); MODULE_IMPORT_NS(SND_SOC_SOF_AMD_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_PCI_DEV); diff --git a/sound/soc/sof/amd/pci-rn.c b/sound/soc/sof/amd/pci-rn.c index a0195e9b400c8989748e1896f4b889fc509372ee..e08875bdfa8b16f3fcd1512223b5c351a735db21 100644 --- a/sound/soc/sof/amd/pci-rn.c +++ b/sound/soc/sof/amd/pci-rn.c @@ -103,5 +103,6 @@ static struct pci_driver snd_sof_pci_amd_rn_driver = { module_pci_driver(snd_sof_pci_amd_rn_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("RENOIR SOF Driver"); MODULE_IMPORT_NS(SND_SOC_SOF_AMD_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_PCI_DEV); diff --git a/sound/soc/sof/amd/pci-vangogh.c b/sound/soc/sof/amd/pci-vangogh.c index 5cd3ac84752fb83fa030c5f91614594c092ffa1c..16eb2994fbab905b8c1f4994f078be4f013611da 100644 --- a/sound/soc/sof/amd/pci-vangogh.c +++ b/sound/soc/sof/amd/pci-vangogh.c @@ -101,5 +101,6 @@ static struct pci_driver snd_sof_pci_amd_vgh_driver = { module_pci_driver(snd_sof_pci_amd_vgh_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("VANGOGH SOF Driver"); MODULE_IMPORT_NS(SND_SOC_SOF_AMD_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_PCI_DEV); diff --git a/sound/soc/sof/amd/rembrandt.c b/sound/soc/sof/amd/rembrandt.c index f1d1ba57ab3a0d716daa42ede437d5cf36518e70..076f2f05a95c2066055dffa26f5ad372cfe6b1c1 100644 --- a/sound/soc/sof/amd/rembrandt.c +++ b/sound/soc/sof/amd/rembrandt.c @@ -140,7 +140,3 @@ int sof_rembrandt_ops_init(struct snd_sof_dev *sdev) return 0; } - -MODULE_IMPORT_NS(SND_SOC_SOF_AMD_COMMON); -MODULE_DESCRIPTION("REMBRANDT SOF Driver"); -MODULE_LICENSE("Dual BSD/GPL"); diff --git a/sound/soc/sof/amd/renoir.c b/sound/soc/sof/amd/renoir.c index 47b863f6258c6d5d922411251155e5eb664151b5..aa2d24dac6f5d5f6a3c3367de868157c2bd49073 100644 --- a/sound/soc/sof/amd/renoir.c +++ b/sound/soc/sof/amd/renoir.c @@ -115,7 +115,3 @@ int sof_renoir_ops_init(struct snd_sof_dev *sdev) return 0; } - -MODULE_IMPORT_NS(SND_SOC_SOF_AMD_COMMON); -MODULE_DESCRIPTION("RENOIR SOF Driver"); -MODULE_LICENSE("Dual BSD/GPL"); diff --git a/sound/soc/sof/amd/vangogh.c b/sound/soc/sof/amd/vangogh.c index bc6ffdb5471a58ae4b30b27ba166fd356a974edd..61372958c09dc8c19343484e1155d986b067a17a 100644 --- a/sound/soc/sof/amd/vangogh.c +++ b/sound/soc/sof/amd/vangogh.c @@ -161,7 +161,3 @@ int sof_vangogh_ops_init(struct snd_sof_dev *sdev) return 0; } - -MODULE_IMPORT_NS(SND_SOC_SOF_AMD_COMMON); -MODULE_DESCRIPTION("VANGOGH SOF Driver"); -MODULE_LICENSE("Dual BSD/GPL"); diff --git a/sound/soc/sof/core.c b/sound/soc/sof/core.c index 0a4917136ff971593fc453acf62fc69ad33783b2..83fe0401baf867b071a7449d71d98c207506a15c 100644 --- a/sound/soc/sof/core.c +++ b/sound/soc/sof/core.c @@ -769,7 +769,7 @@ void sof_machine_unregister(struct snd_sof_dev *sdev, void *pdata) EXPORT_SYMBOL(sof_machine_unregister); MODULE_AUTHOR("Liam Girdwood"); -MODULE_DESCRIPTION("Sound Open Firmware (SOF) Core"); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("Sound Open Firmware (SOF) Core"); MODULE_ALIAS("platform:sof-audio"); MODULE_IMPORT_NS(SND_SOC_SOF_CLIENT); diff --git a/sound/soc/sof/imx/imx-common.c b/sound/soc/sof/imx/imx-common.c index 2981aea123d976cdc0dccc295511cfcb18fb7677..fce6d9cf6a6be839002e5aa45cc3107d57f92f16 100644 --- a/sound/soc/sof/imx/imx-common.c +++ b/sound/soc/sof/imx/imx-common.c @@ -75,3 +75,4 @@ void imx8_dump(struct snd_sof_dev *sdev, u32 flags) EXPORT_SYMBOL(imx8_dump); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF helpers for IMX platforms"); diff --git a/sound/soc/sof/imx/imx8.c b/sound/soc/sof/imx/imx8.c index 3021dc87ab5a23c8b811282e3f12d4e3a784c392..9f24e3c283dd41ef73d89319d623ebacb4c523cc 100644 --- a/sound/soc/sof/imx/imx8.c +++ b/sound/soc/sof/imx/imx8.c @@ -667,5 +667,6 @@ static struct platform_driver snd_sof_of_imx8_driver = { }; module_platform_driver(snd_sof_of_imx8_driver); -MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for IMX8 platforms"); +MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); diff --git a/sound/soc/sof/imx/imx8m.c b/sound/soc/sof/imx/imx8m.c index 4ed415f0434544c3d01a8119b1c0c020a06dbca8..1c7019c3cbd38ccaed5b5aca638a5d299b5ce824 100644 --- a/sound/soc/sof/imx/imx8m.c +++ b/sound/soc/sof/imx/imx8m.c @@ -514,5 +514,6 @@ static struct platform_driver snd_sof_of_imx8m_driver = { }; module_platform_driver(snd_sof_of_imx8m_driver); -MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for IMX8M platforms"); +MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); diff --git a/sound/soc/sof/imx/imx8ulp.c b/sound/soc/sof/imx/imx8ulp.c index 8adfdd00413aa15a2d063bb46199a58b3e0619fc..2585b1beef23f89f65a696ec65cebcb27b579e44 100644 --- a/sound/soc/sof/imx/imx8ulp.c +++ b/sound/soc/sof/imx/imx8ulp.c @@ -516,5 +516,6 @@ static struct platform_driver snd_sof_of_imx8ulp_driver = { }; module_platform_driver(snd_sof_of_imx8ulp_driver); -MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for IMX8ULP platforms"); +MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); diff --git a/sound/soc/sof/intel/atom.c b/sound/soc/sof/intel/atom.c index 86af4e9a716ef776f5e2ffa7c4242625a28a9c29..3505ac3a1b143fcd0586a2782224508afca47357 100644 --- a/sound/soc/sof/intel/atom.c +++ b/sound/soc/sof/intel/atom.c @@ -418,3 +418,4 @@ void atom_set_mach_params(struct snd_soc_acpi_mach *mach, EXPORT_SYMBOL_NS(atom_set_mach_params, SND_SOC_SOF_INTEL_ATOM_HIFI_EP); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for Atom platforms"); diff --git a/sound/soc/sof/intel/bdw.c b/sound/soc/sof/intel/bdw.c index 3262286a9a9d7422b518de2bd96cdf7b31189456..7f18080e4e191834295f35d11ed0c2e2e919d213 100644 --- a/sound/soc/sof/intel/bdw.c +++ b/sound/soc/sof/intel/bdw.c @@ -694,6 +694,7 @@ static struct platform_driver snd_sof_acpi_intel_bdw_driver = { module_platform_driver(snd_sof_acpi_intel_bdw_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for Broadwell platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HIFI_EP_IPC); MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); MODULE_IMPORT_NS(SND_SOC_SOF_ACPI_DEV); diff --git a/sound/soc/sof/intel/byt.c b/sound/soc/sof/intel/byt.c index d78d11d4cfbf6c7b31011f3d135a0e997434a74e..7a57e162fb1c2b323d5eba59180fe3e58db85ca6 100644 --- a/sound/soc/sof/intel/byt.c +++ b/sound/soc/sof/intel/byt.c @@ -475,6 +475,7 @@ static struct platform_driver snd_sof_acpi_intel_byt_driver = { module_platform_driver(snd_sof_acpi_intel_byt_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for Baytrail/Cherrytrail"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HIFI_EP_IPC); MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); MODULE_IMPORT_NS(SND_SOC_SOF_ACPI_DEV); diff --git a/sound/soc/sof/intel/hda-codec.c b/sound/soc/sof/intel/hda-codec.c index da3db3ed379ecb7f350bed63be8535c1eecd04f3..dc46888faa0dc9c3fd3a826615bdeb5b66057b56 100644 --- a/sound/soc/sof/intel/hda-codec.c +++ b/sound/soc/sof/intel/hda-codec.c @@ -457,3 +457,4 @@ EXPORT_SYMBOL_NS_GPL(hda_codec_i915_exit, SND_SOC_SOF_HDA_AUDIO_CODEC_I915); #endif MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for HDaudio codecs"); diff --git a/sound/soc/sof/intel/hda-ctrl.c b/sound/soc/sof/intel/hda-ctrl.c index 262b482dc0a800489eba28725c8b9048424fe9f7..b9a02750ce61311aff49a5674bf5f0cabe7a883e 100644 --- a/sound/soc/sof/intel/hda-ctrl.c +++ b/sound/soc/sof/intel/hda-ctrl.c @@ -328,6 +328,7 @@ void hda_dsp_ctrl_stop_chip(struct snd_sof_dev *sdev) } MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF helpers for HDaudio platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_HDA_MLINK); MODULE_IMPORT_NS(SND_SOC_SOF_HDA_AUDIO_CODEC); MODULE_IMPORT_NS(SND_SOC_SOF_HDA_AUDIO_CODEC_I915); diff --git a/sound/soc/sof/intel/hda-mlink.c b/sound/soc/sof/intel/hda-mlink.c index 04bbc5c9904ce63ab3ad5610ce62ee0ab2e8af17..9a3559c78b6279bd3dbfaf05cca3d5c64cff83f9 100644 --- a/sound/soc/sof/intel/hda-mlink.c +++ b/sound/soc/sof/intel/hda-mlink.c @@ -972,3 +972,4 @@ EXPORT_SYMBOL_NS(hdac_bus_eml_enable_offload, SND_SOC_SOF_HDA_MLINK); #endif MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for HDaudio multi-link"); diff --git a/sound/soc/sof/intel/hda.c b/sound/soc/sof/intel/hda.c index e6a38de0a0aa103a6bf1694820a91d036d07f7c9..dead1c19558bb02af35f23eae1f73c4426a093e4 100644 --- a/sound/soc/sof/intel/hda.c +++ b/sound/soc/sof/intel/hda.c @@ -1522,6 +1522,7 @@ void hda_unregister_clients(struct snd_sof_dev *sdev) } MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for HDaudio platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_PCI_DEV); MODULE_IMPORT_NS(SND_SOC_SOF_HDA_AUDIO_CODEC); MODULE_IMPORT_NS(SND_SOC_SOF_HDA_AUDIO_CODEC_I915); diff --git a/sound/soc/sof/intel/pci-apl.c b/sound/soc/sof/intel/pci-apl.c index df6d897da290b6486e386cff799319976a062daf..f006dcf5458aee93051a2ef24cb79266e2119fef 100644 --- a/sound/soc/sof/intel/pci-apl.c +++ b/sound/soc/sof/intel/pci-apl.c @@ -105,6 +105,7 @@ static struct pci_driver snd_sof_pci_intel_apl_driver = { module_pci_driver(snd_sof_pci_intel_apl_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for ApolloLake platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_GENERIC); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_PCI_DEV); diff --git a/sound/soc/sof/intel/pci-cnl.c b/sound/soc/sof/intel/pci-cnl.c index a39fa3657d5556dc4ad938e071d0dd2e9ab1c12d..a8406342f08bdd8edf8bfeb7e5a2e90b80bf6c3a 100644 --- a/sound/soc/sof/intel/pci-cnl.c +++ b/sound/soc/sof/intel/pci-cnl.c @@ -143,6 +143,7 @@ static struct pci_driver snd_sof_pci_intel_cnl_driver = { module_pci_driver(snd_sof_pci_intel_cnl_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for CannonLake platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_GENERIC); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_PCI_DEV); diff --git a/sound/soc/sof/intel/pci-icl.c b/sound/soc/sof/intel/pci-icl.c index 9f1fe47475fb17a6d53f70a77c4212cdc25d181a..25effca50d9fed1a819501717bb209e0217c7b68 100644 --- a/sound/soc/sof/intel/pci-icl.c +++ b/sound/soc/sof/intel/pci-icl.c @@ -108,6 +108,7 @@ static struct pci_driver snd_sof_pci_intel_icl_driver = { module_pci_driver(snd_sof_pci_intel_icl_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for IceLake platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_GENERIC); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_CNL); diff --git a/sound/soc/sof/intel/pci-lnl.c b/sound/soc/sof/intel/pci-lnl.c index 68e5c90151b27250586e2658281021fca94d6e36..602c574064eb5ca34978be1edd8b0987edef67e0 100644 --- a/sound/soc/sof/intel/pci-lnl.c +++ b/sound/soc/sof/intel/pci-lnl.c @@ -70,6 +70,7 @@ static struct pci_driver snd_sof_pci_intel_lnl_driver = { module_pci_driver(snd_sof_pci_intel_lnl_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for LunarLake platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_GENERIC); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_MTL); diff --git a/sound/soc/sof/intel/pci-mtl.c b/sound/soc/sof/intel/pci-mtl.c index c685cb8d617123ab397b953c2a14baac8360f2dd..8cb0333c033ef19756818c46998e329edc2fee2c 100644 --- a/sound/soc/sof/intel/pci-mtl.c +++ b/sound/soc/sof/intel/pci-mtl.c @@ -133,6 +133,7 @@ static struct pci_driver snd_sof_pci_intel_mtl_driver = { module_pci_driver(snd_sof_pci_intel_mtl_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for MeteorLake platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_GENERIC); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_PCI_DEV); diff --git a/sound/soc/sof/intel/pci-skl.c b/sound/soc/sof/intel/pci-skl.c index 862da8009543bbd699d51434529eaeb09b90fa21..8ca0231d7e4f6e349d98fff90f0e4558920682cc 100644 --- a/sound/soc/sof/intel/pci-skl.c +++ b/sound/soc/sof/intel/pci-skl.c @@ -89,6 +89,7 @@ static struct pci_driver snd_sof_pci_intel_skl_driver = { module_pci_driver(snd_sof_pci_intel_skl_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for SkyLake platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_GENERIC); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_PCI_DEV); diff --git a/sound/soc/sof/intel/pci-tgl.c b/sound/soc/sof/intel/pci-tgl.c index f73bb47cd79e7b79fc372b7e1f439ecbacba6faa..ebe1a7d1668946018e7a3277993c95291dc7b1ee 100644 --- a/sound/soc/sof/intel/pci-tgl.c +++ b/sound/soc/sof/intel/pci-tgl.c @@ -317,6 +317,7 @@ static struct pci_driver snd_sof_pci_intel_tgl_driver = { module_pci_driver(snd_sof_pci_intel_tgl_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for TigerLake platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_GENERIC); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HDA_COMMON); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_CNL); diff --git a/sound/soc/sof/intel/pci-tng.c b/sound/soc/sof/intel/pci-tng.c index 5c3069588bb77111654b69e1c751e4e54ab49462..1375c393827e1167806a748cb4dab28b3b912791 100644 --- a/sound/soc/sof/intel/pci-tng.c +++ b/sound/soc/sof/intel/pci-tng.c @@ -244,6 +244,7 @@ static struct pci_driver snd_sof_pci_intel_tng_driver = { module_pci_driver(snd_sof_pci_intel_tng_driver); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for Tangier platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HIFI_EP_IPC); MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); MODULE_IMPORT_NS(SND_SOC_SOF_PCI_DEV); diff --git a/sound/soc/sof/ipc4-pcm.c b/sound/soc/sof/ipc4-pcm.c index 307bee63756b5c81d2b19ea85ee5ca26bd6323aa..4df2be3d39eba0efba95aa16ebb7514c45dccc96 100644 --- a/sound/soc/sof/ipc4-pcm.c +++ b/sound/soc/sof/ipc4-pcm.c @@ -650,7 +650,7 @@ static int sof_ipc4_pcm_dai_link_fixup(struct snd_soc_pcm_runtime *rtd, struct snd_soc_dai *cpu_dai = snd_soc_rtd_to_cpu(rtd, 0); struct sof_ipc4_audio_format *ipc4_fmt; struct sof_ipc4_copier *ipc4_copier; - bool single_fmt = false; + bool single_bitdepth = false; u32 valid_bits = 0; int dir, ret; @@ -682,18 +682,18 @@ static int sof_ipc4_pcm_dai_link_fixup(struct snd_soc_pcm_runtime *rtd, return 0; if (dir == SNDRV_PCM_STREAM_PLAYBACK) { - if (sof_ipc4_copier_is_single_format(sdev, + if (sof_ipc4_copier_is_single_bitdepth(sdev, available_fmt->output_pin_fmts, available_fmt->num_output_formats)) { ipc4_fmt = &available_fmt->output_pin_fmts->audio_fmt; - single_fmt = true; + single_bitdepth = true; } } else { - if (sof_ipc4_copier_is_single_format(sdev, + if (sof_ipc4_copier_is_single_bitdepth(sdev, available_fmt->input_pin_fmts, available_fmt->num_input_formats)) { ipc4_fmt = &available_fmt->input_pin_fmts->audio_fmt; - single_fmt = true; + single_bitdepth = true; } } } @@ -703,7 +703,7 @@ static int sof_ipc4_pcm_dai_link_fixup(struct snd_soc_pcm_runtime *rtd, if (ret) return ret; - if (single_fmt) { + if (single_bitdepth) { snd_mask_none(fmt); valid_bits = SOF_IPC4_AUDIO_FORMAT_CFG_V_BIT_DEPTH(ipc4_fmt->fmt_cfg); dev_dbg(component->dev, "Set %s to %d bit format\n", dai->name, valid_bits); diff --git a/sound/soc/sof/ipc4-topology.c b/sound/soc/sof/ipc4-topology.c index beff10989324755e68237138e598fb017c47b617..00987039c9720ff0034d4baf536b8cb6c27a0f13 100644 --- a/sound/soc/sof/ipc4-topology.c +++ b/sound/soc/sof/ipc4-topology.c @@ -195,9 +195,10 @@ static void sof_ipc4_dbg_audio_format(struct device *dev, struct sof_ipc4_pin_fo for (i = 0; i < num_formats; i++) { struct sof_ipc4_audio_format *fmt = &pin_fmt[i].audio_fmt; dev_dbg(dev, - "Pin index #%d: %uHz, %ubit (ch_map %#x ch_cfg %u interleaving_style %u fmt_cfg %#x) buffer size %d\n", - pin_fmt[i].pin_index, fmt->sampling_frequency, fmt->bit_depth, fmt->ch_map, - fmt->ch_cfg, fmt->interleaving_style, fmt->fmt_cfg, + "Pin index #%d: %uHz, %ubit, %luch (ch_map %#x ch_cfg %u interleaving_style %u fmt_cfg %#x) buffer size %d\n", + pin_fmt[i].pin_index, fmt->sampling_frequency, fmt->bit_depth, + SOF_IPC4_AUDIO_FORMAT_CFG_CHANNELS_COUNT(fmt->fmt_cfg), + fmt->ch_map, fmt->ch_cfg, fmt->interleaving_style, fmt->fmt_cfg, pin_fmt[i].buffer_size); } } @@ -217,6 +218,14 @@ sof_ipc4_get_input_pin_audio_fmt(struct snd_sof_widget *swidget, int pin_index) } process = swidget->private; + + /* + * For process modules without base config extension, base module config + * format is used for all input pins + */ + if (process->init_config != SOF_IPC4_MODULE_INIT_CONFIG_TYPE_BASE_CFG_WITH_EXT) + return &process->base_config.audio_fmt; + base_cfg_ext = process->base_config_ext; /* @@ -1422,7 +1431,7 @@ static int snd_sof_get_hw_config_params(struct snd_sof_dev *sdev, struct snd_sof static int snd_sof_get_nhlt_endpoint_data(struct snd_sof_dev *sdev, struct snd_sof_dai *dai, - bool single_format, + bool single_bitdepth, struct snd_pcm_hw_params *params, u32 dai_index, u32 linktype, u8 dir, u32 **dst, u32 *len) { @@ -1445,7 +1454,7 @@ snd_sof_get_nhlt_endpoint_data(struct snd_sof_dev *sdev, struct snd_sof_dai *dai * Look for 32-bit blob first instead of 16-bit if copier * supports multiple formats */ - if (bit_depth == 16 && !single_format) { + if (bit_depth == 16 && !single_bitdepth) { dev_dbg(sdev->dev, "Looking for 32-bit blob first for DMIC\n"); format_change = true; bit_depth = 32; @@ -1483,6 +1492,8 @@ snd_sof_get_nhlt_endpoint_data(struct snd_sof_dev *sdev, struct snd_sof_dai *dai dir, dev_type); if (!cfg) { + bool get_new_blob = false; + if (format_change) { /* * The 32-bit blob was not found in NHLT table, try to @@ -1490,7 +1501,20 @@ snd_sof_get_nhlt_endpoint_data(struct snd_sof_dev *sdev, struct snd_sof_dai *dai */ bit_depth = params_width(params); format_change = false; + get_new_blob = true; + } else if (linktype == SOF_DAI_INTEL_DMIC && !single_bitdepth) { + /* + * The requested 32-bit blob (no format change for the + * blob request) was not found in NHLT table, try to + * look for 16-bit blob if the copier supports multiple + * formats + */ + bit_depth = 16; + format_change = true; + get_new_blob = true; + } + if (get_new_blob) { cfg = intel_nhlt_get_endpoint_blob(sdev->dev, ipc4_data->nhlt, dai_index, nhlt_type, bit_depth, bit_depth, @@ -1513,8 +1537,8 @@ out: if (format_change) { /* - * Update the params to reflect that we have loaded 32-bit blob - * instead of the 16-bit. + * Update the params to reflect that different blob was loaded + * instead of the requested bit depth (16 -> 32 or 32 -> 16). * This information is going to be used by the caller to find * matching copier format on the dai side. */ @@ -1522,7 +1546,11 @@ out: m = hw_param_mask(params, SNDRV_PCM_HW_PARAM_FORMAT); snd_mask_none(m); - snd_mask_set_format(m, SNDRV_PCM_FORMAT_S32_LE); + if (bit_depth == 16) + snd_mask_set_format(m, SNDRV_PCM_FORMAT_S16_LE); + else + snd_mask_set_format(m, SNDRV_PCM_FORMAT_S32_LE); + } return 0; @@ -1530,7 +1558,7 @@ out: #else static int snd_sof_get_nhlt_endpoint_data(struct snd_sof_dev *sdev, struct snd_sof_dai *dai, - bool single_format, + bool single_bitdepth, struct snd_pcm_hw_params *params, u32 dai_index, u32 linktype, u8 dir, u32 **dst, u32 *len) { @@ -1538,9 +1566,9 @@ snd_sof_get_nhlt_endpoint_data(struct snd_sof_dev *sdev, struct snd_sof_dai *dai } #endif -bool sof_ipc4_copier_is_single_format(struct snd_sof_dev *sdev, - struct sof_ipc4_pin_format *pin_fmts, - u32 pin_fmts_size) +bool sof_ipc4_copier_is_single_bitdepth(struct snd_sof_dev *sdev, + struct sof_ipc4_pin_format *pin_fmts, + u32 pin_fmts_size) { struct sof_ipc4_audio_format *fmt; u32 valid_bits; @@ -1563,6 +1591,55 @@ bool sof_ipc4_copier_is_single_format(struct snd_sof_dev *sdev, return true; } +static int +sof_ipc4_adjust_params_to_dai_format(struct snd_sof_dev *sdev, + struct snd_pcm_hw_params *params, + struct sof_ipc4_pin_format *pin_fmts, + u32 pin_fmts_size) +{ + u32 params_mask = BIT(SNDRV_PCM_HW_PARAM_RATE) | + BIT(SNDRV_PCM_HW_PARAM_CHANNELS) | + BIT(SNDRV_PCM_HW_PARAM_FORMAT); + struct sof_ipc4_audio_format *fmt; + u32 rate, channels, valid_bits; + int i; + + fmt = &pin_fmts[0].audio_fmt; + rate = fmt->sampling_frequency; + channels = SOF_IPC4_AUDIO_FORMAT_CFG_CHANNELS_COUNT(fmt->fmt_cfg); + valid_bits = SOF_IPC4_AUDIO_FORMAT_CFG_V_BIT_DEPTH(fmt->fmt_cfg); + + /* check if parameters in topology defined formats are the same */ + for (i = 1; i < pin_fmts_size; i++) { + u32 val; + + fmt = &pin_fmts[i].audio_fmt; + + if (params_mask & BIT(SNDRV_PCM_HW_PARAM_RATE)) { + val = fmt->sampling_frequency; + if (val != rate) + params_mask &= ~BIT(SNDRV_PCM_HW_PARAM_RATE); + } + if (params_mask & BIT(SNDRV_PCM_HW_PARAM_CHANNELS)) { + val = SOF_IPC4_AUDIO_FORMAT_CFG_CHANNELS_COUNT(fmt->fmt_cfg); + if (val != channels) + params_mask &= ~BIT(SNDRV_PCM_HW_PARAM_CHANNELS); + } + if (params_mask & BIT(SNDRV_PCM_HW_PARAM_FORMAT)) { + val = SOF_IPC4_AUDIO_FORMAT_CFG_V_BIT_DEPTH(fmt->fmt_cfg); + if (val != valid_bits) + params_mask &= ~BIT(SNDRV_PCM_HW_PARAM_FORMAT); + } + } + + if (params_mask) + return sof_ipc4_update_hw_params(sdev, params, + &pin_fmts[0].audio_fmt, + params_mask); + + return 0; +} + static int sof_ipc4_prepare_dai_copier(struct snd_sof_dev *sdev, struct snd_sof_dai *dai, struct snd_pcm_hw_params *params, int dir) @@ -1570,8 +1647,10 @@ sof_ipc4_prepare_dai_copier(struct snd_sof_dev *sdev, struct snd_sof_dai *dai, struct sof_ipc4_available_audio_format *available_fmt; struct snd_pcm_hw_params dai_params = *params; struct sof_ipc4_copier_data *copier_data; + struct sof_ipc4_pin_format *pin_fmts; struct sof_ipc4_copier *ipc4_copier; - bool single_format; + bool single_bitdepth; + u32 num_pin_fmts; int ret; ipc4_copier = dai->private; @@ -1579,40 +1658,26 @@ sof_ipc4_prepare_dai_copier(struct snd_sof_dev *sdev, struct snd_sof_dai *dai, available_fmt = &ipc4_copier->available_fmt; /* - * If the copier on the DAI side supports only single bit depth then - * this depth (format) should be used to look for the NHLT blob (if - * needed) and in case of capture this should be used for the input - * format lookup + * Fixup the params based on the format parameters of the DAI. If any + * of the RATE, CHANNELS, bit depth is static among the formats then + * narrow the params to only allow that specific parameter value. */ if (dir == SNDRV_PCM_STREAM_PLAYBACK) { - single_format = sof_ipc4_copier_is_single_format(sdev, - available_fmt->output_pin_fmts, - available_fmt->num_output_formats); - - /* Update the dai_params with the only supported format */ - if (single_format) { - ret = sof_ipc4_update_hw_params(sdev, &dai_params, - &available_fmt->output_pin_fmts[0].audio_fmt, - BIT(SNDRV_PCM_HW_PARAM_FORMAT)); - if (ret) - return ret; - } + pin_fmts = available_fmt->output_pin_fmts; + num_pin_fmts = available_fmt->num_output_formats; } else { - single_format = sof_ipc4_copier_is_single_format(sdev, - available_fmt->input_pin_fmts, - available_fmt->num_input_formats); - - /* Update the dai_params with the only supported format */ - if (single_format) { - ret = sof_ipc4_update_hw_params(sdev, &dai_params, - &available_fmt->input_pin_fmts[0].audio_fmt, - BIT(SNDRV_PCM_HW_PARAM_FORMAT)); - if (ret) - return ret; - } + pin_fmts = available_fmt->input_pin_fmts; + num_pin_fmts = available_fmt->num_input_formats; } - ret = snd_sof_get_nhlt_endpoint_data(sdev, dai, single_format, + ret = sof_ipc4_adjust_params_to_dai_format(sdev, &dai_params, pin_fmts, + num_pin_fmts); + if (ret) + return ret; + + single_bitdepth = sof_ipc4_copier_is_single_bitdepth(sdev, pin_fmts, + num_pin_fmts); + ret = snd_sof_get_nhlt_endpoint_data(sdev, dai, single_bitdepth, &dai_params, ipc4_copier->dai_index, ipc4_copier->dai_type, dir, @@ -1647,7 +1712,7 @@ sof_ipc4_prepare_copier_module(struct snd_sof_widget *swidget, u32 out_ref_rate, out_ref_channels; u32 deep_buffer_dma_ms = 0; int output_fmt_index; - bool single_output_format; + bool single_output_bitdepth; int i; dev_dbg(sdev->dev, "copier %s, type %d", swidget->widget->name, swidget->id); @@ -1784,9 +1849,9 @@ sof_ipc4_prepare_copier_module(struct snd_sof_widget *swidget, return ret; /* set the reference params for output format selection */ - single_output_format = sof_ipc4_copier_is_single_format(sdev, - available_fmt->output_pin_fmts, - available_fmt->num_output_formats); + single_output_bitdepth = sof_ipc4_copier_is_single_bitdepth(sdev, + available_fmt->output_pin_fmts, + available_fmt->num_output_formats); switch (swidget->id) { case snd_soc_dapm_aif_in: case snd_soc_dapm_dai_out: @@ -1798,7 +1863,7 @@ sof_ipc4_prepare_copier_module(struct snd_sof_widget *swidget, out_ref_rate = in_fmt->sampling_frequency; out_ref_channels = SOF_IPC4_AUDIO_FORMAT_CFG_CHANNELS_COUNT(in_fmt->fmt_cfg); - if (!single_output_format) + if (!single_output_bitdepth) out_ref_valid_bits = SOF_IPC4_AUDIO_FORMAT_CFG_V_BIT_DEPTH(in_fmt->fmt_cfg); break; @@ -1807,7 +1872,7 @@ sof_ipc4_prepare_copier_module(struct snd_sof_widget *swidget, case snd_soc_dapm_dai_in: out_ref_rate = params_rate(fe_params); out_ref_channels = params_channels(fe_params); - if (!single_output_format) { + if (!single_output_bitdepth) { out_ref_valid_bits = sof_ipc4_get_valid_bits(sdev, fe_params); if (out_ref_valid_bits < 0) return out_ref_valid_bits; @@ -1825,7 +1890,7 @@ sof_ipc4_prepare_copier_module(struct snd_sof_widget *swidget, * if the output format is the same across all available output formats, choose * that as the reference. */ - if (single_output_format) { + if (single_output_bitdepth) { struct sof_ipc4_audio_format *out_fmt; out_fmt = &available_fmt->output_pin_fmts[0].audio_fmt; diff --git a/sound/soc/sof/ipc4-topology.h b/sound/soc/sof/ipc4-topology.h index 4488762f6a71ba47322e0a1b7024833fed727612..f4dc499c0ffe55cb2c2dbdcd57efff4b1f38ba5e 100644 --- a/sound/soc/sof/ipc4-topology.h +++ b/sound/soc/sof/ipc4-topology.h @@ -476,7 +476,7 @@ struct sof_ipc4_process { u32 init_config; }; -bool sof_ipc4_copier_is_single_format(struct snd_sof_dev *sdev, - struct sof_ipc4_pin_format *pin_fmts, - u32 pin_fmts_size); +bool sof_ipc4_copier_is_single_bitdepth(struct snd_sof_dev *sdev, + struct sof_ipc4_pin_format *pin_fmts, + u32 pin_fmts_size); #endif diff --git a/sound/soc/sof/mediatek/mt8186/mt8186.c b/sound/soc/sof/mediatek/mt8186/mt8186.c index c63e0d2f4b9686cc86877715e1893b580aae2c94..bea1b9d9ca2886db2b1f513ab10327b77f10b764 100644 --- a/sound/soc/sof/mediatek/mt8186/mt8186.c +++ b/sound/soc/sof/mediatek/mt8186/mt8186.c @@ -666,6 +666,7 @@ static struct platform_driver snd_sof_of_mt8186_driver = { }; module_platform_driver(snd_sof_of_mt8186_driver); +MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for MT8186/MT8188 platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); MODULE_IMPORT_NS(SND_SOC_SOF_MTK_COMMON); -MODULE_LICENSE("Dual BSD/GPL"); diff --git a/sound/soc/sof/mediatek/mt8195/mt8195.c b/sound/soc/sof/mediatek/mt8195/mt8195.c index fc1c016104aee329d32433d03e19355e0541f2e6..31dc98d1b1d8bddde8f91047125ea034b912f691 100644 --- a/sound/soc/sof/mediatek/mt8195/mt8195.c +++ b/sound/soc/sof/mediatek/mt8195/mt8195.c @@ -619,6 +619,7 @@ static struct platform_driver snd_sof_of_mt8195_driver = { }; module_platform_driver(snd_sof_of_mt8195_driver); +MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for MTL 8195 platforms"); MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); MODULE_IMPORT_NS(SND_SOC_SOF_MTK_COMMON); -MODULE_LICENSE("Dual BSD/GPL"); diff --git a/sound/soc/sof/mediatek/mtk-adsp-common.c b/sound/soc/sof/mediatek/mtk-adsp-common.c index de8dbe27cd0deffc0a6708fcce6bfa0186c73cad..20bcf5590eb8855797d6cc61ef36e676d9dd6ceb 100644 --- a/sound/soc/sof/mediatek/mtk-adsp-common.c +++ b/sound/soc/sof/mediatek/mtk-adsp-common.c @@ -82,3 +82,4 @@ void mtk_adsp_dump(struct snd_sof_dev *sdev, u32 flags) EXPORT_SYMBOL(mtk_adsp_dump); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF helpers for MTK ADSP platforms"); diff --git a/sound/soc/sof/nocodec.c b/sound/soc/sof/nocodec.c index fdcbe33d3dcfb8346dfd2977e7cfcb70887256f6..b12b3d865ae30c9078bb33c5d25ab36b81a72a34 100644 --- a/sound/soc/sof/nocodec.c +++ b/sound/soc/sof/nocodec.c @@ -110,7 +110,7 @@ static struct platform_driver sof_nocodec_audio = { }; module_platform_driver(sof_nocodec_audio) +MODULE_LICENSE("Dual BSD/GPL"); MODULE_DESCRIPTION("ASoC sof nocodec"); MODULE_AUTHOR("Liam Girdwood"); -MODULE_LICENSE("Dual BSD/GPL"); MODULE_ALIAS("platform:sof-nocodec"); diff --git a/sound/soc/sof/sof-acpi-dev.c b/sound/soc/sof/sof-acpi-dev.c index 2d96d00f1c449a7ba98b8cb5f531a107e844e880..b196b2b74c2647124b5876e2ca1a222524d88cfb 100644 --- a/sound/soc/sof/sof-acpi-dev.c +++ b/sound/soc/sof/sof-acpi-dev.c @@ -100,3 +100,4 @@ void sof_acpi_remove(struct platform_device *pdev) EXPORT_SYMBOL_NS(sof_acpi_remove, SND_SOC_SOF_ACPI_DEV); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for ACPI platforms"); diff --git a/sound/soc/sof/sof-client-ipc-flood-test.c b/sound/soc/sof/sof-client-ipc-flood-test.c index 43561492609252e598dc7d474783a9fb285ad195..e7d2001140e81a93e46d8d05d8f1ca45ef05ef14 100644 --- a/sound/soc/sof/sof-client-ipc-flood-test.c +++ b/sound/soc/sof/sof-client-ipc-flood-test.c @@ -394,6 +394,6 @@ static struct auxiliary_driver sof_ipc_flood_client_drv = { module_auxiliary_driver(sof_ipc_flood_client_drv); -MODULE_DESCRIPTION("SOF IPC Flood Test Client Driver"); MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("SOF IPC Flood Test Client Driver"); MODULE_IMPORT_NS(SND_SOC_SOF_CLIENT); diff --git a/sound/soc/sof/sof-client-ipc-kernel-injector.c b/sound/soc/sof/sof-client-ipc-kernel-injector.c index 6973b6690df4b8f838bd8d7c316d69b03d99ed7a..d3f541069b24b8fc4dfaa8a1d4774e6e30db6cc5 100644 --- a/sound/soc/sof/sof-client-ipc-kernel-injector.c +++ b/sound/soc/sof/sof-client-ipc-kernel-injector.c @@ -157,6 +157,6 @@ static struct auxiliary_driver sof_msg_inject_client_drv = { module_auxiliary_driver(sof_msg_inject_client_drv); -MODULE_DESCRIPTION("SOF IPC Kernel Injector Client Driver"); MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("SOF IPC Kernel Injector Client Driver"); MODULE_IMPORT_NS(SND_SOC_SOF_CLIENT); diff --git a/sound/soc/sof/sof-client-ipc-msg-injector.c b/sound/soc/sof/sof-client-ipc-msg-injector.c index af22e6421029c67a4bb498fd16d47857892e5f29..d0f8beb9d0008d1ccfc1a9eadd13324108d5e1de 100644 --- a/sound/soc/sof/sof-client-ipc-msg-injector.c +++ b/sound/soc/sof/sof-client-ipc-msg-injector.c @@ -335,6 +335,6 @@ static struct auxiliary_driver sof_msg_inject_client_drv = { module_auxiliary_driver(sof_msg_inject_client_drv); -MODULE_DESCRIPTION("SOF IPC Message Injector Client Driver"); MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("SOF IPC Message Injector Client Driver"); MODULE_IMPORT_NS(SND_SOC_SOF_CLIENT); diff --git a/sound/soc/sof/sof-client-probes.c b/sound/soc/sof/sof-client-probes.c index b8f297307565cc630b87aaca1cec9973e9a8868b..ccc7d38ddc38340686c93282d83f4e427a5febc8 100644 --- a/sound/soc/sof/sof-client-probes.c +++ b/sound/soc/sof/sof-client-probes.c @@ -540,6 +540,6 @@ static struct auxiliary_driver sof_probes_client_drv = { module_auxiliary_driver(sof_probes_client_drv); -MODULE_DESCRIPTION("SOF Probes Client Driver"); MODULE_LICENSE("GPL v2"); +MODULE_DESCRIPTION("SOF Probes Client Driver"); MODULE_IMPORT_NS(SND_SOC_SOF_CLIENT); diff --git a/sound/soc/sof/sof-of-dev.c b/sound/soc/sof/sof-of-dev.c index b9a499e92b9a57514effeb54dbaf1e8fa43f42dd..71f7153cf79c6915f95d53a3625aa86bd896272a 100644 --- a/sound/soc/sof/sof-of-dev.c +++ b/sound/soc/sof/sof-of-dev.c @@ -93,3 +93,4 @@ void sof_of_shutdown(struct platform_device *pdev) EXPORT_SYMBOL(sof_of_shutdown); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for OF/DT platforms"); diff --git a/sound/soc/sof/sof-pci-dev.c b/sound/soc/sof/sof-pci-dev.c index 4365405783e61de3dbe4ca4f9b71b68996f1d869..38f2187da5de185f70d62dca8a26df5a457cf697 100644 --- a/sound/soc/sof/sof-pci-dev.c +++ b/sound/soc/sof/sof-pci-dev.c @@ -304,3 +304,4 @@ void sof_pci_shutdown(struct pci_dev *pci) EXPORT_SYMBOL_NS(sof_pci_shutdown, SND_SOC_SOF_PCI_DEV); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF support for PCI platforms"); diff --git a/sound/soc/sof/sof-utils.c b/sound/soc/sof/sof-utils.c index cad041bf56ccf62ab2d392690eb43287d6855776..44608682e9f8f678dd7b827145cc97507b4dc217 100644 --- a/sound/soc/sof/sof-utils.c +++ b/sound/soc/sof/sof-utils.c @@ -73,3 +73,4 @@ int snd_sof_create_page_table(struct device *dev, EXPORT_SYMBOL(snd_sof_create_page_table); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF utils"); diff --git a/sound/soc/sof/stream-ipc.c b/sound/soc/sof/stream-ipc.c index eb71303aa24c6ae61f7114b877dd5958a72275cb..794c7bbccbaf9202d836d340229bec9108600aa1 100644 --- a/sound/soc/sof/stream-ipc.c +++ b/sound/soc/sof/stream-ipc.c @@ -125,5 +125,3 @@ int sof_stream_pcm_close(struct snd_sof_dev *sdev, return 0; } EXPORT_SYMBOL(sof_stream_pcm_close); - -MODULE_LICENSE("Dual BSD/GPL"); diff --git a/sound/soc/sof/xtensa/core.c b/sound/soc/sof/xtensa/core.c index ccbc3fcdadd5ec8a6f66f868d504f4d9f539aea2..3cf8c84beff9570f559cc1fc027af9dc5766190d 100644 --- a/sound/soc/sof/xtensa/core.c +++ b/sound/soc/sof/xtensa/core.c @@ -151,5 +151,5 @@ const struct dsp_arch_ops sof_xtensa_arch_ops = { }; EXPORT_SYMBOL_NS(sof_xtensa_arch_ops, SND_SOC_SOF_XTENSA); -MODULE_DESCRIPTION("SOF Xtensa DSP support"); MODULE_LICENSE("Dual BSD/GPL"); +MODULE_DESCRIPTION("SOF Xtensa DSP support"); diff --git a/tools/bpf/bpftool/Documentation/bpftool-btf.rst b/tools/bpf/bpftool/Documentation/bpftool-btf.rst index eaba24320fb268b7fbcda12b8c6881ed637ee47e..3f6bca03ad2e6fe27df3bef4ac92cc3d4676fccb 100644 --- a/tools/bpf/bpftool/Documentation/bpftool-btf.rst +++ b/tools/bpf/bpftool/Documentation/bpftool-btf.rst @@ -28,7 +28,7 @@ BTF COMMANDS | **bpftool** **btf help** | | *BTF_SRC* := { **id** *BTF_ID* | **prog** *PROG* | **map** *MAP* [{**key** | **value** | **kv** | **all**}] | **file** *FILE* } -| *FORMAT* := { **raw** | **c** } +| *FORMAT* := { **raw** | **c** [**unsorted**] } | *MAP* := { **id** *MAP_ID* | **pinned** *FILE* } | *PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* | **name** *PROG_NAME* } @@ -63,7 +63,9 @@ bpftool btf dump *BTF_SRC* pahole. **format** option can be used to override default (raw) output format. Raw - (**raw**) or C-syntax (**c**) output formats are supported. + (**raw**) or C-syntax (**c**) output formats are supported. With C-style + formatting, the output is sorted by default. Use the **unsorted** option + to avoid sorting the output. bpftool btf help Print short help message. diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile index dfa4f1bebbb311ee1b7be53c57d66880729db758..ba927379eb2017630ccc559d25b2a3f6b44c1411 100644 --- a/tools/bpf/bpftool/Makefile +++ b/tools/bpf/bpftool/Makefile @@ -204,10 +204,11 @@ ifeq ($(feature-clang-bpf-co-re),1) BUILD_BPF_SKELS := 1 -$(OUTPUT)vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL_BOOTSTRAP) ifeq ($(VMLINUX_H),) +$(OUTPUT)vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL_BOOTSTRAP) $(QUIET_GEN)$(BPFTOOL_BOOTSTRAP) btf dump file $< format c > $@ else +$(OUTPUT)vmlinux.h: $(VMLINUX_H) $(Q)cp "$(VMLINUX_H)" $@ endif diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool index 04afe2ac2228210f01d834cd450caf8177452ba8..be99d49b871443b9ec6f6dfaa1a752b63de6c066 100644 --- a/tools/bpf/bpftool/bash-completion/bpftool +++ b/tools/bpf/bpftool/bash-completion/bpftool @@ -930,6 +930,9 @@ _bpftool() format) COMPREPLY=( $( compgen -W "c raw" -- "$cur" ) ) ;; + c) + COMPREPLY=( $( compgen -W "unsorted" -- "$cur" ) ) + ;; *) # emit extra options case ${words[3]} in diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c index 91fcb75babe33a9b15b6ae856a884cf2a68c071f..6789c7a4d5ca1de84e6397d302b1d59500f132a6 100644 --- a/tools/bpf/bpftool/btf.c +++ b/tools/bpf/bpftool/btf.c @@ -20,6 +20,8 @@ #include "json_writer.h" #include "main.h" +#define KFUNC_DECL_TAG "bpf_kfunc" + static const char * const btf_kind_str[NR_BTF_KINDS] = { [BTF_KIND_UNKN] = "UNKNOWN", [BTF_KIND_INT] = "INT", @@ -43,6 +45,13 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = { [BTF_KIND_ENUM64] = "ENUM64", }; +struct sort_datum { + int index; + int type_rank; + const char *sort_name; + const char *own_name; +}; + static const char *btf_int_enc_str(__u8 encoding) { switch (encoding) { @@ -454,15 +463,171 @@ static int dump_btf_raw(const struct btf *btf, return 0; } +static int dump_btf_kfuncs(struct btf_dump *d, const struct btf *btf) +{ + LIBBPF_OPTS(btf_dump_emit_type_decl_opts, opts); + int cnt = btf__type_cnt(btf); + int i; + + printf("\n/* BPF kfuncs */\n"); + printf("#ifndef BPF_NO_KFUNC_PROTOTYPES\n"); + + for (i = 1; i < cnt; i++) { + const struct btf_type *t = btf__type_by_id(btf, i); + const char *name; + int err; + + if (!btf_is_decl_tag(t)) + continue; + + if (btf_decl_tag(t)->component_idx != -1) + continue; + + name = btf__name_by_offset(btf, t->name_off); + if (strncmp(name, KFUNC_DECL_TAG, sizeof(KFUNC_DECL_TAG))) + continue; + + t = btf__type_by_id(btf, t->type); + if (!btf_is_func(t)) + continue; + + printf("extern "); + + opts.field_name = btf__name_by_offset(btf, t->name_off); + err = btf_dump__emit_type_decl(d, t->type, &opts); + if (err) + return err; + + printf(" __weak __ksym;\n"); + } + + printf("#endif\n\n"); + + return 0; +} + static void __printf(2, 0) btf_dump_printf(void *ctx, const char *fmt, va_list args) { vfprintf(stdout, fmt, args); } +static int btf_type_rank(const struct btf *btf, __u32 index, bool has_name) +{ + const struct btf_type *t = btf__type_by_id(btf, index); + const int kind = btf_kind(t); + const int max_rank = 10; + + if (t->name_off) + has_name = true; + + switch (kind) { + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + return has_name ? 1 : 0; + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + return 2; + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + return has_name ? 3 : max_rank; + case BTF_KIND_FUNC_PROTO: + return has_name ? 4 : max_rank; + case BTF_KIND_ARRAY: + if (has_name) + return btf_type_rank(btf, btf_array(t)->type, has_name); + return max_rank; + case BTF_KIND_TYPE_TAG: + case BTF_KIND_CONST: + case BTF_KIND_PTR: + case BTF_KIND_VOLATILE: + case BTF_KIND_RESTRICT: + case BTF_KIND_TYPEDEF: + case BTF_KIND_DECL_TAG: + if (has_name) + return btf_type_rank(btf, t->type, has_name); + return max_rank; + default: + return max_rank; + } +} + +static const char *btf_type_sort_name(const struct btf *btf, __u32 index, bool from_ref) +{ + const struct btf_type *t = btf__type_by_id(btf, index); + + switch (btf_kind(t)) { + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: { + int name_off = t->name_off; + + /* Use name of the first element for anonymous enums if allowed */ + if (!from_ref && !t->name_off && btf_vlen(t)) + name_off = btf_enum(t)->name_off; + + return btf__name_by_offset(btf, name_off); + } + case BTF_KIND_ARRAY: + return btf_type_sort_name(btf, btf_array(t)->type, true); + case BTF_KIND_TYPE_TAG: + case BTF_KIND_CONST: + case BTF_KIND_PTR: + case BTF_KIND_VOLATILE: + case BTF_KIND_RESTRICT: + case BTF_KIND_TYPEDEF: + case BTF_KIND_DECL_TAG: + return btf_type_sort_name(btf, t->type, true); + default: + return btf__name_by_offset(btf, t->name_off); + } + return NULL; +} + +static int btf_type_compare(const void *left, const void *right) +{ + const struct sort_datum *d1 = (const struct sort_datum *)left; + const struct sort_datum *d2 = (const struct sort_datum *)right; + int r; + + if (d1->type_rank != d2->type_rank) + return d1->type_rank < d2->type_rank ? -1 : 1; + + r = strcmp(d1->sort_name, d2->sort_name); + if (r) + return r; + + return strcmp(d1->own_name, d2->own_name); +} + +static struct sort_datum *sort_btf_c(const struct btf *btf) +{ + struct sort_datum *datums; + int n; + + n = btf__type_cnt(btf); + datums = malloc(sizeof(struct sort_datum) * n); + if (!datums) + return NULL; + + for (int i = 0; i < n; ++i) { + struct sort_datum *d = datums + i; + const struct btf_type *t = btf__type_by_id(btf, i); + + d->index = i; + d->type_rank = btf_type_rank(btf, i, false); + d->sort_name = btf_type_sort_name(btf, i, false); + d->own_name = btf__name_by_offset(btf, t->name_off); + } + + qsort(datums, n, sizeof(struct sort_datum), btf_type_compare); + + return datums; +} + static int dump_btf_c(const struct btf *btf, - __u32 *root_type_ids, int root_type_cnt) + __u32 *root_type_ids, int root_type_cnt, bool sort_dump) { + struct sort_datum *datums = NULL; struct btf_dump *d; int err = 0, i; @@ -476,6 +641,12 @@ static int dump_btf_c(const struct btf *btf, printf("#ifndef BPF_NO_PRESERVE_ACCESS_INDEX\n"); printf("#pragma clang attribute push (__attribute__((preserve_access_index)), apply_to = record)\n"); printf("#endif\n\n"); + printf("#ifndef __ksym\n"); + printf("#define __ksym __attribute__((section(\".ksyms\")))\n"); + printf("#endif\n\n"); + printf("#ifndef __weak\n"); + printf("#define __weak __attribute__((weak))\n"); + printf("#endif\n\n"); if (root_type_cnt) { for (i = 0; i < root_type_cnt; i++) { @@ -486,11 +657,19 @@ static int dump_btf_c(const struct btf *btf, } else { int cnt = btf__type_cnt(btf); + if (sort_dump) + datums = sort_btf_c(btf); for (i = 1; i < cnt; i++) { - err = btf_dump__dump_type(d, i); + int idx = datums ? datums[i].index : i; + + err = btf_dump__dump_type(d, idx); if (err) goto done; } + + err = dump_btf_kfuncs(d, btf); + if (err) + goto done; } printf("#ifndef BPF_NO_PRESERVE_ACCESS_INDEX\n"); @@ -500,6 +679,7 @@ static int dump_btf_c(const struct btf *btf, printf("#endif /* __VMLINUX_H__ */\n"); done: + free(datums); btf_dump__free(d); return err; } @@ -549,10 +729,10 @@ static bool btf_is_kernel_module(__u32 btf_id) static int do_dump(int argc, char **argv) { + bool dump_c = false, sort_dump_c = true; struct btf *btf = NULL, *base = NULL; __u32 root_type_ids[2]; int root_type_cnt = 0; - bool dump_c = false; __u32 btf_id = -1; const char *src; int fd = -1; @@ -663,6 +843,9 @@ static int do_dump(int argc, char **argv) goto done; } NEXT_ARG(); + } else if (is_prefix(*argv, "unsorted")) { + sort_dump_c = false; + NEXT_ARG(); } else { p_err("unrecognized option: '%s'", *argv); err = -EINVAL; @@ -691,7 +874,7 @@ static int do_dump(int argc, char **argv) err = -ENOTSUP; goto done; } - err = dump_btf_c(btf, root_type_ids, root_type_cnt); + err = dump_btf_c(btf, root_type_ids, root_type_cnt, sort_dump_c); } else { err = dump_btf_raw(btf, root_type_ids, root_type_cnt); } @@ -1063,7 +1246,7 @@ static int do_help(int argc, char **argv) " %1$s %2$s help\n" "\n" " BTF_SRC := { id BTF_ID | prog PROG | map MAP [{key | value | kv | all}] | file FILE }\n" - " FORMAT := { raw | c }\n" + " FORMAT := { raw | c [unsorted] }\n" " " HELP_SPEC_MAP "\n" " " HELP_SPEC_PROGRAM "\n" " " HELP_SPEC_OPTIONS " |\n" diff --git a/tools/bpf/bpftool/cgroup.c b/tools/bpf/bpftool/cgroup.c index af6898c0f388b6a7fb63c021e232622f0033bd18..9af426d4329931ebb1a403db50873adefd2beb38 100644 --- a/tools/bpf/bpftool/cgroup.c +++ b/tools/bpf/bpftool/cgroup.c @@ -19,6 +19,38 @@ #include "main.h" +static const int cgroup_attach_types[] = { + BPF_CGROUP_INET_INGRESS, + BPF_CGROUP_INET_EGRESS, + BPF_CGROUP_INET_SOCK_CREATE, + BPF_CGROUP_INET_SOCK_RELEASE, + BPF_CGROUP_INET4_BIND, + BPF_CGROUP_INET6_BIND, + BPF_CGROUP_INET4_POST_BIND, + BPF_CGROUP_INET6_POST_BIND, + BPF_CGROUP_INET4_CONNECT, + BPF_CGROUP_INET6_CONNECT, + BPF_CGROUP_UNIX_CONNECT, + BPF_CGROUP_INET4_GETPEERNAME, + BPF_CGROUP_INET6_GETPEERNAME, + BPF_CGROUP_UNIX_GETPEERNAME, + BPF_CGROUP_INET4_GETSOCKNAME, + BPF_CGROUP_INET6_GETSOCKNAME, + BPF_CGROUP_UNIX_GETSOCKNAME, + BPF_CGROUP_UDP4_SENDMSG, + BPF_CGROUP_UDP6_SENDMSG, + BPF_CGROUP_UNIX_SENDMSG, + BPF_CGROUP_UDP4_RECVMSG, + BPF_CGROUP_UDP6_RECVMSG, + BPF_CGROUP_UNIX_RECVMSG, + BPF_CGROUP_SOCK_OPS, + BPF_CGROUP_DEVICE, + BPF_CGROUP_SYSCTL, + BPF_CGROUP_GETSOCKOPT, + BPF_CGROUP_SETSOCKOPT, + BPF_LSM_CGROUP +}; + #define HELP_SPEC_ATTACH_FLAGS \ "ATTACH_FLAGS := { multi | override }" @@ -183,13 +215,13 @@ static int count_attached_bpf_progs(int cgroup_fd, enum bpf_attach_type type) static int cgroup_has_attached_progs(int cgroup_fd) { - enum bpf_attach_type type; + unsigned int i = 0; bool no_prog = true; - for (type = 0; type < __MAX_BPF_ATTACH_TYPE; type++) { - int count = count_attached_bpf_progs(cgroup_fd, type); + for (i = 0; i < ARRAY_SIZE(cgroup_attach_types); i++) { + int count = count_attached_bpf_progs(cgroup_fd, cgroup_attach_types[i]); - if (count < 0 && errno != EINVAL) + if (count < 0) return -1; if (count > 0) { diff --git a/tools/bpf/bpftool/common.c b/tools/bpf/bpftool/common.c index 958e92acca8e2476236573064fedf19b862adea8..9b75639434b8157360ef08f82648beeb6f4cd940 100644 --- a/tools/bpf/bpftool/common.c +++ b/tools/bpf/bpftool/common.c @@ -410,7 +410,7 @@ void get_prog_full_name(const struct bpf_prog_info *prog_info, int prog_fd, { const char *prog_name = prog_info->name; const struct btf_type *func_type; - const struct bpf_func_info finfo = {}; + struct bpf_func_info finfo = {}; struct bpf_prog_info info = {}; __u32 info_len = sizeof(info); struct btf *prog_btf = NULL; diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c index b3979ddc01895abb571d54ba2e2f2779f0b31b17..4a4eedfcd47920b313067d42d4b31db5deb0aadd 100644 --- a/tools/bpf/bpftool/gen.c +++ b/tools/bpf/bpftool/gen.c @@ -848,7 +848,7 @@ out: } static void -codegen_maps_skeleton(struct bpf_object *obj, size_t map_cnt, bool mmaped) +codegen_maps_skeleton(struct bpf_object *obj, size_t map_cnt, bool mmaped, bool populate_links) { struct bpf_map *map; char ident[256]; @@ -888,6 +888,14 @@ codegen_maps_skeleton(struct bpf_object *obj, size_t map_cnt, bool mmaped) printf("\ts->maps[%zu].mmaped = (void **)&obj->%s;\n", i, ident); } + + if (populate_links && bpf_map__type(map) == BPF_MAP_TYPE_STRUCT_OPS) { + codegen("\ + \n\ + s->maps[%zu].link = &obj->links.%s;\n\ + ", + i, ident); + } i++; } } @@ -1141,7 +1149,7 @@ static void gen_st_ops_shadow_init(struct btf *btf, struct bpf_object *obj) static int do_skeleton(int argc, char **argv) { char header_guard[MAX_OBJ_NAME_LEN + sizeof("__SKEL_H__")]; - size_t map_cnt = 0, prog_cnt = 0, file_sz, mmap_sz; + size_t map_cnt = 0, prog_cnt = 0, attach_map_cnt = 0, file_sz, mmap_sz; DECLARE_LIBBPF_OPTS(bpf_object_open_opts, opts); char obj_name[MAX_OBJ_NAME_LEN] = "", *obj_data; struct bpf_object *obj = NULL; @@ -1225,6 +1233,10 @@ static int do_skeleton(int argc, char **argv) bpf_map__name(map)); continue; } + + if (bpf_map__type(map) == BPF_MAP_TYPE_STRUCT_OPS) + attach_map_cnt++; + map_cnt++; } bpf_object__for_each_program(prog, obj) { @@ -1297,6 +1309,9 @@ static int do_skeleton(int argc, char **argv) bpf_program__name(prog)); } printf("\t} progs;\n"); + } + + if (prog_cnt + attach_map_cnt) { printf("\tstruct {\n"); bpf_object__for_each_program(prog, obj) { if (use_loader) @@ -1306,6 +1321,19 @@ static int do_skeleton(int argc, char **argv) printf("\t\tstruct bpf_link *%s;\n", bpf_program__name(prog)); } + + bpf_object__for_each_map(map, obj) { + if (!get_map_ident(map, ident, sizeof(ident))) + continue; + if (bpf_map__type(map) != BPF_MAP_TYPE_STRUCT_OPS) + continue; + + if (use_loader) + printf("t\tint %s_fd;\n", ident); + else + printf("\t\tstruct bpf_link *%s;\n", ident); + } + printf("\t} links;\n"); } @@ -1448,7 +1476,7 @@ static int do_skeleton(int argc, char **argv) obj_name ); - codegen_maps_skeleton(obj, map_cnt, true /*mmaped*/); + codegen_maps_skeleton(obj, map_cnt, true /*mmaped*/, true /*links*/); codegen_progs_skeleton(obj, prog_cnt, true /*populate_links*/); codegen("\ @@ -1786,7 +1814,7 @@ static int do_subskeleton(int argc, char **argv) } } - codegen_maps_skeleton(obj, map_cnt, false /*mmaped*/); + codegen_maps_skeleton(obj, map_cnt, false /*mmaped*/, false /*links*/); codegen_progs_skeleton(obj, prog_cnt, false /*links*/); codegen("\ @@ -2379,15 +2407,6 @@ out: return err; } -static int btfgen_remap_id(__u32 *type_id, void *ctx) -{ - unsigned int *ids = ctx; - - *type_id = ids[*type_id]; - - return 0; -} - /* Generate BTF from relocation information previously recorded */ static struct btf *btfgen_get_btf(struct btfgen_info *info) { @@ -2467,10 +2486,15 @@ static struct btf *btfgen_get_btf(struct btfgen_info *info) /* second pass: fix up type ids */ for (i = 1; i < btf__type_cnt(btf_new); i++) { struct btf_type *btf_type = (struct btf_type *) btf__type_by_id(btf_new, i); + struct btf_field_iter it; + __u32 *type_id; - err = btf_type_visit_type_ids(btf_type, btfgen_remap_id, ids); + err = btf_field_iter_init(&it, btf_type, BTF_FIELD_ITER_IDS); if (err) goto err_out; + + while ((type_id = btf_field_iter_next(&it))) + *type_id = ids[*type_id]; } free(ids); diff --git a/tools/bpf/bpftool/skeleton/pid_iter.bpf.c b/tools/bpf/bpftool/skeleton/pid_iter.bpf.c index 7bdbcac3cf628523d664f57bcb20f73a6dccbb39..948dde25034efefc2582509f701903d18fd1e5e4 100644 --- a/tools/bpf/bpftool/skeleton/pid_iter.bpf.c +++ b/tools/bpf/bpftool/skeleton/pid_iter.bpf.c @@ -29,6 +29,7 @@ enum bpf_link_type___local { }; extern const void bpf_link_fops __ksym; +extern const void bpf_link_fops_poll __ksym __weak; extern const void bpf_map_fops __ksym; extern const void bpf_prog_fops __ksym; extern const void btf_fops __ksym; @@ -84,7 +85,11 @@ int iter(struct bpf_iter__task_file *ctx) fops = &btf_fops; break; case BPF_OBJ_LINK: - fops = &bpf_link_fops; + if (&bpf_link_fops_poll && + file->f_op == &bpf_link_fops_poll) + fops = &bpf_link_fops_poll; + else + fops = &bpf_link_fops; break; default: return 0; diff --git a/tools/bpf/bpftool/skeleton/profiler.bpf.c b/tools/bpf/bpftool/skeleton/profiler.bpf.c index 2f80edc682f115e52b7377170c6ec358aadc5f8a..f48c783cb9f7f65edac48cc56055f5e31c02cf3b 100644 --- a/tools/bpf/bpftool/skeleton/profiler.bpf.c +++ b/tools/bpf/bpftool/skeleton/profiler.bpf.c @@ -40,17 +40,17 @@ struct { const volatile __u32 num_cpu = 1; const volatile __u32 num_metric = 1; -#define MAX_NUM_MATRICS 4 +#define MAX_NUM_METRICS 4 SEC("fentry/XXX") int BPF_PROG(fentry_XXX) { - struct bpf_perf_event_value___local *ptrs[MAX_NUM_MATRICS]; + struct bpf_perf_event_value___local *ptrs[MAX_NUM_METRICS]; u32 key = bpf_get_smp_processor_id(); u32 i; /* look up before reading, to reduce error */ - for (i = 0; i < num_metric && i < MAX_NUM_MATRICS; i++) { + for (i = 0; i < num_metric && i < MAX_NUM_METRICS; i++) { u32 flag = i; ptrs[i] = bpf_map_lookup_elem(&fentry_readings, &flag); @@ -58,7 +58,7 @@ int BPF_PROG(fentry_XXX) return 0; } - for (i = 0; i < num_metric && i < MAX_NUM_MATRICS; i++) { + for (i = 0; i < num_metric && i < MAX_NUM_METRICS; i++) { struct bpf_perf_event_value___local reading; int err; @@ -99,14 +99,14 @@ fexit_update_maps(u32 id, struct bpf_perf_event_value___local *after) SEC("fexit/XXX") int BPF_PROG(fexit_XXX) { - struct bpf_perf_event_value___local readings[MAX_NUM_MATRICS]; + struct bpf_perf_event_value___local readings[MAX_NUM_METRICS]; u32 cpu = bpf_get_smp_processor_id(); u32 i, zero = 0; int err; u64 *count; /* read all events before updating the maps, to reduce error */ - for (i = 0; i < num_metric && i < MAX_NUM_MATRICS; i++) { + for (i = 0; i < num_metric && i < MAX_NUM_METRICS; i++) { err = bpf_perf_event_read_value(&events, cpu + i * num_cpu, (void *)(readings + i), sizeof(*readings)); @@ -116,7 +116,7 @@ int BPF_PROG(fexit_XXX) count = bpf_map_lookup_elem(&counts, &zero); if (count) { *count += 1; - for (i = 0; i < num_metric && i < MAX_NUM_MATRICS; i++) + for (i = 0; i < num_metric && i < MAX_NUM_METRICS; i++) fexit_update_maps(i, &readings[i]); } return 0; diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c index d9520cb826b31f545b6604ff770c1c5891bcada0..936ef95c3d3236fec15ef9b0fa5432cfdf3dfe93 100644 --- a/tools/bpf/resolve_btfids/main.c +++ b/tools/bpf/resolve_btfids/main.c @@ -409,6 +409,14 @@ static int elf_collect(struct object *obj) obj->efile.idlist = data; obj->efile.idlist_shndx = idx; obj->efile.idlist_addr = sh.sh_addr; + } else if (!strcmp(name, BTF_BASE_ELF_SEC)) { + /* If a .BTF.base section is found, do not resolve + * BTF ids relative to vmlinux; resolve relative + * to the .BTF.base section instead. btf__parse_split() + * will take care of this once the base BTF it is + * passed is NULL. + */ + obj->base_btf_path = NULL; } if (compressed_section_fix(elf, scn, &sh)) @@ -728,7 +736,7 @@ static int sets_patch(struct object *obj) static int symbols_patch(struct object *obj) { - int err; + off_t err; if (__symbols_patch(obj, &obj->structs) || __symbols_patch(obj, &obj->unions) || diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 90706a47f6ffe974d346502c5511d9ac63a4f86e..35bcf52dbc6526469a933f67af8a2374cad88f50 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1425,6 +1425,8 @@ enum { #define BPF_F_TEST_RUN_ON_CPU (1U << 0) /* If set, XDP frames will be transmitted after processing */ #define BPF_F_TEST_XDP_LIVE_FRAMES (1U << 1) +/* If set, apply CHECKSUM_COMPLETE to skb and validate the checksum */ +#define BPF_F_TEST_SKB_CHECKSUM_COMPLETE (1U << 2) /* type for BPF_ENABLE_STATS */ enum bpf_stats_type { @@ -6207,12 +6209,17 @@ union { \ __u64 :64; \ } __attribute__((aligned(8))) +/* The enum used in skb->tstamp_type. It specifies the clock type + * of the time stored in the skb->tstamp. + */ enum { - BPF_SKB_TSTAMP_UNSPEC, - BPF_SKB_TSTAMP_DELIVERY_MONO, /* tstamp has mono delivery time */ - /* For any BPF_SKB_TSTAMP_* that the bpf prog cannot handle, - * the bpf prog should handle it like BPF_SKB_TSTAMP_UNSPEC - * and try to deduce it by ingress, egress or skb->sk->sk_clockid. + BPF_SKB_TSTAMP_UNSPEC = 0, /* DEPRECATED */ + BPF_SKB_TSTAMP_DELIVERY_MONO = 1, /* DEPRECATED */ + BPF_SKB_CLOCK_REALTIME = 0, + BPF_SKB_CLOCK_MONOTONIC = 1, + BPF_SKB_CLOCK_TAI = 2, + /* For any future BPF_SKB_CLOCK_* that the bpf prog cannot handle, + * the bpf prog can try to deduce it by ingress/egress/skb->sk->sk_clockid. */ }; diff --git a/tools/include/uapi/linux/netdev.h b/tools/include/uapi/linux/netdev.h index a8188202413ec4cab8ea0c1df92637a64750cb4e..43742ac5b00da0aa66660d86cb627561d8eaf298 100644 --- a/tools/include/uapi/linux/netdev.h +++ b/tools/include/uapi/linux/netdev.h @@ -148,6 +148,7 @@ enum { NETDEV_A_QSTATS_RX_ALLOC_FAIL, NETDEV_A_QSTATS_RX_HW_DROPS, NETDEV_A_QSTATS_RX_HW_DROP_OVERRUNS, + NETDEV_A_QSTATS_RX_CSUM_COMPLETE, NETDEV_A_QSTATS_RX_CSUM_UNNECESSARY, NETDEV_A_QSTATS_RX_CSUM_NONE, NETDEV_A_QSTATS_RX_CSUM_BAD, diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build index b6619199a7066a4c80153e4c9cb3ad98d63f5499..336da6844d425ec6a1b41b0445821dbd3544537f 100644 --- a/tools/lib/bpf/Build +++ b/tools/lib/bpf/Build @@ -1,4 +1,4 @@ libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \ netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \ btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \ - usdt.o zip.o elf.o features.o + usdt.o zip.o elf.o features.o btf_relocate.o diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 2d0840ef599aff7d54a1efab818e88e6dca2af2e..ef1b2f573c1bc3d84004e1c56bd257ba97d18a89 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -116,6 +116,9 @@ struct btf { /* whether strings are already deduplicated */ bool strs_deduped; + /* whether base_btf should be freed in btf_free for this instance */ + bool owns_base; + /* BTF object FD, if loaded into kernel */ int fd; @@ -969,6 +972,8 @@ void btf__free(struct btf *btf) free(btf->raw_data); free(btf->raw_data_swapped); free(btf->type_offs); + if (btf->owns_base) + btf__free(btf->base_btf); free(btf); } @@ -1084,53 +1089,38 @@ struct btf *btf__new_split(const void *data, __u32 size, struct btf *base_btf) return libbpf_ptr(btf_new(data, size, base_btf)); } -static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, - struct btf_ext **btf_ext) +struct btf_elf_secs { + Elf_Data *btf_data; + Elf_Data *btf_ext_data; + Elf_Data *btf_base_data; +}; + +static int btf_find_elf_sections(Elf *elf, const char *path, struct btf_elf_secs *secs) { - Elf_Data *btf_data = NULL, *btf_ext_data = NULL; - int err = 0, fd = -1, idx = 0; - struct btf *btf = NULL; Elf_Scn *scn = NULL; - Elf *elf = NULL; + Elf_Data *data; GElf_Ehdr ehdr; size_t shstrndx; + int idx = 0; - if (elf_version(EV_CURRENT) == EV_NONE) { - pr_warn("failed to init libelf for %s\n", path); - return ERR_PTR(-LIBBPF_ERRNO__LIBELF); - } - - fd = open(path, O_RDONLY | O_CLOEXEC); - if (fd < 0) { - err = -errno; - pr_warn("failed to open %s: %s\n", path, strerror(errno)); - return ERR_PTR(err); - } - - err = -LIBBPF_ERRNO__FORMAT; - - elf = elf_begin(fd, ELF_C_READ, NULL); - if (!elf) { - pr_warn("failed to open %s as ELF file\n", path); - goto done; - } if (!gelf_getehdr(elf, &ehdr)) { pr_warn("failed to get EHDR from %s\n", path); - goto done; + goto err; } if (elf_getshdrstrndx(elf, &shstrndx)) { pr_warn("failed to get section names section index for %s\n", path); - goto done; + goto err; } if (!elf_rawdata(elf_getscn(elf, shstrndx), NULL)) { pr_warn("failed to get e_shstrndx from %s\n", path); - goto done; + goto err; } while ((scn = elf_nextscn(elf, scn)) != NULL) { + Elf_Data **field; GElf_Shdr sh; char *name; @@ -1138,42 +1128,102 @@ static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, if (gelf_getshdr(scn, &sh) != &sh) { pr_warn("failed to get section(%d) header from %s\n", idx, path); - goto done; + goto err; } name = elf_strptr(elf, shstrndx, sh.sh_name); if (!name) { pr_warn("failed to get section(%d) name from %s\n", idx, path); - goto done; + goto err; } - if (strcmp(name, BTF_ELF_SEC) == 0) { - btf_data = elf_getdata(scn, 0); - if (!btf_data) { - pr_warn("failed to get section(%d, %s) data from %s\n", - idx, name, path); - goto done; - } - continue; - } else if (btf_ext && strcmp(name, BTF_EXT_ELF_SEC) == 0) { - btf_ext_data = elf_getdata(scn, 0); - if (!btf_ext_data) { - pr_warn("failed to get section(%d, %s) data from %s\n", - idx, name, path); - goto done; - } + + if (strcmp(name, BTF_ELF_SEC) == 0) + field = &secs->btf_data; + else if (strcmp(name, BTF_EXT_ELF_SEC) == 0) + field = &secs->btf_ext_data; + else if (strcmp(name, BTF_BASE_ELF_SEC) == 0) + field = &secs->btf_base_data; + else continue; + + data = elf_getdata(scn, 0); + if (!data) { + pr_warn("failed to get section(%d, %s) data from %s\n", + idx, name, path); + goto err; } + *field = data; + } + + return 0; + +err: + return -LIBBPF_ERRNO__FORMAT; +} + +static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, + struct btf_ext **btf_ext) +{ + struct btf_elf_secs secs = {}; + struct btf *dist_base_btf = NULL; + struct btf *btf = NULL; + int err = 0, fd = -1; + Elf *elf = NULL; + + if (elf_version(EV_CURRENT) == EV_NONE) { + pr_warn("failed to init libelf for %s\n", path); + return ERR_PTR(-LIBBPF_ERRNO__LIBELF); + } + + fd = open(path, O_RDONLY | O_CLOEXEC); + if (fd < 0) { + err = -errno; + pr_warn("failed to open %s: %s\n", path, strerror(errno)); + return ERR_PTR(err); + } + + elf = elf_begin(fd, ELF_C_READ, NULL); + if (!elf) { + pr_warn("failed to open %s as ELF file\n", path); + goto done; } - if (!btf_data) { + err = btf_find_elf_sections(elf, path, &secs); + if (err) + goto done; + + if (!secs.btf_data) { pr_warn("failed to find '%s' ELF section in %s\n", BTF_ELF_SEC, path); err = -ENODATA; goto done; } - btf = btf_new(btf_data->d_buf, btf_data->d_size, base_btf); - err = libbpf_get_error(btf); - if (err) + + if (secs.btf_base_data) { + dist_base_btf = btf_new(secs.btf_base_data->d_buf, secs.btf_base_data->d_size, + NULL); + if (IS_ERR(dist_base_btf)) { + err = PTR_ERR(dist_base_btf); + dist_base_btf = NULL; + goto done; + } + } + + btf = btf_new(secs.btf_data->d_buf, secs.btf_data->d_size, + dist_base_btf ?: base_btf); + if (IS_ERR(btf)) { + err = PTR_ERR(btf); goto done; + } + if (dist_base_btf && base_btf) { + err = btf__relocate(btf, base_btf); + if (err) + goto done; + btf__free(dist_base_btf); + dist_base_btf = NULL; + } + + if (dist_base_btf) + btf->owns_base = true; switch (gelf_getclass(elf)) { case ELFCLASS32: @@ -1187,11 +1237,12 @@ static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, break; } - if (btf_ext && btf_ext_data) { - *btf_ext = btf_ext__new(btf_ext_data->d_buf, btf_ext_data->d_size); - err = libbpf_get_error(*btf_ext); - if (err) + if (btf_ext && secs.btf_ext_data) { + *btf_ext = btf_ext__new(secs.btf_ext_data->d_buf, secs.btf_ext_data->d_size); + if (IS_ERR(*btf_ext)) { + err = PTR_ERR(*btf_ext); goto done; + } } else if (btf_ext) { *btf_ext = NULL; } @@ -1205,6 +1256,7 @@ done: if (btf_ext) btf_ext__free(*btf_ext); + btf__free(dist_base_btf); btf__free(btf); return ERR_PTR(err); @@ -1739,9 +1791,8 @@ struct btf_pipe { struct hashmap *str_off_map; /* map string offsets from src to dst */ }; -static int btf_rewrite_str(__u32 *str_off, void *ctx) +static int btf_rewrite_str(struct btf_pipe *p, __u32 *str_off) { - struct btf_pipe *p = ctx; long mapped_off; int off, err; @@ -1771,10 +1822,11 @@ static int btf_rewrite_str(__u32 *str_off, void *ctx) return 0; } -int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_type *src_type) +static int btf_add_type(struct btf_pipe *p, const struct btf_type *src_type) { - struct btf_pipe p = { .src = src_btf, .dst = btf }; + struct btf_field_iter it; struct btf_type *t; + __u32 *str_off; int sz, err; sz = btf_type_size(src_type); @@ -1782,35 +1834,33 @@ int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_t return libbpf_err(sz); /* deconstruct BTF, if necessary, and invalidate raw_data */ - if (btf_ensure_modifiable(btf)) + if (btf_ensure_modifiable(p->dst)) return libbpf_err(-ENOMEM); - t = btf_add_type_mem(btf, sz); + t = btf_add_type_mem(p->dst, sz); if (!t) return libbpf_err(-ENOMEM); memcpy(t, src_type, sz); - err = btf_type_visit_str_offs(t, btf_rewrite_str, &p); + err = btf_field_iter_init(&it, t, BTF_FIELD_ITER_STRS); if (err) return libbpf_err(err); - return btf_commit_type(btf, sz); + while ((str_off = btf_field_iter_next(&it))) { + err = btf_rewrite_str(p, str_off); + if (err) + return libbpf_err(err); + } + + return btf_commit_type(p->dst, sz); } -static int btf_rewrite_type_ids(__u32 *type_id, void *ctx) +int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_type *src_type) { - struct btf *btf = ctx; - - if (!*type_id) /* nothing to do for VOID references */ - return 0; + struct btf_pipe p = { .src = src_btf, .dst = btf }; - /* we haven't updated btf's type count yet, so - * btf->start_id + btf->nr_types - 1 is the type ID offset we should - * add to all newly added BTF types - */ - *type_id += btf->start_id + btf->nr_types - 1; - return 0; + return btf_add_type(&p, src_type); } static size_t btf_dedup_identity_hash_fn(long key, void *ctx); @@ -1858,6 +1908,9 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf) memcpy(t, src_btf->types_data, data_sz); for (i = 0; i < cnt; i++) { + struct btf_field_iter it; + __u32 *type_id, *str_off; + sz = btf_type_size(t); if (sz < 0) { /* unlikely, has to be corrupted src_btf */ @@ -1869,15 +1922,31 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf) *off = t - btf->types_data; /* add, dedup, and remap strings referenced by this BTF type */ - err = btf_type_visit_str_offs(t, btf_rewrite_str, &p); + err = btf_field_iter_init(&it, t, BTF_FIELD_ITER_STRS); if (err) goto err_out; + while ((str_off = btf_field_iter_next(&it))) { + err = btf_rewrite_str(&p, str_off); + if (err) + goto err_out; + } /* remap all type IDs referenced from this BTF type */ - err = btf_type_visit_type_ids(t, btf_rewrite_type_ids, btf); + err = btf_field_iter_init(&it, t, BTF_FIELD_ITER_IDS); if (err) goto err_out; + while ((type_id = btf_field_iter_next(&it))) { + if (!*type_id) /* nothing to do for VOID references */ + continue; + + /* we haven't updated btf's type count yet, so + * btf->start_id + btf->nr_types - 1 is the type ID offset we should + * add to all newly added BTF types + */ + *type_id += btf->start_id + btf->nr_types - 1; + } + /* go to next type data and type offset index entry */ t += sz; off++; @@ -3453,11 +3522,19 @@ static int btf_for_each_str_off(struct btf_dedup *d, str_off_visit_fn fn, void * int i, r; for (i = 0; i < d->btf->nr_types; i++) { + struct btf_field_iter it; struct btf_type *t = btf_type_by_id(d->btf, d->btf->start_id + i); + __u32 *str_off; - r = btf_type_visit_str_offs(t, fn, ctx); + r = btf_field_iter_init(&it, t, BTF_FIELD_ITER_STRS); if (r) return r; + + while ((str_off = btf_field_iter_next(&it))) { + r = fn(str_off, ctx); + if (r) + return r; + } } if (!d->btf_ext) @@ -4919,10 +4996,23 @@ static int btf_dedup_remap_types(struct btf_dedup *d) for (i = 0; i < d->btf->nr_types; i++) { struct btf_type *t = btf_type_by_id(d->btf, d->btf->start_id + i); + struct btf_field_iter it; + __u32 *type_id; - r = btf_type_visit_type_ids(t, btf_dedup_remap_type_id, d); + r = btf_field_iter_init(&it, t, BTF_FIELD_ITER_IDS); if (r) return r; + + while ((type_id = btf_field_iter_next(&it))) { + __u32 resolved_id, new_id; + + resolved_id = resolve_type_id(d, *type_id); + new_id = d->hypot_map[resolved_id]; + if (new_id > BTF_MAX_NR_TYPES) + return -EINVAL; + + *type_id = new_id; + } } if (!d->btf_ext) @@ -5003,134 +5093,166 @@ struct btf *btf__load_module_btf(const char *module_name, struct btf *vmlinux_bt return btf__parse_split(path, vmlinux_btf); } -int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx) +int btf_field_iter_init(struct btf_field_iter *it, struct btf_type *t, enum btf_field_iter_kind iter_kind) { - int i, n, err; - - switch (btf_kind(t)) { - case BTF_KIND_INT: - case BTF_KIND_FLOAT: - case BTF_KIND_ENUM: - case BTF_KIND_ENUM64: - return 0; + it->p = NULL; + it->m_idx = -1; + it->off_idx = 0; + it->vlen = 0; - case BTF_KIND_FWD: - case BTF_KIND_CONST: - case BTF_KIND_VOLATILE: - case BTF_KIND_RESTRICT: - case BTF_KIND_PTR: - case BTF_KIND_TYPEDEF: - case BTF_KIND_FUNC: - case BTF_KIND_VAR: - case BTF_KIND_DECL_TAG: - case BTF_KIND_TYPE_TAG: - return visit(&t->type, ctx); - - case BTF_KIND_ARRAY: { - struct btf_array *a = btf_array(t); - - err = visit(&a->type, ctx); - err = err ?: visit(&a->index_type, ctx); - return err; - } - - case BTF_KIND_STRUCT: - case BTF_KIND_UNION: { - struct btf_member *m = btf_members(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; - } - return 0; - } - - case BTF_KIND_FUNC_PROTO: { - struct btf_param *m = btf_params(t); - - err = visit(&t->type, ctx); - if (err) - return err; - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; + switch (iter_kind) { + case BTF_FIELD_ITER_IDS: + switch (btf_kind(t)) { + case BTF_KIND_UNKN: + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + it->desc = (struct btf_field_desc) {}; + break; + case BTF_KIND_FWD: + case BTF_KIND_CONST: + case BTF_KIND_VOLATILE: + case BTF_KIND_RESTRICT: + case BTF_KIND_PTR: + case BTF_KIND_TYPEDEF: + case BTF_KIND_FUNC: + case BTF_KIND_VAR: + case BTF_KIND_DECL_TAG: + case BTF_KIND_TYPE_TAG: + it->desc = (struct btf_field_desc) { 1, {offsetof(struct btf_type, type)} }; + break; + case BTF_KIND_ARRAY: + it->desc = (struct btf_field_desc) { + 2, {sizeof(struct btf_type) + offsetof(struct btf_array, type), + sizeof(struct btf_type) + offsetof(struct btf_array, index_type)} + }; + break; + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + it->desc = (struct btf_field_desc) { + 0, {}, + sizeof(struct btf_member), + 1, {offsetof(struct btf_member, type)} + }; + break; + case BTF_KIND_FUNC_PROTO: + it->desc = (struct btf_field_desc) { + 1, {offsetof(struct btf_type, type)}, + sizeof(struct btf_param), + 1, {offsetof(struct btf_param, type)} + }; + break; + case BTF_KIND_DATASEC: + it->desc = (struct btf_field_desc) { + 0, {}, + sizeof(struct btf_var_secinfo), + 1, {offsetof(struct btf_var_secinfo, type)} + }; + break; + default: + return -EINVAL; } - return 0; - } - - case BTF_KIND_DATASEC: { - struct btf_var_secinfo *m = btf_var_secinfos(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; + break; + case BTF_FIELD_ITER_STRS: + switch (btf_kind(t)) { + case BTF_KIND_UNKN: + it->desc = (struct btf_field_desc) {}; + break; + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_FWD: + case BTF_KIND_ARRAY: + case BTF_KIND_CONST: + case BTF_KIND_VOLATILE: + case BTF_KIND_RESTRICT: + case BTF_KIND_PTR: + case BTF_KIND_TYPEDEF: + case BTF_KIND_FUNC: + case BTF_KIND_VAR: + case BTF_KIND_DECL_TAG: + case BTF_KIND_TYPE_TAG: + case BTF_KIND_DATASEC: + it->desc = (struct btf_field_desc) { + 1, {offsetof(struct btf_type, name_off)} + }; + break; + case BTF_KIND_ENUM: + it->desc = (struct btf_field_desc) { + 1, {offsetof(struct btf_type, name_off)}, + sizeof(struct btf_enum), + 1, {offsetof(struct btf_enum, name_off)} + }; + break; + case BTF_KIND_ENUM64: + it->desc = (struct btf_field_desc) { + 1, {offsetof(struct btf_type, name_off)}, + sizeof(struct btf_enum64), + 1, {offsetof(struct btf_enum64, name_off)} + }; + break; + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + it->desc = (struct btf_field_desc) { + 1, {offsetof(struct btf_type, name_off)}, + sizeof(struct btf_member), + 1, {offsetof(struct btf_member, name_off)} + }; + break; + case BTF_KIND_FUNC_PROTO: + it->desc = (struct btf_field_desc) { + 1, {offsetof(struct btf_type, name_off)}, + sizeof(struct btf_param), + 1, {offsetof(struct btf_param, name_off)} + }; + break; + default: + return -EINVAL; } - return 0; - } - + break; default: return -EINVAL; } -} -int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx) -{ - int i, n, err; + if (it->desc.m_sz) + it->vlen = btf_vlen(t); - err = visit(&t->name_off, ctx); - if (err) - return err; + it->p = t; + return 0; +} - switch (btf_kind(t)) { - case BTF_KIND_STRUCT: - case BTF_KIND_UNION: { - struct btf_member *m = btf_members(t); +__u32 *btf_field_iter_next(struct btf_field_iter *it) +{ + if (!it->p) + return NULL; - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; + if (it->m_idx < 0) { + if (it->off_idx < it->desc.t_off_cnt) + return it->p + it->desc.t_offs[it->off_idx++]; + /* move to per-member iteration */ + it->m_idx = 0; + it->p += sizeof(struct btf_type); + it->off_idx = 0; } - case BTF_KIND_ENUM: { - struct btf_enum *m = btf_enum(t); - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; + /* if type doesn't have members, stop */ + if (it->desc.m_sz == 0) { + it->p = NULL; + return NULL; } - case BTF_KIND_ENUM64: { - struct btf_enum64 *m = btf_enum64(t); - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; + if (it->off_idx >= it->desc.m_off_cnt) { + /* exhausted this member's fields, go to the next member */ + it->m_idx++; + it->p += it->desc.m_sz; + it->off_idx = 0; } - case BTF_KIND_FUNC_PROTO: { - struct btf_param *m = btf_params(t); - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - default: - break; - } + if (it->m_idx < it->vlen) + return it->p + it->desc.m_offs[it->off_idx++]; - return 0; + it->p = NULL; + return NULL; } int btf_ext_visit_type_ids(struct btf_ext *btf_ext, type_id_visit_fn visit, void *ctx) @@ -5212,3 +5334,325 @@ int btf_ext_visit_str_offs(struct btf_ext *btf_ext, str_off_visit_fn visit, void return 0; } + +struct btf_distill { + struct btf_pipe pipe; + int *id_map; + unsigned int split_start_id; + unsigned int split_start_str; + int diff_id; +}; + +static int btf_add_distilled_type_ids(struct btf_distill *dist, __u32 i) +{ + struct btf_type *split_t = btf_type_by_id(dist->pipe.src, i); + struct btf_field_iter it; + __u32 *id; + int err; + + err = btf_field_iter_init(&it, split_t, BTF_FIELD_ITER_IDS); + if (err) + return err; + while ((id = btf_field_iter_next(&it))) { + struct btf_type *base_t; + + if (!*id) + continue; + /* split BTF id, not needed */ + if (*id >= dist->split_start_id) + continue; + /* already added ? */ + if (dist->id_map[*id] > 0) + continue; + + /* only a subset of base BTF types should be referenced from + * split BTF; ensure nothing unexpected is referenced. + */ + base_t = btf_type_by_id(dist->pipe.src, *id); + switch (btf_kind(base_t)) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_FWD: + case BTF_KIND_ARRAY: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_TYPEDEF: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + case BTF_KIND_PTR: + case BTF_KIND_CONST: + case BTF_KIND_RESTRICT: + case BTF_KIND_VOLATILE: + case BTF_KIND_FUNC_PROTO: + case BTF_KIND_TYPE_TAG: + dist->id_map[*id] = *id; + break; + default: + pr_warn("unexpected reference to base type[%u] of kind [%u] when creating distilled base BTF.\n", + *id, btf_kind(base_t)); + return -EINVAL; + } + /* If a base type is used, ensure types it refers to are + * marked as used also; so for example if we find a PTR to INT + * we need both the PTR and INT. + * + * The only exception is named struct/unions, since distilled + * base BTF composite types have no members. + */ + if (btf_is_composite(base_t) && base_t->name_off) + continue; + err = btf_add_distilled_type_ids(dist, *id); + if (err) + return err; + } + return 0; +} + +static int btf_add_distilled_types(struct btf_distill *dist) +{ + bool adding_to_base = dist->pipe.dst->start_id == 1; + int id = btf__type_cnt(dist->pipe.dst); + struct btf_type *t; + int i, err = 0; + + + /* Add types for each of the required references to either distilled + * base or split BTF, depending on type characteristics. + */ + for (i = 1; i < dist->split_start_id; i++) { + const char *name; + int kind; + + if (!dist->id_map[i]) + continue; + t = btf_type_by_id(dist->pipe.src, i); + kind = btf_kind(t); + name = btf__name_by_offset(dist->pipe.src, t->name_off); + + switch (kind) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_FWD: + /* Named int, float, fwd are added to base. */ + if (!adding_to_base) + continue; + err = btf_add_type(&dist->pipe, t); + break; + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + /* Named struct/union are added to base as 0-vlen + * struct/union of same size. Anonymous struct/unions + * are added to split BTF as-is. + */ + if (adding_to_base) { + if (!t->name_off) + continue; + err = btf_add_composite(dist->pipe.dst, kind, name, t->size); + } else { + if (t->name_off) + continue; + err = btf_add_type(&dist->pipe, t); + } + break; + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + /* Named enum[64]s are added to base as a sized + * enum; relocation will match with appropriately-named + * and sized enum or enum64. + * + * Anonymous enums are added to split BTF as-is. + */ + if (adding_to_base) { + if (!t->name_off) + continue; + err = btf__add_enum(dist->pipe.dst, name, t->size); + } else { + if (t->name_off) + continue; + err = btf_add_type(&dist->pipe, t); + } + break; + case BTF_KIND_ARRAY: + case BTF_KIND_TYPEDEF: + case BTF_KIND_PTR: + case BTF_KIND_CONST: + case BTF_KIND_RESTRICT: + case BTF_KIND_VOLATILE: + case BTF_KIND_FUNC_PROTO: + case BTF_KIND_TYPE_TAG: + /* All other types are added to split BTF. */ + if (adding_to_base) + continue; + err = btf_add_type(&dist->pipe, t); + break; + default: + pr_warn("unexpected kind when adding base type '%s'[%u] of kind [%u] to distilled base BTF.\n", + name, i, kind); + return -EINVAL; + + } + if (err < 0) + break; + dist->id_map[i] = id++; + } + return err; +} + +/* Split BTF ids without a mapping will be shifted downwards since distilled + * base BTF is smaller than the original base BTF. For those that have a + * mapping (either to base or updated split BTF), update the id based on + * that mapping. + */ +static int btf_update_distilled_type_ids(struct btf_distill *dist, __u32 i) +{ + struct btf_type *t = btf_type_by_id(dist->pipe.dst, i); + struct btf_field_iter it; + __u32 *id; + int err; + + err = btf_field_iter_init(&it, t, BTF_FIELD_ITER_IDS); + if (err) + return err; + while ((id = btf_field_iter_next(&it))) { + if (dist->id_map[*id]) + *id = dist->id_map[*id]; + else if (*id >= dist->split_start_id) + *id -= dist->diff_id; + } + return 0; +} + +/* Create updated split BTF with distilled base BTF; distilled base BTF + * consists of BTF information required to clarify the types that split + * BTF refers to, omitting unneeded details. Specifically it will contain + * base types and memberless definitions of named structs, unions and enumerated + * types. Associated reference types like pointers, arrays and anonymous + * structs, unions and enumerated types will be added to split BTF. + * Size is recorded for named struct/unions to help guide matching to the + * target base BTF during later relocation. + * + * The only case where structs, unions or enumerated types are fully represented + * is when they are anonymous; in such cases, the anonymous type is added to + * split BTF in full. + * + * We return newly-created split BTF where the split BTF refers to a newly-created + * distilled base BTF. Both must be freed separately by the caller. + */ +int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf, + struct btf **new_split_btf) +{ + struct btf *new_base = NULL, *new_split = NULL; + const struct btf *old_base; + unsigned int n = btf__type_cnt(src_btf); + struct btf_distill dist = {}; + struct btf_type *t; + int i, err = 0; + + /* src BTF must be split BTF. */ + old_base = btf__base_btf(src_btf); + if (!new_base_btf || !new_split_btf || !old_base) + return libbpf_err(-EINVAL); + + new_base = btf__new_empty(); + if (!new_base) + return libbpf_err(-ENOMEM); + dist.id_map = calloc(n, sizeof(*dist.id_map)); + if (!dist.id_map) { + err = -ENOMEM; + goto done; + } + dist.pipe.src = src_btf; + dist.pipe.dst = new_base; + dist.pipe.str_off_map = hashmap__new(btf_dedup_identity_hash_fn, btf_dedup_equal_fn, NULL); + if (IS_ERR(dist.pipe.str_off_map)) { + err = -ENOMEM; + goto done; + } + dist.split_start_id = btf__type_cnt(old_base); + dist.split_start_str = old_base->hdr->str_len; + + /* Pass over src split BTF; generate the list of base BTF type ids it + * references; these will constitute our distilled BTF set to be + * distributed over base and split BTF as appropriate. + */ + for (i = src_btf->start_id; i < n; i++) { + err = btf_add_distilled_type_ids(&dist, i); + if (err < 0) + goto done; + } + /* Next add types for each of the required references to base BTF and split BTF + * in turn. + */ + err = btf_add_distilled_types(&dist); + if (err < 0) + goto done; + + /* Create new split BTF with distilled base BTF as its base; the final + * state is split BTF with distilled base BTF that represents enough + * about its base references to allow it to be relocated with the base + * BTF available. + */ + new_split = btf__new_empty_split(new_base); + if (!new_split_btf) { + err = -errno; + goto done; + } + dist.pipe.dst = new_split; + /* First add all split types */ + for (i = src_btf->start_id; i < n; i++) { + t = btf_type_by_id(src_btf, i); + err = btf_add_type(&dist.pipe, t); + if (err < 0) + goto done; + } + /* Now add distilled types to split BTF that are not added to base. */ + err = btf_add_distilled_types(&dist); + if (err < 0) + goto done; + + /* All split BTF ids will be shifted downwards since there are less base + * BTF ids in distilled base BTF. + */ + dist.diff_id = dist.split_start_id - btf__type_cnt(new_base); + + n = btf__type_cnt(new_split); + /* Now update base/split BTF ids. */ + for (i = 1; i < n; i++) { + err = btf_update_distilled_type_ids(&dist, i); + if (err < 0) + break; + } +done: + free(dist.id_map); + hashmap__free(dist.pipe.str_off_map); + if (err) { + btf__free(new_split); + btf__free(new_base); + return libbpf_err(err); + } + *new_base_btf = new_base; + *new_split_btf = new_split; + + return 0; +} + +const struct btf_header *btf_header(const struct btf *btf) +{ + return btf->hdr; +} + +void btf_set_base_btf(struct btf *btf, const struct btf *base_btf) +{ + btf->base_btf = (struct btf *)base_btf; + btf->start_id = btf__type_cnt(base_btf); + btf->start_str_off = base_btf->hdr->str_len; +} + +int btf__relocate(struct btf *btf, const struct btf *base_btf) +{ + int err = btf_relocate(btf, base_btf, NULL); + + if (!err) + btf->owns_base = false; + return libbpf_err(err); +} diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index 8e6880d91c8408261f2fc2071a4376ca279829a1..b68d216837a92470f2da32809402329ef48c7f44 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -18,6 +18,7 @@ extern "C" { #define BTF_ELF_SEC ".BTF" #define BTF_EXT_ELF_SEC ".BTF.ext" +#define BTF_BASE_ELF_SEC ".BTF.base" #define MAPS_ELF_SEC ".maps" struct btf; @@ -107,6 +108,27 @@ LIBBPF_API struct btf *btf__new_empty(void); */ LIBBPF_API struct btf *btf__new_empty_split(struct btf *base_btf); +/** + * @brief **btf__distill_base()** creates new versions of the split BTF + * *src_btf* and its base BTF. The new base BTF will only contain the types + * needed to improve robustness of the split BTF to small changes in base BTF. + * When that split BTF is loaded against a (possibly changed) base, this + * distilled base BTF will help update references to that (possibly changed) + * base BTF. + * + * Both the new split and its associated new base BTF must be freed by + * the caller. + * + * If successful, 0 is returned and **new_base_btf** and **new_split_btf** + * will point at new base/split BTF. Both the new split and its associated + * new base BTF must be freed by the caller. + * + * A negative value is returned on error and the thread-local `errno` variable + * is set to the error code as well. + */ +LIBBPF_API int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf, + struct btf **new_split_btf); + LIBBPF_API struct btf *btf__parse(const char *path, struct btf_ext **btf_ext); LIBBPF_API struct btf *btf__parse_split(const char *path, struct btf *base_btf); LIBBPF_API struct btf *btf__parse_elf(const char *path, struct btf_ext **btf_ext); @@ -231,6 +253,20 @@ struct btf_dedup_opts { LIBBPF_API int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts); +/** + * @brief **btf__relocate()** will check the split BTF *btf* for references + * to base BTF kinds, and verify those references are compatible with + * *base_btf*; if they are, *btf* is adjusted such that is re-parented to + * *base_btf* and type ids and strings are adjusted to accommodate this. + * + * If successful, 0 is returned and **btf** now has **base_btf** as its + * base. + * + * A negative value is returned on error and the thread-local `errno` variable + * is set to the error code as well. + */ +LIBBPF_API int btf__relocate(struct btf *btf, const struct btf *base_btf); + struct btf_dump; struct btf_dump_opts { diff --git a/tools/lib/bpf/btf_relocate.c b/tools/lib/bpf/btf_relocate.c new file mode 100644 index 0000000000000000000000000000000000000000..eabb8755f6625a771ed6718a7c184e767c757ba2 --- /dev/null +++ b/tools/lib/bpf/btf_relocate.c @@ -0,0 +1,506 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif + +#include "btf.h" +#include "bpf.h" +#include "libbpf.h" +#include "libbpf_internal.h" + +struct btf; + +struct btf_relocate { + struct btf *btf; + const struct btf *base_btf; + const struct btf *dist_base_btf; + unsigned int nr_base_types; + unsigned int nr_split_types; + unsigned int nr_dist_base_types; + int dist_str_len; + int base_str_len; + __u32 *id_map; + __u32 *str_map; +}; + +/* Set temporarily in relocation id_map if distilled base struct/union is + * embedded in a split BTF struct/union; in such a case, size information must + * match between distilled base BTF and base BTF representation of type. + */ +#define BTF_IS_EMBEDDED ((__u32)-1) + +/* triple used in sorting/searching distilled base BTF. */ +struct btf_name_info { + const char *name; + /* set when search requires a size match */ + int needs_size:1, + size:31; + __u32 id; +}; + +static int btf_relocate_rewrite_type_id(struct btf_relocate *r, __u32 i) +{ + struct btf_type *t = btf_type_by_id(r->btf, i); + struct btf_field_iter it; + __u32 *id; + int err; + + err = btf_field_iter_init(&it, t, BTF_FIELD_ITER_IDS); + if (err) + return err; + + while ((id = btf_field_iter_next(&it))) + *id = r->id_map[*id]; + return 0; +} + +/* Simple string comparison used for sorting within BTF, since all distilled + * types are named. If strings match, and size is non-zero for both elements + * fall back to using size for ordering. + */ +static int cmp_btf_name_size(const void *n1, const void *n2) +{ + const struct btf_name_info *ni1 = n1; + const struct btf_name_info *ni2 = n2; + int name_diff = strcmp(ni1->name, ni2->name); + + if (!name_diff && ni1->needs_size && ni2->needs_size) + return ni2->size - ni1->size; + return name_diff; +} + +/* Binary search with a small twist; find leftmost element that matches + * so that we can then iterate through all exact matches. So for example + * searching { "a", "bb", "bb", "c" } we would always match on the + * leftmost "bb". + */ +static struct btf_name_info *search_btf_name_size(struct btf_name_info *key, + struct btf_name_info *vals, + int nelems) +{ + struct btf_name_info *ret = NULL; + int high = nelems - 1; + int low = 0; + + while (low <= high) { + int mid = (low + high)/2; + struct btf_name_info *val = &vals[mid]; + int diff = cmp_btf_name_size(key, val); + + if (diff == 0) + ret = val; + /* even if found, keep searching for leftmost match */ + if (diff <= 0) + high = mid - 1; + else + low = mid + 1; + } + return ret; +} + +/* If a member of a split BTF struct/union refers to a base BTF + * struct/union, mark that struct/union id temporarily in the id_map + * with BTF_IS_EMBEDDED. Members can be const/restrict/volatile/typedef + * reference types, but if a pointer is encountered, the type is no longer + * considered embedded. + */ +static int btf_mark_embedded_composite_type_ids(struct btf_relocate *r, __u32 i) +{ + struct btf_type *t = btf_type_by_id(r->btf, i); + struct btf_field_iter it; + __u32 *id; + int err; + + if (!btf_is_composite(t)) + return 0; + + err = btf_field_iter_init(&it, t, BTF_FIELD_ITER_IDS); + if (err) + return err; + + while ((id = btf_field_iter_next(&it))) { + __u32 next_id = *id; + + while (next_id) { + t = btf_type_by_id(r->btf, next_id); + switch (btf_kind(t)) { + case BTF_KIND_CONST: + case BTF_KIND_RESTRICT: + case BTF_KIND_VOLATILE: + case BTF_KIND_TYPEDEF: + case BTF_KIND_TYPE_TAG: + next_id = t->type; + break; + case BTF_KIND_ARRAY: { + struct btf_array *a = btf_array(t); + + next_id = a->type; + break; + } + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + if (next_id < r->nr_dist_base_types) + r->id_map[next_id] = BTF_IS_EMBEDDED; + next_id = 0; + break; + default: + next_id = 0; + break; + } + } + } + + return 0; +} + +/* Build a map from distilled base BTF ids to base BTF ids. To do so, iterate + * through base BTF looking up distilled type (using binary search) equivalents. + */ +static int btf_relocate_map_distilled_base(struct btf_relocate *r) +{ + struct btf_name_info *dist_base_info_sorted, *dist_base_info_sorted_end; + struct btf_type *base_t, *dist_t; + __u8 *base_name_cnt = NULL; + int err = 0; + __u32 id; + + /* generate a sort index array of name/type ids sorted by name for + * distilled base BTF to speed name-based lookups. + */ + dist_base_info_sorted = calloc(r->nr_dist_base_types, sizeof(*dist_base_info_sorted)); + if (!dist_base_info_sorted) { + err = -ENOMEM; + goto done; + } + dist_base_info_sorted_end = dist_base_info_sorted + r->nr_dist_base_types; + for (id = 0; id < r->nr_dist_base_types; id++) { + dist_t = btf_type_by_id(r->dist_base_btf, id); + dist_base_info_sorted[id].name = btf__name_by_offset(r->dist_base_btf, + dist_t->name_off); + dist_base_info_sorted[id].id = id; + dist_base_info_sorted[id].size = dist_t->size; + dist_base_info_sorted[id].needs_size = true; + } + qsort(dist_base_info_sorted, r->nr_dist_base_types, sizeof(*dist_base_info_sorted), + cmp_btf_name_size); + + /* Mark distilled base struct/union members of split BTF structs/unions + * in id_map with BTF_IS_EMBEDDED; this signals that these types + * need to match both name and size, otherwise embeddding the base + * struct/union in the split type is invalid. + */ + for (id = r->nr_dist_base_types; id < r->nr_split_types; id++) { + err = btf_mark_embedded_composite_type_ids(r, id); + if (err) + goto done; + } + + /* Collect name counts for composite types in base BTF. If multiple + * instances of a struct/union of the same name exist, we need to use + * size to determine which to map to since name alone is ambiguous. + */ + base_name_cnt = calloc(r->base_str_len, sizeof(*base_name_cnt)); + if (!base_name_cnt) { + err = -ENOMEM; + goto done; + } + for (id = 1; id < r->nr_base_types; id++) { + base_t = btf_type_by_id(r->base_btf, id); + if (!btf_is_composite(base_t) || !base_t->name_off) + continue; + if (base_name_cnt[base_t->name_off] < 255) + base_name_cnt[base_t->name_off]++; + } + + /* Now search base BTF for matching distilled base BTF types. */ + for (id = 1; id < r->nr_base_types; id++) { + struct btf_name_info *dist_name_info, *dist_name_info_next = NULL; + struct btf_name_info base_name_info = {}; + int dist_kind, base_kind; + + base_t = btf_type_by_id(r->base_btf, id); + /* distilled base consists of named types only. */ + if (!base_t->name_off) + continue; + base_kind = btf_kind(base_t); + base_name_info.id = id; + base_name_info.name = btf__name_by_offset(r->base_btf, base_t->name_off); + switch (base_kind) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + /* These types should match both name and size */ + base_name_info.needs_size = true; + base_name_info.size = base_t->size; + break; + case BTF_KIND_FWD: + /* No size considerations for fwds. */ + break; + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + /* Size only needs to be used for struct/union if there + * are multiple types in base BTF with the same name. + * If there are multiple _distilled_ types with the same + * name (a very unlikely scenario), that doesn't matter + * unless corresponding _base_ types to match them are + * missing. + */ + base_name_info.needs_size = base_name_cnt[base_t->name_off] > 1; + base_name_info.size = base_t->size; + break; + default: + continue; + } + /* iterate over all matching distilled base types */ + for (dist_name_info = search_btf_name_size(&base_name_info, dist_base_info_sorted, + r->nr_dist_base_types); + dist_name_info != NULL; dist_name_info = dist_name_info_next) { + /* Are there more distilled matches to process after + * this one? + */ + dist_name_info_next = dist_name_info + 1; + if (dist_name_info_next >= dist_base_info_sorted_end || + cmp_btf_name_size(&base_name_info, dist_name_info_next)) + dist_name_info_next = NULL; + + if (!dist_name_info->id || dist_name_info->id > r->nr_dist_base_types) { + pr_warn("base BTF id [%d] maps to invalid distilled base BTF id [%d]\n", + id, dist_name_info->id); + err = -EINVAL; + goto done; + } + dist_t = btf_type_by_id(r->dist_base_btf, dist_name_info->id); + dist_kind = btf_kind(dist_t); + + /* Validate that the found distilled type is compatible. + * Do not error out on mismatch as another match may + * occur for an identically-named type. + */ + switch (dist_kind) { + case BTF_KIND_FWD: + switch (base_kind) { + case BTF_KIND_FWD: + if (btf_kflag(dist_t) != btf_kflag(base_t)) + continue; + break; + case BTF_KIND_STRUCT: + if (btf_kflag(base_t)) + continue; + break; + case BTF_KIND_UNION: + if (!btf_kflag(base_t)) + continue; + break; + default: + continue; + } + break; + case BTF_KIND_INT: + if (dist_kind != base_kind || + btf_int_encoding(base_t) != btf_int_encoding(dist_t)) + continue; + break; + case BTF_KIND_FLOAT: + if (dist_kind != base_kind) + continue; + break; + case BTF_KIND_ENUM: + /* ENUM and ENUM64 are encoded as sized ENUM in + * distilled base BTF. + */ + if (base_kind != dist_kind && base_kind != BTF_KIND_ENUM64) + continue; + break; + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + /* size verification is required for embedded + * struct/unions. + */ + if (r->id_map[dist_name_info->id] == BTF_IS_EMBEDDED && + base_t->size != dist_t->size) + continue; + break; + default: + continue; + } + if (r->id_map[dist_name_info->id] && + r->id_map[dist_name_info->id] != BTF_IS_EMBEDDED) { + /* we already have a match; this tells us that + * multiple base types of the same name + * have the same size, since for cases where + * multiple types have the same name we match + * on name and size. In this case, we have + * no way of determining which to relocate + * to in base BTF, so error out. + */ + pr_warn("distilled base BTF type '%s' [%u], size %u has multiple candidates of the same size (ids [%u, %u]) in base BTF\n", + base_name_info.name, dist_name_info->id, + base_t->size, id, r->id_map[dist_name_info->id]); + err = -EINVAL; + goto done; + } + /* map id and name */ + r->id_map[dist_name_info->id] = id; + r->str_map[dist_t->name_off] = base_t->name_off; + } + } + /* ensure all distilled BTF ids now have a mapping... */ + for (id = 1; id < r->nr_dist_base_types; id++) { + const char *name; + + if (r->id_map[id] && r->id_map[id] != BTF_IS_EMBEDDED) + continue; + dist_t = btf_type_by_id(r->dist_base_btf, id); + name = btf__name_by_offset(r->dist_base_btf, dist_t->name_off); + pr_warn("distilled base BTF type '%s' [%d] is not mapped to base BTF id\n", + name, id); + err = -EINVAL; + break; + } +done: + free(base_name_cnt); + free(dist_base_info_sorted); + return err; +} + +/* distilled base should only have named int/float/enum/fwd/struct/union types. */ +static int btf_relocate_validate_distilled_base(struct btf_relocate *r) +{ + unsigned int i; + + for (i = 1; i < r->nr_dist_base_types; i++) { + struct btf_type *t = btf_type_by_id(r->dist_base_btf, i); + int kind = btf_kind(t); + + switch (kind) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_FWD: + if (t->name_off) + break; + pr_warn("type [%d], kind [%d] is invalid for distilled base BTF; it is anonymous\n", + i, kind); + return -EINVAL; + default: + pr_warn("type [%d] in distilled based BTF has unexpected kind [%d]\n", + i, kind); + return -EINVAL; + } + } + return 0; +} + +static int btf_relocate_rewrite_strs(struct btf_relocate *r, __u32 i) +{ + struct btf_type *t = btf_type_by_id(r->btf, i); + struct btf_field_iter it; + __u32 *str_off; + int off, err; + + err = btf_field_iter_init(&it, t, BTF_FIELD_ITER_STRS); + if (err) + return err; + + while ((str_off = btf_field_iter_next(&it))) { + if (!*str_off) + continue; + if (*str_off >= r->dist_str_len) { + *str_off += r->base_str_len - r->dist_str_len; + } else { + off = r->str_map[*str_off]; + if (!off) { + pr_warn("string '%s' [offset %u] is not mapped to base BTF", + btf__str_by_offset(r->btf, off), *str_off); + return -ENOENT; + } + *str_off = off; + } + } + return 0; +} + +/* If successful, output of relocation is updated BTF with base BTF pointing + * at base_btf, and type ids, strings adjusted accordingly. + */ +int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **id_map) +{ + unsigned int nr_types = btf__type_cnt(btf); + const struct btf_header *dist_base_hdr; + const struct btf_header *base_hdr; + struct btf_relocate r = {}; + int err = 0; + __u32 id, i; + + r.dist_base_btf = btf__base_btf(btf); + if (!base_btf || r.dist_base_btf == base_btf) + return -EINVAL; + + r.nr_dist_base_types = btf__type_cnt(r.dist_base_btf); + r.nr_base_types = btf__type_cnt(base_btf); + r.nr_split_types = nr_types - r.nr_dist_base_types; + r.btf = btf; + r.base_btf = base_btf; + + r.id_map = calloc(nr_types, sizeof(*r.id_map)); + r.str_map = calloc(btf_header(r.dist_base_btf)->str_len, sizeof(*r.str_map)); + dist_base_hdr = btf_header(r.dist_base_btf); + base_hdr = btf_header(r.base_btf); + r.dist_str_len = dist_base_hdr->str_len; + r.base_str_len = base_hdr->str_len; + if (!r.id_map || !r.str_map) { + err = -ENOMEM; + goto err_out; + } + + err = btf_relocate_validate_distilled_base(&r); + if (err) + goto err_out; + + /* Split BTF ids need to be adjusted as base and distilled base + * have different numbers of types, changing the start id of split + * BTF. + */ + for (id = r.nr_dist_base_types; id < nr_types; id++) + r.id_map[id] = id + r.nr_base_types - r.nr_dist_base_types; + + /* Build a map from distilled base ids to actual base BTF ids; it is used + * to update split BTF id references. Also build a str_map mapping from + * distilled base BTF names to base BTF names. + */ + err = btf_relocate_map_distilled_base(&r); + if (err) + goto err_out; + + /* Next, rewrite type ids in split BTF, replacing split ids with updated + * ids based on number of types in base BTF, and base ids with + * relocated ids from base_btf. + */ + for (i = 0, id = r.nr_dist_base_types; i < r.nr_split_types; i++, id++) { + err = btf_relocate_rewrite_type_id(&r, id); + if (err) + goto err_out; + } + /* String offsets now need to be updated using the str_map. */ + for (i = 0; i < r.nr_split_types; i++) { + err = btf_relocate_rewrite_strs(&r, i + r.nr_dist_base_types); + if (err) + goto err_out; + } + /* Finally reset base BTF to be base_btf */ + btf_set_base_btf(btf, base_btf); + + if (id_map) { + *id_map = r.id_map; + r.id_map = NULL; + } +err_out: + free(r.id_map); + free(r.str_map); + return err; +} diff --git a/tools/lib/bpf/features.c b/tools/lib/bpf/features.c index a336786a22a38d9b36a9062f733a01f1d1cb312a..50befe125ddc550997fc0c8452c05c8e6f955579 100644 --- a/tools/lib/bpf/features.c +++ b/tools/lib/bpf/features.c @@ -392,11 +392,41 @@ static int probe_uprobe_multi_link(int token_fd) link_fd = bpf_link_create(prog_fd, -1, BPF_TRACE_UPROBE_MULTI, &link_opts); err = -errno; /* close() can clobber errno */ + if (link_fd >= 0 || err != -EBADF) { + if (link_fd >= 0) + close(link_fd); + close(prog_fd); + return 0; + } + + /* Initial multi-uprobe support in kernel didn't handle PID filtering + * correctly (it was doing thread filtering, not process filtering). + * So now we'll detect if PID filtering logic was fixed, and, if not, + * we'll pretend multi-uprobes are not supported, if not. + * Multi-uprobes are used in USDT attachment logic, and we need to be + * conservative here, because multi-uprobe selection happens early at + * load time, while the use of PID filtering is known late at + * attachment time, at which point it's too late to undo multi-uprobe + * selection. + * + * Creating uprobe with pid == -1 for (invalid) '/' binary will fail + * early with -EINVAL on kernels with fixed PID filtering logic; + * otherwise -ESRCH would be returned if passed correct binary path + * (but we'll just get -BADF, of course). + */ + link_opts.uprobe_multi.pid = -1; /* invalid PID */ + link_opts.uprobe_multi.path = "/"; /* invalid path */ + link_opts.uprobe_multi.offsets = &offset; + link_opts.uprobe_multi.cnt = 1; + + link_fd = bpf_link_create(prog_fd, -1, BPF_TRACE_UPROBE_MULTI, &link_opts); + err = -errno; /* close() can clobber errno */ + if (link_fd >= 0) close(link_fd); close(prog_fd); - return link_fd < 0 && err == -EBADF; + return link_fd < 0 && err == -EINVAL; } static int probe_kern_bpf_cookie(int token_fd) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 5401f2df463d24f52b6430252048f4bd5e04e4f5..4a28fac4908a6e119d1e69be2e3a873cc8d20ad4 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -229,7 +229,30 @@ static const char * const prog_type_name[] = { static int __base_pr(enum libbpf_print_level level, const char *format, va_list args) { - if (level == LIBBPF_DEBUG) + const char *env_var = "LIBBPF_LOG_LEVEL"; + static enum libbpf_print_level min_level = LIBBPF_INFO; + static bool initialized; + + if (!initialized) { + char *verbosity; + + initialized = true; + verbosity = getenv(env_var); + if (verbosity) { + if (strcasecmp(verbosity, "warn") == 0) + min_level = LIBBPF_WARN; + else if (strcasecmp(verbosity, "debug") == 0) + min_level = LIBBPF_DEBUG; + else if (strcasecmp(verbosity, "info") == 0) + min_level = LIBBPF_INFO; + else + fprintf(stderr, "libbpf: unrecognized '%s' envvar value: '%s', should be one of 'warn', 'debug', or 'info'.\n", + env_var, verbosity); + } + } + + /* if too verbose, skip logging */ + if (level > min_level) return 0; return vfprintf(stderr, format, args); @@ -549,6 +572,7 @@ struct bpf_map { bool pinned; bool reused; bool autocreate; + bool autoattach; __u64 map_extra; }; @@ -1377,6 +1401,7 @@ static int init_struct_ops_maps(struct bpf_object *obj, const char *sec_name, map->def.value_size = type->size; map->def.max_entries = 1; map->def.map_flags = strcmp(sec_name, STRUCT_OPS_LINK_SEC) == 0 ? BPF_F_LINK : 0; + map->autoattach = true; map->st_ops = calloc(1, sizeof(*map->st_ops)); if (!map->st_ops) @@ -4796,6 +4821,20 @@ int bpf_map__set_autocreate(struct bpf_map *map, bool autocreate) return 0; } +int bpf_map__set_autoattach(struct bpf_map *map, bool autoattach) +{ + if (!bpf_map__is_struct_ops(map)) + return libbpf_err(-EINVAL); + + map->autoattach = autoattach; + return 0; +} + +bool bpf_map__autoattach(const struct bpf_map *map) +{ + return map->autoattach; +} + int bpf_map__reuse_fd(struct bpf_map *map, int fd) { struct bpf_map_info info; @@ -12877,8 +12916,10 @@ struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map) __u32 zero = 0; int err, fd; - if (!bpf_map__is_struct_ops(map)) + if (!bpf_map__is_struct_ops(map)) { + pr_warn("map '%s': can't attach non-struct_ops map\n", map->name); return libbpf_err_ptr(-EINVAL); + } if (map->fd < 0) { pr_warn("map '%s': can't attach BPF map without FD (was it created?)\n", map->name); @@ -13922,6 +13963,35 @@ int bpf_object__attach_skeleton(struct bpf_object_skeleton *s) */ } + /* Skeleton is created with earlier version of bpftool + * which does not support auto-attachment + */ + if (s->map_skel_sz < sizeof(struct bpf_map_skeleton)) + return 0; + + for (i = 0; i < s->map_cnt; i++) { + struct bpf_map *map = *s->maps[i].map; + struct bpf_link **link = s->maps[i].link; + + if (!map->autocreate || !map->autoattach) + continue; + + if (*link) + continue; + + /* only struct_ops maps can be attached */ + if (!bpf_map__is_struct_ops(map)) + continue; + *link = bpf_map__attach_struct_ops(map); + + if (!*link) { + err = -errno; + pr_warn("map '%s': failed to auto-attach: %d\n", + bpf_map__name(map), err); + return libbpf_err(err); + } + } + return 0; } @@ -13935,6 +14005,18 @@ void bpf_object__detach_skeleton(struct bpf_object_skeleton *s) bpf_link__destroy(*link); *link = NULL; } + + if (s->map_skel_sz < sizeof(struct bpf_map_skeleton)) + return; + + for (i = 0; i < s->map_cnt; i++) { + struct bpf_link **link = s->maps[i].link; + + if (link) { + bpf_link__destroy(*link); + *link = NULL; + } + } } void bpf_object__destroy_skeleton(struct bpf_object_skeleton *s) @@ -13942,8 +14024,7 @@ void bpf_object__destroy_skeleton(struct bpf_object_skeleton *s) if (!s) return; - if (s->progs) - bpf_object__detach_skeleton(s); + bpf_object__detach_skeleton(s); if (s->obj) bpf_object__close(*s->obj); free(s->maps); diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index c3f77d9260feaea1181ea49f65e033af03490e34..64a6a3d323e378c3ee9c6152db01d7891a6b98c0 100644 --- a/tools/lib/bpf/libbpf.h +++ b/tools/lib/bpf/libbpf.h @@ -98,7 +98,10 @@ typedef int (*libbpf_print_fn_t)(enum libbpf_print_level level, /** * @brief **libbpf_set_print()** sets user-provided log callback function to - * be used for libbpf warnings and informational messages. + * be used for libbpf warnings and informational messages. If the user callback + * is not set, messages are logged to stderr by default. The verbosity of these + * messages can be controlled by setting the environment variable + * LIBBPF_LOG_LEVEL to either warn, info, or debug. * @param fn The log print function. If NULL, libbpf won't print anything. * @return Pointer to old print function. * @@ -975,6 +978,23 @@ bpf_object__prev_map(const struct bpf_object *obj, const struct bpf_map *map); LIBBPF_API int bpf_map__set_autocreate(struct bpf_map *map, bool autocreate); LIBBPF_API bool bpf_map__autocreate(const struct bpf_map *map); +/** + * @brief **bpf_map__set_autoattach()** sets whether libbpf has to auto-attach + * map during BPF skeleton attach phase. + * @param map the BPF map instance + * @param autoattach whether to attach map during BPF skeleton attach phase + * @return 0 on success; negative error code, otherwise + */ +LIBBPF_API int bpf_map__set_autoattach(struct bpf_map *map, bool autoattach); + +/** + * @brief **bpf_map__autoattach()** returns whether BPF map is configured to + * auto-attach during BPF skeleton attach phase. + * @param map the BPF map instance + * @return true if map is set to auto-attach during skeleton attach phase; false, otherwise + */ +LIBBPF_API bool bpf_map__autoattach(const struct bpf_map *map); + /** * @brief **bpf_map__fd()** gets the file descriptor of the passed * BPF map @@ -1669,6 +1689,7 @@ struct bpf_map_skeleton { const char *name; struct bpf_map **map; void **mmaped; + struct bpf_link **link; }; struct bpf_prog_skeleton { diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index c1ce8aa3520bf4a833fad43de1db5b1caef62c07..8f0d9ea3b1b48f09bf16259a758eddb2eb2ea02b 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -419,6 +419,10 @@ LIBBPF_1.4.0 { LIBBPF_1.5.0 { global: + btf__distill_base; + btf__relocate; + bpf_map__autoattach; + bpf_map__set_autoattach; bpf_program__attach_sockmap; ring__consume_n; ring_buffer__consume_n; diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index a0dcfb82e455d7c7f3dc8ed0b944ee37dad03324..408df59e07719f9f16a866e6e78ca2c1ec188cfe 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -234,6 +234,9 @@ struct btf_type; struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id); const char *btf_kind_str(const struct btf_type *t); const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, __u32 *res_id); +const struct btf_header *btf_header(const struct btf *btf); +void btf_set_base_btf(struct btf *btf, const struct btf *base_btf); +int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **id_map); static inline enum btf_func_linkage btf_func_linkage(const struct btf_type *t) { @@ -508,11 +511,33 @@ struct bpf_line_info_min { __u32 line_col; }; +enum btf_field_iter_kind { + BTF_FIELD_ITER_IDS, + BTF_FIELD_ITER_STRS, +}; + +struct btf_field_desc { + /* once-per-type offsets */ + int t_off_cnt, t_offs[2]; + /* member struct size, or zero, if no members */ + int m_sz; + /* repeated per-member offsets */ + int m_off_cnt, m_offs[1]; +}; + +struct btf_field_iter { + struct btf_field_desc desc; + void *p; + int m_idx; + int off_idx; + int vlen; +}; + +int btf_field_iter_init(struct btf_field_iter *it, struct btf_type *t, enum btf_field_iter_kind iter_kind); +__u32 *btf_field_iter_next(struct btf_field_iter *it); typedef int (*type_id_visit_fn)(__u32 *type_id, void *ctx); typedef int (*str_off_visit_fn)(__u32 *str_off, void *ctx); -int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx); -int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx); int btf_ext_visit_type_ids(struct btf_ext *btf_ext, type_id_visit_fn visit, void *ctx); int btf_ext_visit_str_offs(struct btf_ext *btf_ext, str_off_visit_fn visit, void *ctx); __s32 btf__find_by_name_kind_own(const struct btf *btf, const char *type_name, @@ -597,13 +622,9 @@ static inline int ensure_good_fd(int fd) return fd; } -static inline int sys_dup2(int oldfd, int newfd) +static inline int sys_dup3(int oldfd, int newfd, int flags) { -#ifdef __NR_dup2 - return syscall(__NR_dup2, oldfd, newfd); -#else - return syscall(__NR_dup3, oldfd, newfd, 0); -#endif + return syscall(__NR_dup3, oldfd, newfd, flags); } /* Point *fixed_fd* to the same file that *tmp_fd* points to. @@ -614,7 +635,7 @@ static inline int reuse_fd(int fixed_fd, int tmp_fd) { int err; - err = sys_dup2(tmp_fd, fixed_fd); + err = sys_dup3(tmp_fd, fixed_fd, O_CLOEXEC); err = err < 0 ? -errno : 0; close(tmp_fd); /* clean up temporary FD */ return err; diff --git a/tools/lib/bpf/linker.c b/tools/lib/bpf/linker.c index 0d4be829551b578c01d1be2049d22a1ea7bb4c9b..fa11a671da3e07e03fce06b712e958b251a0b79d 100644 --- a/tools/lib/bpf/linker.c +++ b/tools/lib/bpf/linker.c @@ -957,19 +957,33 @@ static int check_btf_str_off(__u32 *str_off, void *ctx) static int linker_sanity_check_btf(struct src_obj *obj) { struct btf_type *t; - int i, n, err = 0; + int i, n, err; if (!obj->btf) return 0; n = btf__type_cnt(obj->btf); for (i = 1; i < n; i++) { + struct btf_field_iter it; + __u32 *type_id, *str_off; + t = btf_type_by_id(obj->btf, i); - err = err ?: btf_type_visit_type_ids(t, check_btf_type_id, obj->btf); - err = err ?: btf_type_visit_str_offs(t, check_btf_str_off, obj->btf); + err = btf_field_iter_init(&it, t, BTF_FIELD_ITER_IDS); if (err) return err; + while ((type_id = btf_field_iter_next(&it))) { + if (*type_id >= n) + return -EINVAL; + } + + err = btf_field_iter_init(&it, t, BTF_FIELD_ITER_STRS); + if (err) + return err; + while ((str_off = btf_field_iter_next(&it))) { + if (!btf__str_by_offset(obj->btf, *str_off)) + return -EINVAL; + } } return 0; @@ -2234,26 +2248,10 @@ static int linker_fixup_btf(struct src_obj *obj) return 0; } -static int remap_type_id(__u32 *type_id, void *ctx) -{ - int *id_map = ctx; - int new_id = id_map[*type_id]; - - /* Error out if the type wasn't remapped. Ignore VOID which stays VOID. */ - if (new_id == 0 && *type_id != 0) { - pr_warn("failed to find new ID mapping for original BTF type ID %u\n", *type_id); - return -EINVAL; - } - - *type_id = id_map[*type_id]; - - return 0; -} - static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) { const struct btf_type *t; - int i, j, n, start_id, id; + int i, j, n, start_id, id, err; const char *name; if (!obj->btf) @@ -2324,9 +2322,25 @@ static int linker_append_btf(struct bpf_linker *linker, struct src_obj *obj) n = btf__type_cnt(linker->btf); for (i = start_id; i < n; i++) { struct btf_type *dst_t = btf_type_by_id(linker->btf, i); + struct btf_field_iter it; + __u32 *type_id; - if (btf_type_visit_type_ids(dst_t, remap_type_id, obj->btf_type_map)) - return -EINVAL; + err = btf_field_iter_init(&it, dst_t, BTF_FIELD_ITER_IDS); + if (err) + return err; + + while ((type_id = btf_field_iter_next(&it))) { + int new_id = obj->btf_type_map[*type_id]; + + /* Error out if the type wasn't remapped. Ignore VOID which stays VOID. */ + if (new_id == 0 && *type_id != 0) { + pr_warn("failed to find new ID mapping for original BTF type ID %u\n", + *type_id); + return -EINVAL; + } + + *type_id = obj->btf_type_map[*type_id]; + } } /* Rewrite VAR/FUNC underlying types (i.e., FUNC's FUNC_PROTO and VAR's diff --git a/tools/net/ynl/lib/ynl-priv.h b/tools/net/ynl/lib/ynl-priv.h index 6cf890080dc0921710b37dd6d442591dc9b502da..3c09a7bbfba59ff743b9e0b2e2c087db1ee23fc9 100644 --- a/tools/net/ynl/lib/ynl-priv.h +++ b/tools/net/ynl/lib/ynl-priv.h @@ -45,17 +45,17 @@ struct ynl_policy_attr { enum ynl_policy_type type; unsigned int len; const char *name; - struct ynl_policy_nest *nest; + const struct ynl_policy_nest *nest; }; struct ynl_policy_nest { unsigned int max_attr; - struct ynl_policy_attr *table; + const struct ynl_policy_attr *table; }; struct ynl_parse_arg { struct ynl_sock *ys; - struct ynl_policy_nest *rsp_policy; + const struct ynl_policy_nest *rsp_policy; void *data; }; @@ -79,7 +79,7 @@ static inline void *ynl_dump_obj_next(void *obj) struct ynl_dump_list_type *list; uptr -= offsetof(struct ynl_dump_list_type, data); - list = (void *)uptr; + list = (struct ynl_dump_list_type *)uptr; uptr = (unsigned long)list->next; uptr += offsetof(struct ynl_dump_list_type, data); @@ -119,7 +119,7 @@ struct ynl_dump_state { }; struct ynl_ntf_info { - struct ynl_policy_nest *policy; + const struct ynl_policy_nest *policy; ynl_parse_cb_t cb; size_t alloc_sz; void (*free)(struct ynl_ntf_base_type *ntf); @@ -139,7 +139,7 @@ int ynl_error_parse(struct ynl_parse_arg *yarg, const char *msg); static inline struct nlmsghdr *ynl_nlmsg_put_header(void *buf) { - struct nlmsghdr *nlh = buf; + struct nlmsghdr *nlh = (struct nlmsghdr *)buf; memset(nlh, 0, sizeof(*nlh)); nlh->nlmsg_len = NLMSG_HDRLEN; @@ -196,7 +196,7 @@ static inline void *ynl_attr_data(const struct nlattr *attr) static inline void *ynl_attr_data_end(const struct nlattr *attr) { - return ynl_attr_data(attr) + ynl_attr_data_len(attr); + return (char *)ynl_attr_data(attr) + ynl_attr_data_len(attr); } #define ynl_attr_for_each(attr, nlh, fixed_hdr_sz) \ @@ -228,7 +228,7 @@ ynl_attr_next(const void *end, const struct nlattr *prev) { struct nlattr *attr; - attr = (void *)((char *)prev + NLA_ALIGN(prev->nla_len)); + attr = (struct nlattr *)((char *)prev + NLA_ALIGN(prev->nla_len)); return ynl_attr_if_good(end, attr); } @@ -237,8 +237,8 @@ ynl_attr_first(const void *start, size_t len, size_t skip) { struct nlattr *attr; - attr = (void *)((char *)start + NLMSG_ALIGN(skip)); - return ynl_attr_if_good(start + len, attr); + attr = (struct nlattr *)((char *)start + NLMSG_ALIGN(skip)); + return ynl_attr_if_good((char *)start + len, attr); } static inline bool @@ -262,9 +262,9 @@ ynl_attr_nest_start(struct nlmsghdr *nlh, unsigned int attr_type) struct nlattr *attr; if (__ynl_attr_put_overflow(nlh, 0)) - return ynl_nlmsg_end_addr(nlh) - NLA_HDRLEN; + return (struct nlattr *)ynl_nlmsg_end_addr(nlh) - 1; - attr = ynl_nlmsg_end_addr(nlh); + attr = (struct nlattr *)ynl_nlmsg_end_addr(nlh); attr->nla_type = attr_type | NLA_F_NESTED; nlh->nlmsg_len += NLA_HDRLEN; @@ -286,7 +286,7 @@ ynl_attr_put(struct nlmsghdr *nlh, unsigned int attr_type, if (__ynl_attr_put_overflow(nlh, size)) return; - attr = ynl_nlmsg_end_addr(nlh); + attr = (struct nlattr *)ynl_nlmsg_end_addr(nlh); attr->nla_type = attr_type; attr->nla_len = NLA_HDRLEN + size; @@ -305,10 +305,10 @@ ynl_attr_put_str(struct nlmsghdr *nlh, unsigned int attr_type, const char *str) if (__ynl_attr_put_overflow(nlh, len)) return; - attr = ynl_nlmsg_end_addr(nlh); + attr = (struct nlattr *)ynl_nlmsg_end_addr(nlh); attr->nla_type = attr_type; - strcpy(ynl_attr_data(attr), str); + strcpy((char *)ynl_attr_data(attr), str); attr->nla_len = NLA_HDRLEN + NLA_ALIGN(len); nlh->nlmsg_len += NLMSG_ALIGN(attr->nla_len); diff --git a/tools/net/ynl/lib/ynl.c b/tools/net/ynl/lib/ynl.c index 4b9c091fc86b5205d3c4a018ca65b444279cee11..fcb18a5a6d709d55206835e0cfd24da0efcaa419 100644 --- a/tools/net/ynl/lib/ynl.c +++ b/tools/net/ynl/lib/ynl.c @@ -46,7 +46,7 @@ /* -- Netlink boiler plate */ static int -ynl_err_walk_report_one(struct ynl_policy_nest *policy, unsigned int type, +ynl_err_walk_report_one(const struct ynl_policy_nest *policy, unsigned int type, char *str, int str_sz, int *n) { if (!policy) { @@ -75,8 +75,8 @@ ynl_err_walk_report_one(struct ynl_policy_nest *policy, unsigned int type, static int ynl_err_walk(struct ynl_sock *ys, void *start, void *end, unsigned int off, - struct ynl_policy_nest *policy, char *str, int str_sz, - struct ynl_policy_nest **nest_pol) + const struct ynl_policy_nest *policy, char *str, int str_sz, + const struct ynl_policy_nest **nest_pol) { unsigned int astart_off, aend_off; const struct nlattr *attr; @@ -206,7 +206,7 @@ ynl_ext_ack_check(struct ynl_sock *ys, const struct nlmsghdr *nlh, bad_attr[n] = '\0'; } if (tb[NLMSGERR_ATTR_MISS_TYPE]) { - struct ynl_policy_nest *nest_pol = NULL; + const struct ynl_policy_nest *nest_pol = NULL; unsigned int n, off, type; void *start, *end; int n2; @@ -296,7 +296,7 @@ static int ynl_cb_done(const struct nlmsghdr *nlh, struct ynl_parse_arg *yarg) int ynl_attr_validate(struct ynl_parse_arg *yarg, const struct nlattr *attr) { - struct ynl_policy_attr *policy; + const struct ynl_policy_attr *policy; unsigned int type, len; unsigned char *data; diff --git a/tools/net/ynl/lib/ynl.h b/tools/net/ynl/lib/ynl.h index eef7c6324ed48f3f5207af22577671b3e10cd636..6cd570b283eab89bb1a532deefc8f1e4c4229080 100644 --- a/tools/net/ynl/lib/ynl.h +++ b/tools/net/ynl/lib/ynl.h @@ -76,7 +76,7 @@ struct ynl_sock { struct ynl_ntf_base_type **ntf_last_next; struct nlmsghdr *nlh; - struct ynl_policy_nest *req_policy; + const struct ynl_policy_nest *req_policy; unsigned char *tx_buf; unsigned char *rx_buf; unsigned char raw_buf[]; diff --git a/tools/net/ynl/ynl-gen-c.py b/tools/net/ynl/ynl-gen-c.py index a42d62b23ee004338e7ee9796625fb32708c532f..374ca5e86e24d1afc47db4413183e3ab0dc31d1b 100755 --- a/tools/net/ynl/ynl-gen-c.py +++ b/tools/net/ynl/ynl-gen-c.py @@ -1507,12 +1507,12 @@ def print_dump_prototype(ri): def put_typol_fwd(cw, struct): - cw.p(f'extern struct ynl_policy_nest {struct.render_name}_nest;') + cw.p(f'extern const struct ynl_policy_nest {struct.render_name}_nest;') def put_typol(cw, struct): type_max = struct.attr_set.max_name - cw.block_start(line=f'struct ynl_policy_attr {struct.render_name}_policy[{type_max} + 1] =') + cw.block_start(line=f'const struct ynl_policy_attr {struct.render_name}_policy[{type_max} + 1] =') for _, arg in struct.member_list(): arg.attr_typol(cw) @@ -1520,7 +1520,7 @@ def put_typol(cw, struct): cw.block_end(line=';') cw.nl() - cw.block_start(line=f'struct ynl_policy_nest {struct.render_name}_nest =') + cw.block_start(line=f'const struct ynl_policy_nest {struct.render_name}_nest =') cw.p(f'.max_attr = {type_max},') cw.p(f'.table = {struct.render_name}_policy,') cw.block_end(line=';') diff --git a/tools/net/ynl/ynl-gen-rst.py b/tools/net/ynl/ynl-gen-rst.py index 657e881d2ea4a571a2cebbc1b9465a04ab21d804..6c56d0d726b4131957551f4b6c09be8c043db766 100755 --- a/tools/net/ynl/ynl-gen-rst.py +++ b/tools/net/ynl/ynl-gen-rst.py @@ -49,7 +49,7 @@ def inline(text: str) -> str: def sanitize(text: str) -> str: """Remove newlines and multiple spaces""" # This is useful for some fields that are spread across multiple lines - return str(text).replace("\n", "").strip() + return str(text).replace("\n", " ").strip() def rst_fields(key: str, value: str, level: int = 0) -> str: @@ -156,7 +156,10 @@ def parse_do(do_dict: Dict[str, Any], level: int = 0) -> str: lines = [] for key in do_dict.keys(): lines.append(rst_paragraph(bold(key), level + 1)) - lines.append(parse_do_attributes(do_dict[key], level + 1) + "\n") + if key in ['request', 'reply']: + lines.append(parse_do_attributes(do_dict[key], level + 1) + "\n") + else: + lines.append(headroom(level + 2) + do_dict[key] + "\n") return "\n".join(lines) @@ -172,13 +175,13 @@ def parse_do_attributes(attrs: Dict[str, Any], level: int = 0) -> str: def parse_operations(operations: List[Dict[str, Any]], namespace: str) -> str: """Parse operations block""" - preprocessed = ["name", "doc", "title", "do", "dump"] + preprocessed = ["name", "doc", "title", "do", "dump", "flags"] linkable = ["fixed-header", "attribute-set"] lines = [] for operation in operations: lines.append(rst_section(namespace, 'operation', operation["name"])) - lines.append(rst_paragraph(sanitize(operation["doc"])) + "\n") + lines.append(rst_paragraph(operation["doc"]) + "\n") for key in operation.keys(): if key in preprocessed: @@ -188,6 +191,8 @@ def parse_operations(operations: List[Dict[str, Any]], namespace: str) -> str: if key in linkable: value = rst_ref(namespace, key, value) lines.append(rst_fields(key, value, 0)) + if 'flags' in operation: + lines.append(rst_fields('flags', rst_list_inline(operation['flags']))) if "do" in operation: lines.append(rst_paragraph(":do:", 0)) diff --git a/tools/power/cpupower/utils/helpers/amd.c b/tools/power/cpupower/utils/helpers/amd.c index c519cc89c97f42a3849130aebdfcdfb14b44ebf6..0a56e22240fc8b8b026db7d72e0f5055cc420a1a 100644 --- a/tools/power/cpupower/utils/helpers/amd.c +++ b/tools/power/cpupower/utils/helpers/amd.c @@ -41,6 +41,16 @@ union core_pstate { unsigned res1:31; unsigned en:1; } pstatedef; + /* since fam 1Ah: */ + struct { + unsigned fid:12; + unsigned res1:2; + unsigned vid:8; + unsigned iddval:8; + unsigned idddiv:2; + unsigned res2:31; + unsigned en:1; + } pstatedef2; unsigned long long val; }; @@ -48,6 +58,10 @@ static int get_did(union core_pstate pstate) { int t; + /* Fam 1Ah onward do not use did */ + if (cpupower_cpu_info.family >= 0x1A) + return 0; + if (cpupower_cpu_info.caps & CPUPOWER_CAP_AMD_PSTATEDEF) t = pstate.pstatedef.did; else if (cpupower_cpu_info.family == 0x12) @@ -61,12 +75,18 @@ static int get_did(union core_pstate pstate) static int get_cof(union core_pstate pstate) { int t; - int fid, did, cof; + int fid, did, cof = 0; did = get_did(pstate); if (cpupower_cpu_info.caps & CPUPOWER_CAP_AMD_PSTATEDEF) { - fid = pstate.pstatedef.fid; - cof = 200 * fid / did; + if (cpupower_cpu_info.family >= 0x1A) { + fid = pstate.pstatedef2.fid; + if (fid > 0x0f) + cof = (fid * 5); + } else { + fid = pstate.pstatedef.fid; + cof = 200 * fid / did; + } } else { t = 0x10; fid = pstate.pstate.fid; diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 6584443144de89d3a344758e8ccf831c8bc3536d..eaf091a3d33137a4a09598e5d63145b4a7a4e71b 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -3,6 +3,7 @@ #include #include +#include #include #include #include diff --git a/tools/testing/selftests/alsa/Makefile b/tools/testing/selftests/alsa/Makefile index 5af9ba8a4645bc5fb86f724e8619e8e2d25ca11a..c1ce39874e2b5451d1caac769925ca4c0351fb98 100644 --- a/tools/testing/selftests/alsa/Makefile +++ b/tools/testing/selftests/alsa/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 # -CFLAGS += $(shell pkg-config --cflags alsa) +CFLAGS += $(shell pkg-config --cflags alsa) $(KHDR_INCLUDES) LDLIBS += $(shell pkg-config --libs alsa) ifeq ($(LDLIBS),) LDLIBS += -lasound diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 3d9e4b8c6b81a950654bc4a7d759b6e6e0bcf4a0..eede6fc2ccb4043d78591046a0e2bc7c09c014d5 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -163,7 +163,7 @@ struct bpf_iter_task_vma; extern int bpf_iter_task_vma_new(struct bpf_iter_task_vma *it, struct task_struct *task, - unsigned long addr) __ksym; + __u64 addr) __ksym; extern struct vm_area_struct *bpf_iter_task_vma_next(struct bpf_iter_task_vma *it) __ksym; extern void bpf_iter_task_vma_destroy(struct bpf_iter_task_vma *it) __ksym; @@ -351,6 +351,7 @@ l_true: \ l_continue:; \ }) #else +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ #define can_loop \ ({ __label__ l_break, l_continue; \ bool ret = true; \ @@ -376,6 +377,33 @@ l_true: \ l_break: break; \ l_continue:; \ }) +#else +#define can_loop \ + ({ __label__ l_break, l_continue; \ + bool ret = true; \ + asm volatile goto("1:.byte 0xe5; \ + .byte 0; \ + .long (((%l[l_break] - 1b - 8) / 8) & 0xffff) << 16; \ + .short 0" \ + :::: l_break); \ + goto l_continue; \ + l_break: ret = false; \ + l_continue:; \ + ret; \ + }) + +#define cond_break \ + ({ __label__ l_break, l_continue; \ + asm volatile goto("1:.byte 0xe5; \ + .byte 0; \ + .long (((%l[l_break] - 1b - 8) / 8) & 0xffff) << 16; \ + .short 0" \ + :::: l_break); \ + goto l_continue; \ + l_break: break; \ + l_continue:; \ + }) +#endif #endif #ifndef bpf_nop_mov diff --git a/tools/testing/selftests/bpf/bpf_test_no_cfi/bpf_test_no_cfi.c b/tools/testing/selftests/bpf/bpf_test_no_cfi/bpf_test_no_cfi.c index b1dd889d5d7d08ced54fa89e6835235ddd7dc3f3..948eb3962732dee2d17a68732ec899d4265ca4fb 100644 --- a/tools/testing/selftests/bpf/bpf_test_no_cfi/bpf_test_no_cfi.c +++ b/tools/testing/selftests/bpf/bpf_test_no_cfi/bpf_test_no_cfi.c @@ -22,12 +22,12 @@ static int dummy_init_member(const struct btf_type *t, return 0; } -static int dummy_reg(void *kdata) +static int dummy_reg(void *kdata, struct bpf_link *link) { return 0; } -static void dummy_unreg(void *kdata) +static void dummy_unreg(void *kdata, struct bpf_link *link) { } diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c index 2a18bd320e92209de70681a8a8095f50438444e1..49f9a311e49b3bb4905375adf0e6bf701b22020f 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c @@ -154,6 +154,11 @@ __bpf_kfunc void bpf_kfunc_common_test(void) { } +__bpf_kfunc void bpf_kfunc_dynptr_test(struct bpf_dynptr *ptr, + struct bpf_dynptr *ptr__nullable) +{ +} + struct bpf_testmod_btf_type_tag_1 { int a; }; @@ -363,6 +368,7 @@ BTF_ID_FLAGS(func, bpf_iter_testmod_seq_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_testmod_seq_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_testmod_seq_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_kfunc_common_test) +BTF_ID_FLAGS(func, bpf_kfunc_dynptr_test) BTF_KFUNCS_END(bpf_testmod_common_kfunc_ids) static const struct btf_kfunc_id_set bpf_testmod_common_kfunc_set = { @@ -820,7 +826,7 @@ static const struct bpf_verifier_ops bpf_testmod_verifier_ops = { .is_valid_access = bpf_testmod_ops_is_valid_access, }; -static int bpf_dummy_reg(void *kdata) +static int bpf_dummy_reg(void *kdata, struct bpf_link *link) { struct bpf_testmod_ops *ops = kdata; @@ -835,7 +841,7 @@ static int bpf_dummy_reg(void *kdata) return 0; } -static void bpf_dummy_unreg(void *kdata) +static void bpf_dummy_unreg(void *kdata, struct bpf_link *link) { } @@ -871,7 +877,7 @@ struct bpf_struct_ops bpf_bpf_testmod_ops = { .owner = THIS_MODULE, }; -static int bpf_dummy_reg2(void *kdata) +static int bpf_dummy_reg2(void *kdata, struct bpf_link *link) { struct bpf_testmod_ops2 *ops = kdata; diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod_kfunc.h index b0d586a6751fa3b7ef5b0714d054c3f8ecaaaf81..f9809517e7fa0e81795d0243d79f3f5ed7cc3426 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod_kfunc.h +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod_kfunc.h @@ -134,4 +134,5 @@ int bpf_kfunc_call_sock_sendmsg(struct sendmsg_args *args) __ksym; int bpf_kfunc_call_kernel_getsockname(struct addr_args *args) __ksym; int bpf_kfunc_call_kernel_getpeername(struct addr_args *args) __ksym; +void bpf_kfunc_dynptr_test(struct bpf_dynptr *ptr, struct bpf_dynptr *ptr__nullable) __ksym; #endif /* _BPF_TESTMOD_KFUNC_H */ diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config index eeabd798bc3ae8c0054f91f6c02a1db9d69fbe99..2fb16da78dce8805b5302374ae3b165a77489b6d 100644 --- a/tools/testing/selftests/bpf/config +++ b/tools/testing/selftests/bpf/config @@ -80,6 +80,7 @@ CONFIG_NETFILTER_XT_TARGET_CT=y CONFIG_NETKIT=y CONFIG_NF_CONNTRACK=y CONFIG_NF_CONNTRACK_MARK=y +CONFIG_NF_CONNTRACK_ZONES=y CONFIG_NF_DEFRAG_IPV4=y CONFIG_NF_DEFRAG_IPV6=y CONFIG_NF_NAT=y diff --git a/tools/testing/selftests/bpf/network_helpers.c b/tools/testing/selftests/bpf/network_helpers.c index 35250e6cde7fb8dbbe5745486ff0ce9a08312256..e20caef06aae9ebae9f64231144dfd253e87665f 100644 --- a/tools/testing/selftests/bpf/network_helpers.c +++ b/tools/testing/selftests/bpf/network_helpers.c @@ -94,7 +94,8 @@ static int __start_server(int type, const struct sockaddr *addr, socklen_t addrl if (settimeo(fd, opts->timeout_ms)) goto error_close; - if (opts->post_socket_cb && opts->post_socket_cb(fd, NULL)) { + if (opts->post_socket_cb && + opts->post_socket_cb(fd, opts->cb_opts)) { log_err("Failed to call post_socket_cb"); goto error_close; } @@ -118,22 +119,32 @@ error_close: return -1; } -int start_server(int family, int type, const char *addr_str, __u16 port, - int timeout_ms) +int start_server_str(int family, int type, const char *addr_str, __u16 port, + const struct network_helper_opts *opts) { - struct network_helper_opts opts = { - .timeout_ms = timeout_ms, - }; struct sockaddr_storage addr; socklen_t addrlen; + if (!opts) + opts = &default_opts; + if (make_sockaddr(family, addr_str, port, &addr, &addrlen)) return -1; - return __start_server(type, (struct sockaddr *)&addr, addrlen, &opts); + return __start_server(type, (struct sockaddr *)&addr, addrlen, opts); +} + +int start_server(int family, int type, const char *addr_str, __u16 port, + int timeout_ms) +{ + struct network_helper_opts opts = { + .timeout_ms = timeout_ms, + }; + + return start_server_str(family, type, addr_str, port, &opts); } -static int reuseport_cb(int fd, const struct post_socket_opts *opts) +static int reuseport_cb(int fd, void *opts) { int on = 1; @@ -338,9 +349,8 @@ int connect_to_fd_opts(int server_fd, const struct network_helper_opts *opts) if (settimeo(fd, opts->timeout_ms)) goto error_close; - if (opts->cc && opts->cc[0] && - setsockopt(fd, SOL_TCP, TCP_CONGESTION, opts->cc, - strlen(opts->cc) + 1)) + if (opts->post_socket_cb && + opts->post_socket_cb(fd, opts->cb_opts)) goto error_close; if (!opts->noconnect) diff --git a/tools/testing/selftests/bpf/network_helpers.h b/tools/testing/selftests/bpf/network_helpers.h index 883c7ea9d8d5479c6c7662d0344d86804096620c..11eea8e2e4f1e23d1fef674ae02af0d8ea63551c 100644 --- a/tools/testing/selftests/bpf/network_helpers.h +++ b/tools/testing/selftests/bpf/network_helpers.h @@ -21,16 +21,14 @@ typedef __u16 __sum16; #define VIP_NUM 5 #define MAGIC_BYTES 123 -struct post_socket_opts {}; - struct network_helper_opts { - const char *cc; int timeout_ms; bool must_fail; bool noconnect; int type; int proto; - int (*post_socket_cb)(int fd, const struct post_socket_opts *opts); + int (*post_socket_cb)(int fd, void *opts); + void *cb_opts; }; /* ipv4 test vector */ @@ -50,6 +48,8 @@ struct ipv6_packet { extern struct ipv6_packet pkt_v6; int settimeo(int fd, int timeout_ms); +int start_server_str(int family, int type, const char *addr_str, __u16 port, + const struct network_helper_opts *opts); int start_server(int family, int type, const char *addr, __u16 port, int timeout_ms); int *start_reuseport_server(int family, int type, const char *addr_str, diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c index 4407ea428e77c528664b3c5e590d5b8e1e93b36a..070c52c312e5f61f44b042ae027a547cce945870 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c @@ -451,7 +451,7 @@ static void pe_subtest(struct test_bpf_cookie *skel) attr.type = PERF_TYPE_SOFTWARE; attr.config = PERF_COUNT_SW_CPU_CLOCK; attr.freq = 1; - attr.sample_freq = 1000; + attr.sample_freq = 10000; pfd = syscall(__NR_perf_event_open, &attr, -1, 0, -1, PERF_FLAG_FD_CLOEXEC); if (!ASSERT_GE(pfd, 0, "perf_fd")) goto cleanup; diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_nf.c b/tools/testing/selftests/bpf/prog_tests/bpf_nf.c index b30ff6b3b81ae31c4c54968810a55e5a4b1d9424..a4a1f93878d40dceed5cb41f6c03c60b39fbc96d 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_nf.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_nf.c @@ -104,6 +104,7 @@ static void test_bpf_nf_ct(int mode) ASSERT_EQ(skel->bss->test_einval_bpf_tuple, -EINVAL, "Test EINVAL for NULL bpf_tuple"); ASSERT_EQ(skel->bss->test_einval_reserved, -EINVAL, "Test EINVAL for reserved not set to 0"); + ASSERT_EQ(skel->bss->test_einval_reserved_new, -EINVAL, "Test EINVAL for reserved in new struct not set to 0"); ASSERT_EQ(skel->bss->test_einval_netns_id, -EINVAL, "Test EINVAL for netns_id < -1"); ASSERT_EQ(skel->bss->test_einval_len_opts, -EINVAL, "Test EINVAL for len__opts != NF_BPF_CT_OPTS_SZ"); ASSERT_EQ(skel->bss->test_eproto_l4proto, -EPROTO, "Test EPROTO for l4proto != TCP or UDP"); @@ -122,6 +123,12 @@ static void test_bpf_nf_ct(int mode) ASSERT_EQ(skel->bss->test_exist_lookup_mark, 43, "Test existing connection lookup ctmark"); ASSERT_EQ(skel->data->test_snat_addr, 0, "Test for source natting"); ASSERT_EQ(skel->data->test_dnat_addr, 0, "Test for destination natting"); + ASSERT_EQ(skel->data->test_ct_zone_id_alloc_entry, 0, "Test for alloc new entry in specified ct zone"); + ASSERT_EQ(skel->data->test_ct_zone_id_insert_entry, 0, "Test for insert new entry in specified ct zone"); + ASSERT_EQ(skel->data->test_ct_zone_id_succ_lookup, 0, "Test for successful lookup in specified ct_zone"); + ASSERT_EQ(skel->bss->test_ct_zone_dir_enoent_lookup, -ENOENT, "Test ENOENT for lookup with wrong ct zone dir"); + ASSERT_EQ(skel->bss->test_ct_zone_id_enoent_lookup, -ENOENT, "Test ENOENT for lookup in wrong ct zone"); + end: if (client_fd != -1) close(client_fd); diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c index 0aca025327948d80613ba7e289cd76cbfabfae1b..67358adf5db3a5492552ef95e4291bcd5843baf1 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_tcp_ca.c @@ -23,6 +23,11 @@ static const unsigned int total_bytes = 10 * 1024 * 1024; static int expected_stg = 0xeB9F; +struct cb_opts { + const char *cc; + int map_fd; +}; + static int settcpca(int fd, const char *tcp_ca) { int err; @@ -34,55 +39,66 @@ static int settcpca(int fd, const char *tcp_ca) return 0; } -static void do_test(const char *tcp_ca, const struct bpf_map *sk_stg_map) +static bool start_test(char *addr_str, + const struct network_helper_opts *srv_opts, + const struct network_helper_opts *cli_opts, + int *srv_fd, int *cli_fd) { - int lfd = -1, fd = -1; - int err; - - lfd = start_server(AF_INET6, SOCK_STREAM, NULL, 0, 0); - if (!ASSERT_NEQ(lfd, -1, "socket")) - return; + *srv_fd = start_server_str(AF_INET6, SOCK_STREAM, addr_str, 0, srv_opts); + if (!ASSERT_NEQ(*srv_fd, -1, "start_server_str")) + goto err; - fd = socket(AF_INET6, SOCK_STREAM, 0); - if (!ASSERT_NEQ(fd, -1, "socket")) { - close(lfd); - return; - } + /* connect to server */ + *cli_fd = connect_to_fd_opts(*srv_fd, cli_opts); + if (!ASSERT_NEQ(*cli_fd, -1, "connect_to_fd_opts")) + goto err; - if (settcpca(lfd, tcp_ca) || settcpca(fd, tcp_ca)) - goto done; + return true; - if (sk_stg_map) { - err = bpf_map_update_elem(bpf_map__fd(sk_stg_map), &fd, - &expected_stg, BPF_NOEXIST); - if (!ASSERT_OK(err, "bpf_map_update_elem(sk_stg_map)")) - goto done; +err: + if (*srv_fd != -1) { + close(*srv_fd); + *srv_fd = -1; } + if (*cli_fd != -1) { + close(*cli_fd); + *cli_fd = -1; + } + return false; +} - /* connect to server */ - err = connect_fd_to_fd(fd, lfd, 0); - if (!ASSERT_NEQ(err, -1, "connect")) - goto done; - - if (sk_stg_map) { - int tmp_stg; +static void do_test(const struct network_helper_opts *opts) +{ + int lfd = -1, fd = -1; - err = bpf_map_lookup_elem(bpf_map__fd(sk_stg_map), &fd, - &tmp_stg); - if (!ASSERT_ERR(err, "bpf_map_lookup_elem(sk_stg_map)") || - !ASSERT_EQ(errno, ENOENT, "bpf_map_lookup_elem(sk_stg_map)")) - goto done; - } + if (!start_test(NULL, opts, opts, &lfd, &fd)) + goto done; ASSERT_OK(send_recv_data(lfd, fd, total_bytes), "send_recv_data"); done: - close(lfd); - close(fd); + if (lfd != -1) + close(lfd); + if (fd != -1) + close(fd); +} + +static int cc_cb(int fd, void *opts) +{ + struct cb_opts *cb_opts = (struct cb_opts *)opts; + + return settcpca(fd, cb_opts->cc); } static void test_cubic(void) { + struct cb_opts cb_opts = { + .cc = "bpf_cubic", + }; + struct network_helper_opts opts = { + .post_socket_cb = cc_cb, + .cb_opts = &cb_opts, + }; struct bpf_cubic *cubic_skel; struct bpf_link *link; @@ -96,7 +112,7 @@ static void test_cubic(void) return; } - do_test("bpf_cubic", NULL); + do_test(&opts); ASSERT_EQ(cubic_skel->bss->bpf_cubic_acked_called, 1, "pkts_acked called"); @@ -104,8 +120,37 @@ static void test_cubic(void) bpf_cubic__destroy(cubic_skel); } +static int stg_post_socket_cb(int fd, void *opts) +{ + struct cb_opts *cb_opts = (struct cb_opts *)opts; + int err; + + err = settcpca(fd, cb_opts->cc); + if (err) + return err; + + err = bpf_map_update_elem(cb_opts->map_fd, &fd, + &expected_stg, BPF_NOEXIST); + if (!ASSERT_OK(err, "bpf_map_update_elem(sk_stg_map)")) + return err; + + return 0; +} + static void test_dctcp(void) { + struct cb_opts cb_opts = { + .cc = "bpf_dctcp", + }; + struct network_helper_opts opts = { + .post_socket_cb = cc_cb, + .cb_opts = &cb_opts, + }; + struct network_helper_opts cli_opts = { + .post_socket_cb = stg_post_socket_cb, + .cb_opts = &cb_opts, + }; + int lfd = -1, fd = -1, tmp_stg, err; struct bpf_dctcp *dctcp_skel; struct bpf_link *link; @@ -119,11 +164,25 @@ static void test_dctcp(void) return; } - do_test("bpf_dctcp", dctcp_skel->maps.sk_stg_map); + cb_opts.map_fd = bpf_map__fd(dctcp_skel->maps.sk_stg_map); + if (!start_test(NULL, &opts, &cli_opts, &lfd, &fd)) + goto done; + + err = bpf_map_lookup_elem(cb_opts.map_fd, &fd, &tmp_stg); + if (!ASSERT_ERR(err, "bpf_map_lookup_elem(sk_stg_map)") || + !ASSERT_EQ(errno, ENOENT, "bpf_map_lookup_elem(sk_stg_map)")) + goto done; + + ASSERT_OK(send_recv_data(lfd, fd, total_bytes), "send_recv_data"); ASSERT_EQ(dctcp_skel->bss->stg_result, expected_stg, "stg_result"); +done: bpf_link__destroy(link); bpf_dctcp__destroy(dctcp_skel); + if (lfd != -1) + close(lfd); + if (fd != -1) + close(fd); } static char *err_str; @@ -171,11 +230,22 @@ static void test_invalid_license(void) static void test_dctcp_fallback(void) { int err, lfd = -1, cli_fd = -1, srv_fd = -1; - struct network_helper_opts opts = { - .cc = "cubic", - }; struct bpf_dctcp *dctcp_skel; struct bpf_link *link = NULL; + struct cb_opts dctcp = { + .cc = "bpf_dctcp", + }; + struct network_helper_opts srv_opts = { + .post_socket_cb = cc_cb, + .cb_opts = &dctcp, + }; + struct cb_opts cubic = { + .cc = "cubic", + }; + struct network_helper_opts cli_opts = { + .post_socket_cb = cc_cb, + .cb_opts = &cubic, + }; char srv_cc[16]; socklen_t cc_len = sizeof(srv_cc); @@ -190,13 +260,7 @@ static void test_dctcp_fallback(void) if (!ASSERT_OK_PTR(link, "dctcp link")) goto done; - lfd = start_server(AF_INET6, SOCK_STREAM, "::1", 0, 0); - if (!ASSERT_GE(lfd, 0, "lfd") || - !ASSERT_OK(settcpca(lfd, "bpf_dctcp"), "lfd=>bpf_dctcp")) - goto done; - - cli_fd = connect_to_fd_opts(lfd, &opts); - if (!ASSERT_GE(cli_fd, 0, "cli_fd")) + if (!start_test("::1", &srv_opts, &cli_opts, &lfd, &cli_fd)) goto done; srv_fd = accept(lfd, NULL, 0); @@ -297,6 +361,13 @@ static void test_unsupp_cong_op(void) static void test_update_ca(void) { + struct cb_opts cb_opts = { + .cc = "tcp_ca_update", + }; + struct network_helper_opts opts = { + .post_socket_cb = cc_cb, + .cb_opts = &cb_opts, + }; struct tcp_ca_update *skel; struct bpf_link *link; int saved_ca1_cnt; @@ -309,14 +380,14 @@ static void test_update_ca(void) link = bpf_map__attach_struct_ops(skel->maps.ca_update_1); ASSERT_OK_PTR(link, "attach_struct_ops"); - do_test("tcp_ca_update", NULL); + do_test(&opts); saved_ca1_cnt = skel->bss->ca1_cnt; ASSERT_GT(saved_ca1_cnt, 0, "ca1_ca1_cnt"); err = bpf_link__update_map(link, skel->maps.ca_update_2); ASSERT_OK(err, "update_map"); - do_test("tcp_ca_update", NULL); + do_test(&opts); ASSERT_EQ(skel->bss->ca1_cnt, saved_ca1_cnt, "ca2_ca1_cnt"); ASSERT_GT(skel->bss->ca2_cnt, 0, "ca2_ca2_cnt"); @@ -326,6 +397,13 @@ static void test_update_ca(void) static void test_update_wrong(void) { + struct cb_opts cb_opts = { + .cc = "tcp_ca_update", + }; + struct network_helper_opts opts = { + .post_socket_cb = cc_cb, + .cb_opts = &cb_opts, + }; struct tcp_ca_update *skel; struct bpf_link *link; int saved_ca1_cnt; @@ -338,14 +416,14 @@ static void test_update_wrong(void) link = bpf_map__attach_struct_ops(skel->maps.ca_update_1); ASSERT_OK_PTR(link, "attach_struct_ops"); - do_test("tcp_ca_update", NULL); + do_test(&opts); saved_ca1_cnt = skel->bss->ca1_cnt; ASSERT_GT(saved_ca1_cnt, 0, "ca1_ca1_cnt"); err = bpf_link__update_map(link, skel->maps.ca_wrong); ASSERT_ERR(err, "update_map"); - do_test("tcp_ca_update", NULL); + do_test(&opts); ASSERT_GT(skel->bss->ca1_cnt, saved_ca1_cnt, "ca2_ca1_cnt"); bpf_link__destroy(link); @@ -354,6 +432,13 @@ static void test_update_wrong(void) static void test_mixed_links(void) { + struct cb_opts cb_opts = { + .cc = "tcp_ca_update", + }; + struct network_helper_opts opts = { + .post_socket_cb = cc_cb, + .cb_opts = &cb_opts, + }; struct tcp_ca_update *skel; struct bpf_link *link, *link_nl; int err; @@ -368,7 +453,7 @@ static void test_mixed_links(void) link = bpf_map__attach_struct_ops(skel->maps.ca_update_1); ASSERT_OK_PTR(link, "attach_struct_ops"); - do_test("tcp_ca_update", NULL); + do_test(&opts); ASSERT_GT(skel->bss->ca1_cnt, 0, "ca1_ca1_cnt"); err = bpf_link__update_map(link, skel->maps.ca_no_link); @@ -455,6 +540,13 @@ static void test_tcp_ca_kfunc(void) static void test_cc_cubic(void) { + struct cb_opts cb_opts = { + .cc = "bpf_cc_cubic", + }; + struct network_helper_opts opts = { + .post_socket_cb = cc_cb, + .cb_opts = &cb_opts, + }; struct bpf_cc_cubic *cc_cubic_skel; struct bpf_link *link; @@ -468,7 +560,7 @@ static void test_cc_cubic(void) return; } - do_test("bpf_cc_cubic", NULL); + do_test(&opts); bpf_link__destroy(link); bpf_cc_cubic__destroy(cc_cubic_skel); diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c b/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c index 4c6ada5b270b240b6038c6d2ded0d7e3190105e3..73f669014b6923d0d00564de4dd87cde7d7c3104 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c @@ -45,12 +45,6 @@ err_out: return err; } -struct scale_test_def { - const char *file; - enum bpf_prog_type attach_type; - bool fails; -}; - static void scale_test(const char *file, enum bpf_prog_type attach_type, bool should_fail) diff --git a/tools/testing/selftests/bpf/prog_tests/btf_distill.c b/tools/testing/selftests/bpf/prog_tests/btf_distill.c new file mode 100644 index 0000000000000000000000000000000000000000..bfbe795823a29e6dfaf713c2e60a10da821d77e2 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/btf_distill.c @@ -0,0 +1,552 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#include +#include +#include "btf_helpers.h" + +/* Fabricate base, split BTF with references to base types needed; then create + * split BTF with distilled base BTF and ensure expectations are met: + * - only referenced base types from split BTF are present + * - struct/union/enum are represented as empty unless anonymous, when they + * are represented in full in split BTF + */ +static void test_distilled_base(void) +{ + struct btf *btf1 = NULL, *btf2 = NULL, *btf3 = NULL, *btf4 = NULL; + + btf1 = btf__new_empty(); + if (!ASSERT_OK_PTR(btf1, "empty_main_btf")) + return; + + btf__add_int(btf1, "int", 4, BTF_INT_SIGNED); /* [1] int */ + btf__add_ptr(btf1, 1); /* [2] ptr to int */ + btf__add_struct(btf1, "s1", 8); /* [3] struct s1 { */ + btf__add_field(btf1, "f1", 2, 0, 0); /* int *f1; */ + /* } */ + btf__add_struct(btf1, "", 12); /* [4] struct { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + btf__add_field(btf1, "f2", 3, 32, 0); /* struct s1 f2; */ + /* } */ + btf__add_int(btf1, "unsigned int", 4, 0); /* [5] unsigned int */ + btf__add_union(btf1, "u1", 12); /* [6] union u1 { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + btf__add_field(btf1, "f2", 2, 0, 0); /* int *f2; */ + /* } */ + btf__add_union(btf1, "", 4); /* [7] union { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_enum(btf1, "e1", 4); /* [8] enum e1 { */ + btf__add_enum_value(btf1, "v1", 1); /* v1 = 1; */ + /* } */ + btf__add_enum(btf1, "", 4); /* [9] enum { */ + btf__add_enum_value(btf1, "av1", 2); /* av1 = 2; */ + /* } */ + btf__add_enum64(btf1, "e641", 8, true); /* [10] enum64 { */ + btf__add_enum64_value(btf1, "v1", 1024); /* v1 = 1024; */ + /* } */ + btf__add_enum64(btf1, "", 8, true); /* [11] enum64 { */ + btf__add_enum64_value(btf1, "v1", 1025); /* v1 = 1025; */ + /* } */ + btf__add_struct(btf1, "unneeded", 4); /* [12] struct unneeded { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_struct(btf1, "embedded", 4); /* [13] struct embedded { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_func_proto(btf1, 1); /* [14] int (*)(int *p1); */ + btf__add_func_param(btf1, "p1", 1); + + btf__add_array(btf1, 1, 1, 3); /* [15] int [3]; */ + + btf__add_struct(btf1, "from_proto", 4); /* [16] struct from_proto { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_union(btf1, "u1", 4); /* [17] union u1 { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + VALIDATE_RAW_BTF( + btf1, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[16] STRUCT 'from_proto' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[17] UNION 'u1' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0"); + + btf2 = btf__new_empty_split(btf1); + if (!ASSERT_OK_PTR(btf2, "empty_split_btf")) + goto cleanup; + + btf__add_ptr(btf2, 3); /* [18] ptr to struct s1 */ + /* add ptr to struct anon */ + btf__add_ptr(btf2, 4); /* [19] ptr to struct (anon) */ + btf__add_const(btf2, 6); /* [20] const union u1 */ + btf__add_restrict(btf2, 7); /* [21] restrict union (anon) */ + btf__add_volatile(btf2, 8); /* [22] volatile enum e1 */ + btf__add_typedef(btf2, "et", 9); /* [23] typedef enum (anon) */ + btf__add_const(btf2, 10); /* [24] const enum64 e641 */ + btf__add_ptr(btf2, 11); /* [25] restrict enum64 (anon) */ + btf__add_struct(btf2, "with_embedded", 4); /* [26] struct with_embedded { */ + btf__add_field(btf2, "f1", 13, 0, 0); /* struct embedded f1; */ + /* } */ + btf__add_func(btf2, "fn", BTF_FUNC_STATIC, 14); /* [27] int fn(int p1); */ + btf__add_typedef(btf2, "arraytype", 15); /* [28] typedef int[3] foo; */ + btf__add_func_proto(btf2, 1); /* [29] int (*)(struct from proto p1); */ + btf__add_func_param(btf2, "p1", 16); + + VALIDATE_RAW_BTF( + btf2, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[16] STRUCT 'from_proto' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[17] UNION 'u1' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[18] PTR '(anon)' type_id=3", + "[19] PTR '(anon)' type_id=4", + "[20] CONST '(anon)' type_id=6", + "[21] RESTRICT '(anon)' type_id=7", + "[22] VOLATILE '(anon)' type_id=8", + "[23] TYPEDEF 'et' type_id=9", + "[24] CONST '(anon)' type_id=10", + "[25] PTR '(anon)' type_id=11", + "[26] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=13 bits_offset=0", + "[27] FUNC 'fn' type_id=14 linkage=static", + "[28] TYPEDEF 'arraytype' type_id=15", + "[29] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=16"); + + if (!ASSERT_EQ(0, btf__distill_base(btf2, &btf3, &btf4), + "distilled_base") || + !ASSERT_OK_PTR(btf3, "distilled_base") || + !ASSERT_OK_PTR(btf4, "distilled_split") || + !ASSERT_EQ(8, btf__type_cnt(btf3), "distilled_base_type_cnt")) + goto cleanup; + + VALIDATE_RAW_BTF( + btf4, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] STRUCT 's1' size=8 vlen=0", + "[3] UNION 'u1' size=12 vlen=0", + "[4] ENUM 'e1' encoding=UNSIGNED size=4 vlen=0", + "[5] ENUM 'e641' encoding=UNSIGNED size=8 vlen=0", + "[6] STRUCT 'embedded' size=4 vlen=0", + "[7] STRUCT 'from_proto' size=4 vlen=0", + /* split BTF; these types should match split BTF above from 17-28, with + * updated type id references + */ + "[8] PTR '(anon)' type_id=2", + "[9] PTR '(anon)' type_id=20", + "[10] CONST '(anon)' type_id=3", + "[11] RESTRICT '(anon)' type_id=21", + "[12] VOLATILE '(anon)' type_id=4", + "[13] TYPEDEF 'et' type_id=22", + "[14] CONST '(anon)' type_id=5", + "[15] PTR '(anon)' type_id=23", + "[16] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=6 bits_offset=0", + "[17] FUNC 'fn' type_id=24 linkage=static", + "[18] TYPEDEF 'arraytype' type_id=25", + "[19] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=7", + /* split BTF types added from original base BTF below */ + "[20] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=32", + "[21] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[22] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[23] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[24] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[25] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3"); + + if (!ASSERT_EQ(btf__relocate(btf4, btf1), 0, "relocate_split")) + goto cleanup; + + VALIDATE_RAW_BTF( + btf4, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[16] STRUCT 'from_proto' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[17] UNION 'u1' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[18] PTR '(anon)' type_id=3", + "[19] PTR '(anon)' type_id=30", + "[20] CONST '(anon)' type_id=6", + "[21] RESTRICT '(anon)' type_id=31", + "[22] VOLATILE '(anon)' type_id=8", + "[23] TYPEDEF 'et' type_id=32", + "[24] CONST '(anon)' type_id=10", + "[25] PTR '(anon)' type_id=33", + "[26] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=13 bits_offset=0", + "[27] FUNC 'fn' type_id=34 linkage=static", + "[28] TYPEDEF 'arraytype' type_id=35", + "[29] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=16", + /* below here are (duplicate) anon base types added by distill + * process to split BTF. + */ + "[30] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[31] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[32] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[33] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[34] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[35] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3"); + +cleanup: + btf__free(btf4); + btf__free(btf3); + btf__free(btf2); + btf__free(btf1); +} + +/* ensure we can cope with multiple types with the same name in + * distilled base BTF. In this case because sizes are different, + * we can still disambiguate them. + */ +static void test_distilled_base_multi(void) +{ + struct btf *btf1 = NULL, *btf2 = NULL, *btf3 = NULL, *btf4 = NULL; + + btf1 = btf__new_empty(); + if (!ASSERT_OK_PTR(btf1, "empty_main_btf")) + return; + btf__add_int(btf1, "int", 4, BTF_INT_SIGNED); /* [1] int */ + btf__add_int(btf1, "int", 8, BTF_INT_SIGNED); /* [2] int */ + VALIDATE_RAW_BTF( + btf1, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED"); + btf2 = btf__new_empty_split(btf1); + if (!ASSERT_OK_PTR(btf2, "empty_split_btf")) + goto cleanup; + btf__add_ptr(btf2, 1); + btf__add_const(btf2, 2); + VALIDATE_RAW_BTF( + btf2, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED", + "[3] PTR '(anon)' type_id=1", + "[4] CONST '(anon)' type_id=2"); + if (!ASSERT_EQ(0, btf__distill_base(btf2, &btf3, &btf4), + "distilled_base") || + !ASSERT_OK_PTR(btf3, "distilled_base") || + !ASSERT_OK_PTR(btf4, "distilled_split") || + !ASSERT_EQ(3, btf__type_cnt(btf3), "distilled_base_type_cnt")) + goto cleanup; + VALIDATE_RAW_BTF( + btf3, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED"); + if (!ASSERT_EQ(btf__relocate(btf4, btf1), 0, "relocate_split")) + goto cleanup; + + VALIDATE_RAW_BTF( + btf4, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED", + "[3] PTR '(anon)' type_id=1", + "[4] CONST '(anon)' type_id=2"); + +cleanup: + btf__free(btf4); + btf__free(btf3); + btf__free(btf2); + btf__free(btf1); +} + +/* If a needed type is not present in the base BTF we wish to relocate + * with, btf__relocate() should error our. + */ +static void test_distilled_base_missing_err(void) +{ + struct btf *btf1 = NULL, *btf2 = NULL, *btf3 = NULL, *btf4 = NULL, *btf5 = NULL; + + btf1 = btf__new_empty(); + if (!ASSERT_OK_PTR(btf1, "empty_main_btf")) + return; + btf__add_int(btf1, "int", 4, BTF_INT_SIGNED); /* [1] int */ + btf__add_int(btf1, "int", 8, BTF_INT_SIGNED); /* [2] int */ + VALIDATE_RAW_BTF( + btf1, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED"); + btf2 = btf__new_empty_split(btf1); + if (!ASSERT_OK_PTR(btf2, "empty_split_btf")) + goto cleanup; + btf__add_ptr(btf2, 1); + btf__add_const(btf2, 2); + VALIDATE_RAW_BTF( + btf2, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED", + "[3] PTR '(anon)' type_id=1", + "[4] CONST '(anon)' type_id=2"); + if (!ASSERT_EQ(0, btf__distill_base(btf2, &btf3, &btf4), + "distilled_base") || + !ASSERT_OK_PTR(btf3, "distilled_base") || + !ASSERT_OK_PTR(btf4, "distilled_split") || + !ASSERT_EQ(3, btf__type_cnt(btf3), "distilled_base_type_cnt")) + goto cleanup; + VALIDATE_RAW_BTF( + btf3, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED"); + btf5 = btf__new_empty(); + if (!ASSERT_OK_PTR(btf5, "empty_reloc_btf")) + return; + btf__add_int(btf5, "int", 4, BTF_INT_SIGNED); /* [1] int */ + VALIDATE_RAW_BTF( + btf5, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED"); + ASSERT_EQ(btf__relocate(btf4, btf5), -EINVAL, "relocate_split"); + +cleanup: + btf__free(btf5); + btf__free(btf4); + btf__free(btf3); + btf__free(btf2); + btf__free(btf1); +} + +/* With 2 types of same size in distilled base BTF, relocation should + * fail as we have no means to choose between them. + */ +static void test_distilled_base_multi_err(void) +{ + struct btf *btf1 = NULL, *btf2 = NULL, *btf3 = NULL, *btf4 = NULL; + + btf1 = btf__new_empty(); + if (!ASSERT_OK_PTR(btf1, "empty_main_btf")) + return; + btf__add_int(btf1, "int", 4, BTF_INT_SIGNED); /* [1] int */ + btf__add_int(btf1, "int", 4, BTF_INT_SIGNED); /* [2] int */ + VALIDATE_RAW_BTF( + btf1, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED"); + btf2 = btf__new_empty_split(btf1); + if (!ASSERT_OK_PTR(btf2, "empty_split_btf")) + goto cleanup; + btf__add_ptr(btf2, 1); + btf__add_const(btf2, 2); + VALIDATE_RAW_BTF( + btf2, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[3] PTR '(anon)' type_id=1", + "[4] CONST '(anon)' type_id=2"); + if (!ASSERT_EQ(0, btf__distill_base(btf2, &btf3, &btf4), + "distilled_base") || + !ASSERT_OK_PTR(btf3, "distilled_base") || + !ASSERT_OK_PTR(btf4, "distilled_split") || + !ASSERT_EQ(3, btf__type_cnt(btf3), "distilled_base_type_cnt")) + goto cleanup; + VALIDATE_RAW_BTF( + btf3, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED"); + ASSERT_EQ(btf__relocate(btf4, btf1), -EINVAL, "relocate_split"); +cleanup: + btf__free(btf4); + btf__free(btf3); + btf__free(btf2); + btf__free(btf1); +} + +/* With 2 types of same size in base BTF, relocation should + * fail as we have no means to choose between them. + */ +static void test_distilled_base_multi_err2(void) +{ + struct btf *btf1 = NULL, *btf2 = NULL, *btf3 = NULL, *btf4 = NULL, *btf5 = NULL; + + btf1 = btf__new_empty(); + if (!ASSERT_OK_PTR(btf1, "empty_main_btf")) + return; + btf__add_int(btf1, "int", 4, BTF_INT_SIGNED); /* [1] int */ + VALIDATE_RAW_BTF( + btf1, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED"); + btf2 = btf__new_empty_split(btf1); + if (!ASSERT_OK_PTR(btf2, "empty_split_btf")) + goto cleanup; + btf__add_ptr(btf2, 1); + VALIDATE_RAW_BTF( + btf2, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1"); + if (!ASSERT_EQ(0, btf__distill_base(btf2, &btf3, &btf4), + "distilled_base") || + !ASSERT_OK_PTR(btf3, "distilled_base") || + !ASSERT_OK_PTR(btf4, "distilled_split") || + !ASSERT_EQ(2, btf__type_cnt(btf3), "distilled_base_type_cnt")) + goto cleanup; + VALIDATE_RAW_BTF( + btf3, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED"); + btf5 = btf__new_empty(); + if (!ASSERT_OK_PTR(btf5, "empty_reloc_btf")) + return; + btf__add_int(btf5, "int", 4, BTF_INT_SIGNED); /* [1] int */ + btf__add_int(btf5, "int", 4, BTF_INT_SIGNED); /* [2] int */ + VALIDATE_RAW_BTF( + btf5, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED"); + ASSERT_EQ(btf__relocate(btf4, btf5), -EINVAL, "relocate_split"); +cleanup: + btf__free(btf5); + btf__free(btf4); + btf__free(btf3); + btf__free(btf2); + btf__free(btf1); +} + +/* create split reference BTF from vmlinux + split BTF with a few type references; + * ensure the resultant split reference BTF is as expected, containing only types + * needed to disambiguate references from split BTF. + */ +static void test_distilled_base_vmlinux(void) +{ + struct btf *split_btf = NULL, *vmlinux_btf = btf__load_vmlinux_btf(); + struct btf *split_dist = NULL, *base_dist = NULL; + __s32 int_id, myint_id; + + if (!ASSERT_OK_PTR(vmlinux_btf, "load_vmlinux")) + return; + int_id = btf__find_by_name_kind(vmlinux_btf, "int", BTF_KIND_INT); + if (!ASSERT_GT(int_id, 0, "find_int")) + goto cleanup; + split_btf = btf__new_empty_split(vmlinux_btf); + if (!ASSERT_OK_PTR(split_btf, "new_split")) + goto cleanup; + myint_id = btf__add_typedef(split_btf, "myint", int_id); + btf__add_ptr(split_btf, myint_id); + + if (!ASSERT_EQ(btf__distill_base(split_btf, &base_dist, &split_dist), 0, + "distill_vmlinux_base")) + goto cleanup; + + if (!ASSERT_OK_PTR(split_dist, "split_distilled") || + !ASSERT_OK_PTR(base_dist, "base_dist")) + goto cleanup; + VALIDATE_RAW_BTF( + split_dist, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] TYPEDEF 'myint' type_id=1", + "[3] PTR '(anon)' type_id=2"); + +cleanup: + btf__free(split_dist); + btf__free(base_dist); + btf__free(split_btf); + btf__free(vmlinux_btf); +} + +void test_btf_distill(void) +{ + if (test__start_subtest("distilled_base")) + test_distilled_base(); + if (test__start_subtest("distilled_base_multi")) + test_distilled_base_multi(); + if (test__start_subtest("distilled_base_missing_err")) + test_distilled_base_missing_err(); + if (test__start_subtest("distilled_base_multi_err")) + test_distilled_base_multi_err(); + if (test__start_subtest("distilled_base_multi_err2")) + test_distilled_base_multi_err2(); + if (test__start_subtest("distilled_base_vmlinux")) + test_distilled_base_vmlinux(); +} diff --git a/tools/testing/selftests/bpf/prog_tests/btf_field_iter.c b/tools/testing/selftests/bpf/prog_tests/btf_field_iter.c new file mode 100644 index 0000000000000000000000000000000000000000..32159d3eb281b76eabadf4d0fa0b3dbde752c478 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/btf_field_iter.c @@ -0,0 +1,161 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#include +#include +#include "btf_helpers.h" +#include "bpf/libbpf_internal.h" + +struct field_data { + __u32 ids[5]; + const char *strs[5]; +} fields[] = { + { .ids = {}, .strs = {} }, + { .ids = {}, .strs = { "int" } }, + { .ids = {}, .strs = { "int64" } }, + { .ids = { 1 }, .strs = { "" } }, + { .ids = { 2, 1 }, .strs = { "" } }, + { .ids = { 3, 1 }, .strs = { "s1", "f1", "f2" } }, + { .ids = { 1, 5 }, .strs = { "u1", "f1", "f2" } }, + { .ids = {}, .strs = { "e1", "v1", "v2" } }, + { .ids = {}, .strs = { "fw1" } }, + { .ids = { 1 }, .strs = { "t" } }, + { .ids = { 2 }, .strs = { "" } }, + { .ids = { 1 }, .strs = { "" } }, + { .ids = { 3 }, .strs = { "" } }, + { .ids = { 1, 1, 3 }, .strs = { "", "p1", "p2" } }, + { .ids = { 13 }, .strs = { "func" } }, + { .ids = { 1 }, .strs = { "var1" } }, + { .ids = { 3 }, .strs = { "var2" } }, + { .ids = {}, .strs = { "float" } }, + { .ids = { 11 }, .strs = { "decltag" } }, + { .ids = { 6 }, .strs = { "typetag" } }, + { .ids = {}, .strs = { "e64", "eval1", "eval2", "eval3" } }, + { .ids = { 15, 16 }, .strs = { "datasec1" } } + +}; + +/* Fabricate BTF with various types and check BTF field iteration finds types, + * strings expected. + */ +void test_btf_field_iter(void) +{ + struct btf *btf = NULL; + int id; + + btf = btf__new_empty(); + if (!ASSERT_OK_PTR(btf, "empty_btf")) + return; + + btf__add_int(btf, "int", 4, BTF_INT_SIGNED); /* [1] int */ + btf__add_int(btf, "int64", 8, BTF_INT_SIGNED); /* [2] int64 */ + btf__add_ptr(btf, 1); /* [3] int * */ + btf__add_array(btf, 1, 2, 3); /* [4] int64[3] */ + btf__add_struct(btf, "s1", 12); /* [5] struct s1 { */ + btf__add_field(btf, "f1", 3, 0, 0); /* int *f1; */ + btf__add_field(btf, "f2", 1, 0, 0); /* int f2; */ + /* } */ + btf__add_union(btf, "u1", 12); /* [6] union u1 { */ + btf__add_field(btf, "f1", 1, 0, 0); /* int f1; */ + btf__add_field(btf, "f2", 5, 0, 0); /* struct s1 f2; */ + /* } */ + btf__add_enum(btf, "e1", 4); /* [7] enum e1 { */ + btf__add_enum_value(btf, "v1", 1); /* v1 = 1; */ + btf__add_enum_value(btf, "v2", 2); /* v2 = 2; */ + /* } */ + + btf__add_fwd(btf, "fw1", BTF_FWD_STRUCT); /* [8] struct fw1; */ + btf__add_typedef(btf, "t", 1); /* [9] typedef int t; */ + btf__add_volatile(btf, 2); /* [10] volatile int64; */ + btf__add_const(btf, 1); /* [11] const int; */ + btf__add_restrict(btf, 3); /* [12] restrict int *; */ + btf__add_func_proto(btf, 1); /* [13] int (*)(int p1, int *p2); */ + btf__add_func_param(btf, "p1", 1); + btf__add_func_param(btf, "p2", 3); + + btf__add_func(btf, "func", BTF_FUNC_GLOBAL, 13);/* [14] int func(int p1, int *p2); */ + btf__add_var(btf, "var1", BTF_VAR_STATIC, 1); /* [15] static int var1; */ + btf__add_var(btf, "var2", BTF_VAR_STATIC, 3); /* [16] static int *var2; */ + btf__add_float(btf, "float", 4); /* [17] float; */ + btf__add_decl_tag(btf, "decltag", 11, -1); /* [18] decltag const int; */ + btf__add_type_tag(btf, "typetag", 6); /* [19] typetag union u1; */ + btf__add_enum64(btf, "e64", 8, true); /* [20] enum { */ + btf__add_enum64_value(btf, "eval1", 1000); /* eval1 = 1000, */ + btf__add_enum64_value(btf, "eval2", 2000); /* eval2 = 2000, */ + btf__add_enum64_value(btf, "eval3", 3000); /* eval3 = 3000 */ + /* } */ + btf__add_datasec(btf, "datasec1", 12); /* [21] datasec datasec1 */ + btf__add_datasec_var_info(btf, 15, 0, 4); + btf__add_datasec_var_info(btf, 16, 4, 8); + + VALIDATE_RAW_BTF( + btf, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] INT 'int64' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED", + "[3] PTR '(anon)' type_id=1", + "[4] ARRAY '(anon)' type_id=2 index_type_id=1 nr_elems=3", + "[5] STRUCT 's1' size=12 vlen=2\n" + "\t'f1' type_id=3 bits_offset=0\n" + "\t'f2' type_id=1 bits_offset=0", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=5 bits_offset=0", + "[7] ENUM 'e1' encoding=UNSIGNED size=4 vlen=2\n" + "\t'v1' val=1\n" + "\t'v2' val=2", + "[8] FWD 'fw1' fwd_kind=struct", + "[9] TYPEDEF 't' type_id=1", + "[10] VOLATILE '(anon)' type_id=2", + "[11] CONST '(anon)' type_id=1", + "[12] RESTRICT '(anon)' type_id=3", + "[13] FUNC_PROTO '(anon)' ret_type_id=1 vlen=2\n" + "\t'p1' type_id=1\n" + "\t'p2' type_id=3", + "[14] FUNC 'func' type_id=13 linkage=global", + "[15] VAR 'var1' type_id=1, linkage=static", + "[16] VAR 'var2' type_id=3, linkage=static", + "[17] FLOAT 'float' size=4", + "[18] DECL_TAG 'decltag' type_id=11 component_idx=-1", + "[19] TYPE_TAG 'typetag' type_id=6", + "[20] ENUM64 'e64' encoding=SIGNED size=8 vlen=3\n" + "\t'eval1' val=1000\n" + "\t'eval2' val=2000\n" + "\t'eval3' val=3000", + "[21] DATASEC 'datasec1' size=12 vlen=2\n" + "\ttype_id=15 offset=0 size=4\n" + "\ttype_id=16 offset=4 size=8"); + + for (id = 1; id < btf__type_cnt(btf); id++) { + struct btf_type *t = btf_type_by_id(btf, id); + struct btf_field_iter it_strs, it_ids; + int str_idx = 0, id_idx = 0; + __u32 *next_str, *next_id; + + if (!ASSERT_OK_PTR(t, "btf_type_by_id")) + break; + if (!ASSERT_OK(btf_field_iter_init(&it_strs, t, BTF_FIELD_ITER_STRS), + "iter_init_strs")) + break; + if (!ASSERT_OK(btf_field_iter_init(&it_ids, t, BTF_FIELD_ITER_IDS), + "iter_init_ids")) + break; + while ((next_str = btf_field_iter_next(&it_strs))) { + const char *str = btf__str_by_offset(btf, *next_str); + + if (!ASSERT_OK(strcmp(fields[id].strs[str_idx], str), "field_str_match")) + break; + str_idx++; + } + /* ensure no more strings are expected */ + ASSERT_EQ(fields[id].strs[str_idx], NULL, "field_str_cnt"); + + while ((next_id = btf_field_iter_next(&it_ids))) { + if (!ASSERT_EQ(*next_id, fields[id].ids[id_idx], "field_id_match")) + break; + id_idx++; + } + /* ensure no more ids are expected */ + ASSERT_EQ(fields[id].ids[id_idx], 0, "field_id_cnt"); + } + btf__free(btf); +} diff --git a/tools/testing/selftests/bpf/prog_tests/cpumask.c b/tools/testing/selftests/bpf/prog_tests/cpumask.c index ecf89df781098e27fabcfb75341410271b497d54..2570bd4b0cb228d3cdb9aae05a22cb4cb5534758 100644 --- a/tools/testing/selftests/bpf/prog_tests/cpumask.c +++ b/tools/testing/selftests/bpf/prog_tests/cpumask.c @@ -18,6 +18,11 @@ static const char * const cpumask_success_testcases[] = { "test_insert_leave", "test_insert_remove_release", "test_global_mask_rcu", + "test_global_mask_array_one_rcu", + "test_global_mask_array_rcu", + "test_global_mask_array_l2_rcu", + "test_global_mask_nested_rcu", + "test_global_mask_nested_deep_rcu", "test_cpumask_weight", }; diff --git a/tools/testing/selftests/bpf/prog_tests/ctx_rewrite.c b/tools/testing/selftests/bpf/prog_tests/ctx_rewrite.c index 3b7c57fe55a596e3f997cd06609c629ab2358deb..08b6391f2f56ce195e9dcc9d795fb715cfb7e187 100644 --- a/tools/testing/selftests/bpf/prog_tests/ctx_rewrite.c +++ b/tools/testing/selftests/bpf/prog_tests/ctx_rewrite.c @@ -69,15 +69,17 @@ static struct test_case test_cases[] = { { N(SCHED_CLS, struct __sk_buff, tstamp), .read = "r11 = *(u8 *)($ctx + sk_buff::__mono_tc_offset);" - "w11 &= 3;" - "if w11 != 0x3 goto pc+2;" + "if w11 & 0x4 goto pc+1;" + "goto pc+4;" + "if w11 & 0x3 goto pc+1;" + "goto pc+2;" "$dst = 0;" "goto pc+1;" "$dst = *(u64 *)($ctx + sk_buff::tstamp);", .write = "r11 = *(u8 *)($ctx + sk_buff::__mono_tc_offset);" - "if w11 & 0x2 goto pc+1;" + "if w11 & 0x4 goto pc+1;" "goto pc+2;" - "w11 &= -2;" + "w11 &= -4;" "*(u8 *)($ctx + sk_buff::__mono_tc_offset) = r11;" "*(u64 *)($ctx + sk_buff::tstamp) = $src;", }, diff --git a/tools/testing/selftests/bpf/prog_tests/find_vma.c b/tools/testing/selftests/bpf/prog_tests/find_vma.c index 5165b38f0e59fd27890b8fe22c04acd05358daad..f7619e0ade10eadd58215b4aa15dd74c343fe258 100644 --- a/tools/testing/selftests/bpf/prog_tests/find_vma.c +++ b/tools/testing/selftests/bpf/prog_tests/find_vma.c @@ -29,8 +29,8 @@ static int open_pe(void) /* create perf event */ attr.size = sizeof(attr); - attr.type = PERF_TYPE_HARDWARE; - attr.config = PERF_COUNT_HW_CPU_CYCLES; + attr.type = PERF_TYPE_SOFTWARE; + attr.config = PERF_COUNT_SW_CPU_CLOCK; attr.freq = 1; attr.sample_freq = 1000; pfd = syscall(__NR_perf_event_open, &attr, 0, -1, -1, PERF_FLAG_FD_CLOEXEC); diff --git a/tools/testing/selftests/bpf/prog_tests/kfunc_param_nullable.c b/tools/testing/selftests/bpf/prog_tests/kfunc_param_nullable.c new file mode 100644 index 0000000000000000000000000000000000000000..c8f4dcaac7c761f26b540d320d7c0d7cee4aaf91 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/kfunc_param_nullable.c @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright (c) 2024 Meta Platforms, Inc */ + +#include +#include "test_kfunc_param_nullable.skel.h" + +void test_kfunc_param_nullable(void) +{ + RUN_TESTS(test_kfunc_param_nullable); +} diff --git a/tools/testing/selftests/bpf/prog_tests/linked_list.c b/tools/testing/selftests/bpf/prog_tests/linked_list.c index 2fb89de63bd22ae2454eb51a9d05196d2dbcc4e2..77d07e0a4a55c56a20eefa99ab2f3f12f3ef4aca 100644 --- a/tools/testing/selftests/bpf/prog_tests/linked_list.c +++ b/tools/testing/selftests/bpf/prog_tests/linked_list.c @@ -183,6 +183,18 @@ static void test_linked_list_success(int mode, bool leave_in_map) if (!leave_in_map) clear_fields(skel->maps.bss_A); + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.global_list_push_pop_nested), &opts); + ASSERT_OK(ret, "global_list_push_pop_nested"); + ASSERT_OK(opts.retval, "global_list_push_pop_nested retval"); + if (!leave_in_map) + clear_fields(skel->maps.bss_A); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.global_list_array_push_pop), &opts); + ASSERT_OK(ret, "global_list_array_push_pop"); + ASSERT_OK(opts.retval, "global_list_array_push_pop retval"); + if (!leave_in_map) + clear_fields(skel->maps.bss_A); + if (mode == PUSH_POP) goto end; diff --git a/tools/testing/selftests/bpf/prog_tests/rbtree.c b/tools/testing/selftests/bpf/prog_tests/rbtree.c index e9300c96607d3148c2ffc9a241390bb490495f93..9818f06c97c5a11deb0d04cfbf09ba448fda0180 100644 --- a/tools/testing/selftests/bpf/prog_tests/rbtree.c +++ b/tools/testing/selftests/bpf/prog_tests/rbtree.c @@ -31,6 +31,28 @@ static void test_rbtree_add_nodes(void) rbtree__destroy(skel); } +static void test_rbtree_add_nodes_nested(void) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts, + .data_in = &pkt_v4, + .data_size_in = sizeof(pkt_v4), + .repeat = 1, + ); + struct rbtree *skel; + int ret; + + skel = rbtree__open_and_load(); + if (!ASSERT_OK_PTR(skel, "rbtree__open_and_load")) + return; + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.rbtree_add_nodes_nested), &opts); + ASSERT_OK(ret, "rbtree_add_nodes_nested run"); + ASSERT_OK(opts.retval, "rbtree_add_nodes_nested retval"); + ASSERT_EQ(skel->data->less_callback_ran, 1, "rbtree_add_nodes_nested less_callback_ran"); + + rbtree__destroy(skel); +} + static void test_rbtree_add_and_remove(void) { LIBBPF_OPTS(bpf_test_run_opts, opts, @@ -53,6 +75,27 @@ static void test_rbtree_add_and_remove(void) rbtree__destroy(skel); } +static void test_rbtree_add_and_remove_array(void) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts, + .data_in = &pkt_v4, + .data_size_in = sizeof(pkt_v4), + .repeat = 1, + ); + struct rbtree *skel; + int ret; + + skel = rbtree__open_and_load(); + if (!ASSERT_OK_PTR(skel, "rbtree__open_and_load")) + return; + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.rbtree_add_and_remove_array), &opts); + ASSERT_OK(ret, "rbtree_add_and_remove_array"); + ASSERT_OK(opts.retval, "rbtree_add_and_remove_array retval"); + + rbtree__destroy(skel); +} + static void test_rbtree_first_and_remove(void) { LIBBPF_OPTS(bpf_test_run_opts, opts, @@ -104,8 +147,12 @@ void test_rbtree_success(void) { if (test__start_subtest("rbtree_add_nodes")) test_rbtree_add_nodes(); + if (test__start_subtest("rbtree_add_nodes_nested")) + test_rbtree_add_nodes_nested(); if (test__start_subtest("rbtree_add_and_remove")) test_rbtree_add_and_remove(); + if (test__start_subtest("rbtree_add_and_remove_array")) + test_rbtree_add_and_remove_array(); if (test__start_subtest("rbtree_first_and_remove")) test_rbtree_first_and_remove(); if (test__start_subtest("rbtree_api_release_aliasing")) diff --git a/tools/testing/selftests/bpf/prog_tests/send_signal.c b/tools/testing/selftests/bpf/prog_tests/send_signal.c index 920aee41bd58c0a3a4263cfcb190b7f95ffaa6bf..6cc69900b3106f89acb09691a4d340de08e9287c 100644 --- a/tools/testing/selftests/bpf/prog_tests/send_signal.c +++ b/tools/testing/selftests/bpf/prog_tests/send_signal.c @@ -156,7 +156,8 @@ static void test_send_signal_tracepoint(bool signal_thread) static void test_send_signal_perf(bool signal_thread) { struct perf_event_attr attr = { - .sample_period = 1, + .freq = 1, + .sample_freq = 1000, .type = PERF_TYPE_SOFTWARE, .config = PERF_COUNT_SW_CPU_CLOCK, }; diff --git a/tools/testing/selftests/bpf/prog_tests/sockopt_inherit.c b/tools/testing/selftests/bpf/prog_tests/sockopt_inherit.c index 1d3a20f01b607625d9abd110733a4731591c7dbb..7cd8be2780cac74e3d50672228bd6b5952ec1dca 100644 --- a/tools/testing/selftests/bpf/prog_tests/sockopt_inherit.c +++ b/tools/testing/selftests/bpf/prog_tests/sockopt_inherit.c @@ -70,7 +70,7 @@ static void *server_thread(void *arg) return (void *)(long)err; } -static int custom_cb(int fd, const struct post_socket_opts *opts) +static int custom_cb(int fd, void *opts) { char buf; int err; diff --git a/tools/testing/selftests/bpf/prog_tests/tc_netkit.c b/tools/testing/selftests/bpf/prog_tests/tc_netkit.c index 15ee7b2fc4106428b3210de62df53a6c125060ec..b9135720024ccfc37b0a6946840248aa3e7f948f 100644 --- a/tools/testing/selftests/bpf/prog_tests/tc_netkit.c +++ b/tools/testing/selftests/bpf/prog_tests/tc_netkit.c @@ -73,6 +73,16 @@ static int create_netkit(int mode, int policy, int peer_policy, int *ifindex, "up primary"); ASSERT_OK(system("ip addr add dev " netkit_name " 10.0.0.1/24"), "addr primary"); + + if (mode == NETKIT_L3) { + ASSERT_EQ(system("ip link set dev " netkit_name + " addr ee:ff:bb:cc:aa:dd 2> /dev/null"), 512, + "set hwaddress"); + } else { + ASSERT_OK(system("ip link set dev " netkit_name + " addr ee:ff:bb:cc:aa:dd"), + "set hwaddress"); + } if (same_netns) { ASSERT_OK(system("ip link set dev " netkit_peer " up"), "up peer"); @@ -89,6 +99,16 @@ static int create_netkit(int mode, int policy, int peer_policy, int *ifindex, return err; } +static void move_netkit(void) +{ + ASSERT_OK(system("ip link set " netkit_peer " netns foo"), + "move peer"); + ASSERT_OK(system("ip netns exec foo ip link set dev " + netkit_peer " up"), "up peer"); + ASSERT_OK(system("ip netns exec foo ip addr add dev " + netkit_peer " 10.0.0.2/24"), "addr peer"); +} + static void destroy_netkit(void) { ASSERT_OK(system("ip link del dev " netkit_name), "del primary"); @@ -685,3 +705,77 @@ void serial_test_tc_netkit_neigh_links(void) serial_test_tc_netkit_neigh_links_target(NETKIT_L2, BPF_NETKIT_PRIMARY); serial_test_tc_netkit_neigh_links_target(NETKIT_L3, BPF_NETKIT_PRIMARY); } + +static void serial_test_tc_netkit_pkt_type_mode(int mode) +{ + LIBBPF_OPTS(bpf_netkit_opts, optl_nk); + LIBBPF_OPTS(bpf_tcx_opts, optl_tcx); + int err, ifindex, ifindex2; + struct test_tc_link *skel; + struct bpf_link *link; + + err = create_netkit(mode, NETKIT_PASS, NETKIT_PASS, + &ifindex, true); + if (err) + return; + + ifindex2 = if_nametoindex(netkit_peer); + ASSERT_NEQ(ifindex, ifindex2, "ifindex_1_2"); + + skel = test_tc_link__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + + ASSERT_EQ(bpf_program__set_expected_attach_type(skel->progs.tc1, + BPF_NETKIT_PRIMARY), 0, "tc1_attach_type"); + ASSERT_EQ(bpf_program__set_expected_attach_type(skel->progs.tc7, + BPF_TCX_INGRESS), 0, "tc7_attach_type"); + + err = test_tc_link__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + + assert_mprog_count_ifindex(ifindex, BPF_NETKIT_PRIMARY, 0); + assert_mprog_count_ifindex(ifindex2, BPF_TCX_INGRESS, 0); + + link = bpf_program__attach_netkit(skel->progs.tc1, ifindex, &optl_nk); + if (!ASSERT_OK_PTR(link, "link_attach")) + goto cleanup; + + skel->links.tc1 = link; + + assert_mprog_count_ifindex(ifindex, BPF_NETKIT_PRIMARY, 1); + assert_mprog_count_ifindex(ifindex2, BPF_TCX_INGRESS, 0); + + link = bpf_program__attach_tcx(skel->progs.tc7, ifindex2, &optl_tcx); + if (!ASSERT_OK_PTR(link, "link_attach")) + goto cleanup; + + skel->links.tc7 = link; + + assert_mprog_count_ifindex(ifindex, BPF_NETKIT_PRIMARY, 1); + assert_mprog_count_ifindex(ifindex2, BPF_TCX_INGRESS, 1); + + move_netkit(); + + tc_skel_reset_all_seen(skel); + skel->bss->set_type = true; + ASSERT_EQ(send_icmp(), 0, "icmp_pkt"); + + ASSERT_EQ(skel->bss->seen_tc1, true, "seen_tc1"); + ASSERT_EQ(skel->bss->seen_tc7, true, "seen_tc7"); + + ASSERT_EQ(skel->bss->seen_host, true, "seen_host"); + ASSERT_EQ(skel->bss->seen_mcast, true, "seen_mcast"); +cleanup: + test_tc_link__destroy(skel); + + assert_mprog_count_ifindex(ifindex, BPF_NETKIT_PRIMARY, 0); + destroy_netkit(); +} + +void serial_test_tc_netkit_pkt_type(void) +{ + serial_test_tc_netkit_pkt_type_mode(NETKIT_L2); + serial_test_tc_netkit_pkt_type_mode(NETKIT_L3); +} diff --git a/tools/testing/selftests/bpf/prog_tests/tc_redirect.c b/tools/testing/selftests/bpf/prog_tests/tc_redirect.c index b1073d36d77ac298a240bb92d65e7c614a3cf68f..327d51f591427993ffe337919c425878954b68a4 100644 --- a/tools/testing/selftests/bpf/prog_tests/tc_redirect.c +++ b/tools/testing/selftests/bpf/prog_tests/tc_redirect.c @@ -890,9 +890,6 @@ static void test_udp_dtime(struct test_tc_dtime *skel, int family, bool bpf_fwd) ASSERT_EQ(dtimes[INGRESS_FWDNS_P100], 0, dtime_cnt_str(t, INGRESS_FWDNS_P100)); - /* non mono delivery time is not forwarded */ - ASSERT_EQ(dtimes[INGRESS_FWDNS_P101], 0, - dtime_cnt_str(t, INGRESS_FWDNS_P101)); for (i = EGRESS_FWDNS_P100; i < SET_DTIME; i++) ASSERT_GT(dtimes[i], 0, dtime_cnt_str(t, i)); diff --git a/tools/testing/selftests/bpf/prog_tests/test_skb_pkt_end.c b/tools/testing/selftests/bpf/prog_tests/test_skb_pkt_end.c index ae93411fd5824c22b184d252d661e84ad3a968d8..09ca13bdf6ca2796490532d05d676d0bdfccc94e 100644 --- a/tools/testing/selftests/bpf/prog_tests/test_skb_pkt_end.c +++ b/tools/testing/selftests/bpf/prog_tests/test_skb_pkt_end.c @@ -11,6 +11,7 @@ static int sanity_run(struct bpf_program *prog) .data_in = &pkt_v4, .data_size_in = sizeof(pkt_v4), .repeat = 1, + .flags = BPF_F_TEST_SKB_CHECKSUM_COMPLETE, ); prog_fd = bpf_program__fd(prog); diff --git a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c index 29e183a80f49004d35e69bb39df67b5225a6be6e..bbcf12696a6bc8002b024f45bd181f536d53a3c4 100644 --- a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c +++ b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c @@ -3,9 +3,12 @@ #include #include +#include + #include "struct_ops_module.skel.h" #include "struct_ops_nulled_out_cb.skel.h" #include "struct_ops_forgotten_cb.skel.h" +#include "struct_ops_detach.skel.h" static void check_map_info(struct bpf_map_info *info) { @@ -242,6 +245,58 @@ cleanup: struct_ops_forgotten_cb__destroy(skel); } +/* Detach a link from a user space program */ +static void test_detach_link(void) +{ + struct epoll_event ev, events[2]; + struct struct_ops_detach *skel; + struct bpf_link *link = NULL; + int fd, epollfd = -1, nfds; + int err; + + skel = struct_ops_detach__open_and_load(); + if (!ASSERT_OK_PTR(skel, "struct_ops_detach__open_and_load")) + return; + + link = bpf_map__attach_struct_ops(skel->maps.testmod_do_detach); + if (!ASSERT_OK_PTR(link, "attach_struct_ops")) + goto cleanup; + + fd = bpf_link__fd(link); + if (!ASSERT_GE(fd, 0, "link_fd")) + goto cleanup; + + epollfd = epoll_create1(0); + if (!ASSERT_GE(epollfd, 0, "epoll_create1")) + goto cleanup; + + ev.events = EPOLLHUP; + ev.data.fd = fd; + err = epoll_ctl(epollfd, EPOLL_CTL_ADD, fd, &ev); + if (!ASSERT_OK(err, "epoll_ctl")) + goto cleanup; + + err = bpf_link__detach(link); + if (!ASSERT_OK(err, "detach_link")) + goto cleanup; + + /* Wait for EPOLLHUP */ + nfds = epoll_wait(epollfd, events, 2, 500); + if (!ASSERT_EQ(nfds, 1, "epoll_wait")) + goto cleanup; + + if (!ASSERT_EQ(events[0].data.fd, fd, "epoll_wait_fd")) + goto cleanup; + if (!ASSERT_TRUE(events[0].events & EPOLLHUP, "events[0].events")) + goto cleanup; + +cleanup: + if (epollfd >= 0) + close(epollfd); + bpf_link__destroy(link); + struct_ops_detach__destroy(skel); +} + void serial_test_struct_ops_module(void) { if (test__start_subtest("struct_ops_load")) @@ -254,5 +309,7 @@ void serial_test_struct_ops_module(void) test_struct_ops_nulled_out_cb(); if (test__start_subtest("struct_ops_forgotten_cb")) test_struct_ops_forgotten_cb(); + if (test__start_subtest("test_detach_link")) + test_detach_link(); } diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c index 8269cdee33ae97949c89f363b32d3d911b7dac21..bf6ca8e3eb131484300316d8097e9efac9454cbd 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_multi_test.c @@ -1,12 +1,14 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include #include #include "uprobe_multi.skel.h" #include "uprobe_multi_bench.skel.h" #include "uprobe_multi_usdt.skel.h" #include "bpf/libbpf_internal.h" #include "testing_helpers.h" +#include "../sdt.h" static char test_data[] = "test_data"; @@ -25,9 +27,17 @@ noinline void uprobe_multi_func_3(void) asm volatile (""); } +noinline void usdt_trigger(void) +{ + STAP_PROBE(test, pid_filter_usdt); +} + struct child { int go[2]; + int c2p[2]; /* child -> parent channel */ int pid; + int tid; + pthread_t thread; }; static void release_child(struct child *child) @@ -38,6 +48,10 @@ static void release_child(struct child *child) return; close(child->go[1]); close(child->go[0]); + if (child->thread) + pthread_join(child->thread, NULL); + close(child->c2p[0]); + close(child->c2p[1]); if (child->pid > 0) waitpid(child->pid, &child_status, 0); } @@ -63,7 +77,7 @@ static struct child *spawn_child(void) if (pipe(child.go)) return NULL; - child.pid = fork(); + child.pid = child.tid = fork(); if (child.pid < 0) { release_child(&child); errno = EINVAL; @@ -82,6 +96,7 @@ static struct child *spawn_child(void) uprobe_multi_func_1(); uprobe_multi_func_2(); uprobe_multi_func_3(); + usdt_trigger(); exit(errno); } @@ -89,6 +104,67 @@ static struct child *spawn_child(void) return &child; } +static void *child_thread(void *ctx) +{ + struct child *child = ctx; + int c = 0, err; + + child->tid = syscall(SYS_gettid); + + /* let parent know we are ready */ + err = write(child->c2p[1], &c, 1); + if (err != 1) + pthread_exit(&err); + + /* wait for parent's kick */ + err = read(child->go[0], &c, 1); + if (err != 1) + pthread_exit(&err); + + uprobe_multi_func_1(); + uprobe_multi_func_2(); + uprobe_multi_func_3(); + usdt_trigger(); + + err = 0; + pthread_exit(&err); +} + +static struct child *spawn_thread(void) +{ + static struct child child; + int c, err; + + /* pipe to notify child to execute the trigger functions */ + if (pipe(child.go)) + return NULL; + /* pipe to notify parent that child thread is ready */ + if (pipe(child.c2p)) { + close(child.go[0]); + close(child.go[1]); + return NULL; + } + + child.pid = getpid(); + + err = pthread_create(&child.thread, NULL, child_thread, &child); + if (err) { + err = -errno; + close(child.go[0]); + close(child.go[1]); + close(child.c2p[0]); + close(child.c2p[1]); + errno = -err; + return NULL; + } + + err = read(child.c2p[0], &c, 1); + if (!ASSERT_EQ(err, 1, "child_thread_ready")) + return NULL; + + return &child; +} + static void uprobe_multi_test_run(struct uprobe_multi *skel, struct child *child) { skel->bss->uprobe_multi_func_1_addr = (__u64) uprobe_multi_func_1; @@ -103,15 +179,23 @@ static void uprobe_multi_test_run(struct uprobe_multi *skel, struct child *child * passed at the probe attach. */ skel->bss->pid = child ? 0 : getpid(); + skel->bss->expect_pid = child ? child->pid : 0; + + /* trigger all probes, if we are testing child *process*, just to make + * sure that PID filtering doesn't let through activations from wrong + * PIDs; when we test child *thread*, we don't want to do this to + * avoid double counting number of triggering events + */ + if (!child || !child->thread) { + uprobe_multi_func_1(); + uprobe_multi_func_2(); + uprobe_multi_func_3(); + usdt_trigger(); + } if (child) kick_child(child); - /* trigger all probes */ - uprobe_multi_func_1(); - uprobe_multi_func_2(); - uprobe_multi_func_3(); - /* * There are 2 entry and 2 exit probe called for each uprobe_multi_func_[123] * function and each slepable probe (6) increments uprobe_multi_sleep_result. @@ -126,8 +210,12 @@ static void uprobe_multi_test_run(struct uprobe_multi *skel, struct child *child ASSERT_EQ(skel->bss->uprobe_multi_sleep_result, 6, "uprobe_multi_sleep_result"); - if (child) + ASSERT_FALSE(skel->bss->bad_pid_seen, "bad_pid_seen"); + + if (child) { ASSERT_EQ(skel->bss->child_pid, child->pid, "uprobe_multi_child_pid"); + ASSERT_EQ(skel->bss->child_tid, child->tid, "uprobe_multi_child_tid"); + } } static void test_skel_api(void) @@ -190,8 +278,24 @@ __test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_mul if (!ASSERT_OK_PTR(skel->links.uprobe_extra, "bpf_program__attach_uprobe_multi")) goto cleanup; + /* Attach (uprobe-backed) USDTs */ + skel->links.usdt_pid = bpf_program__attach_usdt(skel->progs.usdt_pid, pid, binary, + "test", "pid_filter_usdt", NULL); + if (!ASSERT_OK_PTR(skel->links.usdt_pid, "attach_usdt_pid")) + goto cleanup; + + skel->links.usdt_extra = bpf_program__attach_usdt(skel->progs.usdt_extra, -1, binary, + "test", "pid_filter_usdt", NULL); + if (!ASSERT_OK_PTR(skel->links.usdt_extra, "attach_usdt_extra")) + goto cleanup; + uprobe_multi_test_run(skel, child); + ASSERT_FALSE(skel->bss->bad_pid_seen_usdt, "bad_pid_seen_usdt"); + if (child) { + ASSERT_EQ(skel->bss->child_pid_usdt, child->pid, "usdt_multi_child_pid"); + ASSERT_EQ(skel->bss->child_tid_usdt, child->tid, "usdt_multi_child_tid"); + } cleanup: uprobe_multi__destroy(skel); } @@ -210,6 +314,13 @@ test_attach_api(const char *binary, const char *pattern, struct bpf_uprobe_multi return; __test_attach_api(binary, pattern, opts, child); + + /* pid filter (thread) */ + child = spawn_thread(); + if (!ASSERT_OK_PTR(child, "spawn_thread")) + return; + + __test_attach_api(binary, pattern, opts, child); } static void test_attach_api_pattern(void) @@ -397,7 +508,7 @@ static void test_attach_api_fails(void) link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_UPROBE_MULTI, &opts); if (!ASSERT_ERR(link_fd, "link_fd")) goto cleanup; - ASSERT_EQ(link_fd, -ESRCH, "pid_is_wrong"); + ASSERT_EQ(link_fd, -EINVAL, "pid_is_wrong"); cleanup: if (link_fd >= 0) @@ -495,6 +606,13 @@ static void test_link_api(void) return; __test_link_api(child); + + /* pid filter (thread) */ + child = spawn_thread(); + if (!ASSERT_OK_PTR(child, "spawn_thread")) + return; + + __test_link_api(child); } static void test_bench_attach_uprobe(void) diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c index c60db8beeb734238f4cdb197292dcf1e58578290..6816ff064516808eb1d20ccbc94d28033b4492c7 100644 --- a/tools/testing/selftests/bpf/prog_tests/verifier.c +++ b/tools/testing/selftests/bpf/prog_tests/verifier.c @@ -67,6 +67,7 @@ #include "verifier_search_pruning.skel.h" #include "verifier_sock.skel.h" #include "verifier_sock_addr.skel.h" +#include "verifier_sockmap_mutate.skel.h" #include "verifier_spill_fill.skel.h" #include "verifier_spin_lock.skel.h" #include "verifier_stack_ptr.skel.h" @@ -85,6 +86,7 @@ #include "verifier_xadd.skel.h" #include "verifier_xdp.skel.h" #include "verifier_xdp_direct_packet_access.skel.h" +#include "verifier_bits_iter.skel.h" #define MAX_ENTRIES 11 @@ -183,6 +185,7 @@ void test_verifier_sdiv(void) { RUN(verifier_sdiv); } void test_verifier_search_pruning(void) { RUN(verifier_search_pruning); } void test_verifier_sock(void) { RUN(verifier_sock); } void test_verifier_sock_addr(void) { RUN(verifier_sock_addr); } +void test_verifier_sockmap_mutate(void) { RUN(verifier_sockmap_mutate); } void test_verifier_spill_fill(void) { RUN(verifier_spill_fill); } void test_verifier_spin_lock(void) { RUN(verifier_spin_lock); } void test_verifier_stack_ptr(void) { RUN(verifier_stack_ptr); } @@ -200,6 +203,7 @@ void test_verifier_var_off(void) { RUN(verifier_var_off); } void test_verifier_xadd(void) { RUN(verifier_xadd); } void test_verifier_xdp(void) { RUN(verifier_xdp); } void test_verifier_xdp_direct_packet_access(void) { RUN(verifier_xdp_direct_packet_access); } +void test_verifier_bits_iter(void) { RUN(verifier_bits_iter); } static int init_test_val_map(struct bpf_object *obj, char *map_name) { diff --git a/tools/testing/selftests/bpf/progs/arena_htab.c b/tools/testing/selftests/bpf/progs/arena_htab.c index 1e6ac187a6a0ced2da5294ac6805a38f81047093..81eaa94afeb01c2447fc947aa4d970f49e16657e 100644 --- a/tools/testing/selftests/bpf/progs/arena_htab.c +++ b/tools/testing/selftests/bpf/progs/arena_htab.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#define BPF_NO_KFUNC_PROTOTYPES #include #include #include @@ -18,25 +19,35 @@ void __arena *htab_for_user; bool skip = false; int zero = 0; +char __arena arr1[100000]; +char arr2[1000]; SEC("syscall") int arena_htab_llvm(void *ctx) { #if defined(__BPF_FEATURE_ADDR_SPACE_CAST) || defined(BPF_ARENA_FORCE_ASM) struct htab __arena *htab; + char __arena *arr = arr1; __u64 i; htab = bpf_alloc(sizeof(*htab)); cast_kern(htab); htab_init(htab); + cast_kern(arr); + /* first run. No old elems in the table */ - for (i = zero; i < 1000; i++) + for (i = zero; i < 100000 && can_loop; i++) { htab_update_elem(htab, i, i); + arr[i] = i; + } - /* should replace all elems with new ones */ - for (i = zero; i < 1000; i++) + /* should replace some elems with new ones */ + for (i = zero; i < 1000 && can_loop; i++) { htab_update_elem(htab, i, i); + /* Access mem to make the verifier use bounded loop logic */ + arr2[i] = i; + } cast_user(htab); htab_for_user = htab; #else diff --git a/tools/testing/selftests/bpf/progs/arena_list.c b/tools/testing/selftests/bpf/progs/arena_list.c index 93bd0600eba04423bd85387e651b269265f2b17e..3a2ddcacbea6aee4b84253253fd1a85ee76e5b82 100644 --- a/tools/testing/selftests/bpf/progs/arena_list.c +++ b/tools/testing/selftests/bpf/progs/arena_list.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#define BPF_NO_KFUNC_PROTOTYPES #include #include #include diff --git a/tools/testing/selftests/bpf/progs/bpf_dctcp.c b/tools/testing/selftests/bpf/progs/bpf_dctcp.c index 3c9ffe340312399816c79f7650f65901da650e6d..02f552e7fd4d01b3d82d85ef18892806cccb9afe 100644 --- a/tools/testing/selftests/bpf/progs/bpf_dctcp.c +++ b/tools/testing/selftests/bpf/progs/bpf_dctcp.c @@ -65,7 +65,7 @@ static void dctcp_reset(const struct tcp_sock *tp, struct bpf_dctcp *ca) } SEC("struct_ops") -void BPF_PROG(dctcp_init, struct sock *sk) +void BPF_PROG(bpf_dctcp_init, struct sock *sk) { const struct tcp_sock *tp = tcp_sk(sk); struct bpf_dctcp *ca = inet_csk_ca(sk); @@ -77,7 +77,7 @@ void BPF_PROG(dctcp_init, struct sock *sk) (void *)fallback, sizeof(fallback)) == -EBUSY) ebusy_cnt++; - /* Switch back to myself and the recurred dctcp_init() + /* Switch back to myself and the recurred bpf_dctcp_init() * will get -EBUSY for all bpf_setsockopt(TCP_CONGESTION), * except the last "cdg" one. */ @@ -112,7 +112,7 @@ void BPF_PROG(dctcp_init, struct sock *sk) } SEC("struct_ops") -__u32 BPF_PROG(dctcp_ssthresh, struct sock *sk) +__u32 BPF_PROG(bpf_dctcp_ssthresh, struct sock *sk) { struct bpf_dctcp *ca = inet_csk_ca(sk); struct tcp_sock *tp = tcp_sk(sk); @@ -122,7 +122,7 @@ __u32 BPF_PROG(dctcp_ssthresh, struct sock *sk) } SEC("struct_ops") -void BPF_PROG(dctcp_update_alpha, struct sock *sk, __u32 flags) +void BPF_PROG(bpf_dctcp_update_alpha, struct sock *sk, __u32 flags) { const struct tcp_sock *tp = tcp_sk(sk); struct bpf_dctcp *ca = inet_csk_ca(sk); @@ -161,12 +161,12 @@ static void dctcp_react_to_loss(struct sock *sk) } SEC("struct_ops") -void BPF_PROG(dctcp_state, struct sock *sk, __u8 new_state) +void BPF_PROG(bpf_dctcp_state, struct sock *sk, __u8 new_state) { if (new_state == TCP_CA_Recovery && new_state != BPF_CORE_READ_BITFIELD(inet_csk(sk), icsk_ca_state)) dctcp_react_to_loss(sk); - /* We handle RTO in dctcp_cwnd_event to ensure that we perform only + /* We handle RTO in bpf_dctcp_cwnd_event to ensure that we perform only * one loss-adjustment per RTT. */ } @@ -208,7 +208,7 @@ static void dctcp_ece_ack_update(struct sock *sk, enum tcp_ca_event evt, } SEC("struct_ops") -void BPF_PROG(dctcp_cwnd_event, struct sock *sk, enum tcp_ca_event ev) +void BPF_PROG(bpf_dctcp_cwnd_event, struct sock *sk, enum tcp_ca_event ev) { struct bpf_dctcp *ca = inet_csk_ca(sk); @@ -227,7 +227,7 @@ void BPF_PROG(dctcp_cwnd_event, struct sock *sk, enum tcp_ca_event ev) } SEC("struct_ops") -__u32 BPF_PROG(dctcp_cwnd_undo, struct sock *sk) +__u32 BPF_PROG(bpf_dctcp_cwnd_undo, struct sock *sk) { const struct bpf_dctcp *ca = inet_csk_ca(sk); @@ -237,28 +237,28 @@ __u32 BPF_PROG(dctcp_cwnd_undo, struct sock *sk) extern void tcp_reno_cong_avoid(struct sock *sk, __u32 ack, __u32 acked) __ksym; SEC("struct_ops") -void BPF_PROG(dctcp_cong_avoid, struct sock *sk, __u32 ack, __u32 acked) +void BPF_PROG(bpf_dctcp_cong_avoid, struct sock *sk, __u32 ack, __u32 acked) { tcp_reno_cong_avoid(sk, ack, acked); } SEC(".struct_ops") struct tcp_congestion_ops dctcp_nouse = { - .init = (void *)dctcp_init, - .set_state = (void *)dctcp_state, + .init = (void *)bpf_dctcp_init, + .set_state = (void *)bpf_dctcp_state, .flags = TCP_CONG_NEEDS_ECN, .name = "bpf_dctcp_nouse", }; SEC(".struct_ops") struct tcp_congestion_ops dctcp = { - .init = (void *)dctcp_init, - .in_ack_event = (void *)dctcp_update_alpha, - .cwnd_event = (void *)dctcp_cwnd_event, - .ssthresh = (void *)dctcp_ssthresh, - .cong_avoid = (void *)dctcp_cong_avoid, - .undo_cwnd = (void *)dctcp_cwnd_undo, - .set_state = (void *)dctcp_state, + .init = (void *)bpf_dctcp_init, + .in_ack_event = (void *)bpf_dctcp_update_alpha, + .cwnd_event = (void *)bpf_dctcp_cwnd_event, + .ssthresh = (void *)bpf_dctcp_ssthresh, + .cong_avoid = (void *)bpf_dctcp_cong_avoid, + .undo_cwnd = (void *)bpf_dctcp_cwnd_undo, + .set_state = (void *)bpf_dctcp_state, .flags = TCP_CONG_NEEDS_ECN, .name = "bpf_dctcp", }; diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_bpf_array_map.c b/tools/testing/selftests/bpf/progs/bpf_iter_bpf_array_map.c index c5969ca6f26b5072ebd546dcfa120fd5d8dd9e2b..564835ba7d5166ebdd3306b5dc74a6f1d10e6310 100644 --- a/tools/testing/selftests/bpf/progs/bpf_iter_bpf_array_map.c +++ b/tools/testing/selftests/bpf/progs/bpf_iter_bpf_array_map.c @@ -6,12 +6,6 @@ char _license[] SEC("license") = "GPL"; -struct key_t { - int a; - int b; - int c; -}; - struct { __uint(type, BPF_MAP_TYPE_ARRAY); __uint(max_entries, 3); diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_bpf_percpu_array_map.c b/tools/testing/selftests/bpf/progs/bpf_iter_bpf_percpu_array_map.c index 85fa710fad908a6ad19db76b875a471c8d4817b3..9f0e0705b2bf6d23e62382e48a18685d44139b09 100644 --- a/tools/testing/selftests/bpf/progs/bpf_iter_bpf_percpu_array_map.c +++ b/tools/testing/selftests/bpf/progs/bpf_iter_bpf_percpu_array_map.c @@ -6,12 +6,6 @@ char _license[] SEC("license") = "GPL"; -struct key_t { - int a; - int b; - int c; -}; - struct { __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); __uint(max_entries, 3); diff --git a/tools/testing/selftests/bpf/progs/cpumask_success.c b/tools/testing/selftests/bpf/progs/cpumask_success.c index 7a1e64c6c065ce48df3059332378dc91165402a2..fd8106831c32c356a5a494b09f374a55221e1133 100644 --- a/tools/testing/selftests/bpf/progs/cpumask_success.c +++ b/tools/testing/selftests/bpf/progs/cpumask_success.c @@ -12,6 +12,31 @@ char _license[] SEC("license") = "GPL"; int pid, nr_cpus; +struct kptr_nested { + struct bpf_cpumask __kptr * mask; +}; + +struct kptr_nested_pair { + struct bpf_cpumask __kptr * mask_1; + struct bpf_cpumask __kptr * mask_2; +}; + +struct kptr_nested_mid { + int dummy; + struct kptr_nested m; +}; + +struct kptr_nested_deep { + struct kptr_nested_mid ptrs[2]; + struct kptr_nested_pair ptr_pairs[3]; +}; + +private(MASK) static struct bpf_cpumask __kptr * global_mask_array[2]; +private(MASK) static struct bpf_cpumask __kptr * global_mask_array_l2[2][1]; +private(MASK) static struct bpf_cpumask __kptr * global_mask_array_one[1]; +private(MASK) static struct kptr_nested global_mask_nested[2]; +private(MASK_DEEP) static struct kptr_nested_deep global_mask_nested_deep; + static bool is_test_task(void) { int cur_pid = bpf_get_current_pid_tgid() >> 32; @@ -460,6 +485,152 @@ int BPF_PROG(test_global_mask_rcu, struct task_struct *task, u64 clone_flags) return 0; } +SEC("tp_btf/task_newtask") +int BPF_PROG(test_global_mask_array_one_rcu, struct task_struct *task, u64 clone_flags) +{ + struct bpf_cpumask *local, *prev; + + if (!is_test_task()) + return 0; + + /* Kptr arrays with one element are special cased, being treated + * just like a single pointer. + */ + + local = create_cpumask(); + if (!local) + return 0; + + prev = bpf_kptr_xchg(&global_mask_array_one[0], local); + if (prev) { + bpf_cpumask_release(prev); + err = 3; + return 0; + } + + bpf_rcu_read_lock(); + local = global_mask_array_one[0]; + if (!local) { + err = 4; + bpf_rcu_read_unlock(); + return 0; + } + + bpf_rcu_read_unlock(); + + return 0; +} + +static int _global_mask_array_rcu(struct bpf_cpumask **mask0, + struct bpf_cpumask **mask1) +{ + struct bpf_cpumask *local; + + if (!is_test_task()) + return 0; + + /* Check if two kptrs in the array work and independently */ + + local = create_cpumask(); + if (!local) + return 0; + + bpf_rcu_read_lock(); + + local = bpf_kptr_xchg(mask0, local); + if (local) { + err = 1; + goto err_exit; + } + + /* [, NULL] */ + if (!*mask0 || *mask1) { + err = 2; + goto err_exit; + } + + local = create_cpumask(); + if (!local) { + err = 9; + goto err_exit; + } + + local = bpf_kptr_xchg(mask1, local); + if (local) { + err = 10; + goto err_exit; + } + + /* [, ] */ + if (!*mask0 || !*mask1 || *mask0 == *mask1) { + err = 11; + goto err_exit; + } + +err_exit: + if (local) + bpf_cpumask_release(local); + bpf_rcu_read_unlock(); + return 0; +} + +SEC("tp_btf/task_newtask") +int BPF_PROG(test_global_mask_array_rcu, struct task_struct *task, u64 clone_flags) +{ + return _global_mask_array_rcu(&global_mask_array[0], &global_mask_array[1]); +} + +SEC("tp_btf/task_newtask") +int BPF_PROG(test_global_mask_array_l2_rcu, struct task_struct *task, u64 clone_flags) +{ + return _global_mask_array_rcu(&global_mask_array_l2[0][0], &global_mask_array_l2[1][0]); +} + +SEC("tp_btf/task_newtask") +int BPF_PROG(test_global_mask_nested_rcu, struct task_struct *task, u64 clone_flags) +{ + return _global_mask_array_rcu(&global_mask_nested[0].mask, &global_mask_nested[1].mask); +} + +/* Ensure that the field->offset has been correctly advanced from one + * nested struct or array sub-tree to another. In the case of + * kptr_nested_deep, it comprises two sub-trees: ktpr_1 and kptr_2. By + * calling bpf_kptr_xchg() on every single kptr in both nested sub-trees, + * the verifier should reject the program if the field->offset of any kptr + * is incorrect. + * + * For instance, if we have 10 kptrs in a nested struct and a program that + * accesses each kptr individually with bpf_kptr_xchg(), the compiler + * should emit instructions to access 10 different offsets if it works + * correctly. If the field->offset values of any pair of them are + * incorrectly the same, the number of unique offsets in btf_record for + * this nested struct should be less than 10. The verifier should fail to + * discover some of the offsets emitted by the compiler. + * + * Even if the field->offset values of kptrs are not duplicated, the + * verifier should fail to find a btf_field for the instruction accessing a + * kptr if the corresponding field->offset is pointing to a random + * incorrect offset. + */ +SEC("tp_btf/task_newtask") +int BPF_PROG(test_global_mask_nested_deep_rcu, struct task_struct *task, u64 clone_flags) +{ + int r, i; + + r = _global_mask_array_rcu(&global_mask_nested_deep.ptrs[0].m.mask, + &global_mask_nested_deep.ptrs[1].m.mask); + if (r) + return r; + + for (i = 0; i < 3; i++) { + r = _global_mask_array_rcu(&global_mask_nested_deep.ptr_pairs[i].mask_1, + &global_mask_nested_deep.ptr_pairs[i].mask_2); + if (r) + return r; + } + return 0; +} + SEC("tp_btf/task_newtask") int BPF_PROG(test_cpumask_weight, struct task_struct *task, u64 clone_flags) { diff --git a/tools/testing/selftests/bpf/progs/crypto_bench.c b/tools/testing/selftests/bpf/progs/crypto_bench.c index e61fe08822931a7d01dade17e58e6197b960204a..4ac956b26240dc2cd39fbacaac63da9f7f13b72e 100644 --- a/tools/testing/selftests/bpf/progs/crypto_bench.c +++ b/tools/testing/selftests/bpf/progs/crypto_bench.c @@ -57,7 +57,7 @@ int crypto_encrypt(struct __sk_buff *skb) { struct __crypto_ctx_value *v; struct bpf_crypto_ctx *ctx; - struct bpf_dynptr psrc, pdst, iv; + struct bpf_dynptr psrc, pdst; v = crypto_ctx_value_lookup(); if (!v) { @@ -73,9 +73,8 @@ int crypto_encrypt(struct __sk_buff *skb) bpf_dynptr_from_skb(skb, 0, &psrc); bpf_dynptr_from_mem(dst, len, 0, &pdst); - bpf_dynptr_from_mem(dst, 0, 0, &iv); - status = bpf_crypto_encrypt(ctx, &psrc, &pdst, &iv); + status = bpf_crypto_encrypt(ctx, &psrc, &pdst, NULL); __sync_add_and_fetch(&hits, 1); return 0; @@ -84,7 +83,7 @@ int crypto_encrypt(struct __sk_buff *skb) SEC("tc") int crypto_decrypt(struct __sk_buff *skb) { - struct bpf_dynptr psrc, pdst, iv; + struct bpf_dynptr psrc, pdst; struct __crypto_ctx_value *v; struct bpf_crypto_ctx *ctx; @@ -98,9 +97,8 @@ int crypto_decrypt(struct __sk_buff *skb) bpf_dynptr_from_skb(skb, 0, &psrc); bpf_dynptr_from_mem(dst, len, 0, &pdst); - bpf_dynptr_from_mem(dst, 0, 0, &iv); - status = bpf_crypto_decrypt(ctx, &psrc, &pdst, &iv); + status = bpf_crypto_decrypt(ctx, &psrc, &pdst, NULL); __sync_add_and_fetch(&hits, 1); return 0; diff --git a/tools/testing/selftests/bpf/progs/crypto_sanity.c b/tools/testing/selftests/bpf/progs/crypto_sanity.c index 1be0a3fa5efd9ef095e2e4b16b3a8a4cae4f56e7..645be6cddf36cdf8c5b8796a45c9e8fd96e901e2 100644 --- a/tools/testing/selftests/bpf/progs/crypto_sanity.c +++ b/tools/testing/selftests/bpf/progs/crypto_sanity.c @@ -89,7 +89,7 @@ int decrypt_sanity(struct __sk_buff *skb) { struct __crypto_ctx_value *v; struct bpf_crypto_ctx *ctx; - struct bpf_dynptr psrc, pdst, iv; + struct bpf_dynptr psrc, pdst; int err; err = skb_dynptr_validate(skb, &psrc); @@ -114,12 +114,8 @@ int decrypt_sanity(struct __sk_buff *skb) * production code, a percpu map should be used to store the result. */ bpf_dynptr_from_mem(dst, sizeof(dst), 0, &pdst); - /* iv dynptr has to be initialized with 0 size, but proper memory region - * has to be provided anyway - */ - bpf_dynptr_from_mem(dst, 0, 0, &iv); - status = bpf_crypto_decrypt(ctx, &psrc, &pdst, &iv); + status = bpf_crypto_decrypt(ctx, &psrc, &pdst, NULL); return TC_ACT_SHOT; } @@ -129,7 +125,7 @@ int encrypt_sanity(struct __sk_buff *skb) { struct __crypto_ctx_value *v; struct bpf_crypto_ctx *ctx; - struct bpf_dynptr psrc, pdst, iv; + struct bpf_dynptr psrc, pdst; int err; status = 0; @@ -156,12 +152,8 @@ int encrypt_sanity(struct __sk_buff *skb) * production code, a percpu map should be used to store the result. */ bpf_dynptr_from_mem(dst, sizeof(dst), 0, &pdst); - /* iv dynptr has to be initialized with 0 size, but proper memory region - * has to be provided anyway - */ - bpf_dynptr_from_mem(dst, 0, 0, &iv); - status = bpf_crypto_encrypt(ctx, &psrc, &pdst, &iv); + status = bpf_crypto_encrypt(ctx, &psrc, &pdst, NULL); return TC_ACT_SHOT; } diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_test.c index 8956eb78a226040d604c471aa550238d2138be6a..2011cacdeb183c2cdde678970779a44502f40a67 100644 --- a/tools/testing/selftests/bpf/progs/get_func_ip_test.c +++ b/tools/testing/selftests/bpf/progs/get_func_ip_test.c @@ -5,13 +5,12 @@ char _license[] SEC("license") = "GPL"; -extern const void bpf_fentry_test1 __ksym; +extern int bpf_fentry_test1(int a) __ksym; +extern int bpf_modify_return_test(int a, int *b) __ksym; + extern const void bpf_fentry_test2 __ksym; extern const void bpf_fentry_test3 __ksym; extern const void bpf_fentry_test4 __ksym; -extern const void bpf_modify_return_test __ksym; -extern const void bpf_fentry_test6 __ksym; -extern const void bpf_fentry_test7 __ksym; extern bool CONFIG_X86_KERNEL_IBT __kconfig __weak; diff --git a/tools/testing/selftests/bpf/progs/ip_check_defrag.c b/tools/testing/selftests/bpf/progs/ip_check_defrag.c index 1c2b6c1616b05d145980885385a8c6cdb2dbf2d4..645b2c9f7867c3a84316e3bbe7884ab16003bf5d 100644 --- a/tools/testing/selftests/bpf/progs/ip_check_defrag.c +++ b/tools/testing/selftests/bpf/progs/ip_check_defrag.c @@ -12,7 +12,7 @@ #define IP_OFFSET 0x1FFF #define NEXTHDR_FRAGMENT 44 -extern int bpf_dynptr_from_skb(struct sk_buff *skb, __u64 flags, +extern int bpf_dynptr_from_skb(struct __sk_buff *skb, __u64 flags, struct bpf_dynptr *ptr__uninit) __ksym; extern void *bpf_dynptr_slice(const struct bpf_dynptr *ptr, uint32_t offset, void *buffer, uint32_t buffer__sz) __ksym; @@ -42,7 +42,7 @@ static bool is_frag_v6(struct ipv6hdr *ip6h) return ip6h->nexthdr == NEXTHDR_FRAGMENT; } -static int handle_v4(struct sk_buff *skb) +static int handle_v4(struct __sk_buff *skb) { struct bpf_dynptr ptr; u8 iph_buf[20] = {}; @@ -64,7 +64,7 @@ static int handle_v4(struct sk_buff *skb) return NF_ACCEPT; } -static int handle_v6(struct sk_buff *skb) +static int handle_v6(struct __sk_buff *skb) { struct bpf_dynptr ptr; struct ipv6hdr *ip6h; @@ -89,9 +89,9 @@ static int handle_v6(struct sk_buff *skb) SEC("netfilter") int defrag(struct bpf_nf_ctx *ctx) { - struct sk_buff *skb = ctx->skb; + struct __sk_buff *skb = (struct __sk_buff *)ctx->skb; - switch (bpf_ntohs(skb->protocol)) { + switch (bpf_ntohs(ctx->skb->protocol)) { case ETH_P_IP: return handle_v4(skb); case ETH_P_IPV6: diff --git a/tools/testing/selftests/bpf/progs/linked_list.c b/tools/testing/selftests/bpf/progs/linked_list.c index 26205ca8067968d3b58443257cedc47e2300a6dd..f69bf3e30321a0fde68506b6f9729b258a373d3a 100644 --- a/tools/testing/selftests/bpf/progs/linked_list.c +++ b/tools/testing/selftests/bpf/progs/linked_list.c @@ -11,6 +11,22 @@ #include "linked_list.h" +struct head_nested_inner { + struct bpf_spin_lock lock; + struct bpf_list_head head __contains(foo, node2); +}; + +struct head_nested { + int dummy; + struct head_nested_inner inner; +}; + +private(C) struct bpf_spin_lock glock_c; +private(C) struct bpf_list_head ghead_array[2] __contains(foo, node2); +private(C) struct bpf_list_head ghead_array_one[1] __contains(foo, node2); + +private(D) struct head_nested ghead_nested; + static __always_inline int list_push_pop(struct bpf_spin_lock *lock, struct bpf_list_head *head, bool leave_in_map) { @@ -309,6 +325,32 @@ int global_list_push_pop(void *ctx) return test_list_push_pop(&glock, &ghead); } +SEC("tc") +int global_list_push_pop_nested(void *ctx) +{ + return test_list_push_pop(&ghead_nested.inner.lock, &ghead_nested.inner.head); +} + +SEC("tc") +int global_list_array_push_pop(void *ctx) +{ + int r; + + r = test_list_push_pop(&glock_c, &ghead_array[0]); + if (r) + return r; + + r = test_list_push_pop(&glock_c, &ghead_array[1]); + if (r) + return r; + + /* Arrays with only one element is a special case, being treated + * just like a bpf_list_head variable by the verifier, not an + * array. + */ + return test_list_push_pop(&glock_c, &ghead_array_one[0]); +} + SEC("tc") int map_list_push_pop_multiple(void *ctx) { diff --git a/tools/testing/selftests/bpf/progs/map_percpu_stats.c b/tools/testing/selftests/bpf/progs/map_percpu_stats.c index 10b2325c17207b960e230708731df43006c3d07e..63245785eb69e0b2967ce68941e6e8f4f016de4e 100644 --- a/tools/testing/selftests/bpf/progs/map_percpu_stats.c +++ b/tools/testing/selftests/bpf/progs/map_percpu_stats.c @@ -7,7 +7,7 @@ __u32 target_id; -__s64 bpf_map_sum_elem_count(struct bpf_map *map) __ksym; +__s64 bpf_map_sum_elem_count(const struct bpf_map *map) __ksym; SEC("iter/bpf_map") int dump_bpf_map(struct bpf_iter__bpf_map *ctx) diff --git a/tools/testing/selftests/bpf/progs/nested_trust_common.h b/tools/testing/selftests/bpf/progs/nested_trust_common.h index 83d33931136ea62122504e14f554143983c01cd2..1784b496be2ec9d3ee757582c4aefcf1f5555470 100644 --- a/tools/testing/selftests/bpf/progs/nested_trust_common.h +++ b/tools/testing/selftests/bpf/progs/nested_trust_common.h @@ -7,6 +7,6 @@ #include bool bpf_cpumask_test_cpu(unsigned int cpu, const struct cpumask *cpumask) __ksym; -bool bpf_cpumask_first_zero(const struct cpumask *cpumask) __ksym; +__u32 bpf_cpumask_first_zero(const struct cpumask *cpumask) __ksym; #endif /* _NESTED_TRUST_COMMON_H */ diff --git a/tools/testing/selftests/bpf/progs/rbtree.c b/tools/testing/selftests/bpf/progs/rbtree.c index b09f4fffe57cfde7b42831462371643055f35b9f..a3620c15c13675ec378f7920c525b0b1f7a2afb4 100644 --- a/tools/testing/selftests/bpf/progs/rbtree.c +++ b/tools/testing/selftests/bpf/progs/rbtree.c @@ -13,6 +13,15 @@ struct node_data { struct bpf_rb_node node; }; +struct root_nested_inner { + struct bpf_spin_lock glock; + struct bpf_rb_root root __contains(node_data, node); +}; + +struct root_nested { + struct root_nested_inner inner; +}; + long less_callback_ran = -1; long removed_key = -1; long first_data[2] = {-1, -1}; @@ -20,6 +29,9 @@ long first_data[2] = {-1, -1}; #define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8))) private(A) struct bpf_spin_lock glock; private(A) struct bpf_rb_root groot __contains(node_data, node); +private(A) struct bpf_rb_root groot_array[2] __contains(node_data, node); +private(A) struct bpf_rb_root groot_array_one[1] __contains(node_data, node); +private(B) struct root_nested groot_nested; static bool less(struct bpf_rb_node *a, const struct bpf_rb_node *b) { @@ -71,6 +83,12 @@ long rbtree_add_nodes(void *ctx) return __add_three(&groot, &glock); } +SEC("tc") +long rbtree_add_nodes_nested(void *ctx) +{ + return __add_three(&groot_nested.inner.root, &groot_nested.inner.glock); +} + SEC("tc") long rbtree_add_and_remove(void *ctx) { @@ -109,6 +127,65 @@ err_out: return 1; } +SEC("tc") +long rbtree_add_and_remove_array(void *ctx) +{ + struct bpf_rb_node *res1 = NULL, *res2 = NULL, *res3 = NULL; + struct node_data *nodes[3][2] = {{NULL, NULL}, {NULL, NULL}, {NULL, NULL}}; + struct node_data *n; + long k1 = -1, k2 = -1, k3 = -1; + int i, j; + + for (i = 0; i < 3; i++) { + for (j = 0; j < 2; j++) { + nodes[i][j] = bpf_obj_new(typeof(*nodes[i][j])); + if (!nodes[i][j]) + goto err_out; + nodes[i][j]->key = i * 2 + j; + } + } + + bpf_spin_lock(&glock); + for (i = 0; i < 2; i++) + for (j = 0; j < 2; j++) + bpf_rbtree_add(&groot_array[i], &nodes[i][j]->node, less); + for (j = 0; j < 2; j++) + bpf_rbtree_add(&groot_array_one[0], &nodes[2][j]->node, less); + res1 = bpf_rbtree_remove(&groot_array[0], &nodes[0][0]->node); + res2 = bpf_rbtree_remove(&groot_array[1], &nodes[1][0]->node); + res3 = bpf_rbtree_remove(&groot_array_one[0], &nodes[2][0]->node); + bpf_spin_unlock(&glock); + + if (res1) { + n = container_of(res1, struct node_data, node); + k1 = n->key; + bpf_obj_drop(n); + } + if (res2) { + n = container_of(res2, struct node_data, node); + k2 = n->key; + bpf_obj_drop(n); + } + if (res3) { + n = container_of(res3, struct node_data, node); + k3 = n->key; + bpf_obj_drop(n); + } + if (k1 != 0 || k2 != 2 || k3 != 4) + return 2; + + return 0; + +err_out: + for (i = 0; i < 3; i++) { + for (j = 0; j < 2; j++) { + if (nodes[i][j]) + bpf_obj_drop(nodes[i][j]); + } + } + return 1; +} + SEC("tc") long rbtree_first_and_remove(void *ctx) { diff --git a/tools/testing/selftests/bpf/progs/skb_pkt_end.c b/tools/testing/selftests/bpf/progs/skb_pkt_end.c index db4abd2682fcfd107fc2af1c1b4bbf6d377feb89..3bb4451524a1ad931dc915eaa8dd94529b5c2a6d 100644 --- a/tools/testing/selftests/bpf/progs/skb_pkt_end.c +++ b/tools/testing/selftests/bpf/progs/skb_pkt_end.c @@ -33,6 +33,8 @@ int main_prog(struct __sk_buff *skb) struct iphdr *ip = NULL; struct tcphdr *tcp; __u8 proto = 0; + int urg_ptr; + u32 offset; if (!(ip = get_iphdr(skb))) goto out; @@ -48,7 +50,14 @@ int main_prog(struct __sk_buff *skb) if (!tcp) goto out; - return tcp->urg_ptr; + urg_ptr = tcp->urg_ptr; + + /* Checksum validation part */ + proto++; + offset = sizeof(struct ethhdr) + offsetof(struct iphdr, protocol); + bpf_skb_store_bytes(skb, offset, &proto, sizeof(proto), BPF_F_RECOMPUTE_CSUM); + + return urg_ptr; out: return -1; } diff --git a/tools/testing/selftests/bpf/progs/struct_ops_detach.c b/tools/testing/selftests/bpf/progs/struct_ops_detach.c new file mode 100644 index 0000000000000000000000000000000000000000..56b787a8987610ffe776238801937890244f6184 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_detach.c @@ -0,0 +1,10 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include "../bpf_testmod/bpf_testmod.h" + +char _license[] SEC("license") = "GPL"; + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_do_detach; diff --git a/tools/testing/selftests/bpf/progs/test_bpf_nf.c b/tools/testing/selftests/bpf/progs/test_bpf_nf.c index 77ad8adf68daaae1e648d12e976683069d04da2f..f7b330ddd007e8b14fe5cca353e13ce5c6daf663 100644 --- a/tools/testing/selftests/bpf/progs/test_bpf_nf.c +++ b/tools/testing/selftests/bpf/progs/test_bpf_nf.c @@ -1,4 +1,5 @@ // SPDX-License-Identifier: GPL-2.0 +#define BPF_NO_KFUNC_PROTOTYPES #include #include #include @@ -9,10 +10,14 @@ #define EINVAL 22 #define ENOENT 2 +#define NF_CT_ZONE_DIR_ORIG (1 << IP_CT_DIR_ORIGINAL) +#define NF_CT_ZONE_DIR_REPL (1 << IP_CT_DIR_REPLY) + extern unsigned long CONFIG_HZ __kconfig; int test_einval_bpf_tuple = 0; int test_einval_reserved = 0; +int test_einval_reserved_new = 0; int test_einval_netns_id = 0; int test_einval_len_opts = 0; int test_eproto_l4proto = 0; @@ -22,6 +27,11 @@ int test_eafnosupport = 0; int test_alloc_entry = -EINVAL; int test_insert_entry = -EAFNOSUPPORT; int test_succ_lookup = -ENOENT; +int test_ct_zone_id_alloc_entry = -EINVAL; +int test_ct_zone_id_insert_entry = -EAFNOSUPPORT; +int test_ct_zone_id_succ_lookup = -ENOENT; +int test_ct_zone_dir_enoent_lookup = 0; +int test_ct_zone_id_enoent_lookup = 0; u32 test_delta_timeout = 0; u32 test_status = 0; u32 test_insert_lookup_mark = 0; @@ -45,6 +55,17 @@ struct bpf_ct_opts___local { s32 netns_id; s32 error; u8 l4proto; + u8 dir; + u8 reserved[2]; +}; + +struct bpf_ct_opts___new { + s32 netns_id; + s32 error; + u8 l4proto; + u8 dir; + u16 ct_zone_id; + u8 ct_zone_dir; u8 reserved[3]; } __attribute__((preserve_access_index)); @@ -220,10 +241,97 @@ nf_ct_test(struct nf_conn *(*lookup_fn)(void *, struct bpf_sock_tuple *, u32, } } +static __always_inline void +nf_ct_opts_new_test(struct nf_conn *(*lookup_fn)(void *, struct bpf_sock_tuple *, u32, + struct bpf_ct_opts___new *, u32), + struct nf_conn *(*alloc_fn)(void *, struct bpf_sock_tuple *, u32, + struct bpf_ct_opts___new *, u32), + void *ctx) +{ + struct bpf_ct_opts___new opts_def = { .l4proto = IPPROTO_TCP, .netns_id = -1 }; + struct bpf_sock_tuple bpf_tuple; + struct nf_conn *ct; + + __builtin_memset(&bpf_tuple, 0, sizeof(bpf_tuple.ipv4)); + + opts_def.reserved[0] = 1; + ct = lookup_fn(ctx, &bpf_tuple, sizeof(bpf_tuple.ipv4), &opts_def, + sizeof(opts_def)); + opts_def.reserved[0] = 0; + if (ct) + bpf_ct_release(ct); + else + test_einval_reserved_new = opts_def.error; + + bpf_tuple.ipv4.saddr = bpf_get_prandom_u32(); /* src IP */ + bpf_tuple.ipv4.daddr = bpf_get_prandom_u32(); /* dst IP */ + bpf_tuple.ipv4.sport = bpf_get_prandom_u32(); /* src port */ + bpf_tuple.ipv4.dport = bpf_get_prandom_u32(); /* dst port */ + + /* use non-default ct zone */ + opts_def.ct_zone_id = 10; + opts_def.ct_zone_dir = NF_CT_ZONE_DIR_ORIG; + ct = alloc_fn(ctx, &bpf_tuple, sizeof(bpf_tuple.ipv4), &opts_def, + sizeof(opts_def)); + if (ct) { + __u16 sport = bpf_get_prandom_u32(); + __u16 dport = bpf_get_prandom_u32(); + union nf_inet_addr saddr = {}; + union nf_inet_addr daddr = {}; + struct nf_conn *ct_ins; + + bpf_ct_set_timeout(ct, 10000); + + /* snat */ + saddr.ip = bpf_get_prandom_u32(); + bpf_ct_set_nat_info(ct, &saddr, sport, NF_NAT_MANIP_SRC___local); + /* dnat */ + daddr.ip = bpf_get_prandom_u32(); + bpf_ct_set_nat_info(ct, &daddr, dport, NF_NAT_MANIP_DST___local); + + ct_ins = bpf_ct_insert_entry(ct); + if (ct_ins) { + struct nf_conn *ct_lk; + + /* entry should exist in same ct zone we inserted it */ + ct_lk = lookup_fn(ctx, &bpf_tuple, sizeof(bpf_tuple.ipv4), + &opts_def, sizeof(opts_def)); + if (ct_lk) { + bpf_ct_release(ct_lk); + test_ct_zone_id_succ_lookup = 0; + } + + /* entry should not exist with wrong direction */ + opts_def.ct_zone_dir = NF_CT_ZONE_DIR_REPL; + ct_lk = lookup_fn(ctx, &bpf_tuple, sizeof(bpf_tuple.ipv4), + &opts_def, sizeof(opts_def)); + opts_def.ct_zone_dir = NF_CT_ZONE_DIR_ORIG; + if (ct_lk) + bpf_ct_release(ct_lk); + else + test_ct_zone_dir_enoent_lookup = opts_def.error; + + /* entry should not exist in default ct zone */ + opts_def.ct_zone_id = 0; + ct_lk = lookup_fn(ctx, &bpf_tuple, sizeof(bpf_tuple.ipv4), + &opts_def, sizeof(opts_def)); + if (ct_lk) + bpf_ct_release(ct_lk); + else + test_ct_zone_id_enoent_lookup = opts_def.error; + + bpf_ct_release(ct_ins); + test_ct_zone_id_insert_entry = 0; + } + test_ct_zone_id_alloc_entry = 0; + } +} + SEC("xdp") int nf_xdp_ct_test(struct xdp_md *ctx) { nf_ct_test((void *)bpf_xdp_ct_lookup, (void *)bpf_xdp_ct_alloc, ctx); + nf_ct_opts_new_test((void *)bpf_xdp_ct_lookup, (void *)bpf_xdp_ct_alloc, ctx); return 0; } @@ -231,6 +339,7 @@ SEC("tc") int nf_skb_ct_test(struct __sk_buff *ctx) { nf_ct_test((void *)bpf_skb_ct_lookup, (void *)bpf_skb_ct_alloc, ctx); + nf_ct_opts_new_test((void *)bpf_skb_ct_lookup, (void *)bpf_skb_ct_alloc, ctx); return 0; } diff --git a/tools/testing/selftests/bpf/progs/test_bpf_nf_fail.c b/tools/testing/selftests/bpf/progs/test_bpf_nf_fail.c index 0e4759ab38ff1ac92eb51c505309634b48051c67..a586f087ffeb476b9f421b3843cb264bddad8fa3 100644 --- a/tools/testing/selftests/bpf/progs/test_bpf_nf_fail.c +++ b/tools/testing/selftests/bpf/progs/test_bpf_nf_fail.c @@ -1,4 +1,5 @@ // SPDX-License-Identifier: GPL-2.0 +#define BPF_NO_KFUNC_PROTOTYPES #include #include #include diff --git a/tools/testing/selftests/bpf/progs/test_kfunc_param_nullable.c b/tools/testing/selftests/bpf/progs/test_kfunc_param_nullable.c new file mode 100644 index 0000000000000000000000000000000000000000..7c75e9b8f4554ab12445bb15577ec11e100cb8ce --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_kfunc_param_nullable.c @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc */ +#include +#include +#include "bpf_misc.h" +#include "bpf_kfuncs.h" +#include "../bpf_testmod/bpf_testmod_kfunc.h" + +SEC("tc") +int kfunc_dynptr_nullable_test1(struct __sk_buff *skb) +{ + struct bpf_dynptr data; + + bpf_dynptr_from_skb(skb, 0, &data); + bpf_kfunc_dynptr_test(&data, NULL); + + return 0; +} + +SEC("tc") +int kfunc_dynptr_nullable_test2(struct __sk_buff *skb) +{ + struct bpf_dynptr data; + + bpf_dynptr_from_skb(skb, 0, &data); + bpf_kfunc_dynptr_test(&data, &data); + + return 0; +} + +SEC("tc") +__failure __msg("expected pointer to stack or dynptr_ptr") +int kfunc_dynptr_nullable_test3(struct __sk_buff *skb) +{ + struct bpf_dynptr data; + + bpf_dynptr_from_skb(skb, 0, &data); + bpf_kfunc_dynptr_test(NULL, &data); + + return 0; +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/test_sk_storage_tracing.c b/tools/testing/selftests/bpf/progs/test_sk_storage_tracing.c index 02e718f06e0f5c5b81c14e9d75c635b7c0837a31..40531e56776e426c247af0472a1d1ed6e9c0e49e 100644 --- a/tools/testing/selftests/bpf/progs/test_sk_storage_tracing.c +++ b/tools/testing/selftests/bpf/progs/test_sk_storage_tracing.c @@ -84,7 +84,7 @@ int BPF_PROG(trace_tcp_connect, struct sock *sk) } SEC("fexit/inet_csk_accept") -int BPF_PROG(inet_csk_accept, struct sock *sk, int flags, int *err, bool kern, +int BPF_PROG(inet_csk_accept, struct sock *sk, struct proto_accept_arg *arg, struct sock *accepted_sk) { set_task_info(accepted_sk); diff --git a/tools/testing/selftests/bpf/progs/test_sockmap_kern.h b/tools/testing/selftests/bpf/progs/test_sockmap_kern.h index 99d2ea9fb658f35701f419797f48c722ced85b27..f48f85f1bd7028e8b7a3a140268eec7d4f127d9c 100644 --- a/tools/testing/selftests/bpf/progs/test_sockmap_kern.h +++ b/tools/testing/selftests/bpf/progs/test_sockmap_kern.h @@ -92,7 +92,7 @@ struct { __uint(value_size, sizeof(int)); } tls_sock_map SEC(".maps"); -SEC("sk_skb1") +SEC("sk_skb/stream_parser") int bpf_prog1(struct __sk_buff *skb) { int *f, two = 2; @@ -104,7 +104,7 @@ int bpf_prog1(struct __sk_buff *skb) return skb->len; } -SEC("sk_skb2") +SEC("sk_skb/stream_verdict") int bpf_prog2(struct __sk_buff *skb) { __u32 lport = skb->local_port; @@ -151,7 +151,7 @@ static inline void bpf_write_pass(struct __sk_buff *skb, int offset) memcpy(c + offset, "PASS", 4); } -SEC("sk_skb3") +SEC("sk_skb/stream_verdict") int bpf_prog3(struct __sk_buff *skb) { int err, *f, ret = SK_PASS; @@ -177,9 +177,6 @@ int bpf_prog3(struct __sk_buff *skb) return bpf_sk_redirect_hash(skb, &tls_sock_map, &ret, flags); #endif } - f = bpf_map_lookup_elem(&sock_skb_opts, &one); - if (f && *f) - ret = SK_DROP; err = bpf_skb_adjust_room(skb, 4, 0, 0); if (err) return SK_DROP; @@ -233,7 +230,7 @@ int bpf_sockmap(struct bpf_sock_ops *skops) return 0; } -SEC("sk_msg1") +SEC("sk_msg") int bpf_prog4(struct sk_msg_md *msg) { int *bytes, zero = 0, one = 1, two = 2, three = 3, four = 4, five = 5; @@ -263,7 +260,7 @@ int bpf_prog4(struct sk_msg_md *msg) return SK_PASS; } -SEC("sk_msg2") +SEC("sk_msg") int bpf_prog6(struct sk_msg_md *msg) { int zero = 0, one = 1, two = 2, three = 3, four = 4, five = 5, key = 0; @@ -308,7 +305,7 @@ int bpf_prog6(struct sk_msg_md *msg) #endif } -SEC("sk_msg3") +SEC("sk_msg") int bpf_prog8(struct sk_msg_md *msg) { void *data_end = (void *)(long) msg->data_end; @@ -329,7 +326,8 @@ int bpf_prog8(struct sk_msg_md *msg) return SK_PASS; } -SEC("sk_msg4") + +SEC("sk_msg") int bpf_prog9(struct sk_msg_md *msg) { void *data_end = (void *)(long) msg->data_end; @@ -347,7 +345,7 @@ int bpf_prog9(struct sk_msg_md *msg) return SK_PASS; } -SEC("sk_msg5") +SEC("sk_msg") int bpf_prog10(struct sk_msg_md *msg) { int *bytes, *start, *end, *start_push, *end_push, *start_pop, *pop; diff --git a/tools/testing/selftests/bpf/progs/test_tc_dtime.c b/tools/testing/selftests/bpf/progs/test_tc_dtime.c index 74ec09f040b7a265c9573b96890bfeae6a92b7f8..ca8e8734d901a7633810a23865cec04f94b9cb54 100644 --- a/tools/testing/selftests/bpf/progs/test_tc_dtime.c +++ b/tools/testing/selftests/bpf/progs/test_tc_dtime.c @@ -222,17 +222,21 @@ int egress_host(struct __sk_buff *skb) return TC_ACT_OK; if (skb_proto(skb_type) == IPPROTO_TCP) { - if (skb->tstamp_type == BPF_SKB_TSTAMP_DELIVERY_MONO && + if (skb->tstamp_type == BPF_SKB_CLOCK_MONOTONIC && skb->tstamp) inc_dtimes(EGRESS_ENDHOST); else inc_errs(EGRESS_ENDHOST); - } else { - if (skb->tstamp_type == BPF_SKB_TSTAMP_UNSPEC && + } else if (skb_proto(skb_type) == IPPROTO_UDP) { + if (skb->tstamp_type == BPF_SKB_CLOCK_TAI && skb->tstamp) inc_dtimes(EGRESS_ENDHOST); else inc_errs(EGRESS_ENDHOST); + } else { + if (skb->tstamp_type == BPF_SKB_CLOCK_REALTIME && + skb->tstamp) + inc_errs(EGRESS_ENDHOST); } skb->tstamp = EGRESS_ENDHOST_MAGIC; @@ -252,7 +256,7 @@ int ingress_host(struct __sk_buff *skb) if (!skb_type) return TC_ACT_OK; - if (skb->tstamp_type == BPF_SKB_TSTAMP_DELIVERY_MONO && + if (skb->tstamp_type == BPF_SKB_CLOCK_MONOTONIC && skb->tstamp == EGRESS_FWDNS_MAGIC) inc_dtimes(INGRESS_ENDHOST); else @@ -315,7 +319,6 @@ int egress_fwdns_prio100(struct __sk_buff *skb) SEC("tc") int ingress_fwdns_prio101(struct __sk_buff *skb) { - __u64 expected_dtime = EGRESS_ENDHOST_MAGIC; int skb_type; skb_type = skb_get_type(skb); @@ -323,29 +326,24 @@ int ingress_fwdns_prio101(struct __sk_buff *skb) /* Should have handled in prio100 */ return TC_ACT_SHOT; - if (skb_proto(skb_type) == IPPROTO_UDP) - expected_dtime = 0; - if (skb->tstamp_type) { if (fwdns_clear_dtime() || - skb->tstamp_type != BPF_SKB_TSTAMP_DELIVERY_MONO || - skb->tstamp != expected_dtime) + (skb->tstamp_type != BPF_SKB_CLOCK_MONOTONIC && + skb->tstamp_type != BPF_SKB_CLOCK_TAI) || + skb->tstamp != EGRESS_ENDHOST_MAGIC) inc_errs(INGRESS_FWDNS_P101); else inc_dtimes(INGRESS_FWDNS_P101); } else { - if (!fwdns_clear_dtime() && expected_dtime) + if (!fwdns_clear_dtime()) inc_errs(INGRESS_FWDNS_P101); } - if (skb->tstamp_type == BPF_SKB_TSTAMP_DELIVERY_MONO) { + if (skb->tstamp_type == BPF_SKB_CLOCK_MONOTONIC) { skb->tstamp = INGRESS_FWDNS_MAGIC; } else { if (bpf_skb_set_tstamp(skb, INGRESS_FWDNS_MAGIC, - BPF_SKB_TSTAMP_DELIVERY_MONO)) - inc_errs(SET_DTIME); - if (!bpf_skb_set_tstamp(skb, INGRESS_FWDNS_MAGIC, - BPF_SKB_TSTAMP_UNSPEC)) + BPF_SKB_CLOCK_MONOTONIC)) inc_errs(SET_DTIME); } @@ -370,7 +368,7 @@ int egress_fwdns_prio101(struct __sk_buff *skb) if (skb->tstamp_type) { if (fwdns_clear_dtime() || - skb->tstamp_type != BPF_SKB_TSTAMP_DELIVERY_MONO || + skb->tstamp_type != BPF_SKB_CLOCK_MONOTONIC || skb->tstamp != INGRESS_FWDNS_MAGIC) inc_errs(EGRESS_FWDNS_P101); else @@ -380,14 +378,11 @@ int egress_fwdns_prio101(struct __sk_buff *skb) inc_errs(EGRESS_FWDNS_P101); } - if (skb->tstamp_type == BPF_SKB_TSTAMP_DELIVERY_MONO) { + if (skb->tstamp_type == BPF_SKB_CLOCK_MONOTONIC) { skb->tstamp = EGRESS_FWDNS_MAGIC; } else { if (bpf_skb_set_tstamp(skb, EGRESS_FWDNS_MAGIC, - BPF_SKB_TSTAMP_DELIVERY_MONO)) - inc_errs(SET_DTIME); - if (!bpf_skb_set_tstamp(skb, INGRESS_FWDNS_MAGIC, - BPF_SKB_TSTAMP_UNSPEC)) + BPF_SKB_CLOCK_MONOTONIC)) inc_errs(SET_DTIME); } diff --git a/tools/testing/selftests/bpf/progs/test_tc_link.c b/tools/testing/selftests/bpf/progs/test_tc_link.c index 992400acb9572aa9b6949566e9eeabd2dcd86cd8..ab3eae3d6af8795006f98c1bb8122fffa3a49d5a 100644 --- a/tools/testing/selftests/bpf/progs/test_tc_link.c +++ b/tools/testing/selftests/bpf/progs/test_tc_link.c @@ -4,7 +4,8 @@ #include #include - +#include +#include #include #include @@ -16,7 +17,13 @@ bool seen_tc3; bool seen_tc4; bool seen_tc5; bool seen_tc6; +bool seen_tc7; + +bool set_type; + bool seen_eth; +bool seen_host; +bool seen_mcast; SEC("tc/ingress") int tc1(struct __sk_buff *skb) @@ -28,8 +35,16 @@ int tc1(struct __sk_buff *skb) if (bpf_skb_load_bytes(skb, 0, ð, sizeof(eth))) goto out; seen_eth = eth.h_proto == bpf_htons(ETH_P_IP); + seen_host = skb->pkt_type == PACKET_HOST; + if (seen_host && set_type) { + eth.h_dest[0] = 4; + if (bpf_skb_store_bytes(skb, 0, ð, sizeof(eth), 0)) + goto fail; + bpf_skb_change_type(skb, PACKET_MULTICAST); + } out: seen_tc1 = true; +fail: return TCX_NEXT; } @@ -67,3 +82,21 @@ int tc6(struct __sk_buff *skb) seen_tc6 = true; return TCX_PASS; } + +SEC("tc/ingress") +int tc7(struct __sk_buff *skb) +{ + struct ethhdr eth = {}; + + if (skb->protocol != __bpf_constant_htons(ETH_P_IP)) + goto out; + if (bpf_skb_load_bytes(skb, 0, ð, sizeof(eth))) + goto out; + if (eth.h_dest[0] == 4 && set_type) { + seen_mcast = skb->pkt_type == PACKET_MULTICAST; + bpf_skb_change_type(skb, PACKET_HOST); + } +out: + seen_tc7 = true; + return TCX_PASS; +} diff --git a/tools/testing/selftests/bpf/progs/uprobe_multi.c b/tools/testing/selftests/bpf/progs/uprobe_multi.c index 419d9aa28fce5cdf94e9976e6f8501a3551b4133..44190efcdba217b507e0c5772f9dfc14af72de25 100644 --- a/tools/testing/selftests/bpf/progs/uprobe_multi.c +++ b/tools/testing/selftests/bpf/progs/uprobe_multi.c @@ -1,8 +1,8 @@ // SPDX-License-Identifier: GPL-2.0 -#include +#include "vmlinux.h" #include #include -#include +#include char _license[] SEC("license") = "GPL"; @@ -22,6 +22,13 @@ __u64 uprobe_multi_sleep_result = 0; int pid = 0; int child_pid = 0; +int child_tid = 0; +int child_pid_usdt = 0; +int child_tid_usdt = 0; + +int expect_pid = 0; +bool bad_pid_seen = false; +bool bad_pid_seen_usdt = false; bool test_cookie = false; void *user_ptr = 0; @@ -36,11 +43,19 @@ static __always_inline bool verify_sleepable_user_copy(void) static void uprobe_multi_check(void *ctx, bool is_return, bool is_sleep) { - child_pid = bpf_get_current_pid_tgid() >> 32; + __u64 cur_pid_tgid = bpf_get_current_pid_tgid(); + __u32 cur_pid; - if (pid && child_pid != pid) + cur_pid = cur_pid_tgid >> 32; + if (pid && cur_pid != pid) return; + if (expect_pid && cur_pid != expect_pid) + bad_pid_seen = true; + + child_pid = cur_pid_tgid >> 32; + child_tid = (__u32)cur_pid_tgid; + __u64 cookie = test_cookie ? bpf_get_attach_cookie(ctx) : 0; __u64 addr = bpf_get_func_ip(ctx); @@ -97,5 +112,32 @@ int uretprobe_sleep(struct pt_regs *ctx) SEC("uprobe.multi//proc/self/exe:uprobe_multi_func_*") int uprobe_extra(struct pt_regs *ctx) { + /* we need this one just to mix PID-filtered and global uprobes */ + return 0; +} + +SEC("usdt") +int usdt_pid(struct pt_regs *ctx) +{ + __u64 cur_pid_tgid = bpf_get_current_pid_tgid(); + __u32 cur_pid; + + cur_pid = cur_pid_tgid >> 32; + if (pid && cur_pid != pid) + return 0; + + if (expect_pid && cur_pid != expect_pid) + bad_pid_seen_usdt = true; + + child_pid_usdt = cur_pid_tgid >> 32; + child_tid_usdt = (__u32)cur_pid_tgid; + + return 0; +} + +SEC("usdt") +int usdt_extra(struct pt_regs *ctx) +{ + /* we need this one just to mix PID-filtered and global USDT probes */ return 0; } diff --git a/tools/testing/selftests/bpf/progs/verifier_arena.c b/tools/testing/selftests/bpf/progs/verifier_arena.c index 93144ae6df74124fb7e7bb27084dc1371483d774..67509c5d3982a1699c9b3f1bc0536425335b2e4d 100644 --- a/tools/testing/selftests/bpf/progs/verifier_arena.c +++ b/tools/testing/selftests/bpf/progs/verifier_arena.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#define BPF_NO_KFUNC_PROTOTYPES #include #include #include diff --git a/tools/testing/selftests/bpf/progs/verifier_arena_large.c b/tools/testing/selftests/bpf/progs/verifier_arena_large.c index ef66ea460264c1f587d764d45ecf0cae72b0d4ff..6065f862d96438c7b9c8e583dba06ddec161a7d4 100644 --- a/tools/testing/selftests/bpf/progs/verifier_arena_large.c +++ b/tools/testing/selftests/bpf/progs/verifier_arena_large.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#define BPF_NO_KFUNC_PROTOTYPES #include #include #include diff --git a/tools/testing/selftests/bpf/progs/verifier_bits_iter.c b/tools/testing/selftests/bpf/progs/verifier_bits_iter.c new file mode 100644 index 0000000000000000000000000000000000000000..716113c2bce21873fab00a9ca0977e3de6fc3f01 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/verifier_bits_iter.c @@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2024 Yafang Shao */ + +#include "vmlinux.h" +#include +#include + +#include "bpf_misc.h" +#include "task_kfunc_common.h" + +char _license[] SEC("license") = "GPL"; + +int bpf_iter_bits_new(struct bpf_iter_bits *it, const u64 *unsafe_ptr__ign, + u32 nr_bits) __ksym __weak; +int *bpf_iter_bits_next(struct bpf_iter_bits *it) __ksym __weak; +void bpf_iter_bits_destroy(struct bpf_iter_bits *it) __ksym __weak; + +SEC("iter.s/cgroup") +__description("bits iter without destroy") +__failure __msg("Unreleased reference") +int BPF_PROG(no_destroy, struct bpf_iter_meta *meta, struct cgroup *cgrp) +{ + struct bpf_iter_bits it; + u64 data = 1; + + bpf_iter_bits_new(&it, &data, 1); + bpf_iter_bits_next(&it); + return 0; +} + +SEC("iter/cgroup") +__description("uninitialized iter in ->next()") +__failure __msg("expected an initialized iter_bits as arg #1") +int BPF_PROG(next_uninit, struct bpf_iter_meta *meta, struct cgroup *cgrp) +{ + struct bpf_iter_bits *it = NULL; + + bpf_iter_bits_next(it); + return 0; +} + +SEC("iter/cgroup") +__description("uninitialized iter in ->destroy()") +__failure __msg("expected an initialized iter_bits as arg #1") +int BPF_PROG(destroy_uninit, struct bpf_iter_meta *meta, struct cgroup *cgrp) +{ + struct bpf_iter_bits it = {}; + + bpf_iter_bits_destroy(&it); + return 0; +} + +SEC("syscall") +__description("null pointer") +__success __retval(0) +int null_pointer(void) +{ + int nr = 0; + int *bit; + + bpf_for_each(bits, bit, NULL, 1) + nr++; + return nr; +} + +SEC("syscall") +__description("bits copy") +__success __retval(10) +int bits_copy(void) +{ + u64 data = 0xf7310UL; /* 4 + 3 + 2 + 1 + 0*/ + int nr = 0; + int *bit; + + bpf_for_each(bits, bit, &data, 1) + nr++; + return nr; +} + +SEC("syscall") +__description("bits memalloc") +__success __retval(64) +int bits_memalloc(void) +{ + u64 data[2]; + int nr = 0; + int *bit; + + __builtin_memset(&data, 0xf0, sizeof(data)); /* 4 * 16 */ + bpf_for_each(bits, bit, &data[0], sizeof(data) / sizeof(u64)) + nr++; + return nr; +} + +SEC("syscall") +__description("bit index") +__success __retval(8) +int bit_index(void) +{ + u64 data = 0x100; + int bit_idx = 0; + int *bit; + + bpf_for_each(bits, bit, &data, 1) { + if (*bit == 0) + continue; + bit_idx = *bit; + } + return bit_idx; +} + +SEC("syscall") +__description("bits nomem") +__success __retval(0) +int bits_nomem(void) +{ + u64 data[4]; + int nr = 0; + int *bit; + + __builtin_memset(&data, 0xff, sizeof(data)); + bpf_for_each(bits, bit, &data[0], 513) /* Be greater than 512 */ + nr++; + return nr; +} + +SEC("syscall") +__description("fewer words") +__success __retval(1) +int fewer_words(void) +{ + u64 data[2] = {0x1, 0xff}; + int nr = 0; + int *bit; + + bpf_for_each(bits, bit, &data[0], 1) + nr++; + return nr; +} + +SEC("syscall") +__description("zero words") +__success __retval(0) +int zero_words(void) +{ + u64 data[2] = {0x1, 0xff}; + int nr = 0; + int *bit; + + bpf_for_each(bits, bit, &data[0], 0) + nr++; + return nr; +} diff --git a/tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c b/tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c index bd676d7e615fd538aadaf890b90ec902221e79bc..53679252e8a1d585018dffa9236b2ac1a538bc5b 100644 --- a/tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c +++ b/tools/testing/selftests/bpf/progs/verifier_iterating_callbacks.c @@ -405,4 +405,240 @@ int cond_break5(const void *ctx) return cnt1 > 1 && cnt2 > 1 ? 1 : 0; } +#define ARR2_SZ 1000 +SEC(".data.arr2") +char arr2[ARR2_SZ]; + +SEC("socket") +__success __flag(BPF_F_TEST_STATE_FREQ) +int loop_inside_iter(const void *ctx) +{ + struct bpf_iter_num it; + int *v, sum = 0; + __u64 i = 0; + + bpf_iter_num_new(&it, 0, ARR2_SZ); + while ((v = bpf_iter_num_next(&it))) { + if (i < ARR2_SZ) + sum += arr2[i++]; + } + bpf_iter_num_destroy(&it); + return sum; +} + +SEC("socket") +__success __flag(BPF_F_TEST_STATE_FREQ) +int loop_inside_iter_signed(const void *ctx) +{ + struct bpf_iter_num it; + int *v, sum = 0; + long i = 0; + + bpf_iter_num_new(&it, 0, ARR2_SZ); + while ((v = bpf_iter_num_next(&it))) { + if (i < ARR2_SZ && i >= 0) + sum += arr2[i++]; + } + bpf_iter_num_destroy(&it); + return sum; +} + +volatile const int limit = ARR2_SZ; + +SEC("socket") +__success __flag(BPF_F_TEST_STATE_FREQ) +int loop_inside_iter_volatile_limit(const void *ctx) +{ + struct bpf_iter_num it; + int *v, sum = 0; + __u64 i = 0; + + bpf_iter_num_new(&it, 0, ARR2_SZ); + while ((v = bpf_iter_num_next(&it))) { + if (i < limit) + sum += arr2[i++]; + } + bpf_iter_num_destroy(&it); + return sum; +} + +#define ARR_LONG_SZ 1000 + +SEC(".data.arr_long") +long arr_long[ARR_LONG_SZ]; + +SEC("socket") +__success +int test1(const void *ctx) +{ + long i; + + for (i = 0; i < ARR_LONG_SZ && can_loop; i++) + arr_long[i] = i; + return 0; +} + +SEC("socket") +__success +int test2(const void *ctx) +{ + __u64 i; + + for (i = zero; i < ARR_LONG_SZ && can_loop; i++) { + barrier_var(i); + arr_long[i] = i; + } + return 0; +} + +SEC(".data.arr_foo") +struct { + int a; + int b; +} arr_foo[ARR_LONG_SZ]; + +SEC("socket") +__success +int test3(const void *ctx) +{ + __u64 i; + + for (i = zero; i < ARR_LONG_SZ && can_loop; i++) { + barrier_var(i); + arr_foo[i].a = i; + arr_foo[i].b = i; + } + return 0; +} + +SEC("socket") +__success +int test4(const void *ctx) +{ + long i; + + for (i = zero + ARR_LONG_SZ - 1; i < ARR_LONG_SZ && i >= 0 && can_loop; i--) { + barrier_var(i); + arr_foo[i].a = i; + arr_foo[i].b = i; + } + return 0; +} + +char buf[10] SEC(".data.buf"); + +SEC("socket") +__description("check add const") +__success +__naked void check_add_const(void) +{ + /* typical LLVM generated loop with may_goto */ + asm volatile (" \ + call %[bpf_ktime_get_ns]; \ + if r0 > 9 goto l1_%=; \ +l0_%=: r1 = %[buf]; \ + r2 = r0; \ + r1 += r2; \ + r3 = *(u8 *)(r1 +0); \ + .byte 0xe5; /* may_goto */ \ + .byte 0; /* regs */ \ + .short 4; /* off of l1_%=: */ \ + .long 0; /* imm */ \ + r0 = r2; \ + r0 += 1; \ + if r2 < 9 goto l0_%=; \ + exit; \ +l1_%=: r0 = 0; \ + exit; \ +" : + : __imm(bpf_ktime_get_ns), + __imm_ptr(buf) + : __clobber_common); +} + +SEC("socket") +__failure +__msg("*(u8 *)(r7 +0) = r0") +__msg("invalid access to map value, value_size=10 off=10 size=1") +__naked void check_add_const_3regs(void) +{ + asm volatile ( + "r6 = %[buf];" + "r7 = %[buf];" + "call %[bpf_ktime_get_ns];" + "r1 = r0;" /* link r0.id == r1.id == r2.id */ + "r2 = r0;" + "r1 += 1;" /* r1 == r0+1 */ + "r2 += 2;" /* r2 == r0+2 */ + "if r0 > 8 goto 1f;" /* r0 range [0, 8] */ + "r6 += r1;" /* r1 range [1, 9] */ + "r7 += r2;" /* r2 range [2, 10] */ + "*(u8 *)(r6 +0) = r0;" /* safe, within bounds */ + "*(u8 *)(r7 +0) = r0;" /* unsafe, out of bounds */ + "1: exit;" + : + : __imm(bpf_ktime_get_ns), + __imm_ptr(buf) + : __clobber_common); +} + +SEC("socket") +__failure +__msg("*(u8 *)(r8 -1) = r0") +__msg("invalid access to map value, value_size=10 off=10 size=1") +__naked void check_add_const_3regs_2if(void) +{ + asm volatile ( + "r6 = %[buf];" + "r7 = %[buf];" + "r8 = %[buf];" + "call %[bpf_ktime_get_ns];" + "if r0 < 2 goto 1f;" + "r1 = r0;" /* link r0.id == r1.id == r2.id */ + "r2 = r0;" + "r1 += 1;" /* r1 == r0+1 */ + "r2 += 2;" /* r2 == r0+2 */ + "if r2 > 11 goto 1f;" /* r2 range [0, 11] -> r0 range [-2, 9]; r1 range [-1, 10] */ + "if r0 s< 0 goto 1f;" /* r0 range [0, 9] -> r1 range [1, 10]; r2 range [2, 11]; */ + "r6 += r0;" /* r0 range [0, 9] */ + "r7 += r1;" /* r1 range [1, 10] */ + "r8 += r2;" /* r2 range [2, 11] */ + "*(u8 *)(r6 +0) = r0;" /* safe, within bounds */ + "*(u8 *)(r7 -1) = r0;" /* safe */ + "*(u8 *)(r8 -1) = r0;" /* unsafe */ + "1: exit;" + : + : __imm(bpf_ktime_get_ns), + __imm_ptr(buf) + : __clobber_common); +} + +SEC("socket") +__failure +__flag(BPF_F_TEST_STATE_FREQ) +__naked void check_add_const_regsafe_off(void) +{ + asm volatile ( + "r8 = %[buf];" + "call %[bpf_ktime_get_ns];" + "r6 = r0;" + "call %[bpf_ktime_get_ns];" + "r7 = r0;" + "call %[bpf_ktime_get_ns];" + "r1 = r0;" /* same ids for r1 and r0 */ + "if r6 > r7 goto 1f;" /* this jump can't be predicted */ + "r1 += 1;" /* r1.off == +1 */ + "goto 2f;" + "1: r1 += 100;" /* r1.off == +100 */ + "goto +0;" /* verify r1.off in regsafe() after this insn */ + "2: if r0 > 8 goto 3f;" /* r0 range [0,8], r1 range either [1,9] or [100,108]*/ + "r8 += r1;" + "*(u8 *)(r8 +0) = r0;" /* potentially unsafe, buf size is 10 */ + "3: exit;" + : + : __imm(bpf_ktime_get_ns), + __imm_ptr(buf) + : __clobber_common); +} + char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/verifier_netfilter_ctx.c b/tools/testing/selftests/bpf/progs/verifier_netfilter_ctx.c index 65bba330e7e5c4e83e1403955eccbdee4df4eb61..ab9f9f2620edca00afc9a69f8549921eb392d189 100644 --- a/tools/testing/selftests/bpf/progs/verifier_netfilter_ctx.c +++ b/tools/testing/selftests/bpf/progs/verifier_netfilter_ctx.c @@ -79,7 +79,7 @@ int with_invalid_ctx_access_test5(struct bpf_nf_ctx *ctx) return NF_ACCEPT; } -extern int bpf_dynptr_from_skb(struct sk_buff *skb, __u64 flags, +extern int bpf_dynptr_from_skb(struct __sk_buff *skb, __u64 flags, struct bpf_dynptr *ptr__uninit) __ksym; extern void *bpf_dynptr_slice(const struct bpf_dynptr *ptr, uint32_t offset, void *buffer, uint32_t buffer__sz) __ksym; @@ -90,8 +90,8 @@ __success __failure_unpriv __retval(0) int with_valid_ctx_access_test6(struct bpf_nf_ctx *ctx) { + struct __sk_buff *skb = (struct __sk_buff *)ctx->skb; const struct nf_hook_state *state = ctx->state; - struct sk_buff *skb = ctx->skb; const struct iphdr *iph; const struct tcphdr *th; u8 buffer_iph[20] = {}; @@ -99,7 +99,7 @@ int with_valid_ctx_access_test6(struct bpf_nf_ctx *ctx) struct bpf_dynptr ptr; uint8_t ihl; - if (skb->len <= 20 || bpf_dynptr_from_skb(skb, 0, &ptr)) + if (ctx->skb->len <= 20 || bpf_dynptr_from_skb(skb, 0, &ptr)) return NF_ACCEPT; iph = bpf_dynptr_slice(&ptr, 0, buffer_iph, sizeof(buffer_iph)); diff --git a/tools/testing/selftests/bpf/progs/verifier_sockmap_mutate.c b/tools/testing/selftests/bpf/progs/verifier_sockmap_mutate.c new file mode 100644 index 0000000000000000000000000000000000000000..fe4b123187b86e7ad92d25eda5181bb927408fcd --- /dev/null +++ b/tools/testing/selftests/bpf/progs/verifier_sockmap_mutate.c @@ -0,0 +1,187 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include + +#include "bpf_misc.h" + +#define __always_unused __attribute__((unused)) + +char _license[] SEC("license") = "GPL"; + +struct sock { +} __attribute__((preserve_access_index)); + +struct bpf_iter__sockmap { + union { + struct sock *sk; + }; +} __attribute__((preserve_access_index)); + +struct { + __uint(type, BPF_MAP_TYPE_SOCKHASH); + __uint(max_entries, 1); + __type(key, int); + __type(value, int); +} sockhash SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_SOCKMAP); + __uint(max_entries, 1); + __type(key, int); + __type(value, int); +} sockmap SEC(".maps"); + +enum { CG_OK = 1 }; + +int zero = 0; + +static __always_inline void test_sockmap_delete(void) +{ + bpf_map_delete_elem(&sockmap, &zero); + bpf_map_delete_elem(&sockhash, &zero); +} + +static __always_inline void test_sockmap_update(void *sk) +{ + if (sk) { + bpf_map_update_elem(&sockmap, &zero, sk, BPF_ANY); + bpf_map_update_elem(&sockhash, &zero, sk, BPF_ANY); + } +} + +static __always_inline void test_sockmap_lookup_and_update(void) +{ + struct bpf_sock *sk = bpf_map_lookup_elem(&sockmap, &zero); + + if (sk) { + test_sockmap_update(sk); + bpf_sk_release(sk); + } +} + +static __always_inline void test_sockmap_mutate(void *sk) +{ + test_sockmap_delete(); + test_sockmap_update(sk); +} + +static __always_inline void test_sockmap_lookup_and_mutate(void) +{ + test_sockmap_delete(); + test_sockmap_lookup_and_update(); +} + +SEC("action") +__success +int test_sched_act(struct __sk_buff *skb) +{ + test_sockmap_mutate(skb->sk); + return 0; +} + +SEC("classifier") +__success +int test_sched_cls(struct __sk_buff *skb) +{ + test_sockmap_mutate(skb->sk); + return 0; +} + +SEC("flow_dissector") +__success +int test_flow_dissector_delete(struct __sk_buff *skb __always_unused) +{ + test_sockmap_delete(); + return 0; +} + +SEC("flow_dissector") +__failure __msg("program of this type cannot use helper bpf_sk_release") +int test_flow_dissector_update(struct __sk_buff *skb __always_unused) +{ + test_sockmap_lookup_and_update(); /* no access to skb->sk */ + return 0; +} + +SEC("iter/sockmap") +__success +int test_trace_iter(struct bpf_iter__sockmap *ctx) +{ + test_sockmap_mutate(ctx->sk); + return 0; +} + +SEC("raw_tp/kfree") +__failure __msg("cannot update sockmap in this context") +int test_raw_tp_delete(const void *ctx __always_unused) +{ + test_sockmap_delete(); + return 0; +} + +SEC("raw_tp/kfree") +__failure __msg("cannot update sockmap in this context") +int test_raw_tp_update(const void *ctx __always_unused) +{ + test_sockmap_lookup_and_update(); + return 0; +} + +SEC("sk_lookup") +__success +int test_sk_lookup(struct bpf_sk_lookup *ctx) +{ + test_sockmap_mutate(ctx->sk); + return 0; +} + +SEC("sk_reuseport") +__success +int test_sk_reuseport(struct sk_reuseport_md *ctx) +{ + test_sockmap_mutate(ctx->sk); + return 0; +} + +SEC("socket") +__success +int test_socket_filter(struct __sk_buff *skb) +{ + test_sockmap_mutate(skb->sk); + return 0; +} + +SEC("sockops") +__success +int test_sockops_delete(struct bpf_sock_ops *ctx __always_unused) +{ + test_sockmap_delete(); + return CG_OK; +} + +SEC("sockops") +__failure __msg("cannot update sockmap in this context") +int test_sockops_update(struct bpf_sock_ops *ctx) +{ + test_sockmap_update(ctx->sk); + return CG_OK; +} + +SEC("sockops") +__success +int test_sockops_update_dedicated(struct bpf_sock_ops *ctx) +{ + bpf_sock_map_update(ctx, &sockmap, &zero, BPF_ANY); + bpf_sock_hash_update(ctx, &sockhash, &zero, BPF_ANY); + return CG_OK; +} + +SEC("xdp") +__success +int test_xdp(struct xdp_md *ctx __always_unused) +{ + test_sockmap_lookup_and_mutate(); + return XDP_PASS; +} diff --git a/tools/testing/selftests/bpf/progs/xdp_synproxy_kern.c b/tools/testing/selftests/bpf/progs/xdp_synproxy_kern.c index 7ea9785738b5a0f79fc32af8d9950b4723f18ce5..f8f5dc9f72b8ca7168605d5f8ab3fc245843ffc2 100644 --- a/tools/testing/selftests/bpf/progs/xdp_synproxy_kern.c +++ b/tools/testing/selftests/bpf/progs/xdp_synproxy_kern.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause /* Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ +#define BPF_NO_KFUNC_PROTOTYPES #include "vmlinux.h" #include diff --git a/tools/testing/selftests/bpf/progs/xfrm_info.c b/tools/testing/selftests/bpf/progs/xfrm_info.c index f6a501fbba2bedcd82fc115deab8714930ac6b36..a1d9f106c3f0688e163647c093bcf7ba5e835180 100644 --- a/tools/testing/selftests/bpf/progs/xfrm_info.c +++ b/tools/testing/selftests/bpf/progs/xfrm_info.c @@ -1,4 +1,5 @@ // SPDX-License-Identifier: GPL-2.0 +#define BPF_NO_KFUNC_PROTOTYPES #include "vmlinux.h" #include "bpf_tracing_net.h" #include diff --git a/tools/testing/selftests/bpf/test_sockmap.c b/tools/testing/selftests/bpf/test_sockmap.c index 92752f5eededf90514627bec6d2302be5741c61b..9cba4ec844a53712eb7cb007379faabe139521cb 100644 --- a/tools/testing/selftests/bpf/test_sockmap.c +++ b/tools/testing/selftests/bpf/test_sockmap.c @@ -63,7 +63,8 @@ int passed; int failed; int map_fd[9]; struct bpf_map *maps[9]; -int prog_fd[11]; +struct bpf_program *progs[9]; +struct bpf_link *links[9]; int txmsg_pass; int txmsg_redir; @@ -680,7 +681,8 @@ static int msg_loop(int fd, int iov_count, int iov_length, int cnt, } } - s->bytes_recvd += recv; + if (recv > 0) + s->bytes_recvd += recv; if (opt->check_recved_len && s->bytes_recvd > total_bytes) { errno = EMSGSIZE; @@ -952,7 +954,8 @@ enum { static int run_options(struct sockmap_options *options, int cg_fd, int test) { - int i, key, next_key, err, tx_prog_fd = -1, zero = 0; + int i, key, next_key, err, zero = 0; + struct bpf_program *tx_prog; /* If base test skip BPF setup */ if (test == BASE || test == BASE_SENDPAGE) @@ -960,48 +963,44 @@ static int run_options(struct sockmap_options *options, int cg_fd, int test) /* Attach programs to sockmap */ if (!txmsg_omit_skb_parser) { - err = bpf_prog_attach(prog_fd[0], map_fd[0], - BPF_SK_SKB_STREAM_PARSER, 0); - if (err) { + links[0] = bpf_program__attach_sockmap(progs[0], map_fd[0]); + if (!links[0]) { fprintf(stderr, - "ERROR: bpf_prog_attach (sockmap %i->%i): %d (%s)\n", - prog_fd[0], map_fd[0], err, strerror(errno)); - return err; + "ERROR: bpf_program__attach_sockmap (sockmap %i->%i): (%s)\n", + bpf_program__fd(progs[0]), map_fd[0], strerror(errno)); + return -1; } } - err = bpf_prog_attach(prog_fd[1], map_fd[0], - BPF_SK_SKB_STREAM_VERDICT, 0); - if (err) { - fprintf(stderr, "ERROR: bpf_prog_attach (sockmap): %d (%s)\n", - err, strerror(errno)); - return err; + links[1] = bpf_program__attach_sockmap(progs[1], map_fd[0]); + if (!links[1]) { + fprintf(stderr, "ERROR: bpf_program__attach_sockmap (sockmap): (%s)\n", + strerror(errno)); + return -1; } /* Attach programs to TLS sockmap */ if (txmsg_ktls_skb) { if (!txmsg_omit_skb_parser) { - err = bpf_prog_attach(prog_fd[0], map_fd[8], - BPF_SK_SKB_STREAM_PARSER, 0); - if (err) { + links[2] = bpf_program__attach_sockmap(progs[0], map_fd[8]); + if (!links[2]) { fprintf(stderr, - "ERROR: bpf_prog_attach (TLS sockmap %i->%i): %d (%s)\n", - prog_fd[0], map_fd[8], err, strerror(errno)); - return err; + "ERROR: bpf_program__attach_sockmap (TLS sockmap %i->%i): (%s)\n", + bpf_program__fd(progs[0]), map_fd[8], strerror(errno)); + return -1; } } - err = bpf_prog_attach(prog_fd[2], map_fd[8], - BPF_SK_SKB_STREAM_VERDICT, 0); - if (err) { - fprintf(stderr, "ERROR: bpf_prog_attach (TLS sockmap): %d (%s)\n", - err, strerror(errno)); - return err; + links[3] = bpf_program__attach_sockmap(progs[2], map_fd[8]); + if (!links[3]) { + fprintf(stderr, "ERROR: bpf_program__attach_sockmap (TLS sockmap): (%s)\n", + strerror(errno)); + return -1; } } /* Attach to cgroups */ - err = bpf_prog_attach(prog_fd[3], cg_fd, BPF_CGROUP_SOCK_OPS, 0); + err = bpf_prog_attach(bpf_program__fd(progs[3]), cg_fd, BPF_CGROUP_SOCK_OPS, 0); if (err) { fprintf(stderr, "ERROR: bpf_prog_attach (groups): %d (%s)\n", err, strerror(errno)); @@ -1017,30 +1016,31 @@ run: /* Attach txmsg program to sockmap */ if (txmsg_pass) - tx_prog_fd = prog_fd[4]; + tx_prog = progs[4]; else if (txmsg_redir) - tx_prog_fd = prog_fd[5]; + tx_prog = progs[5]; else if (txmsg_apply) - tx_prog_fd = prog_fd[6]; + tx_prog = progs[6]; else if (txmsg_cork) - tx_prog_fd = prog_fd[7]; + tx_prog = progs[7]; else if (txmsg_drop) - tx_prog_fd = prog_fd[8]; + tx_prog = progs[8]; else - tx_prog_fd = 0; + tx_prog = NULL; - if (tx_prog_fd) { - int redir_fd, i = 0; + if (tx_prog) { + int redir_fd; - err = bpf_prog_attach(tx_prog_fd, - map_fd[1], BPF_SK_MSG_VERDICT, 0); - if (err) { + links[4] = bpf_program__attach_sockmap(tx_prog, map_fd[1]); + if (!links[4]) { fprintf(stderr, - "ERROR: bpf_prog_attach (txmsg): %d (%s)\n", - err, strerror(errno)); + "ERROR: bpf_program__attach_sockmap (txmsg): (%s)\n", + strerror(errno)); + err = -1; goto out; } + i = 0; err = bpf_map_update_elem(map_fd[1], &i, &c1, BPF_ANY); if (err) { fprintf(stderr, @@ -1279,16 +1279,14 @@ run: fprintf(stderr, "unknown test\n"); out: /* Detatch and zero all the maps */ - bpf_prog_detach2(prog_fd[3], cg_fd, BPF_CGROUP_SOCK_OPS); - bpf_prog_detach2(prog_fd[0], map_fd[0], BPF_SK_SKB_STREAM_PARSER); - bpf_prog_detach2(prog_fd[1], map_fd[0], BPF_SK_SKB_STREAM_VERDICT); - bpf_prog_detach2(prog_fd[0], map_fd[8], BPF_SK_SKB_STREAM_PARSER); - bpf_prog_detach2(prog_fd[2], map_fd[8], BPF_SK_SKB_STREAM_VERDICT); + bpf_prog_detach2(bpf_program__fd(progs[3]), cg_fd, BPF_CGROUP_SOCK_OPS); - if (tx_prog_fd >= 0) - bpf_prog_detach2(tx_prog_fd, map_fd[1], BPF_SK_MSG_VERDICT); + for (i = 0; i < ARRAY_SIZE(links); i++) { + if (links[i]) + bpf_link__detach(links[i]); + } - for (i = 0; i < 8; i++) { + for (i = 0; i < ARRAY_SIZE(map_fd); i++) { key = next_key = 0; bpf_map_update_elem(map_fd[i], &key, &zero, BPF_ANY); while (bpf_map_get_next_key(map_fd[i], &key, &next_key) == 0) { @@ -1783,34 +1781,6 @@ char *map_names[] = { "tls_sock_map", }; -int prog_attach_type[] = { - BPF_SK_SKB_STREAM_PARSER, - BPF_SK_SKB_STREAM_VERDICT, - BPF_SK_SKB_STREAM_VERDICT, - BPF_CGROUP_SOCK_OPS, - BPF_SK_MSG_VERDICT, - BPF_SK_MSG_VERDICT, - BPF_SK_MSG_VERDICT, - BPF_SK_MSG_VERDICT, - BPF_SK_MSG_VERDICT, - BPF_SK_MSG_VERDICT, - BPF_SK_MSG_VERDICT, -}; - -int prog_type[] = { - BPF_PROG_TYPE_SK_SKB, - BPF_PROG_TYPE_SK_SKB, - BPF_PROG_TYPE_SK_SKB, - BPF_PROG_TYPE_SOCK_OPS, - BPF_PROG_TYPE_SK_MSG, - BPF_PROG_TYPE_SK_MSG, - BPF_PROG_TYPE_SK_MSG, - BPF_PROG_TYPE_SK_MSG, - BPF_PROG_TYPE_SK_MSG, - BPF_PROG_TYPE_SK_MSG, - BPF_PROG_TYPE_SK_MSG, -}; - static int populate_progs(char *bpf_file) { struct bpf_program *prog; @@ -1829,17 +1799,10 @@ static int populate_progs(char *bpf_file) return -1; } - bpf_object__for_each_program(prog, obj) { - bpf_program__set_type(prog, prog_type[i]); - bpf_program__set_expected_attach_type(prog, - prog_attach_type[i]); - i++; - } - i = bpf_object__load(obj); i = 0; bpf_object__for_each_program(prog, obj) { - prog_fd[i] = bpf_program__fd(prog); + progs[i] = prog; i++; } @@ -1853,6 +1816,9 @@ static int populate_progs(char *bpf_file) } } + for (i = 0; i < ARRAY_SIZE(links); i++) + links[i] = NULL; + return 0; } diff --git a/tools/testing/selftests/bpf/test_tcp_check_syncookie_user.c b/tools/testing/selftests/bpf/test_tcp_check_syncookie_user.c index 7b5fc98838cd3c69d442c85b33def862bb4a6cc9..aebc58c24dc57403177abc08fc743c0aa4dc5433 100644 --- a/tools/testing/selftests/bpf/test_tcp_check_syncookie_user.c +++ b/tools/testing/selftests/bpf/test_tcp_check_syncookie_user.c @@ -139,14 +139,14 @@ out: return ret; } -static int v6only_true(int fd, const struct post_socket_opts *opts) +static int v6only_true(int fd, void *opts) { int mode = true; return setsockopt(fd, IPPROTO_IPV6, IPV6_V6ONLY, &mode, sizeof(mode)); } -static int v6only_false(int fd, const struct post_socket_opts *opts) +static int v6only_false(int fd, void *opts) { int mode = false; diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c index df04bda1c9277bcb1bd55bfc341c3668e322e606..610392dfc4fbc63178237a4032ddc0cccb12251e 100644 --- a/tools/testing/selftests/bpf/test_verifier.c +++ b/tools/testing/selftests/bpf/test_verifier.c @@ -1237,11 +1237,6 @@ static void do_test_fixup(struct bpf_test *test, enum bpf_prog_type prog_type, fixup_prog_kfuncs(prog, fd_array, test->fixup_kfunc_btf_id); } -struct libcap { - struct __user_cap_header_struct hdr; - struct __user_cap_data_struct data[2]; -}; - static int set_admin(bool admin) { int err; diff --git a/tools/testing/selftests/bpf/trace_helpers.c b/tools/testing/selftests/bpf/trace_helpers.c index 70e29f316fe7c43f0a155a244e4f68b28d411b8c..465d196c716514037efef2ef6e229314a93e1f41 100644 --- a/tools/testing/selftests/bpf/trace_helpers.c +++ b/tools/testing/selftests/bpf/trace_helpers.c @@ -211,7 +211,7 @@ long ksym_get_addr(const char *name) */ int kallsyms_find(const char *sym, unsigned long long *addr) { - char type, name[500]; + char type, name[500], *match; unsigned long long value; int err = 0; FILE *f; @@ -221,6 +221,17 @@ int kallsyms_find(const char *sym, unsigned long long *addr) return -EINVAL; while (fscanf(f, "%llx %c %499s%*[^\n]\n", &value, &type, name) > 0) { + /* If CONFIG_LTO_CLANG_THIN is enabled, static variable/function + * symbols could be promoted to global due to cross-file inlining. + * For such cases, clang compiler will add .llvm. suffix + * to those symbols to avoid potential naming conflict. + * Let us ignore .llvm. suffix during symbol comparison. + */ + if (type == 'd') { + match = strstr(name, ".llvm."); + if (match) + *match = '\0'; + } if (strcmp(name, sym) == 0) { *addr = value; goto out; diff --git a/tools/testing/selftests/bpf/verifier/precise.c b/tools/testing/selftests/bpf/verifier/precise.c index 0a9293a572115d3e6c9ba1a4df10824462c323df..90643ccc221d140fe009cf5d4c37dc904cdf6f2a 100644 --- a/tools/testing/selftests/bpf/verifier/precise.c +++ b/tools/testing/selftests/bpf/verifier/precise.c @@ -39,12 +39,12 @@ .result = VERBOSE_ACCEPT, .errstr = "mark_precise: frame0: last_idx 26 first_idx 20\ - mark_precise: frame0: regs=r2 stack= before 25\ - mark_precise: frame0: regs=r2 stack= before 24\ - mark_precise: frame0: regs=r2 stack= before 23\ - mark_precise: frame0: regs=r2 stack= before 22\ - mark_precise: frame0: regs=r2 stack= before 20\ - mark_precise: frame0: parent state regs=r2 stack=:\ + mark_precise: frame0: regs=r2,r9 stack= before 25\ + mark_precise: frame0: regs=r2,r9 stack= before 24\ + mark_precise: frame0: regs=r2,r9 stack= before 23\ + mark_precise: frame0: regs=r2,r9 stack= before 22\ + mark_precise: frame0: regs=r2,r9 stack= before 20\ + mark_precise: frame0: parent state regs=r2,r9 stack=:\ mark_precise: frame0: last_idx 19 first_idx 10\ mark_precise: frame0: regs=r2,r9 stack= before 19\ mark_precise: frame0: regs=r9 stack= before 18\ @@ -100,11 +100,11 @@ .errstr = "26: (85) call bpf_probe_read_kernel#113\ mark_precise: frame0: last_idx 26 first_idx 22\ - mark_precise: frame0: regs=r2 stack= before 25\ - mark_precise: frame0: regs=r2 stack= before 24\ - mark_precise: frame0: regs=r2 stack= before 23\ - mark_precise: frame0: regs=r2 stack= before 22\ - mark_precise: frame0: parent state regs=r2 stack=:\ + mark_precise: frame0: regs=r2,r9 stack= before 25\ + mark_precise: frame0: regs=r2,r9 stack= before 24\ + mark_precise: frame0: regs=r2,r9 stack= before 23\ + mark_precise: frame0: regs=r2,r9 stack= before 22\ + mark_precise: frame0: parent state regs=r2,r9 stack=:\ mark_precise: frame0: last_idx 20 first_idx 20\ mark_precise: frame0: regs=r2,r9 stack= before 20\ mark_precise: frame0: parent state regs=r2,r9 stack=:\ diff --git a/tools/testing/selftests/cachestat/test_cachestat.c b/tools/testing/selftests/cachestat/test_cachestat.c index b171fd53b004e8372bf9e73bdce84e5429d79b11..632ab44737ec3afca5412172e6df67cff87b1e28 100644 --- a/tools/testing/selftests/cachestat/test_cachestat.c +++ b/tools/testing/selftests/cachestat/test_cachestat.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #define _GNU_SOURCE +#define __SANE_USERSPACE_TYPES__ // Use ll64 #include #include diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/tc_flower.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/tc_flower.sh index 31252bc8775e081866a42ac519576b0d7a38a1cb..4994bea5daf80a8053eb382ad85df31eee4c1d9f 100755 --- a/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/tc_flower.sh +++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/tc_flower.sh @@ -11,7 +11,7 @@ ALL_TESTS="single_mask_test identical_filters_test two_masks_test \ multiple_masks_test ctcam_edge_cases_test delta_simple_test \ delta_two_masks_one_key_test delta_simple_rehash_test \ bloom_simple_test bloom_complex_test bloom_delta_test \ - max_erp_entries_test max_group_size_test" + max_erp_entries_test max_group_size_test collision_test" NUM_NETIFS=2 source $lib_dir/lib.sh source $lib_dir/tc_common.sh @@ -457,7 +457,7 @@ delta_two_masks_one_key_test() { # If 2 keys are the same and only differ in mask in a way that # they belong under the same ERP (second is delta of the first), - # there should be no C-TCAM spill. + # there should be C-TCAM spill. RET=0 @@ -474,8 +474,8 @@ delta_two_masks_one_key_test() tp_record "mlxsw:*" "tc filter add dev $h2 ingress protocol ip \ pref 2 handle 102 flower $tcflags dst_ip 192.0.2.2 \ action drop" - tp_check_hits "mlxsw:mlxsw_sp_acl_atcam_entry_add_ctcam_spill" 0 - check_err $? "incorrect C-TCAM spill while inserting the second rule" + tp_check_hits "mlxsw:mlxsw_sp_acl_atcam_entry_add_ctcam_spill" 1 + check_err $? "C-TCAM spill did not happen while inserting the second rule" $MZ $h1 -c 1 -p 64 -a $h1mac -b $h2mac -A 192.0.2.1 -B 192.0.2.2 \ -t ip -q @@ -1087,6 +1087,53 @@ max_group_size_test() log_test "max ACL group size test ($tcflags). max size $max_size" } +collision_test() +{ + # Filters cannot share an eRP if in the common unmasked part (i.e., + # without the delta bits) they have the same values. If the driver does + # not prevent such configuration (by spilling into the C-TCAM), then + # multiple entries will be present in the device with the same key, + # leading to collisions and a reduced scale. + # + # Create such a scenario and make sure all the filters are successfully + # added. + + RET=0 + + local ret + + if [[ "$tcflags" != "skip_sw" ]]; then + return 0; + fi + + # Add a single dst_ip/24 filter and multiple dst_ip/32 filters that all + # have the same values in the common unmasked part (dst_ip/24). + + tc filter add dev $h2 ingress pref 1 proto ipv4 handle 101 \ + flower $tcflags dst_ip 198.51.100.0/24 \ + action drop + + for i in {0..255}; do + tc filter add dev $h2 ingress pref 2 proto ipv4 \ + handle $((102 + i)) \ + flower $tcflags dst_ip 198.51.100.${i}/32 \ + action drop + ret=$? + [[ $ret -ne 0 ]] && break + done + + check_err $ret "failed to add all the filters" + + for i in {255..0}; do + tc filter del dev $h2 ingress pref 2 proto ipv4 \ + handle $((102 + i)) flower + done + + tc filter del dev $h2 ingress pref 1 proto ipv4 handle 101 flower + + log_test "collision test ($tcflags)" +} + setup_prepare() { h1=${NETIFS[p1]} diff --git a/tools/testing/selftests/filesystems/overlayfs/dev_in_maps.c b/tools/testing/selftests/filesystems/overlayfs/dev_in_maps.c index 759f86e7d263e43bcd7438b5979cc9815c2d4c2f..2862aae58b79acbe175ab6b36b42798bb99a2225 100644 --- a/tools/testing/selftests/filesystems/overlayfs/dev_in_maps.c +++ b/tools/testing/selftests/filesystems/overlayfs/dev_in_maps.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #define _GNU_SOURCE +#define __SANE_USERSPACE_TYPES__ // Use ll64 #include #include diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config index e59d985eeff0c8be95c79fd914ac706439952dc7..048a312abf405243da08a2a28216ed8a98dab4d7 100644 --- a/tools/testing/selftests/ftrace/config +++ b/tools/testing/selftests/ftrace/config @@ -1,16 +1,28 @@ -CONFIG_KPROBES=y +CONFIG_BPF_SYSCALL=y +CONFIG_DEBUG_INFO_BTF=y +CONFIG_DEBUG_INFO_DWARF4=y +CONFIG_EPROBE_EVENTS=y +CONFIG_FPROBE=y +CONFIG_FPROBE_EVENTS=y CONFIG_FTRACE=y +CONFIG_FTRACE_SYSCALLS=y +CONFIG_FUNCTION_GRAPH_RETVAL=y CONFIG_FUNCTION_PROFILER=y -CONFIG_TRACER_SNAPSHOT=y -CONFIG_STACK_TRACER=y CONFIG_HIST_TRIGGERS=y -CONFIG_SCHED_TRACER=y -CONFIG_PREEMPT_TRACER=y CONFIG_IRQSOFF_TRACER=y -CONFIG_PREEMPTIRQ_DELAY_TEST=m +CONFIG_KALLSYMS_ALL=y +CONFIG_KPROBES=y +CONFIG_KPROBE_EVENTS=y CONFIG_MODULES=y CONFIG_MODULE_UNLOAD=y +CONFIG_PREEMPTIRQ_DELAY_TEST=m +CONFIG_PREEMPT_TRACER=y +CONFIG_PROBE_EVENTS_BTF_ARGS=y CONFIG_SAMPLES=y CONFIG_SAMPLE_FTRACE_DIRECT=m CONFIG_SAMPLE_TRACE_PRINTK=m -CONFIG_KALLSYMS_ALL=y +CONFIG_SCHED_TRACER=y +CONFIG_STACK_TRACER=y +CONFIG_TRACER_SNAPSHOT=y +CONFIG_UPROBES=y +CONFIG_UPROBE_EVENTS=y diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/test_duplicates.tc b/tools/testing/selftests/ftrace/test.d/dynevent/test_duplicates.tc index d3a79da215c8b08a053c09f3547d3f4fc2f74dcc..5f72abe6fa79bbe0d199a8dcabfdb2bbbbf119fe 100644 --- a/tools/testing/selftests/ftrace/test.d/dynevent/test_duplicates.tc +++ b/tools/testing/selftests/ftrace/test.d/dynevent/test_duplicates.tc @@ -1,7 +1,7 @@ #!/bin/sh # SPDX-License-Identifier: GPL-2.0 # description: Generic dynamic event - check if duplicate events are caught -# requires: dynamic_events "e[:[/][]] . []":README +# requires: dynamic_events "e[:[/][]] . []":README events/syscalls/sys_enter_openat echo 0 > events/enable diff --git a/tools/testing/selftests/ftrace/test.d/filter/event-filter-function.tc b/tools/testing/selftests/ftrace/test.d/filter/event-filter-function.tc index 3f74c09c56b62465189346e6a56f2525cb97bd6b..118247b8dd84d8733ee2147f7e27d0d5635cfe5d 100644 --- a/tools/testing/selftests/ftrace/test.d/filter/event-filter-function.tc +++ b/tools/testing/selftests/ftrace/test.d/filter/event-filter-function.tc @@ -10,7 +10,6 @@ fail() { #msg } sample_events() { - echo > trace echo 1 > events/kmem/kmem_cache_free/enable echo 1 > tracing_on ls > /dev/null @@ -22,6 +21,7 @@ echo 0 > tracing_on echo 0 > events/enable echo "Get the most frequently calling function" +echo > trace sample_events target_func=`cat trace | grep -o 'call_site=\([^+]*\)' | sed 's/call_site=//' | sort | uniq -c | sort | tail -n 1 | sed 's/^[ 0-9]*//'` @@ -32,7 +32,16 @@ echo > trace echo "Test event filter function name" echo "call_site.function == $target_func" > events/kmem/kmem_cache_free/filter + +sample_events +max_retry=10 +while [ `grep kmem_cache_free trace| wc -l` -eq 0 ]; do sample_events +max_retry=$((max_retry - 1)) +if [ $max_retry -eq 0 ]; then + exit_fail +fi +done hitcnt=`grep kmem_cache_free trace| grep $target_func | wc -l` misscnt=`grep kmem_cache_free trace| grep -v $target_func | wc -l` @@ -49,7 +58,16 @@ address=`grep " ${target_func}\$" /proc/kallsyms | cut -d' ' -f1` echo "Test event filter function address" echo "call_site.function == 0x$address" > events/kmem/kmem_cache_free/filter +echo > trace +sample_events +max_retry=10 +while [ `grep kmem_cache_free trace| wc -l` -eq 0 ]; do sample_events +max_retry=$((max_retry - 1)) +if [ $max_retry -eq 0 ]; then + exit_fail +fi +done hitcnt=`grep kmem_cache_free trace| grep $target_func | wc -l` misscnt=`grep kmem_cache_free trace| grep -v $target_func | wc -l` diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_eventname.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_eventname.tc index 1f6981ef7afa063bd8b1185bffa5159ea3e40ce3..ba19b81cef39afc7426c46d3eb1bc4cbe9ead83a 100644 --- a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_eventname.tc +++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_eventname.tc @@ -30,7 +30,8 @@ find_dot_func() { fi grep " [tT] .*\.isra\..*" /proc/kallsyms | cut -f 3 -d " " | while read f; do - if grep -s $f available_filter_functions; then + cnt=`grep -s $f available_filter_functions | wc -l`; + if [ $cnt -eq 1 ]; then echo $f break fi diff --git a/tools/testing/selftests/futex/Makefile b/tools/testing/selftests/futex/Makefile index 11e157d7533b8fbaf5ac691b3df7057e0ff558c1..78ab2cd111f608aca6d35a80937bd72f3ce0c3d4 100644 --- a/tools/testing/selftests/futex/Makefile +++ b/tools/testing/selftests/futex/Makefile @@ -3,8 +3,6 @@ SUBDIRS := functional TEST_PROGS := run.sh -.PHONY: all clean - include ../lib.mk all: diff --git a/tools/testing/selftests/futex/functional/Makefile b/tools/testing/selftests/futex/functional/Makefile index a392d0917b4e55c145b293e1e60674bc904e74c0..994fa3468f170cb05c258c17298819a78af100bf 100644 --- a/tools/testing/selftests/futex/functional/Makefile +++ b/tools/testing/selftests/futex/functional/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 INCLUDES := -I../include -I../../ $(KHDR_INCLUDES) -CFLAGS := $(CFLAGS) -g -O2 -Wall -D_GNU_SOURCE -pthread $(INCLUDES) $(KHDR_INCLUDES) +CFLAGS := $(CFLAGS) -g -O2 -Wall -D_GNU_SOURCE= -pthread $(INCLUDES) $(KHDR_INCLUDES) LDLIBS := -lpthread -lrt LOCAL_HDRS := \ diff --git a/tools/testing/selftests/futex/functional/futex_requeue_pi.c b/tools/testing/selftests/futex/functional/futex_requeue_pi.c index 7f3ca5c78df12968f0aa14168f7dc78001b8fff8..215c6cb539b4abc7a48fb00b545ae26e52fd118a 100644 --- a/tools/testing/selftests/futex/functional/futex_requeue_pi.c +++ b/tools/testing/selftests/futex/functional/futex_requeue_pi.c @@ -360,7 +360,7 @@ out: int main(int argc, char *argv[]) { - const char *test_name; + char *test_name; int c, ret; while ((c = getopt(argc, argv, "bchlot:v:")) != -1) { diff --git a/tools/testing/selftests/landlock/fs_test.c b/tools/testing/selftests/landlock/fs_test.c index 6b5a9ff88c3d715ae14f12635dcbee9b83ee6bcb..7d063c652be164b589887f546095b58752f09ac3 100644 --- a/tools/testing/selftests/landlock/fs_test.c +++ b/tools/testing/selftests/landlock/fs_test.c @@ -35,6 +35,7 @@ * See https://sourceware.org/glibc/wiki/Synchronizing_Headers. */ #include +#include #include "common.h" @@ -47,6 +48,13 @@ int renameat2(int olddirfd, const char *oldpath, int newdirfd, } #endif +#ifndef open_tree +int open_tree(int dfd, const char *filename, unsigned int flags) +{ + return syscall(__NR_open_tree, dfd, filename, flags); +} +#endif + #ifndef RENAME_EXCHANGE #define RENAME_EXCHANGE (1 << 1) #endif @@ -2400,6 +2408,43 @@ TEST_F_FORK(layout1, refer_denied_by_default4) layer_dir_s1d1_refer); } +/* + * Tests walking through a denied root mount. + */ +TEST_F_FORK(layout1, refer_mount_root_deny) +{ + const struct landlock_ruleset_attr ruleset_attr = { + .handled_access_fs = LANDLOCK_ACCESS_FS_MAKE_DIR, + }; + int root_fd, ruleset_fd; + + /* Creates a mount object from a non-mount point. */ + set_cap(_metadata, CAP_SYS_ADMIN); + root_fd = + open_tree(AT_FDCWD, dir_s1d1, + AT_EMPTY_PATH | OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); + clear_cap(_metadata, CAP_SYS_ADMIN); + ASSERT_LE(0, root_fd); + + ruleset_fd = + landlock_create_ruleset(&ruleset_attr, sizeof(ruleset_attr), 0); + ASSERT_LE(0, ruleset_fd); + + ASSERT_EQ(0, prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0)); + ASSERT_EQ(0, landlock_restrict_self(ruleset_fd, 0)); + EXPECT_EQ(0, close(ruleset_fd)); + + /* Link denied by Landlock: EACCES. */ + EXPECT_EQ(-1, linkat(root_fd, ".", root_fd, "does_not_exist", 0)); + EXPECT_EQ(EACCES, errno); + + /* renameat2() always returns EBUSY. */ + EXPECT_EQ(-1, renameat2(root_fd, ".", root_fd, "does_not_exist", 0)); + EXPECT_EQ(EBUSY, errno); + + EXPECT_EQ(0, close(root_fd)); +} + TEST_F_FORK(layout1, reparent_link) { const struct rule layer1[] = { diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile index bd01e4a0be2c29663097f149cc359a892793611e..6da63d1831c11c36dfec93dd5027833a5067c3cb 100644 --- a/tools/testing/selftests/net/Makefile +++ b/tools/testing/selftests/net/Makefile @@ -53,6 +53,7 @@ TEST_PROGS += bind_bhash.sh TEST_PROGS += ip_local_port_range.sh TEST_PROGS += rps_default_mask.sh TEST_PROGS += big_tcp.sh +TEST_PROGS += netns-sysctl.sh TEST_PROGS_EXTENDED := toeplitz_client.sh toeplitz.sh TEST_GEN_FILES = socket nettest TEST_GEN_FILES += psock_fanout psock_tpacket msg_zerocopy reuseport_addr_any diff --git a/tools/testing/selftests/net/hsr/config b/tools/testing/selftests/net/hsr/config index 22061204fb691298e6004aaec1839d011bd17ecb..241542441c5177f60f08963f0b63d456d0ecfdbc 100644 --- a/tools/testing/selftests/net/hsr/config +++ b/tools/testing/selftests/net/hsr/config @@ -2,3 +2,4 @@ CONFIG_IPV6=y CONFIG_NET_SCH_NETEM=m CONFIG_HSR=y CONFIG_VETH=y +CONFIG_BRIDGE=y diff --git a/tools/testing/selftests/net/hsr/hsr_ping.sh b/tools/testing/selftests/net/hsr/hsr_ping.sh index 790294c8af83271d0962e8ddecfc132449bc89fb..f5d207fc770ad14714d902469e68e194de4981ae 100755 --- a/tools/testing/selftests/net/hsr/hsr_ping.sh +++ b/tools/testing/selftests/net/hsr/hsr_ping.sh @@ -152,6 +152,15 @@ setup_hsr_interfaces() ip -net "$ns3" addr add 100.64.0.3/24 dev hsr3 ip -net "$ns3" addr add dead:beef:1::3/64 dev hsr3 nodad + ip -net "$ns1" link set address 00:11:22:00:01:01 dev ns1eth1 + ip -net "$ns1" link set address 00:11:22:00:01:02 dev ns1eth2 + + ip -net "$ns2" link set address 00:11:22:00:02:01 dev ns2eth1 + ip -net "$ns2" link set address 00:11:22:00:02:02 dev ns2eth2 + + ip -net "$ns3" link set address 00:11:22:00:03:01 dev ns3eth1 + ip -net "$ns3" link set address 00:11:22:00:03:02 dev ns3eth2 + # All Links up ip -net "$ns1" link set ns1eth1 up ip -net "$ns1" link set ns1eth2 up @@ -174,6 +183,8 @@ trap cleanup_all_ns EXIT setup_hsr_interfaces 0 do_complete_ping_test +setup_ns ns1 ns2 ns3 + setup_hsr_interfaces 1 do_complete_ping_test diff --git a/tools/testing/selftests/net/hsr/hsr_redbox.sh b/tools/testing/selftests/net/hsr/hsr_redbox.sh index 1f36785347c0b147c94350903e12f6d48c64d1ff..998103502d5d38ee68641e80003303b508672e5c 100755 --- a/tools/testing/selftests/net/hsr/hsr_redbox.sh +++ b/tools/testing/selftests/net/hsr/hsr_redbox.sh @@ -96,6 +96,21 @@ setup_hsr_interfaces() ip -n "${ns4}" link set ns4eth1 up ip -n "${ns5}" link set ns5eth1 up + ip -net "$ns1" link set address 00:11:22:00:01:01 dev ns1eth1 + ip -net "$ns1" link set address 00:11:22:00:01:02 dev ns1eth2 + + ip -net "$ns2" link set address 00:11:22:00:02:01 dev ns2eth1 + ip -net "$ns2" link set address 00:11:22:00:02:02 dev ns2eth2 + ip -net "$ns2" link set address 00:11:22:00:02:03 dev ns2eth3 + + ip -net "$ns3" link set address 00:11:22:00:03:11 dev ns3eth1 + ip -net "$ns3" link set address 00:11:22:00:03:11 dev ns3eth2 + ip -net "$ns3" link set address 00:11:22:00:03:11 dev ns3eth3 + ip -net "$ns3" link set address 00:11:22:00:03:11 dev ns3br1 + + ip -net "$ns4" link set address 00:11:22:00:04:01 dev ns4eth1 + ip -net "$ns5" link set address 00:11:22:00:05:01 dev ns5eth1 + ip -net "${ns1}" link add name hsr1 type hsr slave1 ns1eth1 slave2 ns1eth2 supervision 45 version ${HSRv} proto 0 ip -net "${ns2}" link add name hsr2 type hsr slave1 ns2eth1 slave2 ns2eth2 interlink ns2eth3 supervision 45 version ${HSRv} proto 0 diff --git a/tools/testing/selftests/net/lib.sh b/tools/testing/selftests/net/lib.sh index edc030e81a4649e5245c6bc1a336f3c83c28daed..9155c914c064fe542fae87631561692e48104a26 100644 --- a/tools/testing/selftests/net/lib.sh +++ b/tools/testing/selftests/net/lib.sh @@ -15,7 +15,7 @@ ksft_xfail=2 ksft_skip=4 # namespace list created by setup_ns -NS_LIST="" +NS_LIST=() ############################################################################## # Helpers @@ -27,6 +27,7 @@ __ksft_status_merge() local -A weights local weight=0 + local i for i in "$@"; do weights[$i]=$((weight++)) done @@ -67,9 +68,7 @@ loopy_wait() while true do local out - out=$("$@") - local ret=$? - if ((!ret)); then + if out=$("$@"); then echo -n "$out" return 0 fi @@ -139,6 +138,7 @@ cleanup_ns() fi for ns in "$@"; do + [ -z "${ns}" ] && continue ip netns delete "${ns}" &> /dev/null if ! busywait $BUSYWAIT_TIMEOUT ip netns list \| grep -vq "^$ns$" &> /dev/null; then echo "Warn: Failed to remove namespace $ns" @@ -152,7 +152,7 @@ cleanup_ns() cleanup_all_ns() { - cleanup_ns $NS_LIST + cleanup_ns "${NS_LIST[@]}" } # setup netns with given names as prefix. e.g @@ -161,7 +161,7 @@ setup_ns() { local ns="" local ns_name="" - local ns_list="" + local ns_list=() local ns_exist= for ns_name in "$@"; do # Some test may setup/remove same netns multi times @@ -177,13 +177,13 @@ setup_ns() if ! ip netns add "$ns"; then echo "Failed to create namespace $ns_name" - cleanup_ns "$ns_list" + cleanup_ns "${ns_list[@]}" return $ksft_skip fi ip -n "$ns" link set lo up - ! $ns_exist && ns_list="$ns_list $ns" + ! $ns_exist && ns_list+=("$ns") done - NS_LIST="$NS_LIST $ns_list" + NS_LIST+=("${ns_list[@]}") } tc_rule_stats_get() diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh index fefa9173bdaaabf7182b1bc2c626cbdfbe014efa..2b66c5fa71ebff7bf93f712aecab26e7e77134e3 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh @@ -261,6 +261,8 @@ reset() TEST_NAME="${1}" + MPTCP_LIB_SUBTEST_FLAKY=0 # reset if modified + if skip_test; then MPTCP_LIB_TEST_COUNTER=$((MPTCP_LIB_TEST_COUNTER+1)) last_test_ignored=1 @@ -448,7 +450,9 @@ reset_with_tcp_filter() # $1: err msg fail_test() { - ret=${KSFT_FAIL} + if ! mptcp_lib_subtest_is_flaky; then + ret=${KSFT_FAIL} + fi if [ ${#} -gt 0 ]; then print_fail "${@}" @@ -3069,6 +3073,7 @@ fullmesh_tests() fastclose_tests() { if reset_check_counter "fastclose test" "MPTcpExtMPFastcloseTx"; then + MPTCP_LIB_SUBTEST_FLAKY=1 test_linkfail=1024 fastclose=client \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 0 0 0 @@ -3077,6 +3082,7 @@ fastclose_tests() fi if reset_check_counter "fastclose server test" "MPTcpExtMPFastcloseRx"; then + MPTCP_LIB_SUBTEST_FLAKY=1 test_linkfail=1024 fastclose=server \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 0 0 0 0 0 0 1 @@ -3095,6 +3101,7 @@ fail_tests() { # single subflow if reset_with_fail "Infinite map" 1; then + MPTCP_LIB_SUBTEST_FLAKY=1 test_linkfail=128 \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 0 0 0 +1 +0 1 0 1 "$(pedit_action_pkts)" @@ -3103,6 +3110,7 @@ fail_tests() # multiple subflows if reset_with_fail "MP_FAIL MP_RST" 2; then + MPTCP_LIB_SUBTEST_FLAKY=1 tc -n $ns2 qdisc add dev ns2eth1 root netem rate 1mbit delay 5ms pm_nl_set_limits $ns1 0 1 pm_nl_set_limits $ns2 0 1 diff --git a/tools/testing/selftests/net/mptcp/mptcp_lib.sh b/tools/testing/selftests/net/mptcp/mptcp_lib.sh index ad2ebda5cb64ba5e389ec9f4f3064474547fba95..6ffa9b7a3260dffa2bedf1685155b2c45053ce56 100644 --- a/tools/testing/selftests/net/mptcp/mptcp_lib.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_lib.sh @@ -21,6 +21,7 @@ declare -rx MPTCP_LIB_AF_INET6=10 MPTCP_LIB_SUBTESTS=() MPTCP_LIB_SUBTESTS_DUPLICATED=0 +MPTCP_LIB_SUBTEST_FLAKY=0 MPTCP_LIB_TEST_COUNTER=0 MPTCP_LIB_TEST_FORMAT="%02u %-50s" MPTCP_LIB_IP_MPTCP=0 @@ -41,6 +42,16 @@ else readonly MPTCP_LIB_COLOR_RESET= fi +# SELFTESTS_MPTCP_LIB_OVERRIDE_FLAKY env var can be set not to ignore errors +# from subtests marked as flaky +mptcp_lib_override_flaky() { + [ "${SELFTESTS_MPTCP_LIB_OVERRIDE_FLAKY:-}" = 1 ] +} + +mptcp_lib_subtest_is_flaky() { + [ "${MPTCP_LIB_SUBTEST_FLAKY}" = 1 ] && ! mptcp_lib_override_flaky +} + # $1: color, $2: text mptcp_lib_print_color() { echo -e "${MPTCP_LIB_START_PRINT:-}${*}${MPTCP_LIB_COLOR_RESET}" @@ -72,7 +83,16 @@ mptcp_lib_pr_skip() { } mptcp_lib_pr_fail() { - mptcp_lib_print_err "[FAIL]${1:+ ${*}}" + local title cmt + + if mptcp_lib_subtest_is_flaky; then + title="IGNO" + cmt=" (flaky)" + else + title="FAIL" + fi + + mptcp_lib_print_err "[${title}]${cmt}${1:+ ${*}}" } mptcp_lib_pr_info() { @@ -208,7 +228,13 @@ mptcp_lib_result_pass() { # $1: test name mptcp_lib_result_fail() { - __mptcp_lib_result_add "not ok" "${1}" + if mptcp_lib_subtest_is_flaky; then + # It might sound better to use 'not ok # TODO' or 'ok # SKIP', + # but some CIs don't understand 'TODO' and treat SKIP as errors. + __mptcp_lib_result_add "ok" "${1} # IGNORE Flaky" + else + __mptcp_lib_result_add "not ok" "${1}" + fi } # $1: test name diff --git a/tools/testing/selftests/net/mptcp/simult_flows.sh b/tools/testing/selftests/net/mptcp/simult_flows.sh index 4b14b4412166b5db79ea6be28c009b536a8136b8..f74e1c3c126d18821ebba2968ed3f25b9f90c506 100755 --- a/tools/testing/selftests/net/mptcp/simult_flows.sh +++ b/tools/testing/selftests/net/mptcp/simult_flows.sh @@ -244,7 +244,7 @@ run_test() do_transfer $small $large $time lret=$? mptcp_lib_result_code "${lret}" "${msg}" - if [ $lret -ne 0 ]; then + if [ $lret -ne 0 ] && ! mptcp_lib_subtest_is_flaky; then ret=$lret [ $bail -eq 0 ] || exit $ret fi @@ -254,7 +254,7 @@ run_test() do_transfer $large $small $time lret=$? mptcp_lib_result_code "${lret}" "${msg}" - if [ $lret -ne 0 ]; then + if [ $lret -ne 0 ] && ! mptcp_lib_subtest_is_flaky; then ret=$lret [ $bail -eq 0 ] || exit $ret fi @@ -290,7 +290,7 @@ run_test 10 10 0 0 "balanced bwidth" run_test 10 10 1 25 "balanced bwidth with unbalanced delay" # we still need some additional infrastructure to pass the following test-cases -run_test 10 3 0 0 "unbalanced bwidth" +MPTCP_LIB_SUBTEST_FLAKY=1 run_test 10 3 0 0 "unbalanced bwidth" run_test 10 3 1 25 "unbalanced bwidth with unbalanced delay" run_test 10 3 25 1 "unbalanced bwidth with opposed, unbalanced delay" diff --git a/tools/testing/selftests/net/netns-sysctl.sh b/tools/testing/selftests/net/netns-sysctl.sh new file mode 100755 index 0000000000000000000000000000000000000000..45c34a3b9aae32baa5830b655bd7ceade9e3ff26 --- /dev/null +++ b/tools/testing/selftests/net/netns-sysctl.sh @@ -0,0 +1,40 @@ +#!/bin/bash -e +# SPDX-License-Identifier: GPL-2.0 +# +# This test checks that the network buffer sysctls are present +# in a network namespaces, and that they are readonly. + +source lib.sh + +cleanup() { + cleanup_ns $test_ns +} + +trap cleanup EXIT + +fail() { + echo "ERROR: $*" >&2 + exit 1 +} + +setup_ns test_ns + +for sc in {r,w}mem_{default,max}; do + # check that this is writable in a netns + [ -w "/proc/sys/net/core/$sc" ] || + fail "$sc isn't writable in the init netns!" + + # change the value in the host netns + sysctl -qw "net.core.$sc=300000" || + fail "Can't write $sc in init netns!" + + # check that the value is read from the init netns + [ "$(ip netns exec $test_ns sysctl -n "net.core.$sc")" -eq 300000 ] || + fail "Value for $sc mismatch!" + + # check that this isn't writable in a netns + ip netns exec $test_ns [ -w "/proc/sys/net/core/$sc" ] && + fail "$sc is writable in a netns!" +done + +echo 'Test passed OK' diff --git a/tools/testing/selftests/openat2/openat2_test.c b/tools/testing/selftests/openat2/openat2_test.c index 9024754530b230d0252905c5e01d327908da092a..5790ab446527f1677408ffce4b9cf76271afe9ed 100644 --- a/tools/testing/selftests/openat2/openat2_test.c +++ b/tools/testing/selftests/openat2/openat2_test.c @@ -5,6 +5,7 @@ */ #define _GNU_SOURCE +#define __SANE_USERSPACE_TYPES__ // Use ll64 #include #include #include diff --git a/tools/testing/selftests/tc-testing/tc-tests/qdiscs/taprio.json b/tools/testing/selftests/tc-testing/tc-tests/qdiscs/taprio.json index 12da0a939e3e5f2461eadb48883d4e0d024f7e34..557fb074acf0ca4e7a5ee0e31b306f2e3ac3725a 100644 --- a/tools/testing/selftests/tc-testing/tc-tests/qdiscs/taprio.json +++ b/tools/testing/selftests/tc-testing/tc-tests/qdiscs/taprio.json @@ -132,6 +132,50 @@ "echo \"1\" > /sys/bus/netdevsim/del_device" ] }, + { + "id": "6f62", + "name": "Add taprio Qdisc with too short interval", + "category": [ + "qdisc", + "taprio" + ], + "plugins": { + "requires": "nsPlugin" + }, + "setup": [ + "echo \"1 1 8\" > /sys/bus/netdevsim/new_device" + ], + "cmdUnderTest": "$TC qdisc add dev $ETH root handle 1: taprio num_tc 2 queues 1@0 1@1 sched-entry S 01 300 sched-entry S 02 1700 clockid CLOCK_TAI", + "expExitCode": "2", + "verifyCmd": "$TC qdisc show dev $ETH", + "matchPattern": "qdisc taprio 1: root refcnt", + "matchCount": "0", + "teardown": [ + "echo \"1\" > /sys/bus/netdevsim/del_device" + ] + }, + { + "id": "831f", + "name": "Add taprio Qdisc with too short cycle-time", + "category": [ + "qdisc", + "taprio" + ], + "plugins": { + "requires": "nsPlugin" + }, + "setup": [ + "echo \"1 1 8\" > /sys/bus/netdevsim/new_device" + ], + "cmdUnderTest": "$TC qdisc add dev $ETH root handle 1: taprio num_tc 2 queues 1@0 1@1 sched-entry S 01 200000 sched-entry S 02 200000 cycle-time 100 clockid CLOCK_TAI", + "expExitCode": "2", + "verifyCmd": "$TC qdisc show dev $ETH", + "matchPattern": "qdisc taprio 1: root refcnt", + "matchCount": "0", + "teardown": [ + "echo \"1\" > /sys/bus/netdevsim/del_device" + ] + }, { "id": "3e1e", "name": "Add taprio Qdisc with an invalid cycle-time",