linux-imx

enetc: Replace enetc_gregs with a readers-writer lock

The LS1028A MDIO errata tells us that any MDIO register access must not
be concurrent with any other ENETC register access.

That has been handled so far by a number of per-CPU spinlocks over the
ENETC register map. This came as an optimization over a single spinlock,
because the regular register accesses can still be concurrent with one
another, as long as they aren't concurrent with MDIO.

But this logic is broken in RT, because the enetc_rd_reg_wa and
enetc_wr_reg_wa functions can be preempted in any context, and when they
resume they may not run on the same CPU.

This renders the logic to take the per-CPU spinlock pointless, since the
spinlock may not be the correct one (corresponding to this CPU) after
preemption has occurred.

The following splat is telling us the same thing:

[   19.073928] BUG: using smp_processor_id() in preemptible [00000000] code: systemd-network/3423
[   19.073932] caller is debug_smp_processor_id+0x1c/0x30
[   19.073935] CPU: 1 PID: 3423 Comm: systemd-network Not tainted 4.19.68-rt26 #1
[   19.073936] Hardware name: LS1028A RDB Board (DT)
[   19.073938] Call trace:
[   19.073940]  dump_backtrace+0x0/0x1a0
[   19.073942]  show_stack+0x24/0x30
[   19.073945]  dump_stack+0x9c/0xdc
[   19.073948]  check_preemption_disabled+0xe0/0x100
[   19.073951]  debug_smp_processor_id+0x1c/0x30
[   19.073954]  enetc_open+0x1b0/0xbc0
[   19.073957]  __dev_open+0xdc/0x160
[   19.073960]  __dev_change_flags+0x160/0x1d0
[   19.073963]  dev_change_flags+0x34/0x70
[   19.073966]  do_setlink+0x2a0/0xcd0
[   19.073969]  rtnl_setlink+0xe4/0x140
[   19.073972]  rtnetlink_rcv_msg+0x18c/0x500
[   19.073975]  netlink_rcv_skb+0x60/0x120
[   19.073978]  rtnetlink_rcv+0x28/0x40
[   19.073982]  netlink_unicast+0x194/0x210
[   19.073985]  netlink_sendmsg+0x194/0x330
[   19.073987]  sock_sendmsg+0x34/0x50
[   19.073990]  __sys_sendto+0xe4/0x150
[   19.073992]  __arm64_sys_sendto+0x30/0x40
[   19.073996]  el0_svc_common+0xa4/0x1a0
[   19.073999]  el0_svc_handler+0x38/0x80
[   19.074002]  el0_svc+0x8/0xc

But there already exists a spinlock optimized for the single writer,
multiple readers case: the rwlock_t. The writer in this case is the MDIO
access code (irrelevant whether that MDIO access is a register read or
write), and the reader is everybody else.

This patch also fixes two more existing bugs in the errata workaround:
- The MDIO access code was not unlocking the per-CPU spinlocks in the
  reverse order of their locking order.
- The per-CPU spinlock array was not initialized.

Fixes: 5ec0d668d62e ("enetc: WA for MDIO register access issue")
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Signed-off-by: Claudiu Manoil <claudiu.manoil@nxp.com>