all: remove darwin/arm build-tags and files

This removes all files that are only used on darwin/arm and cleans up
build tags in files that are still used on other platforms.

Updates #37611.

Change-Id: Ic9490cf0edfc157c6276a7ca950c1768b34a998f
Reviewed-on: https://go-review.googlesource.com/c/go/+/227197


Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>