Closes #482
I traced the problem to where buildx parses the cli args
([here](fa4461b9a1/util/buildflags/cache.go (L14))),
and confirmed it applies defaults based on GHA environment variables and
ignores the cacheTo/cacheFrom directive altogether when the variables
aren't available.
The fix is to ignore the GHA cache directive when it the upstream parser
ignores it.