By inspection the mbuf->port is never set for received packets in af_xdp device driver.  Looks like driver needs to record port (in rxq structure) during initialization than do:
      mbuf->port = rxq->port;