The kernel device tree needs to be modified in order to support RGMII to RGMII direct connection.
make sure 'fec1' node is enabled in device tree
make sure phy-mode is changed to
phy-mode = "rgmii-id";
add following section in 'fec1' node
fixed-link { speed = <1000>; full-duplex; };
RGMII internal delay needs to be set for TXC on the peer.
Note that TXC and RXC is enough for RGMII direct connection. ETH_REF_CLK is not mandatory in terms of RGMII link, unless the chip itself needs a source of clock input.