The controller ports are P4 and P5 on
this schematic of the Famicom. The ICs that handle this are U7 and U8, which appear to be 6-pin 40H368 (like 74368?) bus drivers. The CPU puts out three relevant signals: last value written to $4016 D0 (strobe), $4016 /OE (player 1 clock), and $4017 /OE (player 2 clock), which gets sent to the controllers. Then the 368s put signals from the controllers on D0, D3, and D4.